2023-06-09T10:50:38.487 INFO:root:teuthology version: 0.0.1.dev120+g2ef0dcd 2023-06-09T10:50:38.489 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849 --owner scheduled_rishabh@teuthology --description fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} --verbose --name rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/orig.config.yaml 2023-06-09T10:50:38.543 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-09T10:50:38.645 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849 branch: wip-rishabh-2023Jun9-1308 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} email: null first_in_suite: false job_id: '7299849' kernel: flavor: default kdb: true sha1: 10b2f20a4f46c803bb730efc73498f97d45082e7 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jun9-1308 ceph: cephfs: max_mds: 3 session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - Replacing daemon mds sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jun9-1308 sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d owner: scheduled_rishabh@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jun9-1308 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi117.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - mds_thrash: null - workunit: clients: all: - fs/snaps timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61 timestamp: 2023-06-09_10:19:22 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-09T10:50:38.646 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa; will attempt to use it 2023-06-09T10:50:38.647 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks 2023-06-09T10:50:38.648 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-09T10:50:38.658 INFO:teuthology.task.internal:Checking packages... 2023-06-09T10:50:38.693 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '034108821cde95ca1c56d458dfccc7ef9e0d964d' 2023-06-09T10:50:38.694 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-09T10:50:38.694 INFO:teuthology.packaging:ref: None 2023-06-09T10:50:38.694 INFO:teuthology.packaging:tag: None 2023-06-09T10:50:38.695 INFO:teuthology.packaging:branch: wip-rishabh-2023Jun9-1308 2023-06-09T10:50:38.695 INFO:teuthology.packaging:sha1: 034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T10:50:38.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jun9-1308 2023-06-09T10:50:38.841 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3924.g03410882 2023-06-09T10:50:38.841 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-09T10:50:38.850 INFO:teuthology.task.internal:no buildpackages task found 2023-06-09T10:50:38.851 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-09T10:50:38.859 INFO:teuthology.task.internal:Saving configuration 2023-06-09T10:50:38.883 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-09T10:50:38.891 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-09T10:50:38.920 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 10:19:34.133256', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-09T10:50:38.948 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 10:19:34.134781', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-09T10:50:38.948 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-09T10:50:38.963 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-06-09T10:50:38.963 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-06-09T10:50:38.963 INFO:teuthology.run_tasks:Running task console_log... 2023-06-09T10:50:39.056 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fefd8cd4ee0>, signals=[15]) 2023-06-09T10:50:39.057 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-09T10:50:39.065 INFO:teuthology.task.internal:Opening connections... 2023-06-09T10:50:39.066 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T10:50:39.068 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:50:39.139 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T10:50:39.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:50:39.250 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-09T10:50:39.257 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2023-06-09T10:50:39.279 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2023-06-09T10:50:39.280 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2023-06-09T10:50:39.340 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="Red Hat Enterprise Linux" 2023-06-09T10:50:39.341 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="8.6 (Ootpa)" 2023-06-09T10:50:39.341 INFO:teuthology.orchestra.run.smithi007.stdout:ID="rhel" 2023-06-09T10:50:39.341 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="fedora" 2023-06-09T10:50:39.342 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="8.6" 2023-06-09T10:50:39.342 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el8" 2023-06-09T10:50:39.342 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-09T10:50:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2023-06-09T10:50:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-09T10:50:39.343 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://www.redhat.com/" 2023-06-09T10:50:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-09T10:50:39.344 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-09T10:50:39.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T10:50:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-09T10:50:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-09T10:50:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-09T10:50:39.345 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-09T10:50:39.347 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2023-06-09T10:50:39.378 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2023-06-09T10:50:39.406 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2023-06-09T10:50:39.407 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2023-06-09T10:50:39.469 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Red Hat Enterprise Linux" 2023-06-09T10:50:39.469 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="8.6 (Ootpa)" 2023-06-09T10:50:39.469 INFO:teuthology.orchestra.run.smithi117.stdout:ID="rhel" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="fedora" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="8.6" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el8" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-09T10:50:39.470 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.redhat.com/" 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-09T10:50:39.471 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-09T10:50:39.472 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2023-06-09T10:50:39.494 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-09T10:50:39.505 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-09T10:50:39.519 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-09T10:50:39.520 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2023-06-09T10:50:39.522 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2023-06-09T10:50:39.540 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-09T10:50:39.548 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-09T10:50:39.548 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2023-06-09T10:50:39.582 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2023-06-09T10:50:39.610 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-09T10:50:39.674 INFO:teuthology.run_tasks:Running task kernel... 2023-06-09T10:50:39.691 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '10b2f20a4f46c803bb730efc73498f97d45082e7'} 2023-06-09T10:50:39.691 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '10b2f20a4f46c803bb730efc73498f97d45082e7'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '10b2f20a4f46c803bb730efc73498f97d45082e7'}}, timeout 300 2023-06-09T10:50:39.692 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-09T10:50:39.692 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-09T10:50:39.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:50:39.712 DEBUG:teuthology.task.kernel:sha1 for mon.b is 10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.712 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "10b2f20a4f46c803bb730efc73498f97d45082e7"... 2023-06-09T10:50:39.712 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2023-06-09T10:50:39.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:50:39.713 DEBUG:teuthology.task.kernel:sha1 for mon.a is 10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.714 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "10b2f20a4f46c803bb730efc73498f97d45082e7"... 2023-06-09T10:50:39.714 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-06-09T10:50:39.770 INFO:teuthology.orchestra.run.smithi117.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-09T10:50:39.771 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.771 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-06-09T10:50:39.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.774 INFO:teuthology.orchestra.run.smithi007.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-09T10:50:39.775 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.775 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-06-09T10:50:39.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:39.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2023-06-09T10:50:40.001 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum upgrade -y linux-firmware 2023-06-09T10:50:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2023-06-09T10:50:41.135 INFO:teuthology.orchestra.run.smithi007.stdout:Unable to read consumer identity 2023-06-09T10:50:41.206 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-06-09T10:50:41.206 INFO:teuthology.orchestra.run.smithi117.stdout:Unable to read consumer identity 2023-06-09T10:50:44.735 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-09T10:50:44.952 INFO:teuthology.orchestra.run.smithi117.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-09T10:50:45.661 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-06-09T10:50:46.778 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 16 MB 00:01 2023-06-09T10:50:49.369 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 390 kB/s | 24 kB 00:00 2023-06-09T10:50:50.148 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2023-06-09T10:50:50.149 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2023-06-09T10:50:50.149 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2023-06-09T10:50:50.235 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 370 kB/s | 24 kB 00:00 2023-06-09T10:50:50.274 INFO:teuthology.task.kernel:Downloading kernel 10b2f20a4f46c803bb730efc73498f97d45082e7 on mon.a... 2023-06-09T10:50:50.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:50.411 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/ 2023-06-09T10:50:50.412 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_rc2_g10b2f20a4f46-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/ --input-file=- 2023-06-09T10:50:50.415 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2023-06-09T10:50:51.018 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2023-06-09T10:50:51.019 INFO:teuthology.orchestra.run.smithi117.stdout:Nothing to do. 2023-06-09T10:50:51.019 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2023-06-09T10:50:51.146 INFO:teuthology.task.kernel:Downloading kernel 10b2f20a4f46c803bb730efc73498f97d45082e7 on mon.b... 2023-06-09T10:50:51.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=10b2f20a4f46c803bb730efc73498f97d45082e7 2023-06-09T10:50:51.283 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/ 2023-06-09T10:50:51.284 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_rc2_g10b2f20a4f46-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/ --input-file=- 2023-06-09T10:50:51.287 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi117.front.sepia.ceph.com to complete... 2023-06-09T10:51:19.056 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09 10:51:19 URL:https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/kernel-6.4.0_rc2_g10b2f20a4f46-1.x86_64.rpm [379752256/379752256] -> "/tmp/kernel.x86_64.rpm" [1] 2023-06-09T10:51:19.056 INFO:teuthology.orchestra.run.smithi117.stderr:FINISHED --2023-06-09 10:51:19-- 2023-06-09T10:51:19.057 INFO:teuthology.orchestra.run.smithi117.stderr:Total wall clock time: 28s 2023-06-09T10:51:19.057 INFO:teuthology.orchestra.run.smithi117.stderr:Downloaded: 1 files, 362M in 28s (13.2 MB/s) 2023-06-09T10:51:19.058 INFO:teuthology.task.kernel:Installing kernel 10b2f20a4f46c803bb730efc73498f97d45082e7 on mon.b... 2023-06-09T10:51:19.058 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-06-09T10:51:20.635 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying... ######################################## 2023-06-09T10:51:20.698 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing... ######################################## 2023-06-09T10:51:20.781 INFO:teuthology.orchestra.run.smithi117.stdout:Updating / installing... 2023-06-09T10:51:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09 10:51:27 URL:https://2.chacra.ceph.com/r/kernel/testing/10b2f20a4f46c803bb730efc73498f97d45082e7/centos/8/flavors/default/x86_64/kernel-6.4.0_rc2_g10b2f20a4f46-1.x86_64.rpm [379752256/379752256] -> "/tmp/kernel.x86_64.rpm" [1] 2023-06-09T10:51:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2023-06-09 10:51:27-- 2023-06-09T10:51:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 37s 2023-06-09T10:51:27.942 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 362M in 37s (9.72 MB/s) 2023-06-09T10:51:27.944 INFO:teuthology.task.kernel:Installing kernel 10b2f20a4f46c803bb730efc73498f97d45082e7 on mon.a... 2023-06-09T10:51:27.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-06-09T10:51:29.617 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2023-06-09T10:51:29.697 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2023-06-09T10:51:29.785 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2023-06-09T10:51:58.511 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-6.4.0_rc2_g10b2f20a4f46-1 ######################################## 2023-06-09T10:52:10.157 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.4.0_rc2_g10b2f20a4f46-1 ######################################## 2023-06-09T10:52:22.904 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-06-09T10:52:22.904 DEBUG:teuthology.orchestra.run.smithi117:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-06-09T10:52:22.958 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/System.map-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:22.958 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/config-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:22.959 INFO:teuthology.orchestra.run.smithi117.stdout:/boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:22.959 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:22.959 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel 2023-06-09T10:52:22.959 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch 2023-06-09T10:52:22.959 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86 2023-06-09T10:52:22.960 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto 2023-06-09T10:52:22.960 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-06-09T10:52:22.960 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-06-09T10:52:22.960 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-06-09T10:52:22.960 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crc32c-intel.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-06-09T10:52:22.961 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-avx2.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-06-09T10:52:22.962 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu/mce 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm 2023-06-09T10:52:22.963 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm-amd.ko 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm-intel.ko 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm.ko 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/ansi_cprng.ko 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/anubis.ko 2023-06-09T10:52:22.964 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/arc4.ko 2023-06-09T10:52:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx 2023-06-09T10:52:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_memcpy.ko 2023-06-09T10:52:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_pq.ko 2023-06-09T10:52:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_raid6_recov.ko 2023-06-09T10:52:22.965 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_tx.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_xor.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/raid6test.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/authenc.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/authencesn.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blake2b_generic.ko 2023-06-09T10:52:22.966 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blowfish_common.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blowfish_generic.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/camellia_generic.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast5_generic.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast6_generic.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast_common.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/ccm.ko 2023-06-09T10:52:22.967 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cmac.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crc32_generic.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crypto_engine.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crypto_user.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cts.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/des_generic.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/dh_generic.ko 2023-06-09T10:52:22.968 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/echainiv.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/essiv.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/fcrypt.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/khazad.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/lz4.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/lz4hc.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/md4.ko 2023-06-09T10:52:22.969 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/michael_mic.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/pcbc.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/pcrypt.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/rmd160.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/seed.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/serpent_generic.ko 2023-06-09T10:52:22.970 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/tcrypt.ko 2023-06-09T10:52:22.971 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/tea.ko 2023-06-09T10:52:22.971 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/twofish_common.ko 2023-06-09T10:52:22.971 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/twofish_generic.ko 2023-06-09T10:52:22.971 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/vmac.ko 2023-06-09T10:52:22.971 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/wp512.ko 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xcbc.ko 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xor.ko 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xxhash_generic.ko 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/acpi_ipmi.ko 2023-06-09T10:52:22.972 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/acpi_pad.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/custom_method.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/ec_sys.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/platform_profile.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/sbs.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/sbshc.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/video.ko 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata 2023-06-09T10:52:22.973 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/acard-ahci.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/ahci_platform.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/ata_generic.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/libahci_platform.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_acpi.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ali.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_amd.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_artop.ko 2023-06-09T10:52:22.974 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_atiixp.ko 2023-06-09T10:52:22.975 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_atp867x.ko 2023-06-09T10:52:22.975 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cmd640.ko 2023-06-09T10:52:22.975 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cmd64x.ko 2023-06-09T10:52:22.975 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cypress.ko 2023-06-09T10:52:22.975 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_efar.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt366.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt37x.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt3x2n.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt3x3.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_it8213.ko 2023-06-09T10:52:22.976 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_it821x.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_jmicron.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_marvell.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_mpiix.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_netcell.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ninja32.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ns87410.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ns87415.ko 2023-06-09T10:52:22.977 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_oldpiix.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_opti.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_optidma.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pcmcia.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pdc2027x.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_piccolo.ko 2023-06-09T10:52:22.978 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_rdc.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sch.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_serverworks.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sil680.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sis.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sl82c105.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_triflex.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_via.ko 2023-06-09T10:52:22.979 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pdc_adma.ko 2023-06-09T10:52:22.980 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_inic162x.ko 2023-06-09T10:52:22.980 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_mv.ko 2023-06-09T10:52:22.980 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_nv.ko 2023-06-09T10:52:22.980 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_promise.ko 2023-06-09T10:52:22.980 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_qstor.ko 2023-06-09T10:52:22.981 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sil.ko 2023-06-09T10:52:22.981 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sil24.ko 2023-06-09T10:52:22.981 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sis.ko 2023-06-09T10:52:22.981 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_svw.ko 2023-06-09T10:52:22.982 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sx4.ko 2023-06-09T10:52:22.982 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_uli.ko 2023-06-09T10:52:22.982 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_via.ko 2023-06-09T10:52:22.983 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_vsc.ko 2023-06-09T10:52:22.983 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/auxdisplay 2023-06-09T10:52:22.983 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/auxdisplay/ks0108.ko 2023-06-09T10:52:22.983 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base 2023-06-09T10:52:22.984 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base/regmap 2023-06-09T10:52:22.984 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base/regmap/regmap-i2c.ko 2023-06-09T10:52:22.984 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/bcma 2023-06-09T10:52:22.984 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/bcma/bcma.ko 2023-06-09T10:52:22.985 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block 2023-06-09T10:52:22.985 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/aoe 2023-06-09T10:52:22.985 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/aoe/aoe.ko 2023-06-09T10:52:22.986 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/brd.ko 2023-06-09T10:52:22.986 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/drbd 2023-06-09T10:52:22.986 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/drbd/drbd.ko 2023-06-09T10:52:22.986 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/floppy.ko 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/loop.ko 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/mtip32xx 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/nbd.ko 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/null_blk 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/null_blk/null_blk.ko 2023-06-09T10:52:22.987 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/pktcdvd.ko 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/rbd.ko 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/virtio_blk.ko 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkback 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkfront.ko 2023-06-09T10:52:22.988 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/zram 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/zram/zram.ko 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hangcheck-timer.ko 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/amd-rng.ko 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/intel-rng.ko 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-06-09T10:52:22.989 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/via-rng.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/virtio-rng.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_si.ko 2023-06-09T10:52:22.990 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-06-09T10:52:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-06-09T10:52:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/lp.ko 2023-06-09T10:52:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/mwave 2023-06-09T10:52:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/mwave/mwave.ko 2023-06-09T10:52:22.991 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ppdev.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tlclk.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_atmel.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_infineon.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_nsc.ko 2023-06-09T10:52:22.992 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_tis.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/uv_mmtimer.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/virtio_console.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/p4-clockmod.ko 2023-06-09T10:52:22.993 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/powernow-k8.ko 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/speedstep-lib.ko 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp/ccp.ko 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel 2023-06-09T10:52:22.994 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat 2023-06-09T10:52:22.995 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_common 2023-06-09T10:52:22.995 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-06-09T10:52:22.995 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-06-09T10:52:22.995 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-06-09T10:52:22.995 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/padlock-aes.ko 2023-06-09T10:52:22.996 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/padlock-sha.ko 2023-06-09T10:52:22.996 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/virtio 2023-06-09T10:52:22.996 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-06-09T10:52:22.996 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dca 2023-06-09T10:52:22.997 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dca/dca.ko 2023-06-09T10:52:22.997 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/devfreq 2023-06-09T10:52:22.997 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-06-09T10:52:22.998 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma 2023-06-09T10:52:22.998 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/dw 2023-06-09T10:52:22.998 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/dw/dw_dmac.ko 2023-06-09T10:52:22.998 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/ioat 2023-06-09T10:52:22.999 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/ioat/ioatdma.ko 2023-06-09T10:52:22.999 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac 2023-06-09T10:52:22.999 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/amd64_edac.ko 2023-06-09T10:52:22.999 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/e752x_edac.ko 2023-06-09T10:52:23.000 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/edac_mce_amd.ko 2023-06-09T10:52:23.000 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i3000_edac.ko 2023-06-09T10:52:23.000 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i3200_edac.ko 2023-06-09T10:52:23.000 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i5100_edac.ko 2023-06-09T10:52:23.001 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i5400_edac.ko 2023-06-09T10:52:23.001 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i7300_edac.ko 2023-06-09T10:52:23.001 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i7core_edac.ko 2023-06-09T10:52:23.001 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i82975x_edac.ko 2023-06-09T10:52:23.002 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/ie31200_edac.ko 2023-06-09T10:52:23.002 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/sb_edac.ko 2023-06-09T10:52:23.002 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/x38_edac.ko 2023-06-09T10:52:23.002 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware 2023-06-09T10:52:23.003 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware/edd.ko 2023-06-09T10:52:23.003 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware/iscsi_ibft.ko 2023-06-09T10:52:23.003 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio 2023-06-09T10:52:23.004 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio/gpio-ich.ko 2023-06-09T10:52:23.004 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio/gpio-viperboard.ko 2023-06-09T10:52:23.004 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid 2023-06-09T10:52:23.004 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-appleir.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-aureal.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-axff.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-dr.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-elecom.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-elo.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-emsff.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gaff.ko 2023-06-09T10:52:23.005 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gt683r.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gyration.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtek-kbd.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtek-mouse.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtekff.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-hyperv.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-icade.ko 2023-06-09T10:52:23.006 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-keytouch.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-kye.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-lcpower.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-led.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-lenovo.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-logitech-dj.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-multitouch.ko 2023-06-09T10:52:23.007 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-ortek.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-penmount.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-petalynx.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-picolcd.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-pl.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-plantronics.ko 2023-06-09T10:52:23.008 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-primax.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-rmi.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-arvo.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-common.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-isku.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-kone.ko 2023-06-09T10:52:23.009 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-konepure.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-lua.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-pyra.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-ryos.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-savu.ko 2023-06-09T10:52:23.010 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-saitek.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-samsung.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sensor-hub.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sjoy.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sony.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-speedlink.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-steelseries.ko 2023-06-09T10:52:23.011 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sunplus.ko 2023-06-09T10:52:23.012 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-thrustmaster.ko 2023-06-09T10:52:23.012 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-tivo.ko 2023-06-09T10:52:23.012 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-tmff.ko 2023-06-09T10:52:23.012 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-topseed.ko 2023-06-09T10:52:23.012 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-twinhan.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-uclogic.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-waltop.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-wiimote.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-xinmo.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-zpff.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-zydacron.ko 2023-06-09T10:52:23.013 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/uhid.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/wacom.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_balloon.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_utils.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_vmbus.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/abituguru.ko 2023-06-09T10:52:23.014 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/abituguru3.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/acpi_power_meter.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ad7414.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ad7418.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adc128d818.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1025.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1026.ko 2023-06-09T10:52:23.015 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1029.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1031.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm9240.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ads7828.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7410.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7411.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7462.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7470.ko 2023-06-09T10:52:23.016 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7475.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7x10.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/amc6821.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/applesmc.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asb100.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asc7621.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asus_atk0110.ko 2023-06-09T10:52:23.017 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/atxp1.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/coretemp.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/dme1737.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ds1621.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ds620.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/emc1403.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/emc6w201.ko 2023-06-09T10:52:23.018 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f71805f.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f71882fg.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f75375s.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/fam15h_power.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/fschmd.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/g760a.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/g762.ko 2023-06-09T10:52:23.019 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/gl518sm.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/gl520sm.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/hwmon-vid.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/i5500_temp.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/i5k_amb.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ibmaem.ko 2023-06-09T10:52:23.020 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ibmpex.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ina209.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ina2xx.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/it87.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/k10temp.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/k8temp.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lineage-pem.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm63.ko 2023-06-09T10:52:23.021 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm73.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm75.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm77.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm78.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm80.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm83.ko 2023-06-09T10:52:23.022 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm85.ko 2023-06-09T10:52:23.023 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm87.ko 2023-06-09T10:52:23.023 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm90.ko 2023-06-09T10:52:23.023 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm92.ko 2023-06-09T10:52:23.023 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm93.ko 2023-06-09T10:52:23.024 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95234.ko 2023-06-09T10:52:23.024 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95241.ko 2023-06-09T10:52:23.024 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95245.ko 2023-06-09T10:52:23.025 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc2945.ko 2023-06-09T10:52:23.025 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4151.ko 2023-06-09T10:52:23.025 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4215.ko 2023-06-09T10:52:23.026 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4222.ko 2023-06-09T10:52:23.026 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4245.ko 2023-06-09T10:52:23.026 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4260.ko 2023-06-09T10:52:23.026 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4261.ko 2023-06-09T10:52:23.027 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max16065.ko 2023-06-09T10:52:23.027 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max1619.ko 2023-06-09T10:52:23.027 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max1668.ko 2023-06-09T10:52:23.028 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max197.ko 2023-06-09T10:52:23.028 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6639.ko 2023-06-09T10:52:23.028 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6650.ko 2023-06-09T10:52:23.029 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6697.ko 2023-06-09T10:52:23.029 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/mcp3021.ko 2023-06-09T10:52:23.029 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6683.ko 2023-06-09T10:52:23.029 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6775-core.ko 2023-06-09T10:52:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6775.ko 2023-06-09T10:52:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct7802.ko 2023-06-09T10:52:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pc87360.ko 2023-06-09T10:52:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pc87427.ko 2023-06-09T10:52:23.030 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pcf8591.ko 2023-06-09T10:52:23.031 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus 2023-06-09T10:52:23.031 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-06-09T10:52:23.031 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-06-09T10:52:23.031 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-06-09T10:52:23.032 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max16064.ko 2023-06-09T10:52:23.032 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max34440.ko 2023-06-09T10:52:23.032 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max8688.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-06-09T10:52:23.033 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/powr1220.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch5627.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch5636.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch56xx-common.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sht15.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sht21.ko 2023-06-09T10:52:23.034 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/shtc1.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sis5595.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47b397.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47m1.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47m192.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/thmc50.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp102.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp103.ko 2023-06-09T10:52:23.035 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp401.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp421.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/via-cputemp.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/via686a.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/vt1211.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/vt8231.ko 2023-06-09T10:52:23.036 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83627ehf.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83627hf.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83781d.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83791d.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83792d.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83793.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83795.ko 2023-06-09T10:52:23.037 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83l785ts.ko 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83l786ng.ko 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses 2023-06-09T10:52:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-06-09T10:52:23.039 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-i801.ko 2023-06-09T10:52:23.040 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-isch.ko 2023-06-09T10:52:23.040 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-06-09T10:52:23.040 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-06-09T10:52:23.040 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-06-09T10:52:23.040 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-parport.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-via.ko 2023-06-09T10:52:23.041 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-dev.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-mux.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-smbus.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-stub.ko 2023-06-09T10:52:23.042 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_cm.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_core.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_umad.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/iw_cm.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/rdma_cm.ko 2023-06-09T10:52:23.043 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/cxgb4 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx4 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx5 2023-06-09T10:52:23.044 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mthca 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/ipoib 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/iser 2023-06-09T10:52:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/isert 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srp 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srpt 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-06-09T10:52:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/fm801-gp.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/gameport.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/lightning.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/ns558.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/joydev.ko 2023-06-09T10:52:23.047 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/keyboard 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/keyboard/gpio_keys.ko 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/appletouch.ko 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/bcm5974.ko 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/cyapatp.ko 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/elan_i2c.ko 2023-06-09T10:52:23.048 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/sermouse.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/synaptics_usb.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/vsxxxaa.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/rmi4 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/rmi4/rmi_core.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/altera_ps2.ko 2023-06-09T10:52:23.049 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/arc_ps2.ko 2023-06-09T10:52:23.050 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-06-09T10:52:23.050 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/serio_raw.ko 2023-06-09T10:52:23.050 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/sparse-keymap.ko 2023-06-09T10:52:23.050 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu 2023-06-09T10:52:23.051 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu/amd 2023-06-09T10:52:23.051 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu/amd/iommu_v2.ko 2023-06-09T10:52:23.051 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds 2023-06-09T10:52:23.051 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-blinkm.ko 2023-06-09T10:52:23.052 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-lm3530.ko 2023-06-09T10:52:23.052 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-lp3944.ko 2023-06-09T10:52:23.052 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-ss4200.ko 2023-06-09T10:52:23.052 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger 2023-06-09T10:52:23.053 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-06-09T10:52:23.053 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-06-09T10:52:23.053 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-06-09T10:52:23.054 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-06-09T10:52:23.054 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-06-09T10:52:23.054 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-06-09T10:52:23.054 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-06-09T10:52:23.055 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-06-09T10:52:23.055 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md 2023-06-09T10:52:23.055 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/bcache 2023-06-09T10:52:23.055 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/bcache/bcache.ko 2023-06-09T10:52:23.056 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-bio-prison.ko 2023-06-09T10:52:23.056 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-cache-smq.ko 2023-06-09T10:52:23.056 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-cache.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-crypt.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-delay.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-flakey.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-log-userspace.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-multipath.ko 2023-06-09T10:52:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-queue-length.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-raid.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-round-robin.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-service-time.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-switch.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-thin-pool.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-verity.ko 2023-06-09T10:52:23.058 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/faulty.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/linear.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/multipath.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/persistent-data 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid0.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid1.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid10.ko 2023-06-09T10:52:23.059 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid456.ko 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptbase.ko 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptctl.ko 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptfc.ko 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptlan.ko 2023-06-09T10:52:23.060 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptsas.ko 2023-06-09T10:52:23.061 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptscsih.ko 2023-06-09T10:52:23.061 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptspi.ko 2023-06-09T10:52:23.061 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd 2023-06-09T10:52:23.062 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/lpc_ich.ko 2023-06-09T10:52:23.062 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/lpc_sch.ko 2023-06-09T10:52:23.062 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/mfd-core.ko 2023-06-09T10:52:23.062 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/sm501.ko 2023-06-09T10:52:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/viperboard.ko 2023-06-09T10:52:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/vx855.ko 2023-06-09T10:52:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/wl1273-core.ko 2023-06-09T10:52:23.063 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc 2023-06-09T10:52:23.064 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/altera-stapl 2023-06-09T10:52:23.064 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-06-09T10:52:23.064 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/apds9802als.ko 2023-06-09T10:52:23.065 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/apds990x.ko 2023-06-09T10:52:23.065 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/bh1770glc.ko 2023-06-09T10:52:23.065 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/cb710 2023-06-09T10:52:23.065 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/cb710/cb710.ko 2023-06-09T10:52:23.066 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom 2023-06-09T10:52:23.066 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/at24.ko 2023-06-09T10:52:23.066 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/eeprom.ko 2023-06-09T10:52:23.066 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-06-09T10:52:23.067 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/max6875.ko 2023-06-09T10:52:23.067 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/enclosure.ko 2023-06-09T10:52:23.067 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/hpilo.ko 2023-06-09T10:52:23.068 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/isl29003.ko 2023-06-09T10:52:23.068 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/isl29020.ko 2023-06-09T10:52:23.068 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d 2023-06-09T10:52:23.068 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-06-09T10:52:23.069 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-06-09T10:52:23.069 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei 2023-06-09T10:52:23.069 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei-me.ko 2023-06-09T10:52:23.069 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei-txe.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-gru 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-gru/gru.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xp.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xpc.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-06-09T10:52:23.070 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tifm_7xx1.ko 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tifm_core.ko 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tsl2550.ko 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_balloon.ko 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_vmci 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-06-09T10:52:23.071 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/bonding 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/bonding/bonding.ko 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/b53 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/b53/b53_common.ko 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/bcm-sf2.ko 2023-06-09T10:52:23.072 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6060.ko 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6xxx 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dummy.ko 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/eql.ko 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com 2023-06-09T10:52:23.073 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/8390.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-06-09T10:52:23.074 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/adaptec 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/agere 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/agere/et131x.ko 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/alteon 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-06-09T10:52:23.075 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/altera 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-06-09T10:52:23.076 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/alx 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1c 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1e 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-06-09T10:52:23.077 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-06-09T10:52:23.078 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/genet 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade/bna 2023-06-09T10:52:23.079 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-06-09T10:52:23.080 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco/enic 2023-06-09T10:52:23.081 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-06-09T10:52:23.082 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dnet.ko 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex/benet 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-06-09T10:52:23.083 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ethoc.ko 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/fealnx.ko 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e100.ko 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000e 2023-06-09T10:52:23.084 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/fm10k 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/i40e 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/iavf 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igb 2023-06-09T10:52:23.085 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igbvf 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbe 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbevf 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/jme.ko 2023-06-09T10:52:23.086 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/skge.ko 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-06-09T10:52:23.087 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/micrel 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom/myri10ge 2023-06-09T10:52:23.088 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/neterion 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/nvidia 2023-06-09T10:52:23.089 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/netxen 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-06-09T10:52:23.090 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rdc 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/atp.ko 2023-06-09T10:52:23.091 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rocker 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sfc 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/silan 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-06-09T10:52:23.092 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis/sis190.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis/sis900.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-06-09T10:52:23.093 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/cassini.ko 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/niu.ko 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/sungem.ko 2023-06-09T10:52:23.094 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/tehuti 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ti 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ti/tlan.ko 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-06-09T10:52:23.095 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/xircom 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/geneve.ko 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/hyperv 2023-06-09T10:52:23.096 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ieee802154 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ieee802154/fakelb.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ifb.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ipvlan 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ipvlan/ipvlan.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/macvlan.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/macvtap.ko 2023-06-09T10:52:23.097 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mii.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/net_failover.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/netconsole.ko 2023-06-09T10:52:23.098 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/nlmon.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/amd.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-06-09T10:52:23.099 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm7xxx.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm87xx.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/broadcom.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/cicada.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/davicom.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/dp83640.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/et1011c.ko 2023-06-09T10:52:23.100 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/icplus.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/lxt.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/marvell.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/micrel.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/national.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/phylink.ko 2023-06-09T10:52:23.101 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/qsemi.ko 2023-06-09T10:52:23.102 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/realtek.ko 2023-06-09T10:52:23.102 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/smsc.ko 2023-06-09T10:52:23.102 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/ste10Xp.ko 2023-06-09T10:52:23.103 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/vitesse.ko 2023-06-09T10:52:23.103 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp 2023-06-09T10:52:23.103 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/bsd_comp.ko 2023-06-09T10:52:23.103 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_async.ko 2023-06-09T10:52:23.104 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_deflate.ko 2023-06-09T10:52:23.104 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_generic.ko 2023-06-09T10:52:23.104 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_mppe.ko 2023-06-09T10:52:23.105 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_synctty.ko 2023-06-09T10:52:23.105 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pppoe.ko 2023-06-09T10:52:23.105 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pppox.ko 2023-06-09T10:52:23.105 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pptp.ko 2023-06-09T10:52:23.106 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip 2023-06-09T10:52:23.106 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip/slhc.ko 2023-06-09T10:52:23.106 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip/slip.ko 2023-06-09T10:52:23.106 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/sungem_phy.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/tap.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_activebackup.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_broadcast.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_random.ko 2023-06-09T10:52:23.107 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/tun.ko 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/veth.ko 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/virtio_net.ko 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vmxnet3 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vxlan 2023-06-09T10:52:23.108 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vxlan/vxlan.ko 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netback 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netback/xen-netback.ko 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netfront.ko 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ntb 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ntb/ntb.ko 2023-06-09T10:52:23.109 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host/nvme-core.ko 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host/nvme.ko 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport.ko 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_cs.ko 2023-06-09T10:52:23.110 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_pc.ko 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_serial.ko 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/hotplug 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/pcie 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/pcie/aer_inject.ko 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/xen-pcifront.ko 2023-06-09T10:52:23.111 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/i82092.ko 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/pd6729.ko 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/yenta_socket.ko 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl/intel 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-06-09T10:52:23.112 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/acer-wmi.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/acerhdf.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/apple-gmux.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-laptop.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-wmi.ko 2023-06-09T10:52:23.113 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/classmate-laptop.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/hdaps.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-06-09T10:52:23.114 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel_ips.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/msi-wmi.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/mxm-wmi.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/samsung-laptop.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/samsung-q10.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/topstar-laptop.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-06-09T10:52:23.115 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/toshiba_haps.ko 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/wmi-bmof.ko 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/wmi.ko 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap/intel_rapl_common.ko 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap/intel_rapl_msr.ko 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps 2023-06-09T10:52:23.116 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps-gpio.ko 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps-ldisc.ko 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps_parport.ko 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/pps_core.ko 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp/ptp.ko 2023-06-09T10:52:23.117 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp/ptp_kvm.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-bq32k.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-bq4802.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1286.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1307.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1374.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1511.ko 2023-06-09T10:52:23.118 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1553.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1672.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1742.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds2404.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds3232.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-em3027.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-fm3130.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-isl12022.ko 2023-06-09T10:52:23.119 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-isl1208.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m41t80.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m48t35.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m48t59.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-max6900.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-msm6242.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf2127.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf85063.ko 2023-06-09T10:52:23.120 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8523.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8563.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8583.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rp5c01.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rs5c372.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rx8025.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rx8581.ko 2023-06-09T10:52:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-x1205.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-9xxx.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-sas.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-xxxx.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/BusLogic.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/a100u2w.ko 2023-06-09T10:52:23.122 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aacraid 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aacraid/aacraid.ko 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/advansys.ko 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/am53c974.ko 2023-06-09T10:52:23.123 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/arcmsr 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/atp870u.ko 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/be2iscsi 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bfa 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bfa/bfa.ko 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2fc 2023-06-09T10:52:23.124 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2i 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ch.ko 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/csiostor 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/csiostor/csiostor.ko 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi 2023-06-09T10:52:23.125 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb3i 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb4i 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/dc395x.ko 2023-06-09T10:52:23.126 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/dmx3191d.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esas2r 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esas2r/esas2r.ko 2023-06-09T10:52:23.127 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esp_scsi.ko 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe/fcoe.ko 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fnic 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fnic/fnic.ko 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hpsa.ko 2023-06-09T10:52:23.128 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hptiop.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hv_storvsc.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/initio.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ipr.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ips.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/isci 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/isci/isci.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-06-09T10:52:23.129 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/iscsi_tcp.ko 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libfc 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libfc/libfc.ko 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libiscsi.ko 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libiscsi_tcp.ko 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libsas 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libsas/libsas.ko 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/lpfc 2023-06-09T10:52:23.130 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/lpfc/lpfc.ko 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid.ko 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mpt3sas 2023-06-09T10:52:23.131 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvsas 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvsas/mvsas.ko 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvumi.ko 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pm8001 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-06-09T10:52:23.132 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pmcraid.ko 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla1280.ko 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla4xxx 2023-06-09T10:52:23.133 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/raid_class.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_debug.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_fc.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_sas.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_spi.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_srp.ko 2023-06-09T10:52:23.134 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ses.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/st.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/stex.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/sym53c8xx_2 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/virtio_scsi.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/vmw_pvscsi.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/wd719x.ko 2023-06-09T10:52:23.135 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ssb 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ssb/ssb.ko 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/iscsi 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/loopback 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/loopback/tcm_loop.ko 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_file.ko 2023-06-09T10:52:23.136 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_iblock.ko 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_mod.ko 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_pscsi.ko 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_user.ko 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/tcm_fc 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal 2023-06-09T10:52:23.137 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-06-09T10:52:23.138 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-06-09T10:52:23.139 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/ipwireless 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/n_gsm.ko 2023-06-09T10:52:23.140 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/n_hdlc.ko 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/nozomi.ko 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/8250 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/arc_uart.ko 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/jsm 2023-06-09T10:52:23.141 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/jsm/jsm.ko 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/synclink_gt.ko 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/core 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/core/ufshcd-core.ko 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/host 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio 2023-06-09T10:52:23.142 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio.ko 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_aec.ko 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_cif.ko 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_pci_generic.ko 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_sercos3.ko 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb 2023-06-09T10:52:23.143 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class 2023-06-09T10:52:23.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/cdc-acm.ko 2023-06-09T10:52:23.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/cdc-wdm.ko 2023-06-09T10:52:23.144 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/usblp.ko 2023-06-09T10:52:23.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/usbtmc.ko 2023-06-09T10:52:23.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/host 2023-06-09T10:52:23.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/host/sl811-hcd.ko 2023-06-09T10:52:23.145 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image 2023-06-09T10:52:23.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image/mdc800.ko 2023-06-09T10:52:23.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image/microtek.ko 2023-06-09T10:52:23.146 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc 2023-06-09T10:52:23.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/adutux.ko 2023-06-09T10:52:23.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/appledisplay.ko 2023-06-09T10:52:23.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/emi26.ko 2023-06-09T10:52:23.147 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/emi62.ko 2023-06-09T10:52:23.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/ezusb.ko 2023-06-09T10:52:23.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/idmouse.ko 2023-06-09T10:52:23.148 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/iowarrior.ko 2023-06-09T10:52:23.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/isight_firmware.ko 2023-06-09T10:52:23.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/ldusb.ko 2023-06-09T10:52:23.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/legousbtower.ko 2023-06-09T10:52:23.149 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/sisusbvga 2023-06-09T10:52:23.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-06-09T10:52:23.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/trancevibrator.ko 2023-06-09T10:52:23.150 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usb3503.ko 2023-06-09T10:52:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usblcd.ko 2023-06-09T10:52:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usbsevseg.ko 2023-06-09T10:52:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/uss720.ko 2023-06-09T10:52:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/yurex.ko 2023-06-09T10:52:23.151 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/phy 2023-06-09T10:52:23.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/phy/phy-generic.ko 2023-06-09T10:52:23.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial 2023-06-09T10:52:23.152 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/aircable.ko 2023-06-09T10:52:23.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ark3116.ko 2023-06-09T10:52:23.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/belkin_sa.ko 2023-06-09T10:52:23.156 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ch341.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cp210x.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cyberjack.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cypress_m8.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/digi_acceleport.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/empeg.ko 2023-06-09T10:52:23.157 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ftdi_sio.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/garmin_gps.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/io_edgeport.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/io_ti.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ipaq.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ipw.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ir-usb.ko 2023-06-09T10:52:23.158 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/keyspan.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/keyspan_pda.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/kl5kusb105.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/kobil_sct.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mct_u232.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mos7720.ko 2023-06-09T10:52:23.159 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mos7840.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/navman.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/omninet.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/opticon.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/option.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/oti6858.ko 2023-06-09T10:52:23.160 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/pl2303.ko 2023-06-09T10:52:23.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/qcaux.ko 2023-06-09T10:52:23.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/qcserial.ko 2023-06-09T10:52:23.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/quatech2.ko 2023-06-09T10:52:23.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/safe_serial.ko 2023-06-09T10:52:23.161 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/sierra.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/spcp8x5.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ssu100.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/symbolserial.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb_debug.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb_wwan.ko 2023-06-09T10:52:23.162 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/visor.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/whiteheat.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/xsens_mt.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/uas.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-alauda.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-cypress.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-datafab.ko 2023-06-09T10:52:23.163 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-freecom.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-isd200.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-karma.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-onetouch.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-realtek.ko 2023-06-09T10:52:23.164 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-sddr09.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-sddr55.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-usbat.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/usb-storage.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/usbip-core.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/usbip-host.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-06-09T10:52:23.165 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci/vfio-pci.ko 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/vfio.ko 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost.ko 2023-06-09T10:52:23.166 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_iotlb.ko 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_net.ko 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_scsi.ko 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/apple_bl.ko 2023-06-09T10:52:23.167 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/lcd.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/platform_lcd.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_balloon.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_mmio.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci.ko 2023-06-09T10:52:23.168 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_ring.ko 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters/ds2482.ko 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters/ds2490.ko 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves 2023-06-09T10:52:23.169 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-06-09T10:52:23.170 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_smem.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_therm.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/wire.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/alim1535_wdt.ko 2023-06-09T10:52:23.171 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/alim7101_wdt.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/f71808e_wdt.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/hpwdt.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/i6300esb.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/iTCO_wdt.ko 2023-06-09T10:52:23.172 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ib700wdt.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ibmasr.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/it8712f_wdt.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/it87_wdt.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/machzwd.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/nv_tco.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/pcwd_pci.ko 2023-06-09T10:52:23.173 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/pcwd_usb.ko 2023-06-09T10:52:23.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-06-09T10:52:23.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sch311x_wdt.ko 2023-06-09T10:52:23.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/softdog.ko 2023-06-09T10:52:23.174 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sp5100_tco.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/via_wdt.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83877f_wdt.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83977f_wdt.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/wdt_pci.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/xen_wdt.ko 2023-06-09T10:52:23.175 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-acpi-processor.ko 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-evtchn.ko 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-gntalloc.ko 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-gntdev.ko 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-pciback 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-06-09T10:52:23.176 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-privcmd.ko 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xenfs 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xenfs/xenfs.ko 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/9p 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/9p/9p.ko 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/affs 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/affs/affs.ko 2023-06-09T10:52:23.177 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/befs 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/befs/befs.ko 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/binfmt_misc.ko 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/btrfs 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/btrfs/btrfs.ko 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cachefiles 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cachefiles/cachefiles.ko 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ceph 2023-06-09T10:52:23.178 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ceph/ceph.ko 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cifs 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cifs/cifs.ko 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/coda 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/coda/coda.ko 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cramfs 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cramfs/cramfs.ko 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/dlm 2023-06-09T10:52:23.179 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/dlm/dlm.ko 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ecryptfs 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ecryptfs/ecryptfs.ko 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/f2fs 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/f2fs/f2fs.ko 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat 2023-06-09T10:52:23.180 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/fat.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/msdos.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/vfat.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fscache 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fscache/fscache.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse/cuse.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse/fuse.ko 2023-06-09T10:52:23.181 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/gfs2 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/gfs2/gfs2.ko 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfs 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfs/hfs.ko 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfsplus 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfsplus/hfsplus.ko 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/isofs 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/isofs/isofs.ko 2023-06-09T10:52:23.182 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/jfs 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/jfs/jfs.ko 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/lockd 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/lockd/lockd.ko 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/minix 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/minix/minix.ko 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/netfs 2023-06-09T10:52:23.183 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/netfs/netfs.ko 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/blocklayout 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/filelayout 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/flexfilelayout 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-06-09T10:52:23.184 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfs.ko 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfsv3.ko 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfsv4.ko 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common/grace.ko 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common/nfs_acl.ko 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfsd 2023-06-09T10:52:23.185 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfsd/nfsd.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nilfs2 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nilfs2/nilfs2.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-celtic.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-centeuro.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-croatian.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-cyrillic.ko 2023-06-09T10:52:23.186 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-gaelic.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-greek.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-iceland.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-inuit.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-roman.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-romanian.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-turkish.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1250.ko 2023-06-09T10:52:23.187 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1251.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1255.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp737.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp775.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp850.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp852.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp855.ko 2023-06-09T10:52:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp857.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp860.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp861.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp862.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp863.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp864.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp865.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp866.ko 2023-06-09T10:52:23.189 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp869.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp874.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp932.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp936.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp949.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp950.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_euc-jp.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-1.ko 2023-06-09T10:52:23.190 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-13.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-14.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-15.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-2.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-3.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-4.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-5.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-6.ko 2023-06-09T10:52:23.191 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-7.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-9.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-r.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-ru.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-u.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_utf8.ko 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/cluster 2023-06-09T10:52:23.192 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlm 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlmfs 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2.ko 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-06-09T10:52:23.193 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/overlayfs 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/overlayfs/overlay.ko 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/pstore 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/pstore/ramoops.ko 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/reiserfs 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/reiserfs/reiserfs.ko 2023-06-09T10:52:23.194 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/romfs 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/romfs/romfs.ko 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common/cifs_arc4.ko 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common/cifs_md4.ko 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/squashfs 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/squashfs/squashfs.ko 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/sysv 2023-06-09T10:52:23.195 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/sysv/sysv.ko 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/udf 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/udf/udf.ko 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ufs 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ufs/ufs.ko 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/xfs 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/xfs/xfs.ko 2023-06-09T10:52:23.196 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/rcu 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/rcu/rcutorture.ko 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/torture.ko 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/trace 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/asn1_encoder.ko 2023-06-09T10:52:23.197 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crc-itu-t.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crc8.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto/libarc4.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto/libdes.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/libcrc32c.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lru_cache.ko 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4 2023-06-09T10:52:23.198 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4/lz4_compress.ko 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4/lz4hc_compress.ko 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/math 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/math/cordic.ko 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/raid6 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/raid6/raid6_pq.ko 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/reed_solomon 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/reed_solomon/reed_solomon.ko 2023-06-09T10:52:23.199 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_bm.ko 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_fsm.ko 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_kmp.ko 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/zstd 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/zstd/zstd_compress.ko 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/mm 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/mm/hwpoison-inject.ko 2023-06-09T10:52:23.200 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net 2023-06-09T10:52:23.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan 2023-06-09T10:52:23.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/6lowpan.ko 2023-06-09T10:52:23.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_dest.ko 2023-06-09T10:52:23.201 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_fragment.ko 2023-06-09T10:52:23.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_hop.ko 2023-06-09T10:52:23.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_ipv6.ko 2023-06-09T10:52:23.202 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_mobility.ko 2023-06-09T10:52:23.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_routing.ko 2023-06-09T10:52:23.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_udp.ko 2023-06-09T10:52:23.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802 2023-06-09T10:52:23.203 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/garp.ko 2023-06-09T10:52:23.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/mrp.ko 2023-06-09T10:52:23.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/p8022.ko 2023-06-09T10:52:23.204 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/psnap.ko 2023-06-09T10:52:23.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/stp.ko 2023-06-09T10:52:23.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/8021q 2023-06-09T10:52:23.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/8021q/8021q.ko 2023-06-09T10:52:23.205 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p 2023-06-09T10:52:23.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet.ko 2023-06-09T10:52:23.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_fd.ko 2023-06-09T10:52:23.206 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_rdma.ko 2023-06-09T10:52:23.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_virtio.ko 2023-06-09T10:52:23.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/batman-adv 2023-06-09T10:52:23.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/batman-adv/batman-adv.ko 2023-06-09T10:52:23.207 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge 2023-06-09T10:52:23.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/br_netfilter.ko 2023-06-09T10:52:23.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/bridge.ko 2023-06-09T10:52:23.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter 2023-06-09T10:52:23.208 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-06-09T10:52:23.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_among.ko 2023-06-09T10:52:23.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_arp.ko 2023-06-09T10:52:23.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-06-09T10:52:23.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-06-09T10:52:23.209 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_ip.ko 2023-06-09T10:52:23.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-06-09T10:52:23.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_limit.ko 2023-06-09T10:52:23.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_log.ko 2023-06-09T10:52:23.210 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_mark.ko 2023-06-09T10:52:23.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-06-09T10:52:23.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-06-09T10:52:23.211 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_snat.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_stp.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-06-09T10:52:23.212 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtables.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ceph 2023-06-09T10:52:23.213 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ceph/libceph.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core/failover.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core/pktgen.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_diag.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_ipv4.ko 2023-06-09T10:52:23.214 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_ipv6.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dns_resolver 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dns_resolver/dns_resolver.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/dsa_core.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_brcm.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_dsa.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_none.ko 2023-06-09T10:52:23.215 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_trailer.ko 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/6lowpan 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/ieee802154.ko 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/ieee802154_socket.ko 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4 2023-06-09T10:52:23.216 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ah4.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/esp4.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/fou.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/gre.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/inet_diag.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_gre.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_tunnel.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_vti.ko 2023-06-09T10:52:23.217 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ipcomp.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ipip.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arp_tables.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-06-09T10:52:23.218 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_security.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-06-09T10:52:23.219 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-06-09T10:52:23.220 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_bic.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_dctcp.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_diag.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_highspeed.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_htcp.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_hybla.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_illinois.ko 2023-06-09T10:52:23.221 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_lp.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_scalable.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_vegas.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_veno.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_westwood.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_yeah.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tunnel4.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/udp_diag.ko 2023-06-09T10:52:23.222 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/udp_tunnel.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/xfrm4_tunnel.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ah6.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/esp6.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/fou6.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_tunnel.ko 2023-06-09T10:52:23.223 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_vti.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ipcomp6.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-06-09T10:52:23.224 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-06-09T10:52:23.225 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-06-09T10:52:23.226 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-06-09T10:52:23.227 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/sit.ko 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/tunnel6.ko 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/xfrm6_tunnel.ko 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/key 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/key/af_key.ko 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp 2023-06-09T10:52:23.228 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_core.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_debugfs.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_eth.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ip.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ip6.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_netlink.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ppp.ko 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/llc 2023-06-09T10:52:23.229 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/llc/llc.ko 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mac802154 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mac802154/mac802154.ko 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mpls 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mpls/mpls_gso.ko 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set.ko 2023-06-09T10:52:23.230 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-06-09T10:52:23.231 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-06-09T10:52:23.232 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-06-09T10:52:23.233 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conncount.ko 2023-06-09T10:52:23.234 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_h323.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_irc.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-06-09T10:52:23.235 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_sane.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_sip.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_dup_netdev.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_flow_table.ko 2023-06-09T10:52:23.236 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_flow_table_inet.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_log_syslog.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_amanda.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_ftp.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_irc.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_sip.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_tftp.ko 2023-06-09T10:52:23.237 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_synproxy_core.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_tables.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_acct.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_log.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_osf.ko 2023-06-09T10:52:23.238 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_queue.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_chain_nat.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_compat.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_ct.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_dup_netdev.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_fib.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_fwd_netdev.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_hash.ko 2023-06-09T10:52:23.239 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_limit.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_log.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_masq.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_nat.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_numgen.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_queue.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_quota.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_redir.ko 2023-06-09T10:52:23.240 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_reject.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_reject_inet.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_socket.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_tunnel.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_AUDIT.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CHECKSUM.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CLASSIFY.ko 2023-06-09T10:52:23.241 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CT.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_DSCP.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_HL.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_HMARK.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_IDLETIMER.ko 2023-06-09T10:52:23.242 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_LED.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_LOG.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_MASQUERADE.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NETMAP.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NFLOG.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NFQUEUE.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_RATEEST.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_REDIRECT.ko 2023-06-09T10:52:23.243 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_SECMARK.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TCPMSS.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TEE.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TPROXY.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TRACE.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_addrtype.ko 2023-06-09T10:52:23.244 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_bpf.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cgroup.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cluster.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_comment.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connbytes.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connlabel.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connlimit.ko 2023-06-09T10:52:23.245 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connmark.ko 2023-06-09T10:52:23.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_conntrack.ko 2023-06-09T10:52:23.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cpu.ko 2023-06-09T10:52:23.246 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_dccp.ko 2023-06-09T10:52:23.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_devgroup.ko 2023-06-09T10:52:23.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_dscp.ko 2023-06-09T10:52:23.247 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ecn.ko 2023-06-09T10:52:23.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_esp.ko 2023-06-09T10:52:23.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_hashlimit.ko 2023-06-09T10:52:23.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_helper.ko 2023-06-09T10:52:23.248 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_hl.ko 2023-06-09T10:52:23.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ipcomp.ko 2023-06-09T10:52:23.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_iprange.ko 2023-06-09T10:52:23.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ipvs.ko 2023-06-09T10:52:23.249 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_l2tp.ko 2023-06-09T10:52:23.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_length.ko 2023-06-09T10:52:23.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_limit.ko 2023-06-09T10:52:23.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_mac.ko 2023-06-09T10:52:23.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_mark.ko 2023-06-09T10:52:23.250 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_multiport.ko 2023-06-09T10:52:23.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_nat.ko 2023-06-09T10:52:23.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_nfacct.ko 2023-06-09T10:52:23.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_osf.ko 2023-06-09T10:52:23.251 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_owner.ko 2023-06-09T10:52:23.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_physdev.ko 2023-06-09T10:52:23.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_pkttype.ko 2023-06-09T10:52:23.252 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_policy.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_quota.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_rateest.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_realm.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_recent.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_sctp.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_set.ko 2023-06-09T10:52:23.253 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_socket.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_state.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_statistic.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_string.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_tcpmss.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_time.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_u32.ko 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netlink 2023-06-09T10:52:23.254 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netlink/netlink_diag.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/nsh 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/nsh/nsh.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/openvswitch.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-geneve.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-gre.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-vxlan.ko 2023-06-09T10:52:23.255 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/packet 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/packet/af_packet_diag.ko 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds.ko 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds_rdma.ko 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds_tcp.ko 2023-06-09T10:52:23.256 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_csum.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_gact.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_ipt.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_mirred.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_nat.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_pedit.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_police.ko 2023-06-09T10:52:23.257 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_simple.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_skbedit.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_vlan.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_basic.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_bpf.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_flow.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_fw.ko 2023-06-09T10:52:23.258 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_route.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_u32.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_cmp.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_ipset.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_meta.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_nbyte.ko 2023-06-09T10:52:23.259 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_text.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_u32.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_choke.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_codel.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_drr.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_fq.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_gred.ko 2023-06-09T10:52:23.260 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_hfsc.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_hhf.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_htb.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_ingress.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_mqprio.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_mqprio_lib.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_multiq.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_netem.ko 2023-06-09T10:52:23.261 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_pie.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_plug.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_prio.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_qfq.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_red.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_sfb.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_sfq.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_tbf.ko 2023-06-09T10:52:23.262 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_teql.ko 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp/sctp.ko 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp/sctp_diag.ko 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-06-09T10:52:23.263 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/sunrpc.ko 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/xprtrdma 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc/diag.ko 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc/tipc.ko 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/unix 2023-06-09T10:52:23.264 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/unix/unix_diag.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock_diag.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock_loopback.ko 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/xfrm 2023-06-09T10:52:23.265 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/xfrm/xfrm_ipcomp.ko 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/encrypted-keys 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/trusted-keys 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/trusted-keys/trusted.ko 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt 2023-06-09T10:52:23.266 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt/lib 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt/lib/irqbypass.ko 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.alias 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.alias.bin 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin.bin 2023-06-09T10:52:23.267 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin.modinfo 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.dep 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.dep.bin 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.devname 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.order 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.softdep 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.symbols 2023-06-09T10:52:23.268 INFO:teuthology.orchestra.run.smithi117.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.symbols.bin 2023-06-09T10:52:23.269 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:23.270 DEBUG:teuthology.orchestra.run.smithi117:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-06-09T10:52:23.294 INFO:teuthology.orchestra.run.smithi117.stdout:postinstall scriptlet (using /bin/sh): 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 -a -r /boot/System.map-6.4.0-rc2-g10b2f20a4f46 ]; then 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:cp /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:cp /boot/System.map-6.4.0-rc2-g10b2f20a4f46 /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:rm -f /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 /boot/System.map-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/installkernel 6.4.0-rc2-g10b2f20a4f46 /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:23.295 INFO:teuthology.orchestra.run.smithi117.stdout:rm -f /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:preuninstall scriptlet (using /bin/sh): 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:new-kernel-pkg --remove 6.4.0-rc2-g10b2f20a4f46 --rminitrd --initrdfile=/boot/initramfs-6.4.0-rc2-g10b2f20a4f46.img 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-install remove 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:23.296 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2023-06-09T10:52:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:postuninstall scriptlet (using /bin/sh): 2023-06-09T10:52:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:if [ -x /sbin/update-bootloader ]; then 2023-06-09T10:52:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:/sbin/update-bootloader --remove 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:23.297 INFO:teuthology.orchestra.run.smithi117.stdout:fi 2023-06-09T10:52:23.298 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2023-06-09T10:52:23.375 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2023-06-09T10:52:23.376 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2023-06-09T10:52:23.376 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.02 2023-06-09T10:52:23.376 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 123.el8 2023-06-09T10:52:23.377 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2023-06-09T10:52:23.377 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-06-09T10:52:23.377 INFO:teuthology.orchestra.run.smithi117.stdout:Group : System Environment/Base 2023-06-09T10:52:23.377 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 9265450 2023-06-09T10:52:23.377 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2023-06-09T10:52:23.378 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-06-09T10:52:23.378 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-06-09T10:52:23.378 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-06-09T10:52:23.378 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-06-09T10:52:23.379 INFO:teuthology.orchestra.run.smithi117.stdout:Relocations : (not relocatable) 2023-06-09T10:52:23.379 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : Red Hat, Inc. 2023-06-09T10:52:23.379 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : Red Hat, Inc. 2023-06-09T10:52:23.380 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-09T10:52:23.380 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2023-06-09T10:52:23.380 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2023-06-09T10:52:23.380 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T10:52:23.380 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-09T10:52:23.381 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-09T10:52:23.381 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-09T10:52:23.381 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2023-06-09T10:52:23.381 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T10:52:23.381 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2023-06-09T10:52:23.382 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-09T10:52:23.383 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:23.383 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-09T10:52:27.555 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2023-06-09T10:52:29.750 INFO:teuthology.orchestra.run.smithi117.stderr:done 2023-06-09T10:52:29.752 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2023-06-09T10:52:29.769 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.RKPs8LXrZD 2023-06-09T10:52:29.770 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RKPs8LXrZD 2023-06-09T10:52:29.855 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.RKPs8LXrZD 2023-06-09T10:52:29.958 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.RKPs8LXrZD is 6KB 2023-06-09T10:52:30.005 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.RKPs8LXrZD 2023-06-09T10:52:30.021 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2023-06-09T10:52:30.086 INFO:teuthology.orchestra.run.smithi117.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-06-09T10:52:30.086 INFO:teuthology.orchestra.run.smithi117.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-06-09T10:52:30.086 INFO:teuthology.orchestra.run.smithi117.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.4.0-rc2-g10b2f20a4f46.conf 2023-06-09T10:52:30.087 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:30.240 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2023-06-09T10:52:34.066 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-06-09T10:52:34.066 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-06-09T10:52:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.117 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-06-09T10:52:34.118 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-06-09T10:52:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-06-09T10:52:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-06-09T10:52:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crc32c-intel.ko 2023-06-09T10:52:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-06-09T10:52:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-avx2.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-06-09T10:52:34.120 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu 2023-06-09T10:52:34.121 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu/mce 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm-amd.ko 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm-intel.ko 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/arch/x86/kvm/kvm.ko 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/ansi_cprng.ko 2023-06-09T10:52:34.122 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/anubis.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/arc4.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_memcpy.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_pq.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_raid6_recov.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_tx.ko 2023-06-09T10:52:34.123 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/async_xor.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/async_tx/raid6test.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/authenc.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/authencesn.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blake2b_generic.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blowfish_common.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/blowfish_generic.ko 2023-06-09T10:52:34.124 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/camellia_generic.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast5_generic.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast6_generic.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cast_common.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/ccm.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cmac.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crc32_generic.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crypto_engine.ko 2023-06-09T10:52:34.125 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/crypto_user.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/cts.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/des_generic.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/dh_generic.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/echainiv.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/essiv.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/fcrypt.ko 2023-06-09T10:52:34.126 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/khazad.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/lz4.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/lz4hc.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/md4.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/michael_mic.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/pcbc.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/pcrypt.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/rmd160.ko 2023-06-09T10:52:34.127 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/seed.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/serpent_generic.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/tcrypt.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/tea.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/twofish_common.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/twofish_generic.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/vmac.ko 2023-06-09T10:52:34.128 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/wp512.ko 2023-06-09T10:52:34.129 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xcbc.ko 2023-06-09T10:52:34.129 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xor.ko 2023-06-09T10:52:34.129 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/crypto/xxhash_generic.ko 2023-06-09T10:52:34.129 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers 2023-06-09T10:52:34.129 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi 2023-06-09T10:52:34.130 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/acpi_ipmi.ko 2023-06-09T10:52:34.130 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/acpi_pad.ko 2023-06-09T10:52:34.130 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/custom_method.ko 2023-06-09T10:52:34.130 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/ec_sys.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/platform_profile.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/sbs.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/sbshc.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/acpi/video.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/acard-ahci.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/ahci_platform.ko 2023-06-09T10:52:34.131 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/ata_generic.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/libahci_platform.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_acpi.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ali.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_amd.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_artop.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_atiixp.ko 2023-06-09T10:52:34.132 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_atp867x.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cmd640.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cmd64x.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_cypress.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_efar.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt366.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt37x.ko 2023-06-09T10:52:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt3x2n.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_hpt3x3.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_it8213.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_it821x.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_jmicron.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_marvell.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_mpiix.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_netcell.ko 2023-06-09T10:52:34.134 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ninja32.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ns87410.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_ns87415.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_oldpiix.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_opti.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_optidma.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pcmcia.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pdc2027x.ko 2023-06-09T10:52:34.135 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_piccolo.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_rdc.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sch.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_serverworks.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sil680.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sis.ko 2023-06-09T10:52:34.136 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_sl82c105.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_triflex.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pata_via.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/pdc_adma.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_inic162x.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_mv.ko 2023-06-09T10:52:34.137 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_nv.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_promise.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_qstor.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sil.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sil24.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sis.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_svw.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_sx4.ko 2023-06-09T10:52:34.138 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_uli.ko 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_via.ko 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ata/sata_vsc.ko 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/auxdisplay 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/auxdisplay/ks0108.ko 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base/regmap 2023-06-09T10:52:34.139 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/base/regmap/regmap-i2c.ko 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/bcma 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/bcma/bcma.ko 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/aoe 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/aoe/aoe.ko 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/brd.ko 2023-06-09T10:52:34.140 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/drbd 2023-06-09T10:52:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/drbd/drbd.ko 2023-06-09T10:52:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/floppy.ko 2023-06-09T10:52:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/loop.ko 2023-06-09T10:52:34.141 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/mtip32xx 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/nbd.ko 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/null_blk 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/null_blk/null_blk.ko 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/pktcdvd.ko 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/rbd.ko 2023-06-09T10:52:34.142 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/virtio_blk.ko 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkback 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/xen-blkfront.ko 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/zram 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/block/zram/zram.ko 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hangcheck-timer.ko 2023-06-09T10:52:34.143 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/amd-rng.ko 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/intel-rng.ko 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/via-rng.ko 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/hw_random/virtio-rng.ko 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi 2023-06-09T10:52:34.144 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-06-09T10:52:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-06-09T10:52:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-06-09T10:52:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_si.ko 2023-06-09T10:52:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-06-09T10:52:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/lp.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/mwave 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/mwave/mwave.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/ppdev.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tlclk.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm.ko 2023-06-09T10:52:34.146 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_atmel.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_infineon.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_nsc.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_tis.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/uv_mmtimer.ko 2023-06-09T10:52:34.147 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/char/virtio_console.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/p4-clockmod.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/powernow-k8.ko 2023-06-09T10:52:34.148 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/cpufreq/speedstep-lib.ko 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/ccp/ccp.ko 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat 2023-06-09T10:52:34.149 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_common 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/padlock-aes.ko 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/padlock-sha.ko 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/virtio 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-06-09T10:52:34.150 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dca 2023-06-09T10:52:34.151 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dca/dca.ko 2023-06-09T10:52:34.151 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/devfreq 2023-06-09T10:52:34.151 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-06-09T10:52:34.151 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma 2023-06-09T10:52:34.151 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/dw 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/dw/dw_dmac.ko 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/ioat 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/dma/ioat/ioatdma.ko 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/amd64_edac.ko 2023-06-09T10:52:34.152 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/e752x_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/edac_mce_amd.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i3000_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i3200_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i5100_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i5400_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i7300_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i7core_edac.ko 2023-06-09T10:52:34.153 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/i82975x_edac.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/ie31200_edac.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/sb_edac.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/edac/x38_edac.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware/edd.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/firmware/iscsi_ibft.ko 2023-06-09T10:52:34.154 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio/gpio-ich.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/gpio/gpio-viperboard.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-appleir.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-aureal.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-axff.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-dr.ko 2023-06-09T10:52:34.155 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-elecom.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-elo.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-emsff.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gaff.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gt683r.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-gyration.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtek-kbd.ko 2023-06-09T10:52:34.156 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtek-mouse.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-holtekff.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-hyperv.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-icade.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-keytouch.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-kye.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-lcpower.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-led.ko 2023-06-09T10:52:34.157 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-lenovo.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-logitech-dj.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-multitouch.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-ortek.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-penmount.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-petalynx.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-picolcd.ko 2023-06-09T10:52:34.158 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-pl.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-plantronics.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-primax.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-rmi.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-arvo.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-common.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-isku.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-kone.ko 2023-06-09T10:52:34.159 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-konepure.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-lua.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-pyra.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-ryos.ko 2023-06-09T10:52:34.160 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat-savu.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-roccat.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-saitek.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-samsung.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sensor-hub.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sjoy.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sony.ko 2023-06-09T10:52:34.161 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-speedlink.ko 2023-06-09T10:52:34.162 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-steelseries.ko 2023-06-09T10:52:34.162 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-sunplus.ko 2023-06-09T10:52:34.162 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-thrustmaster.ko 2023-06-09T10:52:34.163 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-tivo.ko 2023-06-09T10:52:34.163 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-tmff.ko 2023-06-09T10:52:34.163 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-topseed.ko 2023-06-09T10:52:34.164 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-twinhan.ko 2023-06-09T10:52:34.164 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-uclogic.ko 2023-06-09T10:52:34.164 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-waltop.ko 2023-06-09T10:52:34.165 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-wiimote.ko 2023-06-09T10:52:34.165 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-xinmo.ko 2023-06-09T10:52:34.165 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-zpff.ko 2023-06-09T10:52:34.165 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/hid-zydacron.ko 2023-06-09T10:52:34.166 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/uhid.ko 2023-06-09T10:52:34.166 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hid/wacom.ko 2023-06-09T10:52:34.166 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv 2023-06-09T10:52:34.167 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_balloon.ko 2023-06-09T10:52:34.167 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_utils.ko 2023-06-09T10:52:34.167 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hv/hv_vmbus.ko 2023-06-09T10:52:34.167 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon 2023-06-09T10:52:34.168 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/abituguru.ko 2023-06-09T10:52:34.168 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/abituguru3.ko 2023-06-09T10:52:34.168 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/acpi_power_meter.ko 2023-06-09T10:52:34.169 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ad7414.ko 2023-06-09T10:52:34.169 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ad7418.ko 2023-06-09T10:52:34.169 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adc128d818.ko 2023-06-09T10:52:34.169 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1025.ko 2023-06-09T10:52:34.170 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1026.ko 2023-06-09T10:52:34.170 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1029.ko 2023-06-09T10:52:34.170 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm1031.ko 2023-06-09T10:52:34.170 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adm9240.ko 2023-06-09T10:52:34.171 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ads7828.ko 2023-06-09T10:52:34.171 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7410.ko 2023-06-09T10:52:34.171 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7411.ko 2023-06-09T10:52:34.171 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7462.ko 2023-06-09T10:52:34.171 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7470.ko 2023-06-09T10:52:34.172 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7475.ko 2023-06-09T10:52:34.172 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/adt7x10.ko 2023-06-09T10:52:34.172 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/amc6821.ko 2023-06-09T10:52:34.173 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/applesmc.ko 2023-06-09T10:52:34.173 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asb100.ko 2023-06-09T10:52:34.173 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asc7621.ko 2023-06-09T10:52:34.173 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/asus_atk0110.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/atxp1.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/coretemp.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/dme1737.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ds1621.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ds620.ko 2023-06-09T10:52:34.174 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/emc1403.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/emc6w201.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f71805f.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f71882fg.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/f75375s.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/fam15h_power.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/fschmd.ko 2023-06-09T10:52:34.175 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/g760a.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/g762.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/gl518sm.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/gl520sm.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/hwmon-vid.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/i5500_temp.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/i5k_amb.ko 2023-06-09T10:52:34.176 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ibmaem.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ibmpex.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ina209.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ina2xx.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/it87.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/k10temp.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/k8temp.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lineage-pem.ko 2023-06-09T10:52:34.177 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm63.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm73.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm75.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm77.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm78.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm80.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm83.ko 2023-06-09T10:52:34.178 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm85.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm87.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm90.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm92.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm93.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95234.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95241.ko 2023-06-09T10:52:34.179 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/lm95245.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc2945.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4151.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4215.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4222.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4245.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4260.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/ltc4261.ko 2023-06-09T10:52:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max16065.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max1619.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max1668.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max197.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6639.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6650.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/max6697.ko 2023-06-09T10:52:34.181 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/mcp3021.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6683.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6775-core.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct6775.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/nct7802.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pc87360.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pc87427.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pcf8591.ko 2023-06-09T10:52:34.182 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max16064.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max34440.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/max8688.ko 2023-06-09T10:52:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/powr1220.ko 2023-06-09T10:52:34.184 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch5627.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch5636.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sch56xx-common.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sht15.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sht21.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/shtc1.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/sis5595.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47b397.ko 2023-06-09T10:52:34.185 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47m1.ko 2023-06-09T10:52:34.186 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/smsc47m192.ko 2023-06-09T10:52:34.186 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/thmc50.ko 2023-06-09T10:52:34.186 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp102.ko 2023-06-09T10:52:34.186 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp103.ko 2023-06-09T10:52:34.186 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp401.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/tmp421.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/via-cputemp.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/via686a.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/vt1211.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/vt8231.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83627ehf.ko 2023-06-09T10:52:34.187 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83627hf.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83781d.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83791d.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83792d.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83793.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83795.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83l785ts.ko 2023-06-09T10:52:34.188 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/hwmon/w83l786ng.ko 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-06-09T10:52:34.189 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-06-09T10:52:34.190 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-i801.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-isch.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-parport.ko 2023-06-09T10:52:34.191 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-06-09T10:52:34.192 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-via.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-dev.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-mux.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-smbus.ko 2023-06-09T10:52:34.193 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/i2c/i2c-stub.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_cm.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_core.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_umad.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/iw_cm.ko 2023-06-09T10:52:34.194 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/rdma_cm.ko 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/cxgb4 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx4 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-06-09T10:52:34.195 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx5 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mthca 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/ipoib 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-06-09T10:52:34.196 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/iser 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/isert 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srp 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srpt 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-06-09T10:52:34.197 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/fm801-gp.ko 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/gameport.ko 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/lightning.ko 2023-06-09T10:52:34.198 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/gameport/ns558.ko 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/joydev.ko 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/keyboard 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/keyboard/gpio_keys.ko 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/appletouch.ko 2023-06-09T10:52:34.199 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/bcm5974.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/cyapatp.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/elan_i2c.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/sermouse.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/synaptics_usb.ko 2023-06-09T10:52:34.200 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/mouse/vsxxxaa.ko 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/rmi4 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/rmi4/rmi_core.ko 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/altera_ps2.ko 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/arc_ps2.ko 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-06-09T10:52:34.201 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/serio/serio_raw.ko 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/input/sparse-keymap.ko 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu/amd 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/iommu/amd/iommu_v2.ko 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-blinkm.ko 2023-06-09T10:52:34.202 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-lm3530.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-lp3944.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/leds-ss4200.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-06-09T10:52:34.203 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/bcache 2023-06-09T10:52:34.204 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/bcache/bcache.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-bio-prison.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-cache-smq.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-cache.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-crypt.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-delay.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-flakey.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-log-userspace.ko 2023-06-09T10:52:34.205 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-multipath.ko 2023-06-09T10:52:34.206 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-queue-length.ko 2023-06-09T10:52:34.206 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-raid.ko 2023-06-09T10:52:34.206 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-round-robin.ko 2023-06-09T10:52:34.206 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-service-time.ko 2023-06-09T10:52:34.206 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-switch.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-thin-pool.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/dm-verity.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/faulty.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/linear.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/multipath.ko 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/persistent-data 2023-06-09T10:52:34.207 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid0.ko 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid1.ko 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid10.ko 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/md/raid456.ko 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message 2023-06-09T10:52:34.208 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptbase.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptctl.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptfc.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptlan.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptsas.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptscsih.ko 2023-06-09T10:52:34.209 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/message/fusion/mptspi.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/lpc_ich.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/lpc_sch.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/mfd-core.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/sm501.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/viperboard.ko 2023-06-09T10:52:34.210 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/vx855.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/mfd/wl1273-core.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/altera-stapl 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/apds9802als.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/apds990x.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/bh1770glc.ko 2023-06-09T10:52:34.211 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/cb710 2023-06-09T10:52:34.212 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/cb710/cb710.ko 2023-06-09T10:52:34.212 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom 2023-06-09T10:52:34.212 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/at24.ko 2023-06-09T10:52:34.212 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/eeprom.ko 2023-06-09T10:52:34.212 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/eeprom/max6875.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/enclosure.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/hpilo.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/isl29003.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/isl29020.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-06-09T10:52:34.213 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei-me.ko 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei-txe.ko 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/mei/mei.ko 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-gru 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-gru/gru.ko 2023-06-09T10:52:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xp.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xpc.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tifm_7xx1.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tifm_core.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/tsl2550.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_balloon.ko 2023-06-09T10:52:34.215 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_vmci 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/bonding 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/bonding/bonding.ko 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/b53 2023-06-09T10:52:34.216 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/b53/b53_common.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/bcm-sf2.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6060.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6xxx 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/dummy.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/eql.ko 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet 2023-06-09T10:52:34.217 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/8390.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-06-09T10:52:34.218 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/adaptec 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/agere 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/agere/et131x.ko 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/alteon 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-06-09T10:52:34.219 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/altera 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-06-09T10:52:34.220 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/alx 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1c 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1e 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-06-09T10:52:34.221 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-06-09T10:52:34.222 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/genet 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade/bna 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-06-09T10:52:34.223 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-06-09T10:52:34.224 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco 2023-06-09T10:52:34.225 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco/enic 2023-06-09T10:52:34.226 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-06-09T10:52:34.226 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec 2023-06-09T10:52:34.226 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip 2023-06-09T10:52:34.226 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-06-09T10:52:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-06-09T10:52:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-06-09T10:52:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-06-09T10:52:34.227 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-06-09T10:52:34.228 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-06-09T10:52:34.228 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink 2023-06-09T10:52:34.228 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-06-09T10:52:34.228 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-06-09T10:52:34.229 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/dnet.ko 2023-06-09T10:52:34.229 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex 2023-06-09T10:52:34.229 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex/benet 2023-06-09T10:52:34.230 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-06-09T10:52:34.230 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ethoc.ko 2023-06-09T10:52:34.230 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/fealnx.ko 2023-06-09T10:52:34.230 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel 2023-06-09T10:52:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e100.ko 2023-06-09T10:52:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000 2023-06-09T10:52:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-06-09T10:52:34.231 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000e 2023-06-09T10:52:34.232 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-06-09T10:52:34.232 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/fm10k 2023-06-09T10:52:34.232 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-06-09T10:52:34.232 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/i40e 2023-06-09T10:52:34.233 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-06-09T10:52:34.233 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/iavf 2023-06-09T10:52:34.233 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-06-09T10:52:34.234 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igb 2023-06-09T10:52:34.234 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-06-09T10:52:34.234 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igbvf 2023-06-09T10:52:34.234 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-06-09T10:52:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbe 2023-06-09T10:52:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-06-09T10:52:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbevf 2023-06-09T10:52:34.235 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/jme.ko 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/skge.ko 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-06-09T10:52:34.236 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox 2023-06-09T10:52:34.237 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4 2023-06-09T10:52:34.237 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-06-09T10:52:34.237 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-06-09T10:52:34.237 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5 2023-06-09T10:52:34.237 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/micrel 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom/myri10ge 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-06-09T10:52:34.238 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/neterion 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/nvidia 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines 2023-06-09T10:52:34.239 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-06-09T10:52:34.240 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-06-09T10:52:34.240 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic 2023-06-09T10:52:34.240 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/netxen 2023-06-09T10:52:34.241 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-06-09T10:52:34.241 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-06-09T10:52:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-06-09T10:52:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-06-09T10:52:34.242 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rdc 2023-06-09T10:52:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-06-09T10:52:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek 2023-06-09T10:52:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-06-09T10:52:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-06-09T10:52:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/atp.ko 2023-06-09T10:52:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-06-09T10:52:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rocker 2023-06-09T10:52:34.244 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-06-09T10:52:34.245 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sfc 2023-06-09T10:52:34.245 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-06-09T10:52:34.245 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/silan 2023-06-09T10:52:34.245 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-06-09T10:52:34.246 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis 2023-06-09T10:52:34.246 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis/sis190.ko 2023-06-09T10:52:34.246 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sis/sis900.ko 2023-06-09T10:52:34.246 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro 2023-06-09T10:52:34.247 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/cassini.ko 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/niu.ko 2023-06-09T10:52:34.248 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/sungem.ko 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/tehuti 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ti 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/ti/tlan.ko 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-06-09T10:52:34.249 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/xircom 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/geneve.ko 2023-06-09T10:52:34.250 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/hyperv 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ieee802154 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ieee802154/fakelb.ko 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ifb.ko 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ipvlan 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ipvlan/ipvlan.ko 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/macvlan.ko 2023-06-09T10:52:34.251 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/macvtap.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/mii.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/net_failover.ko 2023-06-09T10:52:34.252 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/netconsole.ko 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/nlmon.ko 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy 2023-06-09T10:52:34.253 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/amd.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm7xxx.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/bcm87xx.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/broadcom.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/cicada.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/davicom.ko 2023-06-09T10:52:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/dp83640.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/et1011c.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/icplus.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/lxt.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/marvell.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/micrel.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/national.ko 2023-06-09T10:52:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/phylink.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/qsemi.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/realtek.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/smsc.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/ste10Xp.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/phy/vitesse.ko 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp 2023-06-09T10:52:34.256 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/bsd_comp.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_async.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_deflate.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_generic.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_mppe.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/ppp_synctty.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pppoe.ko 2023-06-09T10:52:34.257 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pppox.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/ppp/pptp.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip/slhc.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/slip/slip.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/sungem_phy.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/tap.ko 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team 2023-06-09T10:52:34.258 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_activebackup.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_broadcast.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_random.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-06-09T10:52:34.259 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/tun.ko 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/veth.ko 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/virtio_net.ko 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vmxnet3 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vxlan 2023-06-09T10:52:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/vxlan/vxlan.ko 2023-06-09T10:52:34.261 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netback 2023-06-09T10:52:34.261 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netback/xen-netback.ko 2023-06-09T10:52:34.261 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/net/xen-netfront.ko 2023-06-09T10:52:34.261 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ntb 2023-06-09T10:52:34.261 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ntb/ntb.ko 2023-06-09T10:52:34.262 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme 2023-06-09T10:52:34.262 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host 2023-06-09T10:52:34.262 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host/nvme-core.ko 2023-06-09T10:52:34.262 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/nvme/host/nvme.ko 2023-06-09T10:52:34.263 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport 2023-06-09T10:52:34.263 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport.ko 2023-06-09T10:52:34.263 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_cs.ko 2023-06-09T10:52:34.263 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_pc.ko 2023-06-09T10:52:34.264 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/parport/parport_serial.ko 2023-06-09T10:52:34.264 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci 2023-06-09T10:52:34.264 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/hotplug 2023-06-09T10:52:34.265 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-06-09T10:52:34.265 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/pcie 2023-06-09T10:52:34.265 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/pcie/aer_inject.ko 2023-06-09T10:52:34.265 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pci/xen-pcifront.ko 2023-06-09T10:52:34.266 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia 2023-06-09T10:52:34.266 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/i82092.ko 2023-06-09T10:52:34.266 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/pd6729.ko 2023-06-09T10:52:34.267 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pcmcia/yenta_socket.ko 2023-06-09T10:52:34.267 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl 2023-06-09T10:52:34.267 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl/intel 2023-06-09T10:52:34.267 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-06-09T10:52:34.268 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform 2023-06-09T10:52:34.268 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86 2023-06-09T10:52:34.268 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/acer-wmi.ko 2023-06-09T10:52:34.268 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/acerhdf.ko 2023-06-09T10:52:34.269 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/apple-gmux.ko 2023-06-09T10:52:34.269 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-laptop.ko 2023-06-09T10:52:34.269 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-06-09T10:52:34.269 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/asus-wmi.ko 2023-06-09T10:52:34.270 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/classmate-laptop.ko 2023-06-09T10:52:34.270 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-06-09T10:52:34.270 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/hdaps.ko 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-06-09T10:52:34.271 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/intel_ips.ko 2023-06-09T10:52:34.272 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/msi-wmi.ko 2023-06-09T10:52:34.272 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/mxm-wmi.ko 2023-06-09T10:52:34.272 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-06-09T10:52:34.272 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/samsung-laptop.ko 2023-06-09T10:52:34.272 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/samsung-q10.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/topstar-laptop.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/toshiba_haps.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/wmi-bmof.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/platform/x86/wmi.ko 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap 2023-06-09T10:52:34.273 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap/intel_rapl_common.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/powercap/intel_rapl_msr.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps-gpio.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps-ldisc.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/clients/pps_parport.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/pps/pps_core.ko 2023-06-09T10:52:34.274 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp/ptp.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ptp/ptp_kvm.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-bq32k.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-bq4802.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1286.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1307.ko 2023-06-09T10:52:34.275 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1374.ko 2023-06-09T10:52:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1511.ko 2023-06-09T10:52:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1553.ko 2023-06-09T10:52:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1672.ko 2023-06-09T10:52:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds1742.ko 2023-06-09T10:52:34.276 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds2404.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-ds3232.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-em3027.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-fm3130.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-isl12022.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-isl1208.ko 2023-06-09T10:52:34.277 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m41t80.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m48t35.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-m48t59.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-max6900.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-msm6242.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf2127.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf85063.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8523.ko 2023-06-09T10:52:34.278 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8563.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-pcf8583.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rp5c01.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rs5c372.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rx8025.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-rx8581.ko 2023-06-09T10:52:34.279 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/rtc/rtc-x1205.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-9xxx.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-sas.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/3w-xxxx.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/BusLogic.ko 2023-06-09T10:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/a100u2w.ko 2023-06-09T10:52:34.281 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aacraid 2023-06-09T10:52:34.281 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aacraid/aacraid.ko 2023-06-09T10:52:34.281 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/advansys.ko 2023-06-09T10:52:34.281 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx 2023-06-09T10:52:34.282 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-06-09T10:52:34.282 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-06-09T10:52:34.282 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/am53c974.ko 2023-06-09T10:52:34.282 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/arcmsr 2023-06-09T10:52:34.283 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-06-09T10:52:34.283 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/atp870u.ko 2023-06-09T10:52:34.283 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/be2iscsi 2023-06-09T10:52:34.283 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-06-09T10:52:34.283 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bfa 2023-06-09T10:52:34.284 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bfa/bfa.ko 2023-06-09T10:52:34.284 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2fc 2023-06-09T10:52:34.284 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-06-09T10:52:34.284 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2i 2023-06-09T10:52:34.285 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-06-09T10:52:34.285 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ch.ko 2023-06-09T10:52:34.285 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/csiostor 2023-06-09T10:52:34.285 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/csiostor/csiostor.ko 2023-06-09T10:52:34.286 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi 2023-06-09T10:52:34.286 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb3i 2023-06-09T10:52:34.286 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-06-09T10:52:34.286 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb4i 2023-06-09T10:52:34.287 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-06-09T10:52:34.287 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-06-09T10:52:34.287 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/dc395x.ko 2023-06-09T10:52:34.287 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler 2023-06-09T10:52:34.288 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-06-09T10:52:34.288 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-06-09T10:52:34.288 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-06-09T10:52:34.288 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/dmx3191d.ko 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esas2r 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esas2r/esas2r.ko 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/esp_scsi.ko 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe/fcoe.ko 2023-06-09T10:52:34.289 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fnic 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/fnic/fnic.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hpsa.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hptiop.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/hv_storvsc.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/initio.ko 2023-06-09T10:52:34.290 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ipr.ko 2023-06-09T10:52:34.291 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ips.ko 2023-06-09T10:52:34.291 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/isci 2023-06-09T10:52:34.291 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/isci/isci.ko 2023-06-09T10:52:34.291 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-06-09T10:52:34.291 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/iscsi_tcp.ko 2023-06-09T10:52:34.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libfc 2023-06-09T10:52:34.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libfc/libfc.ko 2023-06-09T10:52:34.292 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libiscsi.ko 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libiscsi_tcp.ko 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libsas 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/libsas/libsas.ko 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/lpfc 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/lpfc/lpfc.ko 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid 2023-06-09T10:52:34.293 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid.ko 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mpt3sas 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvsas 2023-06-09T10:52:34.294 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvsas/mvsas.ko 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/mvumi.ko 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pm8001 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/pmcraid.ko 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla1280.ko 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx 2023-06-09T10:52:34.295 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla4xxx 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/raid_class.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_debug.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_fc.ko 2023-06-09T10:52:34.296 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_sas.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_spi.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/scsi_transport_srp.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/ses.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/st.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/stex.ko 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/sym53c8xx_2 2023-06-09T10:52:34.297 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/virtio_scsi.ko 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/vmw_pvscsi.ko 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/scsi/wd719x.ko 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ssb 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ssb/ssb.ko 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target 2023-06-09T10:52:34.298 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/iscsi 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/loopback 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/loopback/tcm_loop.ko 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_file.ko 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_iblock.ko 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_mod.ko 2023-06-09T10:52:34.299 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_pscsi.ko 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/target_core_user.ko 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/tcm_fc 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-06-09T10:52:34.300 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-06-09T10:52:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-06-09T10:52:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/ipwireless 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/n_gsm.ko 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/n_hdlc.ko 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/nozomi.ko 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial 2023-06-09T10:52:34.303 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/8250 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/arc_uart.ko 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/jsm 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/serial/jsm/jsm.ko 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/tty/synclink_gt.ko 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs 2023-06-09T10:52:34.304 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/core 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/core/ufshcd-core.ko 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/host 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio.ko 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_aec.ko 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_cif.ko 2023-06-09T10:52:34.305 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_pci_generic.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/uio/uio_sercos3.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/cdc-acm.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/cdc-wdm.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/usblp.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/class/usbtmc.ko 2023-06-09T10:52:34.306 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/host 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/host/sl811-hcd.ko 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image/mdc800.ko 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/image/microtek.ko 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/adutux.ko 2023-06-09T10:52:34.307 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/appledisplay.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/emi26.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/emi62.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/ezusb.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/idmouse.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/iowarrior.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/isight_firmware.ko 2023-06-09T10:52:34.308 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/ldusb.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/legousbtower.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/sisusbvga 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/trancevibrator.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usb3503.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usblcd.ko 2023-06-09T10:52:34.309 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/usbsevseg.ko 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/uss720.ko 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/misc/yurex.ko 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/phy 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/phy/phy-generic.ko 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/aircable.ko 2023-06-09T10:52:34.310 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ark3116.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/belkin_sa.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ch341.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cp210x.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cyberjack.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/cypress_m8.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/digi_acceleport.ko 2023-06-09T10:52:34.311 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/empeg.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ftdi_sio.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/garmin_gps.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/io_edgeport.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/io_ti.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ipaq.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ipw.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ir-usb.ko 2023-06-09T10:52:34.312 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/keyspan.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/keyspan_pda.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/kl5kusb105.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/kobil_sct.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mct_u232.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mos7720.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/mos7840.ko 2023-06-09T10:52:34.313 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/navman.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/omninet.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/opticon.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/option.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/oti6858.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/pl2303.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/qcaux.ko 2023-06-09T10:52:34.314 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/qcserial.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/quatech2.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/safe_serial.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/sierra.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/spcp8x5.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ssu100.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/symbolserial.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-06-09T10:52:34.315 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb_debug.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/usb_wwan.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/visor.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/whiteheat.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/serial/xsens_mt.ko 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage 2023-06-09T10:52:34.316 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/uas.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-alauda.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-cypress.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-datafab.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-freecom.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-isd200.ko 2023-06-09T10:52:34.317 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-karma.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-onetouch.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-realtek.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-sddr09.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-sddr55.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/ums-usbat.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/storage/usb-storage.ko 2023-06-09T10:52:34.318 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/usbip-core.ko 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/usbip-host.ko 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-06-09T10:52:34.319 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/pci/vfio-pci.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/vfio.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_iotlb.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_net.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/vhost/vhost_scsi.ko 2023-06-09T10:52:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/apple_bl.ko 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/lcd.ko 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/video/backlight/platform_lcd.ko 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio.ko 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_balloon.ko 2023-06-09T10:52:34.321 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_mmio.ko 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci.ko 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/virtio/virtio_ring.ko 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters 2023-06-09T10:52:34.322 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters/ds2482.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/masters/ds2490.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-06-09T10:52:34.323 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_smem.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/slaves/w1_therm.ko 2023-06-09T10:52:34.324 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/w1/wire.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/alim1535_wdt.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/alim7101_wdt.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/f71808e_wdt.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/hpwdt.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/i6300esb.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-06-09T10:52:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/iTCO_wdt.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ib700wdt.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ibmasr.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/it8712f_wdt.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/it87_wdt.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/machzwd.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/nv_tco.ko 2023-06-09T10:52:34.326 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/pcwd_pci.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/pcwd_usb.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sch311x_wdt.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/softdog.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/sp5100_tco.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/via_wdt.ko 2023-06-09T10:52:34.327 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83877f_wdt.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/w83977f_wdt.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/wdt_pci.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/watchdog/xen_wdt.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-acpi-processor.ko 2023-06-09T10:52:34.328 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-evtchn.ko 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-gntalloc.ko 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-gntdev.ko 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-pciback 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xen-privcmd.ko 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xenfs 2023-06-09T10:52:34.329 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/drivers/xen/xenfs/xenfs.ko 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/9p 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/9p/9p.ko 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/affs 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/affs/affs.ko 2023-06-09T10:52:34.330 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/befs 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/befs/befs.ko 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/binfmt_misc.ko 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/btrfs 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/btrfs/btrfs.ko 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cachefiles 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cachefiles/cachefiles.ko 2023-06-09T10:52:34.331 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ceph 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ceph/ceph.ko 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cifs 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cifs/cifs.ko 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/coda 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/coda/coda.ko 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cramfs 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/cramfs/cramfs.ko 2023-06-09T10:52:34.332 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/dlm 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/dlm/dlm.ko 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ecryptfs 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ecryptfs/ecryptfs.ko 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/f2fs 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/f2fs/f2fs.ko 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat 2023-06-09T10:52:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/fat.ko 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/msdos.ko 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fat/vfat.ko 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fscache 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fscache/fscache.ko 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse/cuse.ko 2023-06-09T10:52:34.334 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/fuse/fuse.ko 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/gfs2 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/gfs2/gfs2.ko 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfs 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfs/hfs.ko 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfsplus 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/hfsplus/hfsplus.ko 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/isofs 2023-06-09T10:52:34.335 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/isofs/isofs.ko 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/jfs 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/jfs/jfs.ko 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/lockd 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/lockd/lockd.ko 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/minix 2023-06-09T10:52:34.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/minix/minix.ko 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/netfs 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/netfs/netfs.ko 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/blocklayout 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/filelayout 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-06-09T10:52:34.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/flexfilelayout 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfs.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfsv3.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs/nfsv4.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common/grace.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfs_common/nfs_acl.ko 2023-06-09T10:52:34.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfsd 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nfsd/nfsd.ko 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nilfs2 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nilfs2/nilfs2.ko 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-celtic.ko 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-centeuro.ko 2023-06-09T10:52:34.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-croatian.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-cyrillic.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-gaelic.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-greek.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-iceland.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-inuit.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-roman.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-romanian.ko 2023-06-09T10:52:34.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/mac-turkish.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1250.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1251.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp1255.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp737.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp775.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp850.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp852.ko 2023-06-09T10:52:34.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp855.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp857.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp860.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp861.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp862.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp863.ko 2023-06-09T10:52:34.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp864.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp865.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp866.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp869.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp874.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp932.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp936.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp949.ko 2023-06-09T10:52:34.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_cp950.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_euc-jp.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-1.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-13.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-14.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-15.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-2.ko 2023-06-09T10:52:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-3.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-4.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-5.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-6.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-7.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_iso8859-9.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-r.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-ru.ko 2023-06-09T10:52:34.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_koi8-u.ko 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/nls/nls_utf8.ko 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/cluster 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlm 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-06-09T10:52:34.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlmfs 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/overlayfs 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/overlayfs/overlay.ko 2023-06-09T10:52:34.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/pstore 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/pstore/ramoops.ko 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/reiserfs 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/reiserfs/reiserfs.ko 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/romfs 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/romfs/romfs.ko 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common/cifs_arc4.ko 2023-06-09T10:52:34.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/smbfs_common/cifs_md4.ko 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/squashfs 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/squashfs/squashfs.ko 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/sysv 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/sysv/sysv.ko 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/udf 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/udf/udf.ko 2023-06-09T10:52:34.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ufs 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/ufs/ufs.ko 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/xfs 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/fs/xfs/xfs.ko 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/rcu 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/rcu/rcutorture.ko 2023-06-09T10:52:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/torture.ko 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/trace 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/asn1_encoder.ko 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crc-itu-t.ko 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crc8.ko 2023-06-09T10:52:34.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto/libarc4.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/crypto/libdes.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/libcrc32c.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lru_cache.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4/lz4_compress.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/lz4/lz4hc_compress.ko 2023-06-09T10:52:34.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/math 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/math/cordic.ko 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/raid6 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/raid6/raid6_pq.ko 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/reed_solomon 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/reed_solomon/reed_solomon.ko 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_bm.ko 2023-06-09T10:52:34.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_fsm.ko 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/ts_kmp.ko 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/zstd 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/lib/zstd/zstd_compress.ko 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/mm 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/mm/hwpoison-inject.ko 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan 2023-06-09T10:52:34.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/6lowpan.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_dest.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_fragment.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_hop.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_ipv6.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_mobility.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_routing.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/6lowpan/nhc_udp.ko 2023-06-09T10:52:34.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/garp.ko 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/mrp.ko 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/p8022.ko 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/psnap.ko 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/802/stp.ko 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/8021q 2023-06-09T10:52:34.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/8021q/8021q.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_fd.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_rdma.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/9p/9pnet_virtio.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/batman-adv 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/batman-adv/batman-adv.ko 2023-06-09T10:52:34.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/br_netfilter.ko 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/bridge.ko 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_among.ko 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_arp.ko 2023-06-09T10:52:34.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_ip.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_limit.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_log.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_mark.ko 2023-06-09T10:52:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_snat.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_stp.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-06-09T10:52:34.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/ebtables.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ceph 2023-06-09T10:52:34.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ceph/libceph.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core/failover.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/core/pktgen.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_diag.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_ipv4.ko 2023-06-09T10:52:34.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dccp/dccp_ipv6.ko 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dns_resolver 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dns_resolver/dns_resolver.ko 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/dsa_core.ko 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_brcm.ko 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_dsa.ko 2023-06-09T10:52:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_none.ko 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/dsa/tag_trailer.ko 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/6lowpan 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/ieee802154.ko 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ieee802154/ieee802154_socket.ko 2023-06-09T10:52:34.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ah4.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/esp4.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/fou.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/gre.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/inet_diag.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_gre.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_tunnel.ko 2023-06-09T10:52:34.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ip_vti.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ipcomp.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/ipip.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arp_tables.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-06-09T10:52:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/iptable_security.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-06-09T10:52:34.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-06-09T10:52:34.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_bic.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_dctcp.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_diag.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_highspeed.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_htcp.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_hybla.ko 2023-06-09T10:52:34.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_illinois.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_lp.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_scalable.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_vegas.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_veno.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_westwood.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tcp_yeah.ko 2023-06-09T10:52:34.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/tunnel4.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/udp_diag.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/udp_tunnel.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv4/xfrm4_tunnel.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ah6.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/esp6.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/fou6.ko 2023-06-09T10:52:34.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_tunnel.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ip6_vti.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/ipcomp6.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-06-09T10:52:34.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-06-09T10:52:34.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-06-09T10:52:34.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/sit.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/tunnel6.ko 2023-06-09T10:52:34.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/ipv6/xfrm6_tunnel.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/key 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/key/af_key.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_core.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_debugfs.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_eth.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ip.ko 2023-06-09T10:52:34.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ip6.ko 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_netlink.ko 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/l2tp/l2tp_ppp.ko 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/llc 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/llc/llc.ko 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mac802154 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mac802154/mac802154.ko 2023-06-09T10:52:34.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mpls 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/mpls/mpls_gso.ko 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set.ko 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-06-09T10:52:34.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-06-09T10:52:34.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-06-09T10:52:34.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-06-09T10:52:34.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conncount.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack.ko 2023-06-09T10:52:34.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_h323.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_irc.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-06-09T10:52:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_sane.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_sip.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_dup_netdev.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_flow_table.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_flow_table_inet.ko 2023-06-09T10:52:34.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_log_syslog.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_amanda.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_ftp.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_irc.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_sip.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_nat_tftp.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_synproxy_core.ko 2023-06-09T10:52:34.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nf_tables.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_acct.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_log.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_osf.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nfnetlink_queue.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_chain_nat.ko 2023-06-09T10:52:34.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_compat.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_ct.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_dup_netdev.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_fib.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_fwd_netdev.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_hash.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_limit.ko 2023-06-09T10:52:34.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_log.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_masq.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_nat.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_numgen.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_queue.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_quota.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_redir.ko 2023-06-09T10:52:34.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_reject.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_reject_inet.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_socket.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/nft_tunnel.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_AUDIT.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CHECKSUM.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CLASSIFY.ko 2023-06-09T10:52:34.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_CT.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_DSCP.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_HL.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_HMARK.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_IDLETIMER.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_LED.ko 2023-06-09T10:52:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_LOG.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_MASQUERADE.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NETMAP.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NFLOG.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_NFQUEUE.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_RATEEST.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_REDIRECT.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_SECMARK.ko 2023-06-09T10:52:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TCPMSS.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TEE.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TPROXY.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_TRACE.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_addrtype.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_bpf.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cgroup.ko 2023-06-09T10:52:34.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cluster.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_comment.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connbytes.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connlabel.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connlimit.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_connmark.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_conntrack.ko 2023-06-09T10:52:34.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_cpu.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_dccp.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_devgroup.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_dscp.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ecn.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_esp.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_hashlimit.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_helper.ko 2023-06-09T10:52:34.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_hl.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ipcomp.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_iprange.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_ipvs.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_l2tp.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_length.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_limit.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_mac.ko 2023-06-09T10:52:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_mark.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_multiport.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_nat.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_nfacct.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_osf.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_owner.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_physdev.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_pkttype.ko 2023-06-09T10:52:34.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_policy.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_quota.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_rateest.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_realm.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_recent.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_sctp.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_set.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_socket.ko 2023-06-09T10:52:34.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_state.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_statistic.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_string.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_tcpmss.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_time.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netfilter/xt_u32.ko 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netlink 2023-06-09T10:52:34.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/netlink/netlink_diag.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/nsh 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/nsh/nsh.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/openvswitch.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-geneve.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-gre.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/openvswitch/vport-vxlan.ko 2023-06-09T10:52:34.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/packet 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/packet/af_packet_diag.ko 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds.ko 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds_rdma.ko 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/rds/rds_tcp.ko 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_csum.ko 2023-06-09T10:52:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_gact.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_ipt.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_mirred.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_nat.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_pedit.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_police.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_simple.ko 2023-06-09T10:52:34.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_skbedit.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/act_vlan.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_basic.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_bpf.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_flow.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_fw.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_route.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/cls_u32.ko 2023-06-09T10:52:34.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_cmp.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_ipset.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_meta.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_nbyte.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_text.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/em_u32.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_choke.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_codel.ko 2023-06-09T10:52:34.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_drr.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_fq.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_gred.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_hfsc.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_hhf.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_htb.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_ingress.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_mqprio.ko 2023-06-09T10:52:34.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_mqprio_lib.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_multiq.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_netem.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_pie.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_plug.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_prio.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_qfq.ko 2023-06-09T10:52:34.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_red.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_sfb.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_sfq.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_tbf.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sched/sch_teql.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp/sctp.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sctp/sctp_diag.ko 2023-06-09T10:52:34.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/sunrpc.ko 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/xprtrdma 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc 2023-06-09T10:52:34.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc/diag.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/tipc/tipc.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/unix 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/unix/unix_diag.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock.ko 2023-06-09T10:52:34.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock_diag.ko 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/vmw_vsock/vsock_loopback.ko 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/xfrm 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/net/xfrm/xfrm_ipcomp.ko 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/encrypted-keys 2023-06-09T10:52:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/trusted-keys 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/security/keys/trusted-keys/trusted.ko 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt/lib 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/kernel/virt/lib/irqbypass.ko 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.alias 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.alias.bin 2023-06-09T10:52:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin.bin 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.builtin.modinfo 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.dep 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.dep.bin 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.devname 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.order 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.softdep 2023-06-09T10:52:34.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.symbols 2023-06-09T10:52:34.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.4.0-rc2-g10b2f20a4f46/modules.symbols.bin 2023-06-09T10:52:34.412 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.413 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-06-09T10:52:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2023-06-09T10:52:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 -a -r /boot/System.map-6.4.0-rc2-g10b2f20a4f46 ]; then 2023-06-09T10:52:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:cp /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:cp /boot/System.map-6.4.0-rc2-g10b2f20a4f46 /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:rm -f /boot/vmlinuz-6.4.0-rc2-g10b2f20a4f46 /boot/System.map-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/installkernel 6.4.0-rc2-g10b2f20a4f46 /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:rm -f /boot/.vmlinuz-6.4.0-rc2-g10b2f20a4f46-rpm /boot/.System.map-6.4.0-rc2-g10b2f20a4f46-rpm 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:new-kernel-pkg --remove 6.4.0-rc2-g10b2f20a4f46 --rminitrd --initrdfile=/boot/initramfs-6.4.0-rc2-g10b2f20a4f46.img 2023-06-09T10:52:34.442 INFO:teuthology.orchestra.run.smithi007.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:postuninstall scriptlet (using /bin/sh): 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /sbin/update-bootloader ]; then 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:/sbin/update-bootloader --remove 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.443 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2023-06-09T10:52:34.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2023-06-09T10:52:34.525 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2023-06-09T10:52:34.525 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.02 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 123.el8 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Group : System Environment/Base 2023-06-09T10:52:34.526 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 9265450 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Relocations : (not relocatable) 2023-06-09T10:52:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : Red Hat, Inc. 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : Red Hat, Inc. 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-06-09T10:52:34.528 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2023-06-09T10:52:34.529 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2023-06-09T10:52:34.529 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2023-06-09T10:52:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T10:52:34.529 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2023-06-09T10:52:34.530 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-06-09T10:52:34.530 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:34.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-06-09T10:52:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2023-06-09T10:52:41.083 INFO:teuthology.orchestra.run.smithi007.stderr:done 2023-06-09T10:52:41.085 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-06-09T10:52:41.106 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.eybwGqN9uE 2023-06-09T10:52:41.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eybwGqN9uE 2023-06-09T10:52:41.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.eybwGqN9uE 2023-06-09T10:52:41.327 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.eybwGqN9uE is 6KB 2023-06-09T10:52:41.348 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.eybwGqN9uE 2023-06-09T10:52:41.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2023-06-09T10:52:41.435 INFO:teuthology.orchestra.run.smithi007.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-06-09T10:52:41.435 INFO:teuthology.orchestra.run.smithi007.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-06-09T10:52:41.435 INFO:teuthology.orchestra.run.smithi007.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.4.0-rc2-g10b2f20a4f46.conf 2023-06-09T10:52:41.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:52:41.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2023-06-09T10:53:00.260 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-06-09T10:53:00.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-06-09T10:53:00.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:53:11.584 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-06-09T10:53:11.584 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-06-09T10:53:11.585 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:53:18.576 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2023-06-09T10:53:30.096 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-06-09T10:53:48.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-06-09T10:53:48.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:54:00.108 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-06-09T10:54:00.109 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:54:06.960 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2023-06-09T10:54:18.484 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2023-06-09T10:54:36.976 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-06-09T10:54:36.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:54:37.316 DEBUG:teuthology.orchestra.run.smithi117:> true 2023-06-09T10:54:37.704 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-06-09T10:54:37.704 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.4.0-rc2-g10b2f20a4f46"... 2023-06-09T10:54:37.705 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2023-06-09T10:54:37.746 INFO:teuthology.orchestra.run.smithi117.stdout:6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:54:37.747 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-rc2-g10b2f20a4f46 vs 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:54:37.747 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-09T10:54:37.747 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-09T10:54:38.748 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-09T10:54:38.748 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-09T10:54:39.179 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2023-06-09T10:54:39.204 DEBUG:teuthology.parallel:result is None 2023-06-09T10:54:48.485 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-06-09T10:54:48.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T10:54:48.837 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-06-09T10:54:49.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-06-09T10:54:49.251 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.4.0-rc2-g10b2f20a4f46"... 2023-06-09T10:54:49.252 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2023-06-09T10:54:49.294 INFO:teuthology.orchestra.run.smithi007.stdout:6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:54:49.294 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-rc2-g10b2f20a4f46 vs 6.4.0-rc2-g10b2f20a4f46 2023-06-09T10:54:49.295 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-06-09T10:54:49.295 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-06-09T10:54:50.296 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-09T10:54:50.296 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-09T10:54:50.425 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2023-06-09T10:54:50.452 DEBUG:teuthology.parallel:result is None 2023-06-09T10:54:50.452 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-09T10:54:50.461 INFO:teuthology.task.internal:Creating test directory... 2023-06-09T10:54:50.462 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-09T10:54:50.466 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-09T10:54:50.513 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-09T10:54:50.521 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-09T10:54:50.602 INFO:teuthology.task.internal:Creating archive directory... 2023-06-09T10:54:50.603 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-09T10:54:50.607 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-09T10:54:50.734 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-09T10:54:50.741 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-09T10:54:50.741 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-09T10:54:50.746 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-09T10:54:50.818 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T10:54:50.828 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T10:54:50.846 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T10:54:50.856 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-09T10:54:50.859 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-09T10:54:50.865 INFO:teuthology.task.internal:Configuring sudo... 2023-06-09T10:54:50.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-09T10:54:50.870 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-09T10:54:50.939 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-09T10:54:50.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-09T10:54:50.950 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-09T10:54:50.954 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-09T10:54:50.996 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T10:54:51.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T10:54:51.185 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T10:54:51.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T10:54:51.338 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T10:54:51.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-09T10:54:51.406 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T10:54:51.485 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-09T10:54:51.563 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T10:54:51.634 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-09T10:54:51.714 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T10:54:51.715 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-09T10:54:51.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2023-06-09T10:54:51.785 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2023-06-09T10:54:51.893 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T10:54:51.922 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T10:54:52.304 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-09T10:54:52.311 INFO:teuthology.task.internal:Starting timer... 2023-06-09T10:54:52.312 INFO:teuthology.run_tasks:Running task pcp... 2023-06-09T10:54:52.321 INFO:teuthology.run_tasks:Running task selinux... 2023-06-09T10:54:52.333 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-09T10:54:52.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2023-06-09T10:54:52.447 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2023-06-09T10:54:52.451 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2023-06-09T10:54:52.566 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2023-06-09T10:54:52.569 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-09T10:54:52.569 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2023-06-09T10:54:52.621 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2023-06-09T10:54:52.621 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2023-06-09T10:54:52.672 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2023-06-09T10:54:52.672 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2023-06-09T10:54:52.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T10:54:52.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:54:52.744 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T10:54:52.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T10:54:52.819 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-09T10:54:52.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2023-06-09T10:54:52.888 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2023-06-09T10:54:52.961 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-09T10:54:52.970 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-06-09T10:54:52.976 INFO:teuthology.repo_utils:Fetching main from origin 2023-06-09T10:54:53.099 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-09T10:54:53.121 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-09T10:54:53.123 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-09T11:06:38.417 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2023-06-09T11:06:38.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2023-06-09T11:06:38.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T11:06:38.510 DEBUG:teuthology.orchestra.run.smithi007:> true 2023-06-09T11:06:38.622 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2023-06-09T11:06:38.623 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-09T11:06:38.712 DEBUG:teuthology.orchestra.run.smithi117:> true 2023-06-09T11:06:38.826 INFO:teuthology.run_tasks:Running task clock... 2023-06-09T11:06:38.841 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-09T11:06:38.842 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-09T11:06:38.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T11:06:38.848 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-09T11:06:38.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T11:06:38.918 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-09T11:06:38.938 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-09T11:06:38.953 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-09T11:06:38.972 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-09T11:06:39.045 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2023-06-09T11:06:39.069 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2023-06-09T11:06:39.075 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2023-06-09T11:06:39.098 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2023-06-09T11:06:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-09T11:06:39.131 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-09T11:06:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-09T11:06:39.165 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-09T11:06:39.236 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2023-06-09T11:06:39.243 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T11:06:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2023-06-09T11:06:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.244 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.260 INFO:teuthology.orchestra.run.smithi117.stderr:bash: ntpq: command not found 2023-06-09T11:06:39.268 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T11:06:39.268 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2023-06-09T11:06:39.268 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.269 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.269 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.269 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T11:06:39.270 INFO:teuthology.run_tasks:Running task install... 2023-06-09T11:06:39.284 DEBUG:teuthology.task.install:project ceph 2023-06-09T11:06:39.285 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'}} 2023-06-09T11:06:39.285 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'} 2023-06-09T11:06:39.285 INFO:teuthology.task.install:Using flavor: default 2023-06-09T11:06:39.291 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-09T11:06:39.292 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-09T11:06:39.292 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d', 'tag': None, 'wait_for_package': False} 2023-06-09T11:06:39.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:06:39.296 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d', 'tag': None, 'wait_for_package': False} 2023-06-09T11:06:39.296 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:06:39.439 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/ 2023-06-09T11:06:39.439 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3924.g03410882 2023-06-09T11:06:39.533 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/ 2023-06-09T11:06:39.533 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3924.g03410882 2023-06-09T11:06:39.690 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-09T11:06:39.691 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:06:39.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-09T11:06:39.762 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-09T11:06:39.763 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/034108821cde95ca1c56d458dfccc7ef9e0d964d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-09T11:06:39.792 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-rishabh-2023Jun9-1308/034108821cde95ca1c56d458dfccc7ef9e0d964d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-09T11:06:39.793 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:06:39.793 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-09T11:06:39.864 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-09T11:06:39.865 DEBUG:teuthology.orchestra.run.smithi117:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/034108821cde95ca1c56d458dfccc7ef9e0d964d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-09T11:06:39.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-09T11:06:39.968 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-09T11:06:39.974 DEBUG:teuthology.orchestra.run.smithi117:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-09T11:06:40.035 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2023-06-09T11:06:40.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2023-06-09T11:06:40.069 DEBUG:teuthology.orchestra.run.smithi117:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-09T11:06:40.136 INFO:teuthology.orchestra.run.smithi117.stdout:check_obsoletes = 1 2023-06-09T11:06:40.139 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum clean all 2023-06-09T11:06:40.517 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2023-06-09T11:06:40.579 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-06-09T11:06:40.841 INFO:teuthology.orchestra.run.smithi007.stdout:44 files removed 2023-06-09T11:06:40.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-09T11:06:40.900 INFO:teuthology.orchestra.run.smithi117.stdout:44 files removed 2023-06-09T11:06:40.955 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-09T11:06:41.371 INFO:teuthology.orchestra.run.smithi007.stdout:Updating Subscription Management repositories. 2023-06-09T11:06:41.396 INFO:teuthology.orchestra.run.smithi117.stdout:Updating Subscription Management repositories. 2023-06-09T11:06:42.766 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 88 kB/s | 80 kB 00:00 2023-06-09T11:06:43.095 INFO:teuthology.orchestra.run.smithi117.stdout:ceph packages for x86_64 69 kB/s | 80 kB 00:01 2023-06-09T11:06:43.291 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 19 kB/s | 9.4 kB 00:00 2023-06-09T11:06:43.520 INFO:teuthology.orchestra.run.smithi117.stdout:ceph noarch packages 24 kB/s | 9.4 kB 00:00 2023-06-09T11:06:43.875 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 2.7 kB/s | 1.5 kB 00:00 2023-06-09T11:06:43.979 INFO:teuthology.orchestra.run.smithi117.stdout:ceph source packages 3.4 kB/s | 1.5 kB 00:00 2023-06-09T11:06:46.484 INFO:teuthology.orchestra.run.smithi007.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-09T11:06:46.579 INFO:teuthology.orchestra.run.smithi117.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-09T11:06:47.218 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-06-09T11:06:51.510 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 523 kB/s | 24 kB 00:00 2023-06-09T11:06:52.761 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 53 MB 00:01 2023-06-09T11:06:59.052 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2023-06-09T11:07:01.995 INFO:teuthology.orchestra.run.smithi007.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-06-09T11:07:03.127 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 203 kB/s | 24 kB 00:00 2023-06-09T11:07:04.204 INFO:teuthology.orchestra.run.smithi117.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-06-09T11:07:12.737 INFO:teuthology.orchestra.run.smithi117.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-06-09T11:07:14.206 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T11:07:14.207 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T11:07:14.223 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-09T11:07:14.224 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-09T11:07:14.225 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-09T11:07:14.228 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T11:07:14.229 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T11:07:14.233 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-09T11:07:14.234 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T11:07:14.385 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2023-06-09T11:07:14.399 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2023-06-09T11:07:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2023-06-09T11:07:14.400 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2023-06-09T11:07:14.400 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2023-06-09T11:07:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-09T11:07:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-09T11:07:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:18.0.0-3924.g03410882.el8 ceph 6.4 k 2023-06-09T11:07:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:18.0.0-3924.g03410882.el8 ceph 5.3 M 2023-06-09T11:07:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:18.0.0-3924.g03410882.el8 ceph 848 k 2023-06-09T11:07:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3924.g03410882.el8 ceph 165 k 2023-06-09T11:07:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:18.0.0-3924.g03410882.el8 ceph 1.5 M 2023-06-09T11:07:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 127 k 2023-06-09T11:07:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 1.7 M 2023-06-09T11:07:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 7.4 M 2023-06-09T11:07:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 47 k 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:18.0.0-3924.g03410882.el8 ceph 13 M 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:18.0.0-3924.g03410882.el8 ceph 41 M 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 265 k 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 83 k 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:18.0.0-3924.g03410882.el8 ceph 258 k 2023-06-09T11:07:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 18 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:18.0.0-3924.g03410882.el8 ceph 31 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 771 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:18.0.0-3924.g03410882.el8 ceph 139 k 2023-06-09T11:07:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:18.0.0-3924.g03410882.el8 ceph 197 k 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:18.0.0-3924.g03410882.el8 ceph 371 k 2023-06-09T11:07:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:18.0.0-3924.g03410882.el8 ceph 365 k 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:18.0.0-3924.g03410882.el8 ceph 107 k 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:18.0.0-3924.g03410882.el8 ceph 94 k 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.4 M 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:18.0.0-3924.g03410882.el8 ceph 180 k 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2023-06-09T11:07:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.7 M 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.6 M 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:18.0.0-3924.g03410882.el8 ceph 22 M 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 23 k 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:18.0.0-3924.g03410882.el8 ceph 2.3 M 2023-06-09T11:07:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 240 k 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:18.0.0-3924.g03410882.el8 ceph 4.6 M 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:18.0.0-3924.g03410882.el8 ceph 18 M 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 15 k 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:18.0.0-3924.g03410882.el8 ceph 24 k 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-09T11:07:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:18.0.0-3924.g03410882.el8 ceph 137 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-09T11:07:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-09T11:07:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-09T11:07:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-09T11:07:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:18.0.0-3924.g03410882.el8 ceph 473 k 2023-06-09T11:07:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-09T11:07:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 6.5 M 2023-06-09T11:07:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-09T11:07:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-09T11:07:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-09T11:07:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-09T11:07:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-09T11:07:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-09T11:07:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-09T11:07:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-09T11:07:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-09T11:07:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-09T11:07:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-09T11:07:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-09T11:07:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-09T11:07:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-09T11:07:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-09T11:07:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-09T11:07:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:18.0.0-3924.g03410882.el8 ceph 45 k 2023-06-09T11:07:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:18.0.0-3924.g03410882.el8 ceph 119 k 2023-06-09T11:07:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-09T11:07:14.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-09T11:07:14.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-09T11:07:14.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-09T11:07:14.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-09T11:07:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-09T11:07:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-09T11:07:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-09T11:07:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-09T11:07:14.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-09T11:07:14.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-09T11:07:14.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T11:07:14.419 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-09T11:07:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-09T11:07:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-09T11:07:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-09T11:07:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-09T11:07:14.420 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-09T11:07:14.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-09T11:07:14.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-09T11:07:14.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-09T11:07:14.421 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-09T11:07:14.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-09T11:07:14.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-09T11:07:14.422 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-09T11:07:14.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-09T11:07:14.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-09T11:07:14.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-09T11:07:14.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-09T11:07:14.423 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-09T11:07:14.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T11:07:14.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-09T11:07:14.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-09T11:07:14.424 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-09T11:07:14.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-09T11:07:14.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-09T11:07:14.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-09T11:07:14.425 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-09T11:07:14.426 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-09T11:07:14.426 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-09T11:07:14.426 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-09T11:07:14.426 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-09T11:07:14.426 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-09T11:07:14.427 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-09T11:07:14.427 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-09T11:07:14.427 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-09T11:07:14.427 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-09T11:07:14.428 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-09T11:07:14.428 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-09T11:07:14.428 INFO:teuthology.orchestra.run.smithi007.stdout:Enabling module streams: 2023-06-09T11:07:14.428 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql 10 2023-06-09T11:07:14.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:07:14.429 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2023-06-09T11:07:14.429 INFO:teuthology.orchestra.run.smithi007.stdout:=========================================================================================================================== 2023-06-09T11:07:14.429 INFO:teuthology.orchestra.run.smithi007.stdout:Install 128 Packages 2023-06-09T11:07:14.429 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 3 Packages 2023-06-09T11:07:14.430 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:07:14.431 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 197 M 2023-06-09T11:07:14.431 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2023-06-09T11:07:14.796 INFO:teuthology.orchestra.run.smithi007.stdout:(1/131): ceph-18.0.0-3924.g03410882.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-06-09T11:07:15.163 INFO:teuthology.orchestra.run.smithi007.stdout:(2/131): ceph-base-18.0.0-3924.g03410882.el8.x8 8.6 MB/s | 5.3 MB 00:00 2023-06-09T11:07:15.229 INFO:teuthology.orchestra.run.smithi007.stdout:(3/131): ceph-immutable-object-cache-18.0.0-392 2.4 MB/s | 165 kB 00:00 2023-06-09T11:07:15.488 INFO:teuthology.orchestra.run.smithi007.stdout:(4/131): ceph-mds-18.0.0-3924.g03410882.el8.x86 8.8 MB/s | 2.3 MB 00:00 2023-06-09T11:07:15.689 INFO:teuthology.orchestra.run.smithi007.stdout:(5/131): ceph-mgr-18.0.0-3924.g03410882.el8.x86 7.7 MB/s | 1.5 MB 00:00 2023-06-09T11:07:16.097 INFO:teuthology.orchestra.run.smithi007.stdout:(6/131): ceph-mon-18.0.0-3924.g03410882.el8.x86 11 MB/s | 4.6 MB 00:00 2023-06-09T11:07:16.322 INFO:teuthology.orchestra.run.smithi007.stdout:(7/131): ceph-common-18.0.0-3924.g03410882.el8. 12 MB/s | 22 MB 00:01 2023-06-09T11:07:16.606 INFO:teuthology.orchestra.run.smithi007.stdout:(8/131): ceph-fuse-18.0.0-3924.g03410882.el8.x8 468 kB/s | 848 kB 00:01 2023-06-09T11:07:16.723 INFO:teuthology.orchestra.run.smithi007.stdout:(9/131): ceph-selinux-18.0.0-3924.g03410882.el8 208 kB/s | 24 kB 00:00 2023-06-09T11:07:17.448 INFO:teuthology.orchestra.run.smithi007.stdout:(10/131): ceph-radosgw-18.0.0-3924.g03410882.el 12 MB/s | 13 MB 00:01 2023-06-09T11:07:17.716 INFO:teuthology.orchestra.run.smithi007.stdout:(11/131): ceph-osd-18.0.0-3924.g03410882.el8.x8 11 MB/s | 18 MB 00:01 2023-06-09T11:07:17.740 INFO:teuthology.orchestra.run.smithi007.stdout:(12/131): cephfs-mirror-18.0.0-3924.g03410882.e 885 kB/s | 258 kB 00:00 2023-06-09T11:07:17.757 INFO:teuthology.orchestra.run.smithi007.stdout:(13/131): libcephfs-devel-18.0.0-3924.g03410882 744 kB/s | 31 kB 00:00 2023-06-09T11:07:17.807 INFO:teuthology.orchestra.run.smithi007.stdout:(14/131): libcephsqlite-18.0.0-3924.g03410882.e 2.7 MB/s | 137 kB 00:00 2023-06-09T11:07:17.849 INFO:teuthology.orchestra.run.smithi007.stdout:(15/131): libcephfs2-18.0.0-3924.g03410882.el8. 6.9 MB/s | 771 kB 00:00 2023-06-09T11:07:17.875 INFO:teuthology.orchestra.run.smithi007.stdout:(16/131): librados-devel-18.0.0-3924.g03410882. 2.0 MB/s | 139 kB 00:00 2023-06-09T11:07:17.908 INFO:teuthology.orchestra.run.smithi007.stdout:(17/131): libradosstriper1-18.0.0-3924.g0341088 7.9 MB/s | 473 kB 00:00 2023-06-09T11:07:17.975 INFO:teuthology.orchestra.run.smithi007.stdout:(18/131): python3-ceph-argparse-18.0.0-3924.g03 671 kB/s | 45 kB 00:00 2023-06-09T11:07:18.043 INFO:teuthology.orchestra.run.smithi007.stdout:(19/131): python3-ceph-common-18.0.0-3924.g0341 1.7 MB/s | 119 kB 00:00 2023-06-09T11:07:18.117 INFO:teuthology.orchestra.run.smithi007.stdout:(20/131): python3-cephfs-18.0.0-3924.g03410882. 2.6 MB/s | 197 kB 00:00 2023-06-09T11:07:18.201 INFO:teuthology.orchestra.run.smithi007.stdout:(21/131): python3-rados-18.0.0-3924.g03410882.e 4.4 MB/s | 371 kB 00:00 2023-06-09T11:07:18.293 INFO:teuthology.orchestra.run.smithi007.stdout:(22/131): python3-rbd-18.0.0-3924.g03410882.el8 3.9 MB/s | 365 kB 00:00 2023-06-09T11:07:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:(23/131): python3-rgw-18.0.0-3924.g03410882.el8 1.6 MB/s | 107 kB 00:00 2023-06-09T11:07:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:(24/131): librgw2-18.0.0-3924.g03410882.el8.x86 11 MB/s | 6.5 MB 00:00 2023-06-09T11:07:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:(25/131): rbd-fuse-18.0.0-3924.g03410882.el8.x8 870 kB/s | 94 kB 00:00 2023-06-09T11:07:18.553 INFO:teuthology.orchestra.run.smithi007.stdout:(26/131): rbd-nbd-18.0.0-3924.g03410882.el8.x86 2.1 MB/s | 180 kB 00:00 2023-06-09T11:07:18.628 INFO:teuthology.orchestra.run.smithi007.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3924.g 307 kB/s | 23 kB 00:00 2023-06-09T11:07:18.703 INFO:teuthology.orchestra.run.smithi007.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3924.g0341088 1.7 MB/s | 127 kB 00:00 2023-06-09T11:07:18.770 INFO:teuthology.orchestra.run.smithi007.stdout:(29/131): rbd-mirror-18.0.0-3924.g03410882.el8. 11 MB/s | 3.4 MB 00:00 2023-06-09T11:07:18.912 INFO:teuthology.orchestra.run.smithi007.stdout:(30/131): ceph-mgr-dashboard-18.0.0-3924.g03410 8.2 MB/s | 1.7 MB 00:00 2023-06-09T11:07:19.005 INFO:teuthology.orchestra.run.smithi007.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3924.g03 2.6 MB/s | 240 kB 00:00 2023-06-09T11:07:19.080 INFO:teuthology.orchestra.run.smithi007.stdout:(32/131): ceph-mgr-rook-18.0.0-3924.g03410882.e 635 kB/s | 47 kB 00:00 2023-06-09T11:07:19.155 INFO:teuthology.orchestra.run.smithi007.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3924.g0 194 kB/s | 15 kB 00:00 2023-06-09T11:07:19.248 INFO:teuthology.orchestra.run.smithi007.stdout:(34/131): ceph-volume-18.0.0-3924.g03410882.el8 2.8 MB/s | 265 kB 00:00 2023-06-09T11:07:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:(35/131): cephadm-18.0.0-3924.g03410882.el8.noa 1.0 MB/s | 83 kB 00:00 2023-06-09T11:07:19.406 INFO:teuthology.orchestra.run.smithi007.stdout:(36/131): cephfs-top-18.0.0-3924.g03410882.el8. 240 kB/s | 18 kB 00:00 2023-06-09T11:07:19.498 INFO:teuthology.orchestra.run.smithi007.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 10 MB/s | 7.4 MB 00:00 2023-06-09T11:07:19.732 INFO:teuthology.orchestra.run.smithi007.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 252 kB/s | 59 kB 00:00 2023-06-09T11:07:19.807 INFO:teuthology.orchestra.run.smithi007.stdout:(39/131): gperftools-libs-2.7-9.el8.x86_64.rpm 764 kB/s | 306 kB 00:00 2023-06-09T11:07:19.907 INFO:teuthology.orchestra.run.smithi007.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 429 kB/s | 75 kB 00:00 2023-06-09T11:07:19.957 INFO:teuthology.orchestra.run.smithi007.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-06-09T11:07:19.991 INFO:teuthology.orchestra.run.smithi007.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-06-09T11:07:20.058 INFO:teuthology.orchestra.run.smithi007.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-06-09T11:07:20.083 INFO:teuthology.orchestra.run.smithi007.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2023-06-09T11:07:20.142 INFO:teuthology.orchestra.run.smithi007.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-06-09T11:07:20.226 INFO:teuthology.orchestra.run.smithi007.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-06-09T11:07:20.268 INFO:teuthology.orchestra.run.smithi007.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-06-09T11:07:20.310 INFO:teuthology.orchestra.run.smithi007.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 973 kB/s | 81 kB 00:00 2023-06-09T11:07:20.357 INFO:teuthology.orchestra.run.smithi007.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-06-09T11:07:20.419 INFO:teuthology.orchestra.run.smithi007.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-06-09T11:07:20.544 INFO:teuthology.orchestra.run.smithi007.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-06-09T11:07:20.570 INFO:teuthology.orchestra.run.smithi007.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 324 kB/s | 49 kB 00:00 2023-06-09T11:07:20.636 INFO:teuthology.orchestra.run.smithi007.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 641 kB/s | 59 kB 00:00 2023-06-09T11:07:20.670 INFO:teuthology.orchestra.run.smithi007.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 602 kB/s | 60 kB 00:00 2023-06-09T11:07:20.770 INFO:teuthology.orchestra.run.smithi007.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2023-06-09T11:07:20.797 INFO:teuthology.orchestra.run.smithi007.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 124 kB/s | 16 kB 00:00 2023-06-09T11:07:20.846 INFO:teuthology.orchestra.run.smithi007.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-06-09T11:07:20.905 INFO:teuthology.orchestra.run.smithi007.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-06-09T11:07:20.967 INFO:teuthology.orchestra.run.smithi007.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 526 kB/s | 61 kB 00:00 2023-06-09T11:07:20.997 INFO:teuthology.orchestra.run.smithi007.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-06-09T11:07:21.055 INFO:teuthology.orchestra.run.smithi007.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 260 kB/s | 24 kB 00:00 2023-06-09T11:07:21.114 INFO:teuthology.orchestra.run.smithi007.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2023-06-09T11:07:21.148 INFO:teuthology.orchestra.run.smithi007.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-06-09T11:07:21.206 INFO:teuthology.orchestra.run.smithi007.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-06-09T11:07:21.265 INFO:teuthology.orchestra.run.smithi007.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-06-09T11:07:21.299 INFO:teuthology.orchestra.run.smithi007.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 666 kB/s | 61 kB 00:00 2023-06-09T11:07:21.374 INFO:teuthology.orchestra.run.smithi007.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 786 kB/s | 85 kB 00:00 2023-06-09T11:07:21.391 INFO:teuthology.orchestra.run.smithi007.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 250 kB/s | 23 kB 00:00 2023-06-09T11:07:21.417 INFO:teuthology.orchestra.run.smithi007.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-06-09T11:07:21.492 INFO:teuthology.orchestra.run.smithi007.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 627 kB/s | 47 kB 00:00 2023-06-09T11:07:21.550 INFO:teuthology.orchestra.run.smithi007.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.4 MB/s | 200 kB 00:00 2023-06-09T11:07:21.592 INFO:teuthology.orchestra.run.smithi007.stdout:(72/131): thrift-0.13.0-2.el8.x86_64.rpm 8.0 MB/s | 1.7 MB 00:00 2023-06-09T11:07:21.618 INFO:teuthology.orchestra.run.smithi007.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 640 kB/s | 43 kB 00:00 2023-06-09T11:07:21.676 INFO:teuthology.orchestra.run.smithi007.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 967 kB/s | 48 kB 00:00 2023-06-09T11:07:21.702 INFO:teuthology.orchestra.run.smithi007.stdout:(75/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.0 MB/s | 220 kB 00:00 2023-06-09T11:07:21.727 INFO:teuthology.orchestra.run.smithi007.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-09T11:07:21.752 INFO:teuthology.orchestra.run.smithi007.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2023-06-09T11:07:21.777 INFO:teuthology.orchestra.run.smithi007.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-06-09T11:07:21.811 INFO:teuthology.orchestra.run.smithi007.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-06-09T11:07:21.845 INFO:teuthology.orchestra.run.smithi007.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-06-09T11:07:21.870 INFO:teuthology.orchestra.run.smithi007.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 320 kB/s | 19 kB 00:00 2023-06-09T11:07:21.895 INFO:teuthology.orchestra.run.smithi007.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-06-09T11:07:21.929 INFO:teuthology.orchestra.run.smithi007.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-06-09T11:07:21.954 INFO:teuthology.orchestra.run.smithi007.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 304 kB/s | 18 kB 00:00 2023-06-09T11:07:22.005 INFO:teuthology.orchestra.run.smithi007.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 7.8 MB/s | 605 kB 00:00 2023-06-09T11:07:22.039 INFO:teuthology.orchestra.run.smithi007.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2023-06-09T11:07:22.089 INFO:teuthology.orchestra.run.smithi007.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-06-09T11:07:22.147 INFO:teuthology.orchestra.run.smithi007.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-06-09T11:07:22.173 INFO:teuthology.orchestra.run.smithi007.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-06-09T11:07:22.198 INFO:teuthology.orchestra.run.smithi007.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-06-09T11:07:22.223 INFO:teuthology.orchestra.run.smithi007.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-09T11:07:22.248 INFO:teuthology.orchestra.run.smithi007.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-09T11:07:22.274 INFO:teuthology.orchestra.run.smithi007.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-06-09T11:07:22.316 INFO:teuthology.orchestra.run.smithi007.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-06-09T11:07:22.341 INFO:teuthology.orchestra.run.smithi007.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-06-09T11:07:22.366 INFO:teuthology.orchestra.run.smithi007.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-06-09T11:07:22.417 INFO:teuthology.orchestra.run.smithi007.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.2 MB/s | 110 kB 00:00 2023-06-09T11:07:22.467 INFO:teuthology.orchestra.run.smithi007.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-06-09T11:07:22.526 INFO:teuthology.orchestra.run.smithi007.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.4 MB/s | 259 kB 00:00 2023-06-09T11:07:22.585 INFO:teuthology.orchestra.run.smithi007.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-06-09T11:07:22.626 INFO:teuthology.orchestra.run.smithi007.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 938 kB/s | 39 kB 00:00 2023-06-09T11:07:22.702 INFO:teuthology.orchestra.run.smithi007.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 9.0 MB/s | 688 kB 00:00 2023-06-09T11:07:22.768 INFO:teuthology.orchestra.run.smithi007.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.9 MB/s | 538 kB 00:00 2023-06-09T11:07:22.927 INFO:teuthology.orchestra.run.smithi007.stdout:(104/131): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2023-06-09T11:07:22.952 INFO:teuthology.orchestra.run.smithi007.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 855 kB/s | 157 kB 00:00 2023-06-09T11:07:22.978 INFO:teuthology.orchestra.run.smithi007.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-06-09T11:07:23.028 INFO:teuthology.orchestra.run.smithi007.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-06-09T11:07:23.078 INFO:teuthology.orchestra.run.smithi007.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 533 kB/s | 27 kB 00:00 2023-06-09T11:07:23.137 INFO:teuthology.orchestra.run.smithi007.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 5.4 MB/s | 320 kB 00:00 2023-06-09T11:07:23.229 INFO:teuthology.orchestra.run.smithi007.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2023-06-09T11:07:23.256 INFO:teuthology.orchestra.run.smithi007.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 369 kB/s | 44 kB 00:00 2023-06-09T11:07:23.323 INFO:teuthology.orchestra.run.smithi007.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2023-06-09T11:07:23.399 INFO:teuthology.orchestra.run.smithi007.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 22 MB/s | 3.7 MB 00:00 2023-06-09T11:07:23.424 INFO:teuthology.orchestra.run.smithi007.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-06-09T11:07:23.451 INFO:teuthology.orchestra.run.smithi007.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-06-09T11:07:23.474 INFO:teuthology.orchestra.run.smithi007.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-06-09T11:07:23.500 INFO:teuthology.orchestra.run.smithi007.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-06-09T11:07:23.525 INFO:teuthology.orchestra.run.smithi007.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-06-09T11:07:23.559 INFO:teuthology.orchestra.run.smithi007.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-09T11:07:23.609 INFO:teuthology.orchestra.run.smithi007.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-06-09T11:07:23.801 INFO:teuthology.orchestra.run.smithi007.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 18 MB/s | 4.9 MB 00:00 2023-06-09T11:07:23.851 INFO:teuthology.orchestra.run.smithi007.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2023-06-09T11:07:23.943 INFO:teuthology.orchestra.run.smithi007.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-06-09T11:07:23.977 INFO:teuthology.orchestra.run.smithi007.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-06-09T11:07:24.052 INFO:teuthology.orchestra.run.smithi007.stdout:(125/131): postgresql-contrib-10.21-2.module+el 11 MB/s | 808 kB 00:00 2023-06-09T11:07:24.169 INFO:teuthology.orchestra.run.smithi007.stdout:(126/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-06-09T11:07:24.212 INFO:teuthology.orchestra.run.smithi007.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 9.5 MB/s | 1.5 MB 00:00 2023-06-09T11:07:24.219 INFO:teuthology.orchestra.run.smithi117.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T11:07:24.221 INFO:teuthology.orchestra.run.smithi117.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-09T11:07:24.236 INFO:teuthology.orchestra.run.smithi117.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-09T11:07:24.237 INFO:teuthology.orchestra.run.smithi117.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-09T11:07:24.238 INFO:teuthology.orchestra.run.smithi117.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-09T11:07:24.239 INFO:teuthology.orchestra.run.smithi117.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-09T11:07:24.242 INFO:teuthology.orchestra.run.smithi117.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-09T11:07:24.245 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-09T11:07:24.246 INFO:teuthology.orchestra.run.smithi117.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-09T11:07:24.386 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2023-06-09T11:07:24.398 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout: Package Arch Version Repository Size 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-09T11:07:24.399 INFO:teuthology.orchestra.run.smithi117.stdout: ceph x86_64 2:18.0.0-3924.g03410882.el8 ceph 6.4 k 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base x86_64 2:18.0.0-3924.g03410882.el8 ceph 5.3 M 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse x86_64 2:18.0.0-3924.g03410882.el8 ceph 848 k 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3924.g03410882.el8 ceph 165 k 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr x86_64 2:18.0.0-3924.g03410882.el8 ceph 1.5 M 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 127 k 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 1.7 M 2023-06-09T11:07:24.400 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 7.4 M 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 47 k 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw x86_64 2:18.0.0-3924.g03410882.el8 ceph 13 M 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test x86_64 2:18.0.0-3924.g03410882.el8 ceph 41 M 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 265 k 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 83 k 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror x86_64 2:18.0.0-3924.g03410882.el8 ceph 258 k 2023-06-09T11:07:24.401 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 18 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel x86_64 2:18.0.0-3924.g03410882.el8 ceph 31 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 771 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel x86_64 2:18.0.0-3924.g03410882.el8 ceph 139 k 2023-06-09T11:07:24.402 INFO:teuthology.orchestra.run.smithi117.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs x86_64 2:18.0.0-3924.g03410882.el8 ceph 197 k 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados x86_64 2:18.0.0-3924.g03410882.el8 ceph 371 k 2023-06-09T11:07:24.403 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd x86_64 2:18.0.0-3924.g03410882.el8 ceph 365 k 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw x86_64 2:18.0.0-3924.g03410882.el8 ceph 107 k 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse x86_64 2:18.0.0-3924.g03410882.el8 ceph 94 k 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.4 M 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd x86_64 2:18.0.0-3924.g03410882.el8 ceph 180 k 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrading: 2023-06-09T11:07:24.404 INFO:teuthology.orchestra.run.smithi117.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: librados2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.7 M 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1 x86_64 2:18.0.0-3924.g03410882.el8 ceph 3.6 M 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common x86_64 2:18.0.0-3924.g03410882.el8 ceph 22 M 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 23 k 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds x86_64 2:18.0.0-3924.g03410882.el8 ceph 2.3 M 2023-06-09T11:07:24.405 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 240 k 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon x86_64 2:18.0.0-3924.g03410882.el8 ceph 4.6 M 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd x86_64 2:18.0.0-3924.g03410882.el8 ceph 18 M 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3924.g03410882.el8 ceph-noarch 15 k 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux x86_64 2:18.0.0-3924.g03410882.el8 ceph 24 k 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-09T11:07:24.406 INFO:teuthology.orchestra.run.smithi117.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite x86_64 2:18.0.0-3924.g03410882.el8 ceph 137 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-09T11:07:24.407 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1 x86_64 2:18.0.0-3924.g03410882.el8 ceph 473 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2 x86_64 2:18.0.0-3924.g03410882.el8 ceph 6.5 M 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-09T11:07:24.408 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-09T11:07:24.409 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse x86_64 2:18.0.0-3924.g03410882.el8 ceph 45 k 2023-06-09T11:07:24.410 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common x86_64 2:18.0.0-3924.g03410882.el8 ceph 119 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-09T11:07:24.411 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-09T11:07:24.412 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-09T11:07:24.413 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-09T11:07:24.414 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-09T11:07:24.415 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-09T11:07:24.416 INFO:teuthology.orchestra.run.smithi117.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling module streams: 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql 10 2023-06-09T11:07:24.417 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:07:24.418 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2023-06-09T11:07:24.418 INFO:teuthology.orchestra.run.smithi117.stdout:=========================================================================================================================== 2023-06-09T11:07:24.418 INFO:teuthology.orchestra.run.smithi117.stdout:Install 128 Packages 2023-06-09T11:07:24.418 INFO:teuthology.orchestra.run.smithi117.stdout:Upgrade 3 Packages 2023-06-09T11:07:24.418 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:07:24.419 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 197 M 2023-06-09T11:07:24.419 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2023-06-09T11:07:24.903 INFO:teuthology.orchestra.run.smithi117.stdout:(1/131): ceph-18.0.0-3924.g03410882.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-06-09T11:07:25.511 INFO:teuthology.orchestra.run.smithi007.stdout:(128/131): librbd1-18.0.0-3924.g03410882.el8.x8 2.7 MB/s | 3.6 MB 00:01 2023-06-09T11:07:25.578 INFO:teuthology.orchestra.run.smithi007.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-06-09T11:07:25.886 INFO:teuthology.orchestra.run.smithi117.stdout:(2/131): ceph-fuse-18.0.0-3924.g03410882.el8.x8 863 kB/s | 848 kB 00:00 2023-06-09T11:07:26.028 INFO:teuthology.orchestra.run.smithi117.stdout:(3/131): ceph-immutable-object-cache-18.0.0-392 1.1 MB/s | 165 kB 00:00 2023-06-09T11:07:26.303 INFO:teuthology.orchestra.run.smithi007.stdout:(130/131): librados2-18.0.0-3924.g03410882.el8. 1.7 MB/s | 3.7 MB 00:02 2023-06-09T11:07:26.811 INFO:teuthology.orchestra.run.smithi117.stdout:(4/131): ceph-mds-18.0.0-3924.g03410882.el8.x86 2.9 MB/s | 2.3 MB 00:00 2023-06-09T11:07:27.261 INFO:teuthology.orchestra.run.smithi117.stdout:(5/131): ceph-mgr-18.0.0-3924.g03410882.el8.x86 3.4 MB/s | 1.5 MB 00:00 2023-06-09T11:07:27.336 INFO:teuthology.orchestra.run.smithi117.stdout:(6/131): ceph-base-18.0.0-3924.g03410882.el8.x8 1.9 MB/s | 5.3 MB 00:02 2023-06-09T11:07:28.187 INFO:teuthology.orchestra.run.smithi117.stdout:(7/131): ceph-mon-18.0.0-3924.g03410882.el8.x86 5.0 MB/s | 4.6 MB 00:00 2023-06-09T11:07:29.828 INFO:teuthology.orchestra.run.smithi117.stdout:(8/131): ceph-common-18.0.0-3924.g03410882.el8. 4.1 MB/s | 22 MB 00:05 2023-06-09T11:07:29.913 INFO:teuthology.orchestra.run.smithi117.stdout:(9/131): ceph-selinux-18.0.0-3924.g03410882.el8 293 kB/s | 24 kB 00:00 2023-06-09T11:07:30.447 INFO:teuthology.orchestra.run.smithi117.stdout:(10/131): ceph-radosgw-18.0.0-3924.g03410882.el 5.8 MB/s | 13 MB 00:02 2023-06-09T11:07:30.545 INFO:teuthology.orchestra.run.smithi117.stdout:(11/131): cephfs-mirror-18.0.0-3924.g03410882.e 2.5 MB/s | 258 kB 00:00 2023-06-09T11:07:30.637 INFO:teuthology.orchestra.run.smithi117.stdout:(12/131): libcephfs-devel-18.0.0-3924.g03410882 338 kB/s | 31 kB 00:00 2023-06-09T11:07:31.255 INFO:teuthology.orchestra.run.smithi117.stdout:(13/131): libcephfs2-18.0.0-3924.g03410882.el8. 1.2 MB/s | 771 kB 00:00 2023-06-09T11:07:31.346 INFO:teuthology.orchestra.run.smithi117.stdout:(14/131): libcephsqlite-18.0.0-3924.g03410882.e 1.5 MB/s | 137 kB 00:00 2023-06-09T11:07:31.446 INFO:teuthology.orchestra.run.smithi117.stdout:(15/131): librados-devel-18.0.0-3924.g03410882. 1.4 MB/s | 139 kB 00:00 2023-06-09T11:07:31.688 INFO:teuthology.orchestra.run.smithi117.stdout:(16/131): ceph-osd-18.0.0-3924.g03410882.el8.x8 4.1 MB/s | 18 MB 00:04 2023-06-09T11:07:31.722 INFO:teuthology.orchestra.run.smithi117.stdout:(17/131): libradosstriper1-18.0.0-3924.g0341088 1.7 MB/s | 473 kB 00:00 2023-06-09T11:07:31.872 INFO:teuthology.orchestra.run.smithi117.stdout:(18/131): python3-ceph-argparse-18.0.0-3924.g03 298 kB/s | 45 kB 00:00 2023-06-09T11:07:31.973 INFO:teuthology.orchestra.run.smithi117.stdout:(19/131): python3-ceph-common-18.0.0-3924.g0341 1.2 MB/s | 119 kB 00:00 2023-06-09T11:07:32.082 INFO:teuthology.orchestra.run.smithi117.stdout:(20/131): python3-cephfs-18.0.0-3924.g03410882. 1.8 MB/s | 197 kB 00:00 2023-06-09T11:07:32.456 INFO:teuthology.orchestra.run.smithi117.stdout:(21/131): python3-rados-18.0.0-3924.g03410882.e 990 kB/s | 371 kB 00:00 2023-06-09T11:07:32.623 INFO:teuthology.orchestra.run.smithi117.stdout:(22/131): python3-rbd-18.0.0-3924.g03410882.el8 2.1 MB/s | 365 kB 00:00 2023-06-09T11:07:32.715 INFO:teuthology.orchestra.run.smithi117.stdout:(23/131): python3-rgw-18.0.0-3924.g03410882.el8 1.1 MB/s | 107 kB 00:00 2023-06-09T11:07:32.815 INFO:teuthology.orchestra.run.smithi117.stdout:(24/131): rbd-fuse-18.0.0-3924.g03410882.el8.x8 944 kB/s | 94 kB 00:00 2023-06-09T11:07:33.207 INFO:teuthology.orchestra.run.smithi117.stdout:(25/131): librgw2-18.0.0-3924.g03410882.el8.x86 4.3 MB/s | 6.5 MB 00:01 2023-06-09T11:07:33.291 INFO:teuthology.orchestra.run.smithi117.stdout:(26/131): rbd-nbd-18.0.0-3924.g03410882.el8.x86 2.1 MB/s | 180 kB 00:00 2023-06-09T11:07:33.383 INFO:teuthology.orchestra.run.smithi117.stdout:(27/131): ceph-grafana-dashboards-18.0.0-3924.g 251 kB/s | 23 kB 00:00 2023-06-09T11:07:33.475 INFO:teuthology.orchestra.run.smithi117.stdout:(28/131): ceph-mgr-cephadm-18.0.0-3924.g0341088 1.4 MB/s | 127 kB 00:00 2023-06-09T11:07:34.041 INFO:teuthology.orchestra.run.smithi117.stdout:(29/131): ceph-mgr-dashboard-18.0.0-3924.g03410 3.0 MB/s | 1.7 MB 00:00 2023-06-09T11:07:34.367 INFO:teuthology.orchestra.run.smithi117.stdout:(30/131): rbd-mirror-18.0.0-3924.g03410882.el8. 2.2 MB/s | 3.4 MB 00:01 2023-06-09T11:07:34.625 INFO:teuthology.orchestra.run.smithi117.stdout:(31/131): ceph-mgr-modules-core-18.0.0-3924.g03 931 kB/s | 240 kB 00:00 2023-06-09T11:07:34.725 INFO:teuthology.orchestra.run.smithi117.stdout:(32/131): ceph-mgr-rook-18.0.0-3924.g03410882.e 474 kB/s | 47 kB 00:00 2023-06-09T11:07:34.826 INFO:teuthology.orchestra.run.smithi117.stdout:(33/131): ceph-prometheus-alerts-18.0.0-3924.g0 145 kB/s | 15 kB 00:00 2023-06-09T11:07:35.259 INFO:teuthology.orchestra.run.smithi117.stdout:(34/131): ceph-volume-18.0.0-3924.g03410882.el8 611 kB/s | 265 kB 00:00 2023-06-09T11:07:35.343 INFO:teuthology.orchestra.run.smithi117.stdout:(35/131): cephadm-18.0.0-3924.g03410882.el8.noa 1.0 MB/s | 83 kB 00:00 2023-06-09T11:07:35.443 INFO:teuthology.orchestra.run.smithi117.stdout:(36/131): cephfs-top-18.0.0-3924.g03410882.el8. 180 kB/s | 18 kB 00:00 2023-06-09T11:07:35.860 INFO:teuthology.orchestra.run.smithi117.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 735 kB/s | 306 kB 00:00 2023-06-09T11:07:35.943 INFO:teuthology.orchestra.run.smithi117.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 707 kB/s | 59 kB 00:00 2023-06-09T11:07:36.027 INFO:teuthology.orchestra.run.smithi117.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-06-09T11:07:36.186 INFO:teuthology.orchestra.run.smithi117.stdout:(40/131): ceph-mgr-diskprediction-local-18.0.0- 3.5 MB/s | 7.4 MB 00:02 2023-06-09T11:07:36.211 INFO:teuthology.orchestra.run.smithi117.stdout:(41/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.5 MB/s | 476 kB 00:00 2023-06-09T11:07:36.303 INFO:teuthology.orchestra.run.smithi117.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-06-09T11:07:36.395 INFO:teuthology.orchestra.run.smithi117.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 360 kB/s | 33 kB 00:00 2023-06-09T11:07:36.453 INFO:teuthology.orchestra.run.smithi117.stdout:(44/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 163 kB/s | 44 kB 00:00 2023-06-09T11:07:36.479 INFO:teuthology.orchestra.run.smithi117.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 196 kB/s | 16 kB 00:00 2023-06-09T11:07:36.571 INFO:teuthology.orchestra.run.smithi117.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 4.1 MB/s | 384 kB 00:00 2023-06-09T11:07:36.638 INFO:teuthology.orchestra.run.smithi117.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 940 kB/s | 173 kB 00:00 2023-06-09T11:07:36.665 INFO:teuthology.orchestra.run.smithi117.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 883 kB/s | 81 kB 00:00 2023-06-09T11:07:36.714 INFO:teuthology.orchestra.run.smithi117.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2023-06-09T11:07:36.747 INFO:teuthology.orchestra.run.smithi117.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-06-09T11:07:36.830 INFO:teuthology.orchestra.run.smithi117.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2023-06-09T11:07:37.014 INFO:teuthology.orchestra.run.smithi117.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 320 kB/s | 59 kB 00:00 2023-06-09T11:07:37.065 INFO:teuthology.orchestra.run.smithi117.stdout:(53/131): python3-kubernetes-11.0.0-6.el8.noarc 3.3 MB/s | 1.2 MB 00:00 2023-06-09T11:07:37.098 INFO:teuthology.orchestra.run.smithi117.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 721 kB/s | 60 kB 00:00 2023-06-09T11:07:37.173 INFO:teuthology.orchestra.run.smithi117.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-06-09T11:07:37.190 INFO:teuthology.orchestra.run.smithi117.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 169 kB/s | 16 kB 00:00 2023-06-09T11:07:37.266 INFO:teuthology.orchestra.run.smithi117.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 367 kB/s | 34 kB 00:00 2023-06-09T11:07:37.299 INFO:teuthology.orchestra.run.smithi117.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-06-09T11:07:37.366 INFO:teuthology.orchestra.run.smithi117.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 616 kB/s | 61 kB 00:00 2023-06-09T11:07:37.408 INFO:teuthology.orchestra.run.smithi117.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-06-09T11:07:37.475 INFO:teuthology.orchestra.run.smithi117.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-06-09T11:07:37.500 INFO:teuthology.orchestra.run.smithi117.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-06-09T11:07:37.559 INFO:teuthology.orchestra.run.smithi117.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 329 kB/s | 27 kB 00:00 2023-06-09T11:07:37.592 INFO:teuthology.orchestra.run.smithi117.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-06-09T11:07:37.651 INFO:teuthology.orchestra.run.smithi117.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2023-06-09T11:07:37.693 INFO:teuthology.orchestra.run.smithi117.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 612 kB/s | 61 kB 00:00 2023-06-09T11:07:37.752 INFO:teuthology.orchestra.run.smithi117.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 853 kB/s | 85 kB 00:00 2023-06-09T11:07:37.810 INFO:teuthology.orchestra.run.smithi117.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-06-09T11:07:37.827 INFO:teuthology.orchestra.run.smithi117.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.2 MB/s | 70 kB 00:00 2023-06-09T11:07:37.986 INFO:teuthology.orchestra.run.smithi117.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-06-09T11:07:38.044 INFO:teuthology.orchestra.run.smithi117.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-06-09T11:07:38.078 INFO:teuthology.orchestra.run.smithi117.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-06-09T11:07:38.120 INFO:teuthology.orchestra.run.smithi117.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-09T11:07:38.170 INFO:teuthology.orchestra.run.smithi117.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-06-09T11:07:38.204 INFO:teuthology.orchestra.run.smithi117.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-06-09T11:07:38.354 INFO:teuthology.orchestra.run.smithi117.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-06-09T11:07:38.396 INFO:teuthology.orchestra.run.smithi117.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.5 MB/s | 688 kB 00:00 2023-06-09T11:07:38.438 INFO:teuthology.orchestra.run.smithi117.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 6.3 MB/s | 538 kB 00:00 2023-06-09T11:07:38.464 INFO:teuthology.orchestra.run.smithi117.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-06-09T11:07:38.489 INFO:teuthology.orchestra.run.smithi117.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-06-09T11:07:38.522 INFO:teuthology.orchestra.run.smithi117.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-06-09T11:07:38.564 INFO:teuthology.orchestra.run.smithi117.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 636 kB/s | 27 kB 00:00 2023-06-09T11:07:38.606 INFO:teuthology.orchestra.run.smithi117.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-09T11:07:38.648 INFO:teuthology.orchestra.run.smithi117.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-06-09T11:07:38.732 INFO:teuthology.orchestra.run.smithi117.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-06-09T11:07:38.790 INFO:teuthology.orchestra.run.smithi117.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-06-09T11:07:38.832 INFO:teuthology.orchestra.run.smithi117.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-06-09T11:07:38.908 INFO:teuthology.orchestra.run.smithi117.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-06-09T11:07:38.933 INFO:teuthology.orchestra.run.smithi117.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 638 kB/s | 64 kB 00:00 2023-06-09T11:07:38.958 INFO:teuthology.orchestra.run.smithi117.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-06-09T11:07:38.983 INFO:teuthology.orchestra.run.smithi117.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-06-09T11:07:39.009 INFO:teuthology.orchestra.run.smithi117.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-06-09T11:07:39.042 INFO:teuthology.orchestra.run.smithi117.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-09T11:07:39.084 INFO:teuthology.orchestra.run.smithi117.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-06-09T11:07:39.226 INFO:teuthology.orchestra.run.smithi117.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-06-09T11:07:39.268 INFO:teuthology.orchestra.run.smithi117.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-06-09T11:07:39.343 INFO:teuthology.orchestra.run.smithi117.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-06-09T11:07:39.377 INFO:teuthology.orchestra.run.smithi117.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-06-09T11:07:39.436 INFO:teuthology.orchestra.run.smithi117.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-06-09T11:07:39.511 INFO:teuthology.orchestra.run.smithi117.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 20 MB/s | 1.5 MB 00:00 2023-06-09T11:07:39.544 INFO:teuthology.orchestra.run.smithi117.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-06-09T11:07:39.611 INFO:teuthology.orchestra.run.smithi117.stdout:(102/131): postgresql-server-10.21-2.module+el8 19 MB/s | 5.1 MB 00:00 2023-06-09T11:07:39.637 INFO:teuthology.orchestra.run.smithi117.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2023-06-09T11:07:39.662 INFO:teuthology.orchestra.run.smithi117.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 851 kB/s | 43 kB 00:00 2023-06-09T11:07:39.687 INFO:teuthology.orchestra.run.smithi117.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-09T11:07:39.714 INFO:teuthology.orchestra.run.smithi117.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 961 kB/s | 48 kB 00:00 2023-06-09T11:07:39.738 INFO:teuthology.orchestra.run.smithi117.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-09T11:07:39.755 INFO:teuthology.orchestra.run.smithi117.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-06-09T11:07:39.780 INFO:teuthology.orchestra.run.smithi117.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.6 MB/s | 109 kB 00:00 2023-06-09T11:07:39.805 INFO:teuthology.orchestra.run.smithi117.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-06-09T11:07:39.839 INFO:teuthology.orchestra.run.smithi117.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-06-09T11:07:39.856 INFO:teuthology.orchestra.run.smithi117.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 370 kB/s | 19 kB 00:00 2023-06-09T11:07:39.891 INFO:teuthology.orchestra.run.smithi117.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-06-09T11:07:39.923 INFO:teuthology.orchestra.run.smithi117.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-06-09T11:07:39.949 INFO:teuthology.orchestra.run.smithi117.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 304 kB/s | 18 kB 00:00 2023-06-09T11:07:39.982 INFO:teuthology.orchestra.run.smithi117.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-06-09T11:07:40.008 INFO:teuthology.orchestra.run.smithi117.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-09T11:07:40.057 INFO:teuthology.orchestra.run.smithi117.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-06-09T11:07:40.099 INFO:teuthology.orchestra.run.smithi117.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-06-09T11:07:40.150 INFO:teuthology.orchestra.run.smithi117.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 14 MB/s | 2.3 MB 00:00 2023-06-09T11:07:40.175 INFO:teuthology.orchestra.run.smithi117.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 797 kB/s | 60 kB 00:00 2023-06-09T11:07:40.200 INFO:teuthology.orchestra.run.smithi117.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-06-09T11:07:40.225 INFO:teuthology.orchestra.run.smithi117.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-06-09T11:07:40.251 INFO:teuthology.orchestra.run.smithi117.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 769 kB/s | 39 kB 00:00 2023-06-09T11:07:40.284 INFO:teuthology.orchestra.run.smithi117.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-06-09T11:07:40.310 INFO:teuthology.orchestra.run.smithi117.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-06-09T11:07:40.335 INFO:teuthology.orchestra.run.smithi117.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-06-09T11:07:41.960 INFO:teuthology.orchestra.run.smithi117.stdout:(128/131): librbd1-18.0.0-3924.g03410882.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-06-09T11:07:42.386 INFO:teuthology.orchestra.run.smithi117.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 1.8 MB/s | 798 kB 00:00 2023-06-09T11:07:42.761 INFO:teuthology.orchestra.run.smithi117.stdout:(130/131): ceph-test-18.0.0-3924.g03410882.el8. 3.2 MB/s | 41 MB 00:12 2023-06-09T11:07:43.136 INFO:teuthology.orchestra.run.smithi117.stdout:(131/131): librados2-18.0.0-3924.g03410882.el8. 1.3 MB/s | 3.7 MB 00:02 2023-06-09T11:07:43.139 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2023-06-09T11:07:43.140 INFO:teuthology.orchestra.run.smithi117.stdout:Total 11 MB/s | 197 MB 00:18 2023-06-09T11:07:44.763 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2023-06-09T11:07:44.897 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2023-06-09T11:07:44.898 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2023-06-09T11:07:46.675 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2023-06-09T11:07:46.679 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2023-06-09T11:07:51.503 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2023-06-09T11:07:51.681 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-09T11:07:51.856 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-09T11:07:52.060 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-06-09T11:07:52.224 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-09T11:07:52.509 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-09T11:07:52.707 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-09T11:07:53.167 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-09T11:07:53.366 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librados2-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:07:53.808 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:07:53.974 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : librbd1-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:07:54.119 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:07:54.307 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 8/134 2023-06-09T11:07:54.430 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 8/134 2023-06-09T11:07:54.612 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 9/134 2023-06-09T11:07:54.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-09T11:07:55.070 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-09T11:07:55.516 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-09T11:07:55.737 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-09T11:07:55.907 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 14/134 2023-06-09T11:07:56.030 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 14/134 2023-06-09T11:07:56.158 INFO:teuthology.orchestra.run.smithi007.stdout:(131/131): ceph-test-18.0.0-3924.g03410882.el8. 1.0 MB/s | 41 MB 00:39 2023-06-09T11:07:56.161 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2023-06-09T11:07:56.163 INFO:teuthology.orchestra.run.smithi007.stdout:Total 4.7 MB/s | 197 MB 00:41 2023-06-09T11:07:56.182 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 15/134 2023-06-09T11:07:56.281 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 15/134 2023-06-09T11:07:56.423 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-09T11:07:56.584 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-09T11:07:56.706 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-09T11:07:57.395 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-09T11:07:57.531 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-09T11:07:57.820 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2023-06-09T11:07:57.967 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2023-06-09T11:07:57.967 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2023-06-09T11:07:58.219 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-09T11:07:58.397 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-06-09T11:07:59.235 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-06-09T11:07:59.525 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-06-09T11:07:59.711 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2023-06-09T11:07:59.716 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2023-06-09T11:07:59.756 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-06-09T11:07:59.979 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-06-09T11:08:00.148 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-06-09T11:08:00.364 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-06-09T11:08:00.591 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-09T11:08:00.746 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-common-2:18.0.0-3924.g03410882.el8. 26/134 2023-06-09T11:08:00.877 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-ceph-argparse-2:18.0.0-3924.g03410882.el 27/134 2023-06-09T11:08:01.883 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_6 28/134 2023-06-09T11:08:02.226 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-06-09T11:08:02.496 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-06-09T11:08:02.768 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-06-09T11:08:02.955 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-09T11:08:04.600 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2023-06-09T11:08:04.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-09T11:08:04.972 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-09T11:08:05.017 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-09T11:08:05.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-09T11:08:05.279 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-09T11:08:05.346 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-06-09T11:08:05.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-09T11:08:05.557 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-09T11:08:05.752 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-09T11:08:05.912 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 36/134 2023-06-09T11:08:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-09T11:08:06.068 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librados-devel-2:18.0.0-3924.g03410882.el8.x86_6 37/134 2023-06-09T11:08:06.219 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:18.0.0-3924.g03410882.el8.x86_64 5/134 2023-06-09T11:08:06.225 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-06-09T11:08:06.399 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-06-09T11:08:06.565 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-06-09T11:08:06.701 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-06-09T11:08:06.702 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:18.0.0-3924.g03410882.el8.x86_64 5/134 2023-06-09T11:08:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:08:06.889 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-06-09T11:08:07.011 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:08:07.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-06-09T11:08:07.202 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:08:07.330 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:08:07.488 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-06-09T11:08:07.508 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 8/134 2023-06-09T11:08:07.674 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-09T11:08:07.753 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-06-09T11:08:07.931 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-09T11:08:07.943 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-06-09T11:08:08.173 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-06-09T11:08:08.175 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-09T11:08:08.388 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-06-09T11:08:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-09T11:08:08.703 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-06-09T11:08:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-09T11:08:08.955 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-06-09T11:08:09.059 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 14/134 2023-06-09T11:08:09.123 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-06-09T11:08:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 14/134 2023-06-09T11:08:09.331 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-06-09T11:08:09.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 15/134 2023-06-09T11:08:09.503 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-06-09T11:08:09.541 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 15/134 2023-06-09T11:08:09.659 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-06-09T11:08:09.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-09T11:08:09.945 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-09T11:08:09.976 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-06-09T11:08:10.121 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-09T11:08:10.223 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-06-09T11:08:10.234 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-09T11:08:10.395 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-09T11:08:10.476 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-06-09T11:08:10.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-09T11:08:10.711 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-06-09T11:08:10.879 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-06-09T11:08:11.016 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-06-09T11:08:11.169 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-06-09T11:08:11.245 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-06-09T11:08:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-09T11:08:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-09T11:08:11.463 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-06-09T11:08:11.659 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-06-09T11:08:11.795 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-09T11:08:11.891 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-09T11:08:12.109 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-06-09T11:08:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-09T11:08:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-09T11:08:12.322 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-06-09T11:08:12.496 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-09T11:08:12.701 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-09T11:08:13.196 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-09T11:08:13.310 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-09T11:08:13.451 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-09T11:08:13.519 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-09T11:08:13.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-09T11:08:13.666 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-09T11:08:13.798 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-09T11:08:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-09T11:08:13.982 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-09T11:08:14.059 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:18.0.0-3924.g03410882.el8. 26/134 2023-06-09T11:08:14.151 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-09T11:08:14.181 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:18.0.0-3924.g03410882.el 27/134 2023-06-09T11:08:14.316 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-09T11:08:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_6 28/134 2023-06-09T11:08:14.464 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-09T11:08:14.584 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-09T11:08:14.617 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-09T11:08:14.830 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-09T11:08:15.024 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-09T11:08:15.208 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-09T11:08:15.477 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-09T11:08:15.686 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-09T11:08:15.853 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-09T11:08:16.065 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-09T11:08:16.825 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-09T11:08:17.308 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-09T11:08:17.571 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-09T11:08:17.815 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-09T11:08:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-09T11:08:18.123 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-09T11:08:18.418 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-09T11:08:18.423 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-09T11:08:18.638 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-09T11:08:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-09T11:08:18.905 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-09T11:08:19.149 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-09T11:08:19.391 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-09T11:08:19.436 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-09T11:08:19.551 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 36/134 2023-06-09T11:08:19.576 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : librgw2-2:18.0.0-3924.g03410882.el8.x86_64 92/134 2023-06-09T11:08:19.674 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librgw2-2:18.0.0-3924.g03410882.el8.x86_64 92/134 2023-06-09T11:08:19.725 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:18.0.0-3924.g03410882.el8.x86_6 37/134 2023-06-09T11:08:19.835 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 93/134 2023-06-09T11:08:19.868 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-09T11:08:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-09T11:08:20.458 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-09T11:08:20.552 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-09T11:08:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-09T11:08:21.006 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-09T11:08:21.149 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-09T11:08:21.263 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-09T11:08:21.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-09T11:08:21.663 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-09T11:08:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-09T11:08:22.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-09T11:08:22.211 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-09T11:08:22.356 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-09T11:08:22.541 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-09T11:08:22.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-09T11:08:23.160 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-09T11:08:23.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-09T11:08:23.674 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-09T11:08:23.952 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-09T11:08:24.204 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-09T11:08:24.530 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-09T11:08:24.545 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:24.742 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:24.743 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-09T11:08:24.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-09T11:08:25.112 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-09T11:08:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-09T11:08:25.432 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-09T11:08:25.744 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 95/134 2023-06-09T11:08:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-09T11:08:25.879 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 95/134 2023-06-09T11:08:26.025 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 96/134 2023-06-09T11:08:26.092 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-09T11:08:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-09T11:08:26.611 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-09T11:08:26.759 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-09T11:08:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-09T11:08:27.214 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-09T11:08:27.532 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-09T11:08:27.671 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-09T11:08:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-09T11:08:28.008 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-09T11:08:28.208 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-09T11:08:28.390 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-09T11:08:28.544 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-09T11:08:28.681 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-09T11:08:28.921 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-09T11:08:29.114 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-09T11:08:29.273 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-09T11:08:29.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-09T11:08:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-09T11:08:29.944 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-09T11:08:30.152 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-09T11:08:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-09T11:08:32.014 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-09T11:08:32.248 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-09T11:08:32.560 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-09T11:08:32.863 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-09T11:08:33.054 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-09T11:08:33.893 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-09T11:08:34.047 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:18.0.0-3924.g03410882.el8.x86_64 92/134 2023-06-09T11:08:34.145 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:18.0.0-3924.g03410882.el8.x86_64 92/134 2023-06-09T11:08:34.298 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 93/134 2023-06-09T11:08:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:40.261 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 94/134 2023-06-09T11:08:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 95/134 2023-06-09T11:08:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 95/134 2023-06-09T11:08:41.265 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 96/134 2023-06-09T11:08:46.860 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 96/134 2023-06-09T11:08:46.860 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /sys 2023-06-09T11:08:46.860 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /proc 2023-06-09T11:08:46.861 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2023-06-09T11:08:46.861 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /run 2023-06-09T11:08:46.861 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /mnt 2023-06-09T11:08:46.861 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /var/tmp 2023-06-09T11:08:46.862 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /home 2023-06-09T11:08:46.862 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /tmp 2023-06-09T11:08:46.862 INFO:teuthology.orchestra.run.smithi117.stdout:skipping the directory /dev 2023-06-09T11:08:46.863 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:08:47.221 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 97/134 2023-06-09T11:08:48.948 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 97/134 2023-06-09T11:08:49.106 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 98/134 2023-06-09T11:08:49.373 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 98/134 2023-06-09T11:08:49.520 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 99/134 2023-06-09T11:08:50.044 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 99/134 2023-06-09T11:08:50.228 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 100/134 2023-06-09T11:08:50.269 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 100/134 2023-06-09T11:08:52.830 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephadm-2:18.0.0-3924.g03410882.el8.noarch 101/134 2023-06-09T11:08:53.000 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephadm-2:18.0.0-3924.g03410882.el8.noarch 101/134 2023-06-09T11:08:53.135 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3924.g03410882.e 102/134 2023-06-09T11:08:53.332 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3924.g03410882. 103/134 2023-06-09T11:08:53.502 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 104/134 2023-06-09T11:08:53.767 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 104/134 2023-06-09T11:08:54.046 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 105/134 2023-06-09T11:08:55.204 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 105/134 2023-06-09T11:08:55.360 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 106/134 2023-06-09T11:08:55.490 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 106/134 2023-06-09T11:08:55.864 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el 107/134 2023-06-09T11:08:56.030 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 108/134 2023-06-09T11:08:56.095 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 108/134 2023-06-09T11:08:56.285 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 109/134 2023-06-09T11:08:56.312 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 109/134 2023-06-09T11:08:57.659 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-2:18.0.0-3924.g03410882.el8.x86_64 110/134 2023-06-09T11:08:57.876 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 111/134 2023-06-09T11:08:57.937 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 111/134 2023-06-09T11:08:58.077 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3924.g03410 112/134 2023-06-09T11:08:58.146 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3924.g03410 112/134 2023-06-09T11:08:58.293 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 113/134 2023-06-09T11:08:58.699 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 113/134 2023-06-09T11:08:58.851 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 114/134 2023-06-09T11:09:02.796 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 96/134 2023-06-09T11:09:02.797 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2023-06-09T11:09:02.797 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2023-06-09T11:09:02.797 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2023-06-09T11:09:02.797 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /run 2023-06-09T11:09:02.798 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2023-06-09T11:09:02.798 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2023-06-09T11:09:02.798 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2023-06-09T11:09:02.798 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2023-06-09T11:09:02.799 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /dev 2023-06-09T11:09:02.799 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:03.147 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 114/134 2023-06-09T11:09:03.177 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 97/134 2023-06-09T11:09:03.453 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 115/134 2023-06-09T11:09:03.608 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-06-09T11:09:03.875 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-06-09T11:09:05.037 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 97/134 2023-06-09T11:09:05.084 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-06-09T11:09:05.203 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 98/134 2023-06-09T11:09:05.395 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-06-09T11:09:05.487 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 98/134 2023-06-09T11:09:05.554 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 119/134 2023-06-09T11:09:05.642 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 99/134 2023-06-09T11:09:05.729 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-09T11:09:05.904 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_ 121/134 2023-06-09T11:09:06.034 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T11:09:06.203 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 99/134 2023-06-09T11:09:06.401 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 100/134 2023-06-09T11:09:06.443 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 100/134 2023-06-09T11:09:09.198 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:09.230 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:18.0.0-3924.g03410882.el8.noarch 101/134 2023-06-09T11:09:09.381 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:18.0.0-3924.g03410882.el8.noarch 101/134 2023-06-09T11:09:09.457 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:09.495 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:09.533 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3924.g03410882.e 102/134 2023-06-09T11:09:09.732 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3924.g03410882. 103/134 2023-06-09T11:09:09.793 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 124/134 2023-06-09T11:09:09.883 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 104/134 2023-06-09T11:09:09.975 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 125/134 2023-06-09T11:09:10.160 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 104/134 2023-06-09T11:09:10.283 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 126/134 2023-06-09T11:09:10.402 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 105/134 2023-06-09T11:09:10.575 INFO:teuthology.orchestra.run.smithi117.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-09T11:09:10.748 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T11:09:10.770 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T11:09:10.966 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-06-09T11:09:11.156 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-06-09T11:09:11.263 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-06-09T11:09:11.464 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-06-09T11:09:11.492 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-06-09T11:09:11.493 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:11.593 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 105/134 2023-06-09T11:09:11.705 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:11.705 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T11:09:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 106/134 2023-06-09T11:09:11.906 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 106/134 2023-06-09T11:09:11.937 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T11:09:12.106 INFO:teuthology.orchestra.run.smithi117.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el 107/134 2023-06-09T11:09:12.386 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:12.387 INFO:teuthology.orchestra.run.smithi117.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-06-09T11:09:12.387 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:12.477 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 108/134 2023-06-09T11:09:12.545 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 108/134 2023-06-09T11:09:12.692 INFO:teuthology.orchestra.run.smithi117.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-09T11:09:12.693 INFO:teuthology.orchestra.run.smithi117.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-09T11:09:12.693 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 109/134 2023-06-09T11:09:12.753 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 109/134 2023-06-09T11:09:13.004 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-2:18.0.0-3924.g03410882.el8.x86_64 1/134 2023-06-09T11:09:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 2/134 2023-06-09T11:09:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 3/134 2023-06-09T11:09:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 4/134 2023-06-09T11:09:13.008 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3924.g03410 5/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 8/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 9/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 10/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 11/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 12/134 2023-06-09T11:09:13.009 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 13/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_ 14/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 15/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 16/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados-devel-2:18.0.0-3924.g03410882.el8.x86_6 17/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 18/134 2023-06-09T11:09:13.010 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librgw2-2:18.0.0-3924.g03410882.el8.x86_64 19/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3924.g03410882.el 20/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-ceph-common-2:18.0.0-3924.g03410882.el8. 21/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_6 22/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 23/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 24/134 2023-06-09T11:09:13.011 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 25/134 2023-06-09T11:09:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 26/134 2023-06-09T11:09:13.018 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 27/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 28/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3924.g03410882. 29/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 30/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 31/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 32/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el 33/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 34/134 2023-06-09T11:09:13.019 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3924.g03410882.e 35/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 36/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephadm-2:18.0.0-3924.g03410882.el8.noarch 37/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 38/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-09T11:09:13.020 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-09T11:09:13.021 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-09T11:09:13.022 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-09T11:09:13.023 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-09T11:09:13.024 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-06-09T11:09:13.025 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-06-09T11:09:13.026 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-06-09T11:09:13.027 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-06-09T11:09:13.028 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-06-09T11:09:13.029 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-06-09T11:09:13.030 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-06-09T11:09:13.031 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-2:18.0.0-3924.g03410882.el8.x86_64 129/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-2:18.0.0-3924.g03410882.el8.x86_64 131/134 2023-06-09T11:09:13.032 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:13.033 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-09T11:09:14.195 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:18.0.0-3924.g03410882.el8.x86_64 110/134 2023-06-09T11:09:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 111/134 2023-06-09T11:09:14.479 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 111/134 2023-06-09T11:09:14.624 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3924.g03410 112/134 2023-06-09T11:09:14.697 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3924.g03410 112/134 2023-06-09T11:09:14.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 113/134 2023-06-09T11:09:15.276 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 113/134 2023-06-09T11:09:15.424 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 114/134 2023-06-09T11:09:15.789 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:15.789 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:02:03 ago on Fri 09 Jun 2023 11:07:12 AM UTC. 2023-06-09T11:09:18.443 INFO:teuthology.orchestra.run.smithi117.stdout:Installed products updated. 2023-06-09T11:09:18.690 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:18.690 INFO:teuthology.orchestra.run.smithi117.stdout:Upgraded: 2023-06-09T11:09:18.690 INFO:teuthology.orchestra.run.smithi117.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-09T11:09:18.690 INFO:teuthology.orchestra.run.smithi117.stdout: librados2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: librbd1-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: automake-1.16.1-7.el8.noarch 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: bison-3.0.4-10.el8.x86_64 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.691 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.692 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.692 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.692 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-grafana-dashboards-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.692 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-immutable-object-cache-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.692 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.693 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-prometheus-alerts-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: cephadm-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.694 INFO:teuthology.orchestra.run.smithi117.stdout: cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: flex-2.6.1-9.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: jq-1.6-3.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-09T11:09:18.695 INFO:teuthology.orchestra.run.smithi117.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-09T11:09:18.696 INFO:teuthology.orchestra.run.smithi117.stdout: libpq-13.5-1.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: librados-devel-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: libradosstriper1-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: librgw2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.697 INFO:teuthology.orchestra.run.smithi117.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-09T11:09:18.698 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:18.699 INFO:teuthology.orchestra.run.smithi117.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.700 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-09T11:09:18.701 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-09T11:09:18.702 INFO:teuthology.orchestra.run.smithi117.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-09T11:09:18.703 INFO:teuthology.orchestra.run.smithi117.stdout: python3-portend-2.6-1.el8.noarch 2023-06-09T11:09:18.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-09T11:09:18.704 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.705 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-09T11:09:18.706 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-09T11:09:18.707 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-09T11:09:18.708 INFO:teuthology.orchestra.run.smithi117.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-09T11:09:18.709 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-09T11:09:18.709 INFO:teuthology.orchestra.run.smithi117.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-09T11:09:18.709 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:18.709 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2023-06-09T11:09:18.996 DEBUG:teuthology.parallel:result is None 2023-06-09T11:09:20.039 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 114/134 2023-06-09T11:09:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 115/134 2023-06-09T11:09:21.682 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-09T11:09:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 117/134 2023-06-09T11:09:22.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-09T11:09:22.239 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-09T11:09:22.516 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-09T11:09:22.745 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-09T11:09:22.903 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_ 121/134 2023-06-09T11:09:23.057 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-09T11:09:26.482 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-09T11:09:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 124/134 2023-06-09T11:09:27.141 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 125/134 2023-06-09T11:09:27.440 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 126/134 2023-06-09T11:09:27.793 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-09T11:09:27.955 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-09T11:09:28.071 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-09T11:09:28.263 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-09T11:09:28.292 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-09T11:09:28.480 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-09T11:09:28.502 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-09T11:09:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-09T11:09:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:28.878 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T11:09:29.112 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-09T11:09:29.305 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:29.562 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-06-09T11:09:29.562 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-09T11:09:29.872 INFO:teuthology.orchestra.run.smithi007.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-09T11:09:29.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:18.0.0-3924.g03410882.el8.x86_64 1/134 2023-06-09T11:09:30.181 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 2/134 2023-06-09T11:09:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 3/134 2023-06-09T11:09:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 4/134 2023-06-09T11:09:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3924.g03410 5/134 2023-06-09T11:09:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 6/134 2023-06-09T11:09:30.182 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 7/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 8/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 9/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 10/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 11/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 12/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 13/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_ 14/134 2023-06-09T11:09:30.183 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 15/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 16/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:18.0.0-3924.g03410882.el8.x86_6 17/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:18.0.0-3924.g03410882.el8.x86 18/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:18.0.0-3924.g03410882.el8.x86_64 19/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3924.g03410882.el 20/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:18.0.0-3924.g03410882.el8. 21/134 2023-06-09T11:09:30.184 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_6 22/134 2023-06-09T11:09:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 23/134 2023-06-09T11:09:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 24/134 2023-06-09T11:09:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 25/134 2023-06-09T11:09:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 26/134 2023-06-09T11:09:30.185 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 27/134 2023-06-09T11:09:30.192 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 28/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3924.g03410882. 29/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noa 30/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.n 31/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3924.g034 32/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el 33/134 2023-06-09T11:09:30.193 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 34/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3924.g03410882.e 35/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 36/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:18.0.0-3924.g03410882.el8.noarch 37/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 38/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-09T11:09:30.194 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-09T11:09:30.195 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-09T11:09:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-09T11:09:30.196 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-09T11:09:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-09T11:09:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-09T11:09:30.197 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-09T11:09:30.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-09T11:09:30.199 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-09T11:09:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-09T11:09:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-09T11:09:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-09T11:09:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-09T11:09:30.200 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-09T11:09:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-09T11:09:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-09T11:09:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-09T11:09:30.201 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-09T11:09:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-09T11:09:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-09T11:09:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-09T11:09:30.202 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-09T11:09:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-09T11:09:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-09T11:09:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-09T11:09:30.203 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-09T11:09:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-09T11:09:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-09T11:09:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-09T11:09:30.204 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-09T11:09:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-09T11:09:30.206 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-09T11:09:30.207 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-09T11:09:30.208 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-09T11:09:30.209 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:18.0.0-3924.g03410882.el8.x86_64 129/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-09T11:09:30.210 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:18.0.0-3924.g03410882.el8.x86_64 131/134 2023-06-09T11:09:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-09T11:09:30.211 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-09T11:09:32.844 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-09T11:09:32.845 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:02:31 ago on Fri 09 Jun 2023 11:07:01 AM UTC. 2023-06-09T11:09:35.510 INFO:teuthology.orchestra.run.smithi007.stdout:Installed products updated. 2023-06-09T11:09:35.750 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2023-06-09T11:09:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.1-7.el8.noarch 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.0.4-10.el8.x86_64 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.752 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.753 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.754 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.755 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:18.0.0-3924.g03410882.el8.noarch 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.1-9.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: jq-1.6-3.el8.x86_64 2023-06-09T11:09:35.756 INFO:teuthology.orchestra.run.smithi007.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-09T11:09:35.757 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: libpq-13.5-1.el8.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.758 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-09T11:09:35.759 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-09T11:09:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-09T11:09:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-09T11:09:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-09T11:09:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-09T11:09:35.760 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:35.761 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-09T11:09:35.762 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-09T11:09:35.763 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-09T11:09:35.764 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-09T11:09:35.765 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-2.6-1.el8.noarch 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-09T11:09:35.766 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-09T11:09:35.767 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-09T11:09:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-09T11:09:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-09T11:09:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-09T11:09:35.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-09T11:09:35.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:18.0.0-3924.g03410882.el8.x86_64 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-09T11:09:35.770 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-09T11:09:35.771 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-09T11:09:35.771 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-09T11:09:35.771 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-09T11:09:35.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:35.772 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2023-06-09T11:09:36.769 DEBUG:teuthology.parallel:result is None 2023-06-09T11:09:36.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:09:36.910 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-09T11:09:36.966 INFO:teuthology.orchestra.run.smithi007.stdout:18.0.0-3924.g03410882.el8 2023-06-09T11:09:36.967 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3924.g03410882.el8 2023-06-09T11:09:36.968 INFO:teuthology.task.install:The correct ceph version 18.0.0-3924.g03410882 is installed. 2023-06-09T11:09:36.969 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:09:37.107 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-09T11:09:37.163 INFO:teuthology.orchestra.run.smithi117.stdout:18.0.0-3924.g03410882.el8 2023-06-09T11:09:37.163 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3924.g03410882.el8 2023-06-09T11:09:37.163 INFO:teuthology.task.install:The correct ceph version 18.0.0-3924.g03410882 is installed. 2023-06-09T11:09:37.164 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-09T11:09:37.165 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:37.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-09T11:09:37.256 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:37.257 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-09T11:09:37.345 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-09T11:09:37.346 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:37.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2023-06-09T11:09:37.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-09T11:09:37.526 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:37.527 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2023-06-09T11:09:37.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-09T11:09:37.708 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-09T11:09:37.709 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:37.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-09T11:09:37.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-09T11:09:37.891 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:37.891 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-09T11:09:37.961 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-09T11:09:38.070 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-09T11:09:38.070 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:38.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2023-06-09T11:09:38.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-09T11:09:38.255 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:38.255 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2023-06-09T11:09:38.326 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-09T11:09:38.438 INFO:teuthology.run_tasks:Running task ceph... 2023-06-09T11:09:38.519 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-06-09T11:09:38.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2023-06-09T11:09:38.525 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /var/log/ceph 2023-06-09T11:09:38.594 INFO:tasks.ceph:Disabling ceph logrotate... 2023-06-09T11:09:38.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-09T11:09:38.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/logrotate.d/ceph 2023-06-09T11:09:38.666 INFO:tasks.ceph:Creating extra log directories... 2023-06-09T11:09:38.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-09T11:09:38.672 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-06-09T11:09:38.748 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-06-09T11:09:38.749 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-06-09T11:09:38.750 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-06-09T11:09:38.750 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:38.751 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-09T11:09:38.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:39.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:39.082 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:39.082 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-06-09T11:09:39.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:39.337 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-06-09T11:09:39.412 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-06-09T11:09:39.412 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-06-09T11:09:39.413 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849', 'branch': 'wip-rishabh-2023Jun9-1308', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}}', 'email': None, 'first_in_suite': False, 'job_id': '7299849', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '10b2f20a4f46c803bb730efc73498f97d45082e7'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023Jun9-1308'}, 'ceph': {'cephfs': {'max_mds': 3, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'Replacing daemon mds'], 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023Jun9-1308', 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023Jun9-1308', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi007.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi117.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '10b2f20a4f46c803bb730efc73498f97d45082e7'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '034108821cde95ca1c56d458dfccc7ef9e0d964d'}}, {'ceph': None}, {'ceph-fuse': None}, {'mds_thrash': None}, {'workunit': {'clients': {'all': ['fs/snaps']}, 'timeout': '6h'}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61', 'timestamp': '2023-06-09_10:19:22', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-06-09T11:09:39.413 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-09T11:09:39.418 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-06-09T11:09:39.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2023-06-09T11:09:39.501 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m0777 -- /var/run/ceph 2023-06-09T11:09:39.574 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:39.575 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2023-06-09T11:09:39.618 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-09T11:09:39.619 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2023-06-09T11:09:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-09T11:09:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:39.707 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-06-09T11:09:39.707 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:39.707 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:39.708 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-06-09 11:09:29.675459763 +0000 2023-06-09T11:09:39.708 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-06-09 11:05:57.142969108 +0000 2023-06-09T11:09:39.708 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-06-09 11:05:57.142969108 +0000 2023-06-09T11:09:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2023-06-09 11:05:57.119969443 +0000 2023-06-09T11:09:39.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-09T11:09:39.815 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-06-09T11:09:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-06-09T11:09:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000238259 s, 2.1 MB/s 2023-06-09T11:09:39.818 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-09T11:09:39.865 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2023-06-09T11:09:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-09T11:09:39.908 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:39.908 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-06-09T11:09:39.908 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-06-09 11:09:29.675459763 +0000 2023-06-09T11:09:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-06-09 11:05:57.990956745 +0000 2023-06-09T11:09:39.909 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-06-09 11:05:57.990956745 +0000 2023-06-09T11:09:39.909 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2023-06-09 11:05:57.967957080 +0000 2023-06-09T11:09:39.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-09T11:09:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-06-09T11:09:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-06-09T11:09:40.014 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000180303 s, 2.8 MB/s 2023-06-09T11:09:40.016 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-09T11:09:40.061 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2023-06-09T11:09:40.146 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-06-09 11:09:29.676459747 +0000 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-06-09 11:05:58.836944412 +0000 2023-06-09T11:09:40.147 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-06-09 11:05:58.836944412 +0000 2023-06-09T11:09:40.148 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2023-06-09 11:05:58.814944733 +0000 2023-06-09T11:09:40.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-09T11:09:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-06-09T11:09:40.257 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-06-09T11:09:40.258 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00018589 s, 2.8 MB/s 2023-06-09T11:09:40.260 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-09T11:09:40.305 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2023-06-09T11:09:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-09T11:09:40.391 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:40.391 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-06-09T11:09:40.391 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:40.391 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:40.392 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2023-06-09 11:09:29.676459747 +0000 2023-06-09T11:09:40.392 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-06-09 11:05:59.655932472 +0000 2023-06-09T11:09:40.392 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-06-09 11:05:59.655932472 +0000 2023-06-09T11:09:40.392 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2023-06-09 11:05:59.633932793 +0000 2023-06-09T11:09:40.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-09T11:09:40.501 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2023-06-09T11:09:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2023-06-09T11:09:40.502 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000169341 s, 3.0 MB/s 2023-06-09T11:09:40.504 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-09T11:09:40.552 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:40.553 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:40.553 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2023-06-09T11:09:40.597 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-09T11:09:40.597 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2023-06-09T11:09:40.682 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-09T11:09:40.682 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:40.682 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 451 Links: 1 2023-06-09T11:09:40.682 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:40.683 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:40.683 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-06-09 11:09:12.504515992 +0000 2023-06-09T11:09:40.683 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-06-09 11:05:41.814888603 +0000 2023-06-09T11:09:40.683 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-06-09 11:05:41.814888603 +0000 2023-06-09T11:09:40.683 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2023-06-09 11:05:41.783889211 +0000 2023-06-09T11:09:40.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-09T11:09:40.789 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-06-09T11:09:40.790 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-06-09T11:09:40.790 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000407916 s, 1.3 MB/s 2023-06-09T11:09:40.792 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-09T11:09:40.838 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2023-06-09T11:09:40.922 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 457 Links: 1 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-06-09 11:09:12.504515992 +0000 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-06-09 11:05:42.652872153 +0000 2023-06-09T11:09:40.923 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-06-09 11:05:42.652872153 +0000 2023-06-09T11:09:40.924 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2023-06-09 11:05:42.624872703 +0000 2023-06-09T11:09:40.926 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-09T11:09:41.034 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-06-09T11:09:41.035 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-06-09T11:09:41.035 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000274024 s, 1.9 MB/s 2023-06-09T11:09:41.035 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-09T11:09:41.080 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2023-06-09T11:09:41.165 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-09T11:09:41.165 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:41.165 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 463 Links: 1 2023-06-09T11:09:41.165 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:41.166 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:41.166 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-06-09 11:09:12.504515992 +0000 2023-06-09T11:09:41.166 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-06-09 11:05:43.485855801 +0000 2023-06-09T11:09:41.166 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-06-09 11:05:43.485855801 +0000 2023-06-09T11:09:41.166 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2023-06-09 11:05:43.457856351 +0000 2023-06-09T11:09:41.166 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-09T11:09:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-06-09T11:09:41.274 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-06-09T11:09:41.275 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000293463 s, 1.7 MB/s 2023-06-09T11:09:41.277 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-09T11:09:41.321 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2023-06-09T11:09:41.406 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 469 Links: 1 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2023-06-09 11:09:12.504515992 +0000 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2023-06-09 11:05:44.312839567 +0000 2023-06-09T11:09:41.407 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2023-06-09 11:05:44.312839567 +0000 2023-06-09T11:09:41.408 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2023-06-09 11:05:44.285840097 +0000 2023-06-09T11:09:41.409 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-09T11:09:41.512 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2023-06-09T11:09:41.513 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2023-06-09T11:09:41.513 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000289005 s, 1.8 MB/s 2023-06-09T11:09:41.515 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-09T11:09:41.561 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:41.562 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-09T11:09:41.563 INFO:tasks.ceph:Generating config... 2023-06-09T11:09:41.565 INFO:tasks.ceph:[client] client mount timeout = 600 2023-06-09T11:09:41.566 INFO:tasks.ceph:[client] debug client = 20 2023-06-09T11:09:41.566 INFO:tasks.ceph:[client] debug ms = 1 2023-06-09T11:09:41.566 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-06-09T11:09:41.567 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-06-09T11:09:41.567 INFO:tasks.ceph:[mds] debug mds = 20 2023-06-09T11:09:41.567 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-06-09T11:09:41.568 INFO:tasks.ceph:[mds] debug ms = 1 2023-06-09T11:09:41.568 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2023-06-09T11:09:41.568 INFO:tasks.ceph:[mds] mds bal merge size = 5 2023-06-09T11:09:41.570 INFO:tasks.ceph:[mds] mds bal split bits = 3 2023-06-09T11:09:41.570 INFO:tasks.ceph:[mds] mds bal split size = 100 2023-06-09T11:09:41.571 INFO:tasks.ceph:[mds] mds debug frag = True 2023-06-09T11:09:41.571 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-06-09T11:09:41.571 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2023-06-09T11:09:41.572 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-06-09T11:09:41.572 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-06-09T11:09:41.573 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2023-06-09T11:09:41.573 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2023-06-09T11:09:41.573 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2023-06-09T11:09:41.574 INFO:tasks.ceph:[mds] ms inject delay max = 1 2023-06-09T11:09:41.574 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2023-06-09T11:09:41.574 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2023-06-09T11:09:41.575 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2023-06-09T11:09:41.575 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-06-09T11:09:41.576 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-06-09T11:09:41.576 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-06-09T11:09:41.576 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-06-09T11:09:41.577 INFO:tasks.ceph:[mgr] debug ms = 1 2023-06-09T11:09:41.577 INFO:tasks.ceph:[mon] debug mon = 20 2023-06-09T11:09:41.577 INFO:tasks.ceph:[mon] debug ms = 1 2023-06-09T11:09:41.578 INFO:tasks.ceph:[mon] debug paxos = 20 2023-06-09T11:09:41.578 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-06-09T11:09:41.578 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-06-09T11:09:41.579 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-06-09T11:09:41.579 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-06-09T11:09:41.580 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-06-09T11:09:41.580 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-06-09T11:09:41.580 INFO:tasks.ceph:[osd] debug ms = 1 2023-06-09T11:09:41.581 INFO:tasks.ceph:[osd] debug osd = 20 2023-06-09T11:09:41.581 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-06-09T11:09:41.582 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2023-06-09T11:09:41.582 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-06-09T11:09:41.582 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-06-09T11:09:41.583 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-06-09T11:09:41.583 INFO:tasks.ceph:[osd] ms inject delay max = 1 2023-06-09T11:09:41.584 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2023-06-09T11:09:41.584 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2023-06-09T11:09:41.585 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2023-06-09T11:09:41.585 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2023-06-09T11:09:41.586 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-06-09T11:09:41.586 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-06-09T11:09:41.586 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-06-09T11:09:41.587 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-06-09T11:09:41.587 INFO:tasks.ceph:Setting up mon.a... 2023-06-09T11:09:41.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-06-09T11:09:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2023-06-09T11:09:41.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-06-09T11:09:42.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T11:09:42.072 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.117'), ('mon.c', '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]')] 2023-06-09T11:09:42.073 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-06-09T11:09:42.074 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:42.075 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-06-09T11:09:42.119 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.117 --addv c '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:42.249 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid cb7aac03-4bc4-45fa-8177-804a746951e5 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = pacific 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:fsid cb7aac03-4bc4-45fa-8177-804a746951e5 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2023-06-09T11:09:42.245567+0000 2023-06-09T11:09:42.250 INFO:teuthology.orchestra.run.smithi007.stdout:created 2023-06-09T11:09:42.245567+0000 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 16 (pacific) 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.b 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] mon.c 2023-06-09T11:09:42.251 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-06-09T11:09:42.253 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-06-09T11:09:42.296 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cb7aac03-4bc4-45fa-8177-804a746951e5... 2023-06-09T11:09:42.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-09T11:09:42.341 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-06-09T11:09:42.459 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2023-06-09T11:09:42.460 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2023-06-09T11:09:42.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2023-06-09T11:09:42.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2023-06-09T11:09:42.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2023-06-09T11:09:42.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2023-06-09T11:09:42.465 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = cb7aac03-4bc4-45fa-8177-804a746951e5 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.466 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2023-06-09T11:09:42.467 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2023-06-09T11:09:42.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression mode = aggressive 2023-06-09T11:09:42.469 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2023-06-09T11:09:42.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay max = 1 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay probability = 0.005 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay type = client mds 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 2500 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2023-06-09T11:09:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2023-06-09T11:09:42.472 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-06-09T11:09:42.473 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2023-06-09T11:09:42.474 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-09T11:09:42.475 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: mon op complaint time = 120 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:09:42.476 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 600 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2023-06-09T11:09:42.477 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-06-09T11:09:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e] 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f] 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds = 20 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: debug mds balancer = 20 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2023-06-09T11:09:42.479 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal fragment size max = 10000 2023-06-09T11:09:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal merge size = 5 2023-06-09T11:09:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal split bits = 3 2023-06-09T11:09:42.480 INFO:teuthology.orchestra.run.smithi007.stdout: mds bal split size = 100 2023-06-09T11:09:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug frag = True 2023-06-09T11:09:42.481 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mds oft prefetch dirfrags = True 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mds op complaint time = 180 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mds_dir_prefetch = True 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mds_sleep_rank_change = 5000000.0 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2023-06-09T11:09:42.482 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay max = 1 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay probability = 0.005 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject delay type = client mds osd 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 2500 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: osd op complaint time = 180 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: rados mon op timeout = 900 2023-06-09T11:09:42.483 INFO:teuthology.orchestra.run.smithi007.stdout: rados osd op timeout = 900 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout: chdir = "" 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout: auth supported = cephx 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.485 INFO:teuthology.orchestra.run.smithi117.stdout: filestore xattr use omap = true 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: mon clock drift allowed = 1.000 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: osd crush chooseleaf type = 0 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: auth debug = true 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on old message = true 2023-06-09T11:09:42.486 INFO:teuthology.orchestra.run.smithi117.stdout: ms die on bug = true 2023-06-09T11:09:42.487 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.487 INFO:teuthology.orchestra.run.smithi117.stdout: mon max pg per osd = 10000 # >= luminous 2023-06-09T11:09:42.487 INFO:teuthology.orchestra.run.smithi117.stdout: mon pg warn max object skew = 0 2023-06-09T11:09:42.488 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.488 INFO:teuthology.orchestra.run.smithi117.stdout: # disable pg_autoscaler by default for new pools 2023-06-09T11:09:42.488 INFO:teuthology.orchestra.run.smithi117.stdout: osd_pool_default_pg_autoscale_mode = off 2023-06-09T11:09:42.489 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.489 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default size = 2 2023-06-09T11:09:42.489 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.489 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow primary affinity = true 2023-06-09T11:09:42.490 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd allow pg remap = true 2023-06-09T11:09:42.490 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on legacy crush tunables = false 2023-06-09T11:09:42.490 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on crush straw calc version zero = false 2023-06-09T11:09:42.491 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on no sortbitwise = false 2023-06-09T11:09:42.491 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on osd down out interval zero = false 2023-06-09T11:09:42.491 INFO:teuthology.orchestra.run.smithi117.stdout: mon warn on too few osds = false 2023-06-09T11:09:42.491 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-06-09T11:09:42.492 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_pool_no_redundancy = false 2023-06-09T11:09:42.492 INFO:teuthology.orchestra.run.smithi117.stdout: mon_allow_pool_size_one = true 2023-06-09T11:09:42.492 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.492 INFO:teuthology.orchestra.run.smithi117.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-06-09T11:09:42.493 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.493 INFO:teuthology.orchestra.run.smithi117.stdout: osd default data pool replay window = 5 2023-06-09T11:09:42.493 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.493 INFO:teuthology.orchestra.run.smithi117.stdout: mon allow pool delete = true 2023-06-09T11:09:42.494 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.494 INFO:teuthology.orchestra.run.smithi117.stdout: mon cluster log file level = debug 2023-06-09T11:09:42.494 INFO:teuthology.orchestra.run.smithi117.stdout: debug asserts on shutdown = true 2023-06-09T11:09:42.494 INFO:teuthology.orchestra.run.smithi117.stdout: mon health detail to clog = false 2023-06-09T11:09:42.495 INFO:teuthology.orchestra.run.smithi117.stdout: mon host = "172.21.15.7,172.21.15.117,[v2:172.21.15.117:3301,v1:172.21.15.117:6790]" 2023-06-09T11:09:42.495 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = cb7aac03-4bc4-45fa-8177-804a746951e5 2023-06-09T11:09:42.495 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.495 INFO:teuthology.orchestra.run.smithi117.stdout:[osd] 2023-06-09T11:09:42.496 INFO:teuthology.orchestra.run.smithi117.stdout: osd journal size = 100 2023-06-09T11:09:42.496 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.496 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub load threshold = 5.0 2023-06-09T11:09:42.496 INFO:teuthology.orchestra.run.smithi117.stdout: osd scrub max interval = 600 2023-06-09T11:09:42.497 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.497 INFO:teuthology.orchestra.run.smithi117.stdout: osd recover clone overlap = true 2023-06-09T11:09:42.497 INFO:teuthology.orchestra.run.smithi117.stdout: osd recovery max chunk = 1048576 2023-06-09T11:09:42.497 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.498 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug shutdown = true 2023-06-09T11:09:42.498 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug op order = true 2023-06-09T11:09:42.498 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug verify stray on activate = true 2023-06-09T11:09:42.498 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.498 INFO:teuthology.orchestra.run.smithi117.stdout: osd open classes on start = true 2023-06-09T11:09:42.499 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug pg log writeout = true 2023-06-09T11:09:42.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.499 INFO:teuthology.orchestra.run.smithi117.stdout: osd deep scrub update digest min age = 30 2023-06-09T11:09:42.499 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.499 INFO:teuthology.orchestra.run.smithi117.stdout: osd map max advance = 10 2023-06-09T11:09:42.500 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.500 INFO:teuthology.orchestra.run.smithi117.stdout: journal zero on create = true 2023-06-09T11:09:42.500 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.500 INFO:teuthology.orchestra.run.smithi117.stdout: filestore ondisk finisher threads = 3 2023-06-09T11:09:42.501 INFO:teuthology.orchestra.run.smithi117.stdout: filestore apply finisher threads = 3 2023-06-09T11:09:42.501 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.501 INFO:teuthology.orchestra.run.smithi117.stdout: bdev debug aio = true 2023-06-09T11:09:42.501 INFO:teuthology.orchestra.run.smithi117.stdout: osd debug misdirected ops = true 2023-06-09T11:09:42.502 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore block size = 96636764160 2023-06-09T11:09:42.502 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore compression mode = aggressive 2023-06-09T11:09:42.502 INFO:teuthology.orchestra.run.smithi117.stdout: bluestore fsck on mount = True 2023-06-09T11:09:42.503 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluefs = 20 2023-06-09T11:09:42.503 INFO:teuthology.orchestra.run.smithi117.stdout: debug bluestore = 20 2023-06-09T11:09:42.503 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-06-09T11:09:42.503 INFO:teuthology.orchestra.run.smithi117.stdout: debug osd = 20 2023-06-09T11:09:42.504 INFO:teuthology.orchestra.run.smithi117.stdout: debug rocksdb = 10 2023-06-09T11:09:42.504 INFO:teuthology.orchestra.run.smithi117.stdout: mon client directed command retry = 5 2023-06-09T11:09:42.504 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd backfillfull_ratio = 0.85 2023-06-09T11:09:42.504 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd full ratio = 0.9 2023-06-09T11:09:42.505 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd nearfull ratio = 0.8 2023-06-09T11:09:42.505 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay max = 1 2023-06-09T11:09:42.505 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay probability = 0.005 2023-06-09T11:09:42.505 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay type = client mds 2023-06-09T11:09:42.506 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject socket failures = 2500 2023-06-09T11:09:42.506 INFO:teuthology.orchestra.run.smithi117.stdout: osd blocked scrub grace period = 3600 2023-06-09T11:09:42.506 INFO:teuthology.orchestra.run.smithi117.stdout: osd failsafe full ratio = 0.95 2023-06-09T11:09:42.506 INFO:teuthology.orchestra.run.smithi117.stdout: osd max markdown count = 1000 2023-06-09T11:09:42.507 INFO:teuthology.orchestra.run.smithi117.stdout: osd objectstore = bluestore 2023-06-09T11:09:42.507 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2023-06-09T11:09:42.508 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.508 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr] 2023-06-09T11:09:42.508 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-06-09T11:09:42.508 INFO:teuthology.orchestra.run.smithi117.stdout: debug mgr = 20 2023-06-09T11:09:42.509 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2023-06-09T11:09:42.509 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2023-06-09T11:09:42.509 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2023-06-09T11:09:42.510 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2023-06-09T11:09:42.510 INFO:teuthology.orchestra.run.smithi117.stdout: mgr/telemetry/nag = false 2023-06-09T11:09:42.510 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.510 INFO:teuthology.orchestra.run.smithi117.stdout:[mon] 2023-06-09T11:09:42.511 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-06-09T11:09:42.511 INFO:teuthology.orchestra.run.smithi117.stdout: debug mon = 20 2023-06-09T11:09:42.511 INFO:teuthology.orchestra.run.smithi117.stdout: debug paxos = 20 2023-06-09T11:09:42.511 INFO:teuthology.orchestra.run.smithi117.stdout: debug auth = 20 2023-06-09T11:09:42.511 INFO:teuthology.orchestra.run.smithi117.stdout: mon data avail warn = 5 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: mon mgr mkfs grace = 240 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min pgs per osd = 4 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd reporter subtree level = osd 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: mon osd prime pg temp = true 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: mon reweight min bytes per osd = 10 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-06-09T11:09:42.512 INFO:teuthology.orchestra.run.smithi117.stdout: auth mon ticket ttl = 660 # 11m 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: auth service ticket ttl = 240 # 4m 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: # don't complain about insecure global_id in the test suite 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.513 INFO:teuthology.orchestra.run.smithi117.stdout: # 1m isn't quite enough 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: mon_down_mkfs_grace = 2m 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: mon_warn_on_filestore_osds = false 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: mon op complaint time = 120 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout:[client] 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: rgw cache enabled = true 2023-06-09T11:09:42.514 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable ops log = true 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: rgw enable usage log = true 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: client mount timeout = 600 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: debug client = 20 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2023-06-09T11:09:42.515 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.a] 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.b] 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mon.c] 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2023-06-09T11:09:42.516 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.e] 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.f] 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout:[mds] 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds = 20 2023-06-09T11:09:42.517 INFO:teuthology.orchestra.run.smithi117.stdout: debug mds balancer = 20 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: debug ms = 1 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal fragment size max = 10000 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal merge size = 5 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal split bits = 3 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds bal split size = 100 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug frag = True 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds debug scatterstat = True 2023-06-09T11:09:42.518 INFO:teuthology.orchestra.run.smithi117.stdout: mds oft prefetch dirfrags = True 2023-06-09T11:09:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: mds op complaint time = 180 2023-06-09T11:09:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: mds verify scatter = True 2023-06-09T11:09:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: mds_dir_prefetch = True 2023-06-09T11:09:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: mds_sleep_rank_change = 5000000.0 2023-06-09T11:09:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: mon client directed command retry = 5 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay max = 1 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay probability = 0.005 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject delay type = client mds osd 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: ms inject socket failures = 2500 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: osd op complaint time = 180 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: rados mon op timeout = 900 2023-06-09T11:09:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: rados osd op timeout = 900 2023-06-09T11:09:42.522 INFO:tasks.ceph:Creating admin key on mon.a... 2023-06-09T11:09:42.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-06-09T11:09:42.638 INFO:tasks.ceph:Copying monmap to all nodes... 2023-06-09T11:09:42.638 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:42.638 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-06-09T11:09:42.683 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:42.684 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-06-09T11:09:42.767 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:09:42.767 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:42.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-09T11:09:42.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T11:09:42.913 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:09:42.913 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:42.995 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T11:09:42.996 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:42.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.keyring 2023-06-09T11:09:42.997 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-06-09T11:09:43.089 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:09:43.090 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:43.173 INFO:tasks.ceph:Setting up mon nodes... 2023-06-09T11:09:43.173 INFO:tasks.ceph:Setting up mgr nodes... 2023-06-09T11:09:43.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-06-09T11:09:43.303 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-06-09T11:09:43.307 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-06-09T11:09:43.442 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-06-09T11:09:43.450 INFO:tasks.ceph:Setting up mds nodes... 2023-06-09T11:09:43.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-06-09T11:09:43.581 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-06-09T11:09:43.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-06-09T11:09:43.659 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-06-09T11:09:43.791 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-06-09T11:09:43.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-06-09T11:09:43.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2023-06-09T11:09:43.998 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2023-06-09T11:09:44.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2023-06-09T11:09:44.075 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-06-09T11:09:44.199 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-06-09T11:09:44.206 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-06-09T11:09:44.273 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-06-09T11:09:44.404 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-06-09T11:09:44.410 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-06-09T11:09:44.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2023-06-09T11:09:44.604 INFO:teuthology.orchestra.run.smithi117.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2023-06-09T11:09:44.609 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2023-06-09T11:09:44.678 INFO:tasks.ceph_client:Setting up client nodes... 2023-06-09T11:09:44.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-09T11:09:44.779 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-06-09T11:09:44.812 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-06-09T11:09:44.812 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi117.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-06-09T11:09:44.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-06-09T11:09:44.888 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:44.889 INFO:tasks.ceph:role: osd.0 2023-06-09T11:09:44.889 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:09:44.890 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-09T11:09:44.962 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:44.963 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:09:44.985 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-06-09T11:09:44.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-06-09T11:09:44.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-06-09T11:09:45.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-06-09T11:09:45.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-06-09T11:09:45.254 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:45.254 INFO:tasks.ceph:role: osd.1 2023-06-09T11:09:45.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:09:45.255 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-09T11:09:45.324 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:45.324 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:45.325 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:45.326 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:09:45.348 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-06-09T11:09:45.352 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-06-09T11:09:45.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-06-09T11:09:45.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-06-09T11:09:45.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-06-09T11:09:45.610 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:45.610 INFO:tasks.ceph:role: osd.2 2023-06-09T11:09:45.611 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:09:45.611 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-09T11:09:45.680 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:45.680 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:45.680 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:45.680 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:45.680 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:45.681 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:45.681 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:45.681 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:45.681 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:45.681 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:09:45.703 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-06-09T11:09:45.708 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-06-09T11:09:45.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-06-09T11:09:45.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-06-09T11:09:45.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-06-09T11:09:45.963 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:45.964 INFO:tasks.ceph:role: osd.3 2023-06-09T11:09:45.964 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:09:45.964 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-09T11:09:46.035 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:46.036 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:46.037 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:46.037 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:46.037 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:09:46.059 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2023-06-09T11:09:46.064 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2023-06-09T11:09:46.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-06-09T11:09:46.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-06-09T11:09:46.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:46.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.374+0000 7f4c75602640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-06-09T11:09:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.374+0000 7f4c75602640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-06-09T11:09:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.374+0000 7f4c75602640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T11:09:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.374+0000 7f4c75602640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T11:09:46.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.374+0000 7f4c75602640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-06-09T11:09:46.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:46.377+0000 7f4c75602640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-06-09T11:09:49.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-06-09T11:09:49.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.480+0000 7efcc6737640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-06-09T11:09:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.480+0000 7efcc6737640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-06-09T11:09:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.480+0000 7efcc6737640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T11:09:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.480+0000 7efcc6737640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T11:09:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.481+0000 7efcc6737640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-06-09T11:09:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:49.482+0000 7efcc6737640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-06-09T11:09:52.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-06-09T11:09:52.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:52.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.588+0000 7f015fbbf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-06-09T11:09:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.588+0000 7f015fbbf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-06-09T11:09:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.588+0000 7f015fbbf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T11:09:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.588+0000 7f015fbbf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T11:09:52.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.588+0000 7f015fbbf640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-06-09T11:09:52.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:52.589+0000 7f015fbbf640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-06-09T11:09:55.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-06-09T11:09:55.618 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:09:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.722+0000 7efcedb77640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-06-09T11:09:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.722+0000 7efcedb77640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-06-09T11:09:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.722+0000 7efcedb77640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T11:09:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.722+0000 7efcedb77640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T11:09:55.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.722+0000 7efcedb77640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-06-09T11:09:55.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:09:55.724+0000 7efcedb77640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-06-09T11:09:58.656 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-06-09T11:09:58.725 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-06-09T11:09:58.797 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:58.797 INFO:tasks.ceph:role: osd.4 2023-06-09T11:09:58.797 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T11:09:58.798 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-06-09T11:09:58.870 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:58.870 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:58.871 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:58.871 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:58.871 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:58.871 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:58.872 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:58.872 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:58.872 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:58.872 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:09:59.295 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2023-06-09T11:09:59.300 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-06-09T11:09:59.300 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-06-09T11:09:59.489 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-06-09T11:09:59.570 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-06-09T11:09:59.639 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T11:09:59.639 INFO:tasks.ceph:role: osd.5 2023-06-09T11:09:59.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T11:09:59.640 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-06-09T11:09:59.714 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:09:59.715 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:09:59.716 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:09:59.716 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:09:59.716 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:10:00.119 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2023-06-09T11:10:00.123 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-06-09T11:10:00.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-06-09T11:10:00.261 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-06-09T11:10:00.336 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-06-09T11:10:00.405 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T11:10:00.405 INFO:tasks.ceph:role: osd.6 2023-06-09T11:10:00.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T11:10:00.406 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-06-09T11:10:00.482 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:10:00.482 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:10:00.482 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:10:00.483 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:10:00.483 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:10:00.483 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:10:00.484 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:10:00.484 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:10:00.484 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:10:00.484 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:10:00.936 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2023-06-09T11:10:00.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-06-09T11:10:00.940 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-06-09T11:10:01.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-06-09T11:10:01.201 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-06-09T11:10:01.269 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-06-09T11:10:01.270 INFO:tasks.ceph:role: osd.7 2023-06-09T11:10:01.270 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T11:10:01.270 DEBUG:teuthology.orchestra.run.smithi117:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-06-09T11:10:01.346 INFO:teuthology.orchestra.run.smithi117.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 attr=2, projid32bit=1 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout: = sunit=0 swidth=0 blks 2023-06-09T11:10:01.347 INFO:teuthology.orchestra.run.smithi117.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-06-09T11:10:01.348 INFO:teuthology.orchestra.run.smithi117.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-06-09T11:10:01.348 INFO:teuthology.orchestra.run.smithi117.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-06-09T11:10:01.348 INFO:teuthology.orchestra.run.smithi117.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-06-09T11:10:01.777 INFO:teuthology.orchestra.run.smithi117.stdout:Discarding blocks...Done. 2023-06-09T11:10:01.781 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi117.front.sepia.ceph.com -o noatime 2023-06-09T11:10:01.782 DEBUG:teuthology.orchestra.run.smithi117:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-06-09T11:10:01.927 DEBUG:teuthology.orchestra.run.smithi117:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-06-09T11:10:02.002 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:02.135 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.129+0000 7f761d275640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-06-09T11:10:02.135 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.129+0000 7f761d275640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-06-09T11:10:02.135 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.129+0000 7f761d275640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T11:10:02.136 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.129+0000 7f761d275640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T11:10:02.136 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.129+0000 7f761d275640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-06-09T11:10:02.136 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:02.131+0000 7f761d275640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-06-09T11:10:05.017 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-06-09T11:10:05.085 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:05.188 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.183+0000 7fd6f3a4f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-06-09T11:10:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.183+0000 7fd6f3a4f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-06-09T11:10:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.183+0000 7fd6f3a4f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T11:10:05.190 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.183+0000 7fd6f3a4f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T11:10:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.183+0000 7fd6f3a4f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-06-09T11:10:05.191 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:05.185+0000 7fd6f3a4f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-06-09T11:10:08.074 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-06-09T11:10:08.144 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:08.247 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.241+0000 7fb74526b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-06-09T11:10:08.247 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.241+0000 7fb74526b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-06-09T11:10:08.247 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.241+0000 7fb74526b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T11:10:08.247 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.241+0000 7fb74526b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T11:10:08.248 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.242+0000 7fb74526b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-06-09T11:10:08.248 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:08.243+0000 7fb74526b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-06-09T11:10:09.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:09.420 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:11.134 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-06-09T11:10:11.205 DEBUG:teuthology.orchestra.run.smithi117:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:11.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.303+0000 7fd5969c9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-06-09T11:10:11.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.303+0000 7fd5969c9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-06-09T11:10:11.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.303+0000 7fd5969c9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T11:10:11.309 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.303+0000 7fd5969c9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T11:10:11.310 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.303+0000 7fd5969c9640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-06-09T11:10:11.311 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:11.306+0000 7fd5969c9640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-06-09T11:10:14.187 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-06-09T11:10:14.259 INFO:tasks.ceph:Reading keys from all nodes... 2023-06-09T11:10:14.259 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-06-09T11:10:14.331 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-06-09T11:10:14.442 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-06-09T11:10:14.555 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2023-06-09T11:10:14.664 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-06-09T11:10:14.773 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-06-09T11:10:14.883 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-06-09T11:10:14.991 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:14.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-06-09T11:10:15.105 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.106 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-06-09T11:10:15.180 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.181 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-06-09T11:10:15.290 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.291 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-06-09T11:10:15.399 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.400 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2023-06-09T11:10:15.505 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.506 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-06-09T11:10:15.612 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.612 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-06-09T11:10:15.718 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.719 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-06-09T11:10:15.825 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:15.825 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-06-09T11:10:15.931 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:15.931 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-06-09T11:10:15.974 INFO:tasks.ceph:Adding keys to all mons... 2023-06-09T11:10:15.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-09T11:10:16.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo tee -a /etc/ceph/ceph.keyring 2023-06-09T11:10:16.081 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2023-06-09T11:10:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB3CINkdwOwERAAwkpI11hUZia1aqvhKvqsJQ== 2023-06-09T11:10:16.082 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2023-06-09T11:10:16.082 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB3CINkzHlrIhAAvrIGe8W2kohuKBEYQz0Niw== 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB3CINkwnffLhAAbYcbgep9TrjlPqXV9PWWsg== 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.e] 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB3CINkMohFOxAAeK/vVP6118BVYYPPvK02KQ== 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2023-06-09T11:10:16.083 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB6CINkWWFYFhAAM7QVKGNCM9XJsQw+FIamUg== 2023-06-09T11:10:16.084 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2023-06-09T11:10:16.084 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB9CINkGs2sHBAAEkaZdJgiK7pXvmaUe/ltQg== 2023-06-09T11:10:16.084 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2023-06-09T11:10:16.084 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCACINkb3wWIxAAXCNwOnVrzBTf4DZgOreeZw== 2023-06-09T11:10:16.084 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2023-06-09T11:10:16.085 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCDCINki0QTKxAA2sKcUkwP5N9MPpIqpOAh3Q== 2023-06-09T11:10:16.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2023-06-09T11:10:16.085 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB3CINkZ/UjGhAAbQbOSFrgL41c7O79l+D4UQ== 2023-06-09T11:10:16.085 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2023-06-09T11:10:16.086 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4CINkjJmtCxAAEWwYruSSssJ37ta4lmWwCA== 2023-06-09T11:10:16.086 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2023-06-09T11:10:16.086 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4CINkW97aFxAA7JI8o4XOav1zoRECy/KTGQ== 2023-06-09T11:10:16.086 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.f] 2023-06-09T11:10:16.086 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4CINkAp/EIxAAOmJOQ7t0XNfZ0bZZ+hd1qA== 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCKCINksmvHBxAAn+JMWiPwBN4hFMrH6qMzmg== 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCNCINk2Sf+ChAAgb8S8FAoFbzEIXvu7ZnPUg== 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCQCINkMpp7DhAACbzbAp1uiI5eQdVH1kBWZw== 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2023-06-09T11:10:16.087 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQCTCINkzussEhAAhPmuMpbzmH9DrIsyQIa2kg== 2023-06-09T11:10:16.088 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2023-06-09T11:10:16.088 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4CINkhhk0LhAAma8fyivtOR5WbvqZX+5jcA== 2023-06-09T11:10:16.089 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.y] 2023-06-09T11:10:16.089 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB3CINkdwOwERAAwkpI11hUZia1aqvhKvqsJQ== 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.a] 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB3CINkzHlrIhAAvrIGe8W2kohuKBEYQz0Niw== 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.c] 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB3CINkwnffLhAAbYcbgep9TrjlPqXV9PWWsg== 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.e] 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB3CINkMohFOxAAeK/vVP6118BVYYPPvK02KQ== 2023-06-09T11:10:16.090 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.0] 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB6CINkWWFYFhAAM7QVKGNCM9XJsQw+FIamUg== 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.1] 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB9CINkGs2sHBAAEkaZdJgiK7pXvmaUe/ltQg== 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.2] 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCACINkb3wWIxAAXCNwOnVrzBTf4DZgOreeZw== 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.3] 2023-06-09T11:10:16.091 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCDCINki0QTKxAA2sKcUkwP5N9MPpIqpOAh3Q== 2023-06-09T11:10:16.092 INFO:teuthology.orchestra.run.smithi117.stdout:[mgr.x] 2023-06-09T11:10:16.092 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB3CINkZ/UjGhAAbQbOSFrgL41c7O79l+D4UQ== 2023-06-09T11:10:16.092 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.b] 2023-06-09T11:10:16.092 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB4CINkjJmtCxAAEWwYruSSssJ37ta4lmWwCA== 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.d] 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB4CINkW97aFxAA7JI8o4XOav1zoRECy/KTGQ== 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout:[mds.f] 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB4CINkAp/EIxAAOmJOQ7t0XNfZ0bZZ+hd1qA== 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.4] 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCKCINksmvHBxAAn+JMWiPwBN4hFMrH6qMzmg== 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.5] 2023-06-09T11:10:16.093 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCNCINk2Sf+ChAAgb8S8FAoFbzEIXvu7ZnPUg== 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.6] 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCQCINkMpp7DhAACbzbAp1uiI5eQdVH1kBWZw== 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout:[osd.7] 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCTCINkzussEhAAhPmuMpbzmH9DrIsyQIa2kg== 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2023-06-09T11:10:16.094 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQB4CINkhhk0LhAAma8fyivtOR5WbvqZX+5jcA== 2023-06-09T11:10:16.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T11:10:16.136 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T11:10:16.242 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.246 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.360 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:16.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.810 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:16.925 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T11:10:17.038 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-06-09T11:10:17.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.148 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.265 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-06-09T11:10:17.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.495 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.605 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.717 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.828 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-06-09T11:10:17.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T11:10:17.941 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-06-09T11:10:18.052 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-06-09T11:10:18.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-06-09T11:10:18.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T11:10:18.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-06-09T11:10:18.320 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-06-09T11:10:18.388 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T11:10:18.510 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-06-09T11:10:18.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-06-09T11:10:18.645 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-06-09T11:10:18.765 DEBUG:teuthology.orchestra.run.smithi117:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-06-09T11:10:18.832 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:18.837 DEBUG:teuthology.orchestra.run.smithi117:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-06-09T11:10:18.880 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-06-09T11:10:18.880 INFO:tasks.ceph.mon.a:Restarting daemon 2023-06-09T11:10:18.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-06-09T11:10:18.920 INFO:tasks.ceph.mon.a:Started 2023-06-09T11:10:18.921 INFO:tasks.ceph.mon.b:Restarting daemon 2023-06-09T11:10:18.921 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-06-09T11:10:18.926 INFO:tasks.ceph.mon.b:Started 2023-06-09T11:10:18.926 INFO:tasks.ceph.mon.c:Restarting daemon 2023-06-09T11:10:18.926 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-06-09T11:10:18.930 INFO:tasks.ceph.mon.c:Started 2023-06-09T11:10:18.931 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-06-09T11:10:18.931 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-06-09T11:10:18.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-06-09T11:10:18.938 INFO:tasks.ceph.mgr.y:Started 2023-06-09T11:10:18.938 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-06-09T11:10:18.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-06-09T11:10:18.943 INFO:tasks.ceph.mgr.x:Started 2023-06-09T11:10:18.944 INFO:tasks.ceph:Setting crush tunables to default 2023-06-09T11:10:18.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2023-06-09T11:10:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.164+0000 7f8d013ff700 1 Processor -- start 2023-06-09T11:10:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.164+0000 7f8d013ff700 1 -- start start 2023-06-09T11:10:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.165+0000 7f8d013ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.165+0000 7f8d013ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.165+0000 7f8d013ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0cbc90 con 0x7f8cfc14d070 2023-06-09T11:10:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.165+0000 7f8d013ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c1710 con 0x7f8cfc14d440 2023-06-09T11:10:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.165+0000 7f8d013ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c0b90 con 0x7f8cfc070110 2023-06-09T11:10:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:59504/0 (socket says 172.21.15.7:59504) 2023-06-09T11:10:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 learned_addr learned my addr 172.21.15.7:0/3614934417 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.166+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-09T11:10:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:19.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.367+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:19.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.768+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.768+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:19.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:19.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:19.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:19.769+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-06-09T11:10:20.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-09T11:10:20.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:20.570+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-09T11:10:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8cfc0c1e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:10:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8cfc0c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 msgr2=0x7f8cfc0c1950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:10:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8cfc14d440 0x7f8cfc0c1950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v1:172.21.15.117:6789/0 conn(0x7f8cfc14d070 legacy=0x7f8cfc0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0c0e70 con 0x7f8ce8003770 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c1710 con 0x7f8ce8007460 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:22.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.166+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.367+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.367+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:22.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.367+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.768+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:22.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.768+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:22.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.768+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:22.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.768+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:22.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:22.768+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:23.569+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:23.569+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:23.569+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:23.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:23.569+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:23.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:23.569+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-09T11:10:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 msgr2=0x7f8ce80098f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:10:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce8007460 0x7f8ce80098f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v1:172.21.15.7:6789/0 conn(0x7f8ce8003770 legacy=0x7f8ce8005ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 reap_dead start 2023-06-09T11:10:25.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:25.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce800aff0 con 0x7f8ce80062d0 2023-06-09T11:10:25.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ce800adb0 con 0x7f8cfc14d070 2023-06-09T11:10:25.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c1710 con 0x7f8cfc070110 2023-06-09T11:10:25.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:25.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:25.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:25.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:25.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.368+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.368+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:25.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.769+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.769+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.769+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:25.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.769+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:25.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:25.769+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:26.571+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:26.571+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:26.571+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:26.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:26.571+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:26.571+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-09T11:10:28.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 msgr2=0x7f8ce800a0d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:10:28.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cfc070110 0x7f8ce800a0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:28.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v1:172.21.15.117:6789/0 conn(0x7f8cfc14d070 legacy=0x7f8ce800a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 >> v1:172.21.15.7:6789/0 conn(0x7f8ce80062d0 legacy=0x7f8ce8006740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c1710 con 0x7f8ce80050e0 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cf8ff9700 1 -- 172.21.15.7:0/3614934417 --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c0b90 con 0x7f8ce8009890 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:28.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.167+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:10:28.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:28.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.368+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-09T11:10:28.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-06-09T11:10:28.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.769+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.770+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.770+0000 7f8cfaffd700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:10:28.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.770+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:10:28.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:28.770+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-09T11:10:29.329 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:29.324+0000 7f7f2a848200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-09T11:10:29.414 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:29.409+0000 7f7f2a848200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-09T11:10:29.512 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:29.507+0000 7f7f2a848200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-09T11:10:29.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.570+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.7:3300/0 conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 msgr2=0x7f8ce8004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> v2:172.21.15.117:3300/0 conn(0x7f8ce8009890 0x7f8ce8004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cfa7fc700 1 -- 172.21.15.7:0/3614934417 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce8006ef0 con 0x7f8ce80050e0 2023-06-09T11:10:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce80050e0 0x7f8ce8005580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007230 tx=0x7f8cf000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8de6601fe7ff185 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cf9ffb700 1 -- 172.21.15.7:0/3614934417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0011cc0 con 0x7f8ce80050e0 2023-06-09T11:10:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cf9ffb700 1 -- 172.21.15.7:0/3614934417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf0017040 con 0x7f8ce80050e0 2023-06-09T11:10:29.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.571+0000 7f8cf9ffb700 1 -- 172.21.15.7:0/3614934417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0004270 con 0x7f8ce80050e0 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 -- 172.21.15.7:0/3614934417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce80050e0 msgr2=0x7f8ce8005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce80050e0 0x7f8ce8005580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8007230 tx=0x7f8cf000eda0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/3614934417 reap_dead start 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 -- 172.21.15.7:0/3614934417 shutdown_connections 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 --2- 172.21.15.7:0/3614934417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ce80050e0 0x7f8ce8005580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 -- 172.21.15.7:0/3614934417 >> 172.21.15.7:0/3614934417 conn(0x7f8cfc06a4d0 msgr2=0x7f8cfc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 -- 172.21.15.7:0/3614934417 shutdown_connections 2023-06-09T11:10:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.572+0000 7f8d013ff700 1 -- 172.21.15.7:0/3614934417 wait complete. 2023-06-09T11:10:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.574+0000 7f8d013ff700 1 Processor -- start 2023-06-09T11:10:29.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 -- start start 2023-06-09T11:10:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce80050e0 0x7f8cfc0c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cfc0b0ad0 0x7f8cfc1e1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:29.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc159850 con 0x7f8cfc1e1590 2023-06-09T11:10:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc159360 con 0x7f8cfc0b0ad0 2023-06-09T11:10:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.575+0000 7f8d013ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc0c4790 con 0x7f8ce80050e0 2023-06-09T11:10:29.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46252/0 (socket says 172.21.15.7:46252) 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/1153455925 learned_addr learned my addr 172.21.15.7:0/1153455925 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfaffd700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce80050e0 0x7f8cfc0c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfa7fc700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cfc0b0ad0 0x7f8cfc1e1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce80050e0 msgr2=0x7f8cfc0c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:29.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce80050e0 0x7f8cfc0c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cfc0b0ad0 msgr2=0x7f8cfc1e1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cfc0b0ad0 0x7f8cfc1e1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 -- 172.21.15.7:0/1153455925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce800adb0 con 0x7f8cfc1e1590 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.576+0000 7f8cfb7fe700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009610 tx=0x7f8cf40048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf40118a0 con 0x7f8cfc1e1590 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf400f040 con 0x7f8cfc1e1590 2023-06-09T11:10:29.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4011c40 con 0x7f8cfc1e1590 2023-06-09T11:10:29.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0c1430 con 0x7f8cfc1e1590 2023-06-09T11:10:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc0c1150 con 0x7f8cfc1e1590 2023-06-09T11:10:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8cf4017460 con 0x7f8cfc1e1590 2023-06-09T11:10:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.577+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8cf4011de0 con 0x7f8cfc1e1590 2023-06-09T11:10:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.578+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc0cb170 con 0x7f8cfc1e1590 2023-06-09T11:10:29.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.581+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f8cf40178f0 con 0x7f8cfc1e1590 2023-06-09T11:10:29.615 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:29.609+0000 7f7f2a848200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-09T11:10:29.649 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:29.643+0000 7f7323c6a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-09T11:10:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.662+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8cfc074cf0 con 0x7f8cfc1e1590 2023-06-09T11:10:29.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.665+0000 7f8ce37fe700 1 -- 172.21.15.7:0/1153455925 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8cf4017a90 con 0x7f8cfc1e1590 2023-06-09T11:10:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 msgr2=0x7f8cfc1f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4009610 tx=0x7f8cf40048c0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 shutdown_connections 2023-06-09T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cfc1e1590 0x7f8cfc1f22f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ce80050e0 0x7f8cfc0c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 --2- 172.21.15.7:0/1153455925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cfc0b0ad0 0x7f8cfc1e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 >> 172.21.15.7:0/1153455925 conn(0x7f8cfc06a4d0 msgr2=0x7f8cfc0c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 shutdown_connections 2023-06-09T11:10:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:29.667+0000 7f8d013ff700 1 -- 172.21.15.7:0/1153455925 wait complete. 2023-06-09T11:10:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2023-06-09T11:10:29.696 INFO:tasks.ceph:check_enable_crimson: False 2023-06-09T11:10:29.696 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-06-09T11:10:29.697 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:29.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-06-09T11:10:29.731 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:29.726+0000 7f7323c6a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-09T11:10:29.766 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:29.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-06-09T11:10:29.805 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:29.799+0000 7f7f2a848200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-09T11:10:29.824 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:29.819+0000 7f7323c6a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-09T11:10:29.873 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:29.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-06-09T11:10:29.920 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:29.915+0000 7f7323c6a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-09T11:10:29.984 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:10:29.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-06-09T11:10:30.092 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:30.092 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-06-09T11:10:30.099 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:30.091+0000 7f7323c6a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-09T11:10:30.166 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:30.167 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-06-09T11:10:30.222 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:30.217+0000 7f7f2a848200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-09T11:10:30.273 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:30.274 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-06-09T11:10:30.321 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:30.314+0000 7f7f2a848200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-09T11:10:30.386 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2023-06-09T11:10:30.387 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-06-09T11:10:30.476 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:30.470+0000 7f7f2a848200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-09T11:10:30.493 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 8beb7b04-04e2-4c76-843c-6347fd4b7423 0 2023-06-09T11:10:30.534 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:30.528+0000 7f7323c6a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-09T11:10:30.580 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:30.574+0000 7f7f2a848200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-09T11:10:30.632 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:30.626+0000 7f7323c6a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-09T11:10:30.724 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.718+0000 7f22dc9ff700 1 Processor -- start 2023-06-09T11:10:30.725 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.718+0000 7f22dc9ff700 1 -- start start 2023-06-09T11:10:30.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22dc9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 0x7f22d80c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:30.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22dc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:30.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22dc9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d8157450 con 0x7f22d8071040 2023-06-09T11:10:30.726 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22dc9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d8072050 con 0x7f22d8151600 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22dc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81576f0 con 0x7f22d815b340 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50462/0 (socket says 172.21.15.117:50462) 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 -- 172.21.15.117:0/3848411464 learned_addr learned my addr 172.21.15.117:0/3848411464 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 -- 172.21.15.117:0/3848411464 >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 msgr2=0x7f22d80c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22d659c700 1 --2- 172.21.15.117:0/3848411464 >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 0x7f22d80c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 --2- 172.21.15.117:0/3848411464 >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 0x7f22d80c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.727 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 -- 172.21.15.117:0/3848411464 >> v1:172.21.15.7:6789/0 conn(0x7f22d8071040 legacy=0x7f22d81552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22cffff700 1 -- 172.21.15.117:0/3848411464 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d81579d0 con 0x7f22d815b340 2023-06-09T11:10:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.720+0000 7f22d659c700 1 --2- 172.21.15.117:0/3848411464 >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 0x7f22d80c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:30.728 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.721+0000 7f22cffff700 1 --2- 172.21.15.117:0/3848411464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098f0 tx=0x7f22c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b27efd589465431 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:30.729 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.721+0000 7f22d5d9b700 1 -- 172.21.15.117:0/3848411464 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0004230 con 0x7f22d815b340 2023-06-09T11:10:30.729 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.721+0000 7f22d5d9b700 1 -- 172.21.15.117:0/3848411464 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0004390 con 0x7f22d815b340 2023-06-09T11:10:30.729 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.721+0000 7f22d5d9b700 1 -- 172.21.15.117:0/3848411464 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c00036a0 con 0x7f22d815b340 2023-06-09T11:10:30.729 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 -- 172.21.15.117:0/3848411464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 msgr2=0x7f22d8070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:30.729 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/3848411464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f22c00098f0 tx=0x7f22c0009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 -- 172.21.15.117:0/3848411464 shutdown_connections 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/3848411464 >> v2:172.21.15.117:3300/0 conn(0x7f22d8151600 0x7f22d80c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/3848411464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d815b340 0x7f22d8070b00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 -- 172.21.15.117:0/3848411464 >> 172.21.15.117:0/3848411464 conn(0x7f22d806a4d0 msgr2=0x7f22d806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 -- 172.21.15.117:0/3848411464 shutdown_connections 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.722+0000 7f22dc9ff700 1 -- 172.21.15.117:0/3848411464 wait complete. 2023-06-09T11:10:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.723+0000 7f22dc9ff700 1 Processor -- start 2023-06-09T11:10:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 -- start start 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d815b340 0x7f22d80c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80cc0c0 con 0x7f22d815b340 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80ca580 con 0x7f22d8151600 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.724+0000 7f22dc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80cbde0 con 0x7f22d8071040 2023-06-09T11:10:30.733 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38064/0 (socket says 172.21.15.117:38064) 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50468/0 (socket says 172.21.15.117:50468) 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22cffff700 1 -- 172.21.15.117:0/225688430 learned_addr learned my addr 172.21.15.117:0/225688430 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d6d9d700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d815b340 0x7f22d80c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 -- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 msgr2=0x7f22d8154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:30.734 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 -- 172.21.15.117:0/225688430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d815b340 msgr2=0x7f22d80c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d815b340 0x7f22d80c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 -- 172.21.15.117:0/225688430 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c0009550 con 0x7f22d8071040 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22cffff700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.725+0000 7f22d659c700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c820 tx=0x7f22c800cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8011800 con 0x7f22d8071040 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8011960 con 0x7f22d8071040 2023-06-09T11:10:30.735 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d8157f90 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8011cf0 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8157cb0 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f22c801a450 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.726+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f22c801aaa0 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.727+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d80c6250 con 0x7f22d8071040 2023-06-09T11:10:30.736 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.730+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f22c8039b10 con 0x7f22d8071040 2023-06-09T11:10:30.789 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:30.783+0000 7f7323c6a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-09T11:10:30.824 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.818+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8beb7b04-04e2-4c76-843c-6347fd4b7423", "id": 0} v 0) v1 -- 0x7f22d80c9e70 con 0x7f22d8071040 2023-06-09T11:10:30.828 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.821+0000 7f22ceffd700 1 -- 172.21.15.117:0/225688430 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8beb7b04-04e2-4c76-843c-6347fd4b7423", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f22c8028030 con 0x7f22d8071040 2023-06-09T11:10:30.828 INFO:teuthology.orchestra.run.smithi117.stdout:0 2023-06-09T11:10:30.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 msgr2=0x7f22d80c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:30.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c820 tx=0x7f22c800cb30 comp rx=0 tx=0).stop 2023-06-09T11:10:30.830 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 shutdown_connections 2023-06-09T11:10:30.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22d815b340 0x7f22d80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22d8071040 0x7f22d80c0140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 --2- 172.21.15.117:0/225688430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22d8151600 0x7f22d8154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:30.831 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 >> 172.21.15.117:0/225688430 conn(0x7f22d806a4d0 msgr2=0x7f22d814bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:30.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 shutdown_connections 2023-06-09T11:10:30.832 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:30.823+0000 7f22dc9ff700 1 -- 172.21.15.117:0/225688430 wait complete. 2023-06-09T11:10:30.846 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new c86f3c75-2178-4498-9490-309afe7973e4 1 2023-06-09T11:10:30.895 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:30.889+0000 7f7323c6a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-09T11:10:31.035 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.030+0000 7f12d13ff700 1 Processor -- start 2023-06-09T11:10:31.036 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.030+0000 7f12d13ff700 1 -- start start 2023-06-09T11:10:31.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12d13ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 0x7f12cc06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.037 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12d13ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12cc06b740 0x7f12cc15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12d13ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0c6220 con 0x7f12cc1537a0 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12d13ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0c4f70 con 0x7f12cc06b740 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12d13ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0c5c20 con 0x7f12cc0c64e0 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12ca7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 0x7f12cc06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12caffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f12cc1537a0 0x7f12cc0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32924/0 (socket says 172.21.15.117:32924) 2023-06-09T11:10:31.038 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.031+0000 7f12caffd700 1 -- 172.21.15.117:0/217350755 learned_addr learned my addr 172.21.15.117:0/217350755 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-06-09T11:10:31.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12cb7fe700 1 --2- 172.21.15.117:0/217350755 >> v2:172.21.15.7:3300/0 conn(0x7f12cc06b740 0x7f12cc15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12c9ffb700 1 -- 172.21.15.117:0/217350755 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127514028 0 0) 0x7f12bc001110 con 0x7f12cc1537a0 2023-06-09T11:10:31.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12c9ffb700 1 -- 172.21.15.117:0/217350755 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b40035b0 con 0x7f12cc1537a0 2023-06-09T11:10:31.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12ca7fc700 1 -- 172.21.15.117:0/217350755 >> v2:172.21.15.7:3300/0 conn(0x7f12cc06b740 msgr2=0x7f12cc15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:31.039 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12ca7fc700 1 --2- 172.21.15.117:0/217350755 >> v2:172.21.15.7:3300/0 conn(0x7f12cc06b740 0x7f12cc15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12ca7fc700 1 -- 172.21.15.117:0/217350755 >> v1:172.21.15.117:6789/0 conn(0x7f12cc1537a0 legacy=0x7f12cc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.040 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12ca7fc700 1 -- 172.21.15.117:0/217350755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0c4480 con 0x7f12cc0c64e0 2023-06-09T11:10:31.041 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12ca7fc700 1 --2- 172.21.15.117:0/217350755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 0x7f12cc06b1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009840 tx=0x7f12c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d1e9edf9628f976 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.041 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12c9ffb700 1 -- 172.21.15.117:0/217350755 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011830 con 0x7f12cc0c64e0 2023-06-09T11:10:31.041 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12c9ffb700 1 -- 172.21.15.117:0/217350755 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0011990 con 0x7f12cc0c64e0 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.032+0000 7f12c9ffb700 1 -- 172.21.15.117:0/217350755 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0011d20 con 0x7f12cc0c64e0 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 -- 172.21.15.117:0/217350755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 msgr2=0x7f12cc06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 --2- 172.21.15.117:0/217350755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 0x7f12cc06b1b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009840 tx=0x7f12c0009b50 comp rx=0 tx=0).stop 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 -- 172.21.15.117:0/217350755 shutdown_connections 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 --2- 172.21.15.117:0/217350755 >> v2:172.21.15.7:3300/0 conn(0x7f12cc06b740 0x7f12cc15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 --2- 172.21.15.117:0/217350755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc0c64e0 0x7f12cc06b1b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.042 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 -- 172.21.15.117:0/217350755 >> 172.21.15.117:0/217350755 conn(0x7f12cc072040 msgr2=0x7f12cc072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.043 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 -- 172.21.15.117:0/217350755 shutdown_connections 2023-06-09T11:10:31.043 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.033+0000 7f12d13ff700 1 -- 172.21.15.117:0/217350755 wait complete. 2023-06-09T11:10:31.043 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.036+0000 7f12d13ff700 1 Processor -- start 2023-06-09T11:10:31.043 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.036+0000 7f12d13ff700 1 -- start start 2023-06-09T11:10:31.045 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc06b740 0x7f12cc0cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.045 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.045 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.045 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0c86d0 con 0x7f12cc06b740 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1e9260 con 0x7f12cc0c64e0 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12d13ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc1e34b0 con 0x7f12cc1537a0 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12ca7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12cb7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50480/0 (socket says 172.21.15.117:50480) 2023-06-09T11:10:31.046 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12caffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc06b740 0x7f12cc0cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.047 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12cb7fe700 1 -- 172.21.15.117:0/1465945832 learned_addr learned my addr 172.21.15.117:0/1465945832 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:31.047 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.037+0000 7f12ca7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38076/0 (socket says 172.21.15.117:38076) 2023-06-09T11:10:31.047 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 -- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 msgr2=0x7f12cc14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.047 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.047 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 -- 172.21.15.117:0/1465945832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc06b740 msgr2=0x7f12cc0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc06b740 0x7f12cc0cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 -- 172.21.15.117:0/1465945832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c00094a0 con 0x7f12cc0c64e0 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12cb7fe700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12ca7fc700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f12c0016000 tx=0x7f12c0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c00036a0 con 0x7f12cc0c64e0 2023-06-09T11:10:31.048 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c001f070 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc0c46d0 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.038+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0003a20 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.039+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc15ae20 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.039+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f12c0026480 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.039+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f12c0021070 con 0x7f12cc0c64e0 2023-06-09T11:10:31.049 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.040+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc0c49b0 con 0x7f12cc0c64e0 2023-06-09T11:10:31.050 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.042+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f12c002b020 con 0x7f12cc0c64e0 2023-06-09T11:10:31.124 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:31.118+0000 7f7f2a848200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-09T11:10:31.136 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.131+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c86f3c75-2178-4498-9490-309afe7973e4", "id": 1} v 0) v1 -- 0x7f12cc08d0c0 con 0x7f12cc0c64e0 2023-06-09T11:10:31.140 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.135+0000 7f12b37fe700 1 -- 172.21.15.117:0/1465945832 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c86f3c75-2178-4498-9490-309afe7973e4", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f12c0015020 con 0x7f12cc0c64e0 2023-06-09T11:10:31.140 INFO:teuthology.orchestra.run.smithi117.stdout:1 2023-06-09T11:10:31.141 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 msgr2=0x7f12cc148350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f12c0016000 tx=0x7f12c0004750 comp rx=0 tx=0).stop 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 shutdown_connections 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12cc1537a0 0x7f12cc14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc06b740 0x7f12cc0cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 --2- 172.21.15.117:0/1465945832 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12cc0c64e0 0x7f12cc148350 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.142 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 >> 172.21.15.117:0/1465945832 conn(0x7f12cc072040 msgr2=0x7f12cc15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 shutdown_connections 2023-06-09T11:10:31.143 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.136+0000 7f12d13ff700 1 -- 172.21.15.117:0/1465945832 wait complete. 2023-06-09T11:10:31.159 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb 2 2023-06-09T11:10:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.342+0000 7f0df27ff700 1 Processor -- start 2023-06-09T11:10:31.348 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.342+0000 7f0df27ff700 1 -- start start 2023-06-09T11:10:31.349 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 0x7f0dec073c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.349 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df27ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec15cef0 con 0x7f0dec153b60 2023-06-09T11:10:31.350 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df27ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec15da90 con 0x7f0dec153790 2023-06-09T11:10:31.350 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15cbc0 con 0x7f0dec0737f0 2023-06-09T11:10:31.350 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0debfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0dec153790 0x7f0dec15d380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32936/0 (socket says 172.21.15.117:32936) 2023-06-09T11:10:31.350 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0debfff700 1 -- 172.21.15.117:0/3939422484 learned_addr learned my addr 172.21.15.117:0/3939422484 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-06-09T11:10:31.350 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df0d9c700 1 --2- 172.21.15.117:0/3939422484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 0x7f0dec073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.351 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879514181 0 0) 0x7f0dec15da90 con 0x7f0dec153790 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd80035b0 con 0x7f0dec153790 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056700613 0 0) 0x7f0dec15cef0 con 0x7f0dec153b60 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df0d9c700 1 -- 172.21.15.117:0/3939422484 >> v1:172.21.15.117:6789/0 conn(0x7f0dec153790 legacy=0x7f0dec15d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df0d9c700 1 -- 172.21.15.117:0/3939422484 >> v1:172.21.15.7:6789/0 conn(0x7f0dec153b60 legacy=0x7f0dec06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.344+0000 7f0df0d9c700 1 -- 172.21.15.117:0/3939422484 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec15aac0 con 0x7f0dec0737f0 2023-06-09T11:10:31.352 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0df0d9c700 1 --2- 172.21.15.117:0/3939422484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 0x7f0dec073c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009dc0 tx=0x7f0de400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b85be97e5313dd2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de400e070 con 0x7f0dec0737f0 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de400bb20 con 0x7f0dec0737f0 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0deb7fe700 1 -- 172.21.15.117:0/3939422484 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de400bed0 con 0x7f0dec0737f0 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0df27ff700 1 -- 172.21.15.117:0/3939422484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 msgr2=0x7f0dec073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0df27ff700 1 --2- 172.21.15.117:0/3939422484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 0x7f0dec073c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009dc0 tx=0x7f0de400b330 comp rx=0 tx=0).stop 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.345+0000 7f0df27ff700 1 -- 172.21.15.117:0/3939422484 shutdown_connections 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.346+0000 7f0df27ff700 1 --2- 172.21.15.117:0/3939422484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec0737f0 0x7f0dec073c70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.353 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.346+0000 7f0df27ff700 1 -- 172.21.15.117:0/3939422484 >> 172.21.15.117:0/3939422484 conn(0x7f0dec071410 msgr2=0x7f0dec0722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.346+0000 7f0df27ff700 1 -- 172.21.15.117:0/3939422484 shutdown_connections 2023-06-09T11:10:31.354 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.346+0000 7f0df27ff700 1 -- 172.21.15.117:0/3939422484 wait complete. 2023-06-09T11:10:31.355 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.347+0000 7f0df27ff700 1 Processor -- start 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.347+0000 7f0df27ff700 1 -- start start 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dec0737f0 0x7f0dec150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 0x7f0dec149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec0ce770 con 0x7f0dec0737f0 2023-06-09T11:10:31.356 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1ef530 con 0x7f0dec153790 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0df27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1521b0 con 0x7f0dec153b60 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.348+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0debfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dec0737f0 0x7f0dec150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 0x7f0dec149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0de3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38090/0 (socket says 172.21.15.117:38090) 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0de3fff700 1 -- 172.21.15.117:0/419817976 learned_addr learned my addr 172.21.15.117:0/419817976 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 -- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 msgr2=0x7f0dec1eeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.357 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 -- 172.21.15.117:0/419817976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dec0737f0 msgr2=0x7f0dec150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dec0737f0 0x7f0dec150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 -- 172.21.15.117:0/419817976 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de4009a70 con 0x7f0dec153b60 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0de3fff700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0df0d9c700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 0x7f0dec149a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dec08c7d0 tx=0x7f0de4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.349+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de401c810 con 0x7f0dec153b60 2023-06-09T11:10:31.358 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4020070 con 0x7f0dec153b60 2023-06-09T11:10:31.359 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec0c4a80 con 0x7f0dec153b60 2023-06-09T11:10:31.359 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de401cbb0 con 0x7f0dec153b60 2023-06-09T11:10:31.359 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec0c47a0 con 0x7f0dec153b60 2023-06-09T11:10:31.359 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0de4027780 con 0x7f0dec153b60 2023-06-09T11:10:31.359 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.350+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0de4022070 con 0x7f0dec153b60 2023-06-09T11:10:31.360 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.351+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec0c4d60 con 0x7f0dec153b60 2023-06-09T11:10:31.360 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.353+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f0de4046b10 con 0x7f0dec153b60 2023-06-09T11:10:31.438 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:31.432+0000 7f7323c6a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-09T11:10:31.445 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.440+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb", "id": 2} v 0) v1 -- 0x7f0dec074cf0 con 0x7f0dec153b60 2023-06-09T11:10:31.449 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.444+0000 7f0de97fa700 1 -- 172.21.15.117:0/419817976 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0de4035030 con 0x7f0dec153b60 2023-06-09T11:10:31.450 INFO:teuthology.orchestra.run.smithi117.stdout:2 2023-06-09T11:10:31.451 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.444+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 msgr2=0x7f0dec149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.451 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.444+0000 7f0df27ff700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 0x7f0dec149a90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0dec08c7d0 tx=0x7f0de4004ad0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.452 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 shutdown_connections 2023-06-09T11:10:31.452 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0dec153b60 0x7f0dec149a90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.452 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0dec0737f0 0x7f0dec150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.452 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 --2- 172.21.15.117:0/419817976 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0dec153790 0x7f0dec1eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.453 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 >> 172.21.15.117:0/419817976 conn(0x7f0dec071410 msgr2=0x7f0dec0722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.453 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 shutdown_connections 2023-06-09T11:10:31.453 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.445+0000 7f0df27ff700 1 -- 172.21.15.117:0/419817976 wait complete. 2023-06-09T11:10:31.467 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new caab7ecc-cafe-4ff5-8472-dffc52ac78d1 3 2023-06-09T11:10:31.659 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.653+0000 7fe0d7fff700 1 Processor -- start 2023-06-09T11:10:31.659 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.653+0000 7fe0d7fff700 1 -- start start 2023-06-09T11:10:31.660 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 0x7fe0d006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.661 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d7fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00c6220 con 0x7fe0d006b740 2023-06-09T11:10:31.661 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d7fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00c4480 con 0x7fe0d01537a0 2023-06-09T11:10:31.661 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00c4f70 con 0x7fe0d00c64e0 2023-06-09T11:10:31.661 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d5d9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe0d01537a0 0x7fe0d00cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32948/0 (socket says 172.21.15.117:32948) 2023-06-09T11:10:31.662 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d559a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 0x7fe0d006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.662 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d5d9b700 1 -- 172.21.15.117:0/3047861947 learned_addr learned my addr 172.21.15.117:0/3047861947 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-06-09T11:10:31.663 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d559a700 1 -- 172.21.15.117:0/3047861947 >> v1:172.21.15.117:6789/0 conn(0x7fe0d01537a0 legacy=0x7fe0d00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.663 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d559a700 1 -- 172.21.15.117:0/3047861947 >> v1:172.21.15.7:6789/0 conn(0x7fe0d006b740 legacy=0x7fe0d015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.663 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d559a700 1 -- 172.21.15.117:0/3047861947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d015ae20 con 0x7fe0d00c64e0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.655+0000 7fe0d4d99700 1 -- 172.21.15.117:0/3047861947 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537996884 0 0) 0x7fe0d00c4480 con 0x7fe0d01537a0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d559a700 1 --2- 172.21.15.117:0/3047861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 0x7fe0d006b1b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009870 tx=0x7fe0c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426c556f2309c9d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d4d99700 1 -- 172.21.15.117:0/3047861947 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011830 con 0x7fe0d00c64e0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d4d99700 1 -- 172.21.15.117:0/3047861947 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0011990 con 0x7fe0d00c64e0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d4d99700 1 -- 172.21.15.117:0/3047861947 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011d20 con 0x7fe0d00c64e0 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d7fff700 1 -- 172.21.15.117:0/3047861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 msgr2=0x7fe0d006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.664 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.656+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/3047861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 0x7fe0d006b1b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009870 tx=0x7fe0c0009b80 comp rx=0 tx=0).stop 2023-06-09T11:10:31.665 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.657+0000 7fe0d7fff700 1 -- 172.21.15.117:0/3047861947 shutdown_connections 2023-06-09T11:10:31.665 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.657+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/3047861947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d00c64e0 0x7fe0d006b1b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.665 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.657+0000 7fe0d7fff700 1 -- 172.21.15.117:0/3047861947 >> 172.21.15.117:0/3047861947 conn(0x7fe0d0072040 msgr2=0x7fe0d0072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.665 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.657+0000 7fe0d7fff700 1 -- 172.21.15.117:0/3047861947 shutdown_connections 2023-06-09T11:10:31.665 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.657+0000 7fe0d7fff700 1 -- 172.21.15.117:0/3047861947 wait complete. 2023-06-09T11:10:31.667 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.658+0000 7fe0d7fff700 1 Processor -- start 2023-06-09T11:10:31.668 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.659+0000 7fe0d7fff700 1 -- start start 2023-06-09T11:10:31.668 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.659+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.668 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.659+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d00c64e0 0x7fe0d014e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.659+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 0x7fe0d01492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.659+0000 7fe0d7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00cede0 con 0x7fe0d01537a0 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0148df0 con 0x7fe0d00c64e0 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d01489d0 con 0x7fe0d006b740 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d559a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d00c64e0 0x7fe0d014e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50516/0 (socket says 172.21.15.117:50516) 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 -- 172.21.15.117:0/385857785 learned_addr learned my addr 172.21.15.117:0/385857785 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d659c700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 0x7fe0d01492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 -- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d00c64e0 msgr2=0x7fe0d014e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d00c64e0 0x7fe0d014e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 -- 172.21.15.117:0/385857785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 msgr2=0x7fe0d01492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 0x7fe0d01492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 -- 172.21.15.117:0/385857785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c00094a0 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d659c700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 0x7fe0d01492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.660+0000 7fe0d5d9b700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c550 tx=0x7fe0cc00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc004c80 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc004e00 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d00c49b0 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc007310 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d00c46d0 con 0x7fe0d006b740 2023-06-09T11:10:31.671 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe0cc01d3f0 con 0x7fe0d006b740 2023-06-09T11:10:31.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.661+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe0cc01db30 con 0x7fe0d006b740 2023-06-09T11:10:31.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.662+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d00c4c90 con 0x7fe0d006b740 2023-06-09T11:10:31.672 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.664+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fe0cc01dd80 con 0x7fe0d006b740 2023-06-09T11:10:31.756 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.750+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "caab7ecc-cafe-4ff5-8472-dffc52ac78d1", "id": 3} v 0) v1 -- 0x7fe0d00c5180 con 0x7fe0d006b740 2023-06-09T11:10:31.759 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.754+0000 7fe0c67fc700 1 -- 172.21.15.117:0/385857785 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "caab7ecc-cafe-4ff5-8472-dffc52ac78d1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe0cc01d850 con 0x7fe0d006b740 2023-06-09T11:10:31.760 INFO:teuthology.orchestra.run.smithi117.stdout:3 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 msgr2=0x7fe0d00cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00c550 tx=0x7fe0cc00c860 comp rx=0 tx=0).stop 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 shutdown_connections 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d01537a0 0x7fe0d01492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0d006b740 0x7fe0d00cf060 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.761 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 --2- 172.21.15.117:0/385857785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d00c64e0 0x7fe0d014e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 >> 172.21.15.117:0/385857785 conn(0x7fe0d0072040 msgr2=0x7fe0d015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 shutdown_connections 2023-06-09T11:10:31.762 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.755+0000 7fe0d7fff700 1 -- 172.21.15.117:0/385857785 wait complete. 2023-06-09T11:10:31.778 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 267f3c53-d8f4-4f37-90bd-8b39be8322f3 4 2023-06-09T11:10:31.849 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:31.843+0000 7f7f2a848200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-09T11:10:32.001 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.995+0000 7f7f161ff700 1 Processor -- start 2023-06-09T11:10:32.001 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.995+0000 7f7f161ff700 1 -- start start 2023-06-09T11:10:32.002 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f161ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7f101537a0 0x7f7f1006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.003 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f161ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.003 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f161ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100c6220 con 0x7f7f100c64e0 2023-06-09T11:10:32.003 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f161ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100c4f70 con 0x7f7f101537a0 2023-06-09T11:10:32.003 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f161ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100c5c20 con 0x7f7f1006b740 2023-06-09T11:10:32.003 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50518/0 (socket says 172.21.15.117:50518) 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 -- 172.21.15.117:0/1754760295 learned_addr learned my addr 172.21.15.117:0/1754760295 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0effd700 1 --1- 172.21.15.117:0/1754760295 >> v1:172.21.15.117:6789/0 conn(0x7f7f100c64e0 0x7f7f100cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32954/0 (socket says 172.21.15.117:32954) 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0f7fe700 1 --2- 172.21.15.117:0/1754760295 >> v2:172.21.15.7:3300/0 conn(0x7f7f101537a0 0x7f7f1006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 -- 172.21.15.117:0/1754760295 >> v2:172.21.15.7:3300/0 conn(0x7f7f101537a0 msgr2=0x7f7f1006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:32.004 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 --2- 172.21.15.117:0/1754760295 >> v2:172.21.15.7:3300/0 conn(0x7f7f101537a0 0x7f7f1006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.005 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 -- 172.21.15.117:0/1754760295 >> v1:172.21.15.117:6789/0 conn(0x7f7f100c64e0 legacy=0x7f7f100cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.005 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.997+0000 7f7f0ffff700 1 -- 172.21.15.117:0/1754760295 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100c4480 con 0x7f7f1006b740 2023-06-09T11:10:32.005 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.998+0000 7f7f0ffff700 1 --2- 172.21.15.117:0/1754760295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f040098f0 tx=0x7f7f04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae58d0ca5886ba10 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.998+0000 7f7f0e7fc700 1 -- 172.21.15.117:0/1754760295 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011830 con 0x7f7f1006b740 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.998+0000 7f7f0e7fc700 1 -- 172.21.15.117:0/1754760295 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04011990 con 0x7f7f1006b740 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.998+0000 7f7f0e7fc700 1 -- 172.21.15.117:0/1754760295 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04011d20 con 0x7f7f1006b740 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 -- 172.21.15.117:0/1754760295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 msgr2=0x7f7f1015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 --2- 172.21.15.117:0/1754760295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7f040098f0 tx=0x7f7f04009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:32.006 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 -- 172.21.15.117:0/1754760295 shutdown_connections 2023-06-09T11:10:32.007 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 --2- 172.21.15.117:0/1754760295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f1006b740 0x7f7f1015b2b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.007 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 --2- 172.21.15.117:0/1754760295 >> v2:172.21.15.7:3300/0 conn(0x7f7f101537a0 0x7f7f1006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.007 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 -- 172.21.15.117:0/1754760295 >> 172.21.15.117:0/1754760295 conn(0x7f7f10072040 msgr2=0x7f7f10072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.007 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 -- 172.21.15.117:0/1754760295 shutdown_connections 2023-06-09T11:10:32.007 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:31.999+0000 7f7f161ff700 1 -- 172.21.15.117:0/1754760295 wait complete. 2023-06-09T11:10:32.009 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.001+0000 7f7f161ff700 1 Processor -- start 2023-06-09T11:10:32.009 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.001+0000 7f7f161ff700 1 -- start start 2023-06-09T11:10:32.009 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 0x7f7f101f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 0x7f7f101f0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f101efc90 con 0x7f7f100c64e0 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100cf2f0 con 0x7f7f1006b740 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f161ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10146f60 con 0x7f7f101537a0 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.010 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38106/0 (socket says 172.21.15.117:38106) 2023-06-09T11:10:32.011 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 0x7f7f101f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.011 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0f7fe700 1 -- 172.21.15.117:0/3794320167 learned_addr learned my addr 172.21.15.117:0/3794320167 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.011 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0effd700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 0x7f7f101f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.012 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.002+0000 7f7f0ffff700 1 -- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 msgr2=0x7f7f101483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.012 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0ffff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.012 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0ffff700 1 -- 172.21.15.117:0/3794320167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 msgr2=0x7f7f101f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.012 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0ffff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 0x7f7f101f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.012 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0ffff700 1 -- 172.21.15.117:0/3794320167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04009550 con 0x7f7f101537a0 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0f7fe700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0effd700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 0x7f7f101f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f0ffff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 0x7f7f101f0960 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f04016000 tx=0x7f7f04004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04004510 con 0x7f7f101537a0 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0401e070 con 0x7f7f101537a0 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f100c46d0 con 0x7f7f101537a0 2023-06-09T11:10:32.013 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040038e0 con 0x7f7f101537a0 2023-06-09T11:10:32.014 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.003+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1015ae20 con 0x7f7f101537a0 2023-06-09T11:10:32.014 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.004+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7f04003d40 con 0x7f7f101537a0 2023-06-09T11:10:32.015 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.004+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f7f0402db80 con 0x7f7f101537a0 2023-06-09T11:10:32.015 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.004+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f100c49b0 con 0x7f7f101537a0 2023-06-09T11:10:32.015 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.007+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f7f04020070 con 0x7f7f101537a0 2023-06-09T11:10:32.039 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:32.034+0000 7f7f2a848200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-09T11:10:32.099 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.093+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "267f3c53-d8f4-4f37-90bd-8b39be8322f3", "id": 4} v 0) v1 -- 0x7f7f101f07d0 con 0x7f7f101537a0 2023-06-09T11:10:32.105 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.100+0000 7f7ef7fff700 1 -- 172.21.15.117:0/3794320167 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "267f3c53-d8f4-4f37-90bd-8b39be8322f3", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7f04015020 con 0x7f7f101537a0 2023-06-09T11:10:32.106 INFO:teuthology.orchestra.run.smithi117.stdout:4 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 msgr2=0x7f7f101f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 0x7f7f101f0960 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7f04016000 tx=0x7f7f04004750 comp rx=0 tx=0).stop 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 shutdown_connections 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f101537a0 0x7f7f101f0960 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f1006b740 0x7f7f101483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.107 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 --2- 172.21.15.117:0/3794320167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f100c64e0 0x7f7f101f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 >> 172.21.15.117:0/3794320167 conn(0x7f7f10072040 msgr2=0x7f7f1014f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 shutdown_connections 2023-06-09T11:10:32.108 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.101+0000 7f7f161ff700 1 -- 172.21.15.117:0/3794320167 wait complete. 2023-06-09T11:10:32.125 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 2a285f66-b951-48de-9909-2de669a508bb 5 2023-06-09T11:10:32.166 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:32.161+0000 7f7323c6a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-09T11:10:32.318 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.313+0000 7f99fc7ff700 1 Processor -- start 2023-06-09T11:10:32.319 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.313+0000 7f99fc7ff700 1 -- start start 2023-06-09T11:10:32.320 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fc7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f41537a0 0x7f99f406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.320 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fc7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f99f406b740 0x7f99f415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.320 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fc7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f40c6220 con 0x7f99f40c64e0 2023-06-09T11:10:32.320 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fc7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f40c4f70 con 0x7f99f406b740 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fc7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f40c5c20 con 0x7f99f41537a0 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fad9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f99f406b740 0x7f99f415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fad9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f99f406b740 0x7f99f415b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38108/0 (socket says 172.21.15.117:38108) 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fad9c700 1 -- 172.21.15.117:0/2266407278 learned_addr learned my addr 172.21.15.117:0/2266407278 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99f9d9a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f99f40c64e0 0x7f99f40cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32962/0 (socket says 172.21.15.117:32962) 2023-06-09T11:10:32.321 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.314+0000 7f99fa59b700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f41537a0 0x7f99f406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.322 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99fad9c700 1 -- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f41537a0 msgr2=0x7f99f406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:32.322 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99fad9c700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f41537a0 0x7f99f406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99fad9c700 1 -- 172.21.15.117:0/2266407278 >> v1:172.21.15.117:6789/0 conn(0x7f99f40c64e0 legacy=0x7f99f40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99fad9c700 1 -- 172.21.15.117:0/2266407278 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40c4480 con 0x7f99f406b740 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99fad9c700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f415b2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009e50 tx=0x7f99ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34acd7542c26f39c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.315+0000 7f99f9599700 1 -- 172.21.15.117:0/2266407278 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00e070 con 0x7f99f406b740 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99f9599700 1 -- 172.21.15.117:0/2266407278 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00b910 con 0x7f99f406b740 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99f9599700 1 -- 172.21.15.117:0/2266407278 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00bcc0 con 0x7f99f406b740 2023-06-09T11:10:32.323 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 msgr2=0x7f99f415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f415b2b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009e50 tx=0x7f99ec00b330 comp rx=0 tx=0).stop 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2266407278 shutdown_connections 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f415b2b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2266407278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f41537a0 0x7f99f406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2266407278 >> 172.21.15.117:0/2266407278 conn(0x7f99f4072040 msgr2=0x7f99f4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2266407278 shutdown_connections 2023-06-09T11:10:32.324 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.316+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2266407278 wait complete. 2023-06-09T11:10:32.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.318+0000 7f99fc7ff700 1 Processor -- start 2023-06-09T11:10:32.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.318+0000 7f99fc7ff700 1 -- start start 2023-06-09T11:10:32.326 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 0x7f99f41f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99f41537a0 0x7f99f414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f4149dd0 con 0x7f99f41537a0 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f41f2b10 con 0x7f99f406b740 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fc7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f414a7a0 con 0x7f99f40c64e0 2023-06-09T11:10:32.327 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fa59b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99f9d9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 0x7f99f41f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fa59b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38122/0 (socket says 172.21.15.117:38122) 2023-06-09T11:10:32.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fa59b700 1 -- 172.21.15.117:0/2639517229 learned_addr learned my addr 172.21.15.117:0/2639517229 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.328 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.319+0000 7f99fad9c700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99f41537a0 0x7f99f414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.329 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 -- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 msgr2=0x7f99f41f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.329 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 0x7f99f41f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.329 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 -- 172.21.15.117:0/2639517229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99f41537a0 msgr2=0x7f99f414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99f41537a0 0x7f99f414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 -- 172.21.15.117:0/2639517229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ec009b30 con 0x7f99f406b740 2023-06-09T11:10:32.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99f9d9a700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 0x7f99f41f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.330 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fa59b700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99f000e9b0 tx=0x7f99f000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0009710 con 0x7f99f406b740 2023-06-09T11:10:32.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f000ee50 con 0x7f99f406b740 2023-06-09T11:10:32.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f000ca50 con 0x7f99f406b740 2023-06-09T11:10:32.331 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f40c46d0 con 0x7f99f406b740 2023-06-09T11:10:32.332 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.320+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f415ae20 con 0x7f99f406b740 2023-06-09T11:10:32.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.321+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f99f0005a20 con 0x7f99f406b740 2023-06-09T11:10:32.333 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.321+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f99f0005cd0 con 0x7f99f406b740 2023-06-09T11:10:32.334 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.321+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f40c49b0 con 0x7f99f406b740 2023-06-09T11:10:32.335 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.324+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f99f0021060 con 0x7f99f406b740 2023-06-09T11:10:32.363 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:32.357+0000 7f7323c6a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-09T11:10:32.416 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.410+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2a285f66-b951-48de-9909-2de669a508bb", "id": 5} v 0) v1 -- 0x7f99f4074cf0 con 0x7f99f406b740 2023-06-09T11:10:32.420 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.414+0000 7f99eaffd700 1 -- 172.21.15.117:0/2639517229 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a285f66-b951-48de-9909-2de669a508bb", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f99f0029020 con 0x7f99f406b740 2023-06-09T11:10:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:5 2023-06-09T11:10:32.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 msgr2=0x7f99f40c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f99f000e9b0 tx=0x7f99f000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.421 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 shutdown_connections 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99f41537a0 0x7f99f414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f99f406b740 0x7f99f40c7950 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 --2- 172.21.15.117:0/2639517229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f99f40c64e0 0x7f99f41f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.415+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 >> 172.21.15.117:0/2639517229 conn(0x7f99f4072040 msgr2=0x7f99f414df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.416+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 shutdown_connections 2023-06-09T11:10:32.422 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.416+0000 7f99fc7ff700 1 -- 172.21.15.117:0/2639517229 wait complete. 2023-06-09T11:10:32.438 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new ed9a82da-beba-41bd-a493-90c966e52b1a 6 2023-06-09T11:10:32.667 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.661+0000 7fe8451ff700 1 Processor -- start 2023-06-09T11:10:32.667 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.661+0000 7fe8451ff700 1 -- start start 2023-06-09T11:10:32.668 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.662+0000 7fe8451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 0x7fe8400ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.662+0000 7fe8451ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe84006ac20 0x7fe8400cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.662+0000 7fe8451ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400b40e0 con 0x7fe8400c6590 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.662+0000 7fe8451ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400cca00 con 0x7fe84006ac20 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.662+0000 7fe8451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84006b110 con 0x7fe840153850 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 0x7fe8400ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83e59c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fe8400c6590 0x7fe840070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32964/0 (socket says 172.21.15.117:32964) 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83e59c700 1 -- 172.21.15.117:0/940990066 learned_addr learned my addr 172.21.15.117:0/940990066 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-06-09T11:10:32.669 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83f59e700 1 --2- 172.21.15.117:0/940990066 >> v2:172.21.15.7:3300/0 conn(0x7fe84006ac20 0x7fe8400cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 -- 172.21.15.117:0/940990066 >> v2:172.21.15.7:3300/0 conn(0x7fe84006ac20 msgr2=0x7fe8400cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:32.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 --2- 172.21.15.117:0/940990066 >> v2:172.21.15.7:3300/0 conn(0x7fe84006ac20 0x7fe8400cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 -- 172.21.15.117:0/940990066 >> v1:172.21.15.117:6789/0 conn(0x7fe8400c6590 legacy=0x7fe840070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 -- 172.21.15.117:0/940990066 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400b4330 con 0x7fe840153850 2023-06-09T11:10:32.670 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83dd9b700 1 -- 172.21.15.117:0/940990066 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563615484 0 0) 0x7fe8400b40e0 con 0x7fe8400c6590 2023-06-09T11:10:32.674 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.663+0000 7fe83ed9d700 1 --2- 172.21.15.117:0/940990066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 0x7fe8400ca890 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe830009840 tx=0x7fe830009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79985c6d1d588520 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.675 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe83dd9b700 1 -- 172.21.15.117:0/940990066 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830011830 con 0x7fe840153850 2023-06-09T11:10:32.675 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe83dd9b700 1 -- 172.21.15.117:0/940990066 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830011990 con 0x7fe840153850 2023-06-09T11:10:32.675 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe83dd9b700 1 -- 172.21.15.117:0/940990066 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830011d20 con 0x7fe840153850 2023-06-09T11:10:32.676 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 -- 172.21.15.117:0/940990066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 msgr2=0x7fe8400ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.676 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 --2- 172.21.15.117:0/940990066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 0x7fe8400ca890 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe830009840 tx=0x7fe830009b50 comp rx=0 tx=0).stop 2023-06-09T11:10:32.676 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 -- 172.21.15.117:0/940990066 shutdown_connections 2023-06-09T11:10:32.677 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 --2- 172.21.15.117:0/940990066 >> v2:172.21.15.7:3300/0 conn(0x7fe84006ac20 0x7fe8400cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.677 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 --2- 172.21.15.117:0/940990066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe840153850 0x7fe8400ca890 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.677 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 -- 172.21.15.117:0/940990066 >> 172.21.15.117:0/940990066 conn(0x7fe8400737f0 msgr2=0x7fe840072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.677 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 -- 172.21.15.117:0/940990066 shutdown_connections 2023-06-09T11:10:32.678 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.664+0000 7fe8451ff700 1 -- 172.21.15.117:0/940990066 wait complete. 2023-06-09T11:10:32.678 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.666+0000 7fe8451ff700 1 Processor -- start 2023-06-09T11:10:32.678 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.666+0000 7fe8451ff700 1 -- start start 2023-06-09T11:10:32.678 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 0x7fe8401e2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840153850 0x7fe840149f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84014c8c0 con 0x7fe840153850 2023-06-09T11:10:32.679 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400cffc0 con 0x7fe84006ac20 2023-06-09T11:10:32.680 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe8451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400d02a0 con 0x7fe8400c6590 2023-06-09T11:10:32.680 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe83ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.680 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe83e59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 0x7fe8401e2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe83ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38136/0 (socket says 172.21.15.117:38136) 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe83ed9d700 1 -- 172.21.15.117:0/958212272 learned_addr learned my addr 172.21.15.117:0/958212272 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.667+0000 7fe83f59e700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840153850 0x7fe840149f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 -- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 msgr2=0x7fe8401ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 -- 172.21.15.117:0/958212272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840153850 msgr2=0x7fe840149f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840153850 0x7fe840149f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.681 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 -- 172.21.15.117:0/958212272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300094a0 con 0x7fe8400c6590 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83ed9d700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe83e59c700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 0x7fe8401e2a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c4b0 tx=0x7fe83400c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834004c80 con 0x7fe8400c6590 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe834004e00 con 0x7fe8400c6590 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834007310 con 0x7fe8400c6590 2023-06-09T11:10:32.682 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8400b48f0 con 0x7fe8400c6590 2023-06-09T11:10:32.683 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8400b4610 con 0x7fe8400c6590 2023-06-09T11:10:32.683 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.668+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe83401d3f0 con 0x7fe8400c6590 2023-06-09T11:10:32.683 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.669+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fe83401dc90 con 0x7fe8400c6590 2023-06-09T11:10:32.683 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.669+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe840153c20 con 0x7fe8400c6590 2023-06-09T11:10:32.684 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.672+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fe83401d6a0 con 0x7fe8400c6590 2023-06-09T11:10:32.763 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.757+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed9a82da-beba-41bd-a493-90c966e52b1a", "id": 6} v 0) v1 -- 0x7fe84004e8c0 con 0x7fe8400c6590 2023-06-09T11:10:32.768 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.762+0000 7fe8277fe700 1 -- 172.21.15.117:0/958212272 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed9a82da-beba-41bd-a493-90c966e52b1a", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe83401d910 con 0x7fe8400c6590 2023-06-09T11:10:32.769 INFO:teuthology.orchestra.run.smithi117.stdout:6 2023-06-09T11:10:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 msgr2=0x7fe8401e2a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 0x7fe8401e2a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe83400c4b0 tx=0x7fe83400c7c0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 shutdown_connections 2023-06-09T11:10:32.770 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe840153850 0x7fe840149f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe84006ac20 0x7fe8401ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 --2- 172.21.15.117:0/958212272 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8400c6590 0x7fe8401e2a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 >> 172.21.15.117:0/958212272 conn(0x7fe8400737f0 msgr2=0x7fe8400b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 shutdown_connections 2023-06-09T11:10:32.772 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.763+0000 7fe8451ff700 1 -- 172.21.15.117:0/958212272 wait complete. 2023-06-09T11:10:32.785 DEBUG:teuthology.orchestra.run.smithi117:> sudo ceph --cluster ceph osd new 97300c66-1e58-4029-91ed-e65bc4497fe3 7 2023-06-09T11:10:32.899 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:32.893+0000 7f7f2a848200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-09T11:10:32.973 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.968+0000 7f22877ff700 1 Processor -- start 2023-06-09T11:10:32.974 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.968+0000 7f22877ff700 1 -- start start 2023-06-09T11:10:32.974 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f22877ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22801537a0 0x7f228006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f22877ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22800c6220 con 0x7f228006b740 2023-06-09T11:10:32.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f22877ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22800c4480 con 0x7f22800c64e0 2023-06-09T11:10:32.975 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f22877ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22800c4f70 con 0x7f22801537a0 2023-06-09T11:10:32.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f2284d9a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f22800c64e0 0x7f22800cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.117:32968/0 (socket says 172.21.15.117:32968) 2023-06-09T11:10:32.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f2284d9a700 1 -- 172.21.15.117:0/1300340702 learned_addr learned my addr 172.21.15.117:0/1300340702 (peer_addr_for_me v1:172.21.15.117:0/0) 2023-06-09T11:10:32.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.969+0000 7f228559b700 1 --2- 172.21.15.117:0/1300340702 >> v2:172.21.15.7:3300/0 conn(0x7f22801537a0 0x7f228006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.976 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988556171 0 0) 0x7f22800c4480 con 0x7f22800c64e0 2023-06-09T11:10:32.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f226c0035b0 con 0x7f22800c64e0 2023-06-09T11:10:32.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f228559b700 1 -- 172.21.15.117:0/1300340702 >> v1:172.21.15.117:6789/0 conn(0x7f22800c64e0 legacy=0x7f22800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f228559b700 1 -- 172.21.15.117:0/1300340702 >> v1:172.21.15.7:6789/0 conn(0x7f228006b740 legacy=0x7f228015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.977 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452198633 0 0) 0x7f2278001490 con 0x7f228006b740 2023-06-09T11:10:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f228559b700 1 -- 172.21.15.117:0/1300340702 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228015ae20 con 0x7f22801537a0 2023-06-09T11:10:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.970+0000 7f228559b700 1 --2- 172.21.15.117:0/1300340702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f228006b1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f227c009840 tx=0x7f227c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a943132a4034c2f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c011830 con 0x7f22801537a0 2023-06-09T11:10:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c011990 con 0x7f22801537a0 2023-06-09T11:10:32.978 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f2277fff700 1 -- 172.21.15.117:0/1300340702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c011d00 con 0x7f22801537a0 2023-06-09T11:10:32.979 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 -- 172.21.15.117:0/1300340702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 msgr2=0x7f228006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 --2- 172.21.15.117:0/1300340702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f228006b1b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f227c009840 tx=0x7f227c009b50 comp rx=0 tx=0).stop 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 -- 172.21.15.117:0/1300340702 shutdown_connections 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 --2- 172.21.15.117:0/1300340702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f228006b1b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 -- 172.21.15.117:0/1300340702 >> 172.21.15.117:0/1300340702 conn(0x7f2280072040 msgr2=0x7f2280072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 -- 172.21.15.117:0/1300340702 shutdown_connections 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.971+0000 7f22877ff700 1 -- 172.21.15.117:0/1300340702 wait complete. 2023-06-09T11:10:32.980 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.973+0000 7f22877ff700 1 Processor -- start 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.973+0000 7f22877ff700 1 -- start start 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 0x7f228014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f2280144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228014e950 con 0x7f22801537a0 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228014b020 con 0x7f22800c64e0 2023-06-09T11:10:32.981 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f22877ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22801eea50 con 0x7f228006b740 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f228559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f2284d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 0x7f228014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f228559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50564/0 (socket says 172.21.15.117:50564) 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f228559b700 1 -- 172.21.15.117:0/1163877022 learned_addr learned my addr 172.21.15.117:0/1163877022 (peer_addr_for_me v2:172.21.15.117:0/0) 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.974+0000 7f2285d9c700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f2280144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:32.983 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 -- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 msgr2=0x7f228014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 0x7f228014dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 -- 172.21.15.117:0/1163877022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 msgr2=0x7f2280144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f2280144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 -- 172.21.15.117:0/1163877022 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c0094a0 con 0x7f228006b740 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f2284d9a700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 0x7f228014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.984 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f2285d9c700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f2280144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f228559b700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c004550 con 0x7f228006b740 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c01e070 con 0x7f228006b740 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0038e0 con 0x7f228006b740 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.975+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22800c49b0 con 0x7f228006b740 2023-06-09T11:10:32.985 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.976+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22800c46d0 con 0x7f228006b740 2023-06-09T11:10:32.986 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.976+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f227c003d40 con 0x7f228006b740 2023-06-09T11:10:32.986 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.976+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f227c01a420 con 0x7f228006b740 2023-06-09T11:10:32.986 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.977+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22800c4c90 con 0x7f228006b740 2023-06-09T11:10:32.986 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:32.980+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f227c020070 con 0x7f228006b740 2023-06-09T11:10:33.001 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:32.996+0000 7f7f2a848200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-09T11:10:33.073 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.068+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd new", "uuid": "97300c66-1e58-4029-91ed-e65bc4497fe3", "id": 7} v 0) v1 -- 0x7f2280074cf0 con 0x7f228006b740 2023-06-09T11:10:33.078 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.073+0000 7f2275ffb700 1 -- 172.21.15.117:0/1163877022 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "97300c66-1e58-4029-91ed-e65bc4497fe3", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f227c032070 con 0x7f228006b740 2023-06-09T11:10:33.079 INFO:teuthology.orchestra.run.smithi117.stdout:7 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 msgr2=0x7f22800cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f227c009fd0 tx=0x7f227c004750 comp rx=0 tx=0).stop 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 shutdown_connections 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22801537a0 0x7f2280144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f228006b740 0x7f22800cf800 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.080 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 --2- 172.21.15.117:0/1163877022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22800c64e0 0x7f228014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 >> 172.21.15.117:0/1163877022 conn(0x7f2280072040 msgr2=0x7f2280151050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 shutdown_connections 2023-06-09T11:10:33.081 INFO:teuthology.orchestra.run.smithi117.stderr:2023-06-09T11:10:33.074+0000 7f22877ff700 1 -- 172.21.15.117:0/1163877022 wait complete. 2023-06-09T11:10:33.099 INFO:tasks.ceph.osd.0:Restarting daemon 2023-06-09T11:10:33.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-06-09T11:10:33.103 INFO:tasks.ceph.osd.0:Started 2023-06-09T11:10:33.104 INFO:tasks.ceph.osd.1:Restarting daemon 2023-06-09T11:10:33.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-06-09T11:10:33.111 INFO:tasks.ceph.osd.1:Started 2023-06-09T11:10:33.111 INFO:tasks.ceph.osd.2:Restarting daemon 2023-06-09T11:10:33.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-06-09T11:10:33.117 INFO:tasks.ceph.osd.2:Started 2023-06-09T11:10:33.118 INFO:tasks.ceph.osd.3:Restarting daemon 2023-06-09T11:10:33.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-06-09T11:10:33.126 INFO:tasks.ceph.osd.3:Started 2023-06-09T11:10:33.126 INFO:tasks.ceph.osd.4:Restarting daemon 2023-06-09T11:10:33.127 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-06-09T11:10:33.133 INFO:tasks.ceph.osd.4:Started 2023-06-09T11:10:33.133 INFO:tasks.ceph.osd.5:Restarting daemon 2023-06-09T11:10:33.133 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-06-09T11:10:33.139 INFO:tasks.ceph.osd.5:Started 2023-06-09T11:10:33.140 INFO:tasks.ceph.osd.6:Restarting daemon 2023-06-09T11:10:33.140 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-06-09T11:10:33.145 INFO:tasks.ceph.osd.6:Started 2023-06-09T11:10:33.145 INFO:tasks.ceph.osd.7:Restarting daemon 2023-06-09T11:10:33.146 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-06-09T11:10:33.150 INFO:tasks.ceph.osd.7:Started 2023-06-09T11:10:33.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:33.213 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:33.207+0000 7f7f2a848200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-09T11:10:33.251 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:33.246+0000 7f7323c6a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-09T11:10:33.377 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:33.372+0000 7f7323c6a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-09T11:10:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.395+0000 7f39989ff700 1 Processor -- start 2023-06-09T11:10:33.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.396+0000 7f39989ff700 1 -- start start 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f39989ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3994153880 0x7f399415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f39989ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399415a500 con 0x7f39940c9f40 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f39989ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f399415da00 con 0x7f399415b4a0 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f39989ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940c5f10 con 0x7f3994153880 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f399259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3994153880 0x7f399415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:33.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f399259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3994153880 0x7f399415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44836/0 (socket says 172.21.15.7:44836) 2023-06-09T11:10:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f398bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39940c9f40 0x7f399415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50530/0 (socket says 172.21.15.7:50530) 2023-06-09T11:10:33.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.397+0000 7f399259c700 1 -- 172.21.15.7:0/1417826501 learned_addr learned my addr 172.21.15.7:0/1417826501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f399259c700 1 -- 172.21.15.7:0/1417826501 >> v1:172.21.15.117:6789/0 conn(0x7f399415b4a0 legacy=0x7f39940c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f3991d9b700 1 -- 172.21.15.7:0/1417826501 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367191740 0 0) 0x7f398c001490 con 0x7f399415b4a0 2023-06-09T11:10:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f399259c700 1 -- 172.21.15.7:0/1417826501 >> v1:172.21.15.7:6789/0 conn(0x7f39940c9f40 legacy=0x7f399415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f399259c700 1 -- 172.21.15.7:0/1417826501 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39941596e0 con 0x7f3994153880 2023-06-09T11:10:33.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f399259c700 1 --2- 172.21.15.7:0/1417826501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994153880 0x7f399415d4c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39840098f0 tx=0x7f3984009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2686aee6123de7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f3991d9b700 1 -- 172.21.15.7:0/1417826501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984004230 con 0x7f3994153880 2023-06-09T11:10:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f3991d9b700 1 -- 172.21.15.7:0/1417826501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984004390 con 0x7f3994153880 2023-06-09T11:10:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.398+0000 7f3991d9b700 1 -- 172.21.15.7:0/1417826501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39840036a0 con 0x7f3994153880 2023-06-09T11:10:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 -- 172.21.15.7:0/1417826501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994153880 msgr2=0x7f399415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 --2- 172.21.15.7:0/1417826501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994153880 0x7f399415d4c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f39840098f0 tx=0x7f3984009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 -- 172.21.15.7:0/1417826501 shutdown_connections 2023-06-09T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 --2- 172.21.15.7:0/1417826501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3994153880 0x7f399415d4c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 -- 172.21.15.7:0/1417826501 >> 172.21.15.7:0/1417826501 conn(0x7f39940737f0 msgr2=0x7f39940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 -- 172.21.15.7:0/1417826501 shutdown_connections 2023-06-09T11:10:33.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.399+0000 7f39989ff700 1 -- 172.21.15.7:0/1417826501 wait complete. 2023-06-09T11:10:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.401+0000 7f39989ff700 1 Processor -- start 2023-06-09T11:10:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.401+0000 7f39989ff700 1 -- start start 2023-06-09T11:10:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39940c9f40 0x7f3994158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994153880 0x7f39940cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:33.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39940cc5c0 con 0x7f399415b4a0 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399414aef0 con 0x7f3994153880 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f39989ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940cae80 con 0x7f39940c9f40 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f3992d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f3992d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44844/0 (socket says 172.21.15.7:44844) 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f398bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994153880 0x7f39940cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:33.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f3992d9d700 1 -- 172.21.15.7:0/3898808851 learned_addr learned my addr 172.21.15.7:0/3898808851 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.402+0000 7f399259c700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39940c9f40 0x7f3994158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 -- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39940c9f40 msgr2=0x7f3994158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39940c9f40 0x7f3994158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 -- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994153880 msgr2=0x7f39940cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994153880 0x7f39940cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 -- 172.21.15.7:0/3898808851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009550 con 0x7f399415b4a0 2023-06-09T11:10:33.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f3992d9d700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c760 tx=0x7f398c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c003e30 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c005070 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3994159c60 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c005400 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3994159980 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.403+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f398c01b400 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.404+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f398c01bd80 con 0x7f399415b4a0 2023-06-09T11:10:33.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.404+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3994159f40 con 0x7f399415b4a0 2023-06-09T11:10:33.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.407+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f399415da00 con 0x7f399415b4a0 2023-06-09T11:10:33.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.481+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3994074cf0 con 0x7f399415b4a0 2023-06-09T11:10:33.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.481+0000 7f398affd700 1 -- 172.21.15.7:0/3898808851 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f398c02a090 con 0x7f399415b4a0 2023-06-09T11:10:33.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:33.488 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:33.070734+0000","last_up_change":"0.000000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 msgr2=0x7f39940cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f398c00c760 tx=0x7f398c00ca70 comp rx=0 tx=0).stop 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 shutdown_connections 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f399415b4a0 0x7f39940cda20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39940c9f40 0x7f3994158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 --2- 172.21.15.7:0/3898808851 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3994153880 0x7f39940cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.482+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 >> 172.21.15.7:0/3898808851 conn(0x7f39940737f0 msgr2=0x7f399415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.483+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 shutdown_connections 2023-06-09T11:10:33.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:33.483+0000 7f39989ff700 1 -- 172.21.15.7:0/3898808851 wait complete. 2023-06-09T11:10:33.506 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-06-09T11:10:33.506 INFO:tasks.ceph:Waiting for OSDs to come up 2023-06-09T11:10:33.553 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:33.548+0000 7f7323c6a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-09T11:10:33.566 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:33.560+0000 7f7f2a848200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-09T11:10:33.664 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:33.658+0000 7f7f2a848200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-09T11:10:33.819 INFO:tasks.ceph.osd.4.smithi117.stderr:2023-06-09T11:10:33.813+0000 7fbed5905640 -1 Falling back to public interface 2023-06-09T11:10:33.821 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-06-09T11:10:33.816+0000 7fed66dd5640 -1 Falling back to public interface 2023-06-09T11:10:33.838 INFO:tasks.ceph.osd.6.smithi117.stderr:2023-06-09T11:10:33.833+0000 7fc893766640 -1 Falling back to public interface 2023-06-09T11:10:33.847 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-06-09T11:10:33.841+0000 7ff235c59640 -1 Falling back to public interface 2023-06-09T11:10:33.850 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:33.844+0000 7f7323c6a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-09T11:10:33.858 INFO:tasks.ceph.osd.7.smithi117.stderr:2023-06-09T11:10:33.852+0000 7f5446593640 -1 Falling back to public interface 2023-06-09T11:10:33.859 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-06-09T11:10:33.853+0000 7f55b9cef640 -1 Falling back to public interface 2023-06-09T11:10:33.866 INFO:tasks.ceph.osd.5.smithi117.stderr:2023-06-09T11:10:33.861+0000 7f1bf7740640 -1 Falling back to public interface 2023-06-09T11:10:33.868 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-06-09T11:10:33.863+0000 7ff0d97f1640 -1 Falling back to public interface 2023-06-09T11:10:33.947 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:33.941+0000 7f7323c6a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-09T11:10:34.004 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:33.998+0000 7f7f2a848200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-09T11:10:34.283 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:34.277+0000 7f7323c6a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-09T11:10:34.291 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:34.285+0000 7f7f2a848200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-09T11:10:34.312 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:34.498 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.492+0000 7f163d7ff700 1 Processor -- start 2023-06-09T11:10:34.498 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.492+0000 7f163d7ff700 1 -- start start 2023-06-09T11:10:34.499 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.493+0000 7f163d7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 0x7f16380c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:34.499 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f163d7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163815a520 con 0x7f1638159530 2023-06-09T11:10:34.499 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f163d7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16381599c0 con 0x7f16380ca430 2023-06-09T11:10:34.499 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f163d7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1638159c80 con 0x7f1638153700 2023-06-09T11:10:34.500 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f16367fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1638159530 0x7f16380cbf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50534/0 (socket says 172.21.15.7:50534) 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f16367fc700 1 -- 172.21.15.7:0/464418926 learned_addr learned my addr 172.21.15.7:0/464418926 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1636ffd700 1 --2- 172.21.15.7:0/464418926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 0x7f16380c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3724709513 0 0) 0x7f163815a520 con 0x7f1638159530 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16200035b0 con 0x7f1638159530 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068274291 0 0) 0x7f16381599c0 con 0x7f16380ca430 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1636ffd700 1 -- 172.21.15.7:0/464418926 >> v1:172.21.15.117:6789/0 conn(0x7f16380ca430 legacy=0x7f163815d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.501 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1636ffd700 1 -- 172.21.15.7:0/464418926 >> v1:172.21.15.7:6789/0 conn(0x7f1638159530 legacy=0x7f16380cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.502 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.494+0000 7f1636ffd700 1 -- 172.21.15.7:0/464418926 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1638159f60 con 0x7f1638153700 2023-06-09T11:10:34.502 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.495+0000 7f1636ffd700 1 --2- 172.21.15.7:0/464418926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 0x7f16380c9ef0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f1628009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6eb9e2ab224b0e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:34.502 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.495+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011830 con 0x7f1638153700 2023-06-09T11:10:34.502 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.495+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1628011990 con 0x7f1638153700 2023-06-09T11:10:34.502 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.495+0000 7f1635ffb700 1 -- 172.21.15.7:0/464418926 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1628011d20 con 0x7f1638153700 2023-06-09T11:10:34.503 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 -- 172.21.15.7:0/464418926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 msgr2=0x7f16380c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.503 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 --2- 172.21.15.7:0/464418926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 0x7f16380c9ef0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f1628009b80 comp rx=0 tx=0).stop 2023-06-09T11:10:34.503 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 -- 172.21.15.7:0/464418926 shutdown_connections 2023-06-09T11:10:34.503 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 --2- 172.21.15.7:0/464418926 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638153700 0x7f16380c9ef0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.504 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 -- 172.21.15.7:0/464418926 >> 172.21.15.7:0/464418926 conn(0x7f163806a4d0 msgr2=0x7f163806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:34.504 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 -- 172.21.15.7:0/464418926 shutdown_connections 2023-06-09T11:10:34.504 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.496+0000 7f163d7ff700 1 -- 172.21.15.7:0/464418926 wait complete. 2023-06-09T11:10:34.504 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.497+0000 7f163d7ff700 1 Processor -- start 2023-06-09T11:10:34.504 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.498+0000 7f163d7ff700 1 -- start start 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1638153700 0x7f1638150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638159530 0x7f1638151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16381ef330 con 0x7f16380ca430 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16380c9950 con 0x7f1638153700 2023-06-09T11:10:34.506 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f163d7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16380cefc0 con 0x7f1638159530 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44860/0 (socket says 172.21.15.7:44860) 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 -- 172.21.15.7:0/3083087911 learned_addr learned my addr 172.21.15.7:0/3083087911 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f16367fc700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1638153700 0x7f1638150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f16377fe700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638159530 0x7f1638151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 -- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638159530 msgr2=0x7f1638151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.507 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638159530 0x7f1638151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 -- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1638153700 msgr2=0x7f1638150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1638153700 0x7f1638150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.499+0000 7f1636ffd700 1 -- 172.21.15.7:0/3083087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280094a0 con 0x7f16380ca430 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f1636ffd700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16280097c0 tx=0x7f1628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162801a9f0 con 0x7f16380ca430 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162801ab50 con 0x7f16380ca430 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16280243f0 con 0x7f16380ca430 2023-06-09T11:10:34.508 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163815cc30 con 0x7f16380ca430 2023-06-09T11:10:34.509 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163815a240 con 0x7f16380ca430 2023-06-09T11:10:34.509 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1628024850 con 0x7f16380ca430 2023-06-09T11:10:34.509 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.500+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f162802dd40 con 0x7f16380ca430 2023-06-09T11:10:34.509 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.501+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16380c5990 con 0x7f16380ca430 2023-06-09T11:10:34.509 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.504+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f1628032060 con 0x7f16380ca430 2023-06-09T11:10:34.512 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:34.507+0000 7f7f2a848200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-09T11:10:34.558 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:34.552+0000 7f7323c6a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-09T11:10:34.584 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.578+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16381ef750 con 0x7f16380ca430 2023-06-09T11:10:34.585 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.579+0000 7f161f7fe700 1 -- 172.21.15.7:0/3083087911 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1628015020 con 0x7f16380ca430 2023-06-09T11:10:34.586 INFO:teuthology.misc.health.smithi007.stdout: 2023-06-09T11:10:34.586 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:33.070734+0000","last_up_change":"0.000000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 msgr2=0x7f16380ce510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16280097c0 tx=0x7f1628004750 comp rx=0 tx=0).stop 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 shutdown_connections 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1638159530 0x7f1638151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16380ca430 0x7f16380ce510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 --2- 172.21.15.7:0/3083087911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1638153700 0x7f1638150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:34.588 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 >> 172.21.15.7:0/3083087911 conn(0x7f163806a4d0 msgr2=0x7f16380cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:34.588 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 shutdown_connections 2023-06-09T11:10:34.588 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:34.580+0000 7f163d7ff700 1 -- 172.21.15.7:0/3083087911 wait complete. 2023-06-09T11:10:34.601 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-06-09T11:10:34.663 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:34.658+0000 7f7f2a848200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-09T11:10:34.756 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:34.750+0000 7f7323c6a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-09T11:10:34.897 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:34.892+0000 7f7323c6a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-09T11:10:35.059 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:35.054+0000 7f7f2a848200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-09T11:10:35.189 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:35.183+0000 7f7f2a848200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-09T11:10:35.289 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:35.282+0000 7f7323c6a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-09T11:10:35.411 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:35.405+0000 7f7323c6a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-09T11:10:35.428 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:35.422+0000 7f7f2a848200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-09T11:10:35.642 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:35.636+0000 7f7323c6a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-09T11:10:36.176 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:36.171+0000 7f7f2a848200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-09T11:10:36.388 INFO:tasks.ceph.mgr.x.smithi117.stderr:2023-06-09T11:10:36.383+0000 7f7323c6a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-09T11:10:37.195 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:37.190+0000 7f7ede3fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-06-09T11:10:37.196 INFO:tasks.ceph.mgr.y.smithi007.stderr:2023-06-09T11:10:37.190+0000 7f7ede3fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-06-09T11:10:37.327 INFO:tasks.ceph.osd.4.smithi117.stderr:2023-06-09T11:10:37.322+0000 7fbed5905640 -1 osd.4 0 log_to_monitors true 2023-06-09T11:10:37.362 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-06-09T11:10:37.356+0000 7fed66dd5640 -1 osd.1 0 log_to_monitors true 2023-06-09T11:10:37.393 INFO:tasks.ceph.osd.6.smithi117.stderr:2023-06-09T11:10:37.388+0000 7fc893766640 -1 osd.6 0 log_to_monitors true 2023-06-09T11:10:37.425 INFO:tasks.ceph.osd.7.smithi117.stderr:2023-06-09T11:10:37.418+0000 7f5446593640 -1 osd.7 0 log_to_monitors true 2023-06-09T11:10:37.439 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-06-09T11:10:37.433+0000 7f55b9cef640 -1 osd.2 0 log_to_monitors true 2023-06-09T11:10:37.444 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-06-09T11:10:37.437+0000 7ff235c59640 -1 osd.0 0 log_to_monitors true 2023-06-09T11:10:37.458 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-06-09T11:10:37.452+0000 7ff0d97f1640 -1 osd.3 0 log_to_monitors true 2023-06-09T11:10:37.476 INFO:tasks.ceph.osd.5.smithi117.stderr:2023-06-09T11:10:37.471+0000 7f1bf7740640 -1 osd.5 0 log_to_monitors true 2023-06-09T11:10:39.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:39.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:10:40.526 INFO:tasks.ceph.osd.6.smithi117.stderr:2023-06-09T11:10:40.520+0000 7fc8825f7700 -1 osd.6 0 waiting for initial osdmap 2023-06-09T11:10:40.540 INFO:tasks.ceph.osd.6.smithi117.stderr:2023-06-09T11:10:40.534+0000 7fc879dfa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.541 INFO:tasks.ceph.osd.5.smithi117.stderr:2023-06-09T11:10:40.535+0000 7f1be65f7700 -1 osd.5 0 waiting for initial osdmap 2023-06-09T11:10:40.548 INFO:tasks.ceph.osd.7.smithi117.stderr:2023-06-09T11:10:40.543+0000 7f54353f7700 -1 osd.7 0 waiting for initial osdmap 2023-06-09T11:10:40.552 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-06-09T11:10:40.548+0000 7ff224bf7700 -1 osd.0 0 waiting for initial osdmap 2023-06-09T11:10:40.557 INFO:tasks.ceph.osd.5.smithi117.stderr:2023-06-09T11:10:40.552+0000 7f1bdddfa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.565 INFO:tasks.ceph.osd.7.smithi117.stderr:2023-06-09T11:10:40.560+0000 7f542cbfa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.567 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-06-09T11:10:40.561+0000 7fed55bf7700 -1 osd.1 0 waiting for initial osdmap 2023-06-09T11:10:40.572 INFO:tasks.ceph.osd.4.smithi117.stderr:2023-06-09T11:10:40.566+0000 7fbec47f7700 -1 osd.4 0 waiting for initial osdmap 2023-06-09T11:10:40.584 INFO:tasks.ceph.osd.0.smithi007.stderr:2023-06-09T11:10:40.579+0000 7ff21c3fa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.587 INFO:tasks.ceph.osd.4.smithi117.stderr:2023-06-09T11:10:40.582+0000 7fbebbffa700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.597 INFO:tasks.ceph.osd.1.smithi007.stderr:2023-06-09T11:10:40.592+0000 7fed4d3fa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:40.674 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-06-09T11:10:40.669+0000 7f55a8bf7700 -1 osd.2 0 waiting for initial osdmap 2023-06-09T11:10:40.703 INFO:tasks.ceph.osd.2.smithi007.stderr:2023-06-09T11:10:40.698+0000 7f55a03fa700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:41.259 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-06-09T11:10:41.252+0000 7ff0c85f7700 -1 osd.3 0 waiting for initial osdmap 2023-06-09T11:10:41.271 INFO:tasks.ceph.osd.3.smithi007.stderr:2023-06-09T11:10:41.266+0000 7ff0bfdfa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-09T11:10:41.406 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:41.587 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.582+0000 7fb196dff700 1 Processor -- start 2023-06-09T11:10:41.588 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.583+0000 7fb196dff700 1 -- start start 2023-06-09T11:10:41.588 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb196dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb190153810 0x7fb1900c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.589 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb196dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.589 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb196dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb19006a600 con 0x7fb1900c58b0 2023-06-09T11:10:41.589 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb196dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1900c6200 con 0x7fb190153810 2023-06-09T11:10:41.589 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb196dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb19015cc20 con 0x7fb190153be0 2023-06-09T11:10:41.591 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb18ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.591 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.584+0000 7fb194b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb190153810 0x7fb1900c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.591 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb19539c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fb1900c58b0 0x7fb19006aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:36344/0 (socket says 172.21.15.7:36344) 2023-06-09T11:10:41.592 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb18ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:52326/0 (socket says 172.21.15.7:52326) 2023-06-09T11:10:41.592 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb19539c700 1 -- 172.21.15.7:0/2169254520 learned_addr learned my addr 172.21.15.7:0/2169254520 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:41.593 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb194b9b700 1 -- 172.21.15.7:0/2169254520 >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 msgr2=0x7fb1900cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:41.593 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb194b9b700 1 --2- 172.21.15.7:0/2169254520 >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.593 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb194b9b700 1 -- 172.21.15.7:0/2169254520 >> v1:172.21.15.117:6789/0 conn(0x7fb1900c58b0 legacy=0x7fb19006aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.593 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb194b9b700 1 -- 172.21.15.7:0/2169254520 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19015cec0 con 0x7fb190153810 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb18ffff700 1 --2- 172.21.15.7:0/2169254520 >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb18f7fe700 1 -- 172.21.15.7:0/2169254520 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374714112 0 0) 0x7fb19006a600 con 0x7fb1900c58b0 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb194b9b700 1 --2- 172.21.15.7:0/2169254520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190153810 0x7fb1900c9f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb19008d570 tx=0x7fb184009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f221a77d36732ba2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb18f7fe700 1 -- 172.21.15.7:0/2169254520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011830 con 0x7fb190153810 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.585+0000 7fb18f7fe700 1 -- 172.21.15.7:0/2169254520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb184011990 con 0x7fb190153810 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb18f7fe700 1 -- 172.21.15.7:0/2169254520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb184011d20 con 0x7fb190153810 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 -- 172.21.15.7:0/2169254520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190153810 msgr2=0x7fb1900c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.594 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 --2- 172.21.15.7:0/2169254520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190153810 0x7fb1900c9f40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb19008d570 tx=0x7fb184009b20 comp rx=0 tx=0).stop 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 -- 172.21.15.7:0/2169254520 shutdown_connections 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 --2- 172.21.15.7:0/2169254520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb190153810 0x7fb1900c9f40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 --2- 172.21.15.7:0/2169254520 >> v2:172.21.15.117:3300/0 conn(0x7fb190153be0 0x7fb1900cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 -- 172.21.15.7:0/2169254520 >> 172.21.15.7:0/2169254520 conn(0x7fb1900737f0 msgr2=0x7fb190073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 -- 172.21.15.7:0/2169254520 shutdown_connections 2023-06-09T11:10:41.595 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.586+0000 7fb196dff700 1 -- 172.21.15.7:0/2169254520 wait complete. 2023-06-09T11:10:41.597 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.588+0000 7fb196dff700 1 Processor -- start 2023-06-09T11:10:41.597 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.588+0000 7fb196dff700 1 -- start start 2023-06-09T11:10:41.597 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1900c58b0 0x7fb1901517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb190153810 0x7fb1901ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb19014cb00 con 0x7fb1900cd4c0 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901f2f50 con 0x7fb1900c58b0 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb196dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1901f3210 con 0x7fb190153810 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb19539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.598 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb19539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44966/0 (socket says 172.21.15.7:44966) 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb19539c700 1 -- 172.21.15.7:0/2568333774 learned_addr learned my addr 172.21.15.7:0/2568333774 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb18ffff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb190153810 0x7fb1901ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.589+0000 7fb194b9b700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1900c58b0 0x7fb1901517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 -- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb190153810 msgr2=0x7fb1901ef1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb190153810 0x7fb1901ef1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 -- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1900c58b0 msgr2=0x7fb1901517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1900c58b0 0x7fb1901517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.599 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 -- 172.21.15.7:0/2568333774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0094a0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb19539c700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb188002230 tx=0x7fb18800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18800aac0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18800ac40 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1880070d0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1840094a0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.590+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb19015d460 con 0x7fb1900cd4c0 2023-06-09T11:10:41.600 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.592+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb19015d180 con 0x7fb1900cd4c0 2023-06-09T11:10:41.601 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.592+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7fb188007250 con 0x7fb1900cd4c0 2023-06-09T11:10:41.601 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.594+0000 7fb18d7fa700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 0x7fb17406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:41.601 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.594+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fb1880887b0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.602 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.595+0000 7fb194b9b700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 0x7fb17406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:41.602 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.595+0000 7fb194b9b700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 0x7fb17406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb184008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:41.603 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.599+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb188053ed0 con 0x7fb1900cd4c0 2023-06-09T11:10:41.810 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.806+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb19015d740 con 0x7fb1900cd4c0 2023-06-09T11:10:41.811 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.806+0000 7fb18d7fa700 1 -- 172.21.15.7:0/2568333774 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10812 (secure 0 0 0) 0x7fb188018090 con 0x7fb1900cd4c0 2023-06-09T11:10:41.811 INFO:teuthology.misc.health.smithi007.stdout: 2023-06-09T11:10:41.811 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":13,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:41.542612+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:41.814 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 msgr2=0x7fb17406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.814 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 0x7fb17406d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb184008030 comp rx=0 tx=0).stop 2023-06-09T11:10:41.814 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 msgr2=0x7fb1901ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:41.814 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb188002230 tx=0x7fb18800cf50 comp rx=0 tx=0).stop 2023-06-09T11:10:41.814 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 shutdown_connections 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1900cd4c0 0x7fb1901ef6e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb1900c58b0 0x7fb1901517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb17406ac50 0x7fb17406d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 --2- 172.21.15.7:0/2568333774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb190153810 0x7fb1901ef1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 >> 172.21.15.7:0/2568333774 conn(0x7fb1900737f0 msgr2=0x7fb1900cabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 shutdown_connections 2023-06-09T11:10:41.815 INFO:teuthology.misc.health.smithi007.stderr:2023-06-09T11:10:41.809+0000 7fb196dff700 1 -- 172.21.15.7:0/2568333774 wait complete. 2023-06-09T11:10:41.829 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-06-09T11:10:41.829 INFO:tasks.ceph:Creating RBD pool 2023-06-09T11:10:41.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2023-06-09T11:10:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.017+0000 7f41879ff700 1 Processor -- start 2023-06-09T11:10:42.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.017+0000 7f41879ff700 1 -- start start 2023-06-09T11:10:42.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.018+0000 7f41879ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f41801537a0 0x7f418006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f41879ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800c6220 con 0x7f418006b740 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f41879ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41800c4480 con 0x7f41800c64e0 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f41879ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41800c4f70 con 0x7f41801537a0 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f4185f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f418006b740 0x7f418015b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50538/0 (socket says 172.21.15.7:50538) 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f4185f9c700 1 -- 172.21.15.7:0/3865218466 learned_addr learned my addr 172.21.15.7:0/3865218466 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f418579b700 1 --2- 172.21.15.7:0/3865218466 >> v2:172.21.15.117:3300/0 conn(0x7f41801537a0 0x7f418006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486297520 0 0) 0x7f4178001490 con 0x7f418006b740 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.019+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f416c0035b0 con 0x7f418006b740 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 68265919 0 0) 0x7f416c0035b0 con 0x7f418006b740 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 >> v2:172.21.15.117:3300/0 conn(0x7f41801537a0 msgr2=0x7f418006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 --2- 172.21.15.7:0/3865218466 >> v2:172.21.15.117:3300/0 conn(0x7f41801537a0 0x7f418006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 >> v1:172.21.15.117:6789/0 conn(0x7f41800c64e0 legacy=0x7f41800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418015ae20 con 0x7f418006b740 2023-06-09T11:10:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f4178001b80 con 0x7f418006b740 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 >> v1:172.21.15.7:6789/0 conn(0x7f418006b740 legacy=0x7f418015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c005200 0x7f416c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c4f70 con 0x7f416c003a00 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c00c060 con 0x7f416c005200 2023-06-09T11:10:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c00c220 con 0x7f416c004850 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4184f9a700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4185f9c700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c005200 0x7f416c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 -- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c004850 msgr2=0x7f416c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c004850 0x7f416c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 -- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c005200 msgr2=0x7f416c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c005200 0x7f416c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f418579b700 1 -- 172.21.15.7:0/3865218466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0035b0 con 0x7f416c003a00 2023-06-09T11:10:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4185f9c700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f416c005200 0x7f416c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.020+0000 7f4185f9c700 1 -- 172.21.15.7:0/3865218466 reap_dead start 2023-06-09T11:10:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f418579b700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 0x7f416c003e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f417c009b60 tx=0x7f416c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192c28acdf94a47b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00e6c0 con 0x7f416c003a00 2023-06-09T11:10:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c00e820 con 0x7f416c003a00 2023-06-09T11:10:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f4177fff700 1 -- 172.21.15.7:0/3865218466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c00eb90 con 0x7f416c003a00 2023-06-09T11:10:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f41879ff700 1 -- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 msgr2=0x7f416c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.021+0000 7f41879ff700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 0x7f416c003e10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f417c009b60 tx=0x7f416c005880 comp rx=0 tx=0).stop 2023-06-09T11:10:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.022+0000 7f41879ff700 1 -- 172.21.15.7:0/3865218466 shutdown_connections 2023-06-09T11:10:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.022+0000 7f41879ff700 1 --2- 172.21.15.7:0/3865218466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f416c003a00 0x7f416c003e10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.022+0000 7f41879ff700 1 -- 172.21.15.7:0/3865218466 >> 172.21.15.7:0/3865218466 conn(0x7f4180072040 msgr2=0x7f4180072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.022+0000 7f41879ff700 1 -- 172.21.15.7:0/3865218466 shutdown_connections 2023-06-09T11:10:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.022+0000 7f41879ff700 1 -- 172.21.15.7:0/3865218466 wait complete. 2023-06-09T11:10:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.023+0000 7f41879ff700 1 Processor -- start 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.024+0000 7f41879ff700 1 -- start start 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.024+0000 7f41879ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c003a00 0x7f41801f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.024+0000 7f41879ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.024+0000 7f41879ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41801468a0 0x7f4180146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f41879ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41800c9290 con 0x7f41800ca2c0 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f41879ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4180147a90 con 0x7f41801468a0 2023-06-09T11:10:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f41879ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41801f30c0 con 0x7f416c003a00 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44984/0 (socket says 172.21.15.7:44984) 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 -- 172.21.15.7:0/2864100649 learned_addr learned my addr 172.21.15.7:0/2864100649 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f418579b700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c003a00 0x7f41801f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4185f9c700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41801468a0 0x7f4180146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 -- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c003a00 msgr2=0x7f41801f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c003a00 0x7f41801f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 -- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41801468a0 msgr2=0x7f4180146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41801468a0 0x7f4180146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 -- 172.21.15.7:0/2864100649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41800c49b0 con 0x7f41800ca2c0 2023-06-09T11:10:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.025+0000 7f4184f9a700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f417000e550 tx=0x7f417000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.026+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700052e0 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.026+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170005440 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.026+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41800c46d0 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.026+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41700038a0 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.026+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41800c4c90 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.027+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41800c9440 con 0x7f41800ca2c0 2023-06-09T11:10:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.028+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f4170003a00 con 0x7f41800ca2c0 2023-06-09T11:10:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.029+0000 7f4175ffb700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 0x7f416406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.029+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f4170088290 con 0x7f41800ca2c0 2023-06-09T11:10:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.029+0000 7f418579b700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 0x7f416406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.030+0000 7f418579b700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 0x7f416406d130 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c0090b0 tx=0x7f417c01e040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.034+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f41800c9440 con 0x7f41800ca2c0 2023-06-09T11:10:42.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.244+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f41800c9440 con 0x7f41800ca2c0 2023-06-09T11:10:42.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.548+0000 7f4175ffb700 1 -- 172.21.15.7:0/2864100649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7f41800c9440 con 0x7f41800ca2c0 2023-06-09T11:10:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.551+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 msgr2=0x7f416406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.551+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 0x7f416406d130 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c0090b0 tx=0x7f417c01e040 comp rx=0 tx=0).stop 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.551+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 msgr2=0x7f41801ecd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.551+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f417000e550 tx=0x7f417000e860 comp rx=0 tx=0).stop 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 shutdown_connections 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41801468a0 0x7f4180146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f416406ac80 0x7f416406d130 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f416c003a00 0x7f41801f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 --2- 172.21.15.7:0/2864100649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41800ca2c0 0x7f41801ecd40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 >> 172.21.15.7:0/2864100649 conn(0x7f4180072040 msgr2=0x7f418014f880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 shutdown_connections 2023-06-09T11:10:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.552+0000 7f41879ff700 1 -- 172.21.15.7:0/2864100649 wait complete. 2023-06-09T11:10:42.558 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2023-06-09T11:10:42.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-06-09T11:10:42.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.766+0000 7f8677bff700 1 Processor -- start 2023-06-09T11:10:42.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.766+0000 7f8677bff700 1 -- start start 2023-06-09T11:10:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.767+0000 7f8677bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700cc100 0x7f867015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.767+0000 7f8677bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86700cc4d0 0x7f86700cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.767+0000 7f8677bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867015dd00 con 0x7f86700b5270 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.767+0000 7f8677bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8670071790 con 0x7f86700cc4d0 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.767+0000 7f8677bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86700ca3f0 con 0x7f86700cc100 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86700cc4d0 0x7f86700cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f86700cc4d0 0x7f86700cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44996/0 (socket says 172.21.15.7:44996) 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 -- 172.21.15.7:0/2391598489 learned_addr learned my addr 172.21.15.7:0/2391598489 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:42.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867599b700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700cc100 0x7f867015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 -- 172.21.15.7:0/2391598489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700cc100 msgr2=0x7f867015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:42.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700cc100 0x7f867015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 -- 172.21.15.7:0/2391598489 >> v1:172.21.15.117:6789/0 conn(0x7f86700b5270 legacy=0x7f86700c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 -- 172.21.15.7:0/2391598489 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670159840 con 0x7f86700cc4d0 2023-06-09T11:10:42.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f867519a700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc4d0 0x7f86700cac90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86600098f0 tx=0x7f8660009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504fbd3c84e401f7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.768+0000 7f8674999700 1 -- 172.21.15.7:0/2391598489 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011830 con 0x7f86700cc4d0 2023-06-09T11:10:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8674999700 1 -- 172.21.15.7:0/2391598489 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8660011990 con 0x7f86700cc4d0 2023-06-09T11:10:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8674999700 1 -- 172.21.15.7:0/2391598489 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660011d20 con 0x7f86700cc4d0 2023-06-09T11:10:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 -- 172.21.15.7:0/2391598489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc4d0 msgr2=0x7f86700cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc4d0 0x7f86700cac90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86600098f0 tx=0x7f8660009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:42.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 -- 172.21.15.7:0/2391598489 shutdown_connections 2023-06-09T11:10:42.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700cc100 0x7f867015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 --2- 172.21.15.7:0/2391598489 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc4d0 0x7f86700cac90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 -- 172.21.15.7:0/2391598489 >> 172.21.15.7:0/2391598489 conn(0x7f8670070af0 msgr2=0x7f8670070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:42.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 -- 172.21.15.7:0/2391598489 shutdown_connections 2023-06-09T11:10:42.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.769+0000 7f8677bff700 1 -- 172.21.15.7:0/2391598489 wait complete. 2023-06-09T11:10:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.771+0000 7f8677bff700 1 Processor -- start 2023-06-09T11:10:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.771+0000 7f8677bff700 1 -- start start 2023-06-09T11:10:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700b5270 0x7f86701f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86700cc4d0 0x7f86700cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86701f2b50 con 0x7f86700cc100 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86700c8770 con 0x7f86700cc4d0 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f8677bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86701523a0 con 0x7f86700b5270 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f867519a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f867519a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45006/0 (socket says 172.21.15.7:45006) 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.772+0000 7f867519a700 1 -- 172.21.15.7:0/3763066693 learned_addr learned my addr 172.21.15.7:0/3763066693 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867599b700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700b5270 0x7f86701f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867619c700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86700cc4d0 0x7f86700cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 -- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700b5270 msgr2=0x7f86701f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700b5270 0x7f86701f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 -- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86700cc4d0 msgr2=0x7f86700cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86700cc4d0 0x7f86700cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 -- 172.21.15.7:0/3763066693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8660009550 con 0x7f86700cc100 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f867519a700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f8660004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660004570 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866001f070 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86600038c0 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86700ca6b0 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.773+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86700ca120 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.775+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8670159b00 con 0x7f86700cc100 2023-06-09T11:10:42.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.776+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8660021030 con 0x7f86700cc100 2023-06-09T11:10:42.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.777+0000 7f86667fc700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 0x7f865406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:42.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.777+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f866008a520 con 0x7f86700cc100 2023-06-09T11:10:42.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.777+0000 7f867599b700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 0x7f865406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:42.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.778+0000 7f867599b700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 0x7f865406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f866c009b70 tx=0x7f866c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:42.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.782+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8660055a00 con 0x7f86700cc100 2023-06-09T11:10:42.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:42.993+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86701f2d00 con 0x7f86700cc100 2023-06-09T11:10:43.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.551+0000 7f86667fc700 1 -- 172.21.15.7:0/3763066693 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8660059020 con 0x7f86700cc100 2023-06-09T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.556+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 msgr2=0x7f865406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.556+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 0x7f865406d100 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f866c009b70 tx=0x7f866c008040 comp rx=0 tx=0).stop 2023-06-09T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.556+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 msgr2=0x7f86700c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:43.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.556+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8660000c00 tx=0x7f8660004750 comp rx=0 tx=0).stop 2023-06-09T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 shutdown_connections 2023-06-09T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86700cc4d0 0x7f86700cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f865406ac50 0x7f865406d100 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86700b5270 0x7f86701f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 --2- 172.21.15.7:0/3763066693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86700cc100 0x7f86700c7d30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 >> 172.21.15.7:0/3763066693 conn(0x7f8670070af0 msgr2=0x7f867015d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 shutdown_connections 2023-06-09T11:10:43.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.557+0000 7f8677bff700 1 -- 172.21.15.7:0/3763066693 wait complete. 2023-06-09T11:10:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:enabled application 'rbd' on pool 'rbd' 2023-06-09T11:10:43.583 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-06-09T11:10:43.584 INFO:tasks.ceph.mds.a:Restarting daemon 2023-06-09T11:10:43.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-06-09T11:10:43.589 INFO:tasks.ceph.mds.a:Started 2023-06-09T11:10:43.590 INFO:tasks.ceph.mds.c:Restarting daemon 2023-06-09T11:10:43.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-06-09T11:10:43.596 INFO:tasks.ceph.mds.c:Started 2023-06-09T11:10:43.597 INFO:tasks.ceph.mds.e:Restarting daemon 2023-06-09T11:10:43.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-06-09T11:10:43.605 INFO:tasks.ceph.mds.e:Started 2023-06-09T11:10:43.607 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:10:43.608 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:10:43.613 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:10:43.614 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:10:43.615 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:10:43.620 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:10:43.620 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:10:43.620 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:10:43.624 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:10:43.625 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-06-09T11:10:43.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:43.756 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2023-06-09T11:10:43.757 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2023-06-09T11:10:43.762 INFO:tasks.ceph.mds.e.smithi007.stdout:starting mds.e at 2023-06-09T11:10:43.768 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:10:43.779 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:10:43.783 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:10:43.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.850+0000 7f26fc9ff700 1 Processor -- start 2023-06-09T11:10:43.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.850+0000 7f26fc9ff700 1 -- start start 2023-06-09T11:10:43.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26f8153880 0x7f26f815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f80c9f40 0x7f26f815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26f815b2d0 0x7f26f80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f80c5f10 con 0x7f26f815b2d0 2023-06-09T11:10:43.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f815a500 con 0x7f26f8153880 2023-06-09T11:10:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26fc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f815da00 con 0x7f26f80c9f40 2023-06-09T11:10:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26f815b2d0 0x7f26f80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f26f815b2d0 0x7f26f80c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34028/0 (socket says 172.21.15.7:34028) 2023-06-09T11:10:43.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 -- 172.21.15.7:0/1787372584 learned_addr learned my addr 172.21.15.7:0/1787372584 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:43.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f659c700 1 --2- 172.21.15.7:0/1787372584 >> v2:172.21.15.117:3300/0 conn(0x7f26f8153880 0x7f26f815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f5d9b700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f80c9f40 0x7f26f815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 -- 172.21.15.7:0/1787372584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f80c9f40 msgr2=0x7f26f815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:43.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f80c9f40 0x7f26f815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 -- 172.21.15.7:0/1787372584 >> v2:172.21.15.117:3300/0 conn(0x7f26f8153880 msgr2=0x7f26f815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 --2- 172.21.15.7:0/1787372584 >> v2:172.21.15.117:3300/0 conn(0x7f26f8153880 0x7f26f815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.852+0000 7f26f6d9d700 1 -- 172.21.15.7:0/1787372584 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f815b750 con 0x7f26f815b2d0 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26f6d9d700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f815b2d0 0x7f26f80c4e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f26f808d340 tx=0x7f26f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2eabf4277d47be server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26f559a700 1 -- 172.21.15.7:0/1787372584 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000e070 con 0x7f26f815b2d0 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26f559a700 1 -- 172.21.15.7:0/1787372584 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f000b910 con 0x7f26f815b2d0 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26f559a700 1 -- 172.21.15.7:0/1787372584 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f000bcc0 con 0x7f26f815b2d0 2023-06-09T11:10:43.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 -- 172.21.15.7:0/1787372584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f815b2d0 msgr2=0x7f26f80c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f815b2d0 0x7f26f80c4e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f26f808d340 tx=0x7f26f000b330 comp rx=0 tx=0).stop 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 -- 172.21.15.7:0/1787372584 shutdown_connections 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/1787372584 >> v2:172.21.15.117:3300/0 conn(0x7f26f8153880 0x7f26f815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f815b2d0 0x7f26f80c4e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.853+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/1787372584 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f80c9f40 0x7f26f815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.854+0000 7f26fc9ff700 1 -- 172.21.15.7:0/1787372584 >> 172.21.15.7:0/1787372584 conn(0x7f26f80737f0 msgr2=0x7f26f80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:43.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.854+0000 7f26fc9ff700 1 -- 172.21.15.7:0/1787372584 shutdown_connections 2023-06-09T11:10:43.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.854+0000 7f26fc9ff700 1 -- 172.21.15.7:0/1787372584 wait complete. 2023-06-09T11:10:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.855+0000 7f26fc9ff700 1 Processor -- start 2023-06-09T11:10:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.855+0000 7f26fc9ff700 1 -- start start 2023-06-09T11:10:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f8153880 0x7f26f8150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f815b2d0 0x7f26f80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f8152250 con 0x7f26f80c9f40 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81eee40 con 0x7f26f815b2d0 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26fc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81efe10 con 0x7f26f8153880 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26f659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.856+0000 7f26f659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34038/0 (socket says 172.21.15.7:34038) 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 -- 172.21.15.7:0/3039636824 learned_addr learned my addr 172.21.15.7:0/3039636824 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f5d9b700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f8153880 0x7f26f8150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f6d9d700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f815b2d0 0x7f26f80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 -- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f8153880 msgr2=0x7f26f8150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f8153880 0x7f26f8150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 -- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f815b2d0 msgr2=0x7f26f80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f815b2d0 0x7f26f80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:43.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 -- 172.21.15.7:0/3039636824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009b30 con 0x7f26f80c9f40 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.857+0000 7f26f659c700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26e800e9b0 tx=0x7f26e800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.858+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8009710 con 0x7f26f80c9f40 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.858+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e800ee50 con 0x7f26f80c9f40 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.858+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f8159980 con 0x7f26f80c9f40 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.858+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e800ca50 con 0x7f26f80c9f40 2023-06-09T11:10:43.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.858+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f81596e0 con 0x7f26f80c9f40 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.859+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8159c60 con 0x7f26f80c9f40 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.860+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f26e800cbb0 con 0x7f26f80c9f40 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.862+0000 7f26e6ffd700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 0x7f26d806d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.862+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f26e8089130 con 0x7f26f80c9f40 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.863+0000 7f26f5d9b700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 0x7f26d806d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:43.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.863+0000 7f26f5d9b700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 0x7f26d806d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26f80ccb70 tx=0x7f26ec009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:43.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:43.868+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26e80544e0 con 0x7f26f80c9f40 2023-06-09T11:10:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.086+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f8074cf0 con 0x7f26f80c9f40 2023-06-09T11:10:44.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.087+0000 7f26e6ffd700 1 -- 172.21.15.7:0/3039636824 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+14566 (secure 0 0 0) 0x7f26e8017090 con 0x7f26f80c9f40 2023-06-09T11:10:44.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:44.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:43.546720+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:44.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 msgr2=0x7f26d806d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 0x7f26d806d100 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f26f80ccb70 tx=0x7f26ec009210 comp rx=0 tx=0).stop 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 msgr2=0x7f26f80cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f26e800e9b0 tx=0x7f26e800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 shutdown_connections 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26f815b2d0 0x7f26f80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26f80c9f40 0x7f26f80cf830 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f26d806ac50 0x7f26d806d100 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 --2- 172.21.15.7:0/3039636824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26f8153880 0x7f26f8150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.089+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 >> 172.21.15.7:0/3039636824 conn(0x7f26f80737f0 msgr2=0x7f26f8150780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.090+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 shutdown_connections 2023-06-09T11:10:44.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.090+0000 7f26fc9ff700 1 -- 172.21.15.7:0/3039636824 wait complete. 2023-06-09T11:10:44.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:10:44.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:10:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.318+0000 7f638d3ff700 1 Processor -- start 2023-06-09T11:10:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.318+0000 7f638d3ff700 1 -- start start 2023-06-09T11:10:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.319+0000 7f638d3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f638815b9c0 0x7f638806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f638d3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880c6220 con 0x7f63881537a0 2023-06-09T11:10:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f638d3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63880c4480 con 0x7f63880c64e0 2023-06-09T11:10:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f638d3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63880c4f70 con 0x7f638815b9c0 2023-06-09T11:10:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f638815b9c0 0x7f638806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f638815b9c0 0x7f638806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34068/0 (socket says 172.21.15.7:34068) 2023-06-09T11:10:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 -- 172.21.15.7:0/3945624961 learned_addr learned my addr 172.21.15.7:0/3945624961 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 -- 172.21.15.7:0/3945624961 >> v1:172.21.15.117:6789/0 conn(0x7f63880c64e0 legacy=0x7f638815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 -- 172.21.15.7:0/3945624961 >> v1:172.21.15.7:6789/0 conn(0x7f63881537a0 legacy=0x7f63880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.320+0000 7f63877fe700 1 -- 172.21.15.7:0/3945624961 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638815ae20 con 0x7f638815b9c0 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.321+0000 7f63877fe700 1 --2- 172.21.15.7:0/3945624961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638815b9c0 0x7f638806b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6380009e50 tx=0x7f638000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803d07edccb5223c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.321+0000 7f6385ffb700 1 -- 172.21.15.7:0/3945624961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000e070 con 0x7f638815b9c0 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.321+0000 7f6385ffb700 1 -- 172.21.15.7:0/3945624961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638000b9c0 con 0x7f638815b9c0 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.321+0000 7f6385ffb700 1 -- 172.21.15.7:0/3945624961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638000bd70 con 0x7f638815b9c0 2023-06-09T11:10:44.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 -- 172.21.15.7:0/3945624961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638815b9c0 msgr2=0x7f638806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 --2- 172.21.15.7:0/3945624961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638815b9c0 0x7f638806b1b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6380009e50 tx=0x7f638000b330 comp rx=0 tx=0).stop 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 -- 172.21.15.7:0/3945624961 shutdown_connections 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 --2- 172.21.15.7:0/3945624961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f638815b9c0 0x7f638806b1b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 -- 172.21.15.7:0/3945624961 >> 172.21.15.7:0/3945624961 conn(0x7f6388072040 msgr2=0x7f6388072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 -- 172.21.15.7:0/3945624961 shutdown_connections 2023-06-09T11:10:44.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.322+0000 7f638d3ff700 1 -- 172.21.15.7:0/3945624961 wait complete. 2023-06-09T11:10:44.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.325+0000 7f638d3ff700 1 Processor -- start 2023-06-09T11:10:44.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.325+0000 7f638d3ff700 1 -- start start 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63881537a0 0x7f63881e49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f638815b9c0 0x7f63880ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63881e38d0 con 0x7f63880c64e0 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63880cddc0 con 0x7f63881537a0 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f638d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63880cc900 con 0x7f638815b9c0 2023-06-09T11:10:44.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34074/0 (socket says 172.21.15.7:34074) 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 -- 172.21.15.7:0/1884903531 learned_addr learned my addr 172.21.15.7:0/1884903531 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f63867fc700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63881537a0 0x7f63881e49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f63877fe700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f638815b9c0 0x7f63880ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 -- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f638815b9c0 msgr2=0x7f63880ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f638815b9c0 0x7f63880ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 -- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63881537a0 msgr2=0x7f63881e49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63881537a0 0x7f63881e49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.326+0000 7f6386ffd700 1 -- 172.21.15.7:0/1884903531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6380009b30 con 0x7f63880c64e0 2023-06-09T11:10:44.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f6386ffd700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f637800c4e0 tx=0x7f637800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780052d0 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6378005430 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63880c49b0 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63780038c0 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.327+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63880c46d0 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.328+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63880c4c90 con 0x7f63880c64e0 2023-06-09T11:10:44.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.329+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f6378003a20 con 0x7f63880c64e0 2023-06-09T11:10:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.330+0000 7f636f7fe700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 0x7f637406d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.330+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f63780884c0 con 0x7f63880c64e0 2023-06-09T11:10:44.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.330+0000 7f63867fc700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 0x7f637406d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.331+0000 7f63867fc700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 0x7f637406d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f638815b690 tx=0x7f637c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.336+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6378053870 con 0x7f63880c64e0 2023-06-09T11:10:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.544+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63880c6220 con 0x7f63880c64e0 2023-06-09T11:10:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.545+0000 7f636f7fe700 1 -- 172.21.15.7:0/1884903531 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63880c6220 con 0x7f63880c64e0 2023-06-09T11:10:44.550 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:10:44.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 msgr2=0x7f637406d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 0x7f637406d100 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f638815b690 tx=0x7f637c009040 comp rx=0 tx=0).stop 2023-06-09T11:10:44.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 msgr2=0x7f638814ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f637800c4e0 tx=0x7f637800c8a0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 shutdown_connections 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f637406ac50 0x7f637406d100 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f638815b9c0 0x7f63880ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63880c64e0 0x7f638814ce40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 --2- 172.21.15.7:0/1884903531 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63881537a0 0x7f63881e49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 >> 172.21.15.7:0/1884903531 conn(0x7f6388072040 msgr2=0x7f6388151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.547+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 shutdown_connections 2023-06-09T11:10:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.548+0000 7f638d3ff700 1 -- 172.21.15.7:0/1884903531 wait complete. 2023-06-09T11:10:44.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:10:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.773+0000 7f8ccfbff700 1 Processor -- start 2023-06-09T11:10:44.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.773+0000 7f8ccfbff700 1 -- start start 2023-06-09T11:10:44.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.775+0000 7f8ccfbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.775+0000 7f8ccfbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc8071ca0 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.775+0000 7f8ccfbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc8071120 con 0x7f8cc8153740 2023-06-09T11:10:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.775+0000 7f8ccfbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc8159770 con 0x7f8cc8153370 2023-06-09T11:10:44.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:36012/0 (socket says 172.21.15.7:36012) 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8cce19c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8cc8153740 0x7f8cc8159b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:57332/0 (socket says 172.21.15.7:57332) 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd19a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8cc80c9fa0 0x7f8cc80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59368/0 (socket says 172.21.15.7:59368) 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 -- 172.21.15.7:0/57933782 learned_addr learned my addr 172.21.15.7:0/57933782 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 -- 172.21.15.7:0/57933782 >> v1:172.21.15.117:6789/0 conn(0x7f8cc8153740 legacy=0x7f8cc8159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 -- 172.21.15.7:0/57933782 >> v1:172.21.15.7:6789/0 conn(0x7f8cc80c9fa0 legacy=0x7f8cc80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccc999700 1 -- 172.21.15.7:0/57933782 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706292282 0 0) 0x7f8cc8071120 con 0x7f8cc8153740 2023-06-09T11:10:44.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 -- 172.21.15.7:0/57933782 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8071400 con 0x7f8cc8153370 2023-06-09T11:10:44.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.776+0000 7f8ccd99b700 1 --2- 172.21.15.7:0/57933782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009920 tx=0x7f8cc4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38c6f7433a0bc18d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccc999700 1 -- 172.21.15.7:0/57933782 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4004230 con 0x7f8cc8153370 2023-06-09T11:10:44.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccc999700 1 -- 172.21.15.7:0/57933782 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4004390 con 0x7f8cc8153370 2023-06-09T11:10:44.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccc999700 1 -- 172.21.15.7:0/57933782 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4009e70 con 0x7f8cc8153370 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccfbff700 1 -- 172.21.15.7:0/57933782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 msgr2=0x7f8cc80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/57933782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009920 tx=0x7f8cc4009c30 comp rx=0 tx=0).stop 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccfbff700 1 -- 172.21.15.7:0/57933782 shutdown_connections 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/57933782 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80b4450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.777+0000 7f8ccfbff700 1 -- 172.21.15.7:0/57933782 >> 172.21.15.7:0/57933782 conn(0x7f8cc80737f0 msgr2=0x7f8cc8073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.778+0000 7f8ccfbff700 1 -- 172.21.15.7:0/57933782 shutdown_connections 2023-06-09T11:10:44.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.778+0000 7f8ccfbff700 1 -- 172.21.15.7:0/57933782 wait complete. 2023-06-09T11:10:44.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.779+0000 7f8ccfbff700 1 Processor -- start 2023-06-09T11:10:44.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.779+0000 7f8ccfbff700 1 -- start start 2023-06-09T11:10:44.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cc8153740 0x7f8cc80cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80cce30 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc80ca7e0 con 0x7f8cc8153740 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccfbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81f2c80 con 0x7f8cc8153370 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.780+0000 7f8ccd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34090/0 (socket says 172.21.15.7:34090) 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 -- 172.21.15.7:0/506935437 learned_addr learned my addr 172.21.15.7:0/506935437 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd19a700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8cce19c700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cc8153740 0x7f8cc80cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 -- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 msgr2=0x7f8cc80caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 -- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cc8153740 msgr2=0x7f8cc80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cc8153740 0x7f8cc80cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 -- 172.21.15.7:0/506935437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009550 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8ccd99b700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006fd0 tx=0x7f8cc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.781+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400be30 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.782+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc40036a0 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.782+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc80719c0 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.782+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4003a10 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.782+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc80716e0 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.783+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc80c5040 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.784+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f8cc4022020 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.785+0000 7f8cbe7fc700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 0x7f8cac06d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.785+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8cc408a640 con 0x7f8cc80c9fa0 2023-06-09T11:10:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.786+0000 7f8ccd19a700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 0x7f8cac06d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.787+0000 7f8ccd19a700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 0x7f8cac06d100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ce750 tx=0x7f8cb8004a50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:44.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:44.790+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8cc4055bb0 con 0x7f8cc80c9fa0 2023-06-09T11:10:45.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.004+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8cc8074cf0 con 0x7f8cc80c9fa0 2023-06-09T11:10:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.004+0000 7f8cbe7fc700 1 -- 172.21.15.7:0/506935437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8cc40591d0 con 0x7f8cc80c9fa0 2023-06-09T11:10:45.009 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:10:45.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 msgr2=0x7f8cac06d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 0x7f8cac06d100 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ce750 tx=0x7f8cb8004a50 comp rx=0 tx=0).stop 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 msgr2=0x7f8cc814b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4006fd0 tx=0x7f8cc4004750 comp rx=0 tx=0).stop 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 shutdown_connections 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cc8153740 0x7f8cc80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cc80c9fa0 0x7f8cc814b210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cac06ac50 0x7f8cac06d100 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 --2- 172.21.15.7:0/506935437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cc8153370 0x7f8cc80caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 >> 172.21.15.7:0/506935437 conn(0x7f8cc80737f0 msgr2=0x7f8cc80cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 shutdown_connections 2023-06-09T11:10:45.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.007+0000 7f8ccfbff700 1 -- 172.21.15.7:0/506935437 wait complete. 2023-06-09T11:10:45.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:10:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.227+0000 7fe6a4fff700 1 Processor -- start 2023-06-09T11:10:45.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.227+0000 7fe6a4fff700 1 -- start start 2023-06-09T11:10:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.228+0000 7fe6a4fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6a01539b0 0x7fe6a0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.228+0000 7fe6a4fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a015a420 con 0x7fe6a01535e0 2023-06-09T11:10:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.228+0000 7fe6a4fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a006b460 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.228+0000 7fe6a4fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a00ca670 con 0x7fe6a01539b0 2023-06-09T11:10:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.228+0000 7fe69dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6a01539b0 0x7fe6a0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6a01539b0 0x7fe6a0159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34096/0 (socket says 172.21.15.7:34096) 2023-06-09T11:10:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe6a01535e0 0x7fe6a00c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59372/0 (socket says 172.21.15.7:59372) 2023-06-09T11:10:45.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 -- 172.21.15.7:0/900126395 learned_addr learned my addr 172.21.15.7:0/900126395 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 -- 172.21.15.7:0/900126395 >> v1:172.21.15.117:6789/0 conn(0x7fe6a00c4ce0 legacy=0x7fe6a015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 -- 172.21.15.7:0/900126395 >> v1:172.21.15.7:6789/0 conn(0x7fe6a01535e0 legacy=0x7fe6a00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 -- 172.21.15.7:0/900126395 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a006a570 con 0x7fe6a01539b0 2023-06-09T11:10:45.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69d59a700 1 -- 172.21.15.7:0/900126395 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311273124 0 0) 0x7fe690001110 con 0x7fe6a01535e0 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69dd9b700 1 --2- 172.21.15.7:0/900126395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a01539b0 0x7fe6a0159ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098f0 tx=0x7fe694009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2205f36261442686 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.229+0000 7fe69d59a700 1 -- 172.21.15.7:0/900126395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400b8a0 con 0x7fe6a01539b0 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe69d59a700 1 -- 172.21.15.7:0/900126395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69400ba00 con 0x7fe6a01539b0 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe69d59a700 1 -- 172.21.15.7:0/900126395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69400bd90 con 0x7fe6a01539b0 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 -- 172.21.15.7:0/900126395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a01539b0 msgr2=0x7fe6a0159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/900126395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a01539b0 0x7fe6a0159ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098f0 tx=0x7fe694009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 -- 172.21.15.7:0/900126395 shutdown_connections 2023-06-09T11:10:45.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/900126395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a01539b0 0x7fe6a0159ee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 -- 172.21.15.7:0/900126395 >> 172.21.15.7:0/900126395 conn(0x7fe6a00737f0 msgr2=0x7fe6a0072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 -- 172.21.15.7:0/900126395 shutdown_connections 2023-06-09T11:10:45.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.230+0000 7fe6a4fff700 1 -- 172.21.15.7:0/900126395 wait complete. 2023-06-09T11:10:45.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.232+0000 7fe6a4fff700 1 Processor -- start 2023-06-09T11:10:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.232+0000 7fe6a4fff700 1 -- start start 2023-06-09T11:10:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6a01535e0 0x7fe6a01f1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6a01539b0 0x7fe6a01f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00d00f0 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0158260 con 0x7fe6a01535e0 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe6a4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a0158790 con 0x7fe6a01539b0 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34112/0 (socket says 172.21.15.7:34112) 2023-06-09T11:10:45.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69e59c700 1 -- 172.21.15.7:0/1265558481 learned_addr learned my addr 172.21.15.7:0/1265558481 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69dd9b700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6a01535e0 0x7fe6a01f1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69ed9d700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6a01539b0 0x7fe6a01f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69e59c700 1 -- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6a01539b0 msgr2=0x7fe6a01f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.233+0000 7fe69e59c700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6a01539b0 0x7fe6a01f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe69e59c700 1 -- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6a01535e0 msgr2=0x7fe6a01f1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe69e59c700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6a01535e0 0x7fe6a01f1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe69e59c700 1 -- 172.21.15.7:0/1265558481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694009550 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe69e59c700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c520 tx=0x7fe69000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690004c80 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe690004de0 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690007310 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a006ab30 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.234+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a006a850 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.235+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a006ae10 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.236+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe690007470 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.237+0000 7fe68effd700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 0x7fe68006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.237+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe690088bc0 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.237+0000 7fe69dd9b700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 0x7fe68006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.238+0000 7fe69dd9b700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 0x7fe68006d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0153da0 tx=0x7fe69401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.242+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe690053fb0 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.501+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a015a420 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.502+0000 7fe68effd700 1 -- 172.21.15.7:0/1265558481 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+5033 (secure 0 0 0) 0x7fe6a015a420 con 0x7fe6a00c4ce0 2023-06-09T11:10:45.508 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:45.508 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-09T11:10:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 msgr2=0x7fe68006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 0x7fe68006d190 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0153da0 tx=0x7fe69401c040 comp rx=0 tx=0).stop 2023-06-09T11:10:45.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 msgr2=0x7fe6a01f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe69000c520 tx=0x7fe69000c830 comp rx=0 tx=0).stop 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 shutdown_connections 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe68006ace0 0x7fe68006d190 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6a01539b0 0x7fe6a01f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6a00c4ce0 0x7fe6a01f1600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.504+0000 7fe6a4fff700 1 --2- 172.21.15.7:0/1265558481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6a01535e0 0x7fe6a01f1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.505+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 >> 172.21.15.7:0/1265558481 conn(0x7fe6a00737f0 msgr2=0x7fe6a0152970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.505+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 shutdown_connections 2023-06-09T11:10:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.505+0000 7fe6a4fff700 1 -- 172.21.15.7:0/1265558481 wait complete. 2023-06-09T11:10:45.511 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2023-06-09T11:10:45.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.725+0000 7f4bf11ff700 1 Processor -- start 2023-06-09T11:10:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.725+0000 7f4bf11ff700 1 -- start start 2023-06-09T11:10:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bf11ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4bec153880 0x7f4bec15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bf11ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec0c9f40 0x7f4bec15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bf11ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec15a500 con 0x7f4bec15b2d0 2023-06-09T11:10:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bf11ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0c5f10 con 0x7f4bec153880 2023-06-09T11:10:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bf11ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec15da00 con 0x7f4bec0c9f40 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4bec153880 0x7f4bec15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4bec153880 0x7f4bec15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34136/0 (socket says 172.21.15.7:34136) 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4beb59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4bec15b2d0 0x7f4bec0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59386/0 (socket says 172.21.15.7:59386) 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 -- 172.21.15.7:0/1624422491 learned_addr learned my addr 172.21.15.7:0/1624422491 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bea59c700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec0c9f40 0x7f4bec15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 -- 172.21.15.7:0/1624422491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec0c9f40 msgr2=0x7f4bec15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec0c9f40 0x7f4bec15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4bead9d700 1 -- 172.21.15.7:0/1624422491 >> v1:172.21.15.7:6789/0 conn(0x7f4bec15b2d0 legacy=0x7f4bec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.727+0000 7f4be9d9b700 1 -- 172.21.15.7:0/1624422491 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510480449 0 0) 0x7f4be4001490 con 0x7f4bec15b2d0 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4bead9d700 1 -- 172.21.15.7:0/1624422491 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec15b750 con 0x7f4bec153880 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4bead9d700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec15d4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098f0 tx=0x7f4bdc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a642d2b0413c4ce server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4be9d9b700 1 -- 172.21.15.7:0/1624422491 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00b8a0 con 0x7f4bec153880 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4be9d9b700 1 -- 172.21.15.7:0/1624422491 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00ba00 con 0x7f4bec153880 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4be9d9b700 1 -- 172.21.15.7:0/1624422491 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00bd90 con 0x7f4bec153880 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4bf11ff700 1 -- 172.21.15.7:0/1624422491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 msgr2=0x7f4bec15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.728+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec15d4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098f0 tx=0x7f4bdc009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 -- 172.21.15.7:0/1624422491 shutdown_connections 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec15d4c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/1624422491 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec0c9f40 0x7f4bec15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 -- 172.21.15.7:0/1624422491 >> 172.21.15.7:0/1624422491 conn(0x7f4bec0737f0 msgr2=0x7f4bec0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 -- 172.21.15.7:0/1624422491 shutdown_connections 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.729+0000 7f4bf11ff700 1 -- 172.21.15.7:0/1624422491 wait complete. 2023-06-09T11:10:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.730+0000 7f4bf11ff700 1 Processor -- start 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.730+0000 7f4bf11ff700 1 -- start start 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bec0c9f40 0x7f4bec158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec15b2d0 0x7f4bec148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec158e50 con 0x7f4bec153880 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec158050 con 0x7f4bec0c9f40 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bf11ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec14b570 con 0x7f4bec15b2d0 2023-06-09T11:10:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.731+0000 7f4bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34144/0 (socket says 172.21.15.7:34144) 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 -- 172.21.15.7:0/770589362 learned_addr learned my addr 172.21.15.7:0/770589362 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4beb59e700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec15b2d0 0x7f4bec148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bead9d700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bec0c9f40 0x7f4bec158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 -- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec15b2d0 msgr2=0x7f4bec148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec15b2d0 0x7f4bec148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 -- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bec0c9f40 msgr2=0x7f4bec158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bec0c9f40 0x7f4bec158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 -- 172.21.15.7:0/770589362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009550 con 0x7f4bec153880 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bea59c700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c820 tx=0x7f4be000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.732+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0004220 con 0x7f4bec153880 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.733+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be0004380 con 0x7f4bec153880 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.733+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec159980 con 0x7f4bec153880 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.733+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be00036a0 con 0x7f4bec153880 2023-06-09T11:10:45.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.733+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec1596e0 con 0x7f4bec153880 2023-06-09T11:10:45.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.734+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec159c60 con 0x7f4bec153880 2023-06-09T11:10:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.735+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4be0003800 con 0x7f4bec153880 2023-06-09T11:10:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.736+0000 7f4bd37fe700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 0x7f4bd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.736+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4be0088be0 con 0x7f4bec153880 2023-06-09T11:10:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.736+0000 7f4bead9d700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 0x7f4bd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:45.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.737+0000 7f4bead9d700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 0x7f4bd806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:45.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.741+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4be0057a60 con 0x7f4bec153880 2023-06-09T11:10:45.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.953+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bec14b2f0 con 0x7f4bec153880 2023-06-09T11:10:45.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.954+0000 7f4bd37fe700 1 -- 172.21.15.7:0/770589362 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14540 (secure 0 0 0) 0x7f4be001e090 con 0x7f4bec153880 2023-06-09T11:10:45.959 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:45.959 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:44.549260+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:45.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 msgr2=0x7f4bd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 0x7f4bd806d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc008030 comp rx=0 tx=0).stop 2023-06-09T11:10:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 msgr2=0x7f4bec149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c820 tx=0x7f4be000cb30 comp rx=0 tx=0).stop 2023-06-09T11:10:45.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 shutdown_connections 2023-06-09T11:10:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4bd806ace0 0x7f4bd806d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4bec15b2d0 0x7f4bec148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4bec0c9f40 0x7f4bec158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 --2- 172.21.15.7:0/770589362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4bec153880 0x7f4bec149c70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 >> 172.21.15.7:0/770589362 conn(0x7f4bec0737f0 msgr2=0x7f4bec14d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 shutdown_connections 2023-06-09T11:10:45.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:45.956+0000 7f4bf11ff700 1 -- 172.21.15.7:0/770589362 wait complete. 2023-06-09T11:10:45.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:10:45.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:10:46.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.178+0000 7fb8115ff700 1 Processor -- start 2023-06-09T11:10:46.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.179+0000 7fb8115ff700 1 -- start start 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 0x7fb80c0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb80c071040 0x7fb80c1552b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c072050 con 0x7fb80c071040 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80c157450 con 0x7fb80c151600 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb8115ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1576f0 con 0x7fb80c15b340 2023-06-09T11:10:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb80b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb80c071040 0x7fb80c1552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb80b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb80c071040 0x7fb80c1552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34160/0 (socket says 172.21.15.7:34160) 2023-06-09T11:10:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb80b7fe700 1 -- 172.21.15.7:0/1563403680 learned_addr learned my addr 172.21.15.7:0/1563403680 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb803fff700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.180+0000 7fb80affd700 1 --2- 172.21.15.7:0/1563403680 >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 0x7fb80c0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 -- 172.21.15.7:0/1563403680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 msgr2=0x7fb80c070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 -- 172.21.15.7:0/1563403680 >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 msgr2=0x7fb80c0c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 --2- 172.21.15.7:0/1563403680 >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 0x7fb80c0c5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 -- 172.21.15.7:0/1563403680 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c1579d0 con 0x7fb80c071040 2023-06-09T11:10:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb803fff700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80affd700 1 --2- 172.21.15.7:0/1563403680 >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 0x7fb80c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80b7fe700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c1552b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80c06ff10 tx=0x7fb80400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624e8c799c9874b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80a7fc700 1 -- 172.21.15.7:0/1563403680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400e070 con 0x7fb80c071040 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80a7fc700 1 -- 172.21.15.7:0/1563403680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80400b9c0 con 0x7fb80c071040 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.181+0000 7fb80a7fc700 1 -- 172.21.15.7:0/1563403680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80400bd70 con 0x7fb80c071040 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 -- 172.21.15.7:0/1563403680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 msgr2=0x7fb80c1552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c1552b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb80c06ff10 tx=0x7fb80400b330 comp rx=0 tx=0).stop 2023-06-09T11:10:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 -- 172.21.15.7:0/1563403680 shutdown_connections 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 --2- 172.21.15.7:0/1563403680 >> v2:172.21.15.117:3300/0 conn(0x7fb80c151600 0x7fb80c0c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c1552b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 --2- 172.21.15.7:0/1563403680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 -- 172.21.15.7:0/1563403680 >> 172.21.15.7:0/1563403680 conn(0x7fb80c06a4d0 msgr2=0x7fb80c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 -- 172.21.15.7:0/1563403680 shutdown_connections 2023-06-09T11:10:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.182+0000 7fb8115ff700 1 -- 172.21.15.7:0/1563403680 wait complete. 2023-06-09T11:10:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.183+0000 7fb8115ff700 1 Processor -- start 2023-06-09T11:10:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 -- start start 2023-06-09T11:10:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c151600 0x7fb80c0c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c159ba0 con 0x7fb80c071040 2023-06-09T11:10:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1eee90 con 0x7fb80c151600 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.184+0000 7fb8115ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80c1efe60 con 0x7fb80c15b340 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34166/0 (socket says 172.21.15.7:34166) 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 -- 172.21.15.7:0/3428295511 learned_addr learned my addr 172.21.15.7:0/3428295511 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80b7fe700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb803fff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c151600 0x7fb80c0c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 -- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 msgr2=0x7fb80c0c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c0c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 -- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c151600 msgr2=0x7fb80c0c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c151600 0x7fb80c0c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 -- 172.21.15.7:0/3428295511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804009b30 con 0x7fb80c071040 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.185+0000 7fb80affd700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d830 tx=0x7fb7fc00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.186+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc004030 con 0x7fb80c071040 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.186+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc004190 con 0x7fb80c071040 2023-06-09T11:10:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.186+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c157f90 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.186+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc004520 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.186+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80c157cb0 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.187+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80c0c6250 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.188+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7fc0036a0 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.189+0000 7fb8037fe700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 0x7fb7ec06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.190+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb7fc089800 con 0x7fb80c071040 2023-06-09T11:10:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.190+0000 7fb803fff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 0x7fb7ec06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.190+0000 7fb803fff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 0x7fb7ec06d1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb80c15a900 tx=0x7fb7f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.194+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb7fc054ad0 con 0x7fb80c071040 2023-06-09T11:10:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.399+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb80c074cf0 con 0x7fb80c071040 2023-06-09T11:10:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.400+0000 7fb8037fe700 1 -- 172.21.15.7:0/3428295511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb7fc0580f0 con 0x7fb80c071040 2023-06-09T11:10:46.405 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:10:46.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 msgr2=0x7fb7ec06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 0x7fb7ec06d1c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb80c15a900 tx=0x7fb7f4008040 comp rx=0 tx=0).stop 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 msgr2=0x7fb80c158900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00d830 tx=0x7fb7fc00dbf0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 shutdown_connections 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7ec06ad10 0x7fb7ec06d1c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb80c15b340 0x7fb80c0c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb80c071040 0x7fb80c158900 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 --2- 172.21.15.7:0/3428295511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb80c151600 0x7fb80c0c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 >> 172.21.15.7:0/3428295511 conn(0x7fb80c06a4d0 msgr2=0x7fb80c0661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 shutdown_connections 2023-06-09T11:10:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.402+0000 7fb8115ff700 1 -- 172.21.15.7:0/3428295511 wait complete. 2023-06-09T11:10:46.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:10:46.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.622+0000 7f0d869ff700 1 Processor -- start 2023-06-09T11:10:46.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.622+0000 7f0d869ff700 1 -- start start 2023-06-09T11:10:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d869ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d801537a0 0x7f0d8006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d869ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d8006b740 0x7f0d8015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d869ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800c6220 con 0x7f0d800c64e0 2023-06-09T11:10:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d869ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c4f70 con 0x7f0d801537a0 2023-06-09T11:10:46.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d869ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800c5c20 con 0x7f0d8006b740 2023-06-09T11:10:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.623+0000 7f0d7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d801537a0 0x7f0d8006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0d801537a0 0x7f0d8006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34182/0 (socket says 172.21.15.7:34182) 2023-06-09T11:10:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d800c64e0 0x7f0d800cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59394/0 (socket says 172.21.15.7:59394) 2023-06-09T11:10:46.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 -- 172.21.15.7:0/782846865 learned_addr learned my addr 172.21.15.7:0/782846865 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:46.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d84f9c700 1 --2- 172.21.15.7:0/782846865 >> v2:172.21.15.117:3300/0 conn(0x7f0d8006b740 0x7f0d8015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 -- 172.21.15.7:0/782846865 >> v2:172.21.15.117:3300/0 conn(0x7f0d8006b740 msgr2=0x7f0d8015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 --2- 172.21.15.7:0/782846865 >> v2:172.21.15.117:3300/0 conn(0x7f0d8006b740 0x7f0d8015b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 -- 172.21.15.7:0/782846865 >> v1:172.21.15.7:6789/0 conn(0x7f0d800c64e0 legacy=0x7f0d800cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 -- 172.21.15.7:0/782846865 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800c4480 con 0x7f0d801537a0 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7effd700 1 -- 172.21.15.7:0/782846865 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826587964 0 0) 0x7f0d74001110 con 0x7f0d800c64e0 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7ffff700 1 --2- 172.21.15.7:0/782846865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d801537a0 0x7f0d8006b1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d8008d6c0 tx=0x7f0d70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9d38db858c1515 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.624+0000 7f0d7effd700 1 -- 172.21.15.7:0/782846865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70011830 con 0x7f0d801537a0 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d7effd700 1 -- 172.21.15.7:0/782846865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70011990 con 0x7f0d801537a0 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d7effd700 1 -- 172.21.15.7:0/782846865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70011d20 con 0x7f0d801537a0 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 -- 172.21.15.7:0/782846865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d801537a0 msgr2=0x7f0d8006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 --2- 172.21.15.7:0/782846865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d801537a0 0x7f0d8006b1b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d8008d6c0 tx=0x7f0d70009bd0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 -- 172.21.15.7:0/782846865 shutdown_connections 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 --2- 172.21.15.7:0/782846865 >> v2:172.21.15.117:3300/0 conn(0x7f0d8006b740 0x7f0d8015b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 --2- 172.21.15.7:0/782846865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d801537a0 0x7f0d8006b1b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 -- 172.21.15.7:0/782846865 >> 172.21.15.7:0/782846865 conn(0x7f0d80072040 msgr2=0x7f0d80072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 -- 172.21.15.7:0/782846865 shutdown_connections 2023-06-09T11:10:46.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.625+0000 7f0d869ff700 1 -- 172.21.15.7:0/782846865 wait complete. 2023-06-09T11:10:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.627+0000 7f0d869ff700 1 Processor -- start 2023-06-09T11:10:46.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.627+0000 7f0d869ff700 1 -- start start 2023-06-09T11:10:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 0x7f0d800c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 0x7f0d8014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800c96e0 con 0x7f0d8006b740 2023-06-09T11:10:46.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8015a400 con 0x7f0d800c64e0 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d869ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8014b740 con 0x7f0d801537a0 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d7ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34194/0 (socket says 172.21.15.7:34194) 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.628+0000 7f0d7ffff700 1 -- 172.21.15.7:0/1937226454 learned_addr learned my addr 172.21.15.7:0/1937226454 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d84f9c700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 0x7f0d8014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 -- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 msgr2=0x7f0d8014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7f7fe700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 0x7f0d800c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 0x7f0d8014df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 -- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 msgr2=0x7f0d800c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 0x7f0d800c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 -- 172.21.15.7:0/1937226454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70009550 con 0x7f0d8006b740 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d84f9c700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 0x7f0d8014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7f7fe700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 0x7f0d800c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7ffff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009c10 tx=0x7f0d70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d700036a0 con 0x7f0d8006b740 2023-06-09T11:10:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d70003800 con 0x7f0d8006b740 2023-06-09T11:10:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800c46d0 con 0x7f0d8006b740 2023-06-09T11:10:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d70003b70 con 0x7f0d8006b740 2023-06-09T11:10:46.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.629+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8015ae20 con 0x7f0d8006b740 2023-06-09T11:10:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.630+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d800c49b0 con 0x7f0d8006b740 2023-06-09T11:10:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.633+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d70003cd0 con 0x7f0d8006b740 2023-06-09T11:10:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.634+0000 7f0d7cff9700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 0x7f0d6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.634+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0d70021030 con 0x7f0d8006b740 2023-06-09T11:10:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.634+0000 7f0d7f7fe700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 0x7f0d6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.635+0000 7f0d7f7fe700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 0x7f0d6c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d801f01e0 tx=0x7f0d74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.637+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0d700561a0 con 0x7f0d8006b740 2023-06-09T11:10:46.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.843+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d8014ba20 con 0x7f0d8006b740 2023-06-09T11:10:46.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.843+0000 7f0d7cff9700 1 -- 172.21.15.7:0/1937226454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d7002b020 con 0x7f0d8006b740 2023-06-09T11:10:46.849 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:10:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.845+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 msgr2=0x7f0d6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 0x7f0d6c06d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0d801f01e0 tx=0x7f0d74009040 comp rx=0 tx=0).stop 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 msgr2=0x7f0d8014a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d70009c10 tx=0x7f0d70004750 comp rx=0 tx=0).stop 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 shutdown_connections 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d6c06ace0 0x7f0d6c06d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d801537a0 0x7f0d8014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d8006b740 0x7f0d8014a100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 --2- 172.21.15.7:0/1937226454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d800c64e0 0x7f0d800c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 >> 172.21.15.7:0/1937226454 conn(0x7f0d80072040 msgr2=0x7f0d800cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 shutdown_connections 2023-06-09T11:10:46.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:46.846+0000 7f0d869ff700 1 -- 172.21.15.7:0/1937226454 wait complete. 2023-06-09T11:10:46.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-09T11:10:47.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.064+0000 7fc280fff700 1 Processor -- start 2023-06-09T11:10:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.064+0000 7fc280fff700 1 -- start start 2023-06-09T11:10:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.065+0000 7fc280fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc27c159ee0 0x7fc27c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.065+0000 7fc280fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c06b460 con 0x7fc27c1535e0 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.065+0000 7fc280fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc27c06a570 con 0x7fc27c1539b0 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.065+0000 7fc280fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27c0ca670 con 0x7fc27c159ee0 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc27c1535e0 0x7fc27c0c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59400/0 (socket says 172.21.15.7:59400) 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27a59c700 1 -- 172.21.15.7:0/3032408727 learned_addr learned my addr 172.21.15.7:0/3032408727 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27ad9d700 1 --2- 172.21.15.7:0/3032408727 >> v2:172.21.15.117:3300/0 conn(0x7fc27c159ee0 0x7fc27c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241812384 0 0) 0x7fc27c06b460 con 0x7fc27c1535e0 2023-06-09T11:10:47.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2640035b0 con 0x7fc27c1535e0 2023-06-09T11:10:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858525174 0 0) 0x7fc2640035b0 con 0x7fc27c1535e0 2023-06-09T11:10:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27ad9d700 1 -- 172.21.15.7:0/3032408727 >> v1:172.21.15.117:6789/0 conn(0x7fc27c1539b0 legacy=0x7fc27c0c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27ad9d700 1 -- 172.21.15.7:0/3032408727 >> v1:172.21.15.7:6789/0 conn(0x7fc27c1535e0 legacy=0x7fc27c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.066+0000 7fc27ad9d700 1 -- 172.21.15.7:0/3032408727 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c06a850 con 0x7fc27c159ee0 2023-06-09T11:10:47.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc27ad9d700 1 --2- 172.21.15.7:0/3032408727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c159ee0 0x7fc27c15abb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d90 tx=0x7fc27400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5466407cb8c7dfb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e070 con 0x7fc27c159ee0 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc274003a20 con 0x7fc27c159ee0 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc27959a700 1 -- 172.21.15.7:0/3032408727 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c001670 con 0x7fc27c159ee0 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc280fff700 1 -- 172.21.15.7:0/3032408727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c159ee0 msgr2=0x7fc27c15abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.067+0000 7fc280fff700 1 --2- 172.21.15.7:0/3032408727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c159ee0 0x7fc27c15abb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d90 tx=0x7fc27400b330 comp rx=0 tx=0).stop 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.068+0000 7fc280fff700 1 -- 172.21.15.7:0/3032408727 shutdown_connections 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.068+0000 7fc280fff700 1 --2- 172.21.15.7:0/3032408727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c159ee0 0x7fc27c15abb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.068+0000 7fc280fff700 1 -- 172.21.15.7:0/3032408727 >> 172.21.15.7:0/3032408727 conn(0x7fc27c0737f0 msgr2=0x7fc27c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.068+0000 7fc280fff700 1 -- 172.21.15.7:0/3032408727 shutdown_connections 2023-06-09T11:10:47.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.068+0000 7fc280fff700 1 -- 172.21.15.7:0/3032408727 wait complete. 2023-06-09T11:10:47.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.069+0000 7fc280fff700 1 Processor -- start 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.069+0000 7fc280fff700 1 -- start start 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c1535e0 0x7fc27c1f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27c159ee0 0x7fc27c1efaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c1ef610 con 0x7fc27c1539b0 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c1eefe0 con 0x7fc27c1535e0 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc280fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c1f3000 con 0x7fc27c159ee0 2023-06-09T11:10:47.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.070+0000 7fc279d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34206/0 (socket says 172.21.15.7:34206) 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 -- 172.21.15.7:0/3133118901 learned_addr learned my addr 172.21.15.7:0/3133118901 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc27a59c700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c1535e0 0x7fc27c1f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc27ad9d700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27c159ee0 0x7fc27c1efaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 -- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27c159ee0 msgr2=0x7fc27c1efaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27c159ee0 0x7fc27c1efaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 -- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c1535e0 msgr2=0x7fc27c1f0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c1535e0 0x7fc27c1f0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 -- 172.21.15.7:0/3133118901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009a70 con 0x7fc27c1539b0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.071+0000 7fc279d9b700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c4b0 tx=0x7fc27000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.072+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700052d0 con 0x7fc27c1539b0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.072+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc270005430 con 0x7fc27c1539b0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.072+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2700038c0 con 0x7fc27c1539b0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.072+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c06ae10 con 0x7fc27c1539b0 2023-06-09T11:10:47.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.072+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c06ab30 con 0x7fc27c1539b0 2023-06-09T11:10:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.073+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c06b0f0 con 0x7fc27c1539b0 2023-06-09T11:10:47.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.074+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc270003a20 con 0x7fc27c1539b0 2023-06-09T11:10:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.075+0000 7fc26affd700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 0x7fc25c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.075+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc270088500 con 0x7fc27c1539b0 2023-06-09T11:10:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.075+0000 7fc27a59c700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 0x7fc25c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.076+0000 7fc27a59c700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 0x7fc25c06d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc26c001eb0 tx=0x7fc26c001e40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.079+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc270053860 con 0x7fc27c1539b0 2023-06-09T11:10:47.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.287+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc27c074cf0 con 0x7fc27c1539b0 2023-06-09T11:10:47.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.288+0000 7fc26affd700 1 -- 172.21.15.7:0/3133118901 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+126 (secure 0 0 0) 0x7fc270056e80 con 0x7fc27c1539b0 2023-06-09T11:10:47.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:47.293 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2023-06-09T11:10:47.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-06-09T11:10:47.294 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2023-06-09T11:10:47.294 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2023-06-09T11:10:47.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-06-09T11:10:47.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2023-06-09T11:10:47.295 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 2, 2023-06-09T11:10:47.295 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": "rbd" 2023-06-09T11:10:47.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-06-09T11:10:47.295 INFO:teuthology.orchestra.run.smithi007.stdout:] 2023-06-09T11:10:47.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 msgr2=0x7fc25c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 0x7fc25c06d190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc26c001eb0 tx=0x7fc26c001e40 comp rx=0 tx=0).stop 2023-06-09T11:10:47.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 msgr2=0x7fc27c158890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc27000c4b0 tx=0x7fc27000c7c0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 shutdown_connections 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc25c06ace0 0x7fc25c06d190 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc27c159ee0 0x7fc27c1efaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc27c1535e0 0x7fc27c1f0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 --2- 172.21.15.7:0/3133118901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc27c1539b0 0x7fc27c158890 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 >> 172.21.15.7:0/3133118901 conn(0x7fc27c0737f0 msgr2=0x7fc27c0c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 shutdown_connections 2023-06-09T11:10:47.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.290+0000 7fc280fff700 1 -- 172.21.15.7:0/3133118901 wait complete. 2023-06-09T11:10:47.313 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-09T11:10:47.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-09T11:10:47.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.512+0000 7fd3579ff700 1 Processor -- start 2023-06-09T11:10:47.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.512+0000 7fd3579ff700 1 -- start start 2023-06-09T11:10:47.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.513+0000 7fd3579ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd35015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd3579ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3500b5270 0x7fd3500c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd3579ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35015dd00 con 0x7fd3500cc100 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd3579ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd350071790 con 0x7fd3500b5270 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd3579ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3500ca3f0 con 0x7fd3500cc4d0 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd35579b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd3500cc100 0x7fd3500cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59414/0 (socket says 172.21.15.7:59414) 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd35579b700 1 -- 172.21.15.7:0/4143611743 learned_addr learned my addr 172.21.15.7:0/4143611743 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd355f9c700 1 --2- 172.21.15.7:0/4143611743 >> v2:172.21.15.7:3300/0 conn(0x7fd3500b5270 0x7fd3500c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd354f9a700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd35015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd347fff700 1 -- 172.21.15.7:0/4143611743 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171500597 0 0) 0x7fd34c001110 con 0x7fd3500cc100 2023-06-09T11:10:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd347fff700 1 -- 172.21.15.7:0/4143611743 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd33c0035b0 con 0x7fd3500cc100 2023-06-09T11:10:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd355f9c700 1 -- 172.21.15.7:0/4143611743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 msgr2=0x7fd35015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd355f9c700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd35015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd355f9c700 1 -- 172.21.15.7:0/4143611743 >> v1:172.21.15.7:6789/0 conn(0x7fd3500cc100 legacy=0x7fd3500cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.514+0000 7fd355f9c700 1 -- 172.21.15.7:0/4143611743 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350159840 con 0x7fd3500b5270 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd355f9c700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd3500c8e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918d22304b4d145d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd347fff700 1 -- 172.21.15.7:0/4143611743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34800e070 con 0x7fd3500b5270 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd347fff700 1 -- 172.21.15.7:0/4143611743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34800bb20 con 0x7fd3500b5270 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd347fff700 1 -- 172.21.15.7:0/4143611743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348012430 con 0x7fd3500b5270 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 -- 172.21.15.7:0/4143611743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 msgr2=0x7fd3500c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd3500c8e60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd348009d90 tx=0x7fd34800b330 comp rx=0 tx=0).stop 2023-06-09T11:10:47.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 -- 172.21.15.7:0/4143611743 shutdown_connections 2023-06-09T11:10:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd3500c8e60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 --2- 172.21.15.7:0/4143611743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd35015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.515+0000 7fd3579ff700 1 -- 172.21.15.7:0/4143611743 >> 172.21.15.7:0/4143611743 conn(0x7fd350070af0 msgr2=0x7fd350070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.516+0000 7fd3579ff700 1 -- 172.21.15.7:0/4143611743 shutdown_connections 2023-06-09T11:10:47.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.516+0000 7fd3579ff700 1 -- 172.21.15.7:0/4143611743 wait complete. 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.517+0000 7fd3579ff700 1 Processor -- start 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.517+0000 7fd3579ff700 1 -- start start 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 0x7fd35015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd3500ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd350156150 con 0x7fd3500b5270 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd350155b90 con 0x7fd3500cc100 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd3579ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd350155e70 con 0x7fd3500cc4d0 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd35579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd35579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34224/0 (socket says 172.21.15.7:34224) 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.518+0000 7fd35579b700 1 -- 172.21.15.7:0/3981052969 learned_addr learned my addr 172.21.15.7:0/3981052969 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd355f9c700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd3500ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 -- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 msgr2=0x7fd3500ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd354f9a700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 0x7fd35015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd3500ce540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 -- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 msgr2=0x7fd35015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 0x7fd35015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 -- 172.21.15.7:0/3981052969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd348009a70 con 0x7fd3500b5270 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd354f9a700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 0x7fd35015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd35579b700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd34c007fd0 tx=0x7fd34c00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c005d70 con 0x7fd3500b5270 2023-06-09T11:10:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0070d0 con 0x7fd3500b5270 2023-06-09T11:10:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c007460 con 0x7fd3500b5270 2023-06-09T11:10:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3500ca6b0 con 0x7fd3500b5270 2023-06-09T11:10:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.519+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3500ca120 con 0x7fd3500b5270 2023-06-09T11:10:47.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.520+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd350159b00 con 0x7fd3500b5270 2023-06-09T11:10:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.521+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd34c0075c0 con 0x7fd3500b5270 2023-06-09T11:10:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.522+0000 7fd345ffb700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.522+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd34c0885f0 con 0x7fd3500b5270 2023-06-09T11:10:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.522+0000 7fd354f9a700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:47.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.523+0000 7fd354f9a700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 0x7fd33406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd350153b50 tx=0x7fd340008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:47.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.526+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd34c053950 con 0x7fd3500b5270 2023-06-09T11:10:47.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:47.736+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd3501547f0 con 0x7fd3500b5270 2023-06-09T11:10:48.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.648+0000 7fd345ffb700 1 -- 172.21.15.7:0/3981052969 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd34c007e30 con 0x7fd3500b5270 2023-06-09T11:10:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 msgr2=0x7fd33406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 0x7fd33406d190 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd350153b50 tx=0x7fd340008040 comp rx=0 tx=0).stop 2023-06-09T11:10:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 msgr2=0x7fd350155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd34c007fd0 tx=0x7fd34c00ca90 comp rx=0 tx=0).stop 2023-06-09T11:10:48.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 shutdown_connections 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd33406ace0 0x7fd33406d190 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3500cc4d0 0x7fd3500ce540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3500b5270 0x7fd350155500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 --2- 172.21.15.7:0/3981052969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3500cc100 0x7fd35015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 >> 172.21.15.7:0/3981052969 conn(0x7fd350070af0 msgr2=0x7fd350158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 shutdown_connections 2023-06-09T11:10:48.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.651+0000 7fd3579ff700 1 -- 172.21.15.7:0/3981052969 wait complete. 2023-06-09T11:10:48.663 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2023-06-09T11:10:48.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-09T11:10:48.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.888+0000 7f0f793ff700 1 Processor -- start 2023-06-09T11:10:48.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.888+0000 7f0f793ff700 1 -- start start 2023-06-09T11:10:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f793ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f741537a0 0x7f0f7406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f793ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f740c64e0 0x7f0f740cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f793ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f740c6220 con 0x7f0f7406b740 2023-06-09T11:10:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f793ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f740c4f70 con 0x7f0f741537a0 2023-06-09T11:10:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f793ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f740c5c20 con 0x7f0f740c64e0 2023-06-09T11:10:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f741537a0 0x7f0f7406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f741537a0 0x7f0f7406b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34230/0 (socket says 172.21.15.7:34230) 2023-06-09T11:10:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 -- 172.21.15.7:0/519008443 learned_addr learned my addr 172.21.15.7:0/519008443 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f6bfff700 1 --2- 172.21.15.7:0/519008443 >> v2:172.21.15.117:3300/0 conn(0x7f0f740c64e0 0x7f0f740cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 -- 172.21.15.7:0/519008443 >> v2:172.21.15.117:3300/0 conn(0x7f0f740c64e0 msgr2=0x7f0f740cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 --2- 172.21.15.7:0/519008443 >> v2:172.21.15.117:3300/0 conn(0x7f0f740c64e0 0x7f0f740cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 -- 172.21.15.7:0/519008443 >> v1:172.21.15.117:6789/0 conn(0x7f0f7406b740 legacy=0x7f0f7415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.890+0000 7f0f72ffd700 1 -- 172.21.15.7:0/519008443 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f740c4480 con 0x7f0f741537a0 2023-06-09T11:10:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f72ffd700 1 --2- 172.21.15.7:0/519008443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f7406b1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cd0b0 tx=0x7f0f64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d9fc164b047012 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f727fc700 1 -- 172.21.15.7:0/519008443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64011830 con 0x7f0f741537a0 2023-06-09T11:10:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f727fc700 1 -- 172.21.15.7:0/519008443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f64011990 con 0x7f0f741537a0 2023-06-09T11:10:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f727fc700 1 -- 172.21.15.7:0/519008443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f64011d20 con 0x7f0f741537a0 2023-06-09T11:10:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f793ff700 1 -- 172.21.15.7:0/519008443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 msgr2=0x7f0f7406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.891+0000 7f0f793ff700 1 --2- 172.21.15.7:0/519008443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f7406b1b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0f740cd0b0 tx=0x7f0f64009bd0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 -- 172.21.15.7:0/519008443 shutdown_connections 2023-06-09T11:10:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 --2- 172.21.15.7:0/519008443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f7406b1b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 --2- 172.21.15.7:0/519008443 >> v2:172.21.15.117:3300/0 conn(0x7f0f740c64e0 0x7f0f740cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 -- 172.21.15.7:0/519008443 >> 172.21.15.7:0/519008443 conn(0x7f0f74072040 msgr2=0x7f0f74072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 -- 172.21.15.7:0/519008443 shutdown_connections 2023-06-09T11:10:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.892+0000 7f0f793ff700 1 -- 172.21.15.7:0/519008443 wait complete. 2023-06-09T11:10:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.893+0000 7f0f793ff700 1 Processor -- start 2023-06-09T11:10:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.893+0000 7f0f793ff700 1 -- start start 2023-06-09T11:10:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f7406b740 0x7f0f74150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f740c64e0 0x7f0f740cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74154b60 con 0x7f0f741537a0 2023-06-09T11:10:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74154020 con 0x7f0f740c64e0 2023-06-09T11:10:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.894+0000 7f0f793ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f741542c0 con 0x7f0f7406b740 2023-06-09T11:10:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34236/0 (socket says 172.21.15.7:34236) 2023-06-09T11:10:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 -- 172.21.15.7:0/2231991897 learned_addr learned my addr 172.21.15.7:0/2231991897 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f72ffd700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f7406b740 0x7f0f74150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f6bfff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f740c64e0 0x7f0f740cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 -- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f7406b740 msgr2=0x7f0f74150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f7406b740 0x7f0f74150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 -- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f740c64e0 msgr2=0x7f0f740cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f740c64e0 0x7f0f740cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 -- 172.21.15.7:0/2231991897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f64009550 con 0x7f0f741537a0 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.895+0000 7f0f737fe700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00cb50 tx=0x7f0f6c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.896+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003c00 con 0x7f0f741537a0 2023-06-09T11:10:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.896+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c003d60 con 0x7f0f741537a0 2023-06-09T11:10:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.896+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f740c46d0 con 0x7f0f741537a0 2023-06-09T11:10:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.896+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c00ab70 con 0x7f0f741537a0 2023-06-09T11:10:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.896+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7415ae20 con 0x7f0f741537a0 2023-06-09T11:10:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.897+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f740c49b0 con 0x7f0f741537a0 2023-06-09T11:10:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.898+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f6c00acd0 con 0x7f0f741537a0 2023-06-09T11:10:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.899+0000 7f0f6b7fe700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 0x7f0f5406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.899+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f0f6c089280 con 0x7f0f741537a0 2023-06-09T11:10:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.899+0000 7f0f72ffd700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 0x7f0f5406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.900+0000 7f0f72ffd700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 0x7f0f5406d1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009c10 tx=0x7f0f64008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:48.904+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0f6c0543e0 con 0x7f0f741537a0 2023-06-09T11:10:49.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.113+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0f740c6220 con 0x7f0f741537a0 2023-06-09T11:10:49.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.655+0000 7f0f6b7fe700 1 -- 172.21.15.7:0/2231991897 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0f740c6220 con 0x7f0f741537a0 2023-06-09T11:10:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.662+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 msgr2=0x7f0f5406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.662+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 0x7f0f5406d1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0f64009c10 tx=0x7f0f64008030 comp rx=0 tx=0).stop 2023-06-09T11:10:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.662+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 msgr2=0x7f0f740cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.662+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00cb50 tx=0x7f0f6c00cf10 comp rx=0 tx=0).stop 2023-06-09T11:10:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 shutdown_connections 2023-06-09T11:10:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f741537a0 0x7f0f740cbfb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f5406ad10 0x7f0f5406d1c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f7406b740 0x7f0f74150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 --2- 172.21.15.7:0/2231991897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f740c64e0 0x7f0f740cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.667+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 >> 172.21.15.7:0/2231991897 conn(0x7f0f74072040 msgr2=0x7f0f7406af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:49.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.669+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 shutdown_connections 2023-06-09T11:10:49.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.674+0000 7f0f793ff700 1 -- 172.21.15.7:0/2231991897 wait complete. 2023-06-09T11:10:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2023-06-09T11:10:49.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-09T11:10:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.905+0000 7f5794bff700 1 Processor -- start 2023-06-09T11:10:49.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.905+0000 7f5794bff700 1 -- start start 2023-06-09T11:10:49.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.906+0000 7f5794bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57901537a0 0x7f579006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.906+0000 7f5794bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f579006b740 0x7f579015b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.906+0000 7f5794bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900c6220 con 0x7f57900c64e0 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.906+0000 7f5794bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900c4f70 con 0x7f579006b740 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.906+0000 7f5794bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900c5c20 con 0x7f57901537a0 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f579006b740 0x7f579015b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f579006b740 0x7f579015b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34248/0 (socket says 172.21.15.7:34248) 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 -- 172.21.15.7:0/3268926329 learned_addr learned my addr 172.21.15.7:0/3268926329 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578dd9b700 1 --1- 172.21.15.7:0/3268926329 >> v1:172.21.15.7:6789/0 conn(0x7f57900c64e0 0x7f57900cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59428/0 (socket says 172.21.15.7:59428) 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578e59c700 1 --2- 172.21.15.7:0/3268926329 >> v2:172.21.15.117:3300/0 conn(0x7f57901537a0 0x7f579006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 -- 172.21.15.7:0/3268926329 >> v2:172.21.15.117:3300/0 conn(0x7f57901537a0 msgr2=0x7f579006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 --2- 172.21.15.7:0/3268926329 >> v2:172.21.15.117:3300/0 conn(0x7f57901537a0 0x7f579006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 -- 172.21.15.7:0/3268926329 >> v1:172.21.15.7:6789/0 conn(0x7f57900c64e0 legacy=0x7f57900cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 -- 172.21.15.7:0/3268926329 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900c4480 con 0x7f579006b740 2023-06-09T11:10:49.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578d59a700 1 -- 172.21.15.7:0/3268926329 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428917646 0 0) 0x7f5784001110 con 0x7f57900c64e0 2023-06-09T11:10:49.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.907+0000 7f578ed9d700 1 --2- 172.21.15.7:0/3268926329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f579015b2b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd130 tx=0x7f578800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678c5662bec7126e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f578d59a700 1 -- 172.21.15.7:0/3268926329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800e070 con 0x7f579006b740 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f578d59a700 1 -- 172.21.15.7:0/3268926329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578800b9c0 con 0x7f579006b740 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f578d59a700 1 -- 172.21.15.7:0/3268926329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800bd70 con 0x7f579006b740 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 -- 172.21.15.7:0/3268926329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 msgr2=0x7f579015b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 --2- 172.21.15.7:0/3268926329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f579015b2b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57900cd130 tx=0x7f578800b330 comp rx=0 tx=0).stop 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 -- 172.21.15.7:0/3268926329 shutdown_connections 2023-06-09T11:10:49.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 --2- 172.21.15.7:0/3268926329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f579015b2b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 --2- 172.21.15.7:0/3268926329 >> v2:172.21.15.117:3300/0 conn(0x7f57901537a0 0x7f579006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 -- 172.21.15.7:0/3268926329 >> 172.21.15.7:0/3268926329 conn(0x7f5790072040 msgr2=0x7f5790072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 -- 172.21.15.7:0/3268926329 shutdown_connections 2023-06-09T11:10:49.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.908+0000 7f5794bff700 1 -- 172.21.15.7:0/3268926329 wait complete. 2023-06-09T11:10:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.911+0000 7f5794bff700 1 Processor -- start 2023-06-09T11:10:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.911+0000 7f5794bff700 1 -- start start 2023-06-09T11:10:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c64e0 0x7f57900ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57901537a0 0x7f57901498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790150530 con 0x7f579006b740 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579014fc90 con 0x7f57901537a0 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f5794bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579014ff70 con 0x7f57900c64e0 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f578e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f578e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34264/0 (socket says 172.21.15.7:34264) 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.912+0000 7f578e59c700 1 -- 172.21.15.7:0/1169651774 learned_addr learned my addr 172.21.15.7:0/1169651774 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578ed9d700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57901537a0 0x7f57901498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578dd9b700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c64e0 0x7f57900ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 -- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c64e0 msgr2=0x7f57900ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c64e0 0x7f57900ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 -- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57901537a0 msgr2=0x7f57901498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57901537a0 0x7f57901498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 -- 172.21.15.7:0/1169651774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009b30 con 0x7f579006b740 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f578e59c700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f578000e810 tx=0x7f578000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780013610 con 0x7f579006b740 2023-06-09T11:10:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780013770 con 0x7f579006b740 2023-06-09T11:10:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57900c46d0 con 0x7f579006b740 2023-06-09T11:10:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780013b20 con 0x7f579006b740 2023-06-09T11:10:49.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.913+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579015ae20 con 0x7f579006b740 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.914+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57900c49b0 con 0x7f579006b740 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.916+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5780013c80 con 0x7f579006b740 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.917+0000 7f577effd700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 0x7f577006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.917+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f57800895d0 con 0x7f579006b740 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.917+0000 7f578dd9b700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 0x7f577006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:49.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.917+0000 7f578dd9b700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 0x7f577006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f579014ad80 tx=0x7f5784007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:49.920+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f57800580d0 con 0x7f579006b740 2023-06-09T11:10:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.177+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5790074cf0 con 0x7f579006b740 2023-06-09T11:10:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.184+0000 7f577effd700 1 -- 172.21.15.7:0/1169651774 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5780057c60 con 0x7f579006b740 2023-06-09T11:10:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.189+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 msgr2=0x7f577006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.189+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 0x7f577006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f579014ad80 tx=0x7f5784007a40 comp rx=0 tx=0).stop 2023-06-09T11:10:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.189+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 msgr2=0x7f57901f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.189+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f578000e810 tx=0x7f578000eb20 comp rx=0 tx=0).stop 2023-06-09T11:10:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 shutdown_connections 2023-06-09T11:10:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57901537a0 0x7f57901498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f579006b740 0x7f57901f26c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f577006ace0 0x7f577006d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 --2- 172.21.15.7:0/1169651774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57900c64e0 0x7f57900ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 >> 172.21.15.7:0/1169651774 conn(0x7f5790072040 msgr2=0x7f57900cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.194+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 shutdown_connections 2023-06-09T11:10:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.195+0000 7f5794bff700 1 -- 172.21.15.7:0/1169651774 wait complete. 2023-06-09T11:10:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 3 and data pool 4 2023-06-09T11:10:50.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:50.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.519+0000 7faf62dff700 1 Processor -- start 2023-06-09T11:10:50.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.519+0000 7faf62dff700 1 -- start start 2023-06-09T11:10:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf62dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf5c153760 0x7faf5c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf62dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153b30 0x7faf5c0b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf62dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5c15a390 con 0x7faf5c071fa0 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf62dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5c0c5f40 con 0x7faf5c153760 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf62dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0ca720 con 0x7faf5c153b30 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf60b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf5c153760 0x7faf5c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf60b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf5c153760 0x7faf5c0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34268/0 (socket says 172.21.15.7:34268) 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf60b9b700 1 -- 172.21.15.7:0/3531700023 learned_addr learned my addr 172.21.15.7:0/3531700023 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.520+0000 7faf6139c700 1 --1- 172.21.15.7:0/3531700023 >> v1:172.21.15.7:6789/0 conn(0x7faf5c071fa0 0x7faf5c15cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59436/0 (socket says 172.21.15.7:59436) 2023-06-09T11:10:50.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf5bfff700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153b30 0x7faf5c0b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf60b9b700 1 -- 172.21.15.7:0/3531700023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153b30 msgr2=0x7faf5c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf60b9b700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153b30 0x7faf5c0b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf60b9b700 1 -- 172.21.15.7:0/3531700023 >> v1:172.21.15.7:6789/0 conn(0x7faf5c071fa0 legacy=0x7faf5c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf60b9b700 1 -- 172.21.15.7:0/3531700023 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0c5a20 con 0x7faf5c153760 2023-06-09T11:10:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf60b9b700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c153760 0x7faf5c0ca070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf500098f0 tx=0x7faf50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c596b01560797 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf5b7fe700 1 -- 172.21.15.7:0/3531700023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000b8a0 con 0x7faf5c153760 2023-06-09T11:10:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf5b7fe700 1 -- 172.21.15.7:0/3531700023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000ba00 con 0x7faf5c153760 2023-06-09T11:10:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.521+0000 7faf5b7fe700 1 -- 172.21.15.7:0/3531700023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000bd90 con 0x7faf5c153760 2023-06-09T11:10:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 -- 172.21.15.7:0/3531700023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c153760 msgr2=0x7faf5c0ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c153760 0x7faf5c0ca070 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf500098f0 tx=0x7faf50009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 -- 172.21.15.7:0/3531700023 shutdown_connections 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c153760 0x7faf5c0ca070 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 --2- 172.21.15.7:0/3531700023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153b30 0x7faf5c0b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 -- 172.21.15.7:0/3531700023 >> 172.21.15.7:0/3531700023 conn(0x7faf5c0737f0 msgr2=0x7faf5c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 -- 172.21.15.7:0/3531700023 shutdown_connections 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.522+0000 7faf62dff700 1 -- 172.21.15.7:0/3531700023 wait complete. 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.524+0000 7faf62dff700 1 Processor -- start 2023-06-09T11:10:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.524+0000 7faf62dff700 1 -- start start 2023-06-09T11:10:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153760 0x7faf5c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf5c153b30 0x7faf5c14ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b3440 con 0x7faf5c071fa0 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b3690 con 0x7faf5c153b30 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf62dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5c0b3970 con 0x7faf5c153760 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34272/0 (socket says 172.21.15.7:34272) 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 -- 172.21.15.7:0/4064829715 learned_addr learned my addr 172.21.15.7:0/4064829715 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf5bfff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153760 0x7faf5c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf6139c700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf5c153b30 0x7faf5c14ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 -- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153760 msgr2=0x7faf5c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153760 0x7faf5c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 -- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf5c153b30 msgr2=0x7faf5c14ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf5c153b30 0x7faf5c14ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.525+0000 7faf60b9b700 1 -- 172.21.15.7:0/4064829715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf50009550 con 0x7faf5c071fa0 2023-06-09T11:10:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf60b9b700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf50000c00 tx=0x7faf50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50004480 con 0x7faf5c071fa0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5001f070 con 0x7faf5c071fa0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5c159a60 con 0x7faf5c071fa0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf500038a0 con 0x7faf5c071fa0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.526+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5c159780 con 0x7faf5c071fa0 2023-06-09T11:10:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.527+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5c159d40 con 0x7faf5c071fa0 2023-06-09T11:10:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.528+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf50021030 con 0x7faf5c071fa0 2023-06-09T11:10:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.529+0000 7faf597fa700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 0x7faf4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.529+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf5008aaf0 con 0x7faf5c071fa0 2023-06-09T11:10:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.530+0000 7faf5bfff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 0x7faf4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.530+0000 7faf5bfff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 0x7faf4006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf5c150ce0 tx=0x7faf48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.533+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faf50055c20 con 0x7faf5c071fa0 2023-06-09T11:10:50.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.744+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf5c074cf0 con 0x7faf5c071fa0 2023-06-09T11:10:50.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.746+0000 7faf597fa700 1 -- 172.21.15.7:0/4064829715 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+18196 (secure 0 0 0) 0x7faf50059240 con 0x7faf5c071fa0 2023-06-09T11:10:50.751 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:50.752 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:50.178722+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 msgr2=0x7faf4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 0x7faf4006d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf5c150ce0 tx=0x7faf48008040 comp rx=0 tx=0).stop 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 msgr2=0x7faf5c14bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf50000c00 tx=0x7faf50004750 comp rx=0 tx=0).stop 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 shutdown_connections 2023-06-09T11:10:50.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf5c153b30 0x7faf5c14ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf5c071fa0 0x7faf5c14bde0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf4006ace0 0x7faf4006d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 --2- 172.21.15.7:0/4064829715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf5c153760 0x7faf5c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 >> 172.21.15.7:0/4064829715 conn(0x7faf5c0737f0 msgr2=0x7faf5c15cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 shutdown_connections 2023-06-09T11:10:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.748+0000 7faf62dff700 1 -- 172.21.15.7:0/4064829715 wait complete. 2023-06-09T11:10:50.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.965+0000 7f0b597ff700 1 Processor -- start 2023-06-09T11:10:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.965+0000 7f0b597ff700 1 -- start start 2023-06-09T11:10:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b597ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b54153880 0x7f0b5415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b597ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 0x7f0b5415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b597ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5415a500 con 0x7f0b5415b2d0 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b597ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540c5f10 con 0x7f0b54153880 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b597ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5415da00 con 0x7f0b540c9f40 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.966+0000 7f0b52ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b54153880 0x7f0b5415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b54153880 0x7f0b5415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34288/0 (socket says 172.21.15.7:34288) 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 -- 172.21.15.7:0/325030712 learned_addr learned my addr 172.21.15.7:0/325030712 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b527fc700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 0x7f0b5415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 -- 172.21.15.7:0/325030712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 msgr2=0x7f0b5415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 0x7f0b5415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 -- 172.21.15.7:0/325030712 >> v1:172.21.15.117:6789/0 conn(0x7f0b5415b2d0 legacy=0x7f0b540c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 -- 172.21.15.7:0/325030712 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5415b750 con 0x7f0b54153880 2023-06-09T11:10:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b527fc700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 0x7f0b5415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b52ffd700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b54153880 0x7f0b5415d4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b440098f0 tx=0x7f0b44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832692f9265e390c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b51ffb700 1 -- 172.21.15.7:0/325030712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44004230 con 0x7f0b54153880 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b51ffb700 1 -- 172.21.15.7:0/325030712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44004390 con 0x7f0b54153880 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.967+0000 7f0b51ffb700 1 -- 172.21.15.7:0/325030712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440036a0 con 0x7f0b54153880 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 -- 172.21.15.7:0/325030712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b54153880 msgr2=0x7f0b5415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b54153880 0x7f0b5415d4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b440098f0 tx=0x7f0b44009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 -- 172.21.15.7:0/325030712 shutdown_connections 2023-06-09T11:10:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b54153880 0x7f0b5415d4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 --2- 172.21.15.7:0/325030712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b540c9f40 0x7f0b5415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 -- 172.21.15.7:0/325030712 >> 172.21.15.7:0/325030712 conn(0x7f0b540737f0 msgr2=0x7f0b540720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 -- 172.21.15.7:0/325030712 shutdown_connections 2023-06-09T11:10:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.968+0000 7f0b597ff700 1 -- 172.21.15.7:0/325030712 wait complete. 2023-06-09T11:10:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.970+0000 7f0b597ff700 1 Processor -- start 2023-06-09T11:10:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.970+0000 7f0b597ff700 1 -- start start 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 0x7f0b541f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 0x7f0b540b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b541ef990 con 0x7f0b540c9f40 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b4930 con 0x7f0b5415b2d0 2023-06-09T11:10:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b597ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b541ef0c0 con 0x7f0b54153880 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b52ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b52ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34298/0 (socket says 172.21.15.7:34298) 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.971+0000 7f0b52ffd700 1 -- 172.21.15.7:0/2905181186 learned_addr learned my addr 172.21.15.7:0/2905181186 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b527fc700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 0x7f0b541f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b537fe700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 0x7f0b540b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 -- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 msgr2=0x7f0b541f3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 0x7f0b541f3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 -- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 msgr2=0x7f0b540b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 0x7f0b540b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 -- 172.21.15.7:0/2905181186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b44009550 con 0x7f0b540c9f40 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b537fe700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 0x7f0b540b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b527fc700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 0x7f0b541f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b52ffd700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406b580 tx=0x7f0b44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44003e30 con 0x7f0b540c9f40 2023-06-09T11:10:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4401f070 con 0x7f0b540c9f40 2023-06-09T11:10:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54159980 con 0x7f0b540c9f40 2023-06-09T11:10:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400b8a0 con 0x7f0b540c9f40 2023-06-09T11:10:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.972+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b541596e0 con 0x7f0b540c9f40 2023-06-09T11:10:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.974+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54159c60 con 0x7f0b540c9f40 2023-06-09T11:10:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.974+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b44021030 con 0x7f0b540c9f40 2023-06-09T11:10:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.975+0000 7f0b3b7fe700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 0x7f0b4006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.975+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b4408a9c0 con 0x7f0b540c9f40 2023-06-09T11:10:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.976+0000 7f0b527fc700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 0x7f0b4006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.976+0000 7f0b527fc700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 0x7f0b4006d1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b54152d00 tx=0x7f0b48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:50.980+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0b44055b00 con 0x7f0b540c9f40 2023-06-09T11:10:51.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.189+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b541efc00 con 0x7f0b540c9f40 2023-06-09T11:10:51.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.190+0000 7f0b3b7fe700 1 -- 172.21.15.7:0/2905181186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f0b44010380 con 0x7f0b540c9f40 2023-06-09T11:10:51.195 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:51.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.192+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 msgr2=0x7f0b4006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.192+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 0x7f0b4006d1c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0b54152d00 tx=0x7f0b48008040 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 msgr2=0x7f0b540b43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b5406b580 tx=0x7f0b44004750 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 shutdown_connections 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b5415b2d0 0x7f0b540b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b540c9f40 0x7f0b540b43f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4006ad10 0x7f0b4006d1c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 --2- 172.21.15.7:0/2905181186 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b54153880 0x7f0b541f3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 >> 172.21.15.7:0/2905181186 conn(0x7f0b540737f0 msgr2=0x7f0b5415c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:51.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 shutdown_connections 2023-06-09T11:10:51.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.193+0000 7f0b597ff700 1 -- 172.21.15.7:0/2905181186 wait complete. 2023-06-09T11:10:51.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-09T11:10:51.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.414+0000 7f9f997ff700 1 Processor -- start 2023-06-09T11:10:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.414+0000 7f9f997ff700 1 -- start start 2023-06-09T11:10:51.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f997ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9f941537a0 0x7f9f9406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f997ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940c6220 con 0x7f9f940c64e0 2023-06-09T11:10:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f997ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940c4480 con 0x7f9f9406b740 2023-06-09T11:10:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f997ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f940c4f70 con 0x7f9f941537a0 2023-06-09T11:10:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f927fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9f940c64e0 0x7f9f940cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59446/0 (socket says 172.21.15.7:59446) 2023-06-09T11:10:51.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.415+0000 7f9f927fc700 1 -- 172.21.15.7:0/2905008048 learned_addr learned my addr 172.21.15.7:0/2905008048 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/2905008048 >> v2:172.21.15.117:3300/0 conn(0x7f9f941537a0 0x7f9f9406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845242171 0 0) 0x7f9f88001110 con 0x7f9f940c64e0 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f7c0035b0 con 0x7f9f940c64e0 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815316253 0 0) 0x7f9f7c0035b0 con 0x7f9f940c64e0 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 >> v2:172.21.15.117:3300/0 conn(0x7f9f941537a0 msgr2=0x7f9f9406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 --2- 172.21.15.7:0/2905008048 >> v2:172.21.15.117:3300/0 conn(0x7f9f941537a0 0x7f9f9406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 >> v1:172.21.15.117:6789/0 conn(0x7f9f9406b740 legacy=0x7f9f9415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9415ae20 con 0x7f9f940c64e0 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f9f88001800 con 0x7f9f940c64e0 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 >> v1:172.21.15.7:6789/0 conn(0x7f9f940c64e0 legacy=0x7f9f940cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c003a50 0x7f9f7c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c004850 0x7f9f7c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f7c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940c4f70 con 0x7f9f7c005200 2023-06-09T11:10:51.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c00c0c0 con 0x7f9f7c004850 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c00c280 con 0x7f9f7c003a50 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f927fc700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c004850 0x7f9f7c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.416+0000 7f9f937fe700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f7c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c003a50 0x7f9f7c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 -- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c003a50 msgr2=0x7f9f7c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c003a50 0x7f9f7c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 -- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c004850 msgr2=0x7f9f7c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:51.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f7c004850 0x7f9f7c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 -- 172.21.15.7:0/2905008048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7c0035b0 con 0x7f9f7c005200 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f7c003a50 0x7f9f7c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 -- 172.21.15.7:0/2905008048 reap_dead start 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f937fe700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f7c00ba10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c190 tx=0x7f9f7c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d54bdba37965baf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c001db0 con 0x7f9f7c005200 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c00ccf0 con 0x7f9f7c005200 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.417+0000 7f9f91ffb700 1 -- 172.21.15.7:0/2905008048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003c40 con 0x7f9f7c005200 2023-06-09T11:10:51.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 -- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 msgr2=0x7f9f7c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f7c00ba10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c190 tx=0x7f9f7c005880 comp rx=0 tx=0).stop 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 -- 172.21.15.7:0/2905008048 shutdown_connections 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 --2- 172.21.15.7:0/2905008048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f7c00ba10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 -- 172.21.15.7:0/2905008048 >> 172.21.15.7:0/2905008048 conn(0x7f9f94072040 msgr2=0x7f9f94072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 -- 172.21.15.7:0/2905008048 shutdown_connections 2023-06-09T11:10:51.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.418+0000 7f9f997ff700 1 -- 172.21.15.7:0/2905008048 wait complete. 2023-06-09T11:10:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.420+0000 7f9f997ff700 1 Processor -- start 2023-06-09T11:10:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.420+0000 7f9f997ff700 1 -- start start 2023-06-09T11:10:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 0x7f9f94146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f94156b00 0x7f9f94147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940caea0 con 0x7f9f7c005200 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f940cb260 con 0x7f9f940c6900 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f997ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f941f2cc0 con 0x7f9f94156b00 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34318/0 (socket says 172.21.15.7:34318) 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 -- 172.21.15.7:0/141268481 learned_addr learned my addr 172.21.15.7:0/141268481 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:51.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f927fc700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 0x7f9f94146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f937fe700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f94156b00 0x7f9f94147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 -- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f94156b00 msgr2=0x7f9f94147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f94156b00 0x7f9f94147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 -- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 msgr2=0x7f9f94146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 0x7f9f94146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f92ffd700 1 -- 172.21.15.7:0/141268481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940c49b0 con 0x7f9f7c005200 2023-06-09T11:10:51.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.421+0000 7f9f927fc700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 0x7f9f94146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f92ffd700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f84011cd0 tx=0x7f9f8400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8400fb30 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8400fc90 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940c46d0 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004020 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.422+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f940c4c90 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.423+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f94074cf0 con 0x7f9f7c005200 2023-06-09T11:10:51.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.424+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f84004180 con 0x7f9f7c005200 2023-06-09T11:10:51.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.426+0000 7f9f7b7fe700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 0x7f9f8006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:51.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.426+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f84089a90 con 0x7f9f7c005200 2023-06-09T11:10:51.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.426+0000 7f9f927fc700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 0x7f9f8006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:51.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.426+0000 7f9f927fc700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 0x7f9f8006d1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f880012b0 tx=0x7f9f88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:51.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.430+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9f84058530 con 0x7f9f7c005200 2023-06-09T11:10:51.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:51.685+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9f9414ed70 con 0x7f9f7c005200 2023-06-09T11:10:52.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.666+0000 7f9f7b7fe700 1 -- 172.21.15.7:0/141268481 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9f84022020 con 0x7f9f7c005200 2023-06-09T11:10:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 msgr2=0x7f9f8006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 0x7f9f8006d1c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f880012b0 tx=0x7f9f88009040 comp rx=0 tx=0).stop 2023-06-09T11:10:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 msgr2=0x7f9f941f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:52.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9f84011cd0 tx=0x7f9f8400f520 comp rx=0 tx=0).stop 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 shutdown_connections 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9f8006ad10 0x7f9f8006d1c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9f94156b00 0x7f9f94147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f7c005200 0x7f9f941f2320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 --2- 172.21.15.7:0/141268481 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9f940c6900 0x7f9f94146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 >> 172.21.15.7:0/141268481 conn(0x7f9f94072040 msgr2=0x7f9f9414f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.669+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 shutdown_connections 2023-06-09T11:10:52.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.670+0000 7f9f997ff700 1 -- 172.21.15.7:0/141268481 wait complete. 2023-06-09T11:10:52.695 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300} 2023-06-09T11:10:52.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-06-09T11:10:52.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.897+0000 7fb0451ff700 1 Processor -- start 2023-06-09T11:10:52.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.897+0000 7fb0451ff700 1 -- start start 2023-06-09T11:10:52.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.898+0000 7fb0451ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb0451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0400c64e0 0x7fb0400cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb0451ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb04006b740 0x7fb0400cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb0451ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0400c4f70 con 0x7fb04006b740 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb0451ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0400c5c20 con 0x7fb0401537a0 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb0451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0400c6220 con 0x7fb0400c64e0 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb04006b740 0x7fb0400cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03ed9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03ed9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:59140/0 (socket says 172.21.15.7:59140) 2023-06-09T11:10:52.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03ed9d700 1 -- 172.21.15.7:0/1431211549 learned_addr learned my addr 172.21.15.7:0/1431211549 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03e59c700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0400c64e0 0x7fb0400cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 -- 172.21.15.7:0/1431211549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0400c64e0 msgr2=0x7fb0400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0400c64e0 0x7fb0400cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 -- 172.21.15.7:0/1431211549 >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 msgr2=0x7fb04006b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 --2- 172.21.15.7:0/1431211549 >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03f59e700 1 -- 172.21.15.7:0/1431211549 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0400c4480 con 0x7fb04006b740 2023-06-09T11:10:52.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.899+0000 7fb03ed9d700 1 --2- 172.21.15.7:0/1431211549 >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:52.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb03f59e700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cfad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0400cd130 tx=0x7fb03800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8935684bfdd16241 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:52.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb03dd9b700 1 -- 172.21.15.7:0/1431211549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03800e070 con 0x7fb04006b740 2023-06-09T11:10:52.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb03dd9b700 1 -- 172.21.15.7:0/1431211549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03800bb20 con 0x7fb04006b740 2023-06-09T11:10:52.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb03dd9b700 1 -- 172.21.15.7:0/1431211549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038012430 con 0x7fb04006b740 2023-06-09T11:10:52.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 -- 172.21.15.7:0/1431211549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 msgr2=0x7fb0400cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cfad0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb0400cd130 tx=0x7fb03800b330 comp rx=0 tx=0).stop 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 -- 172.21.15.7:0/1431211549 shutdown_connections 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1431211549 >> v2:172.21.15.117:3300/0 conn(0x7fb0401537a0 0x7fb04006b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cfad0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1431211549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0400c64e0 0x7fb0400cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.900+0000 7fb0451ff700 1 -- 172.21.15.7:0/1431211549 >> 172.21.15.7:0/1431211549 conn(0x7fb040072040 msgr2=0x7fb040072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.901+0000 7fb0451ff700 1 -- 172.21.15.7:0/1431211549 shutdown_connections 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.901+0000 7fb0451ff700 1 -- 172.21.15.7:0/1431211549 wait complete. 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.902+0000 7fb0451ff700 1 Processor -- start 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.902+0000 7fb0451ff700 1 -- start start 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0400c64e0 0x7fb0401e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 0x7fb0400cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401e3460 con 0x7fb04006b740 2023-06-09T11:10:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401e3740 con 0x7fb0400c64e0 2023-06-09T11:10:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.903+0000 7fb0451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0401eec70 con 0x7fb0401537a0 2023-06-09T11:10:52.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34338/0 (socket says 172.21.15.7:34338) 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 -- 172.21.15.7:0/1246301216 learned_addr learned my addr 172.21.15.7:0/1246301216 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03e59c700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0400c64e0 0x7fb0401e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03f59e700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 0x7fb0400cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 -- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 msgr2=0x7fb0400cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 0x7fb0400cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 -- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0400c64e0 msgr2=0x7fb0401e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0400c64e0 0x7fb0401e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 -- 172.21.15.7:0/1246301216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030009550 con 0x7fb04006b740 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03f59e700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 0x7fb0400cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb03ed9d700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9b0 tx=0x7fb03000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.904+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000c850 con 0x7fb04006b740 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.905+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03000c9b0 con 0x7fb04006b740 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.905+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03000cd20 con 0x7fb04006b740 2023-06-09T11:10:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.905+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb038009a70 con 0x7fb04006b740 2023-06-09T11:10:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.905+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0400c46d0 con 0x7fb04006b740 2023-06-09T11:10:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.906+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04015ae20 con 0x7fb04006b740 2023-06-09T11:10:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.907+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb030013470 con 0x7fb04006b740 2023-06-09T11:10:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.908+0000 7fb0277fe700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 0x7fb02c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.908+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb030089550 con 0x7fb04006b740 2023-06-09T11:10:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.908+0000 7fb03e59c700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 0x7fb02c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.909+0000 7fb03e59c700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 0x7fb02c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0401ec750 tx=0x7fb034009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:52.912+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb030054560 con 0x7fb04006b740 2023-06-09T11:10:53.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.161+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7fb0401ef880 con 0x7fb04006b740 2023-06-09T11:10:53.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.669+0000 7fb0277fe700 1 -- 172.21.15.7:0/1246301216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fb030057b80 con 0x7fb04006b740 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 msgr2=0x7fb02c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 0x7fb02c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb0401ec750 tx=0x7fb034009210 comp rx=0 tx=0).stop 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 msgr2=0x7fb0400cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb03000e9b0 tx=0x7fb03000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 shutdown_connections 2023-06-09T11:10:53.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02c06ace0 0x7fb02c06d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0401537a0 0x7fb0400cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb04006b740 0x7fb0400cf280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 --2- 172.21.15.7:0/1246301216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0400c64e0 0x7fb0401e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 >> 172.21.15.7:0/1246301216 conn(0x7fb040072040 msgr2=0x7fb040152990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 shutdown_connections 2023-06-09T11:10:53.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.673+0000 7fb0451ff700 1 -- 172.21.15.7:0/1246301216 wait complete. 2023-06-09T11:10:53.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-06-09T11:10:53.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.901+0000 7fa48d3ff700 1 Processor -- start 2023-06-09T11:10:53.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.901+0000 7fa48d3ff700 1 -- start start 2023-06-09T11:10:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.902+0000 7fa48d3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 0x7fa48806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.902+0000 7fa48d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.902+0000 7fa48d3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4880c6220 con 0x7fa4881537a0 2023-06-09T11:10:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.902+0000 7fa48d3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4880c4f70 con 0x7fa4880c64e0 2023-06-09T11:10:53.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.902+0000 7fa48d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4880c5c20 con 0x7fa48806b740 2023-06-09T11:10:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4867fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 0x7fa48806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47494/0 (socket says 172.21.15.7:47494) 2023-06-09T11:10:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4867fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 0x7fa48806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41076/0 (socket says 172.21.15.7:41076) 2023-06-09T11:10:53.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 -- 172.21.15.7:0/3667170460 learned_addr learned my addr 172.21.15.7:0/3667170460 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa486ffd700 1 --1- 172.21.15.7:0/3667170460 >> v1:172.21.15.117:6789/0 conn(0x7fa4881537a0 0x7fa4880cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34678/0 (socket says 172.21.15.7:34678) 2023-06-09T11:10:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 -- 172.21.15.7:0/3667170460 >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 msgr2=0x7fa48806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 --2- 172.21.15.7:0/3667170460 >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 0x7fa48806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 -- 172.21.15.7:0/3667170460 >> v1:172.21.15.117:6789/0 conn(0x7fa4881537a0 legacy=0x7fa4880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 -- 172.21.15.7:0/3667170460 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4880c4480 con 0x7fa48806b740 2023-06-09T11:10:53.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.903+0000 7fa4877fe700 1 --2- 172.21.15.7:0/3667170460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa480009e50 tx=0x7fa48000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcdedf85250cbdc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:53.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa485ffb700 1 -- 172.21.15.7:0/3667170460 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000e070 con 0x7fa48806b740 2023-06-09T11:10:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa485ffb700 1 -- 172.21.15.7:0/3667170460 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48000b910 con 0x7fa48806b740 2023-06-09T11:10:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa485ffb700 1 -- 172.21.15.7:0/3667170460 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48000bcc0 con 0x7fa48806b740 2023-06-09T11:10:53.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 -- 172.21.15.7:0/3667170460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 msgr2=0x7fa48815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/3667170460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa480009e50 tx=0x7fa48000b330 comp rx=0 tx=0).stop 2023-06-09T11:10:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 -- 172.21.15.7:0/3667170460 shutdown_connections 2023-06-09T11:10:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/3667170460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa48806b740 0x7fa48815b2b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/3667170460 >> v2:172.21.15.117:3300/0 conn(0x7fa4880c64e0 0x7fa48806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 -- 172.21.15.7:0/3667170460 >> 172.21.15.7:0/3667170460 conn(0x7fa488072040 msgr2=0x7fa488072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 -- 172.21.15.7:0/3667170460 shutdown_connections 2023-06-09T11:10:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.904+0000 7fa48d3ff700 1 -- 172.21.15.7:0/3667170460 wait complete. 2023-06-09T11:10:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.906+0000 7fa48d3ff700 1 Processor -- start 2023-06-09T11:10:53.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.906+0000 7fa48d3ff700 1 -- start start 2023-06-09T11:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48806b740 0x7fa488155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880c64e0 0x7fa488156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa488155cb0 con 0x7fa4881537a0 2023-06-09T11:10:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4881e1a30 con 0x7fa48806b740 2023-06-09T11:10:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa48d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4881e0960 con 0x7fa4880c64e0 2023-06-09T11:10:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa4877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa4877fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36340/0 (socket says 172.21.15.7:36340) 2023-06-09T11:10:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.907+0000 7fa4877fe700 1 -- 172.21.15.7:0/1418949112 learned_addr learned my addr 172.21.15.7:0/1418949112 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa486ffd700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48806b740 0x7fa488155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4867fc700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880c64e0 0x7fa488156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 -- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880c64e0 msgr2=0x7fa488156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880c64e0 0x7fa488156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 -- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48806b740 msgr2=0x7fa488155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:53.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48806b740 0x7fa488155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:53.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 -- 172.21.15.7:0/1418949112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa480009b30 con 0x7fa4881537a0 2023-06-09T11:10:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa4877fe700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa488074a10 tx=0x7fa480004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:53.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa480004960 con 0x7fa4881537a0 2023-06-09T11:10:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4800099b0 con 0x7fa4881537a0 2023-06-09T11:10:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4880c46d0 con 0x7fa4881537a0 2023-06-09T11:10:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48001b670 con 0x7fa4881537a0 2023-06-09T11:10:53.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.908+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48815ae20 con 0x7fa4881537a0 2023-06-09T11:10:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.909+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4880c49b0 con 0x7fa4881537a0 2023-06-09T11:10:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.910+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa48001b7d0 con 0x7fa4881537a0 2023-06-09T11:10:53.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.911+0000 7fa46f7fe700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.911+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa48008aaf0 con 0x7fa4881537a0 2023-06-09T11:10:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.911+0000 7fa486ffd700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.912+0000 7fa486ffd700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 0x7fa47406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa478000f60 tx=0x7fa478007d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:53.916+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa480055b00 con 0x7fa4881537a0 2023-06-09T11:10:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:54.169+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa488074cf0 con 0x7fa4881537a0 2023-06-09T11:10:55.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.119+0000 7fa46f7fe700 1 -- 172.21.15.7:0/1418949112 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa480059120 con 0x7fa4881537a0 2023-06-09T11:10:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 msgr2=0x7fa47406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 0x7fa47406d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa478000f60 tx=0x7fa478007d10 comp rx=0 tx=0).stop 2023-06-09T11:10:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 msgr2=0x7fa488156ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa488074a10 tx=0x7fa480004ad0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 shutdown_connections 2023-06-09T11:10:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4881537a0 0x7fa488156ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa48806b740 0x7fa488155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa47406ace0 0x7fa47406d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 --2- 172.21.15.7:0/1418949112 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4880c64e0 0x7fa488156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 >> 172.21.15.7:0/1418949112 conn(0x7fa488072040 msgr2=0x7fa4880cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:55.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 shutdown_connections 2023-06-09T11:10:55.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.122+0000 7fa48d3ff700 1 -- 172.21.15.7:0/1418949112 wait complete. 2023-06-09T11:10:55.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-06-09T11:10:55.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.355+0000 7f84a05ff700 1 Processor -- start 2023-06-09T11:10:55.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.355+0000 7f84a05ff700 1 -- start start 2023-06-09T11:10:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f84a05ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84981537a0 0x7f849806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f84a05ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f84980c64e0 0x7f84980cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f84a05ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84980c6220 con 0x7f849806b740 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f84a05ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980c4f70 con 0x7f84981537a0 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f84a05ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84980c5c20 con 0x7f84980c64e0 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f849e39b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84981537a0 0x7f849806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f849eb9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f849806b740 0x7f849815b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49880/0 (socket says 172.21.15.7:49880) 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.356+0000 7f849eb9c700 1 -- 172.21.15.7:0/969536721 learned_addr learned my addr 172.21.15.7:0/969536721 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:55.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849db9a700 1 --2- 172.21.15.7:0/969536721 >> v2:172.21.15.117:3300/0 conn(0x7f84980c64e0 0x7f84980cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849e39b700 1 -- 172.21.15.7:0/969536721 >> v2:172.21.15.117:3300/0 conn(0x7f84980c64e0 msgr2=0x7f84980cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849e39b700 1 --2- 172.21.15.7:0/969536721 >> v2:172.21.15.117:3300/0 conn(0x7f84980c64e0 0x7f84980cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849e39b700 1 -- 172.21.15.7:0/969536721 >> v1:172.21.15.7:6789/0 conn(0x7f849806b740 legacy=0x7f849815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849e39b700 1 -- 172.21.15.7:0/969536721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980c4480 con 0x7f84981537a0 2023-06-09T11:10:55.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849d399700 1 -- 172.21.15.7:0/969536721 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576887340 0 0) 0x7f8490001490 con 0x7f849806b740 2023-06-09T11:10:55.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849e39b700 1 --2- 172.21.15.7:0/969536721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f849806b1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84980cd0b0 tx=0x7f8494009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153b7d6b9462a84 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849d399700 1 -- 172.21.15.7:0/969536721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011830 con 0x7f84981537a0 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849d399700 1 -- 172.21.15.7:0/969536721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8494011990 con 0x7f84981537a0 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.357+0000 7f849d399700 1 -- 172.21.15.7:0/969536721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494011d20 con 0x7f84981537a0 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 -- 172.21.15.7:0/969536721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 msgr2=0x7f849806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 --2- 172.21.15.7:0/969536721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f849806b1b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84980cd0b0 tx=0x7f8494009b20 comp rx=0 tx=0).stop 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 -- 172.21.15.7:0/969536721 shutdown_connections 2023-06-09T11:10:55.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 --2- 172.21.15.7:0/969536721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f849806b1b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 --2- 172.21.15.7:0/969536721 >> v2:172.21.15.117:3300/0 conn(0x7f84980c64e0 0x7f84980cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 -- 172.21.15.7:0/969536721 >> 172.21.15.7:0/969536721 conn(0x7f8498072040 msgr2=0x7f8498072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 -- 172.21.15.7:0/969536721 shutdown_connections 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.358+0000 7f84a05ff700 1 -- 172.21.15.7:0/969536721 wait complete. 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.360+0000 7f84a05ff700 1 Processor -- start 2023-06-09T11:10:55.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.360+0000 7f84a05ff700 1 -- start start 2023-06-09T11:10:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849806b740 0x7f8498149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84980c64e0 0x7f84981e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84981e4a00 con 0x7f84981537a0 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84980c7bf0 con 0x7f849806b740 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f84a05ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8498149e50 con 0x7f84980c64e0 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f849eb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f849eb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36364/0 (socket says 172.21.15.7:36364) 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f849eb9c700 1 -- 172.21.15.7:0/4117776507 learned_addr learned my addr 172.21.15.7:0/4117776507 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:55.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f849e39b700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849806b740 0x7f8498149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.361+0000 7f849db9a700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84980c64e0 0x7f84981e3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 -- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84980c64e0 msgr2=0x7f84981e3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84980c64e0 0x7f84981e3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 -- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849806b740 msgr2=0x7f8498149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849806b740 0x7f8498149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 -- 172.21.15.7:0/4117776507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940094a0 con 0x7f84981537a0 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f849eb9c700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f849000cbd0 tx=0x7f849000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:55.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849000aac0 con 0x7f84981537a0 2023-06-09T11:10:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849000ac40 con 0x7f84981537a0 2023-06-09T11:10:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84900070d0 con 0x7f84981537a0 2023-06-09T11:10:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84980c46d0 con 0x7f84981537a0 2023-06-09T11:10:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.362+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849815ae20 con 0x7f84981537a0 2023-06-09T11:10:55.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.363+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84980c49b0 con 0x7f84981537a0 2023-06-09T11:10:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.364+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8490007250 con 0x7f84981537a0 2023-06-09T11:10:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.365+0000 7f848effd700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 0x7f847c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.365+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8490088e40 con 0x7f84981537a0 2023-06-09T11:10:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.365+0000 7f849e39b700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 0x7f847c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:55.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.366+0000 7f849e39b700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 0x7f847c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84940090b0 tx=0x7f8494008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:55.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.370+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8490053dc0 con 0x7f84981537a0 2023-06-09T11:10:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:55.629+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f84980c6220 con 0x7f84981537a0 2023-06-09T11:10:56.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.121+0000 7f848effd700 1 -- 172.21.15.7:0/4117776507 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) v1 ==== 115+0+0 (secure 0 0 0) 0x7f84980c6220 con 0x7f84981537a0 2023-06-09T11:10:56.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 msgr2=0x7f847c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 0x7f847c06d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84940090b0 tx=0x7f8494008030 comp rx=0 tx=0).stop 2023-06-09T11:10:56.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 msgr2=0x7f84981f2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f849000cbd0 tx=0x7f849000cee0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 shutdown_connections 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84981537a0 0x7f84981f2050 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f849806b740 0x7f8498149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f847c06ace0 0x7f847c06d190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 --2- 172.21.15.7:0/4117776507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f84980c64e0 0x7f84981e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 >> 172.21.15.7:0/4117776507 conn(0x7f8498072040 msgr2=0x7f84980b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 shutdown_connections 2023-06-09T11:10:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.124+0000 7f84a05ff700 1 -- 172.21.15.7:0/4117776507 wait complete. 2023-06-09T11:10:56.150 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-06-09T11:10:56.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:10:56.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.350+0000 7f362d5ff700 1 Processor -- start 2023-06-09T11:10:56.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.350+0000 7f362d5ff700 1 -- start start 2023-06-09T11:10:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.351+0000 7f362d5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3628153880 0x7f362815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.351+0000 7f362d5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f362815b4a0 0x7f36280c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f362d5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362815a500 con 0x7f36280c9f40 2023-06-09T11:10:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f362d5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36280c5f10 con 0x7f3628153880 2023-06-09T11:10:56.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f362d5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362815da00 con 0x7f362815b4a0 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3628153880 0x7f362815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f36267fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f36280c9f40 0x7f362815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49890/0 (socket says 172.21.15.7:49890) 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f36267fc700 1 -- 172.21.15.7:0/216884927 learned_addr learned my addr 172.21.15.7:0/216884927 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3628153880 0x7f362815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36374/0 (socket says 172.21.15.7:36374) 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f36277fe700 1 --2- 172.21.15.7:0/216884927 >> v2:172.21.15.117:3300/0 conn(0x7f362815b4a0 0x7f36280c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 -- 172.21.15.7:0/216884927 >> v2:172.21.15.117:3300/0 conn(0x7f362815b4a0 msgr2=0x7f36280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:56.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 --2- 172.21.15.7:0/216884927 >> v2:172.21.15.117:3300/0 conn(0x7f362815b4a0 0x7f36280c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 -- 172.21.15.7:0/216884927 >> v1:172.21.15.7:6789/0 conn(0x7f36280c9f40 legacy=0x7f362815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3626ffd700 1 -- 172.21.15.7:0/216884927 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36281596e0 con 0x7f3628153880 2023-06-09T11:10:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.352+0000 7f3625ffb700 1 -- 172.21.15.7:0/216884927 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419953367 0 0) 0x7f361c001110 con 0x7f36280c9f40 2023-06-09T11:10:56.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f3626ffd700 1 --2- 172.21.15.7:0/216884927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f362815d4c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36280c4640 tx=0x7f3618009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71daeaafef9bda2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f3625ffb700 1 -- 172.21.15.7:0/216884927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618011830 con 0x7f3628153880 2023-06-09T11:10:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f3625ffb700 1 -- 172.21.15.7:0/216884927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618011990 con 0x7f3628153880 2023-06-09T11:10:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f3625ffb700 1 -- 172.21.15.7:0/216884927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618011d20 con 0x7f3628153880 2023-06-09T11:10:56.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 -- 172.21.15.7:0/216884927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 msgr2=0x7f362815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 --2- 172.21.15.7:0/216884927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f362815d4c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36280c4640 tx=0x7f3618009b20 comp rx=0 tx=0).stop 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 -- 172.21.15.7:0/216884927 shutdown_connections 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 --2- 172.21.15.7:0/216884927 >> v2:172.21.15.117:3300/0 conn(0x7f362815b4a0 0x7f36280c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 --2- 172.21.15.7:0/216884927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f362815d4c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 -- 172.21.15.7:0/216884927 >> 172.21.15.7:0/216884927 conn(0x7f36280737f0 msgr2=0x7f36280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 -- 172.21.15.7:0/216884927 shutdown_connections 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.353+0000 7f362d5ff700 1 -- 172.21.15.7:0/216884927 wait complete. 2023-06-09T11:10:56.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.355+0000 7f362d5ff700 1 Processor -- start 2023-06-09T11:10:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.355+0000 7f362d5ff700 1 -- start start 2023-06-09T11:10:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36280c9f40 0x7f362814c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362815b4a0 0x7f3628148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36281565d0 con 0x7f3628153880 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362814ce40 con 0x7f36280c9f40 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f362d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3628156010 con 0x7f362815b4a0 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f36267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f36267fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36384/0 (socket says 172.21.15.7:36384) 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.356+0000 7f36267fc700 1 -- 172.21.15.7:0/424185083 learned_addr learned my addr 172.21.15.7:0/424185083 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:56.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f3626ffd700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36280c9f40 0x7f362814c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36277fe700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362815b4a0 0x7f3628148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 -- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362815b4a0 msgr2=0x7f3628148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362815b4a0 0x7f3628148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 -- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36280c9f40 msgr2=0x7f362814c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36280c9f40 0x7f362814c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 -- 172.21.15.7:0/424185083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36180094a0 con 0x7f3628153880 2023-06-09T11:10:56.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f36267fc700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c730 tx=0x7f361c00caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c004c80 con 0x7f3628153880 2023-06-09T11:10:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c004e00 con 0x7f3628153880 2023-06-09T11:10:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3628159c60 con 0x7f3628153880 2023-06-09T11:10:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c007310 con 0x7f3628153880 2023-06-09T11:10:56.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.357+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3628159980 con 0x7f3628153880 2023-06-09T11:10:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.358+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3628159f40 con 0x7f3628153880 2023-06-09T11:10:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.359+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f361c007470 con 0x7f3628153880 2023-06-09T11:10:56.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.360+0000 7f360f7fe700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 0x7f361406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.360+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f361c089090 con 0x7f3628153880 2023-06-09T11:10:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.360+0000 7f3626ffd700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 0x7f361406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.361+0000 7f3626ffd700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 0x7f361406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36180090b0 tx=0x7f3618008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.364+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f361c054140 con 0x7f3628153880 2023-06-09T11:10:56.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.617+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f362815a500 con 0x7f3628153880 2023-06-09T11:10:56.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.617+0000 7f360f7fe700 1 -- 172.21.15.7:0/424185083 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6191 (secure 0 0 0) 0x7f362815a500 con 0x7f3628153880 2023-06-09T11:10:56.624 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:56.625 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:10:56.121566+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4291,"mds_1":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:creating","state_seq":1,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:10:56.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 msgr2=0x7f361406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 0x7f361406d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f36180090b0 tx=0x7f3618008030 comp rx=0 tx=0).stop 2023-06-09T11:10:56.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 msgr2=0x7f36280cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c730 tx=0x7f361c00caf0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 shutdown_connections 2023-06-09T11:10:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361406ace0 0x7f361406d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362815b4a0 0x7f3628148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36280c9f40 0x7f362814c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 --2- 172.21.15.7:0/424185083 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3628153880 0x7f36280cbe70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.619+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 >> 172.21.15.7:0/424185083 conn(0x7f36280737f0 msgr2=0x7f36281528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.620+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 shutdown_connections 2023-06-09T11:10:56.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.620+0000 7f362d5ff700 1 -- 172.21.15.7:0/424185083 wait complete. 2023-06-09T11:10:56.631 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 9 2023-06-09T11:10:56.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.839+0000 7f22065ff700 1 Processor -- start 2023-06-09T11:10:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.839+0000 7f22065ff700 1 -- start start 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f22065ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f220015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f22065ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220015a500 con 0x7f22000c9f40 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f22065ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220015da00 con 0x7f220015b4a0 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f22065ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22000c5f10 con 0x7f2200153880 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f21ff7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f22000c9f40 0x7f220015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49904/0 (socket says 172.21.15.7:49904) 2023-06-09T11:10:56.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.840+0000 7f21ff7fe700 1 -- 172.21.15.7:0/2672665252 learned_addr learned my addr 172.21.15.7:0/2672665252 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21fffff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f220015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017797972 0 0) 0x7f21f4001110 con 0x7f22000c9f40 2023-06-09T11:10:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e80035b0 con 0x7f22000c9f40 2023-06-09T11:10:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801752537 0 0) 0x7f21f8001490 con 0x7f220015b4a0 2023-06-09T11:10:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21f4001110 con 0x7f220015b4a0 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 732812386 0 0) 0x7f21e80035b0 con 0x7f22000c9f40 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 msgr2=0x7f220015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f220015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 >> v1:172.21.15.117:6789/0 conn(0x7f220015b4a0 legacy=0x7f22000c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22001596e0 con 0x7f22000c9f40 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f21f4001800 con 0x7f22000c9f40 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 >> v1:172.21.15.7:6789/0 conn(0x7f22000c9f40 legacy=0x7f220015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e8003bc0 0x7f21e8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8004850 0x7f21e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f21e8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000c5f10 con 0x7f21e8003bc0 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e800baf0 con 0x7f21e8004850 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.841+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e800bcb0 con 0x7f2200153880 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e8003bc0 0x7f21e8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21ff7fe700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8004850 0x7f21e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f2204b9c700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f21e8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 -- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 msgr2=0x7f21e8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200153880 0x7f21e8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 -- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8004850 msgr2=0x7f21e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8004850 0x7f21e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 -- 172.21.15.7:0/2672665252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80035b0 con 0x7f21e8003bc0 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21fffff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e8003bc0 0x7f21e8003fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009b60 tx=0x7f21e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0495db72050f87 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000e6c0 con 0x7f21e8003bc0 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f000e820 con 0x7f21e8003bc0 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.842+0000 7f21feffd700 1 -- 172.21.15.7:0/2672665252 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f000eb90 con 0x7f21e8003bc0 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 -- 172.21.15.7:0/2672665252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e8003bc0 msgr2=0x7f21e8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 --2- 172.21.15.7:0/2672665252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e8003bc0 0x7f21e8003fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009b60 tx=0x7f21e8005880 comp rx=0 tx=0).stop 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f2204b9c700 1 -- 172.21.15.7:0/2672665252 reap_dead start 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 -- 172.21.15.7:0/2672665252 shutdown_connections 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 -- 172.21.15.7:0/2672665252 >> 172.21.15.7:0/2672665252 conn(0x7f22000737f0 msgr2=0x7f22000720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 -- 172.21.15.7:0/2672665252 shutdown_connections 2023-06-09T11:10:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.843+0000 7f22065ff700 1 -- 172.21.15.7:0/2672665252 wait complete. 2023-06-09T11:10:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.845+0000 7f22065ff700 1 Processor -- start 2023-06-09T11:10:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.845+0000 7f22065ff700 1 -- start start 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8003bc0 0x7f22000b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200154930 0x7f220015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22001ef310 con 0x7f22000cc550 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22000b33f0 con 0x7f21e8003bc0 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f22065ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2200156470 con 0x7f2200154930 2023-06-09T11:10:56.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f2204b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f2204b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36408/0 (socket says 172.21.15.7:36408) 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f2204b9c700 1 -- 172.21.15.7:0/742425968 learned_addr learned my addr 172.21.15.7:0/742425968 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f21ff7fe700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200154930 0x7f220015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.846+0000 7f21fffff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8003bc0 0x7f22000b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 -- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200154930 msgr2=0x7f220015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200154930 0x7f220015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 -- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8003bc0 msgr2=0x7f22000b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8003bc0 0x7f22000b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 -- 172.21.15.7:0/742425968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200159c60 con 0x7f22000cc550 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f2204b9c700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21f8001330 tx=0x7f21f800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8003c10 con 0x7f22000cc550 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8003d90 con 0x7f22000cc550 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8005310 con 0x7f22000cc550 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2200159980 con 0x7f22000cc550 2023-06-09T11:10:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.847+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2200159f40 con 0x7f22000cc550 2023-06-09T11:10:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.848+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22001ef5b0 con 0x7f22000cc550 2023-06-09T11:10:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.849+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21f8005470 con 0x7f22000cc550 2023-06-09T11:10:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.850+0000 7f21fcff9700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 0x7f21ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.850+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21f80893c0 con 0x7f22000cc550 2023-06-09T11:10:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.851+0000 7f21fffff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 0x7f21ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.851+0000 7f21fffff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 0x7f21ec06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21f00090b0 tx=0x7f21f001f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:56.855+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f21f8054340 con 0x7f22000cc550 2023-06-09T11:10:57.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.067+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f220015da00 con 0x7f22000cc550 2023-06-09T11:10:57.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.068+0000 7f21fcff9700 1 -- 172.21.15.7:0/742425968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f220015da00 con 0x7f22000cc550 2023-06-09T11:10:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:57.074 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 msgr2=0x7f21ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 0x7f21ec06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21f00090b0 tx=0x7f21f001f040 comp rx=0 tx=0).stop 2023-06-09T11:10:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 msgr2=0x7f220014d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f21f8001330 tx=0x7f21f800ee90 comp rx=0 tx=0).stop 2023-06-09T11:10:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 shutdown_connections 2023-06-09T11:10:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22000cc550 0x7f220014d060 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21e8003bc0 0x7f22000b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f21ec06ace0 0x7f21ec06d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 --2- 172.21.15.7:0/742425968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2200154930 0x7f220015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 >> 172.21.15.7:0/742425968 conn(0x7f22000737f0 msgr2=0x7f22000cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 shutdown_connections 2023-06-09T11:10:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.071+0000 7f22065ff700 1 -- 172.21.15.7:0/742425968 wait complete. 2023-06-09T11:10:57.094 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-09T11:10:57.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:10:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.290+0000 7fadfa9ff700 1 Processor -- start 2023-06-09T11:10:57.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.290+0000 7fadfa9ff700 1 -- start start 2023-06-09T11:10:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadfa9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fadf40c1bc0 0x7fadf41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadfa9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40c0bf0 con 0x7fadf40c0140 2023-06-09T11:10:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadfa9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40702f0 con 0x7fadf40c1f90 2023-06-09T11:10:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadfa9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf4070110 con 0x7fadf40c1bc0 2023-06-09T11:10:57.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadf37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fadf40c0140 0x7fadf41501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49908/0 (socket says 172.21.15.7:49908) 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadf37fe700 1 -- 172.21.15.7:0/3171411697 learned_addr learned my addr 172.21.15.7:0/3171411697 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadf3fff700 1 --2- 172.21.15.7:0/3171411697 >> v2:172.21.15.117:3300/0 conn(0x7fadf40c1bc0 0x7fadf41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953180775 0 0) 0x7fadf40c0bf0 con 0x7fadf40c0140 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.292+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faddc0035b0 con 0x7fadf40c0140 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1311295120 0 0) 0x7faddc0035b0 con 0x7fadf40c0140 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 >> v2:172.21.15.117:3300/0 conn(0x7fadf40c1bc0 msgr2=0x7fadf41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:57.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 --2- 172.21.15.7:0/3171411697 >> v2:172.21.15.117:3300/0 conn(0x7fadf40c1bc0 0x7fadf41599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 >> v1:172.21.15.117:6789/0 conn(0x7fadf40c1f90 legacy=0x7fadf4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40c0e90 con 0x7fadf40c0140 2023-06-09T11:10:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fade8001670 con 0x7fadf40c0140 2023-06-09T11:10:57.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 >> v1:172.21.15.7:6789/0 conn(0x7fadf40c0140 legacy=0x7fadf41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faddc003a00 0x7faddc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc004850 0x7faddc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7faddc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4070110 con 0x7faddc005200 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc00c080 con 0x7faddc004850 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc00c240 con 0x7faddc003a00 2023-06-09T11:10:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf37fe700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc004850 0x7faddc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf3fff700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faddc003a00 0x7faddc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.293+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7faddc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 -- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faddc003a00 msgr2=0x7faddc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faddc003a00 0x7faddc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 -- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc004850 msgr2=0x7faddc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faddc004850 0x7faddc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 -- 172.21.15.7:0/3171411697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0035b0 con 0x7faddc005200 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 -- 172.21.15.7:0/3171411697 reap_dead start 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7faddc00ba10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c190 tx=0x7faddc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1e5510da71e9f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec001db0 con 0x7faddc005200 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec00ccd0 con 0x7faddc005200 2023-06-09T11:10:57.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadf2ffd700 1 -- 172.21.15.7:0/3171411697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec003c40 con 0x7faddc005200 2023-06-09T11:10:57.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 msgr2=0x7faddc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.294+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7faddc00ba10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadec00c190 tx=0x7faddc005880 comp rx=0 tx=0).stop 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.295+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3171411697 shutdown_connections 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.295+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3171411697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7faddc00ba10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.295+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3171411697 >> 172.21.15.7:0/3171411697 conn(0x7fadf406a4d0 msgr2=0x7fadf406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.295+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3171411697 shutdown_connections 2023-06-09T11:10:57.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.295+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3171411697 wait complete. 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.297+0000 7fadfa9ff700 1 Processor -- start 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.297+0000 7fadfa9ff700 1 -- start start 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4151f80 0x7fadf40cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 0x7fadf41eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf41528e0 con 0x7faddc005200 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf40afe20 con 0x7fadf4151f80 2023-06-09T11:10:57.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadfa9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4153ca0 con 0x7fadf414bc90 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36430/0 (socket says 172.21.15.7:36430) 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 -- 172.21.15.7:0/3989951883 learned_addr learned my addr 172.21.15.7:0/3989951883 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 0x7fadf41eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf37fe700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4151f80 0x7fadf40cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 -- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 msgr2=0x7fadf41eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 0x7fadf41eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 -- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4151f80 msgr2=0x7fadf40cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4151f80 0x7fadf40cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 -- 172.21.15.7:0/3989951883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf40c1450 con 0x7faddc005200 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf8f9c700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 0x7fadf41eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.299+0000 7fadf3fff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fade40119a0 tx=0x7fade4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.300+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400f940 con 0x7faddc005200 2023-06-09T11:10:57.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.300+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade400faa0 con 0x7faddc005200 2023-06-09T11:10:57.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.300+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade400fe30 con 0x7faddc005200 2023-06-09T11:10:57.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.300+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf40c1170 con 0x7faddc005200 2023-06-09T11:10:57.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.300+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf40705d0 con 0x7faddc005200 2023-06-09T11:10:57.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.301+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf41eebb0 con 0x7faddc005200 2023-06-09T11:10:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.302+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fade4014440 con 0x7faddc005200 2023-06-09T11:10:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.303+0000 7fadf0ff9700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 0x7fade006d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.303+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fade4089be0 con 0x7faddc005200 2023-06-09T11:10:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.303+0000 7fadf37fe700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 0x7fade006d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.304+0000 7fadf37fe700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 0x7fade006d120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadf41efb00 tx=0x7fade8000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.308+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fade4054d30 con 0x7faddc005200 2023-06-09T11:10:57.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.510+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadf4074d60 con 0x7fade006ac70 2023-06-09T11:10:57.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.520+0000 7fadf0ff9700 1 -- 172.21.15.7:0/3989951883 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286705 (secure 0 0 0) 0x7fadf4074d60 con 0x7fade006ac70 2023-06-09T11:10:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.524+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 msgr2=0x7fade006d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.524+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 0x7fade006d120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fadf41efb00 tx=0x7fade8000ed0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 msgr2=0x7fadf4153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fade40119a0 tx=0x7fade4011cb0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 shutdown_connections 2023-06-09T11:10:57.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fade006ac70 0x7fade006d120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadf414bc90 0x7fadf41eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faddc005200 0x7fadf4153760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 --2- 172.21.15.7:0/3989951883 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadf4151f80 0x7fadf40cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 >> 172.21.15.7:0/3989951883 conn(0x7fadf406a4d0 msgr2=0x7fadf4071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 shutdown_connections 2023-06-09T11:10:57.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.525+0000 7fadfa9ff700 1 -- 172.21.15.7:0/3989951883 wait complete. 2023-06-09T11:10:57.532 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:10:57.548 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-06-09T11:10:56.189262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67612,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":65856,"kb_avail":754907108,"statfs":{"total":773094113280,"available":773024878592,"internally_reserved":0,"allocated":1732608,"data_stored":1756137,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67436544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.534791"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185867+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:10:51.185867+0000","last_peered":"2023-06-09T11:10:51.185867+0000","last_clean":"2023-06-09T11:10:51.185867+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:10:51.185867+0000","last_undegraded":"2023-06-09T11:10:51.185867+0000","last_fullsized":"2023-06-09T11:10:51.185867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462070+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:51.462070+0000","last_peered":"2023-06-09T11:10:51.462070+0000","last_clean":"2023-06-09T11:10:51.462070+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:51.462070+0000","last_undegraded":"2023-06-09T11:10:51.462070+0000","last_fullsized":"2023-06-09T11:10:51.462070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548111+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:10:51.548111+0000","last_peered":"2023-06-09T11:10:51.548111+0000","last_clean":"2023-06-09T11:10:51.548111+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:10:51.548111+0000","last_undegraded":"2023-06-09T11:10:51.548111+0000","last_fullsized":"2023-06-09T11:10:51.548111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689982+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:51.689982+0000","last_peered":"2023-06-09T11:10:51.689982+0000","last_clean":"2023-06-09T11:10:51.689982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:51.689982+0000","last_undegraded":"2023-06-09T11:10:51.689982+0000","last_fullsized":"2023-06-09T11:10:51.689982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190203+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:10:51.190203+0000","last_peered":"2023-06-09T11:10:51.190203+0000","last_clean":"2023-06-09T11:10:51.190203+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:10:51.190203+0000","last_undegraded":"2023-06-09T11:10:51.190203+0000","last_fullsized":"2023-06-09T11:10:51.190203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689997+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:10:51.689997+0000","last_peered":"2023-06-09T11:10:51.689997+0000","last_clean":"2023-06-09T11:10:51.689997+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:10:51.689997+0000","last_undegraded":"2023-06-09T11:10:51.689997+0000","last_fullsized":"2023-06-09T11:10:51.689997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547634+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:10:51.547634+0000","last_peered":"2023-06-09T11:10:51.547634+0000","last_clean":"2023-06-09T11:10:51.547634+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:10:51.547634+0000","last_undegraded":"2023-06-09T11:10:51.547634+0000","last_fullsized":"2023-06-09T11:10:51.547634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689705+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:10:51.689705+0000","last_peered":"2023-06-09T11:10:51.689705+0000","last_clean":"2023-06-09T11:10:51.689705+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:10:51.689705+0000","last_undegraded":"2023-06-09T11:10:51.689705+0000","last_fullsized":"2023-06-09T11:10:51.689705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690218+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:51.690218+0000","last_peered":"2023-06-09T11:10:51.690218+0000","last_clean":"2023-06-09T11:10:51.690218+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:51.690218+0000","last_undegraded":"2023-06-09T11:10:51.690218+0000","last_fullsized":"2023-06-09T11:10:51.690218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689898+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:51.689898+0000","last_peered":"2023-06-09T11:10:51.689898+0000","last_clean":"2023-06-09T11:10:51.689898+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:51.689898+0000","last_undegraded":"2023-06-09T11:10:51.689898+0000","last_fullsized":"2023-06-09T11:10:51.689898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690145+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:51.690145+0000","last_peered":"2023-06-09T11:10:51.690145+0000","last_clean":"2023-06-09T11:10:51.690145+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:51.690145+0000","last_undegraded":"2023-06-09T11:10:51.690145+0000","last_fullsized":"2023-06-09T11:10:51.690145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189143+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:10:51.189143+0000","last_peered":"2023-06-09T11:10:51.189143+0000","last_clean":"2023-06-09T11:10:51.189143+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:10:51.189143+0000","last_undegraded":"2023-06-09T11:10:51.189143+0000","last_fullsized":"2023-06-09T11:10:51.189143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462228+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:10:51.462228+0000","last_peered":"2023-06-09T11:10:51.462228+0000","last_clean":"2023-06-09T11:10:51.462228+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:10:51.462228+0000","last_undegraded":"2023-06-09T11:10:51.462228+0000","last_fullsized":"2023-06-09T11:10:51.462228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197985+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:51.197985+0000","last_peered":"2023-06-09T11:10:51.197985+0000","last_clean":"2023-06-09T11:10:51.197985+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:51.197985+0000","last_undegraded":"2023-06-09T11:10:51.197985+0000","last_fullsized":"2023-06-09T11:10:51.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189298+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:10:51.189298+0000","last_peered":"2023-06-09T11:10:51.189298+0000","last_clean":"2023-06-09T11:10:51.189298+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:10:51.189298+0000","last_undegraded":"2023-06-09T11:10:51.189298+0000","last_fullsized":"2023-06-09T11:10:51.189298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461738+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:10:51.461738+0000","last_peered":"2023-06-09T11:10:51.461738+0000","last_clean":"2023-06-09T11:10:51.461738+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:10:51.461738+0000","last_undegraded":"2023-06-09T11:10:51.461738+0000","last_fullsized":"2023-06-09T11:10:51.461738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189545+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:51.189545+0000","last_peered":"2023-06-09T11:10:51.189545+0000","last_clean":"2023-06-09T11:10:51.189545+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:51.189545+0000","last_undegraded":"2023-06-09T11:10:51.189545+0000","last_fullsized":"2023-06-09T11:10:51.189545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189440+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:10:51.189440+0000","last_peered":"2023-06-09T11:10:51.189440+0000","last_clean":"2023-06-09T11:10:51.189440+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:10:51.189440+0000","last_undegraded":"2023-06-09T11:10:51.189440+0000","last_fullsized":"2023-06-09T11:10:51.189440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185737+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:10:51.185737+0000","last_peered":"2023-06-09T11:10:51.185737+0000","last_clean":"2023-06-09T11:10:51.185737+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:10:51.185737+0000","last_undegraded":"2023-06-09T11:10:51.185737+0000","last_fullsized":"2023-06-09T11:10:51.185737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461810+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:51.461810+0000","last_peered":"2023-06-09T11:10:51.461810+0000","last_clean":"2023-06-09T11:10:51.461810+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:51.461810+0000","last_undegraded":"2023-06-09T11:10:51.461810+0000","last_fullsized":"2023-06-09T11:10:51.461810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547703+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:10:51.547703+0000","last_peered":"2023-06-09T11:10:51.547703+0000","last_clean":"2023-06-09T11:10:51.547703+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:10:51.547703+0000","last_undegraded":"2023-06-09T11:10:51.547703+0000","last_fullsized":"2023-06-09T11:10:51.547703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462280+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:10:51.462280+0000","last_peered":"2023-06-09T11:10:51.462280+0000","last_clean":"2023-06-09T11:10:51.462280+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:10:51.462280+0000","last_undegraded":"2023-06-09T11:10:51.462280+0000","last_fullsized":"2023-06-09T11:10:51.462280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548150+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:10:51.548150+0000","last_peered":"2023-06-09T11:10:51.548150+0000","last_clean":"2023-06-09T11:10:51.548150+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:10:51.548150+0000","last_undegraded":"2023-06-09T11:10:51.548150+0000","last_fullsized":"2023-06-09T11:10:51.548150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547815+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:10:51.547815+0000","last_peered":"2023-06-09T11:10:51.547815+0000","last_clean":"2023-06-09T11:10:51.547815+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:10:51.547815+0000","last_undegraded":"2023-06-09T11:10:51.547815+0000","last_fullsized":"2023-06-09T11:10:51.547815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188999+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:51.188999+0000","last_peered":"2023-06-09T11:10:51.188999+0000","last_clean":"2023-06-09T11:10:51.188999+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:51.188999+0000","last_undegraded":"2023-06-09T11:10:51.188999+0000","last_fullsized":"2023-06-09T11:10:51.188999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189595+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:10:51.189595+0000","last_peered":"2023-06-09T11:10:51.189595+0000","last_clean":"2023-06-09T11:10:51.189595+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:10:51.189595+0000","last_undegraded":"2023-06-09T11:10:51.189595+0000","last_fullsized":"2023-06-09T11:10:51.189595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462046+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:51.462046+0000","last_peered":"2023-06-09T11:10:51.462046+0000","last_clean":"2023-06-09T11:10:51.462046+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:51.462046+0000","last_undegraded":"2023-06-09T11:10:51.462046+0000","last_fullsized":"2023-06-09T11:10:51.462046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548187+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:10:51.548187+0000","last_peered":"2023-06-09T11:10:51.548187+0000","last_clean":"2023-06-09T11:10:51.548187+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:10:51.548187+0000","last_undegraded":"2023-06-09T11:10:51.548187+0000","last_fullsized":"2023-06-09T11:10:51.548187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94363004,"statfs":{"total":96636764160,"available":96627716096,"internally_reserved":0,"allocated":258048,"data_stored":658135,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8912,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362928,"statfs":{"total":96636764160,"available":96627638272,"internally_reserved":0,"allocated":270336,"data_stored":664788,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":192512,"data_stored":67767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8324,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363516,"statfs":{"total":96636764160,"available":96628240384,"internally_reserved":0,"allocated":192512,"data_stored":67767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:10:57.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:10:57.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.785+0000 7f1ec31ff700 1 Processor -- start 2023-06-09T11:10:57.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.785+0000 7f1ec31ff700 1 -- start start 2023-06-09T11:10:57.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.786+0000 7f1ec31ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ebc15b4a0 0x7f1ebc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec31ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ebc0c9f40 0x7f1ebc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec31ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc15a500 con 0x7f1ebc153880 2023-06-09T11:10:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec31ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0c5f10 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec31ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc15da00 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec0f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ebc153880 0x7f1ebc15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49920/0 (socket says 172.21.15.7:49920) 2023-06-09T11:10:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec0f9b700 1 -- 172.21.15.7:0/2361040425 learned_addr learned my addr 172.21.15.7:0/2361040425 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:57.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec179c700 1 --2- 172.21.15.7:0/2361040425 >> v2:172.21.15.7:3300/0 conn(0x7f1ebc0c9f40 0x7f1ebc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ebbfff700 1 --2- 172.21.15.7:0/2361040425 >> v2:172.21.15.117:3300/0 conn(0x7f1ebc15b4a0 0x7f1ebc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ebb7fe700 1 -- 172.21.15.7:0/2361040425 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967624952 0 0) 0x7f1eb0001110 con 0x7f1ebc153880 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ebb7fe700 1 -- 172.21.15.7:0/2361040425 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eac0035b0 con 0x7f1ebc153880 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec179c700 1 -- 172.21.15.7:0/2361040425 >> v2:172.21.15.117:3300/0 conn(0x7f1ebc15b4a0 msgr2=0x7f1ebc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec179c700 1 --2- 172.21.15.7:0/2361040425 >> v2:172.21.15.117:3300/0 conn(0x7f1ebc15b4a0 0x7f1ebc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec179c700 1 -- 172.21.15.7:0/2361040425 >> v1:172.21.15.7:6789/0 conn(0x7f1ebc153880 legacy=0x7f1ebc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.787+0000 7f1ec179c700 1 -- 172.21.15.7:0/2361040425 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc1596e0 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec179c700 1 --2- 172.21.15.7:0/2361040425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc0c9f40 0x7f1ebc0c4e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0cce30 tx=0x7f1eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e4ce607fc263d9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ebb7fe700 1 -- 172.21.15.7:0/2361040425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400e070 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ebb7fe700 1 -- 172.21.15.7:0/2361040425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb400bb20 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ebb7fe700 1 -- 172.21.15.7:0/2361040425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb400be90 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 -- 172.21.15.7:0/2361040425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc0c9f40 msgr2=0x7f1ebc0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/2361040425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc0c9f40 0x7f1ebc0c4e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0cce30 tx=0x7f1eb400b330 comp rx=0 tx=0).stop 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 -- 172.21.15.7:0/2361040425 shutdown_connections 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/2361040425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc0c9f40 0x7f1ebc0c4e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/2361040425 >> v2:172.21.15.117:3300/0 conn(0x7f1ebc15b4a0 0x7f1ebc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 -- 172.21.15.7:0/2361040425 >> 172.21.15.7:0/2361040425 conn(0x7f1ebc0737f0 msgr2=0x7f1ebc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:57.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.788+0000 7f1ec31ff700 1 -- 172.21.15.7:0/2361040425 shutdown_connections 2023-06-09T11:10:57.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.789+0000 7f1ec31ff700 1 -- 172.21.15.7:0/2361040425 wait complete. 2023-06-09T11:10:57.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.791+0000 7f1ec31ff700 1 Processor -- start 2023-06-09T11:10:57.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.791+0000 7f1ec31ff700 1 -- start start 2023-06-09T11:10:57.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ebc0c9f40 0x7f1ebc1f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ebc153880 0x7f1ebc0cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc159370 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1f28d0 con 0x7f1ebc153880 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.792+0000 7f1ec31ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1f32a0 con 0x7f1ebc0c9f40 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36454/0 (socket says 172.21.15.7:36454) 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 -- 172.21.15.7:0/1007489118 learned_addr learned my addr 172.21.15.7:0/1007489118 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ebbfff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ebc153880 0x7f1ebc0cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec0f9b700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ebc0c9f40 0x7f1ebc1f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 -- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ebc0c9f40 msgr2=0x7f1ebc1f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ebc0c9f40 0x7f1ebc1f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 -- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ebc153880 msgr2=0x7f1ebc0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ebc153880 0x7f1ebc0cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 -- 172.21.15.7:0/1007489118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009a70 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1ec179c700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b370 tx=0x7f1eb4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4007000 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4012650 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.793+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4012a00 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.794+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc159c60 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.794+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc159980 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.795+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc159f40 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.795+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1eb4012b60 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.797+0000 7f1eb97fa700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 0x7f1ea006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:57.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.798+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1eb408b230 con 0x7f1ebc15b4a0 2023-06-09T11:10:57.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.798+0000 7f1ec0f9b700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 0x7f1ea006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:57.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.798+0000 7f1ec0f9b700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 0x7f1ea006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007fd0 tx=0x7f1eb0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:57.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:57.802+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1eb40564a0 con 0x7f1ebc15b4a0 2023-06-09T11:10:58.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.006+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ebc0d04d0 con 0x7f1ea006ace0 2023-06-09T11:10:58.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.014+0000 7f1eb97fa700 1 -- 172.21.15.7:0/1007489118 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286705 (secure 0 0 0) 0x7f1ebc0d04d0 con 0x7f1ea006ace0 2023-06-09T11:10:58.022 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 msgr2=0x7f1ea006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 0x7f1ea006d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0007fd0 tx=0x7f1eb0009040 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 msgr2=0x7f1ebc1e5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b370 tx=0x7f1eb4004ad0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 shutdown_connections 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc15b4a0 0x7f1ebc1e5af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ea006ace0 0x7f1ea006d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ebc0c9f40 0x7f1ebc1f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 --2- 172.21.15.7:0/1007489118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ebc153880 0x7f1ebc0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 >> 172.21.15.7:0/1007489118 conn(0x7f1ebc0737f0 msgr2=0x7f1ebc15b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 shutdown_connections 2023-06-09T11:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.019+0000 7f1ec31ff700 1 -- 172.21.15.7:0/1007489118 wait complete. 2023-06-09T11:10:58.026 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:10:58.042 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-06-09T11:10:56.189262+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67612,"kb_used_data":1692,"kb_used_omap":0,"kb_used_meta":65856,"kb_avail":754907108,"statfs":{"total":773094113280,"available":773024878592,"internally_reserved":0,"allocated":1732608,"data_stored":1756137,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67436544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.534791"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185867+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:10:51.185867+0000","last_peered":"2023-06-09T11:10:51.185867+0000","last_clean":"2023-06-09T11:10:51.185867+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:10:51.185867+0000","last_undegraded":"2023-06-09T11:10:51.185867+0000","last_fullsized":"2023-06-09T11:10:51.185867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462070+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:51.462070+0000","last_peered":"2023-06-09T11:10:51.462070+0000","last_clean":"2023-06-09T11:10:51.462070+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:51.462070+0000","last_undegraded":"2023-06-09T11:10:51.462070+0000","last_fullsized":"2023-06-09T11:10:51.462070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548111+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:10:51.548111+0000","last_peered":"2023-06-09T11:10:51.548111+0000","last_clean":"2023-06-09T11:10:51.548111+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:10:51.548111+0000","last_undegraded":"2023-06-09T11:10:51.548111+0000","last_fullsized":"2023-06-09T11:10:51.548111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689982+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:51.689982+0000","last_peered":"2023-06-09T11:10:51.689982+0000","last_clean":"2023-06-09T11:10:51.689982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:51.689982+0000","last_undegraded":"2023-06-09T11:10:51.689982+0000","last_fullsized":"2023-06-09T11:10:51.689982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190203+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:10:51.190203+0000","last_peered":"2023-06-09T11:10:51.190203+0000","last_clean":"2023-06-09T11:10:51.190203+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:10:51.190203+0000","last_undegraded":"2023-06-09T11:10:51.190203+0000","last_fullsized":"2023-06-09T11:10:51.190203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689997+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:10:51.689997+0000","last_peered":"2023-06-09T11:10:51.689997+0000","last_clean":"2023-06-09T11:10:51.689997+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:10:51.689997+0000","last_undegraded":"2023-06-09T11:10:51.689997+0000","last_fullsized":"2023-06-09T11:10:51.689997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547634+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:10:51.547634+0000","last_peered":"2023-06-09T11:10:51.547634+0000","last_clean":"2023-06-09T11:10:51.547634+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:10:51.547634+0000","last_undegraded":"2023-06-09T11:10:51.547634+0000","last_fullsized":"2023-06-09T11:10:51.547634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689705+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:10:51.689705+0000","last_peered":"2023-06-09T11:10:51.689705+0000","last_clean":"2023-06-09T11:10:51.689705+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:10:51.689705+0000","last_undegraded":"2023-06-09T11:10:51.689705+0000","last_fullsized":"2023-06-09T11:10:51.689705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690218+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:51.690218+0000","last_peered":"2023-06-09T11:10:51.690218+0000","last_clean":"2023-06-09T11:10:51.690218+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:51.690218+0000","last_undegraded":"2023-06-09T11:10:51.690218+0000","last_fullsized":"2023-06-09T11:10:51.690218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689898+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:51.689898+0000","last_peered":"2023-06-09T11:10:51.689898+0000","last_clean":"2023-06-09T11:10:51.689898+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:51.689898+0000","last_undegraded":"2023-06-09T11:10:51.689898+0000","last_fullsized":"2023-06-09T11:10:51.689898+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690145+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:51.690145+0000","last_peered":"2023-06-09T11:10:51.690145+0000","last_clean":"2023-06-09T11:10:51.690145+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:51.690145+0000","last_undegraded":"2023-06-09T11:10:51.690145+0000","last_fullsized":"2023-06-09T11:10:51.690145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189143+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:10:51.189143+0000","last_peered":"2023-06-09T11:10:51.189143+0000","last_clean":"2023-06-09T11:10:51.189143+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:10:51.189143+0000","last_undegraded":"2023-06-09T11:10:51.189143+0000","last_fullsized":"2023-06-09T11:10:51.189143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462228+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:10:51.462228+0000","last_peered":"2023-06-09T11:10:51.462228+0000","last_clean":"2023-06-09T11:10:51.462228+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:10:51.462228+0000","last_undegraded":"2023-06-09T11:10:51.462228+0000","last_fullsized":"2023-06-09T11:10:51.462228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197985+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:51.197985+0000","last_peered":"2023-06-09T11:10:51.197985+0000","last_clean":"2023-06-09T11:10:51.197985+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:51.197985+0000","last_undegraded":"2023-06-09T11:10:51.197985+0000","last_fullsized":"2023-06-09T11:10:51.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189298+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:10:51.189298+0000","last_peered":"2023-06-09T11:10:51.189298+0000","last_clean":"2023-06-09T11:10:51.189298+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:10:51.189298+0000","last_undegraded":"2023-06-09T11:10:51.189298+0000","last_fullsized":"2023-06-09T11:10:51.189298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461738+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:10:51.461738+0000","last_peered":"2023-06-09T11:10:51.461738+0000","last_clean":"2023-06-09T11:10:51.461738+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:10:51.461738+0000","last_undegraded":"2023-06-09T11:10:51.461738+0000","last_fullsized":"2023-06-09T11:10:51.461738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189545+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:51.189545+0000","last_peered":"2023-06-09T11:10:51.189545+0000","last_clean":"2023-06-09T11:10:51.189545+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:51.189545+0000","last_undegraded":"2023-06-09T11:10:51.189545+0000","last_fullsized":"2023-06-09T11:10:51.189545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189440+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:10:51.189440+0000","last_peered":"2023-06-09T11:10:51.189440+0000","last_clean":"2023-06-09T11:10:51.189440+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:10:51.189440+0000","last_undegraded":"2023-06-09T11:10:51.189440+0000","last_fullsized":"2023-06-09T11:10:51.189440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185737+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:10:51.185737+0000","last_peered":"2023-06-09T11:10:51.185737+0000","last_clean":"2023-06-09T11:10:51.185737+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:10:51.185737+0000","last_undegraded":"2023-06-09T11:10:51.185737+0000","last_fullsized":"2023-06-09T11:10:51.185737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461810+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:51.461810+0000","last_peered":"2023-06-09T11:10:51.461810+0000","last_clean":"2023-06-09T11:10:51.461810+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:51.461810+0000","last_undegraded":"2023-06-09T11:10:51.461810+0000","last_fullsized":"2023-06-09T11:10:51.461810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547703+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:10:51.547703+0000","last_peered":"2023-06-09T11:10:51.547703+0000","last_clean":"2023-06-09T11:10:51.547703+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:10:51.547703+0000","last_undegraded":"2023-06-09T11:10:51.547703+0000","last_fullsized":"2023-06-09T11:10:51.547703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462280+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:10:51.462280+0000","last_peered":"2023-06-09T11:10:51.462280+0000","last_clean":"2023-06-09T11:10:51.462280+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:10:51.462280+0000","last_undegraded":"2023-06-09T11:10:51.462280+0000","last_fullsized":"2023-06-09T11:10:51.462280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548150+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:10:51.548150+0000","last_peered":"2023-06-09T11:10:51.548150+0000","last_clean":"2023-06-09T11:10:51.548150+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:10:51.548150+0000","last_undegraded":"2023-06-09T11:10:51.548150+0000","last_fullsized":"2023-06-09T11:10:51.548150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547815+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:10:51.547815+0000","last_peered":"2023-06-09T11:10:51.547815+0000","last_clean":"2023-06-09T11:10:51.547815+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:10:51.547815+0000","last_undegraded":"2023-06-09T11:10:51.547815+0000","last_fullsized":"2023-06-09T11:10:51.547815+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188999+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:51.188999+0000","last_peered":"2023-06-09T11:10:51.188999+0000","last_clean":"2023-06-09T11:10:51.188999+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:51.188999+0000","last_undegraded":"2023-06-09T11:10:51.188999+0000","last_fullsized":"2023-06-09T11:10:51.188999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189595+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:10:51.189595+0000","last_peered":"2023-06-09T11:10:51.189595+0000","last_clean":"2023-06-09T11:10:51.189595+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:10:51.189595+0000","last_undegraded":"2023-06-09T11:10:51.189595+0000","last_fullsized":"2023-06-09T11:10:51.189595+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462046+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:51.462046+0000","last_peered":"2023-06-09T11:10:51.462046+0000","last_clean":"2023-06-09T11:10:51.462046+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:51.462046+0000","last_undegraded":"2023-06-09T11:10:51.462046+0000","last_fullsized":"2023-06-09T11:10:51.462046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548187+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:10:51.548187+0000","last_peered":"2023-06-09T11:10:51.548187+0000","last_clean":"2023-06-09T11:10:51.548187+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:10:51.548187+0000","last_undegraded":"2023-06-09T11:10:51.548187+0000","last_fullsized":"2023-06-09T11:10:51.548187+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8836,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94363004,"statfs":{"total":96636764160,"available":96627716096,"internally_reserved":0,"allocated":258048,"data_stored":658135,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8912,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362928,"statfs":{"total":96636764160,"available":96627638272,"internally_reserved":0,"allocated":270336,"data_stored":664788,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":192512,"data_stored":67767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8324,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363516,"statfs":{"total":96636764160,"available":96628240384,"internally_reserved":0,"allocated":192512,"data_stored":67767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1220000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:10:58.048 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-09T11:10:58.049 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-09T11:10:58.049 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-06-09T11:10:58.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-06-09T11:10:58.051 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-06-09T11:10:58.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.280+0000 7fe6289ff700 1 Processor -- start 2023-06-09T11:10:58.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.281+0000 7fe6289ff700 1 -- start start 2023-06-09T11:10:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe6289ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe62406a6b0 0x7fe62415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe6289ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 0x7fe6240b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe6289ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6240b51c0 con 0x7fe62406aa80 2023-06-09T11:10:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe6289ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62406b300 con 0x7fe62406a6b0 2023-06-09T11:10:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe6289ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6240b4180 con 0x7fe62406af00 2023-06-09T11:10:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe61bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe62406aa80 0x7fe624156960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49930/0 (socket says 172.21.15.7:49930) 2023-06-09T11:10:58.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe61bfff700 1 -- 172.21.15.7:0/3900145689 learned_addr learned my addr 172.21.15.7:0/3900145689 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:58.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe62259c700 1 --2- 172.21.15.7:0/3900145689 >> v2:172.21.15.117:3300/0 conn(0x7fe62406a6b0 0x7fe62415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe622d9d700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 0x7fe6240b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.282+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507003971 0 0) 0x7fe6240b51c0 con 0x7fe62406aa80 2023-06-09T11:10:58.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6100035b0 con 0x7fe62406aa80 2023-06-09T11:10:58.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe62259c700 1 -- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 msgr2=0x7fe6240b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe62259c700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 0x7fe6240b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe62259c700 1 -- 172.21.15.7:0/3900145689 >> v1:172.21.15.7:6789/0 conn(0x7fe62406aa80 legacy=0x7fe624156960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3738997858 0 0) 0x7fe6100035b0 con 0x7fe62406aa80 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe62259c700 1 -- 172.21.15.7:0/3900145689 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240b54a0 con 0x7fe62406a6b0 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe622d9d700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 0x7fe6240b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe62259c700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe62415a030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe614009840 tx=0x7fe614009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd771db939d68 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0017f0 con 0x7fe62406a6b0 2023-06-09T11:10:58.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe614004030 con 0x7fe62406a6b0 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.283+0000 7fe621d9b700 1 -- 172.21.15.7:0/3900145689 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6140043c0 con 0x7fe62406a6b0 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 -- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 msgr2=0x7fe62415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe62415a030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe614009840 tx=0x7fe614009b50 comp rx=0 tx=0).stop 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 -- 172.21.15.7:0/3900145689 shutdown_connections 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406af00 0x7fe6240b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 --2- 172.21.15.7:0/3900145689 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe62415a030 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 -- 172.21.15.7:0/3900145689 >> 172.21.15.7:0/3900145689 conn(0x7fe624070af0 msgr2=0x7fe624070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 -- 172.21.15.7:0/3900145689 shutdown_connections 2023-06-09T11:10:58.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.284+0000 7fe6289ff700 1 -- 172.21.15.7:0/3900145689 wait complete. 2023-06-09T11:10:58.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.286+0000 7fe6289ff700 1 Processor -- start 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.286+0000 7fe6289ff700 1 -- start start 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe6241e3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 0x7fe6241e5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6241e5d00 con 0x7fe62406af00 2023-06-09T11:10:58.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62415d4e0 con 0x7fe62406a6b0 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe6289ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62415d6d0 con 0x7fe62406aa80 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe622d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe622d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36456/0 (socket says 172.21.15.7:36456) 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe62259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe6241e3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe622d9d700 1 -- 172.21.15.7:0/834000277 learned_addr learned my addr 172.21.15.7:0/834000277 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe61bfff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 0x7fe6241e5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe622d9d700 1 -- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 msgr2=0x7fe6241e5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.287+0000 7fe622d9d700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 0x7fe6241e5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe622d9d700 1 -- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 msgr2=0x7fe6241e3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe622d9d700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe6241e3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe622d9d700 1 -- 172.21.15.7:0/834000277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c009a70 con 0x7fe62406af00 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe61bfff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 0x7fe6241e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe622d9d700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00efc0 tx=0x7fe61c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00ca90 con 0x7fe62406af00 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c00f040 con 0x7fe62406af00 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6140094a0 con 0x7fe62406af00 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00ce30 con 0x7fe62406af00 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.288+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6240ca240 con 0x7fe62406af00 2023-06-09T11:10:58.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.289+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6240c9f60 con 0x7fe62406af00 2023-06-09T11:10:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.290+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe61c014480 con 0x7fe62406af00 2023-06-09T11:10:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.291+0000 7fe61affd700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 0x7fe60406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.291+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe61c089c20 con 0x7fe62406af00 2023-06-09T11:10:58.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.291+0000 7fe62259c700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 0x7fe60406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.292+0000 7fe62259c700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 0x7fe60406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.295+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe61c054bb0 con 0x7fe62406af00 2023-06-09T11:10:58.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.563+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe6240ca520 con 0x7fe62406af00 2023-06-09T11:10:58.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.566+0000 7fe61affd700 1 -- 172.21.15.7:0/834000277 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170835 (secure 0 0 0) 0x7fe61c0581d0 con 0x7fe62406af00 2023-06-09T11:10:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 msgr2=0x7fe60406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 0x7fe60406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6140090b0 tx=0x7fe614008030 comp rx=0 tx=0).stop 2023-06-09T11:10:58.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 msgr2=0x7fe6241e57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00efc0 tx=0x7fe61c00c3a0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 shutdown_connections 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe62406af00 0x7fe6241e57c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62406a6b0 0x7fe6241e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe60406ace0 0x7fe60406d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 --2- 172.21.15.7:0/834000277 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62406aa80 0x7fe6241e5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 >> 172.21.15.7:0/834000277 conn(0x7fe624070af0 msgr2=0x7fe6240ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 shutdown_connections 2023-06-09T11:10:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.570+0000 7fe6289ff700 1 -- 172.21.15.7:0/834000277 wait complete. 2023-06-09T11:10:58.592 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2153300544},{"type":"v1","addr":"172.21.15.7:6833","nonce":2153300544}]},"active_addr":"172.21.15.7:6833/2153300544","active_change":"2023-06-09T11:10:36.174570+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":3190200300}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1076662415}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":1933848464}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.7:0","nonce":4172559323}]}]} 2023-06-09T11:10:58.597 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-06-09T11:10:58.597 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-06-09T11:10:58.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:58.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.792+0000 7efd463ff700 1 Processor -- start 2023-06-09T11:10:58.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.793+0000 7efd463ff700 1 -- start start 2023-06-09T11:10:58.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efd40153880 0x7efd4015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd400c9f40 0x7efd4015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd4015b2d0 0x7efd400c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd400c5f10 con 0x7efd400c9f40 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4015a500 con 0x7efd40153880 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd463ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4015da00 con 0x7efd4015b2d0 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd400c9f40 0x7efd4015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd400c9f40 0x7efd4015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36470/0 (socket says 172.21.15.7:36470) 2023-06-09T11:10:58.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 -- 172.21.15.7:0/4103388397 learned_addr learned my addr 172.21.15.7:0/4103388397 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:58.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd4499c700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd4015b2d0 0x7efd400c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3ffff700 1 --2- 172.21.15.7:0/4103388397 >> v2:172.21.15.117:3300/0 conn(0x7efd40153880 0x7efd4015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 -- 172.21.15.7:0/4103388397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd4015b2d0 msgr2=0x7efd400c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:58.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd4015b2d0 0x7efd400c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 -- 172.21.15.7:0/4103388397 >> v2:172.21.15.117:3300/0 conn(0x7efd40153880 msgr2=0x7efd4015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/4103388397 >> v2:172.21.15.117:3300/0 conn(0x7efd40153880 0x7efd4015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.794+0000 7efd3f7fe700 1 -- 172.21.15.7:0/4103388397 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd401596e0 con 0x7efd400c9f40 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.795+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd4015ad90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd4008d340 tx=0x7efd34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b003170475aeb689 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.795+0000 7efd3effd700 1 -- 172.21.15.7:0/4103388397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011830 con 0x7efd400c9f40 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.795+0000 7efd3effd700 1 -- 172.21.15.7:0/4103388397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34011990 con 0x7efd400c9f40 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.795+0000 7efd3effd700 1 -- 172.21.15.7:0/4103388397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34011d20 con 0x7efd400c9f40 2023-06-09T11:10:58.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 -- 172.21.15.7:0/4103388397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 msgr2=0x7efd4015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd4015ad90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7efd4008d340 tx=0x7efd34009bd0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 -- 172.21.15.7:0/4103388397 shutdown_connections 2023-06-09T11:10:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 --2- 172.21.15.7:0/4103388397 >> v2:172.21.15.117:3300/0 conn(0x7efd40153880 0x7efd4015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd4015b2d0 0x7efd400c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 --2- 172.21.15.7:0/4103388397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd4015ad90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 -- 172.21.15.7:0/4103388397 >> 172.21.15.7:0/4103388397 conn(0x7efd400737f0 msgr2=0x7efd400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:58.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 -- 172.21.15.7:0/4103388397 shutdown_connections 2023-06-09T11:10:58.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.796+0000 7efd463ff700 1 -- 172.21.15.7:0/4103388397 wait complete. 2023-06-09T11:10:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.798+0000 7efd463ff700 1 Processor -- start 2023-06-09T11:10:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.798+0000 7efd463ff700 1 -- start start 2023-06-09T11:10:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.799+0000 7efd463ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd463ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd40153880 0x7efd401ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd463ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd4015b2d0 0x7efd400b4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd463ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd400c6e40 con 0x7efd400c9f40 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd463ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd400b4500 con 0x7efd4015b2d0 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd463ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd400c78a0 con 0x7efd40153880 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36472/0 (socket says 172.21.15.7:36472) 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 -- 172.21.15.7:0/3810846515 learned_addr learned my addr 172.21.15.7:0/3810846515 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:58.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd4499c700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd4015b2d0 0x7efd400b4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd40153880 0x7efd401ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 -- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd40153880 msgr2=0x7efd401ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd40153880 0x7efd401ef230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 -- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd4015b2d0 msgr2=0x7efd400b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd4015b2d0 0x7efd400b4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.800+0000 7efd3ffff700 1 -- 172.21.15.7:0/3810846515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd34009550 con 0x7efd400c9f40 2023-06-09T11:10:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd3ffff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e930 tx=0x7efd3000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30013680 con 0x7efd400c9f40 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd300137e0 con 0x7efd400c9f40 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd40159980 con 0x7efd400c9f40 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30013b70 con 0x7efd400c9f40 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.801+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4015b730 con 0x7efd400c9f40 2023-06-09T11:10:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.802+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd40159c60 con 0x7efd400c9f40 2023-06-09T11:10:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.803+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd30013cd0 con 0x7efd400c9f40 2023-06-09T11:10:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.804+0000 7efd3cff9700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 0x7efd2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.804+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efd30089500 con 0x7efd400c9f40 2023-06-09T11:10:58.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.805+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 0x7efd2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:58.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.805+0000 7efd3f7fe700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 0x7efd2c06d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd34009c10 tx=0x7efd34008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:58.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:58.809+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efd30054480 con 0x7efd400c9f40 2023-06-09T11:10:59.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.018+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd40074cf0 con 0x7efd400c9f40 2023-06-09T11:10:59.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.019+0000 7efd3cff9700 1 -- 172.21.15.7:0/3810846515 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7efd30057aa0 con 0x7efd400c9f40 2023-06-09T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:59.024 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 msgr2=0x7efd2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 0x7efd2c06d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efd34009c10 tx=0x7efd34008030 comp rx=0 tx=0).stop 2023-06-09T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 msgr2=0x7efd400c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efd3000e930 tx=0x7efd3000ec40 comp rx=0 tx=0).stop 2023-06-09T11:10:59.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 shutdown_connections 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd4015b2d0 0x7efd400b4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd400c9f40 0x7efd400c6560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd2c06ace0 0x7efd2c06d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 --2- 172.21.15.7:0/3810846515 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd40153880 0x7efd401ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 >> 172.21.15.7:0/3810846515 conn(0x7efd400737f0 msgr2=0x7efd400cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 shutdown_connections 2023-06-09T11:10:59.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.021+0000 7efd463ff700 1 -- 172.21.15.7:0/3810846515 wait complete. 2023-06-09T11:10:59.044 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-06-09T11:10:59.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:10:59.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.240+0000 7f75345ff700 1 Processor -- start 2023-06-09T11:10:59.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.240+0000 7f75345ff700 1 -- start start 2023-06-09T11:10:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f752c153880 0x7f752c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c0c9f40 0x7f752c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752c15b2d0 0x7f752c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c0c5f10 con 0x7f752c15b2d0 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c15a500 con 0x7f752c153880 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f75345ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c15da00 con 0x7f752c0c9f40 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752c15b2d0 0x7f752c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f752c15b2d0 0x7f752c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36482/0 (socket says 172.21.15.7:36482) 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 -- 172.21.15.7:0/1743077552 learned_addr learned my addr 172.21.15.7:0/1743077552 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7531b9a700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c0c9f40 0x7f752c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f753239b700 1 --2- 172.21.15.7:0/1743077552 >> v2:172.21.15.117:3300/0 conn(0x7f752c153880 0x7f752c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 -- 172.21.15.7:0/1743077552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c0c9f40 msgr2=0x7f752c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c0c9f40 0x7f752c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 -- 172.21.15.7:0/1743077552 >> v2:172.21.15.117:3300/0 conn(0x7f752c153880 msgr2=0x7f752c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.242+0000 7f7532b9c700 1 --2- 172.21.15.7:0/1743077552 >> v2:172.21.15.117:3300/0 conn(0x7f752c153880 0x7f752c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.243+0000 7f7532b9c700 1 -- 172.21.15.7:0/1743077552 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c15b750 con 0x7f752c15b2d0 2023-06-09T11:10:59.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.243+0000 7f7532b9c700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0c4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f752c08d340 tx=0x7f752400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b09980a9994a61a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.243+0000 7f7531399700 1 -- 172.21.15.7:0/1743077552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400e070 con 0x7f752c15b2d0 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.243+0000 7f7531399700 1 -- 172.21.15.7:0/1743077552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400b910 con 0x7f752c15b2d0 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.243+0000 7f7531399700 1 -- 172.21.15.7:0/1743077552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400bcc0 con 0x7f752c15b2d0 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 -- 172.21.15.7:0/1743077552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 msgr2=0x7f752c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0c4e30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f752c08d340 tx=0x7f752400b330 comp rx=0 tx=0).stop 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 -- 172.21.15.7:0/1743077552 shutdown_connections 2023-06-09T11:10:59.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 --2- 172.21.15.7:0/1743077552 >> v2:172.21.15.117:3300/0 conn(0x7f752c153880 0x7f752c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0c4e30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 --2- 172.21.15.7:0/1743077552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c0c9f40 0x7f752c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 -- 172.21.15.7:0/1743077552 >> 172.21.15.7:0/1743077552 conn(0x7f752c0737f0 msgr2=0x7f752c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 -- 172.21.15.7:0/1743077552 shutdown_connections 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.244+0000 7f75345ff700 1 -- 172.21.15.7:0/1743077552 wait complete. 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.246+0000 7f75345ff700 1 Processor -- start 2023-06-09T11:10:59.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.246+0000 7f75345ff700 1 -- start start 2023-06-09T11:10:59.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f752c0c9f40 0x7f752c0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c153880 0x7f752c150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c152250 con 0x7f752c15b2d0 2023-06-09T11:10:59.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c1eee40 con 0x7f752c0c9f40 2023-06-09T11:10:59.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f75345ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c1efe10 con 0x7f752c153880 2023-06-09T11:10:59.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36494/0 (socket says 172.21.15.7:36494) 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 -- 172.21.15.7:0/2100735594 learned_addr learned my addr 172.21.15.7:0/2100735594 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f753239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f752c0c9f40 0x7f752c0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7531b9a700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c153880 0x7f752c150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 -- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c153880 msgr2=0x7f752c150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c153880 0x7f752c150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 -- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f752c0c9f40 msgr2=0x7f752c0cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f752c0c9f40 0x7f752c0cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 -- 172.21.15.7:0/2100735594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009b30 con 0x7f752c15b2d0 2023-06-09T11:10:59.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.247+0000 7f7532b9c700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f752c06b580 tx=0x7f7524004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.248+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75240099b0 con 0x7f752c15b2d0 2023-06-09T11:10:59.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.248+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400ce80 con 0x7f752c15b2d0 2023-06-09T11:10:59.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.248+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524004490 con 0x7f752c15b2d0 2023-06-09T11:10:59.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.248+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c159980 con 0x7f752c15b2d0 2023-06-09T11:10:59.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.248+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c1596e0 con 0x7f752c15b2d0 2023-06-09T11:10:59.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.249+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c159c60 con 0x7f752c15b2d0 2023-06-09T11:10:59.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.250+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75240045f0 con 0x7f752c15b2d0 2023-06-09T11:10:59.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.251+0000 7f7522ffd700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.251+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f752408a3b0 con 0x7f752c15b2d0 2023-06-09T11:10:59.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.251+0000 7f753239b700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.252+0000 7f753239b700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 0x7f751006d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7528009be0 tx=0x7f7528009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.256+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7524055450 con 0x7f752c15b2d0 2023-06-09T11:10:59.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.464+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f752c074cf0 con 0x7f752c15b2d0 2023-06-09T11:10:59.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.465+0000 7f7522ffd700 1 -- 172.21.15.7:0/2100735594 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f7524058a70 con 0x7f752c15b2d0 2023-06-09T11:10:59.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:10:59.471 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:10:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 msgr2=0x7f751006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 0x7f751006d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7528009be0 tx=0x7f7528009210 comp rx=0 tx=0).stop 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 msgr2=0x7f752c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f752c06b580 tx=0x7f7524004ad0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 shutdown_connections 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f752c15b2d0 0x7f752c0ce670 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f752c0c9f40 0x7f752c0cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f751006ace0 0x7f751006d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 --2- 172.21.15.7:0/2100735594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f752c153880 0x7f752c150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 >> 172.21.15.7:0/2100735594 conn(0x7f752c0737f0 msgr2=0x7f752c150780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 shutdown_connections 2023-06-09T11:10:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.468+0000 7f75345ff700 1 -- 172.21.15.7:0/2100735594 wait complete. 2023-06-09T11:10:59.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-06-09T11:10:59.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-06-09T11:10:59.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-06-09T11:10:59.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-06-09T11:10:59.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-06-09T11:10:59.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-06-09T11:10:59.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-06-09T11:10:59.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-06-09T11:10:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.857+0000 7f8f9ebff700 1 Processor -- start 2023-06-09T11:10:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.857+0000 7f8f9ebff700 1 -- start start 2023-06-09T11:10:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9814f830 0x7f8f980c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 0x7f8f98158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f981591d0 0x7f8f980cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98155720 con 0x7f8f9814f830 2023-06-09T11:10:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98159c20 con 0x7f8f981591d0 2023-06-09T11:10:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.859+0000 7f8f9ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c0bb0 con 0x7f8f981563f0 2023-06-09T11:10:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.860+0000 7f8f9d19c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f981591d0 0x7f8f980cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.860+0000 7f8f9d19c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f981591d0 0x7f8f980cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41282/0 (socket says 172.21.15.7:41282) 2023-06-09T11:10:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.860+0000 7f8f9d19c700 1 -- 172.21.15.7:0/2200911731 learned_addr learned my addr 172.21.15.7:0/2200911731 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 -- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 msgr2=0x7f8f98158c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 --2- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 0x7f8f98158c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 -- 172.21.15.7:0/2200911731 >> v2:172.21.15.7:3300/0 conn(0x7f8f9814f830 msgr2=0x7f8f980c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 --2- 172.21.15.7:0/2200911731 >> v2:172.21.15.7:3300/0 conn(0x7f8f9814f830 0x7f8f980c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 -- 172.21.15.7:0/2200911731 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980c7940 con 0x7f8f981591d0 2023-06-09T11:10:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.861+0000 7f8f9d19c700 1 --2- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 0x7f8f980cb110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f980c73e0 tx=0x7f8f9000d650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c79928ea92393083 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.862+0000 7f8f977fe700 1 -- 172.21.15.7:0/2200911731 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90007d30 con 0x7f8f981591d0 2023-06-09T11:10:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.862+0000 7f8f977fe700 1 -- 172.21.15.7:0/2200911731 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90007e90 con 0x7f8f981591d0 2023-06-09T11:10:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.862+0000 7f8f977fe700 1 -- 172.21.15.7:0/2200911731 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f900126f0 con 0x7f8f981591d0 2023-06-09T11:10:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 msgr2=0x7f8f980cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 0x7f8f980cb110 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f980c73e0 tx=0x7f8f9000d650 comp rx=0 tx=0).stop 2023-06-09T11:10:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2200911731 shutdown_connections 2023-06-09T11:10:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 0x7f8f980cb110 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2200911731 >> v2:172.21.15.7:3300/0 conn(0x7f8f9814f830 0x7f8f980c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2200911731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 0x7f8f98158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2200911731 >> 172.21.15.7:0/2200911731 conn(0x7f8f9806a4d0 msgr2=0x7f8f9806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.863+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2200911731 shutdown_connections 2023-06-09T11:10:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.864+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2200911731 wait complete. 2023-06-09T11:10:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.864+0000 7f00755ff700 1 Processor -- start 2023-06-09T11:10:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.864+0000 7f00755ff700 1 -- start start 2023-06-09T11:10:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f8f9ebff700 1 Processor -- start 2023-06-09T11:10:59.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f00755ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f00755ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00701591d0 0x7f00700cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f00755ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00700c7940 con 0x7f007014f830 2023-06-09T11:10:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f00755ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0070155720 con 0x7f00701591d0 2023-06-09T11:10:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.866+0000 7f00755ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0070159c20 con 0x7f00701563f0 2023-06-09T11:10:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.867+0000 7f8f9ebff700 1 -- start start 2023-06-09T11:10:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47654/0 (socket says 172.21.15.7:47654) 2023-06-09T11:10:59.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 -- 172.21.15.7:0/1033943528 learned_addr learned my addr 172.21.15.7:0/1033943528 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 0x7f8f98149d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 0x7f8f980c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 0x7f8f980ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c4700 con 0x7f8f9814f830 2023-06-09T11:10:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98070ed0 con 0x7f8f981591d0 2023-06-09T11:10:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7f8f9ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980c2730 con 0x7f8f981563f0 2023-06-09T11:10:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7f8f9c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 0x7f8f98149d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7f8f9c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 0x7f8f98149d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36526/0 (socket says 172.21.15.7:36526) 2023-06-09T11:10:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7f8f9c99b700 1 -- 172.21.15.7:0/2868707537 learned_addr learned my addr 172.21.15.7:0/2868707537 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 -- 172.21.15.7:0/2868707537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 msgr2=0x7f8f980c9ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:10:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f981563f0 0x7f8f980c9ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 -- 172.21.15.7:0/2868707537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 msgr2=0x7f8f980ca530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:10:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f981591d0 0x7f8f980ca530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9000d050 con 0x7f8f9814f830 2023-06-09T11:10:59.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f8f9c99b700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 0x7f8f98149d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00b560 tx=0x7f8f8c00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 -- 172.21.15.7:0/1033943528 >> v2:172.21.15.7:3300/0 conn(0x7f00701591d0 msgr2=0x7f00700cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 --2- 172.21.15.7:0/1033943528 >> v2:172.21.15.7:3300/0 conn(0x7f00701591d0 0x7f00700cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 -- 172.21.15.7:0/1033943528 >> v1:172.21.15.7:6789/0 conn(0x7f007014f830 legacy=0x7f00700c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7f006e7fc700 1 -- 172.21.15.7:0/1033943528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070159660 con 0x7f00701563f0 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7f006e7fc700 1 --2- 172.21.15.7:0/1033943528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f00600088f0 tx=0x7f006000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9ee06d9ec8aaf3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f006dffb700 1 -- 172.21.15.7:0/1033943528 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0060010070 con 0x7f00701563f0 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f006dffb700 1 -- 172.21.15.7:0/1033943528 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006000da40 con 0x7f00701563f0 2023-06-09T11:10:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f006dffb700 1 -- 172.21.15.7:0/1033943528 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ddd0 con 0x7f00701563f0 2023-06-09T11:10:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c011830 con 0x7f8f9814f830 2023-06-09T11:10:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c011990 con 0x7f8f9814f830 2023-06-09T11:10:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c011d40 con 0x7f8f9814f830 2023-06-09T11:10:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98159940 con 0x7f8f9814f830 2023-06-09T11:10:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98159660 con 0x7f8f9814f830 2023-06-09T11:10:59.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7f00755ff700 1 -- 172.21.15.7:0/1033943528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 msgr2=0x7f0070158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7f00755ff700 1 --2- 172.21.15.7:0/1033943528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f00600088f0 tx=0x7f006000d330 comp rx=0 tx=0).stop 2023-06-09T11:10:59.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f98155910 con 0x7f8f9814f830 2023-06-09T11:10:59.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 -- 172.21.15.7:0/1033943528 shutdown_connections 2023-06-09T11:10:59.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 --2- 172.21.15.7:0/1033943528 >> v2:172.21.15.7:3300/0 conn(0x7f00701591d0 0x7f00700cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 --2- 172.21.15.7:0/1033943528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f0070158c90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 -- 172.21.15.7:0/1033943528 >> 172.21.15.7:0/1033943528 conn(0x7f007006a4d0 msgr2=0x7f007006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7fef6fbff700 1 Processor -- start 2023-06-09T11:10:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.869+0000 7fef6fbff700 1 -- start start 2023-06-09T11:10:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 -- 172.21.15.7:0/1033943528 shutdown_connections 2023-06-09T11:10:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7fef6fbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fef680c7510 0x7fef680c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7fef6fbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef680c6dc0 0x7fef6815db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7f00755ff700 1 -- 172.21.15.7:0/1033943528 wait complete. 2023-06-09T11:10:59.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7fef6fbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef680c85f0 con 0x7fef68157c70 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7fef6fbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680c71c0 con 0x7fef680c6dc0 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7fef6fbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef680cb1f0 con 0x7fef680c7510 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7fef6e19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef680c6dc0 0x7fef6815db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7fef6e19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fef680c6dc0 0x7fef6815db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36546/0 (socket says 172.21.15.7:36546) 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7fef6e19c700 1 -- 172.21.15.7:0/1112726148 learned_addr learned my addr 172.21.15.7:0/1112726148 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7fef6d19a700 1 --2- 172.21.15.7:0/1112726148 >> v2:172.21.15.117:3300/0 conn(0x7fef680c7510 0x7fef680c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7fef6e19c700 1 -- 172.21.15.7:0/1112726148 >> v2:172.21.15.117:3300/0 conn(0x7fef680c7510 msgr2=0x7fef680c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7fef6e19c700 1 --2- 172.21.15.7:0/1112726148 >> v2:172.21.15.117:3300/0 conn(0x7fef680c7510 0x7fef680c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7fef6e19c700 1 -- 172.21.15.7:0/1112726148 >> v1:172.21.15.117:6789/0 conn(0x7fef68157c70 legacy=0x7fef680c7bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-09T11:10:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7fef6e19c700 1 -- 172.21.15.7:0/1112726148 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680cb4d0 con 0x7fef680c6dc0 2023-06-09T11:10:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7fef6e19c700 1 --2- 172.21.15.7:0/1112726148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c6dc0 0x7fef6815db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef6806d930 tx=0x7fef60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d5aa558defc906 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.867+0000 7faf8c3ff700 1 Processor -- start 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.868+0000 7faf8c3ff700 1 -- start start 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7faf8c3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf84157c70 0x7faf840c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7faf8c3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf840c7bb0 0x7faf840c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7faf8c3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf840c85f0 con 0x7faf840c8030 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7faf8c3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf840cb1f0 con 0x7faf840c7bb0 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.870+0000 7faf8c3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf840cb4d0 con 0x7faf84157c70 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf84157c70 0x7faf840c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faf84157c70 0x7faf840c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41302/0 (socket says 172.21.15.7:41302) 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 -- 172.21.15.7:0/3291435621 learned_addr learned my addr 172.21.15.7:0/3291435621 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 -- 172.21.15.7:0/3291435621 >> v2:172.21.15.7:3300/0 conn(0x7faf840c7bb0 msgr2=0x7faf840c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 --2- 172.21.15.7:0/3291435621 >> v2:172.21.15.7:3300/0 conn(0x7faf840c7bb0 0x7faf840c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 -- 172.21.15.7:0/3291435621 >> v1:172.21.15.117:6789/0 conn(0x7faf840c8030 legacy=0x7faf8415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.871+0000 7faf8a19b700 1 -- 172.21.15.7:0/3291435621 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840cb7b0 con 0x7faf84157c70 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7faf8a19b700 1 --2- 172.21.15.7:0/3291435621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf84157c70 0x7faf840c7510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf8406b100 tx=0x7faf7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f44fd2ab7a8d0c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.872+0000 7faf89199700 1 -- 172.21.15.7:0/3291435621 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00e070 con 0x7faf84157c70 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7faf89199700 1 -- 172.21.15.7:0/3291435621 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c003a20 con 0x7faf84157c70 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.873+0000 7faf89199700 1 -- 172.21.15.7:0/3291435621 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c003dd0 con 0x7faf84157c70 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 -- 172.21.15.7:0/3291435621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf84157c70 msgr2=0x7faf840c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/3291435621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf84157c70 0x7faf840c7510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faf8406b100 tx=0x7faf7c00b330 comp rx=0 tx=0).stop 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 -- 172.21.15.7:0/3291435621 shutdown_connections 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/3291435621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf84157c70 0x7faf840c7510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/3291435621 >> v2:172.21.15.7:3300/0 conn(0x7faf840c7bb0 0x7faf840c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.874+0000 7faf8c3ff700 1 -- 172.21.15.7:0/3291435621 >> 172.21.15.7:0/3291435621 conn(0x7faf8406a4d0 msgr2=0x7faf8406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.875+0000 7faf8c3ff700 1 -- 172.21.15.7:0/3291435621 shutdown_connections 2023-06-09T11:10:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.875+0000 7faf8c3ff700 1 -- 172.21.15.7:0/3291435621 wait complete. 2023-06-09T11:10:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.877+0000 7f00755ff700 1 Processor -- start 2023-06-09T11:10:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.877+0000 7faf8c3ff700 1 Processor -- start 2023-06-09T11:10:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.877+0000 7faf8c3ff700 1 -- start start 2023-06-09T11:10:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.877+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f8c01a400 con 0x7f8f9814f830 2023-06-09T11:10:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.878+0000 7f8f957fa700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8406adb0 0x7f8f8406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.878+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8f8c089c80 con 0x7f8f9814f830 2023-06-09T11:10:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f8f957fa700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f8f84073660 0x7f8f84075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f84076120 con 0x7f8f84073660 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f8c08a030 con 0x7f8f9814f830 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf840c7bb0 0x7faf841eb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf840c8030 0x7faf840c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 0x7faf8414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf841eadc0 con 0x7faf840c7bb0 2023-06-09T11:10:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf84152090 con 0x7faf840c8030 2023-06-09T11:10:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7faf8c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf841ebd10 con 0x7faf84157c70 2023-06-09T11:10:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7fef6c999700 1 -- 172.21.15.7:0/1112726148 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60004230 con 0x7fef680c6dc0 2023-06-09T11:10:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7fef6c999700 1 -- 172.21.15.7:0/1112726148 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef60004390 con 0x7fef680c6dc0 2023-06-09T11:10:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7fef6c999700 1 -- 172.21.15.7:0/1112726148 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600036a0 con 0x7fef680c6dc0 2023-06-09T11:10:59.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f8f9d19c700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f8f84073660 0x7f8f84075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f8f97fff700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8406adb0 0x7f8f8406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.877+0000 7f00755ff700 1 -- start start 2023-06-09T11:10:59.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f007014f830 0x7f00701f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f00700c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701591d0 0x7f00700c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700cad70 con 0x7f00701591d0 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00700c94c0 con 0x7f007014f830 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.879+0000 7f00755ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00701551f0 con 0x7f00701563f0 2023-06-09T11:10:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7faf8a99c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 0x7faf8414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7faf8a99c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 0x7faf8414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47676/0 (socket says 172.21.15.7:47676) 2023-06-09T11:10:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7faf8a99c700 1 -- 172.21.15.7:0/716961162 learned_addr learned my addr 172.21.15.7:0/716961162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f006e7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f00700c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f006effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f007014f830 0x7f00701f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f006effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f007014f830 0x7f00701f5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41326/0 (socket says 172.21.15.7:41326) 2023-06-09T11:10:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f006effd700 1 -- 172.21.15.7:0/155214599 learned_addr learned my addr 172.21.15.7:0/155214599 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 -- 172.21.15.7:0/1112726148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c6dc0 msgr2=0x7fef6815db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1112726148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c6dc0 0x7fef6815db30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef6806d930 tx=0x7fef60009bd0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f8f97fff700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8406adb0 0x7f8f8406d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f9814def0 tx=0x7f8f88006c60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7faf8999a700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf840c8030 0x7faf840c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a19b700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf840c7bb0 0x7faf841eb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 -- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf840c8030 msgr2=0x7faf840c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf840c8030 0x7faf840c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 -- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf840c7bb0 msgr2=0x7faf841eb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf840c7bb0 0x7faf841eb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c009b30 con 0x7faf84157c70 2023-06-09T11:10:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7faf8a99c700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 0x7faf8414d410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c490 tx=0x7faf80001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f006e7fc700 1 -- 172.21.15.7:0/155214599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f007014f830 msgr2=0x7f00701f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f006e7fc700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f007014f830 0x7f00701f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f006e7fc700 1 -- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701591d0 msgr2=0x7f00700c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f006e7fc700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00701591d0 0x7f00700c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.881+0000 7f006e7fc700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600085d0 con 0x7f00701563f0 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 -- 172.21.15.7:0/1112726148 shutdown_connections 2023-06-09T11:10:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1112726148 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c6dc0 0x7fef6815db30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1112726148 >> v2:172.21.15.117:3300/0 conn(0x7fef680c7510 0x7fef680c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7fef6fbff700 1 -- 172.21.15.7:0/1112726148 >> 172.21.15.7:0/1112726148 conn(0x7fef6806a4d0 msgr2=0x7fef6806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf800054e0 con 0x7faf84157c70 2023-06-09T11:10:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf80003680 con 0x7faf84157c70 2023-06-09T11:10:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80003a30 con 0x7faf84157c70 2023-06-09T11:10:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f006e7fc700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f00700c46f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f006000b000 tx=0x7f0060004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7fef6fbff700 1 -- 172.21.15.7:0/1112726148 shutdown_connections 2023-06-09T11:10:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf840cbd70 con 0x7faf84157c70 2023-06-09T11:10:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf840cba90 con 0x7faf84157c70 2023-06-09T11:10:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f3adafff700 1 Processor -- start 2023-06-09T11:10:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.880+0000 7f3adafff700 1 -- start start 2023-06-09T11:10:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f3adafff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f3adafff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ad40c1660 0x7f3ad40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f3adafff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad40c11d0 con 0x7f3ad414fa60 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f3adafff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad40ad900 con 0x7f3ad40c1660 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.882+0000 7f3adafff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40c0bf0 con 0x7f3ad40c6df0 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47700/0 (socket says 172.21.15.7:47700) 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 -- 172.21.15.7:0/781668530 learned_addr learned my addr 172.21.15.7:0/781668530 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad959c700 1 --2- 172.21.15.7:0/781668530 >> v2:172.21.15.7:3300/0 conn(0x7f3ad40c1660 0x7f3ad40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7fef6fbff700 1 -- 172.21.15.7:0/1112726148 wait complete. 2023-06-09T11:10:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 -- 172.21.15.7:0/781668530 >> v2:172.21.15.7:3300/0 conn(0x7f3ad40c1660 msgr2=0x7f3ad40c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 --2- 172.21.15.7:0/781668530 >> v2:172.21.15.7:3300/0 conn(0x7f3ad40c1660 0x7f3ad40c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 -- 172.21.15.7:0/781668530 >> v1:172.21.15.117:6789/0 conn(0x7f3ad414fa60 legacy=0x7f3ad4158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f3ad3fff700 1 -- 172.21.15.7:0/781668530 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40c72c0 con 0x7f3ad40c6df0 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f3ad3fff700 1 --2- 172.21.15.7:0/781668530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009e50 tx=0x7f3acc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3480060ab01a1955 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f3ad37fe700 1 -- 172.21.15.7:0/781668530 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc003c20 con 0x7f3ad40c6df0 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f3ad37fe700 1 -- 172.21.15.7:0/781668530 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc003d80 con 0x7f3ad40c6df0 2023-06-09T11:10:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f3ad37fe700 1 -- 172.21.15.7:0/781668530 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc00bb10 con 0x7f3ad40c6df0 2023-06-09T11:10:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f8f9d19c700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f8f84073660 0x7f8f84075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006000ddb0 con 0x7f00701563f0 2023-06-09T11:10:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006001a070 con 0x7f00701563f0 2023-06-09T11:10:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00600038e0 con 0x7f00701563f0 2023-06-09T11:10:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0070155910 con 0x7f00701563f0 2023-06-09T11:10:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070159940 con 0x7f00701563f0 2023-06-09T11:10:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.884+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faf84158040 con 0x7faf84157c70 2023-06-09T11:10:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 -- 172.21.15.7:0/781668530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 msgr2=0x7f3ad40c0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 --2- 172.21.15.7:0/781668530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009e50 tx=0x7f3acc00b330 comp rx=0 tx=0).stop 2023-06-09T11:10:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7fcaa27ff700 1 Processor -- start 2023-06-09T11:10:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.883+0000 7fcaa27ff700 1 -- start start 2023-06-09T11:10:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0070155bf0 con 0x7f00701563f0 2023-06-09T11:10:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 -- 172.21.15.7:0/781668530 shutdown_connections 2023-06-09T11:10:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 --2- 172.21.15.7:0/781668530 >> v2:172.21.15.7:3300/0 conn(0x7f3ad40c1660 0x7f3ad40c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 --2- 172.21.15.7:0/781668530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c6df0 0x7f3ad40c0140 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.885+0000 7f3adafff700 1 -- 172.21.15.7:0/781668530 >> 172.21.15.7:0/781668530 conn(0x7f3ad406a4d0 msgr2=0x7f3ad406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa27ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca9c157c70 0x7fca9c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== osd.0 v2:172.21.15.7:6808/1473672210 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8f84076120 con 0x7f8f84073660 2023-06-09T11:10:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa27ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c0c85f0 con 0x7fca9c0c7bb0 2023-06-09T11:10:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa27ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c0cb1f0 con 0x7fca9c157c70 2023-06-09T11:10:59.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0cb4d0 con 0x7fca9c0c8030 2023-06-09T11:10:59.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fef6fbff700 1 Processor -- start 2023-06-09T11:10:59.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fca9bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca9c157c70 0x7fca9c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7f3adafff700 1 -- 172.21.15.7:0/781668530 shutdown_connections 2023-06-09T11:10:59.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fca9bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fca9c157c70 0x7fca9c0c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41340/0 (socket says 172.21.15.7:41340) 2023-06-09T11:10:59.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fca9bfff700 1 -- 172.21.15.7:0/1281276527 learned_addr learned my addr 172.21.15.7:0/1281276527 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9bfff700 1 -- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 msgr2=0x7fca9c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9bfff700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9bfff700 1 -- 172.21.15.7:0/1281276527 >> v1:172.21.15.7:6789/0 conn(0x7fca9c0c7bb0 legacy=0x7fca9c0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9bfff700 1 -- 172.21.15.7:0/1281276527 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c0cb7b0 con 0x7fca9c157c70 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9bfff700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c0c7510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098f0 tx=0x7fca8c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ff6550baaf33b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9affd700 1 -- 172.21.15.7:0/1281276527 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00b8a0 con 0x7fca9c157c70 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9affd700 1 -- 172.21.15.7:0/1281276527 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c00ba00 con 0x7fca9c157c70 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.887+0000 7fca9affd700 1 -- 172.21.15.7:0/1281276527 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00bd70 con 0x7fca9c157c70 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 -- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 msgr2=0x7fca9c0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c0c7510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0098f0 tx=0x7fca8c009c00 comp rx=0 tx=0).stop 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.886+0000 7f3adafff700 1 -- 172.21.15.7:0/781668530 wait complete. 2023-06-09T11:10:59.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7f3adafff700 1 Processor -- start 2023-06-09T11:10:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 -- 172.21.15.7:0/1281276527 shutdown_connections 2023-06-09T11:10:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/1281276527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c0c7510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 -- 172.21.15.7:0/1281276527 >> 172.21.15.7:0/1281276527 conn(0x7fca9c06a4d0 msgr2=0x7fca9c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 -- 172.21.15.7:0/1281276527 shutdown_connections 2023-06-09T11:10:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fef6fbff700 1 -- start start 2023-06-09T11:10:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 0x7fef6815b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c7510 0x7fef680c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef68157c70 0x7fef680c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef68141000 con 0x7fef680c7510 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef680c9c00 con 0x7fef680c6dc0 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7fef6fbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6815d090 con 0x7fef68157c70 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7fef6d99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 0x7fef6815b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7fef6d99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 0x7fef6815b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41354/0 (socket says 172.21.15.7:41354) 2023-06-09T11:10:59.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7fef6d99b700 1 -- 172.21.15.7:0/1758301439 learned_addr learned my addr 172.21.15.7:0/1758301439 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7f3adafff700 1 -- start start 2023-06-09T11:10:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c1660 0x7f3ad40ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad40c6df0 0x7f3ad40abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 0x7f3ad4163810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40a9d20 con 0x7f3ad40c6df0 2023-06-09T11:10:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad414f1b0 con 0x7f3ad40c5340 2023-06-09T11:10:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3adafff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad40a9480 con 0x7f3ad40c1660 2023-06-09T11:10:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3ad959c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 0x7f3ad4163810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3ad959c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 0x7f3ad4163810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41364/0 (socket says 172.21.15.7:41364) 2023-06-09T11:10:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f3ad959c700 1 -- 172.21.15.7:0/1369018275 learned_addr learned my addr 172.21.15.7:0/1369018275 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.888+0000 7facd45ff700 1 Processor -- start 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.888+0000 7facd45ff700 1 -- start start 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7facd45ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faccc15db30 0x7faccc15df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7facd45ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc15e470 con 0x7faccc0c7510 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7facd45ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faccc0c85f0 con 0x7faccc157c70 2023-06-09T11:10:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7facd45ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faccc0cb1f0 con 0x7faccc15db30 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.888+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0060003a40 con 0x7f00701563f0 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7f00577fe700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f005c06adb0 0x7f005c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.890+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f006008a250 con 0x7f00701563f0 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f00577fe700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7f005c073660 0x7f005c075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f005c076120 con 0x7f005c073660 2023-06-09T11:10:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006008a600 con 0x7f00701563f0 2023-06-09T11:10:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 -- 172.21.15.7:0/1758301439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef68157c70 msgr2=0x7fef680c96c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:10:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef68157c70 0x7fef680c96c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 -- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c7510 msgr2=0x7fef680c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef680c7510 0x7fef680c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 -- 172.21.15.7:0/1758301439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60009550 con 0x7fef680c6dc0 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 -- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c1660 msgr2=0x7f3ad40ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad8d9b700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c1660 0x7f3ad40ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c1660 0x7f3ad40ab5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 -- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad40c6df0 msgr2=0x7f3ad40abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad40c6df0 0x7f3ad40abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc009b30 con 0x7f3ad40c5340 2023-06-09T11:10:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.891+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faf80003b90 con 0x7faf84157c70 2023-06-09T11:10:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad8d9b700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad40c1660 0x7f3ad40ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7fef6d99b700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 0x7fef6815b6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00ba50 tx=0x7fef5c00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7facd1b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faccc0c7510 0x7faccc0c6880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49950/0 (socket says 172.21.15.7:49950) 2023-06-09T11:10:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7facd1b9a700 1 -- 172.21.15.7:0/915052793 learned_addr learned my addr 172.21.15.7:0/915052793 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7f3ad959c700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 0x7f3ad4163810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e800 tx=0x7f3ac400ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c00cae0 con 0x7fef680c6dc0 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c00cc40 con 0x7fef680c6dc0 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c014440 con 0x7fef680c6dc0 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4013780 con 0x7f3ad40c5340 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac4010040 con 0x7f3ad40c5340 2023-06-09T11:10:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4013b40 con 0x7f3ad40c5340 2023-06-09T11:10:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad40c0e60 con 0x7f3ad40c5340 2023-06-09T11:10:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fc3d33ff700 1 Processor -- start 2023-06-09T11:10:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.892+0000 7faf76ffd700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf6806adb0 0x7faf6806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7faf80088f90 con 0x7faf84157c70 2023-06-09T11:10:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7faf76ffd700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7faf68073660 0x7faf68075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faf68076120 con 0x7faf68073660 2023-06-09T11:10:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf80089340 con 0x7faf84157c70 2023-06-09T11:10:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7faf8999a700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7faf68073660 0x7faf68075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7faf8a19b700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf6806adb0 0x7faf6806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f006f7fe700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7f005c073660 0x7f005c075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7f006effd700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f005c06adb0 0x7f005c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7f006f7fe700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7f005c073660 0x7f005c075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef680cba90 con 0x7fef680c6dc0 2023-06-09T11:10:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef680cb7b0 con 0x7fef680c6dc0 2023-06-09T11:10:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad40c75a0 con 0x7f3ad40c5340 2023-06-09T11:10:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7fc3d33ff700 1 -- start start 2023-06-09T11:10:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fc3d33ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3cc14f830 0x7fc3cc0c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fc3d33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 0x7fc3cc0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fc3d33ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0c0bb0 con 0x7fc3cc1563f0 2023-06-09T11:10:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fc3d33ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc155720 con 0x7fc3cc14f830 2023-06-09T11:10:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fc3d33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc159c20 con 0x7fc3cc14fc00 2023-06-09T11:10:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.889+0000 7fcaa27ff700 1 -- 172.21.15.7:0/1281276527 wait complete. 2023-06-09T11:10:59.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.896+0000 7fcaa27ff700 1 Processor -- start 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.893+0000 7facd2b9c700 1 --2- 172.21.15.7:0/915052793 >> v2:172.21.15.7:3300/0 conn(0x7faccc15db30 0x7faccc15df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.894+0000 7facd1399700 1 -- 172.21.15.7:0/915052793 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679880105 0 0) 0x7faccc15e470 con 0x7faccc0c7510 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.894+0000 7facd1399700 1 -- 172.21.15.7:0/915052793 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb80035b0 con 0x7faccc0c7510 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7facd2b9c700 1 -- 172.21.15.7:0/915052793 >> v1:172.21.15.117:6789/0 conn(0x7faccc157c70 legacy=0x7faccc0c7bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7facd2b9c700 1 -- 172.21.15.7:0/915052793 >> v1:172.21.15.7:6789/0 conn(0x7faccc0c7510 legacy=0x7faccc0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.895+0000 7facd2b9c700 1 -- 172.21.15.7:0/915052793 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc0cb4d0 con 0x7faccc15db30 2023-06-09T11:10:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.897+0000 7fc3d099a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc3cc1563f0 0x7fc3cc158c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49952/0 (socket says 172.21.15.7:49952) 2023-06-09T11:10:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.897+0000 7fc3d099a700 1 -- 172.21.15.7:0/796075911 learned_addr learned my addr 172.21.15.7:0/796075911 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:10:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7faf8999a700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7faf68073660 0x7faf68075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ab4000fb0 con 0x7f3ad40c5340 2023-06-09T11:10:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7facd2b9c700 1 --2- 172.21.15.7:0/915052793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc15db30 0x7faccc15df30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facc8009840 tx=0x7facc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c099d5ce6636a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7fc3d119b700 1 --2- 172.21.15.7:0/796075911 >> v2:172.21.15.7:3300/0 conn(0x7fc3cc14f830 0x7fc3cc0c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== osd.2 v2:172.21.15.7:6816/843671507 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f005c076120 con 0x7f005c073660 2023-06-09T11:10:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7f006effd700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f005c06adb0 0x7f005c06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f006800e270 tx=0x7f0068009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7fcaa27ff700 1 -- start start 2023-06-09T11:10:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7faf8a19b700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf6806adb0 0x7faf6806d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf7c000f80 tx=0x7faf7c009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7facd1399700 1 -- 172.21.15.7:0/915052793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8004230 con 0x7faccc15db30 2023-06-09T11:10:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7facd1399700 1 -- 172.21.15.7:0/915052793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc8004390 con 0x7faccc15db30 2023-06-09T11:10:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7facd1399700 1 -- 172.21.15.7:0/915052793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc80036a0 con 0x7faccc15db30 2023-06-09T11:10:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== osd.1 v2:172.21.15.7:6800/48537347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7faf68076120 con 0x7faf68073660 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3c3fff700 1 -- 172.21.15.7:0/796075911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310479466 0 0) 0x7fc3cc0c0bb0 con 0x7fc3cc1563f0 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3c3fff700 1 -- 172.21.15.7:0/796075911 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b80035b0 con 0x7fc3cc1563f0 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d199c700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 0x7fc3cc0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d119b700 1 -- 172.21.15.7:0/796075911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 msgr2=0x7fc3cc0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d119b700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 0x7fc3cc0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d119b700 1 -- 172.21.15.7:0/796075911 >> v1:172.21.15.7:6789/0 conn(0x7fc3cc1563f0 legacy=0x7fc3cc158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d119b700 1 -- 172.21.15.7:0/796075911 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc0c7940 con 0x7fc3cc14f830 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fc3d199c700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 0x7fc3cc0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.897+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fef5c006d50 con 0x7fef680c6dc0 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7fef667fc700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fef4c06adb0 0x7fef4c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fef5c089060 con 0x7fef680c6dc0 2023-06-09T11:10:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.899+0000 7fef6d19a700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fef4c06adb0 0x7fef4c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca9c0c7bb0 0x7fca9c14ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c155300 con 0x7fca9c0c7bb0 2023-06-09T11:10:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c14f1d0 con 0x7fca9c157c70 2023-06-09T11:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fcaa27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0ca2e0 con 0x7fca9c0c8030 2023-06-09T11:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 -- 172.21.15.7:0/915052793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc15db30 msgr2=0x7faccc15df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 --2- 172.21.15.7:0/915052793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc15db30 0x7faccc15df30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facc8009840 tx=0x7facc8009b50 comp rx=0 tx=0).stop 2023-06-09T11:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 -- 172.21.15.7:0/915052793 shutdown_connections 2023-06-09T11:10:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 --2- 172.21.15.7:0/915052793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc15db30 0x7faccc15df30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 -- 172.21.15.7:0/915052793 >> 172.21.15.7:0/915052793 conn(0x7faccc06a4d0 msgr2=0x7faccc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fef6d19a700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fef4c06adb0 0x7fef4c06d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef6806eeb0 tx=0x7fef54008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fca9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca9c0c7bb0 0x7fca9c14ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fca9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca9c0c7bb0 0x7fca9c14ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36636/0 (socket says 172.21.15.7:36636) 2023-06-09T11:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7fca9bfff700 1 -- 172.21.15.7:0/438774493 learned_addr learned my addr 172.21.15.7:0/438774493 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7facd45ff700 1 -- 172.21.15.7:0/915052793 shutdown_connections 2023-06-09T11:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7facd45ff700 1 -- 172.21.15.7:0/915052793 wait complete. 2023-06-09T11:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7fef500015d0 0x7fef50003a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 --> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef50006bb0 con 0x7fef500015d0 2023-06-09T11:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.898+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ac4013ca0 con 0x7f3ad40c5340 2023-06-09T11:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7f3ad17fa700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ac006adb0 0x7f3ac006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3ac4089100 con 0x7f3ad40c5340 2023-06-09T11:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7f3ad17fa700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7f3ac00735e0 0x7f3ac00759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.900+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3ac00760a0 con 0x7f3ac00735e0 2023-06-09T11:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7f3ad3fff700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7f3ac00735e0 0x7f3ac00759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7f3ad8d9b700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ac006adb0 0x7f3ac006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fef6e19c700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7fef500015d0 0x7fef50003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fca9b7fe700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 -- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 msgr2=0x7fca9c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca9c0c8030 0x7fca9c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 -- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca9c0c7bb0 msgr2=0x7fca9c14ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca9c0c7bb0 0x7fca9c14ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c009550 con 0x7fca9c157c70 2023-06-09T11:10:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7f3ad3fff700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7f3ac00735e0 0x7f3ac00759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fef6e19c700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7fef500015d0 0x7fef50003a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7f3ad8d9b700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ac006adb0 0x7f3ac006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40c8cb0 tx=0x7f3ac8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ac4054190 con 0x7f3ad40c5340 2023-06-09T11:10:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fcaa0d9c700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c15cdc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ce20 tx=0x7fca9400ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.901+0000 7fc3d119b700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14f830 0x7fc3cc0c0670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009cb0 tx=0x7fc3c400c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e0a6ed065cd8dc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fc3c3fff700 1 -- 172.21.15.7:0/796075911 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4003a20 con 0x7fc3cc14f830 2023-06-09T11:10:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fc3c3fff700 1 -- 172.21.15.7:0/796075911 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c4003b80 con 0x7fc3cc14f830 2023-06-09T11:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.902+0000 7fc3c3fff700 1 -- 172.21.15.7:0/796075911 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400c9f0 con 0x7fc3cc14f830 2023-06-09T11:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.903+0000 7fc3d33ff700 1 -- 172.21.15.7:0/796075911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14f830 msgr2=0x7fc3cc0c0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.903+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14f830 0x7fc3cc0c0670 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009cb0 tx=0x7fc3c400c160 comp rx=0 tx=0).stop 2023-06-09T11:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.903+0000 7facd45ff700 1 Processor -- start 2023-06-09T11:10:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.903+0000 7facd45ff700 1 -- start start 2023-06-09T11:10:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== osd.3 v2:172.21.15.7:6824/353233282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fef50006bb0 con 0x7fef500015d0 2023-06-09T11:10:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 -- 172.21.15.7:0/796075911 shutdown_connections 2023-06-09T11:10:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc14fc00 0x7fc3cc0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/796075911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14f830 0x7fc3cc0c0670 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 -- 172.21.15.7:0/796075911 >> 172.21.15.7:0/796075911 conn(0x7fc3cc06a4d0 msgr2=0x7fc3cc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 -- 172.21.15.7:0/796075911 shutdown_connections 2023-06-09T11:10:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9400aca0 con 0x7fca9c157c70 2023-06-09T11:10:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9400ae40 con 0x7fca9c157c70 2023-06-09T11:10:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94015680 con 0x7fca9c157c70 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc0c7510 0x7faccc1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faccc157c70 0x7faccc155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 0x7faccc138930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc1384a0 con 0x7faccc0c7510 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faccc135130 con 0x7faccc15db30 2023-06-09T11:10:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7facd45ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faccc135410 con 0x7faccc157c70 2023-06-09T11:10:59.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fc3d33ff700 1 -- 172.21.15.7:0/796075911 wait complete. 2023-06-09T11:10:59.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.904+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c0cbd70 con 0x7fca9c157c70 2023-06-09T11:10:59.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.905+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c0cba90 con 0x7fca9c157c70 2023-06-09T11:10:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.906+0000 7facd239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc0c7510 0x7faccc1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.906+0000 7facd2b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 0x7faccc138930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.906+0000 7facd2b9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 0x7faccc138930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41384/0 (socket says 172.21.15.7:41384) 2023-06-09T11:10:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.906+0000 7facd2b9c700 1 -- 172.21.15.7:0/2080953327 learned_addr learned my addr 172.21.15.7:0/2080953327 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.906+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca7c000fb0 con 0x7fca9c157c70 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd1b9a700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faccc157c70 0x7faccc155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 -- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faccc157c70 msgr2=0x7faccc155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faccc157c70 0x7faccc155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 -- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc0c7510 msgr2=0x7faccc1553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faccc0c7510 0x7faccc1553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc80094a0 con 0x7faccc15db30 2023-06-09T11:10:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7fc3d33ff700 1 Processor -- start 2023-06-09T11:10:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7facd2b9c700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 0x7faccc138930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facc8004000 tx=0x7facc80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.908+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8011630 con 0x7faccc15db30 2023-06-09T11:10:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.908+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc8011790 con 0x7faccc15db30 2023-06-09T11:10:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.908+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc8011b00 con 0x7faccc15db30 2023-06-09T11:10:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.907+0000 7fc3d33ff700 1 -- start start 2023-06-09T11:10:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.909+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faccc0cba90 con 0x7faccc15db30 2023-06-09T11:10:59.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.909+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faccc0cb7b0 con 0x7faccc15db30 2023-06-09T11:10:59.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fca9400d430 con 0x7fca9c157c70 2023-06-09T11:10:59.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 0x7fc3cc1f6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14fc00 0x7fc3cc1f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc1563f0 0x7fc3cc14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0c8fa0 con 0x7fc3cc14fc00 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc1530c0 con 0x7fc3cc14f830 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc071d10 con 0x7fc3cc1563f0 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d119b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 0x7fc3cc1f6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d119b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 0x7fc3cc1f6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41390/0 (socket says 172.21.15.7:41390) 2023-06-09T11:10:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.910+0000 7fc3d119b700 1 -- 172.21.15.7:0/858972870 learned_addr learned my addr 172.21.15.7:0/858972870 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:10:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.911+0000 7fc3d099a700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14fc00 0x7fc3cc1f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.911+0000 7fca98ff9700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fca8806adb0 0x7fca8806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.911+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fca94089c70 con 0x7fca9c157c70 2023-06-09T11:10:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fca98ff9700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7fca88073660 0x7fca88075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca88076120 con 0x7fca88073660 2023-06-09T11:10:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9408a020 con 0x7fca9c157c70 2023-06-09T11:10:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 -- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc1563f0 msgr2=0x7fc3cc14c970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T11:10:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3cc1563f0 0x7fc3cc14c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 -- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14fc00 msgr2=0x7fc3cc1f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14fc00 0x7fc3cc1f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c400d040 con 0x7fc3cc14f830 2023-06-09T11:10:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.912+0000 7fc3d119b700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 0x7fc3cc1f6610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0c8d40 tx=0x7fc3c4004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fca9b7fe700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7fca88073660 0x7fca88075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.903+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== osd.4 v2:172.21.15.117:6800/3721658998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3ac00760a0 con 0x7f3ac00735e0 2023-06-09T11:10:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.911+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faccc0cbd70 con 0x7faccc15db30 2023-06-09T11:10:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fc3d099a700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3cc14fc00 0x7fc3cc1f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:10:59.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fca9bfff700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fca8806adb0 0x7fca8806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c401f070 con 0x7fc3cc14f830 2023-06-09T11:10:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c4004380 con 0x7fc3cc14f830 2023-06-09T11:10:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4004530 con 0x7fc3cc14f830 2023-06-09T11:10:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.914+0000 7fca9b7fe700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7fca88073660 0x7fca88075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.914+0000 7fca9bfff700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fca8806adb0 0x7fca8806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009c40 tx=0x7fca8c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.913+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7facc8011c60 con 0x7faccc15db30 2023-06-09T11:10:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7facc2ffd700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006adb0 0x7facb006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7facc808ac10 con 0x7faccc15db30 2023-06-09T11:10:59.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7facc2ffd700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7facb0073660 0x7facb0075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7facb0076120 con 0x7facb0073660 2023-06-09T11:10:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7facc808f2a0 con 0x7faccc15db30 2023-06-09T11:10:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.916+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== osd.7 v2:172.21.15.117:6816/1383862628 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fca88076120 con 0x7fca88073660 2023-06-09T11:10:59.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.914+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc159660 con 0x7fc3cc14f830 2023-06-09T11:10:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.915+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc1593a0 con 0x7fc3cc14f830 2023-06-09T11:10:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.916+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3cc159940 con 0x7fc3cc14f830 2023-06-09T11:10:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.916+0000 7facd1b9a700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7facb0073660 0x7facb0075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.917+0000 7facd239b700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006adb0 0x7facb006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.919+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3c4004690 con 0x7fc3cc14f830 2023-06-09T11:10:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.920+0000 7facd239b700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006adb0 0x7facb006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7facc400b450 tx=0x7facc400d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.922+0000 7fc3c1ffb700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3b806b1b0 0x7fc3b806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.922+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3c408afe0 con 0x7fc3cc14f830 2023-06-09T11:10:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.922+0000 7fc3c1ffb700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7fc3b8073a60 0x7fc3b8075e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:10:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.922+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3b8076520 con 0x7fc3b8073a60 2023-06-09T11:10:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.922+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c408b350 con 0x7fc3cc14f830 2023-06-09T11:10:59.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.923+0000 7facd1b9a700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7facb0073660 0x7facb0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.923+0000 7fc3d199c700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7fc3b8073a60 0x7fc3b8075e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.923+0000 7fc3d099a700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3b806b1b0 0x7fc3b806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:10:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.924+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== osd.5 v2:172.21.15.117:6824/644345713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7facb0076120 con 0x7facb0073660 2023-06-09T11:10:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.924+0000 7fc3d099a700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3b806b1b0 0x7fc3b806d660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007c40 tx=0x7fc3bc007b90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.925+0000 7fc3d199c700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7fc3b8073a60 0x7fc3b8075e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:10:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.926+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== osd.6 v2:172.21.15.117:6808/2098282767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc3b8076520 con 0x7fc3b8073a60 2023-06-09T11:10:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.923+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 --> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f98074cf0 con 0x7f8f84073660 2023-06-09T11:10:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.930+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 --> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0070074cf0 con 0x7f005c073660 2023-06-09T11:10:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.930+0000 7f8f957fa700 1 -- 172.21.15.7:0/2868707537 <== osd.0 v2:172.21.15.7:6808/1473672210 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f98074cf0 con 0x7f8f84073660 2023-06-09T11:10:59.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.930+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 --> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faf84074cf0 con 0x7faf68073660 2023-06-09T11:10:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.931+0000 7f00577fe700 1 -- 172.21.15.7:0/155214599 <== osd.2 v2:172.21.15.7:6816/843671507 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0070074cf0 con 0x7f005c073660 2023-06-09T11:10:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.931+0000 7faf76ffd700 1 -- 172.21.15.7:0/716961162 <== osd.1 v2:172.21.15.7:6800/48537347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faf84074cf0 con 0x7faf68073660 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7f005c073660 msgr2=0x7f005c075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7f005c073660 0x7f005c075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f005c06adb0 msgr2=0x7f005c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f005c06adb0 0x7f005c06d260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f006800e270 tx=0x7f0068009040 comp rx=0 tx=0).stop 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 msgr2=0x7f00700c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f00755ff700 1 --2- 172.21.15.7:0/155214599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00701563f0 0x7f00700c46f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f006000b000 tx=0x7f0060004750 comp rx=0 tx=0).stop 2023-06-09T11:10:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.932+0000 7f006f7fe700 1 -- 172.21.15.7:0/155214599 reap_dead start 2023-06-09T11:10:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 shutdown_connections 2023-06-09T11:10:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 >> 172.21.15.7:0/155214599 conn(0x7f007006a4d0 msgr2=0x7f00700c1170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 shutdown_connections 2023-06-09T11:10:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7f00755ff700 1 -- 172.21.15.7:0/155214599 wait complete. 2023-06-09T11:10:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7faf68073660 msgr2=0x7faf68075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.933+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7faf68073660 0x7faf68075a70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf6806adb0 msgr2=0x7faf6806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faf6806adb0 0x7faf6806d260 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faf7c000f80 tx=0x7faf7c009000 comp rx=0 tx=0).stop 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 msgr2=0x7faf8414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 --2- 172.21.15.7:0/716961162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf84157c70 0x7faf8414d410 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faf8000c490 tx=0x7faf80001e70 comp rx=0 tx=0).stop 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8a99c700 1 -- 172.21.15.7:0/716961162 reap_dead start 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 shutdown_connections 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 >> 172.21.15.7:0/716961162 conn(0x7faf8406a4d0 msgr2=0x7faf841525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.934+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 shutdown_connections 2023-06-09T11:10:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f8f84073660 msgr2=0x7f8f84075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f8f84073660 0x7f8f84075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8406adb0 msgr2=0x7f8f8406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8406adb0 0x7f8f8406d260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f9814def0 tx=0x7f8f88006c60 comp rx=0 tx=0).stop 2023-06-09T11:10:59.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 msgr2=0x7f8f98149d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 --2- 172.21.15.7:0/2868707537 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9814f830 0x7f8f98149d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00b560 tx=0x7f8f8c00b920 comp rx=0 tx=0).stop 2023-06-09T11:10:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9d19c700 1 -- 172.21.15.7:0/2868707537 reap_dead start 2023-06-09T11:10:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 shutdown_connections 2023-06-09T11:10:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.938+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 >> 172.21.15.7:0/2868707537 conn(0x7f8f9806a4d0 msgr2=0x7f8f9806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.939+0000 7faf8c3ff700 1 -- 172.21.15.7:0/716961162 wait complete. 2023-06-09T11:10:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.939+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 --> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ab4002d30 con 0x7f3ac00735e0 2023-06-09T11:10:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.939+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 shutdown_connections 2023-06-09T11:10:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.940+0000 7f8f9ebff700 1 -- 172.21.15.7:0/2868707537 wait complete. 2023-06-09T11:10:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.940+0000 7f3ad17fa700 1 -- 172.21.15.7:0/1369018275 <== osd.4 v2:172.21.15.117:6800/3721658998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ab4002d30 con 0x7f3ac00735e0 2023-06-09T11:10:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7f3ac00735e0 msgr2=0x7f3ac00759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7f3ac00735e0 0x7f3ac00759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ac006adb0 msgr2=0x7f3ac006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ac006adb0 0x7f3ac006d260 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40c8cb0 tx=0x7f3ac8009040 comp rx=0 tx=0).stop 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 msgr2=0x7f3ad4163810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3adafff700 1 --2- 172.21.15.7:0/1369018275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad40c5340 0x7f3ad4163810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400e800 tx=0x7f3ac400ebc0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.941+0000 7f3ad959c700 1 -- 172.21.15.7:0/1369018275 reap_dead start 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.942+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 shutdown_connections 2023-06-09T11:10:59.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.942+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 >> 172.21.15.7:0/1369018275 conn(0x7f3ad406a4d0 msgr2=0x7f3ad406aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.942+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 shutdown_connections 2023-06-09T11:10:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.944+0000 7f3adafff700 1 -- 172.21.15.7:0/1369018275 wait complete. 2023-06-09T11:10:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.948+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 --> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fca7c002d30 con 0x7fca88073660 2023-06-09T11:10:59.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.949+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 --> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef50005c30 con 0x7fef500015d0 2023-06-09T11:10:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.949+0000 7fca98ff9700 1 -- 172.21.15.7:0/438774493 <== osd.7 v2:172.21.15.117:6816/1383862628 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fca7c002d30 con 0x7fca88073660 2023-06-09T11:10:59.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 --> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3cc074cf0 con 0x7fc3b8073a60 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7fca88073660 msgr2=0x7fca88075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7fca88073660 0x7fca88075a70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fca8806adb0 msgr2=0x7fca8806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fca8806adb0 0x7fca8806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009c40 tx=0x7fca8c008030 comp rx=0 tx=0).stop 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 msgr2=0x7fca9c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 --2- 172.21.15.7:0/438774493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca9c157c70 0x7fca9c15cdc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fca9400ce20 tx=0x7fca9400ce50 comp rx=0 tx=0).stop 2023-06-09T11:10:59.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa0d9c700 1 -- 172.21.15.7:0/438774493 reap_dead start 2023-06-09T11:10:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fef667fc700 1 -- 172.21.15.7:0/1758301439 <== osd.3 v2:172.21.15.7:6824/353233282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef50005c30 con 0x7fef500015d0 2023-06-09T11:10:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 shutdown_connections 2023-06-09T11:10:59.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 >> 172.21.15.7:0/438774493 conn(0x7fca9c06a4d0 msgr2=0x7fca9c152030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.950+0000 7fc3c1ffb700 1 -- 172.21.15.7:0/858972870 <== osd.6 v2:172.21.15.117:6808/2098282767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3cc074cf0 con 0x7fc3b8073a60 2023-06-09T11:10:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 shutdown_connections 2023-06-09T11:10:59.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fcaa27ff700 1 -- 172.21.15.7:0/438774493 wait complete. 2023-06-09T11:10:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7fc3b8073a60 msgr2=0x7fc3b8075e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7fc3b8073a60 0x7fc3b8075e70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3b806b1b0 msgr2=0x7fc3b806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3b806b1b0 0x7fc3b806d660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc007c40 tx=0x7fc3bc007b90 comp rx=0 tx=0).stop 2023-06-09T11:10:59.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 msgr2=0x7fc3cc1f6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.951+0000 7fc3d33ff700 1 --2- 172.21.15.7:0/858972870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3cc14f830 0x7fc3cc1f6610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0c8d40 tx=0x7fc3c4004b40 comp rx=0 tx=0).stop 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fc3d199c700 1 -- 172.21.15.7:0/858972870 reap_dead start 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 shutdown_connections 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 >> 172.21.15.7:0/858972870 conn(0x7fc3cc06a4d0 msgr2=0x7fc3cc14eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7fef500015d0 msgr2=0x7fef50003a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7fef500015d0 0x7fef50003a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 shutdown_connections 2023-06-09T11:10:59.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.952+0000 7fc3d33ff700 1 -- 172.21.15.7:0/858972870 wait complete. 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fef4c06adb0 msgr2=0x7fef4c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fef4c06adb0 0x7fef4c06d260 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fef6806eeb0 tx=0x7fef54008040 comp rx=0 tx=0).stop 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 msgr2=0x7fef6815b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 --2- 172.21.15.7:0/1758301439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef680c6dc0 0x7fef6815b6d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef5c00ba50 tx=0x7fef5c00be10 comp rx=0 tx=0).stop 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6e19c700 1 -- 172.21.15.7:0/1758301439 reap_dead start 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 shutdown_connections 2023-06-09T11:10:59.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.953+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 >> 172.21.15.7:0/1758301439 conn(0x7fef6806a4d0 msgr2=0x7fef68153d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.954+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 shutdown_connections 2023-06-09T11:10:59.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.954+0000 7fef6fbff700 1 -- 172.21.15.7:0/1758301439 wait complete. 2023-06-09T11:10:59.991 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:10:59.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-09T11:10:59.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.960+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 --> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faccc138610 con 0x7facb0073660 2023-06-09T11:10:59.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.961+0000 7facc2ffd700 1 -- 172.21.15.7:0/2080953327 <== osd.5 v2:172.21.15.117:6824/644345713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faccc138610 con 0x7facb0073660 2023-06-09T11:10:59.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7facb0073660 msgr2=0x7facb0075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7facb0073660 0x7facb0075a70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006adb0 msgr2=0x7facb006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006adb0 0x7facb006d260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7facc400b450 tx=0x7facc400d040 comp rx=0 tx=0).stop 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 msgr2=0x7faccc138930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd45ff700 1 --2- 172.21.15.7:0/2080953327 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faccc15db30 0x7faccc138930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7facc8004000 tx=0x7facc80047c0 comp rx=0 tx=0).stop 2023-06-09T11:10:59.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.962+0000 7facd2b9c700 1 -- 172.21.15.7:0/2080953327 reap_dead start 2023-06-09T11:10:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.963+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 shutdown_connections 2023-06-09T11:10:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.963+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 >> 172.21.15.7:0/2080953327 conn(0x7faccc06a4d0 msgr2=0x7faccc072460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:10:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.963+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 shutdown_connections 2023-06-09T11:10:59.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:10:59.963+0000 7facd45ff700 1 -- 172.21.15.7:0/2080953327 wait complete. 2023-06-09T11:10:59.996 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:10:59.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-09T11:10:59.997 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:10:59.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-09T11:10:59.998 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:10:59.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-09T11:10:59.999 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:10:59.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-09T11:11:00.000 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:00.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-09T11:11:00.001 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:00.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-09T11:11:00.002 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-09T11:11:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.392+0000 7fe776bff700 1 Processor -- start 2023-06-09T11:11:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.392+0000 7fe776bff700 1 -- start start 2023-06-09T11:11:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.394+0000 7fe776bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe7701563f0 0x7fe770158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.394+0000 7fe776bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.394+0000 7fe776bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe770159660 con 0x7fe77006afe0 2023-06-09T11:11:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.394+0000 7fe776bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe770155720 con 0x7fe7701563f0 2023-06-09T11:11:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.394+0000 7fe776bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe770159c20 con 0x7fe7701591d0 2023-06-09T11:11:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.396+0000 7fe77519c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.396+0000 7fe77519c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47752/0 (socket says 172.21.15.7:47752) 2023-06-09T11:11:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.396+0000 7fe77519c700 1 -- 172.21.15.7:0/2728232337 learned_addr learned my addr 172.21.15.7:0/2728232337 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.396+0000 7fe77499b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe77006afe0 0x7fe7700c0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49954/0 (socket says 172.21.15.7:49954) 2023-06-09T11:11:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.397+0000 7fe77519c700 1 -- 172.21.15.7:0/2728232337 >> v2:172.21.15.7:3300/0 conn(0x7fe7701563f0 msgr2=0x7fe770158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.397+0000 7fe77519c700 1 --2- 172.21.15.7:0/2728232337 >> v2:172.21.15.7:3300/0 conn(0x7fe7701563f0 0x7fe770158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7fe76f7fe700 1 -- 172.21.15.7:0/2728232337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525046737 0 0) 0x7fe770159660 con 0x7fe77006afe0 2023-06-09T11:11:00.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.395+0000 7f051e3ff700 1 Processor -- start 2023-06-09T11:11:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7fe77519c700 1 -- 172.21.15.7:0/2728232337 >> v1:172.21.15.7:6789/0 conn(0x7fe77006afe0 legacy=0x7fe7700c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7fe77519c700 1 -- 172.21.15.7:0/2728232337 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770159940 con 0x7fe7701591d0 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.395+0000 7f051e3ff700 1 -- start start 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7f051e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7f051e3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05180c7bb0 0x7f05180c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7f051e3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05180c85f0 con 0x7f05180c6880 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7f051e3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05180cb1f0 con 0x7f05180c7bb0 2023-06-09T11:11:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.398+0000 7f051e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05180cb4d0 con 0x7f0518157c70 2023-06-09T11:11:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.399+0000 7f0517fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.399+0000 7f0517fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47756/0 (socket says 172.21.15.7:47756) 2023-06-09T11:11:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.399+0000 7f0517fff700 1 -- 172.21.15.7:0/612632371 learned_addr learned my addr 172.21.15.7:0/612632371 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.400+0000 7fe77519c700 1 --2- 172.21.15.7:0/2728232337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7600098f0 tx=0x7fe760009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c65d76012b1235 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.400+0000 7f05177fe700 1 --2- 172.21.15.7:0/612632371 >> v2:172.21.15.7:3300/0 conn(0x7f05180c7bb0 0x7f05180c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7f0517fff700 1 -- 172.21.15.7:0/612632371 >> v2:172.21.15.7:3300/0 conn(0x7f05180c7bb0 msgr2=0x7f05180c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7f0517fff700 1 --2- 172.21.15.7:0/612632371 >> v2:172.21.15.7:3300/0 conn(0x7f05180c7bb0 0x7f05180c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7f0517fff700 1 -- 172.21.15.7:0/612632371 >> v1:172.21.15.7:6789/0 conn(0x7f05180c6880 legacy=0x7f051815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7f0517fff700 1 -- 172.21.15.7:0/612632371 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05180cb7b0 con 0x7f0518157c70 2023-06-09T11:11:00.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7f0517fff700 1 --2- 172.21.15.7:0/612632371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05080098f0 tx=0x7f0508009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a1cc683e5e79ca7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.399+0000 7f023b9ff700 1 Processor -- start 2023-06-09T11:11:00.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.400+0000 7f023b9ff700 1 -- start start 2023-06-09T11:11:00.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f023b9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0234157c70 0x7f02340c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f023b9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f023b9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02340c85f0 con 0x7f02340c6880 2023-06-09T11:11:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f023b9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02340cb1f0 con 0x7f0234157c70 2023-06-09T11:11:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f023b9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02340cb4d0 con 0x7f02340c7bb0 2023-06-09T11:11:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7fe76f7fe700 1 -- 172.21.15.7:0/2728232337 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000b8a0 con 0x7fe7701591d0 2023-06-09T11:11:00.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7fe76f7fe700 1 -- 172.21.15.7:0/2728232337 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76000ba00 con 0x7fe7701591d0 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.401+0000 7fe76f7fe700 1 -- 172.21.15.7:0/2728232337 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000bd90 con 0x7fe7701591d0 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 -- 172.21.15.7:0/2728232337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 msgr2=0x7fe77014f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 --2- 172.21.15.7:0/2728232337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7600098f0 tx=0x7fe760009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 -- 172.21.15.7:0/2728232337 shutdown_connections 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 --2- 172.21.15.7:0/2728232337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7701591d0 0x7fe77014f650 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 --2- 172.21.15.7:0/2728232337 >> v2:172.21.15.7:3300/0 conn(0x7fe7701563f0 0x7fe770158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7fe776bff700 1 -- 172.21.15.7:0/2728232337 >> 172.21.15.7:0/2728232337 conn(0x7fe77006a4d0 msgr2=0x7fe77006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f0516ffd700 1 -- 172.21.15.7:0/612632371 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508011830 con 0x7f0518157c70 2023-06-09T11:11:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f0516ffd700 1 -- 172.21.15.7:0/612632371 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0508011990 con 0x7f0518157c70 2023-06-09T11:11:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.402+0000 7f0516ffd700 1 -- 172.21.15.7:0/612632371 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0508011d20 con 0x7f0518157c70 2023-06-09T11:11:00.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7fe776bff700 1 -- 172.21.15.7:0/2728232337 shutdown_connections 2023-06-09T11:11:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 -- 172.21.15.7:0/612632371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 msgr2=0x7f05180c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 --2- 172.21.15.7:0/612632371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f05080098f0 tx=0x7f0508009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7fe776bff700 1 -- 172.21.15.7:0/2728232337 wait complete. 2023-06-09T11:11:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f0238f9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f0238f9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47758/0 (socket says 172.21.15.7:47758) 2023-06-09T11:11:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f0238f9a700 1 -- 172.21.15.7:0/2732056418 learned_addr learned my addr 172.21.15.7:0/2732056418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 -- 172.21.15.7:0/612632371 shutdown_connections 2023-06-09T11:11:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 --2- 172.21.15.7:0/612632371 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0518157c70 0x7f05180c7510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 --2- 172.21.15.7:0/612632371 >> v2:172.21.15.7:3300/0 conn(0x7f05180c7bb0 0x7f05180c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.403+0000 7f051e3ff700 1 -- 172.21.15.7:0/612632371 >> 172.21.15.7:0/612632371 conn(0x7f051806a4d0 msgr2=0x7f051806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f051e3ff700 1 -- 172.21.15.7:0/612632371 shutdown_connections 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f0238f9a700 1 -- 172.21.15.7:0/2732056418 >> v2:172.21.15.117:3300/0 conn(0x7f0234157c70 msgr2=0x7f02340c7510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f0238f9a700 1 --2- 172.21.15.7:0/2732056418 >> v2:172.21.15.117:3300/0 conn(0x7f0234157c70 0x7f02340c7510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f0238f9a700 1 -- 172.21.15.7:0/2732056418 >> v1:172.21.15.7:6789/0 conn(0x7f02340c6880 legacy=0x7f023415db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f0238f9a700 1 -- 172.21.15.7:0/2732056418 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02340cb7b0 con 0x7f02340c7bb0 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.405+0000 7f0238f9a700 1 --2- 172.21.15.7:0/2732056418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02300098f0 tx=0x7f0230009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73a3359bf1db5bf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.406+0000 7fe776bff700 1 Processor -- start 2023-06-09T11:11:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.404+0000 7f051e3ff700 1 -- 172.21.15.7:0/612632371 wait complete. 2023-06-09T11:11:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.406+0000 7f051e3ff700 1 Processor -- start 2023-06-09T11:11:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.407+0000 7f051e3ff700 1 -- start start 2023-06-09T11:11:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.406+0000 7fe776bff700 1 -- start start 2023-06-09T11:11:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180c6880 0x7f0518156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 0x7f0518070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05180ae7c0 con 0x7f0518157c70 2023-06-09T11:11:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05181414a0 con 0x7f05180c7bb0 2023-06-09T11:11:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7f051e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0518142d70 con 0x7f05180c6880 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7701563f0 0x7fe7700c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7701591d0 0x7fe7701520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c41f0 con 0x7fe7701563f0 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700c3c10 con 0x7fe7701591d0 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.408+0000 7fe776bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe770072620 con 0x7fe77006afe0 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47762/0 (socket says 172.21.15.7:47762) 2023-06-09T11:11:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 -- 172.21.15.7:0/2097986078 learned_addr learned my addr 172.21.15.7:0/2097986078 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f05177fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f022bfff700 1 -- 172.21.15.7:0/2732056418 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000b8a0 con 0x7f02340c7bb0 2023-06-09T11:11:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f022bfff700 1 -- 172.21.15.7:0/2732056418 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023000ba00 con 0x7f02340c7bb0 2023-06-09T11:11:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f022bfff700 1 -- 172.21.15.7:0/2732056418 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000bd90 con 0x7f02340c7bb0 2023-06-09T11:11:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f0517fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180c6880 0x7f0518156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f05177fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41402/0 (socket says 172.21.15.7:41402) 2023-06-09T11:11:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7f05177fe700 1 -- 172.21.15.7:0/3217118673 learned_addr learned my addr 172.21.15.7:0/3217118673 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe76ffff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7701563f0 0x7fe7700c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 -- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7701591d0 msgr2=0x7fe7701520b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T11:11:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7701591d0 0x7fe7701520b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 -- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7701563f0 msgr2=0x7fe7700c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7701563f0 0x7fe7700c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.409+0000 7fe77499b700 1 -- 172.21.15.7:0/2097986078 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760009550 con 0x7fe77006afe0 2023-06-09T11:11:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f051c99c700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 0x7f0518070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 -- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180c6880 msgr2=0x7f0518156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180c6880 0x7f0518156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 -- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 msgr2=0x7f0518070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 0x7f0518070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 -- 172.21.15.7:0/3217118673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0508009550 con 0x7f05180c7bb0 2023-06-09T11:11:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f051c99c700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 0x7f0518070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 -- 172.21.15.7:0/2732056418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 msgr2=0x7f02340c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 --2- 172.21.15.7:0/2732056418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f02300098f0 tx=0x7f0230009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 -- 172.21.15.7:0/2732056418 shutdown_connections 2023-06-09T11:11:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 --2- 172.21.15.7:0/2732056418 >> v2:172.21.15.117:3300/0 conn(0x7f0234157c70 0x7f02340c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 --2- 172.21.15.7:0/2732056418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02340c7bb0 0x7f02340c8000 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f023b9ff700 1 -- 172.21.15.7:0/2732056418 >> 172.21.15.7:0/2732056418 conn(0x7f023406a4d0 msgr2=0x7f023406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7f05177fe700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9b0 tx=0x7f050c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.411+0000 7f023b9ff700 1 -- 172.21.15.7:0/2732056418 shutdown_connections 2023-06-09T11:11:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.411+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c009710 con 0x7f05180c7bb0 2023-06-09T11:11:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.411+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c00c850 con 0x7f05180c7bb0 2023-06-09T11:11:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.411+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c00cbc0 con 0x7f05180c7bb0 2023-06-09T11:11:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7fe77499b700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c550 tx=0x7fe76400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764004c80 con 0x7fe77006afe0 2023-06-09T11:11:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe764004e00 con 0x7fe77006afe0 2023-06-09T11:11:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.410+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764007310 con 0x7fe77006afe0 2023-06-09T11:11:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.412+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe770155bf0 con 0x7fe77006afe0 2023-06-09T11:11:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.412+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770155910 con 0x7fe77006afe0 2023-06-09T11:11:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.412+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05180cbd70 con 0x7f05180c7bb0 2023-06-09T11:11:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.412+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05180cba90 con 0x7f05180c7bb0 2023-06-09T11:11:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.414+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe770155ed0 con 0x7fe77006afe0 2023-06-09T11:11:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.413+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0518158040 con 0x7f05180c7bb0 2023-06-09T11:11:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.411+0000 7f023b9ff700 1 -- 172.21.15.7:0/2732056418 wait complete. 2023-06-09T11:11:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.414+0000 7f023b9ff700 1 Processor -- start 2023-06-09T11:11:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.415+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f050c00cd20 con 0x7f05180c7bb0 2023-06-09T11:11:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.414+0000 7f023b9ff700 1 -- start start 2023-06-09T11:11:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02340c6880 0x7f02340716b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0234157c70 0x7f023415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02340c3330 con 0x7f02340c6880 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234156f90 con 0x7f02340c7bb0 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f023b9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0234158330 con 0x7f0234157c70 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f0238f9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f0238f9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41418/0 (socket says 172.21.15.7:41418) 2023-06-09T11:11:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f0238f9a700 1 -- 172.21.15.7:0/3149687921 learned_addr learned my addr 172.21.15.7:0/3149687921 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.416+0000 7f0514ff9700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 0x7f050406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f050c0898e0 con 0x7f05180c7bb0 2023-06-09T11:11:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f22efdff700 1 Processor -- start 2023-06-09T11:11:00.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0239f9c700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0234157c70 0x7f023415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f023979b700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02340c6880 0x7f02340716b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0238f9a700 1 -- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0234157c70 msgr2=0x7f023415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0238f9a700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0234157c70 0x7f023415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0238f9a700 1 -- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02340c6880 msgr2=0x7f02340716b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0517fff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 0x7f050406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0238f9a700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02340c6880 0x7f02340716b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f0238f9a700 1 -- 172.21.15.7:0/3149687921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009550 con 0x7f02340c7bb0 2023-06-09T11:11:00.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.418+0000 7f0238f9a700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02340cb610 tx=0x7f0230004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.417+0000 7f22efdff700 1 -- start start 2023-06-09T11:11:00.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.418+0000 7f0517fff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 0x7f050406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0508009c40 tx=0x7f050801d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.419+0000 7f22efdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22efdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22e814fc00 0x7f22e80cb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22efdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e80c0bb0 con 0x7f22e81563f0 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22efdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e8155720 con 0x7f22e814fc00 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22efdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e8159c20 con 0x7f22e814f830 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22edb9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22edb9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47794/0 (socket says 172.21.15.7:47794) 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7f22edb9b700 1 -- 172.21.15.7:0/1188473825 learned_addr learned my addr 172.21.15.7:0/1188473825 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.420+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe764007470 con 0x7fe77006afe0 2023-06-09T11:11:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.421+0000 7f22edb9b700 1 -- 172.21.15.7:0/1188473825 >> v2:172.21.15.7:3300/0 conn(0x7f22e814fc00 msgr2=0x7f22e80cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.421+0000 7f22edb9b700 1 --2- 172.21.15.7:0/1188473825 >> v2:172.21.15.7:3300/0 conn(0x7f22e814fc00 0x7f22e80cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.421+0000 7f22edb9b700 1 -- 172.21.15.7:0/1188473825 >> v1:172.21.15.117:6789/0 conn(0x7f22e81563f0 legacy=0x7f22e8158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.421+0000 7f22edb9b700 1 -- 172.21.15.7:0/1188473825 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80c7940 con 0x7f22e814f830 2023-06-09T11:11:00.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.421+0000 7f22edb9b700 1 --2- 172.21.15.7:0/1188473825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009bd0 tx=0x7f22e000d3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27cea35e3f54ff4d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300036a0 con 0x7f02340c7bb0 2023-06-09T11:11:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0230003820 con 0x7f02340c7bb0 2023-06-09T11:11:00.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230003bd0 con 0x7f02340c7bb0 2023-06-09T11:11:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f050c054a00 con 0x7f05180c7bb0 2023-06-09T11:11:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7eff53bff700 1 Processor -- start 2023-06-09T11:11:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7fe76d7fa700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 0x7fe75c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe764089480 con 0x7fe77006afe0 2023-06-09T11:11:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7fe4c79ff700 1 Processor -- start 2023-06-09T11:11:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7fe4c79ff700 1 -- start start 2023-06-09T11:11:00.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7eff53bff700 1 -- start start 2023-06-09T11:11:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02340cbd70 con 0x7f02340c7bb0 2023-06-09T11:11:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.422+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02340cba90 con 0x7f02340c7bb0 2023-06-09T11:11:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.424+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234158040 con 0x7f02340c7bb0 2023-06-09T11:11:00.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.424+0000 7fe76ffff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 0x7fe75c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.424+0000 7fe4c79ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.424+0000 7fe4c79ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4c01610f0 0x7fe4c0161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c79ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00c4fc0 con 0x7fe4c0071fa0 2023-06-09T11:11:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c79ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c00c5860 con 0x7fe4c01610f0 2023-06-09T11:11:00.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c79ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c0161ab0 con 0x7fe4c015b120 2023-06-09T11:11:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.424+0000 7eff53bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff4c153760 0x7eff4c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7eff53bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7eff4c071fa0 0x7eff4c072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7eff53bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4c15a390 con 0x7eff4c153b30 2023-06-09T11:11:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7eff53bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c0c5f40 con 0x7eff4c153760 2023-06-09T11:11:00.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7eff53bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c0728b0 con 0x7eff4c071fa0 2023-06-09T11:11:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c579b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe76ffff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 0x7fe75c06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7700709c0 tx=0x7fe76800d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c579b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47800/0 (socket says 172.21.15.7:47800) 2023-06-09T11:11:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.425+0000 7fe4c579b700 1 -- 172.21.15.7:0/1453325928 learned_addr learned my addr 172.21.15.7:0/1453325928 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7fe4c579b700 1 -- 172.21.15.7:0/1453325928 >> v2:172.21.15.7:3300/0 conn(0x7fe4c01610f0 msgr2=0x7fe4c0161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7fe4c579b700 1 --2- 172.21.15.7:0/1453325928 >> v2:172.21.15.7:3300/0 conn(0x7fe4c01610f0 0x7fe4c0161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7eff4c071fa0 0x7eff4c072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7eff4c071fa0 0x7eff4c072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41426/0 (socket says 172.21.15.7:41426) 2023-06-09T11:11:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 -- 172.21.15.7:0/3784270228 learned_addr learned my addr 172.21.15.7:0/3784270228 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7fe4c579b700 1 -- 172.21.15.7:0/1453325928 >> v1:172.21.15.7:6789/0 conn(0x7fe4c0071fa0 legacy=0x7fe4c00cab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7fe4c579b700 1 -- 172.21.15.7:0/1453325928 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0072500 con 0x7fe4c015b120 2023-06-09T11:11:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5199b700 1 --2- 172.21.15.7:0/3784270228 >> v2:172.21.15.7:3300/0 conn(0x7eff4c153760 0x7eff4c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7fe4c579b700 1 --2- 172.21.15.7:0/1453325928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80098f0 tx=0x7fe4b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=863157ee8a44cba2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 -- 172.21.15.7:0/3784270228 >> v2:172.21.15.7:3300/0 conn(0x7eff4c153760 msgr2=0x7eff4c0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 --2- 172.21.15.7:0/3784270228 >> v2:172.21.15.7:3300/0 conn(0x7eff4c153760 0x7eff4c0ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 -- 172.21.15.7:0/3784270228 >> v1:172.21.15.117:6789/0 conn(0x7eff4c153b30 legacy=0x7eff4c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1453325928 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800b8a0 con 0x7fe4c015b120 2023-06-09T11:11:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1453325928 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b800ba00 con 0x7fe4c015b120 2023-06-09T11:11:00.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1453325928 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800bd90 con 0x7fe4c015b120 2023-06-09T11:11:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7f22ecb99700 1 -- 172.21.15.7:0/1188473825 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000dd10 con 0x7f22e814f830 2023-06-09T11:11:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7f22ecb99700 1 -- 172.21.15.7:0/1188473825 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e000de70 con 0x7f22e814f830 2023-06-09T11:11:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7f22ecb99700 1 -- 172.21.15.7:0/1188473825 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e000e650 con 0x7f22e814f830 2023-06-09T11:11:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.426+0000 7eff5219c700 1 -- 172.21.15.7:0/3784270228 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0ca720 con 0x7eff4c071fa0 2023-06-09T11:11:00.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7eff5219c700 1 --2- 172.21.15.7:0/3784270228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c071fa0 0x7eff4c072370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff4c08d540 tx=0x7eff3c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9676693416126ae9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0230003d30 con 0x7f02340c7bb0 2023-06-09T11:11:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.427+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe7640546a0 con 0x7fe77006afe0 2023-06-09T11:11:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 -- 172.21.15.7:0/1188473825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 msgr2=0x7f22e80c0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 --2- 172.21.15.7:0/1188473825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009bd0 tx=0x7f22e000d3c0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1453325928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 msgr2=0x7fe4c00ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1453325928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80098f0 tx=0x7fe4b8009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1453325928 shutdown_connections 2023-06-09T11:11:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1453325928 >> v2:172.21.15.7:3300/0 conn(0x7fe4c01610f0 0x7fe4c0161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1453325928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c00ca070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1453325928 >> 172.21.15.7:0/1453325928 conn(0x7fe4c00737f0 msgr2=0x7fe4c0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 -- 172.21.15.7:0/1188473825 shutdown_connections 2023-06-09T11:11:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 --2- 172.21.15.7:0/1188473825 >> v2:172.21.15.7:3300/0 conn(0x7f22e814fc00 0x7f22e80cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 --2- 172.21.15.7:0/1188473825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e814f830 0x7f22e80c0670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.428+0000 7f22efdff700 1 -- 172.21.15.7:0/1188473825 >> 172.21.15.7:0/1188473825 conn(0x7f22e806a4d0 msgr2=0x7f22e806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1453325928 shutdown_connections 2023-06-09T11:11:00.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7f22efdff700 1 -- 172.21.15.7:0/1188473825 shutdown_connections 2023-06-09T11:11:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7f0229ffb700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 0x7f021806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f023008abf0 con 0x7f02340c7bb0 2023-06-09T11:11:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7f22efdff700 1 -- 172.21.15.7:0/1188473825 wait complete. 2023-06-09T11:11:00.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.430+0000 7f023979b700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 0x7f021806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.431+0000 7eff50999700 1 -- 172.21.15.7:0/3784270228 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c004230 con 0x7eff4c071fa0 2023-06-09T11:11:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.431+0000 7eff50999700 1 -- 172.21.15.7:0/3784270228 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c004390 con 0x7eff4c071fa0 2023-06-09T11:11:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.431+0000 7eff50999700 1 -- 172.21.15.7:0/3784270228 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0036a0 con 0x7eff4c071fa0 2023-06-09T11:11:00.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.429+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1453325928 wait complete. 2023-06-09T11:11:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7fe4c79ff700 1 Processor -- start 2023-06-09T11:11:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7f22efdff700 1 Processor -- start 2023-06-09T11:11:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7f023979b700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 0x7f021806d210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0224009910 tx=0x7f0224008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7fe4c79ff700 1 -- start start 2023-06-09T11:11:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0230055d10 con 0x7f02340c7bb0 2023-06-09T11:11:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.433+0000 7eff53bff700 1 -- 172.21.15.7:0/3784270228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c071fa0 msgr2=0x7eff4c072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.433+0000 7eff53bff700 1 --2- 172.21.15.7:0/3784270228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c071fa0 0x7eff4c072370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7eff4c08d540 tx=0x7eff3c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.432+0000 7f22efdff700 1 -- start start 2023-06-09T11:11:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.433+0000 7f3beffff700 1 Processor -- start 2023-06-09T11:11:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 0x7fe4c01ee070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c01fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 0x7fe4c01fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00b5500 con 0x7fe4c01610f0 2023-06-09T11:11:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00c9420 con 0x7fe4c0071fa0 2023-06-09T11:11:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c79ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00c9990 con 0x7fe4c015b120 2023-06-09T11:11:00.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22e814fc00 0x7f22e814e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 0x7f22e814e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81f34a0 con 0x7f22e814fc00 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c3790 con 0x7f22e814f830 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f22efdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80c39e0 con 0x7f22e81563f0 2023-06-09T11:11:00.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7eff53bff700 1 -- 172.21.15.7:0/3784270228 shutdown_connections 2023-06-09T11:11:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7eff53bff700 1 --2- 172.21.15.7:0/3784270228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c071fa0 0x7eff4c072370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7eff53bff700 1 --2- 172.21.15.7:0/3784270228 >> v2:172.21.15.7:3300/0 conn(0x7eff4c153760 0x7eff4c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7eff53bff700 1 -- 172.21.15.7:0/3784270228 >> 172.21.15.7:0/3784270228 conn(0x7eff4c0737f0 msgr2=0x7eff4c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7f3beffff700 1 -- start start 2023-06-09T11:11:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7f22edb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7f22edb9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41436/0 (socket says 172.21.15.7:41436) 2023-06-09T11:11:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7f22edb9b700 1 -- 172.21.15.7:0/2036411275 learned_addr learned my addr 172.21.15.7:0/2036411275 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7f22ee39c700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 0x7f22e814e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c579b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 0x7fe4c01ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c579b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 0x7fe4c01ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41430/0 (socket says 172.21.15.7:41430) 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7fe4c579b700 1 -- 172.21.15.7:0/1603645941 learned_addr learned my addr 172.21.15.7:0/1603645941 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c01fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c5f9c700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 0x7fe4c01fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 -- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 msgr2=0x7fe4c01ee070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 0x7fe4c01ee070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 -- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 msgr2=0x7fe4c01fbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 0x7fe4c01fbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c4f9a700 1 -- 172.21.15.7:0/1603645941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009550 con 0x7fe4c015b120 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7fe4c5f9c700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 0x7fe4c01fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7fe4c4f9a700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c01fb920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00cb40 tx=0x7fe4bc00cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22edb9b700 1 -- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 msgr2=0x7f22e814e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22edb9b700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 0x7f22e814e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22edb9b700 1 -- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22e814fc00 msgr2=0x7f22e814e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22edb9b700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22e814fc00 0x7f22e814e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22edb9b700 1 -- 172.21.15.7:0/2036411275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0009830 con 0x7f22e814f830 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f22ee39c700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 0x7f22e814e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:00.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc003c00 con 0x7fe4c015b120 2023-06-09T11:11:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc003d60 con 0x7fe4c015b120 2023-06-09T11:11:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00ab50 con 0x7fe4c015b120 2023-06-09T11:11:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7f22edb9b700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d7c0 tx=0x7f22e0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7eff53bff700 1 -- 172.21.15.7:0/3784270228 shutdown_connections 2023-06-09T11:11:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f3beffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3be81563f0 0x7f3be8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f3beffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3be81591d0 0x7f3be814f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f3beffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be8159660 con 0x7f3be806afe0 2023-06-09T11:11:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f3beffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be8155720 con 0x7f3be81591d0 2023-06-09T11:11:00.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.436+0000 7f3beffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be8159c20 con 0x7f3be81563f0 2023-06-09T11:11:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c00c5210 con 0x7fe4c015b120 2023-06-09T11:11:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c00727e0 con 0x7fe4c015b120 2023-06-09T11:11:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0003a20 con 0x7f22e814f830 2023-06-09T11:11:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0003ba0 con 0x7f22e814f830 2023-06-09T11:11:00.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0004050 con 0x7f22e814f830 2023-06-09T11:11:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f3bed59a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3be81563f0 0x7f3be8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f3bedd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3be806afe0 0x7f3be80c0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49994/0 (socket says 172.21.15.7:49994) 2023-06-09T11:11:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f3bedd9b700 1 -- 172.21.15.7:0/583451318 learned_addr learned my addr 172.21.15.7:0/583451318 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:00.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.434+0000 7efddd1ff700 1 Processor -- start 2023-06-09T11:11:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.435+0000 7efddd1ff700 1 -- start start 2023-06-09T11:11:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7efddd1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdd8157c70 0x7efdd80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7efddd1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd80c85f0 con 0x7efdd80c8030 2023-06-09T11:11:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7efddd1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd80cb1f0 con 0x7efdd80c7bb0 2023-06-09T11:11:00.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.437+0000 7efddd1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd80cb4d0 con 0x7efdd8157c70 2023-06-09T11:11:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f3bee59c700 1 --2- 172.21.15.7:0/583451318 >> v2:172.21.15.7:3300/0 conn(0x7f3be81591d0 0x7f3be814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e8159660 con 0x7f22e814f830 2023-06-09T11:11:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.438+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81593a0 con 0x7f22e814f830 2023-06-09T11:11:00.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e8159940 con 0x7f22e814f830 2023-06-09T11:11:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7efdd6d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdd8157c70 0x7efdd80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7efdd6d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efdd8157c70 0x7efdd80c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41458/0 (socket says 172.21.15.7:41458) 2023-06-09T11:11:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7efdd6d9d700 1 -- 172.21.15.7:0/3564597389 learned_addr learned my addr 172.21.15.7:0/3564597389 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7f3bed59a700 1 -- 172.21.15.7:0/583451318 >> v2:172.21.15.7:3300/0 conn(0x7f3be81591d0 msgr2=0x7f3be814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7f3bed59a700 1 --2- 172.21.15.7:0/583451318 >> v2:172.21.15.7:3300/0 conn(0x7f3be81591d0 0x7f3be814f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7f3bed59a700 1 -- 172.21.15.7:0/583451318 >> v1:172.21.15.7:6789/0 conn(0x7f3be806afe0 legacy=0x7f3be80c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7f3bed59a700 1 -- 172.21.15.7:0/583451318 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8159940 con 0x7f3be81563f0 2023-06-09T11:11:00.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.440+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c00c54f0 con 0x7fe4c015b120 2023-06-09T11:11:00.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.439+0000 7eff53bff700 1 -- 172.21.15.7:0/3784270228 wait complete. 2023-06-09T11:11:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.441+0000 7eff53bff700 1 Processor -- start 2023-06-09T11:11:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.442+0000 7eff53bff700 1 -- start start 2023-06-09T11:11:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.441+0000 7efdd6d9d700 1 -- 172.21.15.7:0/3564597389 >> v1:172.21.15.117:6789/0 conn(0x7efdd80c7bb0 legacy=0x7efdd80c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.441+0000 7efdd6d9d700 1 -- 172.21.15.7:0/3564597389 >> v1:172.21.15.7:6789/0 conn(0x7efdd80c8030 legacy=0x7efdd815db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:00.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.441+0000 7efdd6d9d700 1 -- 172.21.15.7:0/3564597389 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd80cb7b0 con 0x7efdd8157c70 2023-06-09T11:11:00.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.442+0000 7f3bed59a700 1 --2- 172.21.15.7:0/583451318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be81563f0 0x7f3be8158c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3be80c7b20 tx=0x7f3bd8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b01eb5211f930062 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.442+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4bc00acb0 con 0x7fe4c015b120 2023-06-09T11:11:00.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.443+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22e00041b0 con 0x7f22e814f830 2023-06-09T11:11:00.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.443+0000 7efdd6d9d700 1 --2- 172.21.15.7:0/3564597389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd80c7510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098f0 tx=0x7efdcc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db11ce03ef950a5b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7fe4b1ffb700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 0x7fe4a406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4bc089470 con 0x7fe4c015b120 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff4c153760 0x7eff4c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c153b30 0x7eff4c0c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1ff520 con 0x7eff4c071fa0 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0c8bd0 con 0x7eff4c153b30 2023-06-09T11:11:00.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff53bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c157690 con 0x7eff4c153760 2023-06-09T11:11:00.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7fe4c579b700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 0x7fe4a406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff5199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff5199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36744/0 (socket says 172.21.15.7:36744) 2023-06-09T11:11:00.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.444+0000 7eff5199b700 1 -- 172.21.15.7:0/3937943818 learned_addr learned my addr 172.21.15.7:0/3937943818 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5219c700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c153b30 0x7eff4c0c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7f22de7fc700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 0x7f22cc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5119a700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff4c153760 0x7eff4c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f22e008c0a0 con 0x7f22e814f830 2023-06-09T11:11:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7f22ed39a700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 0x7f22cc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f22e00571c0 con 0x7f22e814f830 2023-06-09T11:11:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7f22ed39a700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 0x7f22cc06d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000f60 tx=0x7f22d8007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 -- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff4c153760 msgr2=0x7eff4c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff4c153760 0x7eff4c0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 -- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c153b30 msgr2=0x7eff4c0c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c153b30 0x7eff4c0c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 -- 172.21.15.7:0/3937943818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c009550 con 0x7eff4c071fa0 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7eff5199b700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff44007ef0 tx=0x7eff4400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4400eea0 con 0x7eff4c071fa0 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff44014420 con 0x7eff4c071fa0 2023-06-09T11:11:00.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff440147d0 con 0x7eff4c071fa0 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c159780 con 0x7eff4c071fa0 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c0c5a20 con 0x7eff4c071fa0 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7efdd5d9b700 1 -- 172.21.15.7:0/3564597389 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc004230 con 0x7efdd8157c70 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7efdd5d9b700 1 -- 172.21.15.7:0/3564597389 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc004390 con 0x7efdd8157c70 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.446+0000 7efdd5d9b700 1 -- 172.21.15.7:0/3564597389 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0036a0 con 0x7efdd8157c70 2023-06-09T11:11:00.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7efddd1ff700 1 -- 172.21.15.7:0/3564597389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 msgr2=0x7efdd80c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7efddd1ff700 1 --2- 172.21.15.7:0/3564597389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd80c7510 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0098f0 tx=0x7efdcc009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7f3becd99700 1 -- 172.21.15.7:0/583451318 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8011830 con 0x7f3be81563f0 2023-06-09T11:11:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7f3becd99700 1 -- 172.21.15.7:0/583451318 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8011990 con 0x7f3be81563f0 2023-06-09T11:11:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.445+0000 7f3becd99700 1 -- 172.21.15.7:0/583451318 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8011d00 con 0x7f3be81563f0 2023-06-09T11:11:00.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7f3beffff700 1 -- 172.21.15.7:0/583451318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be81563f0 msgr2=0x7f3be8158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7f3beffff700 1 --2- 172.21.15.7:0/583451318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be81563f0 0x7f3be8158c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3be80c7b20 tx=0x7f3bd8009b20 comp rx=0 tx=0).stop 2023-06-09T11:11:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7efddd1ff700 1 -- 172.21.15.7:0/3564597389 shutdown_connections 2023-06-09T11:11:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7efddd1ff700 1 --2- 172.21.15.7:0/3564597389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd80c7510 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.447+0000 7efddd1ff700 1 -- 172.21.15.7:0/3564597389 >> 172.21.15.7:0/3564597389 conn(0x7efdd806a4d0 msgr2=0x7efdd806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7fe4c579b700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 0x7fe4a406d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009c40 tx=0x7fe4b8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff340052e0 con 0x7eff4c071fa0 2023-06-09T11:11:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7f3beffff700 1 -- 172.21.15.7:0/583451318 shutdown_connections 2023-06-09T11:11:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7f3beffff700 1 --2- 172.21.15.7:0/583451318 >> v2:172.21.15.7:3300/0 conn(0x7f3be81591d0 0x7f3be814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7f3beffff700 1 --2- 172.21.15.7:0/583451318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be81563f0 0x7f3be8158c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.448+0000 7f3beffff700 1 -- 172.21.15.7:0/583451318 >> 172.21.15.7:0/583451318 conn(0x7f3be806a4d0 msgr2=0x7f3be806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.449+0000 7efddd1ff700 1 -- 172.21.15.7:0/3564597389 shutdown_connections 2023-06-09T11:11:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.449+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe4bc054590 con 0x7fe4c015b120 2023-06-09T11:11:00.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.451+0000 7f3beffff700 1 -- 172.21.15.7:0/583451318 shutdown_connections 2023-06-09T11:11:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.449+0000 7efddd1ff700 1 -- 172.21.15.7:0/3564597389 wait complete. 2023-06-09T11:11:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.451+0000 7efddd1ff700 1 Processor -- start 2023-06-09T11:11:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.452+0000 7efddd1ff700 1 -- start start 2023-06-09T11:11:00.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 0x7efdd8159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd80c8030 0x7efdd8143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd81596b0 con 0x7efdd80c8030 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd8142da0 con 0x7efdd8157c70 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.453+0000 7efddd1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd80c5510 con 0x7efdd80c7bb0 2023-06-09T11:11:00.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.454+0000 7efdd6d9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 0x7efdd8159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.451+0000 7f3beffff700 1 -- 172.21.15.7:0/583451318 wait complete. 2023-06-09T11:11:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.454+0000 7f3beffff700 1 Processor -- start 2023-06-09T11:11:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.454+0000 7efdd759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.454+0000 7efdd759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd8155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41486/0 (socket says 172.21.15.7:41486) 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.454+0000 7efdd759e700 1 -- 172.21.15.7:0/2838290348 learned_addr learned my addr 172.21.15.7:0/2838290348 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7efdd6d9d700 1 -- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 msgr2=0x7efdd8155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7efdd6d9d700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd8155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7efdd6d9d700 1 -- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd80c8030 msgr2=0x7efdd8143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7efdd6d9d700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd80c8030 0x7efdd8143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7efdd6d9d700 1 -- 172.21.15.7:0/2838290348 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc009550 con 0x7efdd80c7bb0 2023-06-09T11:11:00.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7efdd6d9d700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 0x7efdd8159170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdd80c57c0 tx=0x7efdcc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.457+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff44014930 con 0x7eff4c071fa0 2023-06-09T11:11:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7eff427fc700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 0x7eff3006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff4408aa20 con 0x7eff4c071fa0 2023-06-09T11:11:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011630 con 0x7efdd80c7bb0 2023-06-09T11:11:00.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc011790 con 0x7efdd80c7bb0 2023-06-09T11:11:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.455+0000 7f3beffff700 1 -- start start 2023-06-09T11:11:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be81563f0 0x7f3be8152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 0x7f3be80c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8152b70 con 0x7f3be81563f0 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8144310 con 0x7f3be806afe0 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7f3beffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80c3dc0 con 0x7f3be81591d0 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7f3bee59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 0x7f3be80c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7f3bedd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7f3bedd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41488/0 (socket says 172.21.15.7:41488) 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7f3bee59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 0x7f3be80c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47850/0 (socket says 172.21.15.7:47850) 2023-06-09T11:11:00.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.458+0000 7f3bedd9b700 1 -- 172.21.15.7:0/2790950505 learned_addr learned my addr 172.21.15.7:0/2790950505 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 -- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 msgr2=0x7f3be80c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 0x7f3be80c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 -- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be81563f0 msgr2=0x7f3be8152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be81563f0 0x7f3be8152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 -- 172.21.15.7:0/2790950505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd80094a0 con 0x7f3be806afe0 2023-06-09T11:11:00.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7eff5119a700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 0x7eff3006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.456+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd80cbd70 con 0x7efdd80c7bb0 2023-06-09T11:11:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.457+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd80cba90 con 0x7efdd80c7bb0 2023-06-09T11:11:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdcc011b00 con 0x7efdd80c7bb0 2023-06-09T11:11:00.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7f3bedd9b700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3be000cba0 tx=0x7f3be000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.459+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb80052e0 con 0x7efdd80c7bb0 2023-06-09T11:11:00.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.460+0000 7eff5119a700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 0x7eff3006d260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0c9be0 tx=0x7eff48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.461+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000ab80 con 0x7f3be806afe0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.461+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000ace0 con 0x7f3be806afe0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.461+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0012600 con 0x7f3be806afe0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.461+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be8155bf0 con 0x7f3be806afe0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.461+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be8155910 con 0x7f3be806afe0 2023-06-09T11:11:00.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.462+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7eff44055b40 con 0x7eff4c071fa0 2023-06-09T11:11:00.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.465+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efdcc021070 con 0x7efdd80c7bb0 2023-06-09T11:11:00.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.467+0000 7efdbf7fe700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 0x7efdc406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.467+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efdcc08b1e0 con 0x7efdd80c7bb0 2023-06-09T11:11:00.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.465+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3be001b3b0 con 0x7f3be806afe0 2023-06-09T11:11:00.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.467+0000 7f3bde7fc700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 0x7f3bcc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:00.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.467+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3be00891f0 con 0x7f3be806afe0 2023-06-09T11:11:00.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.468+0000 7efdd659c700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 0x7efdc406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.469+0000 7f3bed59a700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 0x7f3bcc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:00.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.470+0000 7efdd659c700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 0x7efdc406d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efdd8154420 tx=0x7efdc8007a20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.470+0000 7f3bed59a700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 0x7f3bcc06d260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3be80c71d0 tx=0x7f3bd8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:00.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.471+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efdcc056270 con 0x7efdd80c7bb0 2023-06-09T11:11:00.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.472+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd00052e0 con 0x7f3be806afe0 2023-06-09T11:11:00.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.477+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3be0054330 con 0x7f3be806afe0 2023-06-09T11:11:00.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.874+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0518070210 con 0x7f05180c7bb0 2023-06-09T11:11:00.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.875+0000 7f0514ff9700 1 -- 172.21.15.7:0/3217118673 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f050c058020 con 0x7f05180c7bb0 2023-06-09T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.879+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 msgr2=0x7f050406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.879+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 0x7f050406d190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0508009c40 tx=0x7f050801d040 comp rx=0 tx=0).stop 2023-06-09T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.879+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 msgr2=0x7f0518070ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.879+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9b0 tx=0x7f050c00ecc0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 shutdown_connections 2023-06-09T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0518157c70 0x7f0518070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f050406ace0 0x7f050406d190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05180c6880 0x7f0518156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 --2- 172.21.15.7:0/3217118673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05180c7bb0 0x7f0518070ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.880+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 >> 172.21.15.7:0/3217118673 conn(0x7f051806a4d0 msgr2=0x7f05181529b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.881+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 shutdown_connections 2023-06-09T11:11:00.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.881+0000 7f051e3ff700 1 -- 172.21.15.7:0/3217118673 wait complete. 2023-06-09T11:11:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.890+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f023415a180 con 0x7f02340c7bb0 2023-06-09T11:11:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.891+0000 7f0229ffb700 1 -- 172.21.15.7:0/3149687921 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0230059330 con 0x7f02340c7bb0 2023-06-09T11:11:00.897 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.895+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe770074cf0 con 0x7fe77006afe0 2023-06-09T11:11:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.896+0000 7fe76d7fa700 1 -- 172.21.15.7:0/2097986078 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe764057cc0 con 0x7fe77006afe0 2023-06-09T11:11:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.896+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 msgr2=0x7f021806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.896+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 0x7f021806d210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0224009910 tx=0x7f0224008040 comp rx=0 tx=0).stop 2023-06-09T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.896+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 msgr2=0x7f02341591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.896+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f02340cb610 tx=0x7f0230004750 comp rx=0 tx=0).stop 2023-06-09T11:11:00.903 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 shutdown_connections 2023-06-09T11:11:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f021806ad60 0x7f021806d210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0234157c70 0x7f023415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02340c6880 0x7f02340716b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 --2- 172.21.15.7:0/3149687921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02340c7bb0 0x7f02341591e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 >> 172.21.15.7:0/3149687921 conn(0x7f023406a4d0 msgr2=0x7f023406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.899+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 shutdown_connections 2023-06-09T11:11:00.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.900+0000 7f023b9ff700 1 -- 172.21.15.7:0/3149687921 wait complete. 2023-06-09T11:11:00.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 msgr2=0x7fe75c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 0x7fe75c06d1e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7700709c0 tx=0x7fe76800d040 comp rx=0 tx=0).stop 2023-06-09T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 msgr2=0x7fe7700c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe76400c550 tx=0x7fe76400c860 comp rx=0 tx=0).stop 2023-06-09T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 shutdown_connections 2023-06-09T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7701591d0 0x7fe7701520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75c06ad30 0x7fe75c06d1e0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe7700709c0 tx=0x7fe76800d040 comp rx=0 tx=0).stop 2023-06-09T11:11:00.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe77006afe0 0x7fe7700c4680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 --2- 172.21.15.7:0/2097986078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7701563f0 0x7fe7700c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 >> 172.21.15.7:0/2097986078 conn(0x7fe77006a4d0 msgr2=0x7fe770153940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.901+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 shutdown_connections 2023-06-09T11:11:00.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.902+0000 7fe776bff700 1 -- 172.21.15.7:0/2097986078 wait complete. 2023-06-09T11:11:00.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-06-09T11:11:00.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.913+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f22e8074cf0 con 0x7f22e814f830 2023-06-09T11:11:00.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.914+0000 7f22de7fc700 1 -- 172.21.15.7:0/2036411275 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f22e005a7e0 con 0x7f22e814f830 2023-06-09T11:11:00.919 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-06-09T11:11:00.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.918+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 msgr2=0x7f22cc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.918+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 0x7f22cc06d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f22d8000f60 tx=0x7f22d8007a40 comp rx=0 tx=0).stop 2023-06-09T11:11:00.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.918+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 msgr2=0x7f22e80c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.918+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22e000d7c0 tx=0x7f22e0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f3bd0005590 con 0x7f3be806afe0 2023-06-09T11:11:00.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 shutdown_connections 2023-06-09T11:11:00.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22cc06adb0 0x7f22cc06d260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22e81563f0 0x7f22e814e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22e814f830 0x7f22e80c2990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 --2- 172.21.15.7:0/2036411275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22e814fc00 0x7f22e814e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 >> 172.21.15.7:0/2036411275 conn(0x7f22e806a4d0 msgr2=0x7f22e806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 shutdown_connections 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.923+0000 7f22efdff700 1 -- 172.21.15.7:0/2036411275 wait complete. 2023-06-09T11:11:00.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.924+0000 7f3bde7fc700 1 -- 172.21.15.7:0/2790950505 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3be0017020 con 0x7f3be806afe0 2023-06-09T11:11:00.930 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-06-09T11:11:00.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.925+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe4c00c9ad0 con 0x7fe4c015b120 2023-06-09T11:11:00.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.927+0000 7fe4b1ffb700 1 -- 172.21.15.7:0/1603645941 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe4c00c9ad0 con 0x7fe4c015b120 2023-06-09T11:11:00.931 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2023-06-09T11:11:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.927+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efdb8005860 con 0x7efdd80c7bb0 2023-06-09T11:11:00.933 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.928+0000 7efdbf7fe700 1 -- 172.21.15.7:0/2838290348 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efdcc059890 con 0x7efdd80c7bb0 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.928+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 msgr2=0x7f3bcc06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.928+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 0x7f3bcc06d260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3be80c71d0 tx=0x7f3bd8010000 comp rx=0 tx=0).stop 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.928+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 msgr2=0x7f3be80ae040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.928+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3be000cba0 tx=0x7f3be000cf60 comp rx=0 tx=0).stop 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 shutdown_connections 2023-06-09T11:11:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3bcc06adb0 0x7f3bcc06d260 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3be80c71d0 tx=0x7f3bd8010000 comp rx=0 tx=0).stop 2023-06-09T11:11:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3be81591d0 0x7f3be80c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3be806afe0 0x7f3be80ae040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 --2- 172.21.15.7:0/2790950505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3be81563f0 0x7f3be8152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 >> 172.21.15.7:0/2790950505 conn(0x7f3be806a4d0 msgr2=0x7f3be80c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.929+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7eff34005860 con 0x7eff4c071fa0 2023-06-09T11:11:00.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.930+0000 7eff427fc700 1 -- 172.21.15.7:0/3937943818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff44059160 con 0x7eff4c071fa0 2023-06-09T11:11:00.937 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.930+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 shutdown_connections 2023-06-09T11:11:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.930+0000 7f3beffff700 1 -- 172.21.15.7:0/2790950505 wait complete. 2023-06-09T11:11:00.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-06-09T11:11:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 msgr2=0x7fe4a406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 0x7fe4a406d260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009c40 tx=0x7fe4b8008030 comp rx=0 tx=0).stop 2023-06-09T11:11:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 msgr2=0x7fe4c01fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c01fb920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00cb40 tx=0x7fe4bc00cf00 comp rx=0 tx=0).stop 2023-06-09T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 msgr2=0x7efdc406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 0x7efdc406d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efdd8154420 tx=0x7efdc8007a20 comp rx=0 tx=0).stop 2023-06-09T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 msgr2=0x7efdd8159170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 0x7efdd8159170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efdd80c57c0 tx=0x7efdcc004750 comp rx=0 tx=0).stop 2023-06-09T11:11:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 shutdown_connections 2023-06-09T11:11:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4c01610f0 0x7fe4c01fbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4c0071fa0 0x7fe4c01ee070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a406adb0 0x7fe4a406d260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 --2- 172.21.15.7:0/1603645941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4c015b120 0x7fe4c01fb920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.931+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 >> 172.21.15.7:0/1603645941 conn(0x7fe4c00737f0 msgr2=0x7fe4c00c9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 shutdown_connections 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efdd8157c70 0x7efdd8155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efdc406ad30 0x7efdc406d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efdd80c7bb0 0x7efdd8159170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 --2- 172.21.15.7:0/2838290348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efdd80c8030 0x7efdd8143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 >> 172.21.15.7:0/2838290348 conn(0x7efdd806a4d0 msgr2=0x7efdd80720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 shutdown_connections 2023-06-09T11:11:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 shutdown_connections 2023-06-09T11:11:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7fe4c79ff700 1 -- 172.21.15.7:0/1603645941 wait complete. 2023-06-09T11:11:00.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.932+0000 7efddd1ff700 1 -- 172.21.15.7:0/2838290348 wait complete. 2023-06-09T11:11:00.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 msgr2=0x7eff3006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 0x7eff3006d260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0c9be0 tx=0x7eff48008040 comp rx=0 tx=0).stop 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 msgr2=0x7eff4c1fc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7eff44007ef0 tx=0x7eff4400e3c0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 shutdown_connections 2023-06-09T11:11:00.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff4c153b30 0x7eff4c0c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff4c071fa0 0x7eff4c1fc300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff3006adb0 0x7eff3006d260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 --2- 172.21.15.7:0/3937943818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff4c153760 0x7eff4c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 >> 172.21.15.7:0/3937943818 conn(0x7eff4c0737f0 msgr2=0x7eff4c14d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 shutdown_connections 2023-06-09T11:11:00.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:00.935+0000 7eff53bff700 1 -- 172.21.15.7:0/3937943818 wait complete. 2023-06-09T11:11:00.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-06-09T11:11:00.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.5 2023-06-09T11:11:00.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-06-09T11:11:00.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.4 2023-06-09T11:11:00.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-06-09T11:11:01.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-06-09T11:11:01.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-06-09T11:11:01.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-06-09T11:11:01.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-06-09T11:11:01.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-06-09T11:11:01.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-06-09T11:11:01.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-06-09T11:11:01.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-06-09T11:11:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.239+0000 7f44dc1ff700 1 Processor -- start 2023-06-09T11:11:02.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.239+0000 7f44dc1ff700 1 -- start start 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.241+0000 7f44dc1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c5d20 0x7f44d40c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.241+0000 7f44dc1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44d4166b00 0x7f44d4166f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.241+0000 7f44dc1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d4167460 con 0x7f44d40c60f0 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.241+0000 7f44dc1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d4169bc0 con 0x7f44d4166b00 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.241+0000 7f44dc1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d4169ea0 con 0x7f44d40c5d20 2023-06-09T11:11:02.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.243+0000 7f44da79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44d4166b00 0x7f44d4166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.243+0000 7f44da79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44d4166b00 0x7f44d4166f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36770/0 (socket says 172.21.15.7:36770) 2023-06-09T11:11:02.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.243+0000 7f44da79c700 1 -- 172.21.15.7:0/832849065 learned_addr learned my addr 172.21.15.7:0/832849065 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.243+0000 7f44d9f9b700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c5d20 0x7f44d40c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.246+0000 7f44da79c700 1 -- 172.21.15.7:0/832849065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c5d20 msgr2=0x7f44d40c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.246+0000 7f44da79c700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c5d20 0x7f44d40c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.246+0000 7f44da79c700 1 -- 172.21.15.7:0/832849065 >> v1:172.21.15.7:6789/0 conn(0x7f44d40c60f0 legacy=0x7f44d40cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.247+0000 7f44da79c700 1 -- 172.21.15.7:0/832849065 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d416a180 con 0x7f44d4166b00 2023-06-09T11:11:02.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.247+0000 7f44da79c700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d4166b00 0x7f44d4166f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44d0008970 tx=0x7f44d0008c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b771188ce644091d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.244+0000 7f44d8f99700 1 -- 172.21.15.7:0/832849065 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595439142 0 0) 0x7f44d4167460 con 0x7f44d40c60f0 2023-06-09T11:11:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.248+0000 7f44d8f99700 1 -- 172.21.15.7:0/832849065 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0018c60 con 0x7f44d4166b00 2023-06-09T11:11:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.248+0000 7f44d8f99700 1 -- 172.21.15.7:0/832849065 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d0018dc0 con 0x7f44d4166b00 2023-06-09T11:11:02.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.248+0000 7f44d8f99700 1 -- 172.21.15.7:0/832849065 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d000b7d0 con 0x7f44d4166b00 2023-06-09T11:11:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.250+0000 7f44dc1ff700 1 -- 172.21.15.7:0/832849065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d4166b00 msgr2=0x7f44d4166f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.250+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d4166b00 0x7f44d4166f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f44d0008970 tx=0x7f44d0008c80 comp rx=0 tx=0).stop 2023-06-09T11:11:02.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.251+0000 7f44dc1ff700 1 -- 172.21.15.7:0/832849065 shutdown_connections 2023-06-09T11:11:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.251+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d4166b00 0x7f44d4166f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.251+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/832849065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c5d20 0x7f44d40c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.251+0000 7f44dc1ff700 1 -- 172.21.15.7:0/832849065 >> 172.21.15.7:0/832849065 conn(0x7f44d406a4d0 msgr2=0x7f44d406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.252+0000 7f44dc1ff700 1 -- 172.21.15.7:0/832849065 shutdown_connections 2023-06-09T11:11:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.252+0000 7f44dc1ff700 1 -- 172.21.15.7:0/832849065 wait complete. 2023-06-09T11:11:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.255+0000 7f44dc1ff700 1 Processor -- start 2023-06-09T11:11:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.256+0000 7f44dc1ff700 1 -- start start 2023-06-09T11:11:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c60f0 0x7f44d40c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d4166b00 0x7f44d40ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c8fa0 con 0x7f44d40c5d20 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c9e00 con 0x7f44d4166b00 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.258+0000 7f44dc1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c91f0 con 0x7f44d40c60f0 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d979a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c60f0 0x7f44d40c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36774/0 (socket says 172.21.15.7:36774) 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 -- 172.21.15.7:0/3410089202 learned_addr learned my addr 172.21.15.7:0/3410089202 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 -- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c60f0 msgr2=0x7f44d40c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c60f0 0x7f44d40c3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 -- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d4166b00 msgr2=0x7f44d40ca290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.259+0000 7f44d9f9b700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d4166b00 0x7f44d40ca290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.260+0000 7f44d9f9b700 1 -- 172.21.15.7:0/3410089202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00085d0 con 0x7f44d40c5d20 2023-06-09T11:11:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.260+0000 7f44d9f9b700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00be00 tx=0x7f44cc00f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.263+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc010040 con 0x7f44d40c5d20 2023-06-09T11:11:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.263+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00fa20 con 0x7f44d40c5d20 2023-06-09T11:11:02.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.263+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00fdb0 con 0x7f44d40c5d20 2023-06-09T11:11:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.263+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d40cb820 con 0x7f44d40c5d20 2023-06-09T11:11:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.264+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d416a460 con 0x7f44d40c5d20 2023-06-09T11:11:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.269+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44cc009540 con 0x7f44d40c5d20 2023-06-09T11:11:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.271+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc0052e0 con 0x7f44d40c5d20 2023-06-09T11:11:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.273+0000 7f44ca7fc700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 0x7f44b806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.273+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f44cc089bd0 con 0x7f44d40c5d20 2023-06-09T11:11:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.276+0000 7f44d979a700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 0x7f44b806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.277+0000 7f44d979a700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 0x7f44b806d260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44d4166340 tx=0x7f44c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.281+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44cc054bd0 con 0x7f44d40c5d20 2023-06-09T11:11:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.321+0000 7f94c3bff700 1 Processor -- start 2023-06-09T11:11:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.321+0000 7f94c3bff700 1 -- start start 2023-06-09T11:11:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.323+0000 7fb3915ff700 1 Processor -- start 2023-06-09T11:11:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.323+0000 7fb3915ff700 1 -- start start 2023-06-09T11:11:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb3915ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb38c1563f0 0x7fb38c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb3915ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0c7940 con 0x7fb38c14f830 2023-06-09T11:11:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb3915ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c159660 con 0x7fb38c1591d0 2023-06-09T11:11:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb3915ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38c155720 con 0x7fb38c1563f0 2023-06-09T11:11:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.323+0000 7f94c3bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f94bc0c7510 0x7f94bc0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.323+0000 7f94c3bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94bc0c6dc0 0x7f94bc15db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.323+0000 7f94c3bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc0c85f0 con 0x7f94bc157c70 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.324+0000 7f94c3bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0c71c0 con 0x7f94bc0c7510 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.324+0000 7f94c3bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc0cb1f0 con 0x7f94bc0c6dc0 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7f94c219c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94bc0c6dc0 0x7f94bc15db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7f94c199b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94bc157c70 0x7f94bc0c7bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50014/0 (socket says 172.21.15.7:50014) 2023-06-09T11:11:02.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7f94c199b700 1 -- 172.21.15.7:0/1492101582 learned_addr learned my addr 172.21.15.7:0/1492101582 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:02.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb38a7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb38c1563f0 0x7fb38c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb38affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb38c14f830 0x7fb38c0c0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50022/0 (socket says 172.21.15.7:50022) 2023-06-09T11:11:02.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.327+0000 7fb38affd700 1 -- 172.21.15.7:0/409872799 learned_addr learned my addr 172.21.15.7:0/409872799 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:02.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.328+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693905233 0 0) 0x7f94bc0c85f0 con 0x7f94bc157c70 2023-06-09T11:11:02.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.328+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a80035b0 con 0x7f94bc157c70 2023-06-09T11:11:02.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.328+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3838127592 0 0) 0x7fb38c0c7940 con 0x7fb38c14f830 2023-06-09T11:11:02.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.328+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3740035b0 con 0x7fb38c14f830 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c119a700 1 --2- 172.21.15.7:0/1492101582 >> v2:172.21.15.7:3300/0 conn(0x7f94bc0c7510 0x7f94bc0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516256114 0 0) 0x7f94a80035b0 con 0x7f94bc157c70 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 >> v2:172.21.15.117:3300/0 conn(0x7f94bc0c6dc0 msgr2=0x7f94bc15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 --2- 172.21.15.7:0/1492101582 >> v2:172.21.15.117:3300/0 conn(0x7f94bc0c6dc0 0x7f94bc15db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 >> v2:172.21.15.7:3300/0 conn(0x7f94bc0c7510 msgr2=0x7f94bc0c6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 --2- 172.21.15.7:0/1492101582 >> v2:172.21.15.7:3300/0 conn(0x7f94bc0c7510 0x7f94bc0c6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0cb4d0 con 0x7f94bc157c70 2023-06-09T11:11:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f94ac001670 con 0x7f94bc157c70 2023-06-09T11:11:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 >> v1:172.21.15.7:6789/0 conn(0x7f94bc157c70 legacy=0x7f94bc0c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 0x7f94a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a8004850 0x7f94a8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8005200 0x7f94a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0cb1f0 con 0x7f94a8003a00 2023-06-09T11:11:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800c060 con 0x7f94a8004850 2023-06-09T11:11:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a800c220 con 0x7f94a8005200 2023-06-09T11:11:02.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c119a700 1 --2- 172.21.15.7:0/1492101582 >> v2:172.21.15.7:3300/0 conn(0x7f94bc0c7510 0x7f94bc0c6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 615974408 0 0) 0x7fb3740035b0 con 0x7fb38c14f830 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 >> v2:172.21.15.117:3300/0 conn(0x7fb38c1563f0 msgr2=0x7fb38c158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 --2- 172.21.15.7:0/409872799 >> v2:172.21.15.117:3300/0 conn(0x7fb38c1563f0 0x7fb38c158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 >> v1:172.21.15.117:6789/0 conn(0x7fb38c1591d0 legacy=0x7fb38c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c159940 con 0x7fb38c14f830 2023-06-09T11:11:02.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb37c000ed0 con 0x7fb38c14f830 2023-06-09T11:11:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 >> v1:172.21.15.7:6789/0 conn(0x7fb38c14f830 legacy=0x7fb38c0c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7f94c199b700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 0x7f94a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 0x7fb374003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb374004850 0x7fb374004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374005200 0x7fb37400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c050 con 0x7fb374005200 2023-06-09T11:11:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c1c0 con 0x7fb374004850 2023-06-09T11:11:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.329+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37400c380 con 0x7fb374003a00 2023-06-09T11:11:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c219c700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8005200 0x7f94a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7fb38affd700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 0x7fb374003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7fb38b7fe700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374005200 0x7fb37400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c199b700 1 -- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8005200 msgr2=0x7f94a800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c199b700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8005200 0x7f94a800ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c199b700 1 -- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a8004850 msgr2=0x7f94a8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c199b700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94a8004850 0x7f94a8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.330+0000 7f94c199b700 1 -- 172.21.15.7:0/1492101582 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0cb4d0 con 0x7f94a8003a00 2023-06-09T11:11:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38a7fc700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb374004850 0x7fb374004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7f94c199b700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 0x7f94a8003e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c340 tx=0x7f94a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f165ae494f97c77f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 -- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb374004850 msgr2=0x7fb374004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb374004850 0x7fb374004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 -- 172.21.15.7:0/409872799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374005200 msgr2=0x7fb37400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374005200 0x7fb37400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 -- 172.21.15.7:0/409872799 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c159940 con 0x7fb374003a00 2023-06-09T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38b7fe700 1 -- 172.21.15.7:0/409872799 reap_dead start 2023-06-09T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7f94c219c700 1 -- 172.21.15.7:0/1492101582 reap_dead start 2023-06-09T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00ce40 con 0x7f94a8003a00 2023-06-09T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac005960 con 0x7f94a8003a00 2023-06-09T11:11:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7f94c0999700 1 -- 172.21.15.7:0/1492101582 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac005d10 con 0x7f94a8003a00 2023-06-09T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.331+0000 7fb38affd700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 0x7fb374003e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d380 tx=0x7fb374005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18d7c84719dc94a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.332+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c015030 con 0x7fb374003a00 2023-06-09T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.332+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c010450 con 0x7fb374003a00 2023-06-09T11:11:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.332+0000 7fb389ffb700 1 -- 172.21.15.7:0/409872799 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c001990 con 0x7fb374003a00 2023-06-09T11:11:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.332+0000 7f94c3bff700 1 -- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 msgr2=0x7f94a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.332+0000 7f94c3bff700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 0x7f94a8003e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c340 tx=0x7f94a8005880 comp rx=0 tx=0).stop 2023-06-09T11:11:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.333+0000 7f94c3bff700 1 -- 172.21.15.7:0/1492101582 shutdown_connections 2023-06-09T11:11:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.333+0000 7f94c3bff700 1 --2- 172.21.15.7:0/1492101582 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94a8003a00 0x7f94a8003e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.333+0000 7f94c3bff700 1 -- 172.21.15.7:0/1492101582 >> 172.21.15.7:0/1492101582 conn(0x7f94bc06a4d0 msgr2=0x7f94bc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.333+0000 7fb3915ff700 1 -- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 msgr2=0x7fb374003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.333+0000 7fb3915ff700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 0x7fb374003e10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00d380 tx=0x7fb374005880 comp rx=0 tx=0).stop 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7fb3915ff700 1 -- 172.21.15.7:0/409872799 shutdown_connections 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7fb3915ff700 1 --2- 172.21.15.7:0/409872799 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb374003a00 0x7fb374003e10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7fb3915ff700 1 -- 172.21.15.7:0/409872799 >> 172.21.15.7:0/409872799 conn(0x7fb38c06a4d0 msgr2=0x7fb38c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7f94c3bff700 1 -- 172.21.15.7:0/1492101582 shutdown_connections 2023-06-09T11:11:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7fb3915ff700 1 -- 172.21.15.7:0/409872799 shutdown_connections 2023-06-09T11:11:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7fb3915ff700 1 -- 172.21.15.7:0/409872799 wait complete. 2023-06-09T11:11:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.334+0000 7f94c3bff700 1 -- 172.21.15.7:0/1492101582 wait complete. 2023-06-09T11:11:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.336+0000 7fb3915ff700 1 Processor -- start 2023-06-09T11:11:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.336+0000 7fb3915ff700 1 -- start start 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.337+0000 7f94c3bff700 1 Processor -- start 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb38c0cbcd0 0x7fb38c1ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb38c0c93d0 0x7fb38c1ead00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c1eec90 con 0x7fb374003a00 2023-06-09T11:11:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c1ee800 con 0x7fb38c0cbcd0 2023-06-09T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.338+0000 7fb3915ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0c98a0 con 0x7fb38c0c93d0 2023-06-09T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7fb38affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7fb38affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36814/0 (socket says 172.21.15.7:36814) 2023-06-09T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7fb38affd700 1 -- 172.21.15.7:0/1765477995 learned_addr learned my addr 172.21.15.7:0/1765477995 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.337+0000 7f94c3bff700 1 -- start start 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8003a00 0x7f94bc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1570a0 0x7f94bc1f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1f7690 con 0x7f94bc1570a0 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1f3440 con 0x7f94bc158d70 2023-06-09T11:11:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f94c3bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1f6b30 con 0x7f94a8003a00 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 -- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb38c0c93d0 msgr2=0x7fb38c1ead00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb38c0c93d0 0x7fb38c1ead00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 -- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb38c0cbcd0 msgr2=0x7fb38c1ea7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb38c0cbcd0 0x7fb38c1ea7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 -- 172.21.15.7:0/1765477995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c155bf0 con 0x7fb374003a00 2023-06-09T11:11:02.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7fb38affd700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb37c010e30 tx=0x7fb37c0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f94c119a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f94c119a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41562/0 (socket says 172.21.15.7:41562) 2023-06-09T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f94c119a700 1 -- 172.21.15.7:0/651053161 learned_addr learned my addr 172.21.15.7:0/651053161 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f94c219c700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1570a0 0x7f94bc1f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.337+0000 7f08255ff700 1 Processor -- start 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.337+0000 7f08255ff700 1 -- start start 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f08255ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 0x7f08200c0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f08255ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08200c1660 0x7f08200c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f08255ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08200c11d0 con 0x7f082014fa60 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f08255ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200ad900 con 0x7f08200c1660 2023-06-09T11:11:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.339+0000 7f08255ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200c0bf0 con 0x7f08200c6df0 2023-06-09T11:11:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f08167fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 0x7f08200c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f08167fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 0x7f08200c0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41568/0 (socket says 172.21.15.7:41568) 2023-06-09T11:11:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f08167fc700 1 -- 172.21.15.7:0/920191008 learned_addr learned my addr 172.21.15.7:0/920191008 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.340+0000 7f081f7fe700 1 --2- 172.21.15.7:0/920191008 >> v2:172.21.15.7:3300/0 conn(0x7f08200c1660 0x7f08200c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.341+0000 7f94c119a700 1 -- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8003a00 msgr2=0x7f94bc156940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.341+0000 7f94c119a700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8003a00 0x7f94bc156940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.341+0000 7f94c119a700 1 -- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1570a0 msgr2=0x7f94bc1f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.341+0000 7f94c119a700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1570a0 0x7f94bc1f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.341+0000 7f94c119a700 1 -- 172.21.15.7:0/651053161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc0cba90 con 0x7f94bc158d70 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f94c119a700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c7a0 tx=0x7f94b800cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f081f7fe700 1 -- 172.21.15.7:0/920191008 >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 msgr2=0x7f08200c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f081f7fe700 1 --2- 172.21.15.7:0/920191008 >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 0x7f08200c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f081f7fe700 1 -- 172.21.15.7:0/920191008 >> v1:172.21.15.7:6789/0 conn(0x7f082014fa60 legacy=0x7f0820158d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f081f7fe700 1 -- 172.21.15.7:0/920191008 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200c72c0 con 0x7f08200c1660 2023-06-09T11:11:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c00ded0 con 0x7fb374003a00 2023-06-09T11:11:02.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c004850 con 0x7fb374003a00 2023-06-09T11:11:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0266b0 con 0x7fb374003a00 2023-06-09T11:11:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011800 con 0x7f94bc158d70 2023-06-09T11:11:02.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8011960 con 0x7f94bc158d70 2023-06-09T11:11:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c155910 con 0x7fb374003a00 2023-06-09T11:11:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.343+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c155ed0 con 0x7fb374003a00 2023-06-09T11:11:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.344+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c074cf0 con 0x7fb374003a00 2023-06-09T11:11:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.342+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc0cb7b0 con 0x7f94bc158d70 2023-06-09T11:11:02.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.343+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc0cbd70 con 0x7f94bc158d70 2023-06-09T11:11:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.344+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011cd0 con 0x7f94bc158d70 2023-06-09T11:11:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.344+0000 7f3e899ff700 1 Processor -- start 2023-06-09T11:11:02.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.344+0000 7f3e899ff700 1 -- start start 2023-06-09T11:11:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.344+0000 7f081f7fe700 1 --2- 172.21.15.7:0/920191008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200c5b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f082006f6a0 tx=0x7f0818009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe6e7dfa1687231 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.345+0000 7f081e7fc700 1 -- 172.21.15.7:0/920191008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010800 con 0x7f08200c1660 2023-06-09T11:11:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.345+0000 7f081e7fc700 1 -- 172.21.15.7:0/920191008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818010960 con 0x7f08200c1660 2023-06-09T11:11:02.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.345+0000 7f081e7fc700 1 -- 172.21.15.7:0/920191008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010cd0 con 0x7f08200c1660 2023-06-09T11:11:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.345+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc074640 con 0x7f94bc158d70 2023-06-09T11:11:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.346+0000 7f08255ff700 1 -- 172.21.15.7:0/920191008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 msgr2=0x7f08200c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.346+0000 7f08255ff700 1 --2- 172.21.15.7:0/920191008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200c5b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f082006f6a0 tx=0x7f0818009440 comp rx=0 tx=0).stop 2023-06-09T11:11:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f08255ff700 1 -- 172.21.15.7:0/920191008 shutdown_connections 2023-06-09T11:11:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f08255ff700 1 --2- 172.21.15.7:0/920191008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200c5b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f08255ff700 1 --2- 172.21.15.7:0/920191008 >> v2:172.21.15.117:3300/0 conn(0x7f08200c6df0 0x7f08200c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f08255ff700 1 -- 172.21.15.7:0/920191008 >> 172.21.15.7:0/920191008 conn(0x7f082006a4d0 msgr2=0x7f082006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.348+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94b8018360 con 0x7f94bc158d70 2023-06-09T11:11:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.346+0000 7f3e899ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 0x7f3e84158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e899ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e840c7940 con 0x7f3e841591d0 2023-06-09T11:11:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e899ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e84159660 con 0x7f3e8414f830 2023-06-09T11:11:02.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e899ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e84155720 con 0x7f3e841563f0 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e82ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3e8414f830 0x7f3e840c0670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34776/0 (socket says 172.21.15.7:34776) 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e82ffd700 1 -- 172.21.15.7:0/501886539 learned_addr learned my addr 172.21.15.7:0/501886539 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.347+0000 7f3e837fe700 1 --1- 172.21.15.7:0/501886539 >> v1:172.21.15.7:6789/0 conn(0x7f3e841591d0 0x7f3e840cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50038/0 (socket says 172.21.15.7:50038) 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.348+0000 7f3e827fc700 1 -- 172.21.15.7:0/501886539 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757486497 0 0) 0x7f3e84159660 con 0x7f3e8414f830 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.348+0000 7f3e827fc700 1 -- 172.21.15.7:0/501886539 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e700035b0 con 0x7f3e8414f830 2023-06-09T11:11:02.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.348+0000 7f3e7a7fc700 1 --2- 172.21.15.7:0/501886539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 0x7f3e84158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb37c023070 con 0x7fb374003a00 2023-06-09T11:11:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.348+0000 7f08255ff700 1 -- 172.21.15.7:0/920191008 shutdown_connections 2023-06-09T11:11:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/501886539 >> v1:172.21.15.117:6789/0 conn(0x7f3e8414f830 legacy=0x7f3e840c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/501886539 >> v1:172.21.15.7:6789/0 conn(0x7f3e841591d0 legacy=0x7f3e840cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/501886539 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e84159940 con 0x7f3e841563f0 2023-06-09T11:11:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7f94b27fc700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 0x7f94a006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.349+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f94b80892c0 con 0x7f94bc158d70 2023-06-09T11:11:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.350+0000 7fb3737fe700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 0x7fb37806d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.350+0000 7f08255ff700 1 -- 172.21.15.7:0/920191008 wait complete. 2023-06-09T11:11:02.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7f3e7a7fc700 1 --2- 172.21.15.7:0/501886539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 0x7f3e84158c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009870 tx=0x7f3e74009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b84e83c575a7f7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.350+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb37c016070 con 0x7fb374003a00 2023-06-09T11:11:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7fb38a7fc700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 0x7fb37806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.352+0000 7fb38a7fc700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 0x7fb37806d210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0c80f0 tx=0x7fb38000b420 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7f94c199b700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 0x7f94a006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7f94c199b700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 0x7f94a006d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c340 tx=0x7f94ac00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7f3e827fc700 1 -- 172.21.15.7:0/501886539 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011830 con 0x7f3e841563f0 2023-06-09T11:11:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7f3e827fc700 1 -- 172.21.15.7:0/501886539 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e74011990 con 0x7f3e841563f0 2023-06-09T11:11:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.351+0000 7f3e827fc700 1 -- 172.21.15.7:0/501886539 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74011d20 con 0x7f3e841563f0 2023-06-09T11:11:02.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.352+0000 7f3e899ff700 1 -- 172.21.15.7:0/501886539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 msgr2=0x7f3e84158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.352+0000 7f3e899ff700 1 --2- 172.21.15.7:0/501886539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 0x7f3e84158c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009870 tx=0x7f3e74009b80 comp rx=0 tx=0).stop 2023-06-09T11:11:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7f3e899ff700 1 -- 172.21.15.7:0/501886539 shutdown_connections 2023-06-09T11:11:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7f3e899ff700 1 --2- 172.21.15.7:0/501886539 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841563f0 0x7f3e84158c90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7f3e899ff700 1 -- 172.21.15.7:0/501886539 >> 172.21.15.7:0/501886539 conn(0x7f3e8406a4d0 msgr2=0x7f3e8406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.352+0000 7f08255ff700 1 Processor -- start 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.352+0000 7f08255ff700 1 -- start start 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08200c6df0 0x7f08201f4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082014fa60 0x7f08201f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200a9680 con 0x7f08200c1660 2023-06-09T11:11:02.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200afe20 con 0x7f082014fa60 2023-06-09T11:11:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f08255ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08201f3fe0 con 0x7f08200c6df0 2023-06-09T11:11:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7fba83bff700 1 Processor -- start 2023-06-09T11:11:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.353+0000 7fba83bff700 1 -- start start 2023-06-09T11:11:02.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb37c0576b0 con 0x7fb374003a00 2023-06-09T11:11:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.354+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f94b8054350 con 0x7f94bc158d70 2023-06-09T11:11:02.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f3e899ff700 1 -- 172.21.15.7:0/501886539 shutdown_connections 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f081effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f081effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36834/0 (socket says 172.21.15.7:36834) 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f081effd700 1 -- 172.21.15.7:0/1223210454 learned_addr learned my addr 172.21.15.7:0/1223210454 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f08167fc700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08200c6df0 0x7f08201f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.355+0000 7f081f7fe700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082014fa60 0x7f08201f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.356+0000 7f081effd700 1 -- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08200c6df0 msgr2=0x7f08201f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.356+0000 7f081effd700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08200c6df0 0x7f08201f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.356+0000 7f081effd700 1 -- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082014fa60 msgr2=0x7f08201f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.356+0000 7f081effd700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082014fa60 0x7f08201f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.356+0000 7f081effd700 1 -- 172.21.15.7:0/1223210454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009d30 con 0x7f08200c1660 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.357+0000 7f081effd700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0810001ed0 tx=0x7f08100021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.357+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810003c00 con 0x7f08200c1660 2023-06-09T11:11:02.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.357+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0810003d60 con 0x7f08200c1660 2023-06-09T11:11:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.357+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081000ab90 con 0x7f08200c1660 2023-06-09T11:11:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba83bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba7c157c70 0x7fba7c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba83bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba7c0c7bb0 0x7fba7c0c6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba83bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7c0c85f0 con 0x7fba7c0c8030 2023-06-09T11:11:02.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba83bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7c0cb1f0 con 0x7fba7c157c70 2023-06-09T11:11:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba83bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7c0cb4d0 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.359+0000 7fba8219c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fba7c0c8030 0x7fba7c15db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50054/0 (socket says 172.21.15.7:50054) 2023-06-09T11:11:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.360+0000 7fba8219c700 1 -- 172.21.15.7:0/442889231 learned_addr learned my addr 172.21.15.7:0/442889231 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.360+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08200c0e60 con 0x7f08200c1660 2023-06-09T11:11:02.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.360+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08200c75a0 con 0x7f08200c1660 2023-06-09T11:11:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.360+0000 7fba8119a700 1 --2- 172.21.15.7:0/442889231 >> v2:172.21.15.117:3300/0 conn(0x7fba7c0c7bb0 0x7fba7c0c6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.358+0000 7f3e899ff700 1 -- 172.21.15.7:0/501886539 wait complete. 2023-06-09T11:11:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.360+0000 7f3e899ff700 1 Processor -- start 2023-06-09T11:11:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.362+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08200c7880 con 0x7f08200c1660 2023-06-09T11:11:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.361+0000 7fba8119a700 1 -- 172.21.15.7:0/442889231 >> v2:172.21.15.7:3300/0 conn(0x7fba7c157c70 msgr2=0x7fba7c0c7510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-09T11:11:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.361+0000 7fba8119a700 1 --2- 172.21.15.7:0/442889231 >> v2:172.21.15.7:3300/0 conn(0x7fba7c157c70 0x7fba7c0c7510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.361+0000 7fba8119a700 1 -- 172.21.15.7:0/442889231 >> v1:172.21.15.7:6789/0 conn(0x7fba7c0c8030 legacy=0x7fba7c15db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.361+0000 7fba8119a700 1 -- 172.21.15.7:0/442889231 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7c0cb7b0 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.365+0000 7fba8119a700 1 --2- 172.21.15.7:0/442889231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c0c7bb0 0x7fba7c0c6880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba7c06d930 tx=0x7fba78009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fadb926d5f6a2f9b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.365+0000 7f3e899ff700 1 -- start start 2023-06-09T11:11:02.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e8414f830 0x7f3e840c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841591d0 0x7f3e841fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840c2b40 con 0x7f3e841563f0 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e841fb8f0 con 0x7f3e8414f830 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7f3e899ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e840c2860 con 0x7f3e841591d0 2023-06-09T11:11:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7fba80999700 1 -- 172.21.15.7:0/442889231 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78004230 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7fba80999700 1 -- 172.21.15.7:0/442889231 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78004390 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.367+0000 7fba80999700 1 -- 172.21.15.7:0/442889231 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba780036a0 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36836/0 (socket says 172.21.15.7:36836) 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/1015358574 learned_addr learned my addr 172.21.15.7:0/1015358574 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841591d0 msgr2=0x7f3e841fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841591d0 0x7f3e841fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e8414f830 msgr2=0x7f3e840c5000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e8414f830 0x7f3e840c5000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.368+0000 7f3e7a7fc700 1 -- 172.21.15.7:0/1015358574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e740094a0 con 0x7f3e841563f0 2023-06-09T11:11:02.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.369+0000 7f3e7a7fc700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e740090b0 tx=0x7f3e74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.370+0000 7fba83bff700 1 -- 172.21.15.7:0/442889231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c0c7bb0 msgr2=0x7fba7c0c6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.370+0000 7fba83bff700 1 --2- 172.21.15.7:0/442889231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c0c7bb0 0x7fba7c0c6880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba7c06d930 tx=0x7fba78009b20 comp rx=0 tx=0).stop 2023-06-09T11:11:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.369+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f081000acf0 con 0x7f08200c1660 2023-06-09T11:11:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.371+0000 7f0817fff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 0x7f080006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.371+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08100898f0 con 0x7f08200c1660 2023-06-09T11:11:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.372+0000 7f08167fc700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 0x7f080006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.372+0000 7fba83bff700 1 -- 172.21.15.7:0/442889231 shutdown_connections 2023-06-09T11:11:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.372+0000 7fba83bff700 1 --2- 172.21.15.7:0/442889231 >> v2:172.21.15.7:3300/0 conn(0x7fba7c157c70 0x7fba7c0c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.372+0000 7fba83bff700 1 --2- 172.21.15.7:0/442889231 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c0c7bb0 0x7fba7c0c6880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.372+0000 7fba83bff700 1 -- 172.21.15.7:0/442889231 >> 172.21.15.7:0/442889231 conn(0x7fba7c06a4d0 msgr2=0x7fba7c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7f08167fc700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 0x7f080006d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08201ed960 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7fba83bff700 1 -- 172.21.15.7:0/442889231 shutdown_connections 2023-06-09T11:11:02.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0810054a10 con 0x7f08200c1660 2023-06-09T11:11:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e740036a0 con 0x7f3e841563f0 2023-06-09T11:11:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7401f070 con 0x7f3e841563f0 2023-06-09T11:11:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e74003a40 con 0x7f3e841563f0 2023-06-09T11:11:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.374+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e84155bf0 con 0x7f3e841563f0 2023-06-09T11:11:02.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.374+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e84155910 con 0x7f3e841563f0 2023-06-09T11:11:02.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.373+0000 7fba83bff700 1 -- 172.21.15.7:0/442889231 wait complete. 2023-06-09T11:11:02.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.377+0000 7fba83bff700 1 Processor -- start 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.377+0000 7fba83bff700 1 -- start start 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba7c0c7bb0 0x7fba7c159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 0x7fba7c1fbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1ff930 con 0x7fba7c0c8030 2023-06-09T11:11:02.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c159580 con 0x7fba7c157c70 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.379+0000 7fba83bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1ff040 con 0x7fba7c0c7bb0 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.380+0000 7fba8219c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 0x7fba7c1fbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.380+0000 7fba8199b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba7c0c7bb0 0x7fba7c159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.380+0000 7fba8119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.380+0000 7fba8119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36846/0 (socket says 172.21.15.7:36846) 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.380+0000 7fba8119a700 1 -- 172.21.15.7:0/1139051695 learned_addr learned my addr 172.21.15.7:0/1139051695 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8119a700 1 -- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba7c0c7bb0 msgr2=0x7fba7c159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8119a700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba7c0c7bb0 0x7fba7c159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8119a700 1 -- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 msgr2=0x7fba7c1fbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8119a700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 0x7fba7c1fbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8119a700 1 -- 172.21.15.7:0/1139051695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780094a0 con 0x7fba7c0c8030 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.381+0000 7fba8219c700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 0x7fba7c1fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.382+0000 7fba8119a700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba780090b0 tx=0x7fba78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.383+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78003dc0 con 0x7fba7c0c8030 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.383+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7801f070 con 0x7fba7c0c8030 2023-06-09T11:11:02.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.383+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78011870 con 0x7fba7c0c8030 2023-06-09T11:11:02.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.385+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c0cbd70 con 0x7fba7c0c8030 2023-06-09T11:11:02.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.385+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c0cba90 con 0x7fba7c0c8030 2023-06-09T11:11:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.385+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e74003ba0 con 0x7f3e841563f0 2023-06-09T11:11:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.387+0000 7f3e80ff9700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 0x7f3e6406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.387+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e74021070 con 0x7f3e841563f0 2023-06-09T11:11:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.389+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e84155ed0 con 0x7f3e841563f0 2023-06-09T11:11:02.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.390+0000 7f3e82ffd700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 0x7f3e6406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.389+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c158040 con 0x7fba7c0c8030 2023-06-09T11:11:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.389+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba78021030 con 0x7fba7c0c8030 2023-06-09T11:11:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.390+0000 7fba727fc700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 0x7fba6006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.391+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fba7808a960 con 0x7fba7c0c8030 2023-06-09T11:11:02.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.391+0000 7fba8199b700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 0x7fba6006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.392+0000 7f3e82ffd700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 0x7f3e6406d290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a850 tx=0x7f3e6c005d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.393+0000 7fba8199b700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 0x7fba6006d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.400+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fba78055a80 con 0x7fba7c0c8030 2023-06-09T11:11:02.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.408+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3e74055d00 con 0x7f3e841563f0 2023-06-09T11:11:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.408+0000 7f98b2dff700 1 Processor -- start 2023-06-09T11:11:02.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.408+0000 7f98b2dff700 1 -- start start 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.411+0000 7f98b2dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98ac06afe0 0x7f98ac0c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.411+0000 7f98b2dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b2dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98ac1591d0 0x7f98ac14f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b2dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac155720 con 0x7f98ac1591d0 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b2dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac159c20 con 0x7f98ac06afe0 2023-06-09T11:11:02.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b2dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0c0bb0 con 0x7f98ac1563f0 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b0b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98ac06afe0 0x7f98ac0c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b0b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f98ac06afe0 0x7f98ac0c0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41618/0 (socket says 172.21.15.7:41618) 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.412+0000 7f98b0b9b700 1 -- 172.21.15.7:0/55667530 learned_addr learned my addr 172.21.15.7:0/55667530 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98abfff700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b139c700 1 --2- 172.21.15.7:0/55667530 >> v2:172.21.15.7:3300/0 conn(0x7f98ac1591d0 0x7f98ac14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 -- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 msgr2=0x7f98ac158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 -- 172.21.15.7:0/55667530 >> v2:172.21.15.7:3300/0 conn(0x7f98ac1591d0 msgr2=0x7f98ac14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 --2- 172.21.15.7:0/55667530 >> v2:172.21.15.7:3300/0 conn(0x7f98ac1591d0 0x7f98ac14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 -- 172.21.15.7:0/55667530 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac159660 con 0x7f98ac06afe0 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.413+0000 7f98b0b9b700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0c0670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f989c0098f0 tx=0x7f989c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4fe7eb4cc6aaa4a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.414+0000 7f98ab7fe700 1 -- 172.21.15.7:0/55667530 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00b860 con 0x7f98ac06afe0 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.414+0000 7f98ab7fe700 1 -- 172.21.15.7:0/55667530 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989c00b9c0 con 0x7f98ac06afe0 2023-06-09T11:11:02.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.414+0000 7f98ab7fe700 1 -- 172.21.15.7:0/55667530 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f989c00bd30 con 0x7f98ac06afe0 2023-06-09T11:11:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.415+0000 7f98b2dff700 1 -- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 msgr2=0x7f98ac0c0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.415+0000 7f98b2dff700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0c0670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f989c0098f0 tx=0x7f989c009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 -- 172.21.15.7:0/55667530 shutdown_connections 2023-06-09T11:11:02.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0c0670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 --2- 172.21.15.7:0/55667530 >> v2:172.21.15.7:3300/0 conn(0x7f98ac1591d0 0x7f98ac14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 --2- 172.21.15.7:0/55667530 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 -- 172.21.15.7:0/55667530 >> 172.21.15.7:0/55667530 conn(0x7f98ac06a4d0 msgr2=0x7f98ac06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 -- 172.21.15.7:0/55667530 shutdown_connections 2023-06-09T11:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.416+0000 7f98b2dff700 1 -- 172.21.15.7:0/55667530 wait complete. 2023-06-09T11:11:02.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.419+0000 7f98b2dff700 1 Processor -- start 2023-06-09T11:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.419+0000 7f98b2dff700 1 -- start start 2023-06-09T11:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0b0b80 con 0x7f98ac1591d0 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac0ca8a0 con 0x7f98ac06afe0 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.421+0000 7f98b2dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1ffd00 con 0x7f98ac1563f0 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.422+0000 7f98b139c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.422+0000 7f98b139c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36874/0 (socket says 172.21.15.7:36874) 2023-06-09T11:11:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.422+0000 7f98b139c700 1 -- 172.21.15.7:0/2498436318 learned_addr learned my addr 172.21.15.7:0/2498436318 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 -- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 msgr2=0x7f98ac0cad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T11:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac0cad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 -- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 msgr2=0x7f98ac0b0640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-09T11:11:02.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0b0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 -- 172.21.15.7:0/2498436318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f989c009550 con 0x7f98ac1591d0 2023-06-09T11:11:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.423+0000 7f98b139c700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98a4007de0 tx=0x7f98a400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.424+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a400ede0 con 0x7f98ac1591d0 2023-06-09T11:11:02.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.424+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a4014420 con 0x7f98ac1591d0 2023-06-09T11:11:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.424+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a40147d0 con 0x7f98ac1591d0 2023-06-09T11:11:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.425+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac155910 con 0x7f98ac1591d0 2023-06-09T11:11:02.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.425+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac159940 con 0x7f98ac1591d0 2023-06-09T11:11:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.430+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98a4014930 con 0x7f98ac1591d0 2023-06-09T11:11:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.431+0000 7f98a97fa700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 0x7f989806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.431+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f98a4089c50 con 0x7f98ac1591d0 2023-06-09T11:11:02.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.434+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac155bf0 con 0x7f98ac1591d0 2023-06-09T11:11:02.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.434+0000 7f98b0b9b700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 0x7f989806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.436+0000 7f98b0b9b700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 0x7f989806d290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.440+0000 7f4c48dff700 1 Processor -- start 2023-06-09T11:11:02.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.440+0000 7f4c48dff700 1 -- start start 2023-06-09T11:11:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.443+0000 7f4c48dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c4406afe0 0x7f4c440c0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.443+0000 7f4c48dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 0x7f4c44158c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.443+0000 7f4c48dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440c0bb0 con 0x7f4c441563f0 2023-06-09T11:11:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.443+0000 7f4c48dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c44155720 con 0x7f4c4406afe0 2023-06-09T11:11:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.444+0000 7f4c48dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4414ff40 con 0x7f4c440c6cc0 2023-06-09T11:11:02.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.444+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f98a4089e60 con 0x7f98ac1591d0 2023-06-09T11:11:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.445+0000 7f4c41d9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4c441563f0 0x7f4c4414f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34780/0 (socket says 172.21.15.7:34780) 2023-06-09T11:11:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.445+0000 7f4c41d9b700 1 -- 172.21.15.7:0/755679001 learned_addr learned my addr 172.21.15.7:0/755679001 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.445+0000 7f4c4259c700 1 --2- 172.21.15.7:0/755679001 >> v2:172.21.15.7:3300/0 conn(0x7f4c4406afe0 0x7f4c440c0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.445+0000 7f4c42d9d700 1 --2- 172.21.15.7:0/755679001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 0x7f4c44158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.446+0000 7f4c42d9d700 1 -- 172.21.15.7:0/755679001 >> v2:172.21.15.7:3300/0 conn(0x7f4c4406afe0 msgr2=0x7f4c440c0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.446+0000 7f4c42d9d700 1 --2- 172.21.15.7:0/755679001 >> v2:172.21.15.7:3300/0 conn(0x7f4c4406afe0 0x7f4c440c0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.446+0000 7f4c42d9d700 1 -- 172.21.15.7:0/755679001 >> v1:172.21.15.117:6789/0 conn(0x7f4c441563f0 legacy=0x7f4c4414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.446+0000 7f4c42d9d700 1 -- 172.21.15.7:0/755679001 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440c7940 con 0x7f4c440c6cc0 2023-06-09T11:11:02.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.446+0000 7f4c42d9d700 1 --2- 172.21.15.7:0/755679001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 0x7f4c44158c90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d90 tx=0x7f4c3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e64c08447a392faa server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.447+0000 7f4c4159a700 1 -- 172.21.15.7:0/755679001 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00e070 con 0x7f4c440c6cc0 2023-06-09T11:11:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.447+0000 7f4c4159a700 1 -- 172.21.15.7:0/755679001 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c003a20 con 0x7f4c440c6cc0 2023-06-09T11:11:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.447+0000 7f4c4159a700 1 -- 172.21.15.7:0/755679001 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c003dd0 con 0x7f4c440c6cc0 2023-06-09T11:11:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.448+0000 7f4c48dff700 1 -- 172.21.15.7:0/755679001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 msgr2=0x7f4c44158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.448+0000 7f4c48dff700 1 --2- 172.21.15.7:0/755679001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 0x7f4c44158c90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d90 tx=0x7f4c3c00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.449+0000 7f4c48dff700 1 -- 172.21.15.7:0/755679001 shutdown_connections 2023-06-09T11:11:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.449+0000 7f4c48dff700 1 --2- 172.21.15.7:0/755679001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c440c6cc0 0x7f4c44158c90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.449+0000 7f4c48dff700 1 --2- 172.21.15.7:0/755679001 >> v2:172.21.15.7:3300/0 conn(0x7f4c4406afe0 0x7f4c440c0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.449+0000 7f4c48dff700 1 -- 172.21.15.7:0/755679001 >> 172.21.15.7:0/755679001 conn(0x7f4c4406a4d0 msgr2=0x7f4c4406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.450+0000 7f4c48dff700 1 -- 172.21.15.7:0/755679001 shutdown_connections 2023-06-09T11:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.450+0000 7f4c48dff700 1 -- 172.21.15.7:0/755679001 wait complete. 2023-06-09T11:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.453+0000 7f4c48dff700 1 Processor -- start 2023-06-09T11:11:02.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.453+0000 7f4c48dff700 1 -- start start 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4406afe0 0x7f4c440cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441563f0 0x7f4c44153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c441585e0 con 0x7f4c440c6cc0 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c9690 con 0x7f4c4406afe0 2023-06-09T11:11:02.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.455+0000 7f4c48dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c440c9c30 con 0x7f4c441563f0 2023-06-09T11:11:02.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36884/0 (socket says 172.21.15.7:36884) 2023-06-09T11:11:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 -- 172.21.15.7:0/2633197081 learned_addr learned my addr 172.21.15.7:0/2633197081 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:02.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 -- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441563f0 msgr2=0x7f4c44153730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441563f0 0x7f4c44153730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 -- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4406afe0 msgr2=0x7f4c440cbcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:11:02.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4406afe0 0x7f4c440cbcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.457+0000 7f4c41d9b700 1 -- 172.21.15.7:0/2633197081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c009a70 con 0x7f4c440c6cc0 2023-06-09T11:11:02.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.458+0000 7f4c41d9b700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001ce0 tx=0x7f4c3800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.459+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38015e90 con 0x7f4c440c6cc0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.459+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38019070 con 0x7f4c440c6cc0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.459+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38004a90 con 0x7f4c440c6cc0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.459+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c44159940 con 0x7f4c440c6cc0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.459+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c44159660 con 0x7f4c440c6cc0 2023-06-09T11:11:02.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.466+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c38004bf0 con 0x7f4c440c6cc0 2023-06-09T11:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.468+0000 7f4c32ffd700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 0x7f4c2c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.468+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4c38088d60 con 0x7f4c440c6cc0 2023-06-09T11:11:02.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.470+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c280052e0 con 0x7f4c440c6cc0 2023-06-09T11:11:02.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.478+0000 7f4c4259c700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 0x7f4c2c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:02.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.479+0000 7f4c4259c700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 0x7f4c2c06d3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c340099f0 tx=0x7f4c34009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:02.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.482+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c38053df0 con 0x7f4c440c6cc0 2023-06-09T11:11:02.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.726+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f44bc005590 con 0x7f44d40c5d20 2023-06-09T11:11:02.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.727+0000 7f44ca7fc700 1 -- 172.21.15.7:0/3410089202 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44cc0581f0 con 0x7f44d40c5d20 2023-06-09T11:11:02.735 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:02.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.731+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 msgr2=0x7f44b806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.731+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 0x7f44b806d260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f44d4166340 tx=0x7f44c4009040 comp rx=0 tx=0).stop 2023-06-09T11:11:02.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.731+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 msgr2=0x7f44d421e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.731+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00be00 tx=0x7f44cc00f410 comp rx=0 tx=0).stop 2023-06-09T11:11:02.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 shutdown_connections 2023-06-09T11:11:02.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d4166b00 0x7f44d40ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40c5d20 0x7f44d421e180 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b806adb0 0x7f44b806d260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 --2- 172.21.15.7:0/3410089202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40c60f0 0x7f44d40c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.732+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 >> 172.21.15.7:0/3410089202 conn(0x7f44d406a4d0 msgr2=0x7f44d4164680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.735+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 shutdown_connections 2023-06-09T11:11:02.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.735+0000 7f44dc1ff700 1 -- 172.21.15.7:0/3410089202 wait complete. 2023-06-09T11:11:02.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-06-09T11:11:02.776 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.789+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb38c0c6aa0 con 0x7fb374003a00 2023-06-09T11:11:02.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.789+0000 7fb3737fe700 1 -- 172.21.15.7:0/1765477995 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb37c05acd0 con 0x7fb374003a00 2023-06-09T11:11:02.795 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:02.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.793+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 msgr2=0x7fb37806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.793+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 0x7fb37806d210 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0c80f0 tx=0x7fb38000b420 comp rx=0 tx=0).stop 2023-06-09T11:11:02.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.793+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 msgr2=0x7fb38c1dfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.793+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb37c010e30 tx=0x7fb37c0010f0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 shutdown_connections 2023-06-09T11:11:02.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37806ad60 0x7fb37806d210 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb38c0c93d0 0x7fb38c1ead00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb374003a00 0x7fb38c1dfc90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 --2- 172.21.15.7:0/1765477995 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb38c0cbcd0 0x7fb38c1ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 >> 172.21.15.7:0/1765477995 conn(0x7fb38c06a4d0 msgr2=0x7fb38c154fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 shutdown_connections 2023-06-09T11:11:02.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.794+0000 7fb3915ff700 1 -- 172.21.15.7:0/1765477995 wait complete. 2023-06-09T11:11:02.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.810+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f94bc1f3720 con 0x7f94bc158d70 2023-06-09T11:11:02.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.811+0000 7f94b27fc700 1 -- 172.21.15.7:0/651053161 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f94b8057970 con 0x7f94bc158d70 2023-06-09T11:11:02.819 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:02.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 msgr2=0x7f94a006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 0x7f94a006d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c340 tx=0x7f94ac00a000 comp rx=0 tx=0).stop 2023-06-09T11:11:02.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 msgr2=0x7f94bc1ea5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c7a0 tx=0x7f94b800cab0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 shutdown_connections 2023-06-09T11:11:02.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94bc1570a0 0x7f94bc1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94a006adb0 0x7f94a006d260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94a8003a00 0x7f94bc156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 --2- 172.21.15.7:0/651053161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94bc158d70 0x7f94bc1ea5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.816+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 >> 172.21.15.7:0/651053161 conn(0x7f94bc06a4d0 msgr2=0x7f94bc1535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.817+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 shutdown_connections 2023-06-09T11:11:02.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.817+0000 7f94c3bff700 1 -- 172.21.15.7:0/651053161 wait complete. 2023-06-09T11:11:02.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-06-09T11:11:02.830 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.830+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f98ac0caa50 con 0x7f98ac1591d0 2023-06-09T11:11:02.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.831+0000 7f98a97fa700 1 -- 172.21.15.7:0/2498436318 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98ac0caa50 con 0x7f98ac1591d0 2023-06-09T11:11:02.836 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:02.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.833+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 msgr2=0x7f989806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.833+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 0x7f989806d290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c00 tx=0x7f989c008030 comp rx=0 tx=0).stop 2023-06-09T11:11:02.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.833+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 msgr2=0x7f98ac0cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.833+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f98a4007de0 tx=0x7f98a400e3c0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 shutdown_connections 2023-06-09T11:11:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98ac1591d0 0x7f98ac0cb270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98ac06afe0 0x7f98ac0b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f989806ade0 0x7f989806d290 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 --2- 172.21.15.7:0/2498436318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98ac1563f0 0x7f98ac0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 >> 172.21.15.7:0/2498436318 conn(0x7f98ac06a4d0 msgr2=0x7f98ac0c0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 shutdown_connections 2023-06-09T11:11:02.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.834+0000 7f98b2dff700 1 -- 172.21.15.7:0/2498436318 wait complete. 2023-06-09T11:11:02.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.845+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0820074cf0 con 0x7f08200c1660 2023-06-09T11:11:02.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.846+0000 7f0817fff700 1 -- 172.21.15.7:0/1223210454 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0810058030 con 0x7f08200c1660 2023-06-09T11:11:02.851 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:02.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.849+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 msgr2=0x7f080006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.849+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 0x7f080006d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08201ed960 tx=0x7f0808008040 comp rx=0 tx=0).stop 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.849+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 msgr2=0x7f08200af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.849+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0810001ed0 tx=0x7f08100021e0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 shutdown_connections 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f082014fa60 0x7f08201f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08200c1660 0x7f08200af670 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0810001ed0 tx=0x7f08100021e0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006adb0 0x7f080006d260 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f08201ed960 tx=0x7f0808008040 comp rx=0 tx=0).stop 2023-06-09T11:11:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 --2- 172.21.15.7:0/1223210454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08200c6df0 0x7f08201f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 >> 172.21.15.7:0/1223210454 conn(0x7f082006a4d0 msgr2=0x7f08200c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-06-09T11:11:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 shutdown_connections 2023-06-09T11:11:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.850+0000 7f08255ff700 1 -- 172.21.15.7:0/1223210454 wait complete. 2023-06-09T11:11:02.857 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-06-09T11:11:02.861 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.857+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fba7c158040 con 0x7fba7c0c8030 2023-06-09T11:11:02.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.858+0000 7fba727fc700 1 -- 172.21.15.7:0/1139051695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba780590a0 con 0x7fba7c0c8030 2023-06-09T11:11:02.863 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:02.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 msgr2=0x7fba6006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 0x7fba6006d260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fba6c006fd0 tx=0x7fba6c009190 comp rx=0 tx=0).stop 2023-06-09T11:11:02.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 msgr2=0x7fba7c0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fba780090b0 tx=0x7fba78004750 comp rx=0 tx=0).stop 2023-06-09T11:11:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3e84074cf0 con 0x7f3e841563f0 2023-06-09T11:11:02.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 shutdown_connections 2023-06-09T11:11:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba7c157c70 0x7fba7c1fbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba6006adb0 0x7fba6006d260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba7c0c7bb0 0x7fba7c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 --2- 172.21.15.7:0/1139051695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba7c0c8030 0x7fba7c0712e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 >> 172.21.15.7:0/1139051695 conn(0x7fba7c06a4d0 msgr2=0x7fba7c0c0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 shutdown_connections 2023-06-09T11:11:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.860+0000 7fba83bff700 1 -- 172.21.15.7:0/1139051695 wait complete. 2023-06-09T11:11:02.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.861+0000 7f3e80ff9700 1 -- 172.21.15.7:0/1015358574 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e74015070 con 0x7f3e841563f0 2023-06-09T11:11:02.869 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2023-06-09T11:11:02.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.863+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 msgr2=0x7f3e6406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.863+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 0x7f3e6406d290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00a850 tx=0x7f3e6c005d10 comp rx=0 tx=0).stop 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.863+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 msgr2=0x7f3e841ef840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.863+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e740090b0 tx=0x7f3e74004750 comp rx=0 tx=0).stop 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 shutdown_connections 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e6406ade0 0x7f3e6406d290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3e841591d0 0x7f3e841fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3e8414f830 0x7f3e840c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 --2- 172.21.15.7:0/1015358574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3e841563f0 0x7f3e841ef840 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 >> 172.21.15.7:0/1015358574 conn(0x7f3e8406a4d0 msgr2=0x7f3e84153700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 shutdown_connections 2023-06-09T11:11:02.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.864+0000 7f3e899ff700 1 -- 172.21.15.7:0/1015358574 wait complete. 2023-06-09T11:11:02.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2023-06-09T11:11:02.881 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.881+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4c28005590 con 0x7f4c440c6cc0 2023-06-09T11:11:02.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.882+0000 7f4c32ffd700 1 -- 172.21.15.7:0/2633197081 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c38057410 con 0x7f4c440c6cc0 2023-06-09T11:11:02.887 INFO:teuthology.orchestra.run.smithi007.stdout:55834574854 2023-06-09T11:11:02.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-06-09T11:11:02.889 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-06-09T11:11:02.892 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 msgr2=0x7f4c2c06d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 0x7f4c2c06d3f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4c340099f0 tx=0x7f4c34009210 comp rx=0 tx=0).stop 2023-06-09T11:11:02.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 msgr2=0x7f4c441580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c38001ce0 tx=0x7f4c3800ca30 comp rx=0 tx=0).stop 2023-06-09T11:11:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 shutdown_connections 2023-06-09T11:11:02.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06af40 0x7f4c2c06d3f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c441563f0 0x7f4c44153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c4406afe0 0x7f4c440cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 --2- 172.21.15.7:0/2633197081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c440c6cc0 0x7f4c441580a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:02.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 >> 172.21.15.7:0/2633197081 conn(0x7f4c4406a4d0 msgr2=0x7f4c440c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:02.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 shutdown_connections 2023-06-09T11:11:02.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:02.885+0000 7f4c48dff700 1 -- 172.21.15.7:0/2633197081 wait complete. 2023-06-09T11:11:02.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-06-09T11:11:02.908 DEBUG:teuthology.parallel:result is None 2023-06-09T11:11:02.909 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-09T11:11:02.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:11:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.143+0000 7f85a2dff700 1 Processor -- start 2023-06-09T11:11:03.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.144+0000 7f85a2dff700 1 -- start start 2023-06-09T11:11:03.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a2dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859c0cb5d0 0x7f859c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a2dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a2dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f859c159b20 con 0x7f859c0cb9a0 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a2dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859c0cac80 con 0x7f859c0cb5d0 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a2dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c1597d0 con 0x7f859c15cd40 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859c0cb5d0 0x7f859c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f859c0cb5d0 0x7f859c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48964/0 (socket says 172.21.15.7:48964) 2023-06-09T11:11:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 -- 172.21.15.7:0/2338132406 learned_addr learned my addr 172.21.15.7:0/2338132406 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:03.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a139c700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 -- 172.21.15.7:0/2338132406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 msgr2=0x7f859c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:03.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 -- 172.21.15.7:0/2338132406 >> v1:172.21.15.117:6789/0 conn(0x7f859c0cb9a0 legacy=0x7f859c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.145+0000 7f85a0b9b700 1 -- 172.21.15.7:0/2338132406 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c071ab0 con 0x7f859c0cb5d0 2023-06-09T11:11:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f85a0b9b700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c0b52b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85900098f0 tx=0x7f8590009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f080bea066c704ba server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f859b7fe700 1 -- 172.21.15.7:0/2338132406 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590011830 con 0x7f859c0cb5d0 2023-06-09T11:11:03.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f859b7fe700 1 -- 172.21.15.7:0/2338132406 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8590011990 con 0x7f859c0cb5d0 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f859b7fe700 1 -- 172.21.15.7:0/2338132406 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590011d20 con 0x7f859c0cb5d0 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f85a2dff700 1 -- 172.21.15.7:0/2338132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 msgr2=0x7f859c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.146+0000 7f85a2dff700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c0b52b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f85900098f0 tx=0x7f8590009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 -- 172.21.15.7:0/2338132406 shutdown_connections 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 --2- 172.21.15.7:0/2338132406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c0b52b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 -- 172.21.15.7:0/2338132406 >> 172.21.15.7:0/2338132406 conn(0x7f859c070af0 msgr2=0x7f859c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:03.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 -- 172.21.15.7:0/2338132406 shutdown_connections 2023-06-09T11:11:03.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.147+0000 7f85a2dff700 1 -- 172.21.15.7:0/2338132406 wait complete. 2023-06-09T11:11:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.148+0000 7f85a2dff700 1 Processor -- start 2023-06-09T11:11:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.149+0000 7f85a2dff700 1 -- start start 2023-06-09T11:11:03.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.149+0000 7f85a2dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a2dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 0x7f859c0cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a2dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a2dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c0cd600 con 0x7f859c0cb5d0 2023-06-09T11:11:03.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a2dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859c15afc0 con 0x7f859c0cb9a0 2023-06-09T11:11:03.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a2dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f859c152870 con 0x7f859c15cd40 2023-06-09T11:11:03.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48968/0 (socket says 172.21.15.7:48968) 2023-06-09T11:11:03.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 -- 172.21.15.7:0/841392117 learned_addr learned my addr 172.21.15.7:0/841392117 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:03.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f859bfff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 0x7f859c0cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a139c700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 -- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 msgr2=0x7f859c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c1537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 -- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 msgr2=0x7f859c0cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 0x7f859c0cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 -- 172.21.15.7:0/841392117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009550 con 0x7f859c0cb5d0 2023-06-09T11:11:03.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a139c700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:03.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f859bfff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 0x7f859c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.150+0000 7f85a0b9b700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8590000c00 tx=0x7f8590004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.151+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590004570 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.151+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859001f070 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.151+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859c15d4f0 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.151+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85900038c0 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.151+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859c15d210 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.152+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859c15d7d0 con 0x7f859c0cb5d0 2023-06-09T11:11:03.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.153+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8590021030 con 0x7f859c0cb5d0 2023-06-09T11:11:03.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.154+0000 7f85997fa700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 0x7f858006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.154+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f859008aa70 con 0x7f859c0cb5d0 2023-06-09T11:11:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.154+0000 7f859bfff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 0x7f858006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.155+0000 7f859bfff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 0x7f858006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f859c158aa0 tx=0x7f8588009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.158+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8590055a80 con 0x7f859c0cb5d0 2023-06-09T11:11:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.362+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f859c074cf0 con 0x7f858006ace0 2023-06-09T11:11:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.371+0000 7f85997fa700 1 -- 172.21.15.7:0/841392117 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286610 (secure 0 0 0) 0x7f859c074cf0 con 0x7f858006ace0 2023-06-09T11:11:03.379 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:03.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 msgr2=0x7f858006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 0x7f858006d190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f859c158aa0 tx=0x7f8588009040 comp rx=0 tx=0).stop 2023-06-09T11:11:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 msgr2=0x7f859c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8590000c00 tx=0x7f8590004750 comp rx=0 tx=0).stop 2023-06-09T11:11:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 shutdown_connections 2023-06-09T11:11:03.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f858006ace0 0x7f858006d190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f859c15cd40 0x7f859c1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f859c0cb5d0 0x7f859c15a930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 --2- 172.21.15.7:0/841392117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f859c0cb9a0 0x7f859c0cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 >> 172.21.15.7:0/841392117 conn(0x7f859c070af0 msgr2=0x7f859c070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:03.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 shutdown_connections 2023-06-09T11:11:03.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.375+0000 7f85a2dff700 1 -- 172.21.15.7:0/841392117 wait complete. 2023-06-09T11:11:03.384 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:11:03.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-06-09T11:11:02.192453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67904,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":66048,"kb_avail":754906816,"statfs":{"total":773094113280,"available":773024579584,"internally_reserved":0,"allocated":1835008,"data_stored":1780820,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67633152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.008720"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.250974+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:10:52.250974+0000","last_peered":"2023-06-09T11:10:52.250974+0000","last_clean":"2023-06-09T11:10:52.250974+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:10:52.250974+0000","last_undegraded":"2023-06-09T11:10:52.250974+0000","last_fullsized":"2023-06-09T11:10:52.250974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185867+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:10:51.185867+0000","last_peered":"2023-06-09T11:10:51.185867+0000","last_clean":"2023-06-09T11:10:51.185867+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:10:51.185867+0000","last_undegraded":"2023-06-09T11:10:51.185867+0000","last_fullsized":"2023-06-09T11:10:51.185867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462070+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:51.462070+0000","last_peered":"2023-06-09T11:10:51.462070+0000","last_clean":"2023-06-09T11:10:51.462070+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:51.462070+0000","last_undegraded":"2023-06-09T11:10:51.462070+0000","last_fullsized":"2023-06-09T11:10:51.462070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548111+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:10:51.548111+0000","last_peered":"2023-06-09T11:10:51.548111+0000","last_clean":"2023-06-09T11:10:51.548111+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:10:51.548111+0000","last_undegraded":"2023-06-09T11:10:51.548111+0000","last_fullsized":"2023-06-09T11:10:51.548111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199112+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:55.199112+0000","last_peered":"2023-06-09T11:10:55.199112+0000","last_clean":"2023-06-09T11:10:55.199112+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:55.199112+0000","last_undegraded":"2023-06-09T11:10:55.199112+0000","last_fullsized":"2023-06-09T11:10:55.199112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200262+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:10:55.200262+0000","last_peered":"2023-06-09T11:10:55.200262+0000","last_clean":"2023-06-09T11:10:55.200262+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:10:55.200262+0000","last_undegraded":"2023-06-09T11:10:55.200262+0000","last_fullsized":"2023-06-09T11:10:55.200262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199602+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:10:55.199602+0000","last_peered":"2023-06-09T11:10:55.199602+0000","last_clean":"2023-06-09T11:10:55.199602+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:10:55.199602+0000","last_undegraded":"2023-06-09T11:10:55.199602+0000","last_fullsized":"2023-06-09T11:10:55.199602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547634+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:10:51.547634+0000","last_peered":"2023-06-09T11:10:51.547634+0000","last_clean":"2023-06-09T11:10:51.547634+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:10:51.547634+0000","last_undegraded":"2023-06-09T11:10:51.547634+0000","last_fullsized":"2023-06-09T11:10:51.547634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689705+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:10:51.689705+0000","last_peered":"2023-06-09T11:10:51.689705+0000","last_clean":"2023-06-09T11:10:51.689705+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:10:51.689705+0000","last_undegraded":"2023-06-09T11:10:51.689705+0000","last_fullsized":"2023-06-09T11:10:51.689705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199269+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:10:55.199269+0000","last_peered":"2023-06-09T11:10:55.199269+0000","last_clean":"2023-06-09T11:10:55.199269+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:10:55.199269+0000","last_undegraded":"2023-06-09T11:10:55.199269+0000","last_fullsized":"2023-06-09T11:10:55.199269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251581+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:52.251581+0000","last_peered":"2023-06-09T11:10:52.251581+0000","last_clean":"2023-06-09T11:10:52.251581+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:52.251581+0000","last_undegraded":"2023-06-09T11:10:52.251581+0000","last_fullsized":"2023-06-09T11:10:52.251581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199981+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:55.199981+0000","last_peered":"2023-06-09T11:10:55.199981+0000","last_clean":"2023-06-09T11:10:55.199981+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:55.199981+0000","last_undegraded":"2023-06-09T11:10:55.199981+0000","last_fullsized":"2023-06-09T11:10:55.199981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251545+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:52.251545+0000","last_peered":"2023-06-09T11:10:52.251545+0000","last_clean":"2023-06-09T11:10:52.251545+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:52.251545+0000","last_undegraded":"2023-06-09T11:10:52.251545+0000","last_fullsized":"2023-06-09T11:10:52.251545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189143+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:10:51.189143+0000","last_peered":"2023-06-09T11:10:51.189143+0000","last_clean":"2023-06-09T11:10:51.189143+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:10:51.189143+0000","last_undegraded":"2023-06-09T11:10:51.189143+0000","last_fullsized":"2023-06-09T11:10:51.189143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.193076+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:10:55.193076+0000","last_peered":"2023-06-09T11:10:55.193076+0000","last_clean":"2023-06-09T11:10:55.193076+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:10:55.193076+0000","last_undegraded":"2023-06-09T11:10:55.193076+0000","last_fullsized":"2023-06-09T11:10:55.193076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197985+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:51.197985+0000","last_peered":"2023-06-09T11:10:51.197985+0000","last_clean":"2023-06-09T11:10:51.197985+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:51.197985+0000","last_undegraded":"2023-06-09T11:10:51.197985+0000","last_fullsized":"2023-06-09T11:10:51.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189298+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:10:51.189298+0000","last_peered":"2023-06-09T11:10:51.189298+0000","last_clean":"2023-06-09T11:10:51.189298+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:10:51.189298+0000","last_undegraded":"2023-06-09T11:10:51.189298+0000","last_fullsized":"2023-06-09T11:10:51.189298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461738+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:10:51.461738+0000","last_peered":"2023-06-09T11:10:51.461738+0000","last_clean":"2023-06-09T11:10:51.461738+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:10:51.461738+0000","last_undegraded":"2023-06-09T11:10:51.461738+0000","last_fullsized":"2023-06-09T11:10:51.461738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189545+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:51.189545+0000","last_peered":"2023-06-09T11:10:51.189545+0000","last_clean":"2023-06-09T11:10:51.189545+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:51.189545+0000","last_undegraded":"2023-06-09T11:10:51.189545+0000","last_fullsized":"2023-06-09T11:10:51.189545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189440+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:10:51.189440+0000","last_peered":"2023-06-09T11:10:51.189440+0000","last_clean":"2023-06-09T11:10:51.189440+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:10:51.189440+0000","last_undegraded":"2023-06-09T11:10:51.189440+0000","last_fullsized":"2023-06-09T11:10:51.189440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185737+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:10:51.185737+0000","last_peered":"2023-06-09T11:10:51.185737+0000","last_clean":"2023-06-09T11:10:51.185737+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:10:51.185737+0000","last_undegraded":"2023-06-09T11:10:51.185737+0000","last_fullsized":"2023-06-09T11:10:51.185737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461810+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:51.461810+0000","last_peered":"2023-06-09T11:10:51.461810+0000","last_clean":"2023-06-09T11:10:51.461810+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:51.461810+0000","last_undegraded":"2023-06-09T11:10:51.461810+0000","last_fullsized":"2023-06-09T11:10:51.461810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547703+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:10:51.547703+0000","last_peered":"2023-06-09T11:10:51.547703+0000","last_clean":"2023-06-09T11:10:51.547703+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:10:51.547703+0000","last_undegraded":"2023-06-09T11:10:51.547703+0000","last_fullsized":"2023-06-09T11:10:51.547703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199313+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:10:55.199313+0000","last_peered":"2023-06-09T11:10:55.199313+0000","last_clean":"2023-06-09T11:10:55.199313+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:10:55.199313+0000","last_undegraded":"2023-06-09T11:10:55.199313+0000","last_fullsized":"2023-06-09T11:10:55.199313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548150+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:10:51.548150+0000","last_peered":"2023-06-09T11:10:51.548150+0000","last_clean":"2023-06-09T11:10:51.548150+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:10:51.548150+0000","last_undegraded":"2023-06-09T11:10:51.548150+0000","last_fullsized":"2023-06-09T11:10:51.548150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.254052+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:10:52.254052+0000","last_peered":"2023-06-09T11:10:52.254052+0000","last_clean":"2023-06-09T11:10:52.254052+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:10:52.254052+0000","last_undegraded":"2023-06-09T11:10:52.254052+0000","last_fullsized":"2023-06-09T11:10:52.254052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198929+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:55.198929+0000","last_peered":"2023-06-09T11:10:55.198929+0000","last_clean":"2023-06-09T11:10:55.198929+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:55.198929+0000","last_undegraded":"2023-06-09T11:10:55.198929+0000","last_fullsized":"2023-06-09T11:10:55.198929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198635+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:10:55.198635+0000","last_peered":"2023-06-09T11:10:55.198635+0000","last_clean":"2023-06-09T11:10:55.198635+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:10:55.198635+0000","last_undegraded":"2023-06-09T11:10:55.198635+0000","last_fullsized":"2023-06-09T11:10:55.198635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.260044+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:52.260044+0000","last_peered":"2023-06-09T11:10:52.260044+0000","last_clean":"2023-06-09T11:10:52.260044+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:52.260044+0000","last_undegraded":"2023-06-09T11:10:52.260044+0000","last_fullsized":"2023-06-09T11:10:52.260044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.201285+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:10:55.201285+0000","last_peered":"2023-06-09T11:10:55.201285+0000","last_clean":"2023-06-09T11:10:55.201285+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:10:55.201285+0000","last_undegraded":"2023-06-09T11:10:55.201285+0000","last_fullsized":"2023-06-09T11:10:55.201285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8940,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362900,"statfs":{"total":96636764160,"available":96627609600,"internally_reserved":0,"allocated":299008,"data_stored":666568,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":274432,"data_stored":665370,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8404,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363436,"statfs":{"total":96636764160,"available":96628158464,"internally_reserved":0,"allocated":208896,"data_stored":74510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8420,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363420,"statfs":{"total":96636764160,"available":96628142080,"internally_reserved":0,"allocated":225280,"data_stored":76040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":212992,"data_stored":75072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:11:03.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:11:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.633+0000 7f50319ff700 1 Processor -- start 2023-06-09T11:11:03.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.633+0000 7f50319ff700 1 -- start start 2023-06-09T11:11:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f50319ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f50319ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f502c15b4a0 0x7f502c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f50319ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c15a500 con 0x7f502c0c9f40 2023-06-09T11:11:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f50319ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c0c5f10 con 0x7f502c15b4a0 2023-06-09T11:11:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f50319ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c15da00 con 0x7f502c153880 2023-06-09T11:11:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.634+0000 7f502b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f502c15b4a0 0x7f502c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f502c15b4a0 0x7f502c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48972/0 (socket says 172.21.15.7:48972) 2023-06-09T11:11:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 -- 172.21.15.7:0/4064285609 learned_addr learned my addr 172.21.15.7:0/4064285609 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502affd700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 -- 172.21.15.7:0/4064285609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 msgr2=0x7f502c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 -- 172.21.15.7:0/4064285609 >> v1:172.21.15.117:6789/0 conn(0x7f502c0c9f40 legacy=0x7f502c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 -- 172.21.15.7:0/4064285609 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c1596e0 con 0x7f502c15b4a0 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f502b7fe700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0c4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5024009e50 tx=0x7f502400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9786043a60e98f61 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f5029ffb700 1 -- 172.21.15.7:0/4064285609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502400e070 con 0x7f502c15b4a0 2023-06-09T11:11:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f5029ffb700 1 -- 172.21.15.7:0/4064285609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502400b910 con 0x7f502c15b4a0 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.635+0000 7f5029ffb700 1 -- 172.21.15.7:0/4064285609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502400bcc0 con 0x7f502c15b4a0 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 -- 172.21.15.7:0/4064285609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 msgr2=0x7f502c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0c4e30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5024009e50 tx=0x7f502400b330 comp rx=0 tx=0).stop 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 -- 172.21.15.7:0/4064285609 shutdown_connections 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0c4e30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 --2- 172.21.15.7:0/4064285609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 -- 172.21.15.7:0/4064285609 >> 172.21.15.7:0/4064285609 conn(0x7f502c0737f0 msgr2=0x7f502c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 -- 172.21.15.7:0/4064285609 shutdown_connections 2023-06-09T11:11:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.636+0000 7f50319ff700 1 -- 172.21.15.7:0/4064285609 wait complete. 2023-06-09T11:11:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.638+0000 7f50319ff700 1 Processor -- start 2023-06-09T11:11:03.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.638+0000 7f50319ff700 1 -- start start 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c1f2ca0 con 0x7f502c15b4a0 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0b5370 con 0x7f502c0c9f40 2023-06-09T11:11:03.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f50319ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0b3030 con 0x7f502c153880 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f502affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f502b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f502affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:51248/0 (socket says 172.21.15.7:51248) 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f502affd700 1 -- 172.21.15.7:0/4276491823 learned_addr learned my addr 172.21.15.7:0/4276491823 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.639+0000 7f502a7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 -- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 msgr2=0x7f502c151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 -- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 msgr2=0x7f502c0b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 -- 172.21.15.7:0/4276491823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5024009b30 con 0x7f502c0c9f40 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502b7fe700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f502affd700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f501c00e9e0 tx=0x7f501c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c009710 con 0x7f502c0c9f40 2023-06-09T11:11:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00ee80 con 0x7f502c0c9f40 2023-06-09T11:11:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c159c60 con 0x7f502c0c9f40 2023-06-09T11:11:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.640+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00ca50 con 0x7f502c0c9f40 2023-06-09T11:11:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.641+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c159980 con 0x7f502c0c9f40 2023-06-09T11:11:03.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.642+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c159f40 con 0x7f502c0c9f40 2023-06-09T11:11:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.643+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f501c00cbb0 con 0x7f502c0c9f40 2023-06-09T11:11:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.643+0000 7f5013fff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 0x7f501806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.644+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f501c0893a0 con 0x7f502c0c9f40 2023-06-09T11:11:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.644+0000 7f502a7fc700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 0x7f501806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:03.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.644+0000 7f502a7fc700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 0x7f501806d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f502c0cf470 tx=0x7f5020009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:03.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.649+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f501c057ed0 con 0x7f502c0c9f40 2023-06-09T11:11:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.852+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f502c074cf0 con 0x7f501806ad10 2023-06-09T11:11:03.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.861+0000 7f5013fff700 1 -- 172.21.15.7:0/4276491823 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286610 (secure 0 0 0) 0x7f502c074cf0 con 0x7f501806ad10 2023-06-09T11:11:03.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.864+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 msgr2=0x7f501806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.864+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 0x7f501806d1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f502c0cf470 tx=0x7f5020009040 comp rx=0 tx=0).stop 2023-06-09T11:11:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.864+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 msgr2=0x7f502c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.864+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f501c00e9e0 tx=0x7f501c00ecf0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 shutdown_connections 2023-06-09T11:11:03.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f502c15b4a0 0x7f502c0b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f502c0c9f40 0x7f502c0b4e30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f501806ad10 0x7f501806d1c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 --2- 172.21.15.7:0/4276491823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f502c153880 0x7f502c151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 >> 172.21.15.7:0/4276491823 conn(0x7f502c0737f0 msgr2=0x7f502c1e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 shutdown_connections 2023-06-09T11:11:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:03.865+0000 7f50319ff700 1 -- 172.21.15.7:0/4276491823 wait complete. 2023-06-09T11:11:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:11:03.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-06-09T11:11:02.192453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67904,"kb_used_data":1792,"kb_used_omap":0,"kb_used_meta":66048,"kb_avail":754906816,"statfs":{"total":773094113280,"available":773024579584,"internally_reserved":0,"allocated":1835008,"data_stored":1780820,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67633152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.008720"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.250974+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:10:52.250974+0000","last_peered":"2023-06-09T11:10:52.250974+0000","last_clean":"2023-06-09T11:10:52.250974+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:10:52.250974+0000","last_undegraded":"2023-06-09T11:10:52.250974+0000","last_fullsized":"2023-06-09T11:10:52.250974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185867+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:10:51.185867+0000","last_peered":"2023-06-09T11:10:51.185867+0000","last_clean":"2023-06-09T11:10:51.185867+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:10:51.185867+0000","last_undegraded":"2023-06-09T11:10:51.185867+0000","last_fullsized":"2023-06-09T11:10:51.185867+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462070+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:51.462070+0000","last_peered":"2023-06-09T11:10:51.462070+0000","last_clean":"2023-06-09T11:10:51.462070+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:51.462070+0000","last_undegraded":"2023-06-09T11:10:51.462070+0000","last_fullsized":"2023-06-09T11:10:51.462070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548111+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:10:51.548111+0000","last_peered":"2023-06-09T11:10:51.548111+0000","last_clean":"2023-06-09T11:10:51.548111+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:10:51.548111+0000","last_undegraded":"2023-06-09T11:10:51.548111+0000","last_fullsized":"2023-06-09T11:10:51.548111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189507+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:10:51.189507+0000","last_peered":"2023-06-09T11:10:51.189507+0000","last_clean":"2023-06-09T11:10:51.189507+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:10:51.189507+0000","last_undegraded":"2023-06-09T11:10:51.189507+0000","last_fullsized":"2023-06-09T11:10:51.189507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199112+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:55.199112+0000","last_peered":"2023-06-09T11:10:55.199112+0000","last_clean":"2023-06-09T11:10:55.199112+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:55.199112+0000","last_undegraded":"2023-06-09T11:10:55.199112+0000","last_fullsized":"2023-06-09T11:10:55.199112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200262+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:10:55.200262+0000","last_peered":"2023-06-09T11:10:55.200262+0000","last_clean":"2023-06-09T11:10:55.200262+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:10:55.200262+0000","last_undegraded":"2023-06-09T11:10:55.200262+0000","last_fullsized":"2023-06-09T11:10:55.200262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199602+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:10:55.199602+0000","last_peered":"2023-06-09T11:10:55.199602+0000","last_clean":"2023-06-09T11:10:55.199602+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:10:55.199602+0000","last_undegraded":"2023-06-09T11:10:55.199602+0000","last_fullsized":"2023-06-09T11:10:55.199602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547634+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:10:51.547634+0000","last_peered":"2023-06-09T11:10:51.547634+0000","last_clean":"2023-06-09T11:10:51.547634+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:10:51.547634+0000","last_undegraded":"2023-06-09T11:10:51.547634+0000","last_fullsized":"2023-06-09T11:10:51.547634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689705+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:10:51.689705+0000","last_peered":"2023-06-09T11:10:51.689705+0000","last_clean":"2023-06-09T11:10:51.689705+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:10:51.689705+0000","last_undegraded":"2023-06-09T11:10:51.689705+0000","last_fullsized":"2023-06-09T11:10:51.689705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199269+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:10:55.199269+0000","last_peered":"2023-06-09T11:10:55.199269+0000","last_clean":"2023-06-09T11:10:55.199269+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:10:55.199269+0000","last_undegraded":"2023-06-09T11:10:55.199269+0000","last_fullsized":"2023-06-09T11:10:55.199269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251581+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:52.251581+0000","last_peered":"2023-06-09T11:10:52.251581+0000","last_clean":"2023-06-09T11:10:52.251581+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:52.251581+0000","last_undegraded":"2023-06-09T11:10:52.251581+0000","last_fullsized":"2023-06-09T11:10:52.251581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199981+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:55.199981+0000","last_peered":"2023-06-09T11:10:55.199981+0000","last_clean":"2023-06-09T11:10:55.199981+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:55.199981+0000","last_undegraded":"2023-06-09T11:10:55.199981+0000","last_fullsized":"2023-06-09T11:10:55.199981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251545+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:52.251545+0000","last_peered":"2023-06-09T11:10:52.251545+0000","last_clean":"2023-06-09T11:10:52.251545+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:52.251545+0000","last_undegraded":"2023-06-09T11:10:52.251545+0000","last_fullsized":"2023-06-09T11:10:52.251545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189143+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:10:51.189143+0000","last_peered":"2023-06-09T11:10:51.189143+0000","last_clean":"2023-06-09T11:10:51.189143+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:10:51.189143+0000","last_undegraded":"2023-06-09T11:10:51.189143+0000","last_fullsized":"2023-06-09T11:10:51.189143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.193076+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:10:55.193076+0000","last_peered":"2023-06-09T11:10:55.193076+0000","last_clean":"2023-06-09T11:10:55.193076+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:10:55.193076+0000","last_undegraded":"2023-06-09T11:10:55.193076+0000","last_fullsized":"2023-06-09T11:10:55.193076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197985+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:51.197985+0000","last_peered":"2023-06-09T11:10:51.197985+0000","last_clean":"2023-06-09T11:10:51.197985+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:51.197985+0000","last_undegraded":"2023-06-09T11:10:51.197985+0000","last_fullsized":"2023-06-09T11:10:51.197985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189298+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:10:51.189298+0000","last_peered":"2023-06-09T11:10:51.189298+0000","last_clean":"2023-06-09T11:10:51.189298+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:10:51.189298+0000","last_undegraded":"2023-06-09T11:10:51.189298+0000","last_fullsized":"2023-06-09T11:10:51.189298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461738+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:10:51.461738+0000","last_peered":"2023-06-09T11:10:51.461738+0000","last_clean":"2023-06-09T11:10:51.461738+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:10:51.461738+0000","last_undegraded":"2023-06-09T11:10:51.461738+0000","last_fullsized":"2023-06-09T11:10:51.461738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189545+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:51.189545+0000","last_peered":"2023-06-09T11:10:51.189545+0000","last_clean":"2023-06-09T11:10:51.189545+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:51.189545+0000","last_undegraded":"2023-06-09T11:10:51.189545+0000","last_fullsized":"2023-06-09T11:10:51.189545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189440+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:10:51.189440+0000","last_peered":"2023-06-09T11:10:51.189440+0000","last_clean":"2023-06-09T11:10:51.189440+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:10:51.189440+0000","last_undegraded":"2023-06-09T11:10:51.189440+0000","last_fullsized":"2023-06-09T11:10:51.189440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185737+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:10:51.185737+0000","last_peered":"2023-06-09T11:10:51.185737+0000","last_clean":"2023-06-09T11:10:51.185737+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:10:51.185737+0000","last_undegraded":"2023-06-09T11:10:51.185737+0000","last_fullsized":"2023-06-09T11:10:51.185737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461810+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:51.461810+0000","last_peered":"2023-06-09T11:10:51.461810+0000","last_clean":"2023-06-09T11:10:51.461810+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:51.461810+0000","last_undegraded":"2023-06-09T11:10:51.461810+0000","last_fullsized":"2023-06-09T11:10:51.461810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547703+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:10:51.547703+0000","last_peered":"2023-06-09T11:10:51.547703+0000","last_clean":"2023-06-09T11:10:51.547703+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:10:51.547703+0000","last_undegraded":"2023-06-09T11:10:51.547703+0000","last_fullsized":"2023-06-09T11:10:51.547703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199313+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:10:55.199313+0000","last_peered":"2023-06-09T11:10:55.199313+0000","last_clean":"2023-06-09T11:10:55.199313+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:10:55.199313+0000","last_undegraded":"2023-06-09T11:10:55.199313+0000","last_fullsized":"2023-06-09T11:10:55.199313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548150+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:10:51.548150+0000","last_peered":"2023-06-09T11:10:51.548150+0000","last_clean":"2023-06-09T11:10:51.548150+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:10:51.548150+0000","last_undegraded":"2023-06-09T11:10:51.548150+0000","last_fullsized":"2023-06-09T11:10:51.548150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.254052+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:10:52.254052+0000","last_peered":"2023-06-09T11:10:52.254052+0000","last_clean":"2023-06-09T11:10:52.254052+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:10:52.254052+0000","last_undegraded":"2023-06-09T11:10:52.254052+0000","last_fullsized":"2023-06-09T11:10:52.254052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198929+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:55.198929+0000","last_peered":"2023-06-09T11:10:55.198929+0000","last_clean":"2023-06-09T11:10:55.198929+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:55.198929+0000","last_undegraded":"2023-06-09T11:10:55.198929+0000","last_fullsized":"2023-06-09T11:10:55.198929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198635+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:10:55.198635+0000","last_peered":"2023-06-09T11:10:55.198635+0000","last_clean":"2023-06-09T11:10:55.198635+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:10:55.198635+0000","last_undegraded":"2023-06-09T11:10:55.198635+0000","last_fullsized":"2023-06-09T11:10:55.198635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.260044+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:52.260044+0000","last_peered":"2023-06-09T11:10:52.260044+0000","last_clean":"2023-06-09T11:10:52.260044+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:52.260044+0000","last_undegraded":"2023-06-09T11:10:52.260044+0000","last_fullsized":"2023-06-09T11:10:52.260044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.201285+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:10:55.201285+0000","last_peered":"2023-06-09T11:10:55.201285+0000","last_clean":"2023-06-09T11:10:55.201285+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:10:55.201285+0000","last_undegraded":"2023-06-09T11:10:55.201285+0000","last_fullsized":"2023-06-09T11:10:55.201285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8940,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362900,"statfs":{"total":96636764160,"available":96627609600,"internally_reserved":0,"allocated":299008,"data_stored":666568,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8336,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363504,"statfs":{"total":96636764160,"available":96628228096,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8916,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362924,"statfs":{"total":96636764160,"available":96627634176,"internally_reserved":0,"allocated":274432,"data_stored":665370,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0089999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8272,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363568,"statfs":{"total":96636764160,"available":96628293632,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8404,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363436,"statfs":{"total":96636764160,"available":96628158464,"internally_reserved":0,"allocated":208896,"data_stored":74510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8420,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363420,"statfs":{"total":96636764160,"available":96628142080,"internally_reserved":0,"allocated":225280,"data_stored":76040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":212992,"data_stored":75072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.085}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:11:03.898 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-09T11:11:03.898 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-09T11:11:03.898 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-06-09T11:11:03.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-06-09T11:11:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.119+0000 7f2de53ff700 1 Processor -- start 2023-06-09T11:11:04.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.120+0000 7f2de53ff700 1 -- start start 2023-06-09T11:11:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.121+0000 7f2de53ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2de01593d0 0x7f2de01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.121+0000 7f2de53ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2de0072020 0x7f2de0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.121+0000 7f2de53ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de00cc140 con 0x7f2de00c08d0 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.121+0000 7f2de53ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de00714f0 con 0x7f2de0072020 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2de53ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de00c1830 con 0x7f2de01593d0 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2de0072020 0x7f2de0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2de0072020 0x7f2de0159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48992/0 (socket says 172.21.15.7:48992) 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 -- 172.21.15.7:0/315807560 learned_addr learned my addr 172.21.15.7:0/315807560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:04.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddeffd700 1 --2- 172.21.15.7:0/315807560 >> v2:172.21.15.117:3300/0 conn(0x7f2de01593d0 0x7f2de01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 -- 172.21.15.7:0/315807560 >> v2:172.21.15.117:3300/0 conn(0x7f2de01593d0 msgr2=0x7f2de01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 --2- 172.21.15.7:0/315807560 >> v2:172.21.15.117:3300/0 conn(0x7f2de01593d0 0x7f2de01597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 -- 172.21.15.7:0/315807560 >> v1:172.21.15.117:6789/0 conn(0x7f2de00c08d0 legacy=0x7f2de0071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 -- 172.21.15.7:0/315807560 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00c0560 con 0x7f2de0072020 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.122+0000 7f2ddf7fe700 1 --2- 172.21.15.7:0/315807560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de0159ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2de006ff10 tx=0x7f2dd800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd3433b8e9c46 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2dddffb700 1 -- 172.21.15.7:0/315807560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8010070 con 0x7f2de0072020 2023-06-09T11:11:04.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2dddffb700 1 -- 172.21.15.7:0/315807560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd800dce0 con 0x7f2de0072020 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2dddffb700 1 -- 172.21.15.7:0/315807560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800b310 con 0x7f2de0072020 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 -- 172.21.15.7:0/315807560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 msgr2=0x7f2de0159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 --2- 172.21.15.7:0/315807560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de0159ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2de006ff10 tx=0x7f2dd800d430 comp rx=0 tx=0).stop 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 -- 172.21.15.7:0/315807560 shutdown_connections 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 --2- 172.21.15.7:0/315807560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de0159ce0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 --2- 172.21.15.7:0/315807560 >> v2:172.21.15.117:3300/0 conn(0x7f2de01593d0 0x7f2de01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 -- 172.21.15.7:0/315807560 >> 172.21.15.7:0/315807560 conn(0x7f2de006a4d0 msgr2=0x7f2de006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 -- 172.21.15.7:0/315807560 shutdown_connections 2023-06-09T11:11:04.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.123+0000 7f2de53ff700 1 -- 172.21.15.7:0/315807560 wait complete. 2023-06-09T11:11:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.125+0000 7f2de53ff700 1 Processor -- start 2023-06-09T11:11:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.125+0000 7f2de53ff700 1 -- start start 2023-06-09T11:11:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de00c08d0 0x7f2de014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de01593d0 0x7f2de014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de01efb30 con 0x7f2de0072020 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de0156e10 con 0x7f2de00c08d0 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2de53ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de01e6040 con 0x7f2de01593d0 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2ddeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2ddeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49006/0 (socket says 172.21.15.7:49006) 2023-06-09T11:11:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.126+0000 7f2ddeffd700 1 -- 172.21.15.7:0/794186634 learned_addr learned my addr 172.21.15.7:0/794186634 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 -- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de01593d0 msgr2=0x7f2de014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de01593d0 0x7f2de014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 -- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de00c08d0 msgr2=0x7f2de014df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de00c08d0 0x7f2de014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 -- 172.21.15.7:0/794186634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd800bd40 con 0x7f2de0072020 2023-06-09T11:11:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2ddeffd700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000e850 tx=0x7f2dd000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00095b0 con 0x7f2de0072020 2023-06-09T11:11:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000ee50 con 0x7f2de0072020 2023-06-09T11:11:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd000ca70 con 0x7f2de0072020 2023-06-09T11:11:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de00c1d60 con 0x7f2de0072020 2023-06-09T11:11:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.127+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de00c1a80 con 0x7f2de0072020 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.128+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de00c2040 con 0x7f2de0072020 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.129+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2dd000cbd0 con 0x7f2de0072020 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.130+0000 7f2dc77fe700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 0x7f2dcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.130+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2dd0089920 con 0x7f2de0072020 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.130+0000 7f2dde7fc700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 0x7f2dcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.131+0000 7f2dde7fc700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 0x7f2dcc06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4000f60 tx=0x7f2dd4007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.135+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2dd0058330 con 0x7f2de0072020 2023-06-09T11:11:04.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.386+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2de0074cf0 con 0x7f2de0072020 2023-06-09T11:11:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.386+0000 7f2dc77fe700 1 -- 172.21.15.7:0/794186634 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2dd0021040 con 0x7f2de0072020 2023-06-09T11:11:04.391 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:04.391 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 msgr2=0x7f2dcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 0x7f2dcc06d190 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4000f60 tx=0x7f2dd4007a40 comp rx=0 tx=0).stop 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 msgr2=0x7f2de01f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000e850 tx=0x7f2dd000ec10 comp rx=0 tx=0).stop 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 shutdown_connections 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2dcc06ace0 0x7f2dcc06d190 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2de01593d0 0x7f2de014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2de0072020 0x7f2de01f22f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 --2- 172.21.15.7:0/794186634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2de00c08d0 0x7f2de014df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 >> 172.21.15.7:0/794186634 conn(0x7f2de006a4d0 msgr2=0x7f2de00c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 shutdown_connections 2023-06-09T11:11:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.388+0000 7f2de53ff700 1 -- 172.21.15.7:0/794186634 wait complete. 2023-06-09T11:11:04.411 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-06-09T11:11:04.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.605+0000 7f7f293ff700 1 Processor -- start 2023-06-09T11:11:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.605+0000 7f7f293ff700 1 -- start start 2023-06-09T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f293ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f293ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f240c5a80 con 0x7f7f24153880 2023-06-09T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f293ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2415a500 con 0x7f7f2415b4a0 2023-06-09T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f293ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240c5f10 con 0x7f7f240c9f40 2023-06-09T11:11:04.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f22ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f24153880 0x7f7f2415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52804/0 (socket says 172.21.15.7:52804) 2023-06-09T11:11:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.606+0000 7f7f22ffd700 1 -- 172.21.15.7:0/3171584585 learned_addr learned my addr 172.21.15.7:0/3171584585 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:04.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f237fe700 1 --2- 172.21.15.7:0/3171584585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1761372829 0 0) 0x7f7f240c5a80 con 0x7f7f24153880 2023-06-09T11:11:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f0c0035b0 con 0x7f7f24153880 2023-06-09T11:11:04.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3780077805 0 0) 0x7f7f18001110 con 0x7f7f2415b4a0 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f240c5a80 con 0x7f7f2415b4a0 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f237fe700 1 -- 172.21.15.7:0/3171584585 >> v1:172.21.15.117:6789/0 conn(0x7f7f2415b4a0 legacy=0x7f7f240c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f237fe700 1 -- 172.21.15.7:0/3171584585 >> v1:172.21.15.7:6789/0 conn(0x7f7f24153880 legacy=0x7f7f2415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f237fe700 1 -- 172.21.15.7:0/3171584585 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f241596e0 con 0x7f7f240c9f40 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.607+0000 7f7f237fe700 1 --2- 172.21.15.7:0/3171584585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240c5540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009dc0 tx=0x7f7f1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82506ec353f01e94 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00e070 con 0x7f7f240c9f40 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c00bb20 con 0x7f7f240c9f40 2023-06-09T11:11:04.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f21ffb700 1 -- 172.21.15.7:0/3171584585 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c00be90 con 0x7f7f240c9f40 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 -- 172.21.15.7:0/3171584585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 msgr2=0x7f7f240c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 --2- 172.21.15.7:0/3171584585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240c5540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009dc0 tx=0x7f7f1c00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 -- 172.21.15.7:0/3171584585 shutdown_connections 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 --2- 172.21.15.7:0/3171584585 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240c5540 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 -- 172.21.15.7:0/3171584585 >> 172.21.15.7:0/3171584585 conn(0x7f7f240737f0 msgr2=0x7f7f240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:04.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 -- 172.21.15.7:0/3171584585 shutdown_connections 2023-06-09T11:11:04.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.608+0000 7f7f293ff700 1 -- 172.21.15.7:0/3171584585 wait complete. 2023-06-09T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.610+0000 7f7f293ff700 1 Processor -- start 2023-06-09T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.610+0000 7f7f293ff700 1 -- start start 2023-06-09T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2415b4a0 0x7f7f241f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241e3c80 con 0x7f7f24153880 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241527d0 con 0x7f7f2415b4a0 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f293ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241e2b80 con 0x7f7f240c9f40 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49018/0 (socket says 172.21.15.7:49018) 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f227fc700 1 -- 172.21.15.7:0/2042063217 learned_addr learned my addr 172.21.15.7:0/2042063217 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f22ffd700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.611+0000 7f7f237fe700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2415b4a0 0x7f7f241f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 -- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 msgr2=0x7f7f240cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 -- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2415b4a0 msgr2=0x7f7f241f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2415b4a0 0x7f7f241f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 -- 172.21.15.7:0/2042063217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009a70 con 0x7f7f24153880 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f227fc700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c760 tx=0x7f7f1800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f180052d0 con 0x7f7f24153880 2023-06-09T11:11:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18005430 con 0x7f7f24153880 2023-06-09T11:11:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f180038c0 con 0x7f7f24153880 2023-06-09T11:11:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f24159c60 con 0x7f7f24153880 2023-06-09T11:11:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.612+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f24159980 con 0x7f7f24153880 2023-06-09T11:11:04.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.613+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f24159f40 con 0x7f7f24153880 2023-06-09T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.614+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f18003a20 con 0x7f7f24153880 2023-06-09T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.615+0000 7f7f0b7fe700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.615+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7f180887d0 con 0x7f7f24153880 2023-06-09T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.616+0000 7f7f22ffd700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:04.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.616+0000 7f7f22ffd700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f14007fd0 tx=0x7f7f14007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:04.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.619+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7f18053750 con 0x7f7f24153880 2023-06-09T11:11:04.791 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:11:04.786+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:11:04.791 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:11:04.786+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:11:04.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.829+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f2415a500 con 0x7f7f24153880 2023-06-09T11:11:04.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.830+0000 7f7f0b7fe700 1 -- 172.21.15.7:0/2042063217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f7f2415a500 con 0x7f7f24153880 2023-06-09T11:11:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:04.835 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 msgr2=0x7f7f1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 0x7f7f1006d190 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f14007fd0 tx=0x7f7f14007de0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 msgr2=0x7f7f241f2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f1800c760 tx=0x7f7f1800ca70 comp rx=0 tx=0).stop 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 shutdown_connections 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f2415b4a0 0x7f7f241f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7f1006ace0 0x7f7f1006d190 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f240c9f40 0x7f7f240cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 --2- 172.21.15.7:0/2042063217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f24153880 0x7f7f241f2810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 >> 172.21.15.7:0/2042063217 conn(0x7f7f240737f0 msgr2=0x7f7f2414dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:04.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 shutdown_connections 2023-06-09T11:11:04.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:04.833+0000 7f7f293ff700 1 -- 172.21.15.7:0/2042063217 wait complete. 2023-06-09T11:11:04.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:11:04.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:11:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.055+0000 7f36027ff700 1 Processor -- start 2023-06-09T11:11:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.055+0000 7f36027ff700 1 -- start start 2023-06-09T11:11:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.056+0000 7f36027ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f35fc1537a0 0x7f35fc06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.056+0000 7f36027ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35fc0c64e0 0x7f35fc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.056+0000 7f36027ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0c6220 con 0x7f35fc06b740 2023-06-09T11:11:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.056+0000 7f36027ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0c4f70 con 0x7f35fc0c64e0 2023-06-09T11:11:05.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.056+0000 7f36027ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc0c5c20 con 0x7f35fc1537a0 2023-06-09T11:11:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35fc0c64e0 0x7f35fc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35fc0c64e0 0x7f35fc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49022/0 (socket says 172.21.15.7:49022) 2023-06-09T11:11:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1217370732 learned_addr learned my addr 172.21.15.7:0/1217370732 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1217370732 >> v2:172.21.15.117:3300/0 conn(0x7f35fc1537a0 msgr2=0x7f35fc06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 --2- 172.21.15.7:0/1217370732 >> v2:172.21.15.117:3300/0 conn(0x7f35fc1537a0 0x7f35fc06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1217370732 >> v1:172.21.15.117:6789/0 conn(0x7f35fc06b740 legacy=0x7f35fc15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1217370732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc0c4480 con 0x7f35fc0c64e0 2023-06-09T11:11:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35fb7fe700 1 --2- 172.21.15.7:0/1217370732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0cf590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0cd130 tx=0x7f35f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a44b3a41ffc2db server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.057+0000 7f35faffd700 1 -- 172.21.15.7:0/1217370732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0004230 con 0x7f35fc0c64e0 2023-06-09T11:11:05.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f35faffd700 1 -- 172.21.15.7:0/1217370732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0004390 con 0x7f35fc0c64e0 2023-06-09T11:11:05.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f35faffd700 1 -- 172.21.15.7:0/1217370732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f00036a0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 -- 172.21.15.7:0/1217370732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 msgr2=0x7f35fc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 --2- 172.21.15.7:0/1217370732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0cf590 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0cd130 tx=0x7f35f0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 -- 172.21.15.7:0/1217370732 shutdown_connections 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 --2- 172.21.15.7:0/1217370732 >> v2:172.21.15.117:3300/0 conn(0x7f35fc1537a0 0x7f35fc06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 --2- 172.21.15.7:0/1217370732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0cf590 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 -- 172.21.15.7:0/1217370732 >> 172.21.15.7:0/1217370732 conn(0x7f35fc072040 msgr2=0x7f35fc072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 -- 172.21.15.7:0/1217370732 shutdown_connections 2023-06-09T11:11:05.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.058+0000 7f36027ff700 1 -- 172.21.15.7:0/1217370732 wait complete. 2023-06-09T11:11:05.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.060+0000 7f36027ff700 1 Processor -- start 2023-06-09T11:11:05.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.060+0000 7f36027ff700 1 -- start start 2023-06-09T11:11:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 0x7f35fc0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 0x7f35fc1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0caee0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0cbb60 con 0x7f35fc1537a0 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f36027ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc0cc080 con 0x7f35fc06b740 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f35fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f35fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49026/0 (socket says 172.21.15.7:49026) 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.061+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1920765629 learned_addr learned my addr 172.21.15.7:0/1920765629 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f3600d9c700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 0x7f35fc1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 msgr2=0x7f35fc0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fbfff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 0x7f35fc0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 0x7f35fc0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 msgr2=0x7f35fc1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 0x7f35fc1f2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 -- 172.21.15.7:0/1920765629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0009550 con 0x7f35fc0c64e0 2023-06-09T11:11:05.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f3600d9c700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 0x7f35fc1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fbfff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 0x7f35fc0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35fb7fe700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009fd0 tx=0x7f35f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0003db0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f001f070 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc0c46d0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0011870 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.062+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc15ae20 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.063+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35fc0c49b0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.064+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35f0021030 con 0x7f35fc0c64e0 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.066+0000 7f35f8ff9700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 0x7f35e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.066+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f35f008a9d0 con 0x7f35fc0c64e0 2023-06-09T11:11:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.066+0000 7f35fbfff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 0x7f35e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.067+0000 7f35fbfff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 0x7f35e806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007900 tx=0x7f35ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.070+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f35f0055a70 con 0x7f35fc0c64e0 2023-06-09T11:11:05.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.277+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35fc0cb020 con 0x7f35fc0c64e0 2023-06-09T11:11:05.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.278+0000 7f35f8ff9700 1 -- 172.21.15.7:0/1920765629 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35fc0cb020 con 0x7f35fc0c64e0 2023-06-09T11:11:05.283 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:11:05.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 msgr2=0x7f35e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 0x7f35e806d190 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f35ec007900 tx=0x7f35ec008040 comp rx=0 tx=0).stop 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 msgr2=0x7f35fc0c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009fd0 tx=0x7f35f0004750 comp rx=0 tx=0).stop 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 shutdown_connections 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35fc1537a0 0x7f35fc1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f35e806ace0 0x7f35e806d190 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35fc06b740 0x7f35fc0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 --2- 172.21.15.7:0/1920765629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35fc0c64e0 0x7f35fc0c8a20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 >> 172.21.15.7:0/1920765629 conn(0x7f35fc072040 msgr2=0x7f35fc151610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:05.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 shutdown_connections 2023-06-09T11:11:05.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.280+0000 7f36027ff700 1 -- 172.21.15.7:0/1920765629 wait complete. 2023-06-09T11:11:05.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:11:05.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.502+0000 7f3d36fff700 1 Processor -- start 2023-06-09T11:11:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.502+0000 7f3d36fff700 1 -- start start 2023-06-09T11:11:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d36fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d300c64e0 0x7f3d3006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d36fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300c6220 con 0x7f3d3006b740 2023-06-09T11:11:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d36fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d300c4480 con 0x7f3d301537a0 2023-06-09T11:11:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d36fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d300c4f70 con 0x7f3d300c64e0 2023-06-09T11:11:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d2ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d300c64e0 0x7f3d3006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d2ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3d300c64e0 0x7f3d3006b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49040/0 (socket says 172.21.15.7:49040) 2023-06-09T11:11:05.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d3559c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d3006b740 0x7f3d3015b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52808/0 (socket says 172.21.15.7:52808) 2023-06-09T11:11:05.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.503+0000 7f3d2ffff700 1 -- 172.21.15.7:0/846873777 learned_addr learned my addr 172.21.15.7:0/846873777 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:05.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2ffff700 1 -- 172.21.15.7:0/846873777 >> v1:172.21.15.117:6789/0 conn(0x7f3d301537a0 legacy=0x7f3d300cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2ffff700 1 -- 172.21.15.7:0/846873777 >> v1:172.21.15.7:6789/0 conn(0x7f3d3006b740 legacy=0x7f3d3015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2ffff700 1 -- 172.21.15.7:0/846873777 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3015ae20 con 0x7f3d300c64e0 2023-06-09T11:11:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2ffff700 1 --2- 172.21.15.7:0/846873777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d300c64e0 0x7f3d3006b1b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0098f0 tx=0x7f3d1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fa003ffc69dae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2f7fe700 1 -- 172.21.15.7:0/846873777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00b8a0 con 0x7f3d300c64e0 2023-06-09T11:11:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2f7fe700 1 -- 172.21.15.7:0/846873777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c00ba00 con 0x7f3d300c64e0 2023-06-09T11:11:05.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.504+0000 7f3d2f7fe700 1 -- 172.21.15.7:0/846873777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00bd90 con 0x7f3d300c64e0 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 -- 172.21.15.7:0/846873777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d300c64e0 msgr2=0x7f3d3006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 --2- 172.21.15.7:0/846873777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d300c64e0 0x7f3d3006b1b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0098f0 tx=0x7f3d1c009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 -- 172.21.15.7:0/846873777 shutdown_connections 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 --2- 172.21.15.7:0/846873777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d300c64e0 0x7f3d3006b1b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 -- 172.21.15.7:0/846873777 >> 172.21.15.7:0/846873777 conn(0x7f3d30072040 msgr2=0x7f3d30072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:05.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 -- 172.21.15.7:0/846873777 shutdown_connections 2023-06-09T11:11:05.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.505+0000 7f3d36fff700 1 -- 172.21.15.7:0/846873777 wait complete. 2023-06-09T11:11:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.507+0000 7f3d36fff700 1 Processor -- start 2023-06-09T11:11:05.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.508+0000 7f3d36fff700 1 -- start start 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d3006b740 0x7f3d301ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d300c64e0 0x7f3d300ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300cc8b0 con 0x7f3d301537a0 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d301eff20 con 0x7f3d3006b740 2023-06-09T11:11:05.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d36fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300c92f0 con 0x7f3d300c64e0 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d3559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d3559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49050/0 (socket says 172.21.15.7:49050) 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d3559c700 1 -- 172.21.15.7:0/3328890016 learned_addr learned my addr 172.21.15.7:0/3328890016 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d34d9b700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d3006b740 0x7f3d301ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.509+0000 7f3d2ffff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d300c64e0 0x7f3d300ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 -- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d300c64e0 msgr2=0x7f3d300ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d300c64e0 0x7f3d300ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 -- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d3006b740 msgr2=0x7f3d301ef9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d3006b740 0x7f3d301ef9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 -- 172.21.15.7:0/3328890016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1c009550 con 0x7f3d301537a0 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d3559c700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800cc00 tx=0x7f3d2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2800aae0 con 0x7f3d301537a0 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2800ac40 con 0x7f3d301537a0 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28007070 con 0x7f3d301537a0 2023-06-09T11:11:05.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300c49b0 con 0x7f3d301537a0 2023-06-09T11:11:05.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.510+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d300c46d0 con 0x7f3d301537a0 2023-06-09T11:11:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.511+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d300c4c90 con 0x7f3d301537a0 2023-06-09T11:11:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.512+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d280071d0 con 0x7f3d301537a0 2023-06-09T11:11:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.513+0000 7f3d2d7fa700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 0x7f3d1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.513+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3d28089080 con 0x7f3d301537a0 2023-06-09T11:11:05.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.514+0000 7f3d34d9b700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 0x7f3d1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.514+0000 7f3d34d9b700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 0x7f3d1406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000f60 tx=0x7f3d24009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.517+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3d280bb2d0 con 0x7f3d301537a0 2023-06-09T11:11:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.725+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d300c6220 con 0x7f3d301537a0 2023-06-09T11:11:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.726+0000 7f3d2d7fa700 1 -- 172.21.15.7:0/3328890016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d300c6220 con 0x7f3d301537a0 2023-06-09T11:11:05.731 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:11:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 msgr2=0x7f3d1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 0x7f3d1406d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3d24000f60 tx=0x7f3d24009040 comp rx=0 tx=0).stop 2023-06-09T11:11:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 msgr2=0x7f3d300ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800cc00 tx=0x7f3d2800a060 comp rx=0 tx=0).stop 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 shutdown_connections 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d301537a0 0x7f3d300ca440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d3006b740 0x7f3d301ef9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d1406ace0 0x7f3d1406d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 --2- 172.21.15.7:0/3328890016 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d300c64e0 0x7f3d300ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 >> 172.21.15.7:0/3328890016 conn(0x7f3d30072040 msgr2=0x7f3d30152380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 shutdown_connections 2023-06-09T11:11:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.728+0000 7f3d36fff700 1 -- 172.21.15.7:0/3328890016 wait complete. 2023-06-09T11:11:05.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:11:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.949+0000 7f65537ff700 1 Processor -- start 2023-06-09T11:11:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.949+0000 7f65537ff700 1 -- start start 2023-06-09T11:11:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f65537ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f654c1537a0 0x7f654c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f65537ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c0c6220 con 0x7f654c06b740 2023-06-09T11:11:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f65537ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654c0c4480 con 0x7f654c0c64e0 2023-06-09T11:11:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f65537ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654c0c4f70 con 0x7f654c1537a0 2023-06-09T11:11:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f6551d9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f654c06b740 0x7f654c15b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52818/0 (socket says 172.21.15.7:52818) 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f6551d9c700 1 -- 172.21.15.7:0/888608626 learned_addr learned my addr 172.21.15.7:0/888608626 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f655159b700 1 --2- 172.21.15.7:0/888608626 >> v2:172.21.15.117:3300/0 conn(0x7f654c1537a0 0x7f654c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881547293 0 0) 0x7f6544001490 con 0x7f654c06b740 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.951+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65380035b0 con 0x7f654c06b740 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258252349 0 0) 0x7f65380035b0 con 0x7f654c06b740 2023-06-09T11:11:05.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 >> v2:172.21.15.117:3300/0 conn(0x7f654c1537a0 msgr2=0x7f654c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 --2- 172.21.15.7:0/888608626 >> v2:172.21.15.117:3300/0 conn(0x7f654c1537a0 0x7f654c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 >> v1:172.21.15.117:6789/0 conn(0x7f654c0c64e0 legacy=0x7f654c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c15ae20 con 0x7f654c06b740 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f6544001b80 con 0x7f654c06b740 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 >> v1:172.21.15.7:6789/0 conn(0x7f654c06b740 legacy=0x7f654c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6538003a00 0x7f6538003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f65380048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65380050b0 0x7f653800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c0c4f70 con 0x7f65380043f0 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653800bf50 con 0x7f65380050b0 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653800c110 con 0x7f6538003a00 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f65380048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f655159b700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6538003a00 0x7f6538003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6551d9c700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65380050b0 0x7f653800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 -- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6538003a00 msgr2=0x7f6538003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6538003a00 0x7f6538003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 -- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65380050b0 msgr2=0x7f653800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65380050b0 0x7f653800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6550d9a700 1 -- 172.21.15.7:0/888608626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380035b0 con 0x7f65380043f0 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.952+0000 7f6551d9c700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65380050b0 0x7f653800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f6551d9c700 1 -- 172.21.15.7:0/888608626 reap_dead start 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f6550d9a700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f65380048b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f653c0018d0 tx=0x7f6538005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf35ad571d63ec2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c0052d0 con 0x7f65380043f0 2023-06-09T11:11:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c005430 con 0x7f65380043f0 2023-06-09T11:11:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f6543fff700 1 -- 172.21.15.7:0/888608626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c0038a0 con 0x7f65380043f0 2023-06-09T11:11:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f65537ff700 1 -- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 msgr2=0x7f65380048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f65537ff700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f65380048b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f653c0018d0 tx=0x7f6538005880 comp rx=0 tx=0).stop 2023-06-09T11:11:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f65537ff700 1 -- 172.21.15.7:0/888608626 shutdown_connections 2023-06-09T11:11:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f65537ff700 1 --2- 172.21.15.7:0/888608626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f65380048b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.953+0000 7f65537ff700 1 -- 172.21.15.7:0/888608626 >> 172.21.15.7:0/888608626 conn(0x7f654c072040 msgr2=0x7f654c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.954+0000 7f65537ff700 1 -- 172.21.15.7:0/888608626 shutdown_connections 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.954+0000 7f65537ff700 1 -- 172.21.15.7:0/888608626 wait complete. 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.956+0000 7f65537ff700 1 Processor -- start 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.956+0000 7f65537ff700 1 -- start start 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 0x7f654c0c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f654c144660 0x7f654c144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c0c69c0 con 0x7f65380043f0 2023-06-09T11:11:05.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654c145040 con 0x7f654c14f560 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f65537ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654c1ef190 con 0x7f654c144660 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49070/0 (socket says 172.21.15.7:49070) 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 -- 172.21.15.7:0/3908282700 learned_addr learned my addr 172.21.15.7:0/3908282700 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f6551d9c700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f654c144660 0x7f654c144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f6550d9a700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 0x7f654c0c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 -- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f654c144660 msgr2=0x7f654c144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f654c144660 0x7f654c144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 -- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 msgr2=0x7f654c0c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.958+0000 7f655159b700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 0x7f654c0c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f655159b700 1 -- 172.21.15.7:0/3908282700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c0c49b0 con 0x7f65380043f0 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f6550d9a700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 0x7f654c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f655159b700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6548011ad0 tx=0x7f6548011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654800fa00 con 0x7f65380043f0 2023-06-09T11:11:05.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654800fb60 con 0x7f65380043f0 2023-06-09T11:11:05.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548014440 con 0x7f65380043f0 2023-06-09T11:11:05.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654c0c46d0 con 0x7f65380043f0 2023-06-09T11:11:05.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.959+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654c0c4c90 con 0x7f65380043f0 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.960+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654c074cf0 con 0x7f65380043f0 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.961+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65480145a0 con 0x7f65380043f0 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.962+0000 7f6541ffb700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 0x7f653006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.962+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6548089cf0 con 0x7f65380043f0 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.963+0000 7f6550d9a700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 0x7f653006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:05.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.963+0000 7f6550d9a700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 0x7f653006d1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f654c145410 tx=0x7f653c005000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:05.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:05.966+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6548054c70 con 0x7f65380043f0 2023-06-09T11:11:06.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.202+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f654c06aaf0 con 0x7f65380043f0 2023-06-09T11:11:06.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.202+0000 7f6541ffb700 1 -- 172.21.15.7:0/3908282700 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6548022020 con 0x7f65380043f0 2023-06-09T11:11:06.208 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:11:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 msgr2=0x7f653006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 0x7f653006d1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f654c145410 tx=0x7f653c005000 comp rx=0 tx=0).stop 2023-06-09T11:11:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 msgr2=0x7f654c0c8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6548011ad0 tx=0x7f6548011e90 comp rx=0 tx=0).stop 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 shutdown_connections 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f653006ad10 0x7f653006d1c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f654c144660 0x7f654c144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65380043f0 0x7f654c0c8420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 --2- 172.21.15.7:0/3908282700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f654c14f560 0x7f654c0c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 >> 172.21.15.7:0/3908282700 conn(0x7f654c072040 msgr2=0x7f654c0ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 shutdown_connections 2023-06-09T11:11:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.205+0000 7f65537ff700 1 -- 172.21.15.7:0/3908282700 wait complete. 2023-06-09T11:11:06.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:11:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.426+0000 7fe7aa3ff700 1 Processor -- start 2023-06-09T11:11:06.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.427+0000 7fe7aa3ff700 1 -- start start 2023-06-09T11:11:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7aa3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7a41537a0 0x7fe7a406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7aa3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7aa3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a40c6220 con 0x7fe7a406b740 2023-06-09T11:11:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7aa3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a40c4f70 con 0x7fe7a41537a0 2023-06-09T11:11:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7aa3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c5c20 con 0x7fe7a40c64e0 2023-06-09T11:11:06.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7a3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7a41537a0 0x7fe7a406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7a37fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:58252/0 (socket says 172.21.15.7:58252) 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.428+0000 7fe7a37fe700 1 -- 172.21.15.7:0/2964114541 learned_addr learned my addr 172.21.15.7:0/2964114541 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 msgr2=0x7fe7a40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a3fff700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2964114541 >> v1:172.21.15.117:6789/0 conn(0x7fe7a406b740 legacy=0x7fe7a415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2964114541 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a40c4480 con 0x7fe7a41537a0 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a37fe700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.429+0000 7fe7a3fff700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a406b1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe794009840 tx=0x7fe794009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f08701ea9db9df server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7a2ffd700 1 -- 172.21.15.7:0/2964114541 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794011830 con 0x7fe7a41537a0 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7a2ffd700 1 -- 172.21.15.7:0/2964114541 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe794011990 con 0x7fe7a41537a0 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7a2ffd700 1 -- 172.21.15.7:0/2964114541 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794011d20 con 0x7fe7a41537a0 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 msgr2=0x7fe7a406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a406b1b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe794009840 tx=0x7fe794009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2964114541 shutdown_connections 2023-06-09T11:11:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a406b1b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2964114541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2964114541 >> 172.21.15.7:0/2964114541 conn(0x7fe7a4072040 msgr2=0x7fe7a4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2964114541 shutdown_connections 2023-06-09T11:11:06.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.430+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2964114541 wait complete. 2023-06-09T11:11:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.433+0000 7fe7aa3ff700 1 Processor -- start 2023-06-09T11:11:06.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.433+0000 7fe7aa3ff700 1 -- start start 2023-06-09T11:11:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a41ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a414e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c6fb0 con 0x7fe7a406b740 2023-06-09T11:11:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a40c8080 con 0x7fe7a41537a0 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7aa3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a41efa30 con 0x7fe7a40c64e0 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49086/0 (socket says 172.21.15.7:49086) 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2263743318 learned_addr learned my addr 172.21.15.7:0/2263743318 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.434+0000 7fe7a37fe700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a41ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a899c700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a414e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 msgr2=0x7fe7a41ef4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a41ef4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 msgr2=0x7fe7a414e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a414e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 -- 172.21.15.7:0/2263743318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009550 con 0x7fe7a406b740 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a899c700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a3fff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe794009fd0 tx=0x7fe794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940036a0 con 0x7fe7a406b740 2023-06-09T11:11:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79401e070 con 0x7fe7a406b740 2023-06-09T11:11:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7940094a0 con 0x7fe7a406b740 2023-06-09T11:11:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.435+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794003a40 con 0x7fe7a406b740 2023-06-09T11:11:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.436+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a40c46d0 con 0x7fe7a406b740 2023-06-09T11:11:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.437+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a415ae20 con 0x7fe7a406b740 2023-06-09T11:11:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.437+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe794003ba0 con 0x7fe7a406b740 2023-06-09T11:11:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.438+0000 7fe7a0ff9700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.438+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe79408a5c0 con 0x7fe7a406b740 2023-06-09T11:11:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.439+0000 7fe7a37fe700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.439+0000 7fe7a37fe700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 0x7fe79006d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7a41475d0 tx=0x7fe7980092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.443+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe794055540 con 0x7fe7a406b740 2023-06-09T11:11:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.649+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe7a40c49b0 con 0x7fe7a406b740 2023-06-09T11:11:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.650+0000 7fe7a0ff9700 1 -- 172.21.15.7:0/2263743318 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe7940200a0 con 0x7fe7a406b740 2023-06-09T11:11:06.656 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:11:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 msgr2=0x7fe79006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 0x7fe79006d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7a41475d0 tx=0x7fe7980092c0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 msgr2=0x7fe7a40c7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe794009fd0 tx=0x7fe794004750 comp rx=0 tx=0).stop 2023-06-09T11:11:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 shutdown_connections 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7a41537a0 0x7fe7a414e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7a406b740 0x7fe7a40c7b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe79006ace0 0x7fe79006d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 --2- 172.21.15.7:0/2263743318 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7a40c64e0 0x7fe7a41ef4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 >> 172.21.15.7:0/2263743318 conn(0x7fe7a4072040 msgr2=0x7fe7a40ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 shutdown_connections 2023-06-09T11:11:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.652+0000 7fe7aa3ff700 1 -- 172.21.15.7:0/2263743318 wait complete. 2023-06-09T11:11:06.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.870+0000 7f62d87ff700 1 Processor -- start 2023-06-09T11:11:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.870+0000 7f62d87ff700 1 -- start start 2023-06-09T11:11:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d87ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62d0153880 0x7f62d015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d00c9f40 0x7f62d015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d87ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d015a500 con 0x7f62d015b2d0 2023-06-09T11:11:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d87ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d00c5f10 con 0x7f62d0153880 2023-06-09T11:11:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d015da00 con 0x7f62d00c9f40 2023-06-09T11:11:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.871+0000 7f62d659b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62d0153880 0x7f62d015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d5d9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d00c9f40 0x7f62d015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62d0153880 0x7f62d015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:51372/0 (socket says 172.21.15.7:51372) 2023-06-09T11:11:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 -- 172.21.15.7:0/3433309774 learned_addr learned my addr 172.21.15.7:0/3433309774 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 -- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d00c9f40 msgr2=0x7f62d015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 --2- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d00c9f40 0x7f62d015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 -- 172.21.15.7:0/3433309774 >> v1:172.21.15.117:6789/0 conn(0x7f62d015b2d0 legacy=0x7f62d00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 -- 172.21.15.7:0/3433309774 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d015b750 con 0x7f62d0153880 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d659b700 1 --2- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d0153880 0x7f62d015d4c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098f0 tx=0x7f62cc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=754cadf5a530aa07 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d5599700 1 -- 172.21.15.7:0/3433309774 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc011830 con 0x7f62d0153880 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d5599700 1 -- 172.21.15.7:0/3433309774 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc011990 con 0x7f62d0153880 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.872+0000 7f62d5599700 1 -- 172.21.15.7:0/3433309774 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc011d20 con 0x7f62d0153880 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 -- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d0153880 msgr2=0x7f62d015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 --2- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d0153880 0x7f62d015d4c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f62cc0098f0 tx=0x7f62cc009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 -- 172.21.15.7:0/3433309774 shutdown_connections 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 --2- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d0153880 0x7f62d015d4c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 --2- 172.21.15.7:0/3433309774 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d00c9f40 0x7f62d015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 -- 172.21.15.7:0/3433309774 >> 172.21.15.7:0/3433309774 conn(0x7f62d00737f0 msgr2=0x7f62d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 -- 172.21.15.7:0/3433309774 shutdown_connections 2023-06-09T11:11:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.873+0000 7f62d87ff700 1 -- 172.21.15.7:0/3433309774 wait complete. 2023-06-09T11:11:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.875+0000 7f62d87ff700 1 Processor -- start 2023-06-09T11:11:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.875+0000 7f62d87ff700 1 -- start start 2023-06-09T11:11:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d00c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d00cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0158360 con 0x7f62d015b2d0 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d00c69d0 con 0x7f62d00c9f40 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01f2730 con 0x7f62d0153880 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49090/0 (socket says 172.21.15.7:49090) 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d6d9c700 1 -- 172.21.15.7:0/859320314 learned_addr learned my addr 172.21.15.7:0/859320314 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d659b700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d00c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.876+0000 7f62d5d9a700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d00cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 -- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 msgr2=0x7f62d00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d00cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 -- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 msgr2=0x7f62d00c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d00c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 -- 172.21.15.7:0/859320314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc009550 con 0x7f62d015b2d0 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d5d9a700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d6d9c700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f62c800cb50 tx=0x7f62c800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003c00 con 0x7f62d015b2d0 2023-06-09T11:11:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8003d60 con 0x7f62d015b2d0 2023-06-09T11:11:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800aac0 con 0x7f62d015b2d0 2023-06-09T11:11:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0159980 con 0x7f62d015b2d0 2023-06-09T11:11:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.877+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d01596e0 con 0x7f62d015b2d0 2023-06-09T11:11:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.879+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0159c60 con 0x7f62d015b2d0 2023-06-09T11:11:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.879+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62c800ac20 con 0x7f62d015b2d0 2023-06-09T11:11:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.880+0000 7f62c6ffd700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.880+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f62c80892c0 con 0x7f62d015b2d0 2023-06-09T11:11:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.881+0000 7f62d659b700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:06.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.881+0000 7f62d659b700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 0x7f62b406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c40 tx=0x7f62cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:06.885+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62c8054370 con 0x7f62d015b2d0 2023-06-09T11:11:07.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.135+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d015a500 con 0x7f62d015b2d0 2023-06-09T11:11:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.135+0000 7f62c6ffd700 1 -- 172.21.15.7:0/859320314 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f62d015a500 con 0x7f62d015b2d0 2023-06-09T11:11:07.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:07.141 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:07.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.137+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 msgr2=0x7f62b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.137+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 0x7f62b406d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009c40 tx=0x7f62cc008030 comp rx=0 tx=0).stop 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.137+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 msgr2=0x7f62d01f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.137+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f62c800cb50 tx=0x7f62c800ce60 comp rx=0 tx=0).stop 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 shutdown_connections 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b2d0 0x7f62d01f2b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d00c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62b406ace0 0x7f62b406d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 --2- 172.21.15.7:0/859320314 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 >> 172.21.15.7:0/859320314 conn(0x7f62d00737f0 msgr2=0x7f62d015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 shutdown_connections 2023-06-09T11:11:07.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.138+0000 7f62d87ff700 1 -- 172.21.15.7:0/859320314 wait complete. 2023-06-09T11:11:07.146 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:07.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:07.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.358+0000 7f5747dff700 1 Processor -- start 2023-06-09T11:11:07.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.358+0000 7f5747dff700 1 -- start start 2023-06-09T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5740153880 0x7f574015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57400c9f40 0x7f574015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015b2d0 0x7f57400c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57400c5f10 con 0x7f574015b2d0 2023-06-09T11:11:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f574015a500 con 0x7f57400c9f40 2023-06-09T11:11:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f5747dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574015da00 con 0x7f5740153880 2023-06-09T11:11:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f574639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015b2d0 0x7f57400c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f574639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015b2d0 0x7f57400c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49106/0 (socket says 172.21.15.7:49106) 2023-06-09T11:11:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.359+0000 7f574639c700 1 -- 172.21.15.7:0/1306338120 learned_addr learned my addr 172.21.15.7:0/1306338120 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574539a700 1 --2- 172.21.15.7:0/1306338120 >> v2:172.21.15.117:3300/0 conn(0x7f57400c9f40 0x7f574015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f5745b9b700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5740153880 0x7f574015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 -- 172.21.15.7:0/1306338120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5740153880 msgr2=0x7f574015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5740153880 0x7f574015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 -- 172.21.15.7:0/1306338120 >> v2:172.21.15.117:3300/0 conn(0x7f57400c9f40 msgr2=0x7f574015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 --2- 172.21.15.7:0/1306338120 >> v2:172.21.15.117:3300/0 conn(0x7f57400c9f40 0x7f574015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 -- 172.21.15.7:0/1306338120 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574015b750 con 0x7f574015b2d0 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f574639c700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015b2d0 0x7f57400c4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57400cd140 tx=0x7f573800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd53cb2f9b3621dd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f5744b99700 1 -- 172.21.15.7:0/1306338120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800e070 con 0x7f574015b2d0 2023-06-09T11:11:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f5744b99700 1 -- 172.21.15.7:0/1306338120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573800b9c0 con 0x7f574015b2d0 2023-06-09T11:11:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.360+0000 7f5744b99700 1 -- 172.21.15.7:0/1306338120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800bd70 con 0x7f574015b2d0 2023-06-09T11:11:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 -- 172.21.15.7:0/1306338120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015b2d0 msgr2=0x7f57400c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015b2d0 0x7f57400c4e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57400cd140 tx=0x7f573800b330 comp rx=0 tx=0).stop 2023-06-09T11:11:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 -- 172.21.15.7:0/1306338120 shutdown_connections 2023-06-09T11:11:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015b2d0 0x7f57400c4e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 --2- 172.21.15.7:0/1306338120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5740153880 0x7f574015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 --2- 172.21.15.7:0/1306338120 >> v2:172.21.15.117:3300/0 conn(0x7f57400c9f40 0x7f574015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 -- 172.21.15.7:0/1306338120 >> 172.21.15.7:0/1306338120 conn(0x7f57400737f0 msgr2=0x7f57400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 -- 172.21.15.7:0/1306338120 shutdown_connections 2023-06-09T11:11:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.361+0000 7f5747dff700 1 -- 172.21.15.7:0/1306338120 wait complete. 2023-06-09T11:11:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.363+0000 7f5747dff700 1 Processor -- start 2023-06-09T11:11:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.364+0000 7f5747dff700 1 -- start start 2023-06-09T11:11:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400c9f40 0x7f574014d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 0x7f57400b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5740150ba0 con 0x7f5740153880 2023-06-09T11:11:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401523c0 con 0x7f57400c9f40 2023-06-09T11:11:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5747dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400c69a0 con 0x7f574015b2d0 2023-06-09T11:11:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49116/0 (socket says 172.21.15.7:49116) 2023-06-09T11:11:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 -- 172.21.15.7:0/2690952352 learned_addr learned my addr 172.21.15.7:0/2690952352 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f5745b9b700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400c9f40 0x7f574014d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574639c700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 0x7f57400b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 -- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 msgr2=0x7f57400b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 0x7f57400b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 -- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400c9f40 msgr2=0x7f574014d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400c9f40 0x7f574014d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574539a700 1 -- 172.21.15.7:0/2690952352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738009b30 con 0x7f5740153880 2023-06-09T11:11:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.365+0000 7f574639c700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 0x7f57400b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f574539a700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f573000e810 tx=0x7f573000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730013610 con 0x7f5740153880 2023-06-09T11:11:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730013790 con 0x7f5740153880 2023-06-09T11:11:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730013b40 con 0x7f5740153880 2023-06-09T11:11:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5740159980 con 0x7f5740153880 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.366+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57401596e0 con 0x7f5740153880 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.367+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5740159c60 con 0x7f5740153880 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.368+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5730013ca0 con 0x7f5740153880 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.369+0000 7f57367fc700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.369+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5730088fd0 con 0x7f5740153880 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.370+0000 7f5745b9b700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.370+0000 7f5745b9b700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f573c006b20 tx=0x7f573c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.373+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5730057b00 con 0x7f5740153880 2023-06-09T11:11:07.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.582+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57400cc1e0 con 0x7f5740153880 2023-06-09T11:11:07.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.583+0000 7f57367fc700 1 -- 172.21.15.7:0/2690952352 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f57400cc1e0 con 0x7f5740153880 2023-06-09T11:11:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:07.589 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.585+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 msgr2=0x7f572406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.585+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f573c006b20 tx=0x7f573c008040 comp rx=0 tx=0).stop 2023-06-09T11:11:07.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.585+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 msgr2=0x7f57400cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.585+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f573000e810 tx=0x7f573000eb20 comp rx=0 tx=0).stop 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 shutdown_connections 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574015b2d0 0x7f57400b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400c9f40 0x7f574014d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 --2- 172.21.15.7:0/2690952352 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5740153880 0x7f57400cc4c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 >> 172.21.15.7:0/2690952352 conn(0x7f57400737f0 msgr2=0x7f574006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:07.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 shutdown_connections 2023-06-09T11:11:07.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.586+0000 7f5747dff700 1 -- 172.21.15.7:0/2690952352 wait complete. 2023-06-09T11:11:07.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:11:07.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:11:07.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.808+0000 7f56b9fff700 1 Processor -- start 2023-06-09T11:11:07.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.809+0000 7f56b9fff700 1 -- start start 2023-06-09T11:11:07.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b9fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56b41537a0 0x7f56b406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b9fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56b406b740 0x7f56b415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b9fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b40c6220 con 0x7f56b40c64e0 2023-06-09T11:11:07.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b9fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40c4f70 con 0x7f56b406b740 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b9fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b40c5c20 con 0x7f56b41537a0 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56b406b740 0x7f56b415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f56b406b740 0x7f56b415b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49118/0 (socket says 172.21.15.7:49118) 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b3fff700 1 -- 172.21.15.7:0/3094705834 learned_addr learned my addr 172.21.15.7:0/3094705834 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b37fe700 1 --2- 172.21.15.7:0/3094705834 >> v2:172.21.15.117:3300/0 conn(0x7f56b41537a0 0x7f56b406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b3fff700 1 -- 172.21.15.7:0/3094705834 >> v2:172.21.15.117:3300/0 conn(0x7f56b41537a0 msgr2=0x7f56b406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:07.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.810+0000 7f56b3fff700 1 --2- 172.21.15.7:0/3094705834 >> v2:172.21.15.117:3300/0 conn(0x7f56b41537a0 0x7f56b406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b3fff700 1 -- 172.21.15.7:0/3094705834 >> v1:172.21.15.117:6789/0 conn(0x7f56b40c64e0 legacy=0x7f56b40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:07.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b3fff700 1 -- 172.21.15.7:0/3094705834 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40c4480 con 0x7f56b406b740 2023-06-09T11:11:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b3fff700 1 --2- 172.21.15.7:0/3094705834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b415b2b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cd130 tx=0x7f56ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9502ecf01219b1c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b27fc700 1 -- 172.21.15.7:0/3094705834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac00e070 con 0x7f56b406b740 2023-06-09T11:11:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b27fc700 1 -- 172.21.15.7:0/3094705834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac003a20 con 0x7f56b406b740 2023-06-09T11:11:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b27fc700 1 -- 172.21.15.7:0/3094705834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac003dd0 con 0x7f56b406b740 2023-06-09T11:11:07.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b9fff700 1 -- 172.21.15.7:0/3094705834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 msgr2=0x7f56b415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.811+0000 7f56b9fff700 1 --2- 172.21.15.7:0/3094705834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b415b2b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f56b40cd130 tx=0x7f56ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 -- 172.21.15.7:0/3094705834 shutdown_connections 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 --2- 172.21.15.7:0/3094705834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b415b2b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 --2- 172.21.15.7:0/3094705834 >> v2:172.21.15.117:3300/0 conn(0x7f56b41537a0 0x7f56b406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 -- 172.21.15.7:0/3094705834 >> 172.21.15.7:0/3094705834 conn(0x7f56b4072040 msgr2=0x7f56b4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:07.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 -- 172.21.15.7:0/3094705834 shutdown_connections 2023-06-09T11:11:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.812+0000 7f56b9fff700 1 -- 172.21.15.7:0/3094705834 wait complete. 2023-06-09T11:11:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.813+0000 7f56b9fff700 1 Processor -- start 2023-06-09T11:11:07.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.814+0000 7f56b9fff700 1 -- start start 2023-06-09T11:11:07.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b40c64e0 0x7f56b40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b41537a0 0x7f56b40cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4153d90 con 0x7f56b406b740 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b41eeb70 con 0x7f56b40c64e0 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b9fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4154000 con 0x7f56b41537a0 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49120/0 (socket says 172.21.15.7:49120) 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 -- 172.21.15.7:0/997160812 learned_addr learned my addr 172.21.15.7:0/997160812 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b2ffd700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b40c64e0 0x7f56b40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b3fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b41537a0 0x7f56b40cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 -- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b41537a0 msgr2=0x7f56b40cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b41537a0 0x7f56b40cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 -- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b40c64e0 msgr2=0x7f56b40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:07.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b40c64e0 0x7f56b40cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.815+0000 7f56b37fe700 1 -- 172.21.15.7:0/997160812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac009b30 con 0x7f56b406b740 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f56b37fe700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e810 tx=0x7f56a400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a400c070 con 0x7f56b406b740 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a4013610 con 0x7f56b406b740 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b40c46d0 con 0x7f56b406b740 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40139c0 con 0x7f56b406b740 2023-06-09T11:11:07.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.816+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b415ae20 con 0x7f56b406b740 2023-06-09T11:11:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.817+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b40c49b0 con 0x7f56b406b740 2023-06-09T11:11:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.818+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56a4013b20 con 0x7f56b406b740 2023-06-09T11:11:07.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.819+0000 7f569bfff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 0x7f56a006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:07.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.819+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f56a40891f0 con 0x7f56b406b740 2023-06-09T11:11:07.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.819+0000 7f56b2ffd700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 0x7f56a006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:07.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.820+0000 7f56b2ffd700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 0x7f56a006d1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56b40d0680 tx=0x7f56a8007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:07.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:07.823+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f56a4054290 con 0x7f56b406b740 2023-06-09T11:11:08.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.030+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56b40b4650 con 0x7f56b406b740 2023-06-09T11:11:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.030+0000 7f569bfff700 1 -- 172.21.15.7:0/997160812 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56a40578b0 con 0x7f56b406b740 2023-06-09T11:11:08.036 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:11:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 msgr2=0x7f56a006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 0x7f56a006d1e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56b40d0680 tx=0x7f56a8007ec0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 msgr2=0x7f56b414a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f56a400e810 tx=0x7f56a400eb20 comp rx=0 tx=0).stop 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 shutdown_connections 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56a006ad30 0x7f56a006d1e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56b41537a0 0x7f56b40cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56b406b740 0x7f56b414a500 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 --2- 172.21.15.7:0/997160812 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f56b40c64e0 0x7f56b40cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.032+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 >> 172.21.15.7:0/997160812 conn(0x7f56b4072040 msgr2=0x7f56b414c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:08.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.033+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 shutdown_connections 2023-06-09T11:11:08.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.033+0000 7f56b9fff700 1 -- 172.21.15.7:0/997160812 wait complete. 2023-06-09T11:11:08.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:11:08.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.253+0000 7f08233ff700 1 Processor -- start 2023-06-09T11:11:08.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.253+0000 7f08233ff700 1 -- start start 2023-06-09T11:11:08.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f08233ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f081c1537a0 0x7f081c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f08233ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c0c64e0 0x7f081c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f08233ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081c0c6220 con 0x7f081c06b740 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f08233ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081c0c4f70 con 0x7f081c1537a0 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f08233ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c5c20 con 0x7f081c0c64e0 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f082119b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f081c1537a0 0x7f081c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f082119b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f081c1537a0 0x7f081c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49126/0 (socket says 172.21.15.7:49126) 2023-06-09T11:11:08.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f082119b700 1 -- 172.21.15.7:0/4238075411 learned_addr learned my addr 172.21.15.7:0/4238075411 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.254+0000 7f082099a700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c0c64e0 0x7f081c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f082119b700 1 -- 172.21.15.7:0/4238075411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c0c64e0 msgr2=0x7f081c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f082119b700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c0c64e0 0x7f081c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f082119b700 1 -- 172.21.15.7:0/4238075411 >> v1:172.21.15.117:6789/0 conn(0x7f081c06b740 legacy=0x7f081c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f082119b700 1 -- 172.21.15.7:0/4238075411 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0c4480 con 0x7f081c1537a0 2023-06-09T11:11:08.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f082119b700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c1537a0 0x7f081c06b1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08180098f0 tx=0x7f0818009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2a102fc873088f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f0813fff700 1 -- 172.21.15.7:0/4238075411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800b8a0 con 0x7f081c1537a0 2023-06-09T11:11:08.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f0813fff700 1 -- 172.21.15.7:0/4238075411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800ba00 con 0x7f081c1537a0 2023-06-09T11:11:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.255+0000 7f0813fff700 1 -- 172.21.15.7:0/4238075411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800bd90 con 0x7f081c1537a0 2023-06-09T11:11:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 -- 172.21.15.7:0/4238075411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c1537a0 msgr2=0x7f081c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c1537a0 0x7f081c06b1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f08180098f0 tx=0x7f0818009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 -- 172.21.15.7:0/4238075411 shutdown_connections 2023-06-09T11:11:08.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c1537a0 0x7f081c06b1b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 --2- 172.21.15.7:0/4238075411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c0c64e0 0x7f081c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 -- 172.21.15.7:0/4238075411 >> 172.21.15.7:0/4238075411 conn(0x7f081c072040 msgr2=0x7f081c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 -- 172.21.15.7:0/4238075411 shutdown_connections 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.256+0000 7f08233ff700 1 -- 172.21.15.7:0/4238075411 wait complete. 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.258+0000 7f08233ff700 1 Processor -- start 2023-06-09T11:11:08.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.258+0000 7f08233ff700 1 -- start start 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c0c64e0 0x7f081c0c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c1537a0 0x7f081c14e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c1f2c20 con 0x7f081c06b740 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c7340 con 0x7f081c0c64e0 2023-06-09T11:11:08.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f08233ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081c0c7700 con 0x7f081c1537a0 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49140/0 (socket says 172.21.15.7:49140) 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 -- 172.21.15.7:0/2322338286 learned_addr learned my addr 172.21.15.7:0/2322338286 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082099a700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c0c64e0 0x7f081c0c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082199c700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c1537a0 0x7f081c14e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 -- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c1537a0 msgr2=0x7f081c14e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c1537a0 0x7f081c14e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 -- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c0c64e0 msgr2=0x7f081c0c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c0c64e0 0x7f081c0c6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 -- 172.21.15.7:0/2322338286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818009550 con 0x7f081c06b740 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.259+0000 7f082119b700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0818009c40 tx=0x7f0818004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.260+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08180036a0 con 0x7f081c06b740 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.260+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081801e070 con 0x7f081c06b740 2023-06-09T11:11:08.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.260+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818003a40 con 0x7f081c06b740 2023-06-09T11:11:08.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.260+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081c0c46d0 con 0x7f081c06b740 2023-06-09T11:11:08.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.260+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f081c15ae20 con 0x7f081c06b740 2023-06-09T11:11:08.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.261+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081c0c49b0 con 0x7f081c06b740 2023-06-09T11:11:08.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.262+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0818003ba0 con 0x7f081c06b740 2023-06-09T11:11:08.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.263+0000 7f0811ffb700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 0x7f080006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.263+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0818020070 con 0x7f081c06b740 2023-06-09T11:11:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.263+0000 7f082099a700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 0x7f080006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.264+0000 7f082099a700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 0x7f080006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f081c156fb0 tx=0x7f080c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.267+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0818055ae0 con 0x7f081c06b740 2023-06-09T11:11:08.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.472+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f081c0c79e0 con 0x7f081c06b740 2023-06-09T11:11:08.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.472+0000 7f0811ffb700 1 -- 172.21.15.7:0/2322338286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0818059100 con 0x7f081c06b740 2023-06-09T11:11:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:11:08.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 msgr2=0x7f080006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 0x7f080006d190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f081c156fb0 tx=0x7f080c008040 comp rx=0 tx=0).stop 2023-06-09T11:11:08.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 msgr2=0x7f081c156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0818009c40 tx=0x7f0818004750 comp rx=0 tx=0).stop 2023-06-09T11:11:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 shutdown_connections 2023-06-09T11:11:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f080006ace0 0x7f080006d190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f081c1537a0 0x7f081c14e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f081c06b740 0x7f081c156a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 --2- 172.21.15.7:0/2322338286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f081c0c64e0 0x7f081c0c6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.474+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 >> 172.21.15.7:0/2322338286 conn(0x7f081c072040 msgr2=0x7f081c0ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:08.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.475+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 shutdown_connections 2023-06-09T11:11:08.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.475+0000 7f08233ff700 1 -- 172.21.15.7:0/2322338286 wait complete. 2023-06-09T11:11:08.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:11:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.695+0000 7fe312dff700 1 Processor -- start 2023-06-09T11:11:08.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.695+0000 7fe312dff700 1 -- start start 2023-06-09T11:11:08.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.696+0000 7fe312dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe30c15b9c0 0x7fe30c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.696+0000 7fe312dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0c6220 con 0x7fe30c1537a0 2023-06-09T11:11:08.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.696+0000 7fe312dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0c4480 con 0x7fe30c0c64e0 2023-06-09T11:11:08.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.696+0000 7fe312dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c0c4f70 con 0x7fe30c15b9c0 2023-06-09T11:11:08.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe310b9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe30c1537a0 0x7fe30c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52828/0 (socket says 172.21.15.7:52828) 2023-06-09T11:11:08.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe310b9b700 1 -- 172.21.15.7:0/565943662 learned_addr learned my addr 172.21.15.7:0/565943662 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:08.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe31139c700 1 --2- 172.21.15.7:0/565943662 >> v2:172.21.15.7:3300/0 conn(0x7fe30c15b9c0 0x7fe30c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe31139c700 1 -- 172.21.15.7:0/565943662 >> v1:172.21.15.117:6789/0 conn(0x7fe30c0c64e0 legacy=0x7fe30c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe31139c700 1 -- 172.21.15.7:0/565943662 >> v1:172.21.15.7:6789/0 conn(0x7fe30c1537a0 legacy=0x7fe30c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe31139c700 1 -- 172.21.15.7:0/565943662 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c15ae20 con 0x7fe30c15b9c0 2023-06-09T11:11:08.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.697+0000 7fe31139c700 1 --2- 172.21.15.7:0/565943662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c15b9c0 0x7fe30c06b1b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe304009d90 tx=0x7fe30400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0497091ea157d7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe30b7fe700 1 -- 172.21.15.7:0/565943662 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400e070 con 0x7fe30c15b9c0 2023-06-09T11:11:08.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe30b7fe700 1 -- 172.21.15.7:0/565943662 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30400bbd0 con 0x7fe30c15b9c0 2023-06-09T11:11:08.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe30b7fe700 1 -- 172.21.15.7:0/565943662 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304012430 con 0x7fe30c15b9c0 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 -- 172.21.15.7:0/565943662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c15b9c0 msgr2=0x7fe30c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 --2- 172.21.15.7:0/565943662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c15b9c0 0x7fe30c06b1b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe304009d90 tx=0x7fe30400b330 comp rx=0 tx=0).stop 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 -- 172.21.15.7:0/565943662 shutdown_connections 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 --2- 172.21.15.7:0/565943662 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c15b9c0 0x7fe30c06b1b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 -- 172.21.15.7:0/565943662 >> 172.21.15.7:0/565943662 conn(0x7fe30c072040 msgr2=0x7fe30c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:08.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 -- 172.21.15.7:0/565943662 shutdown_connections 2023-06-09T11:11:08.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.698+0000 7fe312dff700 1 -- 172.21.15.7:0/565943662 wait complete. 2023-06-09T11:11:08.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.700+0000 7fe312dff700 1 Processor -- start 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.700+0000 7fe312dff700 1 -- start start 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30c0c64e0 0x7fe30c148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe30c15b9c0 0x7fe30c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c68e0 con 0x7fe30c1537a0 2023-06-09T11:11:08.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c6e60 con 0x7fe30c15b9c0 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe312dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c1490d0 con 0x7fe30c0c64e0 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe30bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe30bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49162/0 (socket says 172.21.15.7:49162) 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.701+0000 7fe30bfff700 1 -- 172.21.15.7:0/3298781956 learned_addr learned my addr 172.21.15.7:0/3298781956 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe310b9b700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30c0c64e0 0x7fe30c148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe31139c700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe30c15b9c0 0x7fe30c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 -- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30c0c64e0 msgr2=0x7fe30c148b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30c0c64e0 0x7fe30c148b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 -- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe30c15b9c0 msgr2=0x7fe30c0c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe30c15b9c0 0x7fe30c0c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 -- 172.21.15.7:0/3298781956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe304009a70 con 0x7fe30c1537a0 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe30bfff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c540 tx=0x7fe2f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80052d0 con 0x7fe30c1537a0 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8005430 con 0x7fe30c1537a0 2023-06-09T11:11:08.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c0c49b0 con 0x7fe30c1537a0 2023-06-09T11:11:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.702+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80038c0 con 0x7fe30c1537a0 2023-06-09T11:11:08.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.703+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c0c46d0 con 0x7fe30c1537a0 2023-06-09T11:11:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.704+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c0c4c90 con 0x7fe30c1537a0 2023-06-09T11:11:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.705+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2f8003a20 con 0x7fe30c1537a0 2023-06-09T11:11:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.706+0000 7fe3097fa700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 0x7fe2f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.706+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe2f8088830 con 0x7fe30c1537a0 2023-06-09T11:11:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.706+0000 7fe310b9b700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 0x7fe2f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.707+0000 7fe310b9b700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 0x7fe2f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe300005910 tx=0x7fe300009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:08.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.710+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe2f80538d0 con 0x7fe30c1537a0 2023-06-09T11:11:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.918+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe30c0c7530 con 0x7fe30c1537a0 2023-06-09T11:11:08.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.919+0000 7fe3097fa700 1 -- 172.21.15.7:0/3298781956 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe30c0c7530 con 0x7fe30c1537a0 2023-06-09T11:11:08.925 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:11:08.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 msgr2=0x7fe2f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 0x7fe2f006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe300005910 tx=0x7fe300009040 comp rx=0 tx=0).stop 2023-06-09T11:11:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 msgr2=0x7fe30c0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c540 tx=0x7fe2f800c850 comp rx=0 tx=0).stop 2023-06-09T11:11:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 shutdown_connections 2023-06-09T11:11:08.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe30c15b9c0 0x7fe30c0c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe2f006ace0 0x7fe2f006d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe30c0c64e0 0x7fe30c148b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 --2- 172.21.15.7:0/3298781956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c1537a0 0x7fe30c0ce7d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.921+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 >> 172.21.15.7:0/3298781956 conn(0x7fe30c072040 msgr2=0x7fe30c15b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.922+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 shutdown_connections 2023-06-09T11:11:08.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:08.922+0000 7fe312dff700 1 -- 172.21.15.7:0/3298781956 wait complete. 2023-06-09T11:11:08.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:11:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.142+0000 7f2e2adff700 1 Processor -- start 2023-06-09T11:11:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.142+0000 7f2e2adff700 1 -- start start 2023-06-09T11:11:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.143+0000 7f2e2adff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 0x7f2e2406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.143+0000 7f2e2adff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e240c64e0 0x7f2e240cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.143+0000 7f2e2adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.143+0000 7f2e2adff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240c4f70 con 0x7f2e240c64e0 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e2adff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e240c5c20 con 0x7f2e241537a0 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e2adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240c6220 con 0x7f2e2406b740 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e240c64e0 0x7f2e240cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2e240c64e0 0x7f2e240cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49176/0 (socket says 172.21.15.7:49176) 2023-06-09T11:11:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 -- 172.21.15.7:0/3903905288 learned_addr learned my addr 172.21.15.7:0/3903905288 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 -- 172.21.15.7:0/3903905288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 msgr2=0x7f2e240cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e2939c700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e28b9b700 1 --2- 172.21.15.7:0/3903905288 >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 0x7f2e2406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 -- 172.21.15.7:0/3903905288 >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 msgr2=0x7f2e2406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 --2- 172.21.15.7:0/3903905288 >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 0x7f2e2406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 -- 172.21.15.7:0/3903905288 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240c4480 con 0x7f2e240c64e0 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e2939c700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e28b9b700 1 --2- 172.21.15.7:0/3903905288 >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 0x7f2e2406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.144+0000 7f2e23fff700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e240c64e0 0x7f2e240cf590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e240cd130 tx=0x7f2e10009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bccf38129da6893 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e237fe700 1 -- 172.21.15.7:0/3903905288 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000b870 con 0x7f2e240c64e0 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e237fe700 1 -- 172.21.15.7:0/3903905288 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1000b9d0 con 0x7f2e240c64e0 2023-06-09T11:11:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e237fe700 1 -- 172.21.15.7:0/3903905288 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1000bd60 con 0x7f2e240c64e0 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 -- 172.21.15.7:0/3903905288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e240c64e0 msgr2=0x7f2e240cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e240c64e0 0x7f2e240cf590 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e240cd130 tx=0x7f2e10009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 -- 172.21.15.7:0/3903905288 shutdown_connections 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3903905288 >> v2:172.21.15.117:3300/0 conn(0x7f2e241537a0 0x7f2e2406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3903905288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e240c64e0 0x7f2e240cf590 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 -- 172.21.15.7:0/3903905288 >> 172.21.15.7:0/3903905288 conn(0x7f2e24072040 msgr2=0x7f2e24072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 -- 172.21.15.7:0/3903905288 shutdown_connections 2023-06-09T11:11:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.145+0000 7f2e2adff700 1 -- 172.21.15.7:0/3903905288 wait complete. 2023-06-09T11:11:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.147+0000 7f2e2adff700 1 Processor -- start 2023-06-09T11:11:09.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.147+0000 7f2e2adff700 1 -- start start 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 0x7f2e241ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241f1b10 con 0x7f2e241537a0 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241e1eb0 con 0x7f2e240c64e0 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e241e2150 con 0x7f2e2406b740 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49178/0 (socket says 172.21.15.7:49178) 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.148+0000 7f2e2939c700 1 -- 172.21.15.7:0/3535779095 learned_addr learned my addr 172.21.15.7:0/3535779095 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e23fff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 0x7f2e241ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e28b9b700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 -- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 msgr2=0x7f2e240c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 -- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 msgr2=0x7f2e241ee5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 0x7f2e241ee5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 -- 172.21.15.7:0/3535779095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10009550 con 0x7f2e241537a0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e23fff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 0x7f2e241ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2939c700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cef0 tx=0x7f2e1c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c00add0 con 0x7f2e241537a0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c015440 con 0x7f2e241537a0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240c46d0 con 0x7f2e241537a0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0157f0 con 0x7f2e241537a0 2023-06-09T11:11:09.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.149+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2415ae20 con 0x7f2e241537a0 2023-06-09T11:11:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.150+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e240c49b0 con 0x7f2e241537a0 2023-06-09T11:11:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.151+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2e1c00d430 con 0x7f2e241537a0 2023-06-09T11:11:09.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.152+0000 7f2e217fa700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.153+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2e1c089a00 con 0x7f2e241537a0 2023-06-09T11:11:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.153+0000 7f2e28b9b700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.153+0000 7f2e28b9b700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 0x7f2e0806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.157+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2e1c054ab0 con 0x7f2e241537a0 2023-06-09T11:11:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.363+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e24074cf0 con 0x7f2e241537a0 2023-06-09T11:11:09.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.364+0000 7f2e217fa700 1 -- 172.21.15.7:0/3535779095 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e1c01a0a0 con 0x7f2e241537a0 2023-06-09T11:11:09.370 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:11:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 msgr2=0x7f2e0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 0x7f2e0806d190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2e18006fd0 tx=0x7f2e18008040 comp rx=0 tx=0).stop 2023-06-09T11:11:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 msgr2=0x7f2e241f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00cef0 tx=0x7f2e1c00a340 comp rx=0 tx=0).stop 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 shutdown_connections 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e241537a0 0x7f2e241f2560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2e0806ace0 0x7f2e0806d190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2e2406b740 0x7f2e240c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 --2- 172.21.15.7:0/3535779095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2e240c64e0 0x7f2e241ee5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 >> 172.21.15.7:0/3535779095 conn(0x7f2e24072040 msgr2=0x7f2e24152b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:09.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 shutdown_connections 2023-06-09T11:11:09.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.366+0000 7f2e2adff700 1 -- 172.21.15.7:0/3535779095 wait complete. 2023-06-09T11:11:09.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.584+0000 7f11bbbff700 1 Processor -- start 2023-06-09T11:11:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.584+0000 7f11bbbff700 1 -- start start 2023-06-09T11:11:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11bbbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f11b4153880 0x7f11b415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11bbbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11b415b4a0 0x7f11b40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11bbbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b415a500 con 0x7f11b40c9f40 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11bbbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b40c5f10 con 0x7f11b415b4a0 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11bbbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b415da00 con 0x7f11b4153880 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11ba19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11b415b4a0 0x7f11b40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11b919a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f11b40c9f40 0x7f11b415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52844/0 (socket says 172.21.15.7:52844) 2023-06-09T11:11:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11ba19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f11b415b4a0 0x7f11b40c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49182/0 (socket says 172.21.15.7:49182) 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.585+0000 7f11b919a700 1 -- 172.21.15.7:0/390155208 learned_addr learned my addr 172.21.15.7:0/390155208 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11b999b700 1 --2- 172.21.15.7:0/390155208 >> v2:172.21.15.117:3300/0 conn(0x7f11b4153880 0x7f11b415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11ba19c700 1 -- 172.21.15.7:0/390155208 >> v2:172.21.15.117:3300/0 conn(0x7f11b4153880 msgr2=0x7f11b415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11ba19c700 1 --2- 172.21.15.7:0/390155208 >> v2:172.21.15.117:3300/0 conn(0x7f11b4153880 0x7f11b415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11ba19c700 1 -- 172.21.15.7:0/390155208 >> v1:172.21.15.7:6789/0 conn(0x7f11b40c9f40 legacy=0x7f11b415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11ba19c700 1 -- 172.21.15.7:0/390155208 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b41596e0 con 0x7f11b415b4a0 2023-06-09T11:11:09.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11ba19c700 1 --2- 172.21.15.7:0/390155208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b40c4e30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b408d340 tx=0x7f11ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b7c4de44ded8ed server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11b8999700 1 -- 172.21.15.7:0/390155208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac00e070 con 0x7f11b415b4a0 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11b8999700 1 -- 172.21.15.7:0/390155208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac00bbd0 con 0x7f11b415b4a0 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.586+0000 7f11b8999700 1 -- 172.21.15.7:0/390155208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac012430 con 0x7f11b415b4a0 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 -- 172.21.15.7:0/390155208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 msgr2=0x7f11b40c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 --2- 172.21.15.7:0/390155208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b40c4e30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11b408d340 tx=0x7f11ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:09.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 -- 172.21.15.7:0/390155208 shutdown_connections 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 --2- 172.21.15.7:0/390155208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b40c4e30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 --2- 172.21.15.7:0/390155208 >> v2:172.21.15.117:3300/0 conn(0x7f11b4153880 0x7f11b415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 -- 172.21.15.7:0/390155208 >> 172.21.15.7:0/390155208 conn(0x7f11b40737f0 msgr2=0x7f11b40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 -- 172.21.15.7:0/390155208 shutdown_connections 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.587+0000 7f11bbbff700 1 -- 172.21.15.7:0/390155208 wait complete. 2023-06-09T11:11:09.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.589+0000 7f11bbbff700 1 Processor -- start 2023-06-09T11:11:09.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.589+0000 7f11bbbff700 1 -- start start 2023-06-09T11:11:09.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11b40c9f40 0x7f11b41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b4153880 0x7f11b40cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c9c20 con 0x7f11b415b4a0 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40cd3a0 con 0x7f11b4153880 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11bbbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40cc0e0 con 0x7f11b40c9f40 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49198/0 (socket says 172.21.15.7:49198) 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 -- 172.21.15.7:0/816246175 learned_addr learned my addr 172.21.15.7:0/816246175 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11b999b700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11b40c9f40 0x7f11b41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11b919a700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b4153880 0x7f11b40cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 -- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11b40c9f40 msgr2=0x7f11b41f23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11b40c9f40 0x7f11b41f23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 -- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b4153880 msgr2=0x7f11b40cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b4153880 0x7f11b40cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.590+0000 7f11ba19c700 1 -- 172.21.15.7:0/816246175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac009a70 con 0x7f11b415b4a0 2023-06-09T11:11:09.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11ba19c700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11b406b580 tx=0x7f11ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac012430 con 0x7f11b415b4a0 2023-06-09T11:11:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac020070 con 0x7f11b415b4a0 2023-06-09T11:11:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0098f0 con 0x7f11b415b4a0 2023-06-09T11:11:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b4159c60 con 0x7f11b415b4a0 2023-06-09T11:11:09.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.591+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b4159980 con 0x7f11b415b4a0 2023-06-09T11:11:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.592+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4159f40 con 0x7f11b415b4a0 2023-06-09T11:11:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.593+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f11ac004040 con 0x7f11b415b4a0 2023-06-09T11:11:09.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.595+0000 7f11aa7fc700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 0x7f119806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.595+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f11ac022070 con 0x7f11b415b4a0 2023-06-09T11:11:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.595+0000 7f11b999b700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 0x7f119806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:09.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.596+0000 7f11b999b700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 0x7f119806d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006b20 tx=0x7f11b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:09.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.599+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f11ac056230 con 0x7f11b415b4a0 2023-06-09T11:11:09.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:09.775 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:09.791 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:11:09.787+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:11:09.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.862+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11b4074cf0 con 0x7f11b415b4a0 2023-06-09T11:11:09.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.863+0000 7f11aa7fc700 1 -- 172.21.15.7:0/816246175 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f11ac059850 con 0x7f11b415b4a0 2023-06-09T11:11:09.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:09.870 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 msgr2=0x7f119806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 0x7f119806d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006b20 tx=0x7f11b0008040 comp rx=0 tx=0).stop 2023-06-09T11:11:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 msgr2=0x7f11b414b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:09.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11b406b580 tx=0x7f11ac004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 shutdown_connections 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11b415b4a0 0x7f11b414b380 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f119806ace0 0x7f119806d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f11b40c9f40 0x7f11b41f23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.865+0000 7f11bbbff700 1 --2- 172.21.15.7:0/816246175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f11b4153880 0x7f11b40cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.866+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 >> 172.21.15.7:0/816246175 conn(0x7f11b40737f0 msgr2=0x7f11b415b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:09.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.866+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 shutdown_connections 2023-06-09T11:11:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:09.866+0000 7f11bbbff700 1 -- 172.21.15.7:0/816246175 wait complete. 2023-06-09T11:11:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:09.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:10.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.123+0000 7f576c9ff700 1 Processor -- start 2023-06-09T11:11:10.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.123+0000 7f576c9ff700 1 -- start start 2023-06-09T11:11:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f576c9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f576815b4a0 0x7f576815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f576c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57680c9f40 0x7f57680c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f576c9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576815a500 con 0x7f5768153880 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f576c9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57680c5f10 con 0x7f576815b4a0 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f576c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576815da00 con 0x7f57680c9f40 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.124+0000 7f5765d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f576815b4a0 0x7f576815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f576815b4a0 0x7f576815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49212/0 (socket says 172.21.15.7:49212) 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 -- 172.21.15.7:0/2601118508 learned_addr learned my addr 172.21.15.7:0/2601118508 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5766d9d700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57680c9f40 0x7f57680c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 -- 172.21.15.7:0/2601118508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57680c9f40 msgr2=0x7f57680c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57680c9f40 0x7f57680c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 -- 172.21.15.7:0/2601118508 >> v1:172.21.15.117:6789/0 conn(0x7f5768153880 legacy=0x7f576815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 -- 172.21.15.7:0/2601118508 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57681596e0 con 0x7f576815b4a0 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.125+0000 7f5765d9b700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576815b4a0 0x7f576815d4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f575c0098f0 tx=0x7f575c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78467616d2cdab44 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576559a700 1 -- 172.21.15.7:0/2601118508 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00b8a0 con 0x7f576815b4a0 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576559a700 1 -- 172.21.15.7:0/2601118508 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00ba00 con 0x7f576815b4a0 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576559a700 1 -- 172.21.15.7:0/2601118508 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00bd90 con 0x7f576815b4a0 2023-06-09T11:11:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 -- 172.21.15.7:0/2601118508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576815b4a0 msgr2=0x7f576815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576815b4a0 0x7f576815d4c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f575c0098f0 tx=0x7f575c009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 -- 172.21.15.7:0/2601118508 shutdown_connections 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57680c9f40 0x7f57680c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 --2- 172.21.15.7:0/2601118508 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f576815b4a0 0x7f576815d4c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 -- 172.21.15.7:0/2601118508 >> 172.21.15.7:0/2601118508 conn(0x7f57680737f0 msgr2=0x7f57680720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 -- 172.21.15.7:0/2601118508 shutdown_connections 2023-06-09T11:11:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.126+0000 7f576c9ff700 1 -- 172.21.15.7:0/2601118508 wait complete. 2023-06-09T11:11:10.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.128+0000 7f576c9ff700 1 Processor -- start 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.128+0000 7f576c9ff700 1 -- start start 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5768153880 0x7f57680cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576815b4a0 0x7f57680caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5768145fa0 con 0x7f57680c9f40 2023-06-09T11:11:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681ef9a0 con 0x7f576815b4a0 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57681efe30 con 0x7f5768153880 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49216/0 (socket says 172.21.15.7:49216) 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f576659c700 1 -- 172.21.15.7:0/3803563306 learned_addr learned my addr 172.21.15.7:0/3803563306 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.129+0000 7f5765d9b700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5768153880 0x7f57680cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f5766d9d700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576815b4a0 0x7f57680caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 -- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5768153880 msgr2=0x7f57680cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5768153880 0x7f57680cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 -- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576815b4a0 msgr2=0x7f57680caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576815b4a0 0x7f57680caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 -- 172.21.15.7:0/3803563306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009550 con 0x7f57680c9f40 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576659c700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f575800c510 tx=0x7f575800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580052d0 con 0x7f57680c9f40 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758005430 con 0x7f57680c9f40 2023-06-09T11:11:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57580038c0 con 0x7f57680c9f40 2023-06-09T11:11:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5768159c60 con 0x7f57680c9f40 2023-06-09T11:11:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.130+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5768159980 con 0x7f57680c9f40 2023-06-09T11:11:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.132+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768159f40 con 0x7f57680c9f40 2023-06-09T11:11:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.132+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5758003a20 con 0x7f57680c9f40 2023-06-09T11:11:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.133+0000 7f5756ffd700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 0x7f574806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.133+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5758088650 con 0x7f57680c9f40 2023-06-09T11:11:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.134+0000 7f5765d9b700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 0x7f574806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.134+0000 7f5765d9b700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 0x7f574806d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f575c009c40 tx=0x7f575c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.138+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5758053790 con 0x7f57680c9f40 2023-06-09T11:11:10.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.347+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f576815a500 con 0x7f57680c9f40 2023-06-09T11:11:10.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.348+0000 7f5756ffd700 1 -- 172.21.15.7:0/3803563306 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f576815a500 con 0x7f57680c9f40 2023-06-09T11:11:10.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:10.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:10.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 msgr2=0x7f574806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 0x7f574806d1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f575c009c40 tx=0x7f575c008030 comp rx=0 tx=0).stop 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 msgr2=0x7f5768157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f575800c510 tx=0x7f575800c820 comp rx=0 tx=0).stop 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 shutdown_connections 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f576815b4a0 0x7f57680caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57680c9f40 0x7f5768157b60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f574806ad10 0x7f574806d1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 --2- 172.21.15.7:0/3803563306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5768153880 0x7f57680cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 >> 172.21.15.7:0/3803563306 conn(0x7f57680737f0 msgr2=0x7f57680720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 shutdown_connections 2023-06-09T11:11:10.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.351+0000 7f576c9ff700 1 -- 172.21.15.7:0/3803563306 wait complete. 2023-06-09T11:11:10.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:10.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.569+0000 7f99345ff700 1 Processor -- start 2023-06-09T11:11:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.569+0000 7f99345ff700 1 -- start start 2023-06-09T11:11:10.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f99345ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f992c153880 0x7f992c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f99345ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f992c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f99345ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c15a500 con 0x7f992c0c9f40 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f99345ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0c5f10 con 0x7f992c153880 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f99345ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15da00 con 0x7f992c15b4a0 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f9931b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f992c0c9f40 0x7f992c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52846/0 (socket says 172.21.15.7:52846) 2023-06-09T11:11:10.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.570+0000 7f9931b9a700 1 -- 172.21.15.7:0/2685090753 learned_addr learned my addr 172.21.15.7:0/2685090753 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f993239b700 1 --2- 172.21.15.7:0/2685090753 >> v2:172.21.15.117:3300/0 conn(0x7f992c153880 0x7f992c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280536890 0 0) 0x7f991c001110 con 0x7f992c0c9f40 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9932b9c700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f992c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99180035b0 con 0x7f992c0c9f40 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2569426302 0 0) 0x7f99180035b0 con 0x7f992c0c9f40 2023-06-09T11:11:10.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 msgr2=0x7f992c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f992c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 >> v2:172.21.15.117:3300/0 conn(0x7f992c153880 msgr2=0x7f992c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 --2- 172.21.15.7:0/2685090753 >> v2:172.21.15.117:3300/0 conn(0x7f992c153880 0x7f992c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c1596e0 con 0x7f992c0c9f40 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f991c001800 con 0x7f992c0c9f40 2023-06-09T11:11:10.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 >> v1:172.21.15.7:6789/0 conn(0x7f992c0c9f40 legacy=0x7f992c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9918003a00 0x7f9918003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9918004430 0x7f99180048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f9918005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0c5f10 con 0x7f9918004430 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15da00 con 0x7f9918003a00 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991800ba10 con 0x7f992c15b4a0 2023-06-09T11:11:10.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9931b9a700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9918004430 0x7f99180048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f993239b700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9918003a00 0x7f9918003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.571+0000 7f9932b9c700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f9918005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 -- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 msgr2=0x7f9918005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c15b4a0 0x7f9918005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 -- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9918003a00 msgr2=0x7f9918003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9918003a00 0x7f9918003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 -- 172.21.15.7:0/2685090753 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180035b0 con 0x7f9918004430 2023-06-09T11:11:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f993239b700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9918003a00 0x7f9918003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931b9a700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9918004430 0x7f99180048f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c320 tx=0x7f9918005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fefb48ee4ec877 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00ce20 con 0x7f9918004430 2023-06-09T11:11:10.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c005960 con 0x7f9918004430 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.572+0000 7f9931399700 1 -- 172.21.15.7:0/2685090753 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c005d10 con 0x7f9918004430 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 -- 172.21.15.7:0/2685090753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9918004430 msgr2=0x7f99180048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 --2- 172.21.15.7:0/2685090753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9918004430 0x7f99180048f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c320 tx=0x7f9918005880 comp rx=0 tx=0).stop 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f9932b9c700 1 -- 172.21.15.7:0/2685090753 reap_dead start 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 -- 172.21.15.7:0/2685090753 shutdown_connections 2023-06-09T11:11:10.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 -- 172.21.15.7:0/2685090753 >> 172.21.15.7:0/2685090753 conn(0x7f992c0737f0 msgr2=0x7f992c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 -- 172.21.15.7:0/2685090753 shutdown_connections 2023-06-09T11:11:10.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.573+0000 7f99345ff700 1 -- 172.21.15.7:0/2685090753 wait complete. 2023-06-09T11:11:10.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.575+0000 7f99345ff700 1 Processor -- start 2023-06-09T11:11:10.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.576+0000 7f99345ff700 1 -- start start 2023-06-09T11:11:10.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9918004430 0x7f992c157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c4e00 0x7f992c14cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c1520c0 con 0x7f992c0cc5e0 2023-06-09T11:11:10.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c1584e0 con 0x7f992c0c4e00 2023-06-09T11:11:10.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f99345ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0c59f0 con 0x7f9918004430 2023-06-09T11:11:10.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f9931b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f9931b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49244/0 (socket says 172.21.15.7:49244) 2023-06-09T11:11:10.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f9931b9a700 1 -- 172.21.15.7:0/4267943712 learned_addr learned my addr 172.21.15.7:0/4267943712 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:10.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f993239b700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9918004430 0x7f992c157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.577+0000 7f9932b9c700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c4e00 0x7f992c14cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 -- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9918004430 msgr2=0x7f992c157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9918004430 0x7f992c157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 -- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c4e00 msgr2=0x7f992c14cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c4e00 0x7f992c14cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 -- 172.21.15.7:0/4267943712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c159c60 con 0x7f992c0cc5e0 2023-06-09T11:11:10.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9931b9a700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9918005770 tx=0x7f991c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c004870 con 0x7f992c0cc5e0 2023-06-09T11:11:10.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c0049d0 con 0x7f992c0cc5e0 2023-06-09T11:11:10.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c159980 con 0x7f992c0cc5e0 2023-06-09T11:11:10.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c004d80 con 0x7f992c0cc5e0 2023-06-09T11:11:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.578+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c159f40 con 0x7f992c0cc5e0 2023-06-09T11:11:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.579+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c1ef8c0 con 0x7f992c0cc5e0 2023-06-09T11:11:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.580+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f991c005040 con 0x7f992c0cc5e0 2023-06-09T11:11:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.581+0000 7f9922ffd700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 0x7f991006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:10.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.581+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f991c089e40 con 0x7f992c0cc5e0 2023-06-09T11:11:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.582+0000 7f993239b700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 0x7f991006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:10.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.582+0000 7f993239b700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 0x7f991006d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9928009d80 tx=0x7f9928009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:10.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.586+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f991c055000 con 0x7f992c0cc5e0 2023-06-09T11:11:10.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.833+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f992c15a500 con 0x7f992c0cc5e0 2023-06-09T11:11:10.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.834+0000 7f9922ffd700 1 -- 172.21.15.7:0/4267943712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f992c15a500 con 0x7f992c0cc5e0 2023-06-09T11:11:10.840 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:10.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:10.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.836+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 msgr2=0x7f991006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.836+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 0x7f991006d190 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9928009d80 tx=0x7f9928009210 comp rx=0 tx=0).stop 2023-06-09T11:11:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.836+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 msgr2=0x7f992c1efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.836+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9918005770 tx=0x7f991c000fe0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.836+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 shutdown_connections 2023-06-09T11:11:10.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c4e00 0x7f992c14cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f991006ace0 0x7f991006d190 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9918004430 0x7f992c157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 --2- 172.21.15.7:0/4267943712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c0cc5e0 0x7f992c1efa70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:10.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 >> 172.21.15.7:0/4267943712 conn(0x7f992c0737f0 msgr2=0x7f992c14ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 shutdown_connections 2023-06-09T11:11:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:10.837+0000 7f99345ff700 1 -- 172.21.15.7:0/4267943712 wait complete. 2023-06-09T11:11:10.845 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:11.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-06-09T11:10:50.179197+0000', 'modified': '2023-06-09T11:11:06.332140+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4291, 'mds_1': 4288, 'mds_2': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 2, 'incarnation': 11, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.117:6833/1113843600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6832', 'nonce': 1113843600}, {'type': 'v1', 'addr': '172.21.15.117:6833', 'nonce': 1113843600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4288': {'gid': 4288, 'name': 'd', 'rank': 1, 'incarnation': 9, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.117:6835/4051500902', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6834', 'nonce': 4051500902}, {'type': 'v1', 'addr': '172.21.15.117:6835', 'nonce': 4051500902}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_4291': {'gid': 4291, 'name': 'f', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.117:6837/2733808306', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.117:6836', 'nonce': 2733808306}, {'type': 'v1', 'addr': '172.21.15.117:6837', 'nonce': 2733808306}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-09T11:11:11.464 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-06-09T11:11:11.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-06-09T11:11:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.675+0000 7f13ba7ff700 1 Processor -- start 2023-06-09T11:11:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.675+0000 7f13ba7ff700 1 -- start start 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b4153880 0x7f13b415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13b40c9f40 0x7f13b415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 0x7f13b40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b40c5f10 con 0x7f13b4153880 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b415a500 con 0x7f13b40c9f40 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13ba7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b415da00 con 0x7f13b415b2d0 2023-06-09T11:11:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.676+0000 7f13b3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b4153880 0x7f13b415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13b4153880 0x7f13b415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49256/0 (socket says 172.21.15.7:49256) 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 -- 172.21.15.7:0/1294303915 learned_addr learned my addr 172.21.15.7:0/1294303915 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1294303915 >> v2:172.21.15.117:3300/0 conn(0x7f13b40c9f40 0x7f13b415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 0x7f13b40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 -- 172.21.15.7:0/1294303915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 msgr2=0x7f13b40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 0x7f13b40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 -- 172.21.15.7:0/1294303915 >> v2:172.21.15.117:3300/0 conn(0x7f13b40c9f40 msgr2=0x7f13b415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1294303915 >> v2:172.21.15.117:3300/0 conn(0x7f13b40c9f40 0x7f13b415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 -- 172.21.15.7:0/1294303915 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b41596e0 con 0x7f13b4153880 2023-06-09T11:11:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 0x7f13b40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4153880 0x7f13b415d4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13b40cce30 tx=0x7f13a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116807ddc41933ae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.677+0000 7f13b2ffd700 1 -- 172.21.15.7:0/1294303915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400b870 con 0x7f13b4153880 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13b2ffd700 1 -- 172.21.15.7:0/1294303915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a400b9d0 con 0x7f13b4153880 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13b2ffd700 1 -- 172.21.15.7:0/1294303915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400bd60 con 0x7f13b4153880 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 -- 172.21.15.7:0/1294303915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4153880 msgr2=0x7f13b415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4153880 0x7f13b415d4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f13b40cce30 tx=0x7f13a4009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 -- 172.21.15.7:0/1294303915 shutdown_connections 2023-06-09T11:11:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b415b2d0 0x7f13b40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/1294303915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b4153880 0x7f13b415d4c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/1294303915 >> v2:172.21.15.117:3300/0 conn(0x7f13b40c9f40 0x7f13b415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 -- 172.21.15.7:0/1294303915 >> 172.21.15.7:0/1294303915 conn(0x7f13b40737f0 msgr2=0x7f13b40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 -- 172.21.15.7:0/1294303915 shutdown_connections 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.678+0000 7f13ba7ff700 1 -- 172.21.15.7:0/1294303915 wait complete. 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.680+0000 7f13ba7ff700 1 Processor -- start 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.680+0000 7f13ba7ff700 1 -- start start 2023-06-09T11:11:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b4153880 0x7f13b406ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b415b2d0 0x7f13b414cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41531c0 con 0x7f13b40c9f40 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41f0400 con 0x7f13b415b2d0 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13ba7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40cabb0 con 0x7f13b4153880 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49258/0 (socket says 172.21.15.7:49258) 2023-06-09T11:11:11.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.681+0000 7f13b3fff700 1 -- 172.21.15.7:0/907592407 learned_addr learned my addr 172.21.15.7:0/907592407 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 -- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b4153880 msgr2=0x7f13b406ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b4153880 0x7f13b406ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 -- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b415b2d0 msgr2=0x7f13b414cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b415b2d0 0x7f13b414cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 -- 172.21.15.7:0/907592407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009550 con 0x7f13b40c9f40 2023-06-09T11:11:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b3fff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f13a4000c00 tx=0x7f13a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b0ff9700 1 -- 172.21.15.7:0/907592407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004540 con 0x7f13b40c9f40 2023-06-09T11:11:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b0ff9700 1 -- 172.21.15.7:0/907592407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a40036a0 con 0x7f13b40c9f40 2023-06-09T11:11:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b4159980 con 0x7f13b40c9f40 2023-06-09T11:11:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13b0ff9700 1 -- 172.21.15.7:0/907592407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4003a50 con 0x7f13b40c9f40 2023-06-09T11:11:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.682+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b415b730 con 0x7f13b40c9f40 2023-06-09T11:11:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.685+0000 7f13b0ff9700 1 -- 172.21.15.7:0/907592407 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13a4021070 con 0x7f13b40c9f40 2023-06-09T11:11:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.686+0000 7f13b0ff9700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 0x7f13a006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.686+0000 7f13b0ff9700 1 -- 172.21.15.7:0/907592407 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13a408afd0 con 0x7f13b40c9f40 2023-06-09T11:11:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.686+0000 7f13b37fe700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 0x7f13a006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.687+0000 7f13b37fe700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 0x7f13a006d1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13b40c9ce0 tx=0x7f13a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.688+0000 7f13ce0f8b80 1 Processor -- start 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.688+0000 7f13ce0f8b80 1 -- start start 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd304580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd2fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd209010 con 0x55edcd300840 2023-06-09T11:11:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd25b850 con 0x55edcd3041b0 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edcd254ce0 con 0x55edcd2fe160 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49272/0 (socket says 172.21.15.7:49272) 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.689+0000 7f13b37fe700 1 -- 172.21.15.7:0/1865304768 learned_addr learned my addr 172.21.15.7:0/1865304768 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd304580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd2fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 -- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 msgr2=0x55edcd2fe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd2fe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 -- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 msgr2=0x55edcd304580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd304580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 -- 172.21.15.7:0/1865304768 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edcd0d65e0 con 0x55edcd300840 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd304580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd2fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f13a8015830 tx=0x7f13a8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0035d9f3a1f3a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f1399ffb700 1 -- 172.21.15.7:0/1865304768 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a80136d0 con 0x55edcd300840 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f1399ffb700 1 -- 172.21.15.7:0/1865304768 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8013830 con 0x55edcd300840 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.690+0000 7f1399ffb700 1 -- 172.21.15.7:0/1865304768 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8013ba0 con 0x55edcd300840 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1865304768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 msgr2=0x55edcd3030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f13a8015830 tx=0x7f13a8009f90 comp rx=0 tx=0).stop 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1865304768 shutdown_connections 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd2fe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd304580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1865304768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd3030c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1865304768 >> 172.21.15.7:0/1865304768 conn(0x55edcd2f8f30 msgr2=0x55edcd301840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1865304768 shutdown_connections 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.691+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1865304768 wait complete. 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.692+0000 7f13ce0f8b80 1 -- start start 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.692+0000 7f13ce0f8b80 1 Processor -- start 2023-06-09T11:11:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd300840 0x55edcd357260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd2fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd1f6890 con 0x55edcd2fe160 2023-06-09T11:11:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd1f6530 con 0x55edcd3041b0 2023-06-09T11:11:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edcd0d88d0 con 0x55edcd300840 2023-06-09T11:11:11.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13ce0f8b80 10 client.0 _ll_register_callbacks cb 0x55edcd214730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49288/0 (socket says 172.21.15.7:49288) 2023-06-09T11:11:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 -- 172.21.15.7:0/1991435572 learned_addr learned my addr 172.21.15.7:0/1991435572 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd300840 0x55edcd357260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd2fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 -- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd300840 msgr2=0x55edcd357260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd300840 0x55edcd357260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 -- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 msgr2=0x55edcd2fe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd2fe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.693+0000 7f13b3fff700 1 -- 172.21.15.7:0/1991435572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a80155a0 con 0x55edcd2fe160 2023-06-09T11:11:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f13b3fff700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13a4061b50 tx=0x7f13a402c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab25ff01ea83a78 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a405d040 con 0x55edcd2fe160 2023-06-09T11:11:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edcd2f66f0 con 0x55edcd2fe160 2023-06-09T11:11:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13a4062080 con 0x55edcd2fe160 2023-06-09T11:11:11.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a4061df0 con 0x55edcd2fe160 2023-06-09T11:11:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f13ce0f8b80 10 client.4695 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55edcd21b140 con 0x55edcd2fe160 2023-06-09T11:11:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4077030 con 0x55edcd2fe160 2023-06-09T11:11:11.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7f13a40711e0 con 0x55edcd2fe160 2023-06-09T11:11:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.694+0000 7f13ce0f8b80 10 client.4695 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13ce0f8b80 10 client.4695 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-09T11:11:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13ce0f8b80 20 client.4695 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd340e00 0x55edcd32b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13ce0f8b80 4 client.4695 mds_command: new command op to 4282 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-09T11:11:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 --> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edcd217050 con 0x55edcd340e00 2023-06-09T11:11:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd340e00 0x55edcd32b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f1392ffd700 10 client.4695 ms_handle_connect on v2:172.21.15.117:6832/1113843600 2023-06-09T11:11:11.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.695+0000 7f13b37fe700 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd340e00 0x55edcd32b460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d18e196e4a1a4ce2 server_cookie=f4929c303851c487 in_seq=0 out_seq=0 2023-06-09T11:11:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.696+0000 7f1392ffd700 1 -- 172.21.15.7:0/1991435572 <== mds.2 v2:172.21.15.117:6832/1113843600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55edcd217050 con 0x55edcd340e00 2023-06-09T11:11:11.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.696+0000 7f1392ffd700 10 client.4695 handle_command_reply: tid=0 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.696+0000 7f13ce0f8b80 1 client.4695 shutdown 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 msgr2=0x55edcd33dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f13a4061b50 tx=0x7f13a402c390 comp rx=0 tx=0).stop 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 shutdown_connections 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd340e00 0x55edcd32b460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd2fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd2fe160 0x55edcd33dec0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/1991435572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd300840 0x55edcd357260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f1392ffd700 0 client.4695 ms_handle_reset on v2:172.21.15.117:6832/1113843600 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 >> 172.21.15.7:0/1991435572 conn(0x55edcd2f8f30 msgr2=0x55edcd303470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 shutdown_connections 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/1991435572 wait complete. 2023-06-09T11:11:11.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.697+0000 7f13ce0f8b80 20 client.4695 trim_cache size 0 max 16384 2023-06-09T11:11:11.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 Processor -- start 2023-06-09T11:11:11.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 -- start start 2023-06-09T11:11:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd330460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd33cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd2ffed0 con 0x55edcd300840 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd305930 con 0x55edcd3041b0 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.716+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edcd3408b0 con 0x55edcd2fe160 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49300/0 (socket says 172.21.15.7:49300) 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 -- 172.21.15.7:0/3698982683 learned_addr learned my addr 172.21.15.7:0/3698982683 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 -- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 msgr2=0x55edcd330460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd330460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 -- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 msgr2=0x55edcd33cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd33cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd33cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 -- 172.21.15.7:0/3698982683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edcd21ae90 con 0x55edcd300840 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd33cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f13b37fe700 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13a8008010 tx=0x7f13a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897641c4bfd8e8d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.717+0000 7f139a7fc700 1 -- 172.21.15.7:0/3698982683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8005da0 con 0x55edcd300840 2023-06-09T11:11:11.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f139a7fc700 1 -- 172.21.15.7:0/3698982683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a80139a0 con 0x55edcd300840 2023-06-09T11:11:11.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f139a7fc700 1 -- 172.21.15.7:0/3698982683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8013d10 con 0x55edcd300840 2023-06-09T11:11:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/3698982683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 msgr2=0x55edcd33c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f13a8008010 tx=0x7f13a8015000 comp rx=0 tx=0).stop 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/3698982683 shutdown_connections 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd3041b0 0x55edcd33cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd330460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/3698982683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd300840 0x55edcd33c9a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/3698982683 >> 172.21.15.7:0/3698982683 conn(0x55edcd2f8f30 msgr2=0x55edcd3017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/3698982683 shutdown_connections 2023-06-09T11:11:11.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.718+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/3698982683 wait complete. 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.719+0000 7f13ce0f8b80 1 -- start start 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.719+0000 7f13ce0f8b80 1 Processor -- start 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd34b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd300840 0x55edcd357220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd357760 con 0x55edcd3041b0 2023-06-09T11:11:11.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edcd340120 con 0x55edcd300840 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55edcd303870 con 0x55edcd2fe160 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13ce0f8b80 10 client.0 _ll_register_callbacks cb 0x7f13a4059900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13b8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13b8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49304/0 (socket says 172.21.15.7:49304) 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13b8d9c700 1 -- 172.21.15.7:0/2012666525 learned_addr learned my addr 172.21.15.7:0/2012666525 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd34b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.720+0000 7f13b37fe700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd300840 0x55edcd357220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 -- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 msgr2=0x55edcd34b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd34b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 -- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd300840 msgr2=0x55edcd357220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd300840 0x55edcd357220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 -- 172.21.15.7:0/2012666525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edcd0d65e0 con 0x55edcd3041b0 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13b8d9c700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13ac008fd0 tx=0x7f13ac011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4d7598db931200 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f1392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:11.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac003cf0 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13ac01bc30 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13a8013830 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13ac003e70 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13ce0f8b80 10 client.4701 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.721+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55edcd06bc10 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a8013830 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7f13ac004030 con 0x55edcd3041b0 2023-06-09T11:11:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 10 client.4701 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 20 client.4701 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 4 client.4701 mds_command: new command op to 4282 tid=0[{"prefix": "status"}] 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 --> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55edcd223610 con 0x55edcd3411d0 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.722+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e60d7c94e6a9be89 server_cookie=34a19acd68eea130 in_seq=0 out_seq=0 2023-06-09T11:11:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f1392ffd700 10 client.4701 ms_handle_connect on v2:172.21.15.117:6832/1113843600 2023-06-09T11:11:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 -- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 msgr2=0x55edcd3415a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 18 2023-06-09T11:11:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 -- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 msgr2=0x55edcd3415a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:11:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:11:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 unknown :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2023-06-09T11:11:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.723+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 unknown :-1 s=BANNER_CONNECTING pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f1392ffd700 10 client.4701 ms_handle_connect on v2:172.21.15.117:6832/1113843600 2023-06-09T11:11:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13b3fff700 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 crc :-1 s=READY pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e60d7c94e6a9be89 server_cookie=34a19acd68eea130 in_seq=0 out_seq=1 2023-06-09T11:11:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f1392ffd700 1 -- 172.21.15.7:0/2012666525 <== mds.2 v2:172.21.15.117:6832/1113843600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55edcd223610 con 0x55edcd3411d0 2023-06-09T11:11:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f1392ffd700 10 client.4701 handle_command_reply: tid=0 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 client.4701 shutdown 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 msgr2=0x55edcd33f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f13ac008fd0 tx=0x7f13ac011970 comp rx=0 tx=0).stop 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 shutdown_connections 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:6832/1113843600,v1:172.21.15.117:6833/1113843600] conn(0x55edcd3411d0 0x55edcd3415a0 crc :-1 s=READY pgs=15 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55edcd3041b0 0x55edcd33f710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55edcd2fe160 0x55edcd34b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f1392ffd700 0 client.4701 ms_handle_reset on v2:172.21.15.117:6832/1113843600 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 --2- 172.21.15.7:0/2012666525 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55edcd300840 0x55edcd357220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 >> 172.21.15.7:0/2012666525 conn(0x55edcd2f8f30 msgr2=0x55edcd304920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 shutdown_connections 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 1 -- 172.21.15.7:0/2012666525 wait complete. 2023-06-09T11:11:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.724+0000 7f13ce0f8b80 20 client.4701 trim_cache size 0 max 16384 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "cb7aac03-4bc4-45fa-8177-804a746951e5", 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 2, 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4282, 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-06-09T11:11:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.565189460999999, 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 12, 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 20, 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 20, 2023-06-09T11:11:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 27.955949661999998 2023-06-09T11:11:11.742 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:11:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 msgr2=0x7f13a006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 0x7f13a006d1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13b40c9ce0 tx=0x7f13a8008040 comp rx=0 tx=0).stop 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 msgr2=0x7f13b40c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f13a4000c00 tx=0x7f13a4004750 comp rx=0 tx=0).stop 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 shutdown_connections 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13b415b2d0 0x7f13b414cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13b40c9f40 0x7f13b40c97a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13a006ad10 0x7f13a006d1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 --2- 172.21.15.7:0/907592407 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13b4153880 0x7f13b406ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 >> 172.21.15.7:0/907592407 conn(0x7f13b40737f0 msgr2=0x7f13b40cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 shutdown_connections 2023-06-09T11:11:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.725+0000 7f13ba7ff700 1 -- 172.21.15.7:0/907592407 wait complete. 2023-06-09T11:11:11.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-06-09T11:11:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.942+0000 7fcbc51ff700 1 Processor -- start 2023-06-09T11:11:11.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.942+0000 7fcbc51ff700 1 -- start start 2023-06-09T11:11:11.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.943+0000 7fcbc51ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbc0153880 0x7fcbc015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.943+0000 7fcbc51ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc015a500 con 0x7fcbc015b4a0 2023-06-09T11:11:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.943+0000 7fcbc51ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc015da00 con 0x7fcbc00c9f40 2023-06-09T11:11:11.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.943+0000 7fcbc51ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc00c5f10 con 0x7fcbc0153880 2023-06-09T11:11:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbc0153880 0x7fcbc015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbf59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcbc015b4a0 0x7fcbc00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52850/0 (socket says 172.21.15.7:52850) 2023-06-09T11:11:11.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbc0153880 0x7fcbc015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49308/0 (socket says 172.21.15.7:49308) 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbf59e700 1 -- 172.21.15.7:0/1023928750 learned_addr learned my addr 172.21.15.7:0/1023928750 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 -- 172.21.15.7:0/1023928750 >> v1:172.21.15.117:6789/0 conn(0x7fcbc00c9f40 legacy=0x7fcbc015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 -- 172.21.15.7:0/1023928750 >> v1:172.21.15.7:6789/0 conn(0x7fcbc015b4a0 legacy=0x7fcbc00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 -- 172.21.15.7:0/1023928750 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc01596e0 con 0x7fcbc0153880 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbdd9b700 1 -- 172.21.15.7:0/1023928750 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530956271 0 0) 0x7fcbb8001490 con 0x7fcbc015b4a0 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.944+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/1023928750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0153880 0x7fcbc015d4c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009840 tx=0x7fcbb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8cba748e192b61 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbbdd9b700 1 -- 172.21.15.7:0/1023928750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0011830 con 0x7fcbc0153880 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbbdd9b700 1 -- 172.21.15.7:0/1023928750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb0011990 con 0x7fcbc0153880 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbbdd9b700 1 -- 172.21.15.7:0/1023928750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0011d20 con 0x7fcbc0153880 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 -- 172.21.15.7:0/1023928750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0153880 msgr2=0x7fcbc015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/1023928750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0153880 0x7fcbc015d4c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0009840 tx=0x7fcbb0009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 -- 172.21.15.7:0/1023928750 shutdown_connections 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/1023928750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc0153880 0x7fcbc015d4c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 -- 172.21.15.7:0/1023928750 >> 172.21.15.7:0/1023928750 conn(0x7fcbc00737f0 msgr2=0x7fcbc00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 -- 172.21.15.7:0/1023928750 shutdown_connections 2023-06-09T11:11:11.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.945+0000 7fcbc51ff700 1 -- 172.21.15.7:0/1023928750 wait complete. 2023-06-09T11:11:11.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.948+0000 7fcbc51ff700 1 Processor -- start 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.948+0000 7fcbc51ff700 1 -- start start 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 0x7fcbc014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbc015b4a0 0x7fcbc00b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc01541c0 con 0x7fcbc00c9f40 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc01ef260 con 0x7fcbc0153880 2023-06-09T11:11:11.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbc51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc01ef540 con 0x7fcbc015b4a0 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49320/0 (socket says 172.21.15.7:49320) 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.949+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3205021444 learned_addr learned my addr 172.21.15.7:0/3205021444 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbc015b4a0 0x7fcbc00b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 0x7fcbc014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbc015b4a0 msgr2=0x7fcbc00b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbc015b4a0 0x7fcbc00b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 msgr2=0x7fcbc014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 0x7fcbc014c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3205021444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00094a0 con 0x7fcbc00c9f40 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 0x7fcbc014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00090b0 tx=0x7fcbb0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcba77fe700 1 -- 172.21.15.7:0/3205021444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001a900 con 0x7fcbc00c9f40 2023-06-09T11:11:11.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcba77fe700 1 -- 172.21.15.7:0/3205021444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb001aa60 con 0x7fcbc00c9f40 2023-06-09T11:11:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc0159c60 con 0x7fcbc00c9f40 2023-06-09T11:11:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.950+0000 7fcba77fe700 1 -- 172.21.15.7:0/3205021444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb001add0 con 0x7fcbc00c9f40 2023-06-09T11:11:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.951+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc0159980 con 0x7fcbc00c9f40 2023-06-09T11:11:11.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.953+0000 7fcba77fe700 1 -- 172.21.15.7:0/3205021444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbb0022070 con 0x7fcbc00c9f40 2023-06-09T11:11:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.953+0000 7fcba77fe700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.954+0000 7fcba77fe700 1 -- 172.21.15.7:0/3205021444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbb008a660 con 0x7fcbc00c9f40 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.954+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.954+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 0x7fcbac06d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00ce890 tx=0x7fcbb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.956+0000 7fcbd8b40b80 1 Processor -- start 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.956+0000 7fcbd8b40b80 1 -- start start 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.956+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e27671a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.956+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27661f010 con 0x55e276716840 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e276671850 con 0x55e27671a1b0 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e27666ace0 con 0x55e276714160 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49332/0 (socket says 172.21.15.7:49332) 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 -- 172.21.15.7:0/1883257012 learned_addr learned my addr 172.21.15.7:0/1883257012 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e27671a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 -- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 msgr2=0x55e2767145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 -- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 msgr2=0x55e27671a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e27671a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 -- 172.21.15.7:0/1883257012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2764ec5e0 con 0x55e276716840 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.957+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4000f60 tx=0x7fcbb4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e1a62829411c62 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcba4ff9700 1 -- 172.21.15.7:0/1883257012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401b070 con 0x55e276716840 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcba4ff9700 1 -- 172.21.15.7:0/1883257012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40054f0 con 0x55e276716840 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcba4ff9700 1 -- 172.21.15.7:0/1883257012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4020620 con 0x55e276716840 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1883257012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 msgr2=0x55e2767190c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4000f60 tx=0x7fcbb4010000 comp rx=0 tx=0).stop 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1883257012 shutdown_connections 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e27671a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1883257012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767190c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1883257012 >> 172.21.15.7:0/1883257012 conn(0x55e27670ef30 msgr2=0x55e276717840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.958+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1883257012 shutdown_connections 2023-06-09T11:11:11.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.959+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1883257012 wait complete. 2023-06-09T11:11:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.959+0000 7fcbd8b40b80 1 -- start start 2023-06-09T11:11:11.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.960+0000 7fcbd8b40b80 1 Processor -- start 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276716840 0x55e276742b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e27671a1b0 0x55e27676d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27660c890 con 0x55e276714160 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27660c530 con 0x55e276716840 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2764ee8d0 con 0x55e27671a1b0 2023-06-09T11:11:11.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbd8b40b80 10 client.0 _ll_register_callbacks cb 0x55e27662a730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.961+0000 7fcbbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49334/0 (socket says 172.21.15.7:49334) 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3757755039 learned_addr learned my addr 172.21.15.7:0/3757755039 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276716840 0x55e276742b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e27671a1b0 0x55e27676d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e27671a1b0 msgr2=0x55e27676d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e27671a1b0 0x55e27676d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276716840 msgr2=0x55e276742b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276716840 0x55e276742b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 -- 172.21.15.7:0/3757755039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40051d0 con 0x55e276714160 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0061280 tx=0x7fcbb005d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba92a13cf6f2647 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcb9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb0058c00 con 0x55e276714160 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbb0057970 con 0x55e276714160 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e27670c6f0 con 0x55e276714160 2023-06-09T11:11:11.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.962+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbb0061320 con 0x55e276714160 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 10 client.4716 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55e276631140 con 0x55e276714160 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb0074030 con 0x55e276714160 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7fcbb0055300 con 0x55e276714160 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 10 client.4716 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 10 client.4716 resolve_mds: resolved name 'd' to daemon mds.d 2023-06-09T11:11:11.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 20 client.4716 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 4 client.4716 mds_command: new command op to 4288 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 --> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e27662d050 con 0x55e276744340 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbbe59c700 1 -- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 msgr2=0x55e276756cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 19 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbbe59c700 1 -- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 msgr2=0x55e276756cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-06-09T11:11:11.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-09T11:11:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:11.963+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-09T11:11:12.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.165+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.166+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d055593be55e89d server_cookie=19a2c6bcb4c08595 in_seq=0 out_seq=0 2023-06-09T11:11:12.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.166+0000 7fcb9dffb700 10 client.4716 ms_handle_connect on v2:172.21.15.117:6834/4051500902 2023-06-09T11:11:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcb9dffb700 1 -- 172.21.15.7:0/3757755039 <== mds.1 v2:172.21.15.117:6834/4051500902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e27662d050 con 0x55e276744340 2023-06-09T11:11:12.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcb9dffb700 10 client.4716 handle_command_reply: tid=0 2023-06-09T11:11:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 client.4716 shutdown 2023-06-09T11:11:12.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 msgr2=0x55e276743cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0061280 tx=0x7fcbb005d550 comp rx=0 tx=0).stop 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 shutdown_connections 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e27671a1b0 0x55e27676d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276714160 0x55e276743cb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276744340 0x55e276756cb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/3757755039 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276716840 0x55e276742b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcb9dffb700 0 client.4716 ms_handle_reset on v2:172.21.15.117:6834/4051500902 2023-06-09T11:11:12.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 >> 172.21.15.7:0/3757755039 conn(0x55e27670ef30 msgr2=0x55e276753fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 shutdown_connections 2023-06-09T11:11:12.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.521+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/3757755039 wait complete. 2023-06-09T11:11:12.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.522+0000 7fcbd8b40b80 20 client.4716 trim_cache size 0 max 16384 2023-06-09T11:11:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.541+0000 7fcbd8b40b80 1 Processor -- start 2023-06-09T11:11:12.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.541+0000 7fcbd8b40b80 1 -- start start 2023-06-09T11:11:12.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276714160 0x55e2767536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276716840 0x55e27671a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2767569d0 con 0x55e27671a1b0 2023-06-09T11:11:12.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e27671bbf0 con 0x55e276714160 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e276717d10 con 0x55e276716840 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.542+0000 7fcbbf59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49348/0 (socket says 172.21.15.7:49348) 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 -- 172.21.15.7:0/1994501365 learned_addr learned my addr 172.21.15.7:0/1994501365 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276714160 0x55e2767536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276716840 0x55e27671a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 -- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276716840 msgr2=0x55e27671a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276716840 0x55e27671a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 -- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276714160 msgr2=0x55e2767536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276714160 0x55e2767536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 -- 172.21.15.7:0/1994501365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e276630e90 con 0x55e27671a1b0 2023-06-09T11:11:12.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55e27661bdb0 tx=0x7fcbb8007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452641ad90ac122e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.543+0000 7fcba57fa700 1 -- 172.21.15.7:0/1994501365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8007750 con 0x55e27671a1b0 2023-06-09T11:11:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcba57fa700 1 -- 172.21.15.7:0/1994501365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb80078b0 con 0x55e27671a1b0 2023-06-09T11:11:12.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcba57fa700 1 -- 172.21.15.7:0/1994501365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8016480 con 0x55e27671a1b0 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1994501365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 msgr2=0x55e27671aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55e27661bdb0 tx=0x7fcbb8007050 comp rx=0 tx=0).stop 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1994501365 shutdown_connections 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e27671a1b0 0x55e27671aac0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e276714160 0x55e2767536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/1994501365 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276716840 0x55e27671a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.544+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1994501365 >> 172.21.15.7:0/1994501365 conn(0x55e27670ef30 msgr2=0x55e27675b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.545+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1994501365 shutdown_connections 2023-06-09T11:11:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.545+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/1994501365 wait complete. 2023-06-09T11:11:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.545+0000 7fcbd8b40b80 1 -- start start 2023-06-09T11:11:12.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.546+0000 7fcbd8b40b80 1 Processor -- start 2023-06-09T11:11:12.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767bb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e276718b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2767642e0 con 0x55e276716840 2023-06-09T11:11:12.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2767c4b70 con 0x55e27671a1b0 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e27671af60 con 0x55e276714160 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.547+0000 7fcbd8b40b80 10 client.0 _ll_register_callbacks cb 0x7fcbb0006d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49356/0 (socket says 172.21.15.7:49356) 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 -- 172.21.15.7:0/2164766442 learned_addr learned my addr 172.21.15.7:0/2164766442 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbf59e700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e276718b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767bb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 -- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 msgr2=0x55e2767bb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767bb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 -- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 msgr2=0x55e276718b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e276718b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 -- 172.21.15.7:0/2164766442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb800f040 con 0x55e276716840 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcbbe59c700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40054c0 tx=0x7fcbb4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4061e4edb472fdca server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.548+0000 7fcb9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb402ba60 con 0x55e276716840 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcbb402bbc0 con 0x55e276716840 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbb40054f0 con 0x55e276716840 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbb402bd80 con 0x55e276716840 2023-06-09T11:11:12.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 10 client.4722 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55e276481c10 con 0x55e276716840 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40054f0 con 0x55e276716840 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7fcbb4004990 con 0x55e276716840 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 10 client.4722 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 10 client.4722 resolve_mds: resolved name 'd' to daemon mds.d 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.549+0000 7fcbd8b40b80 20 client.4722 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:12.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276713120 0x55e2767134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcbd8b40b80 4 client.4722 mds_command: new command op to 4288 tid=0[{"prefix": "status"}] 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 --> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e276639610 con 0x55e276713120 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276713120 0x55e2767134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcbbed9d700 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276713120 0x55e2767134f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cc243c6b8d6cbad server_cookie=d5727057429676d in_seq=0 out_seq=0 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.550+0000 7fcb9dffb700 10 client.4722 ms_handle_connect on v2:172.21.15.117:6834/4051500902 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcb9dffb700 1 -- 172.21.15.7:0/2164766442 <== mds.1 v2:172.21.15.117:6834/4051500902 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e276639610 con 0x55e276713120 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcb9dffb700 10 client.4722 handle_command_reply: tid=0 2023-06-09T11:11:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 client.4722 shutdown 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 msgr2=0x55e2767c1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40054c0 tx=0x7fcbb4000ed0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 shutdown_connections 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:6834/4051500902,v1:172.21.15.117:6835/4051500902] conn(0x55e276713120 0x55e2767134f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55e27671a1b0 0x55e276718b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55e276714160 0x55e2767bb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 --2- 172.21.15.7:0/2164766442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e276716840 0x55e2767c1c30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcb9dffb700 0 client.4722 ms_handle_reset on v2:172.21.15.117:6834/4051500902 2023-06-09T11:11:12.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 >> 172.21.15.7:0/2164766442 conn(0x55e27670ef30 msgr2=0x55e276752c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 shutdown_connections 2023-06-09T11:11:12.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 1 -- 172.21.15.7:0/2164766442 wait complete. 2023-06-09T11:11:12.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.551+0000 7fcbd8b40b80 20 client.4722 trim_cache size 0 max 16384 2023-06-09T11:11:12.564 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "cb7aac03-4bc4-45fa-8177-804a746951e5", 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 1, 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4288, 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-06-09T11:11:12.565 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 16.428332731000001, 2023-06-09T11:11:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 12, 2023-06-09T11:11:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 20, 2023-06-09T11:11:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 20, 2023-06-09T11:11:12.566 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 28.773879502 2023-06-09T11:11:12.566 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:11:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 msgr2=0x7fcbac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 0x7fcbac06d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcbc00ce890 tx=0x7fcbb4009040 comp rx=0 tx=0).stop 2023-06-09T11:11:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 msgr2=0x7fcbc0155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00090b0 tx=0x7fcbb0004750 comp rx=0 tx=0).stop 2023-06-09T11:11:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 shutdown_connections 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbac06ace0 0x7fcbac06d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbc015b4a0 0x7fcbc00b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbc00c9f40 0x7fcbc0155d60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 --2- 172.21.15.7:0/3205021444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbc0153880 0x7fcbc014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 >> 172.21.15.7:0/3205021444 conn(0x7fcbc00737f0 msgr2=0x7fcbc00cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 shutdown_connections 2023-06-09T11:11:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.552+0000 7fcbc51ff700 1 -- 172.21.15.7:0/3205021444 wait complete. 2023-06-09T11:11:12.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-06-09T11:11:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.774+0000 7f03547ff700 1 Processor -- start 2023-06-09T11:11:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.774+0000 7f03547ff700 1 -- start start 2023-06-09T11:11:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.775+0000 7f03547ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f034c0c5b40 0x7f034c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.775+0000 7f03547ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.775+0000 7f03547ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034c1580a0 con 0x7f034c151510 2023-06-09T11:11:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.775+0000 7f03547ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034c0714e0 con 0x7f034c0c5b40 2023-06-09T11:11:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.775+0000 7f03547ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c1518e0 con 0x7f034c071060 2023-06-09T11:11:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f034c0c5b40 0x7f034c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f034c0c5b40 0x7f034c070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49360/0 (socket says 172.21.15.7:49360) 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 -- 172.21.15.7:0/4037280008 learned_addr learned my addr 172.21.15.7:0/4037280008 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0352d9c700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 -- 172.21.15.7:0/4037280008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 msgr2=0x7f034c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 -- 172.21.15.7:0/4037280008 >> v1:172.21.15.117:6789/0 conn(0x7f034c151510 legacy=0x7f034c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 -- 172.21.15.7:0/4037280008 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c0c6180 con 0x7f034c0c5b40 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0352d9c700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.776+0000 7f0351d9a700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c070b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098f0 tx=0x7f033c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805128f09f6b362f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f0351599700 1 -- 172.21.15.7:0/4037280008 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c004230 con 0x7f034c0c5b40 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f0351599700 1 -- 172.21.15.7:0/4037280008 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c004390 con 0x7f034c0c5b40 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f0351599700 1 -- 172.21.15.7:0/4037280008 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c0036a0 con 0x7f034c0c5b40 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 -- 172.21.15.7:0/4037280008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 msgr2=0x7f034c070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c070b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f033c0098f0 tx=0x7f033c009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 -- 172.21.15.7:0/4037280008 shutdown_connections 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 --2- 172.21.15.7:0/4037280008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c070b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 -- 172.21.15.7:0/4037280008 >> 172.21.15.7:0/4037280008 conn(0x7f034c06a4d0 msgr2=0x7f034c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 -- 172.21.15.7:0/4037280008 shutdown_connections 2023-06-09T11:11:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.777+0000 7f03547ff700 1 -- 172.21.15.7:0/4037280008 wait complete. 2023-06-09T11:11:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.779+0000 7f03547ff700 1 Processor -- start 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.779+0000 7f03547ff700 1 -- start start 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c0c1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 0x7f034c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c153390 con 0x7f034c0c5b40 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034c153950 con 0x7f034c151510 2023-06-09T11:11:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f03547ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034c153650 con 0x7f034c071060 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49362/0 (socket says 172.21.15.7:49362) 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.780+0000 7f0351d9a700 1 -- 172.21.15.7:0/4133364399 learned_addr learned my addr 172.21.15.7:0/4133364399 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0352d9c700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 0x7f034c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f035259b700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c0c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 -- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 msgr2=0x7f034c0c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c0c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 -- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 msgr2=0x7f034c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 0x7f034c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 -- 172.21.15.7:0/4133364399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c009550 con 0x7f034c0c5b40 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0352d9c700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 0x7f034c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f035259b700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:12.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0351d9a700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f034c0cc2c0 tx=0x7f033c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0342ffd700 1 -- 172.21.15.7:0/4133364399 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c003e30 con 0x7f034c0c5b40 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0342ffd700 1 -- 172.21.15.7:0/4133364399 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c01f070 con 0x7f034c0c5b40 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034c157770 con 0x7f034c0c5b40 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.781+0000 7f0342ffd700 1 -- 172.21.15.7:0/4133364399 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00b8a0 con 0x7f034c0c5b40 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.782+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c157490 con 0x7f034c0c5b40 2023-06-09T11:11:12.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.784+0000 7f0342ffd700 1 -- 172.21.15.7:0/4133364399 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f033c021030 con 0x7f034c0c5b40 2023-06-09T11:11:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.785+0000 7f0342ffd700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 0x7f033006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.785+0000 7f0342ffd700 1 -- 172.21.15.7:0/4133364399 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f033c08ab80 con 0x7f034c0c5b40 2023-06-09T11:11:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.785+0000 7f035259b700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 0x7f033006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.786+0000 7f035259b700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 0x7f033006d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.787+0000 7f03680f3b80 1 Processor -- start 2023-06-09T11:11:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.787+0000 7f03680f3b80 1 -- start start 2023-06-09T11:11:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2018840 0x55ddd201b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1f21010 con 0x55ddd201c1b0 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1f73850 con 0x55ddd2018840 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1f6cce0 con 0x55ddd2016160 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49378/0 (socket says 172.21.15.7:49378) 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 -- 172.21.15.7:0/2943598022 learned_addr learned my addr 172.21.15.7:0/2943598022 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f0351d9a700 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2018840 0x55ddd201b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f0352d9c700 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 -- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 msgr2=0x55ddd20165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 -- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2018840 msgr2=0x55ddd201b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2018840 0x55ddd201b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.788+0000 7f035259b700 1 -- 172.21.15.7:0/2943598022 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd1dee5e0 con 0x55ddd201c1b0 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f035259b700 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f034800c9a0 tx=0x7f034800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecf7aea93e130c8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f032ffff700 1 -- 172.21.15.7:0/2943598022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034800ade0 con 0x55ddd201c1b0 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f032ffff700 1 -- 172.21.15.7:0/2943598022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03480136a0 con 0x55ddd201c1b0 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f032ffff700 1 -- 172.21.15.7:0/2943598022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348013a50 con 0x55ddd201c1b0 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f03680f3b80 1 -- 172.21.15.7:0/2943598022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 msgr2=0x55ddd201c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.789+0000 7f03680f3b80 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f034800c9a0 tx=0x7f034800f000 comp rx=0 tx=0).stop 2023-06-09T11:11:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 -- 172.21.15.7:0/2943598022 shutdown_connections 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd201c1b0 0x55ddd201c580 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 --2- 172.21.15.7:0/2943598022 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2018840 0x55ddd201b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 -- 172.21.15.7:0/2943598022 >> 172.21.15.7:0/2943598022 conn(0x55ddd2010f30 msgr2=0x55ddd2019840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 -- 172.21.15.7:0/2943598022 shutdown_connections 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 -- 172.21.15.7:0/2943598022 wait complete. 2023-06-09T11:11:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.790+0000 7f03680f3b80 1 -- start start 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.791+0000 7f03680f3b80 1 Processor -- start 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2018840 0x55ddd206c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd20437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1f0e890 con 0x55ddd2016160 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1f0e530 con 0x55ddd201c1b0 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddd1df08d0 con 0x55ddd2018840 2023-06-09T11:11:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f03680f3b80 10 client.0 _ll_register_callbacks cb 0x55ddd1f2c730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49380/0 (socket says 172.21.15.7:49380) 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 -- 172.21.15.7:0/3228731055 learned_addr learned my addr 172.21.15.7:0/3228731055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f0352d9c700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd20437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f0351d9a700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2018840 0x55ddd206c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 -- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2018840 msgr2=0x55ddd206c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2018840 0x55ddd206c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 -- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 msgr2=0x55ddd20437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd20437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.792+0000 7f035259b700 1 -- 172.21.15.7:0/3228731055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034800c680 con 0x55ddd2016160 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f035259b700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0348004040 tx=0x7f0348004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31651ce1052c406 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f032d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0348004520 con 0x55ddd2016160 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0348031c40 con 0x55ddd2016160 2023-06-09T11:11:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddd200e6f0 con 0x55ddd2016160 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0348031e00 con 0x55ddd2016160 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f03680f3b80 10 client.4734 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55ddd1f33140 con 0x55ddd2016160 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.793+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddd200e6f0 con 0x55ddd2016160 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7f0348013810 con 0x55ddd2016160 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 10 client.4734 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 10 client.4734 resolve_mds: resolved name 'f' to daemon mds.f 2023-06-09T11:11:12.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 20 client.4734 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd206f2a0 0x55ddd206f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 4 client.4734 mds_command: new command op to 4291 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 --> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddd1f2f050 con 0x55ddd206f2a0 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.794+0000 7f0351d9a700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd206f2a0 0x55ddd206f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.795+0000 7f0351d9a700 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd206f2a0 0x55ddd206f670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71aafc7e5aa374e8 server_cookie=dfd416e23b07fb0e in_seq=0 out_seq=0 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.795+0000 7f032d7fa700 10 client.4734 ms_handle_connect on v2:172.21.15.117:6836/2733808306 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.795+0000 7f032d7fa700 1 -- 172.21.15.7:0/3228731055 <== mds.0 v2:172.21.15.117:6836/2733808306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ddd1f2f050 con 0x55ddd206f2a0 2023-06-09T11:11:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.795+0000 7f032d7fa700 10 client.4734 handle_command_reply: tid=0 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.795+0000 7f03680f3b80 1 client.4734 shutdown 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 msgr2=0x55ddd2044b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0348004040 tx=0x7f0348004710 comp rx=0 tx=0).stop 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 shutdown_connections 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd206f2a0 0x55ddd206f670 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd20437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2016160 0x55ddd2044b70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 --2- 172.21.15.7:0/3228731055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2018840 0x55ddd206c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f032d7fa700 0 client.4734 ms_handle_reset on v2:172.21.15.117:6836/2733808306 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 >> 172.21.15.7:0/3228731055 conn(0x55ddd2010f30 msgr2=0x55ddd201a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 shutdown_connections 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 1 -- 172.21.15.7:0/3228731055 wait complete. 2023-06-09T11:11:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.796+0000 7f03680f3b80 20 client.4734 trim_cache size 0 max 16384 2023-06-09T11:11:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.814+0000 7f03680f3b80 1 Processor -- start 2023-06-09T11:11:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 -- start start 2023-06-09T11:11:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2016160 0x55ddd205a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd201c1b0 0x55ddd201b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd207e9a0 con 0x55ddd2018840 2023-06-09T11:11:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2059be0 con 0x55ddd2016160 2023-06-09T11:11:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.815+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddd20435e0 con 0x55ddd201c1b0 2023-06-09T11:11:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49388/0 (socket says 172.21.15.7:49388) 2023-06-09T11:11:12.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 -- 172.21.15.7:0/58241220 learned_addr learned my addr 172.21.15.7:0/58241220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 -- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd201c1b0 msgr2=0x55ddd201b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd201c1b0 0x55ddd201b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 -- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2016160 msgr2=0x55ddd205a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2016160 0x55ddd205a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 -- 172.21.15.7:0/58241220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddd1f32e90 con 0x55ddd2018840 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0351d9a700 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f033c009c40 tx=0x7f033c05f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8849c50d252ab28 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0340ff9700 1 -- 172.21.15.7:0/58241220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c088fc0 con 0x55ddd2018840 2023-06-09T11:11:12.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0340ff9700 1 -- 172.21.15.7:0/58241220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c057da0 con 0x55ddd2018840 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.816+0000 7f0340ff9700 1 -- 172.21.15.7:0/58241220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c088360 con 0x55ddd2018840 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 -- 172.21.15.7:0/58241220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 msgr2=0x55ddd205a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f033c009c40 tx=0x7f033c05f040 comp rx=0 tx=0).stop 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 -- 172.21.15.7:0/58241220 shutdown_connections 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd201c1b0 0x55ddd201b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd2016160 0x55ddd205a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 --2- 172.21.15.7:0/58241220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd205a980 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 -- 172.21.15.7:0/58241220 >> 172.21.15.7:0/58241220 conn(0x55ddd2010f30 msgr2=0x55ddd2012400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 -- 172.21.15.7:0/58241220 shutdown_connections 2023-06-09T11:11:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.817+0000 7f03680f3b80 1 -- 172.21.15.7:0/58241220 wait complete. 2023-06-09T11:11:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.818+0000 7f03680f3b80 1 -- start start 2023-06-09T11:11:12.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.818+0000 7f03680f3b80 1 Processor -- start 2023-06-09T11:11:12.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20bc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd2046310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd20c64a0 con 0x55ddd2018840 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddd20c6010 con 0x55ddd201c1b0 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ddd2046fa0 con 0x55ddd2016160 2023-06-09T11:11:12.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f03680f3b80 10 client.0 _ll_register_callbacks cb 0x7f033c004680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f0351d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49394/0 (socket says 172.21.15.7:49394) 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f0351d9a700 1 -- 172.21.15.7:0/102062368 learned_addr learned my addr 172.21.15.7:0/102062368 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f035259b700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20bc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.819+0000 7f0352d9c700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd2046310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 -- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 msgr2=0x55ddd20bc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20bc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 -- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 msgr2=0x55ddd2046310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd2046310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 -- 172.21.15.7:0/102062368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034c157490 con 0x55ddd2018840 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f0351d9a700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f033c059990 tx=0x7f0348011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7a903d733e8fbc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f032d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c056110 con 0x55ddd2018840 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f033c056270 con 0x55ddd2018840 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f03480136a0 con 0x55ddd2018840 2023-06-09T11:11:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f033c088360 con 0x55ddd2018840 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f03680f3b80 10 client.4740 fetch_fsmap learned FSMap version 12 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x55ddd1d83c10 con 0x55ddd2018840 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.820+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03480136a0 con 0x55ddd2018840 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 12) v1 ==== 4228+0+0 (secure 0 0 0) 0x7f033c058ef0 con 0x55ddd2018840 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 10 client.4740 fetch_fsmap finished waiting for FSMap version 12 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 10 client.4740 resolve_mds: resolved name 'f' to daemon mds.f 2023-06-09T11:11:12.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 20 client.4740 populate_metadata read hostname 'smithi007' 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd2046850 0x55ddd205b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 4 client.4740 mds_command: new command op to 4291 tid=0[{"prefix": "status"}] 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 --> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ddd1f3b610 con 0x55ddd2046850 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.821+0000 7f035259b700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd2046850 0x55ddd205b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f032d7fa700 10 client.4740 ms_handle_connect on v2:172.21.15.117:6836/2733808306 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f035259b700 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd2046850 0x55ddd205b4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71be5dbf4b01d4c6 server_cookie=29b4e5779fb7291f in_seq=0 out_seq=0 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f032d7fa700 1 -- 172.21.15.7:0/102062368 <== mds.0 v2:172.21.15.117:6836/2733808306 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ddd1f3b610 con 0x55ddd2046850 2023-06-09T11:11:12.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f032d7fa700 10 client.4740 handle_command_reply: tid=0 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 client.4740 shutdown 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 msgr2=0x55ddd20c5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f033c059990 tx=0x7f0348011000 comp rx=0 tx=0).stop 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 shutdown_connections 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:6836/2733808306,v1:172.21.15.117:6837/2733808306] conn(0x55ddd2046850 0x55ddd205b4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x55ddd201c1b0 0x55ddd2046310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x55ddd2016160 0x55ddd20bc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f032d7fa700 0 client.4740 ms_handle_reset on v2:172.21.15.117:6836/2733808306 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 --2- 172.21.15.7:0/102062368 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ddd2018840 0x55ddd20c5ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 >> 172.21.15.7:0/102062368 conn(0x55ddd2010f30 msgr2=0x55ddd2044690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.822+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 shutdown_connections 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03680f3b80 1 -- 172.21.15.7:0/102062368 wait complete. 2023-06-09T11:11:12.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03680f3b80 20 client.4740 trim_cache size 0 max 16384 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "cb7aac03-4bc4-45fa-8177-804a746951e5", 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4291, 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2023-06-09T11:11:12.836 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 22.640600705000001, 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 12, 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 20, 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 20, 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 29.037424595000001 2023-06-09T11:11:12.837 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:11:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 msgr2=0x7f033006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 0x7f033006d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0348006fd0 tx=0x7f0348008040 comp rx=0 tx=0).stop 2023-06-09T11:11:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 msgr2=0x7f034c153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:12.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f034c0cc2c0 tx=0x7f033c004750 comp rx=0 tx=0).stop 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 shutdown_connections 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f034c151510 0x7f034c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f033006ace0 0x7f033006d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f034c071060 0x7f034c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 --2- 172.21.15.7:0/4133364399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f034c0c5b40 0x7f034c153de0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 >> 172.21.15.7:0/4133364399 conn(0x7f034c06a4d0 msgr2=0x7f034c14d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.823+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 shutdown_connections 2023-06-09T11:11:12.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:12.824+0000 7f03547ff700 1 -- 172.21.15.7:0/4133364399 wait complete. 2023-06-09T11:11:12.846 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-06-09T11:11:12.881 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-06-09T11:11:12.882 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-06-09T11:11:12.882 INFO:tasks.ceph_fuse:client.0 config is {} 2023-06-09T11:11:12.882 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-09T11:11:12.883 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:12.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:12.884 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-06-09T11:11:12.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:12.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-06-09T11:11:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2023-06-09T11:11:13.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T11:11:13.042 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-06-09T11:11:13.042 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-06-09T11:11:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.237+0000 7f618d3ff700 1 Processor -- start 2023-06-09T11:11:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.237+0000 7f618d3ff700 1 -- start start 2023-06-09T11:11:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.238+0000 7f618d3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f61881598b0 0x7f6188159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.238+0000 7f618d3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618815cec0 con 0x7f61880c9fe0 2023-06-09T11:11:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.238+0000 7f618d3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618815d160 con 0x7f61880c9090 2023-06-09T11:11:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.238+0000 7f618d3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61880ca440 con 0x7f61881598b0 2023-06-09T11:11:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61867fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f61880c9fe0 0x7f61880cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59848/0 (socket says 172.21.15.7:59848) 2023-06-09T11:11:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61867fc700 1 -- 172.21.15.7:0/1662194707 learned_addr learned my addr 172.21.15.7:0/1662194707 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:13.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61877fe700 1 --2- 172.21.15.7:0/1662194707 >> v2:172.21.15.117:3300/0 conn(0x7f61881598b0 0x7f6188159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466180026 0 0) 0x7f618815cec0 con 0x7f61880c9fe0 2023-06-09T11:11:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61700035b0 con 0x7f61880c9fe0 2023-06-09T11:11:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917253561 0 0) 0x7f618815d160 con 0x7f61880c9090 2023-06-09T11:11:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f618815cec0 con 0x7f61880c9090 2023-06-09T11:11:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61877fe700 1 -- 172.21.15.7:0/1662194707 >> v1:172.21.15.117:6789/0 conn(0x7f61880c9090 legacy=0x7f61880cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61877fe700 1 -- 172.21.15.7:0/1662194707 >> v1:172.21.15.7:6789/0 conn(0x7f61880c9fe0 legacy=0x7f61880cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.239+0000 7f61877fe700 1 -- 172.21.15.7:0/1662194707 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618815d440 con 0x7f61881598b0 2023-06-09T11:11:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f61877fe700 1 --2- 172.21.15.7:0/1662194707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f6188159cf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6180009d90 tx=0x7f618000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6d7c574849874a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f618000e070 con 0x7f61881598b0 2023-06-09T11:11:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618000bb20 con 0x7f61881598b0 2023-06-09T11:11:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f6185ffb700 1 -- 172.21.15.7:0/1662194707 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6180012430 con 0x7f61881598b0 2023-06-09T11:11:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f618d3ff700 1 -- 172.21.15.7:0/1662194707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 msgr2=0x7f6188159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1662194707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f6188159cf0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6180009d90 tx=0x7f618000b330 comp rx=0 tx=0).stop 2023-06-09T11:11:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f618d3ff700 1 -- 172.21.15.7:0/1662194707 shutdown_connections 2023-06-09T11:11:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1662194707 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f6188159cf0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.240+0000 7f618d3ff700 1 -- 172.21.15.7:0/1662194707 >> 172.21.15.7:0/1662194707 conn(0x7f6188070af0 msgr2=0x7f6188070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.241+0000 7f618d3ff700 1 -- 172.21.15.7:0/1662194707 shutdown_connections 2023-06-09T11:11:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.241+0000 7f618d3ff700 1 -- 172.21.15.7:0/1662194707 wait complete. 2023-06-09T11:11:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.242+0000 7f618d3ff700 1 Processor -- start 2023-06-09T11:11:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.242+0000 7f618d3ff700 1 -- start start 2023-06-09T11:11:13.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61880c9fe0 0x7f61880b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f618815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880c5f60 con 0x7f61880c9090 2023-06-09T11:11:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61880b1c10 con 0x7f61881598b0 2023-06-09T11:11:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f618d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618815a480 con 0x7f61880c9fe0 2023-06-09T11:11:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f6186ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f6186ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33668/0 (socket says 172.21.15.7:33668) 2023-06-09T11:11:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.243+0000 7f6186ffd700 1 -- 172.21.15.7:0/1336600719 learned_addr learned my addr 172.21.15.7:0/1336600719 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f61867fc700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61880c9fe0 0x7f61880b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f61877fe700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f618815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 -- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61880c9fe0 msgr2=0x7f61880b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61880c9fe0 0x7f61880b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 -- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 msgr2=0x7f618815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f618815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 -- 172.21.15.7:0/1336600719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6180009a70 con 0x7f61880c9090 2023-06-09T11:11:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f6186ffd700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f617800c530 tx=0x7f617800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:13.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780052d0 con 0x7f61880c9090 2023-06-09T11:11:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61880ca620 con 0x7f61880c9090 2023-06-09T11:11:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6178005450 con 0x7f61880c9090 2023-06-09T11:11:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.244+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61780038c0 con 0x7f61880c9090 2023-06-09T11:11:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.245+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618815d720 con 0x7f61880c9090 2023-06-09T11:11:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.246+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618815a230 con 0x7f61880c9090 2023-06-09T11:11:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.247+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6178003a20 con 0x7f61880c9090 2023-06-09T11:11:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.247+0000 7f616f7fe700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.248+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6178088a40 con 0x7f61880c9090 2023-06-09T11:11:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.248+0000 7f61867fc700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.248+0000 7f61867fc700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 0x7f617406d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f61880c2da0 tx=0x7f617c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.252+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f617808ae60 con 0x7f61880c9090 2023-06-09T11:11:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.502+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6188154600 con 0x7f61880c9090 2023-06-09T11:11:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.502+0000 7f616f7fe700 1 -- 172.21.15.7:0/1336600719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6178053a70 con 0x7f61880c9090 2023-06-09T11:11:13.509 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-09T11:11:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 msgr2=0x7f617406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 0x7f617406d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f61880c2da0 tx=0x7f617c009040 comp rx=0 tx=0).stop 2023-06-09T11:11:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 msgr2=0x7f61881ef430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f617800c530 tx=0x7f617800c8f0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 shutdown_connections 2023-06-09T11:11:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f61881598b0 0x7f618815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61880c9090 0x7f61881ef430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f617406ace0 0x7f617406d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 --2- 172.21.15.7:0/1336600719 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f61880c9fe0 0x7f61880b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 >> 172.21.15.7:0/1336600719 conn(0x7f6188070af0 msgr2=0x7f61880cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 shutdown_connections 2023-06-09T11:11:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:13.505+0000 7f618d3ff700 1 -- 172.21.15.7:0/1336600719 wait complete. 2023-06-09T11:11:13.528 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-09T11:11:13.529 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-09T11:11:13.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2023-06-09T11:11:13.529 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-09T11:11:13.529 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:13.530 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-09T11:11:13.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-09T11:11:13.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-09T11:11:13.530 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-06-09T11:11:13.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:13.531 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2023-06-09T11:11:13.573 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-09T11:11:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-09T11:11:13.573 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2023-06-09T11:11:13.574 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-06-09T11:11:13.575 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42187sec preferred_lft 42187sec 2023-06-09T11:11:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2023-06-09T11:11:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2023-06-09T11:11:13.576 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-09T11:11:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2023-06-09T11:11:13.576 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2023-06-09T11:11:13.577 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-09T11:11:13.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:13.578 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-06-09T11:11:13.578 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-06-09T11:11:13.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2023-06-09T11:11:13.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2023-06-09T11:11:13.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2023-06-09T11:11:13.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-09T11:11:13.579 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-06-09T11:11:14.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.045 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-09T11:11:14.117 INFO:teuthology.orchestra.run.smithi007.stdout:1 2023-06-09T11:11:14.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.121 DEBUG:teuthology.orchestra.run.smithi007:> route 2023-06-09T11:11:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2023-06-09T11:11:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-09T11:11:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-06-09T11:11:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-06-09T11:11:14.173 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-09T11:11:14.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.174 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-06-09T11:11:14.175 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-06-09T11:11:14.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-09T11:11:14.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-09T11:11:14.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-09T11:11:14.176 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-06-09T11:11:14.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.858 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-06-09T11:11:14.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.902 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2023-06-09T11:11:14.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:14.986 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-06-09T11:11:14.986 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-06-09T11:11:14.986 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-09T11:11:14.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-06-09T11:11:14.987 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-06-09T11:11:15.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-06-09T11:11:15.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:15.169 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-06-09T11:11:15.170 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-06-09T11:11:15.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-06-09T11:11:15.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-09T11:11:15.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-06-09T11:11:15.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-06-09T11:11:15.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-06-09T11:11:15.171 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-06-09T11:11:15.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:15.443 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec bash -c ' 2023-06-09T11:11:15.443 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2023-06-09T11:11:15.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2023-06-09T11:11:15.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2023-06-09T11:11:15.444 DEBUG:teuthology.orchestra.run.smithi007:> ') 2023-06-09T11:11:15.625 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-06-09T11:11:15.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T11:11:15.626 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:15.668 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-06-09T11:11:15.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-09T11:11:15.669 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:15.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-06-09T11:11:15.863 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-09T11:11:15.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-09T11:11:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-09T11:11:15.933 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-09T11:11:15.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:15.934 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-06-09T11:11:15.977 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-09T11:11:15.977 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-06-09T11:11:16.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-06-09T11:11:16.094 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-09T11:11:16.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-09T11:11:16.166 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-09T11:11:16.169 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-09T11:11:16.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:16.171 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-06-09T11:11:16.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-06-09T11:11:16.202+0000 7f59911055c0 -1 init, newargv = 0x55b9794dc590 newargc=15 2023-06-09T11:11:16.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[137538]: starting ceph client 2023-06-09T11:11:16.532 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:ceph-fuse[137538]: starting fuse 2023-06-09T11:11:17.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo modprobe fuse 2023-06-09T11:11:17.394 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-09T11:11:17.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-09T11:11:17.464 INFO:teuthology.orchestra.run.smithi007.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-09T11:11:17.466 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-09T11:11:17.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:17.468 DEBUG:teuthology.orchestra.run.smithi007:> ls /sys/fs/fuse/connections 2023-06-09T11:11:17.512 INFO:teuthology.orchestra.run.smithi007.stdout:43 2023-06-09T11:11:17.512 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-06-09T11:11:17.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-09T11:11:17.513 DEBUG:teuthology.orchestra.run.smithi007:> import glob 2023-06-09T11:11:17.515 DEBUG:teuthology.orchestra.run.smithi007:> import re 2023-06-09T11:11:17.516 DEBUG:teuthology.orchestra.run.smithi007:> import os 2023-06-09T11:11:17.516 DEBUG:teuthology.orchestra.run.smithi007:> import subprocess 2023-06-09T11:11:17.516 DEBUG:teuthology.orchestra.run.smithi007:> 2023-06-09T11:11:17.517 DEBUG:teuthology.orchestra.run.smithi007:> def _find_admin_socket(client_name): 2023-06-09T11:11:17.517 DEBUG:teuthology.orchestra.run.smithi007:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-09T11:11:17.517 DEBUG:teuthology.orchestra.run.smithi007:> files = glob.glob(asok_path) 2023-06-09T11:11:17.517 DEBUG:teuthology.orchestra.run.smithi007:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-06-09T11:11:17.518 DEBUG:teuthology.orchestra.run.smithi007:> 2023-06-09T11:11:17.519 DEBUG:teuthology.orchestra.run.smithi007:> # Given a non-glob path, it better be there 2023-06-09T11:11:17.519 DEBUG:teuthology.orchestra.run.smithi007:> if "*" not in asok_path: 2023-06-09T11:11:17.520 DEBUG:teuthology.orchestra.run.smithi007:> assert(len(files) == 1) 2023-06-09T11:11:17.520 DEBUG:teuthology.orchestra.run.smithi007:> return files[0] 2023-06-09T11:11:17.521 DEBUG:teuthology.orchestra.run.smithi007:> 2023-06-09T11:11:17.521 DEBUG:teuthology.orchestra.run.smithi007:> for f in files: 2023-06-09T11:11:17.522 DEBUG:teuthology.orchestra.run.smithi007:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-09T11:11:17.522 DEBUG:teuthology.orchestra.run.smithi007:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-09T11:11:17.523 DEBUG:teuthology.orchestra.run.smithi007:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-09T11:11:17.523 DEBUG:teuthology.orchestra.run.smithi007:> contents = proc_f.read() 2023-06-09T11:11:17.524 DEBUG:teuthology.orchestra.run.smithi007:> if mountpoint in contents: 2023-06-09T11:11:17.524 DEBUG:teuthology.orchestra.run.smithi007:> return f 2023-06-09T11:11:17.524 DEBUG:teuthology.orchestra.run.smithi007:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-09T11:11:17.525 DEBUG:teuthology.orchestra.run.smithi007:> 2023-06-09T11:11:17.525 DEBUG:teuthology.orchestra.run.smithi007:> print(_find_admin_socket("client.0")) 2023-06-09T11:11:17.528 DEBUG:teuthology.orchestra.run.smithi007:> ' 2023-06-09T11:11:17.696 INFO:teuthology.orchestra.run.smithi007.stdout:/var/run/ceph/ceph-client.0.137538.asok 2023-06-09T11:11:17.874 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137538.asok 2023-06-09T11:11:17.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:17.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137538.asok status 2023-06-09T11:11:18.103 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:11:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "metadata": { 2023-06-09T11:11:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_sha1": "034108821cde95ca1c56d458dfccc7ef9e0d964d", 2023-06-09T11:11:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "ceph_version": "ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev)", 2023-06-09T11:11:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "entity_id": "0", 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "hostname": "smithi007", 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "pid": "137538", 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "root": "/" 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_count": 0, 2023-06-09T11:11:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "dentry_pinned_count": 0, 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 4752, 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "inst": { 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "name": { 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "client", 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4752 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-06-09T11:11:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1078574456 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": { 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2023-06-09T11:11:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "192.168.0.1:0", 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 1078574456 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "inst_str": "client.4752 192.168.0.1:0/1078574456", 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "addr_str": "192.168.0.1:0/1078574456", 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "inode_count": 1, 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_epoch": 12, 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch": 20, 2023-06-09T11:11:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_epoch_barrier": 0, 2023-06-09T11:11:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklisted": false, 2023-06-09T11:11:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs" 2023-06-09T11:11:18.108 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:11:18.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:18.118 DEBUG:teuthology.orchestra.run.smithi007:> stat --file-system '--printf=%T 2023-06-09T11:11:18.118 DEBUG:teuthology.orchestra.run.smithi007:> ' -- /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:18.166 INFO:teuthology.orchestra.run.smithi007.stdout:fuseblk 2023-06-09T11:11:18.166 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:18.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:11:18.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:18.282 INFO:teuthology.run_tasks:Running task mds_thrash... 2023-06-09T11:11:18.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.498+0000 7fab5d7ff700 1 Processor -- start 2023-06-09T11:11:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.498+0000 7fab5d7ff700 1 -- start start 2023-06-09T11:11:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab5d7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58151510 0x7fab580c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab5d7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab58071230 0x7fab58158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab5d7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab581580a0 con 0x7fab5815b380 2023-06-09T11:11:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab5d7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab581518e0 con 0x7fab58071230 2023-06-09T11:11:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab5d7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c6180 con 0x7fab58151510 2023-06-09T11:11:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.499+0000 7fab577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab58071230 0x7fab58158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fab58071230 0x7fab58158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33676/0 (socket says 172.21.15.7:33676) 2023-06-09T11:11:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 -- 172.21.15.7:0/3554429971 learned_addr learned my addr 172.21.15.7:0/3554429971 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab56ffd700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58151510 0x7fab580c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 -- 172.21.15.7:0/3554429971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58151510 msgr2=0x7fab580c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58151510 0x7fab580c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 -- 172.21.15.7:0/3554429971 >> v1:172.21.15.117:6789/0 conn(0x7fab5815b380 legacy=0x7fab58070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 -- 172.21.15.7:0/3554429971 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58157490 con 0x7fab58071230 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab577fe700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab58071230 0x7fab58158870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e50 tx=0x7fab5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5e21c5fc7fe020 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab55ffb700 1 -- 172.21.15.7:0/3554429971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000e070 con 0x7fab58071230 2023-06-09T11:11:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab55ffb700 1 -- 172.21.15.7:0/3554429971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5000b910 con 0x7fab58071230 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.500+0000 7fab55ffb700 1 -- 172.21.15.7:0/3554429971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5000bcc0 con 0x7fab58071230 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 -- 172.21.15.7:0/3554429971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab58071230 msgr2=0x7fab58158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab58071230 0x7fab58158870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e50 tx=0x7fab5000b330 comp rx=0 tx=0).stop 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 -- 172.21.15.7:0/3554429971 shutdown_connections 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab58071230 0x7fab58158870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/3554429971 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58151510 0x7fab580c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 -- 172.21.15.7:0/3554429971 >> 172.21.15.7:0/3554429971 conn(0x7fab5806a4d0 msgr2=0x7fab5806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 -- 172.21.15.7:0/3554429971 shutdown_connections 2023-06-09T11:11:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.501+0000 7fab5d7ff700 1 -- 172.21.15.7:0/3554429971 wait complete. 2023-06-09T11:11:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.503+0000 7fab5d7ff700 1 Processor -- start 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.503+0000 7fab5d7ff700 1 -- start start 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58071230 0x7fab580c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 0x7fab581e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab58158a60 con 0x7fab5815b380 2023-06-09T11:11:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab580c0140 con 0x7fab58151510 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab5d7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab581598f0 con 0x7fab58071230 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33680/0 (socket says 172.21.15.7:33680) 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 -- 172.21.15.7:0/1886914643 learned_addr learned my addr 172.21.15.7:0/1886914643 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab567fc700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 0x7fab581e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab56ffd700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58071230 0x7fab580c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 -- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58071230 msgr2=0x7fab580c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58071230 0x7fab580c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 -- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 msgr2=0x7fab581e4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 0x7fab581e4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab577fe700 1 -- 172.21.15.7:0/1886914643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50009b30 con 0x7fab5815b380 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.504+0000 7fab567fc700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 0x7fab581e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab577fe700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab50012000 tx=0x7fab50004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50004960 con 0x7fab5815b380 2023-06-09T11:11:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab500099b0 con 0x7fab5815b380 2023-06-09T11:11:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5001c680 con 0x7fab5815b380 2023-06-09T11:11:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab58157a50 con 0x7fab5815b380 2023-06-09T11:11:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.505+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab58157770 con 0x7fab5815b380 2023-06-09T11:11:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.506+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab58157d30 con 0x7fab5815b380 2023-06-09T11:11:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.507+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab5001c7e0 con 0x7fab5815b380 2023-06-09T11:11:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.508+0000 7fab3f7fe700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 0x7fab4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.508+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fab5008b190 con 0x7fab5815b380 2023-06-09T11:11:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.509+0000 7fab56ffd700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 0x7fab4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.509+0000 7fab56ffd700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 0x7fab4406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab48009b70 tx=0x7fab48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.513+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fab50056230 con 0x7fab5815b380 2023-06-09T11:11:18.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.720+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab580c1370 con 0x7fab5815b380 2023-06-09T11:11:18.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.722+0000 7fab3f7fe700 1 -- 172.21.15.7:0/1886914643 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7fab50059850 con 0x7fab5815b380 2023-06-09T11:11:18.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:18.731 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:18.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 msgr2=0x7fab4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 0x7fab4406d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fab48009b70 tx=0x7fab48008040 comp rx=0 tx=0).stop 2023-06-09T11:11:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 msgr2=0x7fab581f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fab50012000 tx=0x7fab50004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 shutdown_connections 2023-06-09T11:11:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab5815b380 0x7fab581f2550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab4406ace0 0x7fab4406d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fab58071230 0x7fab580c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 --2- 172.21.15.7:0/1886914643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fab58151510 0x7fab581e4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 >> 172.21.15.7:0/1886914643 conn(0x7fab5806a4d0 msgr2=0x7fab58149c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 shutdown_connections 2023-06-09T11:11:18.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.724+0000 7fab5d7ff700 1 -- 172.21.15.7:0/1886914643 wait complete. 2023-06-09T11:11:18.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:11:18.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:11:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.948+0000 7f0b707ff700 1 Processor -- start 2023-06-09T11:11:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.948+0000 7f0b707ff700 1 -- start start 2023-06-09T11:11:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b707ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b680cc7d0 0x7f0b6806b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b707ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b707ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b680b4800 con 0x7f0b6806b2a0 2023-06-09T11:11:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b707ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6806be10 con 0x7f0b680cc7d0 2023-06-09T11:11:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b707ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6806bbb0 con 0x7f0b680c9fb0 2023-06-09T11:11:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b6ed9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b6dd9a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b680cc7d0 0x7f0b6806b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b6ed9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39950/0 (socket says 172.21.15.7:39950) 2023-06-09T11:11:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b6dd9a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b680cc7d0 0x7f0b6806b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:44302/0 (socket says 172.21.15.7:44302) 2023-06-09T11:11:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.950+0000 7f0b6ed9c700 1 -- 172.21.15.7:0/1706711307 learned_addr learned my addr 172.21.15.7:0/1706711307 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 msgr2=0x7f0b68159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6dd9a700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1706711307 >> v1:172.21.15.117:6789/0 conn(0x7f0b6806b2a0 legacy=0x7f0b680cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1706711307 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6815cb50 con 0x7f0b680cc7d0 2023-06-09T11:11:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6ed9c700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6dd9a700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b6806b670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009870 tx=0x7f0b58009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90e484ff521f9086 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6d599700 1 -- 172.21.15.7:0/1706711307 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58011830 con 0x7f0b680cc7d0 2023-06-09T11:11:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6d599700 1 -- 172.21.15.7:0/1706711307 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58011990 con 0x7f0b680cc7d0 2023-06-09T11:11:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.951+0000 7f0b6d599700 1 -- 172.21.15.7:0/1706711307 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58011d20 con 0x7f0b680cc7d0 2023-06-09T11:11:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 -- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 msgr2=0x7f0b6806b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b6806b670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009870 tx=0x7f0b58009b80 comp rx=0 tx=0).stop 2023-06-09T11:11:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 -- 172.21.15.7:0/1706711307 shutdown_connections 2023-06-09T11:11:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b680c9fb0 0x7f0b68159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1706711307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b6806b670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 -- 172.21.15.7:0/1706711307 >> 172.21.15.7:0/1706711307 conn(0x7f0b68072050 msgr2=0x7f0b68072420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 -- 172.21.15.7:0/1706711307 shutdown_connections 2023-06-09T11:11:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.952+0000 7f0b707ff700 1 -- 172.21.15.7:0/1706711307 wait complete. 2023-06-09T11:11:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.954+0000 7f0b707ff700 1 Processor -- start 2023-06-09T11:11:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.954+0000 7f0b707ff700 1 -- start start 2023-06-09T11:11:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 0x7f0b681449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b680cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680cf050 con 0x7f0b680c9fb0 2023-06-09T11:11:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b68152610 con 0x7f0b680cc7d0 2023-06-09T11:11:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b707ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680cfa70 con 0x7f0b6806b2a0 2023-06-09T11:11:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b6dd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b6dd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33692/0 (socket says 172.21.15.7:33692) 2023-06-09T11:11:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1616680033 learned_addr learned my addr 172.21.15.7:0/1616680033 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b6e59b700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 0x7f0b681449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.955+0000 7f0b6ed9c700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b680cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 msgr2=0x7f0b681449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 0x7f0b681449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 msgr2=0x7f0b680cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b680cdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 -- 172.21.15.7:0/1616680033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60009b30 con 0x7f0b680c9fb0 2023-06-09T11:11:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6e59b700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 0x7f0b681449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b6dd9a700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097c0 tx=0x7f0b58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b580036a0 con 0x7f0b680c9fb0 2023-06-09T11:11:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5801e070 con 0x7f0b680c9fb0 2023-06-09T11:11:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58003a60 con 0x7f0b680c9fb0 2023-06-09T11:11:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.956+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b580094a0 con 0x7f0b680c9fb0 2023-06-09T11:11:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.957+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6815d110 con 0x7f0b680c9fb0 2023-06-09T11:11:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.958+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6815ce30 con 0x7f0b680c9fb0 2023-06-09T11:11:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.959+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b58003bc0 con 0x7f0b680c9fb0 2023-06-09T11:11:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.960+0000 7f0b5effd700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 0x7f0b4c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.960+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0b5808a390 con 0x7f0b680c9fb0 2023-06-09T11:11:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.960+0000 7f0b6e59b700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 0x7f0b4c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.961+0000 7f0b6e59b700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 0x7f0b4c06d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0b64007fd0 tx=0x7f0b64009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:18.964+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0b58055430 con 0x7f0b680c9fb0 2023-06-09T11:11:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.173+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b6815d3f0 con 0x7f0b680c9fb0 2023-06-09T11:11:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.173+0000 7f0b5effd700 1 -- 172.21.15.7:0/1616680033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b5800f2e0 con 0x7f0b680c9fb0 2023-06-09T11:11:19.179 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:11:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 msgr2=0x7f0b4c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 0x7f0b4c06d1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0b64007fd0 tx=0x7f0b64009040 comp rx=0 tx=0).stop 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 msgr2=0x7f0b680cd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097c0 tx=0x7f0b58004750 comp rx=0 tx=0).stop 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 shutdown_connections 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b680cc7d0 0x7f0b680cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4c06ad10 0x7f0b4c06d1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6806b2a0 0x7f0b681449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 --2- 172.21.15.7:0/1616680033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b680c9fb0 0x7f0b680cd7f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 >> 172.21.15.7:0/1616680033 conn(0x7f0b68072050 msgr2=0x7f0b680b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 shutdown_connections 2023-06-09T11:11:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.176+0000 7f0b707ff700 1 -- 172.21.15.7:0/1616680033 wait complete. 2023-06-09T11:11:19.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:11:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.396+0000 7f7e7b3ff700 1 Processor -- start 2023-06-09T11:11:19.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.396+0000 7f7e7b3ff700 1 -- start start 2023-06-09T11:11:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.397+0000 7f7e7b3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e741537a0 0x7f7e7406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.397+0000 7f7e7b3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e740c64e0 0x7f7e740cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7b3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7406b740 0x7f7e740cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7b3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740c4f70 con 0x7f7e741537a0 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7b3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e740c5c20 con 0x7f7e740c64e0 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7b3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e740c6220 con 0x7f7e7406b740 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e741537a0 0x7f7e7406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e741537a0 0x7f7e7406b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33706/0 (socket says 172.21.15.7:33706) 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 -- 172.21.15.7:0/3987645246 learned_addr learned my addr 172.21.15.7:0/3987645246 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:19.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7899a700 1 --2- 172.21.15.7:0/3987645246 >> v2:172.21.15.117:3300/0 conn(0x7f7e740c64e0 0x7f7e740cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7999c700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7406b740 0x7f7e740cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 -- 172.21.15.7:0/3987645246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7406b740 msgr2=0x7f7e740cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7406b740 0x7f7e740cfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 -- 172.21.15.7:0/3987645246 >> v2:172.21.15.117:3300/0 conn(0x7f7e740c64e0 msgr2=0x7f7e740cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 --2- 172.21.15.7:0/3987645246 >> v2:172.21.15.117:3300/0 conn(0x7f7e740c64e0 0x7f7e740cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.398+0000 7f7e7919b700 1 -- 172.21.15.7:0/3987645246 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e740c4480 con 0x7f7e741537a0 2023-06-09T11:11:19.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e7919b700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e741537a0 0x7f7e7406b1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cd0b0 tx=0x7f7e70009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76a5930d064526f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e6bfff700 1 -- 172.21.15.7:0/3987645246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70011830 con 0x7f7e741537a0 2023-06-09T11:11:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e6bfff700 1 -- 172.21.15.7:0/3987645246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e70011990 con 0x7f7e741537a0 2023-06-09T11:11:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e6bfff700 1 -- 172.21.15.7:0/3987645246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70011d20 con 0x7f7e741537a0 2023-06-09T11:11:19.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/3987645246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e741537a0 msgr2=0x7f7e7406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.399+0000 7f7e7b3ff700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e741537a0 0x7f7e7406b1b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e740cd0b0 tx=0x7f7e70009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/3987645246 shutdown_connections 2023-06-09T11:11:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e7406b740 0x7f7e740cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 --2- 172.21.15.7:0/3987645246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e741537a0 0x7f7e7406b1b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 --2- 172.21.15.7:0/3987645246 >> v2:172.21.15.117:3300/0 conn(0x7f7e740c64e0 0x7f7e740cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/3987645246 >> 172.21.15.7:0/3987645246 conn(0x7f7e74072040 msgr2=0x7f7e74072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:19.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/3987645246 shutdown_connections 2023-06-09T11:11:19.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.400+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/3987645246 wait complete. 2023-06-09T11:11:19.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.401+0000 7f7e7b3ff700 1 Processor -- start 2023-06-09T11:11:19.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.402+0000 7f7e7b3ff700 1 -- start start 2023-06-09T11:11:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.402+0000 7f7e7b3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7b3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e740c64e0 0x7f7e740cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7b3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e741537a0 0x7f7e7414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7b3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e74150e90 con 0x7f7e7406b740 2023-06-09T11:11:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7b3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ee720 con 0x7f7e740c64e0 2023-06-09T11:11:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7b3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e741ef540 con 0x7f7e741537a0 2023-06-09T11:11:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33718/0 (socket says 172.21.15.7:33718) 2023-06-09T11:11:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 -- 172.21.15.7:0/1917462868 learned_addr learned my addr 172.21.15.7:0/1917462868 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7899a700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e740c64e0 0x7f7e740cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7999c700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e741537a0 0x7f7e7414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 -- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e741537a0 msgr2=0x7f7e7414e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e741537a0 0x7f7e7414e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 -- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e740c64e0 msgr2=0x7f7e740cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e740c64e0 0x7f7e740cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.403+0000 7f7e7919b700 1 -- 172.21.15.7:0/1917462868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70009550 con 0x7f7e7406b740 2023-06-09T11:11:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e7919b700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c10 tx=0x7f7e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70004450 con 0x7f7e7406b740 2023-06-09T11:11:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7001e070 con 0x7f7e7406b740 2023-06-09T11:11:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e700038e0 con 0x7f7e7406b740 2023-06-09T11:11:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/1917462868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e740c46d0 con 0x7f7e7406b740 2023-06-09T11:11:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.404+0000 7f7e7b3ff700 1 -- 172.21.15.7:0/1917462868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7415ae20 con 0x7f7e7406b740 2023-06-09T11:11:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.405+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e500052e0 con 0x7f7e7406b740 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.406+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e70003a40 con 0x7f7e7406b740 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.407+0000 7f7e69ffb700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 0x7f7e5806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.408+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7e7008a260 con 0x7f7e7406b740 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.408+0000 7f7e7899a700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 0x7f7e5806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.408+0000 7f7e7899a700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 0x7f7e5806d1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e741e4500 tx=0x7f7e64009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.412+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7e70055270 con 0x7f7e7406b740 2023-06-09T11:11:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.619+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7e50005b10 con 0x7f7e7406b740 2023-06-09T11:11:19.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.619+0000 7f7e69ffb700 1 -- 172.21.15.7:0/1917462868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7e70058890 con 0x7f7e7406b740 2023-06-09T11:11:19.625 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:11:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 msgr2=0x7f7e5806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 0x7f7e5806d1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7e741e4500 tx=0x7f7e64009210 comp rx=0 tx=0).stop 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 msgr2=0x7f7e7414a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009c10 tx=0x7f7e70004750 comp rx=0 tx=0).stop 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 shutdown_connections 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ad10 0x7f7e5806d1c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e741537a0 0x7f7e7414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e7406b740 0x7f7e7414a3e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 --2- 172.21.15.7:0/1917462868 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e740c64e0 0x7f7e740cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 >> 172.21.15.7:0/1917462868 conn(0x7f7e74072040 msgr2=0x7f7e74152570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 shutdown_connections 2023-06-09T11:11:19.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.622+0000 7f7e577fe700 1 -- 172.21.15.7:0/1917462868 wait complete. 2023-06-09T11:11:19.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:11:19.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.842+0000 7f18ac7ff700 1 Processor -- start 2023-06-09T11:11:19.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.842+0000 7f18ac7ff700 1 -- start start 2023-06-09T11:11:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18ac7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18a41537a0 0x7f18a406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18ac7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18a406b740 0x7f18a415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18ac7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18a40c6220 con 0x7f18a40c64e0 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18ac7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c4f70 con 0x7f18a41537a0 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18ac7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a40c5c20 con 0x7f18a406b740 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.843+0000 7f18aa59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18a41537a0 0x7f18a406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18a9d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18a40c64e0 0x7f18a40cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59860/0 (socket says 172.21.15.7:59860) 2023-06-09T11:11:19.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18a9d9a700 1 -- 172.21.15.7:0/2479720706 learned_addr learned my addr 172.21.15.7:0/2479720706 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aad9c700 1 --2- 172.21.15.7:0/2479720706 >> v2:172.21.15.117:3300/0 conn(0x7f18a406b740 0x7f18a415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aa59b700 1 -- 172.21.15.7:0/2479720706 >> v2:172.21.15.117:3300/0 conn(0x7f18a406b740 msgr2=0x7f18a415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aa59b700 1 --2- 172.21.15.7:0/2479720706 >> v2:172.21.15.117:3300/0 conn(0x7f18a406b740 0x7f18a415b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aa59b700 1 -- 172.21.15.7:0/2479720706 >> v1:172.21.15.7:6789/0 conn(0x7f18a40c64e0 legacy=0x7f18a40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aa59b700 1 -- 172.21.15.7:0/2479720706 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a40c4480 con 0x7f18a41537a0 2023-06-09T11:11:19.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18aa59b700 1 --2- 172.21.15.7:0/2479720706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a406b1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18a408d6c0 tx=0x7f18a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba518e32975a000d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18a9599700 1 -- 172.21.15.7:0/2479720706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011830 con 0x7f18a41537a0 2023-06-09T11:11:19.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.844+0000 7f18a9599700 1 -- 172.21.15.7:0/2479720706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a0011990 con 0x7f18a41537a0 2023-06-09T11:11:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18a9599700 1 -- 172.21.15.7:0/2479720706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011d20 con 0x7f18a41537a0 2023-06-09T11:11:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 -- 172.21.15.7:0/2479720706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 msgr2=0x7f18a406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/2479720706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a406b1b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18a408d6c0 tx=0x7f18a0009b20 comp rx=0 tx=0).stop 2023-06-09T11:11:19.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 -- 172.21.15.7:0/2479720706 shutdown_connections 2023-06-09T11:11:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/2479720706 >> v2:172.21.15.117:3300/0 conn(0x7f18a406b740 0x7f18a415b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/2479720706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a406b1b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 -- 172.21.15.7:0/2479720706 >> 172.21.15.7:0/2479720706 conn(0x7f18a4072040 msgr2=0x7f18a4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 -- 172.21.15.7:0/2479720706 shutdown_connections 2023-06-09T11:11:19.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.845+0000 7f18ac7ff700 1 -- 172.21.15.7:0/2479720706 wait complete. 2023-06-09T11:11:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.847+0000 7f18ac7ff700 1 Processor -- start 2023-06-09T11:11:19.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.847+0000 7f18ac7ff700 1 -- start start 2023-06-09T11:11:19.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18a406b740 0x7f18a41ef510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a40c64e0 0x7f18a414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41ef080 con 0x7f18a41537a0 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a41eea70 con 0x7f18a40c64e0 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18ac7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a415a150 con 0x7f18a406b740 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18aad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18aad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33740/0 (socket says 172.21.15.7:33740) 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.848+0000 7f18aad9c700 1 -- 172.21.15.7:0/4270065028 learned_addr learned my addr 172.21.15.7:0/4270065028 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aa59b700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18a406b740 0x7f18a41ef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18a9d9a700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a40c64e0 0x7f18a414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 -- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18a406b740 msgr2=0x7f18a41ef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18a406b740 0x7f18a41ef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 -- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a40c64e0 msgr2=0x7f18a414cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a40c64e0 0x7f18a414cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:19.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 -- 172.21.15.7:0/4270065028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a00094a0 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.849+0000 7f18aad9c700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f189c007e60 tx=0x7f189c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.850+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00ed30 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.850+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189c014420 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.850+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c0147d0 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.850+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a40c46d0 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.850+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a415ae20 con 0x7f18a41537a0 2023-06-09T11:11:19.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.851+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a40c49b0 con 0x7f18a41537a0 2023-06-09T11:11:19.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.852+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f189c014930 con 0x7f18a41537a0 2023-06-09T11:11:19.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.853+0000 7f189affd700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 0x7f188806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.853+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f189c089b40 con 0x7f18a41537a0 2023-06-09T11:11:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.853+0000 7f18aa59b700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 0x7f188806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:19.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.854+0000 7f18aa59b700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 0x7f188806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18a00090b0 tx=0x7f18a0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:19.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:19.857+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f189c054c10 con 0x7f18a41537a0 2023-06-09T11:11:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.062+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f18a4074cf0 con 0x7f18a41537a0 2023-06-09T11:11:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.062+0000 7f189affd700 1 -- 172.21.15.7:0/4270065028 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f189c058230 con 0x7f18a41537a0 2023-06-09T11:11:20.068 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:11:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 msgr2=0x7f188806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 0x7f188806d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f18a00090b0 tx=0x7f18a0008030 comp rx=0 tx=0).stop 2023-06-09T11:11:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 msgr2=0x7f18a4159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f189c007e60 tx=0x7f189c00e3c0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 shutdown_connections 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18a41537a0 0x7f18a4159340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f188806ace0 0x7f188806d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18a406b740 0x7f18a41ef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 --2- 172.21.15.7:0/4270065028 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18a40c64e0 0x7f18a414cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 >> 172.21.15.7:0/4270065028 conn(0x7f18a4072040 msgr2=0x7f18a414fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.064+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 shutdown_connections 2023-06-09T11:11:20.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.065+0000 7f18ac7ff700 1 -- 172.21.15.7:0/4270065028 wait complete. 2023-06-09T11:11:20.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:11:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.287+0000 7fa1475ff700 1 Processor -- start 2023-06-09T11:11:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.287+0000 7fa1475ff700 1 -- start start 2023-06-09T11:11:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa1475ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1400c64e0 0x7fa14006b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa1475ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400c6220 con 0x7fa1401537a0 2023-06-09T11:11:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa1475ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1400c4480 con 0x7fa14006b740 2023-06-09T11:11:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa1475ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1400c4f70 con 0x7fa1400c64e0 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa14539b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa1401537a0 0x7fa1400cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59866/0 (socket says 172.21.15.7:59866) 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa14539b700 1 -- 172.21.15.7:0/2544419840 learned_addr learned my addr 172.21.15.7:0/2544419840 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.288+0000 7fa144b9a700 1 --2- 172.21.15.7:0/2544419840 >> v2:172.21.15.117:3300/0 conn(0x7fa1400c64e0 0x7fa14006b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa137fff700 1 -- 172.21.15.7:0/2544419840 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255759313 0 0) 0x7fa13c001110 con 0x7fa1401537a0 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa137fff700 1 -- 172.21.15.7:0/2544419840 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa12c0035b0 con 0x7fa1401537a0 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa144b9a700 1 -- 172.21.15.7:0/2544419840 >> v1:172.21.15.117:6789/0 conn(0x7fa14006b740 legacy=0x7fa14015b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa144b9a700 1 -- 172.21.15.7:0/2544419840 >> v1:172.21.15.7:6789/0 conn(0x7fa1401537a0 legacy=0x7fa1400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa144b9a700 1 -- 172.21.15.7:0/2544419840 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14015ae20 con 0x7fa1400c64e0 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.289+0000 7fa144b9a700 1 --2- 172.21.15.7:0/2544419840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa14006b1b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa130009840 tx=0x7fa130009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10fdc114c35545f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa137fff700 1 -- 172.21.15.7:0/2544419840 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130011830 con 0x7fa1400c64e0 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa137fff700 1 -- 172.21.15.7:0/2544419840 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa130011990 con 0x7fa1400c64e0 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa137fff700 1 -- 172.21.15.7:0/2544419840 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa130011d00 con 0x7fa1400c64e0 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 -- 172.21.15.7:0/2544419840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 msgr2=0x7fa14006b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 --2- 172.21.15.7:0/2544419840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa14006b1b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa130009840 tx=0x7fa130009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 -- 172.21.15.7:0/2544419840 shutdown_connections 2023-06-09T11:11:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 --2- 172.21.15.7:0/2544419840 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa14006b1b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 -- 172.21.15.7:0/2544419840 >> 172.21.15.7:0/2544419840 conn(0x7fa140072040 msgr2=0x7fa140072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 -- 172.21.15.7:0/2544419840 shutdown_connections 2023-06-09T11:11:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.290+0000 7fa1475ff700 1 -- 172.21.15.7:0/2544419840 wait complete. 2023-06-09T11:11:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.292+0000 7fa1475ff700 1 Processor -- start 2023-06-09T11:11:20.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.292+0000 7fa1475ff700 1 -- start start 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 0x7fa140151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa1400cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400c7500 con 0x7fa1401537a0 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400c7780 con 0x7fa1400c64e0 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa1475ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1400c7a60 con 0x7fa14006b740 2023-06-09T11:11:20.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa145b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa145b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33756/0 (socket says 172.21.15.7:33756) 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.293+0000 7fa145b9c700 1 -- 172.21.15.7:0/1105465094 learned_addr learned my addr 172.21.15.7:0/1105465094 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa144b9a700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa1400cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa14539b700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 0x7fa140151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 -- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 msgr2=0x7fa140151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 0x7fa140151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 -- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 msgr2=0x7fa1400cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa1400cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 -- 172.21.15.7:0/1105465094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300094a0 con 0x7fa1401537a0 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa144b9a700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa1400cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa14539b700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 0x7fa140151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa145b9c700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa13800cc50 tx=0x7fa13800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800ad80 con 0x7fa1401537a0 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13800ce50 con 0x7fa1401537a0 2023-06-09T11:11:20.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa138003c40 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1400c49b0 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.294+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1400c46d0 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.295+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1400c4c90 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.296+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1380050d0 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.297+0000 7fa135ffb700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 0x7fa12406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.298+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa138089670 con 0x7fa1401537a0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.298+0000 7fa14539b700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 0x7fa12406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.298+0000 7fa14539b700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 0x7fa12406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007fd0 tx=0x7fa13c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.302+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa138054680 con 0x7fa1401537a0 2023-06-09T11:11:20.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.507+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa140074cf0 con 0x7fa1401537a0 2023-06-09T11:11:20.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.507+0000 7fa135ffb700 1 -- 172.21.15.7:0/1105465094 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa138057ca0 con 0x7fa1401537a0 2023-06-09T11:11:20.514 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:11:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.509+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 msgr2=0x7fa12406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.509+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 0x7fa12406d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa13c007fd0 tx=0x7fa13c009040 comp rx=0 tx=0).stop 2023-06-09T11:11:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 msgr2=0x7fa1400cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa13800cc50 tx=0x7fa13800a340 comp rx=0 tx=0).stop 2023-06-09T11:11:20.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 shutdown_connections 2023-06-09T11:11:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1401537a0 0x7fa1400cb500 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa12406ace0 0x7fa12406d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa14006b740 0x7fa140151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 --2- 172.21.15.7:0/1105465094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1400c64e0 0x7fa1400cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 >> 172.21.15.7:0/1105465094 conn(0x7fa140072040 msgr2=0x7fa14015c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 shutdown_connections 2023-06-09T11:11:20.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.510+0000 7fa1475ff700 1 -- 172.21.15.7:0/1105465094 wait complete. 2023-06-09T11:11:20.533 INFO:tasks.mds_thrash:mds thrasher using random seed: 1686309080 2023-06-09T11:11:20.534 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:20.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.729+0000 7f38ed9ff700 1 Processor -- start 2023-06-09T11:11:20.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.729+0000 7f38ed9ff700 1 -- start start 2023-06-09T11:11:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38ed9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38ed9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e815a500 con 0x7f38e80c9f40 2023-06-09T11:11:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38ed9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e815da00 con 0x7f38e8153880 2023-06-09T11:11:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38ed9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80c5f10 con 0x7f38e815b4a0 2023-06-09T11:11:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38e77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f38e80c9f40 0x7f38e80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59882/0 (socket says 172.21.15.7:59882) 2023-06-09T11:11:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38e77fe700 1 -- 172.21.15.7:0/436488942 learned_addr learned my addr 172.21.15.7:0/436488942 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.731+0000 7f38dffff700 1 --2- 172.21.15.7:0/436488942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541791620 0 0) 0x7f38e0001490 con 0x7f38e80c9f40 2023-06-09T11:11:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d40035b0 con 0x7f38e80c9f40 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492303369 0 0) 0x7f38d8001110 con 0x7f38e8153880 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38dffff700 1 -- 172.21.15.7:0/436488942 >> v1:172.21.15.117:6789/0 conn(0x7f38e8153880 legacy=0x7f38e815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38dffff700 1 -- 172.21.15.7:0/436488942 >> v1:172.21.15.7:6789/0 conn(0x7f38e80c9f40 legacy=0x7f38e80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38dffff700 1 -- 172.21.15.7:0/436488942 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e81596e0 con 0x7f38e815b4a0 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38dffff700 1 --2- 172.21.15.7:0/436488942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e815d4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb1eec1389c198fa server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011830 con 0x7f38e815b4a0 2023-06-09T11:11:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0011990 con 0x7f38e815b4a0 2023-06-09T11:11:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.732+0000 7f38e67fc700 1 -- 172.21.15.7:0/436488942 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d0011d20 con 0x7f38e815b4a0 2023-06-09T11:11:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 -- 172.21.15.7:0/436488942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 msgr2=0x7f38e815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/436488942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e815d4c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009870 tx=0x7f38d0009b80 comp rx=0 tx=0).stop 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 -- 172.21.15.7:0/436488942 shutdown_connections 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/436488942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e815d4c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 -- 172.21.15.7:0/436488942 >> 172.21.15.7:0/436488942 conn(0x7f38e80737f0 msgr2=0x7f38e80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 -- 172.21.15.7:0/436488942 shutdown_connections 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.733+0000 7f38ed9ff700 1 -- 172.21.15.7:0/436488942 wait complete. 2023-06-09T11:11:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.735+0000 7f38ed9ff700 1 Processor -- start 2023-06-09T11:11:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.735+0000 7f38ed9ff700 1 -- start start 2023-06-09T11:11:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e80c9f40 0x7f38e81584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e814bdb0 con 0x7f38e8153880 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e81571a0 con 0x7f38e80c9f40 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38ed9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e814bb40 con 0x7f38e815b4a0 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38e6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e80c9f40 0x7f38e81584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38dffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33762/0 (socket says 172.21.15.7:33762) 2023-06-09T11:11:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38dffff700 1 -- 172.21.15.7:0/3933352501 learned_addr learned my addr 172.21.15.7:0/3933352501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38e77fe700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38dffff700 1 -- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 msgr2=0x7f38e80cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.736+0000 7f38dffff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e80cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38dffff700 1 -- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e80c9f40 msgr2=0x7f38e81584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38dffff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e80c9f40 0x7f38e81584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38dffff700 1 -- 172.21.15.7:0/3933352501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d00094a0 con 0x7f38e8153880 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38dffff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097c0 tx=0x7f38d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d001a9f0 con 0x7f38e8153880 2023-06-09T11:11:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d001ab50 con 0x7f38e8153880 2023-06-09T11:11:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00243f0 con 0x7f38e8153880 2023-06-09T11:11:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e8159c60 con 0x7f38e8153880 2023-06-09T11:11:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.737+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e8159980 con 0x7f38e8153880 2023-06-09T11:11:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.738+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8159f40 con 0x7f38e8153880 2023-06-09T11:11:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.739+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f38d0024550 con 0x7f38e8153880 2023-06-09T11:11:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.740+0000 7f38e4ff9700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 0x7f38c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.740+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38d008a970 con 0x7f38e8153880 2023-06-09T11:11:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.740+0000 7f38e6ffd700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 0x7f38c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.741+0000 7f38e6ffd700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 0x7f38c806d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38d8000f60 tx=0x7f38d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.744+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f38d0055ab0 con 0x7f38e8153880 2023-06-09T11:11:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.952+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e8074cf0 con 0x7f38e8153880 2023-06-09T11:11:20.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.953+0000 7f38e4ff9700 1 -- 172.21.15.7:0/3933352501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7f38d00590d0 con 0x7f38e8153880 2023-06-09T11:11:20.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:20.991 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:20.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 msgr2=0x7f38c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 0x7f38c806d1c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f38d8000f60 tx=0x7f38d8009040 comp rx=0 tx=0).stop 2023-06-09T11:11:20.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 msgr2=0x7f38e80cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:20.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f38d00097c0 tx=0x7f38d0004750 comp rx=0 tx=0).stop 2023-06-09T11:11:20.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 shutdown_connections 2023-06-09T11:11:20.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f38c806ad10 0x7f38c806d1c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38e815b4a0 0x7f38e80cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f38e80c9f40 0x7f38e81584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 --2- 172.21.15.7:0/3933352501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38e8153880 0x7f38e80cddb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 >> 172.21.15.7:0/3933352501 conn(0x7f38e80737f0 msgr2=0x7f38e815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 shutdown_connections 2023-06-09T11:11:20.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:20.956+0000 7f38ed9ff700 1 -- 172.21.15.7:0/3933352501 wait complete. 2023-06-09T11:11:20.996 INFO:tasks.mds_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:11:20.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:11:21.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.194+0000 7f01613ff700 1 Processor -- start 2023-06-09T11:11:21.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.194+0000 7f01613ff700 1 -- start start 2023-06-09T11:11:21.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f01613ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f015c1537a0 0x7f015c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f01613ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f015c06b740 0x7f015c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f01613ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f015c0c6220 con 0x7f015c0c64e0 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f01613ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0c4f70 con 0x7f015c1537a0 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f01613ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015c0c5c20 con 0x7f015c06b740 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f015c1537a0 0x7f015c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f015c1537a0 0x7f015c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33766/0 (socket says 172.21.15.7:33766) 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 -- 172.21.15.7:0/3461278599 learned_addr learned my addr 172.21.15.7:0/3461278599 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015b7fe700 1 --2- 172.21.15.7:0/3461278599 >> v2:172.21.15.117:3300/0 conn(0x7f015c06b740 0x7f015c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 -- 172.21.15.7:0/3461278599 >> v2:172.21.15.117:3300/0 conn(0x7f015c06b740 msgr2=0x7f015c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 --2- 172.21.15.7:0/3461278599 >> v2:172.21.15.117:3300/0 conn(0x7f015c06b740 0x7f015c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 -- 172.21.15.7:0/3461278599 >> v1:172.21.15.117:6789/0 conn(0x7f015c0c64e0 legacy=0x7f015c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.196+0000 7f015affd700 1 -- 172.21.15.7:0/3461278599 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015c0c4480 con 0x7f015c1537a0 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f015affd700 1 --2- 172.21.15.7:0/3461278599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c1537a0 0x7f015c06b1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f015c08d6c0 tx=0x7f014c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f810f8d4661541 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f015a7fc700 1 -- 172.21.15.7:0/3461278599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00b870 con 0x7f015c1537a0 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f015a7fc700 1 -- 172.21.15.7:0/3461278599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c00b9d0 con 0x7f015c1537a0 2023-06-09T11:11:21.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f015a7fc700 1 -- 172.21.15.7:0/3461278599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c00bd60 con 0x7f015c1537a0 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f01613ff700 1 -- 172.21.15.7:0/3461278599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c1537a0 msgr2=0x7f015c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.197+0000 7f01613ff700 1 --2- 172.21.15.7:0/3461278599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c1537a0 0x7f015c06b1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f015c08d6c0 tx=0x7f014c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 -- 172.21.15.7:0/3461278599 shutdown_connections 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 --2- 172.21.15.7:0/3461278599 >> v2:172.21.15.117:3300/0 conn(0x7f015c06b740 0x7f015c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 --2- 172.21.15.7:0/3461278599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c1537a0 0x7f015c06b1b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 -- 172.21.15.7:0/3461278599 >> 172.21.15.7:0/3461278599 conn(0x7f015c072040 msgr2=0x7f015c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 -- 172.21.15.7:0/3461278599 shutdown_connections 2023-06-09T11:11:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.198+0000 7f01613ff700 1 -- 172.21.15.7:0/3461278599 wait complete. 2023-06-09T11:11:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.200+0000 7f01613ff700 1 Processor -- start 2023-06-09T11:11:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.200+0000 7f01613ff700 1 -- start start 2023-06-09T11:11:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.201+0000 7f01613ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.201+0000 7f01613ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 0x7f015c149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.201+0000 7f01613ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c1537a0 0x7f015c14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f01613ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1498c0 con 0x7f015c06b740 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f01613ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1efc60 con 0x7f015c0c64e0 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f01613ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015c1ef0e0 con 0x7f015c1537a0 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33776/0 (socket says 172.21.15.7:33776) 2023-06-09T11:11:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 -- 172.21.15.7:0/1963837415 learned_addr learned my addr 172.21.15.7:0/1963837415 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015b7fe700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c1537a0 0x7f015c14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f0153fff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 0x7f015c149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 -- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c1537a0 msgr2=0x7f015c14a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c1537a0 0x7f015c14a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 -- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 msgr2=0x7f015c149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 0x7f015c149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 -- 172.21.15.7:0/1963837415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c009550 con 0x7f015c06b740 2023-06-09T11:11:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f0153fff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 0x7f015c149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.202+0000 7f015affd700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f014c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.203+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c004540 con 0x7f015c06b740 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.203+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c0036a0 con 0x7f015c06b740 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.203+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c003a10 con 0x7f015c06b740 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.203+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015c0c46d0 con 0x7f015c06b740 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.203+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015c15ae20 con 0x7f015c06b740 2023-06-09T11:11:21.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.204+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015c0c49b0 con 0x7f015c06b740 2023-06-09T11:11:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.205+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f014c021070 con 0x7f015c06b740 2023-06-09T11:11:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.206+0000 7f01537fe700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 0x7f013c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.206+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f014c08ac00 con 0x7f015c06b740 2023-06-09T11:11:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.206+0000 7f0153fff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 0x7f013c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.207+0000 7f0153fff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 0x7f013c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f015c0cd690 tx=0x7f0144009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.211+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f014c055c10 con 0x7f015c06b740 2023-06-09T11:11:21.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.417+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f015c074cf0 con 0x7f015c06b740 2023-06-09T11:11:21.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.418+0000 7f01537fe700 1 -- 172.21.15.7:0/1963837415 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f014c0150a0 con 0x7f015c06b740 2023-06-09T11:11:21.424 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:11:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 msgr2=0x7f013c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 0x7f013c06d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f015c0cd690 tx=0x7f0144009040 comp rx=0 tx=0).stop 2023-06-09T11:11:21.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 msgr2=0x7f015c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f014c000c00 tx=0x7f014c004750 comp rx=0 tx=0).stop 2023-06-09T11:11:21.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 shutdown_connections 2023-06-09T11:11:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f013c06ace0 0x7f013c06d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f015c1537a0 0x7f015c14a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f015c06b740 0x7f015c0cdb70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 --2- 172.21.15.7:0/1963837415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f015c0c64e0 0x7f015c149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 >> 172.21.15.7:0/1963837415 conn(0x7f015c072040 msgr2=0x7f015c14f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 shutdown_connections 2023-06-09T11:11:21.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.420+0000 7f01613ff700 1 -- 172.21.15.7:0/1963837415 wait complete. 2023-06-09T11:11:21.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:11:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.641+0000 7f6f5dbff700 1 Processor -- start 2023-06-09T11:11:21.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.641+0000 7f6f5dbff700 1 -- start start 2023-06-09T11:11:21.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f5dbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f580c72b0 0x7f6f5806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f5dbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5806b600 con 0x7f6f580c6ee0 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f5dbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5806aff0 con 0x7f6f5806ba90 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f5dbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f58074780 con 0x7f6f580c72b0 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f577fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6f580c6ee0 0x7f6f5815ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59896/0 (socket says 172.21.15.7:59896) 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f577fe700 1 -- 172.21.15.7:0/2224109181 learned_addr learned my addr 172.21.15.7:0/2224109181 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:21.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f57fff700 1 --2- 172.21.15.7:0/2224109181 >> v2:172.21.15.117:3300/0 conn(0x7f6f580c72b0 0x7f6f5806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.643+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 62259358 0 0) 0x7f6f5806b600 con 0x7f6f580c6ee0 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f400035b0 con 0x7f6f580c6ee0 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877658511 0 0) 0x7f6f5806aff0 con 0x7f6f5806ba90 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f5806b600 con 0x7f6f5806ba90 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f57fff700 1 -- 172.21.15.7:0/2224109181 >> v1:172.21.15.117:6789/0 conn(0x7f6f5806ba90 legacy=0x7f6f580737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f57fff700 1 -- 172.21.15.7:0/2224109181 >> v1:172.21.15.7:6789/0 conn(0x7f6f580c6ee0 legacy=0x7f6f5815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f57fff700 1 -- 172.21.15.7:0/2224109181 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58073f00 con 0x7f6f580c72b0 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f57fff700 1 --2- 172.21.15.7:0/2224109181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f5806a770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009dc0 tx=0x7f6f5000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577c2d3bd50c27ae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000e070 con 0x7f6f580c72b0 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5000bae0 con 0x7f6f580c72b0 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.644+0000 7f6f567fc700 1 -- 172.21.15.7:0/2224109181 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000be90 con 0x7f6f580c72b0 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 -- 172.21.15.7:0/2224109181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 msgr2=0x7f6f5806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/2224109181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f5806a770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009dc0 tx=0x7f6f5000b330 comp rx=0 tx=0).stop 2023-06-09T11:11:21.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 -- 172.21.15.7:0/2224109181 shutdown_connections 2023-06-09T11:11:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/2224109181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f5806a770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 -- 172.21.15.7:0/2224109181 >> 172.21.15.7:0/2224109181 conn(0x7f6f580cc380 msgr2=0x7f6f580cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 -- 172.21.15.7:0/2224109181 shutdown_connections 2023-06-09T11:11:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.645+0000 7f6f5dbff700 1 -- 172.21.15.7:0/2224109181 wait complete. 2023-06-09T11:11:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.647+0000 7f6f5dbff700 1 Processor -- start 2023-06-09T11:11:21.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.647+0000 7f6f5dbff700 1 -- start start 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f580c6ee0 0x7f6f581eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f58141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581ef130 con 0x7f6f5806ba90 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f58155af0 con 0x7f6f580c72b0 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f5dbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f581582d0 con 0x7f6f580c6ee0 2023-06-09T11:11:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 -- 172.21.15.7:0/438027899 learned_addr learned my addr 172.21.15.7:0/438027899 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f56ffd700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f580c6ee0 0x7f6f581eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f57fff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f58141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 -- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f580c6ee0 msgr2=0x7f6f581eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f580c6ee0 0x7f6f581eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 -- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 msgr2=0x7f6f58141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f58141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.648+0000 7f6f577fe700 1 -- 172.21.15.7:0/438027899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50009a70 con 0x7f6f5806ba90 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f577fe700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c700 tx=0x7f6f4800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48005d70 con 0x7f6f5806ba90 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f480070d0 con 0x7f6f5806ba90 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f48007440 con 0x7f6f5806ba90 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f580744a0 con 0x7f6f5806ba90 2023-06-09T11:11:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.649+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f580741c0 con 0x7f6f5806ba90 2023-06-09T11:11:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.650+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5806b290 con 0x7f6f5806ba90 2023-06-09T11:11:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.651+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f480075a0 con 0x7f6f5806ba90 2023-06-09T11:11:21.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.652+0000 7f6f3ffff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 0x7f6f4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.652+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6f48088850 con 0x7f6f5806ba90 2023-06-09T11:11:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.652+0000 7f6f56ffd700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 0x7f6f4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:21.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.653+0000 7f6f56ffd700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 0x7f6f4406d1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f5813f890 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:21.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.656+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6f48053910 con 0x7f6f5806ba90 2023-06-09T11:11:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.862+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f58074cf0 con 0x7f6f5806ba90 2023-06-09T11:11:21.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.863+0000 7f6f3ffff700 1 -- 172.21.15.7:0/438027899 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f480170a0 con 0x7f6f5806ba90 2023-06-09T11:11:21.869 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:11:21.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 msgr2=0x7f6f4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 0x7f6f4406d1c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6f5813f890 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 msgr2=0x7f6f581f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6f4800c700 tx=0x7f6f4800ca10 comp rx=0 tx=0).stop 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 shutdown_connections 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f580c72b0 0x7f6f58141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f5806ba90 0x7f6f581f2ce0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f4406ad10 0x7f6f4406d1c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 --2- 172.21.15.7:0/438027899 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f580c6ee0 0x7f6f581eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 >> 172.21.15.7:0/438027899 conn(0x7f6f580cc380 msgr2=0x7f6f58140260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 shutdown_connections 2023-06-09T11:11:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:21.865+0000 7f6f5dbff700 1 -- 172.21.15.7:0/438027899 wait complete. 2023-06-09T11:11:21.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:11:22.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.086+0000 7f3802bff700 1 Processor -- start 2023-06-09T11:11:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.087+0000 7f3802bff700 1 -- start start 2023-06-09T11:11:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f3802bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37fc1537a0 0x7f37fc06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f3802bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0c6220 con 0x7f37fc06b740 2023-06-09T11:11:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f3802bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc0c4480 con 0x7f37fc0c64e0 2023-06-09T11:11:22.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f3802bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc0c4f70 con 0x7f37fc1537a0 2023-06-09T11:11:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f380099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37fc1537a0 0x7f37fc06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f380099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37fc1537a0 0x7f37fc06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33794/0 (socket says 172.21.15.7:33794) 2023-06-09T11:11:22.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.088+0000 7f380099b700 1 -- 172.21.15.7:0/2190632440 learned_addr learned my addr 172.21.15.7:0/2190632440 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f380099b700 1 -- 172.21.15.7:0/2190632440 >> v1:172.21.15.117:6789/0 conn(0x7f37fc0c64e0 legacy=0x7f37fc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f380099b700 1 -- 172.21.15.7:0/2190632440 >> v1:172.21.15.7:6789/0 conn(0x7f37fc06b740 legacy=0x7f37fc15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f380099b700 1 -- 172.21.15.7:0/2190632440 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc15ae20 con 0x7f37fc1537a0 2023-06-09T11:11:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f380099b700 1 --2- 172.21.15.7:0/2190632440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc1537a0 0x7f37fc06b1b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37f00098f0 tx=0x7f37f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cda77a462618b5d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f37fb7fe700 1 -- 172.21.15.7:0/2190632440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0004230 con 0x7f37fc1537a0 2023-06-09T11:11:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f37fb7fe700 1 -- 172.21.15.7:0/2190632440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0004390 con 0x7f37fc1537a0 2023-06-09T11:11:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.089+0000 7f37fb7fe700 1 -- 172.21.15.7:0/2190632440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00036a0 con 0x7f37fc1537a0 2023-06-09T11:11:22.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 -- 172.21.15.7:0/2190632440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc1537a0 msgr2=0x7f37fc06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 --2- 172.21.15.7:0/2190632440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc1537a0 0x7f37fc06b1b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f37f00098f0 tx=0x7f37f0009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 -- 172.21.15.7:0/2190632440 shutdown_connections 2023-06-09T11:11:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 --2- 172.21.15.7:0/2190632440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc1537a0 0x7f37fc06b1b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 -- 172.21.15.7:0/2190632440 >> 172.21.15.7:0/2190632440 conn(0x7f37fc072040 msgr2=0x7f37fc072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 -- 172.21.15.7:0/2190632440 shutdown_connections 2023-06-09T11:11:22.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.090+0000 7f3802bff700 1 -- 172.21.15.7:0/2190632440 wait complete. 2023-06-09T11:11:22.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.091+0000 7f3802bff700 1 Processor -- start 2023-06-09T11:11:22.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.092+0000 7f3802bff700 1 -- start start 2023-06-09T11:11:22.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 0x7f37fc14a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37fc1537a0 0x7f37fc14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc149d70 con 0x7f37fc06b740 2023-06-09T11:11:22.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc14bf60 con 0x7f37fc1537a0 2023-06-09T11:11:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f3802bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0cef40 con 0x7f37fc0c64e0 2023-06-09T11:11:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33806/0 (socket says 172.21.15.7:33806) 2023-06-09T11:11:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 -- 172.21.15.7:0/3253242069 learned_addr learned my addr 172.21.15.7:0/3253242069 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f37fbfff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 0x7f37fc14a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380119c700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37fc1537a0 0x7f37fc14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 -- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 msgr2=0x7f37fc14a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 0x7f37fc14a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 -- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37fc1537a0 msgr2=0x7f37fc14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37fc1537a0 0x7f37fc14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f380099b700 1 -- 172.21.15.7:0/3253242069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f0009550 con 0x7f37fc06b740 2023-06-09T11:11:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.093+0000 7f37fbfff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 0x7f37fc14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f380099b700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b0e0 tx=0x7f37f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000b660 con 0x7f37fc06b740 2023-06-09T11:11:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000b7e0 con 0x7f37fc06b740 2023-06-09T11:11:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000bb90 con 0x7f37fc06b740 2023-06-09T11:11:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc0c49b0 con 0x7f37fc06b740 2023-06-09T11:11:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.094+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc0c46d0 con 0x7f37fc06b740 2023-06-09T11:11:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.095+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc0c4c90 con 0x7f37fc06b740 2023-06-09T11:11:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.096+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f37f000bcf0 con 0x7f37fc06b740 2023-06-09T11:11:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.097+0000 7f37f97fa700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 0x7f37e006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.097+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37f008a890 con 0x7f37fc06b740 2023-06-09T11:11:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.097+0000 7f37fbfff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 0x7f37e006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.098+0000 7f37fbfff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 0x7f37e006d1c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0cee40 tx=0x7f37e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.101+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f37f00558a0 con 0x7f37fc06b740 2023-06-09T11:11:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.313+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37fc074cf0 con 0x7f37fc06b740 2023-06-09T11:11:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.314+0000 7f37f97fa700 1 -- 172.21.15.7:0/3253242069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37f0058ec0 con 0x7f37fc06b740 2023-06-09T11:11:22.320 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:11:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 msgr2=0x7f37e006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 0x7f37e006d1c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f37fc0cee40 tx=0x7f37e8009040 comp rx=0 tx=0).stop 2023-06-09T11:11:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 msgr2=0x7f37fc0c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37f000b0e0 tx=0x7f37f0004750 comp rx=0 tx=0).stop 2023-06-09T11:11:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 shutdown_connections 2023-06-09T11:11:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37fc1537a0 0x7f37fc14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37fc06b740 0x7f37fc0c9320 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37e006ad10 0x7f37e006d1c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.316+0000 7f3802bff700 1 --2- 172.21.15.7:0/3253242069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37fc0c64e0 0x7f37fc14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.317+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 >> 172.21.15.7:0/3253242069 conn(0x7f37fc072040 msgr2=0x7f37fc0cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.317+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 shutdown_connections 2023-06-09T11:11:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.317+0000 7f3802bff700 1 -- 172.21.15.7:0/3253242069 wait complete. 2023-06-09T11:11:22.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:11:22.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.537+0000 7fc448fff700 1 Processor -- start 2023-06-09T11:11:22.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.537+0000 7fc448fff700 1 -- start start 2023-06-09T11:11:22.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc448fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc44415b9c0 0x7fc44406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc448fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4440c6220 con 0x7fc4441537a0 2023-06-09T11:11:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc448fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4440c4480 con 0x7fc4440c64e0 2023-06-09T11:11:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc448fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440c4f70 con 0x7fc44415b9c0 2023-06-09T11:11:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc44259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc4441537a0 0x7fc4440cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59922/0 (socket says 172.21.15.7:59922) 2023-06-09T11:11:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc442d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc44415b9c0 0x7fc44406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc44259c700 1 -- 172.21.15.7:0/25678222 learned_addr learned my addr 172.21.15.7:0/25678222 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:22.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc44159a700 1 -- 172.21.15.7:0/25678222 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842636445 0 0) 0x7fc434001110 con 0x7fc4441537a0 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc44159a700 1 -- 172.21.15.7:0/25678222 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c0035b0 con 0x7fc4441537a0 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc442d9d700 1 -- 172.21.15.7:0/25678222 >> v1:172.21.15.117:6789/0 conn(0x7fc4440c64e0 legacy=0x7fc44415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc442d9d700 1 -- 172.21.15.7:0/25678222 >> v1:172.21.15.7:6789/0 conn(0x7fc4441537a0 legacy=0x7fc4440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.539+0000 7fc442d9d700 1 -- 172.21.15.7:0/25678222 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44415ae20 con 0x7fc44415b9c0 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc442d9d700 1 --2- 172.21.15.7:0/25678222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc44406b1b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbf58498f80e5b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc44159a700 1 -- 172.21.15.7:0/25678222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c00e070 con 0x7fc44415b9c0 2023-06-09T11:11:22.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc44159a700 1 -- 172.21.15.7:0/25678222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c00bb20 con 0x7fc44415b9c0 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc44159a700 1 -- 172.21.15.7:0/25678222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c012430 con 0x7fc44415b9c0 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc448fff700 1 -- 172.21.15.7:0/25678222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 msgr2=0x7fc44406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.540+0000 7fc448fff700 1 --2- 172.21.15.7:0/25678222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc44406b1b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc43c009d90 tx=0x7fc43c00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.541+0000 7fc448fff700 1 -- 172.21.15.7:0/25678222 shutdown_connections 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.541+0000 7fc448fff700 1 --2- 172.21.15.7:0/25678222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc44406b1b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.541+0000 7fc448fff700 1 -- 172.21.15.7:0/25678222 >> 172.21.15.7:0/25678222 conn(0x7fc444072040 msgr2=0x7fc444072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:22.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.541+0000 7fc448fff700 1 -- 172.21.15.7:0/25678222 shutdown_connections 2023-06-09T11:11:22.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.541+0000 7fc448fff700 1 -- 172.21.15.7:0/25678222 wait complete. 2023-06-09T11:11:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.542+0000 7fc448fff700 1 Processor -- start 2023-06-09T11:11:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 -- start start 2023-06-09T11:11:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4440c64e0 0x7fc4440ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4441537a0 0x7fc4440cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cee70 con 0x7fc44415b9c0 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440c7960 con 0x7fc4441537a0 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.543+0000 7fc448fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440c7f20 con 0x7fc4440c64e0 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc441d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4441537a0 0x7fc4440cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc44259c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4440c64e0 0x7fc4440ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33826/0 (socket says 172.21.15.7:33826) 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 -- 172.21.15.7:0/874579233 learned_addr learned my addr 172.21.15.7:0/874579233 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 -- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4440c64e0 msgr2=0x7fc4440ca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4440c64e0 0x7fc4440ca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 -- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4441537a0 msgr2=0x7fc4440cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4441537a0 0x7fc4440cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.544+0000 7fc442d9d700 1 -- 172.21.15.7:0/874579233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009a70 con 0x7fc44415b9c0 2023-06-09T11:11:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc442d9d700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc4440d00e0 tx=0x7fc43c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c012430 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c020070 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4440c49b0 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c004280 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.545+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4440c46d0 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.546+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4440c4c90 con 0x7fc44415b9c0 2023-06-09T11:11:22.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.547+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc43c0043e0 con 0x7fc44415b9c0 2023-06-09T11:11:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.548+0000 7fc432ffd700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 0x7fc42406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.548+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc43c022070 con 0x7fc44415b9c0 2023-06-09T11:11:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.548+0000 7fc44259c700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 0x7fc42406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.549+0000 7fc44259c700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 0x7fc42406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc434007fd0 tx=0x7fc434007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.552+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc43c056190 con 0x7fc44415b9c0 2023-06-09T11:11:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.759+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4440c8090 con 0x7fc44415b9c0 2023-06-09T11:11:22.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.759+0000 7fc432ffd700 1 -- 172.21.15.7:0/874579233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc43c02c0a0 con 0x7fc44415b9c0 2023-06-09T11:11:22.766 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:11:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 msgr2=0x7fc42406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 0x7fc42406d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc434007fd0 tx=0x7fc434007c40 comp rx=0 tx=0).stop 2023-06-09T11:11:22.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 msgr2=0x7fc4440cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc4440d00e0 tx=0x7fc43c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 shutdown_connections 2023-06-09T11:11:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc44415b9c0 0x7fc4440cdb90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc42406ace0 0x7fc42406d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4440c64e0 0x7fc4440ca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 --2- 172.21.15.7:0/874579233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4441537a0 0x7fc4440cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 >> 172.21.15.7:0/874579233 conn(0x7fc444072040 msgr2=0x7fc444151d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 shutdown_connections 2023-06-09T11:11:22.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.762+0000 7fc448fff700 1 -- 172.21.15.7:0/874579233 wait complete. 2023-06-09T11:11:22.786 INFO:tasks.mds_thrash:Wait for all MDSs to reach steady state... 2023-06-09T11:11:22.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.979+0000 7f7966fff700 1 Processor -- start 2023-06-09T11:11:22.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.979+0000 7f7966fff700 1 -- start start 2023-06-09T11:11:22.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.980+0000 7f7966fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7960153880 0x7f796015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7966fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f796015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7966fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796015a500 con 0x7f796015b2d0 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7966fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79600c5f10 con 0x7f7960153880 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7966fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796015da00 con 0x7f79600c9f40 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7960153880 0x7f796015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7960153880 0x7f796015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33842/0 (socket says 172.21.15.7:33842) 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 -- 172.21.15.7:0/3520161264 learned_addr learned my addr 172.21.15.7:0/3520161264 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f795ffff700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f796015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 -- 172.21.15.7:0/3520161264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 msgr2=0x7f796015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f796015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 -- 172.21.15.7:0/3520161264 >> v1:172.21.15.117:6789/0 conn(0x7f796015b2d0 legacy=0x7f79600c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.981+0000 7f7964d9b700 1 -- 172.21.15.7:0/3520161264 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796015b750 con 0x7f7960153880 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f7964d9b700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7960153880 0x7f796015d4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79540098f0 tx=0x7f7954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff28ef9db49b1e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f795f7fe700 1 -- 172.21.15.7:0/3520161264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954011830 con 0x7f7960153880 2023-06-09T11:11:22.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f795f7fe700 1 -- 172.21.15.7:0/3520161264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954011990 con 0x7f7960153880 2023-06-09T11:11:22.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f795f7fe700 1 -- 172.21.15.7:0/3520161264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7954011d20 con 0x7f7960153880 2023-06-09T11:11:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f7966fff700 1 -- 172.21.15.7:0/3520161264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7960153880 msgr2=0x7f796015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.982+0000 7f7966fff700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7960153880 0x7f796015d4c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79540098f0 tx=0x7f7954009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 -- 172.21.15.7:0/3520161264 shutdown_connections 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7960153880 0x7f796015d4c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 --2- 172.21.15.7:0/3520161264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f796015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 -- 172.21.15.7:0/3520161264 >> 172.21.15.7:0/3520161264 conn(0x7f79600737f0 msgr2=0x7f79600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 -- 172.21.15.7:0/3520161264 shutdown_connections 2023-06-09T11:11:22.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.983+0000 7f7966fff700 1 -- 172.21.15.7:0/3520161264 wait complete. 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.984+0000 7f7966fff700 1 Processor -- start 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.984+0000 7f7966fff700 1 -- start start 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f79600c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 0x7f79600cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7960158360 con 0x7f796015b2d0 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79600c69d0 con 0x7f7960153880 2023-06-09T11:11:22.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.985+0000 7f7966fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79601f2730 con 0x7f79600c9f40 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33854/0 (socket says 172.21.15.7:33854) 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 -- 172.21.15.7:0/3727045375 learned_addr learned my addr 172.21.15.7:0/3727045375 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f7964d9b700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f79600c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f795ffff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 0x7f79600cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 -- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 msgr2=0x7f79600c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f79600c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 -- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 msgr2=0x7f79600cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 0x7f79600cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 -- 172.21.15.7:0/3727045375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7954009550 con 0x7f796015b2d0 2023-06-09T11:11:22.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f795ffff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 0x7f79600cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:22.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f796559c700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f795800cb50 tx=0x7f795800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958003c00 con 0x7f796015b2d0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958003d60 con 0x7f796015b2d0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.986+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800aac0 con 0x7f796015b2d0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.987+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7960159980 con 0x7f796015b2d0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.987+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79601596e0 con 0x7f796015b2d0 2023-06-09T11:11:22.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.988+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7960159c60 con 0x7f796015b2d0 2023-06-09T11:11:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.988+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f795800ac20 con 0x7f796015b2d0 2023-06-09T11:11:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.990+0000 7f795d7fa700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 0x7f794406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.990+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f79580892c0 con 0x7f796015b2d0 2023-06-09T11:11:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.990+0000 7f7964d9b700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 0x7f794406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.991+0000 7f7964d9b700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 0x7f794406d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7954009c40 tx=0x7f7954008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:22.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:22.994+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7958054370 con 0x7f796015b2d0 2023-06-09T11:11:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.246+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f796015a500 con 0x7f796015b2d0 2023-06-09T11:11:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.247+0000 7f795d7fa700 1 -- 172.21.15.7:0/3727045375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f796015a500 con 0x7f796015b2d0 2023-06-09T11:11:23.254 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 msgr2=0x7f794406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 0x7f794406d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7954009c40 tx=0x7f7954008030 comp rx=0 tx=0).stop 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 msgr2=0x7f79601f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f795800cb50 tx=0x7f795800ce60 comp rx=0 tx=0).stop 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 shutdown_connections 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f796015b2d0 0x7f79601f2b80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f794406ace0 0x7f794406d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79600c9f40 0x7f79600c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 --2- 172.21.15.7:0/3727045375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7960153880 0x7f79600cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 >> 172.21.15.7:0/3727045375 conn(0x7f79600737f0 msgr2=0x7f796015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 shutdown_connections 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.249+0000 7f7966fff700 1 -- 172.21.15.7:0/3727045375 wait complete. 2023-06-09T11:11:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:23.273 INFO:tasks.mds_thrash:Ready to start thrashing 2023-06-09T11:11:23.273 INFO:tasks.mds_thrash.ceph_manager:waiting for clean 2023-06-09T11:11:23.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:11:23.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.467+0000 7fc30d3ff700 1 Processor -- start 2023-06-09T11:11:23.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.467+0000 7fc30d3ff700 1 -- start start 2023-06-09T11:11:23.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc30d3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc30815b4a0 0x7fc30815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc30d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc30d3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30815a500 con 0x7fc308153880 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc30d3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3080c5f10 con 0x7fc30815b4a0 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc30d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc30815da00 con 0x7fc3080c9f40 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc3067fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc30815b4a0 0x7fc30815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc306ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc308153880 0x7fc30815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57388/0 (socket says 172.21.15.7:57388) 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc306ffd700 1 -- 172.21.15.7:0/2547572605 learned_addr learned my addr 172.21.15.7:0/2547572605 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:23.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc3077fe700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc3067fc700 1 -- 172.21.15.7:0/2547572605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 msgr2=0x7fc3080c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc3067fc700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.469+0000 7fc3067fc700 1 -- 172.21.15.7:0/2547572605 >> v1:172.21.15.7:6789/0 conn(0x7fc308153880 legacy=0x7fc30815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc3067fc700 1 -- 172.21.15.7:0/2547572605 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3081596e0 con 0x7fc30815b4a0 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc305ffb700 1 -- 172.21.15.7:0/2547572605 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593047472 0 0) 0x7fc2f8001110 con 0x7fc308153880 2023-06-09T11:11:23.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc3067fc700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30815b4a0 0x7fc30815d4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009840 tx=0x7fc2fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d346849a2851789 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc305ffb700 1 -- 172.21.15.7:0/2547572605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc011830 con 0x7fc30815b4a0 2023-06-09T11:11:23.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc305ffb700 1 -- 172.21.15.7:0/2547572605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc011990 con 0x7fc30815b4a0 2023-06-09T11:11:23.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.470+0000 7fc305ffb700 1 -- 172.21.15.7:0/2547572605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc011d20 con 0x7fc30815b4a0 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 -- 172.21.15.7:0/2547572605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30815b4a0 msgr2=0x7fc30815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30815b4a0 0x7fc30815d4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009840 tx=0x7fc2fc009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 -- 172.21.15.7:0/2547572605 shutdown_connections 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/2547572605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30815b4a0 0x7fc30815d4c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 -- 172.21.15.7:0/2547572605 >> 172.21.15.7:0/2547572605 conn(0x7fc3080737f0 msgr2=0x7fc3080720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:23.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 -- 172.21.15.7:0/2547572605 shutdown_connections 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.471+0000 7fc30d3ff700 1 -- 172.21.15.7:0/2547572605 wait complete. 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.472+0000 7fc30d3ff700 1 Processor -- start 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 -- start start 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 0x7fc308148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081eedd0 con 0x7fc308153880 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3081ef070 con 0x7fc30815b4a0 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.473+0000 7fc30d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3080cc0d0 con 0x7fc3080c9f40 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41406/0 (socket says 172.21.15.7:41406) 2023-06-09T11:11:23.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 -- 172.21.15.7:0/4216096413 learned_addr learned my addr 172.21.15.7:0/4216096413 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc306ffd700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3077fe700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 0x7fc308148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 -- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 msgr2=0x7fc3080cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 -- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 msgr2=0x7fc308148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 0x7fc308148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 -- 172.21.15.7:0/4216096413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0094a0 con 0x7fc308153880 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3077fe700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 0x7fc308148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.474+0000 7fc3067fc700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009fd0 tx=0x7fc2fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.475+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0038a0 con 0x7fc308153880 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.475+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc01f070 con 0x7fc308153880 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.475+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc003c40 con 0x7fc308153880 2023-06-09T11:11:23.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.475+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc308159c60 con 0x7fc308153880 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.475+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc308159980 con 0x7fc308153880 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.476+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc308159f40 con 0x7fc308153880 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.477+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2fc026480 con 0x7fc308153880 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.478+0000 7fc2ef7fe700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 0x7fc2f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.478+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc2fc021070 con 0x7fc308153880 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.478+0000 7fc306ffd700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 0x7fc2f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.478+0000 7fc306ffd700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 0x7fc2f406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000f60 tx=0x7fc2f8007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.482+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc2fc055dc0 con 0x7fc308153880 2023-06-09T11:11:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.686+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3081ef1e0 con 0x7fc2f406ace0 2023-06-09T11:11:23.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.695+0000 7fc2ef7fe700 1 -- 172.21.15.7:0/4216096413 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286774 (secure 0 0 0) 0x7fc3081ef1e0 con 0x7fc2f406ace0 2023-06-09T11:11:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 msgr2=0x7fc2f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 0x7fc2f406d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000f60 tx=0x7fc2f8007a40 comp rx=0 tx=0).stop 2023-06-09T11:11:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 msgr2=0x7fc3081ef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc009fd0 tx=0x7fc2fc004750 comp rx=0 tx=0).stop 2023-06-09T11:11:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 shutdown_connections 2023-06-09T11:11:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30815b4a0 0x7fc308148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2f406ace0 0x7fc2f406d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3080c9f40 0x7fc3080cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 --2- 172.21.15.7:0/4216096413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc308153880 0x7fc3081ef500 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 >> 172.21.15.7:0/4216096413 conn(0x7fc3080737f0 msgr2=0x7fc30815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 shutdown_connections 2023-06-09T11:11:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.699+0000 7fc30d3ff700 1 -- 172.21.15.7:0/4216096413 wait complete. 2023-06-09T11:11:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:11:23.723 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-06-09T11:11:22.196237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595887,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":286,"num_write_kb":2456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":68192,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":66240,"kb_avail":754906528,"statfs":{"total":773094113280,"available":773024284672,"internally_reserved":0,"allocated":1933312,"data_stored":1786252,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67829760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":441,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002169"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170937+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:11:06.170937+0000","last_peered":"2023-06-09T11:11:06.170937+0000","last_clean":"2023-06-09T11:11:06.170937+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:11:06.170937+0000","last_undegraded":"2023-06-09T11:11:06.170937+0000","last_fullsized":"2023-06-09T11:11:06.170937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.169478+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:11:06.169478+0000","last_peered":"2023-06-09T11:11:06.169478+0000","last_clean":"2023-06-09T11:11:06.169478+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:11:06.169478+0000","last_undegraded":"2023-06-09T11:11:06.169478+0000","last_fullsized":"2023-06-09T11:11:06.169478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259827+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:52.259827+0000","last_peered":"2023-06-09T11:10:52.259827+0000","last_clean":"2023-06-09T11:10:52.259827+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:52.259827+0000","last_undegraded":"2023-06-09T11:10:52.259827+0000","last_fullsized":"2023-06-09T11:10:52.259827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137650+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:11:01.137650+0000","last_peered":"2023-06-09T11:11:01.137650+0000","last_clean":"2023-06-09T11:11:01.137650+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:11:01.137650+0000","last_undegraded":"2023-06-09T11:11:01.137650+0000","last_fullsized":"2023-06-09T11:11:01.137650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134392+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:11:01.134392+0000","last_peered":"2023-06-09T11:11:01.134392+0000","last_clean":"2023-06-09T11:11:01.134392+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:11:01.134392+0000","last_undegraded":"2023-06-09T11:11:01.134392+0000","last_fullsized":"2023-06-09T11:11:01.134392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200221+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:55.200221+0000","last_peered":"2023-06-09T11:10:55.200221+0000","last_clean":"2023-06-09T11:10:55.200221+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:55.200221+0000","last_undegraded":"2023-06-09T11:10:55.200221+0000","last_fullsized":"2023-06-09T11:10:55.200221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.136176+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:11:01.136176+0000","last_peered":"2023-06-09T11:11:01.136176+0000","last_clean":"2023-06-09T11:11:01.136176+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:11:01.136176+0000","last_undegraded":"2023-06-09T11:11:01.136176+0000","last_fullsized":"2023-06-09T11:11:01.136176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171741+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:11:06.171741+0000","last_peered":"2023-06-09T11:11:06.171741+0000","last_clean":"2023-06-09T11:11:06.171741+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:11:06.171741+0000","last_undegraded":"2023-06-09T11:11:06.171741+0000","last_fullsized":"2023-06-09T11:11:06.171741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137302+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:11:01.137302+0000","last_peered":"2023-06-09T11:11:01.137302+0000","last_clean":"2023-06-09T11:11:01.137302+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:11:01.137302+0000","last_undegraded":"2023-06-09T11:11:01.137302+0000","last_fullsized":"2023-06-09T11:11:01.137302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174010+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:11:06.174010+0000","last_peered":"2023-06-09T11:11:06.174010+0000","last_clean":"2023-06-09T11:11:06.174010+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:11:06.174010+0000","last_undegraded":"2023-06-09T11:11:06.174010+0000","last_fullsized":"2023-06-09T11:11:06.174010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'15","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.172253+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:11:06.172253+0000","last_peered":"2023-06-09T11:11:06.172253+0000","last_clean":"2023-06-09T11:11:06.172253+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:11:06.172253+0000","last_undegraded":"2023-06-09T11:11:06.172253+0000","last_fullsized":"2023-06-09T11:11:06.172253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200205+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:55.200205+0000","last_peered":"2023-06-09T11:10:55.200205+0000","last_clean":"2023-06-09T11:10:55.200205+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:55.200205+0000","last_undegraded":"2023-06-09T11:10:55.200205+0000","last_fullsized":"2023-06-09T11:10:55.200205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199981+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:55.199981+0000","last_peered":"2023-06-09T11:10:55.199981+0000","last_clean":"2023-06-09T11:10:55.199981+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:55.199981+0000","last_undegraded":"2023-06-09T11:10:55.199981+0000","last_fullsized":"2023-06-09T11:10:55.199981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251545+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:52.251545+0000","last_peered":"2023-06-09T11:10:52.251545+0000","last_clean":"2023-06-09T11:10:52.251545+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:52.251545+0000","last_undegraded":"2023-06-09T11:10:52.251545+0000","last_fullsized":"2023-06-09T11:10:52.251545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170742+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:11:06.170742+0000","last_peered":"2023-06-09T11:11:06.170742+0000","last_clean":"2023-06-09T11:11:06.170742+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:11:06.170742+0000","last_undegraded":"2023-06-09T11:11:06.170742+0000","last_fullsized":"2023-06-09T11:11:06.170742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134150+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:11:01.134150+0000","last_peered":"2023-06-09T11:11:01.134150+0000","last_clean":"2023-06-09T11:11:01.134150+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:11:01.134150+0000","last_undegraded":"2023-06-09T11:11:01.134150+0000","last_fullsized":"2023-06-09T11:11:01.134150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.250804+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:52.250804+0000","last_peered":"2023-06-09T11:10:52.250804+0000","last_clean":"2023-06-09T11:10:52.250804+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:52.250804+0000","last_undegraded":"2023-06-09T11:10:52.250804+0000","last_fullsized":"2023-06-09T11:10:52.250804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135398+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:11:01.135398+0000","last_peered":"2023-06-09T11:11:01.135398+0000","last_clean":"2023-06-09T11:11:01.135398+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:11:01.135398+0000","last_undegraded":"2023-06-09T11:11:01.135398+0000","last_fullsized":"2023-06-09T11:11:01.135398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.133151+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:11:01.133151+0000","last_peered":"2023-06-09T11:11:01.133151+0000","last_clean":"2023-06-09T11:11:01.133151+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:11:01.133151+0000","last_undegraded":"2023-06-09T11:11:01.133151+0000","last_fullsized":"2023-06-09T11:11:01.133151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251095+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:52.251095+0000","last_peered":"2023-06-09T11:10:52.251095+0000","last_clean":"2023-06-09T11:10:52.251095+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:52.251095+0000","last_undegraded":"2023-06-09T11:10:52.251095+0000","last_fullsized":"2023-06-09T11:10:52.251095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135947+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:11:01.135947+0000","last_peered":"2023-06-09T11:11:01.135947+0000","last_clean":"2023-06-09T11:11:01.135947+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:11:01.135947+0000","last_undegraded":"2023-06-09T11:11:01.135947+0000","last_fullsized":"2023-06-09T11:11:01.135947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170165+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:11:06.170165+0000","last_peered":"2023-06-09T11:11:06.170165+0000","last_clean":"2023-06-09T11:11:06.170165+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:11:06.170165+0000","last_undegraded":"2023-06-09T11:11:06.170165+0000","last_fullsized":"2023-06-09T11:11:06.170165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259923+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:52.259923+0000","last_peered":"2023-06-09T11:10:52.259923+0000","last_clean":"2023-06-09T11:10:52.259923+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:52.259923+0000","last_undegraded":"2023-06-09T11:10:52.259923+0000","last_fullsized":"2023-06-09T11:10:52.259923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171980+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:11:06.171980+0000","last_peered":"2023-06-09T11:11:06.171980+0000","last_clean":"2023-06-09T11:11:06.171980+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:11:06.171980+0000","last_undegraded":"2023-06-09T11:11:06.171980+0000","last_fullsized":"2023-06-09T11:11:06.171980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134155+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:11:01.134155+0000","last_peered":"2023-06-09T11:11:01.134155+0000","last_clean":"2023-06-09T11:11:01.134155+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:11:01.134155+0000","last_undegraded":"2023-06-09T11:11:01.134155+0000","last_fullsized":"2023-06-09T11:11:01.134155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171372+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:11:06.171372+0000","last_peered":"2023-06-09T11:11:06.171372+0000","last_clean":"2023-06-09T11:11:06.171372+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:11:06.171372+0000","last_undegraded":"2023-06-09T11:11:06.171372+0000","last_fullsized":"2023-06-09T11:11:06.171372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.139116+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:11:01.139116+0000","last_peered":"2023-06-09T11:11:01.139116+0000","last_clean":"2023-06-09T11:11:01.139116+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:11:01.139116+0000","last_undegraded":"2023-06-09T11:11:01.139116+0000","last_fullsized":"2023-06-09T11:11:01.139116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198929+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:55.198929+0000","last_peered":"2023-06-09T11:10:55.198929+0000","last_clean":"2023-06-09T11:10:55.198929+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:55.198929+0000","last_undegraded":"2023-06-09T11:10:55.198929+0000","last_fullsized":"2023-06-09T11:10:55.198929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171753+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:11:06.171753+0000","last_peered":"2023-06-09T11:11:06.171753+0000","last_clean":"2023-06-09T11:11:06.171753+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:11:06.171753+0000","last_undegraded":"2023-06-09T11:11:06.171753+0000","last_fullsized":"2023-06-09T11:11:06.171753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.195108+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:55.195108+0000","last_peered":"2023-06-09T11:10:55.195108+0000","last_clean":"2023-06-09T11:10:55.195108+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:55.195108+0000","last_undegraded":"2023-06-09T11:10:55.195108+0000","last_fullsized":"2023-06-09T11:10:55.195108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'9","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174340+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:11:06.174340+0000","last_peered":"2023-06-09T11:11:06.174340+0000","last_clean":"2023-06-09T11:11:06.174340+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:11:06.174340+0000","last_undegraded":"2023-06-09T11:11:06.174340+0000","last_fullsized":"2023-06-09T11:11:06.174340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":5519,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8948,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362892,"statfs":{"total":96636764160,"available":96627601408,"internally_reserved":0,"allocated":307200,"data_stored":667690,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":212992,"data_stored":74532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8400,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363440,"statfs":{"total":96636764160,"available":96628162560,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9008,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362832,"statfs":{"total":96636764160,"available":96627539968,"internally_reserved":0,"allocated":303104,"data_stored":666270,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8276,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363564,"statfs":{"total":96636764160,"available":96628289536,"internally_reserved":0,"allocated":208896,"data_stored":74510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8416,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363424,"statfs":{"total":96636764160,"available":96628146176,"internally_reserved":0,"allocated":221184,"data_stored":75696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8444,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363396,"statfs":{"total":96636764160,"available":96628117504,"internally_reserved":0,"allocated":249856,"data_stored":77368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8356,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363484,"statfs":{"total":96636764160,"available":96628207616,"internally_reserved":0,"allocated":225280,"data_stored":75766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:11:23.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:11:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.957+0000 7f5799dff700 1 Processor -- start 2023-06-09T11:11:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.957+0000 7f5799dff700 1 -- start start 2023-06-09T11:11:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.958+0000 7f5799dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 0x7f57941599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.958+0000 7f5799dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.958+0000 7f5799dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57940c0bf0 con 0x7f57940c0140 2023-06-09T11:11:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.958+0000 7f5799dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5794070110 con 0x7f57940c1f90 2023-06-09T11:11:23.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.958+0000 7f5799dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5794071300 con 0x7f57940c1bc0 2023-06-09T11:11:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f5793fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f57937fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 0x7f57941599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f5793fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:60170/0 (socket says 172.21.15.7:60170) 2023-06-09T11:11:23.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f5793fff700 1 -- 172.21.15.7:0/760293400 learned_addr learned my addr 172.21.15.7:0/760293400 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f57937fe700 1 -- 172.21.15.7:0/760293400 >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 msgr2=0x7f5794159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f57937fe700 1 --2- 172.21.15.7:0/760293400 >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f57937fe700 1 -- 172.21.15.7:0/760293400 >> v1:172.21.15.117:6789/0 conn(0x7f57940c0140 legacy=0x7f57941501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f57937fe700 1 -- 172.21.15.7:0/760293400 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940702f0 con 0x7f57940c1bc0 2023-06-09T11:11:23.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f5793fff700 1 --2- 172.21.15.7:0/760293400 >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.959+0000 7f57927fc700 1 -- 172.21.15.7:0/760293400 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326554449 0 0) 0x7f57940c0bf0 con 0x7f57940c0140 2023-06-09T11:11:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f57937fe700 1 --2- 172.21.15.7:0/760293400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 0x7f57941599b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5784009840 tx=0x7f5784009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3485e897b4bbbd1d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f57927fc700 1 -- 172.21.15.7:0/760293400 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011830 con 0x7f57940c1bc0 2023-06-09T11:11:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f57927fc700 1 -- 172.21.15.7:0/760293400 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5784011990 con 0x7f57940c1bc0 2023-06-09T11:11:23.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.960+0000 7f57927fc700 1 -- 172.21.15.7:0/760293400 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784011d20 con 0x7f57940c1bc0 2023-06-09T11:11:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 -- 172.21.15.7:0/760293400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 msgr2=0x7f57941599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 --2- 172.21.15.7:0/760293400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 0x7f57941599b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5784009840 tx=0x7f5784009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:23.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 -- 172.21.15.7:0/760293400 shutdown_connections 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 --2- 172.21.15.7:0/760293400 >> v2:172.21.15.117:3300/0 conn(0x7f57940c1f90 0x7f5794159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 --2- 172.21.15.7:0/760293400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1bc0 0x7f57941599b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 -- 172.21.15.7:0/760293400 >> 172.21.15.7:0/760293400 conn(0x7f579406a4d0 msgr2=0x7f579406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 -- 172.21.15.7:0/760293400 shutdown_connections 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.961+0000 7f5799dff700 1 -- 172.21.15.7:0/760293400 wait complete. 2023-06-09T11:11:23.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.962+0000 7f5799dff700 1 Processor -- start 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 -- start start 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57940c1bc0 0x7f57941452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1f90 0x7f57941df470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940ca2c0 con 0x7f57940c0140 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940ca5a0 con 0x7f57940c1bc0 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.963+0000 7f5799dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940ca880 con 0x7f57940c1f90 2023-06-09T11:11:23.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41410/0 (socket says 172.21.15.7:41410) 2023-06-09T11:11:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 -- 172.21.15.7:0/2058463221 learned_addr learned my addr 172.21.15.7:0/2058463221 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f5793fff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1f90 0x7f57941df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f5792ffd700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57940c1bc0 0x7f57941452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 -- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1f90 msgr2=0x7f57941df470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1f90 0x7f57941df470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 -- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57940c1bc0 msgr2=0x7f57941452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57940c1bc0 0x7f57941452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:23.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.964+0000 7f57937fe700 1 -- 172.21.15.7:0/2058463221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c00bd40 con 0x7f57940c0140 2023-06-09T11:11:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f57937fe700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fd0 tx=0x7f5784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57840036a0 con 0x7f57940c0140 2023-06-09T11:11:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578401f070 con 0x7f57940c0140 2023-06-09T11:11:23.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5784003a20 con 0x7f57940c0140 2023-06-09T11:11:23.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57840094a0 con 0x7f57940c0140 2023-06-09T11:11:23.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.965+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57940c1170 con 0x7f57940c0140 2023-06-09T11:11:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.966+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57940c0e90 con 0x7f57940c0140 2023-06-09T11:11:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.967+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5784003b80 con 0x7f57940c0140 2023-06-09T11:11:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.968+0000 7f577bfff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 0x7f578006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.968+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5784021070 con 0x7f57940c0140 2023-06-09T11:11:23.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.968+0000 7f5792ffd700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 0x7f578006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.969+0000 7f5792ffd700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 0x7f578006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57940cb570 tx=0x7f5788009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:23.972+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5784055e70 con 0x7f57940c0140 2023-06-09T11:11:24.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.181+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57940c1450 con 0x7f578006ace0 2023-06-09T11:11:24.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.189+0000 7f577bfff700 1 -- 172.21.15.7:0/2058463221 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286774 (secure 0 0 0) 0x7f57940c1450 con 0x7f578006ace0 2023-06-09T11:11:24.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:24.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 msgr2=0x7f578006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 0x7f578006d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f57940cb570 tx=0x7f5788009040 comp rx=0 tx=0).stop 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 msgr2=0x7f5794144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5784009fd0 tx=0x7f5784004750 comp rx=0 tx=0).stop 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 shutdown_connections 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f578006ace0 0x7f578006d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57940c1f90 0x7f57941df470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57940c0140 0x7f5794144db0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 --2- 172.21.15.7:0/2058463221 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57940c1bc0 0x7f57941452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 >> 172.21.15.7:0/2058463221 conn(0x7f579406a4d0 msgr2=0x7f579414d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.193+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 shutdown_connections 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.194+0000 7f5799dff700 1 -- 172.21.15.7:0/2058463221 wait complete. 2023-06-09T11:11:24.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:11:24.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-06-09T11:11:22.196237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595887,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":286,"num_write_kb":2456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":68192,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":66240,"kb_avail":754906528,"statfs":{"total":773094113280,"available":773024284672,"internally_reserved":0,"allocated":1933312,"data_stored":1786252,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":67829760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":441,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002169"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170937+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:11:06.170937+0000","last_peered":"2023-06-09T11:11:06.170937+0000","last_clean":"2023-06-09T11:11:06.170937+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:11:06.170937+0000","last_undegraded":"2023-06-09T11:11:06.170937+0000","last_fullsized":"2023-06-09T11:11:06.170937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.169478+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:11:06.169478+0000","last_peered":"2023-06-09T11:11:06.169478+0000","last_clean":"2023-06-09T11:11:06.169478+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:11:06.169478+0000","last_undegraded":"2023-06-09T11:11:06.169478+0000","last_fullsized":"2023-06-09T11:11:06.169478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259827+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:52.259827+0000","last_peered":"2023-06-09T11:10:52.259827+0000","last_clean":"2023-06-09T11:10:52.259827+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:52.259827+0000","last_undegraded":"2023-06-09T11:10:52.259827+0000","last_fullsized":"2023-06-09T11:10:52.259827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137650+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:11:01.137650+0000","last_peered":"2023-06-09T11:11:01.137650+0000","last_clean":"2023-06-09T11:11:01.137650+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:11:01.137650+0000","last_undegraded":"2023-06-09T11:11:01.137650+0000","last_fullsized":"2023-06-09T11:11:01.137650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134392+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:11:01.134392+0000","last_peered":"2023-06-09T11:11:01.134392+0000","last_clean":"2023-06-09T11:11:01.134392+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:11:01.134392+0000","last_undegraded":"2023-06-09T11:11:01.134392+0000","last_fullsized":"2023-06-09T11:11:01.134392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200221+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:55.200221+0000","last_peered":"2023-06-09T11:10:55.200221+0000","last_clean":"2023-06-09T11:10:55.200221+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:55.200221+0000","last_undegraded":"2023-06-09T11:10:55.200221+0000","last_fullsized":"2023-06-09T11:10:55.200221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'8","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.136176+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:11:01.136176+0000","last_peered":"2023-06-09T11:11:01.136176+0000","last_clean":"2023-06-09T11:11:01.136176+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:11:01.136176+0000","last_undegraded":"2023-06-09T11:11:01.136176+0000","last_fullsized":"2023-06-09T11:11:01.136176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171741+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:11:06.171741+0000","last_peered":"2023-06-09T11:11:06.171741+0000","last_clean":"2023-06-09T11:11:06.171741+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:11:06.171741+0000","last_undegraded":"2023-06-09T11:11:06.171741+0000","last_fullsized":"2023-06-09T11:11:06.171741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137302+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:11:01.137302+0000","last_peered":"2023-06-09T11:11:01.137302+0000","last_clean":"2023-06-09T11:11:01.137302+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:11:01.137302+0000","last_undegraded":"2023-06-09T11:11:01.137302+0000","last_fullsized":"2023-06-09T11:11:01.137302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174010+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:11:06.174010+0000","last_peered":"2023-06-09T11:11:06.174010+0000","last_clean":"2023-06-09T11:11:06.174010+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:11:06.174010+0000","last_undegraded":"2023-06-09T11:11:06.174010+0000","last_fullsized":"2023-06-09T11:11:06.174010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'15","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.172253+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:11:06.172253+0000","last_peered":"2023-06-09T11:11:06.172253+0000","last_clean":"2023-06-09T11:11:06.172253+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:11:06.172253+0000","last_undegraded":"2023-06-09T11:11:06.172253+0000","last_fullsized":"2023-06-09T11:11:06.172253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200205+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:55.200205+0000","last_peered":"2023-06-09T11:10:55.200205+0000","last_clean":"2023-06-09T11:10:55.200205+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:55.200205+0000","last_undegraded":"2023-06-09T11:10:55.200205+0000","last_fullsized":"2023-06-09T11:10:55.200205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199981+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:55.199981+0000","last_peered":"2023-06-09T11:10:55.199981+0000","last_clean":"2023-06-09T11:10:55.199981+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:55.199981+0000","last_undegraded":"2023-06-09T11:10:55.199981+0000","last_fullsized":"2023-06-09T11:10:55.199981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251545+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:52.251545+0000","last_peered":"2023-06-09T11:10:52.251545+0000","last_clean":"2023-06-09T11:10:52.251545+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:52.251545+0000","last_undegraded":"2023-06-09T11:10:52.251545+0000","last_fullsized":"2023-06-09T11:10:52.251545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170742+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:11:06.170742+0000","last_peered":"2023-06-09T11:11:06.170742+0000","last_clean":"2023-06-09T11:11:06.170742+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:11:06.170742+0000","last_undegraded":"2023-06-09T11:11:06.170742+0000","last_fullsized":"2023-06-09T11:11:06.170742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134150+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:11:01.134150+0000","last_peered":"2023-06-09T11:11:01.134150+0000","last_clean":"2023-06-09T11:11:01.134150+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:11:01.134150+0000","last_undegraded":"2023-06-09T11:11:01.134150+0000","last_fullsized":"2023-06-09T11:11:01.134150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.250804+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:52.250804+0000","last_peered":"2023-06-09T11:10:52.250804+0000","last_clean":"2023-06-09T11:10:52.250804+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:52.250804+0000","last_undegraded":"2023-06-09T11:10:52.250804+0000","last_fullsized":"2023-06-09T11:10:52.250804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135398+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:11:01.135398+0000","last_peered":"2023-06-09T11:11:01.135398+0000","last_clean":"2023-06-09T11:11:01.135398+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:11:01.135398+0000","last_undegraded":"2023-06-09T11:11:01.135398+0000","last_fullsized":"2023-06-09T11:11:01.135398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.133151+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:11:01.133151+0000","last_peered":"2023-06-09T11:11:01.133151+0000","last_clean":"2023-06-09T11:11:01.133151+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:11:01.133151+0000","last_undegraded":"2023-06-09T11:11:01.133151+0000","last_fullsized":"2023-06-09T11:11:01.133151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251095+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:52.251095+0000","last_peered":"2023-06-09T11:10:52.251095+0000","last_clean":"2023-06-09T11:10:52.251095+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:52.251095+0000","last_undegraded":"2023-06-09T11:10:52.251095+0000","last_fullsized":"2023-06-09T11:10:52.251095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135947+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:11:01.135947+0000","last_peered":"2023-06-09T11:11:01.135947+0000","last_clean":"2023-06-09T11:11:01.135947+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:11:01.135947+0000","last_undegraded":"2023-06-09T11:11:01.135947+0000","last_fullsized":"2023-06-09T11:11:01.135947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170165+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:11:06.170165+0000","last_peered":"2023-06-09T11:11:06.170165+0000","last_clean":"2023-06-09T11:11:06.170165+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:11:06.170165+0000","last_undegraded":"2023-06-09T11:11:06.170165+0000","last_fullsized":"2023-06-09T11:11:06.170165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259923+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:52.259923+0000","last_peered":"2023-06-09T11:10:52.259923+0000","last_clean":"2023-06-09T11:10:52.259923+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:52.259923+0000","last_undegraded":"2023-06-09T11:10:52.259923+0000","last_fullsized":"2023-06-09T11:10:52.259923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171980+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:11:06.171980+0000","last_peered":"2023-06-09T11:11:06.171980+0000","last_clean":"2023-06-09T11:11:06.171980+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:11:06.171980+0000","last_undegraded":"2023-06-09T11:11:06.171980+0000","last_fullsized":"2023-06-09T11:11:06.171980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134155+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:11:01.134155+0000","last_peered":"2023-06-09T11:11:01.134155+0000","last_clean":"2023-06-09T11:11:01.134155+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:11:01.134155+0000","last_undegraded":"2023-06-09T11:11:01.134155+0000","last_fullsized":"2023-06-09T11:11:01.134155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171372+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:11:06.171372+0000","last_peered":"2023-06-09T11:11:06.171372+0000","last_clean":"2023-06-09T11:11:06.171372+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:11:06.171372+0000","last_undegraded":"2023-06-09T11:11:06.171372+0000","last_fullsized":"2023-06-09T11:11:06.171372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.139116+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:11:01.139116+0000","last_peered":"2023-06-09T11:11:01.139116+0000","last_clean":"2023-06-09T11:11:01.139116+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:11:01.139116+0000","last_undegraded":"2023-06-09T11:11:01.139116+0000","last_fullsized":"2023-06-09T11:11:01.139116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198929+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:55.198929+0000","last_peered":"2023-06-09T11:10:55.198929+0000","last_clean":"2023-06-09T11:10:55.198929+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:55.198929+0000","last_undegraded":"2023-06-09T11:10:55.198929+0000","last_fullsized":"2023-06-09T11:10:55.198929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171753+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:11:06.171753+0000","last_peered":"2023-06-09T11:11:06.171753+0000","last_clean":"2023-06-09T11:11:06.171753+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:11:06.171753+0000","last_undegraded":"2023-06-09T11:11:06.171753+0000","last_fullsized":"2023-06-09T11:11:06.171753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.195108+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:55.195108+0000","last_peered":"2023-06-09T11:10:55.195108+0000","last_clean":"2023-06-09T11:10:55.195108+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:55.195108+0000","last_undegraded":"2023-06-09T11:10:55.195108+0000","last_fullsized":"2023-06-09T11:10:55.195108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'9","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174340+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:11:06.174340+0000","last_peered":"2023-06-09T11:11:06.174340+0000","last_clean":"2023-06-09T11:11:06.174340+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:11:06.174340+0000","last_undegraded":"2023-06-09T11:11:06.174340+0000","last_fullsized":"2023-06-09T11:11:06.174340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":5519,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":127,"num_write_kb":40,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":11038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8948,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362892,"statfs":{"total":96636764160,"available":96627601408,"internally_reserved":0,"allocated":307200,"data_stored":667690,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":212992,"data_stored":74532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8400,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363440,"statfs":{"total":96636764160,"available":96628162560,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9008,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362832,"statfs":{"total":96636764160,"available":96627539968,"internally_reserved":0,"allocated":303104,"data_stored":666270,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8276,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363564,"statfs":{"total":96636764160,"available":96628289536,"internally_reserved":0,"allocated":208896,"data_stored":74510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8416,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363424,"statfs":{"total":96636764160,"available":96628146176,"internally_reserved":0,"allocated":221184,"data_stored":75696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8444,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363396,"statfs":{"total":96636764160,"available":96628117504,"internally_reserved":0,"allocated":249856,"data_stored":77368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8356,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363484,"statfs":{"total":96636764160,"available":96628207616,"internally_reserved":0,"allocated":225280,"data_stored":75766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:11:24.225 INFO:tasks.mds_thrash.ceph_manager:clean! 2023-06-09T11:11:24.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:11:24.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.453+0000 7f0f1edff700 1 Processor -- start 2023-06-09T11:11:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.453+0000 7f0f1edff700 1 -- start start 2023-06-09T11:11:24.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1edff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f180c1bc0 0x7f0f181599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1edff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 0x7f0f18159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1edff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f180c0bf0 con 0x7f0f180c0140 2023-06-09T11:11:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1edff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f18070110 con 0x7f0f180c1bc0 2023-06-09T11:11:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1edff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f18071300 con 0x7f0f180c1f90 2023-06-09T11:11:24.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1cb9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f180c1bc0 0x7f0f181599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f17fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0f180c0140 0x7f0f181501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57402/0 (socket says 172.21.15.7:57402) 2023-06-09T11:11:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f1cb9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f180c1bc0 0x7f0f181599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41414/0 (socket says 172.21.15.7:41414) 2023-06-09T11:11:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.454+0000 7f0f17fff700 1 -- 172.21.15.7:0/962596514 learned_addr learned my addr 172.21.15.7:0/962596514 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:24.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1d39c700 1 --2- 172.21.15.7:0/962596514 >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 0x7f0f18159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f177fe700 1 -- 172.21.15.7:0/962596514 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887748050 0 0) 0x7f0f180c0bf0 con 0x7f0f180c0140 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f177fe700 1 -- 172.21.15.7:0/962596514 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f080035b0 con 0x7f0f180c0140 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/962596514 >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 msgr2=0x7f0f18159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1cb9b700 1 --2- 172.21.15.7:0/962596514 >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 0x7f0f18159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/962596514 >> v1:172.21.15.7:6789/0 conn(0x7f0f180c0140 legacy=0x7f0f181501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/962596514 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180702f0 con 0x7f0f180c1bc0 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1d39c700 1 --2- 172.21.15.7:0/962596514 >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 0x7f0f18159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:24.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f1cb9b700 1 --2- 172.21.15.7:0/962596514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c1bc0 0x7f0f181599b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f18071ae0 tx=0x7f0f0c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cef43468cb2767 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f177fe700 1 -- 172.21.15.7:0/962596514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c011830 con 0x7f0f180c1bc0 2023-06-09T11:11:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f177fe700 1 -- 172.21.15.7:0/962596514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c011990 con 0x7f0f180c1bc0 2023-06-09T11:11:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.455+0000 7f0f177fe700 1 -- 172.21.15.7:0/962596514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c011d20 con 0x7f0f180c1bc0 2023-06-09T11:11:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 -- 172.21.15.7:0/962596514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c1bc0 msgr2=0x7f0f181599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 --2- 172.21.15.7:0/962596514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c1bc0 0x7f0f181599b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0f18071ae0 tx=0x7f0f0c009b20 comp rx=0 tx=0).stop 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 -- 172.21.15.7:0/962596514 shutdown_connections 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 --2- 172.21.15.7:0/962596514 >> v2:172.21.15.117:3300/0 conn(0x7f0f180c1f90 0x7f0f18159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 --2- 172.21.15.7:0/962596514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c1bc0 0x7f0f181599b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 -- 172.21.15.7:0/962596514 >> 172.21.15.7:0/962596514 conn(0x7f0f1806a4d0 msgr2=0x7f0f1806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 -- 172.21.15.7:0/962596514 shutdown_connections 2023-06-09T11:11:24.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.456+0000 7f0f1edff700 1 -- 172.21.15.7:0/962596514 wait complete. 2023-06-09T11:11:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.458+0000 7f0f1edff700 1 Processor -- start 2023-06-09T11:11:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.458+0000 7f0f1edff700 1 -- start start 2023-06-09T11:11:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 0x7f0f1814ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 0x7f0f1814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f181f2f70 con 0x7f0f180c0140 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180ca9f0 con 0x7f0f180c1f90 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f180c98d0 con 0x7f0f180c1bc0 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1cb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1cb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41428/0 (socket says 172.21.15.7:41428) 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/2897343047 learned_addr learned my addr 172.21.15.7:0/2897343047 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f17fff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 0x7f0f1814ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1d39c700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 0x7f0f1814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.459+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 msgr2=0x7f0f1814ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1cb9b700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 0x7f0f1814ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 msgr2=0x7f0f1814f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1cb9b700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 0x7f0f1814f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1cb9b700 1 -- 172.21.15.7:0/2897343047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c0094a0 con 0x7f0f180c0140 2023-06-09T11:11:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1d39c700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 0x7f0f1814f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f17fff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 0x7f0f1814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1cb9b700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0090b0 tx=0x7f0f0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0036a0 con 0x7f0f180c0140 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0c01f070 con 0x7f0f180c0140 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f180c1170 con 0x7f0f180c0140 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c003a40 con 0x7f0f180c0140 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.460+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f180c0e90 con 0x7f0f180c0140 2023-06-09T11:11:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.461+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f180c1450 con 0x7f0f180c0140 2023-06-09T11:11:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.462+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f0c003ba0 con 0x7f0f180c0140 2023-06-09T11:11:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.463+0000 7f0f157fa700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 0x7f0efc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.463+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f0c021070 con 0x7f0f180c0140 2023-06-09T11:11:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.463+0000 7f0f17fff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 0x7f0efc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.464+0000 7f0f17fff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 0x7f0efc06d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f180c7010 tx=0x7f0f04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.468+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0f0c055e60 con 0x7f0f180c0140 2023-06-09T11:11:24.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.674+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f18074cf0 con 0x7f0efc06ace0 2023-06-09T11:11:24.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.682+0000 7f0f157fa700 1 -- 172.21.15.7:0/2897343047 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286726 (secure 0 0 0) 0x7f0f18074cf0 con 0x7f0efc06ace0 2023-06-09T11:11:24.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 msgr2=0x7f0efc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 0x7f0efc06d190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0f180c7010 tx=0x7f0f04009040 comp rx=0 tx=0).stop 2023-06-09T11:11:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 msgr2=0x7f0f180c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0090b0 tx=0x7f0f0c004750 comp rx=0 tx=0).stop 2023-06-09T11:11:24.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 shutdown_connections 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f180c1f90 0x7f0f1814f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f180c0140 0x7f0f180c9bb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0efc06ace0 0x7f0efc06d190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 --2- 172.21.15.7:0/2897343047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f180c1bc0 0x7f0f1814ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.686+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 >> 172.21.15.7:0/2897343047 conn(0x7f0f1806a4d0 msgr2=0x7f0f180c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.687+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 shutdown_connections 2023-06-09T11:11:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.687+0000 7f0f1edff700 1 -- 172.21.15.7:0/2897343047 wait complete. 2023-06-09T11:11:24.695 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:11:24.710 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-06-09T11:11:24.196629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597143,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":289,"num_write_kb":2459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":68384,"kb_used_data":1888,"kb_used_omap":0,"kb_used_meta":66432,"kb_avail":754906336,"statfs":{"total":773094113280,"available":773024088064,"internally_reserved":0,"allocated":1933312,"data_stored":1789646,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":68026368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1697,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002259"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981047+0000","last_change":"2023-06-09T11:10:50.214139+0000","last_active":"2023-06-09T11:10:51.981047+0000","last_peered":"2023-06-09T11:10:51.981047+0000","last_clean":"2023-06-09T11:10:51.981047+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:10:51.981047+0000","last_undegraded":"2023-06-09T11:10:51.981047+0000","last_fullsized":"2023-06-09T11:10:51.981047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:34:25.935483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189338+0000","last_change":"2023-06-09T11:10:50.203878+0000","last_active":"2023-06-09T11:10:51.189338+0000","last_peered":"2023-06-09T11:10:51.189338+0000","last_clean":"2023-06-09T11:10:51.189338+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:10:51.189338+0000","last_undegraded":"2023-06-09T11:10:51.189338+0000","last_fullsized":"2023-06-09T11:10:51.189338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:38.924075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189829+0000","last_change":"2023-06-09T11:10:50.198316+0000","last_active":"2023-06-09T11:10:51.189829+0000","last_peered":"2023-06-09T11:10:51.189829+0000","last_clean":"2023-06-09T11:10:51.189829+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:10:51.189829+0000","last_undegraded":"2023-06-09T11:10:51.189829+0000","last_fullsized":"2023-06-09T11:10:51.189829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:02:46.559031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197982+0000","last_change":"2023-06-09T11:10:50.202426+0000","last_active":"2023-06-09T11:10:51.197982+0000","last_peered":"2023-06-09T11:10:51.197982+0000","last_clean":"2023-06-09T11:10:51.197982+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:10:51.197982+0000","last_undegraded":"2023-06-09T11:10:51.197982+0000","last_fullsized":"2023-06-09T11:10:51.197982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:17.911344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547998+0000","last_change":"2023-06-09T11:10:50.215708+0000","last_active":"2023-06-09T11:10:51.547998+0000","last_peered":"2023-06-09T11:10:51.547998+0000","last_clean":"2023-06-09T11:10:51.547998+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:10:51.547998+0000","last_undegraded":"2023-06-09T11:10:51.547998+0000","last_fullsized":"2023-06-09T11:10:51.547998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:41:48.069613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189242+0000","last_change":"2023-06-09T11:10:50.197031+0000","last_active":"2023-06-09T11:10:51.189242+0000","last_peered":"2023-06-09T11:10:51.189242+0000","last_clean":"2023-06-09T11:10:51.189242+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:10:51.189242+0000","last_undegraded":"2023-06-09T11:10:51.189242+0000","last_fullsized":"2023-06-09T11:10:51.189242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:07.193459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689932+0000","last_change":"2023-06-09T11:10:50.210370+0000","last_active":"2023-06-09T11:10:51.689932+0000","last_peered":"2023-06-09T11:10:51.689932+0000","last_clean":"2023-06-09T11:10:51.689932+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:10:51.689932+0000","last_undegraded":"2023-06-09T11:10:51.689932+0000","last_fullsized":"2023-06-09T11:10:51.689932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:18:13.331321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189750+0000","last_change":"2023-06-09T11:10:50.200284+0000","last_active":"2023-06-09T11:10:51.189750+0000","last_peered":"2023-06-09T11:10:51.189750+0000","last_clean":"2023-06-09T11:10:51.189750+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:10:51.189750+0000","last_undegraded":"2023-06-09T11:10:51.189750+0000","last_fullsized":"2023-06-09T11:10:51.189750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:45:55.452752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548018+0000","last_change":"2023-06-09T11:10:50.216091+0000","last_active":"2023-06-09T11:10:51.548018+0000","last_peered":"2023-06-09T11:10:51.548018+0000","last_clean":"2023-06-09T11:10:51.548018+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:10:51.548018+0000","last_undegraded":"2023-06-09T11:10:51.548018+0000","last_fullsized":"2023-06-09T11:10:51.548018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:32.072919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189623+0000","last_change":"2023-06-09T11:10:50.202660+0000","last_active":"2023-06-09T11:10:51.189623+0000","last_peered":"2023-06-09T11:10:51.189623+0000","last_clean":"2023-06-09T11:10:51.189623+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:10:51.189623+0000","last_undegraded":"2023-06-09T11:10:51.189623+0000","last_fullsized":"2023-06-09T11:10:51.189623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:33:51.862803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189969+0000","last_change":"2023-06-09T11:10:50.208411+0000","last_active":"2023-06-09T11:10:51.189969+0000","last_peered":"2023-06-09T11:10:51.189969+0000","last_clean":"2023-06-09T11:10:51.189969+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:10:51.189969+0000","last_undegraded":"2023-06-09T11:10:51.189969+0000","last_fullsized":"2023-06-09T11:10:51.189969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:59:46.187552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981121+0000","last_change":"2023-06-09T11:10:50.199404+0000","last_active":"2023-06-09T11:10:51.981121+0000","last_peered":"2023-06-09T11:10:51.981121+0000","last_clean":"2023-06-09T11:10:51.981121+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:10:51.981121+0000","last_undegraded":"2023-06-09T11:10:51.981121+0000","last_fullsized":"2023-06-09T11:10:51.981121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:59:10.806517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981012+0000","last_change":"2023-06-09T11:10:50.207845+0000","last_active":"2023-06-09T11:10:51.981012+0000","last_peered":"2023-06-09T11:10:51.981012+0000","last_clean":"2023-06-09T11:10:51.981012+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:10:51.981012+0000","last_undegraded":"2023-06-09T11:10:51.981012+0000","last_fullsized":"2023-06-09T11:10:51.981012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:06:52.229948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189240+0000","last_change":"2023-06-09T11:10:50.196552+0000","last_active":"2023-06-09T11:10:51.189240+0000","last_peered":"2023-06-09T11:10:51.189240+0000","last_clean":"2023-06-09T11:10:51.189240+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:10:51.189240+0000","last_undegraded":"2023-06-09T11:10:51.189240+0000","last_fullsized":"2023-06-09T11:10:51.189240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:39.243661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690080+0000","last_change":"2023-06-09T11:10:50.210103+0000","last_active":"2023-06-09T11:10:51.690080+0000","last_peered":"2023-06-09T11:10:51.690080+0000","last_clean":"2023-06-09T11:10:51.690080+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:10:51.690080+0000","last_undegraded":"2023-06-09T11:10:51.690080+0000","last_fullsized":"2023-06-09T11:10:51.690080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:54:34.158225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188825+0000","last_change":"2023-06-09T11:10:50.198128+0000","last_active":"2023-06-09T11:10:51.188825+0000","last_peered":"2023-06-09T11:10:51.188825+0000","last_clean":"2023-06-09T11:10:51.188825+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:10:51.188825+0000","last_undegraded":"2023-06-09T11:10:51.188825+0000","last_fullsized":"2023-06-09T11:10:51.188825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:58.514710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548154+0000","last_change":"2023-06-09T11:10:50.215541+0000","last_active":"2023-06-09T11:10:51.548154+0000","last_peered":"2023-06-09T11:10:51.548154+0000","last_clean":"2023-06-09T11:10:51.548154+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:10:51.548154+0000","last_undegraded":"2023-06-09T11:10:51.548154+0000","last_fullsized":"2023-06-09T11:10:51.548154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:54.027446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189783+0000","last_change":"2023-06-09T11:10:50.198990+0000","last_active":"2023-06-09T11:10:51.189783+0000","last_peered":"2023-06-09T11:10:51.189783+0000","last_clean":"2023-06-09T11:10:51.189783+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:10:51.189783+0000","last_undegraded":"2023-06-09T11:10:51.189783+0000","last_fullsized":"2023-06-09T11:10:51.189783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:32.394020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981060+0000","last_change":"2023-06-09T11:10:50.205294+0000","last_active":"2023-06-09T11:10:51.981060+0000","last_peered":"2023-06-09T11:10:51.981060+0000","last_clean":"2023-06-09T11:10:51.981060+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:10:51.981060+0000","last_undegraded":"2023-06-09T11:10:51.981060+0000","last_fullsized":"2023-06-09T11:10:51.981060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:21:47.975170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189962+0000","last_change":"2023-06-09T11:10:50.206220+0000","last_active":"2023-06-09T11:10:51.189962+0000","last_peered":"2023-06-09T11:10:51.189962+0000","last_clean":"2023-06-09T11:10:51.189962+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:10:51.189962+0000","last_undegraded":"2023-06-09T11:10:51.189962+0000","last_fullsized":"2023-06-09T11:10:51.189962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:28:56.411185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.690048+0000","last_change":"2023-06-09T11:10:50.213572+0000","last_active":"2023-06-09T11:10:51.690048+0000","last_peered":"2023-06-09T11:10:51.690048+0000","last_clean":"2023-06-09T11:10:51.690048+0000","last_became_active":"2023-06-09T11:10:50.213193+0000","last_became_peered":"2023-06-09T11:10:50.213193+0000","last_unstale":"2023-06-09T11:10:51.690048+0000","last_undegraded":"2023-06-09T11:10:51.690048+0000","last_fullsized":"2023-06-09T11:10:51.690048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:15:59.628623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547743+0000","last_change":"2023-06-09T11:10:50.210372+0000","last_active":"2023-06-09T11:10:51.547743+0000","last_peered":"2023-06-09T11:10:51.547743+0000","last_clean":"2023-06-09T11:10:51.547743+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:10:51.547743+0000","last_undegraded":"2023-06-09T11:10:51.547743+0000","last_fullsized":"2023-06-09T11:10:51.547743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:29:59.026877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461921+0000","last_change":"2023-06-09T11:10:50.203962+0000","last_active":"2023-06-09T11:10:51.461921+0000","last_peered":"2023-06-09T11:10:51.461921+0000","last_clean":"2023-06-09T11:10:51.461921+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:10:51.461921+0000","last_undegraded":"2023-06-09T11:10:51.461921+0000","last_fullsized":"2023-06-09T11:10:51.461921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:51:10.533627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689723+0000","last_change":"2023-06-09T11:10:50.211596+0000","last_active":"2023-06-09T11:10:51.689723+0000","last_peered":"2023-06-09T11:10:51.689723+0000","last_clean":"2023-06-09T11:10:51.689723+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:10:51.689723+0000","last_undegraded":"2023-06-09T11:10:51.689723+0000","last_fullsized":"2023-06-09T11:10:51.689723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:36:47.319739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186005+0000","last_change":"2023-06-09T11:10:50.204688+0000","last_active":"2023-06-09T11:10:51.186005+0000","last_peered":"2023-06-09T11:10:51.186005+0000","last_clean":"2023-06-09T11:10:51.186005+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:10:51.186005+0000","last_undegraded":"2023-06-09T11:10:51.186005+0000","last_fullsized":"2023-06-09T11:10:51.186005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:53:26.427452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186244+0000","last_change":"2023-06-09T11:10:50.203098+0000","last_active":"2023-06-09T11:10:51.186244+0000","last_peered":"2023-06-09T11:10:51.186244+0000","last_clean":"2023-06-09T11:10:51.186244+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:10:51.186244+0000","last_undegraded":"2023-06-09T11:10:51.186244+0000","last_fullsized":"2023-06-09T11:10:51.186244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:04:28.887986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189397+0000","last_change":"2023-06-09T11:10:50.199635+0000","last_active":"2023-06-09T11:10:51.189397+0000","last_peered":"2023-06-09T11:10:51.189397+0000","last_clean":"2023-06-09T11:10:51.189397+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:10:51.189397+0000","last_undegraded":"2023-06-09T11:10:51.189397+0000","last_fullsized":"2023-06-09T11:10:51.189397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:28:39.156310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170937+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:11:06.170937+0000","last_peered":"2023-06-09T11:11:06.170937+0000","last_clean":"2023-06-09T11:11:06.170937+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:11:06.170937+0000","last_undegraded":"2023-06-09T11:11:06.170937+0000","last_fullsized":"2023-06-09T11:11:06.170937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:45:04.488032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461971+0000","last_change":"2023-06-09T11:10:50.218400+0000","last_active":"2023-06-09T11:10:51.461971+0000","last_peered":"2023-06-09T11:10:51.461971+0000","last_clean":"2023-06-09T11:10:51.461971+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:10:51.461971+0000","last_undegraded":"2023-06-09T11:10:51.461971+0000","last_fullsized":"2023-06-09T11:10:51.461971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:38:01.011511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.169478+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:11:06.169478+0000","last_peered":"2023-06-09T11:11:06.169478+0000","last_clean":"2023-06-09T11:11:06.169478+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:11:06.169478+0000","last_undegraded":"2023-06-09T11:11:06.169478+0000","last_fullsized":"2023-06-09T11:11:06.169478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:25:50.289182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548068+0000","last_change":"2023-06-09T11:10:50.201152+0000","last_active":"2023-06-09T11:10:51.548068+0000","last_peered":"2023-06-09T11:10:51.548068+0000","last_clean":"2023-06-09T11:10:51.548068+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:10:51.548068+0000","last_undegraded":"2023-06-09T11:10:51.548068+0000","last_fullsized":"2023-06-09T11:10:51.548068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:47:52.716524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259827+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:10:52.259827+0000","last_peered":"2023-06-09T11:10:52.259827+0000","last_clean":"2023-06-09T11:10:52.259827+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:10:52.259827+0000","last_undegraded":"2023-06-09T11:10:52.259827+0000","last_fullsized":"2023-06-09T11:10:52.259827+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:59:45.000964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.198118+0000","last_change":"2023-06-09T11:10:50.202996+0000","last_active":"2023-06-09T11:10:51.198118+0000","last_peered":"2023-06-09T11:10:51.198118+0000","last_clean":"2023-06-09T11:10:51.198118+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:10:51.198118+0000","last_undegraded":"2023-06-09T11:10:51.198118+0000","last_fullsized":"2023-06-09T11:10:51.198118+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:33:55.291759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137650+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:11:01.137650+0000","last_peered":"2023-06-09T11:11:01.137650+0000","last_clean":"2023-06-09T11:11:01.137650+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:11:01.137650+0000","last_undegraded":"2023-06-09T11:11:01.137650+0000","last_fullsized":"2023-06-09T11:11:01.137650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:19:40.730392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461906+0000","last_change":"2023-06-09T11:10:50.204978+0000","last_active":"2023-06-09T11:10:51.461906+0000","last_peered":"2023-06-09T11:10:51.461906+0000","last_clean":"2023-06-09T11:10:51.461906+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:10:51.461906+0000","last_undegraded":"2023-06-09T11:10:51.461906+0000","last_fullsized":"2023-06-09T11:10:51.461906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:48.109225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134392+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:11:01.134392+0000","last_peered":"2023-06-09T11:11:01.134392+0000","last_clean":"2023-06-09T11:11:01.134392+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:11:01.134392+0000","last_undegraded":"2023-06-09T11:11:01.134392+0000","last_fullsized":"2023-06-09T11:11:01.134392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:29:29.081601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462156+0000","last_change":"2023-06-09T11:10:50.211044+0000","last_active":"2023-06-09T11:10:51.462156+0000","last_peered":"2023-06-09T11:10:51.462156+0000","last_clean":"2023-06-09T11:10:51.462156+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:10:51.462156+0000","last_undegraded":"2023-06-09T11:10:51.462156+0000","last_fullsized":"2023-06-09T11:10:51.462156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:18:36.738323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200221+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:10:55.200221+0000","last_peered":"2023-06-09T11:10:55.200221+0000","last_clean":"2023-06-09T11:10:55.200221+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:10:55.200221+0000","last_undegraded":"2023-06-09T11:10:55.200221+0000","last_fullsized":"2023-06-09T11:10:55.200221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:48:51.367100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.185959+0000","last_change":"2023-06-09T11:10:50.209749+0000","last_active":"2023-06-09T11:10:51.185959+0000","last_peered":"2023-06-09T11:10:51.185959+0000","last_clean":"2023-06-09T11:10:51.185959+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:10:51.185959+0000","last_undegraded":"2023-06-09T11:10:51.185959+0000","last_fullsized":"2023-06-09T11:10:51.185959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:56:00.306406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'11","reported_seq":24,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:18.273099+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:11:18.273099+0000","last_peered":"2023-06-09T11:11:18.273099+0000","last_clean":"2023-06-09T11:11:18.273099+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:11:18.273099+0000","last_undegraded":"2023-06-09T11:11:18.273099+0000","last_fullsized":"2023-06-09T11:11:18.273099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:46.308363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2820,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189592+0000","last_change":"2023-06-09T11:10:50.201683+0000","last_active":"2023-06-09T11:10:51.189592+0000","last_peered":"2023-06-09T11:10:51.189592+0000","last_clean":"2023-06-09T11:10:51.189592+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:10:51.189592+0000","last_undegraded":"2023-06-09T11:10:51.189592+0000","last_fullsized":"2023-06-09T11:10:51.189592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:41.624452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171741+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:11:06.171741+0000","last_peered":"2023-06-09T11:11:06.171741+0000","last_clean":"2023-06-09T11:11:06.171741+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:11:06.171741+0000","last_undegraded":"2023-06-09T11:11:06.171741+0000","last_fullsized":"2023-06-09T11:11:06.171741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:16:13.739709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461910+0000","last_change":"2023-06-09T11:10:50.206158+0000","last_active":"2023-06-09T11:10:51.461910+0000","last_peered":"2023-06-09T11:10:51.461910+0000","last_clean":"2023-06-09T11:10:51.461910+0000","last_became_active":"2023-06-09T11:10:50.205877+0000","last_became_peered":"2023-06-09T11:10:50.205877+0000","last_unstale":"2023-06-09T11:10:51.461910+0000","last_undegraded":"2023-06-09T11:10:51.461910+0000","last_fullsized":"2023-06-09T11:10:51.461910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:23:44.660890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.137302+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:11:01.137302+0000","last_peered":"2023-06-09T11:11:01.137302+0000","last_clean":"2023-06-09T11:11:01.137302+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:11:01.137302+0000","last_undegraded":"2023-06-09T11:11:01.137302+0000","last_fullsized":"2023-06-09T11:11:01.137302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:35:13.447294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189443+0000","last_change":"2023-06-09T11:10:50.199596+0000","last_active":"2023-06-09T11:10:51.189443+0000","last_peered":"2023-06-09T11:10:51.189443+0000","last_clean":"2023-06-09T11:10:51.189443+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:10:51.189443+0000","last_undegraded":"2023-06-09T11:10:51.189443+0000","last_fullsized":"2023-06-09T11:10:51.189443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:37:37.470059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174010+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:11:06.174010+0000","last_peered":"2023-06-09T11:11:06.174010+0000","last_clean":"2023-06-09T11:11:06.174010+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:11:06.174010+0000","last_undegraded":"2023-06-09T11:11:06.174010+0000","last_fullsized":"2023-06-09T11:11:06.174010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:52:05.731204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548045+0000","last_change":"2023-06-09T11:10:50.216509+0000","last_active":"2023-06-09T11:10:51.548045+0000","last_peered":"2023-06-09T11:10:51.548045+0000","last_clean":"2023-06-09T11:10:51.548045+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:10:51.548045+0000","last_undegraded":"2023-06-09T11:10:51.548045+0000","last_fullsized":"2023-06-09T11:10:51.548045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:38:33.670539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"20'15","reported_seq":28,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.172253+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:11:06.172253+0000","last_peered":"2023-06-09T11:11:06.172253+0000","last_clean":"2023-06-09T11:11:06.172253+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:11:06.172253+0000","last_undegraded":"2023-06-09T11:11:06.172253+0000","last_fullsized":"2023-06-09T11:11:06.172253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:06:39.542883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462335+0000","last_change":"2023-06-09T11:10:50.213424+0000","last_active":"2023-06-09T11:10:51.462335+0000","last_peered":"2023-06-09T11:10:51.462335+0000","last_clean":"2023-06-09T11:10:51.462335+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:10:51.462335+0000","last_undegraded":"2023-06-09T11:10:51.462335+0000","last_fullsized":"2023-06-09T11:10:51.462335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:53.803610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.200205+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:10:55.200205+0000","last_peered":"2023-06-09T11:10:55.200205+0000","last_clean":"2023-06-09T11:10:55.200205+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:10:55.200205+0000","last_undegraded":"2023-06-09T11:10:55.200205+0000","last_fullsized":"2023-06-09T11:10:55.200205+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:54.534058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189905+0000","last_change":"2023-06-09T11:10:50.197487+0000","last_active":"2023-06-09T11:10:51.189905+0000","last_peered":"2023-06-09T11:10:51.189905+0000","last_clean":"2023-06-09T11:10:51.189905+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:10:51.189905+0000","last_undegraded":"2023-06-09T11:10:51.189905+0000","last_fullsized":"2023-06-09T11:10:51.189905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:38.664479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.199981+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:10:55.199981+0000","last_peered":"2023-06-09T11:10:55.199981+0000","last_clean":"2023-06-09T11:10:55.199981+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:10:55.199981+0000","last_undegraded":"2023-06-09T11:10:55.199981+0000","last_fullsized":"2023-06-09T11:10:55.199981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:43:49.312343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.188991+0000","last_change":"2023-06-09T11:10:50.207656+0000","last_active":"2023-06-09T11:10:51.188991+0000","last_peered":"2023-06-09T11:10:51.188991+0000","last_clean":"2023-06-09T11:10:51.188991+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:10:51.188991+0000","last_undegraded":"2023-06-09T11:10:51.188991+0000","last_fullsized":"2023-06-09T11:10:51.188991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:28:09.331841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251545+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:10:52.251545+0000","last_peered":"2023-06-09T11:10:52.251545+0000","last_clean":"2023-06-09T11:10:52.251545+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:10:52.251545+0000","last_undegraded":"2023-06-09T11:10:52.251545+0000","last_fullsized":"2023-06-09T11:10:52.251545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:39:42.880611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197929+0000","last_change":"2023-06-09T11:10:50.212884+0000","last_active":"2023-06-09T11:10:51.197929+0000","last_peered":"2023-06-09T11:10:51.197929+0000","last_clean":"2023-06-09T11:10:51.197929+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:10:51.197929+0000","last_undegraded":"2023-06-09T11:10:51.197929+0000","last_fullsized":"2023-06-09T11:10:51.197929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:21.200587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170742+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:11:06.170742+0000","last_peered":"2023-06-09T11:11:06.170742+0000","last_clean":"2023-06-09T11:11:06.170742+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:11:06.170742+0000","last_undegraded":"2023-06-09T11:11:06.170742+0000","last_fullsized":"2023-06-09T11:11:06.170742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:22.666248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.547735+0000","last_change":"2023-06-09T11:10:50.217609+0000","last_active":"2023-06-09T11:10:51.547735+0000","last_peered":"2023-06-09T11:10:51.547735+0000","last_clean":"2023-06-09T11:10:51.547735+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:10:51.547735+0000","last_undegraded":"2023-06-09T11:10:51.547735+0000","last_fullsized":"2023-06-09T11:10:51.547735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:45.268881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134150+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:11:01.134150+0000","last_peered":"2023-06-09T11:11:01.134150+0000","last_clean":"2023-06-09T11:11:01.134150+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:11:01.134150+0000","last_undegraded":"2023-06-09T11:11:01.134150+0000","last_fullsized":"2023-06-09T11:11:01.134150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:51.359837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190116+0000","last_change":"2023-06-09T11:10:50.211406+0000","last_active":"2023-06-09T11:10:51.190116+0000","last_peered":"2023-06-09T11:10:51.190116+0000","last_clean":"2023-06-09T11:10:51.190116+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:10:51.190116+0000","last_undegraded":"2023-06-09T11:10:51.190116+0000","last_fullsized":"2023-06-09T11:10:51.190116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:01:55.296265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.250804+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:10:52.250804+0000","last_peered":"2023-06-09T11:10:52.250804+0000","last_clean":"2023-06-09T11:10:52.250804+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:10:52.250804+0000","last_undegraded":"2023-06-09T11:10:52.250804+0000","last_fullsized":"2023-06-09T11:10:52.250804+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:38:47.090070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190095+0000","last_change":"2023-06-09T11:10:50.199825+0000","last_active":"2023-06-09T11:10:51.190095+0000","last_peered":"2023-06-09T11:10:51.190095+0000","last_clean":"2023-06-09T11:10:51.190095+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:10:51.190095+0000","last_undegraded":"2023-06-09T11:10:51.190095+0000","last_fullsized":"2023-06-09T11:10:51.190095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:52:12.035010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135398+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:11:01.135398+0000","last_peered":"2023-06-09T11:11:01.135398+0000","last_clean":"2023-06-09T11:11:01.135398+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:11:01.135398+0000","last_undegraded":"2023-06-09T11:11:01.135398+0000","last_fullsized":"2023-06-09T11:11:01.135398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:44:50.874243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190037+0000","last_change":"2023-06-09T11:10:50.206753+0000","last_active":"2023-06-09T11:10:51.190037+0000","last_peered":"2023-06-09T11:10:51.190037+0000","last_clean":"2023-06-09T11:10:51.190037+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:10:51.190037+0000","last_undegraded":"2023-06-09T11:10:51.190037+0000","last_fullsized":"2023-06-09T11:10:51.190037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:05.443892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.133151+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:11:01.133151+0000","last_peered":"2023-06-09T11:11:01.133151+0000","last_clean":"2023-06-09T11:11:01.133151+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:11:01.133151+0000","last_undegraded":"2023-06-09T11:11:01.133151+0000","last_fullsized":"2023-06-09T11:11:01.133151+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:50.649020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186164+0000","last_change":"2023-06-09T11:10:50.211570+0000","last_active":"2023-06-09T11:10:51.186164+0000","last_peered":"2023-06-09T11:10:51.186164+0000","last_clean":"2023-06-09T11:10:51.186164+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:10:51.186164+0000","last_undegraded":"2023-06-09T11:10:51.186164+0000","last_fullsized":"2023-06-09T11:10:51.186164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:26:00.017731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186409+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:10:51.186409+0000","last_peered":"2023-06-09T11:10:51.186409+0000","last_clean":"2023-06-09T11:10:51.186409+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:10:51.186409+0000","last_undegraded":"2023-06-09T11:10:51.186409+0000","last_fullsized":"2023-06-09T11:10:51.186409+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:01:34.605429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.251095+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:10:52.251095+0000","last_peered":"2023-06-09T11:10:52.251095+0000","last_clean":"2023-06-09T11:10:52.251095+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:10:52.251095+0000","last_undegraded":"2023-06-09T11:10:52.251095+0000","last_fullsized":"2023-06-09T11:10:52.251095+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:50.716934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186047+0000","last_change":"2023-06-09T11:10:50.209538+0000","last_active":"2023-06-09T11:10:51.186047+0000","last_peered":"2023-06-09T11:10:51.186047+0000","last_clean":"2023-06-09T11:10:51.186047+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:10:51.186047+0000","last_undegraded":"2023-06-09T11:10:51.186047+0000","last_fullsized":"2023-06-09T11:10:51.186047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:37.032610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189355+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:10:51.189355+0000","last_peered":"2023-06-09T11:10:51.189355+0000","last_clean":"2023-06-09T11:10:51.189355+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:10:51.189355+0000","last_undegraded":"2023-06-09T11:10:51.189355+0000","last_fullsized":"2023-06-09T11:10:51.189355+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:12:51.314698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.135947+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:11:01.135947+0000","last_peered":"2023-06-09T11:11:01.135947+0000","last_clean":"2023-06-09T11:11:01.135947+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:11:01.135947+0000","last_undegraded":"2023-06-09T11:11:01.135947+0000","last_fullsized":"2023-06-09T11:11:01.135947+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:33.667856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186395+0000","last_change":"2023-06-09T11:10:50.203898+0000","last_active":"2023-06-09T11:10:51.186395+0000","last_peered":"2023-06-09T11:10:51.186395+0000","last_clean":"2023-06-09T11:10:51.186395+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:10:51.186395+0000","last_undegraded":"2023-06-09T11:10:51.186395+0000","last_fullsized":"2023-06-09T11:10:51.186395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:07:31.635295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.980746+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:10:51.980746+0000","last_peered":"2023-06-09T11:10:51.980746+0000","last_clean":"2023-06-09T11:10:51.980746+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:10:51.980746+0000","last_undegraded":"2023-06-09T11:10:51.980746+0000","last_fullsized":"2023-06-09T11:10:51.980746+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:05.503188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.170165+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:11:06.170165+0000","last_peered":"2023-06-09T11:11:06.170165+0000","last_clean":"2023-06-09T11:11:06.170165+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:11:06.170165+0000","last_undegraded":"2023-06-09T11:11:06.170165+0000","last_fullsized":"2023-06-09T11:11:06.170165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:53:56.269056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186120+0000","last_change":"2023-06-09T11:10:50.202809+0000","last_active":"2023-06-09T11:10:51.186120+0000","last_peered":"2023-06-09T11:10:51.186120+0000","last_clean":"2023-06-09T11:10:51.186120+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:10:51.186120+0000","last_undegraded":"2023-06-09T11:10:51.186120+0000","last_fullsized":"2023-06-09T11:10:51.186120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:52.136639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189110+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:10:51.189110+0000","last_peered":"2023-06-09T11:10:51.189110+0000","last_clean":"2023-06-09T11:10:51.189110+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:10:51.189110+0000","last_undegraded":"2023-06-09T11:10:51.189110+0000","last_fullsized":"2023-06-09T11:10:51.189110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:20:13.635505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689635+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:10:51.689635+0000","last_peered":"2023-06-09T11:10:51.689635+0000","last_clean":"2023-06-09T11:10:51.689635+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:10:51.689635+0000","last_undegraded":"2023-06-09T11:10:51.689635+0000","last_fullsized":"2023-06-09T11:10:51.689635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:06:09.577359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:52.259923+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:10:52.259923+0000","last_peered":"2023-06-09T11:10:52.259923+0000","last_clean":"2023-06-09T11:10:52.259923+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:10:52.259923+0000","last_undegraded":"2023-06-09T11:10:52.259923+0000","last_fullsized":"2023-06-09T11:10:52.259923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:36:00.656005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197885+0000","last_change":"2023-06-09T11:10:50.202500+0000","last_active":"2023-06-09T11:10:51.197885+0000","last_peered":"2023-06-09T11:10:51.197885+0000","last_clean":"2023-06-09T11:10:51.197885+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:10:51.197885+0000","last_undegraded":"2023-06-09T11:10:51.197885+0000","last_fullsized":"2023-06-09T11:10:51.197885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:39.319622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689437+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:10:51.689437+0000","last_peered":"2023-06-09T11:10:51.689437+0000","last_clean":"2023-06-09T11:10:51.689437+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:10:51.689437+0000","last_undegraded":"2023-06-09T11:10:51.689437+0000","last_fullsized":"2023-06-09T11:10:51.689437+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:02:07.826169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171980+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:11:06.171980+0000","last_peered":"2023-06-09T11:11:06.171980+0000","last_clean":"2023-06-09T11:11:06.171980+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:11:06.171980+0000","last_undegraded":"2023-06-09T11:11:06.171980+0000","last_fullsized":"2023-06-09T11:11:06.171980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:01:16.722489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548205+0000","last_change":"2023-06-09T11:10:50.216954+0000","last_active":"2023-06-09T11:10:51.548205+0000","last_peered":"2023-06-09T11:10:51.548205+0000","last_clean":"2023-06-09T11:10:51.548205+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:10:51.548205+0000","last_undegraded":"2023-06-09T11:10:51.548205+0000","last_fullsized":"2023-06-09T11:10:51.548205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:35.747041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.689987+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:10:51.689987+0000","last_peered":"2023-06-09T11:10:51.689987+0000","last_clean":"2023-06-09T11:10:51.689987+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:10:51.689987+0000","last_undegraded":"2023-06-09T11:10:51.689987+0000","last_fullsized":"2023-06-09T11:10:51.689987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:08:19.840370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186520+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:10:51.186520+0000","last_peered":"2023-06-09T11:10:51.186520+0000","last_clean":"2023-06-09T11:10:51.186520+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:10:51.186520+0000","last_undegraded":"2023-06-09T11:10:51.186520+0000","last_fullsized":"2023-06-09T11:10:51.186520+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:21:54.640229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.548051+0000","last_change":"2023-06-09T11:10:50.201342+0000","last_active":"2023-06-09T11:10:51.548051+0000","last_peered":"2023-06-09T11:10:51.548051+0000","last_clean":"2023-06-09T11:10:51.548051+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:10:51.548051+0000","last_undegraded":"2023-06-09T11:10:51.548051+0000","last_fullsized":"2023-06-09T11:10:51.548051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:41.651344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.134155+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:11:01.134155+0000","last_peered":"2023-06-09T11:11:01.134155+0000","last_clean":"2023-06-09T11:11:01.134155+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:11:01.134155+0000","last_undegraded":"2023-06-09T11:11:01.134155+0000","last_fullsized":"2023-06-09T11:11:01.134155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:54.599921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186279+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:10:51.186279+0000","last_peered":"2023-06-09T11:10:51.186279+0000","last_clean":"2023-06-09T11:10:51.186279+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:10:51.186279+0000","last_undegraded":"2023-06-09T11:10:51.186279+0000","last_fullsized":"2023-06-09T11:10:51.186279+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:00:27.710625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462103+0000","last_change":"2023-06-09T11:10:50.204802+0000","last_active":"2023-06-09T11:10:51.462103+0000","last_peered":"2023-06-09T11:10:51.462103+0000","last_clean":"2023-06-09T11:10:51.462103+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:10:51.462103+0000","last_undegraded":"2023-06-09T11:10:51.462103+0000","last_fullsized":"2023-06-09T11:10:51.462103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:39:55.356673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197901+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:10:51.197901+0000","last_peered":"2023-06-09T11:10:51.197901+0000","last_clean":"2023-06-09T11:10:51.197901+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:10:51.197901+0000","last_undegraded":"2023-06-09T11:10:51.197901+0000","last_fullsized":"2023-06-09T11:10:51.197901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:27:22.847646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"20'3","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171372+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:11:06.171372+0000","last_peered":"2023-06-09T11:11:06.171372+0000","last_clean":"2023-06-09T11:11:06.171372+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:11:06.171372+0000","last_undegraded":"2023-06-09T11:11:06.171372+0000","last_fullsized":"2023-06-09T11:11:06.171372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:07:48.431632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189154+0000","last_change":"2023-06-09T11:10:50.198712+0000","last_active":"2023-06-09T11:10:51.189154+0000","last_peered":"2023-06-09T11:10:51.189154+0000","last_clean":"2023-06-09T11:10:51.189154+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:10:51.189154+0000","last_undegraded":"2023-06-09T11:10:51.189154+0000","last_fullsized":"2023-06-09T11:10:51.189154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:28.937424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"20'6","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:01.139116+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:11:01.139116+0000","last_peered":"2023-06-09T11:11:01.139116+0000","last_clean":"2023-06-09T11:11:01.139116+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:11:01.139116+0000","last_undegraded":"2023-06-09T11:11:01.139116+0000","last_fullsized":"2023-06-09T11:11:01.139116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:17:24.785625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.981057+0000","last_change":"2023-06-09T11:10:50.205986+0000","last_active":"2023-06-09T11:10:51.981057+0000","last_peered":"2023-06-09T11:10:51.981057+0000","last_clean":"2023-06-09T11:10:51.981057+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:10:51.981057+0000","last_undegraded":"2023-06-09T11:10:51.981057+0000","last_fullsized":"2023-06-09T11:10:51.981057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:55:36.115622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.198929+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:10:55.198929+0000","last_peered":"2023-06-09T11:10:55.198929+0000","last_clean":"2023-06-09T11:10:55.198929+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:10:55.198929+0000","last_undegraded":"2023-06-09T11:10:55.198929+0000","last_fullsized":"2023-06-09T11:10:55.198929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:04:24.618421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189279+0000","last_change":"2023-06-09T11:10:50.212751+0000","last_active":"2023-06-09T11:10:51.189279+0000","last_peered":"2023-06-09T11:10:51.189279+0000","last_clean":"2023-06-09T11:10:51.189279+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:10:51.189279+0000","last_undegraded":"2023-06-09T11:10:51.189279+0000","last_fullsized":"2023-06-09T11:10:51.189279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:50.540558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"20'4","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.171753+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:11:06.171753+0000","last_peered":"2023-06-09T11:11:06.171753+0000","last_clean":"2023-06-09T11:11:06.171753+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:11:06.171753+0000","last_undegraded":"2023-06-09T11:11:06.171753+0000","last_fullsized":"2023-06-09T11:11:06.171753+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:17:20.887019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.190299+0000","last_change":"2023-06-09T11:10:50.210697+0000","last_active":"2023-06-09T11:10:51.190299+0000","last_peered":"2023-06-09T11:10:51.190299+0000","last_clean":"2023-06-09T11:10:51.190299+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:10:51.190299+0000","last_undegraded":"2023-06-09T11:10:51.190299+0000","last_fullsized":"2023-06-09T11:10:51.190299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:01:10.484031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'2","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:55.195108+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:10:55.195108+0000","last_peered":"2023-06-09T11:10:55.195108+0000","last_clean":"2023-06-09T11:10:55.195108+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:10:55.195108+0000","last_undegraded":"2023-06-09T11:10:55.195108+0000","last_fullsized":"2023-06-09T11:10:55.195108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:16.369507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461739+0000","last_change":"2023-06-09T11:10:50.215674+0000","last_active":"2023-06-09T11:10:51.461739+0000","last_peered":"2023-06-09T11:10:51.461739+0000","last_clean":"2023-06-09T11:10:51.461739+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:10:51.461739+0000","last_undegraded":"2023-06-09T11:10:51.461739+0000","last_fullsized":"2023-06-09T11:10:51.461739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:45:52.800131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"20'9","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:11:06.174340+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:11:06.174340+0000","last_peered":"2023-06-09T11:11:06.174340+0000","last_clean":"2023-06-09T11:11:06.174340+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:11:06.174340+0000","last_undegraded":"2023-06-09T11:11:06.174340+0000","last_fullsized":"2023-06-09T11:11:06.174340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:17.517425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189985+0000","last_change":"2023-06-09T11:10:50.208019+0000","last_active":"2023-06-09T11:10:51.189985+0000","last_peered":"2023-06-09T11:10:51.189985+0000","last_clean":"2023-06-09T11:10:51.189985+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:10:51.189985+0000","last_undegraded":"2023-06-09T11:10:51.189985+0000","last_fullsized":"2023-06-09T11:10:51.189985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:45:49.034537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.197838+0000","last_change":"2023-06-09T11:10:50.201710+0000","last_active":"2023-06-09T11:10:51.197838+0000","last_peered":"2023-06-09T11:10:51.197838+0000","last_clean":"2023-06-09T11:10:51.197838+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:10:51.197838+0000","last_undegraded":"2023-06-09T11:10:51.197838+0000","last_fullsized":"2023-06-09T11:10:51.197838+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:39:08.790612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.461690+0000","last_change":"2023-06-09T11:10:50.219019+0000","last_active":"2023-06-09T11:10:51.461690+0000","last_peered":"2023-06-09T11:10:51.461690+0000","last_clean":"2023-06-09T11:10:51.461690+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:10:51.461690+0000","last_undegraded":"2023-06-09T11:10:51.461690+0000","last_fullsized":"2023-06-09T11:10:51.461690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:35:01.064974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.189421+0000","last_change":"2023-06-09T11:10:50.207563+0000","last_active":"2023-06-09T11:10:51.189421+0000","last_peered":"2023-06-09T11:10:51.189421+0000","last_clean":"2023-06-09T11:10:51.189421+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:10:51.189421+0000","last_undegraded":"2023-06-09T11:10:51.189421+0000","last_fullsized":"2023-06-09T11:10:51.189421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:05:21.013690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.462161+0000","last_change":"2023-06-09T11:10:50.211936+0000","last_active":"2023-06-09T11:10:51.462161+0000","last_peered":"2023-06-09T11:10:51.462161+0000","last_clean":"2023-06-09T11:10:51.462161+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:10:51.462161+0000","last_undegraded":"2023-06-09T11:10:51.462161+0000","last_fullsized":"2023-06-09T11:10:51.462161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:09:34.714309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-06-09T11:10:51.186204+0000","last_change":"2023-06-09T11:10:50.204095+0000","last_active":"2023-06-09T11:10:51.186204+0000","last_peered":"2023-06-09T11:10:51.186204+0000","last_clean":"2023-06-09T11:10:51.186204+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:10:51.186204+0000","last_undegraded":"2023-06-09T11:10:51.186204+0000","last_fullsized":"2023-06-09T11:10:51.186204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:14:56.308208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":6775,"num_objects":60,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":130,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":13550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9012,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362828,"statfs":{"total":96636764160,"available":96627535872,"internally_reserved":0,"allocated":307200,"data_stored":668946,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":212992,"data_stored":74532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8400,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363440,"statfs":{"total":96636764160,"available":96628162560,"internally_reserved":0,"allocated":204800,"data_stored":74420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9008,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362832,"statfs":{"total":96636764160,"available":96627539968,"internally_reserved":0,"allocated":303104,"data_stored":666711,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8276,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363564,"statfs":{"total":96636764160,"available":96628289536,"internally_reserved":0,"allocated":208896,"data_stored":74510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8416,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363424,"statfs":{"total":96636764160,"available":96628146176,"internally_reserved":0,"allocated":221184,"data_stored":75696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.751}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8508,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363332,"statfs":{"total":96636764160,"available":96628051968,"internally_reserved":0,"allocated":249856,"data_stored":79065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8420,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363420,"statfs":{"total":96636764160,"available":96628142080,"internally_reserved":0,"allocated":225280,"data_stored":75766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":4645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:11:24.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:24.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.943+0000 7fda663ff700 1 Processor -- start 2023-06-09T11:11:24.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.944+0000 7fda663ff700 1 -- start start 2023-06-09T11:11:24.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.945+0000 7fda663ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda6015b4a0 0x7fda6015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.945+0000 7fda663ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fda600c9f40 0x7fda600c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.945+0000 7fda663ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda6015a500 con 0x7fda60153880 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.945+0000 7fda663ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda600c5f10 con 0x7fda6015b4a0 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda663ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6015da00 con 0x7fda600c9f40 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda6015b4a0 0x7fda6015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fda6015b4a0 0x7fda6015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41442/0 (socket says 172.21.15.7:41442) 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 -- 172.21.15.7:0/2304817045 learned_addr learned my addr 172.21.15.7:0/2304817045 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:24.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda6499c700 1 --2- 172.21.15.7:0/2304817045 >> v2:172.21.15.117:3300/0 conn(0x7fda600c9f40 0x7fda600c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 -- 172.21.15.7:0/2304817045 >> v2:172.21.15.117:3300/0 conn(0x7fda600c9f40 msgr2=0x7fda600c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 --2- 172.21.15.7:0/2304817045 >> v2:172.21.15.117:3300/0 conn(0x7fda600c9f40 0x7fda600c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 -- 172.21.15.7:0/2304817045 >> v1:172.21.15.117:6789/0 conn(0x7fda60153880 legacy=0x7fda6015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 -- 172.21.15.7:0/2304817045 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda601596e0 con 0x7fda6015b4a0 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.946+0000 7fda5f7fe700 1 --2- 172.21.15.7:0/2304817045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6015d4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda600749f0 tx=0x7fda54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2310ea49cb40c6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda5effd700 1 -- 172.21.15.7:0/2304817045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54011830 con 0x7fda6015b4a0 2023-06-09T11:11:24.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda5effd700 1 -- 172.21.15.7:0/2304817045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54011990 con 0x7fda6015b4a0 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda5effd700 1 -- 172.21.15.7:0/2304817045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54011d20 con 0x7fda6015b4a0 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 -- 172.21.15.7:0/2304817045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 msgr2=0x7fda6015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 --2- 172.21.15.7:0/2304817045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6015d4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fda600749f0 tx=0x7fda54009bd0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 -- 172.21.15.7:0/2304817045 shutdown_connections 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 --2- 172.21.15.7:0/2304817045 >> v2:172.21.15.117:3300/0 conn(0x7fda600c9f40 0x7fda600c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 --2- 172.21.15.7:0/2304817045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6015d4c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 -- 172.21.15.7:0/2304817045 >> 172.21.15.7:0/2304817045 conn(0x7fda600737f0 msgr2=0x7fda600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:24.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 -- 172.21.15.7:0/2304817045 shutdown_connections 2023-06-09T11:11:24.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.947+0000 7fda663ff700 1 -- 172.21.15.7:0/2304817045 wait complete. 2023-06-09T11:11:24.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.949+0000 7fda663ff700 1 Processor -- start 2023-06-09T11:11:24.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.949+0000 7fda663ff700 1 -- start start 2023-06-09T11:11:24.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda600c9f40 0x7fda600caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 0x7fda6014a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda601594b0 con 0x7fda6015b4a0 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda600cea30 con 0x7fda600c9f40 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.950+0000 7fda663ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda6014b630 con 0x7fda60153880 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda5f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 0x7fda6014a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda5ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda600c9f40 0x7fda600caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41450/0 (socket says 172.21.15.7:41450) 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda5f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 0x7fda6014a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:43576/0 (socket says 172.21.15.7:43576) 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 -- 172.21.15.7:0/1237317484 learned_addr learned my addr 172.21.15.7:0/1237317484 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 -- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 msgr2=0x7fda6014a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 0x7fda6014a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 -- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda600c9f40 msgr2=0x7fda600caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda600c9f40 0x7fda600caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:24.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 -- 172.21.15.7:0/1237317484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda54009550 con 0x7fda6015b4a0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.951+0000 7fda6499c700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda58009dd0 tx=0x7fda5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.952+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58009c40 con 0x7fda6015b4a0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.952+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5800f040 con 0x7fda6015b4a0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.952+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5800ceb0 con 0x7fda6015b4a0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.952+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda60159c60 con 0x7fda6015b4a0 2023-06-09T11:11:24.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.952+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda60159980 con 0x7fda6015b4a0 2023-06-09T11:11:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.953+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda60159f40 con 0x7fda6015b4a0 2023-06-09T11:11:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.954+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda58014480 con 0x7fda6015b4a0 2023-06-09T11:11:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.955+0000 7fda5cff9700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 0x7fda4c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:24.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.955+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda58089f40 con 0x7fda6015b4a0 2023-06-09T11:11:24.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.955+0000 7fda5ffff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 0x7fda4c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:24.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.955+0000 7fda5ffff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 0x7fda4c06d1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda50007fd0 tx=0x7fda50009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:24.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:24.959+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fda58058a70 con 0x7fda6015b4a0 2023-06-09T11:11:25.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.170+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda600cf030 con 0x7fda6015b4a0 2023-06-09T11:11:25.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.172+0000 7fda5cff9700 1 -- 172.21.15.7:0/1237317484 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7fda600cf030 con 0x7fda6015b4a0 2023-06-09T11:11:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:25.178 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 msgr2=0x7fda4c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 0x7fda4c06d1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fda50007fd0 tx=0x7fda50009040 comp rx=0 tx=0).stop 2023-06-09T11:11:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 msgr2=0x7fda6014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fda58009dd0 tx=0x7fda5800c3a0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 shutdown_connections 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda6015b4a0 0x7fda6014b0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fda600c9f40 0x7fda600caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fda4c06ad10 0x7fda4c06d1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 --2- 172.21.15.7:0/1237317484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fda60153880 0x7fda6014a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 >> 172.21.15.7:0/1237317484 conn(0x7fda600737f0 msgr2=0x7fda6014c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.174+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 shutdown_connections 2023-06-09T11:11:25.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.175+0000 7fda663ff700 1 -- 172.21.15.7:0/1237317484 wait complete. 2023-06-09T11:11:25.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:11:25.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:11:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.397+0000 7fd6fd7ff700 1 Processor -- start 2023-06-09T11:11:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.397+0000 7fd6fd7ff700 1 -- start start 2023-06-09T11:11:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6fd7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 0x7fd6f806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6fd7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6f806b740 0x7fd6f815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6fd7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80c6220 con 0x7fd6f80c64e0 2023-06-09T11:11:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6fd7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80c4f70 con 0x7fd6f806b740 2023-06-09T11:11:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6fd7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80c5c20 con 0x7fd6f81537a0 2023-06-09T11:11:25.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6f806b740 0x7fd6f815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6f77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd6f806b740 0x7fd6f815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41460/0 (socket says 172.21.15.7:41460) 2023-06-09T11:11:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6f77fe700 1 -- 172.21.15.7:0/1925144712 learned_addr learned my addr 172.21.15.7:0/1925144712 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.398+0000 7fd6f67fc700 1 --1- 172.21.15.7:0/1925144712 >> v1:172.21.15.7:6789/0 conn(0x7fd6f80c64e0 0x7fd6f80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57414/0 (socket says 172.21.15.7:57414) 2023-06-09T11:11:25.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f6ffd700 1 --2- 172.21.15.7:0/1925144712 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 0x7fd6f806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f77fe700 1 -- 172.21.15.7:0/1925144712 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 msgr2=0x7fd6f806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f77fe700 1 --2- 172.21.15.7:0/1925144712 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 0x7fd6f806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f77fe700 1 -- 172.21.15.7:0/1925144712 >> v1:172.21.15.7:6789/0 conn(0x7fd6f80c64e0 legacy=0x7fd6f80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f77fe700 1 -- 172.21.15.7:0/1925144712 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80c4480 con 0x7fd6f806b740 2023-06-09T11:11:25.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f6ffd700 1 --2- 172.21.15.7:0/1925144712 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 0x7fd6f806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f5ffb700 1 -- 172.21.15.7:0/1925144712 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381510415 0 0) 0x7fd6ec001110 con 0x7fd6f80c64e0 2023-06-09T11:11:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f77fe700 1 --2- 172.21.15.7:0/1925144712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f806b740 0x7fd6f815b2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80cd130 tx=0x7fd6f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e66eb42b4502a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f5ffb700 1 -- 172.21.15.7:0/1925144712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000e070 con 0x7fd6f806b740 2023-06-09T11:11:25.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f5ffb700 1 -- 172.21.15.7:0/1925144712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f000b9c0 con 0x7fd6f806b740 2023-06-09T11:11:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.399+0000 7fd6f5ffb700 1 -- 172.21.15.7:0/1925144712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f000bd70 con 0x7fd6f806b740 2023-06-09T11:11:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/1925144712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f806b740 msgr2=0x7fd6f815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/1925144712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f806b740 0x7fd6f815b2b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd6f80cd130 tx=0x7fd6f000b330 comp rx=0 tx=0).stop 2023-06-09T11:11:25.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/1925144712 shutdown_connections 2023-06-09T11:11:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/1925144712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f806b740 0x7fd6f815b2b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/1925144712 >> v2:172.21.15.117:3300/0 conn(0x7fd6f81537a0 0x7fd6f806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/1925144712 >> 172.21.15.7:0/1925144712 conn(0x7fd6f8072040 msgr2=0x7fd6f8072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:25.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/1925144712 shutdown_connections 2023-06-09T11:11:25.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.400+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/1925144712 wait complete. 2023-06-09T11:11:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.402+0000 7fd6fd7ff700 1 Processor -- start 2023-06-09T11:11:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.402+0000 7fd6fd7ff700 1 -- start start 2023-06-09T11:11:25.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f806b740 0x7fd6f81483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f80c64e0 0x7fd6f80cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f81491f0 con 0x7fd6f81537a0 2023-06-09T11:11:25.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80cb1c0 con 0x7fd6f80c64e0 2023-06-09T11:11:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6fd7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8148ec0 con 0x7fd6f806b740 2023-06-09T11:11:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41462/0 (socket says 172.21.15.7:41462) 2023-06-09T11:11:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 -- 172.21.15.7:0/799174919 learned_addr learned my addr 172.21.15.7:0/799174919 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:25.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f67fc700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f80c64e0 0x7fd6f80cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f6ffd700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f806b740 0x7fd6f81483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 -- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f806b740 msgr2=0x7fd6f81483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f806b740 0x7fd6f81483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 -- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f80c64e0 msgr2=0x7fd6f80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f80c64e0 0x7fd6f80cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.403+0000 7fd6f77fe700 1 -- 172.21.15.7:0/799174919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0009b30 con 0x7fd6f81537a0 2023-06-09T11:11:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6f77fe700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000b730 tx=0x7fd6f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00099b0 con 0x7fd6f81537a0 2023-06-09T11:11:25.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00092e0 con 0x7fd6f81537a0 2023-06-09T11:11:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f80c46d0 con 0x7fd6f81537a0 2023-06-09T11:11:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f001b630 con 0x7fd6f81537a0 2023-06-09T11:11:25.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.404+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f815ae20 con 0x7fd6f81537a0 2023-06-09T11:11:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.405+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f80c49b0 con 0x7fd6f81537a0 2023-06-09T11:11:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.406+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd6f001b790 con 0x7fd6f81537a0 2023-06-09T11:11:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.407+0000 7fd6df7fe700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.407+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd6f008a4d0 con 0x7fd6f81537a0 2023-06-09T11:11:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.407+0000 7fd6f6ffd700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.408+0000 7fd6f6ffd700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 0x7fd6e406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006fd0 tx=0x7fd6e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.411+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd6f0055450 con 0x7fd6f81537a0 2023-06-09T11:11:25.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.621+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6f80cbed0 con 0x7fd6f81537a0 2023-06-09T11:11:25.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.622+0000 7fd6df7fe700 1 -- 172.21.15.7:0/799174919 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6f0058a70 con 0x7fd6f81537a0 2023-06-09T11:11:25.628 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 msgr2=0x7fd6e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 0x7fd6e406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8006fd0 tx=0x7fd6e8008040 comp rx=0 tx=0).stop 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 msgr2=0x7fd6f8149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000b730 tx=0x7fd6f0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 shutdown_connections 2023-06-09T11:11:25.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6f81537a0 0x7fd6f8149680 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd6e406ace0 0x7fd6e406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6f806b740 0x7fd6f81483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 --2- 172.21.15.7:0/799174919 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6f80c64e0 0x7fd6f80cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 >> 172.21.15.7:0/799174919 conn(0x7fd6f8072040 msgr2=0x7fd6f8150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 shutdown_connections 2023-06-09T11:11:25.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.624+0000 7fd6fd7ff700 1 -- 172.21.15.7:0/799174919 wait complete. 2023-06-09T11:11:25.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:11:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.844+0000 7f3dcf5ff700 1 Processor -- start 2023-06-09T11:11:25.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.844+0000 7f3dcf5ff700 1 -- start start 2023-06-09T11:11:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.845+0000 7f3dcf5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 0x7f3dc80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.845+0000 7f3dcf5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 0x7f3dc8070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.845+0000 7f3dcf5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc80b40e0 con 0x7f3dc806ac20 2023-06-09T11:11:25.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.845+0000 7f3dcf5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc806b110 con 0x7f3dc8153850 2023-06-09T11:11:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.845+0000 7f3dcf5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80b4330 con 0x7f3dc80c6590 2023-06-09T11:11:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dcdb9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3dc806ac20 0x7f3dc8071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57430/0 (socket says 172.21.15.7:57430) 2023-06-09T11:11:25.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dcdb9c700 1 -- 172.21.15.7:0/1724038042 learned_addr learned my addr 172.21.15.7:0/1724038042 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:25.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dcd39b700 1 --2- 172.21.15.7:0/1724038042 >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 0x7f3dc80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dccb9a700 1 --2- 172.21.15.7:0/1724038042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 0x7f3dc8070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dbffff700 1 -- 172.21.15.7:0/1724038042 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484715120 0 0) 0x7f3dc80b40e0 con 0x7f3dc806ac20 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dbffff700 1 -- 172.21.15.7:0/1724038042 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db40035b0 con 0x7f3dc806ac20 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dccb9a700 1 -- 172.21.15.7:0/1724038042 >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 msgr2=0x7f3dc80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dccb9a700 1 --2- 172.21.15.7:0/1724038042 >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 0x7f3dc80ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dccb9a700 1 -- 172.21.15.7:0/1724038042 >> v1:172.21.15.7:6789/0 conn(0x7f3dc806ac20 legacy=0x7f3dc8071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dccb9a700 1 -- 172.21.15.7:0/1724038042 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80b4610 con 0x7f3dc80c6590 2023-06-09T11:11:25.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.846+0000 7f3dcd39b700 1 --2- 172.21.15.7:0/1724038042 >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 0x7f3dc80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dccb9a700 1 --2- 172.21.15.7:0/1724038042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 0x7f3dc8070af0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009840 tx=0x7f3db8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8e811b37442d7c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dbffff700 1 -- 172.21.15.7:0/1724038042 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8011830 con 0x7f3dc80c6590 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dbffff700 1 -- 172.21.15.7:0/1724038042 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8011990 con 0x7f3dc80c6590 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dbffff700 1 -- 172.21.15.7:0/1724038042 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8011d20 con 0x7f3dc80c6590 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/1724038042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 msgr2=0x7f3dc8070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/1724038042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 0x7f3dc8070af0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3db8009840 tx=0x7f3db8009b50 comp rx=0 tx=0).stop 2023-06-09T11:11:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/1724038042 shutdown_connections 2023-06-09T11:11:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/1724038042 >> v2:172.21.15.117:3300/0 conn(0x7f3dc8153850 0x7f3dc80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/1724038042 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc80c6590 0x7f3dc8070af0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.847+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/1724038042 >> 172.21.15.7:0/1724038042 conn(0x7f3dc80737f0 msgr2=0x7f3dc8072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.848+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/1724038042 shutdown_connections 2023-06-09T11:11:25.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.848+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/1724038042 wait complete. 2023-06-09T11:11:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.849+0000 7f3dcf5ff700 1 Processor -- start 2023-06-09T11:11:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.849+0000 7f3dcf5ff700 1 -- start start 2023-06-09T11:11:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc806ac20 0x7f3dc80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 0x7f3dc814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc814b450 con 0x7f3dc8153850 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80cd470 con 0x7f3dc80c6590 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.850+0000 7f3dcf5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc814aab0 con 0x7f3dc806ac20 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41468/0 (socket says 172.21.15.7:41468) 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 -- 172.21.15.7:0/4011122354 learned_addr learned my addr 172.21.15.7:0/4011122354 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:25.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcd39b700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc806ac20 0x7f3dc80ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dccb9a700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 0x7f3dc814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 -- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc806ac20 msgr2=0x7f3dc80ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc806ac20 0x7f3dc80ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 -- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 msgr2=0x7f3dc814da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:25.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 0x7f3dc814da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 -- 172.21.15.7:0/4011122354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc40094a0 con 0x7f3dc8153850 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dccb9a700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 0x7f3dc814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dcdb9c700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cbb0 tx=0x7f3dc000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.851+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000ab50 con 0x7f3dc8153850 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.852+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db80094a0 con 0x7f3dc8153850 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.852+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc000acb0 con 0x7f3dc8153850 2023-06-09T11:11:25.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.852+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0012600 con 0x7f3dc8153850 2023-06-09T11:11:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.852+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc8153c20 con 0x7f3dc8153850 2023-06-09T11:11:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.853+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc80b48f0 con 0x7f3dc8153850 2023-06-09T11:11:25.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.854+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3dc0012760 con 0x7f3dc8153850 2023-06-09T11:11:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.855+0000 7f3dbdffb700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 0x7f3dac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.855+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3dc0089200 con 0x7f3dc8153850 2023-06-09T11:11:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.855+0000 7f3dcd39b700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 0x7f3dac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:25.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.855+0000 7f3dcd39b700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 0x7f3dac06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009c60 tx=0x7f3dc4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:25.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:25.859+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3dc00542c0 con 0x7f3dc8153850 2023-06-09T11:11:26.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.064+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3dc814b6c0 con 0x7f3dc8153850 2023-06-09T11:11:26.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.065+0000 7f3dbdffb700 1 -- 172.21.15.7:0/4011122354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3dc00578e0 con 0x7f3dc8153850 2023-06-09T11:11:26.071 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:11:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 msgr2=0x7f3dac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 0x7f3dac06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009c60 tx=0x7f3dc4008040 comp rx=0 tx=0).stop 2023-06-09T11:11:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 msgr2=0x7f3dc8148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cbb0 tx=0x7f3dc000cf70 comp rx=0 tx=0).stop 2023-06-09T11:11:26.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 shutdown_connections 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dc8153850 0x7f3dc8148fc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dac06ace0 0x7f3dac06d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3dc806ac20 0x7f3dc80ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 --2- 172.21.15.7:0/4011122354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3dc80c6590 0x7f3dc814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 >> 172.21.15.7:0/4011122354 conn(0x7f3dc80737f0 msgr2=0x7f3dc81515c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 shutdown_connections 2023-06-09T11:11:26.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.067+0000 7f3dcf5ff700 1 -- 172.21.15.7:0/4011122354 wait complete. 2023-06-09T11:11:26.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:11:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.291+0000 7f6a73fff700 1 Processor -- start 2023-06-09T11:11:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.291+0000 7f6a73fff700 1 -- start start 2023-06-09T11:11:26.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c1537a0 0x7f6a6c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a73fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a73fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0c6220 con 0x7f6a6c06b740 2023-06-09T11:11:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a73fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c4f70 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a73fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0c5c20 con 0x7f6a6c1537a0 2023-06-09T11:11:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41484/0 (socket says 172.21.15.7:41484) 2023-06-09T11:11:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 -- 172.21.15.7:0/758959525 learned_addr learned my addr 172.21.15.7:0/758959525 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:26.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a71d9b700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c1537a0 0x7f6a6c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 -- 172.21.15.7:0/758959525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c1537a0 msgr2=0x7f6a6c06b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c1537a0 0x7f6a6c06b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 -- 172.21.15.7:0/758959525 >> v1:172.21.15.117:6789/0 conn(0x7f6a6c06b740 legacy=0x7f6a6c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.293+0000 7f6a7159a700 1 -- 172.21.15.7:0/758959525 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0c4480 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a7159a700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0098f0 tx=0x7f6a5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973b30d9011d9065 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a70d99700 1 -- 172.21.15.7:0/758959525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00b8a0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a70d99700 1 -- 172.21.15.7:0/758959525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c00ba00 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a70d99700 1 -- 172.21.15.7:0/758959525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c00bd90 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 -- 172.21.15.7:0/758959525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 msgr2=0x7f6a6c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0098f0 tx=0x7f6a5c009c00 comp rx=0 tx=0).stop 2023-06-09T11:11:26.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 -- 172.21.15.7:0/758959525 shutdown_connections 2023-06-09T11:11:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c1537a0 0x7f6a6c06b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 --2- 172.21.15.7:0/758959525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0cf590 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.294+0000 7f6a73fff700 1 -- 172.21.15.7:0/758959525 >> 172.21.15.7:0/758959525 conn(0x7f6a6c072040 msgr2=0x7f6a6c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.295+0000 7f6a73fff700 1 -- 172.21.15.7:0/758959525 shutdown_connections 2023-06-09T11:11:26.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.295+0000 7f6a73fff700 1 -- 172.21.15.7:0/758959525 wait complete. 2023-06-09T11:11:26.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.296+0000 7f6a73fff700 1 Processor -- start 2023-06-09T11:11:26.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.296+0000 7f6a73fff700 1 -- start start 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c06b740 0x7f6a6c1eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c1537a0 0x7f6a6c1f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1481d0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1ef4c0 con 0x7f6a6c1537a0 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.297+0000 7f6a73fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1ef780 con 0x7f6a6c06b740 2023-06-09T11:11:26.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41490/0 (socket says 172.21.15.7:41490) 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 -- 172.21.15.7:0/3479281027 learned_addr learned my addr 172.21.15.7:0/3479281027 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7259c700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c1537a0 0x7f6a6c1f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a71d9b700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c06b740 0x7f6a6c1eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 -- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c06b740 msgr2=0x7f6a6c1eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c06b740 0x7f6a6c1eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 -- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c1537a0 msgr2=0x7f6a6c1f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c1537a0 0x7f6a6c1f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 -- 172.21.15.7:0/3479281027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c009550 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.298+0000 7f6a7159a700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c000c00 tx=0x7f6a5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.299+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c004380 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.299+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c01f070 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.299+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c0036a0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.299+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c0c46d0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.299+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c15ae20 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.300+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c0c49b0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.301+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a5c021030 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.302+0000 7f6a627fc700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 0x7f6a5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.302+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6a5c08a740 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.302+0000 7f6a71d9b700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 0x7f6a5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.303+0000 7f6a71d9b700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 0x7f6a5006d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a68007a30 tx=0x7f6a68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.306+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6a5c0556c0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.512+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a6c0c5180 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.512+0000 7f6a627fc700 1 -- 172.21.15.7:0/3479281027 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a5c02a0a0 con 0x7f6a6c0c64e0 2023-06-09T11:11:26.518 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:11:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.514+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 msgr2=0x7f6a5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.514+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 0x7f6a5006d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a68007a30 tx=0x7f6a68008040 comp rx=0 tx=0).stop 2023-06-09T11:11:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 msgr2=0x7f6a6c0c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c000c00 tx=0x7f6a5c004750 comp rx=0 tx=0).stop 2023-06-09T11:11:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 shutdown_connections 2023-06-09T11:11:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a6c1537a0 0x7f6a6c1f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a5006ace0 0x7f6a5006d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a6c06b740 0x7f6a6c1eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 --2- 172.21.15.7:0/3479281027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a6c0c64e0 0x7f6a6c0c8110 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 >> 172.21.15.7:0/3479281027 conn(0x7f6a6c072040 msgr2=0x7f6a6c0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 shutdown_connections 2023-06-09T11:11:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.515+0000 7f6a73fff700 1 -- 172.21.15.7:0/3479281027 wait complete. 2023-06-09T11:11:26.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:11:26.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.736+0000 7f2bc91ff700 1 Processor -- start 2023-06-09T11:11:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.736+0000 7f2bc91ff700 1 -- start start 2023-06-09T11:11:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.737+0000 7f2bc91ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bc415b9c0 0x7f2bc406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.737+0000 7f2bc91ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40c6220 con 0x7f2bc40c64e0 2023-06-09T11:11:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.737+0000 7f2bc91ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40c4480 con 0x7f2bc41537a0 2023-06-09T11:11:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.737+0000 7f2bc91ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc40c4f70 con 0x7f2bc415b9c0 2023-06-09T11:11:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bc40c64e0 0x7f2bc415b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57440/0 (socket says 172.21.15.7:57440) 2023-06-09T11:11:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc259c700 1 -- 172.21.15.7:0/2380046888 learned_addr learned my addr 172.21.15.7:0/2380046888 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc359e700 1 --2- 172.21.15.7:0/2380046888 >> v2:172.21.15.7:3300/0 conn(0x7f2bc415b9c0 0x7f2bc406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739922191 0 0) 0x7f2bb8001110 con 0x7f2bc40c64e0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac0035b0 con 0x7f2bc40c64e0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828483330 0 0) 0x7f2bc40c4480 con 0x7f2bc41537a0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb8001110 con 0x7f2bc41537a0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc359e700 1 -- 172.21.15.7:0/2380046888 >> v1:172.21.15.117:6789/0 conn(0x7f2bc41537a0 legacy=0x7f2bc40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc359e700 1 -- 172.21.15.7:0/2380046888 >> v1:172.21.15.7:6789/0 conn(0x7f2bc40c64e0 legacy=0x7f2bc415b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc359e700 1 -- 172.21.15.7:0/2380046888 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc415ae20 con 0x7f2bc415b9c0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.738+0000 7f2bc359e700 1 --2- 172.21.15.7:0/2380046888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc415b9c0 0x7f2bc406b1b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dca58ad87fbf66 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00e070 con 0x7f2bc415b9c0 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc00bb20 con 0x7f2bc415b9c0 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc1d9b700 1 -- 172.21.15.7:0/2380046888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc012430 con 0x7f2bc415b9c0 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2380046888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc415b9c0 msgr2=0x7f2bc406b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2380046888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc415b9c0 0x7f2bc406b1b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2380046888 shutdown_connections 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2380046888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc415b9c0 0x7f2bc406b1b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2380046888 >> 172.21.15.7:0/2380046888 conn(0x7f2bc4072040 msgr2=0x7f2bc4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2380046888 shutdown_connections 2023-06-09T11:11:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.739+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2380046888 wait complete. 2023-06-09T11:11:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.742+0000 7f2bc91ff700 1 Processor -- start 2023-06-09T11:11:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.742+0000 7f2bc91ff700 1 -- start start 2023-06-09T11:11:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc40c64e0 0x7f2bc4147240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc415b9c0 0x7f2bc415be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41f2680 con 0x7f2bc41537a0 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41f1790 con 0x7f2bc40c64e0 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.743+0000 7f2bc91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41f1a70 con 0x7f2bc415b9c0 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41502/0 (socket says 172.21.15.7:41502) 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 -- 172.21.15.7:0/2866606386 learned_addr learned my addr 172.21.15.7:0/2866606386 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc2d9d700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc40c64e0 0x7f2bc4147240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc359e700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc415b9c0 0x7f2bc415be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 -- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc415b9c0 msgr2=0x7f2bc415be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc415b9c0 0x7f2bc415be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 -- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc40c64e0 msgr2=0x7f2bc4147240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc40c64e0 0x7f2bc4147240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 -- 172.21.15.7:0/2866606386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009a70 con 0x7f2bc41537a0 2023-06-09T11:11:26.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bc259c700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8001eb0 tx=0x7f2bb800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.744+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8005d70 con 0x7f2bc41537a0 2023-06-09T11:11:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.745+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80070d0 con 0x7f2bc41537a0 2023-06-09T11:11:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.745+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc40c49b0 con 0x7f2bc41537a0 2023-06-09T11:11:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.745+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8007440 con 0x7f2bc41537a0 2023-06-09T11:11:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.745+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc40c46d0 con 0x7f2bc41537a0 2023-06-09T11:11:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.746+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40c4c90 con 0x7f2bc41537a0 2023-06-09T11:11:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.747+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bb80075a0 con 0x7f2bc41537a0 2023-06-09T11:11:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.747+0000 7f2bab7fe700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 0x7f2bb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.748+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2bb8088700 con 0x7f2bc41537a0 2023-06-09T11:11:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.748+0000 7f2bc2d9d700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 0x7f2bb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.748+0000 7f2bc2d9d700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 0x7f2bb006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40011e0 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:26.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.752+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bb80538f0 con 0x7f2bc41537a0 2023-06-09T11:11:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.962+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc40c6220 con 0x7f2bc41537a0 2023-06-09T11:11:26.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.962+0000 7f2bab7fe700 1 -- 172.21.15.7:0/2866606386 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bc40c6220 con 0x7f2bc41537a0 2023-06-09T11:11:26.969 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:11:26.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 msgr2=0x7f2bb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 0x7f2bb006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40011e0 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 msgr2=0x7f2bc40cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8001eb0 tx=0x7f2bb800c9e0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 shutdown_connections 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2bb006ace0 0x7f2bb006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bc415b9c0 0x7f2bc415be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bc40c64e0 0x7f2bc4147240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 --2- 172.21.15.7:0/2866606386 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bc41537a0 0x7f2bc40cbb30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 >> 172.21.15.7:0/2866606386 conn(0x7f2bc4072040 msgr2=0x7f2bc4150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 shutdown_connections 2023-06-09T11:11:26.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:26.965+0000 7f2bc91ff700 1 -- 172.21.15.7:0/2866606386 wait complete. 2023-06-09T11:11:26.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:27.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.184+0000 7f62525ff700 1 Processor -- start 2023-06-09T11:11:27.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.184+0000 7f62525ff700 1 -- start start 2023-06-09T11:11:27.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.185+0000 7f62525ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f624c0cb5d0 0x7f624c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.185+0000 7f62525ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f624c15cd40 0x7f624c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.185+0000 7f62525ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624c159b20 con 0x7f624c0cb9a0 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.185+0000 7f62525ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624c0cac80 con 0x7f624c15cd40 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.185+0000 7f62525ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624c1597d0 con 0x7f624c0cb5d0 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f624c15cd40 0x7f624c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f624c15cd40 0x7f624c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41514/0 (socket says 172.21.15.7:41514) 2023-06-09T11:11:27.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 -- 172.21.15.7:0/968628078 learned_addr learned my addr 172.21.15.7:0/968628078 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:27.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f624bfff700 1 --2- 172.21.15.7:0/968628078 >> v2:172.21.15.117:3300/0 conn(0x7f624c0cb5d0 0x7f624c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 -- 172.21.15.7:0/968628078 >> v2:172.21.15.117:3300/0 conn(0x7f624c0cb5d0 msgr2=0x7f624c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 --2- 172.21.15.7:0/968628078 >> v2:172.21.15.117:3300/0 conn(0x7f624c0cb5d0 0x7f624c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 -- 172.21.15.7:0/968628078 >> v1:172.21.15.117:6789/0 conn(0x7f624c0cb9a0 legacy=0x7f624c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.186+0000 7f6250b9c700 1 -- 172.21.15.7:0/968628078 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c071ab0 con 0x7f624c15cd40 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f6250b9c700 1 --2- 172.21.15.7:0/968628078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c0cbea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f624c08d420 tx=0x7f624400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265e8359f489b0ac server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f624affd700 1 -- 172.21.15.7:0/968628078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624400e070 con 0x7f624c15cd40 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f624affd700 1 -- 172.21.15.7:0/968628078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400b910 con 0x7f624c15cd40 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f624affd700 1 -- 172.21.15.7:0/968628078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624400bcc0 con 0x7f624c15cd40 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 -- 172.21.15.7:0/968628078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 msgr2=0x7f624c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 --2- 172.21.15.7:0/968628078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c0cbea0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f624c08d420 tx=0x7f624400b330 comp rx=0 tx=0).stop 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 -- 172.21.15.7:0/968628078 shutdown_connections 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 --2- 172.21.15.7:0/968628078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c0cbea0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 --2- 172.21.15.7:0/968628078 >> v2:172.21.15.117:3300/0 conn(0x7f624c0cb5d0 0x7f624c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 -- 172.21.15.7:0/968628078 >> 172.21.15.7:0/968628078 conn(0x7f624c070af0 msgr2=0x7f624c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:27.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.187+0000 7f62525ff700 1 -- 172.21.15.7:0/968628078 shutdown_connections 2023-06-09T11:11:27.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.188+0000 7f62525ff700 1 -- 172.21.15.7:0/968628078 wait complete. 2023-06-09T11:11:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.189+0000 7f62525ff700 1 Processor -- start 2023-06-09T11:11:27.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.189+0000 7f62525ff700 1 -- start start 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 0x7f624c1efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 0x7f624c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1f0370 con 0x7f624c15cd40 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1548c0 con 0x7f624c0cb5d0 2023-06-09T11:11:27.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f62525ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624c152c70 con 0x7f624c0cb9a0 2023-06-09T11:11:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f6250b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f6250b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41526/0 (socket says 172.21.15.7:41526) 2023-06-09T11:11:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.190+0000 7f6250b9c700 1 -- 172.21.15.7:0/2547601017 learned_addr learned my addr 172.21.15.7:0/2547601017 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:27.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f624bfff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 0x7f624c1efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f624b7fe700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 0x7f624c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 -- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 msgr2=0x7f624c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 0x7f624c152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 -- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 msgr2=0x7f624c1efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 0x7f624c1efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 -- 172.21.15.7:0/2547601017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244009b30 con 0x7f624c15cd40 2023-06-09T11:11:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f624bfff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 0x7f624c1efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f624b7fe700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 0x7f624c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:11:27.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6250b9c700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6244012000 tx=0x7f6244004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62440092e0 con 0x7f624c15cd40 2023-06-09T11:11:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624401b430 con 0x7f624c15cd40 2023-06-09T11:11:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c15d4f0 con 0x7f624c15cd40 2023-06-09T11:11:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.191+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624401b7e0 con 0x7f624c15cd40 2023-06-09T11:11:27.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.192+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624c15d210 con 0x7f624c15cd40 2023-06-09T11:11:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.193+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c15d7d0 con 0x7f624c15cd40 2023-06-09T11:11:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.194+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f624401b940 con 0x7f624c15cd40 2023-06-09T11:11:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.195+0000 7f6248ff9700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 0x7f623806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.195+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f624408a7d0 con 0x7f624c15cd40 2023-06-09T11:11:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.195+0000 7f624bfff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 0x7f623806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.195+0000 7f624bfff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 0x7f623806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f623c009b60 tx=0x7f623c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.199+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62440557e0 con 0x7f624c15cd40 2023-06-09T11:11:27.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.453+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f624c0b4570 con 0x7f624c15cd40 2023-06-09T11:11:27.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.453+0000 7f6248ff9700 1 -- 172.21.15.7:0/2547601017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f6244058e00 con 0x7f624c15cd40 2023-06-09T11:11:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:27.459 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:27.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 msgr2=0x7f623806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 0x7f623806d190 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f623c009b60 tx=0x7f623c008040 comp rx=0 tx=0).stop 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 msgr2=0x7f624c153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6244012000 tx=0x7f6244004ad0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 shutdown_connections 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f624c15cd40 0x7f624c153740 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f624c0cb5d0 0x7f624c1efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f623806ace0 0x7f623806d190 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 --2- 172.21.15.7:0/2547601017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f624c0cb9a0 0x7f624c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 >> 172.21.15.7:0/2547601017 conn(0x7f624c070af0 msgr2=0x7f624c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 shutdown_connections 2023-06-09T11:11:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.456+0000 7f62525ff700 1 -- 172.21.15.7:0/2547601017 wait complete. 2023-06-09T11:11:27.463 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:27.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:11:27.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.674+0000 7fb6fd1ff700 1 Processor -- start 2023-06-09T11:11:27.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.674+0000 7fb6fd1ff700 1 -- start start 2023-06-09T11:11:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.675+0000 7fb6fd1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6fd1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80c5a80 con 0x7fb6f8153880 2023-06-09T11:11:27.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6fd1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f815a500 con 0x7fb6f815b4a0 2023-06-09T11:11:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6fd1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80c5f10 con 0x7fb6f80c9f40 2023-06-09T11:11:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb6f8153880 0x7fb6f815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57452/0 (socket says 172.21.15.7:57452) 2023-06-09T11:11:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f6d9d700 1 -- 172.21.15.7:0/4131304017 learned_addr learned my addr 172.21.15.7:0/4131304017 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f759e700 1 --2- 172.21.15.7:0/4131304017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f659c700 1 --1- 172.21.15.7:0/4131304017 >> v1:172.21.15.117:6789/0 conn(0x7fb6f815b4a0 0x7fb6f80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:42534/0 (socket says 172.21.15.7:42534) 2023-06-09T11:11:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f5d9b700 1 -- 172.21.15.7:0/4131304017 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3602722712 0 0) 0x7fb6f80c5a80 con 0x7fb6f8153880 2023-06-09T11:11:27.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f759e700 1 -- 172.21.15.7:0/4131304017 >> v1:172.21.15.117:6789/0 conn(0x7fb6f815b4a0 legacy=0x7fb6f80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f759e700 1 -- 172.21.15.7:0/4131304017 >> v1:172.21.15.7:6789/0 conn(0x7fb6f8153880 legacy=0x7fb6f815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.676+0000 7fb6f759e700 1 -- 172.21.15.7:0/4131304017 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f81596e0 con 0x7fb6f80c9f40 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6f759e700 1 --2- 172.21.15.7:0/4131304017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80c5540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009dc0 tx=0x7fb6f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4aceef1784610f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6f5d9b700 1 -- 172.21.15.7:0/4131304017 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f000e070 con 0x7fb6f80c9f40 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6f5d9b700 1 -- 172.21.15.7:0/4131304017 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f000bbd0 con 0x7fb6f80c9f40 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6f5d9b700 1 -- 172.21.15.7:0/4131304017 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0012430 con 0x7fb6f80c9f40 2023-06-09T11:11:27.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/4131304017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 msgr2=0x7fb6f80c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/4131304017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80c5540 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009dc0 tx=0x7fb6f000b330 comp rx=0 tx=0).stop 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/4131304017 shutdown_connections 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/4131304017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80c5540 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/4131304017 >> 172.21.15.7:0/4131304017 conn(0x7fb6f80737f0 msgr2=0x7fb6f80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/4131304017 shutdown_connections 2023-06-09T11:11:27.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.677+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/4131304017 wait complete. 2023-06-09T11:11:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.679+0000 7fb6fd1ff700 1 Processor -- start 2023-06-09T11:11:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.679+0000 7fb6fd1ff700 1 -- start start 2023-06-09T11:11:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f815b4a0 0x7fb6f81ef320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80b3270 con 0x7fb6f8153880 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80b5120 con 0x7fb6f815b4a0 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6fd1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f80b4c90 con 0x7fb6f80c9f40 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6f659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6f659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41528/0 (socket says 172.21.15.7:41528) 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6f659c700 1 -- 172.21.15.7:0/3782223106 learned_addr learned my addr 172.21.15.7:0/3782223106 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:27.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.680+0000 7fb6f6d9d700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f759e700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f815b4a0 0x7fb6f81ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 -- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 msgr2=0x7fb6f80ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 -- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f815b4a0 msgr2=0x7fb6f81ef320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f815b4a0 0x7fb6f81ef320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 -- 172.21.15.7:0/3782223106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009a70 con 0x7fb6f8153880 2023-06-09T11:11:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6f659c700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c530 tx=0x7fb6ec00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec005cb0 con 0x7fb6f8153880 2023-06-09T11:11:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec005e10 con 0x7fb6f8153880 2023-06-09T11:11:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f8159c60 con 0x7fb6f8153880 2023-06-09T11:11:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0072d0 con 0x7fb6f8153880 2023-06-09T11:11:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.681+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f8159980 con 0x7fb6f8153880 2023-06-09T11:11:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.682+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f8159f40 con 0x7fb6f8153880 2023-06-09T11:11:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.683+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6ec007430 con 0x7fb6f8153880 2023-06-09T11:11:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.684+0000 7fb6df7fe700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.684+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb6ec088600 con 0x7fb6f8153880 2023-06-09T11:11:27.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.684+0000 7fb6f6d9d700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:27.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.685+0000 7fb6f6d9d700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8007b10 tx=0x7fb6e8007a60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:27.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.689+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb6ec0536c0 con 0x7fb6f8153880 2023-06-09T11:11:27.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.896+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6f815a500 con 0x7fb6f8153880 2023-06-09T11:11:27.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.898+0000 7fb6df7fe700 1 -- 172.21.15.7:0/3782223106 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+18183 (secure 0 0 0) 0x7fb6f815a500 con 0x7fb6f8153880 2023-06-09T11:11:27.904 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:27.904 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:10:51.181462+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:11:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 msgr2=0x7fb6e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8007b10 tx=0x7fb6e8007a60 comp rx=0 tx=0).stop 2023-06-09T11:11:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 msgr2=0x7fb6f81eede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c530 tx=0x7fb6ec00c840 comp rx=0 tx=0).stop 2023-06-09T11:11:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 shutdown_connections 2023-06-09T11:11:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6f815b4a0 0x7fb6f81ef320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6f80c9f40 0x7fb6f80ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 --2- 172.21.15.7:0/3782223106 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f8153880 0x7fb6f81eede0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 >> 172.21.15.7:0/3782223106 conn(0x7fb6f80737f0 msgr2=0x7fb6f814e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 shutdown_connections 2023-06-09T11:11:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:27.900+0000 7fb6fd1ff700 1 -- 172.21.15.7:0/3782223106 wait complete. 2023-06-09T11:11:27.924 DEBUG:tasks.mds_thrash:Yielding 2023-06-09T11:11:27.925 INFO:teuthology.run_tasks:Running task workunit... 2023-06-09T11:11:27.937 INFO:tasks.workunit:Pulling workunits from ref 034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:11:27.938 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-06-09T11:11:27.938 INFO:tasks.workunit:timeout=6h 2023-06-09T11:11:27.938 INFO:tasks.workunit:cleanup=True 2023-06-09T11:11:27.939 DEBUG:teuthology.orchestra.run.smithi007:> stat -- /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:27.939 INFO:tasks.mds_thrash.fs.[cephfs]:starting mds_do_thrash for fs cephfs 2023-06-09T11:11:27.940 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 27 secs before thrashing 2023-06-09T11:11:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2023-06-09T11:11:27.985 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 2bh/43d Inode: 1 Links: 2 2023-06-09T11:11:27.985 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-09T11:11:27.985 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:fusefs_t:s0 2023-06-09T11:11:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2023-06-09T11:11:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2023-06-09 11:10:55.185001931 +0000 2023-06-09T11:11:27.986 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2023-06-09 11:11:18.268028837 +0000 2023-06-09T11:11:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: - 2023-06-09T11:11:27.986 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-06-09T11:11:27.986 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-06-09T11:11:28.508 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 034108821cde95ca1c56d458dfccc7ef9e0d964d 2023-06-09T11:11:28.555 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-06-09T11:11:39.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:39.953 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:11:54.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.140+0000 7f4a1cdff700 1 Processor -- start 2023-06-09T11:11:55.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.141+0000 7f4a1cdff700 1 -- start start 2023-06-09T11:11:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a1cdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1815a0c0 0x7f4a181501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a1cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c0140 0x7f4a181506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a1cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a180c0bf0 con 0x7f4a180c1bc0 2023-06-09T11:11:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a1cdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a18070110 con 0x7f4a1815a0c0 2023-06-09T11:11:55.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a1cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a18071300 con 0x7f4a180c0140 2023-06-09T11:11:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a15d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1815a0c0 0x7f4a181501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a15d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a1815a0c0 0x7f4a181501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48192/0 (socket says 172.21.15.7:48192) 2023-06-09T11:11:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a15d9b700 1 -- 172.21.15.7:0/2556850484 learned_addr learned my addr 172.21.15.7:0/2556850484 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:55.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.142+0000 7f4a16d9d700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c0140 0x7f4a181506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a15d9b700 1 -- 172.21.15.7:0/2556850484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c0140 msgr2=0x7f4a181506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:55.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a15d9b700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c0140 0x7f4a181506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a15d9b700 1 -- 172.21.15.7:0/2556850484 >> v1:172.21.15.117:6789/0 conn(0x7f4a180c1bc0 legacy=0x7f4a181599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a15d9b700 1 -- 172.21.15.7:0/2556850484 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180702f0 con 0x7f4a1815a0c0 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a15d9b700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a181501b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00bbc0 tx=0x7f4a0c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee30e2495df0cc35 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a1559a700 1 -- 172.21.15.7:0/2556850484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c00dcf0 con 0x7f4a1815a0c0 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a1559a700 1 -- 172.21.15.7:0/2556850484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c00de50 con 0x7f4a1815a0c0 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.143+0000 7f4a1559a700 1 -- 172.21.15.7:0/2556850484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c010690 con 0x7f4a1815a0c0 2023-06-09T11:11:55.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 -- 172.21.15.7:0/2556850484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 msgr2=0x7f4a181501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a181501b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00bbc0 tx=0x7f4a0c00bed0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 -- 172.21.15.7:0/2556850484 shutdown_connections 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c0140 0x7f4a181506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/2556850484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a181501b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 -- 172.21.15.7:0/2556850484 >> 172.21.15.7:0/2556850484 conn(0x7f4a1806a4d0 msgr2=0x7f4a1806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 -- 172.21.15.7:0/2556850484 shutdown_connections 2023-06-09T11:11:55.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.144+0000 7f4a1cdff700 1 -- 172.21.15.7:0/2556850484 wait complete. 2023-06-09T11:11:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.146+0000 7f4a1cdff700 1 Processor -- start 2023-06-09T11:11:55.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.146+0000 7f4a1cdff700 1 -- start start 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a180c0140 0x7f4a181ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c1bc0 0x7f4a181efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c4c80 con 0x7f4a1815a0c0 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c38a0 con 0x7f4a180c0140 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a1cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a180c2730 con 0x7f4a180c1bc0 2023-06-09T11:11:55.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a16d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a16d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48204/0 (socket says 172.21.15.7:48204) 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.147+0000 7f4a16d9d700 1 -- 172.21.15.7:0/930433405 learned_addr learned my addr 172.21.15.7:0/930433405 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a15d9b700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c1bc0 0x7f4a181efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a1659c700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a180c0140 0x7f4a181ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 -- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c1bc0 msgr2=0x7f4a181efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c1bc0 0x7f4a181efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 -- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a180c0140 msgr2=0x7f4a181ee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a180c0140 0x7f4a181ee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 -- 172.21.15.7:0/930433405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c00b7f0 con 0x7f4a1815a0c0 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.148+0000 7f4a16d9d700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000efc0 tx=0x7f4a1000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.149+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10009ea0 con 0x7f4a1815a0c0 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.149+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1000f040 con 0x7f4a1815a0c0 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.149+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a180c1170 con 0x7f4a1815a0c0 2023-06-09T11:11:55.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.149+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000ced0 con 0x7f4a1815a0c0 2023-06-09T11:11:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.149+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a180c0e90 con 0x7f4a1815a0c0 2023-06-09T11:11:55.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.150+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a180c1450 con 0x7f4a1815a0c0 2023-06-09T11:11:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.151+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a10014480 con 0x7f4a1815a0c0 2023-06-09T11:11:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.152+0000 7f4a06ffd700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 0x7f49f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.152+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4a10089d20 con 0x7f4a1815a0c0 2023-06-09T11:11:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.152+0000 7f4a1659c700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 0x7f49f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.153+0000 7f4a1659c700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 0x7f49f806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007fd0 tx=0x7f4a08009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.157+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4a10054d90 con 0x7f4a1815a0c0 2023-06-09T11:11:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.414+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a180c3a50 con 0x7f4a1815a0c0 2023-06-09T11:11:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.415+0000 7f4a06ffd700 1 -- 172.21.15.7:0/930433405 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f4a180c3a50 con 0x7f4a1815a0c0 2023-06-09T11:11:55.420 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:55.420 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 msgr2=0x7f49f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 0x7f49f806d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a08007fd0 tx=0x7f4a08009040 comp rx=0 tx=0).stop 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 msgr2=0x7f4a180ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000efc0 tx=0x7f4a1000c3a0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 shutdown_connections 2023-06-09T11:11:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a1815a0c0 0x7f4a180ca260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a180c0140 0x7f4a181ee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49f806ace0 0x7f49f806d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 --2- 172.21.15.7:0/930433405 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a180c1bc0 0x7f4a181efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 >> 172.21.15.7:0/930433405 conn(0x7f4a1806a4d0 msgr2=0x7f4a1814e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 shutdown_connections 2023-06-09T11:11:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.417+0000 7f4a1cdff700 1 -- 172.21.15.7:0/930433405 wait complete. 2023-06-09T11:11:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:55.445 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2023-06-09T11:11:55.446 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:11:55.446 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:11:55.464 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:11:55.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.675+0000 7f819b1ff700 1 Processor -- start 2023-06-09T11:11:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.675+0000 7f819b1ff700 1 -- start start 2023-06-09T11:11:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.676+0000 7f819b1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 0x7f81940ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.676+0000 7f819b1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f81940723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.676+0000 7f819b1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819415a390 con 0x7f8194153b30 2023-06-09T11:11:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.676+0000 7f819b1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81940c5f40 con 0x7f8194153760 2023-06-09T11:11:55.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.676+0000 7f819b1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81940ca720 con 0x7f8194071fa0 2023-06-09T11:11:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f8193fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8194153b30 0x7f81940b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60874/0 (socket says 172.21.15.7:60874) 2023-06-09T11:11:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f8193fff700 1 -- 172.21.15.7:0/2602943302 learned_addr learned my addr 172.21.15.7:0/2602943302 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:11:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2602943302 >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 0x7f81940ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f819979c700 1 --2- 172.21.15.7:0/2602943302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f81940723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f819979c700 1 -- 172.21.15.7:0/2602943302 >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 msgr2=0x7f81940ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f819979c700 1 --2- 172.21.15.7:0/2602943302 >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 0x7f81940ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f819979c700 1 -- 172.21.15.7:0/2602943302 >> v1:172.21.15.7:6789/0 conn(0x7f8194153b30 legacy=0x7f81940b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f819979c700 1 -- 172.21.15.7:0/2602943302 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940c5a20 con 0x7f8194071fa0 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2602943302 >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 0x7f81940ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.677+0000 7f81937fe700 1 -- 172.21.15.7:0/2602943302 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300427720 0 0) 0x7f819415a390 con 0x7f8194153b30 2023-06-09T11:11:55.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.678+0000 7f819979c700 1 --2- 172.21.15.7:0/2602943302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f81940723c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f818c009d90 tx=0x7f818c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3d0f265267c655 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.678+0000 7f81937fe700 1 -- 172.21.15.7:0/2602943302 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c00e070 con 0x7f8194071fa0 2023-06-09T11:11:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.678+0000 7f81937fe700 1 -- 172.21.15.7:0/2602943302 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c00bb20 con 0x7f8194071fa0 2023-06-09T11:11:55.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.678+0000 7f81937fe700 1 -- 172.21.15.7:0/2602943302 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c012430 con 0x7f8194071fa0 2023-06-09T11:11:55.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 -- 172.21.15.7:0/2602943302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 msgr2=0x7f81940723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2602943302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f81940723c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f818c009d90 tx=0x7f818c00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 -- 172.21.15.7:0/2602943302 shutdown_connections 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2602943302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f81940723c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2602943302 >> v2:172.21.15.117:3300/0 conn(0x7f8194153760 0x7f81940ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 -- 172.21.15.7:0/2602943302 >> 172.21.15.7:0/2602943302 conn(0x7f81940737f0 msgr2=0x7f8194073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 -- 172.21.15.7:0/2602943302 shutdown_connections 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.679+0000 7f819b1ff700 1 -- 172.21.15.7:0/2602943302 wait complete. 2023-06-09T11:11:55.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.681+0000 7f819b1ff700 1 Processor -- start 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.681+0000 7f819b1ff700 1 -- start start 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.681+0000 7f819b1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f819414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.681+0000 7f819b1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f819b1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8194153b30 0x7f81941f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f819b1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81941f3060 con 0x7f8194153760 2023-06-09T11:11:55.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f819b1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8194157590 con 0x7f8194153b30 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f819b1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8194157830 con 0x7f8194071fa0 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8193fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48220/0 (socket says 172.21.15.7:48220) 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8193fff700 1 -- 172.21.15.7:0/2306132067 learned_addr learned my addr 172.21.15.7:0/2306132067 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f819979c700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8194153b30 0x7f81941f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f819414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 -- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8194153b30 msgr2=0x7f81941f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8194153b30 0x7f81941f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 -- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 msgr2=0x7f81941f25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8198f9b700 1 -- 172.21.15.7:0/2306132067 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880094a0 con 0x7f8194071fa0 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.682+0000 7f8193fff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:55.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f8198f9b700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f819414ab90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f818800e9e0 tx=0x7f818800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800ee80 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818800c850 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800cc00 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818c009a70 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.683+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8194159a60 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.684+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8194159780 con 0x7f8194071fa0 2023-06-09T11:11:55.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.686+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f818800cd60 con 0x7f8194071fa0 2023-06-09T11:11:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.687+0000 7f81917fa700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 0x7f817806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.687+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f81880898e0 con 0x7f8194071fa0 2023-06-09T11:11:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.688+0000 7f8193fff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 0x7f817806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:55.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.688+0000 7f8193fff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 0x7f817806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8194158920 tx=0x7f8180007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:55.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.691+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8188054920 con 0x7f8194071fa0 2023-06-09T11:11:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.951+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81941f2430 con 0x7f8194071fa0 2023-06-09T11:11:55.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.952+0000 7f81917fa700 1 -- 172.21.15.7:0/2306132067 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f8188017090 con 0x7f8194071fa0 2023-06-09T11:11:55.957 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:55.957 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:55.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 msgr2=0x7f817806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 0x7f817806d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8194158920 tx=0x7f8180007e70 comp rx=0 tx=0).stop 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 msgr2=0x7f819414ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f819414ab90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f818800e9e0 tx=0x7f818800ecf0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 shutdown_connections 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8194153b30 0x7f81941f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f817806ace0 0x7f817806d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8194071fa0 0x7f819414ab90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 --2- 172.21.15.7:0/2306132067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8194153760 0x7f81941f25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 >> 172.21.15.7:0/2306132067 conn(0x7f81940737f0 msgr2=0x7f819414dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 shutdown_connections 2023-06-09T11:11:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:55.955+0000 7f819b1ff700 1 -- 172.21.15.7:0/2306132067 wait complete. 2023-06-09T11:11:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:55.978 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:11:57.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:11:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.191+0000 7f758abff700 1 Processor -- start 2023-06-09T11:11:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.192+0000 7f758abff700 1 -- start start 2023-06-09T11:11:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.193+0000 7f758abff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f75840ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.193+0000 7f758abff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7584071fa0 0x7f75840723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.193+0000 7f758abff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758415a390 con 0x7f7584153b30 2023-06-09T11:11:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.193+0000 7f758abff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75840c5f40 con 0x7f7584071fa0 2023-06-09T11:11:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758abff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840ca720 con 0x7f7584153760 2023-06-09T11:11:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7584071fa0 0x7f75840723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758899b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f75840ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7584071fa0 0x7f75840723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:32936/0 (socket says 172.21.15.7:32936) 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 -- 172.21.15.7:0/3974635798 learned_addr learned my addr 172.21.15.7:0/3974635798 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 -- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 msgr2=0x7f75840ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 --2- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f75840ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 -- 172.21.15.7:0/3974635798 >> v1:172.21.15.117:6789/0 conn(0x7f7584153b30 legacy=0x7f75840b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f758919c700 1 -- 172.21.15.7:0/3974635798 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75840c5a20 con 0x7f7584071fa0 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.194+0000 7f75837fe700 1 -- 172.21.15.7:0/3974635798 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001737976 0 0) 0x7f758415a390 con 0x7f7584153b30 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.195+0000 7f758919c700 1 --2- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75840723c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f757c009e50 tx=0x7f757c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2706f0e1d580edd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.195+0000 7f75837fe700 1 -- 172.21.15.7:0/3974635798 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00e070 con 0x7f7584071fa0 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.195+0000 7f75837fe700 1 -- 172.21.15.7:0/3974635798 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c00b910 con 0x7f7584071fa0 2023-06-09T11:11:58.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.195+0000 7f75837fe700 1 -- 172.21.15.7:0/3974635798 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00bcc0 con 0x7f7584071fa0 2023-06-09T11:11:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 -- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 msgr2=0x7f75840723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 --2- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75840723c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f757c009e50 tx=0x7f757c00b330 comp rx=0 tx=0).stop 2023-06-09T11:11:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 -- 172.21.15.7:0/3974635798 shutdown_connections 2023-06-09T11:11:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 --2- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75840723c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 --2- 172.21.15.7:0/3974635798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f75840ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 -- 172.21.15.7:0/3974635798 >> 172.21.15.7:0/3974635798 conn(0x7f75840737f0 msgr2=0x7f7584073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 -- 172.21.15.7:0/3974635798 shutdown_connections 2023-06-09T11:11:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.196+0000 7f758abff700 1 -- 172.21.15.7:0/3974635798 wait complete. 2023-06-09T11:11:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.199+0000 7f758abff700 1 Processor -- start 2023-06-09T11:11:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.199+0000 7f758abff700 1 -- start start 2023-06-09T11:11:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.200+0000 7f758abff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75841424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.200+0000 7f758abff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f7584158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758abff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 0x7f75840c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758abff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7584159530 con 0x7f7584153b30 2023-06-09T11:11:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758abff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75841f2bb0 con 0x7f7584071fa0 2023-06-09T11:11:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758abff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75840c8190 con 0x7f7584153760 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758919c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 0x7f75840c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758919c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 0x7f75840c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48224/0 (socket says 172.21.15.7:48224) 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758919c700 1 -- 172.21.15.7:0/3149211213 learned_addr learned my addr 172.21.15.7:0/3149211213 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f758899b700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75841424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.201+0000 7f7583fff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f7584158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 -- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 msgr2=0x7f7584158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f7584158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 -- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 msgr2=0x7f75840c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 0x7f75840c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 -- 172.21.15.7:0/3149211213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c009b30 con 0x7f7584071fa0 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f7583fff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f7584158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758899b700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75841424e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f757800e900 tx=0x7f757800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:58.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578013680 con 0x7f7584071fa0 2023-06-09T11:11:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75780137e0 con 0x7f7584071fa0 2023-06-09T11:11:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7584159a60 con 0x7f7584071fa0 2023-06-09T11:11:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578013b50 con 0x7f7584071fa0 2023-06-09T11:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.202+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7584159780 con 0x7f7584071fa0 2023-06-09T11:11:58.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.204+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7584159d40 con 0x7f7584071fa0 2023-06-09T11:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.205+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7578013cb0 con 0x7f7584071fa0 2023-06-09T11:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.206+0000 7f75817fa700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.206+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f75780898b0 con 0x7f7584071fa0 2023-06-09T11:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.206+0000 7f7583fff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:11:58.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.207+0000 7f7583fff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 0x7f756806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75840b2890 tx=0x7f7570009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:11:58.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.212+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f75780547f0 con 0x7f7584071fa0 2023-06-09T11:11:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.476+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7584074cf0 con 0x7f7584071fa0 2023-06-09T11:11:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.477+0000 7f75817fa700 1 -- 172.21.15.7:0/3149211213 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f7578021090 con 0x7f7584071fa0 2023-06-09T11:11:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:11:58.485 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:11:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 msgr2=0x7f756806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 0x7f756806d190 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75840b2890 tx=0x7f7570009040 comp rx=0 tx=0).stop 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 msgr2=0x7f75841424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75841424e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f757800e900 tx=0x7f757800ec10 comp rx=0 tx=0).stop 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 shutdown_connections 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7584153b30 0x7f75840c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7584071fa0 0x7f75841424e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 --2- 172.21.15.7:0/3149211213 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7584153760 0x7f7584158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:11:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 >> 172.21.15.7:0/3149211213 conn(0x7f75840737f0 msgr2=0x7f758414d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:11:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 shutdown_connections 2023-06-09T11:11:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:11:58.480+0000 7f758abff700 1 -- 172.21.15.7:0/3149211213 wait complete. 2023-06-09T11:11:58.488 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:11:58.509 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:00.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:00.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.745+0000 7fd74a99d700 1 Processor -- start 2023-06-09T11:12:00.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.745+0000 7fd74a99d700 1 -- start start 2023-06-09T11:12:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.747+0000 7fd74a99d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd744156f20 0x7fd7440cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.747+0000 7fd74a99d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd744071d50 0x7fd744072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.747+0000 7fd74a99d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7440d01a0 con 0x7fd744156b50 2023-06-09T11:12:00.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.747+0000 7fd74a99d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd744072690 con 0x7fd744071d50 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.747+0000 7fd74a99d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7440cb2b0 con 0x7fd744156f20 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd744071d50 0x7fd744072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd744071d50 0x7fd744072150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48228/0 (socket says 172.21.15.7:48228) 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 -- 172.21.15.7:0/518727728 learned_addr learned my addr 172.21.15.7:0/518727728 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74919a700 1 --2- 172.21.15.7:0/518727728 >> v2:172.21.15.117:3300/0 conn(0x7fd744156f20 0x7fd7440cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 -- 172.21.15.7:0/518727728 >> v2:172.21.15.117:3300/0 conn(0x7fd744156f20 msgr2=0x7fd7440cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:00.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 --2- 172.21.15.7:0/518727728 >> v2:172.21.15.117:3300/0 conn(0x7fd744156f20 0x7fd7440cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:00.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 -- 172.21.15.7:0/518727728 >> v1:172.21.15.117:6789/0 conn(0x7fd744156b50 legacy=0x7fd7440caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:00.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.748+0000 7fd74a19c700 1 -- 172.21.15.7:0/518727728 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440b54a0 con 0x7fd744071d50 2023-06-09T11:12:00.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd74a19c700 1 --2- 172.21.15.7:0/518727728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744071d50 0x7fd744072150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd7440cd350 tx=0x7fd73c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a0b02498996937 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd748999700 1 -- 172.21.15.7:0/518727728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00e070 con 0x7fd744071d50 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd748999700 1 -- 172.21.15.7:0/518727728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c00b910 con 0x7fd744071d50 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd748999700 1 -- 172.21.15.7:0/518727728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00bcc0 con 0x7fd744071d50 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd74a99d700 1 -- 172.21.15.7:0/518727728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744071d50 msgr2=0x7fd744072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.749+0000 7fd74a99d700 1 --2- 172.21.15.7:0/518727728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744071d50 0x7fd744072150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd7440cd350 tx=0x7fd73c00b330 comp rx=0 tx=0).stop 2023-06-09T11:12:00.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 -- 172.21.15.7:0/518727728 shutdown_connections 2023-06-09T11:12:00.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 --2- 172.21.15.7:0/518727728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744071d50 0x7fd744072150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:00.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 --2- 172.21.15.7:0/518727728 >> v2:172.21.15.117:3300/0 conn(0x7fd744156f20 0x7fd7440cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:00.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 -- 172.21.15.7:0/518727728 >> 172.21.15.7:0/518727728 conn(0x7fd7440737f0 msgr2=0x7fd744073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:00.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 -- 172.21.15.7:0/518727728 shutdown_connections 2023-06-09T11:12:00.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.750+0000 7fd74a99d700 1 -- 172.21.15.7:0/518727728 wait complete. 2023-06-09T11:12:00.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.752+0000 7fd74a99d700 1 Processor -- start 2023-06-09T11:12:00.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.753+0000 7fd74a99d700 1 -- start start 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 0x7fd7441e2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd744156f20 0x7fd7441e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441ef580 con 0x7fd744156b50 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441e1910 con 0x7fd744071d50 2023-06-09T11:12:00.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.754+0000 7fd74a99d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7441efa90 con 0x7fd744156f20 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74919a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 0x7fd7441e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:32954/0 (socket says 172.21.15.7:32954) 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 -- 172.21.15.7:0/1006016737 learned_addr learned my addr 172.21.15.7:0/1006016737 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74919a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 0x7fd7441e2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48236/0 (socket says 172.21.15.7:48236) 2023-06-09T11:12:00.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74a19c700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd744156f20 0x7fd7441e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 -- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd744156f20 msgr2=0x7fd7441e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd744156f20 0x7fd7441e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 -- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 msgr2=0x7fd7441e2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:00.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 0x7fd7441e2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.755+0000 7fd74999b700 1 -- 172.21.15.7:0/1006016737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c009b30 con 0x7fd744071d50 2023-06-09T11:12:00.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.756+0000 7fd74999b700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c4d0 tx=0x7fd74000c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.756+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400052d0 con 0x7fd744071d50 2023-06-09T11:12:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.756+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740005430 con 0x7fd744071d50 2023-06-09T11:12:00.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.756+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7400038f0 con 0x7fd744071d50 2023-06-09T11:12:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.756+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7440cf590 con 0x7fd744071d50 2023-06-09T11:12:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.757+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7440cba50 con 0x7fd744071d50 2023-06-09T11:12:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.757+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7440cf870 con 0x7fd744071d50 2023-06-09T11:12:00.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.760+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd740003a50 con 0x7fd744071d50 2023-06-09T11:12:00.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.761+0000 7fd73a7fc700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:00.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.761+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fd7400888a0 con 0x7fd744071d50 2023-06-09T11:12:00.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.761+0000 7fd74919a700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:00.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.762+0000 7fd74919a700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 0x7fd72806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7440b3320 tx=0x7fd734009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:00.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:00.766+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd740053900 con 0x7fd744071d50 2023-06-09T11:12:01.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.060+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd74404f760 con 0x7fd744071d50 2023-06-09T11:12:01.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.060+0000 7fd73a7fc700 1 -- 172.21.15.7:0/1006016737 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7fd740056f20 con 0x7fd744071d50 2023-06-09T11:12:01.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:01.066 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 msgr2=0x7fd72806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 0x7fd72806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd7440b3320 tx=0x7fd734009210 comp rx=0 tx=0).stop 2023-06-09T11:12:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 msgr2=0x7fd7441e0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c4d0 tx=0x7fd74000c7e0 comp rx=0 tx=0).stop 2023-06-09T11:12:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 shutdown_connections 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd72806ace0 0x7fd72806d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd744156f20 0x7fd7441e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd744071d50 0x7fd7441e0730 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 --2- 172.21.15.7:0/1006016737 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd744156b50 0x7fd7441e2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.063+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 >> 172.21.15.7:0/1006016737 conn(0x7fd7440737f0 msgr2=0x7fd74415a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.064+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 shutdown_connections 2023-06-09T11:12:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:01.064+0000 7fd74a99d700 1 -- 172.21.15.7:0/1006016737 wait complete. 2023-06-09T11:12:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:12:01.091 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:03.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.356+0000 7f5e1dd9d700 1 Processor -- start 2023-06-09T11:12:03.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.356+0000 7f5e1dd9d700 1 -- start start 2023-06-09T11:12:03.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1dd9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 0x7f5e180ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1dd9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e180c74d0 0x7f5e1806ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1dd9d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e18156b10 con 0x7f5e18150300 2023-06-09T11:12:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1dd9d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e180c7950 con 0x7f5e180c74d0 2023-06-09T11:12:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1dd9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e181506d0 con 0x7f5e180722e0 2023-06-09T11:12:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1cd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e18150300 0x7f5e1806c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58734/0 (socket says 172.21.15.7:58734) 2023-06-09T11:12:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1cd9b700 1 -- 172.21.15.7:0/3507692136 learned_addr learned my addr 172.21.15.7:0/3507692136 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/3507692136 >> v2:172.21.15.117:3300/0 conn(0x7f5e180c74d0 0x7f5e1806ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.359+0000 7f5e17fff700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 0x7f5e180ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e177fe700 1 -- 172.21.15.7:0/3507692136 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305239589 0 0) 0x7f5e18156b10 con 0x7f5e18150300 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e177fe700 1 -- 172.21.15.7:0/3507692136 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e080035b0 con 0x7f5e18150300 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e1d59c700 1 -- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 msgr2=0x7f5e180ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 0x7f5e180ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e1d59c700 1 -- 172.21.15.7:0/3507692136 >> v1:172.21.15.7:6789/0 conn(0x7f5e18150300 legacy=0x7f5e1806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e1d59c700 1 -- 172.21.15.7:0/3507692136 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e180aee00 con 0x7f5e180c74d0 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e17fff700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 0x7f5e180ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:03.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.360+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180c74d0 0x7f5e1806ae90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009dc0 tx=0x7f5e1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a388a4ab31c5cf5d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e177fe700 1 -- 172.21.15.7:0/3507692136 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000e070 con 0x7f5e180c74d0 2023-06-09T11:12:03.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e177fe700 1 -- 172.21.15.7:0/3507692136 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1000bae0 con 0x7f5e180c74d0 2023-06-09T11:12:03.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e177fe700 1 -- 172.21.15.7:0/3507692136 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1000be90 con 0x7f5e180c74d0 2023-06-09T11:12:03.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180c74d0 msgr2=0x7f5e1806ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180c74d0 0x7f5e1806ae90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009dc0 tx=0x7f5e1000b330 comp rx=0 tx=0).stop 2023-06-09T11:12:03.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/3507692136 shutdown_connections 2023-06-09T11:12:03.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180c74d0 0x7f5e1806ae90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/3507692136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180722e0 0x7f5e180ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/3507692136 >> 172.21.15.7:0/3507692136 conn(0x7f5e180cc140 msgr2=0x7f5e180cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.361+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/3507692136 shutdown_connections 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.362+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/3507692136 wait complete. 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.364+0000 7f5e1dd9d700 1 Processor -- start 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.364+0000 7f5e1dd9d700 1 -- start start 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 0x7f5e180b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180c74d0 0x7f5e181eda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 0x7f5e180adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180afa80 con 0x7f5e18150300 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180aa490 con 0x7f5e180722e0 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1dd9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e180b0300 con 0x7f5e180c74d0 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1cd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 0x7f5e180b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1d59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 0x7f5e180adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1cd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 0x7f5e180b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:40432/0 (socket says 172.21.15.7:40432) 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e1cd9b700 1 -- 172.21.15.7:0/2986740131 learned_addr learned my addr 172.21.15.7:0/2986740131 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.366+0000 7f5e17fff700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180c74d0 0x7f5e181eda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 -- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180c74d0 msgr2=0x7f5e181eda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180c74d0 0x7f5e181eda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 -- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 msgr2=0x7f5e180b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 0x7f5e180b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 -- 172.21.15.7:0/2986740131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e040094a0 con 0x7f5e18150300 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1d59c700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 0x7f5e180adb80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e10016000 tx=0x7f5e10004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004550 con 0x7f5e18150300 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e10020070 con 0x7f5e18150300 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e10009a70 con 0x7f5e18150300 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.367+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004910 con 0x7f5e18150300 2023-06-09T11:12:03.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.368+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e181561e0 con 0x7f5e18150300 2023-06-09T11:12:03.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.369+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e18155f20 con 0x7f5e18150300 2023-06-09T11:12:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.370+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5e10027480 con 0x7f5e18150300 2023-06-09T11:12:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.371+0000 7f5e157fa700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 0x7f5dfc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.371+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5e10022070 con 0x7f5e18150300 2023-06-09T11:12:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.372+0000 7f5e1cd9b700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 0x7f5dfc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:03.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.372+0000 7f5e1cd9b700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 0x7f5dfc06d1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0011e0 tx=0x7f5e0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:03.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.376+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5e10056420 con 0x7f5e18150300 2023-06-09T11:12:03.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.734+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e181564c0 con 0x7f5e18150300 2023-06-09T11:12:03.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.735+0000 7f5e157fa700 1 -- 172.21.15.7:0/2986740131 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f5e10059a40 con 0x7f5e18150300 2023-06-09T11:12:03.740 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:03.740 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 msgr2=0x7f5dfc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 0x7f5dfc06d1c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0011e0 tx=0x7f5e0c009040 comp rx=0 tx=0).stop 2023-06-09T11:12:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 msgr2=0x7f5e180adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 0x7f5e180adb80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5e10016000 tx=0x7f5e10004ad0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 shutdown_connections 2023-06-09T11:12:03.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e18150300 0x7f5e180adb80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e180722e0 0x7f5e180b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dfc06ad10 0x7f5dfc06d1c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 --2- 172.21.15.7:0/2986740131 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e180c74d0 0x7f5e181eda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 >> 172.21.15.7:0/2986740131 conn(0x7f5e180cc140 msgr2=0x7f5e1806b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.738+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 shutdown_connections 2023-06-09T11:12:03.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:03.739+0000 7f5e1dd9d700 1 -- 172.21.15.7:0/2986740131 wait complete. 2023-06-09T11:12:03.747 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:12:03.769 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:05.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.107+0000 7f2ee93ff700 1 Processor -- start 2023-06-09T11:12:06.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.108+0000 7f2ee93ff700 1 -- start start 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee93ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ee4153880 0x7f2ee415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee93ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee415a500 con 0x7f2ee40c9f40 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee93ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee415da00 con 0x7f2ee415b4a0 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee93ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee40c5f10 con 0x7f2ee4153880 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2ee40c9f40 0x7f2ee415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58738/0 (socket says 172.21.15.7:58738) 2023-06-09T11:12:06.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee27fc700 1 -- 172.21.15.7:0/4287481842 learned_addr learned my addr 172.21.15.7:0/4287481842 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:06.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.110+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/4287481842 >> v2:172.21.15.117:3300/0 conn(0x7f2ee4153880 0x7f2ee415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee1ffb700 1 -- 172.21.15.7:0/4287481842 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277744298 0 0) 0x7f2ed8001110 con 0x7f2ee40c9f40 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee1ffb700 1 -- 172.21.15.7:0/4287481842 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ecc0035b0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/4287481842 >> v1:172.21.15.117:6789/0 conn(0x7f2ee415b4a0 legacy=0x7f2ee40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/4287481842 >> v1:172.21.15.7:6789/0 conn(0x7f2ee40c9f40 legacy=0x7f2ee415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/4287481842 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee41596e0 con 0x7f2ee4153880 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.111+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/4287481842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee4153880 0x7f2ee415d4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009840 tx=0x7f2ed4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e93dca220a4f9536 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:06.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.112+0000 7f2ee1ffb700 1 -- 172.21.15.7:0/4287481842 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011830 con 0x7f2ee4153880 2023-06-09T11:12:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.112+0000 7f2ee1ffb700 1 -- 172.21.15.7:0/4287481842 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed4011990 con 0x7f2ee4153880 2023-06-09T11:12:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.112+0000 7f2ee1ffb700 1 -- 172.21.15.7:0/4287481842 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011d00 con 0x7f2ee4153880 2023-06-09T11:12:06.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 -- 172.21.15.7:0/4287481842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee4153880 msgr2=0x7f2ee415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/4287481842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee4153880 0x7f2ee415d4c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009840 tx=0x7f2ed4009b50 comp rx=0 tx=0).stop 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 -- 172.21.15.7:0/4287481842 shutdown_connections 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/4287481842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee4153880 0x7f2ee415d4c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 -- 172.21.15.7:0/4287481842 >> 172.21.15.7:0/4287481842 conn(0x7f2ee40737f0 msgr2=0x7f2ee40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 -- 172.21.15.7:0/4287481842 shutdown_connections 2023-06-09T11:12:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.113+0000 7f2ee93ff700 1 -- 172.21.15.7:0/4287481842 wait complete. 2023-06-09T11:12:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.116+0000 7f2ee93ff700 1 Processor -- start 2023-06-09T11:12:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.116+0000 7f2ee93ff700 1 -- start start 2023-06-09T11:12:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 0x7f2ee414b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:06.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 0x7f2ee406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 0x7f2ee406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee415b4a0 0x7f2ee4145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 0x7f2ee406a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43626/0 (socket says 172.21.15.7:43626) 2023-06-09T11:12:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee27fc700 1 -- 172.21.15.7:0/1304259712 learned_addr learned my addr 172.21.15.7:0/1304259712 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4149e90 con 0x7f2ee4153880 2023-06-09T11:12:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ceed0 con 0x7f2ee415b4a0 2023-06-09T11:12:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee414a0e0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.118+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 0x7f2ee414b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee37fe700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee415b4a0 0x7f2ee4145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee415b4a0 msgr2=0x7f2ee4145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee415b4a0 0x7f2ee4145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 msgr2=0x7f2ee406a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 0x7f2ee406a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40094a0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.119+0000 7f2ee2ffd700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 0x7f2ee414b530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4016000 tx=0x7f2ed4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.120+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40036a0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.120+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed401f070 con 0x7f2ee40c9f40 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.120+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee4159c60 con 0x7f2ee40c9f40 2023-06-09T11:12:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.120+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003a60 con 0x7f2ee40c9f40 2023-06-09T11:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.120+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee4159980 con 0x7f2ee40c9f40 2023-06-09T11:12:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.122+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4159f40 con 0x7f2ee40c9f40 2023-06-09T11:12:06.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.124+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ed4003bc0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.125+0000 7f2ecb7fe700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.125+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2ed4021070 con 0x7f2ee40c9f40 2023-06-09T11:12:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.126+0000 7f2ee27fc700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:06.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.126+0000 7f2ee27fc700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 0x7f2ed006d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8007fd0 tx=0x7f2ed8007d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:06.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.130+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2ed4055be0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.637+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ee4074cf0 con 0x7f2ee40c9f40 2023-06-09T11:12:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.638+0000 7f2ecb7fe700 1 -- 172.21.15.7:0/1304259712 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f2ed4015090 con 0x7f2ee40c9f40 2023-06-09T11:12:06.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:06.644 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 msgr2=0x7f2ed006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 0x7f2ed006d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8007fd0 tx=0x7f2ed8007d00 comp rx=0 tx=0).stop 2023-06-09T11:12:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 msgr2=0x7f2ee414b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 0x7f2ee414b530 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4016000 tx=0x7f2ed4004750 comp rx=0 tx=0).stop 2023-06-09T11:12:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 shutdown_connections 2023-06-09T11:12:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee415b4a0 0x7f2ee4145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed006ace0 0x7f2ed006d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee40c9f40 0x7f2ee414b530 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 --2- 172.21.15.7:0/1304259712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee4153880 0x7f2ee406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.641+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 >> 172.21.15.7:0/1304259712 conn(0x7f2ee40737f0 msgr2=0x7f2ee415b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.642+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 shutdown_connections 2023-06-09T11:12:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:06.642+0000 7f2ee93ff700 1 -- 172.21.15.7:0/1304259712 wait complete. 2023-06-09T11:12:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:12:06.675 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:08.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.007+0000 7f00439ff700 1 Processor -- start 2023-06-09T11:12:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.007+0000 7f00439ff700 1 -- start start 2023-06-09T11:12:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.009+0000 7f00439ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f003c0c5b40 0x7f003c0c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.009+0000 7f00439ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c15dab0 con 0x7f003c142850 2023-06-09T11:12:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.010+0000 7f00439ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c159aa0 con 0x7f003c0c6980 2023-06-09T11:12:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.010+0000 7f00439ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c15d210 con 0x7f003c0c5b40 2023-06-09T11:12:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.010+0000 7f004179b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f003c0c5b40 0x7f003c0c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.010+0000 7f0041f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f003c142850 0x7f003c0b08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58744/0 (socket says 172.21.15.7:58744) 2023-06-09T11:12:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.010+0000 7f0041f9c700 1 -- 172.21.15.7:0/2225631488 learned_addr learned my addr 172.21.15.7:0/2225631488 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f004179b700 1 -- 172.21.15.7:0/2225631488 >> v1:172.21.15.117:6789/0 conn(0x7f003c0c6980 legacy=0x7f003c159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f004179b700 1 -- 172.21.15.7:0/2225631488 >> v1:172.21.15.7:6789/0 conn(0x7f003c142850 legacy=0x7f003c0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f004179b700 1 -- 172.21.15.7:0/2225631488 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c15d4f0 con 0x7f003c0c5b40 2023-06-09T11:12:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f0033fff700 1 -- 172.21.15.7:0/2225631488 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454257497 0 0) 0x7f003c15dab0 con 0x7f003c142850 2023-06-09T11:12:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f004179b700 1 --2- 172.21.15.7:0/2225631488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c0c5b40 0x7f003c0c0b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00380097c0 tx=0x7f0038009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ebef52bfae1e32 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.011+0000 7f0033fff700 1 -- 172.21.15.7:0/2225631488 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038004230 con 0x7f003c0c5b40 2023-06-09T11:12:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.012+0000 7f0033fff700 1 -- 172.21.15.7:0/2225631488 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0038004390 con 0x7f003c0c5b40 2023-06-09T11:12:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.012+0000 7f0033fff700 1 -- 172.21.15.7:0/2225631488 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380036a0 con 0x7f003c0c5b40 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.012+0000 7f00439ff700 1 -- 172.21.15.7:0/2225631488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c0c5b40 msgr2=0x7f003c0c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.012+0000 7f00439ff700 1 --2- 172.21.15.7:0/2225631488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c0c5b40 0x7f003c0c0b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f00380097c0 tx=0x7f0038009ad0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.013+0000 7f00439ff700 1 -- 172.21.15.7:0/2225631488 shutdown_connections 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.013+0000 7f00439ff700 1 --2- 172.21.15.7:0/2225631488 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c0c5b40 0x7f003c0c0b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.013+0000 7f00439ff700 1 -- 172.21.15.7:0/2225631488 >> 172.21.15.7:0/2225631488 conn(0x7f003c06a4d0 msgr2=0x7f003c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.013+0000 7f00439ff700 1 -- 172.21.15.7:0/2225631488 shutdown_connections 2023-06-09T11:12:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.013+0000 7f00439ff700 1 -- 172.21.15.7:0/2225631488 wait complete. 2023-06-09T11:12:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.016+0000 7f00439ff700 1 Processor -- start 2023-06-09T11:12:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.016+0000 7f00439ff700 1 -- start start 2023-06-09T11:12:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f003c0c5b40 0x7f003c15a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f003c0c6980 0x7f003c138ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c15cd10 con 0x7f003c142850 2023-06-09T11:12:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0caeb0 con 0x7f003c0c5b40 2023-06-09T11:12:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.017+0000 7f00439ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0cb190 con 0x7f003c0c6980 2023-06-09T11:12:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43634/0 (socket says 172.21.15.7:43634) 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 -- 172.21.15.7:0/2290163727 learned_addr learned my addr 172.21.15.7:0/2290163727 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f004179b700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f003c0c5b40 0x7f003c15a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0040f9a700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f003c0c6980 0x7f003c138ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 -- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f003c0c6980 msgr2=0x7f003c138ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f003c0c6980 0x7f003c138ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 -- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f003c0c5b40 msgr2=0x7f003c15a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f003c0c5b40 0x7f003c15a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.018+0000 7f0041f9c700 1 -- 172.21.15.7:0/2290163727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380094a0 con 0x7f003c142850 2023-06-09T11:12:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f0041f9c700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f003400cc20 tx=0x7f003400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400aaa0 con 0x7f003c142850 2023-06-09T11:12:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400ac20 con 0x7f003c142850 2023-06-09T11:12:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c159730 con 0x7f003c142850 2023-06-09T11:12:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00340070d0 con 0x7f003c142850 2023-06-09T11:12:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.019+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c15d7d0 con 0x7f003c142850 2023-06-09T11:12:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.021+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c0b0fd0 con 0x7f003c142850 2023-06-09T11:12:09.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.023+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0034007230 con 0x7f003c142850 2023-06-09T11:12:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.024+0000 7f0031ffb700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 0x7f002006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.025+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0034089120 con 0x7f003c142850 2023-06-09T11:12:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.025+0000 7f004179b700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 0x7f002006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.026+0000 7f004179b700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 0x7f002006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00380090b0 tx=0x7f0038008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:09.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.033+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0034054060 con 0x7f003c142850 2023-06-09T11:12:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.533+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f003c074cf0 con 0x7f003c142850 2023-06-09T11:12:09.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.534+0000 7f0031ffb700 1 -- 172.21.15.7:0/2290163727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f0034057680 con 0x7f003c142850 2023-06-09T11:12:09.539 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:09.539 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.537+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 msgr2=0x7f002006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.537+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 0x7f002006d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f00380090b0 tx=0x7f0038008030 comp rx=0 tx=0).stop 2023-06-09T11:12:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.537+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 msgr2=0x7f003c139010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:09.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.537+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f003400cc20 tx=0x7f003400cf30 comp rx=0 tx=0).stop 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 shutdown_connections 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f003c142850 0x7f003c139010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f003c0c5b40 0x7f003c15a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f002006ace0 0x7f002006d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 --2- 172.21.15.7:0/2290163727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f003c0c6980 0x7f003c138ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 >> 172.21.15.7:0/2290163727 conn(0x7f003c06a4d0 msgr2=0x7f003c0c5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 shutdown_connections 2023-06-09T11:12:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:09.538+0000 7f00439ff700 1 -- 172.21.15.7:0/2290163727 wait complete. 2023-06-09T11:12:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:12:09.572 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:10.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:10.141 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:11.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.904+0000 7f8b03bff700 1 Processor -- start 2023-06-09T11:12:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.904+0000 7f8b03bff700 1 -- start start 2023-06-09T11:12:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.906+0000 7f8b03bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 0x7f8afc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.906+0000 7f8b03bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8afc06e460 0x7f8afc06cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.906+0000 7f8b03bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc15cac0 con 0x7f8afc06a4d0 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.906+0000 7f8b03bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0c8870 con 0x7f8afc0c6280 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.906+0000 7f8b03bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc0c37d0 con 0x7f8afc06e460 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0199b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 0x7f8afc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0219c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8afc06e460 0x7f8afc06cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0199b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 0x7f8afc0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43640/0 (socket says 172.21.15.7:43640) 2023-06-09T11:12:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0199b700 1 -- 172.21.15.7:0/2451512604 learned_addr learned my addr 172.21.15.7:0/2451512604 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0219c700 1 -- 172.21.15.7:0/2451512604 >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 msgr2=0x7f8afc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0219c700 1 --2- 172.21.15.7:0/2451512604 >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 0x7f8afc0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0219c700 1 -- 172.21.15.7:0/2451512604 >> v1:172.21.15.117:6789/0 conn(0x7f8afc06a4d0 legacy=0x7f8afc0737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.907+0000 7f8b0219c700 1 -- 172.21.15.7:0/2451512604 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc15cd30 con 0x7f8afc06e460 2023-06-09T11:12:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.908+0000 7f8b0219c700 1 --2- 172.21.15.7:0/2451512604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc06e460 0x7f8afc06cf30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8afc13f730 tx=0x7f8af400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138879aefbb80f46 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.908+0000 7f8b00999700 1 -- 172.21.15.7:0/2451512604 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400e070 con 0x7f8afc06e460 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.908+0000 7f8b00999700 1 -- 172.21.15.7:0/2451512604 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af400bb20 con 0x7f8afc06e460 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.908+0000 7f8b00999700 1 -- 172.21.15.7:0/2451512604 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4012430 con 0x7f8afc06e460 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 -- 172.21.15.7:0/2451512604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc06e460 msgr2=0x7f8afc06cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 --2- 172.21.15.7:0/2451512604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc06e460 0x7f8afc06cf30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8afc13f730 tx=0x7f8af400b330 comp rx=0 tx=0).stop 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 -- 172.21.15.7:0/2451512604 shutdown_connections 2023-06-09T11:12:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 --2- 172.21.15.7:0/2451512604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc06e460 0x7f8afc06cf30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 --2- 172.21.15.7:0/2451512604 >> v2:172.21.15.7:3300/0 conn(0x7f8afc0c6280 0x7f8afc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 -- 172.21.15.7:0/2451512604 >> 172.21.15.7:0/2451512604 conn(0x7f8afc06f980 msgr2=0x7f8afc0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.909+0000 7f8b03bff700 1 -- 172.21.15.7:0/2451512604 shutdown_connections 2023-06-09T11:12:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.910+0000 7f8b03bff700 1 -- 172.21.15.7:0/2451512604 wait complete. 2023-06-09T11:12:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.912+0000 7f8b03bff700 1 Processor -- start 2023-06-09T11:12:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.913+0000 7f8b03bff700 1 -- start start 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8afc06a4d0 0x7f8afc13f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc0c6280 0x7f8afc0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc156d00 con 0x7f8afc06a4d0 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc157990 con 0x7f8afc0c6280 2023-06-09T11:12:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.914+0000 7f8b03bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc156f10 con 0x7f8afc06e460 2023-06-09T11:12:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8afc06a4d0 0x7f8afc13f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:41222/0 (socket says 172.21.15.7:41222) 2023-06-09T11:12:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 -- 172.21.15.7:0/3219924653 learned_addr learned my addr 172.21.15.7:0/3219924653 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0219c700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc0c6280 0x7f8afc0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 -- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc0c6280 msgr2=0x7f8afc0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc0c6280 0x7f8afc0cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 -- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8afc06a4d0 msgr2=0x7f8afc13f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8afc06a4d0 0x7f8afc13f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.915+0000 7f8b0119a700 1 -- 172.21.15.7:0/3219924653 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4009a70 con 0x7f8afc06e460 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8b0119a700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c4e0 tx=0x7f8aec00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0052d0 con 0x7f8afc06e460 2023-06-09T11:12:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec005430 con 0x7f8afc06e460 2023-06-09T11:12:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0038c0 con 0x7f8afc06e460 2023-06-09T11:12:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc15d2f0 con 0x7f8afc06e460 2023-06-09T11:12:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.916+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc15d010 con 0x7f8afc06e460 2023-06-09T11:12:11.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.918+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc15d5d0 con 0x7f8afc06e460 2023-06-09T11:12:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.920+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8aec003a20 con 0x7f8afc06e460 2023-06-09T11:12:11.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.922+0000 7f8af27fc700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 0x7f8ae006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:11.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.922+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8aec088cf0 con 0x7f8afc06e460 2023-06-09T11:12:11.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.923+0000 7f8b0199b700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 0x7f8ae006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:11.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.923+0000 7f8b0199b700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 0x7f8ae006d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009b30 tx=0x7f8af80092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:11.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:11.924+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8aec053d40 con 0x7f8afc06e460 2023-06-09T11:12:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.421+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8afc13fe90 con 0x7f8afc06e460 2023-06-09T11:12:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.422+0000 7f8af27fc700 1 -- 172.21.15.7:0/3219924653 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+6206 (secure 0 0 0) 0x7f8aec088070 con 0x7f8afc06e460 2023-06-09T11:12:12.428 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:12.429 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":12,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4263,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:11:06.332140+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":2,"incarnation":11,"state":"up:active","state_seq":7,"addr":"172.21.15.117:6833/1113843600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1113843600},{"type":"v1","addr":"172.21.15.117:6833","nonce":1113843600}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 msgr2=0x7f8ae006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 0x7f8ae006d1c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009b30 tx=0x7f8af80092c0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 msgr2=0x7f8afc0c44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c4e0 tx=0x7f8aec00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 shutdown_connections 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8afc0c6280 0x7f8afc0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8afc06a4d0 0x7f8afc13f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ae006ad10 0x7f8ae006d1c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 --2- 172.21.15.7:0/3219924653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8afc06e460 0x7f8afc0c44e0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c4e0 tx=0x7f8aec00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:12:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.425+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 >> 172.21.15.7:0/3219924653 conn(0x7f8afc06f980 msgr2=0x7f8afc06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.426+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 shutdown_connections 2023-06-09T11:12:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:12.426+0000 7f8b03bff700 1 -- 172.21.15.7:0/3219924653 wait complete. 2023-06-09T11:12:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 12 2023-06-09T11:12:12.461 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:12:14.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.794+0000 7f662ffff700 1 Processor -- start 2023-06-09T11:12:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.794+0000 7f662ffff700 1 -- start start 2023-06-09T11:12:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.796+0000 7f662ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6628153880 0x7f662815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.796+0000 7f662ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66280c9f40 0x7f662815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.796+0000 7f662ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662815a500 con 0x7f662815b2d0 2023-06-09T11:12:14.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.796+0000 7f662ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280c5f10 con 0x7f6628153880 2023-06-09T11:12:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.796+0000 7f662ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662815da00 con 0x7f66280c9f40 2023-06-09T11:12:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662dd9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6628153880 0x7f662815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662d59a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66280c9f40 0x7f662815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f662815b2d0 0x7f66280c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37372/0 (socket says 172.21.15.7:37372) 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662e59c700 1 -- 172.21.15.7:0/1550292602 learned_addr learned my addr 172.21.15.7:0/1550292602 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662dd9b700 1 -- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66280c9f40 msgr2=0x7f662815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662dd9b700 1 --2- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66280c9f40 0x7f662815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.797+0000 7f662dd9b700 1 -- 172.21.15.7:0/1550292602 >> v1:172.21.15.7:6789/0 conn(0x7f662815b2d0 legacy=0x7f66280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.798+0000 7f662dd9b700 1 -- 172.21.15.7:0/1550292602 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662815b750 con 0x7f6628153880 2023-06-09T11:12:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.798+0000 7f662dd9b700 1 --2- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6628153880 0x7f662815d4c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6624009840 tx=0x7f6624009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4c2c20f29b0ad6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.798+0000 7f662cd99700 1 -- 172.21.15.7:0/1550292602 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624011830 con 0x7f6628153880 2023-06-09T11:12:14.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.798+0000 7f662cd99700 1 -- 172.21.15.7:0/1550292602 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624011990 con 0x7f6628153880 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.798+0000 7f662cd99700 1 -- 172.21.15.7:0/1550292602 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624011d20 con 0x7f6628153880 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 -- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6628153880 msgr2=0x7f662815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 --2- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6628153880 0x7f662815d4c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6624009840 tx=0x7f6624009b50 comp rx=0 tx=0).stop 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 -- 172.21.15.7:0/1550292602 shutdown_connections 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 --2- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6628153880 0x7f662815d4c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 --2- 172.21.15.7:0/1550292602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66280c9f40 0x7f662815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:14.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.799+0000 7f662ffff700 1 -- 172.21.15.7:0/1550292602 >> 172.21.15.7:0/1550292602 conn(0x7f66280737f0 msgr2=0x7f66280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.800+0000 7f662ffff700 1 -- 172.21.15.7:0/1550292602 shutdown_connections 2023-06-09T11:12:14.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.800+0000 7f662ffff700 1 -- 172.21.15.7:0/1550292602 wait complete. 2023-06-09T11:12:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.802+0000 7f662ffff700 1 Processor -- start 2023-06-09T11:12:14.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.803+0000 7f662ffff700 1 -- start start 2023-06-09T11:12:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 0x7f66280b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815b2d0 0x7f66280cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b4f20 con 0x7f6628153880 2023-06-09T11:12:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662814a0d0 con 0x7f66280c9f40 2023-06-09T11:12:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.804+0000 7f662ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280b4550 con 0x7f662815b2d0 2023-06-09T11:12:14.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.805+0000 7f662d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.805+0000 7f662dd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 0x7f66280b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.805+0000 7f662d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51976/0 (socket says 172.21.15.7:51976) 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.805+0000 7f662d59a700 1 -- 172.21.15.7:0/3788097696 learned_addr learned my addr 172.21.15.7:0/3788097696 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.805+0000 7f662e59c700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815b2d0 0x7f66280cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 -- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815b2d0 msgr2=0x7f66280cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:14.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815b2d0 0x7f66280cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 -- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 msgr2=0x7f66280cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 -- 172.21.15.7:0/3788097696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240094a0 con 0x7f66280c9f40 2023-06-09T11:12:14.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662d59a700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.806+0000 7f662dd9b700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 0x7f66280b49e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6628071db0 tx=0x7f6624004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.807+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66240036a0 con 0x7f66280c9f40 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.807+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662401f070 con 0x7f66280c9f40 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.807+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6628159980 con 0x7f66280c9f40 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.807+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624003a40 con 0x7f66280c9f40 2023-06-09T11:12:14.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.807+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66281596e0 con 0x7f66280c9f40 2023-06-09T11:12:14.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.809+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6628159c60 con 0x7f66280c9f40 2023-06-09T11:12:14.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.811+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6624003ba0 con 0x7f66280c9f40 2023-06-09T11:12:14.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.812+0000 7f661e7fc700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 0x7f660c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:14.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.812+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f6624021070 con 0x7f66280c9f40 2023-06-09T11:12:14.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.813+0000 7f662d59a700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 0x7f660c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:14.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.814+0000 7f662d59a700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 0x7f660c06d1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66280ccb70 tx=0x7f6618009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:14.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:14.817+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6624055d10 con 0x7f66280c9f40 2023-06-09T11:12:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.318+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6628074cf0 con 0x7f66280c9f40 2023-06-09T11:12:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.319+0000 7f661e7fc700 1 -- 172.21.15.7:0/3788097696 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5448 (secure 0 0 0) 0x7f662402b090 con 0x7f66280c9f40 2023-06-09T11:12:15.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:15.325 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:14.126254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 msgr2=0x7f660c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 0x7f660c06d1c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f66280ccb70 tx=0x7f6618009210 comp rx=0 tx=0).stop 2023-06-09T11:12:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 msgr2=0x7f66280b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 0x7f66280b49e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6628071db0 tx=0x7f6624004750 comp rx=0 tx=0).stop 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 shutdown_connections 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660c06ad10 0x7f660c06d1c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815b2d0 0x7f66280cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280c9f40 0x7f66280b49e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 --2- 172.21.15.7:0/3788097696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6628153880 0x7f66280cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.322+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 >> 172.21.15.7:0/3788097696 conn(0x7f66280737f0 msgr2=0x7f662815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.323+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 shutdown_connections 2023-06-09T11:12:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.323+0000 7f662ffff700 1 -- 172.21.15.7:0/3788097696 wait complete. 2023-06-09T11:12:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2023-06-09T11:12:15.357 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:12:15.357 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:12:15.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.688+0000 7f3da07ff700 1 Processor -- start 2023-06-09T11:12:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.688+0000 7f3da07ff700 1 -- start start 2023-06-09T11:12:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.690+0000 7f3da07ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98151510 0x7f3d980c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.690+0000 7f3da07ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3d9815b380 0x7f3d98070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.690+0000 7f3da07ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d981580a0 con 0x7f3d98071060 2023-06-09T11:12:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.690+0000 7f3da07ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d980714e0 con 0x7f3d9815b380 2023-06-09T11:12:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.690+0000 7f3da07ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981518e0 con 0x7f3d98151510 2023-06-09T11:12:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9ed9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d98071060 0x7f3d98158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37378/0 (socket says 172.21.15.7:37378) 2023-06-09T11:12:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9ed9c700 1 -- 172.21.15.7:0/2353255047 learned_addr learned my addr 172.21.15.7:0/2353255047 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/2353255047 >> v2:172.21.15.117:3300/0 conn(0x7f3d9815b380 0x7f3d98070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9e59b700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98151510 0x7f3d980c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9d599700 1 -- 172.21.15.7:0/2353255047 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 530516066 0 0) 0x7f3d981580a0 con 0x7f3d98071060 2023-06-09T11:12:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.691+0000 7f3d9d599700 1 -- 172.21.15.7:0/2353255047 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d840035b0 con 0x7f3d98071060 2023-06-09T11:12:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9dd9a700 1 -- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98151510 msgr2=0x7f3d980c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98151510 0x7f3d980c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9dd9a700 1 -- 172.21.15.7:0/2353255047 >> v1:172.21.15.7:6789/0 conn(0x7f3d98071060 legacy=0x7f3d98158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9dd9a700 1 -- 172.21.15.7:0/2353255047 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d980c6180 con 0x7f3d9815b380 2023-06-09T11:12:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98070b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009840 tx=0x7f3d88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6593ad966e85e171 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9d599700 1 -- 172.21.15.7:0/2353255047 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88004230 con 0x7f3d9815b380 2023-06-09T11:12:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9d599700 1 -- 172.21.15.7:0/2353255047 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88004390 con 0x7f3d9815b380 2023-06-09T11:12:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.692+0000 7f3d9d599700 1 -- 172.21.15.7:0/2353255047 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880036a0 con 0x7f3d9815b380 2023-06-09T11:12:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 -- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 msgr2=0x7f3d98070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98070b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009840 tx=0x7f3d88009b50 comp rx=0 tx=0).stop 2023-06-09T11:12:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 -- 172.21.15.7:0/2353255047 shutdown_connections 2023-06-09T11:12:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98151510 0x7f3d980c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 --2- 172.21.15.7:0/2353255047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98070b20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.693+0000 7f3da07ff700 1 -- 172.21.15.7:0/2353255047 >> 172.21.15.7:0/2353255047 conn(0x7f3d9806a4d0 msgr2=0x7f3d9806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.694+0000 7f3da07ff700 1 -- 172.21.15.7:0/2353255047 shutdown_connections 2023-06-09T11:12:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.694+0000 7f3da07ff700 1 -- 172.21.15.7:0/2353255047 wait complete. 2023-06-09T11:12:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.696+0000 7f3da07ff700 1 Processor -- start 2023-06-09T11:12:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.696+0000 7f3da07ff700 1 -- start start 2023-06-09T11:12:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 0x7f3d980c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980cbd70 con 0x7f3d98151510 2023-06-09T11:12:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d980c8480 con 0x7f3d9815b380 2023-06-09T11:12:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3da07ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d981f2f30 con 0x7f3d98071060 2023-06-09T11:12:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3d9e59b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.698+0000 7f3d9dd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 0x7f3d980c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40172/0 (socket says 172.21.15.7:40172) 2023-06-09T11:12:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 -- 172.21.15.7:0/1564886577 learned_addr learned my addr 172.21.15.7:0/1564886577 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9ed9c700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 -- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 msgr2=0x7f3d98148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 -- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 msgr2=0x7f3d980c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 0x7f3d980c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 -- 172.21.15.7:0/1564886577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d880094a0 con 0x7f3d98071060 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9ed9c700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 0x7f3d980c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.699+0000 7f3d9e59b700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e9b0 tx=0x7f3d9400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.700+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d94009710 con 0x7f3d98071060 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.700+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9400c850 con 0x7f3d98071060 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.700+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9400cbc0 con 0x7f3d98071060 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.700+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d98157770 con 0x7f3d98071060 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.700+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d98157490 con 0x7f3d98071060 2023-06-09T11:12:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.701+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d98157a50 con 0x7f3d98071060 2023-06-09T11:12:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.703+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d9400cd20 con 0x7f3d98071060 2023-06-09T11:12:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.705+0000 7f3d8effd700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 0x7f3d7c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.705+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f3d94089890 con 0x7f3d98071060 2023-06-09T11:12:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.705+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 0x7f3d7c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.706+0000 7f3d9dd9a700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 0x7f3d7c06d1e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d880090b0 tx=0x7f3d88010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:15.709+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3d94054920 con 0x7f3d98071060 2023-06-09T11:12:16.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.211+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d980b0ec0 con 0x7f3d98071060 2023-06-09T11:12:16.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.212+0000 7f3d8effd700 1 -- 172.21.15.7:0/1564886577 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5448 (secure 0 0 0) 0x7f3d94057f40 con 0x7f3d98071060 2023-06-09T11:12:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:16.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:14.126254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 msgr2=0x7f3d7c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 0x7f3d7c06d1e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3d880090b0 tx=0x7f3d88010000 comp rx=0 tx=0).stop 2023-06-09T11:12:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 msgr2=0x7f3d981f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d9400e9b0 tx=0x7f3d9400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:12:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 shutdown_connections 2023-06-09T11:12:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3d9815b380 0x7f3d98148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3d7c06ad30 0x7f3d7c06d1e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3d98071060 0x7f3d981f2470 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 --2- 172.21.15.7:0/1564886577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d98151510 0x7f3d980c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.215+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 >> 172.21.15.7:0/1564886577 conn(0x7f3d9806a4d0 msgr2=0x7f3d9814ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.216+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 shutdown_connections 2023-06-09T11:12:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:16.216+0000 7f3da07ff700 1 -- 172.21.15.7:0/1564886577 wait complete. 2023-06-09T11:12:16.223 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2023-06-09T11:12:16.252 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 13, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 13, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4263": { "addr": "172.21.15.7:6839/967742498", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 967742498, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 967742498, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4263, "incarnation": 13, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_4288": { "addr": "172.21.15.117:6835/4051500902", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 4051500902, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 4051500902, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4288, "incarnation": 9, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 23, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:12:14.126254+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 4288, "mds_2": 4263 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:12:18.254 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.583+0000 7f0954dff700 1 Processor -- start 2023-06-09T11:12:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.583+0000 7f0954dff700 1 -- start start 2023-06-09T11:12:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f0954dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 0x7f095015b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f0954dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950161b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f0954dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095015b750 con 0x7f095006db60 2023-06-09T11:12:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f0954dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095015a800 con 0x7f0950151560 2023-06-09T11:12:18.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f0954dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095006d270 con 0x7f09500ccf50 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f094e59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 0x7f095015b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f094ed9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f094dd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f095006db60 0x7f09500c5930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:51508/0 (socket says 172.21.15.7:51508) 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.586+0000 7f094dd9b700 1 -- 172.21.15.7:0/1311335315 learned_addr learned my addr 172.21.15.7:0/1311335315 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094d59a700 1 -- 172.21.15.7:0/1311335315 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533263792 0 0) 0x7f0944001110 con 0x7f095006db60 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094d59a700 1 -- 172.21.15.7:0/1311335315 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09380035b0 con 0x7f095006db60 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094ed9d700 1 -- 172.21.15.7:0/1311335315 >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 msgr2=0x7f095015b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:18.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094ed9d700 1 --2- 172.21.15.7:0/1311335315 >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 0x7f095015b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094ed9d700 1 -- 172.21.15.7:0/1311335315 >> v1:172.21.15.117:6789/0 conn(0x7f095006db60 legacy=0x7f09500c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094ed9d700 1 -- 172.21.15.7:0/1311335315 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095006e460 con 0x7f09500ccf50 2023-06-09T11:12:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094e59c700 1 --2- 172.21.15.7:0/1311335315 >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 0x7f095015b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:18.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.587+0000 7f094ed9d700 1 --2- 172.21.15.7:0/1311335315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950161b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0948009d90 tx=0x7f094800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c178eca24e0ea3c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.588+0000 7f094d59a700 1 -- 172.21.15.7:0/1311335315 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800e070 con 0x7f09500ccf50 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.588+0000 7f094d59a700 1 -- 172.21.15.7:0/1311335315 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094800bb20 con 0x7f09500ccf50 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.588+0000 7f094d59a700 1 -- 172.21.15.7:0/1311335315 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948012430 con 0x7f09500ccf50 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 -- 172.21.15.7:0/1311335315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 msgr2=0x7f0950161b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 --2- 172.21.15.7:0/1311335315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950161b30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0948009d90 tx=0x7f094800b330 comp rx=0 tx=0).stop 2023-06-09T11:12:18.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 -- 172.21.15.7:0/1311335315 shutdown_connections 2023-06-09T11:12:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 --2- 172.21.15.7:0/1311335315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950161b30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 --2- 172.21.15.7:0/1311335315 >> v2:172.21.15.117:3300/0 conn(0x7f0950151560 0x7f095015b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 -- 172.21.15.7:0/1311335315 >> 172.21.15.7:0/1311335315 conn(0x7f095006f980 msgr2=0x7f09500cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 -- 172.21.15.7:0/1311335315 shutdown_connections 2023-06-09T11:12:18.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.589+0000 7f0954dff700 1 -- 172.21.15.7:0/1311335315 wait complete. 2023-06-09T11:12:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.592+0000 7f0954dff700 1 Processor -- start 2023-06-09T11:12:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.592+0000 7f0954dff700 1 -- start start 2023-06-09T11:12:18.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0950151560 0x7f0950155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c2230 con 0x7f095006db60 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0950165ba0 con 0x7f0950151560 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f0954dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c0790 con 0x7f09500ccf50 2023-06-09T11:12:18.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f094e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f094e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51992/0 (socket says 172.21.15.7:51992) 2023-06-09T11:12:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f094e59c700 1 -- 172.21.15.7:0/500636633 learned_addr learned my addr 172.21.15.7:0/500636633 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f094dd9b700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.594+0000 7f094ed9d700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0950151560 0x7f0950155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 -- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 msgr2=0x7f0950155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 -- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0950151560 msgr2=0x7f0950155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:18.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0950151560 0x7f0950155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 -- 172.21.15.7:0/500636633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400094a0 con 0x7f095006db60 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094dd9b700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.595+0000 7f094e59c700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f094000ebe0 tx=0x7f094000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.596+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940009760 con 0x7f095006db60 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.596+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000c850 con 0x7f095006db60 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.596+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000cbe0 con 0x7f095006db60 2023-06-09T11:12:18.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.596+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0948009a70 con 0x7f095006db60 2023-06-09T11:12:18.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.596+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095015aa10 con 0x7f095006db60 2023-06-09T11:12:18.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.597+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095006e740 con 0x7f095006db60 2023-06-09T11:12:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.600+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f094000cd40 con 0x7f095006db60 2023-06-09T11:12:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.601+0000 7f093effd700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 0x7f093006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.602+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f0940089dc0 con 0x7f095006db60 2023-06-09T11:12:18.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.602+0000 7f094dd9b700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 0x7f093006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:18.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.603+0000 7f094dd9b700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 0x7f093006d260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f09500c7820 tx=0x7f0944007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:18.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:18.608+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0940054dc0 con 0x7f095006db60 2023-06-09T11:12:19.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.109+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f095015acf0 con 0x7f095006db60 2023-06-09T11:12:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.110+0000 7f093effd700 1 -- 172.21.15.7:0/500636633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+5448 (secure 0 0 0) 0x7f09400583e0 con 0x7f095006db60 2023-06-09T11:12:19.116 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:19.116 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:14.126254+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.113+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 msgr2=0x7f093006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.113+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 0x7f093006d260 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f09500c7820 tx=0x7f0944007c40 comp rx=0 tx=0).stop 2023-06-09T11:12:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.113+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 msgr2=0x7f095014d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:19.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.113+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f094000ebe0 tx=0x7f094000eef0 comp rx=0 tx=0).stop 2023-06-09T11:12:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 shutdown_connections 2023-06-09T11:12:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0950151560 0x7f0950155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095006db60 0x7f095014d480 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006adb0 0x7f093006d260 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:19.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 --2- 172.21.15.7:0/500636633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09500ccf50 0x7f0950155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 >> 172.21.15.7:0/500636633 conn(0x7f095006f980 msgr2=0x7f095015a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 shutdown_connections 2023-06-09T11:12:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:19.114+0000 7f0954dff700 1 -- 172.21.15.7:0/500636633 wait complete. 2023-06-09T11:12:19.121 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 13 2023-06-09T11:12:19.149 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:21.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.481+0000 7f93cc5ff700 1 Processor -- start 2023-06-09T11:12:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.481+0000 7f93cc5ff700 1 -- start start 2023-06-09T11:12:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.483+0000 7f93cc5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f93c406a4d0 0x7f93c40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.483+0000 7f93cc5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f93c406e460 0x7f93c406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93cc5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c415cac0 con 0x7f93c40c6280 2023-06-09T11:12:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93cc5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40c8870 con 0x7f93c406a4d0 2023-06-09T11:12:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93cc5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40c37d0 con 0x7f93c406e460 2023-06-09T11:12:21.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93ca39b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f93c40c6280 0x7f93c40737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37380/0 (socket says 172.21.15.7:37380) 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93ca39b700 1 -- 172.21.15.7:0/730948262 learned_addr learned my addr 172.21.15.7:0/730948262 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93c9b9a700 1 --2- 172.21.15.7:0/730948262 >> v2:172.21.15.7:3300/0 conn(0x7f93c406a4d0 0x7f93c40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.484+0000 7f93cab9c700 1 --2- 172.21.15.7:0/730948262 >> v2:172.21.15.117:3300/0 conn(0x7f93c406e460 0x7f93c406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.485+0000 7f93c9b9a700 1 -- 172.21.15.7:0/730948262 >> v2:172.21.15.117:3300/0 conn(0x7f93c406e460 msgr2=0x7f93c406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.485+0000 7f93c9b9a700 1 --2- 172.21.15.7:0/730948262 >> v2:172.21.15.117:3300/0 conn(0x7f93c406e460 0x7f93c406cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:21.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.485+0000 7f93c9b9a700 1 -- 172.21.15.7:0/730948262 >> v1:172.21.15.7:6789/0 conn(0x7f93c40c6280 legacy=0x7f93c40737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:21.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.485+0000 7f93c9b9a700 1 -- 172.21.15.7:0/730948262 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c415cd30 con 0x7f93c406a4d0 2023-06-09T11:12:21.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.485+0000 7f93c9b9a700 1 --2- 172.21.15.7:0/730948262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406a4d0 0x7f93c40c76b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93c413f730 tx=0x7f93b4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1383ea2426e9d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.486+0000 7f93c9399700 1 -- 172.21.15.7:0/730948262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4011830 con 0x7f93c406a4d0 2023-06-09T11:12:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.486+0000 7f93c9399700 1 -- 172.21.15.7:0/730948262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4011990 con 0x7f93c406a4d0 2023-06-09T11:12:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.486+0000 7f93c9399700 1 -- 172.21.15.7:0/730948262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4011d20 con 0x7f93c406a4d0 2023-06-09T11:12:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.486+0000 7f93cc5ff700 1 -- 172.21.15.7:0/730948262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406a4d0 msgr2=0x7f93c40c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:21.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.486+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/730948262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406a4d0 0x7f93c40c76b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93c413f730 tx=0x7f93b4009b20 comp rx=0 tx=0).stop 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 -- 172.21.15.7:0/730948262 shutdown_connections 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/730948262 >> v2:172.21.15.117:3300/0 conn(0x7f93c406e460 0x7f93c406cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/730948262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406a4d0 0x7f93c40c76b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 -- 172.21.15.7:0/730948262 >> 172.21.15.7:0/730948262 conn(0x7f93c406f980 msgr2=0x7f93c40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 -- 172.21.15.7:0/730948262 shutdown_connections 2023-06-09T11:12:21.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.487+0000 7f93cc5ff700 1 -- 172.21.15.7:0/730948262 wait complete. 2023-06-09T11:12:21.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.490+0000 7f93cc5ff700 1 Processor -- start 2023-06-09T11:12:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.490+0000 7f93cc5ff700 1 -- start start 2023-06-09T11:12:21.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406e460 0x7f93c40cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4155420 con 0x7f93c406e460 2023-06-09T11:12:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4142a20 con 0x7f93c40c6280 2023-06-09T11:12:21.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cc5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c406af70 con 0x7f93c406a4d0 2023-06-09T11:12:21.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93ca39b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cab9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93ca39b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40184/0 (socket says 172.21.15.7:40184) 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93cab9c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:49846/0 (socket says 172.21.15.7:49846) 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.492+0000 7f93ca39b700 1 -- 172.21.15.7:0/4100177855 learned_addr learned my addr 172.21.15.7:0/4100177855 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93c9b9a700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406e460 0x7f93c40cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 -- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 msgr2=0x7f93c4141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:21.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 -- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406e460 msgr2=0x7f93c40cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406e460 0x7f93c40cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 -- 172.21.15.7:0/4100177855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b40094a0 con 0x7f93c40c6280 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93ca39b700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.493+0000 7f93cab9c700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93bc007dc0 tx=0x7f93bc00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:21.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.494+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc018070 con 0x7f93c40c6280 2023-06-09T11:12:21.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.494+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc00eb30 con 0x7f93c40c6280 2023-06-09T11:12:21.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.494+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00eec0 con 0x7f93c40c6280 2023-06-09T11:12:21.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.494+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c415d2f0 con 0x7f93c40c6280 2023-06-09T11:12:21.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.494+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c415d010 con 0x7f93c40c6280 2023-06-09T11:12:21.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.495+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c415d5d0 con 0x7f93c40c6280 2023-06-09T11:12:21.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.497+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93bc003a20 con 0x7f93c40c6280 2023-06-09T11:12:21.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.499+0000 7f93baffd700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 0x7f93a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:21.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.499+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f93bc0894f0 con 0x7f93c40c6280 2023-06-09T11:12:21.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.500+0000 7f93ca39b700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 0x7f93a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:21.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.500+0000 7f93ca39b700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 0x7f93a806d1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005910 tx=0x7f93c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:21.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:21.503+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f93bc057f00 con 0x7f93c40c6280 2023-06-09T11:12:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.005+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93c4074cf0 con 0x7f93c40c6280 2023-06-09T11:12:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.006+0000 7f93baffd700 1 -- 172.21.15.7:0/4100177855 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5450 (secure 0 0 0) 0x7f93bc009950 con 0x7f93c40c6280 2023-06-09T11:12:22.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:22.012 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:20.206791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:resolve","state_seq":25,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 msgr2=0x7f93a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 0x7f93a806d1c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005910 tx=0x7f93c0009040 comp rx=0 tx=0).stop 2023-06-09T11:12:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 msgr2=0x7f93c4153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93bc007dc0 tx=0x7f93bc00e3c0 comp rx=0 tx=0).stop 2023-06-09T11:12:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 shutdown_connections 2023-06-09T11:12:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c40c6280 0x7f93c4153d20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93a806ad10 0x7f93a806d1c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c406a4d0 0x7f93c4141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 --2- 172.21.15.7:0/4100177855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c406e460 0x7f93c40cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 >> 172.21.15.7:0/4100177855 conn(0x7f93c406f980 msgr2=0x7f93c413d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 shutdown_connections 2023-06-09T11:12:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:22.009+0000 7f93cc5ff700 1 -- 172.21.15.7:0/4100177855 wait complete. 2023-06-09T11:12:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 14 2023-06-09T11:12:22.044 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:24.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:24.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.249+0000 7f99323ff700 1 Processor -- start 2023-06-09T11:12:24.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.250+0000 7f99323ff700 1 -- start start 2023-06-09T11:12:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.251+0000 7f99323ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f992c159eb0 0x7f992c071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.251+0000 7f99323ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c0cc140 con 0x7f992c0c08d0 2023-06-09T11:12:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f99323ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c0c1830 con 0x7f992c1593d0 2023-06-09T11:12:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f99323ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0714f0 con 0x7f992c159eb0 2023-06-09T11:12:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f992c159eb0 0x7f992c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f993099c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f992c0c08d0 0x7f992c14f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40528/0 (socket says 172.21.15.7:40528) 2023-06-09T11:12:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f993099c700 1 -- 172.21.15.7:0/1358607418 learned_addr learned my addr 172.21.15.7:0/1358607418 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f992c159eb0 0x7f992c071910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58154/0 (socket says 172.21.15.7:58154) 2023-06-09T11:12:24.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992b7fe700 1 -- 172.21.15.7:0/1358607418 >> v1:172.21.15.117:6789/0 conn(0x7f992c1593d0 legacy=0x7f992c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992b7fe700 1 -- 172.21.15.7:0/1358607418 >> v1:172.21.15.7:6789/0 conn(0x7f992c0c08d0 legacy=0x7f992c14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992b7fe700 1 -- 172.21.15.7:0/1358607418 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c0c0560 con 0x7f992c159eb0 2023-06-09T11:12:24.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.252+0000 7f992affd700 1 -- 172.21.15.7:0/1358607418 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581432011 0 0) 0x7f992c0cc140 con 0x7f992c0c08d0 2023-06-09T11:12:24.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f992b7fe700 1 --2- 172.21.15.7:0/1358607418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c159eb0 0x7f992c071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f991c00bae0 tx=0x7f991c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2022feccb280f0e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:24.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f992affd700 1 -- 172.21.15.7:0/1358607418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c00dd30 con 0x7f992c159eb0 2023-06-09T11:12:24.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f992affd700 1 -- 172.21.15.7:0/1358607418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c00de90 con 0x7f992c159eb0 2023-06-09T11:12:24.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f992affd700 1 -- 172.21.15.7:0/1358607418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c010690 con 0x7f992c159eb0 2023-06-09T11:12:24.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f99323ff700 1 -- 172.21.15.7:0/1358607418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c159eb0 msgr2=0x7f992c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.253+0000 7f99323ff700 1 --2- 172.21.15.7:0/1358607418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c159eb0 0x7f992c071910 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f991c00bae0 tx=0x7f991c00bdf0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.254+0000 7f99323ff700 1 -- 172.21.15.7:0/1358607418 shutdown_connections 2023-06-09T11:12:24.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.254+0000 7f99323ff700 1 --2- 172.21.15.7:0/1358607418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c159eb0 0x7f992c071910 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.254+0000 7f99323ff700 1 -- 172.21.15.7:0/1358607418 >> 172.21.15.7:0/1358607418 conn(0x7f992c06a4d0 msgr2=0x7f992c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:24.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.254+0000 7f99323ff700 1 -- 172.21.15.7:0/1358607418 shutdown_connections 2023-06-09T11:12:24.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.254+0000 7f99323ff700 1 -- 172.21.15.7:0/1358607418 wait complete. 2023-06-09T11:12:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.256+0000 7f99323ff700 1 Processor -- start 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.256+0000 7f99323ff700 1 -- start start 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c08d0 0x7f992c0c2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 0x7f992c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c1eba60 con 0x7f992c1593d0 2023-06-09T11:12:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0c3570 con 0x7f992c0c08d0 2023-06-09T11:12:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f99323ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0c3850 con 0x7f992c159eb0 2023-06-09T11:12:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:24.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58164/0 (socket says 172.21.15.7:58164) 2023-06-09T11:12:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 -- 172.21.15.7:0/3476640319 learned_addr learned my addr 172.21.15.7:0/3476640319 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992bfff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c08d0 0x7f992c0c2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f993099c700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 0x7f992c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 -- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 msgr2=0x7f992c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 0x7f992c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 -- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c08d0 msgr2=0x7f992c0c2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c08d0 0x7f992c0c2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f992b7fe700 1 -- 172.21.15.7:0/3476640319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c00b740 con 0x7f992c1593d0 2023-06-09T11:12:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.258+0000 7f993099c700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 0x7f992c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:12:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f992b7fe700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f991c00bf90 tx=0x7f991c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c01e040 con 0x7f992c1593d0 2023-06-09T11:12:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991c019600 con 0x7f992c1593d0 2023-06-09T11:12:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991c019970 con 0x7f992c1593d0 2023-06-09T11:12:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c0c1d60 con 0x7f992c1593d0 2023-06-09T11:12:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.259+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c0c1a80 con 0x7f992c1593d0 2023-06-09T11:12:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.260+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c0c2040 con 0x7f992c1593d0 2023-06-09T11:12:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.261+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f991c019ce0 con 0x7f992c1593d0 2023-06-09T11:12:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.262+0000 7f9928ff9700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 0x7f990c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.262+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f991c08d520 con 0x7f992c1593d0 2023-06-09T11:12:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.262+0000 7f992bfff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 0x7f990c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.263+0000 7f992bfff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 0x7f990c06d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9914007fd0 tx=0x7f9914009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.271+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f991c058650 con 0x7f992c1593d0 2023-06-09T11:12:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.525+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f992c1eb830 con 0x7f992c1593d0 2023-06-09T11:12:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.526+0000 7f9928ff9700 1 -- 172.21.15.7:0/3476640319 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+5450 (secure 0 0 0) 0x7f991c05bc70 con 0x7f992c1593d0 2023-06-09T11:12:24.532 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:24.532 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:20.206791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:resolve","state_seq":25,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:24.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 msgr2=0x7f990c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 0x7f990c06d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9914007fd0 tx=0x7f9914009040 comp rx=0 tx=0).stop 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 msgr2=0x7f992c0b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f991c00bf90 tx=0x7f991c0047c0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 shutdown_connections 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f990c06ace0 0x7f990c06d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f992c159eb0 0x7f992c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f992c0c08d0 0x7f992c0c2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 --2- 172.21.15.7:0/3476640319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f992c1593d0 0x7f992c0b0480 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 >> 172.21.15.7:0/3476640319 conn(0x7f992c06a4d0 msgr2=0x7f992c158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 shutdown_connections 2023-06-09T11:12:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:24.529+0000 7f99323ff700 1 -- 172.21.15.7:0/3476640319 wait complete. 2023-06-09T11:12:24.536 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 14 2023-06-09T11:12:24.554 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:26.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:26.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.754+0000 7fc5121ff700 1 Processor -- start 2023-06-09T11:12:26.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.755+0000 7fc5121ff700 1 -- start start 2023-06-09T11:12:26.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc5121ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc5121ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc50c0c9f40 0x7fc50c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc5121ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c15a500 con 0x7fc50c15b2d0 2023-06-09T11:12:26.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc5121ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c0c5f10 con 0x7fc50c0c9f40 2023-06-09T11:12:26.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc5121ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c15da00 con 0x7fc50c153880 2023-06-09T11:12:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc50b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc50affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc50c0c9f40 0x7fc50c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc50b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:34442/0 (socket says 172.21.15.7:34442) 2023-06-09T11:12:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.756+0000 7fc50b7fe700 1 -- 172.21.15.7:0/3181406760 learned_addr learned my addr 172.21.15.7:0/3181406760 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:26.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50b7fe700 1 -- 172.21.15.7:0/3181406760 >> v2:172.21.15.117:3300/0 conn(0x7fc50c0c9f40 msgr2=0x7fc50c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50b7fe700 1 --2- 172.21.15.7:0/3181406760 >> v2:172.21.15.117:3300/0 conn(0x7fc50c0c9f40 0x7fc50c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50b7fe700 1 -- 172.21.15.7:0/3181406760 >> v1:172.21.15.117:6789/0 conn(0x7fc50c15b2d0 legacy=0x7fc50c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50b7fe700 1 -- 172.21.15.7:0/3181406760 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c15b750 con 0x7fc50c153880 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50b7fe700 1 --2- 172.21.15.7:0/3181406760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0098f0 tx=0x7fc4fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66e19067aa1a6631 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50a7fc700 1 -- 172.21.15.7:0/3181406760 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc011830 con 0x7fc50c153880 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50a7fc700 1 -- 172.21.15.7:0/3181406760 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc011990 con 0x7fc50c153880 2023-06-09T11:12:26.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.757+0000 7fc50a7fc700 1 -- 172.21.15.7:0/3181406760 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc011d20 con 0x7fc50c153880 2023-06-09T11:12:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 -- 172.21.15.7:0/3181406760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 msgr2=0x7fc50c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 --2- 172.21.15.7:0/3181406760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0098f0 tx=0x7fc4fc009c00 comp rx=0 tx=0).stop 2023-06-09T11:12:26.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 -- 172.21.15.7:0/3181406760 shutdown_connections 2023-06-09T11:12:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 --2- 172.21.15.7:0/3181406760 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c153880 0x7fc50c15d4c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 --2- 172.21.15.7:0/3181406760 >> v2:172.21.15.117:3300/0 conn(0x7fc50c0c9f40 0x7fc50c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 -- 172.21.15.7:0/3181406760 >> 172.21.15.7:0/3181406760 conn(0x7fc50c0737f0 msgr2=0x7fc50c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 -- 172.21.15.7:0/3181406760 shutdown_connections 2023-06-09T11:12:26.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.758+0000 7fc5121ff700 1 -- 172.21.15.7:0/3181406760 wait complete. 2023-06-09T11:12:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.761+0000 7fc5121ff700 1 Processor -- start 2023-06-09T11:12:26.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.761+0000 7fc5121ff700 1 -- start start 2023-06-09T11:12:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c0c9f40 0x7fc50c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc50c153880 0x7fc50c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c151c10 con 0x7fc50c15b2d0 2023-06-09T11:12:26.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c151130 con 0x7fc50c153880 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc5121ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0cc660 con 0x7fc50c0c9f40 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc50bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc50bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58168/0 (socket says 172.21.15.7:58168) 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc50bfff700 1 -- 172.21.15.7:0/731653109 learned_addr learned my addr 172.21.15.7:0/731653109 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc50affd700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc50c153880 0x7fc50c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.763+0000 7fc50b7fe700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c0c9f40 0x7fc50c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 -- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c0c9f40 msgr2=0x7fc50c06a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:26.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c0c9f40 0x7fc50c06a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 -- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc50c153880 msgr2=0x7fc50c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:26.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc50c153880 0x7fc50c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 -- 172.21.15.7:0/731653109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc009550 con 0x7fc50c15b2d0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc50bfff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc50400cb80 tx=0x7fc50400ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504003c00 con 0x7fc50c15b2d0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504003d60 con 0x7fc50c15b2d0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400aaa0 con 0x7fc50c15b2d0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.764+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c159980 con 0x7fc50c15b2d0 2023-06-09T11:12:26.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.765+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c1596e0 con 0x7fc50c15b2d0 2023-06-09T11:12:26.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.766+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c159c60 con 0x7fc50c15b2d0 2023-06-09T11:12:26.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.766+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc50400ac00 con 0x7fc50c15b2d0 2023-06-09T11:12:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.767+0000 7fc4f3fff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 0x7fc4f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.767+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fc5040892d0 con 0x7fc50c15b2d0 2023-06-09T11:12:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.768+0000 7fc50b7fe700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 0x7fc4f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.768+0000 7fc50b7fe700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 0x7fc4f806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009c40 tx=0x7fc4fc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:26.772+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc504054250 con 0x7fc50c15b2d0 2023-06-09T11:12:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.024+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc50c15a500 con 0x7fc50c15b2d0 2023-06-09T11:12:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.025+0000 7fc4f3fff700 1 -- 172.21.15.7:0/731653109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5452 (secure 0 0 0) 0x7fc50c15a500 con 0x7fc50c15b2d0 2023-06-09T11:12:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:27.031 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:26.135724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.027+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 msgr2=0x7fc4f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.027+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 0x7fc4f806d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009c40 tx=0x7fc4fc008030 comp rx=0 tx=0).stop 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.027+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 msgr2=0x7fc50c0cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc50400cb80 tx=0x7fc50400ce90 comp rx=0 tx=0).stop 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 shutdown_connections 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc50c15b2d0 0x7fc50c0cd8e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4f806ace0 0x7fc4f806d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc50c0c9f40 0x7fc50c06a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 --2- 172.21.15.7:0/731653109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc50c153880 0x7fc50c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 >> 172.21.15.7:0/731653109 conn(0x7fc50c0737f0 msgr2=0x7fc50c14c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 shutdown_connections 2023-06-09T11:12:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:27.028+0000 7fc5121ff700 1 -- 172.21.15.7:0/731653109 wait complete. 2023-06-09T11:12:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2023-06-09T11:12:27.051 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:27.416 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 62% (7360/11710) Updating files: 63% (7378/11710) Updating files: 64% (7495/11710) Updating files: 65% (7612/11710) Updating files: 66% (7729/11710) Updating files: 67% (7846/11710) Updating files: 68% (7963/11710) Updating files: 69% (8080/11710) Updating files: 70% (8197/11710) Updating files: 71% (8315/11710) Updating files: 72% (8432/11710) Updating files: 73% (8549/11710) Updating files: 74% (8666/11710) Updating files: 75% (8783/11710) Updating files: 76% (8900/11710) Updating files: 77% (9017/11710) Updating files: 78% (9134/11710) Updating files: 79% (9251/11710) Updating files: 80% (9368/11710) Updating files: 81% (9486/11710) Updating files: 82% (9603/11710) Updating files: 83% (9720/11710) Updating files: 84% (9837/11710) Updating files: 85% (9954/11710) Updating files: 86% (10071/11710) Updating files: 87% (10188/11710) Updating files: 88% (10305/11710) Updating files: 89% (10422/11710) Updating files: 90% (10539/11710) Updating files: 91% (10657/11710) Updating files: 92% (10774/11710) Updating files: 93% (10891/11710) Updating files: 94% (11008/11710) Updating files: 95% (11125/11710) Updating files: 96% (11242/11710) Updating files: 97% (11359/11710) Updating files: 98% (11476/11710) Updating files: 98% (11553/11710) Updating files: 99% (11593/11710) Updating files: 100% (11710/11710) Updating files: 100% (11710/11710), done. 2023-06-09T11:12:28.118 INFO:tasks.workunit.client.0.smithi007.stderr:Note: switching to '034108821cde95ca1c56d458dfccc7ef9e0d964d'. 2023-06-09T11:12:28.118 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.118 INFO:tasks.workunit.client.0.smithi007.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-06-09T11:12:28.118 INFO:tasks.workunit.client.0.smithi007.stderr:changes and commit them, and you can discard any commits you make in this 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr:state without impacting any branches by switching back to a branch. 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr:If you want to create a new branch to retain commits you create, you may 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr:do so (now or later) by using -c with the switch command. Example: 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr: git switch -c 2023-06-09T11:12:28.119 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.120 INFO:tasks.workunit.client.0.smithi007.stderr:Or undo this operation with: 2023-06-09T11:12:28.120 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.120 INFO:tasks.workunit.client.0.smithi007.stderr: git switch - 2023-06-09T11:12:28.121 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.121 INFO:tasks.workunit.client.0.smithi007.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-06-09T11:12:28.121 INFO:tasks.workunit.client.0.smithi007.stderr: 2023-06-09T11:12:28.121 INFO:tasks.workunit.client.0.smithi007.stderr:HEAD is now at 034108821cd temp 2023-06-09T11:12:28.131 DEBUG:teuthology.orchestra.run.smithi007:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-06-09T11:12:28.225 INFO:tasks.workunit.client.0.smithi007.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-06-09T11:12:28.230 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-06-09T11:12:28.230 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-06-09T11:12:28.790 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-06-09T11:12:28.870 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-06-09T11:12:28.934 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-06-09T11:12:28.938 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-06-09T11:12:28.938 INFO:tasks.workunit.client.0.smithi007.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-06-09T11:12:29.001 INFO:tasks.workunit.client.0.smithi007.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-06-09T11:12:29.008 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:12:29.008 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-06-09T11:12:29.051 INFO:tasks.workunit:Running workunits matching fs/snaps on client.0... 2023-06-09T11:12:29.052 INFO:tasks.workunit:Running workunit fs/snaps/snap-rm-diff.sh... 2023-06-09T11:12:29.052 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snap-rm-diff.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snap-rm-diff.sh 2023-06-09T11:12:29.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:29.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.291+0000 7f9ee03ff700 1 Processor -- start 2023-06-09T11:12:29.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.291+0000 7f9ee03ff700 1 -- start start 2023-06-09T11:12:29.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.292+0000 7f9ee03ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b4a0 0x7f9ed815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.292+0000 7f9ee03ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c9f40 0x7f9ed80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9ee03ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed815a500 con 0x7f9ed8153880 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9ee03ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80c5f10 con 0x7f9ed815b4a0 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9ee03ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed815da00 con 0x7f9ed80c9f40 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b4a0 0x7f9ed815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b4a0 0x7f9ed815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58172/0 (socket says 172.21.15.7:58172) 2023-06-09T11:12:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 -- 172.21.15.7:0/3823748639 learned_addr learned my addr 172.21.15.7:0/3823748639 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9ede99c700 1 --2- 172.21.15.7:0/3823748639 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c9f40 0x7f9ed80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 -- 172.21.15.7:0/3823748639 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c9f40 msgr2=0x7f9ed80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:29.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 --2- 172.21.15.7:0/3823748639 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c9f40 0x7f9ed80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 -- 172.21.15.7:0/3823748639 >> v1:172.21.15.117:6789/0 conn(0x7f9ed8153880 legacy=0x7f9ed815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:29.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.293+0000 7f9edd99a700 1 -- 172.21.15.7:0/3823748639 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed81596e0 con 0x7f9ed815b4a0 2023-06-09T11:12:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9edd99a700 1 --2- 172.21.15.7:0/3823748639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed815d4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80749f0 tx=0x7f9ec8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ae20138ac205a2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9edd199700 1 -- 172.21.15.7:0/3823748639 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8004230 con 0x7f9ed815b4a0 2023-06-09T11:12:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9edd199700 1 -- 172.21.15.7:0/3823748639 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8004390 con 0x7f9ed815b4a0 2023-06-09T11:12:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9edd199700 1 -- 172.21.15.7:0/3823748639 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80036a0 con 0x7f9ed815b4a0 2023-06-09T11:12:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9ee03ff700 1 -- 172.21.15.7:0/3823748639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 msgr2=0x7f9ed815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/3823748639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed815d4c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9ed80749f0 tx=0x7f9ec8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9ee03ff700 1 -- 172.21.15.7:0/3823748639 shutdown_connections 2023-06-09T11:12:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/3823748639 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c9f40 0x7f9ed80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.294+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/3823748639 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed815d4c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.295+0000 7f9ee03ff700 1 -- 172.21.15.7:0/3823748639 >> 172.21.15.7:0/3823748639 conn(0x7f9ed80737f0 msgr2=0x7f9ed80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.295+0000 7f9ee03ff700 1 -- 172.21.15.7:0/3823748639 shutdown_connections 2023-06-09T11:12:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.295+0000 7f9ee03ff700 1 -- 172.21.15.7:0/3823748639 wait complete. 2023-06-09T11:12:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.296+0000 7f9ee03ff700 1 Processor -- start 2023-06-09T11:12:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.296+0000 7f9ee03ff700 1 -- start start 2023-06-09T11:12:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.297+0000 7f9ee03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed80c9f40 0x7f9ed80caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.297+0000 7f9ee03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed8153880 0x7f9ed81ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.297+0000 7f9ee03ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.297+0000 7f9ee03ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81529a0 con 0x7f9ed815b4a0 2023-06-09T11:12:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ee03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81efe00 con 0x7f9ed80c9f40 2023-06-09T11:12:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ee03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed80ce150 con 0x7f9ed8153880 2023-06-09T11:12:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58186/0 (socket says 172.21.15.7:58186) 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 -- 172.21.15.7:0/734425473 learned_addr learned my addr 172.21.15.7:0/734425473 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9edd99a700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed8153880 0x7f9ed81ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede19b700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed80c9f40 0x7f9ed80caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 -- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed8153880 msgr2=0x7f9ed81ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed8153880 0x7f9ed81ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 -- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed80c9f40 msgr2=0x7f9ed80caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed80c9f40 0x7f9ed80caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 -- 172.21.15.7:0/734425473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009550 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.298+0000 7f9ede99c700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007e40 tx=0x7f9ed000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.299+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000ed30 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.299+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed0014420 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.299+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00147d0 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.299+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8159c60 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.299+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed8159980 con 0x7f9ed815b4a0 2023-06-09T11:12:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.300+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed8159f40 con 0x7f9ed815b4a0 2023-06-09T11:12:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.301+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ed0014930 con 0x7f9ed815b4a0 2023-06-09T11:12:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.302+0000 7f9eceffd700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 0x7f9ebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.302+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f9ed0089d90 con 0x7f9ed815b4a0 2023-06-09T11:12:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.302+0000 7f9ede19b700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 0x7f9ebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.303+0000 7f9ede19b700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 0x7f9ebc06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007fd0 tx=0x7f9ed4007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.306+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9ed0054d30 con 0x7f9ed815b4a0 2023-06-09T11:12:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.554+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ed8074cf0 con 0x7f9ed815b4a0 2023-06-09T11:12:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.554+0000 7f9eceffd700 1 -- 172.21.15.7:0/734425473 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+5452 (secure 0 0 0) 0x7f9ed0058350 con 0x7f9ed815b4a0 2023-06-09T11:12:29.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:29.561 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:26.135724+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 msgr2=0x7f9ebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 0x7f9ebc06d190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4007fd0 tx=0x7f9ed4007ec0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 msgr2=0x7f9ed8148e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0007e40 tx=0x7f9ed000e3c0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 shutdown_connections 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b4a0 0x7f9ed8148e60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed80c9f40 0x7f9ed80caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ace0 0x7f9ebc06d190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 --2- 172.21.15.7:0/734425473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed8153880 0x7f9ed81ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 >> 172.21.15.7:0/734425473 conn(0x7f9ed80737f0 msgr2=0x7f9ed80cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 shutdown_connections 2023-06-09T11:12:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:29.557+0000 7f9ee03ff700 1 -- 172.21.15.7:0/734425473 wait complete. 2023-06-09T11:12:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 15 2023-06-09T11:12:29.582 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 15, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 15, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4263": { "addr": "172.21.15.7:6839/967742498", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 967742498, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 967742498, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4263, "incarnation": 13, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 27 }, "gid_4288": { "addr": "172.21.15.117:6835/4051500902", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 4051500902, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 4051500902, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4288, "incarnation": 9, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 23, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:12:26.135724+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 4288, "mds_2": 4263 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:12:31.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.789+0000 7fbb2f1ff700 1 Processor -- start 2023-06-09T11:12:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.789+0000 7fbb2f1ff700 1 -- start start 2023-06-09T11:12:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2f1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c1bc0 0x7fbb281599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 0x7fbb28159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2f1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb280c0bf0 con 0x7fbb280c1f90 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2f1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb28070110 con 0x7fbb280c1bc0 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28071300 con 0x7fbb280c0140 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2cf9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c1bc0 0x7fbb281599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2d79c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbb280c1f90 0x7fbb281501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40542/0 (socket says 172.21.15.7:40542) 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2d79c700 1 -- 172.21.15.7:0/3316521843 learned_addr learned my addr 172.21.15.7:0/3316521843 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.790+0000 7fbb2cf9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbb280c1bc0 0x7fbb281599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58188/0 (socket says 172.21.15.7:58188) 2023-06-09T11:12:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb27fff700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 0x7fbb28159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3316521843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 msgr2=0x7fbb28159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb2cf9b700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 0x7fbb28159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3316521843 >> v1:172.21.15.7:6789/0 conn(0x7fbb280c1f90 legacy=0x7fbb281501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3316521843 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb280702f0 con 0x7fbb280c1bc0 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb27fff700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 0x7fbb28159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb2cf9b700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c1bc0 0x7fbb281599b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00bae0 tx=0x7fbb1c00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a850efc61223967 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb277fe700 1 -- 172.21.15.7:0/3316521843 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c00dd30 con 0x7fbb280c1bc0 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb277fe700 1 -- 172.21.15.7:0/3316521843 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c00de90 con 0x7fbb280c1bc0 2023-06-09T11:12:31.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.791+0000 7fbb277fe700 1 -- 172.21.15.7:0/3316521843 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c010690 con 0x7fbb280c1bc0 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3316521843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c1bc0 msgr2=0x7fbb281599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c1bc0 0x7fbb281599b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c00bae0 tx=0x7fbb1c00bdf0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3316521843 shutdown_connections 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c1bc0 0x7fbb281599b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3316521843 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c0140 0x7fbb28159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3316521843 >> 172.21.15.7:0/3316521843 conn(0x7fbb2806a4d0 msgr2=0x7fbb2806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:31.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3316521843 shutdown_connections 2023-06-09T11:12:31.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.792+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3316521843 wait complete. 2023-06-09T11:12:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.794+0000 7fbb2f1ff700 1 Processor -- start 2023-06-09T11:12:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.795+0000 7fbb2f1ff700 1 -- start start 2023-06-09T11:12:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c1bc0 0x7fbb280ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb280c1f90 0x7fbb280ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280ada30 con 0x7fbb280c0140 2023-06-09T11:12:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c33c0 con 0x7fbb280c1f90 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb280c3c60 con 0x7fbb280c1bc0 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58202/0 (socket says 172.21.15.7:58202) 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3130264836 learned_addr learned my addr 172.21.15.7:0/3130264836 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c1bc0 msgr2=0x7fbb280ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c1bc0 0x7fbb280ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb280c1f90 msgr2=0x7fbb280ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb280c1f90 0x7fbb280ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.796+0000 7fbb2cf9b700 1 -- 172.21.15.7:0/3130264836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c00b740 con 0x7fbb280c0140 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb2cf9b700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c000c00 tx=0x7fbb1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c003dd0 con 0x7fbb280c0140 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1c010470 con 0x7fbb280c0140 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb280c1170 con 0x7fbb280c0140 2023-06-09T11:12:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1c010800 con 0x7fbb280c0140 2023-06-09T11:12:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.797+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb280c0e90 con 0x7fbb280c0140 2023-06-09T11:12:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.798+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb280c1450 con 0x7fbb280c0140 2023-06-09T11:12:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.799+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb1c019400 con 0x7fbb280c0140 2023-06-09T11:12:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.800+0000 7fbb257fa700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 0x7fbb0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.800+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fbb1c08aba0 con 0x7fbb280c0140 2023-06-09T11:12:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.800+0000 7fbb27fff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 0x7fbb0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.801+0000 7fbb27fff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 0x7fbb0c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007920 tx=0x7fbb14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:31.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:31.804+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbb1c055bb0 con 0x7fbb280c0140 2023-06-09T11:12:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.059+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb280adb70 con 0x7fbb280c0140 2023-06-09T11:12:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.060+0000 7fbb257fa700 1 -- 172.21.15.7:0/3130264836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5449 (secure 0 0 0) 0x7fbb280adb70 con 0x7fbb280c0140 2023-06-09T11:12:32.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:32.066 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:31.153105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 msgr2=0x7fbb0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 0x7fbb0c06d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb14007920 tx=0x7fbb14008040 comp rx=0 tx=0).stop 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 msgr2=0x7fbb281f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c000c00 tx=0x7fbb1c004750 comp rx=0 tx=0).stop 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 shutdown_connections 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbb280c1f90 0x7fbb280ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbb280c0140 0x7fbb281f2530 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbb0c06ace0 0x7fbb0c06d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 --2- 172.21.15.7:0/3130264836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbb280c1bc0 0x7fbb280ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 >> 172.21.15.7:0/3130264836 conn(0x7fbb2806a4d0 msgr2=0x7fbb2814e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 shutdown_connections 2023-06-09T11:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:32.063+0000 7fbb2f1ff700 1 -- 172.21.15.7:0/3130264836 wait complete. 2023-06-09T11:12:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-06-09T11:12:32.087 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:34.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.297+0000 7f6f13dff700 1 Processor -- start 2023-06-09T11:12:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.297+0000 7f6f13dff700 1 -- start start 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f13dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f0c0cb5d0 0x7f6f0c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f13dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c159b20 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f13dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c0cac80 con 0x7f6f0c15cd40 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f13dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c1597d0 con 0x7f6f0c0cb5d0 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f1139a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6f0c0cb9a0 0x7f6f0c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46628/0 (socket says 172.21.15.7:46628) 2023-06-09T11:12:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.298+0000 7f6f1139a700 1 -- 172.21.15.7:0/3468125846 learned_addr learned my addr 172.21.15.7:0/3468125846 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/3468125846 >> v2:172.21.15.117:3300/0 conn(0x7f6f0c0cb5d0 0x7f6f0c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669911116 0 0) 0x7f6f0c159b20 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef80035b0 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323743973 0 0) 0x7f6ef80035b0 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 >> v2:172.21.15.117:3300/0 conn(0x7f6f0c0cb5d0 msgr2=0x7f6f0c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 --2- 172.21.15.7:0/3468125846 >> v2:172.21.15.117:3300/0 conn(0x7f6f0c0cb5d0 0x7f6f0c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 >> v1:172.21.15.117:6789/0 conn(0x7f6f0c15cd40 legacy=0x7f6f0c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c071ab0 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f6efc001670 con 0x7f6f0c0cb9a0 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/3468125846 >> v2:172.21.15.117:3300/0 conn(0x7f6f0c0cb5d0 0x7f6f0c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 >> v1:172.21.15.7:6789/0 conn(0x7f6f0c0cb9a0 legacy=0x7f6f0c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8003a00 0x7f6ef8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef8004850 0x7f6ef8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 0x7f6ef800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c1597d0 con 0x7f6ef8005200 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef800c060 con 0x7f6ef8003a00 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.299+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef800c220 con 0x7f6ef8004850 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 0x7f6ef800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8003a00 0x7f6ef8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1139a700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef8004850 0x7f6ef8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 -- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef8004850 msgr2=0x7f6ef8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ef8004850 0x7f6ef8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 -- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8003a00 msgr2=0x7f6ef8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8003a00 0x7f6ef8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 -- 172.21.15.7:0/3468125846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef80035b0 con 0x7f6ef8005200 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 -- 172.21.15.7:0/3468125846 reap_dead start 2023-06-09T11:12:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.300+0000 7f6f1239c700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 0x7f6ef800ba10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c160 tx=0x7f6ef8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ce463c33925bda server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04001db0 con 0x7f6ef8005200 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400cca0 con 0x7f6ef8005200 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f10b99700 1 -- 172.21.15.7:0/3468125846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04003c40 con 0x7f6ef8005200 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 -- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 msgr2=0x7f6ef800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 0x7f6ef800ba10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c160 tx=0x7f6ef8005880 comp rx=0 tx=0).stop 2023-06-09T11:12:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 -- 172.21.15.7:0/3468125846 shutdown_connections 2023-06-09T11:12:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 --2- 172.21.15.7:0/3468125846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ef8005200 0x7f6ef800ba10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 -- 172.21.15.7:0/3468125846 >> 172.21.15.7:0/3468125846 conn(0x7f6f0c070af0 msgr2=0x7f6f0c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 -- 172.21.15.7:0/3468125846 shutdown_connections 2023-06-09T11:12:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.301+0000 7f6f13dff700 1 -- 172.21.15.7:0/3468125846 wait complete. 2023-06-09T11:12:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.303+0000 7f6f13dff700 1 Processor -- start 2023-06-09T11:12:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.303+0000 7f6f13dff700 1 -- start start 2023-06-09T11:12:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8005200 0x7f6f0c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f0c0d0200 0x7f6f0c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c80e0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0b1780 con 0x7f6ef8005200 2023-06-09T11:12:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f13dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c8670 con 0x7f6f0c0d0200 2023-06-09T11:12:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f1139a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f1139a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47396/0 (socket says 172.21.15.7:47396) 2023-06-09T11:12:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f1139a700 1 -- 172.21.15.7:0/2175263974 learned_addr learned my addr 172.21.15.7:0/2175263974 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.304+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8005200 0x7f6f0c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1239c700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f0c0d0200 0x7f6f0c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 -- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f0c0d0200 msgr2=0x7f6f0c155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f0c0d0200 0x7f6f0c155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 -- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8005200 msgr2=0x7f6f0c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8005200 0x7f6f0c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 -- 172.21.15.7:0/2175263974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c15d4f0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f1139a700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0072c0 tx=0x7f6efc0075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011580 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc0116e0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c15d210 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.305+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011a70 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.306+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c15d7d0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.306+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c074cf0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.308+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6efc011bd0 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.309+0000 7f6f027fc700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 0x7f6ef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.309+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f6efc089590 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.309+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 0x7f6ef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:34.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.309+0000 7f6f11b9b700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 0x7f6ef006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009470 tx=0x7f6f08009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.313+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6efc054510 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.571+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f0c0ce570 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.571+0000 7f6f027fc700 1 -- 172.21.15.7:0/2175263974 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+5449 (secure 0 0 0) 0x7f6efc057b30 con 0x7f6f0c0b1d90 2023-06-09T11:12:34.577 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:34.578 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:31.153105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:34.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 msgr2=0x7f6ef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 0x7f6ef006d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009470 tx=0x7f6f08009210 comp rx=0 tx=0).stop 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 msgr2=0x7f6f0c0c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0072c0 tx=0x7f6efc0075d0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 shutdown_connections 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ef006ace0 0x7f6ef006d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f0c0d0200 0x7f6f0c155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ef8005200 0x7f6f0c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 --2- 172.21.15.7:0/2175263974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f0c0b1d90 0x7f6f0c0c41c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 >> 172.21.15.7:0/2175263974 conn(0x7f6f0c070af0 msgr2=0x7f6f0c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 shutdown_connections 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:34.574+0000 7f6f13dff700 1 -- 172.21.15.7:0/2175263974 wait complete. 2023-06-09T11:12:34.581 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 16 2023-06-09T11:12:34.598 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:12:36.187 INFO:tasks.workunit.client.0.smithi007.stderr:+ wget -q http://download.ceph.com/qa/linux-2.6.33.tar.bz2 2023-06-09T11:12:36.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.809+0000 7f804c1ff700 1 Processor -- start 2023-06-09T11:12:36.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.810+0000 7f804c1ff700 1 -- start start 2023-06-09T11:12:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804c1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f80440cb9a0 0x7f80440b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804c1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044159b20 con 0x7f804415cd40 2023-06-09T11:12:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804c1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440cac80 con 0x7f80440cb5d0 2023-06-09T11:12:36.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804c1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80441597d0 con 0x7f80440cb9a0 2023-06-09T11:12:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804a79c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f804415cd40 0x7f80440cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46630/0 (socket says 172.21.15.7:46630) 2023-06-09T11:12:36.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.811+0000 7f804a79c700 1 -- 172.21.15.7:0/2307349142 learned_addr learned my addr 172.21.15.7:0/2307349142 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:12:36.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f804979a700 1 --2- 172.21.15.7:0/2307349142 >> v2:172.21.15.117:3300/0 conn(0x7f80440cb9a0 0x7f80440b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f8048f99700 1 -- 172.21.15.7:0/2307349142 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924442462 0 0) 0x7f8044159b20 con 0x7f804415cd40 2023-06-09T11:12:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f8048f99700 1 -- 172.21.15.7:0/2307349142 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80300035b0 con 0x7f804415cd40 2023-06-09T11:12:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f804979a700 1 -- 172.21.15.7:0/2307349142 >> v1:172.21.15.117:6789/0 conn(0x7f80440cb5d0 legacy=0x7f8044159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f804979a700 1 -- 172.21.15.7:0/2307349142 >> v1:172.21.15.7:6789/0 conn(0x7f804415cd40 legacy=0x7f80440cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:36.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f804979a700 1 -- 172.21.15.7:0/2307349142 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044071ab0 con 0x7f80440cb9a0 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.812+0000 7f804979a700 1 --2- 172.21.15.7:0/2307349142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80440cb9a0 0x7f80440b52b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8034009840 tx=0x7f8034009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1937ef6ee9b15e9b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.813+0000 7f8048f99700 1 -- 172.21.15.7:0/2307349142 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034011830 con 0x7f80440cb9a0 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.813+0000 7f8048f99700 1 -- 172.21.15.7:0/2307349142 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034011990 con 0x7f80440cb9a0 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.813+0000 7f8048f99700 1 -- 172.21.15.7:0/2307349142 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034011d20 con 0x7f80440cb9a0 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.813+0000 7f804c1ff700 1 -- 172.21.15.7:0/2307349142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80440cb9a0 msgr2=0x7f80440b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.813+0000 7f804c1ff700 1 --2- 172.21.15.7:0/2307349142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80440cb9a0 0x7f80440b52b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8034009840 tx=0x7f8034009b50 comp rx=0 tx=0).stop 2023-06-09T11:12:36.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.814+0000 7f804c1ff700 1 -- 172.21.15.7:0/2307349142 shutdown_connections 2023-06-09T11:12:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.814+0000 7f804c1ff700 1 --2- 172.21.15.7:0/2307349142 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80440cb9a0 0x7f80440b52b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.814+0000 7f804c1ff700 1 -- 172.21.15.7:0/2307349142 >> 172.21.15.7:0/2307349142 conn(0x7f8044070af0 msgr2=0x7f8044070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.814+0000 7f804c1ff700 1 -- 172.21.15.7:0/2307349142 shutdown_connections 2023-06-09T11:12:36.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.814+0000 7f804c1ff700 1 -- 172.21.15.7:0/2307349142 wait complete. 2023-06-09T11:12:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.816+0000 7f804c1ff700 1 Processor -- start 2023-06-09T11:12:36.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.816+0000 7f804c1ff700 1 -- start start 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80440cb5d0 0x7f80441f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 0x7f80440c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441f23d0 con 0x7f80440cb9a0 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804415da80 con 0x7f804415cd40 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.817+0000 7f804c1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440c3660 con 0x7f80440cb5d0 2023-06-09T11:12:36.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47404/0 (socket says 172.21.15.7:47404) 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 -- 172.21.15.7:0/1539917580 learned_addr learned my addr 172.21.15.7:0/1539917580 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f8049f9b700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80440cb5d0 0x7f80441f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804a79c700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 0x7f80440c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 -- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80440cb5d0 msgr2=0x7f80441f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80440cb5d0 0x7f80441f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:36.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 -- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 msgr2=0x7f80440c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 0x7f80440c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 -- 172.21.15.7:0/1539917580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340094a0 con 0x7f80440cb9a0 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804a79c700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 0x7f80440c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.818+0000 7f804979a700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8034009fd0 tx=0x7f8034004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.819+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80340038a0 con 0x7f80440cb9a0 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.819+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803401f070 con 0x7f80440cb9a0 2023-06-09T11:12:36.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.819+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034003c40 con 0x7f80440cb9a0 2023-06-09T11:12:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.819+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804415d4f0 con 0x7f80440cb9a0 2023-06-09T11:12:36.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.819+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804415d210 con 0x7f80440cb9a0 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.820+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804415d7d0 con 0x7f80440cb9a0 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.821+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8034026480 con 0x7f80440cb9a0 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.822+0000 7f803a7fc700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.822+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f8034021070 con 0x7f80440cb9a0 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.822+0000 7f8049f9b700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:36.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.823+0000 7f8049f9b700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 0x7f802806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:36.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:36.828+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8034055de0 con 0x7f80440cb9a0 2023-06-09T11:12:37.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.121+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8044074cf0 con 0x7f80440cb9a0 2023-06-09T11:12:37.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.122+0000 7f803a7fc700 1 -- 172.21.15.7:0/1539917580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5449 (secure 0 0 0) 0x7f8034015090 con 0x7f80440cb9a0 2023-06-09T11:12:37.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:37.128 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:36.168701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 msgr2=0x7f802806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 0x7f802806d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040009040 comp rx=0 tx=0).stop 2023-06-09T11:12:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 msgr2=0x7f80440c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:37.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8034009fd0 tx=0x7f8034004750 comp rx=0 tx=0).stop 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 shutdown_connections 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f804415cd40 0x7f80440c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f802806ace0 0x7f802806d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80440cb5d0 0x7f80441f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 --2- 172.21.15.7:0/1539917580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80440cb9a0 0x7f80440c7ac0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.125+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 >> 172.21.15.7:0/1539917580 conn(0x7f8044070af0 msgr2=0x7f8044156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:37.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.126+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 shutdown_connections 2023-06-09T11:12:37.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:37.126+0000 7f804c1ff700 1 -- 172.21.15.7:0/1539917580 wait complete. 2023-06-09T11:12:37.133 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-06-09T11:12:37.154 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=4282 2023-06-09T11:12:37.154 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.b 2023-06-09T11:12:39.797 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:12:39.792+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:12:40.048 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir foo 2023-06-09T11:12:40.062 INFO:tasks.workunit.client.0.smithi007.stderr:+ cp linux-2.6.33.tar.bz2 foo 2023-06-09T11:12:40.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:40.295 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:12:40.733 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir foo/.snap/barsnap 2023-06-09T11:12:41.189 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm foo/linux-2.6.33.tar.bz2 2023-06-09T11:12:41.219 INFO:tasks.workunit.client.0.smithi007.stderr:+ diff -q foo/.snap/barsnap/linux-2.6.33.tar.bz2 linux-2.6.33.tar.bz2 2023-06-09T11:12:41.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 'passed: files are identical' 2023-06-09T11:12:41.407 INFO:tasks.workunit.client.0.smithi007.stdout:passed: files are identical 2023-06-09T11:12:41.407 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir foo/.snap/barsnap 2023-06-09T11:12:41.641 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:12:41.642 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:12:41.642 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:12:41.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:12:41.749 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-1.sh... 2023-06-09T11:12:41.749 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-1.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-1.sh 2023-06-09T11:12:41.814 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 1 2023-06-09T11:12:41.817 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 2 2023-06-09T11:12:41.819 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 3 2023-06-09T11:12:41.821 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' -e file4 ']' 2023-06-09T11:12:41.822 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/snap1 2023-06-09T11:12:41.840 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 4 2023-06-09T11:12:41.843 INFO:tasks.workunit.client.0.smithi007.stderr:++ ls 2023-06-09T11:12:41.851 INFO:tasks.workunit.client.0.smithi007.stderr:+ now='file1 2023-06-09T11:12:41.852 INFO:tasks.workunit.client.0.smithi007.stderr:file2 2023-06-09T11:12:41.852 INFO:tasks.workunit.client.0.smithi007.stderr:file3 2023-06-09T11:12:41.852 INFO:tasks.workunit.client.0.smithi007.stderr:file4' 2023-06-09T11:12:41.852 INFO:tasks.workunit.client.0.smithi007.stderr:++ ls .snap/snap1 2023-06-09T11:12:41.862 INFO:tasks.workunit.client.0.smithi007.stderr:+ then='file1 2023-06-09T11:12:41.862 INFO:tasks.workunit.client.0.smithi007.stderr:file2 2023-06-09T11:12:41.863 INFO:tasks.workunit.client.0.smithi007.stderr:file3' 2023-06-09T11:12:41.863 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir .snap/snap1 2023-06-09T11:12:41.874 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 'file1 2023-06-09T11:12:41.874 INFO:tasks.workunit.client.0.smithi007.stderr:file2 2023-06-09T11:12:41.874 INFO:tasks.workunit.client.0.smithi007.stderr:file3 2023-06-09T11:12:41.874 INFO:tasks.workunit.client.0.smithi007.stderr:file4' = 'file1 2023-06-09T11:12:41.874 INFO:tasks.workunit.client.0.smithi007.stderr:file2 2023-06-09T11:12:41.875 INFO:tasks.workunit.client.0.smithi007.stderr:file3' ']' 2023-06-09T11:12:41.875 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 1 2023-06-09T11:12:41.881 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 2 2023-06-09T11:12:41.888 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 3 2023-06-09T11:12:41.900 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/snap1 2023-06-09T11:12:41.912 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 4 2023-06-09T11:12:41.922 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir .snap/snap1 2023-06-09T11:12:41.935 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm file1 file2 file3 file4 2023-06-09T11:12:41.948 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:12:41.949 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:12:41.949 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:12:41.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:12:42.031 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-2.sh... 2023-06-09T11:12:42.031 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-2.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-2.sh 2023-06-09T11:12:42.096 INFO:tasks.workunit.client.0.smithi007.stdout:Create dir 100 to 199 ... 2023-06-09T11:12:42.106 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 100 2023-06-09T11:12:42.139 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 101 2023-06-09T11:12:42.155 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:12:42.156 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:12:42.156 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:12:42.161 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:12:42.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:42.171 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 102 2023-06-09T11:12:42.206 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 103 2023-06-09T11:12:42.255 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 104 2023-06-09T11:12:42.288 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 105 2023-06-09T11:12:42.316 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:12:42.334 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 106 2023-06-09T11:12:42.367 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 107 2023-06-09T11:12:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.391+0000 7f640d5ff700 1 Processor -- start 2023-06-09T11:12:42.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.392+0000 7f640d5ff700 1 -- start start 2023-06-09T11:12:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f640d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64081501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f640d5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64080c0140 0x7f64081506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f640d5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64080c0bf0 con 0x7f64080c1bc0 2023-06-09T11:12:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f640d5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6408070110 con 0x7f64080c0140 2023-06-09T11:12:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f640d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408071300 con 0x7f640815a0c0 2023-06-09T11:12:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f64077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64080c0140 0x7f64081506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f64077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f64080c0140 0x7f64081506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47408/0 (socket says 172.21.15.7:47408) 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f64077fe700 1 -- 172.21.15.7:0/1001571123 learned_addr learned my addr 172.21.15.7:0/1001571123 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.393+0000 7f64067fc700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64081501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f64077fe700 1 -- 172.21.15.7:0/1001571123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 msgr2=0x7f64081501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f64077fe700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64081501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f64077fe700 1 -- 172.21.15.7:0/1001571123 >> v1:172.21.15.117:6789/0 conn(0x7f64080c1bc0 legacy=0x7f64081599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f64077fe700 1 -- 172.21.15.7:0/1001571123 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080702f0 con 0x7f64080c0140 2023-06-09T11:12:42.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f64077fe700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f64081506f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f640000d150 tx=0x7f640000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8128f02748bbb04d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:42.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f6405ffb700 1 -- 172.21.15.7:0/1001571123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400010070 con 0x7f64080c0140 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f6405ffb700 1 -- 172.21.15.7:0/1001571123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640000dd10 con 0x7f64080c0140 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.394+0000 7f6405ffb700 1 -- 172.21.15.7:0/1001571123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640000b310 con 0x7f64080c0140 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 -- 172.21.15.7:0/1001571123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 msgr2=0x7f64081506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f64081506f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f640000d150 tx=0x7f640000d460 comp rx=0 tx=0).stop 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 -- 172.21.15.7:0/1001571123 shutdown_connections 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f64081506f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 --2- 172.21.15.7:0/1001571123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64081501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 -- 172.21.15.7:0/1001571123 >> 172.21.15.7:0/1001571123 conn(0x7f640806a4d0 msgr2=0x7f640806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 -- 172.21.15.7:0/1001571123 shutdown_connections 2023-06-09T11:12:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.395+0000 7f640d5ff700 1 -- 172.21.15.7:0/1001571123 wait complete. 2023-06-09T11:12:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.396+0000 7f640d5ff700 1 Processor -- start 2023-06-09T11:12:42.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.397+0000 7f640d5ff700 1 -- start start 2023-06-09T11:12:42.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64080c1bc0 0x7f6408158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64080ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c29f0 con 0x7f64080c0140 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081eef70 con 0x7f64080c1bc0 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f640d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080c3580 con 0x7f640815a0c0 2023-06-09T11:12:42.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47414/0 (socket says 172.21.15.7:47414) 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 -- 172.21.15.7:0/831887150 learned_addr learned my addr 172.21.15.7:0/831887150 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f64067fc700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64080c1bc0 0x7f6408158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f64077fe700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64080ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 -- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 msgr2=0x7f64080ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64080ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 -- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64080c1bc0 msgr2=0x7f6408158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64080c1bc0 0x7f6408158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.398+0000 7f6406ffd700 1 -- 172.21.15.7:0/831887150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640000bd40 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f6406ffd700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c4e0 tx=0x7f63f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80052d0 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f8005430 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080c1170 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80038c0 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.399+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080c0e90 con 0x7f64080c0140 2023-06-09T11:12:42.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.400+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64080c1450 con 0x7f64080c0140 2023-06-09T11:12:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.401+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f63f8003a20 con 0x7f64080c0140 2023-06-09T11:12:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.402+0000 7f63ef7fe700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 0x7f63f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:42.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.403+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f63f80888b0 con 0x7f64080c0140 2023-06-09T11:12:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.403+0000 7f64067fc700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 0x7f63f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:42.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.403+0000 7f64067fc700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 0x7f63f406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64080cb020 tx=0x7f63fc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:42.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.407+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f63f80538c0 con 0x7f64080c0140 2023-06-09T11:12:42.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.658+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6408074cf0 con 0x7f64080c0140 2023-06-09T11:12:42.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.658+0000 7f63ef7fe700 1 -- 172.21.15.7:0/831887150 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+5449 (secure 0 0 0) 0x7f63f8056ee0 con 0x7f64080c0140 2023-06-09T11:12:42.665 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:42.665 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:36.168701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 msgr2=0x7f63f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 0x7f63f406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f64080cb020 tx=0x7f63fc008040 comp rx=0 tx=0).stop 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 msgr2=0x7f640814b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c4e0 tx=0x7f63f800c8a0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 shutdown_connections 2023-06-09T11:12:42.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f63f406ace0 0x7f63f406d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f640815a0c0 0x7f64080ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64080c0140 0x7f640814b7d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 --2- 172.21.15.7:0/831887150 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64080c1bc0 0x7f6408158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 >> 172.21.15.7:0/831887150 conn(0x7f640806a4d0 msgr2=0x7f64080c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 shutdown_connections 2023-06-09T11:12:42.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:42.661+0000 7f640d5ff700 1 -- 172.21.15.7:0/831887150 wait complete. 2023-06-09T11:12:42.669 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2023-06-09T11:12:42.685 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:12:43.230 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 108 2023-06-09T11:12:43.265 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 109 2023-06-09T11:12:43.298 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 110 2023-06-09T11:12:43.330 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 111 2023-06-09T11:12:43.362 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 112 2023-06-09T11:12:43.395 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 113 2023-06-09T11:12:43.428 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 114 2023-06-09T11:12:43.459 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 115 2023-06-09T11:12:43.488 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 116 2023-06-09T11:12:43.523 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 117 2023-06-09T11:12:43.553 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 118 2023-06-09T11:12:43.584 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 119 2023-06-09T11:12:43.616 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 120 2023-06-09T11:12:43.647 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 121 2023-06-09T11:12:43.680 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 122 2023-06-09T11:12:43.712 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 123 2023-06-09T11:12:43.746 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 124 2023-06-09T11:12:43.779 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 125 2023-06-09T11:12:43.812 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 126 2023-06-09T11:12:43.847 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 127 2023-06-09T11:12:43.879 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 128 2023-06-09T11:12:43.910 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 129 2023-06-09T11:12:43.944 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 130 2023-06-09T11:12:43.972 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 131 2023-06-09T11:12:44.003 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 132 2023-06-09T11:12:44.034 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 133 2023-06-09T11:12:44.066 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 134 2023-06-09T11:12:44.096 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 135 2023-06-09T11:12:44.128 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 136 2023-06-09T11:12:44.159 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 137 2023-06-09T11:12:44.190 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 138 2023-06-09T11:12:44.232 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 139 2023-06-09T11:12:44.267 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 140 2023-06-09T11:12:44.299 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 141 2023-06-09T11:12:44.331 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 142 2023-06-09T11:12:44.364 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 143 2023-06-09T11:12:44.398 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 144 2023-06-09T11:12:44.431 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 145 2023-06-09T11:12:44.465 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 146 2023-06-09T11:12:44.496 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 147 2023-06-09T11:12:44.528 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 148 2023-06-09T11:12:44.560 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 149 2023-06-09T11:12:44.591 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 150 2023-06-09T11:12:44.622 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 151 2023-06-09T11:12:44.655 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 152 2023-06-09T11:12:44.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:12:44.692 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 153 2023-06-09T11:12:44.725 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 154 2023-06-09T11:12:44.759 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 155 2023-06-09T11:12:44.792 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 156 2023-06-09T11:12:44.825 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 157 2023-06-09T11:12:44.857 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 158 2023-06-09T11:12:44.891 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 159 2023-06-09T11:12:44.926 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 160 2023-06-09T11:12:44.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.926+0000 7f6f7c3ff700 1 Processor -- start 2023-06-09T11:12:44.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.926+0000 7f6f7c3ff700 1 -- start start 2023-06-09T11:12:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f74153880 0x7f6f7415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f740c9f40 0x7f6f7415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f7415b2d0 0x7f6f740c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f740c5f10 con 0x7f6f740c9f40 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7415a500 con 0x7f6f7415b2d0 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7415da00 con 0x7f6f74153880 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.927+0000 7f6f7999a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f740c9f40 0x7f6f7415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6f740c9f40 0x7f6f7415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45466/0 (socket says 172.21.15.7:45466) 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 -- 172.21.15.7:0/3352928594 learned_addr learned my addr 172.21.15.7:0/3352928594 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7a19b700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f74153880 0x7f6f7415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7a99c700 1 --2- 172.21.15.7:0/3352928594 >> v2:172.21.15.117:3300/0 conn(0x7f6f7415b2d0 0x7f6f740c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 -- 172.21.15.7:0/3352928594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f74153880 msgr2=0x7f6f7415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f74153880 0x7f6f7415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 -- 172.21.15.7:0/3352928594 >> v2:172.21.15.117:3300/0 conn(0x7f6f7415b2d0 msgr2=0x7f6f740c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 --2- 172.21.15.7:0/3352928594 >> v2:172.21.15.117:3300/0 conn(0x7f6f7415b2d0 0x7f6f740c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 -- 172.21.15.7:0/3352928594 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7415b750 con 0x7f6f740c9f40 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.928+0000 7f6f7999a700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f740c9f40 0x7f6f7415ad90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f74071db0 tx=0x7f6f64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2c5b548d90080b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:44.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f79199700 1 -- 172.21.15.7:0/3352928594 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64011830 con 0x7f6f740c9f40 2023-06-09T11:12:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f79199700 1 -- 172.21.15.7:0/3352928594 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f64011990 con 0x7f6f740c9f40 2023-06-09T11:12:44.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f79199700 1 -- 172.21.15.7:0/3352928594 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64011d20 con 0x7f6f740c9f40 2023-06-09T11:12:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/3352928594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f740c9f40 msgr2=0x7f6f7415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:44.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f740c9f40 0x7f6f7415ad90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6f74071db0 tx=0x7f6f64009bd0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/3352928594 shutdown_connections 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/3352928594 >> v2:172.21.15.117:3300/0 conn(0x7f6f7415b2d0 0x7f6f740c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f74153880 0x7f6f7415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/3352928594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f740c9f40 0x7f6f7415ad90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/3352928594 >> 172.21.15.7:0/3352928594 conn(0x7f6f740737f0 msgr2=0x7f6f740720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:44.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.929+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/3352928594 shutdown_connections 2023-06-09T11:12:44.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.930+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/3352928594 wait complete. 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.931+0000 7f6f7c3ff700 1 Processor -- start 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.932+0000 7f6f7c3ff700 1 -- start start 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 0x7f6f741493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 0x7f6f740b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f741498e0 con 0x7f6f7415b2d0 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740cb0e0 con 0x7f6f74153880 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f740cab20 con 0x7f6f740c9f40 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.933+0000 7f6f7a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7999a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 0x7f6f740b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45474/0 (socket says 172.21.15.7:45474) 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 -- 172.21.15.7:0/2169509380 learned_addr learned my addr 172.21.15.7:0/2169509380 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:12:44.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a19b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 0x7f6f741493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 -- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 msgr2=0x7f6f741493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 0x7f6f741493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 -- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 msgr2=0x7f6f740b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 0x7f6f740b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 -- 172.21.15.7:0/2169509380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009550 con 0x7f6f7415b2d0 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7999a700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 0x7f6f740b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a19b700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 0x7f6f741493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.934+0000 7f6f7a99c700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00efc0 tx=0x7f6f6c00c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.935+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00cad0 con 0x7f6f7415b2d0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.935+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00f040 con 0x7f6f7415b2d0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.935+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00ce70 con 0x7f6f7415b2d0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.935+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f74159980 con 0x7f6f7415b2d0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.935+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f741596e0 con 0x7f6f7415b2d0 2023-06-09T11:12:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.936+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f74159c60 con 0x7f6f7415b2d0 2023-06-09T11:12:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.937+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f6c014480 con 0x7f6f7415b2d0 2023-06-09T11:12:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.939+0000 7f6f6affd700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 0x7f6f5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:12:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.939+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f6f6c089c80 con 0x7f6f7415b2d0 2023-06-09T11:12:44.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.939+0000 7f6f7a19b700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 0x7f6f5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:12:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.940+0000 7f6f7a19b700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 0x7f6f5806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009f50 tx=0x7f6f70009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:12:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:44.945+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6f6c054c90 con 0x7f6f7415b2d0 2023-06-09T11:12:44.960 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 161 2023-06-09T11:12:44.993 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 162 2023-06-09T11:12:45.025 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 163 2023-06-09T11:12:45.056 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 164 2023-06-09T11:12:45.090 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 165 2023-06-09T11:12:45.122 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 166 2023-06-09T11:12:45.155 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 167 2023-06-09T11:12:45.186 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 168 2023-06-09T11:12:45.220 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 169 2023-06-09T11:12:45.257 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 170 2023-06-09T11:12:45.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.262+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f74074cf0 con 0x7f6f7415b2d0 2023-06-09T11:12:45.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.262+0000 7f6f6affd700 1 -- 172.21.15.7:0/2169509380 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+6209 (secure 0 0 0) 0x7f6f6c0582b0 con 0x7f6f7415b2d0 2023-06-09T11:12:45.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:12:45.270 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:12:36.168701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:12:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 msgr2=0x7f6f5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 0x7f6f5806d190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009f50 tx=0x7f6f70009210 comp rx=0 tx=0).stop 2023-06-09T11:12:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 msgr2=0x7f6f741f6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:12:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00efc0 tx=0x7f6f6c00c0c0 comp rx=0 tx=0).stop 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 shutdown_connections 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f7415b2d0 0x7f6f741f6bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f5806ace0 0x7f6f5806d190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f740c9f40 0x7f6f741493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 --2- 172.21.15.7:0/2169509380 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f74153880 0x7f6f740b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.265+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 >> 172.21.15.7:0/2169509380 conn(0x7f6f740737f0 msgr2=0x7f6f7414f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:12:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.266+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 shutdown_connections 2023-06-09T11:12:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:12:45.266+0000 7f6f7c3ff700 1 -- 172.21.15.7:0/2169509380 wait complete. 2023-06-09T11:12:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 18 2023-06-09T11:12:45.295 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:12:45.295 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 18 secs before thrashing 2023-06-09T11:12:45.295 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 171 2023-06-09T11:12:45.325 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 172 2023-06-09T11:12:45.357 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 173 2023-06-09T11:12:45.392 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 174 2023-06-09T11:12:45.426 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 175 2023-06-09T11:12:45.460 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 176 2023-06-09T11:12:45.491 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 177 2023-06-09T11:12:46.285 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 178 2023-06-09T11:12:46.331 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 179 2023-06-09T11:12:46.375 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 180 2023-06-09T11:12:46.414 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 181 2023-06-09T11:12:46.453 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 182 2023-06-09T11:12:46.487 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 183 2023-06-09T11:12:46.519 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 184 2023-06-09T11:12:46.552 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 185 2023-06-09T11:12:46.584 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 186 2023-06-09T11:12:46.617 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 187 2023-06-09T11:12:46.649 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 188 2023-06-09T11:12:46.681 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 189 2023-06-09T11:12:46.716 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 190 2023-06-09T11:12:46.748 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 191 2023-06-09T11:12:46.781 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 192 2023-06-09T11:12:46.812 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 193 2023-06-09T11:12:46.845 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 194 2023-06-09T11:12:46.877 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 195 2023-06-09T11:12:46.908 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 196 2023-06-09T11:12:46.943 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 197 2023-06-09T11:12:46.975 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 198 2023-06-09T11:12:47.007 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 199 2023-06-09T11:12:47.039 INFO:tasks.workunit.client.0.smithi007.stdout:Take first snapshot .snap/test1 2023-06-09T11:12:47.189 INFO:tasks.workunit.client.0.smithi007.stdout:Create dir 200 to 299 ... 2023-06-09T11:12:47.191 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 200 2023-06-09T11:12:47.981 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 201 2023-06-09T11:12:48.012 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 202 2023-06-09T11:12:48.043 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 203 2023-06-09T11:12:48.074 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 204 2023-06-09T11:12:48.104 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 205 2023-06-09T11:12:48.133 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 206 2023-06-09T11:12:48.165 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 207 2023-06-09T11:12:48.194 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 208 2023-06-09T11:12:48.971 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 209 2023-06-09T11:12:49.103 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 210 2023-06-09T11:12:49.137 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 211 2023-06-09T11:12:49.170 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 212 2023-06-09T11:12:49.202 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 213 2023-06-09T11:12:49.233 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 214 2023-06-09T11:12:49.264 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 215 2023-06-09T11:12:49.426 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 216 2023-06-09T11:12:49.457 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 217 2023-06-09T11:12:49.488 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 218 2023-06-09T11:12:49.518 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 219 2023-06-09T11:12:49.548 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 220 2023-06-09T11:12:49.578 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 221 2023-06-09T11:12:49.606 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 222 2023-06-09T11:12:49.637 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 223 2023-06-09T11:12:49.668 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 224 2023-06-09T11:12:50.661 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 225 2023-06-09T11:12:50.693 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 226 2023-06-09T11:12:50.726 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 227 2023-06-09T11:12:50.757 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 228 2023-06-09T11:12:50.788 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 229 2023-06-09T11:12:50.817 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 230 2023-06-09T11:12:50.846 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 231 2023-06-09T11:12:50.877 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 232 2023-06-09T11:12:50.907 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 233 2023-06-09T11:12:50.936 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 234 2023-06-09T11:12:50.965 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 235 2023-06-09T11:12:50.994 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 236 2023-06-09T11:12:51.026 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 237 2023-06-09T11:12:51.059 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 238 2023-06-09T11:12:51.092 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 239 2023-06-09T11:12:51.125 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 240 2023-06-09T11:12:51.463 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 241 2023-06-09T11:12:51.493 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 242 2023-06-09T11:12:51.522 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 243 2023-06-09T11:12:51.551 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 244 2023-06-09T11:12:51.581 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 245 2023-06-09T11:12:51.613 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 246 2023-06-09T11:12:51.641 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 247 2023-06-09T11:12:51.670 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 248 2023-06-09T11:12:51.700 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 249 2023-06-09T11:12:51.732 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 250 2023-06-09T11:12:51.768 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 251 2023-06-09T11:12:56.246 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 252 2023-06-09T11:12:56.276 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 253 2023-06-09T11:12:56.307 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 254 2023-06-09T11:12:56.337 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 255 2023-06-09T11:12:57.339 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 256 2023-06-09T11:12:57.412 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 257 2023-06-09T11:12:57.443 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 258 2023-06-09T11:12:57.473 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 259 2023-06-09T11:12:57.505 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 260 2023-06-09T11:12:57.591 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 261 2023-06-09T11:12:57.620 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 262 2023-06-09T11:12:57.650 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 263 2023-06-09T11:12:57.680 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 264 2023-06-09T11:12:57.710 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 265 2023-06-09T11:12:57.739 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 266 2023-06-09T11:12:57.768 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 267 2023-06-09T11:12:57.798 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 268 2023-06-09T11:12:57.826 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 269 2023-06-09T11:12:57.858 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 270 2023-06-09T11:12:57.888 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 271 2023-06-09T11:12:57.918 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 272 2023-06-09T11:12:57.949 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 273 2023-06-09T11:12:57.979 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 274 2023-06-09T11:12:58.008 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 275 2023-06-09T11:12:58.037 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 276 2023-06-09T11:12:58.067 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 277 2023-06-09T11:12:58.097 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 278 2023-06-09T11:12:58.126 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 279 2023-06-09T11:12:58.156 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 280 2023-06-09T11:12:58.688 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 281 2023-06-09T11:12:58.719 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 282 2023-06-09T11:12:58.749 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 283 2023-06-09T11:12:58.779 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 284 2023-06-09T11:12:58.808 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 285 2023-06-09T11:12:58.837 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 286 2023-06-09T11:12:58.868 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 287 2023-06-09T11:12:58.897 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 288 2023-06-09T11:12:58.926 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 289 2023-06-09T11:12:58.956 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 290 2023-06-09T11:12:58.986 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 291 2023-06-09T11:12:59.015 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 292 2023-06-09T11:12:59.044 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 293 2023-06-09T11:12:59.074 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 294 2023-06-09T11:12:59.103 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 295 2023-06-09T11:12:59.133 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 296 2023-06-09T11:12:59.164 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 297 2023-06-09T11:12:59.194 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 298 2023-06-09T11:12:59.225 INFO:tasks.workunit.client.0.smithi007.stdout: create dir 299 2023-06-09T11:12:59.255 INFO:tasks.workunit.client.0.smithi007.stdout:Create a snapshot in every first level dir ... 2023-06-09T11:12:59.449 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/.snap/snap-subdir-test 2023-06-09T11:12:59.472 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/30 file after the snapshot 2023-06-09T11:12:59.475 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/31 file after the snapshot 2023-06-09T11:12:59.478 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/32 file after the snapshot 2023-06-09T11:12:59.482 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/33 file after the snapshot 2023-06-09T11:12:59.485 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/34 file after the snapshot 2023-06-09T11:13:00.431 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/35 file after the snapshot 2023-06-09T11:13:00.435 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/36 file after the snapshot 2023-06-09T11:13:00.439 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/37 file after the snapshot 2023-06-09T11:13:00.444 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/38 file after the snapshot 2023-06-09T11:13:00.448 INFO:tasks.workunit.client.0.smithi007.stdout: create 100/39 file after the snapshot 2023-06-09T11:13:00.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/.snap/snap-subdir-test 2023-06-09T11:13:00.481 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/30 file after the snapshot 2023-06-09T11:13:00.484 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/31 file after the snapshot 2023-06-09T11:13:00.488 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/32 file after the snapshot 2023-06-09T11:13:00.492 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/33 file after the snapshot 2023-06-09T11:13:00.495 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/34 file after the snapshot 2023-06-09T11:13:00.499 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/35 file after the snapshot 2023-06-09T11:13:00.503 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/36 file after the snapshot 2023-06-09T11:13:00.507 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/37 file after the snapshot 2023-06-09T11:13:00.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/38 file after the snapshot 2023-06-09T11:13:00.514 INFO:tasks.workunit.client.0.smithi007.stdout: create 101/39 file after the snapshot 2023-06-09T11:13:00.517 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/.snap/snap-subdir-test 2023-06-09T11:13:00.544 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/30 file after the snapshot 2023-06-09T11:13:00.547 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/31 file after the snapshot 2023-06-09T11:13:00.552 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/32 file after the snapshot 2023-06-09T11:13:00.555 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/33 file after the snapshot 2023-06-09T11:13:00.559 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/34 file after the snapshot 2023-06-09T11:13:01.192 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/35 file after the snapshot 2023-06-09T11:13:01.266 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/36 file after the snapshot 2023-06-09T11:13:01.270 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/37 file after the snapshot 2023-06-09T11:13:01.273 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/38 file after the snapshot 2023-06-09T11:13:01.277 INFO:tasks.workunit.client.0.smithi007.stdout: create 102/39 file after the snapshot 2023-06-09T11:13:01.281 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/.snap/snap-subdir-test 2023-06-09T11:13:01.473 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/30 file after the snapshot 2023-06-09T11:13:01.476 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/31 file after the snapshot 2023-06-09T11:13:01.479 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/32 file after the snapshot 2023-06-09T11:13:01.481 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/33 file after the snapshot 2023-06-09T11:13:01.483 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/34 file after the snapshot 2023-06-09T11:13:01.485 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/35 file after the snapshot 2023-06-09T11:13:01.487 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/36 file after the snapshot 2023-06-09T11:13:01.490 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/37 file after the snapshot 2023-06-09T11:13:01.493 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/38 file after the snapshot 2023-06-09T11:13:01.495 INFO:tasks.workunit.client.0.smithi007.stdout: create 103/39 file after the snapshot 2023-06-09T11:13:01.498 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/.snap/snap-subdir-test 2023-06-09T11:13:01.686 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/30 file after the snapshot 2023-06-09T11:13:01.688 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/31 file after the snapshot 2023-06-09T11:13:01.691 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/32 file after the snapshot 2023-06-09T11:13:01.693 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/33 file after the snapshot 2023-06-09T11:13:01.696 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/34 file after the snapshot 2023-06-09T11:13:01.697 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/35 file after the snapshot 2023-06-09T11:13:01.700 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/36 file after the snapshot 2023-06-09T11:13:01.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/37 file after the snapshot 2023-06-09T11:13:01.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/38 file after the snapshot 2023-06-09T11:13:01.707 INFO:tasks.workunit.client.0.smithi007.stdout: create 104/39 file after the snapshot 2023-06-09T11:13:01.709 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/.snap/snap-subdir-test 2023-06-09T11:13:01.733 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/30 file after the snapshot 2023-06-09T11:13:01.735 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/31 file after the snapshot 2023-06-09T11:13:01.738 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/32 file after the snapshot 2023-06-09T11:13:01.740 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/33 file after the snapshot 2023-06-09T11:13:01.742 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/34 file after the snapshot 2023-06-09T11:13:01.745 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/35 file after the snapshot 2023-06-09T11:13:01.747 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/36 file after the snapshot 2023-06-09T11:13:01.749 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/37 file after the snapshot 2023-06-09T11:13:01.751 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/38 file after the snapshot 2023-06-09T11:13:01.754 INFO:tasks.workunit.client.0.smithi007.stdout: create 105/39 file after the snapshot 2023-06-09T11:13:01.756 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/.snap/snap-subdir-test 2023-06-09T11:13:01.780 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/30 file after the snapshot 2023-06-09T11:13:01.782 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/31 file after the snapshot 2023-06-09T11:13:01.784 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/32 file after the snapshot 2023-06-09T11:13:01.787 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/33 file after the snapshot 2023-06-09T11:13:01.789 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/34 file after the snapshot 2023-06-09T11:13:01.792 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/35 file after the snapshot 2023-06-09T11:13:01.794 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/36 file after the snapshot 2023-06-09T11:13:01.797 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/37 file after the snapshot 2023-06-09T11:13:01.800 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/38 file after the snapshot 2023-06-09T11:13:01.803 INFO:tasks.workunit.client.0.smithi007.stdout: create 106/39 file after the snapshot 2023-06-09T11:13:01.805 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/.snap/snap-subdir-test 2023-06-09T11:13:01.830 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/30 file after the snapshot 2023-06-09T11:13:01.832 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/31 file after the snapshot 2023-06-09T11:13:01.836 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/32 file after the snapshot 2023-06-09T11:13:01.837 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/33 file after the snapshot 2023-06-09T11:13:01.840 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/34 file after the snapshot 2023-06-09T11:13:01.842 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/35 file after the snapshot 2023-06-09T11:13:01.845 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/36 file after the snapshot 2023-06-09T11:13:01.848 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/37 file after the snapshot 2023-06-09T11:13:01.849 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/38 file after the snapshot 2023-06-09T11:13:01.851 INFO:tasks.workunit.client.0.smithi007.stdout: create 107/39 file after the snapshot 2023-06-09T11:13:01.853 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/.snap/snap-subdir-test 2023-06-09T11:13:01.877 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/30 file after the snapshot 2023-06-09T11:13:01.879 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/31 file after the snapshot 2023-06-09T11:13:01.882 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/32 file after the snapshot 2023-06-09T11:13:01.884 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/33 file after the snapshot 2023-06-09T11:13:01.887 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/34 file after the snapshot 2023-06-09T11:13:01.889 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/35 file after the snapshot 2023-06-09T11:13:01.892 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/36 file after the snapshot 2023-06-09T11:13:01.894 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/37 file after the snapshot 2023-06-09T11:13:01.897 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/38 file after the snapshot 2023-06-09T11:13:01.900 INFO:tasks.workunit.client.0.smithi007.stdout: create 108/39 file after the snapshot 2023-06-09T11:13:01.902 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/.snap/snap-subdir-test 2023-06-09T11:13:01.935 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/30 file after the snapshot 2023-06-09T11:13:01.937 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/31 file after the snapshot 2023-06-09T11:13:01.939 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/32 file after the snapshot 2023-06-09T11:13:01.941 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/33 file after the snapshot 2023-06-09T11:13:01.944 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/34 file after the snapshot 2023-06-09T11:13:01.946 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/35 file after the snapshot 2023-06-09T11:13:01.948 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/36 file after the snapshot 2023-06-09T11:13:01.950 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/37 file after the snapshot 2023-06-09T11:13:01.953 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/38 file after the snapshot 2023-06-09T11:13:01.955 INFO:tasks.workunit.client.0.smithi007.stdout: create 109/39 file after the snapshot 2023-06-09T11:13:01.957 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/.snap/snap-subdir-test 2023-06-09T11:13:01.981 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/30 file after the snapshot 2023-06-09T11:13:01.985 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/31 file after the snapshot 2023-06-09T11:13:01.988 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/32 file after the snapshot 2023-06-09T11:13:01.990 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/33 file after the snapshot 2023-06-09T11:13:01.992 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/34 file after the snapshot 2023-06-09T11:13:01.994 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/35 file after the snapshot 2023-06-09T11:13:01.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/36 file after the snapshot 2023-06-09T11:13:01.999 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/37 file after the snapshot 2023-06-09T11:13:02.001 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/38 file after the snapshot 2023-06-09T11:13:02.003 INFO:tasks.workunit.client.0.smithi007.stdout: create 110/39 file after the snapshot 2023-06-09T11:13:02.006 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/.snap/snap-subdir-test 2023-06-09T11:13:02.028 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/30 file after the snapshot 2023-06-09T11:13:02.031 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/31 file after the snapshot 2023-06-09T11:13:02.033 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/32 file after the snapshot 2023-06-09T11:13:02.036 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/33 file after the snapshot 2023-06-09T11:13:02.038 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/34 file after the snapshot 2023-06-09T11:13:02.040 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/35 file after the snapshot 2023-06-09T11:13:02.043 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/36 file after the snapshot 2023-06-09T11:13:02.045 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/37 file after the snapshot 2023-06-09T11:13:02.047 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/38 file after the snapshot 2023-06-09T11:13:02.049 INFO:tasks.workunit.client.0.smithi007.stdout: create 111/39 file after the snapshot 2023-06-09T11:13:02.052 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/.snap/snap-subdir-test 2023-06-09T11:13:02.076 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/30 file after the snapshot 2023-06-09T11:13:02.078 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/31 file after the snapshot 2023-06-09T11:13:02.080 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/32 file after the snapshot 2023-06-09T11:13:02.082 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/33 file after the snapshot 2023-06-09T11:13:02.085 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/34 file after the snapshot 2023-06-09T11:13:02.087 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/35 file after the snapshot 2023-06-09T11:13:02.089 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/36 file after the snapshot 2023-06-09T11:13:02.091 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/37 file after the snapshot 2023-06-09T11:13:02.094 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/38 file after the snapshot 2023-06-09T11:13:02.097 INFO:tasks.workunit.client.0.smithi007.stdout: create 112/39 file after the snapshot 2023-06-09T11:13:02.099 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/.snap/snap-subdir-test 2023-06-09T11:13:02.132 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/30 file after the snapshot 2023-06-09T11:13:02.134 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/31 file after the snapshot 2023-06-09T11:13:02.137 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/32 file after the snapshot 2023-06-09T11:13:02.139 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/33 file after the snapshot 2023-06-09T11:13:02.141 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/34 file after the snapshot 2023-06-09T11:13:02.143 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/35 file after the snapshot 2023-06-09T11:13:02.146 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/36 file after the snapshot 2023-06-09T11:13:02.148 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/37 file after the snapshot 2023-06-09T11:13:02.151 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/38 file after the snapshot 2023-06-09T11:13:02.153 INFO:tasks.workunit.client.0.smithi007.stdout: create 113/39 file after the snapshot 2023-06-09T11:13:02.155 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/.snap/snap-subdir-test 2023-06-09T11:13:02.176 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/30 file after the snapshot 2023-06-09T11:13:02.178 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/31 file after the snapshot 2023-06-09T11:13:02.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/32 file after the snapshot 2023-06-09T11:13:02.184 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/33 file after the snapshot 2023-06-09T11:13:02.186 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/34 file after the snapshot 2023-06-09T11:13:02.189 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/35 file after the snapshot 2023-06-09T11:13:02.191 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/36 file after the snapshot 2023-06-09T11:13:02.193 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/37 file after the snapshot 2023-06-09T11:13:02.195 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/38 file after the snapshot 2023-06-09T11:13:02.198 INFO:tasks.workunit.client.0.smithi007.stdout: create 114/39 file after the snapshot 2023-06-09T11:13:02.200 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/.snap/snap-subdir-test 2023-06-09T11:13:02.223 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/30 file after the snapshot 2023-06-09T11:13:02.226 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/31 file after the snapshot 2023-06-09T11:13:02.229 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/32 file after the snapshot 2023-06-09T11:13:02.231 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/33 file after the snapshot 2023-06-09T11:13:02.235 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/34 file after the snapshot 2023-06-09T11:13:02.236 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/35 file after the snapshot 2023-06-09T11:13:02.238 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/36 file after the snapshot 2023-06-09T11:13:02.240 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/37 file after the snapshot 2023-06-09T11:13:02.243 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/38 file after the snapshot 2023-06-09T11:13:02.245 INFO:tasks.workunit.client.0.smithi007.stdout: create 115/39 file after the snapshot 2023-06-09T11:13:02.247 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/.snap/snap-subdir-test 2023-06-09T11:13:02.272 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/30 file after the snapshot 2023-06-09T11:13:02.274 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/31 file after the snapshot 2023-06-09T11:13:02.277 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/32 file after the snapshot 2023-06-09T11:13:02.279 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/33 file after the snapshot 2023-06-09T11:13:02.281 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/34 file after the snapshot 2023-06-09T11:13:02.283 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/35 file after the snapshot 2023-06-09T11:13:02.286 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/36 file after the snapshot 2023-06-09T11:13:02.289 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/37 file after the snapshot 2023-06-09T11:13:02.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/38 file after the snapshot 2023-06-09T11:13:02.293 INFO:tasks.workunit.client.0.smithi007.stdout: create 116/39 file after the snapshot 2023-06-09T11:13:02.295 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/.snap/snap-subdir-test 2023-06-09T11:13:02.320 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/30 file after the snapshot 2023-06-09T11:13:02.322 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/31 file after the snapshot 2023-06-09T11:13:02.325 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/32 file after the snapshot 2023-06-09T11:13:02.327 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/33 file after the snapshot 2023-06-09T11:13:02.330 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/34 file after the snapshot 2023-06-09T11:13:02.332 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/35 file after the snapshot 2023-06-09T11:13:02.334 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/36 file after the snapshot 2023-06-09T11:13:02.336 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/37 file after the snapshot 2023-06-09T11:13:02.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/38 file after the snapshot 2023-06-09T11:13:02.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 117/39 file after the snapshot 2023-06-09T11:13:02.343 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/.snap/snap-subdir-test 2023-06-09T11:13:03.273 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/30 file after the snapshot 2023-06-09T11:13:03.276 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/31 file after the snapshot 2023-06-09T11:13:03.278 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/32 file after the snapshot 2023-06-09T11:13:03.281 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/33 file after the snapshot 2023-06-09T11:13:03.283 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/34 file after the snapshot 2023-06-09T11:13:03.286 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/35 file after the snapshot 2023-06-09T11:13:03.288 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/36 file after the snapshot 2023-06-09T11:13:03.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/37 file after the snapshot 2023-06-09T11:13:03.293 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/38 file after the snapshot 2023-06-09T11:13:03.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:03.296 INFO:tasks.workunit.client.0.smithi007.stdout: create 118/39 file after the snapshot 2023-06-09T11:13:03.301 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/.snap/snap-subdir-test 2023-06-09T11:13:03.323 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/30 file after the snapshot 2023-06-09T11:13:03.326 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/31 file after the snapshot 2023-06-09T11:13:03.328 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/32 file after the snapshot 2023-06-09T11:13:03.331 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/33 file after the snapshot 2023-06-09T11:13:03.333 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/34 file after the snapshot 2023-06-09T11:13:03.336 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/35 file after the snapshot 2023-06-09T11:13:03.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/36 file after the snapshot 2023-06-09T11:13:03.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/37 file after the snapshot 2023-06-09T11:13:03.343 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/38 file after the snapshot 2023-06-09T11:13:03.346 INFO:tasks.workunit.client.0.smithi007.stdout: create 119/39 file after the snapshot 2023-06-09T11:13:03.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/.snap/snap-subdir-test 2023-06-09T11:13:03.368 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/30 file after the snapshot 2023-06-09T11:13:03.372 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/31 file after the snapshot 2023-06-09T11:13:03.375 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/32 file after the snapshot 2023-06-09T11:13:03.377 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/33 file after the snapshot 2023-06-09T11:13:03.380 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/34 file after the snapshot 2023-06-09T11:13:03.382 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/35 file after the snapshot 2023-06-09T11:13:03.385 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/36 file after the snapshot 2023-06-09T11:13:03.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/37 file after the snapshot 2023-06-09T11:13:03.416 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/38 file after the snapshot 2023-06-09T11:13:03.419 INFO:tasks.workunit.client.0.smithi007.stdout: create 120/39 file after the snapshot 2023-06-09T11:13:03.422 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/.snap/snap-subdir-test 2023-06-09T11:13:03.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/30 file after the snapshot 2023-06-09T11:13:03.512 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/31 file after the snapshot 2023-06-09T11:13:03.514 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/32 file after the snapshot 2023-06-09T11:13:03.517 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/33 file after the snapshot 2023-06-09T11:13:03.519 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/34 file after the snapshot 2023-06-09T11:13:03.521 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/35 file after the snapshot 2023-06-09T11:13:03.524 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/36 file after the snapshot 2023-06-09T11:13:03.526 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/37 file after the snapshot 2023-06-09T11:13:03.529 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/38 file after the snapshot 2023-06-09T11:13:03.531 INFO:tasks.workunit.client.0.smithi007.stdout: create 121/39 file after the snapshot 2023-06-09T11:13:03.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.528+0000 7f42c23ff700 1 Processor -- start 2023-06-09T11:13:03.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.528+0000 7f42c23ff700 1 -- start start 2023-06-09T11:13:03.533 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/.snap/snap-subdir-test 2023-06-09T11:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c23ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42bc153880 0x7f42bc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c23ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c23ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc15a500 con 0x7f42bc15b2d0 2023-06-09T11:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c23ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0c5f10 con 0x7f42bc0c9f40 2023-06-09T11:13:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c23ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc15da00 con 0x7f42bc153880 2023-06-09T11:13:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42bb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42bbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42bc153880 0x7f42bc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42bbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42bc153880 0x7f42bc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:44444/0 (socket says 172.21.15.7:44444) 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42bb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55296/0 (socket says 172.21.15.7:55296) 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42bbfff700 1 -- 172.21.15.7:0/3187115568 learned_addr learned my addr 172.21.15.7:0/3187115568 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.530+0000 7f42c099c700 1 --1- 172.21.15.7:0/3187115568 >> v1:172.21.15.117:6789/0 conn(0x7f42bc15b2d0 0x7f42bc0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:58204/0 (socket says 172.21.15.7:58204) 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bbfff700 1 -- 172.21.15.7:0/3187115568 >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 msgr2=0x7f42bc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bbfff700 1 --2- 172.21.15.7:0/3187115568 >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bbfff700 1 -- 172.21.15.7:0/3187115568 >> v1:172.21.15.117:6789/0 conn(0x7f42bc15b2d0 legacy=0x7f42bc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bbfff700 1 -- 172.21.15.7:0/3187115568 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc1596e0 con 0x7f42bc153880 2023-06-09T11:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bb7fe700 1 --2- 172.21.15.7:0/3187115568 >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42bbfff700 1 --2- 172.21.15.7:0/3187115568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc153880 0x7f42bc15d4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f42bc08d340 tx=0x7f42ac009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02d06ea7790c130 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42baffd700 1 -- 172.21.15.7:0/3187115568 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac011830 con 0x7f42bc153880 2023-06-09T11:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42baffd700 1 -- 172.21.15.7:0/3187115568 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac011990 con 0x7f42bc153880 2023-06-09T11:13:03.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.531+0000 7f42baffd700 1 -- 172.21.15.7:0/3187115568 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac011d20 con 0x7f42bc153880 2023-06-09T11:13:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 -- 172.21.15.7:0/3187115568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc153880 msgr2=0x7f42bc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 --2- 172.21.15.7:0/3187115568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc153880 0x7f42bc15d4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f42bc08d340 tx=0x7f42ac009bd0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 -- 172.21.15.7:0/3187115568 shutdown_connections 2023-06-09T11:13:03.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 --2- 172.21.15.7:0/3187115568 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc153880 0x7f42bc15d4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 --2- 172.21.15.7:0/3187115568 >> v2:172.21.15.7:3300/0 conn(0x7f42bc0c9f40 0x7f42bc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 -- 172.21.15.7:0/3187115568 >> 172.21.15.7:0/3187115568 conn(0x7f42bc0737f0 msgr2=0x7f42bc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 -- 172.21.15.7:0/3187115568 shutdown_connections 2023-06-09T11:13:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.532+0000 7f42c23ff700 1 -- 172.21.15.7:0/3187115568 wait complete. 2023-06-09T11:13:03.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.534+0000 7f42c23ff700 1 Processor -- start 2023-06-09T11:13:03.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.534+0000 7f42c23ff700 1 -- start start 2023-06-09T11:13:03.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42bc0c9f40 0x7f42bc06abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc15b2d0 0x7f42bc146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1f0070 con 0x7f42bc153880 2023-06-09T11:13:03.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c7a30 con 0x7f42bc15b2d0 2023-06-09T11:13:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42c23ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0c6c20 con 0x7f42bc0c9f40 2023-06-09T11:13:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.535+0000 7f42bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55304/0 (socket says 172.21.15.7:55304) 2023-06-09T11:13:03.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 -- 172.21.15.7:0/2467467700 learned_addr learned my addr 172.21.15.7:0/2467467700 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bbfff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42bc0c9f40 0x7f42bc06abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42c099c700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc15b2d0 0x7f42bc146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 -- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42bc0c9f40 msgr2=0x7f42bc06abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42bc0c9f40 0x7f42bc06abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 -- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc15b2d0 msgr2=0x7f42bc146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc15b2d0 0x7f42bc146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 -- 172.21.15.7:0/2467467700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00094a0 con 0x7f42bc153880 2023-06-09T11:13:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.536+0000 7f42bb7fe700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ebe0 tx=0x7f42b000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.537+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000c850 con 0x7f42bc153880 2023-06-09T11:13:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.537+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b000c9b0 con 0x7f42bc153880 2023-06-09T11:13:03.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.537+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac009550 con 0x7f42bc153880 2023-06-09T11:13:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.537+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000cd40 con 0x7f42bc153880 2023-06-09T11:13:03.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.537+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc159c60 con 0x7f42bc153880 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.538+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42bc159980 con 0x7f42bc153880 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.540+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42b0013470 con 0x7f42bc153880 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.541+0000 7f42b8ff9700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 0x7f42a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.541+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f42b00895d0 con 0x7f42bc153880 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.541+0000 7f42bbfff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 0x7f42a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:03.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.542+0000 7f42bbfff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 0x7f42a806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009c10 tx=0x7f42ac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:03.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.546+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f42b00545c0 con 0x7f42bc153880 2023-06-09T11:13:03.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/30 file after the snapshot 2023-06-09T11:13:03.560 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/31 file after the snapshot 2023-06-09T11:13:03.563 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/32 file after the snapshot 2023-06-09T11:13:03.566 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/33 file after the snapshot 2023-06-09T11:13:03.568 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/34 file after the snapshot 2023-06-09T11:13:03.571 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/35 file after the snapshot 2023-06-09T11:13:03.574 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/36 file after the snapshot 2023-06-09T11:13:03.576 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/37 file after the snapshot 2023-06-09T11:13:03.579 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/38 file after the snapshot 2023-06-09T11:13:03.581 INFO:tasks.workunit.client.0.smithi007.stdout: create 122/39 file after the snapshot 2023-06-09T11:13:03.583 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/.snap/snap-subdir-test 2023-06-09T11:13:03.608 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/30 file after the snapshot 2023-06-09T11:13:03.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/31 file after the snapshot 2023-06-09T11:13:03.615 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/32 file after the snapshot 2023-06-09T11:13:03.616 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/33 file after the snapshot 2023-06-09T11:13:03.618 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/34 file after the snapshot 2023-06-09T11:13:03.621 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/35 file after the snapshot 2023-06-09T11:13:03.624 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/36 file after the snapshot 2023-06-09T11:13:03.626 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/37 file after the snapshot 2023-06-09T11:13:03.632 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/38 file after the snapshot 2023-06-09T11:13:03.633 INFO:tasks.workunit.client.0.smithi007.stdout: create 123/39 file after the snapshot 2023-06-09T11:13:03.634 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/.snap/snap-subdir-test 2023-06-09T11:13:03.659 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/30 file after the snapshot 2023-06-09T11:13:03.662 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/31 file after the snapshot 2023-06-09T11:13:03.665 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/32 file after the snapshot 2023-06-09T11:13:03.668 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/33 file after the snapshot 2023-06-09T11:13:03.670 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/34 file after the snapshot 2023-06-09T11:13:03.672 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/35 file after the snapshot 2023-06-09T11:13:03.675 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/36 file after the snapshot 2023-06-09T11:13:03.677 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/37 file after the snapshot 2023-06-09T11:13:03.680 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/38 file after the snapshot 2023-06-09T11:13:03.683 INFO:tasks.workunit.client.0.smithi007.stdout: create 124/39 file after the snapshot 2023-06-09T11:13:03.685 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/.snap/snap-subdir-test 2023-06-09T11:13:03.719 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/30 file after the snapshot 2023-06-09T11:13:03.722 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/31 file after the snapshot 2023-06-09T11:13:03.724 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/32 file after the snapshot 2023-06-09T11:13:03.726 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/33 file after the snapshot 2023-06-09T11:13:03.729 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/34 file after the snapshot 2023-06-09T11:13:03.731 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/35 file after the snapshot 2023-06-09T11:13:03.733 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/36 file after the snapshot 2023-06-09T11:13:03.736 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/37 file after the snapshot 2023-06-09T11:13:03.738 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/38 file after the snapshot 2023-06-09T11:13:03.741 INFO:tasks.workunit.client.0.smithi007.stdout: create 125/39 file after the snapshot 2023-06-09T11:13:03.743 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/.snap/snap-subdir-test 2023-06-09T11:13:03.765 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/30 file after the snapshot 2023-06-09T11:13:03.767 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/31 file after the snapshot 2023-06-09T11:13:03.770 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/32 file after the snapshot 2023-06-09T11:13:03.772 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/33 file after the snapshot 2023-06-09T11:13:03.774 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/34 file after the snapshot 2023-06-09T11:13:03.777 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/35 file after the snapshot 2023-06-09T11:13:03.779 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/36 file after the snapshot 2023-06-09T11:13:03.781 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/37 file after the snapshot 2023-06-09T11:13:03.784 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/38 file after the snapshot 2023-06-09T11:13:03.786 INFO:tasks.workunit.client.0.smithi007.stdout: create 126/39 file after the snapshot 2023-06-09T11:13:03.789 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/.snap/snap-subdir-test 2023-06-09T11:13:03.812 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/30 file after the snapshot 2023-06-09T11:13:03.815 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/31 file after the snapshot 2023-06-09T11:13:03.818 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/32 file after the snapshot 2023-06-09T11:13:03.820 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/33 file after the snapshot 2023-06-09T11:13:03.822 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/34 file after the snapshot 2023-06-09T11:13:03.824 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/35 file after the snapshot 2023-06-09T11:13:03.827 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/36 file after the snapshot 2023-06-09T11:13:03.829 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/37 file after the snapshot 2023-06-09T11:13:03.831 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/38 file after the snapshot 2023-06-09T11:13:03.834 INFO:tasks.workunit.client.0.smithi007.stdout: create 127/39 file after the snapshot 2023-06-09T11:13:03.836 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/.snap/snap-subdir-test 2023-06-09T11:13:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.846+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42bc159f40 con 0x7f42bc153880 2023-06-09T11:13:03.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.846+0000 7f42b8ff9700 1 -- 172.21.15.7:0/2467467700 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f42b000cea0 con 0x7f42bc153880 2023-06-09T11:13:03.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:03.851 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 msgr2=0x7f42a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 0x7f42a806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009c10 tx=0x7f42ac008030 comp rx=0 tx=0).stop 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 msgr2=0x7f42bc158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f42b000ebe0 tx=0x7f42b000eef0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 shutdown_connections 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42bc15b2d0 0x7f42bc146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42a806ace0 0x7f42a806d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42bc0c9f40 0x7f42bc06abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 --2- 172.21.15.7:0/2467467700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42bc153880 0x7f42bc158b70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 >> 172.21.15.7:0/2467467700 conn(0x7f42bc0737f0 msgr2=0x7f42bc15c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 shutdown_connections 2023-06-09T11:13:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:03.849+0000 7f42c23ff700 1 -- 172.21.15.7:0/2467467700 wait complete. 2023-06-09T11:13:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:03.859 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/30 file after the snapshot 2023-06-09T11:13:03.862 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/31 file after the snapshot 2023-06-09T11:13:03.865 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/32 file after the snapshot 2023-06-09T11:13:03.867 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/33 file after the snapshot 2023-06-09T11:13:03.870 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/34 file after the snapshot 2023-06-09T11:13:03.872 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/35 file after the snapshot 2023-06-09T11:13:03.874 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2023-06-09T11:13:03.875 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-06-09T11:13:03.875 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:13:03.875 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/36 file after the snapshot 2023-06-09T11:13:03.877 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/37 file after the snapshot 2023-06-09T11:13:03.880 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/38 file after the snapshot 2023-06-09T11:13:03.883 INFO:tasks.workunit.client.0.smithi007.stdout: create 128/39 file after the snapshot 2023-06-09T11:13:03.885 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/.snap/snap-subdir-test 2023-06-09T11:13:03.891 INFO:tasks.ceph.mds.e:Stopped 2023-06-09T11:13:03.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:04.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.126+0000 7f79effff700 1 Processor -- start 2023-06-09T11:13:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.127+0000 7f79effff700 1 -- start start 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79effff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79e806a4d0 0x7f79e80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79effff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79e806e460 0x7f79e806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79effff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e815cac0 con 0x7f79e80c6280 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79effff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80c8870 con 0x7f79e806e460 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79effff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e80c37d0 con 0x7f79e806a4d0 2023-06-09T11:13:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79ee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79e806e460 0x7f79e806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79ed59a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79e806a4d0 0x7f79e80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79edd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f79e80c6280 0x7f79e80737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:58216/0 (socket says 172.21.15.7:58216) 2023-06-09T11:13:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79ee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79e806e460 0x7f79e806cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55306/0 (socket says 172.21.15.7:55306) 2023-06-09T11:13:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.128+0000 7f79edd9b700 1 -- 172.21.15.7:0/3578367068 learned_addr learned my addr 172.21.15.7:0/3578367068 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ee59c700 1 -- 172.21.15.7:0/3578367068 >> v2:172.21.15.117:3300/0 conn(0x7f79e806a4d0 msgr2=0x7f79e80c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ee59c700 1 --2- 172.21.15.7:0/3578367068 >> v2:172.21.15.117:3300/0 conn(0x7f79e806a4d0 0x7f79e80c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ee59c700 1 -- 172.21.15.7:0/3578367068 >> v1:172.21.15.117:6789/0 conn(0x7f79e80c6280 legacy=0x7f79e80737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ee59c700 1 -- 172.21.15.7:0/3578367068 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e815cd30 con 0x7f79e806e460 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ee59c700 1 --2- 172.21.15.7:0/3578367068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806e460 0x7f79e806cf30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79e815a580 tx=0x7f79e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be910ac14e9621 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ecd99700 1 -- 172.21.15.7:0/3578367068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000e070 con 0x7f79e806e460 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ecd99700 1 -- 172.21.15.7:0/3578367068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000bb20 con 0x7f79e806e460 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.129+0000 7f79ecd99700 1 -- 172.21.15.7:0/3578367068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0012430 con 0x7f79e806e460 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 -- 172.21.15.7:0/3578367068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806e460 msgr2=0x7f79e806cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 --2- 172.21.15.7:0/3578367068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806e460 0x7f79e806cf30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f79e815a580 tx=0x7f79e000b330 comp rx=0 tx=0).stop 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 -- 172.21.15.7:0/3578367068 shutdown_connections 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 --2- 172.21.15.7:0/3578367068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806e460 0x7f79e806cf30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 --2- 172.21.15.7:0/3578367068 >> v2:172.21.15.117:3300/0 conn(0x7f79e806a4d0 0x7f79e80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 -- 172.21.15.7:0/3578367068 >> 172.21.15.7:0/3578367068 conn(0x7f79e806f980 msgr2=0x7f79e80cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 -- 172.21.15.7:0/3578367068 shutdown_connections 2023-06-09T11:13:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.130+0000 7f79effff700 1 -- 172.21.15.7:0/3578367068 wait complete. 2023-06-09T11:13:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.132+0000 7f79effff700 1 Processor -- start 2023-06-09T11:13:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.133+0000 7f79effff700 1 -- start start 2023-06-09T11:13:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e806e460 0x7f79e8158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e80c6280 0x7f79e80ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8159960 con 0x7f79e806a4d0 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8159eb0 con 0x7f79e806e460 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79effff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8159250 con 0x7f79e80c6280 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55318/0 (socket says 172.21.15.7:55318) 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79edd9b700 1 -- 172.21.15.7:0/868881068 learned_addr learned my addr 172.21.15.7:0/868881068 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79ee59c700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e80c6280 0x7f79e80ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.134+0000 7f79ed59a700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e806e460 0x7f79e8158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 -- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e80c6280 msgr2=0x7f79e80ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e80c6280 0x7f79e80ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 -- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e806e460 msgr2=0x7f79e8158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e806e460 0x7f79e8158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 -- 172.21.15.7:0/868881068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009a70 con 0x7f79e806a4d0 2023-06-09T11:13:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79edd9b700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c480 tx=0x7f79e400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e4005d70 con 0x7f79e806a4d0 2023-06-09T11:13:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e400ce90 con 0x7f79e806a4d0 2023-06-09T11:13:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e40072d0 con 0x7f79e806a4d0 2023-06-09T11:13:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e815d2f0 con 0x7f79e806a4d0 2023-06-09T11:13:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.135+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e815d010 con 0x7f79e806a4d0 2023-06-09T11:13:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.136+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e815d5d0 con 0x7f79e806a4d0 2023-06-09T11:13:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.138+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79e4007430 con 0x7f79e806a4d0 2023-06-09T11:13:04.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.139+0000 7f79de7fc700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 0x7f79cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.139+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f79e40886a0 con 0x7f79e806a4d0 2023-06-09T11:13:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.139+0000 7f79ed59a700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 0x7f79cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:04.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.139+0000 7f79ed59a700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 0x7f79cc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79e80c5c60 tx=0x7f79d8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:04.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.145+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f79e4053740 con 0x7f79e806a4d0 2023-06-09T11:13:04.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.405+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79e8073f40 con 0x7f79e806a4d0 2023-06-09T11:13:04.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.405+0000 7f79de7fc700 1 -- 172.21.15.7:0/868881068 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f79e4056d60 con 0x7f79e806a4d0 2023-06-09T11:13:04.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:04.410 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.407+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 msgr2=0x7f79cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.407+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 0x7f79cc06d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f79e80c5c60 tx=0x7f79d8009210 comp rx=0 tx=0).stop 2023-06-09T11:13:04.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.407+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 msgr2=0x7f79e81f3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.407+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f79e400c480 tx=0x7f79e400c790 comp rx=0 tx=0).stop 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 shutdown_connections 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79cc06ace0 0x7f79cc06d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79e80c6280 0x7f79e80ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79e806a4d0 0x7f79e81f3180 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 --2- 172.21.15.7:0/868881068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79e806e460 0x7f79e8158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:04.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 >> 172.21.15.7:0/868881068 conn(0x7f79e806f980 msgr2=0x7f79e80c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 shutdown_connections 2023-06-09T11:13:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:04.408+0000 7f79effff700 1 -- 172.21.15.7:0/868881068 wait complete. 2023-06-09T11:13:04.414 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:04.430 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:06.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.637+0000 7fd448dff700 1 Processor -- start 2023-06-09T11:13:06.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.637+0000 7fd448dff700 1 -- start start 2023-06-09T11:13:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.638+0000 7fd448dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd44415b4a0 0x7fd44415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.638+0000 7fd448dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd4440c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.638+0000 7fd448dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44415a500 con 0x7fd444153880 2023-06-09T11:13:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.638+0000 7fd448dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4440c5f10 con 0x7fd44415b4a0 2023-06-09T11:13:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.638+0000 7fd448dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44415da00 con 0x7fd4440c9f40 2023-06-09T11:13:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd444153880 0x7fd44415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56414/0 (socket says 172.21.15.7:56414) 2023-06-09T11:13:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44259c700 1 -- 172.21.15.7:0/86304951 learned_addr learned my addr 172.21.15.7:0/86304951 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd442d9d700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd4440c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd441d9b700 1 --2- 172.21.15.7:0/86304951 >> v2:172.21.15.117:3300/0 conn(0x7fd44415b4a0 0x7fd44415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019536675 0 0) 0x7fd434001110 con 0x7fd444153880 2023-06-09T11:13:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd42c0035b0 con 0x7fd444153880 2023-06-09T11:13:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2040427126 0 0) 0x7fd42c0035b0 con 0x7fd444153880 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 msgr2=0x7fd4440c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd4440c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 >> v2:172.21.15.117:3300/0 conn(0x7fd44415b4a0 msgr2=0x7fd44415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 --2- 172.21.15.7:0/86304951 >> v2:172.21.15.117:3300/0 conn(0x7fd44415b4a0 0x7fd44415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4441596e0 con 0x7fd444153880 2023-06-09T11:13:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fd434001800 con 0x7fd444153880 2023-06-09T11:13:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 >> v1:172.21.15.7:6789/0 conn(0x7fd444153880 legacy=0x7fd44415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd42c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c003fd0 0x7fd42c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd42c004d90 0x7fd42c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.639+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440c5f10 con 0x7fd42c003fd0 2023-06-09T11:13:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44415da00 con 0x7fd42c004d90 2023-06-09T11:13:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42c00ba90 con 0x7fd4440c9f40 2023-06-09T11:13:06.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c003fd0 0x7fd42c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd44259c700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd42c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd442d9d700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd42c004d90 0x7fd42c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 -- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 msgr2=0x7fd42c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4440c9f40 0x7fd42c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 -- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd42c004d90 msgr2=0x7fd42c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd42c004d90 0x7fd42c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 -- 172.21.15.7:0/86304951 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c0035b0 con 0x7fd42c003fd0 2023-06-09T11:13:06.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd442d9d700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd42c004d90 0x7fd42c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.640+0000 7fd441d9b700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c003fd0 0x7fd42c004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd438009b60 tx=0x7fd42c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b91705664882107 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800e6c0 con 0x7fd42c003fd0 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43800e820 con 0x7fd42c003fd0 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd44159a700 1 -- 172.21.15.7:0/86304951 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43800eb90 con 0x7fd42c003fd0 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 -- 172.21.15.7:0/86304951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c003fd0 msgr2=0x7fd42c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 --2- 172.21.15.7:0/86304951 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd42c003fd0 0x7fd42c004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd438009b60 tx=0x7fd42c005880 comp rx=0 tx=0).stop 2023-06-09T11:13:06.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd442d9d700 1 -- 172.21.15.7:0/86304951 reap_dead start 2023-06-09T11:13:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 -- 172.21.15.7:0/86304951 shutdown_connections 2023-06-09T11:13:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 -- 172.21.15.7:0/86304951 >> 172.21.15.7:0/86304951 conn(0x7fd4440737f0 msgr2=0x7fd4440720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 -- 172.21.15.7:0/86304951 shutdown_connections 2023-06-09T11:13:06.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.641+0000 7fd448dff700 1 -- 172.21.15.7:0/86304951 wait complete. 2023-06-09T11:13:06.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.644+0000 7fd448dff700 1 Processor -- start 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.644+0000 7fd448dff700 1 -- start start 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 0x7fd444158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 0x7fd444158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4441531b0 con 0x7fd4441ef590 2023-06-09T11:13:06.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4440c8cf0 con 0x7fd4440c6560 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd448dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd444152bf0 con 0x7fd42c003fd0 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd442d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd442d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55340/0 (socket says 172.21.15.7:55340) 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.645+0000 7fd442d9d700 1 -- 172.21.15.7:0/3236115385 learned_addr learned my addr 172.21.15.7:0/3236115385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd44259c700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 0x7fd444158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd441d9b700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 0x7fd444158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 -- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 msgr2=0x7fd444158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 0x7fd444158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 -- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 msgr2=0x7fd444158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 0x7fd444158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 -- 172.21.15.7:0/3236115385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444159c60 con 0x7fd4441ef590 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd44259c700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 0x7fd444158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd441d9b700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 0x7fd444158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd442d9d700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd44408d250 tx=0x7fd43c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:06.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00e050 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c00f040 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd444159980 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c010db0 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.646+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd444159f40 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.647+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4440c8b40 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.648+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd43c015440 con 0x7fd4441ef590 2023-06-09T11:13:06.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.649+0000 7fd432ffd700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:06.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.649+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7fd43c0899d0 con 0x7fd4441ef590 2023-06-09T11:13:06.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.649+0000 7fd44259c700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:06.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.650+0000 7fd44259c700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 0x7fd42406d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd434001050 tx=0x7fd434009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:06.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.654+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd43c0549c0 con 0x7fd4441ef590 2023-06-09T11:13:06.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.905+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd444074cf0 con 0x7fd4441ef590 2023-06-09T11:13:06.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.906+0000 7fd432ffd700 1 -- 172.21.15.7:0/3236115385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7fd43c057fe0 con 0x7fd4441ef590 2023-06-09T11:13:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:06.912 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:06.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 msgr2=0x7fd42406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 0x7fd42406d190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd434001050 tx=0x7fd434009040 comp rx=0 tx=0).stop 2023-06-09T11:13:06.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 msgr2=0x7fd4441ef9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd44408d250 tx=0x7fd43c0104a0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 shutdown_connections 2023-06-09T11:13:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4441ef590 0x7fd4441ef9e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd42406ace0 0x7fd42406d190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd42c003fd0 0x7fd444158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 --2- 172.21.15.7:0/3236115385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4440c6560 0x7fd444158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 >> 172.21.15.7:0/3236115385 conn(0x7fd4440737f0 msgr2=0x7fd44414df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 shutdown_connections 2023-06-09T11:13:06.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:06.908+0000 7fd448dff700 1 -- 172.21.15.7:0/3236115385 wait complete. 2023-06-09T11:13:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:06.931 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:08.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.137+0000 7f96c19ff700 1 Processor -- start 2023-06-09T11:13:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.137+0000 7f96c19ff700 1 -- start start 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96c19ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96bc0c5b40 0x7f96bc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96c19ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f96bc071060 0x7f96bc158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96c19ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc1580a0 con 0x7f96bc151510 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96c19ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc1518e0 con 0x7f96bc0c5b40 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96c19ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0c5f70 con 0x7f96bc071060 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96ba7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96bc0c5b40 0x7f96bc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96ba7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96bc0c5b40 0x7f96bc070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55348/0 (socket says 172.21.15.7:55348) 2023-06-09T11:13:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96ba7fc700 1 -- 172.21.15.7:0/2234128615 learned_addr learned my addr 172.21.15.7:0/2234128615 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.139+0000 7f96bb7fe700 1 --2- 172.21.15.7:0/2234128615 >> v2:172.21.15.117:3300/0 conn(0x7f96bc071060 0x7f96bc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.140+0000 7f96ba7fc700 1 -- 172.21.15.7:0/2234128615 >> v2:172.21.15.117:3300/0 conn(0x7f96bc071060 msgr2=0x7f96bc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.140+0000 7f96ba7fc700 1 --2- 172.21.15.7:0/2234128615 >> v2:172.21.15.117:3300/0 conn(0x7f96bc071060 0x7f96bc158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.140+0000 7f96ba7fc700 1 -- 172.21.15.7:0/2234128615 >> v1:172.21.15.7:6789/0 conn(0x7f96bc151510 legacy=0x7f96bc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.140+0000 7f96ba7fc700 1 -- 172.21.15.7:0/2234128615 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc0c6180 con 0x7f96bc0c5b40 2023-06-09T11:13:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.141+0000 7f96ba7fc700 1 --2- 172.21.15.7:0/2234128615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc0c5b40 0x7f96bc070b20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96bc06ff60 tx=0x7f96b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413fb3cb756c2334 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.141+0000 7f96b9ffb700 1 -- 172.21.15.7:0/2234128615 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011830 con 0x7f96bc0c5b40 2023-06-09T11:13:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.141+0000 7f96b9ffb700 1 -- 172.21.15.7:0/2234128615 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b0011990 con 0x7f96bc0c5b40 2023-06-09T11:13:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.141+0000 7f96b9ffb700 1 -- 172.21.15.7:0/2234128615 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0011d20 con 0x7f96bc0c5b40 2023-06-09T11:13:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 -- 172.21.15.7:0/2234128615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc0c5b40 msgr2=0x7f96bc070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 --2- 172.21.15.7:0/2234128615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc0c5b40 0x7f96bc070b20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f96bc06ff60 tx=0x7f96b0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 -- 172.21.15.7:0/2234128615 shutdown_connections 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 --2- 172.21.15.7:0/2234128615 >> v2:172.21.15.117:3300/0 conn(0x7f96bc071060 0x7f96bc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 --2- 172.21.15.7:0/2234128615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc0c5b40 0x7f96bc070b20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 -- 172.21.15.7:0/2234128615 >> 172.21.15.7:0/2234128615 conn(0x7f96bc06a4d0 msgr2=0x7f96bc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 -- 172.21.15.7:0/2234128615 shutdown_connections 2023-06-09T11:13:09.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.142+0000 7f96c19ff700 1 -- 172.21.15.7:0/2234128615 wait complete. 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.144+0000 7f96c19ff700 1 Processor -- start 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.144+0000 7f96c19ff700 1 -- start start 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 0x7f96bc1f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 0x7f96bc155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1f0160 con 0x7f96bc151510 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1ef170 con 0x7f96bc071060 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96c19ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc1f3630 con 0x7f96bc0c5b40 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96bb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55364/0 (socket says 172.21.15.7:55364) 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96bb7fe700 1 -- 172.21.15.7:0/1445215888 learned_addr learned my addr 172.21.15.7:0/1445215888 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96baffd700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 0x7f96bc1f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.145+0000 7f96ba7fc700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 0x7f96bc155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 -- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 msgr2=0x7f96bc155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 0x7f96bc155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 -- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 msgr2=0x7f96bc1f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 0x7f96bc1f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 -- 172.21.15.7:0/1445215888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0009550 con 0x7f96bc151510 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96baffd700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 0x7f96bc1f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96ba7fc700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 0x7f96bc155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96bb7fe700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f96b400efc0 tx=0x7f96b400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:09.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4009ba0 con 0x7f96bc151510 2023-06-09T11:13:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b400f040 con 0x7f96bc151510 2023-06-09T11:13:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b4014480 con 0x7f96bc151510 2023-06-09T11:13:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc157770 con 0x7f96bc151510 2023-06-09T11:13:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.146+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc157490 con 0x7f96bc151510 2023-06-09T11:13:09.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.147+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc157a50 con 0x7f96bc151510 2023-06-09T11:13:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.148+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f96b40145e0 con 0x7f96bc151510 2023-06-09T11:13:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.149+0000 7f96a3fff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 0x7f96a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.149+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f96b4089d80 con 0x7f96bc151510 2023-06-09T11:13:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.150+0000 7f96baffd700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 0x7f96a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:09.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.150+0000 7f96baffd700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 0x7f96a806d1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000f60 tx=0x7f96ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:09.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.154+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f96b4054d70 con 0x7f96bc151510 2023-06-09T11:13:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.408+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96bc150bb0 con 0x7f96bc151510 2023-06-09T11:13:09.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.409+0000 7f96a3fff700 1 -- 172.21.15.7:0/1445215888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f96b4018090 con 0x7f96bc151510 2023-06-09T11:13:09.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:09.414 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 msgr2=0x7f96a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 0x7f96a806d1c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96ac000f60 tx=0x7f96ac009040 comp rx=0 tx=0).stop 2023-06-09T11:13:09.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 msgr2=0x7f96bc0c0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f96b400efc0 tx=0x7f96b400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 shutdown_connections 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96bc151510 0x7f96bc0c0540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96bc071060 0x7f96bc1f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f96a806ad10 0x7f96a806d1c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 --2- 172.21.15.7:0/1445215888 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96bc0c5b40 0x7f96bc155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.411+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 >> 172.21.15.7:0/1445215888 conn(0x7f96bc06a4d0 msgr2=0x7f96bc156930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.412+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 shutdown_connections 2023-06-09T11:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:09.412+0000 7f96c19ff700 1 -- 172.21.15.7:0/1445215888 wait complete. 2023-06-09T11:13:09.418 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:09.434 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:10.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:10.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:11.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.640+0000 7f7b4f9ff700 1 Processor -- start 2023-06-09T11:13:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.640+0000 7f7b4f9ff700 1 -- start start 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4f9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c1bc0 0x7f7b481599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4f9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b480c0140 0x7f7b48159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4f9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b480c0bf0 con 0x7f7b480c1f90 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4f9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b48070110 con 0x7f7b480c0140 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4f9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48071300 con 0x7f7b480c1bc0 2023-06-09T11:13:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4cf9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b480c0140 0x7f7b48159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4cf9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7b480c0140 0x7f7b48159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55368/0 (socket says 172.21.15.7:55368) 2023-06-09T11:13:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4cf9a700 1 -- 172.21.15.7:0/374140937 learned_addr learned my addr 172.21.15.7:0/374140937 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:11.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4d79b700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c1bc0 0x7f7b481599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.642+0000 7f7b4cf9a700 1 -- 172.21.15.7:0/374140937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c1bc0 msgr2=0x7f7b481599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b4cf9a700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c1bc0 0x7f7b481599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b4cf9a700 1 -- 172.21.15.7:0/374140937 >> v1:172.21.15.7:6789/0 conn(0x7f7b480c1f90 legacy=0x7f7b481501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b4cf9a700 1 -- 172.21.15.7:0/374140937 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b480702f0 con 0x7f7b480c0140 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b4cf9a700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c0140 0x7f7b48159ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800bb90 tx=0x7f7b3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3863cc235ff438 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b3ffff700 1 -- 172.21.15.7:0/374140937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800dd30 con 0x7f7b480c0140 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b3ffff700 1 -- 172.21.15.7:0/374140937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3800de90 con 0x7f7b480c0140 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.643+0000 7f7b3ffff700 1 -- 172.21.15.7:0/374140937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38010690 con 0x7f7b480c0140 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/374140937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c0140 msgr2=0x7f7b48159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c0140 0x7f7b48159ef0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800bb90 tx=0x7f7b3800bea0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/374140937 shutdown_connections 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c1bc0 0x7f7b481599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/374140937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c0140 0x7f7b48159ef0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/374140937 >> 172.21.15.7:0/374140937 conn(0x7f7b4806a4d0 msgr2=0x7f7b4806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/374140937 shutdown_connections 2023-06-09T11:13:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.644+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/374140937 wait complete. 2023-06-09T11:13:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.645+0000 7f7b4f9ff700 1 Processor -- start 2023-06-09T11:13:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 -- start start 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c0140 0x7f7b480caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b480c1bc0 0x7f7b480c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b48158930 con 0x7f7b480c1f90 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480c54d0 con 0x7f7b480c1bc0 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.646+0000 7f7b4f9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b480cb5f0 con 0x7f7b480c0140 2023-06-09T11:13:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55374/0 (socket says 172.21.15.7:55374) 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 -- 172.21.15.7:0/1486452560 learned_addr learned my addr 172.21.15.7:0/1486452560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4cf9a700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b480c1bc0 0x7f7b480c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4d79b700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c0140 0x7f7b480caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 -- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c0140 msgr2=0x7f7b480caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c0140 0x7f7b480caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 -- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b480c1bc0 msgr2=0x7f7b480c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b480c1bc0 0x7f7b480c2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 -- 172.21.15.7:0/1486452560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3800b7f0 con 0x7f7b480c1f90 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.647+0000 7f7b4df9c700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000cb50 tx=0x7f7b4000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.648+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40003c00 con 0x7f7b480c1f90 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.648+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b40003d60 con 0x7f7b480c1f90 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.648+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000aa20 con 0x7f7b480c1f90 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.648+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b480c1170 con 0x7f7b480c1f90 2023-06-09T11:13:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.648+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b480c0e90 con 0x7f7b480c1f90 2023-06-09T11:13:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.649+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b480c1450 con 0x7f7b480c1f90 2023-06-09T11:13:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.650+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b4000ab80 con 0x7f7b480c1f90 2023-06-09T11:13:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.651+0000 7f7b3dffb700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 0x7f7b2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.651+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f7b400895e0 con 0x7f7b480c1f90 2023-06-09T11:13:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.651+0000 7f7b4d79b700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 0x7f7b2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.651+0000 7f7b4d79b700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 0x7f7b2c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009be0 tx=0x7f7b44009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.655+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7b40054540 con 0x7f7b480c1f90 2023-06-09T11:13:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.907+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b48074cf0 con 0x7f7b480c1f90 2023-06-09T11:13:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.908+0000 7f7b3dffb700 1 -- 172.21.15.7:0/1486452560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f7b40057b60 con 0x7f7b480c1f90 2023-06-09T11:13:11.913 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:11.913 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 msgr2=0x7f7b2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 0x7f7b2c06d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009be0 tx=0x7f7b44009210 comp rx=0 tx=0).stop 2023-06-09T11:13:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 msgr2=0x7f7b481edcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000cb50 tx=0x7f7b4000ce60 comp rx=0 tx=0).stop 2023-06-09T11:13:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 shutdown_connections 2023-06-09T11:13:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b480c1f90 0x7f7b481edcb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b2c06ace0 0x7f7b2c06d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b480c0140 0x7f7b480caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 --2- 172.21.15.7:0/1486452560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b480c1bc0 0x7f7b480c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.910+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 >> 172.21.15.7:0/1486452560 conn(0x7f7b4806a4d0 msgr2=0x7f7b4814db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.911+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 shutdown_connections 2023-06-09T11:13:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:11.911+0000 7f7b4f9ff700 1 -- 172.21.15.7:0/1486452560 wait complete. 2023-06-09T11:13:11.918 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:11.933 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:13.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.143+0000 7f398f59e700 1 Processor -- start 2023-06-09T11:13:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.143+0000 7f398f59e700 1 -- start start 2023-06-09T11:13:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.144+0000 7f398f59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 0x7f39900caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.144+0000 7f398f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990156f20 0x7f39900cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3990071d50 0x7f3990072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398f59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39900cb2b0 con 0x7f3990071d50 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398f59e700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39900d01a0 con 0x7f3990156b50 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39900cb020 con 0x7f3990156f20 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3990071d50 0x7f3990072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3990071d50 0x7f3990072170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53936/0 (socket says 172.21.15.7:53936) 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 -- 172.21.15.7:0/339567456 learned_addr learned my addr 172.21.15.7:0/339567456 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398e59c700 1 --2- 172.21.15.7:0/339567456 >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 0x7f39900caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398dd9b700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990156f20 0x7f39900cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 -- 172.21.15.7:0/339567456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990156f20 msgr2=0x7f39900cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990156f20 0x7f39900cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 -- 172.21.15.7:0/339567456 >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 msgr2=0x7f39900caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 --2- 172.21.15.7:0/339567456 >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 0x7f39900caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398ed9d700 1 -- 172.21.15.7:0/339567456 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39900b54a0 con 0x7f3990071d50 2023-06-09T11:13:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.145+0000 7f398e59c700 1 --2- 172.21.15.7:0/339567456 >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 0x7f39900caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:14.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398ed9d700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990071d50 0x7f3990072170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39900729a0 tx=0x7f398800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fef033fc9fd772 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398d59a700 1 -- 172.21.15.7:0/339567456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398800e070 con 0x7f3990071d50 2023-06-09T11:13:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398d59a700 1 -- 172.21.15.7:0/339567456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398800b9c0 con 0x7f3990071d50 2023-06-09T11:13:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398d59a700 1 -- 172.21.15.7:0/339567456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398800bd70 con 0x7f3990071d50 2023-06-09T11:13:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 -- 172.21.15.7:0/339567456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990071d50 msgr2=0x7f3990072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:14.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990071d50 0x7f3990072170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39900729a0 tx=0x7f398800b330 comp rx=0 tx=0).stop 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 -- 172.21.15.7:0/339567456 shutdown_connections 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 --2- 172.21.15.7:0/339567456 >> v2:172.21.15.117:3300/0 conn(0x7f3990156b50 0x7f39900caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990071d50 0x7f3990072170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 --2- 172.21.15.7:0/339567456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990156f20 0x7f39900cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.146+0000 7f398f59e700 1 -- 172.21.15.7:0/339567456 >> 172.21.15.7:0/339567456 conn(0x7f39900737f0 msgr2=0x7f3990073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.147+0000 7f398f59e700 1 -- 172.21.15.7:0/339567456 shutdown_connections 2023-06-09T11:13:14.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.147+0000 7f398f59e700 1 -- 172.21.15.7:0/339567456 wait complete. 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.148+0000 7f398f59e700 1 Processor -- start 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.148+0000 7f398f59e700 1 -- start start 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990071d50 0x7f39900cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3990156b50 0x7f39900ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39901efc80 con 0x7f3990156f20 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39900c7090 con 0x7f3990156b50 2023-06-09T11:13:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.149+0000 7f398f59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39901e12d0 con 0x7f3990071d50 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53950/0 (socket says 172.21.15.7:53950) 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 -- 172.21.15.7:0/2619503054 learned_addr learned my addr 172.21.15.7:0/2619503054 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398dd9b700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3990156b50 0x7f39900ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398e59c700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990071d50 0x7f39900cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 -- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990071d50 msgr2=0x7f39900cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990071d50 0x7f39900cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 -- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3990156b50 msgr2=0x7f39900ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3990156b50 0x7f39900ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 -- 172.21.15.7:0/2619503054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3988009b30 con 0x7f3990156f20 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f398ed9d700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f398800b730 tx=0x7f3988004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988004250 con 0x7f3990156f20 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.150+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39880043d0 con 0x7f3990156f20 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.151+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39900cf590 con 0x7f3990156f20 2023-06-09T11:13:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.151+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988004780 con 0x7f3990156f20 2023-06-09T11:13:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.151+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39900cba50 con 0x7f3990156f20 2023-06-09T11:13:14.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.152+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39900cf870 con 0x7f3990156f20 2023-06-09T11:13:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.153+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f398801c440 con 0x7f3990156f20 2023-06-09T11:13:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.154+0000 7f397effd700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 0x7f397006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.154+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f398808abb0 con 0x7f3990156f20 2023-06-09T11:13:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.154+0000 7f398e59c700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 0x7f397006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:14.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.155+0000 7f398e59c700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 0x7f397006d1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f398000c8a0 tx=0x7f3980008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:14.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.159+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3988055a80 con 0x7f3990156f20 2023-06-09T11:13:14.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.413+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39901efdc0 con 0x7f3990156f20 2023-06-09T11:13:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.413+0000 7f397effd700 1 -- 172.21.15.7:0/2619503054 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f39901efdc0 con 0x7f3990156f20 2023-06-09T11:13:14.418 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:14.419 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 msgr2=0x7f397006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 0x7f397006d1c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f398000c8a0 tx=0x7f3980008040 comp rx=0 tx=0).stop 2023-06-09T11:13:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 msgr2=0x7f39901f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:14.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f398800b730 tx=0x7f3988004ad0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 shutdown_connections 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3990156f20 0x7f39901f2390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f397006ad10 0x7f397006d1c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3990071d50 0x7f39900cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 --2- 172.21.15.7:0/2619503054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3990156b50 0x7f39900ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 >> 172.21.15.7:0/2619503054 conn(0x7f39900737f0 msgr2=0x7f3990158b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 shutdown_connections 2023-06-09T11:13:14.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:14.416+0000 7f398f59e700 1 -- 172.21.15.7:0/2619503054 wait complete. 2023-06-09T11:13:14.423 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:14.439 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:16.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:16.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.647+0000 7f93e29ff700 1 Processor -- start 2023-06-09T11:13:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.648+0000 7f93e29ff700 1 -- start start 2023-06-09T11:13:16.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93e29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc159ee0 0x7f93dc15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93e29ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93dc06b460 con 0x7f93dc1539b0 2023-06-09T11:13:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93e29ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93dc06a570 con 0x7f93dc1535e0 2023-06-09T11:13:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93e29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc0ca670 con 0x7f93dc159ee0 2023-06-09T11:13:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93db7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f93dc1539b0 0x7f93dc0c4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53172/0 (socket says 172.21.15.7:53172) 2023-06-09T11:13:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93db7fe700 1 -- 172.21.15.7:0/2060714392 learned_addr learned my addr 172.21.15.7:0/2060714392 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.649+0000 7f93e0f9c700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc159ee0 0x7f93dc15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866999535 0 0) 0x7f93dc06b460 con 0x7f93dc1539b0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c40035b0 con 0x7f93dc1539b0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165405054 0 0) 0x7f93dc06a570 con 0x7f93dc1535e0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93dc06b460 con 0x7f93dc1535e0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 468195233 0 0) 0x7f93c40035b0 con 0x7f93dc1539b0 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc159ee0 msgr2=0x7f93dc15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc159ee0 0x7f93dc15abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 >> v1:172.21.15.117:6789/0 conn(0x7f93dc1535e0 legacy=0x7f93dc0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc06a850 con 0x7f93dc1539b0 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f93d0001670 con 0x7f93dc1539b0 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 >> v1:172.21.15.7:6789/0 conn(0x7f93dc1539b0 legacy=0x7f93dc0c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c4003bc0 0x7f93c4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93c4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4005200 0x7f93c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c400bfa0 con 0x7f93c4004850 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c400c110 con 0x7f93c4005200 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c400c2d0 con 0x7f93c4003bc0 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93e0f9c700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc159ee0 0x7f93dc15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93db7fe700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93c4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.650+0000 7f93dbfff700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c4003bc0 0x7f93c4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93e0f9c700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4005200 0x7f93c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 -- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c4003bc0 msgr2=0x7f93c4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93c4003bc0 0x7f93c4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 -- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4005200 msgr2=0x7f93c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93c4005200 0x7f93c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 -- 172.21.15.7:0/2060714392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40035b0 con 0x7f93c4004850 2023-06-09T11:13:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93db7fe700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93c4004cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93dc153da0 tx=0x7f93c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace726d043b89b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d000cc60 con 0x7f93c4004850 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d000cde0 con 0x7f93c4004850 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.651+0000 7f93daffd700 1 -- 172.21.15.7:0/2060714392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d0005ba0 con 0x7f93c4004850 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 -- 172.21.15.7:0/2060714392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 msgr2=0x7f93c4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 --2- 172.21.15.7:0/2060714392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93c4004cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93dc153da0 tx=0x7f93c4005880 comp rx=0 tx=0).stop 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e0f9c700 1 -- 172.21.15.7:0/2060714392 reap_dead start 2023-06-09T11:13:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 -- 172.21.15.7:0/2060714392 shutdown_connections 2023-06-09T11:13:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 -- 172.21.15.7:0/2060714392 >> 172.21.15.7:0/2060714392 conn(0x7f93dc0737f0 msgr2=0x7f93dc072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 -- 172.21.15.7:0/2060714392 shutdown_connections 2023-06-09T11:13:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.652+0000 7f93e29ff700 1 -- 172.21.15.7:0/2060714392 wait complete. 2023-06-09T11:13:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.654+0000 7f93e29ff700 1 Processor -- start 2023-06-09T11:13:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.655+0000 7f93e29ff700 1 -- start start 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc157700 0x7f93dc0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93dc0b3700 0x7f93dc14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc157db0 con 0x7f93c4004850 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc1f33c0 con 0x7f93dc0b3700 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93dc158a60 con 0x7f93dc157700 2023-06-09T11:13:16.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53970/0 (socket says 172.21.15.7:53970) 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93dbfff700 1 -- 172.21.15.7:0/3929604623 learned_addr learned my addr 172.21.15.7:0/3929604623 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93db7fe700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc157700 0x7f93dc0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.656+0000 7f93e0f9c700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93dc0b3700 0x7f93dc14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 -- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc157700 msgr2=0x7f93dc0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc157700 0x7f93dc0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 -- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93dc0b3700 msgr2=0x7f93dc14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93dc0b3700 0x7f93dc14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 -- 172.21.15.7:0/3929604623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc06ae10 con 0x7f93c4004850 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93dbfff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00f770 tx=0x7f93cc00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc005320 con 0x7f93c4004850 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc0054a0 con 0x7f93c4004850 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc006aa0 con 0x7f93c4004850 2023-06-09T11:13:16.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93dc06ab30 con 0x7f93c4004850 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.657+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93dc06b0f0 con 0x7f93c4004850 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.658+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93dc158d40 con 0x7f93c4004850 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.659+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93cc006c00 con 0x7f93c4004850 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.660+0000 7f93d8ff9700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 0x7f93c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.660+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6234+0+0 (secure 0 0 0) 0x7f93cc088930 con 0x7f93c4004850 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.660+0000 7f93db7fe700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 0x7f93c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:16.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.661+0000 7f93db7fe700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 0x7f93c806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f93dc1f0030 tx=0x7f93d0007f90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:16.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.664+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f93cc053920 con 0x7f93c4004850 2023-06-09T11:13:16.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.918+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93dc0b3470 con 0x7f93c4004850 2023-06-09T11:13:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.919+0000 7f93d8ff9700 1 -- 172.21.15.7:0/3929604623 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+6212 (secure 0 0 0) 0x7f93cc056f40 con 0x7f93c4004850 2023-06-09T11:13:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:16.925 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4936,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:01.177797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":23,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"e","rank":2,"incarnation":13,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/967742498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":967742498},{"type":"v1","addr":"172.21.15.7:6839","nonce":967742498}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.921+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 msgr2=0x7f93c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.921+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 0x7f93c806d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f93dc1f0030 tx=0x7f93d0007f90 comp rx=0 tx=0).stop 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 msgr2=0x7f93dc1f2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00f770 tx=0x7f93cc00fa80 comp rx=0 tx=0).stop 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 shutdown_connections 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f93dc0b3700 0x7f93dc14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c4004850 0x7f93dc1f2e80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f93c806ace0 0x7f93c806d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 --2- 172.21.15.7:0/3929604623 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93dc157700 0x7f93dc0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 >> 172.21.15.7:0/3929604623 conn(0x7f93dc0737f0 msgr2=0x7f93dc0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 shutdown_connections 2023-06-09T11:13:16.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:16.922+0000 7f93e29ff700 1 -- 172.21.15.7:0/3929604623 wait complete. 2023-06-09T11:13:16.929 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 20 2023-06-09T11:13:16.945 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:13:18.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.154+0000 7f0d669ff700 1 Processor -- start 2023-06-09T11:13:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.155+0000 7f0d669ff700 1 -- start start 2023-06-09T11:13:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.156+0000 7f0d669ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 0x7f0d600b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.156+0000 7f0d669ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0d6015cd40 0x7f0d600cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d669ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d60159b20 con 0x7f0d600cb9a0 2023-06-09T11:13:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d669ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d600cac80 con 0x7f0d6015cd40 2023-06-09T11:13:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d669ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601597d0 con 0x7f0d600cb5d0 2023-06-09T11:13:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d5f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0d600cb9a0 0x7f0d60159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53182/0 (socket says 172.21.15.7:53182) 2023-06-09T11:13:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d5f7fe700 1 -- 172.21.15.7:0/2866714029 learned_addr learned my addr 172.21.15.7:0/2866714029 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 0x7f0d600b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d64f9c700 1 --2- 172.21.15.7:0/2866714029 >> v2:172.21.15.117:3300/0 conn(0x7f0d6015cd40 0x7f0d600cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d64f9c700 1 -- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 msgr2=0x7f0d600b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d64f9c700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 0x7f0d600b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d64f9c700 1 -- 172.21.15.7:0/2866714029 >> v1:172.21.15.7:6789/0 conn(0x7f0d600cb9a0 legacy=0x7f0d60159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d64f9c700 1 -- 172.21.15.7:0/2866714029 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60071ab0 con 0x7f0d6015cd40 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.157+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 0x7f0d600b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.158+0000 7f0d64f9c700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d6015cd40 0x7f0d600cbea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009d90 tx=0x7f0d5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d12e74693d29c30 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.158+0000 7f0d5effd700 1 -- 172.21.15.7:0/2866714029 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5800e070 con 0x7f0d6015cd40 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.158+0000 7f0d5effd700 1 -- 172.21.15.7:0/2866714029 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5800bb20 con 0x7f0d6015cd40 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.158+0000 7f0d5effd700 1 -- 172.21.15.7:0/2866714029 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d58012430 con 0x7f0d6015cd40 2023-06-09T11:13:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 -- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d6015cd40 msgr2=0x7f0d600cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d6015cd40 0x7f0d600cbea0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009d90 tx=0x7f0d5800b330 comp rx=0 tx=0).stop 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 -- 172.21.15.7:0/2866714029 shutdown_connections 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d6015cd40 0x7f0d600cbea0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 --2- 172.21.15.7:0/2866714029 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d600cb5d0 0x7f0d600b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 -- 172.21.15.7:0/2866714029 >> 172.21.15.7:0/2866714029 conn(0x7f0d60070af0 msgr2=0x7f0d60070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 -- 172.21.15.7:0/2866714029 shutdown_connections 2023-06-09T11:13:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.159+0000 7f0d669ff700 1 -- 172.21.15.7:0/2866714029 wait complete. 2023-06-09T11:13:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.161+0000 7f0d669ff700 1 Processor -- start 2023-06-09T11:13:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.161+0000 7f0d669ff700 1 -- start start 2023-06-09T11:13:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d600cb5d0 0x7f0d600ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 0x7f0d60153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d600c8e00 con 0x7f0d600cb5d0 2023-06-09T11:13:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601ef540 con 0x7f0d600cb9a0 2023-06-09T11:13:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.162+0000 7f0d669ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d601ef820 con 0x7f0d6015cd40 2023-06-09T11:13:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d64f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 0x7f0d60153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:42292/0 (socket says 172.21.15.7:42292) 2023-06-09T11:13:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 -- 172.21.15.7:0/3223685501 learned_addr learned my addr 172.21.15.7:0/3223685501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d600cb5d0 0x7f0d600ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 -- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 msgr2=0x7f0d60153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 0x7f0d60153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 -- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d600cb5d0 msgr2=0x7f0d600ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d600cb5d0 0x7f0d600ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d5f7fe700 1 -- 172.21.15.7:0/3223685501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500094a0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.163+0000 7f0d64f9c700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 0x7f0d60153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.164+0000 7f0d5f7fe700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c500 tx=0x7f0d5400c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.166+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005d70 con 0x7f0d600cb9a0 2023-06-09T11:13:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.166+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d540070d0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.166+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d58009a70 con 0x7f0d600cb9a0 2023-06-09T11:13:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.166+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54007440 con 0x7f0d600cb9a0 2023-06-09T11:13:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.166+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6015d4f0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.167+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6015d210 con 0x7f0d600cb9a0 2023-06-09T11:13:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.169+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d540075a0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.170+0000 7f0d5cff9700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 0x7f0d4c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.171+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f0d54088780 con 0x7f0d600cb9a0 2023-06-09T11:13:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.171+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 0x7f0d4c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.171+0000 7f0d5ffff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 0x7f0d4c06d1e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d70 tx=0x7f0d50009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.175+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0d540538d0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/30 file after the snapshot 2023-06-09T11:13:19.203 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/31 file after the snapshot 2023-06-09T11:13:19.207 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/32 file after the snapshot 2023-06-09T11:13:19.213 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/33 file after the snapshot 2023-06-09T11:13:19.216 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/34 file after the snapshot 2023-06-09T11:13:19.219 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/35 file after the snapshot 2023-06-09T11:13:19.222 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/36 file after the snapshot 2023-06-09T11:13:19.224 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/37 file after the snapshot 2023-06-09T11:13:19.227 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/38 file after the snapshot 2023-06-09T11:13:19.229 INFO:tasks.workunit.client.0.smithi007.stdout: create 129/39 file after the snapshot 2023-06-09T11:13:19.232 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/.snap/snap-subdir-test 2023-06-09T11:13:19.265 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/30 file after the snapshot 2023-06-09T11:13:19.268 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/31 file after the snapshot 2023-06-09T11:13:19.270 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/32 file after the snapshot 2023-06-09T11:13:19.273 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/33 file after the snapshot 2023-06-09T11:13:19.276 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/34 file after the snapshot 2023-06-09T11:13:19.278 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/35 file after the snapshot 2023-06-09T11:13:19.281 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/36 file after the snapshot 2023-06-09T11:13:19.283 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/37 file after the snapshot 2023-06-09T11:13:19.285 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/38 file after the snapshot 2023-06-09T11:13:19.288 INFO:tasks.workunit.client.0.smithi007.stdout: create 130/39 file after the snapshot 2023-06-09T11:13:19.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/.snap/snap-subdir-test 2023-06-09T11:13:19.314 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/30 file after the snapshot 2023-06-09T11:13:19.316 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/31 file after the snapshot 2023-06-09T11:13:19.319 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/32 file after the snapshot 2023-06-09T11:13:19.323 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/33 file after the snapshot 2023-06-09T11:13:19.326 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/34 file after the snapshot 2023-06-09T11:13:19.328 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/35 file after the snapshot 2023-06-09T11:13:19.330 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/36 file after the snapshot 2023-06-09T11:13:19.333 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/37 file after the snapshot 2023-06-09T11:13:19.336 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/38 file after the snapshot 2023-06-09T11:13:19.339 INFO:tasks.workunit.client.0.smithi007.stdout: create 131/39 file after the snapshot 2023-06-09T11:13:19.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/.snap/snap-subdir-test 2023-06-09T11:13:19.365 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/30 file after the snapshot 2023-06-09T11:13:19.368 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/31 file after the snapshot 2023-06-09T11:13:19.371 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/32 file after the snapshot 2023-06-09T11:13:19.373 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/33 file after the snapshot 2023-06-09T11:13:19.376 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/34 file after the snapshot 2023-06-09T11:13:19.378 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/35 file after the snapshot 2023-06-09T11:13:19.381 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/36 file after the snapshot 2023-06-09T11:13:19.383 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/37 file after the snapshot 2023-06-09T11:13:19.386 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/38 file after the snapshot 2023-06-09T11:13:19.389 INFO:tasks.workunit.client.0.smithi007.stdout: create 132/39 file after the snapshot 2023-06-09T11:13:19.391 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/.snap/snap-subdir-test 2023-06-09T11:13:19.413 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/30 file after the snapshot 2023-06-09T11:13:19.416 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/31 file after the snapshot 2023-06-09T11:13:19.418 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/32 file after the snapshot 2023-06-09T11:13:19.420 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/33 file after the snapshot 2023-06-09T11:13:19.423 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/34 file after the snapshot 2023-06-09T11:13:19.425 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/35 file after the snapshot 2023-06-09T11:13:19.443 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/36 file after the snapshot 2023-06-09T11:13:19.445 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/37 file after the snapshot 2023-06-09T11:13:19.448 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/38 file after the snapshot 2023-06-09T11:13:19.450 INFO:tasks.workunit.client.0.smithi007.stdout: create 133/39 file after the snapshot 2023-06-09T11:13:19.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/.snap/snap-subdir-test 2023-06-09T11:13:19.479 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/30 file after the snapshot 2023-06-09T11:13:19.481 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/31 file after the snapshot 2023-06-09T11:13:19.484 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/32 file after the snapshot 2023-06-09T11:13:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.479+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d6015d7d0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.480+0000 7f0d5cff9700 1 -- 172.21.15.7:0/3223685501 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f0d54056ef0 con 0x7f0d600cb9a0 2023-06-09T11:13:19.486 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:19.486 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:19.140592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:19.487 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/33 file after the snapshot 2023-06-09T11:13:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 msgr2=0x7f0d4c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 0x7f0d4c06d1e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d70 tx=0x7f0d50009210 comp rx=0 tx=0).stop 2023-06-09T11:13:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 msgr2=0x7f0d600cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400c500 tx=0x7f0d5400c810 comp rx=0 tx=0).stop 2023-06-09T11:13:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 shutdown_connections 2023-06-09T11:13:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d4c06ad30 0x7f0d4c06d1e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0d6015cd40 0x7f0d60153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d600cb5d0 0x7f0d600ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 --2- 172.21.15.7:0/3223685501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0d600cb9a0 0x7f0d600cfd20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 >> 172.21.15.7:0/3223685501 conn(0x7f0d60070af0 msgr2=0x7f0d600cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 shutdown_connections 2023-06-09T11:13:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.483+0000 7f0d669ff700 1 -- 172.21.15.7:0/3223685501 wait complete. 2023-06-09T11:13:19.493 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/34 file after the snapshot 2023-06-09T11:13:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2023-06-09T11:13:19.494 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/35 file after the snapshot 2023-06-09T11:13:19.495 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/36 file after the snapshot 2023-06-09T11:13:19.497 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/37 file after the snapshot 2023-06-09T11:13:19.499 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/38 file after the snapshot 2023-06-09T11:13:19.502 INFO:tasks.workunit.client.0.smithi007.stdout: create 134/39 file after the snapshot 2023-06-09T11:13:19.504 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/.snap/snap-subdir-test 2023-06-09T11:13:19.511 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-06-09T11:13:19.511 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:13:19.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:19.527 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/30 file after the snapshot 2023-06-09T11:13:19.530 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/31 file after the snapshot 2023-06-09T11:13:19.532 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/32 file after the snapshot 2023-06-09T11:13:19.535 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/33 file after the snapshot 2023-06-09T11:13:19.537 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/34 file after the snapshot 2023-06-09T11:13:19.540 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/35 file after the snapshot 2023-06-09T11:13:19.542 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/36 file after the snapshot 2023-06-09T11:13:19.545 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/37 file after the snapshot 2023-06-09T11:13:19.548 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/38 file after the snapshot 2023-06-09T11:13:19.550 INFO:tasks.workunit.client.0.smithi007.stdout: create 135/39 file after the snapshot 2023-06-09T11:13:19.553 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/.snap/snap-subdir-test 2023-06-09T11:13:19.575 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/30 file after the snapshot 2023-06-09T11:13:19.577 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/31 file after the snapshot 2023-06-09T11:13:19.580 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/32 file after the snapshot 2023-06-09T11:13:19.583 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/33 file after the snapshot 2023-06-09T11:13:19.585 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/34 file after the snapshot 2023-06-09T11:13:19.588 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/35 file after the snapshot 2023-06-09T11:13:19.591 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/36 file after the snapshot 2023-06-09T11:13:19.593 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/37 file after the snapshot 2023-06-09T11:13:19.596 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/38 file after the snapshot 2023-06-09T11:13:19.598 INFO:tasks.workunit.client.0.smithi007.stdout: create 136/39 file after the snapshot 2023-06-09T11:13:19.600 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/.snap/snap-subdir-test 2023-06-09T11:13:19.620 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/30 file after the snapshot 2023-06-09T11:13:19.622 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/31 file after the snapshot 2023-06-09T11:13:19.625 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/32 file after the snapshot 2023-06-09T11:13:19.627 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/33 file after the snapshot 2023-06-09T11:13:19.629 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/34 file after the snapshot 2023-06-09T11:13:19.632 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/35 file after the snapshot 2023-06-09T11:13:19.634 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/36 file after the snapshot 2023-06-09T11:13:19.636 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/37 file after the snapshot 2023-06-09T11:13:19.639 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/38 file after the snapshot 2023-06-09T11:13:19.641 INFO:tasks.workunit.client.0.smithi007.stdout: create 137/39 file after the snapshot 2023-06-09T11:13:19.644 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/.snap/snap-subdir-test 2023-06-09T11:13:19.665 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/30 file after the snapshot 2023-06-09T11:13:19.668 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/31 file after the snapshot 2023-06-09T11:13:19.670 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/32 file after the snapshot 2023-06-09T11:13:19.673 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/33 file after the snapshot 2023-06-09T11:13:19.675 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/34 file after the snapshot 2023-06-09T11:13:19.678 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/35 file after the snapshot 2023-06-09T11:13:19.680 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/36 file after the snapshot 2023-06-09T11:13:19.682 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/37 file after the snapshot 2023-06-09T11:13:19.685 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/38 file after the snapshot 2023-06-09T11:13:19.687 INFO:tasks.workunit.client.0.smithi007.stdout: create 138/39 file after the snapshot 2023-06-09T11:13:19.690 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/.snap/snap-subdir-test 2023-06-09T11:13:19.712 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/30 file after the snapshot 2023-06-09T11:13:19.715 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/31 file after the snapshot 2023-06-09T11:13:19.717 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/32 file after the snapshot 2023-06-09T11:13:19.719 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/33 file after the snapshot 2023-06-09T11:13:19.722 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/34 file after the snapshot 2023-06-09T11:13:19.724 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/35 file after the snapshot 2023-06-09T11:13:19.726 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/36 file after the snapshot 2023-06-09T11:13:19.729 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/37 file after the snapshot 2023-06-09T11:13:19.731 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/38 file after the snapshot 2023-06-09T11:13:19.734 INFO:tasks.workunit.client.0.smithi007.stdout: create 139/39 file after the snapshot 2023-06-09T11:13:19.736 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/.snap/snap-subdir-test 2023-06-09T11:13:19.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.738+0000 7f823dfff700 1 Processor -- start 2023-06-09T11:13:19.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.738+0000 7f823dfff700 1 -- start start 2023-06-09T11:13:19.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.739+0000 7f823dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 0x7f82380ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.739+0000 7f823dfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823815a390 con 0x7f8238153b30 2023-06-09T11:13:19.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.739+0000 7f823dfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380ca720 con 0x7f8238071fa0 2023-06-09T11:13:19.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.739+0000 7f823dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82380c5f40 con 0x7f8238153760 2023-06-09T11:13:19.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f8236ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8238153b30 0x7f82380b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53194/0 (socket says 172.21.15.7:53194) 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82377fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 0x7f82380ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f8236ffd700 1 -- 172.21.15.7:0/4108156050 learned_addr learned my addr 172.21.15.7:0/4108156050 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82367fc700 1 -- 172.21.15.7:0/4108156050 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162490835 0 0) 0x7f82380ca720 con 0x7f8238071fa0 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82367fc700 1 -- 172.21.15.7:0/4108156050 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82200035b0 con 0x7f8238071fa0 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82377fe700 1 -- 172.21.15.7:0/4108156050 >> v1:172.21.15.117:6789/0 conn(0x7f8238071fa0 legacy=0x7f823815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82377fe700 1 -- 172.21.15.7:0/4108156050 >> v1:172.21.15.7:6789/0 conn(0x7f8238153b30 legacy=0x7f82380b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.740+0000 7f82377fe700 1 -- 172.21.15.7:0/4108156050 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380c5a20 con 0x7f8238153760 2023-06-09T11:13:19.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f82377fe700 1 --2- 172.21.15.7:0/4108156050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 0x7f82380ca070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8228009870 tx=0x7f8228009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8db0e42cf9e758e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f82367fc700 1 -- 172.21.15.7:0/4108156050 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228011830 con 0x7f8238153760 2023-06-09T11:13:19.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f82367fc700 1 -- 172.21.15.7:0/4108156050 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8228011990 con 0x7f8238153760 2023-06-09T11:13:19.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f82367fc700 1 -- 172.21.15.7:0/4108156050 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228011d20 con 0x7f8238153760 2023-06-09T11:13:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f823dfff700 1 -- 172.21.15.7:0/4108156050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 msgr2=0x7f82380ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.741+0000 7f823dfff700 1 --2- 172.21.15.7:0/4108156050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 0x7f82380ca070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8228009870 tx=0x7f8228009b80 comp rx=0 tx=0).stop 2023-06-09T11:13:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.742+0000 7f823dfff700 1 -- 172.21.15.7:0/4108156050 shutdown_connections 2023-06-09T11:13:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.742+0000 7f823dfff700 1 --2- 172.21.15.7:0/4108156050 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238153760 0x7f82380ca070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.742+0000 7f823dfff700 1 -- 172.21.15.7:0/4108156050 >> 172.21.15.7:0/4108156050 conn(0x7f82380737f0 msgr2=0x7f8238073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:19.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.742+0000 7f823dfff700 1 -- 172.21.15.7:0/4108156050 shutdown_connections 2023-06-09T11:13:19.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.742+0000 7f823dfff700 1 -- 172.21.15.7:0/4108156050 wait complete. 2023-06-09T11:13:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.744+0000 7f823dfff700 1 Processor -- start 2023-06-09T11:13:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.744+0000 7f823dfff700 1 -- start start 2023-06-09T11:13:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 0x7f8238157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8238153760 0x7f823815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 0x7f82380cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381580e0 con 0x7f8238153b30 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381583c0 con 0x7f8238153760 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f823dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381589f0 con 0x7f8238071fa0 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f8237fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 0x7f82380cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f82377fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 0x7f8238157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f82377fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 0x7f8238157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46978/0 (socket says 172.21.15.7:46978) 2023-06-09T11:13:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f8236ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8238153760 0x7f823815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.745+0000 7f82377fe700 1 -- 172.21.15.7:0/3121679301 learned_addr learned my addr 172.21.15.7:0/3121679301 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 -- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 msgr2=0x7f8238157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 0x7f8238157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 -- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8238153760 msgr2=0x7f823815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8238153760 0x7f823815a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 -- 172.21.15.7:0/3121679301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82280094a0 con 0x7f8238153b30 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f8237fff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 0x7f82380cae30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f823000cbc0 tx=0x7f823000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230003c00 con 0x7f8238153b30 2023-06-09T11:13:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230003d60 con 0x7f8238153b30 2023-06-09T11:13:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8238159a60 con 0x7f8238153b30 2023-06-09T11:13:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000aa80 con 0x7f8238153b30 2023-06-09T11:13:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.746+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8238159780 con 0x7f8238153b30 2023-06-09T11:13:19.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.748+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8238159d40 con 0x7f8238153b30 2023-06-09T11:13:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.749+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f823000abe0 con 0x7f8238153b30 2023-06-09T11:13:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.750+0000 7f821ffff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 0x7f822406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.750+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f8230089630 con 0x7f8238153b30 2023-06-09T11:13:19.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.750+0000 7f82377fe700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 0x7f822406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:19.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.751+0000 7f82377fe700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 0x7f822406d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f8228008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:19.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:19.754+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f82300544e0 con 0x7f8238153b30 2023-06-09T11:13:19.760 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/30 file after the snapshot 2023-06-09T11:13:19.763 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/31 file after the snapshot 2023-06-09T11:13:19.765 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/32 file after the snapshot 2023-06-09T11:13:19.767 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/33 file after the snapshot 2023-06-09T11:13:19.769 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/34 file after the snapshot 2023-06-09T11:13:19.771 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/35 file after the snapshot 2023-06-09T11:13:19.774 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/36 file after the snapshot 2023-06-09T11:13:19.776 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/37 file after the snapshot 2023-06-09T11:13:19.779 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/38 file after the snapshot 2023-06-09T11:13:19.781 INFO:tasks.workunit.client.0.smithi007.stdout: create 140/39 file after the snapshot 2023-06-09T11:13:19.784 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/.snap/snap-subdir-test 2023-06-09T11:13:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.016+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8238074cf0 con 0x7f8238153b30 2023-06-09T11:13:20.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.016+0000 7f821ffff700 1 -- 172.21.15.7:0/3121679301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f8230057b00 con 0x7f8238153b30 2023-06-09T11:13:20.022 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:20.023 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:19.140592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 msgr2=0x7f822406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:20.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 0x7f822406d190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8228000c00 tx=0x7f8228008030 comp rx=0 tx=0).stop 2023-06-09T11:13:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 msgr2=0x7f82380cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 0x7f82380cae30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f823000cbc0 tx=0x7f823000ced0 comp rx=0 tx=0).stop 2023-06-09T11:13:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 shutdown_connections 2023-06-09T11:13:20.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8238153b30 0x7f82380cae30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f822406ace0 0x7f822406d190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8238071fa0 0x7f8238157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 --2- 172.21.15.7:0/3121679301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8238153760 0x7f823815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:20.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 >> 172.21.15.7:0/3121679301 conn(0x7f82380737f0 msgr2=0x7f823814f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:20.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 shutdown_connections 2023-06-09T11:13:20.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:20.019+0000 7f823dfff700 1 -- 172.21.15.7:0/3121679301 wait complete. 2023-06-09T11:13:20.029 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2023-06-09T11:13:20.044 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 21, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 21, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4288": { "addr": "172.21.15.117:6835/4051500902", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 4051500902, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 4051500902, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4288, "incarnation": 9, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:13:19.140592+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 4288, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:13:20.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/30 file after the snapshot 2023-06-09T11:13:20.391 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/31 file after the snapshot 2023-06-09T11:13:20.393 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/32 file after the snapshot 2023-06-09T11:13:20.396 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/33 file after the snapshot 2023-06-09T11:13:20.399 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/34 file after the snapshot 2023-06-09T11:13:20.401 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/35 file after the snapshot 2023-06-09T11:13:20.403 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/36 file after the snapshot 2023-06-09T11:13:20.406 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/37 file after the snapshot 2023-06-09T11:13:20.408 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/38 file after the snapshot 2023-06-09T11:13:20.411 INFO:tasks.workunit.client.0.smithi007.stdout: create 141/39 file after the snapshot 2023-06-09T11:13:20.413 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/.snap/snap-subdir-test 2023-06-09T11:13:20.438 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/30 file after the snapshot 2023-06-09T11:13:20.441 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/31 file after the snapshot 2023-06-09T11:13:20.444 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/32 file after the snapshot 2023-06-09T11:13:20.446 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/33 file after the snapshot 2023-06-09T11:13:20.449 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/34 file after the snapshot 2023-06-09T11:13:20.451 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/35 file after the snapshot 2023-06-09T11:13:20.454 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/36 file after the snapshot 2023-06-09T11:13:20.456 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/37 file after the snapshot 2023-06-09T11:13:20.459 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/38 file after the snapshot 2023-06-09T11:13:20.461 INFO:tasks.workunit.client.0.smithi007.stdout: create 142/39 file after the snapshot 2023-06-09T11:13:20.464 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/.snap/snap-subdir-test 2023-06-09T11:13:20.488 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/30 file after the snapshot 2023-06-09T11:13:20.491 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/31 file after the snapshot 2023-06-09T11:13:20.493 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/32 file after the snapshot 2023-06-09T11:13:20.496 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/33 file after the snapshot 2023-06-09T11:13:20.499 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/34 file after the snapshot 2023-06-09T11:13:20.502 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/35 file after the snapshot 2023-06-09T11:13:20.504 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/36 file after the snapshot 2023-06-09T11:13:20.507 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/37 file after the snapshot 2023-06-09T11:13:20.509 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/38 file after the snapshot 2023-06-09T11:13:20.511 INFO:tasks.workunit.client.0.smithi007.stdout: create 143/39 file after the snapshot 2023-06-09T11:13:20.514 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/.snap/snap-subdir-test 2023-06-09T11:13:20.538 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/30 file after the snapshot 2023-06-09T11:13:20.541 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/31 file after the snapshot 2023-06-09T11:13:20.543 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/32 file after the snapshot 2023-06-09T11:13:20.545 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/33 file after the snapshot 2023-06-09T11:13:20.548 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/34 file after the snapshot 2023-06-09T11:13:20.550 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/35 file after the snapshot 2023-06-09T11:13:20.552 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/36 file after the snapshot 2023-06-09T11:13:20.554 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/37 file after the snapshot 2023-06-09T11:13:20.556 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/38 file after the snapshot 2023-06-09T11:13:20.559 INFO:tasks.workunit.client.0.smithi007.stdout: create 144/39 file after the snapshot 2023-06-09T11:13:20.561 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/.snap/snap-subdir-test 2023-06-09T11:13:20.584 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/30 file after the snapshot 2023-06-09T11:13:20.587 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/31 file after the snapshot 2023-06-09T11:13:20.589 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/32 file after the snapshot 2023-06-09T11:13:20.592 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/33 file after the snapshot 2023-06-09T11:13:20.594 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/34 file after the snapshot 2023-06-09T11:13:20.596 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/35 file after the snapshot 2023-06-09T11:13:20.599 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/36 file after the snapshot 2023-06-09T11:13:20.601 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/37 file after the snapshot 2023-06-09T11:13:20.603 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/38 file after the snapshot 2023-06-09T11:13:20.606 INFO:tasks.workunit.client.0.smithi007.stdout: create 145/39 file after the snapshot 2023-06-09T11:13:20.608 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/.snap/snap-subdir-test 2023-06-09T11:13:20.633 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/30 file after the snapshot 2023-06-09T11:13:20.635 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/31 file after the snapshot 2023-06-09T11:13:20.637 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/32 file after the snapshot 2023-06-09T11:13:20.640 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/33 file after the snapshot 2023-06-09T11:13:20.642 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/34 file after the snapshot 2023-06-09T11:13:20.644 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/35 file after the snapshot 2023-06-09T11:13:20.646 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/36 file after the snapshot 2023-06-09T11:13:20.649 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/37 file after the snapshot 2023-06-09T11:13:20.651 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/38 file after the snapshot 2023-06-09T11:13:20.653 INFO:tasks.workunit.client.0.smithi007.stdout: create 146/39 file after the snapshot 2023-06-09T11:13:20.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/.snap/snap-subdir-test 2023-06-09T11:13:20.681 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/30 file after the snapshot 2023-06-09T11:13:20.683 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/31 file after the snapshot 2023-06-09T11:13:20.685 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/32 file after the snapshot 2023-06-09T11:13:20.688 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/33 file after the snapshot 2023-06-09T11:13:20.690 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/34 file after the snapshot 2023-06-09T11:13:20.693 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/35 file after the snapshot 2023-06-09T11:13:20.695 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/36 file after the snapshot 2023-06-09T11:13:20.697 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/37 file after the snapshot 2023-06-09T11:13:20.700 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/38 file after the snapshot 2023-06-09T11:13:20.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 147/39 file after the snapshot 2023-06-09T11:13:20.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/.snap/snap-subdir-test 2023-06-09T11:13:20.727 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/30 file after the snapshot 2023-06-09T11:13:20.730 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/31 file after the snapshot 2023-06-09T11:13:20.732 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/32 file after the snapshot 2023-06-09T11:13:20.735 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/33 file after the snapshot 2023-06-09T11:13:20.737 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/34 file after the snapshot 2023-06-09T11:13:20.740 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/35 file after the snapshot 2023-06-09T11:13:20.742 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/36 file after the snapshot 2023-06-09T11:13:20.744 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/37 file after the snapshot 2023-06-09T11:13:20.747 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/38 file after the snapshot 2023-06-09T11:13:20.749 INFO:tasks.workunit.client.0.smithi007.stdout: create 148/39 file after the snapshot 2023-06-09T11:13:20.752 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/.snap/snap-subdir-test 2023-06-09T11:13:20.775 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/30 file after the snapshot 2023-06-09T11:13:20.778 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/31 file after the snapshot 2023-06-09T11:13:20.780 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/32 file after the snapshot 2023-06-09T11:13:20.783 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/33 file after the snapshot 2023-06-09T11:13:20.785 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/34 file after the snapshot 2023-06-09T11:13:20.788 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/35 file after the snapshot 2023-06-09T11:13:20.791 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/36 file after the snapshot 2023-06-09T11:13:20.794 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/37 file after the snapshot 2023-06-09T11:13:20.796 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/38 file after the snapshot 2023-06-09T11:13:20.799 INFO:tasks.workunit.client.0.smithi007.stdout: create 149/39 file after the snapshot 2023-06-09T11:13:20.801 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/.snap/snap-subdir-test 2023-06-09T11:13:20.824 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/30 file after the snapshot 2023-06-09T11:13:20.826 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/31 file after the snapshot 2023-06-09T11:13:20.828 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/32 file after the snapshot 2023-06-09T11:13:20.831 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/33 file after the snapshot 2023-06-09T11:13:20.834 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/34 file after the snapshot 2023-06-09T11:13:20.836 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/35 file after the snapshot 2023-06-09T11:13:20.838 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/36 file after the snapshot 2023-06-09T11:13:20.841 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/37 file after the snapshot 2023-06-09T11:13:20.843 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/38 file after the snapshot 2023-06-09T11:13:20.846 INFO:tasks.workunit.client.0.smithi007.stdout: create 150/39 file after the snapshot 2023-06-09T11:13:20.848 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/.snap/snap-subdir-test 2023-06-09T11:13:20.870 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/30 file after the snapshot 2023-06-09T11:13:20.873 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/31 file after the snapshot 2023-06-09T11:13:20.875 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/32 file after the snapshot 2023-06-09T11:13:20.878 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/33 file after the snapshot 2023-06-09T11:13:20.881 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/34 file after the snapshot 2023-06-09T11:13:20.883 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/35 file after the snapshot 2023-06-09T11:13:20.886 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/36 file after the snapshot 2023-06-09T11:13:20.889 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/37 file after the snapshot 2023-06-09T11:13:20.891 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/38 file after the snapshot 2023-06-09T11:13:20.894 INFO:tasks.workunit.client.0.smithi007.stdout: create 151/39 file after the snapshot 2023-06-09T11:13:20.896 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/.snap/snap-subdir-test 2023-06-09T11:13:20.920 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/30 file after the snapshot 2023-06-09T11:13:20.924 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/31 file after the snapshot 2023-06-09T11:13:20.926 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/32 file after the snapshot 2023-06-09T11:13:20.928 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/33 file after the snapshot 2023-06-09T11:13:20.931 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/34 file after the snapshot 2023-06-09T11:13:20.933 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/35 file after the snapshot 2023-06-09T11:13:20.935 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/36 file after the snapshot 2023-06-09T11:13:20.938 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/37 file after the snapshot 2023-06-09T11:13:20.940 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/38 file after the snapshot 2023-06-09T11:13:20.942 INFO:tasks.workunit.client.0.smithi007.stdout: create 152/39 file after the snapshot 2023-06-09T11:13:20.945 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/.snap/snap-subdir-test 2023-06-09T11:13:20.968 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/30 file after the snapshot 2023-06-09T11:13:20.970 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/31 file after the snapshot 2023-06-09T11:13:20.973 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/32 file after the snapshot 2023-06-09T11:13:20.975 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/33 file after the snapshot 2023-06-09T11:13:20.977 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/34 file after the snapshot 2023-06-09T11:13:20.980 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/35 file after the snapshot 2023-06-09T11:13:20.982 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/36 file after the snapshot 2023-06-09T11:13:20.984 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/37 file after the snapshot 2023-06-09T11:13:20.986 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/38 file after the snapshot 2023-06-09T11:13:20.989 INFO:tasks.workunit.client.0.smithi007.stdout: create 153/39 file after the snapshot 2023-06-09T11:13:20.991 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/.snap/snap-subdir-test 2023-06-09T11:13:21.014 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/30 file after the snapshot 2023-06-09T11:13:21.017 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/31 file after the snapshot 2023-06-09T11:13:21.019 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/32 file after the snapshot 2023-06-09T11:13:21.022 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/33 file after the snapshot 2023-06-09T11:13:21.024 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/34 file after the snapshot 2023-06-09T11:13:21.027 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/35 file after the snapshot 2023-06-09T11:13:21.029 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/36 file after the snapshot 2023-06-09T11:13:21.032 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/37 file after the snapshot 2023-06-09T11:13:21.034 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/38 file after the snapshot 2023-06-09T11:13:21.036 INFO:tasks.workunit.client.0.smithi007.stdout: create 154/39 file after the snapshot 2023-06-09T11:13:21.038 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/.snap/snap-subdir-test 2023-06-09T11:13:21.059 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/30 file after the snapshot 2023-06-09T11:13:21.062 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/31 file after the snapshot 2023-06-09T11:13:21.064 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/32 file after the snapshot 2023-06-09T11:13:21.067 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/33 file after the snapshot 2023-06-09T11:13:21.070 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/34 file after the snapshot 2023-06-09T11:13:21.072 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/35 file after the snapshot 2023-06-09T11:13:21.074 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/36 file after the snapshot 2023-06-09T11:13:21.077 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/37 file after the snapshot 2023-06-09T11:13:21.079 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/38 file after the snapshot 2023-06-09T11:13:21.081 INFO:tasks.workunit.client.0.smithi007.stdout: create 155/39 file after the snapshot 2023-06-09T11:13:21.084 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/.snap/snap-subdir-test 2023-06-09T11:13:21.104 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/30 file after the snapshot 2023-06-09T11:13:21.106 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/31 file after the snapshot 2023-06-09T11:13:21.109 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/32 file after the snapshot 2023-06-09T11:13:21.111 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/33 file after the snapshot 2023-06-09T11:13:21.113 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/34 file after the snapshot 2023-06-09T11:13:21.115 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/35 file after the snapshot 2023-06-09T11:13:21.117 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/36 file after the snapshot 2023-06-09T11:13:21.119 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/37 file after the snapshot 2023-06-09T11:13:21.121 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/38 file after the snapshot 2023-06-09T11:13:21.124 INFO:tasks.workunit.client.0.smithi007.stdout: create 156/39 file after the snapshot 2023-06-09T11:13:21.126 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/.snap/snap-subdir-test 2023-06-09T11:13:21.146 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/30 file after the snapshot 2023-06-09T11:13:21.149 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/31 file after the snapshot 2023-06-09T11:13:21.152 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/32 file after the snapshot 2023-06-09T11:13:21.155 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/33 file after the snapshot 2023-06-09T11:13:21.157 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/34 file after the snapshot 2023-06-09T11:13:21.160 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/35 file after the snapshot 2023-06-09T11:13:21.162 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/36 file after the snapshot 2023-06-09T11:13:21.165 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/37 file after the snapshot 2023-06-09T11:13:21.168 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/38 file after the snapshot 2023-06-09T11:13:21.170 INFO:tasks.workunit.client.0.smithi007.stdout: create 157/39 file after the snapshot 2023-06-09T11:13:21.173 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/.snap/snap-subdir-test 2023-06-09T11:13:21.194 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/30 file after the snapshot 2023-06-09T11:13:21.197 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/31 file after the snapshot 2023-06-09T11:13:21.199 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/32 file after the snapshot 2023-06-09T11:13:21.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/33 file after the snapshot 2023-06-09T11:13:21.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/34 file after the snapshot 2023-06-09T11:13:21.207 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/35 file after the snapshot 2023-06-09T11:13:21.209 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/36 file after the snapshot 2023-06-09T11:13:21.211 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/37 file after the snapshot 2023-06-09T11:13:21.213 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/38 file after the snapshot 2023-06-09T11:13:21.215 INFO:tasks.workunit.client.0.smithi007.stdout: create 158/39 file after the snapshot 2023-06-09T11:13:21.217 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/.snap/snap-subdir-test 2023-06-09T11:13:21.645 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/30 file after the snapshot 2023-06-09T11:13:21.647 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/31 file after the snapshot 2023-06-09T11:13:21.650 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/32 file after the snapshot 2023-06-09T11:13:21.652 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/33 file after the snapshot 2023-06-09T11:13:21.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/34 file after the snapshot 2023-06-09T11:13:21.658 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/35 file after the snapshot 2023-06-09T11:13:21.660 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/36 file after the snapshot 2023-06-09T11:13:21.663 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/37 file after the snapshot 2023-06-09T11:13:21.665 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/38 file after the snapshot 2023-06-09T11:13:21.668 INFO:tasks.workunit.client.0.smithi007.stdout: create 159/39 file after the snapshot 2023-06-09T11:13:21.671 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/.snap/snap-subdir-test 2023-06-09T11:13:21.696 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/30 file after the snapshot 2023-06-09T11:13:21.698 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/31 file after the snapshot 2023-06-09T11:13:21.701 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/32 file after the snapshot 2023-06-09T11:13:21.703 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/33 file after the snapshot 2023-06-09T11:13:21.705 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/34 file after the snapshot 2023-06-09T11:13:21.707 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/35 file after the snapshot 2023-06-09T11:13:21.709 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/36 file after the snapshot 2023-06-09T11:13:21.712 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/37 file after the snapshot 2023-06-09T11:13:21.715 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/38 file after the snapshot 2023-06-09T11:13:21.717 INFO:tasks.workunit.client.0.smithi007.stdout: create 160/39 file after the snapshot 2023-06-09T11:13:21.719 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/.snap/snap-subdir-test 2023-06-09T11:13:21.743 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/30 file after the snapshot 2023-06-09T11:13:21.745 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/31 file after the snapshot 2023-06-09T11:13:21.747 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/32 file after the snapshot 2023-06-09T11:13:21.750 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/33 file after the snapshot 2023-06-09T11:13:21.752 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/34 file after the snapshot 2023-06-09T11:13:21.754 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/35 file after the snapshot 2023-06-09T11:13:21.756 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/36 file after the snapshot 2023-06-09T11:13:21.758 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/37 file after the snapshot 2023-06-09T11:13:21.760 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/38 file after the snapshot 2023-06-09T11:13:21.763 INFO:tasks.workunit.client.0.smithi007.stdout: create 161/39 file after the snapshot 2023-06-09T11:13:21.766 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/.snap/snap-subdir-test 2023-06-09T11:13:21.795 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/30 file after the snapshot 2023-06-09T11:13:21.797 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/31 file after the snapshot 2023-06-09T11:13:21.799 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/32 file after the snapshot 2023-06-09T11:13:21.801 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/33 file after the snapshot 2023-06-09T11:13:21.804 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/34 file after the snapshot 2023-06-09T11:13:21.806 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/35 file after the snapshot 2023-06-09T11:13:21.808 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/36 file after the snapshot 2023-06-09T11:13:21.811 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/37 file after the snapshot 2023-06-09T11:13:21.813 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/38 file after the snapshot 2023-06-09T11:13:21.815 INFO:tasks.workunit.client.0.smithi007.stdout: create 162/39 file after the snapshot 2023-06-09T11:13:21.818 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/.snap/snap-subdir-test 2023-06-09T11:13:21.845 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/30 file after the snapshot 2023-06-09T11:13:21.848 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/31 file after the snapshot 2023-06-09T11:13:21.850 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/32 file after the snapshot 2023-06-09T11:13:21.852 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/33 file after the snapshot 2023-06-09T11:13:21.854 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/34 file after the snapshot 2023-06-09T11:13:21.856 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/35 file after the snapshot 2023-06-09T11:13:21.859 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/36 file after the snapshot 2023-06-09T11:13:21.861 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/37 file after the snapshot 2023-06-09T11:13:21.863 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/38 file after the snapshot 2023-06-09T11:13:21.865 INFO:tasks.workunit.client.0.smithi007.stdout: create 163/39 file after the snapshot 2023-06-09T11:13:21.868 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/.snap/snap-subdir-test 2023-06-09T11:13:21.897 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/30 file after the snapshot 2023-06-09T11:13:21.899 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/31 file after the snapshot 2023-06-09T11:13:21.902 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/32 file after the snapshot 2023-06-09T11:13:21.904 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/33 file after the snapshot 2023-06-09T11:13:21.907 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/34 file after the snapshot 2023-06-09T11:13:21.909 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/35 file after the snapshot 2023-06-09T11:13:21.911 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/36 file after the snapshot 2023-06-09T11:13:21.913 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/37 file after the snapshot 2023-06-09T11:13:21.916 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/38 file after the snapshot 2023-06-09T11:13:21.918 INFO:tasks.workunit.client.0.smithi007.stdout: create 164/39 file after the snapshot 2023-06-09T11:13:21.920 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/.snap/snap-subdir-test 2023-06-09T11:13:22.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:22.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/30 file after the snapshot 2023-06-09T11:13:22.183 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/31 file after the snapshot 2023-06-09T11:13:22.186 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/32 file after the snapshot 2023-06-09T11:13:22.188 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/33 file after the snapshot 2023-06-09T11:13:22.190 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/34 file after the snapshot 2023-06-09T11:13:22.192 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/35 file after the snapshot 2023-06-09T11:13:22.195 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/36 file after the snapshot 2023-06-09T11:13:22.197 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/37 file after the snapshot 2023-06-09T11:13:22.199 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/38 file after the snapshot 2023-06-09T11:13:22.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 165/39 file after the snapshot 2023-06-09T11:13:22.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/.snap/snap-subdir-test 2023-06-09T11:13:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.250+0000 7f55343ff700 1 Processor -- start 2023-06-09T11:13:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.250+0000 7f55343ff700 1 -- start start 2023-06-09T11:13:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f55343ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 0x7f552c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f55343ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 0x7f552c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f55343ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552c15cec0 con 0x7f552c0c9090 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f55343ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552c0ca410 con 0x7f552c0cfaf0 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f55343ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c15d160 con 0x7f552c0c9fe0 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f553219b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f552c0c9090 0x7f552c0cad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53200/0 (socket says 172.21.15.7:53200) 2023-06-09T11:13:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f553219b700 1 -- 172.21.15.7:0/2542523265 learned_addr learned my addr 172.21.15.7:0/2542523265 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:22.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f553199a700 1 --2- 172.21.15.7:0/2542523265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 0x7f552c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.252+0000 7f553299c700 1 --2- 172.21.15.7:0/2542523265 >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 0x7f552c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f5531199700 1 -- 172.21.15.7:0/2542523265 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890855230 0 0) 0x7f552c15cec0 con 0x7f552c0c9090 2023-06-09T11:13:22.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f5531199700 1 -- 172.21.15.7:0/2542523265 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55180035b0 con 0x7f552c0c9090 2023-06-09T11:13:22.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553199a700 1 -- 172.21.15.7:0/2542523265 >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 msgr2=0x7f552c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:22.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553199a700 1 --2- 172.21.15.7:0/2542523265 >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 0x7f552c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553199a700 1 -- 172.21.15.7:0/2542523265 >> v1:172.21.15.7:6789/0 conn(0x7f552c0c9090 legacy=0x7f552c0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553199a700 1 -- 172.21.15.7:0/2542523265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c15d440 con 0x7f552c0c9fe0 2023-06-09T11:13:22.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553299c700 1 --2- 172.21.15.7:0/2542523265 >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 0x7f552c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f553199a700 1 --2- 172.21.15.7:0/2542523265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 0x7f552c0cf5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f551c009840 tx=0x7f551c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd4458f6cf4b7f71 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f5531199700 1 -- 172.21.15.7:0/2542523265 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c011830 con 0x7f552c0c9fe0 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f5531199700 1 -- 172.21.15.7:0/2542523265 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c011990 con 0x7f552c0c9fe0 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.253+0000 7f5531199700 1 -- 172.21.15.7:0/2542523265 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c011d20 con 0x7f552c0c9fe0 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 -- 172.21.15.7:0/2542523265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 msgr2=0x7f552c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 --2- 172.21.15.7:0/2542523265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 0x7f552c0cf5b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f551c009840 tx=0x7f551c009b50 comp rx=0 tx=0).stop 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 -- 172.21.15.7:0/2542523265 shutdown_connections 2023-06-09T11:13:22.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 --2- 172.21.15.7:0/2542523265 >> v2:172.21.15.117:3300/0 conn(0x7f552c0cfaf0 0x7f552c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 --2- 172.21.15.7:0/2542523265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9fe0 0x7f552c0cf5b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 -- 172.21.15.7:0/2542523265 >> 172.21.15.7:0/2542523265 conn(0x7f552c070af0 msgr2=0x7f552c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 -- 172.21.15.7:0/2542523265 shutdown_connections 2023-06-09T11:13:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.254+0000 7f55343ff700 1 -- 172.21.15.7:0/2542523265 wait complete. 2023-06-09T11:13:22.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.256+0000 7f55343ff700 1 Processor -- start 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.256+0000 7f55343ff700 1 -- start start 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9090 0x7f552c1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 0x7f552c1f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 0x7f552c1e4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1e3340 con 0x7f552c0cfaf0 2023-06-09T11:13:22.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1e3e10 con 0x7f552c0c9fe0 2023-06-09T11:13:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f55343ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c1e4680 con 0x7f552c0c9090 2023-06-09T11:13:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 0x7f552c1e4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553219b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9090 0x7f552c1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 0x7f552c1f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 0x7f552c1e4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53984/0 (socket says 172.21.15.7:53984) 2023-06-09T11:13:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553299c700 1 -- 172.21.15.7:0/3327430558 learned_addr learned my addr 172.21.15.7:0/3327430558 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 -- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9090 msgr2=0x7f552c1f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9090 0x7f552c1f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 -- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 msgr2=0x7f552c1e4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 0x7f552c1e4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.257+0000 7f553199a700 1 -- 172.21.15.7:0/3327430558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009a70 con 0x7f552c0c9fe0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f553199a700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 0x7f552c1f2ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f551c016000 tx=0x7f551c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c004590 con 0x7f552c0c9fe0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c01e070 con 0x7f552c0c9fe0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c0094a0 con 0x7f552c0c9fe0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c0038e0 con 0x7f552c0c9fe0 2023-06-09T11:13:22.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.258+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f552c0ca620 con 0x7f552c0c9fe0 2023-06-09T11:13:22.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.259+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552c15d720 con 0x7f552c0c9fe0 2023-06-09T11:13:22.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.260+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f551c003a40 con 0x7f552c0c9fe0 2023-06-09T11:13:22.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.261+0000 7f5522ffd700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:22.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.261+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f551c08a3d0 con 0x7f552c0c9fe0 2023-06-09T11:13:22.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.261+0000 7f553219b700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:22.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.262+0000 7f553219b700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 0x7f551006d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5528007fd0 tx=0x7f5528007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:22.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.265+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f551c055300 con 0x7f552c0c9fe0 2023-06-09T11:13:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.517+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f552c0b53f0 con 0x7f552c0c9fe0 2023-06-09T11:13:22.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.518+0000 7f5522ffd700 1 -- 172.21.15.7:0/3327430558 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f551c058920 con 0x7f552c0c9fe0 2023-06-09T11:13:22.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:22.524 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:19.140592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 msgr2=0x7f551006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 0x7f551006d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5528007fd0 tx=0x7f5528007dc0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 msgr2=0x7f552c1f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:22.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 0x7f552c1f2ab0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f551c016000 tx=0x7f551c004750 comp rx=0 tx=0).stop 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 shutdown_connections 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f552c0cfaf0 0x7f552c1e4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f551006ace0 0x7f551006d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f552c0c9090 0x7f552c1f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 --2- 172.21.15.7:0/3327430558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f552c0c9fe0 0x7f552c1f2ab0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 >> 172.21.15.7:0/3327430558 conn(0x7f552c070af0 msgr2=0x7f552c0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 shutdown_connections 2023-06-09T11:13:22.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:22.520+0000 7f55343ff700 1 -- 172.21.15.7:0/3327430558 wait complete. 2023-06-09T11:13:22.528 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2023-06-09T11:13:22.544 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:22.799 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/30 file after the snapshot 2023-06-09T11:13:22.802 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/31 file after the snapshot 2023-06-09T11:13:22.805 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/32 file after the snapshot 2023-06-09T11:13:22.807 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/33 file after the snapshot 2023-06-09T11:13:22.810 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/34 file after the snapshot 2023-06-09T11:13:22.812 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/35 file after the snapshot 2023-06-09T11:13:22.814 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/36 file after the snapshot 2023-06-09T11:13:22.816 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/37 file after the snapshot 2023-06-09T11:13:22.819 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/38 file after the snapshot 2023-06-09T11:13:22.821 INFO:tasks.workunit.client.0.smithi007.stdout: create 166/39 file after the snapshot 2023-06-09T11:13:22.824 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/.snap/snap-subdir-test 2023-06-09T11:13:23.809 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/30 file after the snapshot 2023-06-09T11:13:23.811 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/31 file after the snapshot 2023-06-09T11:13:23.814 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/32 file after the snapshot 2023-06-09T11:13:23.817 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/33 file after the snapshot 2023-06-09T11:13:23.820 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/34 file after the snapshot 2023-06-09T11:13:23.822 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/35 file after the snapshot 2023-06-09T11:13:23.825 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/36 file after the snapshot 2023-06-09T11:13:23.828 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/37 file after the snapshot 2023-06-09T11:13:23.831 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/38 file after the snapshot 2023-06-09T11:13:23.833 INFO:tasks.workunit.client.0.smithi007.stdout: create 167/39 file after the snapshot 2023-06-09T11:13:23.836 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/.snap/snap-subdir-test 2023-06-09T11:13:23.861 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/30 file after the snapshot 2023-06-09T11:13:23.864 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/31 file after the snapshot 2023-06-09T11:13:23.867 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/32 file after the snapshot 2023-06-09T11:13:23.869 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/33 file after the snapshot 2023-06-09T11:13:23.872 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/34 file after the snapshot 2023-06-09T11:13:23.874 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/35 file after the snapshot 2023-06-09T11:13:23.877 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/36 file after the snapshot 2023-06-09T11:13:23.880 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/37 file after the snapshot 2023-06-09T11:13:23.882 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/38 file after the snapshot 2023-06-09T11:13:23.885 INFO:tasks.workunit.client.0.smithi007.stdout: create 168/39 file after the snapshot 2023-06-09T11:13:23.887 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/.snap/snap-subdir-test 2023-06-09T11:13:23.911 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/30 file after the snapshot 2023-06-09T11:13:23.914 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/31 file after the snapshot 2023-06-09T11:13:23.917 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/32 file after the snapshot 2023-06-09T11:13:23.920 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/33 file after the snapshot 2023-06-09T11:13:23.922 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/34 file after the snapshot 2023-06-09T11:13:23.925 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/35 file after the snapshot 2023-06-09T11:13:23.927 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/36 file after the snapshot 2023-06-09T11:13:23.930 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/37 file after the snapshot 2023-06-09T11:13:23.932 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/38 file after the snapshot 2023-06-09T11:13:23.935 INFO:tasks.workunit.client.0.smithi007.stdout: create 169/39 file after the snapshot 2023-06-09T11:13:23.937 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/.snap/snap-subdir-test 2023-06-09T11:13:23.960 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/30 file after the snapshot 2023-06-09T11:13:23.963 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/31 file after the snapshot 2023-06-09T11:13:23.965 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/32 file after the snapshot 2023-06-09T11:13:23.968 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/33 file after the snapshot 2023-06-09T11:13:23.970 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/34 file after the snapshot 2023-06-09T11:13:24.055 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/35 file after the snapshot 2023-06-09T11:13:24.058 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/36 file after the snapshot 2023-06-09T11:13:24.060 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/37 file after the snapshot 2023-06-09T11:13:24.062 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/38 file after the snapshot 2023-06-09T11:13:24.065 INFO:tasks.workunit.client.0.smithi007.stdout: create 170/39 file after the snapshot 2023-06-09T11:13:24.068 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/.snap/snap-subdir-test 2023-06-09T11:13:24.369 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/30 file after the snapshot 2023-06-09T11:13:24.372 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/31 file after the snapshot 2023-06-09T11:13:24.375 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/32 file after the snapshot 2023-06-09T11:13:24.378 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/33 file after the snapshot 2023-06-09T11:13:24.380 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/34 file after the snapshot 2023-06-09T11:13:24.382 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/35 file after the snapshot 2023-06-09T11:13:24.385 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/36 file after the snapshot 2023-06-09T11:13:24.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/37 file after the snapshot 2023-06-09T11:13:24.390 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/38 file after the snapshot 2023-06-09T11:13:24.393 INFO:tasks.workunit.client.0.smithi007.stdout: create 171/39 file after the snapshot 2023-06-09T11:13:24.395 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/.snap/snap-subdir-test 2023-06-09T11:13:24.417 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/30 file after the snapshot 2023-06-09T11:13:24.420 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/31 file after the snapshot 2023-06-09T11:13:24.422 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/32 file after the snapshot 2023-06-09T11:13:24.425 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/33 file after the snapshot 2023-06-09T11:13:24.427 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/34 file after the snapshot 2023-06-09T11:13:24.429 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/35 file after the snapshot 2023-06-09T11:13:24.432 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/36 file after the snapshot 2023-06-09T11:13:24.435 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/37 file after the snapshot 2023-06-09T11:13:24.437 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/38 file after the snapshot 2023-06-09T11:13:24.439 INFO:tasks.workunit.client.0.smithi007.stdout: create 172/39 file after the snapshot 2023-06-09T11:13:24.442 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/.snap/snap-subdir-test 2023-06-09T11:13:24.486 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/30 file after the snapshot 2023-06-09T11:13:24.488 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/31 file after the snapshot 2023-06-09T11:13:24.491 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/32 file after the snapshot 2023-06-09T11:13:24.494 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/33 file after the snapshot 2023-06-09T11:13:24.497 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/34 file after the snapshot 2023-06-09T11:13:24.499 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/35 file after the snapshot 2023-06-09T11:13:24.502 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/36 file after the snapshot 2023-06-09T11:13:24.504 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/37 file after the snapshot 2023-06-09T11:13:24.507 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/38 file after the snapshot 2023-06-09T11:13:24.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 173/39 file after the snapshot 2023-06-09T11:13:24.512 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/.snap/snap-subdir-test 2023-06-09T11:13:24.537 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/30 file after the snapshot 2023-06-09T11:13:24.540 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/31 file after the snapshot 2023-06-09T11:13:24.542 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/32 file after the snapshot 2023-06-09T11:13:24.545 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/33 file after the snapshot 2023-06-09T11:13:24.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:24.550 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/34 file after the snapshot 2023-06-09T11:13:24.550 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/35 file after the snapshot 2023-06-09T11:13:24.553 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/36 file after the snapshot 2023-06-09T11:13:24.556 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/37 file after the snapshot 2023-06-09T11:13:24.559 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/38 file after the snapshot 2023-06-09T11:13:24.562 INFO:tasks.workunit.client.0.smithi007.stdout: create 174/39 file after the snapshot 2023-06-09T11:13:24.564 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/.snap/snap-subdir-test 2023-06-09T11:13:24.586 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/30 file after the snapshot 2023-06-09T11:13:24.588 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/31 file after the snapshot 2023-06-09T11:13:24.590 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/32 file after the snapshot 2023-06-09T11:13:24.593 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/33 file after the snapshot 2023-06-09T11:13:24.596 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/34 file after the snapshot 2023-06-09T11:13:24.598 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/35 file after the snapshot 2023-06-09T11:13:24.601 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/36 file after the snapshot 2023-06-09T11:13:24.604 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/37 file after the snapshot 2023-06-09T11:13:24.607 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/38 file after the snapshot 2023-06-09T11:13:24.609 INFO:tasks.workunit.client.0.smithi007.stdout: create 175/39 file after the snapshot 2023-06-09T11:13:24.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/.snap/snap-subdir-test 2023-06-09T11:13:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.749+0000 7f8c0f1ff700 1 Processor -- start 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.749+0000 7f8c0f1ff700 1 -- start start 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.750+0000 7f8c0f1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c0815b4a0 0x7f8c0815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.750+0000 7f8c0f1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0815a500 con 0x7f8c08153880 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.750+0000 7f8c0f1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0815da00 con 0x7f8c080c9f40 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.750+0000 7f8c0f1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080c5f10 con 0x7f8c0815b4a0 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c0815b4a0 0x7f8c0815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:24.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c0815b4a0 0x7f8c0815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42604/0 (socket says 172.21.15.7:42604) 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 -- 172.21.15.7:0/1950878321 learned_addr learned my addr 172.21.15.7:0/1950878321 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 -- 172.21.15.7:0/1950878321 >> v1:172.21.15.117:6789/0 conn(0x7f8c080c9f40 legacy=0x7f8c080c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 -- 172.21.15.7:0/1950878321 >> v1:172.21.15.7:6789/0 conn(0x7f8c08153880 legacy=0x7f8c0815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 -- 172.21.15.7:0/1950878321 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c081596e0 con 0x7f8c0815b4a0 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.751+0000 7f8c07fff700 1 --2- 172.21.15.7:0/1950878321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c0815b4a0 0x7f8c0815d4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098f0 tx=0x7f8bf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb58de2121960ef server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c077fe700 1 -- 172.21.15.7:0/1950878321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400b8a0 con 0x7f8c0815b4a0 2023-06-09T11:13:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c077fe700 1 -- 172.21.15.7:0/1950878321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf400ba00 con 0x7f8c0815b4a0 2023-06-09T11:13:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c077fe700 1 -- 172.21.15.7:0/1950878321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400bd90 con 0x7f8c0815b4a0 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/1950878321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c0815b4a0 msgr2=0x7f8c0815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/1950878321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c0815b4a0 0x7f8c0815d4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40098f0 tx=0x7f8bf4009c00 comp rx=0 tx=0).stop 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/1950878321 shutdown_connections 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/1950878321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c0815b4a0 0x7f8c0815d4c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/1950878321 >> 172.21.15.7:0/1950878321 conn(0x7f8c080737f0 msgr2=0x7f8c080720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/1950878321 shutdown_connections 2023-06-09T11:13:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.752+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/1950878321 wait complete. 2023-06-09T11:13:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.755+0000 7f8c0f1ff700 1 Processor -- start 2023-06-09T11:13:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.755+0000 7f8c0f1ff700 1 -- start start 2023-06-09T11:13:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c080c9f40 0x7f8c08151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 0x7f8c081f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081f2650 con 0x7f8c08153880 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08152610 con 0x7f8c080c9f40 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.756+0000 7f8c0f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c081f2370 con 0x7f8c0815b4a0 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:24.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42608/0 (socket says 172.21.15.7:42608) 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 -- 172.21.15.7:0/4227172578 learned_addr learned my addr 172.21.15.7:0/4227172578 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c0cf9b700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c080c9f40 0x7f8c08151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c0d79c700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 0x7f8c081f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 -- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 msgr2=0x7f8c081f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 0x7f8c081f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 -- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c080c9f40 msgr2=0x7f8c08151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c080c9f40 0x7f8c08151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 -- 172.21.15.7:0/4227172578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009550 con 0x7f8c08153880 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c0d79c700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 0x7f8c081f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c07fff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4014040 tx=0x7f8bf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40036a0 con 0x7f8c08153880 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf401e070 con 0x7f8c08153880 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4003a60 con 0x7f8c08153880 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.757+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c08159c60 con 0x7f8c08153880 2023-06-09T11:13:24.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.758+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08159980 con 0x7f8c08153880 2023-06-09T11:13:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.759+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08159f40 con 0x7f8c08153880 2023-06-09T11:13:24.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.760+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bf4003bc0 con 0x7f8c08153880 2023-06-09T11:13:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.761+0000 7f8c057fa700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 0x7f8bec06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.761+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f8bf4020070 con 0x7f8c08153880 2023-06-09T11:13:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.761+0000 7f8c0cf9b700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 0x7f8bec06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:24.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.762+0000 7f8c0cf9b700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 0x7f8bec06d1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007fd0 tx=0x7f8bfc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:24.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:24.765+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8bf4055980 con 0x7f8c08153880 2023-06-09T11:13:25.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.017+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c081f2800 con 0x7f8c08153880 2023-06-09T11:13:25.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.017+0000 7f8c057fa700 1 -- 172.21.15.7:0/4227172578 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f8c081f2800 con 0x7f8c08153880 2023-06-09T11:13:25.023 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:25.023 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:19.140592+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:25.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 msgr2=0x7f8bec06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:25.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 0x7f8bec06d1c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007fd0 tx=0x7f8bfc009040 comp rx=0 tx=0).stop 2023-06-09T11:13:25.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 msgr2=0x7f8c081e6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:25.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4014040 tx=0x7f8bf4004750 comp rx=0 tx=0).stop 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 shutdown_connections 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bec06ad10 0x7f8bec06d1c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c0815b4a0 0x7f8c081f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c080c9f40 0x7f8c08151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 --2- 172.21.15.7:0/4227172578 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c08153880 0x7f8c081e6550 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.019+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 >> 172.21.15.7:0/4227172578 conn(0x7f8c080737f0 msgr2=0x7f8c0806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:25.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.020+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 shutdown_connections 2023-06-09T11:13:25.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:25.020+0000 7f8c0f1ff700 1 -- 172.21.15.7:0/4227172578 wait complete. 2023-06-09T11:13:25.028 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 21 2023-06-09T11:13:25.043 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:27.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.249+0000 7fb5b63ff700 1 Processor -- start 2023-06-09T11:13:27.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.249+0000 7fb5b63ff700 1 -- start start 2023-06-09T11:13:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b00c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b63ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb5b006e460 0x7fb5b006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b63ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b015cac0 con 0x7fb5b006a4d0 2023-06-09T11:13:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b63ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b00c8870 con 0x7fb5b006e460 2023-06-09T11:13:27.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b00c37d0 con 0x7fb5b00c6280 2023-06-09T11:13:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5affff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b00c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5b499c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb5b006e460 0x7fb5b006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5affff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b00c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:33204/0 (socket says 172.21.15.7:33204) 2023-06-09T11:13:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.251+0000 7fb5affff700 1 -- 172.21.15.7:0/3803945127 learned_addr learned my addr 172.21.15.7:0/3803945127 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:27.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5b499c700 1 -- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 msgr2=0x7fb5b00c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5b499c700 1 --2- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b00c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5b499c700 1 -- 172.21.15.7:0/3803945127 >> v1:172.21.15.117:6789/0 conn(0x7fb5b006a4d0 legacy=0x7fb5b00737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5b499c700 1 -- 172.21.15.7:0/3803945127 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b015cd30 con 0x7fb5b006e460 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5aeffd700 1 -- 172.21.15.7:0/3803945127 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243884815 0 0) 0x7fb5b015cac0 con 0x7fb5b006a4d0 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5b499c700 1 --2- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006e460 0x7fb5b006cf30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009d90 tx=0x7fb5a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad1c4e24862e02cf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5aeffd700 1 -- 172.21.15.7:0/3803945127 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a800e070 con 0x7fb5b006e460 2023-06-09T11:13:27.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5aeffd700 1 -- 172.21.15.7:0/3803945127 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a800bb20 con 0x7fb5b006e460 2023-06-09T11:13:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.252+0000 7fb5aeffd700 1 -- 172.21.15.7:0/3803945127 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8012430 con 0x7fb5b006e460 2023-06-09T11:13:27.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 -- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006e460 msgr2=0x7fb5b006cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006e460 0x7fb5b006cf30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009d90 tx=0x7fb5a800b330 comp rx=0 tx=0).stop 2023-06-09T11:13:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 -- 172.21.15.7:0/3803945127 shutdown_connections 2023-06-09T11:13:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006e460 0x7fb5b006cf30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/3803945127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b00c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 -- 172.21.15.7:0/3803945127 >> 172.21.15.7:0/3803945127 conn(0x7fb5b006f980 msgr2=0x7fb5b00cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 -- 172.21.15.7:0/3803945127 shutdown_connections 2023-06-09T11:13:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.253+0000 7fb5b63ff700 1 -- 172.21.15.7:0/3803945127 wait complete. 2023-06-09T11:13:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.255+0000 7fb5b63ff700 1 Processor -- start 2023-06-09T11:13:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.255+0000 7fb5b63ff700 1 -- start start 2023-06-09T11:13:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006a4d0 0x7fb5b0153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b01f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b01421d0 con 0x7fb5b006e460 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b006e0b0 con 0x7fb5b006a4d0 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5b63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b01f2930 con 0x7fb5b00c6280 2023-06-09T11:13:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42612/0 (socket says 172.21.15.7:42612) 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5af7fe700 1 -- 172.21.15.7:0/2072282554 learned_addr learned my addr 172.21.15.7:0/2072282554 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.256+0000 7fb5affff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006a4d0 0x7fb5b0153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5b499c700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b01f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 -- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 msgr2=0x7fb5b01f1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b01f1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 -- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006a4d0 msgr2=0x7fb5b0153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006a4d0 0x7fb5b0153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 -- 172.21.15.7:0/2072282554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a8009a70 con 0x7fb5b006e460 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5b499c700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b01f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5af7fe700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40011e0 tx=0x7fb5a400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a40054e0 con 0x7fb5b006e460 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a4003680 con 0x7fb5b006e460 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4003a30 con 0x7fb5b006e460 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b015d2f0 con 0x7fb5b006e460 2023-06-09T11:13:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.257+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b015d010 con 0x7fb5b006e460 2023-06-09T11:13:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.258+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b015d5d0 con 0x7fb5b006e460 2023-06-09T11:13:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.259+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5a4003b90 con 0x7fb5b006e460 2023-06-09T11:13:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.260+0000 7fb5acff9700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 0x7fb59c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.260+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7fb5a4088b20 con 0x7fb5b006e460 2023-06-09T11:13:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.260+0000 7fb5affff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 0x7fb59c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.261+0000 7fb5affff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 0x7fb59c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009ac0 tx=0x7fb5a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.265+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb5a4053a40 con 0x7fb5b006e460 2023-06-09T11:13:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.522+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5b01f10b0 con 0x7fb5b006e460 2023-06-09T11:13:27.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.523+0000 7fb5acff9700 1 -- 172.21.15.7:0/2072282554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5462 (secure 0 0 0) 0x7fb5a4017020 con 0x7fb5b006e460 2023-06-09T11:13:27.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:27.529 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:25.175627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.525+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 msgr2=0x7fb59c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.525+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 0x7fb59c06d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009ac0 tx=0x7fb5a0008040 comp rx=0 tx=0).stop 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.525+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 msgr2=0x7fb5b01f2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.525+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb5a40011e0 tx=0x7fb5a400cae0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 shutdown_connections 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb59c06ace0 0x7fb59c06d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5b00c6280 0x7fb5b01f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5b006a4d0 0x7fb5b0153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 --2- 172.21.15.7:0/2072282554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5b006e460 0x7fb5b01f2070 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 >> 172.21.15.7:0/2072282554 conn(0x7fb5b006f980 msgr2=0x7fb5b015c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 shutdown_connections 2023-06-09T11:13:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:27.526+0000 7fb5b63ff700 1 -- 172.21.15.7:0/2072282554 wait complete. 2023-06-09T11:13:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2023-06-09T11:13:27.549 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:29.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:29.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.753+0000 7ff672bff700 1 Processor -- start 2023-06-09T11:13:29.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.753+0000 7ff672bff700 1 -- start start 2023-06-09T11:13:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff672bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff66c153880 0x7ff66c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff672bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff66c0c9f40 0x7ff66c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff672bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66c15a500 con 0x7ff66c15b2d0 2023-06-09T11:13:29.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff672bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c0c5f10 con 0x7ff66c153880 2023-06-09T11:13:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff672bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff66c15da00 con 0x7ff66c0c9f40 2023-06-09T11:13:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff67099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff66c153880 0x7ff66c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff67099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff66c153880 0x7ff66c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42616/0 (socket says 172.21.15.7:42616) 2023-06-09T11:13:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.754+0000 7ff67099b700 1 -- 172.21.15.7:0/3249792150 learned_addr learned my addr 172.21.15.7:0/3249792150 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:29.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff66bfff700 1 --2- 172.21.15.7:0/3249792150 >> v2:172.21.15.117:3300/0 conn(0x7ff66c0c9f40 0x7ff66c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff67099b700 1 -- 172.21.15.7:0/3249792150 >> v2:172.21.15.117:3300/0 conn(0x7ff66c0c9f40 msgr2=0x7ff66c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:29.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff67099b700 1 --2- 172.21.15.7:0/3249792150 >> v2:172.21.15.117:3300/0 conn(0x7ff66c0c9f40 0x7ff66c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff67099b700 1 -- 172.21.15.7:0/3249792150 >> v1:172.21.15.117:6789/0 conn(0x7ff66c15b2d0 legacy=0x7ff66c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff67099b700 1 -- 172.21.15.7:0/3249792150 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c1596e0 con 0x7ff66c153880 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff67099b700 1 --2- 172.21.15.7:0/3249792150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c153880 0x7ff66c15d4c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0cce30 tx=0x7ff660009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a96220e6310da3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff66b7fe700 1 -- 172.21.15.7:0/3249792150 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660011830 con 0x7ff66c153880 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff66b7fe700 1 -- 172.21.15.7:0/3249792150 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff660011990 con 0x7ff66c153880 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.755+0000 7ff66b7fe700 1 -- 172.21.15.7:0/3249792150 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660011d20 con 0x7ff66c153880 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 -- 172.21.15.7:0/3249792150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c153880 msgr2=0x7ff66c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 --2- 172.21.15.7:0/3249792150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c153880 0x7ff66c15d4c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff66c0cce30 tx=0x7ff660009bd0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 -- 172.21.15.7:0/3249792150 shutdown_connections 2023-06-09T11:13:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 --2- 172.21.15.7:0/3249792150 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c153880 0x7ff66c15d4c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 --2- 172.21.15.7:0/3249792150 >> v2:172.21.15.117:3300/0 conn(0x7ff66c0c9f40 0x7ff66c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 -- 172.21.15.7:0/3249792150 >> 172.21.15.7:0/3249792150 conn(0x7ff66c0737f0 msgr2=0x7ff66c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 -- 172.21.15.7:0/3249792150 shutdown_connections 2023-06-09T11:13:29.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.756+0000 7ff672bff700 1 -- 172.21.15.7:0/3249792150 wait complete. 2023-06-09T11:13:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.758+0000 7ff672bff700 1 Processor -- start 2023-06-09T11:13:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.758+0000 7ff672bff700 1 -- start start 2023-06-09T11:13:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff66c0c9f40 0x7ff66c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66c153880 0x7ff66c14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c0b3560 con 0x7ff66c15b2d0 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1f3470 con 0x7ff66c153880 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff672bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66c1f2860 con 0x7ff66c0c9f40 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff67119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff67119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42618/0 (socket says 172.21.15.7:42618) 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.759+0000 7ff67119c700 1 -- 172.21.15.7:0/501566113 learned_addr learned my addr 172.21.15.7:0/501566113 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:29.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff66bfff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66c153880 0x7ff66c14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67099b700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff66c0c9f40 0x7ff66c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 -- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff66c0c9f40 msgr2=0x7ff66c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff66c0c9f40 0x7ff66c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 -- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66c153880 msgr2=0x7ff66c14b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66c153880 0x7ff66c14b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 -- 172.21.15.7:0/501566113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660009550 con 0x7ff66c15b2d0 2023-06-09T11:13:29.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff67119c700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff66400cb50 tx=0x7ff66400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664003c00 con 0x7ff66c15b2d0 2023-06-09T11:13:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff664003d60 con 0x7ff66c15b2d0 2023-06-09T11:13:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66c159c60 con 0x7ff66c15b2d0 2023-06-09T11:13:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.760+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400aa20 con 0x7ff66c15b2d0 2023-06-09T11:13:29.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.761+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff66c159980 con 0x7ff66c15b2d0 2023-06-09T11:13:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.762+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c159f40 con 0x7ff66c15b2d0 2023-06-09T11:13:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.763+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff66400ab80 con 0x7ff66c15b2d0 2023-06-09T11:13:29.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.764+0000 7ff6697fa700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 0x7ff65006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:29.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.764+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7ff664089540 con 0x7ff66c15b2d0 2023-06-09T11:13:29.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.764+0000 7ff67099b700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 0x7ff65006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:29.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.765+0000 7ff67099b700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 0x7ff65006d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff660009c10 tx=0x7ff660008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:29.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:29.768+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff6640543d0 con 0x7ff66c15b2d0 2023-06-09T11:13:30.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.022+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66c15a500 con 0x7ff66c15b2d0 2023-06-09T11:13:30.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.023+0000 7ff6697fa700 1 -- 172.21.15.7:0/501566113 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5462 (secure 0 0 0) 0x7ff66c15a500 con 0x7ff66c15b2d0 2023-06-09T11:13:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:30.029 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:25.175627+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 msgr2=0x7ff65006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 0x7ff65006d1c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff660009c10 tx=0x7ff660008030 comp rx=0 tx=0).stop 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 msgr2=0x7ff66c1ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff66400cb50 tx=0x7ff66400ce60 comp rx=0 tx=0).stop 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 shutdown_connections 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff66c15b2d0 0x7ff66c1ef370 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff65006ad10 0x7ff65006d1c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff66c0c9f40 0x7ff66c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 --2- 172.21.15.7:0/501566113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff66c153880 0x7ff66c14b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 >> 172.21.15.7:0/501566113 conn(0x7ff66c0737f0 msgr2=0x7ff66c15b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 shutdown_connections 2023-06-09T11:13:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:30.025+0000 7ff672bff700 1 -- 172.21.15.7:0/501566113 wait complete. 2023-06-09T11:13:30.032 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 22 2023-06-09T11:13:30.048 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:32.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.253+0000 7f0f8ebff700 1 Processor -- start 2023-06-09T11:13:32.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.253+0000 7f0f8ebff700 1 -- start start 2023-06-09T11:13:32.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb5d0 0x7f0f880b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8ebff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f880cb9a0 0x7f0f88159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8ebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f88159b20 con 0x7f0f8815cd40 2023-06-09T11:13:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8ebff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f880cac80 con 0x7f0f880cb9a0 2023-06-09T11:13:32.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881597d0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f880cb9a0 0x7f0f88159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f880cb9a0 0x7f0f88159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42628/0 (socket says 172.21.15.7:42628) 2023-06-09T11:13:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 -- 172.21.15.7:0/2628618562 learned_addr learned my addr 172.21.15.7:0/2628618562 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:32.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f8c99b700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb5d0 0x7f0f880b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:32.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 -- 172.21.15.7:0/2628618562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb5d0 msgr2=0x7f0f880b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb5d0 0x7f0f880b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 -- 172.21.15.7:0/2628618562 >> v1:172.21.15.117:6789/0 conn(0x7f0f8815cd40 legacy=0x7f0f880cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.255+0000 7f0f87fff700 1 -- 172.21.15.7:0/2628618562 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88071ab0 con 0x7f0f880cb9a0 2023-06-09T11:13:32.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f87fff700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb9a0 0x7f0f88159fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098f0 tx=0x7f0f74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3a76160754bb33 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f877fe700 1 -- 172.21.15.7:0/2628618562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011830 con 0x7f0f880cb9a0 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f877fe700 1 -- 172.21.15.7:0/2628618562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f74011990 con 0x7f0f880cb9a0 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f877fe700 1 -- 172.21.15.7:0/2628618562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f74011d20 con 0x7f0f880cb9a0 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 -- 172.21.15.7:0/2628618562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb9a0 msgr2=0x7f0f88159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb9a0 0x7f0f88159fb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f740098f0 tx=0x7f0f74009c00 comp rx=0 tx=0).stop 2023-06-09T11:13:32.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 -- 172.21.15.7:0/2628618562 shutdown_connections 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb5d0 0x7f0f880b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/2628618562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb9a0 0x7f0f88159fb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.256+0000 7f0f8ebff700 1 -- 172.21.15.7:0/2628618562 >> 172.21.15.7:0/2628618562 conn(0x7f0f88070af0 msgr2=0x7f0f88070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.257+0000 7f0f8ebff700 1 -- 172.21.15.7:0/2628618562 shutdown_connections 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.257+0000 7f0f8ebff700 1 -- 172.21.15.7:0/2628618562 wait complete. 2023-06-09T11:13:32.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.258+0000 7f0f8ebff700 1 Processor -- start 2023-06-09T11:13:32.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.258+0000 7f0f8ebff700 1 -- start start 2023-06-09T11:13:32.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb9a0 0x7f0f881e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8815cd40 0x7f0f881f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881e5230 con 0x7f0f880cb5d0 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881e5590 con 0x7f0f8815cd40 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880c8b10 con 0x7f0f880cb9a0 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42630/0 (socket says 172.21.15.7:42630) 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.259+0000 7f0f8c99b700 1 -- 172.21.15.7:0/3211734246 learned_addr learned my addr 172.21.15.7:0/3211734246 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 -- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb9a0 msgr2=0x7f0f881e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb9a0 0x7f0f881e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 -- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8815cd40 msgr2=0x7f0f881f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8815cd40 0x7f0f881f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 -- 172.21.15.7:0/3211734246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f74009550 con 0x7f0f880cb5d0 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8c99b700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00e9e0 tx=0x7f0f7c00eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c009710 con 0x7f0f880cb5d0 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c00c850 con 0x7f0f880cb5d0 2023-06-09T11:13:32.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8815d4f0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00cbc0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.260+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8815d210 con 0x7f0f880cb5d0 2023-06-09T11:13:32.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.262+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8815d7d0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.262+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f7c00cd20 con 0x7f0f880cb5d0 2023-06-09T11:13:32.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.263+0000 7f0f857fa700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 0x7f0f6c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:32.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.264+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f0f7c089620 con 0x7f0f880cb5d0 2023-06-09T11:13:32.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.264+0000 7f0f87fff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 0x7f0f6c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:32.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.264+0000 7f0f87fff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 0x7f0f6c06d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f880cbd70 tx=0x7f0f74010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:32.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.268+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0f7c058060 con 0x7f0f880cb5d0 2023-06-09T11:13:32.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.518+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f88074cf0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.519+0000 7f0f857fa700 1 -- 172.21.15.7:0/3211734246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5464 (secure 0 0 0) 0x7f0f7c057bf0 con 0x7f0f880cb5d0 2023-06-09T11:13:32.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:32.525 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:30.187896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 msgr2=0x7f0f6c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 0x7f0f6c06d1c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0f880cbd70 tx=0x7f0f74010000 comp rx=0 tx=0).stop 2023-06-09T11:13:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 msgr2=0x7f0f881e4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:32.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00e9e0 tx=0x7f0f7c00eda0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 shutdown_connections 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f8815cd40 0x7f0f881f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f880cb5d0 0x7f0f881e4cf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f6c06ad10 0x7f0f6c06d1c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 --2- 172.21.15.7:0/3211734246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f880cb9a0 0x7f0f881e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 >> 172.21.15.7:0/3211734246 conn(0x7f0f88070af0 msgr2=0x7f0f880b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:32.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.521+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 shutdown_connections 2023-06-09T11:13:32.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:32.522+0000 7f0f8ebff700 1 -- 172.21.15.7:0/3211734246 wait complete. 2023-06-09T11:13:32.529 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2023-06-09T11:13:32.546 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 23, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 23, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4288": { "addr": "172.21.15.117:6835/4051500902", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 4051500902, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 4051500902, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4288, "incarnation": 9, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 6 }, "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:reconnect", "state_seq": 14 } }, "last_failure": 0, "last_failure_osd_epoch": 26, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:13:30.187896+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 4288, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:13:34.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:34.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.751+0000 7faa23dff700 1 Processor -- start 2023-06-09T11:13:34.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.751+0000 7faa23dff700 1 -- start start 2023-06-09T11:13:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.752+0000 7faa23dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 0x7faa1c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.752+0000 7faa23dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15a500 con 0x7faa1c153880 2023-06-09T11:13:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.752+0000 7faa23dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c15da00 con 0x7faa1c0c9f40 2023-06-09T11:13:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.752+0000 7faa23dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0c5f10 con 0x7faa1c15b4a0 2023-06-09T11:13:34.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa21b9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7faa1c153880 0x7faa1c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51170/0 (socket says 172.21.15.7:51170) 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa21b9b700 1 -- 172.21.15.7:0/2457689532 learned_addr learned my addr 172.21.15.7:0/2457689532 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa2139a700 1 --2- 172.21.15.7:0/2457689532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 0x7faa1c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132053306 0 0) 0x7faa18001110 con 0x7faa1c153880 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa080035b0 con 0x7faa1c153880 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 523351950 0 0) 0x7faa14001490 con 0x7faa1c0c9f40 2023-06-09T11:13:34.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa18001110 con 0x7faa1c0c9f40 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa2139a700 1 -- 172.21.15.7:0/2457689532 >> v1:172.21.15.117:6789/0 conn(0x7faa1c0c9f40 legacy=0x7faa1c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa2139a700 1 -- 172.21.15.7:0/2457689532 >> v1:172.21.15.7:6789/0 conn(0x7faa1c153880 legacy=0x7faa1c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.753+0000 7faa2139a700 1 -- 172.21.15.7:0/2457689532 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c1596e0 con 0x7faa1c15b4a0 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa2139a700 1 --2- 172.21.15.7:0/2457689532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 0x7faa1c15d4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009870 tx=0x7faa0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21b01f4d5cee191c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011830 con 0x7faa1c15b4a0 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c011990 con 0x7faa1c15b4a0 2023-06-09T11:13:34.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa20b99700 1 -- 172.21.15.7:0/2457689532 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0c011d20 con 0x7faa1c15b4a0 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 -- 172.21.15.7:0/2457689532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 msgr2=0x7faa1c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 --2- 172.21.15.7:0/2457689532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 0x7faa1c15d4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7faa0c009870 tx=0x7faa0c009b80 comp rx=0 tx=0).stop 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 -- 172.21.15.7:0/2457689532 shutdown_connections 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 --2- 172.21.15.7:0/2457689532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c15b4a0 0x7faa1c15d4c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 -- 172.21.15.7:0/2457689532 >> 172.21.15.7:0/2457689532 conn(0x7faa1c0737f0 msgr2=0x7faa1c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:34.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.754+0000 7faa23dff700 1 -- 172.21.15.7:0/2457689532 shutdown_connections 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.755+0000 7faa23dff700 1 -- 172.21.15.7:0/2457689532 wait complete. 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.756+0000 7faa23dff700 1 Processor -- start 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.756+0000 7faa23dff700 1 -- start start 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1c0c9f40 0x7faa1c153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c153880 0x7faa1c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:34.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c1f35b0 con 0x7faa1c15b4a0 2023-06-09T11:13:34.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c1546f0 con 0x7faa1c0c9f40 2023-06-09T11:13:34.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa23dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c1f29a0 con 0x7faa1c153880 2023-06-09T11:13:34.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.757+0000 7faa2239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:34.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2139a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c153880 0x7faa1c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:34.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43898/0 (socket says 172.21.15.7:43898) 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 -- 172.21.15.7:0/2991408580 learned_addr learned my addr 172.21.15.7:0/2991408580 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa21b9b700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1c0c9f40 0x7faa1c153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 -- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c153880 msgr2=0x7faa1c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c153880 0x7faa1c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 -- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1c0c9f40 msgr2=0x7faa1c153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1c0c9f40 0x7faa1c153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:34.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 -- 172.21.15.7:0/2991408580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c0094a0 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa2239c700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cc50 tx=0x7faa1400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14003c00 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14003d80 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c159c60 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.758+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400aad0 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.759+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c159980 con 0x7faa1c15b4a0 2023-06-09T11:13:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.760+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c159f40 con 0x7faa1c15b4a0 2023-06-09T11:13:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.761+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa1400ac30 con 0x7faa1c15b4a0 2023-06-09T11:13:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.762+0000 7faa127fc700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 0x7faa0006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.762+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7faa140894b0 con 0x7faa1c15b4a0 2023-06-09T11:13:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.762+0000 7faa21b9b700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 0x7faa0006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:34.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.762+0000 7faa21b9b700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 0x7faa0006d1c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa18007fd0 tx=0x7faa18007e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:34.766+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faa140543d0 con 0x7faa1c15b4a0 2023-06-09T11:13:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.018+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa1c15da00 con 0x7faa1c15b4a0 2023-06-09T11:13:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.019+0000 7faa127fc700 1 -- 172.21.15.7:0/2991408580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5464 (secure 0 0 0) 0x7faa1c15da00 con 0x7faa1c15b4a0 2023-06-09T11:13:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:35.025 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:30.187896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:35.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 msgr2=0x7faa0006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:35.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 0x7faa0006d1c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faa18007fd0 tx=0x7faa18007e00 comp rx=0 tx=0).stop 2023-06-09T11:13:35.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 msgr2=0x7faa1c0b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:35.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7faa1400cc50 tx=0x7faa1400cf60 comp rx=0 tx=0).stop 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 shutdown_connections 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa1c15b4a0 0x7faa1c0b3c30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa1c0c9f40 0x7faa1c153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa0006ad10 0x7faa0006d1c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 --2- 172.21.15.7:0/2991408580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa1c153880 0x7faa1c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:35.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 >> 172.21.15.7:0/2991408580 conn(0x7faa1c0737f0 msgr2=0x7faa1c14c780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:35.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 shutdown_connections 2023-06-09T11:13:35.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:35.021+0000 7faa23dff700 1 -- 172.21.15.7:0/2991408580 wait complete. 2023-06-09T11:13:35.029 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 23 2023-06-09T11:13:35.045 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:37.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.252+0000 7fea32bff700 1 Processor -- start 2023-06-09T11:13:37.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.253+0000 7fea32bff700 1 -- start start 2023-06-09T11:13:37.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.254+0000 7fea32bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fea2c1539b0 0x7fea2c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.254+0000 7fea32bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c0c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.254+0000 7fea32bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c15a420 con 0x7fea2c1535e0 2023-06-09T11:13:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.254+0000 7fea32bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c0ca670 con 0x7fea2c1539b0 2023-06-09T11:13:37.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.254+0000 7fea32bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0c5a00 con 0x7fea2c0c4ce0 2023-06-09T11:13:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fea2c1539b0 0x7fea2c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea3119c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c0c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fea2c1539b0 0x7fea2c159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:49836/0 (socket says 172.21.15.7:49836) 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea3119c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c0c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:60220/0 (socket says 172.21.15.7:60220) 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 -- 172.21.15.7:0/1464007494 learned_addr learned my addr 172.21.15.7:0/1464007494 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea3099b700 1 --1- 172.21.15.7:0/1464007494 >> v1:172.21.15.117:6789/0 conn(0x7fea2c1535e0 0x7fea2c0c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:42666/0 (socket says 172.21.15.7:42666) 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 -- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 msgr2=0x7fea2c0c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 --2- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c0c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 -- 172.21.15.7:0/1464007494 >> v1:172.21.15.117:6789/0 conn(0x7fea2c1535e0 legacy=0x7fea2c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 -- 172.21.15.7:0/1464007494 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c06b460 con 0x7fea2c1539b0 2023-06-09T11:13:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2b7fe700 1 -- 172.21.15.7:0/1464007494 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3831733458 0 0) 0x7fea20001110 con 0x7fea2c1535e0 2023-06-09T11:13:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.255+0000 7fea2bfff700 1 --2- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1539b0 0x7fea2c159ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea180098f0 tx=0x7fea18009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fcb59271f691950 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea2b7fe700 1 -- 172.21.15.7:0/1464007494 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011830 con 0x7fea2c1539b0 2023-06-09T11:13:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea2b7fe700 1 -- 172.21.15.7:0/1464007494 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18011990 con 0x7fea2c1539b0 2023-06-09T11:13:37.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea2b7fe700 1 -- 172.21.15.7:0/1464007494 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18011d20 con 0x7fea2c1539b0 2023-06-09T11:13:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 -- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1539b0 msgr2=0x7fea2c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 --2- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1539b0 0x7fea2c159ee0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fea180098f0 tx=0x7fea18009c00 comp rx=0 tx=0).stop 2023-06-09T11:13:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 -- 172.21.15.7:0/1464007494 shutdown_connections 2023-06-09T11:13:37.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 --2- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c0c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 --2- 172.21.15.7:0/1464007494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1539b0 0x7fea2c159ee0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 -- 172.21.15.7:0/1464007494 >> 172.21.15.7:0/1464007494 conn(0x7fea2c0737f0 msgr2=0x7fea2c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.256+0000 7fea32bff700 1 -- 172.21.15.7:0/1464007494 shutdown_connections 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.257+0000 7fea32bff700 1 -- 172.21.15.7:0/1464007494 wait complete. 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.258+0000 7fea32bff700 1 Processor -- start 2023-06-09T11:13:37.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.258+0000 7fea32bff700 1 -- start start 2023-06-09T11:13:37.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c1f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1535e0 0x7fea2c1f05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0c89e0 con 0x7fea2c1539b0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0c84a0 con 0x7fea2c1535e0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.259+0000 7fea32bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0c72f0 con 0x7fea2c0c4ce0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43910/0 (socket says 172.21.15.7:43910) 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 -- 172.21.15.7:0/4006631844 learned_addr learned my addr 172.21.15.7:0/4006631844 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3099b700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c1f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea2bfff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1535e0 0x7fea2c1f05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 -- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 msgr2=0x7fea2c1f00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c1f00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 -- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1535e0 msgr2=0x7fea2c1f05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1535e0 0x7fea2c1f05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 -- 172.21.15.7:0/4006631844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18009550 con 0x7fea2c1539b0 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.260+0000 7fea3119c700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea2400efc0 tx=0x7fea2400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.261+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24009ea0 con 0x7fea2c1539b0 2023-06-09T11:13:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.261+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2400f040 con 0x7fea2c1539b0 2023-06-09T11:13:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.261+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2400ced0 con 0x7fea2c1539b0 2023-06-09T11:13:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.261+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c06a850 con 0x7fea2c1539b0 2023-06-09T11:13:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.261+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c06a570 con 0x7fea2c1539b0 2023-06-09T11:13:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.262+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2c06ab30 con 0x7fea2c1539b0 2023-06-09T11:13:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.263+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fea24014480 con 0x7fea2c1539b0 2023-06-09T11:13:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.265+0000 7fea297fa700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 0x7fea1006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.265+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7fea24089ff0 con 0x7fea2c1539b0 2023-06-09T11:13:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.265+0000 7fea3099b700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 0x7fea1006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:37.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.266+0000 7fea3099b700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 0x7fea1006d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea20000f60 tx=0x7fea20007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:37.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.269+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fea240bc2d0 con 0x7fea2c1539b0 2023-06-09T11:13:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.523+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea2c074560 con 0x7fea2c1539b0 2023-06-09T11:13:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.523+0000 7fea297fa700 1 -- 172.21.15.7:0/4006631844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5461 (secure 0 0 0) 0x7fea24018090 con 0x7fea2c1539b0 2023-06-09T11:13:37.529 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:37.530 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:35.212004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 msgr2=0x7fea1006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 0x7fea1006d1c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fea20000f60 tx=0x7fea20007c20 comp rx=0 tx=0).stop 2023-06-09T11:13:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 msgr2=0x7fea2c156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fea2400efc0 tx=0x7fea2400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 shutdown_connections 2023-06-09T11:13:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea2c1539b0 0x7fea2c156440 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fea1006ad10 0x7fea1006d1c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fea2c0c4ce0 0x7fea2c1f00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 --2- 172.21.15.7:0/4006631844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fea2c1535e0 0x7fea2c1f05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 >> 172.21.15.7:0/4006631844 conn(0x7fea2c0737f0 msgr2=0x7fea2c0b4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 shutdown_connections 2023-06-09T11:13:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:37.526+0000 7fea32bff700 1 -- 172.21.15.7:0/4006631844 wait complete. 2023-06-09T11:13:37.535 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2023-06-09T11:13:37.549 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:39.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.755+0000 7f52679ff700 1 Processor -- start 2023-06-09T11:13:39.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.756+0000 7f52679ff700 1 -- start start 2023-06-09T11:13:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f52679ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52600c8e90 0x7f526015d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f52679ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f52600c9260 0x7f52600cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f52679ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f526015dc90 con 0x7f52600b5370 2023-06-09T11:13:39.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f52679ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52601597d0 con 0x7f52600c8e90 2023-06-09T11:13:39.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f52679ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52600cfaf0 con 0x7f52600c9260 2023-06-09T11:13:39.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52600c8e90 0x7f526015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f52600c8e90 0x7f526015d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43924/0 (socket says 172.21.15.7:43924) 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 -- 172.21.15.7:0/3112044037 learned_addr learned my addr 172.21.15.7:0/3112044037 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f5264f9a700 1 --2- 172.21.15.7:0/3112044037 >> v2:172.21.15.117:3300/0 conn(0x7f52600c9260 0x7f52600cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 -- 172.21.15.7:0/3112044037 >> v2:172.21.15.117:3300/0 conn(0x7f52600c9260 msgr2=0x7f52600cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 --2- 172.21.15.7:0/3112044037 >> v2:172.21.15.117:3300/0 conn(0x7f52600c9260 0x7f52600cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 -- 172.21.15.7:0/3112044037 >> v1:172.21.15.117:6789/0 conn(0x7f52600b5370 legacy=0x7f52600c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.757+0000 7f526579b700 1 -- 172.21.15.7:0/3112044037 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600cfdd0 con 0x7f52600c8e90 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f526579b700 1 --2- 172.21.15.7:0/3112044037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c8e90 0x7f526015d750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52600c3160 tx=0x7f525c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f8496bd1b0a2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f5257fff700 1 -- 172.21.15.7:0/3112044037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c011830 con 0x7f52600c8e90 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f5257fff700 1 -- 172.21.15.7:0/3112044037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f525c011990 con 0x7f52600c8e90 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f5257fff700 1 -- 172.21.15.7:0/3112044037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525c011d20 con 0x7f52600c8e90 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f52679ff700 1 -- 172.21.15.7:0/3112044037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c8e90 msgr2=0x7f526015d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.758+0000 7f52679ff700 1 --2- 172.21.15.7:0/3112044037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c8e90 0x7f526015d750 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52600c3160 tx=0x7f525c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 -- 172.21.15.7:0/3112044037 shutdown_connections 2023-06-09T11:13:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 --2- 172.21.15.7:0/3112044037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c8e90 0x7f526015d750 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 --2- 172.21.15.7:0/3112044037 >> v2:172.21.15.117:3300/0 conn(0x7f52600c9260 0x7f52600cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 -- 172.21.15.7:0/3112044037 >> 172.21.15.7:0/3112044037 conn(0x7f5260070af0 msgr2=0x7f5260070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 -- 172.21.15.7:0/3112044037 shutdown_connections 2023-06-09T11:13:39.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.759+0000 7f52679ff700 1 -- 172.21.15.7:0/3112044037 wait complete. 2023-06-09T11:13:39.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.760+0000 7f52679ff700 1 Processor -- start 2023-06-09T11:13:39.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.761+0000 7f52679ff700 1 -- start start 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.761+0000 7f52679ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b5370 0x7f52601f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f52679ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c8e90 0x7f52600c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f52679ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f52679ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52601f28a0 con 0x7f52600c9260 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f52679ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526015bca0 con 0x7f52600b5370 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f52679ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52601f3270 con 0x7f52600c8e90 2023-06-09T11:13:39.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f526579b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b5370 0x7f52601f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43940/0 (socket says 172.21.15.7:43940) 2023-06-09T11:13:39.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 -- 172.21.15.7:0/2041952785 learned_addr learned my addr 172.21.15.7:0/2041952785 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:39.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5264f9a700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c8e90 0x7f52600c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 -- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c8e90 msgr2=0x7f52600c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c8e90 0x7f52600c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 -- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b5370 msgr2=0x7f52601f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b5370 0x7f52601f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.762+0000 7f5265f9c700 1 -- 172.21.15.7:0/2041952785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c009550 con 0x7f52600c9260 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f5265f9c700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f525800cbc0 tx=0x7f525800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:39.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258003c00 con 0x7f52600c9260 2023-06-09T11:13:39.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5258003d60 con 0x7f52600c9260 2023-06-09T11:13:39.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f525800aae0 con 0x7f52600c9260 2023-06-09T11:13:39.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5260159ca0 con 0x7f52600c9260 2023-06-09T11:13:39.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.763+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52601599c0 con 0x7f52600c9260 2023-06-09T11:13:39.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.764+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5260159f80 con 0x7f52600c9260 2023-06-09T11:13:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.765+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f525800ac40 con 0x7f52600c9260 2023-06-09T11:13:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.766+0000 7f5255ffb700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 0x7f524406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.766+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f5258089410 con 0x7f52600c9260 2023-06-09T11:13:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.766+0000 7f526579b700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 0x7f524406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:39.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.767+0000 7f526579b700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 0x7f524406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f525c009c10 tx=0x7f525c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:39.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:39.770+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52580542a0 con 0x7f52600c9260 2023-06-09T11:13:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.023+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f526015dc90 con 0x7f52600c9260 2023-06-09T11:13:40.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.024+0000 7f5255ffb700 1 -- 172.21.15.7:0/2041952785 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+5461 (secure 0 0 0) 0x7f526015dc90 con 0x7f52600c9260 2023-06-09T11:13:40.029 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:40.030 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:35.212004+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:40.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 msgr2=0x7f524406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 0x7f524406d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f525c009c10 tx=0x7f525c008030 comp rx=0 tx=0).stop 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 msgr2=0x7f52600c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f525800cbc0 tx=0x7f525800ced0 comp rx=0 tx=0).stop 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 shutdown_connections 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52600c9260 0x7f52600c5850 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:40.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52600b5370 0x7f52601f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f524406ace0 0x7f524406d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 --2- 172.21.15.7:0/2041952785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52600c8e90 0x7f52600c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 >> 172.21.15.7:0/2041952785 conn(0x7f5260070af0 msgr2=0x7f52600b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 shutdown_connections 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:40.026+0000 7f52679ff700 1 -- 172.21.15.7:0/2041952785 wait complete. 2023-06-09T11:13:40.033 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 24 2023-06-09T11:13:40.049 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:13:40.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:40.640 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:13:42.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.256+0000 7fd5e9fff700 1 Processor -- start 2023-06-09T11:13:42.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.257+0000 7fd5e9fff700 1 -- start start 2023-06-09T11:13:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e9fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5e40cb5d0 0x7fd5e40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e9fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5e415cd40 0x7fd5e40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e9fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e4159b20 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e9fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e40cac80 con 0x7fd5e415cd40 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e9fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e41597d0 con 0x7fd5e40cb5d0 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5e415cd40 0x7fd5e40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd5e415cd40 0x7fd5e40cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43948/0 (socket says 172.21.15.7:43948) 2023-06-09T11:13:42.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 -- 172.21.15.7:0/4163025543 learned_addr learned my addr 172.21.15.7:0/4163025543 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e37fe700 1 --2- 172.21.15.7:0/4163025543 >> v2:172.21.15.117:3300/0 conn(0x7fd5e40cb5d0 0x7fd5e40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 -- 172.21.15.7:0/4163025543 >> v2:172.21.15.117:3300/0 conn(0x7fd5e40cb5d0 msgr2=0x7fd5e40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 --2- 172.21.15.7:0/4163025543 >> v2:172.21.15.117:3300/0 conn(0x7fd5e40cb5d0 0x7fd5e40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 -- 172.21.15.7:0/4163025543 >> v1:172.21.15.117:6789/0 conn(0x7fd5e40cb9a0 legacy=0x7fd5e4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.258+0000 7fd5e3fff700 1 -- 172.21.15.7:0/4163025543 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4071ab0 con 0x7fd5e415cd40 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e3fff700 1 --2- 172.21.15.7:0/4163025543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e415cd40 0x7fd5e40cbea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd5e408d420 tx=0x7fd5dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f392079423c9cc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:42.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e27fc700 1 -- 172.21.15.7:0/4163025543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc00e070 con 0x7fd5e415cd40 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e27fc700 1 -- 172.21.15.7:0/4163025543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5dc003a20 con 0x7fd5e415cd40 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e27fc700 1 -- 172.21.15.7:0/4163025543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc003dd0 con 0x7fd5e415cd40 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e9fff700 1 -- 172.21.15.7:0/4163025543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e415cd40 msgr2=0x7fd5e40cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.259+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/4163025543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e415cd40 0x7fd5e40cbea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd5e408d420 tx=0x7fd5dc00b330 comp rx=0 tx=0).stop 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 -- 172.21.15.7:0/4163025543 shutdown_connections 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/4163025543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e415cd40 0x7fd5e40cbea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/4163025543 >> v2:172.21.15.117:3300/0 conn(0x7fd5e40cb5d0 0x7fd5e40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 -- 172.21.15.7:0/4163025543 >> 172.21.15.7:0/4163025543 conn(0x7fd5e4070af0 msgr2=0x7fd5e4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:42.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 -- 172.21.15.7:0/4163025543 shutdown_connections 2023-06-09T11:13:42.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.260+0000 7fd5e9fff700 1 -- 172.21.15.7:0/4163025543 wait complete. 2023-06-09T11:13:42.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.261+0000 7fd5e9fff700 1 Processor -- start 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.262+0000 7fd5e9fff700 1 -- start start 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.262+0000 7fd5e9fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40cb5d0 0x7fd5e40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.262+0000 7fd5e9fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e9fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e415cd40 0x7fd5e40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e9fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c3aa0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e9fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e41e6930 con 0x7fd5e40cb5d0 2023-06-09T11:13:42.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e9fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e40c3190 con 0x7fd5e415cd40 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43954/0 (socket says 172.21.15.7:43954) 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 -- 172.21.15.7:0/3511099257 learned_addr learned my addr 172.21.15.7:0/3511099257 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e3fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e415cd40 0x7fd5e40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e37fe700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40cb5d0 0x7fd5e40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 -- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e415cd40 msgr2=0x7fd5e40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e415cd40 0x7fd5e40c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 -- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40cb5d0 msgr2=0x7fd5e40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40cb5d0 0x7fd5e40c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 -- 172.21.15.7:0/3511099257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5dc009b30 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.263+0000 7fd5e2ffd700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c4d0 tx=0x7fd5d800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.264+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80054e0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.264+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d800ce40 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.264+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e415d4f0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.264+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80038a0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.264+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e415d210 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.265+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e415d7d0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.266+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd5d8003a00 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.267+0000 7fd5cbfff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.267+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7fd5d8088490 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.267+0000 7fd5e37fe700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.268+0000 7fd5e37fe700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 0x7fd5d006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006b20 tx=0x7fd5d4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:42.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.272+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd5d80534f0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.523+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5e41f1bc0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.524+0000 7fd5cbfff700 1 -- 172.21.15.7:0/3511099257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5461 (secure 0 0 0) 0x7fd5d8003cb0 con 0x7fd5e40cb9a0 2023-06-09T11:13:42.530 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:42.530 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 msgr2=0x7fd5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 0x7fd5d006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4006b20 tx=0x7fd5d4008040 comp rx=0 tx=0).stop 2023-06-09T11:13:42.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 msgr2=0x7fd5e4153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800c4d0 tx=0x7fd5d800c7e0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 shutdown_connections 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd5d006ace0 0x7fd5d006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5e415cd40 0x7fd5e40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5e40cb5d0 0x7fd5e40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 --2- 172.21.15.7:0/3511099257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5e40cb9a0 0x7fd5e4153460 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:42.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.526+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 >> 172.21.15.7:0/3511099257 conn(0x7fd5e4070af0 msgr2=0x7fd5e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.527+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 shutdown_connections 2023-06-09T11:13:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:42.527+0000 7fd5e9fff700 1 -- 172.21.15.7:0/3511099257 wait complete. 2023-06-09T11:13:42.534 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2023-06-09T11:13:42.550 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=4263 2023-06-09T11:13:42.550 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.e 2023-06-09T11:13:44.801 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:13:44.797+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:13:47.034 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/30 file after the snapshot 2023-06-09T11:13:47.045 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/31 file after the snapshot 2023-06-09T11:13:47.056 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/32 file after the snapshot 2023-06-09T11:13:47.076 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/33 file after the snapshot 2023-06-09T11:13:47.091 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/34 file after the snapshot 2023-06-09T11:13:47.105 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/35 file after the snapshot 2023-06-09T11:13:47.122 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/36 file after the snapshot 2023-06-09T11:13:47.132 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/37 file after the snapshot 2023-06-09T11:13:47.145 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/38 file after the snapshot 2023-06-09T11:13:47.156 INFO:tasks.workunit.client.0.smithi007.stdout: create 176/39 file after the snapshot 2023-06-09T11:13:47.169 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/.snap/snap-subdir-test 2023-06-09T11:13:47.195 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/30 file after the snapshot 2023-06-09T11:13:47.197 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/31 file after the snapshot 2023-06-09T11:13:47.200 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/32 file after the snapshot 2023-06-09T11:13:47.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/33 file after the snapshot 2023-06-09T11:13:47.205 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/34 file after the snapshot 2023-06-09T11:13:47.208 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/35 file after the snapshot 2023-06-09T11:13:47.211 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/36 file after the snapshot 2023-06-09T11:13:47.214 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/37 file after the snapshot 2023-06-09T11:13:47.216 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/38 file after the snapshot 2023-06-09T11:13:47.219 INFO:tasks.workunit.client.0.smithi007.stdout: create 177/39 file after the snapshot 2023-06-09T11:13:47.221 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/.snap/snap-subdir-test 2023-06-09T11:13:47.243 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/30 file after the snapshot 2023-06-09T11:13:47.250 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/31 file after the snapshot 2023-06-09T11:13:47.262 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/32 file after the snapshot 2023-06-09T11:13:47.272 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/33 file after the snapshot 2023-06-09T11:13:47.284 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/34 file after the snapshot 2023-06-09T11:13:47.295 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/35 file after the snapshot 2023-06-09T11:13:47.305 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/36 file after the snapshot 2023-06-09T11:13:47.316 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/37 file after the snapshot 2023-06-09T11:13:47.326 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/38 file after the snapshot 2023-06-09T11:13:47.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 178/39 file after the snapshot 2023-06-09T11:13:47.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/.snap/snap-subdir-test 2023-06-09T11:13:47.367 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/30 file after the snapshot 2023-06-09T11:13:47.370 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/31 file after the snapshot 2023-06-09T11:13:47.373 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/32 file after the snapshot 2023-06-09T11:13:47.377 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/33 file after the snapshot 2023-06-09T11:13:47.380 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/34 file after the snapshot 2023-06-09T11:13:47.384 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/35 file after the snapshot 2023-06-09T11:13:47.387 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/36 file after the snapshot 2023-06-09T11:13:47.391 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/37 file after the snapshot 2023-06-09T11:13:47.394 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/38 file after the snapshot 2023-06-09T11:13:47.398 INFO:tasks.workunit.client.0.smithi007.stdout: create 179/39 file after the snapshot 2023-06-09T11:13:47.401 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/.snap/snap-subdir-test 2023-06-09T11:13:47.423 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/30 file after the snapshot 2023-06-09T11:13:47.427 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/31 file after the snapshot 2023-06-09T11:13:47.430 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/32 file after the snapshot 2023-06-09T11:13:47.433 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/33 file after the snapshot 2023-06-09T11:13:47.437 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/34 file after the snapshot 2023-06-09T11:13:47.440 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/35 file after the snapshot 2023-06-09T11:13:47.443 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/36 file after the snapshot 2023-06-09T11:13:47.446 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/37 file after the snapshot 2023-06-09T11:13:47.449 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/38 file after the snapshot 2023-06-09T11:13:47.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 180/39 file after the snapshot 2023-06-09T11:13:47.456 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/.snap/snap-subdir-test 2023-06-09T11:13:47.477 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/30 file after the snapshot 2023-06-09T11:13:47.480 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/31 file after the snapshot 2023-06-09T11:13:47.483 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/32 file after the snapshot 2023-06-09T11:13:47.486 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/33 file after the snapshot 2023-06-09T11:13:47.490 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/34 file after the snapshot 2023-06-09T11:13:47.493 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/35 file after the snapshot 2023-06-09T11:13:47.496 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/36 file after the snapshot 2023-06-09T11:13:47.500 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/37 file after the snapshot 2023-06-09T11:13:47.504 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/38 file after the snapshot 2023-06-09T11:13:47.507 INFO:tasks.workunit.client.0.smithi007.stdout: create 181/39 file after the snapshot 2023-06-09T11:13:47.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/.snap/snap-subdir-test 2023-06-09T11:13:47.532 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/30 file after the snapshot 2023-06-09T11:13:47.535 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/31 file after the snapshot 2023-06-09T11:13:47.538 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/32 file after the snapshot 2023-06-09T11:13:47.541 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/33 file after the snapshot 2023-06-09T11:13:47.544 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/34 file after the snapshot 2023-06-09T11:13:47.547 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/35 file after the snapshot 2023-06-09T11:13:47.551 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/36 file after the snapshot 2023-06-09T11:13:47.554 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/37 file after the snapshot 2023-06-09T11:13:47.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/38 file after the snapshot 2023-06-09T11:13:47.561 INFO:tasks.workunit.client.0.smithi007.stdout: create 182/39 file after the snapshot 2023-06-09T11:13:47.565 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/.snap/snap-subdir-test 2023-06-09T11:13:47.826 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/30 file after the snapshot 2023-06-09T11:13:47.830 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/31 file after the snapshot 2023-06-09T11:13:47.833 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/32 file after the snapshot 2023-06-09T11:13:47.837 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/33 file after the snapshot 2023-06-09T11:13:47.840 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/34 file after the snapshot 2023-06-09T11:13:47.844 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/35 file after the snapshot 2023-06-09T11:13:47.848 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/36 file after the snapshot 2023-06-09T11:13:47.853 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/37 file after the snapshot 2023-06-09T11:13:47.857 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/38 file after the snapshot 2023-06-09T11:13:47.861 INFO:tasks.workunit.client.0.smithi007.stdout: create 183/39 file after the snapshot 2023-06-09T11:13:47.865 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/.snap/snap-subdir-test 2023-06-09T11:13:47.888 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/30 file after the snapshot 2023-06-09T11:13:47.892 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/31 file after the snapshot 2023-06-09T11:13:47.895 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/32 file after the snapshot 2023-06-09T11:13:47.899 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/33 file after the snapshot 2023-06-09T11:13:47.902 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/34 file after the snapshot 2023-06-09T11:13:47.906 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/35 file after the snapshot 2023-06-09T11:13:47.910 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/36 file after the snapshot 2023-06-09T11:13:47.913 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/37 file after the snapshot 2023-06-09T11:13:47.916 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/38 file after the snapshot 2023-06-09T11:13:47.920 INFO:tasks.workunit.client.0.smithi007.stdout: create 184/39 file after the snapshot 2023-06-09T11:13:47.923 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/.snap/snap-subdir-test 2023-06-09T11:13:47.944 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/30 file after the snapshot 2023-06-09T11:13:47.946 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/31 file after the snapshot 2023-06-09T11:13:47.948 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/32 file after the snapshot 2023-06-09T11:13:47.950 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/33 file after the snapshot 2023-06-09T11:13:47.953 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/34 file after the snapshot 2023-06-09T11:13:47.955 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/35 file after the snapshot 2023-06-09T11:13:47.957 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/36 file after the snapshot 2023-06-09T11:13:47.960 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/37 file after the snapshot 2023-06-09T11:13:47.962 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/38 file after the snapshot 2023-06-09T11:13:47.964 INFO:tasks.workunit.client.0.smithi007.stdout: create 185/39 file after the snapshot 2023-06-09T11:13:47.966 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/.snap/snap-subdir-test 2023-06-09T11:13:47.987 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/30 file after the snapshot 2023-06-09T11:13:47.990 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/31 file after the snapshot 2023-06-09T11:13:47.993 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/32 file after the snapshot 2023-06-09T11:13:47.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/33 file after the snapshot 2023-06-09T11:13:48.000 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/34 file after the snapshot 2023-06-09T11:13:48.003 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/35 file after the snapshot 2023-06-09T11:13:48.007 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/36 file after the snapshot 2023-06-09T11:13:48.965 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/37 file after the snapshot 2023-06-09T11:13:48.969 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/38 file after the snapshot 2023-06-09T11:13:48.973 INFO:tasks.workunit.client.0.smithi007.stdout: create 186/39 file after the snapshot 2023-06-09T11:13:48.978 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/.snap/snap-subdir-test 2023-06-09T11:13:49.005 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/30 file after the snapshot 2023-06-09T11:13:49.008 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/31 file after the snapshot 2023-06-09T11:13:49.010 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/32 file after the snapshot 2023-06-09T11:13:49.013 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/33 file after the snapshot 2023-06-09T11:13:49.016 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/34 file after the snapshot 2023-06-09T11:13:49.019 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/35 file after the snapshot 2023-06-09T11:13:49.022 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/36 file after the snapshot 2023-06-09T11:13:49.025 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/37 file after the snapshot 2023-06-09T11:13:49.028 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/38 file after the snapshot 2023-06-09T11:13:49.030 INFO:tasks.workunit.client.0.smithi007.stdout: create 187/39 file after the snapshot 2023-06-09T11:13:49.033 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/.snap/snap-subdir-test 2023-06-09T11:13:49.055 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/30 file after the snapshot 2023-06-09T11:13:49.059 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/31 file after the snapshot 2023-06-09T11:13:49.064 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/32 file after the snapshot 2023-06-09T11:13:49.067 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/33 file after the snapshot 2023-06-09T11:13:49.071 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/34 file after the snapshot 2023-06-09T11:13:49.075 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/35 file after the snapshot 2023-06-09T11:13:49.079 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/36 file after the snapshot 2023-06-09T11:13:49.083 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/37 file after the snapshot 2023-06-09T11:13:49.088 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/38 file after the snapshot 2023-06-09T11:13:49.091 INFO:tasks.workunit.client.0.smithi007.stdout: create 188/39 file after the snapshot 2023-06-09T11:13:49.096 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/.snap/snap-subdir-test 2023-06-09T11:13:49.118 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/30 file after the snapshot 2023-06-09T11:13:49.121 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/31 file after the snapshot 2023-06-09T11:13:49.124 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/32 file after the snapshot 2023-06-09T11:13:49.126 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/33 file after the snapshot 2023-06-09T11:13:49.129 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/34 file after the snapshot 2023-06-09T11:13:49.132 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/35 file after the snapshot 2023-06-09T11:13:49.134 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/36 file after the snapshot 2023-06-09T11:13:49.137 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/37 file after the snapshot 2023-06-09T11:13:49.139 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/38 file after the snapshot 2023-06-09T11:13:49.141 INFO:tasks.workunit.client.0.smithi007.stdout: create 189/39 file after the snapshot 2023-06-09T11:13:49.144 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/.snap/snap-subdir-test 2023-06-09T11:13:49.168 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/30 file after the snapshot 2023-06-09T11:13:49.171 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/31 file after the snapshot 2023-06-09T11:13:49.174 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/32 file after the snapshot 2023-06-09T11:13:49.178 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/33 file after the snapshot 2023-06-09T11:13:49.182 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/34 file after the snapshot 2023-06-09T11:13:49.186 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/35 file after the snapshot 2023-06-09T11:13:49.189 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/36 file after the snapshot 2023-06-09T11:13:49.193 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/37 file after the snapshot 2023-06-09T11:13:49.197 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/38 file after the snapshot 2023-06-09T11:13:49.201 INFO:tasks.workunit.client.0.smithi007.stdout: create 190/39 file after the snapshot 2023-06-09T11:13:49.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/.snap/snap-subdir-test 2023-06-09T11:13:49.225 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/30 file after the snapshot 2023-06-09T11:13:49.228 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/31 file after the snapshot 2023-06-09T11:13:49.232 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/32 file after the snapshot 2023-06-09T11:13:49.236 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/33 file after the snapshot 2023-06-09T11:13:49.239 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/34 file after the snapshot 2023-06-09T11:13:49.243 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/35 file after the snapshot 2023-06-09T11:13:49.248 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/36 file after the snapshot 2023-06-09T11:13:49.252 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/37 file after the snapshot 2023-06-09T11:13:49.255 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/38 file after the snapshot 2023-06-09T11:13:49.258 INFO:tasks.workunit.client.0.smithi007.stdout: create 191/39 file after the snapshot 2023-06-09T11:13:49.262 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/.snap/snap-subdir-test 2023-06-09T11:13:49.284 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/30 file after the snapshot 2023-06-09T11:13:49.288 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/31 file after the snapshot 2023-06-09T11:13:49.293 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/32 file after the snapshot 2023-06-09T11:13:49.296 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/33 file after the snapshot 2023-06-09T11:13:49.299 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/34 file after the snapshot 2023-06-09T11:13:49.303 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/35 file after the snapshot 2023-06-09T11:13:49.306 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/36 file after the snapshot 2023-06-09T11:13:49.310 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/37 file after the snapshot 2023-06-09T11:13:49.313 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/38 file after the snapshot 2023-06-09T11:13:49.316 INFO:tasks.workunit.client.0.smithi007.stdout: create 192/39 file after the snapshot 2023-06-09T11:13:49.320 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/.snap/snap-subdir-test 2023-06-09T11:13:49.342 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/30 file after the snapshot 2023-06-09T11:13:49.346 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/31 file after the snapshot 2023-06-09T11:13:49.349 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/32 file after the snapshot 2023-06-09T11:13:49.352 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/33 file after the snapshot 2023-06-09T11:13:49.355 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/34 file after the snapshot 2023-06-09T11:13:49.359 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/35 file after the snapshot 2023-06-09T11:13:49.363 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/36 file after the snapshot 2023-06-09T11:13:49.366 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/37 file after the snapshot 2023-06-09T11:13:49.369 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/38 file after the snapshot 2023-06-09T11:13:49.373 INFO:tasks.workunit.client.0.smithi007.stdout: create 193/39 file after the snapshot 2023-06-09T11:13:49.376 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/.snap/snap-subdir-test 2023-06-09T11:13:49.396 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/30 file after the snapshot 2023-06-09T11:13:49.398 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/31 file after the snapshot 2023-06-09T11:13:49.401 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/32 file after the snapshot 2023-06-09T11:13:49.403 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/33 file after the snapshot 2023-06-09T11:13:49.405 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/34 file after the snapshot 2023-06-09T11:13:49.408 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/35 file after the snapshot 2023-06-09T11:13:49.410 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/36 file after the snapshot 2023-06-09T11:13:49.412 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/37 file after the snapshot 2023-06-09T11:13:49.414 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/38 file after the snapshot 2023-06-09T11:13:49.416 INFO:tasks.workunit.client.0.smithi007.stdout: create 194/39 file after the snapshot 2023-06-09T11:13:49.419 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/.snap/snap-subdir-test 2023-06-09T11:13:50.168 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/30 file after the snapshot 2023-06-09T11:13:50.172 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/31 file after the snapshot 2023-06-09T11:13:50.177 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/32 file after the snapshot 2023-06-09T11:13:50.180 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/33 file after the snapshot 2023-06-09T11:13:50.184 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/34 file after the snapshot 2023-06-09T11:13:50.190 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/35 file after the snapshot 2023-06-09T11:13:50.195 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/36 file after the snapshot 2023-06-09T11:13:50.199 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/37 file after the snapshot 2023-06-09T11:13:50.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/38 file after the snapshot 2023-06-09T11:13:50.206 INFO:tasks.workunit.client.0.smithi007.stdout: create 195/39 file after the snapshot 2023-06-09T11:13:50.211 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/.snap/snap-subdir-test 2023-06-09T11:13:50.235 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/30 file after the snapshot 2023-06-09T11:13:50.239 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/31 file after the snapshot 2023-06-09T11:13:50.243 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/32 file after the snapshot 2023-06-09T11:13:50.247 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/33 file after the snapshot 2023-06-09T11:13:50.250 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/34 file after the snapshot 2023-06-09T11:13:50.254 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/35 file after the snapshot 2023-06-09T11:13:50.258 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/36 file after the snapshot 2023-06-09T11:13:50.262 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/37 file after the snapshot 2023-06-09T11:13:50.266 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/38 file after the snapshot 2023-06-09T11:13:50.270 INFO:tasks.workunit.client.0.smithi007.stdout: create 196/39 file after the snapshot 2023-06-09T11:13:50.274 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/.snap/snap-subdir-test 2023-06-09T11:13:50.298 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/30 file after the snapshot 2023-06-09T11:13:50.301 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/31 file after the snapshot 2023-06-09T11:13:50.305 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/32 file after the snapshot 2023-06-09T11:13:50.308 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/33 file after the snapshot 2023-06-09T11:13:50.312 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/34 file after the snapshot 2023-06-09T11:13:50.315 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/35 file after the snapshot 2023-06-09T11:13:50.319 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/36 file after the snapshot 2023-06-09T11:13:50.322 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/37 file after the snapshot 2023-06-09T11:13:50.326 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/38 file after the snapshot 2023-06-09T11:13:50.329 INFO:tasks.workunit.client.0.smithi007.stdout: create 197/39 file after the snapshot 2023-06-09T11:13:50.333 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/.snap/snap-subdir-test 2023-06-09T11:13:50.355 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/30 file after the snapshot 2023-06-09T11:13:50.358 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/31 file after the snapshot 2023-06-09T11:13:50.360 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/32 file after the snapshot 2023-06-09T11:13:50.363 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/33 file after the snapshot 2023-06-09T11:13:50.366 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/34 file after the snapshot 2023-06-09T11:13:50.368 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/35 file after the snapshot 2023-06-09T11:13:50.370 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/36 file after the snapshot 2023-06-09T11:13:50.373 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/37 file after the snapshot 2023-06-09T11:13:50.375 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/38 file after the snapshot 2023-06-09T11:13:50.377 INFO:tasks.workunit.client.0.smithi007.stdout: create 198/39 file after the snapshot 2023-06-09T11:13:50.380 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/.snap/snap-subdir-test 2023-06-09T11:13:50.398 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/30 file after the snapshot 2023-06-09T11:13:50.401 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/31 file after the snapshot 2023-06-09T11:13:50.403 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/32 file after the snapshot 2023-06-09T11:13:50.406 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/33 file after the snapshot 2023-06-09T11:13:50.408 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/34 file after the snapshot 2023-06-09T11:13:50.411 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/35 file after the snapshot 2023-06-09T11:13:50.413 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/36 file after the snapshot 2023-06-09T11:13:50.416 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/37 file after the snapshot 2023-06-09T11:13:50.418 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/38 file after the snapshot 2023-06-09T11:13:50.421 INFO:tasks.workunit.client.0.smithi007.stdout: create 199/39 file after the snapshot 2023-06-09T11:13:50.423 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/.snap/snap-subdir-test 2023-06-09T11:13:50.551 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-06-09T11:13:50.552 INFO:tasks.ceph.mds.e:Restarting daemon 2023-06-09T11:13:50.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-06-09T11:13:50.557 INFO:tasks.ceph.mds.e:Started 2023-06-09T11:13:50.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:50.637 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/30 file after the snapshot 2023-06-09T11:13:50.640 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/31 file after the snapshot 2023-06-09T11:13:50.643 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/32 file after the snapshot 2023-06-09T11:13:50.645 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/33 file after the snapshot 2023-06-09T11:13:50.648 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/34 file after the snapshot 2023-06-09T11:13:50.650 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/35 file after the snapshot 2023-06-09T11:13:50.653 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/36 file after the snapshot 2023-06-09T11:13:50.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/37 file after the snapshot 2023-06-09T11:13:50.657 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/38 file after the snapshot 2023-06-09T11:13:50.660 INFO:tasks.workunit.client.0.smithi007.stdout: create 200/39 file after the snapshot 2023-06-09T11:13:50.662 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/.snap/snap-subdir-test 2023-06-09T11:13:50.685 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/30 file after the snapshot 2023-06-09T11:13:50.688 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/31 file after the snapshot 2023-06-09T11:13:50.690 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/32 file after the snapshot 2023-06-09T11:13:50.693 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/33 file after the snapshot 2023-06-09T11:13:50.696 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/34 file after the snapshot 2023-06-09T11:13:50.699 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/35 file after the snapshot 2023-06-09T11:13:50.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/36 file after the snapshot 2023-06-09T11:13:50.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/37 file after the snapshot 2023-06-09T11:13:50.707 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/38 file after the snapshot 2023-06-09T11:13:50.709 INFO:tasks.workunit.client.0.smithi007.stdout: create 201/39 file after the snapshot 2023-06-09T11:13:50.712 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/.snap/snap-subdir-test 2023-06-09T11:13:50.716 INFO:tasks.ceph.mds.e.smithi007.stdout:starting mds.e at 2023-06-09T11:13:50.737 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/30 file after the snapshot 2023-06-09T11:13:50.739 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/31 file after the snapshot 2023-06-09T11:13:50.741 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/32 file after the snapshot 2023-06-09T11:13:50.744 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/33 file after the snapshot 2023-06-09T11:13:50.746 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/34 file after the snapshot 2023-06-09T11:13:50.749 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/35 file after the snapshot 2023-06-09T11:13:50.751 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/36 file after the snapshot 2023-06-09T11:13:50.753 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/37 file after the snapshot 2023-06-09T11:13:50.756 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/38 file after the snapshot 2023-06-09T11:13:50.759 INFO:tasks.workunit.client.0.smithi007.stdout: create 202/39 file after the snapshot 2023-06-09T11:13:50.761 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/.snap/snap-subdir-test 2023-06-09T11:13:50.782 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/30 file after the snapshot 2023-06-09T11:13:50.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.779+0000 7fb7313ff700 1 Processor -- start 2023-06-09T11:13:50.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.779+0000 7fb7313ff700 1 -- start start 2023-06-09T11:13:50.785 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/31 file after the snapshot 2023-06-09T11:13:50.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.780+0000 7fb7313ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 0x7fb72c06b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.780+0000 7fb7313ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb72c159670 0x7fb72c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.780+0000 7fb7313ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72c06dce0 con 0x7fb72c06aee0 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.780+0000 7fb7313ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72c0c3db0 con 0x7fb72c159670 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.780+0000 7fb7313ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c0c7950 con 0x7fb72c150390 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb72c06aee0 0x7fb72c0c7240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52484/0 (socket says 172.21.15.7:52484) 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72a7fc700 1 -- 172.21.15.7:0/2996346748 learned_addr learned my addr 172.21.15.7:0/2996346748 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:13:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 0x7fb72c06b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72b7fe700 1 --2- 172.21.15.7:0/2996346748 >> v2:172.21.15.117:3300/0 conn(0x7fb72c159670 0x7fb72c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb729ffb700 1 -- 172.21.15.7:0/2996346748 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314630977 0 0) 0x7fb72c06dce0 con 0x7fb72c06aee0 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb729ffb700 1 -- 172.21.15.7:0/2996346748 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7140035b0 con 0x7fb72c06aee0 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72b7fe700 1 -- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 msgr2=0x7fb72c06b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72b7fe700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 0x7fb72c06b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72b7fe700 1 -- 172.21.15.7:0/2996346748 >> v1:172.21.15.7:6789/0 conn(0x7fb72c06aee0 legacy=0x7fb72c0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72b7fe700 1 -- 172.21.15.7:0/2996346748 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c06d0d0 con 0x7fb72c159670 2023-06-09T11:13:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.781+0000 7fb72affd700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 0x7fb72c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:13:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.782+0000 7fb72b7fe700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c159ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb724009dc0 tx=0x7fb72400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae4df5204703300 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:50.789 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/32 file after the snapshot 2023-06-09T11:13:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.782+0000 7fb729ffb700 1 -- 172.21.15.7:0/2996346748 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400e070 con 0x7fb72c159670 2023-06-09T11:13:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.782+0000 7fb729ffb700 1 -- 172.21.15.7:0/2996346748 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400bae0 con 0x7fb72c159670 2023-06-09T11:13:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.782+0000 7fb729ffb700 1 -- 172.21.15.7:0/2996346748 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400be90 con 0x7fb72c159670 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 -- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 msgr2=0x7fb72c159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c159ad0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb724009dc0 tx=0x7fb72400b330 comp rx=0 tx=0).stop 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 -- 172.21.15.7:0/2996346748 shutdown_connections 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c159ad0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 --2- 172.21.15.7:0/2996346748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c150390 0x7fb72c06b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 -- 172.21.15.7:0/2996346748 >> 172.21.15.7:0/2996346748 conn(0x7fb72c06f980 msgr2=0x7fb72c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 -- 172.21.15.7:0/2996346748 shutdown_connections 2023-06-09T11:13:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.783+0000 7fb7313ff700 1 -- 172.21.15.7:0/2996346748 wait complete. 2023-06-09T11:13:50.792 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/33 file after the snapshot 2023-06-09T11:13:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.785+0000 7fb7313ff700 1 Processor -- start 2023-06-09T11:13:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.785+0000 7fb7313ff700 1 -- start start 2023-06-09T11:13:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c150390 0x7fb72c149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1f0240 con 0x7fb72c150390 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1f0520 con 0x7fb72c159670 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb7313ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72c1f0800 con 0x7fb72c06aee0 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb72affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb72a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c150390 0x7fb72c149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb72affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:52564/0 (socket says 172.21.15.7:52564) 2023-06-09T11:13:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.786+0000 7fb72affd700 1 -- 172.21.15.7:0/1021925759 learned_addr learned my addr 172.21.15.7:0/1021925759 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72b7fe700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 -- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 msgr2=0x7fb72c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 -- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c150390 msgr2=0x7fb72c149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c150390 0x7fb72c149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 -- 172.21.15.7:0/1021925759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0094a0 con 0x7fb72c06aee0 2023-06-09T11:13:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb72affd700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00ebe0 tx=0x7fb71c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:50.796 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/34 file after the snapshot 2023-06-09T11:13:50.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00c850 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c00c9b0 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00cd40 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.787+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb724009a70 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.788+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72c06d690 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.789+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72c06d3b0 con 0x7fb72c06aee0 2023-06-09T11:13:50.797 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/35 file after the snapshot 2023-06-09T11:13:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.790+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb71c013470 con 0x7fb72c06aee0 2023-06-09T11:13:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.791+0000 7fb7137fe700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 0x7fb71806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.791+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7fb71c0897b0 con 0x7fb72c06aee0 2023-06-09T11:13:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.791+0000 7fb72a7fc700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 0x7fb71806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.792+0000 7fb72a7fc700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 0x7fb71806d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0c3680 tx=0x7fb720007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:50.799 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/36 file after the snapshot 2023-06-09T11:13:50.800 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/37 file after the snapshot 2023-06-09T11:13:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:50.797+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb71c054640 con 0x7fb72c06aee0 2023-06-09T11:13:50.803 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/38 file after the snapshot 2023-06-09T11:13:50.806 INFO:tasks.workunit.client.0.smithi007.stdout: create 203/39 file after the snapshot 2023-06-09T11:13:50.808 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/.snap/snap-subdir-test 2023-06-09T11:13:50.830 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/30 file after the snapshot 2023-06-09T11:13:50.833 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/31 file after the snapshot 2023-06-09T11:13:50.835 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/32 file after the snapshot 2023-06-09T11:13:50.837 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/33 file after the snapshot 2023-06-09T11:13:50.840 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/34 file after the snapshot 2023-06-09T11:13:50.842 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/35 file after the snapshot 2023-06-09T11:13:50.845 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/36 file after the snapshot 2023-06-09T11:13:50.847 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/37 file after the snapshot 2023-06-09T11:13:50.850 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/38 file after the snapshot 2023-06-09T11:13:50.852 INFO:tasks.workunit.client.0.smithi007.stdout: create 204/39 file after the snapshot 2023-06-09T11:13:50.855 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/.snap/snap-subdir-test 2023-06-09T11:13:50.882 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/30 file after the snapshot 2023-06-09T11:13:50.885 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/31 file after the snapshot 2023-06-09T11:13:50.887 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/32 file after the snapshot 2023-06-09T11:13:50.890 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/33 file after the snapshot 2023-06-09T11:13:50.892 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/34 file after the snapshot 2023-06-09T11:13:50.895 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/35 file after the snapshot 2023-06-09T11:13:50.898 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/36 file after the snapshot 2023-06-09T11:13:50.900 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/37 file after the snapshot 2023-06-09T11:13:50.903 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/38 file after the snapshot 2023-06-09T11:13:50.906 INFO:tasks.workunit.client.0.smithi007.stdout: create 205/39 file after the snapshot 2023-06-09T11:13:50.909 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/.snap/snap-subdir-test 2023-06-09T11:13:50.931 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/30 file after the snapshot 2023-06-09T11:13:50.933 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/31 file after the snapshot 2023-06-09T11:13:50.936 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/32 file after the snapshot 2023-06-09T11:13:50.938 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/33 file after the snapshot 2023-06-09T11:13:50.941 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/34 file after the snapshot 2023-06-09T11:13:50.943 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/35 file after the snapshot 2023-06-09T11:13:50.945 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/36 file after the snapshot 2023-06-09T11:13:50.949 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/37 file after the snapshot 2023-06-09T11:13:50.951 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/38 file after the snapshot 2023-06-09T11:13:50.954 INFO:tasks.workunit.client.0.smithi007.stdout: create 206/39 file after the snapshot 2023-06-09T11:13:50.956 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/.snap/snap-subdir-test 2023-06-09T11:13:50.980 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/30 file after the snapshot 2023-06-09T11:13:50.982 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/31 file after the snapshot 2023-06-09T11:13:50.985 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/32 file after the snapshot 2023-06-09T11:13:50.987 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/33 file after the snapshot 2023-06-09T11:13:50.990 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/34 file after the snapshot 2023-06-09T11:13:50.992 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/35 file after the snapshot 2023-06-09T11:13:50.995 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/36 file after the snapshot 2023-06-09T11:13:50.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/37 file after the snapshot 2023-06-09T11:13:50.999 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/38 file after the snapshot 2023-06-09T11:13:51.001 INFO:tasks.workunit.client.0.smithi007.stdout: create 207/39 file after the snapshot 2023-06-09T11:13:51.004 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/.snap/snap-subdir-test 2023-06-09T11:13:51.027 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/30 file after the snapshot 2023-06-09T11:13:51.029 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/31 file after the snapshot 2023-06-09T11:13:51.031 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/32 file after the snapshot 2023-06-09T11:13:51.034 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/33 file after the snapshot 2023-06-09T11:13:51.037 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/34 file after the snapshot 2023-06-09T11:13:51.040 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/35 file after the snapshot 2023-06-09T11:13:51.042 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/36 file after the snapshot 2023-06-09T11:13:51.045 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/37 file after the snapshot 2023-06-09T11:13:51.048 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/38 file after the snapshot 2023-06-09T11:13:51.050 INFO:tasks.workunit.client.0.smithi007.stdout: create 208/39 file after the snapshot 2023-06-09T11:13:51.053 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/.snap/snap-subdir-test 2023-06-09T11:13:51.075 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/30 file after the snapshot 2023-06-09T11:13:51.077 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/31 file after the snapshot 2023-06-09T11:13:51.079 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/32 file after the snapshot 2023-06-09T11:13:51.082 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/33 file after the snapshot 2023-06-09T11:13:51.084 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/34 file after the snapshot 2023-06-09T11:13:51.087 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/35 file after the snapshot 2023-06-09T11:13:51.089 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/36 file after the snapshot 2023-06-09T11:13:51.092 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/37 file after the snapshot 2023-06-09T11:13:51.094 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/38 file after the snapshot 2023-06-09T11:13:51.097 INFO:tasks.workunit.client.0.smithi007.stdout: create 209/39 file after the snapshot 2023-06-09T11:13:51.099 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/.snap/snap-subdir-test 2023-06-09T11:13:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.096+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb72c06d970 con 0x7fb72c06aee0 2023-06-09T11:13:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.097+0000 7fb7137fe700 1 -- 172.21.15.7:0/1021925759 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5461 (secure 0 0 0) 0x7fb71c00cea0 con 0x7fb72c06aee0 2023-06-09T11:13:51.103 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:51.103 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 msgr2=0x7fb71806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 0x7fb71806d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb72c0c3680 tx=0x7fb720007dc0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 msgr2=0x7fb72c157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00ebe0 tx=0x7fb71c00eef0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 shutdown_connections 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb72c159670 0x7fb72c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71806ace0 0x7fb71806d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb72c06aee0 0x7fb72c157ca0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 --2- 172.21.15.7:0/1021925759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb72c150390 0x7fb72c149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 >> 172.21.15.7:0/1021925759 conn(0x7fb72c06f980 msgr2=0x7fb72c14ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 shutdown_connections 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:51.100+0000 7fb7313ff700 1 -- 172.21.15.7:0/1021925759 wait complete. 2023-06-09T11:13:51.107 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 25 2023-06-09T11:13:51.120 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/30 file after the snapshot 2023-06-09T11:13:51.122 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/31 file after the snapshot 2023-06-09T11:13:51.125 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/32 file after the snapshot 2023-06-09T11:13:51.125 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-06-09T11:13:51.127 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/33 file after the snapshot 2023-06-09T11:13:51.129 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/34 file after the snapshot 2023-06-09T11:13:51.132 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/35 file after the snapshot 2023-06-09T11:13:51.135 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/36 file after the snapshot 2023-06-09T11:13:51.137 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/37 file after the snapshot 2023-06-09T11:13:51.139 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/38 file after the snapshot 2023-06-09T11:13:51.142 INFO:tasks.workunit.client.0.smithi007.stdout: create 210/39 file after the snapshot 2023-06-09T11:13:51.144 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/.snap/snap-subdir-test 2023-06-09T11:13:51.164 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/30 file after the snapshot 2023-06-09T11:13:51.166 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/31 file after the snapshot 2023-06-09T11:13:51.169 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/32 file after the snapshot 2023-06-09T11:13:51.171 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/33 file after the snapshot 2023-06-09T11:13:51.173 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/34 file after the snapshot 2023-06-09T11:13:51.176 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/35 file after the snapshot 2023-06-09T11:13:51.179 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/36 file after the snapshot 2023-06-09T11:13:51.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/37 file after the snapshot 2023-06-09T11:13:51.184 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/38 file after the snapshot 2023-06-09T11:13:51.186 INFO:tasks.workunit.client.0.smithi007.stdout: create 211/39 file after the snapshot 2023-06-09T11:13:51.188 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/.snap/snap-subdir-test 2023-06-09T11:13:51.210 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/30 file after the snapshot 2023-06-09T11:13:51.212 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/31 file after the snapshot 2023-06-09T11:13:51.215 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/32 file after the snapshot 2023-06-09T11:13:51.217 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/33 file after the snapshot 2023-06-09T11:13:51.219 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/34 file after the snapshot 2023-06-09T11:13:51.222 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/35 file after the snapshot 2023-06-09T11:13:51.224 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/36 file after the snapshot 2023-06-09T11:13:51.226 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/37 file after the snapshot 2023-06-09T11:13:51.229 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/38 file after the snapshot 2023-06-09T11:13:51.231 INFO:tasks.workunit.client.0.smithi007.stdout: create 212/39 file after the snapshot 2023-06-09T11:13:51.233 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/.snap/snap-subdir-test 2023-06-09T11:13:51.257 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/30 file after the snapshot 2023-06-09T11:13:51.260 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/31 file after the snapshot 2023-06-09T11:13:51.262 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/32 file after the snapshot 2023-06-09T11:13:51.265 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/33 file after the snapshot 2023-06-09T11:13:51.267 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/34 file after the snapshot 2023-06-09T11:13:51.269 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/35 file after the snapshot 2023-06-09T11:13:51.272 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/36 file after the snapshot 2023-06-09T11:13:51.274 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/37 file after the snapshot 2023-06-09T11:13:51.276 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/38 file after the snapshot 2023-06-09T11:13:51.279 INFO:tasks.workunit.client.0.smithi007.stdout: create 213/39 file after the snapshot 2023-06-09T11:13:51.282 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/.snap/snap-subdir-test 2023-06-09T11:13:51.301 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/30 file after the snapshot 2023-06-09T11:13:51.304 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/31 file after the snapshot 2023-06-09T11:13:51.306 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/32 file after the snapshot 2023-06-09T11:13:51.308 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/33 file after the snapshot 2023-06-09T11:13:51.310 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/34 file after the snapshot 2023-06-09T11:13:52.091 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/35 file after the snapshot 2023-06-09T11:13:52.094 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/36 file after the snapshot 2023-06-09T11:13:52.097 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/37 file after the snapshot 2023-06-09T11:13:52.100 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/38 file after the snapshot 2023-06-09T11:13:52.102 INFO:tasks.workunit.client.0.smithi007.stdout: create 214/39 file after the snapshot 2023-06-09T11:13:52.105 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/.snap/snap-subdir-test 2023-06-09T11:13:52.131 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/30 file after the snapshot 2023-06-09T11:13:52.134 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/31 file after the snapshot 2023-06-09T11:13:52.137 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/32 file after the snapshot 2023-06-09T11:13:52.140 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/33 file after the snapshot 2023-06-09T11:13:52.143 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/34 file after the snapshot 2023-06-09T11:13:52.145 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/35 file after the snapshot 2023-06-09T11:13:52.148 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/36 file after the snapshot 2023-06-09T11:13:52.152 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/37 file after the snapshot 2023-06-09T11:13:52.154 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/38 file after the snapshot 2023-06-09T11:13:52.157 INFO:tasks.workunit.client.0.smithi007.stdout: create 215/39 file after the snapshot 2023-06-09T11:13:52.160 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/.snap/snap-subdir-test 2023-06-09T11:13:52.183 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/30 file after the snapshot 2023-06-09T11:13:52.186 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/31 file after the snapshot 2023-06-09T11:13:52.189 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/32 file after the snapshot 2023-06-09T11:13:52.191 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/33 file after the snapshot 2023-06-09T11:13:52.194 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/34 file after the snapshot 2023-06-09T11:13:52.196 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/35 file after the snapshot 2023-06-09T11:13:52.199 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/36 file after the snapshot 2023-06-09T11:13:52.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/37 file after the snapshot 2023-06-09T11:13:52.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/38 file after the snapshot 2023-06-09T11:13:52.206 INFO:tasks.workunit.client.0.smithi007.stdout: create 216/39 file after the snapshot 2023-06-09T11:13:52.209 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/.snap/snap-subdir-test 2023-06-09T11:13:52.231 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/30 file after the snapshot 2023-06-09T11:13:52.234 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/31 file after the snapshot 2023-06-09T11:13:52.236 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/32 file after the snapshot 2023-06-09T11:13:52.239 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/33 file after the snapshot 2023-06-09T11:13:52.241 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/34 file after the snapshot 2023-06-09T11:13:52.244 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/35 file after the snapshot 2023-06-09T11:13:52.246 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/36 file after the snapshot 2023-06-09T11:13:52.249 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/37 file after the snapshot 2023-06-09T11:13:52.252 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/38 file after the snapshot 2023-06-09T11:13:52.255 INFO:tasks.workunit.client.0.smithi007.stdout: create 217/39 file after the snapshot 2023-06-09T11:13:52.257 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/.snap/snap-subdir-test 2023-06-09T11:13:52.280 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/30 file after the snapshot 2023-06-09T11:13:52.282 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/31 file after the snapshot 2023-06-09T11:13:52.285 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/32 file after the snapshot 2023-06-09T11:13:52.287 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/33 file after the snapshot 2023-06-09T11:13:52.289 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/34 file after the snapshot 2023-06-09T11:13:52.292 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/35 file after the snapshot 2023-06-09T11:13:52.294 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/36 file after the snapshot 2023-06-09T11:13:52.296 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/37 file after the snapshot 2023-06-09T11:13:52.299 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/38 file after the snapshot 2023-06-09T11:13:52.301 INFO:tasks.workunit.client.0.smithi007.stdout: create 218/39 file after the snapshot 2023-06-09T11:13:52.304 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/.snap/snap-subdir-test 2023-06-09T11:13:52.326 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/30 file after the snapshot 2023-06-09T11:13:52.329 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/31 file after the snapshot 2023-06-09T11:13:52.332 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/32 file after the snapshot 2023-06-09T11:13:52.334 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/33 file after the snapshot 2023-06-09T11:13:52.336 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/34 file after the snapshot 2023-06-09T11:13:52.339 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/35 file after the snapshot 2023-06-09T11:13:52.342 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/36 file after the snapshot 2023-06-09T11:13:52.344 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/37 file after the snapshot 2023-06-09T11:13:52.347 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/38 file after the snapshot 2023-06-09T11:13:52.349 INFO:tasks.workunit.client.0.smithi007.stdout: create 219/39 file after the snapshot 2023-06-09T11:13:52.351 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/.snap/snap-subdir-test 2023-06-09T11:13:53.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:13:53.288 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/30 file after the snapshot 2023-06-09T11:13:53.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/31 file after the snapshot 2023-06-09T11:13:53.295 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/32 file after the snapshot 2023-06-09T11:13:53.298 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/33 file after the snapshot 2023-06-09T11:13:53.300 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/34 file after the snapshot 2023-06-09T11:13:53.303 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/35 file after the snapshot 2023-06-09T11:13:53.305 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/36 file after the snapshot 2023-06-09T11:13:53.308 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/37 file after the snapshot 2023-06-09T11:13:53.310 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/38 file after the snapshot 2023-06-09T11:13:53.313 INFO:tasks.workunit.client.0.smithi007.stdout: create 220/39 file after the snapshot 2023-06-09T11:13:53.315 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/.snap/snap-subdir-test 2023-06-09T11:13:53.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.339+0000 7f15f85ff700 1 Processor -- start 2023-06-09T11:13:53.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.339+0000 7f15f85ff700 1 -- start start 2023-06-09T11:13:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f85ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f15f0153880 0x7f15f015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f85ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f015a500 con 0x7f15f015b2d0 2023-06-09T11:13:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f85ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f00c5f10 con 0x7f15f0153880 2023-06-09T11:13:53.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f015da00 con 0x7f15f00c9f40 2023-06-09T11:13:53.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f5b9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f5b9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:59418/0 (socket says 172.21.15.7:59418) 2023-06-09T11:13:53.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f5b9a700 1 -- 172.21.15.7:0/1069190118 learned_addr learned my addr 172.21.15.7:0/1069190118 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:53.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.341+0000 7f15f639b700 1 --2- 172.21.15.7:0/1069190118 >> v2:172.21.15.117:3300/0 conn(0x7f15f0153880 0x7f15f015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5b9a700 1 -- 172.21.15.7:0/1069190118 >> v2:172.21.15.117:3300/0 conn(0x7f15f0153880 msgr2=0x7f15f015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5b9a700 1 --2- 172.21.15.7:0/1069190118 >> v2:172.21.15.117:3300/0 conn(0x7f15f0153880 0x7f15f015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5b9a700 1 -- 172.21.15.7:0/1069190118 >> v1:172.21.15.117:6789/0 conn(0x7f15f015b2d0 legacy=0x7f15f00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5b9a700 1 -- 172.21.15.7:0/1069190118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f015b750 con 0x7f15f00c9f40 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5b9a700 1 --2- 172.21.15.7:0/1069190118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3fd129be7b0af71 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5399700 1 -- 172.21.15.7:0/1069190118 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011830 con 0x7f15f00c9f40 2023-06-09T11:13:53.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5399700 1 -- 172.21.15.7:0/1069190118 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4011990 con 0x7f15f00c9f40 2023-06-09T11:13:53.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.342+0000 7f15f5399700 1 -- 172.21.15.7:0/1069190118 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011d20 con 0x7f15f00c9f40 2023-06-09T11:13:53.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 -- 172.21.15.7:0/1069190118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 msgr2=0x7f15f015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 --2- 172.21.15.7:0/1069190118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).stop 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 -- 172.21.15.7:0/1069190118 shutdown_connections 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 --2- 172.21.15.7:0/1069190118 >> v2:172.21.15.117:3300/0 conn(0x7f15f0153880 0x7f15f015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 --2- 172.21.15.7:0/1069190118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f00c9f40 0x7f15f015ad90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 -- 172.21.15.7:0/1069190118 >> 172.21.15.7:0/1069190118 conn(0x7f15f00737f0 msgr2=0x7f15f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 -- 172.21.15.7:0/1069190118 shutdown_connections 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.343+0000 7f15f85ff700 1 -- 172.21.15.7:0/1069190118 wait complete. 2023-06-09T11:13:53.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.344+0000 7f15f85ff700 1 Processor -- start 2023-06-09T11:13:53.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.345+0000 7f15f85ff700 1 -- start start 2023-06-09T11:13:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.345+0000 7f15f85ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f00c9f40 0x7f15f00b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f85ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 0x7f15f01f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f85ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f01f3640 con 0x7f15f0153880 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f85ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00b50c0 con 0x7f15f00c9f40 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f00caf00 con 0x7f15f015b2d0 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40136/0 (socket says 172.21.15.7:40136) 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 -- 172.21.15.7:0/2288151705 learned_addr learned my addr 172.21.15.7:0/2288151705 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f639b700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f00c9f40 0x7f15f00b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f6b9c700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 0x7f15f01f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 -- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 msgr2=0x7f15f01f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 0x7f15f01f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 -- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f00c9f40 msgr2=0x7f15f00b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f00c9f40 0x7f15f00b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f5b9a700 1 -- 172.21.15.7:0/2288151705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4009550 con 0x7f15f0153880 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.346+0000 7f15f6b9c700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 0x7f15f01f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:13:53.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15f5b9a700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009c40 tx=0x7f15e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4004480 con 0x7f15f0153880 2023-06-09T11:13:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e401f070 con 0x7f15f0153880 2023-06-09T11:13:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e40038a0 con 0x7f15f0153880 2023-06-09T11:13:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f0159980 con 0x7f15f0153880 2023-06-09T11:13:53.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.347+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f01596e0 con 0x7f15f0153880 2023-06-09T11:13:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.348+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f0159c60 con 0x7f15f0153880 2023-06-09T11:13:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.349+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15e4021030 con 0x7f15f0153880 2023-06-09T11:13:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.350+0000 7f15e2ffd700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 0x7f15d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:13:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.350+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f15e408abc0 con 0x7f15f0153880 2023-06-09T11:13:53.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.350+0000 7f15f639b700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 0x7f15d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:13:53.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.351+0000 7f15f639b700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 0x7f15d406d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009be0 tx=0x7f15ec009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:13:53.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.354+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f15e4055c90 con 0x7f15f0153880 2023-06-09T11:13:53.472 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/30 file after the snapshot 2023-06-09T11:13:53.474 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/31 file after the snapshot 2023-06-09T11:13:53.477 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/32 file after the snapshot 2023-06-09T11:13:53.479 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/33 file after the snapshot 2023-06-09T11:13:53.482 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/34 file after the snapshot 2023-06-09T11:13:53.485 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/35 file after the snapshot 2023-06-09T11:13:53.487 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/36 file after the snapshot 2023-06-09T11:13:53.490 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/37 file after the snapshot 2023-06-09T11:13:53.492 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/38 file after the snapshot 2023-06-09T11:13:53.498 INFO:tasks.workunit.client.0.smithi007.stdout: create 221/39 file after the snapshot 2023-06-09T11:13:53.503 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/.snap/snap-subdir-test 2023-06-09T11:13:53.555 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/30 file after the snapshot 2023-06-09T11:13:53.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/31 file after the snapshot 2023-06-09T11:13:53.560 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/32 file after the snapshot 2023-06-09T11:13:53.563 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/33 file after the snapshot 2023-06-09T11:13:53.565 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/34 file after the snapshot 2023-06-09T11:13:53.568 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/35 file after the snapshot 2023-06-09T11:13:53.571 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/36 file after the snapshot 2023-06-09T11:13:53.573 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/37 file after the snapshot 2023-06-09T11:13:53.576 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/38 file after the snapshot 2023-06-09T11:13:53.579 INFO:tasks.workunit.client.0.smithi007.stdout: create 222/39 file after the snapshot 2023-06-09T11:13:53.581 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/.snap/snap-subdir-test 2023-06-09T11:13:53.604 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/30 file after the snapshot 2023-06-09T11:13:53.607 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/31 file after the snapshot 2023-06-09T11:13:53.609 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/32 file after the snapshot 2023-06-09T11:13:53.612 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/33 file after the snapshot 2023-06-09T11:13:53.615 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/34 file after the snapshot 2023-06-09T11:13:53.618 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/35 file after the snapshot 2023-06-09T11:13:53.620 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/36 file after the snapshot 2023-06-09T11:13:53.623 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/37 file after the snapshot 2023-06-09T11:13:53.625 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/38 file after the snapshot 2023-06-09T11:13:53.627 INFO:tasks.workunit.client.0.smithi007.stdout: create 223/39 file after the snapshot 2023-06-09T11:13:53.630 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/.snap/snap-subdir-test 2023-06-09T11:13:53.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.629+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15f00b38c0 con 0x7f15f0153880 2023-06-09T11:13:53.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.630+0000 7f15e2ffd700 1 -- 172.21.15.7:0/2288151705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f15e40592b0 con 0x7f15f0153880 2023-06-09T11:13:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:13:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 msgr2=0x7f15d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 0x7f15d406d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009be0 tx=0x7f15ec009210 comp rx=0 tx=0).stop 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 msgr2=0x7f15f00cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009c40 tx=0x7f15e4004750 comp rx=0 tx=0).stop 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 shutdown_connections 2023-06-09T11:13:53.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15d406ace0 0x7f15d406d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f015b2d0 0x7f15f01f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f00c9f40 0x7f15f00b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 --2- 172.21.15.7:0/2288151705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0153880 0x7f15f00cb6a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 >> 172.21.15.7:0/2288151705 conn(0x7f15f00737f0 msgr2=0x7f15f00720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 shutdown_connections 2023-06-09T11:13:53.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:13:53.632+0000 7f15f85ff700 1 -- 172.21.15.7:0/2288151705 wait complete. 2023-06-09T11:13:53.640 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:13:53.652 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/30 file after the snapshot 2023-06-09T11:13:53.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/31 file after the snapshot 2023-06-09T11:13:53.657 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/32 file after the snapshot 2023-06-09T11:13:53.658 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-06-09T11:13:53.658 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 70 secs before thrashing 2023-06-09T11:13:53.659 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/33 file after the snapshot 2023-06-09T11:13:53.662 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/34 file after the snapshot 2023-06-09T11:13:53.664 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/35 file after the snapshot 2023-06-09T11:13:53.667 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/36 file after the snapshot 2023-06-09T11:13:53.906 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/37 file after the snapshot 2023-06-09T11:13:53.908 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/38 file after the snapshot 2023-06-09T11:13:53.910 INFO:tasks.workunit.client.0.smithi007.stdout: create 224/39 file after the snapshot 2023-06-09T11:13:53.913 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/.snap/snap-subdir-test 2023-06-09T11:13:53.934 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/30 file after the snapshot 2023-06-09T11:13:53.937 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/31 file after the snapshot 2023-06-09T11:13:53.940 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/32 file after the snapshot 2023-06-09T11:13:53.942 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/33 file after the snapshot 2023-06-09T11:13:53.945 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/34 file after the snapshot 2023-06-09T11:13:53.947 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/35 file after the snapshot 2023-06-09T11:13:53.949 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/36 file after the snapshot 2023-06-09T11:13:53.952 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/37 file after the snapshot 2023-06-09T11:13:53.955 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/38 file after the snapshot 2023-06-09T11:13:53.957 INFO:tasks.workunit.client.0.smithi007.stdout: create 225/39 file after the snapshot 2023-06-09T11:13:53.959 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/.snap/snap-subdir-test 2023-06-09T11:13:53.984 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/30 file after the snapshot 2023-06-09T11:13:53.986 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/31 file after the snapshot 2023-06-09T11:13:53.989 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/32 file after the snapshot 2023-06-09T11:13:53.991 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/33 file after the snapshot 2023-06-09T11:13:53.993 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/34 file after the snapshot 2023-06-09T11:13:53.995 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/35 file after the snapshot 2023-06-09T11:13:53.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/36 file after the snapshot 2023-06-09T11:13:54.000 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/37 file after the snapshot 2023-06-09T11:13:54.002 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/38 file after the snapshot 2023-06-09T11:13:54.005 INFO:tasks.workunit.client.0.smithi007.stdout: create 226/39 file after the snapshot 2023-06-09T11:13:54.007 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/.snap/snap-subdir-test 2023-06-09T11:13:54.026 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/30 file after the snapshot 2023-06-09T11:13:54.028 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/31 file after the snapshot 2023-06-09T11:13:54.030 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/32 file after the snapshot 2023-06-09T11:13:54.033 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/33 file after the snapshot 2023-06-09T11:13:54.035 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/34 file after the snapshot 2023-06-09T11:13:54.037 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/35 file after the snapshot 2023-06-09T11:13:54.039 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/36 file after the snapshot 2023-06-09T11:13:54.042 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/37 file after the snapshot 2023-06-09T11:13:54.044 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/38 file after the snapshot 2023-06-09T11:13:54.046 INFO:tasks.workunit.client.0.smithi007.stdout: create 227/39 file after the snapshot 2023-06-09T11:13:54.048 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/.snap/snap-subdir-test 2023-06-09T11:13:54.072 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/30 file after the snapshot 2023-06-09T11:13:54.074 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/31 file after the snapshot 2023-06-09T11:13:54.077 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/32 file after the snapshot 2023-06-09T11:13:54.079 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/33 file after the snapshot 2023-06-09T11:13:54.082 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/34 file after the snapshot 2023-06-09T11:13:54.084 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/35 file after the snapshot 2023-06-09T11:13:54.086 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/36 file after the snapshot 2023-06-09T11:13:54.089 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/37 file after the snapshot 2023-06-09T11:13:54.092 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/38 file after the snapshot 2023-06-09T11:13:54.094 INFO:tasks.workunit.client.0.smithi007.stdout: create 228/39 file after the snapshot 2023-06-09T11:13:54.096 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/.snap/snap-subdir-test 2023-06-09T11:13:54.114 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/30 file after the snapshot 2023-06-09T11:13:54.116 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/31 file after the snapshot 2023-06-09T11:13:54.119 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/32 file after the snapshot 2023-06-09T11:13:54.121 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/33 file after the snapshot 2023-06-09T11:13:54.123 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/34 file after the snapshot 2023-06-09T11:13:54.126 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/35 file after the snapshot 2023-06-09T11:13:54.128 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/36 file after the snapshot 2023-06-09T11:13:54.131 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/37 file after the snapshot 2023-06-09T11:13:54.133 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/38 file after the snapshot 2023-06-09T11:13:54.135 INFO:tasks.workunit.client.0.smithi007.stdout: create 229/39 file after the snapshot 2023-06-09T11:13:54.138 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/.snap/snap-subdir-test 2023-06-09T11:13:54.159 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/30 file after the snapshot 2023-06-09T11:13:54.161 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/31 file after the snapshot 2023-06-09T11:13:54.164 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/32 file after the snapshot 2023-06-09T11:13:54.166 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/33 file after the snapshot 2023-06-09T11:13:54.168 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/34 file after the snapshot 2023-06-09T11:13:54.171 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/35 file after the snapshot 2023-06-09T11:13:54.173 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/36 file after the snapshot 2023-06-09T11:13:54.176 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/37 file after the snapshot 2023-06-09T11:13:54.178 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/38 file after the snapshot 2023-06-09T11:13:54.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 230/39 file after the snapshot 2023-06-09T11:13:54.184 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/.snap/snap-subdir-test 2023-06-09T11:13:54.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/30 file after the snapshot 2023-06-09T11:13:54.207 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/31 file after the snapshot 2023-06-09T11:13:54.210 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/32 file after the snapshot 2023-06-09T11:13:54.212 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/33 file after the snapshot 2023-06-09T11:13:54.215 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/34 file after the snapshot 2023-06-09T11:13:54.218 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/35 file after the snapshot 2023-06-09T11:13:54.220 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/36 file after the snapshot 2023-06-09T11:13:54.223 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/37 file after the snapshot 2023-06-09T11:13:54.225 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/38 file after the snapshot 2023-06-09T11:13:54.228 INFO:tasks.workunit.client.0.smithi007.stdout: create 231/39 file after the snapshot 2023-06-09T11:13:54.230 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/.snap/snap-subdir-test 2023-06-09T11:13:54.249 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/30 file after the snapshot 2023-06-09T11:13:54.250 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/31 file after the snapshot 2023-06-09T11:13:54.253 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/32 file after the snapshot 2023-06-09T11:13:54.255 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/33 file after the snapshot 2023-06-09T11:13:54.257 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/34 file after the snapshot 2023-06-09T11:13:54.260 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/35 file after the snapshot 2023-06-09T11:13:54.262 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/36 file after the snapshot 2023-06-09T11:13:54.265 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/37 file after the snapshot 2023-06-09T11:13:54.267 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/38 file after the snapshot 2023-06-09T11:13:54.269 INFO:tasks.workunit.client.0.smithi007.stdout: create 232/39 file after the snapshot 2023-06-09T11:13:54.272 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/.snap/snap-subdir-test 2023-06-09T11:13:54.294 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/30 file after the snapshot 2023-06-09T11:13:54.297 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/31 file after the snapshot 2023-06-09T11:13:54.300 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/32 file after the snapshot 2023-06-09T11:13:54.302 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/33 file after the snapshot 2023-06-09T11:13:54.305 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/34 file after the snapshot 2023-06-09T11:13:54.307 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/35 file after the snapshot 2023-06-09T11:13:54.309 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/36 file after the snapshot 2023-06-09T11:13:54.312 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/37 file after the snapshot 2023-06-09T11:13:54.314 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/38 file after the snapshot 2023-06-09T11:13:54.316 INFO:tasks.workunit.client.0.smithi007.stdout: create 233/39 file after the snapshot 2023-06-09T11:13:54.318 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/.snap/snap-subdir-test 2023-06-09T11:13:54.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/30 file after the snapshot 2023-06-09T11:13:54.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/31 file after the snapshot 2023-06-09T11:13:54.343 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/32 file after the snapshot 2023-06-09T11:13:54.345 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/33 file after the snapshot 2023-06-09T11:13:54.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/34 file after the snapshot 2023-06-09T11:13:54.350 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/35 file after the snapshot 2023-06-09T11:13:54.352 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/36 file after the snapshot 2023-06-09T11:13:54.355 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/37 file after the snapshot 2023-06-09T11:13:54.357 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/38 file after the snapshot 2023-06-09T11:13:54.360 INFO:tasks.workunit.client.0.smithi007.stdout: create 234/39 file after the snapshot 2023-06-09T11:13:54.362 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/.snap/snap-subdir-test 2023-06-09T11:13:54.383 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/30 file after the snapshot 2023-06-09T11:13:54.386 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/31 file after the snapshot 2023-06-09T11:13:54.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/32 file after the snapshot 2023-06-09T11:13:54.391 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/33 file after the snapshot 2023-06-09T11:13:54.393 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/34 file after the snapshot 2023-06-09T11:13:54.396 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/35 file after the snapshot 2023-06-09T11:13:54.398 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/36 file after the snapshot 2023-06-09T11:13:54.400 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/37 file after the snapshot 2023-06-09T11:13:54.403 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/38 file after the snapshot 2023-06-09T11:13:54.406 INFO:tasks.workunit.client.0.smithi007.stdout: create 235/39 file after the snapshot 2023-06-09T11:13:54.408 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/.snap/snap-subdir-test 2023-06-09T11:13:54.433 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/30 file after the snapshot 2023-06-09T11:13:54.435 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/31 file after the snapshot 2023-06-09T11:13:54.438 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/32 file after the snapshot 2023-06-09T11:13:54.440 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/33 file after the snapshot 2023-06-09T11:13:54.443 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/34 file after the snapshot 2023-06-09T11:13:54.445 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/35 file after the snapshot 2023-06-09T11:13:54.447 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/36 file after the snapshot 2023-06-09T11:13:54.450 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/37 file after the snapshot 2023-06-09T11:13:54.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/38 file after the snapshot 2023-06-09T11:13:54.454 INFO:tasks.workunit.client.0.smithi007.stdout: create 236/39 file after the snapshot 2023-06-09T11:13:54.457 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/.snap/snap-subdir-test 2023-06-09T11:13:54.478 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/30 file after the snapshot 2023-06-09T11:13:54.481 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/31 file after the snapshot 2023-06-09T11:13:54.483 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/32 file after the snapshot 2023-06-09T11:13:54.485 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/33 file after the snapshot 2023-06-09T11:13:54.488 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/34 file after the snapshot 2023-06-09T11:13:54.491 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/35 file after the snapshot 2023-06-09T11:13:54.493 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/36 file after the snapshot 2023-06-09T11:13:54.495 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/37 file after the snapshot 2023-06-09T11:13:54.497 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/38 file after the snapshot 2023-06-09T11:13:54.521 INFO:tasks.workunit.client.0.smithi007.stdout: create 237/39 file after the snapshot 2023-06-09T11:13:54.523 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/.snap/snap-subdir-test 2023-06-09T11:13:54.556 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/30 file after the snapshot 2023-06-09T11:13:54.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/31 file after the snapshot 2023-06-09T11:13:54.561 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/32 file after the snapshot 2023-06-09T11:13:54.563 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/33 file after the snapshot 2023-06-09T11:13:54.565 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/34 file after the snapshot 2023-06-09T11:13:54.568 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/35 file after the snapshot 2023-06-09T11:13:54.570 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/36 file after the snapshot 2023-06-09T11:13:54.572 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/37 file after the snapshot 2023-06-09T11:13:54.575 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/38 file after the snapshot 2023-06-09T11:13:54.577 INFO:tasks.workunit.client.0.smithi007.stdout: create 238/39 file after the snapshot 2023-06-09T11:13:54.580 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/.snap/snap-subdir-test 2023-06-09T11:13:54.602 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/30 file after the snapshot 2023-06-09T11:13:54.604 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/31 file after the snapshot 2023-06-09T11:13:54.606 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/32 file after the snapshot 2023-06-09T11:13:54.608 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/33 file after the snapshot 2023-06-09T11:13:54.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/34 file after the snapshot 2023-06-09T11:13:54.613 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/35 file after the snapshot 2023-06-09T11:13:54.616 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/36 file after the snapshot 2023-06-09T11:13:54.618 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/37 file after the snapshot 2023-06-09T11:13:54.620 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/38 file after the snapshot 2023-06-09T11:13:54.622 INFO:tasks.workunit.client.0.smithi007.stdout: create 239/39 file after the snapshot 2023-06-09T11:13:54.625 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/.snap/snap-subdir-test 2023-06-09T11:13:54.646 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/30 file after the snapshot 2023-06-09T11:13:54.648 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/31 file after the snapshot 2023-06-09T11:13:54.650 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/32 file after the snapshot 2023-06-09T11:13:54.652 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/33 file after the snapshot 2023-06-09T11:13:54.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/34 file after the snapshot 2023-06-09T11:13:54.657 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/35 file after the snapshot 2023-06-09T11:13:54.659 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/36 file after the snapshot 2023-06-09T11:13:54.661 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/37 file after the snapshot 2023-06-09T11:13:54.664 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/38 file after the snapshot 2023-06-09T11:13:54.666 INFO:tasks.workunit.client.0.smithi007.stdout: create 240/39 file after the snapshot 2023-06-09T11:13:54.668 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/.snap/snap-subdir-test 2023-06-09T11:13:54.689 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/30 file after the snapshot 2023-06-09T11:13:54.691 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/31 file after the snapshot 2023-06-09T11:13:54.694 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/32 file after the snapshot 2023-06-09T11:13:54.696 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/33 file after the snapshot 2023-06-09T11:13:54.699 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/34 file after the snapshot 2023-06-09T11:13:54.701 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/35 file after the snapshot 2023-06-09T11:13:54.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/36 file after the snapshot 2023-06-09T11:13:54.707 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/37 file after the snapshot 2023-06-09T11:13:54.710 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/38 file after the snapshot 2023-06-09T11:13:54.713 INFO:tasks.workunit.client.0.smithi007.stdout: create 241/39 file after the snapshot 2023-06-09T11:13:54.715 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/.snap/snap-subdir-test 2023-06-09T11:13:54.739 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/30 file after the snapshot 2023-06-09T11:13:55.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/31 file after the snapshot 2023-06-09T11:13:55.875 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/32 file after the snapshot 2023-06-09T11:13:55.878 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/33 file after the snapshot 2023-06-09T11:13:55.881 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/34 file after the snapshot 2023-06-09T11:13:55.884 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/35 file after the snapshot 2023-06-09T11:13:55.887 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/36 file after the snapshot 2023-06-09T11:13:55.890 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/37 file after the snapshot 2023-06-09T11:13:55.893 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/38 file after the snapshot 2023-06-09T11:13:55.896 INFO:tasks.workunit.client.0.smithi007.stdout: create 242/39 file after the snapshot 2023-06-09T11:13:55.899 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/.snap/snap-subdir-test 2023-06-09T11:13:55.932 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/30 file after the snapshot 2023-06-09T11:13:55.935 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/31 file after the snapshot 2023-06-09T11:13:55.938 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/32 file after the snapshot 2023-06-09T11:13:55.941 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/33 file after the snapshot 2023-06-09T11:13:55.943 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/34 file after the snapshot 2023-06-09T11:13:55.947 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/35 file after the snapshot 2023-06-09T11:13:55.950 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/36 file after the snapshot 2023-06-09T11:13:55.953 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/37 file after the snapshot 2023-06-09T11:13:55.956 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/38 file after the snapshot 2023-06-09T11:13:55.958 INFO:tasks.workunit.client.0.smithi007.stdout: create 243/39 file after the snapshot 2023-06-09T11:13:55.961 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/.snap/snap-subdir-test 2023-06-09T11:13:55.994 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/30 file after the snapshot 2023-06-09T11:13:55.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/31 file after the snapshot 2023-06-09T11:13:55.999 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/32 file after the snapshot 2023-06-09T11:13:56.002 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/33 file after the snapshot 2023-06-09T11:13:56.004 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/34 file after the snapshot 2023-06-09T11:13:56.006 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/35 file after the snapshot 2023-06-09T11:13:56.009 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/36 file after the snapshot 2023-06-09T11:13:56.011 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/37 file after the snapshot 2023-06-09T11:13:56.013 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/38 file after the snapshot 2023-06-09T11:13:56.016 INFO:tasks.workunit.client.0.smithi007.stdout: create 244/39 file after the snapshot 2023-06-09T11:13:56.018 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/.snap/snap-subdir-test 2023-06-09T11:13:56.039 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/30 file after the snapshot 2023-06-09T11:13:56.041 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/31 file after the snapshot 2023-06-09T11:13:56.044 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/32 file after the snapshot 2023-06-09T11:13:56.046 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/33 file after the snapshot 2023-06-09T11:13:56.049 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/34 file after the snapshot 2023-06-09T11:13:56.051 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/35 file after the snapshot 2023-06-09T11:13:56.054 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/36 file after the snapshot 2023-06-09T11:13:56.056 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/37 file after the snapshot 2023-06-09T11:13:56.059 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/38 file after the snapshot 2023-06-09T11:13:56.061 INFO:tasks.workunit.client.0.smithi007.stdout: create 245/39 file after the snapshot 2023-06-09T11:13:56.063 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/.snap/snap-subdir-test 2023-06-09T11:13:56.086 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/30 file after the snapshot 2023-06-09T11:13:56.089 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/31 file after the snapshot 2023-06-09T11:13:56.091 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/32 file after the snapshot 2023-06-09T11:13:56.094 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/33 file after the snapshot 2023-06-09T11:13:56.097 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/34 file after the snapshot 2023-06-09T11:13:56.099 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/35 file after the snapshot 2023-06-09T11:13:56.102 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/36 file after the snapshot 2023-06-09T11:13:56.104 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/37 file after the snapshot 2023-06-09T11:13:56.106 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/38 file after the snapshot 2023-06-09T11:13:56.109 INFO:tasks.workunit.client.0.smithi007.stdout: create 246/39 file after the snapshot 2023-06-09T11:13:56.111 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/.snap/snap-subdir-test 2023-06-09T11:13:56.129 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/30 file after the snapshot 2023-06-09T11:13:56.132 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/31 file after the snapshot 2023-06-09T11:13:56.136 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/32 file after the snapshot 2023-06-09T11:13:56.142 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/33 file after the snapshot 2023-06-09T11:13:56.270 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/34 file after the snapshot 2023-06-09T11:13:56.273 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/35 file after the snapshot 2023-06-09T11:13:56.276 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/36 file after the snapshot 2023-06-09T11:13:56.279 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/37 file after the snapshot 2023-06-09T11:13:56.283 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/38 file after the snapshot 2023-06-09T11:13:56.286 INFO:tasks.workunit.client.0.smithi007.stdout: create 247/39 file after the snapshot 2023-06-09T11:13:56.289 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/.snap/snap-subdir-test 2023-06-09T11:13:56.853 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/30 file after the snapshot 2023-06-09T11:13:57.273 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/31 file after the snapshot 2023-06-09T11:13:57.276 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/32 file after the snapshot 2023-06-09T11:13:57.279 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/33 file after the snapshot 2023-06-09T11:13:57.282 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/34 file after the snapshot 2023-06-09T11:13:57.285 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/35 file after the snapshot 2023-06-09T11:13:57.288 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/36 file after the snapshot 2023-06-09T11:13:57.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/37 file after the snapshot 2023-06-09T11:13:57.294 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/38 file after the snapshot 2023-06-09T11:13:57.297 INFO:tasks.workunit.client.0.smithi007.stdout: create 248/39 file after the snapshot 2023-06-09T11:13:57.300 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/.snap/snap-subdir-test 2023-06-09T11:13:57.327 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/30 file after the snapshot 2023-06-09T11:13:57.330 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/31 file after the snapshot 2023-06-09T11:13:57.333 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/32 file after the snapshot 2023-06-09T11:13:57.335 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/33 file after the snapshot 2023-06-09T11:13:57.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/34 file after the snapshot 2023-06-09T11:13:57.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/35 file after the snapshot 2023-06-09T11:13:57.343 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/36 file after the snapshot 2023-06-09T11:13:57.346 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/37 file after the snapshot 2023-06-09T11:13:57.349 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/38 file after the snapshot 2023-06-09T11:13:57.351 INFO:tasks.workunit.client.0.smithi007.stdout: create 249/39 file after the snapshot 2023-06-09T11:13:57.354 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/.snap/snap-subdir-test 2023-06-09T11:13:57.376 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/30 file after the snapshot 2023-06-09T11:13:57.378 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/31 file after the snapshot 2023-06-09T11:13:57.381 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/32 file after the snapshot 2023-06-09T11:13:57.383 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/33 file after the snapshot 2023-06-09T11:13:57.385 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/34 file after the snapshot 2023-06-09T11:13:57.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/35 file after the snapshot 2023-06-09T11:13:57.391 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/36 file after the snapshot 2023-06-09T11:13:57.393 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/37 file after the snapshot 2023-06-09T11:13:57.396 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/38 file after the snapshot 2023-06-09T11:13:57.399 INFO:tasks.workunit.client.0.smithi007.stdout: create 250/39 file after the snapshot 2023-06-09T11:13:57.401 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/.snap/snap-subdir-test 2023-06-09T11:13:57.658 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/30 file after the snapshot 2023-06-09T11:13:57.661 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/31 file after the snapshot 2023-06-09T11:13:57.663 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/32 file after the snapshot 2023-06-09T11:13:57.666 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/33 file after the snapshot 2023-06-09T11:13:57.669 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/34 file after the snapshot 2023-06-09T11:13:57.672 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/35 file after the snapshot 2023-06-09T11:13:57.674 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/36 file after the snapshot 2023-06-09T11:13:57.677 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/37 file after the snapshot 2023-06-09T11:13:57.680 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/38 file after the snapshot 2023-06-09T11:13:57.682 INFO:tasks.workunit.client.0.smithi007.stdout: create 251/39 file after the snapshot 2023-06-09T11:13:57.685 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/.snap/snap-subdir-test 2023-06-09T11:13:57.845 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/30 file after the snapshot 2023-06-09T11:13:57.849 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/31 file after the snapshot 2023-06-09T11:13:57.850 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/32 file after the snapshot 2023-06-09T11:13:57.853 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/33 file after the snapshot 2023-06-09T11:13:57.856 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/34 file after the snapshot 2023-06-09T11:13:57.858 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/35 file after the snapshot 2023-06-09T11:13:57.861 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/36 file after the snapshot 2023-06-09T11:13:57.863 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/37 file after the snapshot 2023-06-09T11:13:57.865 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/38 file after the snapshot 2023-06-09T11:13:57.868 INFO:tasks.workunit.client.0.smithi007.stdout: create 252/39 file after the snapshot 2023-06-09T11:13:57.870 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/.snap/snap-subdir-test 2023-06-09T11:13:59.173 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/30 file after the snapshot 2023-06-09T11:13:59.176 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/31 file after the snapshot 2023-06-09T11:13:59.179 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/32 file after the snapshot 2023-06-09T11:13:59.182 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/33 file after the snapshot 2023-06-09T11:13:59.184 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/34 file after the snapshot 2023-06-09T11:13:59.187 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/35 file after the snapshot 2023-06-09T11:13:59.190 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/36 file after the snapshot 2023-06-09T11:13:59.192 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/37 file after the snapshot 2023-06-09T11:13:59.195 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/38 file after the snapshot 2023-06-09T11:13:59.198 INFO:tasks.workunit.client.0.smithi007.stdout: create 253/39 file after the snapshot 2023-06-09T11:13:59.201 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/.snap/snap-subdir-test 2023-06-09T11:13:59.222 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/30 file after the snapshot 2023-06-09T11:13:59.225 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/31 file after the snapshot 2023-06-09T11:13:59.228 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/32 file after the snapshot 2023-06-09T11:13:59.231 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/33 file after the snapshot 2023-06-09T11:13:59.234 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/34 file after the snapshot 2023-06-09T11:13:59.237 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/35 file after the snapshot 2023-06-09T11:13:59.239 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/36 file after the snapshot 2023-06-09T11:13:59.241 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/37 file after the snapshot 2023-06-09T11:13:59.244 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/38 file after the snapshot 2023-06-09T11:13:59.246 INFO:tasks.workunit.client.0.smithi007.stdout: create 254/39 file after the snapshot 2023-06-09T11:13:59.249 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/.snap/snap-subdir-test 2023-06-09T11:13:59.278 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/30 file after the snapshot 2023-06-09T11:13:59.282 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/31 file after the snapshot 2023-06-09T11:13:59.286 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/32 file after the snapshot 2023-06-09T11:13:59.290 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/33 file after the snapshot 2023-06-09T11:13:59.294 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/34 file after the snapshot 2023-06-09T11:13:59.298 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/35 file after the snapshot 2023-06-09T11:13:59.302 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/36 file after the snapshot 2023-06-09T11:13:59.307 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/37 file after the snapshot 2023-06-09T11:13:59.311 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/38 file after the snapshot 2023-06-09T11:13:59.315 INFO:tasks.workunit.client.0.smithi007.stdout: create 255/39 file after the snapshot 2023-06-09T11:13:59.320 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/.snap/snap-subdir-test 2023-06-09T11:13:59.339 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/30 file after the snapshot 2023-06-09T11:13:59.342 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/31 file after the snapshot 2023-06-09T11:13:59.344 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/32 file after the snapshot 2023-06-09T11:13:59.346 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/33 file after the snapshot 2023-06-09T11:13:59.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/34 file after the snapshot 2023-06-09T11:13:59.351 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/35 file after the snapshot 2023-06-09T11:13:59.353 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/36 file after the snapshot 2023-06-09T11:13:59.355 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/37 file after the snapshot 2023-06-09T11:13:59.357 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/38 file after the snapshot 2023-06-09T11:13:59.360 INFO:tasks.workunit.client.0.smithi007.stdout: create 256/39 file after the snapshot 2023-06-09T11:13:59.362 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/.snap/snap-subdir-test 2023-06-09T11:13:59.380 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/30 file after the snapshot 2023-06-09T11:13:59.382 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/31 file after the snapshot 2023-06-09T11:13:59.385 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/32 file after the snapshot 2023-06-09T11:13:59.387 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/33 file after the snapshot 2023-06-09T11:13:59.389 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/34 file after the snapshot 2023-06-09T11:13:59.392 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/35 file after the snapshot 2023-06-09T11:13:59.394 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/36 file after the snapshot 2023-06-09T11:13:59.397 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/37 file after the snapshot 2023-06-09T11:13:59.399 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/38 file after the snapshot 2023-06-09T11:13:59.402 INFO:tasks.workunit.client.0.smithi007.stdout: create 257/39 file after the snapshot 2023-06-09T11:13:59.404 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/.snap/snap-subdir-test 2023-06-09T11:13:59.429 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/30 file after the snapshot 2023-06-09T11:13:59.432 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/31 file after the snapshot 2023-06-09T11:13:59.435 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/32 file after the snapshot 2023-06-09T11:13:59.438 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/33 file after the snapshot 2023-06-09T11:13:59.440 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/34 file after the snapshot 2023-06-09T11:13:59.445 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/35 file after the snapshot 2023-06-09T11:13:59.445 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/36 file after the snapshot 2023-06-09T11:13:59.447 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/37 file after the snapshot 2023-06-09T11:13:59.450 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/38 file after the snapshot 2023-06-09T11:13:59.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 258/39 file after the snapshot 2023-06-09T11:13:59.455 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/.snap/snap-subdir-test 2023-06-09T11:13:59.473 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/30 file after the snapshot 2023-06-09T11:13:59.476 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/31 file after the snapshot 2023-06-09T11:13:59.479 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/32 file after the snapshot 2023-06-09T11:13:59.482 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/33 file after the snapshot 2023-06-09T11:13:59.484 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/34 file after the snapshot 2023-06-09T11:13:59.487 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/35 file after the snapshot 2023-06-09T11:13:59.489 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/36 file after the snapshot 2023-06-09T11:13:59.492 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/37 file after the snapshot 2023-06-09T11:13:59.494 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/38 file after the snapshot 2023-06-09T11:13:59.496 INFO:tasks.workunit.client.0.smithi007.stdout: create 259/39 file after the snapshot 2023-06-09T11:13:59.499 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/.snap/snap-subdir-test 2023-06-09T11:13:59.524 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/30 file after the snapshot 2023-06-09T11:13:59.552 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/31 file after the snapshot 2023-06-09T11:13:59.555 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/32 file after the snapshot 2023-06-09T11:13:59.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/33 file after the snapshot 2023-06-09T11:13:59.560 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/34 file after the snapshot 2023-06-09T11:13:59.562 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/35 file after the snapshot 2023-06-09T11:13:59.565 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/36 file after the snapshot 2023-06-09T11:13:59.567 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/37 file after the snapshot 2023-06-09T11:13:59.569 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/38 file after the snapshot 2023-06-09T11:13:59.572 INFO:tasks.workunit.client.0.smithi007.stdout: create 260/39 file after the snapshot 2023-06-09T11:13:59.574 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/.snap/snap-subdir-test 2023-06-09T11:14:00.027 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/30 file after the snapshot 2023-06-09T11:14:00.029 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/31 file after the snapshot 2023-06-09T11:14:00.032 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/32 file after the snapshot 2023-06-09T11:14:00.034 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/33 file after the snapshot 2023-06-09T11:14:00.037 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/34 file after the snapshot 2023-06-09T11:14:00.040 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/35 file after the snapshot 2023-06-09T11:14:00.042 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/36 file after the snapshot 2023-06-09T11:14:00.045 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/37 file after the snapshot 2023-06-09T11:14:00.048 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/38 file after the snapshot 2023-06-09T11:14:00.050 INFO:tasks.workunit.client.0.smithi007.stdout: create 261/39 file after the snapshot 2023-06-09T11:14:00.053 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/.snap/snap-subdir-test 2023-06-09T11:14:00.099 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/30 file after the snapshot 2023-06-09T11:14:00.102 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/31 file after the snapshot 2023-06-09T11:14:00.104 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/32 file after the snapshot 2023-06-09T11:14:00.106 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/33 file after the snapshot 2023-06-09T11:14:00.109 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/34 file after the snapshot 2023-06-09T11:14:00.111 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/35 file after the snapshot 2023-06-09T11:14:00.113 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/36 file after the snapshot 2023-06-09T11:14:00.115 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/37 file after the snapshot 2023-06-09T11:14:00.118 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/38 file after the snapshot 2023-06-09T11:14:00.120 INFO:tasks.workunit.client.0.smithi007.stdout: create 262/39 file after the snapshot 2023-06-09T11:14:00.123 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/.snap/snap-subdir-test 2023-06-09T11:14:00.156 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/30 file after the snapshot 2023-06-09T11:14:00.159 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/31 file after the snapshot 2023-06-09T11:14:00.161 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/32 file after the snapshot 2023-06-09T11:14:00.164 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/33 file after the snapshot 2023-06-09T11:14:00.166 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/34 file after the snapshot 2023-06-09T11:14:00.168 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/35 file after the snapshot 2023-06-09T11:14:00.171 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/36 file after the snapshot 2023-06-09T11:14:00.173 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/37 file after the snapshot 2023-06-09T11:14:00.176 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/38 file after the snapshot 2023-06-09T11:14:00.178 INFO:tasks.workunit.client.0.smithi007.stdout: create 263/39 file after the snapshot 2023-06-09T11:14:00.181 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/.snap/snap-subdir-test 2023-06-09T11:14:00.199 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/30 file after the snapshot 2023-06-09T11:14:00.202 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/31 file after the snapshot 2023-06-09T11:14:00.204 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/32 file after the snapshot 2023-06-09T11:14:00.207 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/33 file after the snapshot 2023-06-09T11:14:00.209 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/34 file after the snapshot 2023-06-09T11:14:00.212 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/35 file after the snapshot 2023-06-09T11:14:00.214 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/36 file after the snapshot 2023-06-09T11:14:00.216 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/37 file after the snapshot 2023-06-09T11:14:00.219 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/38 file after the snapshot 2023-06-09T11:14:00.221 INFO:tasks.workunit.client.0.smithi007.stdout: create 264/39 file after the snapshot 2023-06-09T11:14:00.223 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/.snap/snap-subdir-test 2023-06-09T11:14:00.244 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/30 file after the snapshot 2023-06-09T11:14:00.246 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/31 file after the snapshot 2023-06-09T11:14:00.249 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/32 file after the snapshot 2023-06-09T11:14:00.251 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/33 file after the snapshot 2023-06-09T11:14:00.253 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/34 file after the snapshot 2023-06-09T11:14:00.255 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/35 file after the snapshot 2023-06-09T11:14:00.258 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/36 file after the snapshot 2023-06-09T11:14:00.260 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/37 file after the snapshot 2023-06-09T11:14:00.263 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/38 file after the snapshot 2023-06-09T11:14:00.265 INFO:tasks.workunit.client.0.smithi007.stdout: create 265/39 file after the snapshot 2023-06-09T11:14:00.267 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/.snap/snap-subdir-test 2023-06-09T11:14:00.286 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/30 file after the snapshot 2023-06-09T11:14:00.289 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/31 file after the snapshot 2023-06-09T11:14:00.291 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/32 file after the snapshot 2023-06-09T11:14:00.294 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/33 file after the snapshot 2023-06-09T11:14:00.297 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/34 file after the snapshot 2023-06-09T11:14:00.300 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/35 file after the snapshot 2023-06-09T11:14:00.302 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/36 file after the snapshot 2023-06-09T11:14:00.305 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/37 file after the snapshot 2023-06-09T11:14:00.308 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/38 file after the snapshot 2023-06-09T11:14:00.310 INFO:tasks.workunit.client.0.smithi007.stdout: create 266/39 file after the snapshot 2023-06-09T11:14:00.313 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/.snap/snap-subdir-test 2023-06-09T11:14:00.336 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/30 file after the snapshot 2023-06-09T11:14:00.338 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/31 file after the snapshot 2023-06-09T11:14:00.341 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/32 file after the snapshot 2023-06-09T11:14:00.343 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/33 file after the snapshot 2023-06-09T11:14:00.346 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/34 file after the snapshot 2023-06-09T11:14:00.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/35 file after the snapshot 2023-06-09T11:14:00.350 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/36 file after the snapshot 2023-06-09T11:14:00.352 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/37 file after the snapshot 2023-06-09T11:14:00.354 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/38 file after the snapshot 2023-06-09T11:14:00.357 INFO:tasks.workunit.client.0.smithi007.stdout: create 267/39 file after the snapshot 2023-06-09T11:14:00.359 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/.snap/snap-subdir-test 2023-06-09T11:14:00.379 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/30 file after the snapshot 2023-06-09T11:14:00.381 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/31 file after the snapshot 2023-06-09T11:14:00.383 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/32 file after the snapshot 2023-06-09T11:14:00.386 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/33 file after the snapshot 2023-06-09T11:14:00.388 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/34 file after the snapshot 2023-06-09T11:14:00.390 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/35 file after the snapshot 2023-06-09T11:14:00.392 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/36 file after the snapshot 2023-06-09T11:14:00.395 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/37 file after the snapshot 2023-06-09T11:14:00.397 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/38 file after the snapshot 2023-06-09T11:14:00.400 INFO:tasks.workunit.client.0.smithi007.stdout: create 268/39 file after the snapshot 2023-06-09T11:14:00.402 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/.snap/snap-subdir-test 2023-06-09T11:14:00.422 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/30 file after the snapshot 2023-06-09T11:14:00.424 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/31 file after the snapshot 2023-06-09T11:14:00.427 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/32 file after the snapshot 2023-06-09T11:14:00.429 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/33 file after the snapshot 2023-06-09T11:14:00.432 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/34 file after the snapshot 2023-06-09T11:14:00.435 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/35 file after the snapshot 2023-06-09T11:14:00.437 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/36 file after the snapshot 2023-06-09T11:14:00.440 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/37 file after the snapshot 2023-06-09T11:14:00.443 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/38 file after the snapshot 2023-06-09T11:14:00.446 INFO:tasks.workunit.client.0.smithi007.stdout: create 269/39 file after the snapshot 2023-06-09T11:14:00.448 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/.snap/snap-subdir-test 2023-06-09T11:14:00.467 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/30 file after the snapshot 2023-06-09T11:14:00.470 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/31 file after the snapshot 2023-06-09T11:14:00.472 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/32 file after the snapshot 2023-06-09T11:14:00.475 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/33 file after the snapshot 2023-06-09T11:14:00.477 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/34 file after the snapshot 2023-06-09T11:14:00.479 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/35 file after the snapshot 2023-06-09T11:14:00.482 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/36 file after the snapshot 2023-06-09T11:14:00.484 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/37 file after the snapshot 2023-06-09T11:14:00.486 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/38 file after the snapshot 2023-06-09T11:14:00.489 INFO:tasks.workunit.client.0.smithi007.stdout: create 270/39 file after the snapshot 2023-06-09T11:14:00.491 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/.snap/snap-subdir-test 2023-06-09T11:14:00.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/30 file after the snapshot 2023-06-09T11:14:00.512 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/31 file after the snapshot 2023-06-09T11:14:00.515 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/32 file after the snapshot 2023-06-09T11:14:00.517 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/33 file after the snapshot 2023-06-09T11:14:00.519 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/34 file after the snapshot 2023-06-09T11:14:00.522 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/35 file after the snapshot 2023-06-09T11:14:00.524 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/36 file after the snapshot 2023-06-09T11:14:00.526 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/37 file after the snapshot 2023-06-09T11:14:00.529 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/38 file after the snapshot 2023-06-09T11:14:00.531 INFO:tasks.workunit.client.0.smithi007.stdout: create 271/39 file after the snapshot 2023-06-09T11:14:00.533 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/.snap/snap-subdir-test 2023-06-09T11:14:00.562 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/30 file after the snapshot 2023-06-09T11:14:00.564 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/31 file after the snapshot 2023-06-09T11:14:00.566 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/32 file after the snapshot 2023-06-09T11:14:00.568 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/33 file after the snapshot 2023-06-09T11:14:00.571 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/34 file after the snapshot 2023-06-09T11:14:00.573 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/35 file after the snapshot 2023-06-09T11:14:00.575 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/36 file after the snapshot 2023-06-09T11:14:00.578 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/37 file after the snapshot 2023-06-09T11:14:00.580 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/38 file after the snapshot 2023-06-09T11:14:00.582 INFO:tasks.workunit.client.0.smithi007.stdout: create 272/39 file after the snapshot 2023-06-09T11:14:00.585 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/.snap/snap-subdir-test 2023-06-09T11:14:00.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/30 file after the snapshot 2023-06-09T11:14:00.613 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/31 file after the snapshot 2023-06-09T11:14:00.615 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/32 file after the snapshot 2023-06-09T11:14:00.618 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/33 file after the snapshot 2023-06-09T11:14:00.620 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/34 file after the snapshot 2023-06-09T11:14:00.622 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/35 file after the snapshot 2023-06-09T11:14:00.625 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/36 file after the snapshot 2023-06-09T11:14:00.627 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/37 file after the snapshot 2023-06-09T11:14:00.629 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/38 file after the snapshot 2023-06-09T11:14:00.632 INFO:tasks.workunit.client.0.smithi007.stdout: create 273/39 file after the snapshot 2023-06-09T11:14:00.634 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/.snap/snap-subdir-test 2023-06-09T11:14:00.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/30 file after the snapshot 2023-06-09T11:14:00.658 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/31 file after the snapshot 2023-06-09T11:14:00.660 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/32 file after the snapshot 2023-06-09T11:14:00.663 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/33 file after the snapshot 2023-06-09T11:14:00.665 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/34 file after the snapshot 2023-06-09T11:14:00.668 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/35 file after the snapshot 2023-06-09T11:14:00.670 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/36 file after the snapshot 2023-06-09T11:14:00.673 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/37 file after the snapshot 2023-06-09T11:14:00.676 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/38 file after the snapshot 2023-06-09T11:14:00.679 INFO:tasks.workunit.client.0.smithi007.stdout: create 274/39 file after the snapshot 2023-06-09T11:14:00.681 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/.snap/snap-subdir-test 2023-06-09T11:14:00.699 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/30 file after the snapshot 2023-06-09T11:14:00.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/31 file after the snapshot 2023-06-09T11:14:00.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/32 file after the snapshot 2023-06-09T11:14:00.706 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/33 file after the snapshot 2023-06-09T11:14:00.709 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/34 file after the snapshot 2023-06-09T11:14:00.711 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/35 file after the snapshot 2023-06-09T11:14:00.714 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/36 file after the snapshot 2023-06-09T11:14:00.716 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/37 file after the snapshot 2023-06-09T11:14:00.718 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/38 file after the snapshot 2023-06-09T11:14:00.720 INFO:tasks.workunit.client.0.smithi007.stdout: create 275/39 file after the snapshot 2023-06-09T11:14:00.722 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/.snap/snap-subdir-test 2023-06-09T11:14:01.340 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/30 file after the snapshot 2023-06-09T11:14:01.342 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/31 file after the snapshot 2023-06-09T11:14:01.345 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/32 file after the snapshot 2023-06-09T11:14:01.348 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/33 file after the snapshot 2023-06-09T11:14:01.350 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/34 file after the snapshot 2023-06-09T11:14:01.353 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/35 file after the snapshot 2023-06-09T11:14:01.356 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/36 file after the snapshot 2023-06-09T11:14:01.358 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/37 file after the snapshot 2023-06-09T11:14:01.361 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/38 file after the snapshot 2023-06-09T11:14:01.363 INFO:tasks.workunit.client.0.smithi007.stdout: create 276/39 file after the snapshot 2023-06-09T11:14:01.366 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/.snap/snap-subdir-test 2023-06-09T11:14:01.384 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/30 file after the snapshot 2023-06-09T11:14:01.387 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/31 file after the snapshot 2023-06-09T11:14:01.390 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/32 file after the snapshot 2023-06-09T11:14:01.393 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/33 file after the snapshot 2023-06-09T11:14:01.395 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/34 file after the snapshot 2023-06-09T11:14:01.398 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/35 file after the snapshot 2023-06-09T11:14:01.400 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/36 file after the snapshot 2023-06-09T11:14:01.403 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/37 file after the snapshot 2023-06-09T11:14:01.405 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/38 file after the snapshot 2023-06-09T11:14:01.408 INFO:tasks.workunit.client.0.smithi007.stdout: create 277/39 file after the snapshot 2023-06-09T11:14:01.410 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/.snap/snap-subdir-test 2023-06-09T11:14:01.432 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/30 file after the snapshot 2023-06-09T11:14:01.438 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/31 file after the snapshot 2023-06-09T11:14:01.443 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/32 file after the snapshot 2023-06-09T11:14:01.447 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/33 file after the snapshot 2023-06-09T11:14:01.452 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/34 file after the snapshot 2023-06-09T11:14:01.456 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/35 file after the snapshot 2023-06-09T11:14:01.460 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/36 file after the snapshot 2023-06-09T11:14:01.464 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/37 file after the snapshot 2023-06-09T11:14:01.468 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/38 file after the snapshot 2023-06-09T11:14:01.472 INFO:tasks.workunit.client.0.smithi007.stdout: create 278/39 file after the snapshot 2023-06-09T11:14:01.477 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/.snap/snap-subdir-test 2023-06-09T11:14:01.495 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/30 file after the snapshot 2023-06-09T11:14:01.498 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/31 file after the snapshot 2023-06-09T11:14:01.500 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/32 file after the snapshot 2023-06-09T11:14:01.502 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/33 file after the snapshot 2023-06-09T11:14:01.505 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/34 file after the snapshot 2023-06-09T11:14:01.508 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/35 file after the snapshot 2023-06-09T11:14:01.510 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/36 file after the snapshot 2023-06-09T11:14:01.512 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/37 file after the snapshot 2023-06-09T11:14:01.515 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/38 file after the snapshot 2023-06-09T11:14:01.517 INFO:tasks.workunit.client.0.smithi007.stdout: create 279/39 file after the snapshot 2023-06-09T11:14:01.520 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/.snap/snap-subdir-test 2023-06-09T11:14:01.538 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/30 file after the snapshot 2023-06-09T11:14:01.545 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/31 file after the snapshot 2023-06-09T11:14:01.547 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/32 file after the snapshot 2023-06-09T11:14:01.550 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/33 file after the snapshot 2023-06-09T11:14:01.552 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/34 file after the snapshot 2023-06-09T11:14:01.555 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/35 file after the snapshot 2023-06-09T11:14:01.558 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/36 file after the snapshot 2023-06-09T11:14:01.560 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/37 file after the snapshot 2023-06-09T11:14:01.563 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/38 file after the snapshot 2023-06-09T11:14:01.565 INFO:tasks.workunit.client.0.smithi007.stdout: create 280/39 file after the snapshot 2023-06-09T11:14:01.567 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/.snap/snap-subdir-test 2023-06-09T11:14:01.602 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/30 file after the snapshot 2023-06-09T11:14:01.604 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/31 file after the snapshot 2023-06-09T11:14:01.606 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/32 file after the snapshot 2023-06-09T11:14:01.609 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/33 file after the snapshot 2023-06-09T11:14:01.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/34 file after the snapshot 2023-06-09T11:14:01.614 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/35 file after the snapshot 2023-06-09T11:14:01.616 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/36 file after the snapshot 2023-06-09T11:14:01.618 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/37 file after the snapshot 2023-06-09T11:14:01.621 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/38 file after the snapshot 2023-06-09T11:14:01.624 INFO:tasks.workunit.client.0.smithi007.stdout: create 281/39 file after the snapshot 2023-06-09T11:14:01.626 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/.snap/snap-subdir-test 2023-06-09T11:14:01.644 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/30 file after the snapshot 2023-06-09T11:14:01.646 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/31 file after the snapshot 2023-06-09T11:14:01.649 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/32 file after the snapshot 2023-06-09T11:14:01.651 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/33 file after the snapshot 2023-06-09T11:14:01.653 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/34 file after the snapshot 2023-06-09T11:14:01.656 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/35 file after the snapshot 2023-06-09T11:14:01.658 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/36 file after the snapshot 2023-06-09T11:14:01.660 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/37 file after the snapshot 2023-06-09T11:14:01.663 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/38 file after the snapshot 2023-06-09T11:14:01.665 INFO:tasks.workunit.client.0.smithi007.stdout: create 282/39 file after the snapshot 2023-06-09T11:14:01.667 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/.snap/snap-subdir-test 2023-06-09T11:14:01.690 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/30 file after the snapshot 2023-06-09T11:14:01.692 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/31 file after the snapshot 2023-06-09T11:14:01.695 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/32 file after the snapshot 2023-06-09T11:14:01.697 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/33 file after the snapshot 2023-06-09T11:14:01.699 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/34 file after the snapshot 2023-06-09T11:14:01.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/35 file after the snapshot 2023-06-09T11:14:01.704 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/36 file after the snapshot 2023-06-09T11:14:01.706 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/37 file after the snapshot 2023-06-09T11:14:01.708 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/38 file after the snapshot 2023-06-09T11:14:01.711 INFO:tasks.workunit.client.0.smithi007.stdout: create 283/39 file after the snapshot 2023-06-09T11:14:01.713 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/.snap/snap-subdir-test 2023-06-09T11:14:01.731 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/30 file after the snapshot 2023-06-09T11:14:01.733 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/31 file after the snapshot 2023-06-09T11:14:01.735 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/32 file after the snapshot 2023-06-09T11:14:01.737 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/33 file after the snapshot 2023-06-09T11:14:01.739 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/34 file after the snapshot 2023-06-09T11:14:01.741 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/35 file after the snapshot 2023-06-09T11:14:01.744 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/36 file after the snapshot 2023-06-09T11:14:01.746 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/37 file after the snapshot 2023-06-09T11:14:01.749 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/38 file after the snapshot 2023-06-09T11:14:01.751 INFO:tasks.workunit.client.0.smithi007.stdout: create 284/39 file after the snapshot 2023-06-09T11:14:01.753 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/.snap/snap-subdir-test 2023-06-09T11:14:01.776 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/30 file after the snapshot 2023-06-09T11:14:01.778 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/31 file after the snapshot 2023-06-09T11:14:01.781 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/32 file after the snapshot 2023-06-09T11:14:01.783 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/33 file after the snapshot 2023-06-09T11:14:01.785 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/34 file after the snapshot 2023-06-09T11:14:01.787 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/35 file after the snapshot 2023-06-09T11:14:01.790 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/36 file after the snapshot 2023-06-09T11:14:01.792 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/37 file after the snapshot 2023-06-09T11:14:01.794 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/38 file after the snapshot 2023-06-09T11:14:01.797 INFO:tasks.workunit.client.0.smithi007.stdout: create 285/39 file after the snapshot 2023-06-09T11:14:01.799 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/.snap/snap-subdir-test 2023-06-09T11:14:01.817 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/30 file after the snapshot 2023-06-09T11:14:01.820 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/31 file after the snapshot 2023-06-09T11:14:01.822 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/32 file after the snapshot 2023-06-09T11:14:01.824 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/33 file after the snapshot 2023-06-09T11:14:01.827 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/34 file after the snapshot 2023-06-09T11:14:01.829 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/35 file after the snapshot 2023-06-09T11:14:01.831 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/36 file after the snapshot 2023-06-09T11:14:01.833 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/37 file after the snapshot 2023-06-09T11:14:01.836 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/38 file after the snapshot 2023-06-09T11:14:01.838 INFO:tasks.workunit.client.0.smithi007.stdout: create 286/39 file after the snapshot 2023-06-09T11:14:01.840 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/.snap/snap-subdir-test 2023-06-09T11:14:01.856 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/30 file after the snapshot 2023-06-09T11:14:01.859 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/31 file after the snapshot 2023-06-09T11:14:01.861 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/32 file after the snapshot 2023-06-09T11:14:01.864 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/33 file after the snapshot 2023-06-09T11:14:01.866 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/34 file after the snapshot 2023-06-09T11:14:01.869 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/35 file after the snapshot 2023-06-09T11:14:01.872 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/36 file after the snapshot 2023-06-09T11:14:01.875 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/37 file after the snapshot 2023-06-09T11:14:01.877 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/38 file after the snapshot 2023-06-09T11:14:01.880 INFO:tasks.workunit.client.0.smithi007.stdout: create 287/39 file after the snapshot 2023-06-09T11:14:01.882 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/.snap/snap-subdir-test 2023-06-09T11:14:01.903 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/30 file after the snapshot 2023-06-09T11:14:01.905 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/31 file after the snapshot 2023-06-09T11:14:01.908 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/32 file after the snapshot 2023-06-09T11:14:01.910 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/33 file after the snapshot 2023-06-09T11:14:01.913 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/34 file after the snapshot 2023-06-09T11:14:01.915 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/35 file after the snapshot 2023-06-09T11:14:01.918 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/36 file after the snapshot 2023-06-09T11:14:01.920 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/37 file after the snapshot 2023-06-09T11:14:01.922 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/38 file after the snapshot 2023-06-09T11:14:01.924 INFO:tasks.workunit.client.0.smithi007.stdout: create 288/39 file after the snapshot 2023-06-09T11:14:01.927 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/.snap/snap-subdir-test 2023-06-09T11:14:01.945 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/30 file after the snapshot 2023-06-09T11:14:01.947 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/31 file after the snapshot 2023-06-09T11:14:01.949 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/32 file after the snapshot 2023-06-09T11:14:01.952 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/33 file after the snapshot 2023-06-09T11:14:01.954 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/34 file after the snapshot 2023-06-09T11:14:01.956 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/35 file after the snapshot 2023-06-09T11:14:01.959 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/36 file after the snapshot 2023-06-09T11:14:01.961 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/37 file after the snapshot 2023-06-09T11:14:01.963 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/38 file after the snapshot 2023-06-09T11:14:01.965 INFO:tasks.workunit.client.0.smithi007.stdout: create 289/39 file after the snapshot 2023-06-09T11:14:01.967 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/.snap/snap-subdir-test 2023-06-09T11:14:01.985 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/30 file after the snapshot 2023-06-09T11:14:01.988 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/31 file after the snapshot 2023-06-09T11:14:01.990 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/32 file after the snapshot 2023-06-09T11:14:01.993 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/33 file after the snapshot 2023-06-09T11:14:01.995 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/34 file after the snapshot 2023-06-09T11:14:01.997 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/35 file after the snapshot 2023-06-09T11:14:01.999 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/36 file after the snapshot 2023-06-09T11:14:02.001 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/37 file after the snapshot 2023-06-09T11:14:02.004 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/38 file after the snapshot 2023-06-09T11:14:02.006 INFO:tasks.workunit.client.0.smithi007.stdout: create 290/39 file after the snapshot 2023-06-09T11:14:02.009 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/.snap/snap-subdir-test 2023-06-09T11:14:02.527 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/30 file after the snapshot 2023-06-09T11:14:02.529 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/31 file after the snapshot 2023-06-09T11:14:02.532 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/32 file after the snapshot 2023-06-09T11:14:02.534 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/33 file after the snapshot 2023-06-09T11:14:02.537 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/34 file after the snapshot 2023-06-09T11:14:02.539 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/35 file after the snapshot 2023-06-09T11:14:02.544 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/36 file after the snapshot 2023-06-09T11:14:02.557 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/37 file after the snapshot 2023-06-09T11:14:02.560 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/38 file after the snapshot 2023-06-09T11:14:02.562 INFO:tasks.workunit.client.0.smithi007.stdout: create 291/39 file after the snapshot 2023-06-09T11:14:02.564 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/.snap/snap-subdir-test 2023-06-09T11:14:02.595 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/30 file after the snapshot 2023-06-09T11:14:02.598 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/31 file after the snapshot 2023-06-09T11:14:02.601 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/32 file after the snapshot 2023-06-09T11:14:02.604 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/33 file after the snapshot 2023-06-09T11:14:02.606 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/34 file after the snapshot 2023-06-09T11:14:02.609 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/35 file after the snapshot 2023-06-09T11:14:02.611 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/36 file after the snapshot 2023-06-09T11:14:02.614 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/37 file after the snapshot 2023-06-09T11:14:02.616 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/38 file after the snapshot 2023-06-09T11:14:02.619 INFO:tasks.workunit.client.0.smithi007.stdout: create 292/39 file after the snapshot 2023-06-09T11:14:02.621 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/.snap/snap-subdir-test 2023-06-09T11:14:02.639 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/30 file after the snapshot 2023-06-09T11:14:02.641 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/31 file after the snapshot 2023-06-09T11:14:02.644 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/32 file after the snapshot 2023-06-09T11:14:02.646 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/33 file after the snapshot 2023-06-09T11:14:02.648 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/34 file after the snapshot 2023-06-09T11:14:02.651 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/35 file after the snapshot 2023-06-09T11:14:02.653 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/36 file after the snapshot 2023-06-09T11:14:02.655 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/37 file after the snapshot 2023-06-09T11:14:02.658 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/38 file after the snapshot 2023-06-09T11:14:02.660 INFO:tasks.workunit.client.0.smithi007.stdout: create 293/39 file after the snapshot 2023-06-09T11:14:02.662 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/.snap/snap-subdir-test 2023-06-09T11:14:02.681 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/30 file after the snapshot 2023-06-09T11:14:02.684 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/31 file after the snapshot 2023-06-09T11:14:02.686 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/32 file after the snapshot 2023-06-09T11:14:02.688 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/33 file after the snapshot 2023-06-09T11:14:02.691 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/34 file after the snapshot 2023-06-09T11:14:02.693 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/35 file after the snapshot 2023-06-09T11:14:02.696 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/36 file after the snapshot 2023-06-09T11:14:02.698 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/37 file after the snapshot 2023-06-09T11:14:02.700 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/38 file after the snapshot 2023-06-09T11:14:02.702 INFO:tasks.workunit.client.0.smithi007.stdout: create 294/39 file after the snapshot 2023-06-09T11:14:02.705 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/.snap/snap-subdir-test 2023-06-09T11:14:02.723 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/30 file after the snapshot 2023-06-09T11:14:02.726 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/31 file after the snapshot 2023-06-09T11:14:02.728 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/32 file after the snapshot 2023-06-09T11:14:02.731 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/33 file after the snapshot 2023-06-09T11:14:02.733 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/34 file after the snapshot 2023-06-09T11:14:02.735 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/35 file after the snapshot 2023-06-09T11:14:02.738 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/36 file after the snapshot 2023-06-09T11:14:02.740 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/37 file after the snapshot 2023-06-09T11:14:02.742 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/38 file after the snapshot 2023-06-09T11:14:02.745 INFO:tasks.workunit.client.0.smithi007.stdout: create 295/39 file after the snapshot 2023-06-09T11:14:02.747 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/.snap/snap-subdir-test 2023-06-09T11:14:02.766 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/30 file after the snapshot 2023-06-09T11:14:02.768 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/31 file after the snapshot 2023-06-09T11:14:02.770 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/32 file after the snapshot 2023-06-09T11:14:02.773 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/33 file after the snapshot 2023-06-09T11:14:02.775 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/34 file after the snapshot 2023-06-09T11:14:02.777 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/35 file after the snapshot 2023-06-09T11:14:02.779 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/36 file after the snapshot 2023-06-09T11:14:02.781 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/37 file after the snapshot 2023-06-09T11:14:02.784 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/38 file after the snapshot 2023-06-09T11:14:02.786 INFO:tasks.workunit.client.0.smithi007.stdout: create 296/39 file after the snapshot 2023-06-09T11:14:02.788 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/.snap/snap-subdir-test 2023-06-09T11:14:02.810 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/30 file after the snapshot 2023-06-09T11:14:02.812 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/31 file after the snapshot 2023-06-09T11:14:02.815 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/32 file after the snapshot 2023-06-09T11:14:02.817 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/33 file after the snapshot 2023-06-09T11:14:02.819 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/34 file after the snapshot 2023-06-09T11:14:02.821 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/35 file after the snapshot 2023-06-09T11:14:02.823 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/36 file after the snapshot 2023-06-09T11:14:02.826 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/37 file after the snapshot 2023-06-09T11:14:02.828 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/38 file after the snapshot 2023-06-09T11:14:03.072 INFO:tasks.workunit.client.0.smithi007.stdout: create 297/39 file after the snapshot 2023-06-09T11:14:03.074 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/.snap/snap-subdir-test 2023-06-09T11:14:03.095 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/30 file after the snapshot 2023-06-09T11:14:03.097 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/31 file after the snapshot 2023-06-09T11:14:03.100 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/32 file after the snapshot 2023-06-09T11:14:03.102 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/33 file after the snapshot 2023-06-09T11:14:03.105 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/34 file after the snapshot 2023-06-09T11:14:03.107 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/35 file after the snapshot 2023-06-09T11:14:03.110 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/36 file after the snapshot 2023-06-09T11:14:03.112 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/37 file after the snapshot 2023-06-09T11:14:03.114 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/38 file after the snapshot 2023-06-09T11:14:03.116 INFO:tasks.workunit.client.0.smithi007.stdout: create 298/39 file after the snapshot 2023-06-09T11:14:03.119 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/.snap/snap-subdir-test 2023-06-09T11:14:03.139 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/30 file after the snapshot 2023-06-09T11:14:03.142 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/31 file after the snapshot 2023-06-09T11:14:03.144 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/32 file after the snapshot 2023-06-09T11:14:03.147 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/33 file after the snapshot 2023-06-09T11:14:03.149 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/34 file after the snapshot 2023-06-09T11:14:03.151 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/35 file after the snapshot 2023-06-09T11:14:03.154 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/36 file after the snapshot 2023-06-09T11:14:03.156 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/37 file after the snapshot 2023-06-09T11:14:03.158 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/38 file after the snapshot 2023-06-09T11:14:03.160 INFO:tasks.workunit.client.0.smithi007.stdout: create 299/39 file after the snapshot 2023-06-09T11:14:03.163 INFO:tasks.workunit.client.0.smithi007.stdout:Take second snapshot .snap/test2 2023-06-09T11:14:03.256 INFO:tasks.workunit.client.0.smithi007.stdout:Copy content of .snap/test1 to copyofsnap1 ... 2023-06-09T11:14:06.231 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1' -> 'copyofsnap1/test1' 2023-06-09T11:14:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100' -> 'copyofsnap1/test1/100' 2023-06-09T11:14:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/10' -> 'copyofsnap1/test1/100/10' 2023-06-09T11:14:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/11' -> 'copyofsnap1/test1/100/11' 2023-06-09T11:14:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/12' -> 'copyofsnap1/test1/100/12' 2023-06-09T11:14:06.232 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/13' -> 'copyofsnap1/test1/100/13' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/14' -> 'copyofsnap1/test1/100/14' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/15' -> 'copyofsnap1/test1/100/15' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/16' -> 'copyofsnap1/test1/100/16' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/17' -> 'copyofsnap1/test1/100/17' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/18' -> 'copyofsnap1/test1/100/18' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/19' -> 'copyofsnap1/test1/100/19' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/100/20' -> 'copyofsnap1/test1/100/20' 2023-06-09T11:14:06.233 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101' -> 'copyofsnap1/test1/101' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/10' -> 'copyofsnap1/test1/101/10' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/11' -> 'copyofsnap1/test1/101/11' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/12' -> 'copyofsnap1/test1/101/12' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/13' -> 'copyofsnap1/test1/101/13' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/14' -> 'copyofsnap1/test1/101/14' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/15' -> 'copyofsnap1/test1/101/15' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/16' -> 'copyofsnap1/test1/101/16' 2023-06-09T11:14:06.234 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/17' -> 'copyofsnap1/test1/101/17' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/18' -> 'copyofsnap1/test1/101/18' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/19' -> 'copyofsnap1/test1/101/19' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/101/20' -> 'copyofsnap1/test1/101/20' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102' -> 'copyofsnap1/test1/102' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/10' -> 'copyofsnap1/test1/102/10' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/11' -> 'copyofsnap1/test1/102/11' 2023-06-09T11:14:06.235 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/12' -> 'copyofsnap1/test1/102/12' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/13' -> 'copyofsnap1/test1/102/13' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/14' -> 'copyofsnap1/test1/102/14' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/15' -> 'copyofsnap1/test1/102/15' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/16' -> 'copyofsnap1/test1/102/16' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/17' -> 'copyofsnap1/test1/102/17' 2023-06-09T11:14:06.236 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/18' -> 'copyofsnap1/test1/102/18' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/19' -> 'copyofsnap1/test1/102/19' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/102/20' -> 'copyofsnap1/test1/102/20' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103' -> 'copyofsnap1/test1/103' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/10' -> 'copyofsnap1/test1/103/10' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/11' -> 'copyofsnap1/test1/103/11' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/12' -> 'copyofsnap1/test1/103/12' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/13' -> 'copyofsnap1/test1/103/13' 2023-06-09T11:14:06.237 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/14' -> 'copyofsnap1/test1/103/14' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/15' -> 'copyofsnap1/test1/103/15' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/16' -> 'copyofsnap1/test1/103/16' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/17' -> 'copyofsnap1/test1/103/17' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/18' -> 'copyofsnap1/test1/103/18' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/19' -> 'copyofsnap1/test1/103/19' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/103/20' -> 'copyofsnap1/test1/103/20' 2023-06-09T11:14:06.238 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104' -> 'copyofsnap1/test1/104' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/10' -> 'copyofsnap1/test1/104/10' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/11' -> 'copyofsnap1/test1/104/11' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/12' -> 'copyofsnap1/test1/104/12' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/13' -> 'copyofsnap1/test1/104/13' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/14' -> 'copyofsnap1/test1/104/14' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/15' -> 'copyofsnap1/test1/104/15' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/16' -> 'copyofsnap1/test1/104/16' 2023-06-09T11:14:06.239 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/17' -> 'copyofsnap1/test1/104/17' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/18' -> 'copyofsnap1/test1/104/18' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/19' -> 'copyofsnap1/test1/104/19' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/104/20' -> 'copyofsnap1/test1/104/20' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105' -> 'copyofsnap1/test1/105' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/10' -> 'copyofsnap1/test1/105/10' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/11' -> 'copyofsnap1/test1/105/11' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/12' -> 'copyofsnap1/test1/105/12' 2023-06-09T11:14:06.240 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/13' -> 'copyofsnap1/test1/105/13' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/14' -> 'copyofsnap1/test1/105/14' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/15' -> 'copyofsnap1/test1/105/15' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/16' -> 'copyofsnap1/test1/105/16' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/17' -> 'copyofsnap1/test1/105/17' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/18' -> 'copyofsnap1/test1/105/18' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/19' -> 'copyofsnap1/test1/105/19' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/105/20' -> 'copyofsnap1/test1/105/20' 2023-06-09T11:14:06.241 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106' -> 'copyofsnap1/test1/106' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/10' -> 'copyofsnap1/test1/106/10' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/11' -> 'copyofsnap1/test1/106/11' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/12' -> 'copyofsnap1/test1/106/12' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/13' -> 'copyofsnap1/test1/106/13' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/14' -> 'copyofsnap1/test1/106/14' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/15' -> 'copyofsnap1/test1/106/15' 2023-06-09T11:14:06.242 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/16' -> 'copyofsnap1/test1/106/16' 2023-06-09T11:14:09.957 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/17' -> 'copyofsnap1/test1/106/17' 2023-06-09T11:14:09.957 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/18' -> 'copyofsnap1/test1/106/18' 2023-06-09T11:14:09.957 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/19' -> 'copyofsnap1/test1/106/19' 2023-06-09T11:14:09.957 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/106/20' -> 'copyofsnap1/test1/106/20' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107' -> 'copyofsnap1/test1/107' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/10' -> 'copyofsnap1/test1/107/10' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/11' -> 'copyofsnap1/test1/107/11' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/12' -> 'copyofsnap1/test1/107/12' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/13' -> 'copyofsnap1/test1/107/13' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/14' -> 'copyofsnap1/test1/107/14' 2023-06-09T11:14:09.958 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/15' -> 'copyofsnap1/test1/107/15' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/16' -> 'copyofsnap1/test1/107/16' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/17' -> 'copyofsnap1/test1/107/17' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/18' -> 'copyofsnap1/test1/107/18' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/19' -> 'copyofsnap1/test1/107/19' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/107/20' -> 'copyofsnap1/test1/107/20' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108' -> 'copyofsnap1/test1/108' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/10' -> 'copyofsnap1/test1/108/10' 2023-06-09T11:14:09.959 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/11' -> 'copyofsnap1/test1/108/11' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/12' -> 'copyofsnap1/test1/108/12' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/13' -> 'copyofsnap1/test1/108/13' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/14' -> 'copyofsnap1/test1/108/14' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/15' -> 'copyofsnap1/test1/108/15' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/16' -> 'copyofsnap1/test1/108/16' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/17' -> 'copyofsnap1/test1/108/17' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/18' -> 'copyofsnap1/test1/108/18' 2023-06-09T11:14:09.960 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/19' -> 'copyofsnap1/test1/108/19' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/108/20' -> 'copyofsnap1/test1/108/20' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109' -> 'copyofsnap1/test1/109' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/10' -> 'copyofsnap1/test1/109/10' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/11' -> 'copyofsnap1/test1/109/11' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/12' -> 'copyofsnap1/test1/109/12' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/13' -> 'copyofsnap1/test1/109/13' 2023-06-09T11:14:09.961 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/14' -> 'copyofsnap1/test1/109/14' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/15' -> 'copyofsnap1/test1/109/15' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/16' -> 'copyofsnap1/test1/109/16' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/17' -> 'copyofsnap1/test1/109/17' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/18' -> 'copyofsnap1/test1/109/18' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/19' -> 'copyofsnap1/test1/109/19' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/109/20' -> 'copyofsnap1/test1/109/20' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110' -> 'copyofsnap1/test1/110' 2023-06-09T11:14:09.962 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/10' -> 'copyofsnap1/test1/110/10' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/11' -> 'copyofsnap1/test1/110/11' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/12' -> 'copyofsnap1/test1/110/12' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/13' -> 'copyofsnap1/test1/110/13' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/14' -> 'copyofsnap1/test1/110/14' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/15' -> 'copyofsnap1/test1/110/15' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/16' -> 'copyofsnap1/test1/110/16' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/17' -> 'copyofsnap1/test1/110/17' 2023-06-09T11:14:09.963 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/18' -> 'copyofsnap1/test1/110/18' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/19' -> 'copyofsnap1/test1/110/19' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/110/20' -> 'copyofsnap1/test1/110/20' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111' -> 'copyofsnap1/test1/111' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/10' -> 'copyofsnap1/test1/111/10' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/11' -> 'copyofsnap1/test1/111/11' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/12' -> 'copyofsnap1/test1/111/12' 2023-06-09T11:14:09.964 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/13' -> 'copyofsnap1/test1/111/13' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/14' -> 'copyofsnap1/test1/111/14' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/15' -> 'copyofsnap1/test1/111/15' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/16' -> 'copyofsnap1/test1/111/16' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/17' -> 'copyofsnap1/test1/111/17' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/18' -> 'copyofsnap1/test1/111/18' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/19' -> 'copyofsnap1/test1/111/19' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/111/20' -> 'copyofsnap1/test1/111/20' 2023-06-09T11:14:09.965 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112' -> 'copyofsnap1/test1/112' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/10' -> 'copyofsnap1/test1/112/10' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/11' -> 'copyofsnap1/test1/112/11' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/12' -> 'copyofsnap1/test1/112/12' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/13' -> 'copyofsnap1/test1/112/13' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/14' -> 'copyofsnap1/test1/112/14' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/15' -> 'copyofsnap1/test1/112/15' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/16' -> 'copyofsnap1/test1/112/16' 2023-06-09T11:14:09.966 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/17' -> 'copyofsnap1/test1/112/17' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/18' -> 'copyofsnap1/test1/112/18' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/19' -> 'copyofsnap1/test1/112/19' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/112/20' -> 'copyofsnap1/test1/112/20' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113' -> 'copyofsnap1/test1/113' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/10' -> 'copyofsnap1/test1/113/10' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/11' -> 'copyofsnap1/test1/113/11' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/12' -> 'copyofsnap1/test1/113/12' 2023-06-09T11:14:09.967 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/13' -> 'copyofsnap1/test1/113/13' 2023-06-09T11:14:10.770 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/14' -> 'copyofsnap1/test1/113/14' 2023-06-09T11:14:10.771 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/15' -> 'copyofsnap1/test1/113/15' 2023-06-09T11:14:10.771 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/16' -> 'copyofsnap1/test1/113/16' 2023-06-09T11:14:10.771 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/17' -> 'copyofsnap1/test1/113/17' 2023-06-09T11:14:10.772 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/18' -> 'copyofsnap1/test1/113/18' 2023-06-09T11:14:10.772 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/19' -> 'copyofsnap1/test1/113/19' 2023-06-09T11:14:10.772 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/113/20' -> 'copyofsnap1/test1/113/20' 2023-06-09T11:14:10.772 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114' -> 'copyofsnap1/test1/114' 2023-06-09T11:14:10.773 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/10' -> 'copyofsnap1/test1/114/10' 2023-06-09T11:14:10.773 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/11' -> 'copyofsnap1/test1/114/11' 2023-06-09T11:14:10.773 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/12' -> 'copyofsnap1/test1/114/12' 2023-06-09T11:14:10.773 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/13' -> 'copyofsnap1/test1/114/13' 2023-06-09T11:14:10.773 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/14' -> 'copyofsnap1/test1/114/14' 2023-06-09T11:14:10.774 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/15' -> 'copyofsnap1/test1/114/15' 2023-06-09T11:14:10.774 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/16' -> 'copyofsnap1/test1/114/16' 2023-06-09T11:14:10.774 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/17' -> 'copyofsnap1/test1/114/17' 2023-06-09T11:14:10.775 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/18' -> 'copyofsnap1/test1/114/18' 2023-06-09T11:14:10.775 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/19' -> 'copyofsnap1/test1/114/19' 2023-06-09T11:14:10.775 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/114/20' -> 'copyofsnap1/test1/114/20' 2023-06-09T11:14:10.775 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115' -> 'copyofsnap1/test1/115' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/10' -> 'copyofsnap1/test1/115/10' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/11' -> 'copyofsnap1/test1/115/11' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/12' -> 'copyofsnap1/test1/115/12' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/13' -> 'copyofsnap1/test1/115/13' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/14' -> 'copyofsnap1/test1/115/14' 2023-06-09T11:14:10.776 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/15' -> 'copyofsnap1/test1/115/15' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/16' -> 'copyofsnap1/test1/115/16' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/17' -> 'copyofsnap1/test1/115/17' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/18' -> 'copyofsnap1/test1/115/18' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/19' -> 'copyofsnap1/test1/115/19' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/115/20' -> 'copyofsnap1/test1/115/20' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116' -> 'copyofsnap1/test1/116' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/10' -> 'copyofsnap1/test1/116/10' 2023-06-09T11:14:10.777 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/11' -> 'copyofsnap1/test1/116/11' 2023-06-09T11:14:10.778 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/12' -> 'copyofsnap1/test1/116/12' 2023-06-09T11:14:10.778 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/13' -> 'copyofsnap1/test1/116/13' 2023-06-09T11:14:10.778 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/14' -> 'copyofsnap1/test1/116/14' 2023-06-09T11:14:10.778 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/15' -> 'copyofsnap1/test1/116/15' 2023-06-09T11:14:10.778 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/16' -> 'copyofsnap1/test1/116/16' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/17' -> 'copyofsnap1/test1/116/17' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/18' -> 'copyofsnap1/test1/116/18' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/19' -> 'copyofsnap1/test1/116/19' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/116/20' -> 'copyofsnap1/test1/116/20' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117' -> 'copyofsnap1/test1/117' 2023-06-09T11:14:10.779 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/10' -> 'copyofsnap1/test1/117/10' 2023-06-09T11:14:10.780 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/11' -> 'copyofsnap1/test1/117/11' 2023-06-09T11:14:10.781 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/12' -> 'copyofsnap1/test1/117/12' 2023-06-09T11:14:10.781 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/13' -> 'copyofsnap1/test1/117/13' 2023-06-09T11:14:10.781 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/14' -> 'copyofsnap1/test1/117/14' 2023-06-09T11:14:10.782 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/15' -> 'copyofsnap1/test1/117/15' 2023-06-09T11:14:10.782 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/16' -> 'copyofsnap1/test1/117/16' 2023-06-09T11:14:10.782 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/17' -> 'copyofsnap1/test1/117/17' 2023-06-09T11:14:10.782 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/18' -> 'copyofsnap1/test1/117/18' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/19' -> 'copyofsnap1/test1/117/19' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/117/20' -> 'copyofsnap1/test1/117/20' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118' -> 'copyofsnap1/test1/118' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/10' -> 'copyofsnap1/test1/118/10' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/11' -> 'copyofsnap1/test1/118/11' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/12' -> 'copyofsnap1/test1/118/12' 2023-06-09T11:14:10.783 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/13' -> 'copyofsnap1/test1/118/13' 2023-06-09T11:14:10.784 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/14' -> 'copyofsnap1/test1/118/14' 2023-06-09T11:14:10.784 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/15' -> 'copyofsnap1/test1/118/15' 2023-06-09T11:14:10.784 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/16' -> 'copyofsnap1/test1/118/16' 2023-06-09T11:14:10.784 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/17' -> 'copyofsnap1/test1/118/17' 2023-06-09T11:14:10.785 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/18' -> 'copyofsnap1/test1/118/18' 2023-06-09T11:14:10.785 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/19' -> 'copyofsnap1/test1/118/19' 2023-06-09T11:14:10.785 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/118/20' -> 'copyofsnap1/test1/118/20' 2023-06-09T11:14:10.785 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119' -> 'copyofsnap1/test1/119' 2023-06-09T11:14:10.785 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/10' -> 'copyofsnap1/test1/119/10' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/11' -> 'copyofsnap1/test1/119/11' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/12' -> 'copyofsnap1/test1/119/12' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/13' -> 'copyofsnap1/test1/119/13' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/14' -> 'copyofsnap1/test1/119/14' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/15' -> 'copyofsnap1/test1/119/15' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/16' -> 'copyofsnap1/test1/119/16' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/17' -> 'copyofsnap1/test1/119/17' 2023-06-09T11:14:10.786 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/18' -> 'copyofsnap1/test1/119/18' 2023-06-09T11:14:10.787 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/19' -> 'copyofsnap1/test1/119/19' 2023-06-09T11:14:10.787 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/119/20' -> 'copyofsnap1/test1/119/20' 2023-06-09T11:14:10.787 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120' -> 'copyofsnap1/test1/120' 2023-06-09T11:14:10.787 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/10' -> 'copyofsnap1/test1/120/10' 2023-06-09T11:14:10.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:10.835 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:13.421 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/11' -> 'copyofsnap1/test1/120/11' 2023-06-09T11:14:13.421 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/12' -> 'copyofsnap1/test1/120/12' 2023-06-09T11:14:13.421 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/13' -> 'copyofsnap1/test1/120/13' 2023-06-09T11:14:13.421 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/14' -> 'copyofsnap1/test1/120/14' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/15' -> 'copyofsnap1/test1/120/15' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/16' -> 'copyofsnap1/test1/120/16' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/17' -> 'copyofsnap1/test1/120/17' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/18' -> 'copyofsnap1/test1/120/18' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/19' -> 'copyofsnap1/test1/120/19' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/120/20' -> 'copyofsnap1/test1/120/20' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121' -> 'copyofsnap1/test1/121' 2023-06-09T11:14:13.422 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/10' -> 'copyofsnap1/test1/121/10' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/11' -> 'copyofsnap1/test1/121/11' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/12' -> 'copyofsnap1/test1/121/12' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/13' -> 'copyofsnap1/test1/121/13' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/14' -> 'copyofsnap1/test1/121/14' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/15' -> 'copyofsnap1/test1/121/15' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/16' -> 'copyofsnap1/test1/121/16' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/17' -> 'copyofsnap1/test1/121/17' 2023-06-09T11:14:13.423 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/18' -> 'copyofsnap1/test1/121/18' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/19' -> 'copyofsnap1/test1/121/19' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/121/20' -> 'copyofsnap1/test1/121/20' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122' -> 'copyofsnap1/test1/122' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/10' -> 'copyofsnap1/test1/122/10' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/11' -> 'copyofsnap1/test1/122/11' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/12' -> 'copyofsnap1/test1/122/12' 2023-06-09T11:14:13.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/13' -> 'copyofsnap1/test1/122/13' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/14' -> 'copyofsnap1/test1/122/14' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/15' -> 'copyofsnap1/test1/122/15' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/16' -> 'copyofsnap1/test1/122/16' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/17' -> 'copyofsnap1/test1/122/17' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/18' -> 'copyofsnap1/test1/122/18' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/19' -> 'copyofsnap1/test1/122/19' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/122/20' -> 'copyofsnap1/test1/122/20' 2023-06-09T11:14:13.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123' -> 'copyofsnap1/test1/123' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/10' -> 'copyofsnap1/test1/123/10' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/11' -> 'copyofsnap1/test1/123/11' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/12' -> 'copyofsnap1/test1/123/12' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/13' -> 'copyofsnap1/test1/123/13' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/14' -> 'copyofsnap1/test1/123/14' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/15' -> 'copyofsnap1/test1/123/15' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/16' -> 'copyofsnap1/test1/123/16' 2023-06-09T11:14:13.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/17' -> 'copyofsnap1/test1/123/17' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/18' -> 'copyofsnap1/test1/123/18' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/19' -> 'copyofsnap1/test1/123/19' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/123/20' -> 'copyofsnap1/test1/123/20' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124' -> 'copyofsnap1/test1/124' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/10' -> 'copyofsnap1/test1/124/10' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/11' -> 'copyofsnap1/test1/124/11' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/12' -> 'copyofsnap1/test1/124/12' 2023-06-09T11:14:13.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/13' -> 'copyofsnap1/test1/124/13' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/14' -> 'copyofsnap1/test1/124/14' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/15' -> 'copyofsnap1/test1/124/15' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/16' -> 'copyofsnap1/test1/124/16' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/17' -> 'copyofsnap1/test1/124/17' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/18' -> 'copyofsnap1/test1/124/18' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/19' -> 'copyofsnap1/test1/124/19' 2023-06-09T11:14:13.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/124/20' -> 'copyofsnap1/test1/124/20' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125' -> 'copyofsnap1/test1/125' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/10' -> 'copyofsnap1/test1/125/10' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/11' -> 'copyofsnap1/test1/125/11' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/12' -> 'copyofsnap1/test1/125/12' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/13' -> 'copyofsnap1/test1/125/13' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/14' -> 'copyofsnap1/test1/125/14' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/15' -> 'copyofsnap1/test1/125/15' 2023-06-09T11:14:13.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/16' -> 'copyofsnap1/test1/125/16' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/17' -> 'copyofsnap1/test1/125/17' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/18' -> 'copyofsnap1/test1/125/18' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/19' -> 'copyofsnap1/test1/125/19' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/125/20' -> 'copyofsnap1/test1/125/20' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126' -> 'copyofsnap1/test1/126' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/10' -> 'copyofsnap1/test1/126/10' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/11' -> 'copyofsnap1/test1/126/11' 2023-06-09T11:14:13.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/12' -> 'copyofsnap1/test1/126/12' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/13' -> 'copyofsnap1/test1/126/13' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/14' -> 'copyofsnap1/test1/126/14' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/15' -> 'copyofsnap1/test1/126/15' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/16' -> 'copyofsnap1/test1/126/16' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/17' -> 'copyofsnap1/test1/126/17' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/18' -> 'copyofsnap1/test1/126/18' 2023-06-09T11:14:13.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/19' -> 'copyofsnap1/test1/126/19' 2023-06-09T11:14:18.698 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/126/20' -> 'copyofsnap1/test1/126/20' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127' -> 'copyofsnap1/test1/127' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/10' -> 'copyofsnap1/test1/127/10' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/11' -> 'copyofsnap1/test1/127/11' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/12' -> 'copyofsnap1/test1/127/12' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/13' -> 'copyofsnap1/test1/127/13' 2023-06-09T11:14:18.699 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/14' -> 'copyofsnap1/test1/127/14' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/15' -> 'copyofsnap1/test1/127/15' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/16' -> 'copyofsnap1/test1/127/16' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/17' -> 'copyofsnap1/test1/127/17' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/18' -> 'copyofsnap1/test1/127/18' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/19' -> 'copyofsnap1/test1/127/19' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/127/20' -> 'copyofsnap1/test1/127/20' 2023-06-09T11:14:18.700 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128' -> 'copyofsnap1/test1/128' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/10' -> 'copyofsnap1/test1/128/10' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/11' -> 'copyofsnap1/test1/128/11' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/12' -> 'copyofsnap1/test1/128/12' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/13' -> 'copyofsnap1/test1/128/13' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/14' -> 'copyofsnap1/test1/128/14' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/15' -> 'copyofsnap1/test1/128/15' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/16' -> 'copyofsnap1/test1/128/16' 2023-06-09T11:14:18.701 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/17' -> 'copyofsnap1/test1/128/17' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/18' -> 'copyofsnap1/test1/128/18' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/19' -> 'copyofsnap1/test1/128/19' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/128/20' -> 'copyofsnap1/test1/128/20' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129' -> 'copyofsnap1/test1/129' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/10' -> 'copyofsnap1/test1/129/10' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/11' -> 'copyofsnap1/test1/129/11' 2023-06-09T11:14:18.702 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/12' -> 'copyofsnap1/test1/129/12' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/13' -> 'copyofsnap1/test1/129/13' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/14' -> 'copyofsnap1/test1/129/14' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/15' -> 'copyofsnap1/test1/129/15' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/16' -> 'copyofsnap1/test1/129/16' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/17' -> 'copyofsnap1/test1/129/17' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/18' -> 'copyofsnap1/test1/129/18' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/19' -> 'copyofsnap1/test1/129/19' 2023-06-09T11:14:18.703 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/129/20' -> 'copyofsnap1/test1/129/20' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130' -> 'copyofsnap1/test1/130' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/10' -> 'copyofsnap1/test1/130/10' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/11' -> 'copyofsnap1/test1/130/11' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/12' -> 'copyofsnap1/test1/130/12' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/13' -> 'copyofsnap1/test1/130/13' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/14' -> 'copyofsnap1/test1/130/14' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/15' -> 'copyofsnap1/test1/130/15' 2023-06-09T11:14:18.704 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/16' -> 'copyofsnap1/test1/130/16' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/17' -> 'copyofsnap1/test1/130/17' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/18' -> 'copyofsnap1/test1/130/18' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/19' -> 'copyofsnap1/test1/130/19' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/130/20' -> 'copyofsnap1/test1/130/20' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131' -> 'copyofsnap1/test1/131' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/10' -> 'copyofsnap1/test1/131/10' 2023-06-09T11:14:18.705 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/11' -> 'copyofsnap1/test1/131/11' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/12' -> 'copyofsnap1/test1/131/12' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/13' -> 'copyofsnap1/test1/131/13' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/14' -> 'copyofsnap1/test1/131/14' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/15' -> 'copyofsnap1/test1/131/15' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/16' -> 'copyofsnap1/test1/131/16' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/17' -> 'copyofsnap1/test1/131/17' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/18' -> 'copyofsnap1/test1/131/18' 2023-06-09T11:14:18.706 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/19' -> 'copyofsnap1/test1/131/19' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/131/20' -> 'copyofsnap1/test1/131/20' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132' -> 'copyofsnap1/test1/132' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/10' -> 'copyofsnap1/test1/132/10' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/11' -> 'copyofsnap1/test1/132/11' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/12' -> 'copyofsnap1/test1/132/12' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/13' -> 'copyofsnap1/test1/132/13' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/14' -> 'copyofsnap1/test1/132/14' 2023-06-09T11:14:18.707 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/15' -> 'copyofsnap1/test1/132/15' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/16' -> 'copyofsnap1/test1/132/16' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/17' -> 'copyofsnap1/test1/132/17' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/18' -> 'copyofsnap1/test1/132/18' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/19' -> 'copyofsnap1/test1/132/19' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/132/20' -> 'copyofsnap1/test1/132/20' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133' -> 'copyofsnap1/test1/133' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/10' -> 'copyofsnap1/test1/133/10' 2023-06-09T11:14:18.708 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/11' -> 'copyofsnap1/test1/133/11' 2023-06-09T11:14:18.709 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/12' -> 'copyofsnap1/test1/133/12' 2023-06-09T11:14:18.709 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/13' -> 'copyofsnap1/test1/133/13' 2023-06-09T11:14:18.709 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/14' -> 'copyofsnap1/test1/133/14' 2023-06-09T11:14:18.709 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/15' -> 'copyofsnap1/test1/133/15' 2023-06-09T11:14:18.709 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/16' -> 'copyofsnap1/test1/133/16' 2023-06-09T11:14:19.969 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/17' -> 'copyofsnap1/test1/133/17' 2023-06-09T11:14:19.969 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/18' -> 'copyofsnap1/test1/133/18' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/19' -> 'copyofsnap1/test1/133/19' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/133/20' -> 'copyofsnap1/test1/133/20' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134' -> 'copyofsnap1/test1/134' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/10' -> 'copyofsnap1/test1/134/10' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/11' -> 'copyofsnap1/test1/134/11' 2023-06-09T11:14:19.970 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/12' -> 'copyofsnap1/test1/134/12' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/13' -> 'copyofsnap1/test1/134/13' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/14' -> 'copyofsnap1/test1/134/14' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/15' -> 'copyofsnap1/test1/134/15' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/16' -> 'copyofsnap1/test1/134/16' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/17' -> 'copyofsnap1/test1/134/17' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/18' -> 'copyofsnap1/test1/134/18' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/19' -> 'copyofsnap1/test1/134/19' 2023-06-09T11:14:19.971 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/134/20' -> 'copyofsnap1/test1/134/20' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135' -> 'copyofsnap1/test1/135' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/10' -> 'copyofsnap1/test1/135/10' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/11' -> 'copyofsnap1/test1/135/11' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/12' -> 'copyofsnap1/test1/135/12' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/13' -> 'copyofsnap1/test1/135/13' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/14' -> 'copyofsnap1/test1/135/14' 2023-06-09T11:14:19.972 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/15' -> 'copyofsnap1/test1/135/15' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/16' -> 'copyofsnap1/test1/135/16' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/17' -> 'copyofsnap1/test1/135/17' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/18' -> 'copyofsnap1/test1/135/18' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/19' -> 'copyofsnap1/test1/135/19' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/135/20' -> 'copyofsnap1/test1/135/20' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136' -> 'copyofsnap1/test1/136' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/10' -> 'copyofsnap1/test1/136/10' 2023-06-09T11:14:19.973 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/11' -> 'copyofsnap1/test1/136/11' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/12' -> 'copyofsnap1/test1/136/12' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/13' -> 'copyofsnap1/test1/136/13' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/14' -> 'copyofsnap1/test1/136/14' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/15' -> 'copyofsnap1/test1/136/15' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/16' -> 'copyofsnap1/test1/136/16' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/17' -> 'copyofsnap1/test1/136/17' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/18' -> 'copyofsnap1/test1/136/18' 2023-06-09T11:14:19.974 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/19' -> 'copyofsnap1/test1/136/19' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/136/20' -> 'copyofsnap1/test1/136/20' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137' -> 'copyofsnap1/test1/137' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/10' -> 'copyofsnap1/test1/137/10' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/11' -> 'copyofsnap1/test1/137/11' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/12' -> 'copyofsnap1/test1/137/12' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/13' -> 'copyofsnap1/test1/137/13' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/14' -> 'copyofsnap1/test1/137/14' 2023-06-09T11:14:19.975 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/15' -> 'copyofsnap1/test1/137/15' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/16' -> 'copyofsnap1/test1/137/16' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/17' -> 'copyofsnap1/test1/137/17' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/18' -> 'copyofsnap1/test1/137/18' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/19' -> 'copyofsnap1/test1/137/19' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/137/20' -> 'copyofsnap1/test1/137/20' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138' -> 'copyofsnap1/test1/138' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/10' -> 'copyofsnap1/test1/138/10' 2023-06-09T11:14:19.976 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/11' -> 'copyofsnap1/test1/138/11' 2023-06-09T11:14:19.977 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/12' -> 'copyofsnap1/test1/138/12' 2023-06-09T11:14:19.977 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/13' -> 'copyofsnap1/test1/138/13' 2023-06-09T11:14:19.977 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/14' -> 'copyofsnap1/test1/138/14' 2023-06-09T11:14:19.977 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/15' -> 'copyofsnap1/test1/138/15' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/16' -> 'copyofsnap1/test1/138/16' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/17' -> 'copyofsnap1/test1/138/17' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/18' -> 'copyofsnap1/test1/138/18' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/19' -> 'copyofsnap1/test1/138/19' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/138/20' -> 'copyofsnap1/test1/138/20' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139' -> 'copyofsnap1/test1/139' 2023-06-09T11:14:19.978 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/10' -> 'copyofsnap1/test1/139/10' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/11' -> 'copyofsnap1/test1/139/11' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/12' -> 'copyofsnap1/test1/139/12' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/13' -> 'copyofsnap1/test1/139/13' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/14' -> 'copyofsnap1/test1/139/14' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/15' -> 'copyofsnap1/test1/139/15' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/16' -> 'copyofsnap1/test1/139/16' 2023-06-09T11:14:19.979 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/17' -> 'copyofsnap1/test1/139/17' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/18' -> 'copyofsnap1/test1/139/18' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/19' -> 'copyofsnap1/test1/139/19' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/139/20' -> 'copyofsnap1/test1/139/20' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140' -> 'copyofsnap1/test1/140' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/10' -> 'copyofsnap1/test1/140/10' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/11' -> 'copyofsnap1/test1/140/11' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/12' -> 'copyofsnap1/test1/140/12' 2023-06-09T11:14:19.980 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/13' -> 'copyofsnap1/test1/140/13' 2023-06-09T11:14:23.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/14' -> 'copyofsnap1/test1/140/14' 2023-06-09T11:14:23.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/15' -> 'copyofsnap1/test1/140/15' 2023-06-09T11:14:23.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/16' -> 'copyofsnap1/test1/140/16' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/17' -> 'copyofsnap1/test1/140/17' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/18' -> 'copyofsnap1/test1/140/18' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/19' -> 'copyofsnap1/test1/140/19' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/140/20' -> 'copyofsnap1/test1/140/20' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141' -> 'copyofsnap1/test1/141' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/10' -> 'copyofsnap1/test1/141/10' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/11' -> 'copyofsnap1/test1/141/11' 2023-06-09T11:14:23.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/12' -> 'copyofsnap1/test1/141/12' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/13' -> 'copyofsnap1/test1/141/13' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/14' -> 'copyofsnap1/test1/141/14' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/15' -> 'copyofsnap1/test1/141/15' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/16' -> 'copyofsnap1/test1/141/16' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/17' -> 'copyofsnap1/test1/141/17' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/18' -> 'copyofsnap1/test1/141/18' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/19' -> 'copyofsnap1/test1/141/19' 2023-06-09T11:14:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/141/20' -> 'copyofsnap1/test1/141/20' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142' -> 'copyofsnap1/test1/142' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/10' -> 'copyofsnap1/test1/142/10' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/11' -> 'copyofsnap1/test1/142/11' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/12' -> 'copyofsnap1/test1/142/12' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/13' -> 'copyofsnap1/test1/142/13' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/14' -> 'copyofsnap1/test1/142/14' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/15' -> 'copyofsnap1/test1/142/15' 2023-06-09T11:14:23.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/16' -> 'copyofsnap1/test1/142/16' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/17' -> 'copyofsnap1/test1/142/17' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/18' -> 'copyofsnap1/test1/142/18' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/19' -> 'copyofsnap1/test1/142/19' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/142/20' -> 'copyofsnap1/test1/142/20' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143' -> 'copyofsnap1/test1/143' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/10' -> 'copyofsnap1/test1/143/10' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/11' -> 'copyofsnap1/test1/143/11' 2023-06-09T11:14:23.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/12' -> 'copyofsnap1/test1/143/12' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/13' -> 'copyofsnap1/test1/143/13' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/14' -> 'copyofsnap1/test1/143/14' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/15' -> 'copyofsnap1/test1/143/15' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/16' -> 'copyofsnap1/test1/143/16' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/17' -> 'copyofsnap1/test1/143/17' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/18' -> 'copyofsnap1/test1/143/18' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/19' -> 'copyofsnap1/test1/143/19' 2023-06-09T11:14:23.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/143/20' -> 'copyofsnap1/test1/143/20' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144' -> 'copyofsnap1/test1/144' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/10' -> 'copyofsnap1/test1/144/10' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/11' -> 'copyofsnap1/test1/144/11' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/12' -> 'copyofsnap1/test1/144/12' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/13' -> 'copyofsnap1/test1/144/13' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/14' -> 'copyofsnap1/test1/144/14' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/15' -> 'copyofsnap1/test1/144/15' 2023-06-09T11:14:23.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/16' -> 'copyofsnap1/test1/144/16' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/17' -> 'copyofsnap1/test1/144/17' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/18' -> 'copyofsnap1/test1/144/18' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/19' -> 'copyofsnap1/test1/144/19' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/144/20' -> 'copyofsnap1/test1/144/20' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145' -> 'copyofsnap1/test1/145' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/10' -> 'copyofsnap1/test1/145/10' 2023-06-09T11:14:23.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/11' -> 'copyofsnap1/test1/145/11' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/12' -> 'copyofsnap1/test1/145/12' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/13' -> 'copyofsnap1/test1/145/13' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/14' -> 'copyofsnap1/test1/145/14' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/15' -> 'copyofsnap1/test1/145/15' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/16' -> 'copyofsnap1/test1/145/16' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/17' -> 'copyofsnap1/test1/145/17' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/18' -> 'copyofsnap1/test1/145/18' 2023-06-09T11:14:23.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/19' -> 'copyofsnap1/test1/145/19' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/145/20' -> 'copyofsnap1/test1/145/20' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146' -> 'copyofsnap1/test1/146' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/10' -> 'copyofsnap1/test1/146/10' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/11' -> 'copyofsnap1/test1/146/11' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/12' -> 'copyofsnap1/test1/146/12' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/13' -> 'copyofsnap1/test1/146/13' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/14' -> 'copyofsnap1/test1/146/14' 2023-06-09T11:14:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/15' -> 'copyofsnap1/test1/146/15' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/16' -> 'copyofsnap1/test1/146/16' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/17' -> 'copyofsnap1/test1/146/17' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/18' -> 'copyofsnap1/test1/146/18' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/19' -> 'copyofsnap1/test1/146/19' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/146/20' -> 'copyofsnap1/test1/146/20' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147' -> 'copyofsnap1/test1/147' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/10' -> 'copyofsnap1/test1/147/10' 2023-06-09T11:14:23.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/11' -> 'copyofsnap1/test1/147/11' 2023-06-09T11:14:27.879 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/12' -> 'copyofsnap1/test1/147/12' 2023-06-09T11:14:27.879 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/13' -> 'copyofsnap1/test1/147/13' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/14' -> 'copyofsnap1/test1/147/14' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/15' -> 'copyofsnap1/test1/147/15' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/16' -> 'copyofsnap1/test1/147/16' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/17' -> 'copyofsnap1/test1/147/17' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/18' -> 'copyofsnap1/test1/147/18' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/19' -> 'copyofsnap1/test1/147/19' 2023-06-09T11:14:27.880 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/147/20' -> 'copyofsnap1/test1/147/20' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148' -> 'copyofsnap1/test1/148' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/10' -> 'copyofsnap1/test1/148/10' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/11' -> 'copyofsnap1/test1/148/11' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/12' -> 'copyofsnap1/test1/148/12' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/13' -> 'copyofsnap1/test1/148/13' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/14' -> 'copyofsnap1/test1/148/14' 2023-06-09T11:14:27.881 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/15' -> 'copyofsnap1/test1/148/15' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/16' -> 'copyofsnap1/test1/148/16' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/17' -> 'copyofsnap1/test1/148/17' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/18' -> 'copyofsnap1/test1/148/18' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/19' -> 'copyofsnap1/test1/148/19' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/148/20' -> 'copyofsnap1/test1/148/20' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149' -> 'copyofsnap1/test1/149' 2023-06-09T11:14:27.882 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/10' -> 'copyofsnap1/test1/149/10' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/11' -> 'copyofsnap1/test1/149/11' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/12' -> 'copyofsnap1/test1/149/12' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/13' -> 'copyofsnap1/test1/149/13' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/14' -> 'copyofsnap1/test1/149/14' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/15' -> 'copyofsnap1/test1/149/15' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/16' -> 'copyofsnap1/test1/149/16' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/17' -> 'copyofsnap1/test1/149/17' 2023-06-09T11:14:27.883 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/18' -> 'copyofsnap1/test1/149/18' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/19' -> 'copyofsnap1/test1/149/19' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/149/20' -> 'copyofsnap1/test1/149/20' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150' -> 'copyofsnap1/test1/150' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/10' -> 'copyofsnap1/test1/150/10' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/11' -> 'copyofsnap1/test1/150/11' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/12' -> 'copyofsnap1/test1/150/12' 2023-06-09T11:14:27.884 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/13' -> 'copyofsnap1/test1/150/13' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/14' -> 'copyofsnap1/test1/150/14' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/15' -> 'copyofsnap1/test1/150/15' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/16' -> 'copyofsnap1/test1/150/16' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/17' -> 'copyofsnap1/test1/150/17' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/18' -> 'copyofsnap1/test1/150/18' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/19' -> 'copyofsnap1/test1/150/19' 2023-06-09T11:14:27.885 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/150/20' -> 'copyofsnap1/test1/150/20' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151' -> 'copyofsnap1/test1/151' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/10' -> 'copyofsnap1/test1/151/10' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/11' -> 'copyofsnap1/test1/151/11' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/12' -> 'copyofsnap1/test1/151/12' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/13' -> 'copyofsnap1/test1/151/13' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/14' -> 'copyofsnap1/test1/151/14' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/15' -> 'copyofsnap1/test1/151/15' 2023-06-09T11:14:27.886 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/16' -> 'copyofsnap1/test1/151/16' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/17' -> 'copyofsnap1/test1/151/17' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/18' -> 'copyofsnap1/test1/151/18' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/19' -> 'copyofsnap1/test1/151/19' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/151/20' -> 'copyofsnap1/test1/151/20' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152' -> 'copyofsnap1/test1/152' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/10' -> 'copyofsnap1/test1/152/10' 2023-06-09T11:14:27.887 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/11' -> 'copyofsnap1/test1/152/11' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/12' -> 'copyofsnap1/test1/152/12' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/13' -> 'copyofsnap1/test1/152/13' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/14' -> 'copyofsnap1/test1/152/14' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/15' -> 'copyofsnap1/test1/152/15' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/16' -> 'copyofsnap1/test1/152/16' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/17' -> 'copyofsnap1/test1/152/17' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/18' -> 'copyofsnap1/test1/152/18' 2023-06-09T11:14:27.888 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/19' -> 'copyofsnap1/test1/152/19' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/152/20' -> 'copyofsnap1/test1/152/20' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153' -> 'copyofsnap1/test1/153' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/10' -> 'copyofsnap1/test1/153/10' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/11' -> 'copyofsnap1/test1/153/11' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/12' -> 'copyofsnap1/test1/153/12' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/13' -> 'copyofsnap1/test1/153/13' 2023-06-09T11:14:27.889 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/14' -> 'copyofsnap1/test1/153/14' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/15' -> 'copyofsnap1/test1/153/15' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/16' -> 'copyofsnap1/test1/153/16' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/17' -> 'copyofsnap1/test1/153/17' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/18' -> 'copyofsnap1/test1/153/18' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/19' -> 'copyofsnap1/test1/153/19' 2023-06-09T11:14:27.890 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/153/20' -> 'copyofsnap1/test1/153/20' 2023-06-09T11:14:33.480 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154' -> 'copyofsnap1/test1/154' 2023-06-09T11:14:33.481 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/10' -> 'copyofsnap1/test1/154/10' 2023-06-09T11:14:33.481 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/11' -> 'copyofsnap1/test1/154/11' 2023-06-09T11:14:33.482 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/12' -> 'copyofsnap1/test1/154/12' 2023-06-09T11:14:33.482 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/13' -> 'copyofsnap1/test1/154/13' 2023-06-09T11:14:33.482 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/14' -> 'copyofsnap1/test1/154/14' 2023-06-09T11:14:33.483 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/15' -> 'copyofsnap1/test1/154/15' 2023-06-09T11:14:33.483 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/16' -> 'copyofsnap1/test1/154/16' 2023-06-09T11:14:33.483 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/17' -> 'copyofsnap1/test1/154/17' 2023-06-09T11:14:33.484 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/18' -> 'copyofsnap1/test1/154/18' 2023-06-09T11:14:33.484 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/19' -> 'copyofsnap1/test1/154/19' 2023-06-09T11:14:33.484 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/154/20' -> 'copyofsnap1/test1/154/20' 2023-06-09T11:14:33.484 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155' -> 'copyofsnap1/test1/155' 2023-06-09T11:14:33.485 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/10' -> 'copyofsnap1/test1/155/10' 2023-06-09T11:14:33.485 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/11' -> 'copyofsnap1/test1/155/11' 2023-06-09T11:14:33.485 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/12' -> 'copyofsnap1/test1/155/12' 2023-06-09T11:14:33.486 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/13' -> 'copyofsnap1/test1/155/13' 2023-06-09T11:14:33.486 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/14' -> 'copyofsnap1/test1/155/14' 2023-06-09T11:14:33.486 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/15' -> 'copyofsnap1/test1/155/15' 2023-06-09T11:14:33.486 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/16' -> 'copyofsnap1/test1/155/16' 2023-06-09T11:14:33.487 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/17' -> 'copyofsnap1/test1/155/17' 2023-06-09T11:14:33.487 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/18' -> 'copyofsnap1/test1/155/18' 2023-06-09T11:14:33.487 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/19' -> 'copyofsnap1/test1/155/19' 2023-06-09T11:14:33.488 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/155/20' -> 'copyofsnap1/test1/155/20' 2023-06-09T11:14:33.488 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156' -> 'copyofsnap1/test1/156' 2023-06-09T11:14:33.488 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/10' -> 'copyofsnap1/test1/156/10' 2023-06-09T11:14:33.488 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/11' -> 'copyofsnap1/test1/156/11' 2023-06-09T11:14:33.489 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/12' -> 'copyofsnap1/test1/156/12' 2023-06-09T11:14:33.489 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/13' -> 'copyofsnap1/test1/156/13' 2023-06-09T11:14:33.489 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/14' -> 'copyofsnap1/test1/156/14' 2023-06-09T11:14:33.490 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/15' -> 'copyofsnap1/test1/156/15' 2023-06-09T11:14:33.490 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/16' -> 'copyofsnap1/test1/156/16' 2023-06-09T11:14:33.490 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/17' -> 'copyofsnap1/test1/156/17' 2023-06-09T11:14:33.490 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/18' -> 'copyofsnap1/test1/156/18' 2023-06-09T11:14:33.491 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/19' -> 'copyofsnap1/test1/156/19' 2023-06-09T11:14:33.491 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/156/20' -> 'copyofsnap1/test1/156/20' 2023-06-09T11:14:33.491 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157' -> 'copyofsnap1/test1/157' 2023-06-09T11:14:33.492 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/10' -> 'copyofsnap1/test1/157/10' 2023-06-09T11:14:33.492 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/11' -> 'copyofsnap1/test1/157/11' 2023-06-09T11:14:33.492 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/12' -> 'copyofsnap1/test1/157/12' 2023-06-09T11:14:33.492 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/13' -> 'copyofsnap1/test1/157/13' 2023-06-09T11:14:33.493 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/14' -> 'copyofsnap1/test1/157/14' 2023-06-09T11:14:33.493 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/15' -> 'copyofsnap1/test1/157/15' 2023-06-09T11:14:33.493 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/16' -> 'copyofsnap1/test1/157/16' 2023-06-09T11:14:33.494 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/17' -> 'copyofsnap1/test1/157/17' 2023-06-09T11:14:33.494 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/18' -> 'copyofsnap1/test1/157/18' 2023-06-09T11:14:33.494 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/19' -> 'copyofsnap1/test1/157/19' 2023-06-09T11:14:33.494 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/157/20' -> 'copyofsnap1/test1/157/20' 2023-06-09T11:14:33.495 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158' -> 'copyofsnap1/test1/158' 2023-06-09T11:14:33.495 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/10' -> 'copyofsnap1/test1/158/10' 2023-06-09T11:14:33.495 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/11' -> 'copyofsnap1/test1/158/11' 2023-06-09T11:14:33.495 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/12' -> 'copyofsnap1/test1/158/12' 2023-06-09T11:14:33.496 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/13' -> 'copyofsnap1/test1/158/13' 2023-06-09T11:14:33.496 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/14' -> 'copyofsnap1/test1/158/14' 2023-06-09T11:14:33.496 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/15' -> 'copyofsnap1/test1/158/15' 2023-06-09T11:14:33.497 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/16' -> 'copyofsnap1/test1/158/16' 2023-06-09T11:14:33.497 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/17' -> 'copyofsnap1/test1/158/17' 2023-06-09T11:14:33.497 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/18' -> 'copyofsnap1/test1/158/18' 2023-06-09T11:14:33.497 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/19' -> 'copyofsnap1/test1/158/19' 2023-06-09T11:14:33.498 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/158/20' -> 'copyofsnap1/test1/158/20' 2023-06-09T11:14:33.498 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159' -> 'copyofsnap1/test1/159' 2023-06-09T11:14:33.498 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/10' -> 'copyofsnap1/test1/159/10' 2023-06-09T11:14:33.499 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/11' -> 'copyofsnap1/test1/159/11' 2023-06-09T11:14:33.499 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/12' -> 'copyofsnap1/test1/159/12' 2023-06-09T11:14:33.499 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/13' -> 'copyofsnap1/test1/159/13' 2023-06-09T11:14:33.500 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/14' -> 'copyofsnap1/test1/159/14' 2023-06-09T11:14:33.500 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/15' -> 'copyofsnap1/test1/159/15' 2023-06-09T11:14:33.500 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/16' -> 'copyofsnap1/test1/159/16' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/17' -> 'copyofsnap1/test1/159/17' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/18' -> 'copyofsnap1/test1/159/18' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/19' -> 'copyofsnap1/test1/159/19' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/159/20' -> 'copyofsnap1/test1/159/20' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160' -> 'copyofsnap1/test1/160' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/10' -> 'copyofsnap1/test1/160/10' 2023-06-09T11:14:33.501 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/11' -> 'copyofsnap1/test1/160/11' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/12' -> 'copyofsnap1/test1/160/12' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/13' -> 'copyofsnap1/test1/160/13' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/14' -> 'copyofsnap1/test1/160/14' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/15' -> 'copyofsnap1/test1/160/15' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/16' -> 'copyofsnap1/test1/160/16' 2023-06-09T11:14:33.502 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/17' -> 'copyofsnap1/test1/160/17' 2023-06-09T11:14:35.424 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/18' -> 'copyofsnap1/test1/160/18' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/19' -> 'copyofsnap1/test1/160/19' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/160/20' -> 'copyofsnap1/test1/160/20' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161' -> 'copyofsnap1/test1/161' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/10' -> 'copyofsnap1/test1/161/10' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/11' -> 'copyofsnap1/test1/161/11' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/12' -> 'copyofsnap1/test1/161/12' 2023-06-09T11:14:35.425 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/13' -> 'copyofsnap1/test1/161/13' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/14' -> 'copyofsnap1/test1/161/14' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/15' -> 'copyofsnap1/test1/161/15' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/16' -> 'copyofsnap1/test1/161/16' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/17' -> 'copyofsnap1/test1/161/17' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/18' -> 'copyofsnap1/test1/161/18' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/19' -> 'copyofsnap1/test1/161/19' 2023-06-09T11:14:35.426 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/161/20' -> 'copyofsnap1/test1/161/20' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162' -> 'copyofsnap1/test1/162' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/10' -> 'copyofsnap1/test1/162/10' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/11' -> 'copyofsnap1/test1/162/11' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/12' -> 'copyofsnap1/test1/162/12' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/13' -> 'copyofsnap1/test1/162/13' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/14' -> 'copyofsnap1/test1/162/14' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/15' -> 'copyofsnap1/test1/162/15' 2023-06-09T11:14:35.427 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/16' -> 'copyofsnap1/test1/162/16' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/17' -> 'copyofsnap1/test1/162/17' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/18' -> 'copyofsnap1/test1/162/18' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/19' -> 'copyofsnap1/test1/162/19' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/162/20' -> 'copyofsnap1/test1/162/20' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163' -> 'copyofsnap1/test1/163' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/10' -> 'copyofsnap1/test1/163/10' 2023-06-09T11:14:35.428 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/11' -> 'copyofsnap1/test1/163/11' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/12' -> 'copyofsnap1/test1/163/12' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/13' -> 'copyofsnap1/test1/163/13' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/14' -> 'copyofsnap1/test1/163/14' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/15' -> 'copyofsnap1/test1/163/15' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/16' -> 'copyofsnap1/test1/163/16' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/17' -> 'copyofsnap1/test1/163/17' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/18' -> 'copyofsnap1/test1/163/18' 2023-06-09T11:14:35.429 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/19' -> 'copyofsnap1/test1/163/19' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/163/20' -> 'copyofsnap1/test1/163/20' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164' -> 'copyofsnap1/test1/164' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/10' -> 'copyofsnap1/test1/164/10' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/11' -> 'copyofsnap1/test1/164/11' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/12' -> 'copyofsnap1/test1/164/12' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/13' -> 'copyofsnap1/test1/164/13' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/14' -> 'copyofsnap1/test1/164/14' 2023-06-09T11:14:35.430 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/15' -> 'copyofsnap1/test1/164/15' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/16' -> 'copyofsnap1/test1/164/16' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/17' -> 'copyofsnap1/test1/164/17' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/18' -> 'copyofsnap1/test1/164/18' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/19' -> 'copyofsnap1/test1/164/19' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/164/20' -> 'copyofsnap1/test1/164/20' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165' -> 'copyofsnap1/test1/165' 2023-06-09T11:14:35.431 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/10' -> 'copyofsnap1/test1/165/10' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/11' -> 'copyofsnap1/test1/165/11' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/12' -> 'copyofsnap1/test1/165/12' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/13' -> 'copyofsnap1/test1/165/13' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/14' -> 'copyofsnap1/test1/165/14' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/15' -> 'copyofsnap1/test1/165/15' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/16' -> 'copyofsnap1/test1/165/16' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/17' -> 'copyofsnap1/test1/165/17' 2023-06-09T11:14:35.432 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/18' -> 'copyofsnap1/test1/165/18' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/19' -> 'copyofsnap1/test1/165/19' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/165/20' -> 'copyofsnap1/test1/165/20' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166' -> 'copyofsnap1/test1/166' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/10' -> 'copyofsnap1/test1/166/10' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/11' -> 'copyofsnap1/test1/166/11' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/12' -> 'copyofsnap1/test1/166/12' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/13' -> 'copyofsnap1/test1/166/13' 2023-06-09T11:14:35.433 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/14' -> 'copyofsnap1/test1/166/14' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/15' -> 'copyofsnap1/test1/166/15' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/16' -> 'copyofsnap1/test1/166/16' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/17' -> 'copyofsnap1/test1/166/17' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/18' -> 'copyofsnap1/test1/166/18' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/19' -> 'copyofsnap1/test1/166/19' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/166/20' -> 'copyofsnap1/test1/166/20' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167' -> 'copyofsnap1/test1/167' 2023-06-09T11:14:35.434 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/10' -> 'copyofsnap1/test1/167/10' 2023-06-09T11:14:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/11' -> 'copyofsnap1/test1/167/11' 2023-06-09T11:14:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/12' -> 'copyofsnap1/test1/167/12' 2023-06-09T11:14:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/13' -> 'copyofsnap1/test1/167/13' 2023-06-09T11:14:35.435 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/14' -> 'copyofsnap1/test1/167/14' 2023-06-09T11:14:36.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/15' -> 'copyofsnap1/test1/167/15' 2023-06-09T11:14:36.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/16' -> 'copyofsnap1/test1/167/16' 2023-06-09T11:14:36.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/17' -> 'copyofsnap1/test1/167/17' 2023-06-09T11:14:36.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/18' -> 'copyofsnap1/test1/167/18' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/19' -> 'copyofsnap1/test1/167/19' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/167/20' -> 'copyofsnap1/test1/167/20' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168' -> 'copyofsnap1/test1/168' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/10' -> 'copyofsnap1/test1/168/10' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/11' -> 'copyofsnap1/test1/168/11' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/12' -> 'copyofsnap1/test1/168/12' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/13' -> 'copyofsnap1/test1/168/13' 2023-06-09T11:14:36.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/14' -> 'copyofsnap1/test1/168/14' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/15' -> 'copyofsnap1/test1/168/15' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/16' -> 'copyofsnap1/test1/168/16' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/17' -> 'copyofsnap1/test1/168/17' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/18' -> 'copyofsnap1/test1/168/18' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/19' -> 'copyofsnap1/test1/168/19' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/168/20' -> 'copyofsnap1/test1/168/20' 2023-06-09T11:14:36.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169' -> 'copyofsnap1/test1/169' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/10' -> 'copyofsnap1/test1/169/10' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/11' -> 'copyofsnap1/test1/169/11' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/12' -> 'copyofsnap1/test1/169/12' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/13' -> 'copyofsnap1/test1/169/13' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/14' -> 'copyofsnap1/test1/169/14' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/15' -> 'copyofsnap1/test1/169/15' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/16' -> 'copyofsnap1/test1/169/16' 2023-06-09T11:14:36.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/17' -> 'copyofsnap1/test1/169/17' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/18' -> 'copyofsnap1/test1/169/18' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/19' -> 'copyofsnap1/test1/169/19' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/169/20' -> 'copyofsnap1/test1/169/20' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170' -> 'copyofsnap1/test1/170' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/10' -> 'copyofsnap1/test1/170/10' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/11' -> 'copyofsnap1/test1/170/11' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/12' -> 'copyofsnap1/test1/170/12' 2023-06-09T11:14:36.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/13' -> 'copyofsnap1/test1/170/13' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/14' -> 'copyofsnap1/test1/170/14' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/15' -> 'copyofsnap1/test1/170/15' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/16' -> 'copyofsnap1/test1/170/16' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/17' -> 'copyofsnap1/test1/170/17' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/18' -> 'copyofsnap1/test1/170/18' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/19' -> 'copyofsnap1/test1/170/19' 2023-06-09T11:14:36.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/170/20' -> 'copyofsnap1/test1/170/20' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171' -> 'copyofsnap1/test1/171' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/10' -> 'copyofsnap1/test1/171/10' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/11' -> 'copyofsnap1/test1/171/11' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/12' -> 'copyofsnap1/test1/171/12' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/13' -> 'copyofsnap1/test1/171/13' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/14' -> 'copyofsnap1/test1/171/14' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/15' -> 'copyofsnap1/test1/171/15' 2023-06-09T11:14:36.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/16' -> 'copyofsnap1/test1/171/16' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/17' -> 'copyofsnap1/test1/171/17' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/18' -> 'copyofsnap1/test1/171/18' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/19' -> 'copyofsnap1/test1/171/19' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/171/20' -> 'copyofsnap1/test1/171/20' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172' -> 'copyofsnap1/test1/172' 2023-06-09T11:14:36.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/10' -> 'copyofsnap1/test1/172/10' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/11' -> 'copyofsnap1/test1/172/11' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/12' -> 'copyofsnap1/test1/172/12' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/13' -> 'copyofsnap1/test1/172/13' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/14' -> 'copyofsnap1/test1/172/14' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/15' -> 'copyofsnap1/test1/172/15' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/16' -> 'copyofsnap1/test1/172/16' 2023-06-09T11:14:36.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/17' -> 'copyofsnap1/test1/172/17' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/18' -> 'copyofsnap1/test1/172/18' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/19' -> 'copyofsnap1/test1/172/19' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/172/20' -> 'copyofsnap1/test1/172/20' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173' -> 'copyofsnap1/test1/173' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/10' -> 'copyofsnap1/test1/173/10' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/11' -> 'copyofsnap1/test1/173/11' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/12' -> 'copyofsnap1/test1/173/12' 2023-06-09T11:14:36.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/13' -> 'copyofsnap1/test1/173/13' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/14' -> 'copyofsnap1/test1/173/14' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/15' -> 'copyofsnap1/test1/173/15' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/16' -> 'copyofsnap1/test1/173/16' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/17' -> 'copyofsnap1/test1/173/17' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/18' -> 'copyofsnap1/test1/173/18' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/19' -> 'copyofsnap1/test1/173/19' 2023-06-09T11:14:36.748 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/173/20' -> 'copyofsnap1/test1/173/20' 2023-06-09T11:14:36.749 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174' -> 'copyofsnap1/test1/174' 2023-06-09T11:14:36.749 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/10' -> 'copyofsnap1/test1/174/10' 2023-06-09T11:14:36.749 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/11' -> 'copyofsnap1/test1/174/11' 2023-06-09T11:14:39.714 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/12' -> 'copyofsnap1/test1/174/12' 2023-06-09T11:14:39.714 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/13' -> 'copyofsnap1/test1/174/13' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/14' -> 'copyofsnap1/test1/174/14' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/15' -> 'copyofsnap1/test1/174/15' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/16' -> 'copyofsnap1/test1/174/16' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/17' -> 'copyofsnap1/test1/174/17' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/18' -> 'copyofsnap1/test1/174/18' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/19' -> 'copyofsnap1/test1/174/19' 2023-06-09T11:14:39.715 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/174/20' -> 'copyofsnap1/test1/174/20' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175' -> 'copyofsnap1/test1/175' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/10' -> 'copyofsnap1/test1/175/10' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/11' -> 'copyofsnap1/test1/175/11' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/12' -> 'copyofsnap1/test1/175/12' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/13' -> 'copyofsnap1/test1/175/13' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/14' -> 'copyofsnap1/test1/175/14' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/15' -> 'copyofsnap1/test1/175/15' 2023-06-09T11:14:39.716 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/16' -> 'copyofsnap1/test1/175/16' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/17' -> 'copyofsnap1/test1/175/17' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/18' -> 'copyofsnap1/test1/175/18' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/19' -> 'copyofsnap1/test1/175/19' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/175/20' -> 'copyofsnap1/test1/175/20' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176' -> 'copyofsnap1/test1/176' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/10' -> 'copyofsnap1/test1/176/10' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/11' -> 'copyofsnap1/test1/176/11' 2023-06-09T11:14:39.717 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/12' -> 'copyofsnap1/test1/176/12' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/13' -> 'copyofsnap1/test1/176/13' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/14' -> 'copyofsnap1/test1/176/14' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/15' -> 'copyofsnap1/test1/176/15' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/16' -> 'copyofsnap1/test1/176/16' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/17' -> 'copyofsnap1/test1/176/17' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/18' -> 'copyofsnap1/test1/176/18' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/19' -> 'copyofsnap1/test1/176/19' 2023-06-09T11:14:39.718 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/176/20' -> 'copyofsnap1/test1/176/20' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177' -> 'copyofsnap1/test1/177' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/10' -> 'copyofsnap1/test1/177/10' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/11' -> 'copyofsnap1/test1/177/11' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/12' -> 'copyofsnap1/test1/177/12' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/13' -> 'copyofsnap1/test1/177/13' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/14' -> 'copyofsnap1/test1/177/14' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/15' -> 'copyofsnap1/test1/177/15' 2023-06-09T11:14:39.719 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/16' -> 'copyofsnap1/test1/177/16' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/17' -> 'copyofsnap1/test1/177/17' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/18' -> 'copyofsnap1/test1/177/18' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/19' -> 'copyofsnap1/test1/177/19' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/177/20' -> 'copyofsnap1/test1/177/20' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178' -> 'copyofsnap1/test1/178' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/10' -> 'copyofsnap1/test1/178/10' 2023-06-09T11:14:39.720 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/11' -> 'copyofsnap1/test1/178/11' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/12' -> 'copyofsnap1/test1/178/12' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/13' -> 'copyofsnap1/test1/178/13' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/14' -> 'copyofsnap1/test1/178/14' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/15' -> 'copyofsnap1/test1/178/15' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/16' -> 'copyofsnap1/test1/178/16' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/17' -> 'copyofsnap1/test1/178/17' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/18' -> 'copyofsnap1/test1/178/18' 2023-06-09T11:14:39.721 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/19' -> 'copyofsnap1/test1/178/19' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/178/20' -> 'copyofsnap1/test1/178/20' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179' -> 'copyofsnap1/test1/179' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/10' -> 'copyofsnap1/test1/179/10' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/11' -> 'copyofsnap1/test1/179/11' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/12' -> 'copyofsnap1/test1/179/12' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/13' -> 'copyofsnap1/test1/179/13' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/14' -> 'copyofsnap1/test1/179/14' 2023-06-09T11:14:39.722 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/15' -> 'copyofsnap1/test1/179/15' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/16' -> 'copyofsnap1/test1/179/16' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/17' -> 'copyofsnap1/test1/179/17' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/18' -> 'copyofsnap1/test1/179/18' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/19' -> 'copyofsnap1/test1/179/19' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/179/20' -> 'copyofsnap1/test1/179/20' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180' -> 'copyofsnap1/test1/180' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/10' -> 'copyofsnap1/test1/180/10' 2023-06-09T11:14:39.723 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/11' -> 'copyofsnap1/test1/180/11' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/12' -> 'copyofsnap1/test1/180/12' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/13' -> 'copyofsnap1/test1/180/13' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/14' -> 'copyofsnap1/test1/180/14' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/15' -> 'copyofsnap1/test1/180/15' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/16' -> 'copyofsnap1/test1/180/16' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/17' -> 'copyofsnap1/test1/180/17' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/18' -> 'copyofsnap1/test1/180/18' 2023-06-09T11:14:39.724 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/19' -> 'copyofsnap1/test1/180/19' 2023-06-09T11:14:39.725 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/180/20' -> 'copyofsnap1/test1/180/20' 2023-06-09T11:14:41.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:41.013 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181' -> 'copyofsnap1/test1/181' 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/10' -> 'copyofsnap1/test1/181/10' 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/11' -> 'copyofsnap1/test1/181/11' 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/12' -> 'copyofsnap1/test1/181/12' 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/13' -> 'copyofsnap1/test1/181/13' 2023-06-09T11:14:42.737 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/14' -> 'copyofsnap1/test1/181/14' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/15' -> 'copyofsnap1/test1/181/15' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/16' -> 'copyofsnap1/test1/181/16' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/17' -> 'copyofsnap1/test1/181/17' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/18' -> 'copyofsnap1/test1/181/18' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/19' -> 'copyofsnap1/test1/181/19' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/181/20' -> 'copyofsnap1/test1/181/20' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182' -> 'copyofsnap1/test1/182' 2023-06-09T11:14:42.738 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/10' -> 'copyofsnap1/test1/182/10' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/11' -> 'copyofsnap1/test1/182/11' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/12' -> 'copyofsnap1/test1/182/12' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/13' -> 'copyofsnap1/test1/182/13' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/14' -> 'copyofsnap1/test1/182/14' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/15' -> 'copyofsnap1/test1/182/15' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/16' -> 'copyofsnap1/test1/182/16' 2023-06-09T11:14:42.739 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/17' -> 'copyofsnap1/test1/182/17' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/18' -> 'copyofsnap1/test1/182/18' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/19' -> 'copyofsnap1/test1/182/19' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/182/20' -> 'copyofsnap1/test1/182/20' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183' -> 'copyofsnap1/test1/183' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/10' -> 'copyofsnap1/test1/183/10' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/11' -> 'copyofsnap1/test1/183/11' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/12' -> 'copyofsnap1/test1/183/12' 2023-06-09T11:14:42.740 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/13' -> 'copyofsnap1/test1/183/13' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/14' -> 'copyofsnap1/test1/183/14' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/15' -> 'copyofsnap1/test1/183/15' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/16' -> 'copyofsnap1/test1/183/16' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/17' -> 'copyofsnap1/test1/183/17' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/18' -> 'copyofsnap1/test1/183/18' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/19' -> 'copyofsnap1/test1/183/19' 2023-06-09T11:14:42.741 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/183/20' -> 'copyofsnap1/test1/183/20' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184' -> 'copyofsnap1/test1/184' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/10' -> 'copyofsnap1/test1/184/10' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/11' -> 'copyofsnap1/test1/184/11' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/12' -> 'copyofsnap1/test1/184/12' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/13' -> 'copyofsnap1/test1/184/13' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/14' -> 'copyofsnap1/test1/184/14' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/15' -> 'copyofsnap1/test1/184/15' 2023-06-09T11:14:42.742 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/16' -> 'copyofsnap1/test1/184/16' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/17' -> 'copyofsnap1/test1/184/17' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/18' -> 'copyofsnap1/test1/184/18' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/19' -> 'copyofsnap1/test1/184/19' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/184/20' -> 'copyofsnap1/test1/184/20' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185' -> 'copyofsnap1/test1/185' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/10' -> 'copyofsnap1/test1/185/10' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/11' -> 'copyofsnap1/test1/185/11' 2023-06-09T11:14:42.743 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/12' -> 'copyofsnap1/test1/185/12' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/13' -> 'copyofsnap1/test1/185/13' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/14' -> 'copyofsnap1/test1/185/14' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/15' -> 'copyofsnap1/test1/185/15' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/16' -> 'copyofsnap1/test1/185/16' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/17' -> 'copyofsnap1/test1/185/17' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/18' -> 'copyofsnap1/test1/185/18' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/19' -> 'copyofsnap1/test1/185/19' 2023-06-09T11:14:42.744 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/185/20' -> 'copyofsnap1/test1/185/20' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186' -> 'copyofsnap1/test1/186' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/10' -> 'copyofsnap1/test1/186/10' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/11' -> 'copyofsnap1/test1/186/11' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/12' -> 'copyofsnap1/test1/186/12' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/13' -> 'copyofsnap1/test1/186/13' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/14' -> 'copyofsnap1/test1/186/14' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/15' -> 'copyofsnap1/test1/186/15' 2023-06-09T11:14:42.745 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/16' -> 'copyofsnap1/test1/186/16' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/17' -> 'copyofsnap1/test1/186/17' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/18' -> 'copyofsnap1/test1/186/18' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/19' -> 'copyofsnap1/test1/186/19' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/186/20' -> 'copyofsnap1/test1/186/20' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187' -> 'copyofsnap1/test1/187' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/10' -> 'copyofsnap1/test1/187/10' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/11' -> 'copyofsnap1/test1/187/11' 2023-06-09T11:14:42.746 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/12' -> 'copyofsnap1/test1/187/12' 2023-06-09T11:14:42.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/13' -> 'copyofsnap1/test1/187/13' 2023-06-09T11:14:42.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/14' -> 'copyofsnap1/test1/187/14' 2023-06-09T11:14:42.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/15' -> 'copyofsnap1/test1/187/15' 2023-06-09T11:14:42.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/16' -> 'copyofsnap1/test1/187/16' 2023-06-09T11:14:42.747 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/17' -> 'copyofsnap1/test1/187/17' 2023-06-09T11:14:46.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/18' -> 'copyofsnap1/test1/187/18' 2023-06-09T11:14:46.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/19' -> 'copyofsnap1/test1/187/19' 2023-06-09T11:14:46.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/187/20' -> 'copyofsnap1/test1/187/20' 2023-06-09T11:14:46.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188' -> 'copyofsnap1/test1/188' 2023-06-09T11:14:46.080 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/10' -> 'copyofsnap1/test1/188/10' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/11' -> 'copyofsnap1/test1/188/11' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/12' -> 'copyofsnap1/test1/188/12' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/13' -> 'copyofsnap1/test1/188/13' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/14' -> 'copyofsnap1/test1/188/14' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/15' -> 'copyofsnap1/test1/188/15' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/16' -> 'copyofsnap1/test1/188/16' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/17' -> 'copyofsnap1/test1/188/17' 2023-06-09T11:14:46.081 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/18' -> 'copyofsnap1/test1/188/18' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/19' -> 'copyofsnap1/test1/188/19' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/188/20' -> 'copyofsnap1/test1/188/20' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189' -> 'copyofsnap1/test1/189' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/10' -> 'copyofsnap1/test1/189/10' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/11' -> 'copyofsnap1/test1/189/11' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/12' -> 'copyofsnap1/test1/189/12' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/13' -> 'copyofsnap1/test1/189/13' 2023-06-09T11:14:46.082 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/14' -> 'copyofsnap1/test1/189/14' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/15' -> 'copyofsnap1/test1/189/15' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/16' -> 'copyofsnap1/test1/189/16' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/17' -> 'copyofsnap1/test1/189/17' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/18' -> 'copyofsnap1/test1/189/18' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/19' -> 'copyofsnap1/test1/189/19' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/189/20' -> 'copyofsnap1/test1/189/20' 2023-06-09T11:14:46.083 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190' -> 'copyofsnap1/test1/190' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/10' -> 'copyofsnap1/test1/190/10' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/11' -> 'copyofsnap1/test1/190/11' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/12' -> 'copyofsnap1/test1/190/12' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/13' -> 'copyofsnap1/test1/190/13' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/14' -> 'copyofsnap1/test1/190/14' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/15' -> 'copyofsnap1/test1/190/15' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/16' -> 'copyofsnap1/test1/190/16' 2023-06-09T11:14:46.084 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/17' -> 'copyofsnap1/test1/190/17' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/18' -> 'copyofsnap1/test1/190/18' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/19' -> 'copyofsnap1/test1/190/19' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/190/20' -> 'copyofsnap1/test1/190/20' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191' -> 'copyofsnap1/test1/191' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/10' -> 'copyofsnap1/test1/191/10' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/11' -> 'copyofsnap1/test1/191/11' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/12' -> 'copyofsnap1/test1/191/12' 2023-06-09T11:14:46.085 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/13' -> 'copyofsnap1/test1/191/13' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/14' -> 'copyofsnap1/test1/191/14' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/15' -> 'copyofsnap1/test1/191/15' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/16' -> 'copyofsnap1/test1/191/16' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/17' -> 'copyofsnap1/test1/191/17' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/18' -> 'copyofsnap1/test1/191/18' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/19' -> 'copyofsnap1/test1/191/19' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/191/20' -> 'copyofsnap1/test1/191/20' 2023-06-09T11:14:46.086 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192' -> 'copyofsnap1/test1/192' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/10' -> 'copyofsnap1/test1/192/10' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/11' -> 'copyofsnap1/test1/192/11' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/12' -> 'copyofsnap1/test1/192/12' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/13' -> 'copyofsnap1/test1/192/13' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/14' -> 'copyofsnap1/test1/192/14' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/15' -> 'copyofsnap1/test1/192/15' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/16' -> 'copyofsnap1/test1/192/16' 2023-06-09T11:14:46.087 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/17' -> 'copyofsnap1/test1/192/17' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/18' -> 'copyofsnap1/test1/192/18' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/19' -> 'copyofsnap1/test1/192/19' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/192/20' -> 'copyofsnap1/test1/192/20' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193' -> 'copyofsnap1/test1/193' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/10' -> 'copyofsnap1/test1/193/10' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/11' -> 'copyofsnap1/test1/193/11' 2023-06-09T11:14:46.088 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/12' -> 'copyofsnap1/test1/193/12' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/13' -> 'copyofsnap1/test1/193/13' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/14' -> 'copyofsnap1/test1/193/14' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/15' -> 'copyofsnap1/test1/193/15' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/16' -> 'copyofsnap1/test1/193/16' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/17' -> 'copyofsnap1/test1/193/17' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/18' -> 'copyofsnap1/test1/193/18' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/19' -> 'copyofsnap1/test1/193/19' 2023-06-09T11:14:46.089 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/193/20' -> 'copyofsnap1/test1/193/20' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194' -> 'copyofsnap1/test1/194' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/10' -> 'copyofsnap1/test1/194/10' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/11' -> 'copyofsnap1/test1/194/11' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/12' -> 'copyofsnap1/test1/194/12' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/13' -> 'copyofsnap1/test1/194/13' 2023-06-09T11:14:46.090 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/14' -> 'copyofsnap1/test1/194/14' 2023-06-09T11:14:49.345 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/15' -> 'copyofsnap1/test1/194/15' 2023-06-09T11:14:49.345 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/16' -> 'copyofsnap1/test1/194/16' 2023-06-09T11:14:49.345 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/17' -> 'copyofsnap1/test1/194/17' 2023-06-09T11:14:49.345 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/18' -> 'copyofsnap1/test1/194/18' 2023-06-09T11:14:49.345 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/19' -> 'copyofsnap1/test1/194/19' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/194/20' -> 'copyofsnap1/test1/194/20' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195' -> 'copyofsnap1/test1/195' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/10' -> 'copyofsnap1/test1/195/10' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/11' -> 'copyofsnap1/test1/195/11' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/12' -> 'copyofsnap1/test1/195/12' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/13' -> 'copyofsnap1/test1/195/13' 2023-06-09T11:14:49.346 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/14' -> 'copyofsnap1/test1/195/14' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/15' -> 'copyofsnap1/test1/195/15' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/16' -> 'copyofsnap1/test1/195/16' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/17' -> 'copyofsnap1/test1/195/17' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/18' -> 'copyofsnap1/test1/195/18' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/19' -> 'copyofsnap1/test1/195/19' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/195/20' -> 'copyofsnap1/test1/195/20' 2023-06-09T11:14:49.347 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196' -> 'copyofsnap1/test1/196' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/10' -> 'copyofsnap1/test1/196/10' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/11' -> 'copyofsnap1/test1/196/11' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/12' -> 'copyofsnap1/test1/196/12' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/13' -> 'copyofsnap1/test1/196/13' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/14' -> 'copyofsnap1/test1/196/14' 2023-06-09T11:14:49.348 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/15' -> 'copyofsnap1/test1/196/15' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/16' -> 'copyofsnap1/test1/196/16' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/17' -> 'copyofsnap1/test1/196/17' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/18' -> 'copyofsnap1/test1/196/18' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/19' -> 'copyofsnap1/test1/196/19' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/196/20' -> 'copyofsnap1/test1/196/20' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197' -> 'copyofsnap1/test1/197' 2023-06-09T11:14:49.349 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/10' -> 'copyofsnap1/test1/197/10' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/11' -> 'copyofsnap1/test1/197/11' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/12' -> 'copyofsnap1/test1/197/12' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/13' -> 'copyofsnap1/test1/197/13' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/14' -> 'copyofsnap1/test1/197/14' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/15' -> 'copyofsnap1/test1/197/15' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/16' -> 'copyofsnap1/test1/197/16' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/17' -> 'copyofsnap1/test1/197/17' 2023-06-09T11:14:49.350 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/18' -> 'copyofsnap1/test1/197/18' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/19' -> 'copyofsnap1/test1/197/19' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/197/20' -> 'copyofsnap1/test1/197/20' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198' -> 'copyofsnap1/test1/198' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/10' -> 'copyofsnap1/test1/198/10' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/11' -> 'copyofsnap1/test1/198/11' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/12' -> 'copyofsnap1/test1/198/12' 2023-06-09T11:14:49.351 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/13' -> 'copyofsnap1/test1/198/13' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/14' -> 'copyofsnap1/test1/198/14' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/15' -> 'copyofsnap1/test1/198/15' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/16' -> 'copyofsnap1/test1/198/16' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/17' -> 'copyofsnap1/test1/198/17' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/18' -> 'copyofsnap1/test1/198/18' 2023-06-09T11:14:49.352 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/19' -> 'copyofsnap1/test1/198/19' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/198/20' -> 'copyofsnap1/test1/198/20' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199' -> 'copyofsnap1/test1/199' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/10' -> 'copyofsnap1/test1/199/10' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/11' -> 'copyofsnap1/test1/199/11' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/12' -> 'copyofsnap1/test1/199/12' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/13' -> 'copyofsnap1/test1/199/13' 2023-06-09T11:14:49.353 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/14' -> 'copyofsnap1/test1/199/14' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/15' -> 'copyofsnap1/test1/199/15' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/16' -> 'copyofsnap1/test1/199/16' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/17' -> 'copyofsnap1/test1/199/17' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/18' -> 'copyofsnap1/test1/199/18' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/19' -> 'copyofsnap1/test1/199/19' 2023-06-09T11:14:49.354 INFO:tasks.workunit.client.0.smithi007.stdout:'.snap/test1/199/20' -> 'copyofsnap1/test1/199/20' 2023-06-09T11:14:49.355 INFO:tasks.workunit.client.0.smithi007.stdout:Take third snapshot .snap/test3 2023-06-09T11:14:49.425 INFO:tasks.workunit.client.0.smithi007.stdout:Delete the snapshots... 2023-06-09T11:14:57.596 INFO:tasks.workunit.client.0.smithi007.stdout:.//.snap/test1 2023-06-09T11:14:57.596 INFO:tasks.workunit.client.0.smithi007.stdout:.//.snap/test2 2023-06-09T11:14:57.596 INFO:tasks.workunit.client.0.smithi007.stdout:.//.snap/test3 2023-06-09T11:14:57.691 INFO:tasks.workunit.client.0.smithi007.stdout:./274/.snap/snap-subdir-test 2023-06-09T11:14:57.718 INFO:tasks.workunit.client.0.smithi007.stdout:./166/.snap/snap-subdir-test 2023-06-09T11:14:57.736 INFO:tasks.workunit.client.0.smithi007.stdout:./256/.snap/snap-subdir-test 2023-06-09T11:14:57.756 INFO:tasks.workunit.client.0.smithi007.stdout:./247/.snap/snap-subdir-test 2023-06-09T11:14:58.504 INFO:tasks.workunit.client.0.smithi007.stdout:./248/.snap/snap-subdir-test 2023-06-09T11:14:58.530 INFO:tasks.workunit.client.0.smithi007.stdout:./147/.snap/snap-subdir-test 2023-06-09T11:14:58.557 INFO:tasks.workunit.client.0.smithi007.stdout:./151/.snap/snap-subdir-test 2023-06-09T11:14:58.580 INFO:tasks.workunit.client.0.smithi007.stdout:./170/.snap/snap-subdir-test 2023-06-09T11:14:58.603 INFO:tasks.workunit.client.0.smithi007.stdout:./167/.snap/snap-subdir-test 2023-06-09T11:14:58.627 INFO:tasks.workunit.client.0.smithi007.stdout:./140/.snap/snap-subdir-test 2023-06-09T11:14:58.648 INFO:tasks.workunit.client.0.smithi007.stdout:./142/.snap/snap-subdir-test 2023-06-09T11:14:58.671 INFO:tasks.workunit.client.0.smithi007.stdout:./150/.snap/snap-subdir-test 2023-06-09T11:14:58.692 INFO:tasks.workunit.client.0.smithi007.stdout:./273/.snap/snap-subdir-test 2023-06-09T11:14:59.238 INFO:tasks.workunit.client.0.smithi007.stdout:./269/.snap/snap-subdir-test 2023-06-09T11:14:59.265 INFO:tasks.workunit.client.0.smithi007.stdout:./105/.snap/snap-subdir-test 2023-06-09T11:15:00.038 INFO:tasks.workunit.client.0.smithi007.stdout:./210/.snap/snap-subdir-test 2023-06-09T11:15:00.060 INFO:tasks.workunit.client.0.smithi007.stdout:./265/.snap/snap-subdir-test 2023-06-09T11:15:00.769 INFO:tasks.workunit.client.0.smithi007.stdout:./218/.snap/snap-subdir-test 2023-06-09T11:15:00.791 INFO:tasks.workunit.client.0.smithi007.stdout:./290/.snap/snap-subdir-test 2023-06-09T11:15:00.812 INFO:tasks.workunit.client.0.smithi007.stdout:./149/.snap/snap-subdir-test 2023-06-09T11:15:00.834 INFO:tasks.workunit.client.0.smithi007.stdout:./109/.snap/snap-subdir-test 2023-06-09T11:15:00.854 INFO:tasks.workunit.client.0.smithi007.stdout:./292/.snap/snap-subdir-test 2023-06-09T11:15:00.874 INFO:tasks.workunit.client.0.smithi007.stdout:./242/.snap/snap-subdir-test 2023-06-09T11:15:00.896 INFO:tasks.workunit.client.0.smithi007.stdout:./158/.snap/snap-subdir-test 2023-06-09T11:15:00.915 INFO:tasks.workunit.client.0.smithi007.stdout:./243/.snap/snap-subdir-test 2023-06-09T11:15:00.935 INFO:tasks.workunit.client.0.smithi007.stdout:./106/.snap/snap-subdir-test 2023-06-09T11:15:00.954 INFO:tasks.workunit.client.0.smithi007.stdout:./250/.snap/snap-subdir-test 2023-06-09T11:15:00.972 INFO:tasks.workunit.client.0.smithi007.stdout:./221/.snap/snap-subdir-test 2023-06-09T11:15:00.991 INFO:tasks.workunit.client.0.smithi007.stdout:./244/.snap/snap-subdir-test 2023-06-09T11:15:01.010 INFO:tasks.workunit.client.0.smithi007.stdout:./258/.snap/snap-subdir-test 2023-06-09T11:15:01.027 INFO:tasks.workunit.client.0.smithi007.stdout:./268/.snap/snap-subdir-test 2023-06-09T11:15:01.046 INFO:tasks.workunit.client.0.smithi007.stdout:./189/.snap/snap-subdir-test 2023-06-09T11:15:01.064 INFO:tasks.workunit.client.0.smithi007.stdout:./285/.snap/snap-subdir-test 2023-06-09T11:15:01.081 INFO:tasks.workunit.client.0.smithi007.stdout:./208/.snap/snap-subdir-test 2023-06-09T11:15:01.102 INFO:tasks.workunit.client.0.smithi007.stdout:./288/.snap/snap-subdir-test 2023-06-09T11:15:01.121 INFO:tasks.workunit.client.0.smithi007.stdout:./209/.snap/snap-subdir-test 2023-06-09T11:15:01.140 INFO:tasks.workunit.client.0.smithi007.stdout:./253/.snap/snap-subdir-test 2023-06-09T11:15:01.411 INFO:tasks.workunit.client.0.smithi007.stdout:./249/.snap/snap-subdir-test 2023-06-09T11:15:01.434 INFO:tasks.workunit.client.0.smithi007.stdout:./133/.snap/snap-subdir-test 2023-06-09T11:15:01.455 INFO:tasks.workunit.client.0.smithi007.stdout:./291/.snap/snap-subdir-test 2023-06-09T11:15:01.474 INFO:tasks.workunit.client.0.smithi007.stdout:./226/.snap/snap-subdir-test 2023-06-09T11:15:01.498 INFO:tasks.workunit.client.0.smithi007.stdout:./173/.snap/snap-subdir-test 2023-06-09T11:15:01.520 INFO:tasks.workunit.client.0.smithi007.stdout:./174/.snap/snap-subdir-test 2023-06-09T11:15:01.538 INFO:tasks.workunit.client.0.smithi007.stdout:./299/.snap/snap-subdir-test 2023-06-09T11:15:01.555 INFO:tasks.workunit.client.0.smithi007.stdout:./262/.snap/snap-subdir-test 2023-06-09T11:15:01.573 INFO:tasks.workunit.client.0.smithi007.stdout:./231/.snap/snap-subdir-test 2023-06-09T11:15:01.590 INFO:tasks.workunit.client.0.smithi007.stdout:./214/.snap/snap-subdir-test 2023-06-09T11:15:01.608 INFO:tasks.workunit.client.0.smithi007.stdout:./281/.snap/snap-subdir-test 2023-06-09T11:15:01.626 INFO:tasks.workunit.client.0.smithi007.stdout:./289/.snap/snap-subdir-test 2023-06-09T11:15:01.644 INFO:tasks.workunit.client.0.smithi007.stdout:./283/.snap/snap-subdir-test 2023-06-09T11:15:01.767 INFO:tasks.workunit.client.0.smithi007.stdout:./232/.snap/snap-subdir-test 2023-06-09T11:15:01.787 INFO:tasks.workunit.client.0.smithi007.stdout:./298/.snap/snap-subdir-test 2023-06-09T11:15:01.804 INFO:tasks.workunit.client.0.smithi007.stdout:./233/.snap/snap-subdir-test 2023-06-09T11:15:01.824 INFO:tasks.workunit.client.0.smithi007.stdout:./155/.snap/snap-subdir-test 2023-06-09T11:15:01.846 INFO:tasks.workunit.client.0.smithi007.stdout:./114/.snap/snap-subdir-test 2023-06-09T11:15:01.867 INFO:tasks.workunit.client.0.smithi007.stdout:./159/.snap/snap-subdir-test 2023-06-09T11:15:02.609 INFO:tasks.workunit.client.0.smithi007.stdout:./187/.snap/snap-subdir-test 2023-06-09T11:15:02.632 INFO:tasks.workunit.client.0.smithi007.stdout:./185/.snap/snap-subdir-test 2023-06-09T11:15:02.653 INFO:tasks.workunit.client.0.smithi007.stdout:./293/.snap/snap-subdir-test 2023-06-09T11:15:02.671 INFO:tasks.workunit.client.0.smithi007.stdout:./228/.snap/snap-subdir-test 2023-06-09T11:15:02.692 INFO:tasks.workunit.client.0.smithi007.stdout:./120/.snap/snap-subdir-test 2023-06-09T11:15:03.020 INFO:tasks.workunit.client.0.smithi007.stdout:./110/.snap/snap-subdir-test 2023-06-09T11:15:03.042 INFO:tasks.workunit.client.0.smithi007.stdout:./267/.snap/snap-subdir-test 2023-06-09T11:15:03.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.865+0000 7f22f2bff700 1 Processor -- start 2023-06-09T11:15:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.865+0000 7f22f2bff700 1 -- start start 2023-06-09T11:15:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.866+0000 7f22f2bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22ec0cb5d0 0x7f22ec0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.866+0000 7f22f2bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15cd40 0x7f22ec0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.866+0000 7f22f2bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec159b20 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.866+0000 7f22f2bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec0cac80 con 0x7f22ec15cd40 2023-06-09T11:15:03.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.866+0000 7f22f2bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec1597d0 con 0x7f22ec0cb5d0 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15cd40 0x7f22ec0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22ec15cd40 0x7f22ec0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40782/0 (socket says 172.21.15.7:40782) 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f099b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22ec0cb5d0 0x7f22ec0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 -- 172.21.15.7:0/819464206 learned_addr learned my addr 172.21.15.7:0/819464206 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 -- 172.21.15.7:0/819464206 >> v2:172.21.15.117:3300/0 conn(0x7f22ec0cb5d0 msgr2=0x7f22ec0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:03.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 --2- 172.21.15.7:0/819464206 >> v2:172.21.15.117:3300/0 conn(0x7f22ec0cb5d0 0x7f22ec0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 -- 172.21.15.7:0/819464206 >> v1:172.21.15.117:6789/0 conn(0x7f22ec0cb9a0 legacy=0x7f22ec159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 -- 172.21.15.7:0/819464206 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec071ab0 con 0x7f22ec15cd40 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.867+0000 7f22f119c700 1 --2- 172.21.15.7:0/819464206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15cd40 0x7f22ec0cbea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22ec08d420 tx=0x7f22e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f231f4e992cebd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22eb7fe700 1 -- 172.21.15.7:0/819464206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400e070 con 0x7f22ec15cd40 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22eb7fe700 1 -- 172.21.15.7:0/819464206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400b910 con 0x7f22ec15cd40 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22eb7fe700 1 -- 172.21.15.7:0/819464206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400bcc0 con 0x7f22ec15cd40 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 -- 172.21.15.7:0/819464206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15cd40 msgr2=0x7f22ec0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:03.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 --2- 172.21.15.7:0/819464206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15cd40 0x7f22ec0cbea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f22ec08d420 tx=0x7f22e400b330 comp rx=0 tx=0).stop 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 -- 172.21.15.7:0/819464206 shutdown_connections 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 --2- 172.21.15.7:0/819464206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec15cd40 0x7f22ec0cbea0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 --2- 172.21.15.7:0/819464206 >> v2:172.21.15.117:3300/0 conn(0x7f22ec0cb5d0 0x7f22ec0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 -- 172.21.15.7:0/819464206 >> 172.21.15.7:0/819464206 conn(0x7f22ec070af0 msgr2=0x7f22ec070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 -- 172.21.15.7:0/819464206 shutdown_connections 2023-06-09T11:15:03.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.868+0000 7f22f2bff700 1 -- 172.21.15.7:0/819464206 wait complete. 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.870+0000 7f22f2bff700 1 Processor -- start 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.870+0000 7f22f2bff700 1 -- start start 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec0cb5d0 0x7f22ec0c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22ec15cd40 0x7f22ec0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c2fd0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0c4320 con 0x7f22ec0cb5d0 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f2bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec1f0720 con 0x7f22ec15cd40 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40788/0 (socket says 172.21.15.7:40788) 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22ebfff700 1 -- 172.21.15.7:0/241999558 learned_addr learned my addr 172.21.15.7:0/241999558 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.871+0000 7f22f099b700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec0cb5d0 0x7f22ec0c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22f119c700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22ec15cd40 0x7f22ec0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 -- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22ec15cd40 msgr2=0x7f22ec0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:03.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22ec15cd40 0x7f22ec0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:03.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 -- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec0cb5d0 msgr2=0x7f22ec0c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:03.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec0cb5d0 0x7f22ec0c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:03.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 -- 172.21.15.7:0/241999558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009b30 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22ebfff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c4b0 tx=0x7f22d800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80052d0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d8005430 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80038c0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.872+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec15d4f0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.873+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec15d210 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.875+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec15d7d0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.876+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22d8003a20 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.877+0000 7f22e97fa700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 0x7f22d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.877+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f22d80889a0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.877+0000 7f22f099b700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 0x7f22d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.877+0000 7f22f099b700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 0x7f22d006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009be0 tx=0x7f22e0009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:03.881+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f22d80538c0 con 0x7f22ec0cb9a0 2023-06-09T11:15:03.926 INFO:tasks.workunit.client.0.smithi007.stdout:./202/.snap/snap-subdir-test 2023-06-09T11:15:03.945 INFO:tasks.workunit.client.0.smithi007.stdout:./137/.snap/snap-subdir-test 2023-06-09T11:15:03.964 INFO:tasks.workunit.client.0.smithi007.stdout:./246/.snap/snap-subdir-test 2023-06-09T11:15:03.983 INFO:tasks.workunit.client.0.smithi007.stdout:./219/.snap/snap-subdir-test 2023-06-09T11:15:04.003 INFO:tasks.workunit.client.0.smithi007.stdout:./198/.snap/snap-subdir-test 2023-06-09T11:15:04.023 INFO:tasks.workunit.client.0.smithi007.stdout:./136/.snap/snap-subdir-test 2023-06-09T11:15:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.143+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ec0c9cb0 con 0x7f22ec0cb9a0 2023-06-09T11:15:04.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.144+0000 7f22e97fa700 1 -- 172.21.15.7:0/241999558 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f22d8056ee0 con 0x7f22ec0cb9a0 2023-06-09T11:15:04.150 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:04.150 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:04.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 msgr2=0x7f22d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 0x7f22d006d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009be0 tx=0x7f22e0009210 comp rx=0 tx=0).stop 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 msgr2=0x7f22ec1f01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f22d800c4b0 tx=0x7f22d800c7c0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 shutdown_connections 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22d006ace0 0x7f22d006d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22ec15cd40 0x7f22ec0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22ec0cb5d0 0x7f22ec0c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 --2- 172.21.15.7:0/241999558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22ec0cb9a0 0x7f22ec1f01e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 >> 172.21.15.7:0/241999558 conn(0x7f22ec070af0 msgr2=0x7f22ec0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 shutdown_connections 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.146+0000 7f22f2bff700 1 -- 172.21.15.7:0/241999558 wait complete. 2023-06-09T11:15:04.153 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:04.170 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-06-09T11:15:04.170 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:15:04.170 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:15:04.190 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:15:04.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.386+0000 7f244b1ff700 1 Processor -- start 2023-06-09T11:15:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.386+0000 7f244b1ff700 1 -- start start 2023-06-09T11:15:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f244b1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2444151510 0x7f24440c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f244b1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f244415b380 0x7f2444070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f244b1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24441580a0 con 0x7f2444071060 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f244b1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24441518e0 con 0x7f2444151510 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f244b1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2444071480 con 0x7f244415b380 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2444151510 0x7f24440c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2444151510 0x7f24440c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40802/0 (socket says 172.21.15.7:40802) 2023-06-09T11:15:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 -- 172.21.15.7:0/726092643 learned_addr learned my addr 172.21.15.7:0/726092643 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2443fff700 1 --2- 172.21.15.7:0/726092643 >> v2:172.21.15.117:3300/0 conn(0x7f244415b380 0x7f2444070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 -- 172.21.15.7:0/726092643 >> v2:172.21.15.117:3300/0 conn(0x7f244415b380 msgr2=0x7f2444070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 --2- 172.21.15.7:0/726092643 >> v2:172.21.15.117:3300/0 conn(0x7f244415b380 0x7f2444070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 -- 172.21.15.7:0/726092643 >> v1:172.21.15.7:6789/0 conn(0x7f2444071060 legacy=0x7f2444158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.387+0000 7f2448f9b700 1 -- 172.21.15.7:0/726092643 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440c6180 con 0x7f2444151510 2023-06-09T11:15:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f2448f9b700 1 --2- 172.21.15.7:0/726092643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c5b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f244406b170 tx=0x7f2438009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6019bf56f4983f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f24437fe700 1 -- 172.21.15.7:0/726092643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800b870 con 0x7f2444151510 2023-06-09T11:15:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f24437fe700 1 -- 172.21.15.7:0/726092643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800b9d0 con 0x7f2444151510 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f24437fe700 1 -- 172.21.15.7:0/726092643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800bd60 con 0x7f2444151510 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f244b1ff700 1 -- 172.21.15.7:0/726092643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 msgr2=0x7f24440c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.388+0000 7f244b1ff700 1 --2- 172.21.15.7:0/726092643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c5b40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f244406b170 tx=0x7f2438009bd0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 -- 172.21.15.7:0/726092643 shutdown_connections 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 --2- 172.21.15.7:0/726092643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c5b40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 --2- 172.21.15.7:0/726092643 >> v2:172.21.15.117:3300/0 conn(0x7f244415b380 0x7f2444070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 -- 172.21.15.7:0/726092643 >> 172.21.15.7:0/726092643 conn(0x7f244406a4d0 msgr2=0x7f244406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 -- 172.21.15.7:0/726092643 shutdown_connections 2023-06-09T11:15:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.389+0000 7f244b1ff700 1 -- 172.21.15.7:0/726092643 wait complete. 2023-06-09T11:15:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.390+0000 7f244b1ff700 1 Processor -- start 2023-06-09T11:15:04.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 -- start start 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2444071060 0x7f24441f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244415b380 0x7f24440c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6670 con 0x7f2444151510 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6950 con 0x7f244415b380 2023-06-09T11:15:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.391+0000 7f244b1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c6c30 con 0x7f2444071060 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40808/0 (socket says 172.21.15.7:40808) 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 -- 172.21.15.7:0/4035086829 learned_addr learned my addr 172.21.15.7:0/4035086829 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f244979c700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244415b380 0x7f24440c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2448f9b700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2444071060 0x7f24441f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 -- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2444071060 msgr2=0x7f24441f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2444071060 0x7f24441f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 -- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244415b380 msgr2=0x7f24440c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244415b380 0x7f24440c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 -- 172.21.15.7:0/4035086829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009550 con 0x7f2444151510 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.392+0000 7f2443fff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f243000d800 tx=0x7f243000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.393+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24300125f0 con 0x7f2444151510 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.393+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430012750 con 0x7f2444151510 2023-06-09T11:15:04.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.393+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430012b00 con 0x7f2444151510 2023-06-09T11:15:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.393+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444157770 con 0x7f2444151510 2023-06-09T11:15:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.393+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2444157490 con 0x7f2444151510 2023-06-09T11:15:04.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.394+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444157a50 con 0x7f2444151510 2023-06-09T11:15:04.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.396+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2430012c60 con 0x7f2444151510 2023-06-09T11:15:04.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.396+0000 7f24417fa700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 0x7f242806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:04.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.396+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f24300895f0 con 0x7f2444151510 2023-06-09T11:15:04.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.396+0000 7f2448f9b700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 0x7f242806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:04.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.397+0000 7f2448f9b700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 0x7f242806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f2438008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:04.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.401+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2430054520 con 0x7f2444151510 2023-06-09T11:15:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.652+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2444074cf0 con 0x7f2444151510 2023-06-09T11:15:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.652+0000 7f24417fa700 1 -- 172.21.15.7:0/4035086829 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f2430057b40 con 0x7f2444151510 2023-06-09T11:15:04.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:04.658 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:04.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 msgr2=0x7f242806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 0x7f242806d1c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f2438008030 comp rx=0 tx=0).stop 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 msgr2=0x7f24440c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f243000d800 tx=0x7f243000db10 comp rx=0 tx=0).stop 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 shutdown_connections 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f244415b380 0x7f24440c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f242806ad10 0x7f242806d1c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2444071060 0x7f24441f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 --2- 172.21.15.7:0/4035086829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2444151510 0x7f24440c1400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 >> 172.21.15.7:0/4035086829 conn(0x7f244406a4d0 msgr2=0x7f244414cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 shutdown_connections 2023-06-09T11:15:04.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:04.655+0000 7f244b1ff700 1 -- 172.21.15.7:0/4035086829 wait complete. 2023-06-09T11:15:04.663 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:04.677 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:06.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.885+0000 7f6931bff700 1 Processor -- start 2023-06-09T11:15:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.885+0000 7f6931bff700 1 -- start start 2023-06-09T11:15:06.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f6931bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f692c151510 0x7f692c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f6931bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 0x7f692c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f6931bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c1580a0 con 0x7f692c15b380 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f6931bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c1518e0 con 0x7f692c151510 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f6931bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c6180 con 0x7f692c071230 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f692b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f692c151510 0x7f692c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f692b7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f692c151510 0x7f692c0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:44216/0 (socket says 172.21.15.7:44216) 2023-06-09T11:15:06.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f692bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 0x7f692c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.887+0000 7f692b7fe700 1 -- 172.21.15.7:0/3614940663 learned_addr learned my addr 172.21.15.7:0/3614940663 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692b7fe700 1 -- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 msgr2=0x7f692c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692b7fe700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 0x7f692c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692b7fe700 1 -- 172.21.15.7:0/3614940663 >> v1:172.21.15.117:6789/0 conn(0x7f692c15b380 legacy=0x7f692c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692b7fe700 1 -- 172.21.15.7:0/3614940663 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c157490 con 0x7f692c151510 2023-06-09T11:15:06.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692bfff700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 0x7f692c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692a7fc700 1 -- 172.21.15.7:0/3614940663 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675082647 0 0) 0x7f692c1580a0 con 0x7f692c15b380 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692b7fe700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c5b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098f0 tx=0x7f691c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de87fafbe8c699e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692a7fc700 1 -- 172.21.15.7:0/3614940663 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011830 con 0x7f692c151510 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692a7fc700 1 -- 172.21.15.7:0/3614940663 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c011990 con 0x7f692c151510 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.888+0000 7f692a7fc700 1 -- 172.21.15.7:0/3614940663 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c011d20 con 0x7f692c151510 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 -- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 msgr2=0x7f692c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:06.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c5b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f691c0098f0 tx=0x7f691c009c00 comp rx=0 tx=0).stop 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 -- 172.21.15.7:0/3614940663 shutdown_connections 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c071230 0x7f692c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 --2- 172.21.15.7:0/3614940663 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c5b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 -- 172.21.15.7:0/3614940663 >> 172.21.15.7:0/3614940663 conn(0x7f692c06a4d0 msgr2=0x7f692c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 -- 172.21.15.7:0/3614940663 shutdown_connections 2023-06-09T11:15:06.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.889+0000 7f6931bff700 1 -- 172.21.15.7:0/3614940663 wait complete. 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.891+0000 7f6931bff700 1 Processor -- start 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.891+0000 7f6931bff700 1 -- start start 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 0x7f692c0c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c0aa0 con 0x7f692c071230 2023-06-09T11:15:06.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c0140 con 0x7f692c151510 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f6931bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c03a0 con 0x7f692c15b380 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f692b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f692b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40810/0 (socket says 172.21.15.7:40810) 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.892+0000 7f692b7fe700 1 -- 172.21.15.7:0/627831617 learned_addr learned my addr 172.21.15.7:0/627831617 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692bfff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 0x7f692c0c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692affd700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 -- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 msgr2=0x7f692c0c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 0x7f692c0c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 -- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 msgr2=0x7f692c0c0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 -- 172.21.15.7:0/627831617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009a70 con 0x7f692c071230 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692bfff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 0x7f692c0c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692affd700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f692b7fe700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f691c009c40 tx=0x7f691c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c004480 con 0x7f692c071230 2023-06-09T11:15:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c01f070 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0038c0 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.893+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691c009550 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.894+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c157a50 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.895+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c157770 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.895+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f691c021030 con 0x7f692c071230 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.896+0000 7f6913fff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 0x7f691806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:06.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.896+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f691c08aaa0 con 0x7f692c071230 2023-06-09T11:15:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.897+0000 7f692affd700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 0x7f691806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.897+0000 7f692affd700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 0x7f691806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f692c0b04a0 tx=0x7f6920009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:06.901+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f691c055a50 con 0x7f692c071230 2023-06-09T11:15:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.153+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f692c157d30 con 0x7f692c071230 2023-06-09T11:15:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.154+0000 7f6913fff700 1 -- 172.21.15.7:0/627831617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f691c015090 con 0x7f692c071230 2023-06-09T11:15:07.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:07.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 msgr2=0x7f691806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 0x7f691806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f692c0b04a0 tx=0x7f6920009040 comp rx=0 tx=0).stop 2023-06-09T11:15:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 msgr2=0x7f692c14ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f691c009c40 tx=0x7f691c004750 comp rx=0 tx=0).stop 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 shutdown_connections 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f691806ace0 0x7f691806d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f692c15b380 0x7f692c0c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f692c071230 0x7f692c14ff30 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 --2- 172.21.15.7:0/627831617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f692c151510 0x7f692c0c0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 >> 172.21.15.7:0/627831617 conn(0x7f692c06a4d0 msgr2=0x7f692c14d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 shutdown_connections 2023-06-09T11:15:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:07.156+0000 7f6931bff700 1 -- 172.21.15.7:0/627831617 wait complete. 2023-06-09T11:15:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:07.179 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:09.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.385+0000 7f59bd7ff700 1 Processor -- start 2023-06-09T11:15:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.386+0000 7f59bd7ff700 1 -- start start 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59bd7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59b80c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59bd7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b80c5a80 con 0x7f59b8153880 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59bd7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b815a500 con 0x7f59b815b4a0 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59bd7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80c5f10 con 0x7f59b80c9f40 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59b6ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f59b8153880 0x7f59b815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39338/0 (socket says 172.21.15.7:39338) 2023-06-09T11:15:09.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59b6ffd700 1 -- 172.21.15.7:0/652288003 learned_addr learned my addr 172.21.15.7:0/652288003 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.387+0000 7f59b77fe700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59b80c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827881900 0 0) 0x7f59b80c5a80 con 0x7f59b8153880 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59a00035b0 con 0x7f59b8153880 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133915707 0 0) 0x7f59a00035b0 con 0x7f59b8153880 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 msgr2=0x7f59b80c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59b80c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 >> v1:172.21.15.117:6789/0 conn(0x7f59b815b4a0 legacy=0x7f59b80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b81596e0 con 0x7f59b8153880 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f59a8001670 con 0x7f59b8153880 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 >> v1:172.21.15.7:6789/0 conn(0x7f59b8153880 legacy=0x7f59b815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59a0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59a0003f70 0x7f59a0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0004d90 0x7f59a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80c5f10 con 0x7f59a0003f70 2023-06-09T11:15:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a000baf0 con 0x7f59a0004d90 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a000bcb0 con 0x7f59b80c9f40 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b67fc700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59a0003f70 0x7f59a0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b6ffd700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59a0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.388+0000 7f59b77fe700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0004d90 0x7f59a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 -- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 msgr2=0x7f59a0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b80c9f40 0x7f59a0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 -- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0004d90 msgr2=0x7f59a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0004d90 0x7f59a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 -- 172.21.15.7:0/652288003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00035b0 con 0x7f59a0003f70 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b77fe700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0004d90 0x7f59a0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b67fc700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59a0003f70 0x7f59a0004850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f59ac001870 tx=0x7f59a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67a79ab2ba9dc1aa server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac001500 con 0x7f59a0003f70 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac012640 con 0x7f59a0003f70 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.389+0000 7f59b5ffb700 1 -- 172.21.15.7:0/652288003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac0129f0 con 0x7f59a0003f70 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 -- 172.21.15.7:0/652288003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59a0003f70 msgr2=0x7f59a0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/652288003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59a0003f70 0x7f59a0004850 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f59ac001870 tx=0x7f59a0005880 comp rx=0 tx=0).stop 2023-06-09T11:15:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59b77fe700 1 -- 172.21.15.7:0/652288003 reap_dead start 2023-06-09T11:15:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 -- 172.21.15.7:0/652288003 shutdown_connections 2023-06-09T11:15:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 -- 172.21.15.7:0/652288003 >> 172.21.15.7:0/652288003 conn(0x7f59b80737f0 msgr2=0x7f59b80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 -- 172.21.15.7:0/652288003 shutdown_connections 2023-06-09T11:15:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.390+0000 7f59bd7ff700 1 -- 172.21.15.7:0/652288003 wait complete. 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.392+0000 7f59bd7ff700 1 Processor -- start 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.392+0000 7f59bd7ff700 1 -- start start 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0003f70 0x7f59b80cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b81e3610 0x7f59b815c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81eef80 con 0x7f59b81527e0 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8153190 con 0x7f59a0003f70 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59bd7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81ef190 con 0x7f59b81e3610 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40836/0 (socket says 172.21.15.7:40836) 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 -- 172.21.15.7:0/3968336361 learned_addr learned my addr 172.21.15.7:0/3968336361 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b67fc700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b81e3610 0x7f59b815c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b6ffd700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0003f70 0x7f59b80cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 -- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b81e3610 msgr2=0x7f59b815c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b81e3610 0x7f59b815c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 -- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0003f70 msgr2=0x7f59b80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0003f70 0x7f59b80cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.394+0000 7f59b77fe700 1 -- 172.21.15.7:0/3968336361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b8159c60 con 0x7f59b81527e0 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f59b77fe700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f59b806b800 tx=0x7f59b00104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b001b070 con 0x7f59b81527e0 2023-06-09T11:15:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b000f040 con 0x7f59b81527e0 2023-06-09T11:15:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b8159980 con 0x7f59b81527e0 2023-06-09T11:15:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000e050 con 0x7f59b81527e0 2023-06-09T11:15:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.395+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b8159f40 con 0x7f59b81527e0 2023-06-09T11:15:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.396+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b81f2510 con 0x7f59b81527e0 2023-06-09T11:15:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.397+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f59b0003a20 con 0x7f59b81527e0 2023-06-09T11:15:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.398+0000 7f599f7fe700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 0x7f59a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.398+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f59b0089810 con 0x7f59b81527e0 2023-06-09T11:15:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.398+0000 7f59b6ffd700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 0x7f59a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.399+0000 7f59b6ffd700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 0x7f59a406d1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59a8001eb0 tx=0x7f59a8000fe0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:09.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.403+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f59b00547c0 con 0x7f59b81527e0 2023-06-09T11:15:09.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.656+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59b814fdb0 con 0x7f59b81527e0 2023-06-09T11:15:09.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.657+0000 7f599f7fe700 1 -- 172.21.15.7:0/3968336361 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f59b0057de0 con 0x7f59b81527e0 2023-06-09T11:15:09.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:09.662 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:09.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 msgr2=0x7f59a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 0x7f59a406d1c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59a8001eb0 tx=0x7f59a8000fe0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 msgr2=0x7f59b8152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f59b806b800 tx=0x7f59b00104a0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 shutdown_connections 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b81527e0 0x7f59b8152c50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f59a0003f70 0x7f59b80cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f59a406ad10 0x7f59a406d1c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 --2- 172.21.15.7:0/3968336361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f59b81e3610 0x7f59b815c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 >> 172.21.15.7:0/3968336361 conn(0x7f59b80737f0 msgr2=0x7f59b80cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:09.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.659+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 shutdown_connections 2023-06-09T11:15:09.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:09.660+0000 7f59bd7ff700 1 -- 172.21.15.7:0/3968336361 wait complete. 2023-06-09T11:15:09.666 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:09.682 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:11.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:11.153 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:11.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.888+0000 7febcf1ff700 1 Processor -- start 2023-06-09T11:15:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.889+0000 7febcf1ff700 1 -- start start 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febcf1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febc8161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febcf1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80c0bf0 con 0x7febc80c1bc0 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febcf1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80702f0 con 0x7febc815a0c0 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febcf1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8070110 con 0x7febc80c0140 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febccf9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febc80c1bc0 0x7febc81599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39350/0 (socket says 172.21.15.7:39350) 2023-06-09T11:15:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.890+0000 7febccf9b700 1 -- 172.21.15.7:0/3630362461 learned_addr learned my addr 172.21.15.7:0/3630362461 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febcd79c700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febc8161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007110402 0 0) 0x7febc80c0bf0 con 0x7febc80c1bc0 2023-06-09T11:15:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb80057c0 con 0x7febc80c1bc0 2023-06-09T11:15:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3467789838 0 0) 0x7febb80057c0 con 0x7febc80c1bc0 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 msgr2=0x7febc8161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febc8161cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 >> v1:172.21.15.117:6789/0 conn(0x7febc815a0c0 legacy=0x7febc81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80c0e90 con 0x7febc80c1bc0 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7febbc001670 con 0x7febc80c1bc0 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 >> v1:172.21.15.7:6789/0 conn(0x7febc80c1bc0 legacy=0x7febc81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febb8006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febb8006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febb8006fa0 0x7febb8007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8070110 con 0x7febb8005c40 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb800dcd0 con 0x7febb8006fa0 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.891+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb800de60 con 0x7febc80c0140 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febb8006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febc7fff700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febb8006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febcd79c700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febb8006fa0 0x7febb8007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 -- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 msgr2=0x7febb8006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80c0140 0x7febb8006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 -- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febb8006fa0 msgr2=0x7febb8007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febb8006fa0 0x7febb8007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 -- 172.21.15.7:0/3630362461 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80057c0 con 0x7febb8005c40 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febccf9b700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febb8006050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c2a0 tx=0x7febb8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1837c17d8858e56 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007c10 con 0x7febb8005c40 2023-06-09T11:15:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc00cde0 con 0x7febb8005c40 2023-06-09T11:15:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.892+0000 7febc77fe700 1 -- 172.21.15.7:0/3630362461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc005ba0 con 0x7febb8005c40 2023-06-09T11:15:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 -- 172.21.15.7:0/3630362461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 msgr2=0x7febb8006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 --2- 172.21.15.7:0/3630362461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febb8006050 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c2a0 tx=0x7febb8007a90 comp rx=0 tx=0).stop 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcd79c700 1 -- 172.21.15.7:0/3630362461 reap_dead start 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 -- 172.21.15.7:0/3630362461 shutdown_connections 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 -- 172.21.15.7:0/3630362461 >> 172.21.15.7:0/3630362461 conn(0x7febc806a4d0 msgr2=0x7febc806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 -- 172.21.15.7:0/3630362461 shutdown_connections 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.893+0000 7febcf1ff700 1 -- 172.21.15.7:0/3630362461 wait complete. 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.895+0000 7febcf1ff700 1 Processor -- start 2023-06-09T11:15:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.895+0000 7febcf1ff700 1 -- start start 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80b0590 0x7febc81ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc81f16e0 0x7febc81efc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80afd40 con 0x7febb8005c40 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81f1b50 con 0x7febc81f16e0 2023-06-09T11:15:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcf1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc81f0360 con 0x7febc80b0590 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40858/0 (socket says 172.21.15.7:40858) 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 -- 172.21.15.7:0/1111211353 learned_addr learned my addr 172.21.15.7:0/1111211353 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febc7fff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80b0590 0x7febc81ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febcd79c700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc81f16e0 0x7febc81efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 -- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80b0590 msgr2=0x7febc81ee740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80b0590 0x7febc81ee740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 -- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc81f16e0 msgr2=0x7febc81efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.896+0000 7febccf9b700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc81f16e0 0x7febc81efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febccf9b700 1 -- 172.21.15.7:0/1111211353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80c1450 con 0x7febb8005c40 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febccf9b700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c890 tx=0x7febc8156440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc001630 con 0x7febb8005c40 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc0070d0 con 0x7febb8005c40 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc80c1170 con 0x7febb8005c40 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007460 con 0x7febb8005c40 2023-06-09T11:15:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.897+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc80705d0 con 0x7febb8005c40 2023-06-09T11:15:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.898+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8071720 con 0x7febb8005c40 2023-06-09T11:15:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.899+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febbc019030 con 0x7febb8005c40 2023-06-09T11:15:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.901+0000 7febc57fa700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 0x7febac06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.901+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7febbc08a870 con 0x7febb8005c40 2023-06-09T11:15:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.901+0000 7febc7fff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 0x7febac06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.902+0000 7febc7fff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 0x7febac06d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7febc8071450 tx=0x7febb4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:11.905+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7febbc055820 con 0x7febb8005c40 2023-06-09T11:15:12.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.153+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc8074cf0 con 0x7febb8005c40 2023-06-09T11:15:12.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.154+0000 7febc57fa700 1 -- 172.21.15.7:0/1111211353 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7febbc01ae60 con 0x7febb8005c40 2023-06-09T11:15:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:12.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:12.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 msgr2=0x7febac06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 0x7febac06d1c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7febc8071450 tx=0x7febb4009040 comp rx=0 tx=0).stop 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 msgr2=0x7febc81ec6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c890 tx=0x7febc8156440 comp rx=0 tx=0).stop 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 shutdown_connections 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febc81f16e0 0x7febc81efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febb8005c40 0x7febc81ec6f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:12.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febac06ad10 0x7febac06d1c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 --2- 172.21.15.7:0/1111211353 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febc80b0590 0x7febc81ee740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 >> 172.21.15.7:0/1111211353 conn(0x7febc806a4d0 msgr2=0x7febc80c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 shutdown_connections 2023-06-09T11:15:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:12.157+0000 7febcf1ff700 1 -- 172.21.15.7:0/1111211353 wait complete. 2023-06-09T11:15:12.163 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:12.180 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:14.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.389+0000 7f51a77ff700 1 Processor -- start 2023-06-09T11:15:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.390+0000 7f51a77ff700 1 -- start start 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51a00cb5d0 0x7f51a00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51a015cd40 0x7f51a00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a0159b20 con 0x7f51a015cd40 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00cac80 con 0x7f51a00cb5d0 2023-06-09T11:15:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a01597d0 con 0x7f51a00cb9a0 2023-06-09T11:15:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a5d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51a015cd40 0x7f51a00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a5d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f51a015cd40 0x7f51a00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47032/0 (socket says 172.21.15.7:47032) 2023-06-09T11:15:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a5d9c700 1 -- 172.21.15.7:0/3009963429 learned_addr learned my addr 172.21.15.7:0/3009963429 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a559b700 1 --2- 172.21.15.7:0/3009963429 >> v2:172.21.15.117:3300/0 conn(0x7f51a00cb5d0 0x7f51a00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.391+0000 7f51a4d9a700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 -- 172.21.15.7:0/3009963429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 msgr2=0x7f51a0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 -- 172.21.15.7:0/3009963429 >> v2:172.21.15.117:3300/0 conn(0x7f51a00cb5d0 msgr2=0x7f51a00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/3009963429 >> v2:172.21.15.117:3300/0 conn(0x7f51a00cb5d0 0x7f51a00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 -- 172.21.15.7:0/3009963429 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0071ab0 con 0x7f51a015cd40 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00cbea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f51a008d420 tx=0x7f519800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96409fea80859ef3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:14.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f5197fff700 1 -- 172.21.15.7:0/3009963429 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800e070 con 0x7f51a015cd40 2023-06-09T11:15:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f5197fff700 1 -- 172.21.15.7:0/3009963429 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800b9c0 con 0x7f51a015cd40 2023-06-09T11:15:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.392+0000 7f5197fff700 1 -- 172.21.15.7:0/3009963429 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800bd70 con 0x7f51a015cd40 2023-06-09T11:15:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 -- 172.21.15.7:0/3009963429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 msgr2=0x7f51a00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00cbea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f51a008d420 tx=0x7f519800b330 comp rx=0 tx=0).stop 2023-06-09T11:15:14.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 -- 172.21.15.7:0/3009963429 shutdown_connections 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 --2- 172.21.15.7:0/3009963429 >> v2:172.21.15.117:3300/0 conn(0x7f51a00cb5d0 0x7f51a00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00cbea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 --2- 172.21.15.7:0/3009963429 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 -- 172.21.15.7:0/3009963429 >> 172.21.15.7:0/3009963429 conn(0x7f51a0070af0 msgr2=0x7f51a0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 -- 172.21.15.7:0/3009963429 shutdown_connections 2023-06-09T11:15:14.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.393+0000 7f51a77ff700 1 -- 172.21.15.7:0/3009963429 wait complete. 2023-06-09T11:15:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.395+0000 7f51a77ff700 1 Processor -- start 2023-06-09T11:15:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.395+0000 7f51a77ff700 1 -- start start 2023-06-09T11:15:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a00cb5d0 0x7f51a00c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a01ef480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00c7990 con 0x7f51a015cd40 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00c36b0 con 0x7f51a00cb5d0 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a00c7be0 con 0x7f51a00cb9a0 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47034/0 (socket says 172.21.15.7:47034) 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 -- 172.21.15.7:0/115108078 learned_addr learned my addr 172.21.15.7:0/115108078 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:14.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a559b700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a00cb5d0 0x7f51a00c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a4d9a700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a01ef480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 -- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 msgr2=0x7f51a01ef480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a01ef480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 -- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a00cb5d0 msgr2=0x7f51a00c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a00cb5d0 0x7f51a00c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a5d9c700 1 -- 172.21.15.7:0/115108078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5198009b30 con 0x7f51a015cd40 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.397+0000 7f51a4d9a700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a01ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:14.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f51a5d9c700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5198012000 tx=0x7f5198004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:14.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980092e0 con 0x7f51a015cd40 2023-06-09T11:15:14.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5198004250 con 0x7f51a015cd40 2023-06-09T11:15:14.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a015d4f0 con 0x7f51a015cd40 2023-06-09T11:15:14.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198004600 con 0x7f51a015cd40 2023-06-09T11:15:14.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.398+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a015d210 con 0x7f51a015cd40 2023-06-09T11:15:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.399+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a015d7d0 con 0x7f51a015cd40 2023-06-09T11:15:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.400+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5198004760 con 0x7f51a015cd40 2023-06-09T11:15:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.401+0000 7f5195ffb700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 0x7f518406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:14.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.402+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f519808a750 con 0x7f51a015cd40 2023-06-09T11:15:14.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.402+0000 7f51a559b700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 0x7f518406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:14.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.402+0000 7f51a559b700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 0x7f518406d1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f519c009950 tx=0x7f519c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.406+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5198055670 con 0x7f51a015cd40 2023-06-09T11:15:14.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.664+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51a01eeff0 con 0x7f51a015cd40 2023-06-09T11:15:14.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.664+0000 7f5195ffb700 1 -- 172.21.15.7:0/115108078 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f5198058c90 con 0x7f51a015cd40 2023-06-09T11:15:14.669 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:14.670 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:14.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 msgr2=0x7f518406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 0x7f518406d1c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f519c009950 tx=0x7f519c008040 comp rx=0 tx=0).stop 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 msgr2=0x7f51a00ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5198012000 tx=0x7f5198004ad0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 shutdown_connections 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51a015cd40 0x7f51a00ce8a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51a00cb5d0 0x7f51a00c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f518406ad10 0x7f518406d1c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 --2- 172.21.15.7:0/115108078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51a00cb9a0 0x7f51a01ef480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 >> 172.21.15.7:0/115108078 conn(0x7f51a0070af0 msgr2=0x7f51a0156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 shutdown_connections 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:14.667+0000 7f51a77ff700 1 -- 172.21.15.7:0/115108078 wait complete. 2023-06-09T11:15:14.673 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:14.689 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:16.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:16.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.897+0000 7f215d1ff700 1 Processor -- start 2023-06-09T11:15:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.897+0000 7f215d1ff700 1 -- start start 2023-06-09T11:15:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215d1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2158153880 0x7f215815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215815b4a0 0x7f21580c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215d1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215815a500 con 0x7f21580c9f40 2023-06-09T11:15:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215d1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580c5f10 con 0x7f2158153880 2023-06-09T11:15:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215815da00 con 0x7f215815b4a0 2023-06-09T11:15:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f21580c9f40 0x7f215815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57030/0 (socket says 172.21.15.7:57030) 2023-06-09T11:15:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.898+0000 7f215659c700 1 -- 172.21.15.7:0/1834207507 learned_addr learned my addr 172.21.15.7:0/1834207507 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2156d9d700 1 --2- 172.21.15.7:0/1834207507 >> v2:172.21.15.117:3300/0 conn(0x7f2158153880 0x7f215815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f215759e700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215815b4a0 0x7f21580c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844944488 0 0) 0x7f214c001110 con 0x7f21580c9f40 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21400035b0 con 0x7f21580c9f40 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2085764596 0 0) 0x7f21400035b0 con 0x7f21580c9f40 2023-06-09T11:15:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215815b4a0 msgr2=0x7f21580c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215815b4a0 0x7f21580c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 >> v2:172.21.15.117:3300/0 conn(0x7f2158153880 msgr2=0x7f215815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 --2- 172.21.15.7:0/1834207507 >> v2:172.21.15.117:3300/0 conn(0x7f2158153880 0x7f215815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21581596e0 con 0x7f21580c9f40 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f214c001800 con 0x7f21580c9f40 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 >> v1:172.21.15.7:6789/0 conn(0x7f21580c9f40 legacy=0x7f215815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2140003a00 0x7f2140003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21400048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2140004fc0 0x7f214000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21580c5f10 con 0x7f21400043e0 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215815da00 con 0x7f2140004fc0 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214000bf50 con 0x7f2140003a00 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f215759e700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215815b4a0 0x7f21580c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.899+0000 7f2156d9d700 1 --2- 172.21.15.7:0/1834207507 >> v2:172.21.15.117:3300/0 conn(0x7f2158153880 0x7f215815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:15:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21400048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215759e700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2140004fc0 0x7f214000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f2156d9d700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2140003a00 0x7f2140003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 -- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2140003a00 msgr2=0x7f2140003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2140003a00 0x7f2140003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 -- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2140004fc0 msgr2=0x7f214000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2140004fc0 0x7f214000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 -- 172.21.15.7:0/1834207507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400035b0 con 0x7f21400043e0 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.900+0000 7f215659c700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21400048b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f214c00c320 tx=0x7f2140005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c906e218a1b48b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00ce20 con 0x7f21400043e0 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c005960 con 0x7f21400043e0 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f2155d9b700 1 -- 172.21.15.7:0/1834207507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c005d10 con 0x7f21400043e0 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 -- 172.21.15.7:0/1834207507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 msgr2=0x7f21400048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 --2- 172.21.15.7:0/1834207507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21400048b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f214c00c320 tx=0x7f2140005880 comp rx=0 tx=0).stop 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215759e700 1 -- 172.21.15.7:0/1834207507 reap_dead start 2023-06-09T11:15:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 -- 172.21.15.7:0/1834207507 shutdown_connections 2023-06-09T11:15:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 -- 172.21.15.7:0/1834207507 >> 172.21.15.7:0/1834207507 conn(0x7f21580737f0 msgr2=0x7f21580720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 -- 172.21.15.7:0/1834207507 shutdown_connections 2023-06-09T11:15:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.901+0000 7f215d1ff700 1 -- 172.21.15.7:0/1834207507 wait complete. 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.903+0000 7f215d1ff700 1 Processor -- start 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.903+0000 7f215d1ff700 1 -- start start 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 0x7f21580b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 0x7f215814ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21580cba80 con 0x7f21400043e0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21581f02b0 con 0x7f215814c9e0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f215d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21580cc400 con 0x7f21580b43a0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f2156d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f2156d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47060/0 (socket says 172.21.15.7:47060) 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.904+0000 7f2156d9d700 1 -- 172.21.15.7:0/3743558877 learned_addr learned my addr 172.21.15.7:0/3743558877 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215659c700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 0x7f21580b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215759e700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 0x7f215814ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 -- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 msgr2=0x7f21580b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 0x7f21580b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 -- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 msgr2=0x7f215814ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 0x7f215814ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 -- 172.21.15.7:0/3743558877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158159c60 con 0x7f21400043e0 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215659c700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 0x7f21580b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215759e700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 0x7f215814ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f2156d9d700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2148011ca0 tx=0x7f214800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214800fb30 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214800fc90 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2158159980 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148004020 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.905+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2158159f40 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.906+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21580cca30 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.907+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2148004180 con 0x7f21400043e0 2023-06-09T11:15:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.908+0000 7f213f7fe700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.908+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6292+0+0 (secure 0 0 0) 0x7f2148089c40 con 0x7f21400043e0 2023-06-09T11:15:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.908+0000 7f215659c700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.909+0000 7f215659c700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f21400047d0 tx=0x7f214c00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:16.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:16.912+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2148054b70 con 0x7f21400043e0 2023-06-09T11:15:17.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.164+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2158074cf0 con 0x7f21400043e0 2023-06-09T11:15:17.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.164+0000 7f213f7fe700 1 -- 172.21.15.7:0/3743558877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+6215 (secure 0 0 0) 0x7f2148022070 con 0x7f21400043e0 2023-06-09T11:15:17.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:17.169 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5071,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:13:41.149204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":26,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":4288,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"d","rank":1,"incarnation":9,"state":"up:active","state_seq":6,"addr":"172.21.15.117:6835/4051500902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":4051500902},{"type":"v1","addr":"172.21.15.117:6835","nonce":4051500902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:17.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 msgr2=0x7f214406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 0x7f214406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f21400047d0 tx=0x7f214c00a000 comp rx=0 tx=0).stop 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 msgr2=0x7f21581efd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2148011ca0 tx=0x7f214800f520 comp rx=0 tx=0).stop 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 shutdown_connections 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215814c9e0 0x7f215814ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21400043e0 0x7f21581efd70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:17.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f214406ace0 0x7f214406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 --2- 172.21.15.7:0/3743558877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21580b43a0 0x7f21580b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 >> 172.21.15.7:0/3743558877 conn(0x7f21580737f0 msgr2=0x7f215815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 shutdown_connections 2023-06-09T11:15:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:17.167+0000 7f215d1ff700 1 -- 172.21.15.7:0/3743558877 wait complete. 2023-06-09T11:15:17.173 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2023-06-09T11:15:17.190 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:15:19.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:19.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.408+0000 7ff5cf7ff700 1 Processor -- start 2023-06-09T11:15:19.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.408+0000 7ff5cf7ff700 1 -- start start 2023-06-09T11:15:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.409+0000 7ff5cf7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.409+0000 7ff5cf7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5c815b380 0x7ff5c8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.409+0000 7ff5cf7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c81580a0 con 0x7ff5c8071060 2023-06-09T11:15:19.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.409+0000 7ff5cf7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80714e0 con 0x7ff5c815b380 2023-06-09T11:15:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.409+0000 7ff5cf7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81518e0 con 0x7ff5c8151510 2023-06-09T11:15:19.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5c815b380 0x7ff5c8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff5c815b380 0x7ff5c8070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47068/0 (socket says 172.21.15.7:47068) 2023-06-09T11:15:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 -- 172.21.15.7:0/380722569 learned_addr learned my addr 172.21.15.7:0/380722569 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5cd59b700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 -- 172.21.15.7:0/380722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 msgr2=0x7ff5c80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 -- 172.21.15.7:0/380722569 >> v1:172.21.15.117:6789/0 conn(0x7ff5c8071060 legacy=0x7ff5c8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 -- 172.21.15.7:0/380722569 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80c6180 con 0x7ff5c815b380 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.410+0000 7ff5ccd9a700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c815b380 0x7ff5c8070b20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80098f0 tx=0x7ff5b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ba97b71b92d3bc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5bffff700 1 -- 172.21.15.7:0/380722569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8011830 con 0x7ff5c815b380 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5bffff700 1 -- 172.21.15.7:0/380722569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b8011990 con 0x7ff5c815b380 2023-06-09T11:15:19.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5bffff700 1 -- 172.21.15.7:0/380722569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8011d20 con 0x7ff5c815b380 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/380722569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c815b380 msgr2=0x7ff5c8070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c815b380 0x7ff5c8070b20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80098f0 tx=0x7ff5b8009c00 comp rx=0 tx=0).stop 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/380722569 shutdown_connections 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/380722569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c815b380 0x7ff5c8070b20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/380722569 >> 172.21.15.7:0/380722569 conn(0x7ff5c806a4d0 msgr2=0x7ff5c806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.411+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/380722569 shutdown_connections 2023-06-09T11:15:19.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.412+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/380722569 wait complete. 2023-06-09T11:15:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.413+0000 7ff5cf7ff700 1 Processor -- start 2023-06-09T11:15:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.413+0000 7ff5cf7ff700 1 -- start start 2023-06-09T11:15:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c815b380 0x7ff5c80c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81edd90 con 0x7ff5c8071060 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8159190 con 0x7ff5c815b380 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cf7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c81edab0 con 0x7ff5c8151510 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cd59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.414+0000 7ff5cd59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47080/0 (socket says 172.21.15.7:47080) 2023-06-09T11:15:19.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 -- 172.21.15.7:0/3946411938 learned_addr learned my addr 172.21.15.7:0/3946411938 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5ccd9a700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cdd9c700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c815b380 0x7ff5c80c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 -- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 msgr2=0x7ff5c80c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 -- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c815b380 msgr2=0x7ff5c80c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c815b380 0x7ff5c80c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 -- 172.21.15.7:0/3946411938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b8009550 con 0x7ff5c8071060 2023-06-09T11:15:19.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cd59b700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e9b0 tx=0x7ff5c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4009710 con 0x7ff5c8071060 2023-06-09T11:15:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400c850 con 0x7ff5c8071060 2023-06-09T11:15:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c8157770 con 0x7ff5c8071060 2023-06-09T11:15:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.415+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400cbc0 con 0x7ff5c8071060 2023-06-09T11:15:19.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.416+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c8157490 con 0x7ff5c8071060 2023-06-09T11:15:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.416+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8157a50 con 0x7ff5c8071060 2023-06-09T11:15:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.417+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff5c400cd20 con 0x7ff5c8071060 2023-06-09T11:15:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.418+0000 7ff5bdffb700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 0x7ff5ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.418+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7ff5c4089af0 con 0x7ff5c8071060 2023-06-09T11:15:19.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.419+0000 7ff5ccd9a700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 0x7ff5ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.419+0000 7ff5ccd9a700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 0x7ff5ac06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8014040 tx=0x7ff5b8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.422+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff5c4054930 con 0x7ff5c8071060 2023-06-09T11:15:19.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.678+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5c8074cf0 con 0x7ff5c8071060 2023-06-09T11:15:19.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.678+0000 7ff5bdffb700 1 -- 172.21.15.7:0/3946411938 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5456 (secure 0 0 0) 0x7ff5c4021090 con 0x7ff5c8071060 2023-06-09T11:15:19.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:19.683 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:19.166725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.680+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 msgr2=0x7ff5ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.680+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 0x7ff5ac06d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8014040 tx=0x7ff5b8008030 comp rx=0 tx=0).stop 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.680+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 msgr2=0x7ff5c8159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.680+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5c400e9b0 tx=0x7ff5c400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 shutdown_connections 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5c815b380 0x7ff5c80c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5c8071060 0x7ff5c8159b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff5ac06ace0 0x7ff5ac06d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 --2- 172.21.15.7:0/3946411938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5c8151510 0x7ff5c80c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 >> 172.21.15.7:0/3946411938 conn(0x7ff5c806a4d0 msgr2=0x7ff5c81553f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:19.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 shutdown_connections 2023-06-09T11:15:19.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.681+0000 7ff5cf7ff700 1 -- 172.21.15.7:0/3946411938 wait complete. 2023-06-09T11:15:19.687 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2023-06-09T11:15:19.703 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:15:19.703 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:15:19.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.901+0000 7f7d617ff700 1 Processor -- start 2023-06-09T11:15:19.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.902+0000 7f7d617ff700 1 -- start start 2023-06-09T11:15:19.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d617ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c153880 0x7f7d5c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d617ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d5c0c9f40 0x7f7d5c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d617ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d5c15a500 con 0x7f7d5c15b2d0 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d617ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c0c5f10 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d617ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c15da00 con 0x7f7d5c153880 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d5a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d5c0c9f40 0x7f7d5c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d5b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7d5c15b2d0 0x7f7d5c0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57042/0 (socket says 172.21.15.7:57042) 2023-06-09T11:15:19.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d5b7fe700 1 -- 172.21.15.7:0/1297657721 learned_addr learned my addr 172.21.15.7:0/1297657721 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:19.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.903+0000 7f7d5affd700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c153880 0x7f7d5c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d59ffb700 1 -- 172.21.15.7:0/1297657721 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898212789 0 0) 0x7f7d54001490 con 0x7f7d5c15b2d0 2023-06-09T11:15:19.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/1297657721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c153880 msgr2=0x7f7d5c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:19.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d5a7fc700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c153880 0x7f7d5c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/1297657721 >> v1:172.21.15.7:6789/0 conn(0x7f7d5c15b2d0 legacy=0x7f7d5c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/1297657721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c15b750 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d5a7fc700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c0c9f40 0x7f7d5c15ad90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009840 tx=0x7f7d50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51fa5066c9bc48a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d59ffb700 1 -- 172.21.15.7:0/1297657721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50011830 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d59ffb700 1 -- 172.21.15.7:0/1297657721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50011990 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.904+0000 7f7d59ffb700 1 -- 172.21.15.7:0/1297657721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50011d20 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 -- 172.21.15.7:0/1297657721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c0c9f40 msgr2=0x7f7d5c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c0c9f40 0x7f7d5c15ad90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009840 tx=0x7f7d50009b50 comp rx=0 tx=0).stop 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 -- 172.21.15.7:0/1297657721 shutdown_connections 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c153880 0x7f7d5c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 --2- 172.21.15.7:0/1297657721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c0c9f40 0x7f7d5c15ad90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 -- 172.21.15.7:0/1297657721 >> 172.21.15.7:0/1297657721 conn(0x7f7d5c0737f0 msgr2=0x7f7d5c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 -- 172.21.15.7:0/1297657721 shutdown_connections 2023-06-09T11:15:19.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.905+0000 7f7d617ff700 1 -- 172.21.15.7:0/1297657721 wait complete. 2023-06-09T11:15:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.907+0000 7f7d617ff700 1 Processor -- start 2023-06-09T11:15:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.907+0000 7f7d617ff700 1 -- start start 2023-06-09T11:15:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0c9f40 0x7f7d5c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c15b2d0 0x7f7d5c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c1518c0 con 0x7f7d5c153880 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c152f40 con 0x7f7d5c15b2d0 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d617ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0cbbb0 con 0x7f7d5c0c9f40 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47100/0 (socket says 172.21.15.7:47100) 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/3906324981 learned_addr learned my addr 172.21.15.7:0/3906324981 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5b7fe700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c15b2d0 0x7f7d5c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5affd700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0c9f40 0x7f7d5c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0c9f40 msgr2=0x7f7d5c151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0c9f40 0x7f7d5c151ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c15b2d0 msgr2=0x7f7d5c06a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c15b2d0 0x7f7d5c06a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.908+0000 7f7d5a7fc700 1 -- 172.21.15.7:0/3906324981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d500094a0 con 0x7f7d5c153880 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d5a7fc700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009fd0 tx=0x7f7d50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d500036a0 con 0x7f7d5c153880 2023-06-09T11:15:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5001f070 con 0x7f7d5c153880 2023-06-09T11:15:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50003a40 con 0x7f7d5c153880 2023-06-09T11:15:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5c159980 con 0x7f7d5c153880 2023-06-09T11:15:19.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.909+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c1596e0 con 0x7f7d5c153880 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.910+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c159c60 con 0x7f7d5c153880 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.911+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d50003ba0 con 0x7f7d5c153880 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.912+0000 7f7d437fe700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 0x7f7d4806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.912+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f7d50021070 con 0x7f7d5c153880 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.912+0000 7f7d5affd700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 0x7f7d4806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.913+0000 7f7d5affd700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 0x7f7d4806d1c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b1f20 tx=0x7f7d4c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:19.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:19.916+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7d500559f0 con 0x7f7d5c153880 2023-06-09T11:15:20.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.164+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d5c151a00 con 0x7f7d5c153880 2023-06-09T11:15:20.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.165+0000 7f7d437fe700 1 -- 172.21.15.7:0/3906324981 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f7d5c151a00 con 0x7f7d5c153880 2023-06-09T11:15:20.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:20.170 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:19.166725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 msgr2=0x7f7d4806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 0x7f7d4806d1c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b1f20 tx=0x7f7d4c009210 comp rx=0 tx=0).stop 2023-06-09T11:15:20.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 msgr2=0x7f7d5c0b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009fd0 tx=0x7f7d50004750 comp rx=0 tx=0).stop 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 shutdown_connections 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d5c15b2d0 0x7f7d5c06a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7d4806ad10 0x7f7d4806d1c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d5c0c9f40 0x7f7d5c151ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 --2- 172.21.15.7:0/3906324981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d5c153880 0x7f7d5c0b3170 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:20.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.167+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 >> 172.21.15.7:0/3906324981 conn(0x7f7d5c0737f0 msgr2=0x7f7d5c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:20.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.168+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 shutdown_connections 2023-06-09T11:15:20.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:20.168+0000 7f7d617ff700 1 -- 172.21.15.7:0/3906324981 wait complete. 2023-06-09T11:15:20.174 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2023-06-09T11:15:20.191 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 27, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 27, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:15:19.166725+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:15:22.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:22.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.400+0000 7f4e1ddff700 1 Processor -- start 2023-06-09T11:15:22.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.400+0000 7f4e1ddff700 1 -- start start 2023-06-09T11:15:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e1ddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 0x7f4e1815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e1ddff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1815a500 con 0x7f4e180c9f40 2023-06-09T11:15:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e1ddff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1815da00 con 0x7f4e1815b4a0 2023-06-09T11:15:22.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e1ddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e180c5f10 con 0x7f4e18153880 2023-06-09T11:15:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e16ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e180c9f40 0x7f4e1815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57058/0 (socket says 172.21.15.7:57058) 2023-06-09T11:15:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.401+0000 7f4e16ffd700 1 -- 172.21.15.7:0/2751497060 learned_addr learned my addr 172.21.15.7:0/2751497060 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:22.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e177fe700 1 --2- 172.21.15.7:0/2751497060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 0x7f4e1815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967425609 0 0) 0x7f4e0c001110 con 0x7f4e180c9f40 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e000035b0 con 0x7f4e180c9f40 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607539727 0 0) 0x7f4e10001490 con 0x7f4e1815b4a0 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e0c001110 con 0x7f4e1815b4a0 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e177fe700 1 -- 172.21.15.7:0/2751497060 >> v1:172.21.15.117:6789/0 conn(0x7f4e1815b4a0 legacy=0x7f4e180c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e177fe700 1 -- 172.21.15.7:0/2751497060 >> v1:172.21.15.7:6789/0 conn(0x7f4e180c9f40 legacy=0x7f4e1815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.402+0000 7f4e177fe700 1 -- 172.21.15.7:0/2751497060 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e181596e0 con 0x7f4e18153880 2023-06-09T11:15:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e177fe700 1 --2- 172.21.15.7:0/2751497060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 0x7f4e1815d4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009870 tx=0x7f4e08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a691f2acd581b4d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:22.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08011830 con 0x7f4e18153880 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08011990 con 0x7f4e18153880 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e167fc700 1 -- 172.21.15.7:0/2751497060 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08011d20 con 0x7f4e18153880 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2751497060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 msgr2=0x7f4e1815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2751497060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 0x7f4e1815d4c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009870 tx=0x7f4e08009b80 comp rx=0 tx=0).stop 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2751497060 shutdown_connections 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2751497060 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e18153880 0x7f4e1815d4c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.403+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2751497060 >> 172.21.15.7:0/2751497060 conn(0x7f4e180737f0 msgr2=0x7f4e180720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:22.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.404+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2751497060 shutdown_connections 2023-06-09T11:15:22.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.404+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2751497060 wait complete. 2023-06-09T11:15:22.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.405+0000 7f4e1ddff700 1 Processor -- start 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.405+0000 7f4e1ddff700 1 -- start start 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e180c9f40 0x7f4e1814c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1815b4a0 0x7f4e18146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1814ae40 con 0x7f4e18153880 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181531d0 con 0x7f4e1815b4a0 2023-06-09T11:15:22.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.406+0000 7f4e1ddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e18149aa0 con 0x7f4e180c9f40 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47102/0 (socket says 172.21.15.7:47102) 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 -- 172.21.15.7:0/2734113270 learned_addr learned my addr 172.21.15.7:0/2734113270 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e177fe700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e180c9f40 0x7f4e1814c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e17fff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1815b4a0 0x7f4e18146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 -- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e180c9f40 msgr2=0x7f4e1814c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e180c9f40 0x7f4e1814c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 -- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1815b4a0 msgr2=0x7f4e18146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1815b4a0 0x7f4e18146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 -- 172.21.15.7:0/2734113270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e080094a0 con 0x7f4e18153880 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.407+0000 7f4e16ffd700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c730 tx=0x7f4e0c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.408+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c005d70 con 0x7f4e18153880 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.408+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c0070d0 con 0x7f4e18153880 2023-06-09T11:15:22.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.408+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c007460 con 0x7f4e18153880 2023-06-09T11:15:22.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.408+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e18159c60 con 0x7f4e18153880 2023-06-09T11:15:22.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.408+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e18159980 con 0x7f4e18153880 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.409+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e18159f40 con 0x7f4e18153880 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.410+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e0c0075c0 con 0x7f4e18153880 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.411+0000 7f4dfffff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.411+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4e0c0889a0 con 0x7f4e18153880 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.411+0000 7f4e177fe700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:22.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.412+0000 7f4e177fe700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 0x7f4e0406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097c0 tx=0x7f4e0801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:22.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.415+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4e0c053870 con 0x7f4e18153880 2023-06-09T11:15:22.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.669+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e1814b640 con 0x7f4e18153880 2023-06-09T11:15:22.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.669+0000 7f4dfffff700 1 -- 172.21.15.7:0/2734113270 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f4e1814b640 con 0x7f4e18153880 2023-06-09T11:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:22.674 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:19.166725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.671+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 msgr2=0x7f4e0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.671+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 0x7f4e0406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e080097c0 tx=0x7f4e0801c040 comp rx=0 tx=0).stop 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.671+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 msgr2=0x7f4e18146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.671+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00c730 tx=0x7f4e0c00ca40 comp rx=0 tx=0).stop 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 shutdown_connections 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e1815b4a0 0x7f4e18146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e0406ace0 0x7f4e0406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e180c9f40 0x7f4e1814c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 --2- 172.21.15.7:0/2734113270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18153880 0x7f4e18146640 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:22.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 >> 172.21.15.7:0/2734113270 conn(0x7f4e180737f0 msgr2=0x7f4e1814ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:22.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 shutdown_connections 2023-06-09T11:15:22.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:22.672+0000 7f4e1ddff700 1 -- 172.21.15.7:0/2734113270 wait complete. 2023-06-09T11:15:22.678 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2023-06-09T11:15:22.694 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:24.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.908+0000 7f78c67ff700 1 Processor -- start 2023-06-09T11:15:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.908+0000 7f78c67ff700 1 -- start start 2023-06-09T11:15:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.909+0000 7f78c67ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78c00b5360 0x7f78c015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.909+0000 7f78c67ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca580 0x7f78c00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78c67ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c015dcf0 con 0x7f78c00ca1b0 2023-06-09T11:15:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78c67ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c00c5e00 con 0x7f78c00b5360 2023-06-09T11:15:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78c67ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c015d450 con 0x7f78c00ca580 2023-06-09T11:15:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78bffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78c00ca1b0 0x7f78c01536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56118/0 (socket says 172.21.15.7:56118) 2023-06-09T11:15:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78bffff700 1 -- 172.21.15.7:0/4270356701 learned_addr learned my addr 172.21.15.7:0/4270356701 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/4270356701 >> v2:172.21.15.7:3300/0 conn(0x7f78c00b5360 0x7f78c015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78c4d9c700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca580 0x7f78c00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78beffd700 1 -- 172.21.15.7:0/4270356701 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239450281 0 0) 0x7f78c015dcf0 con 0x7f78c00ca1b0 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.910+0000 7f78beffd700 1 -- 172.21.15.7:0/4270356701 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a80035b0 con 0x7f78c00ca1b0 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78bf7fe700 1 -- 172.21.15.7:0/4270356701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca580 msgr2=0x7f78c00c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca580 0x7f78c00c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78bf7fe700 1 -- 172.21.15.7:0/4270356701 >> v1:172.21.15.7:6789/0 conn(0x7f78c00ca1b0 legacy=0x7f78c01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78bf7fe700 1 -- 172.21.15.7:0/4270356701 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c015d730 con 0x7f78c00b5360 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c015cdc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009840 tx=0x7f78b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7a1c43cdc61737 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78beffd700 1 -- 172.21.15.7:0/4270356701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011830 con 0x7f78c00b5360 2023-06-09T11:15:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78beffd700 1 -- 172.21.15.7:0/4270356701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b4011990 con 0x7f78c00b5360 2023-06-09T11:15:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.911+0000 7f78beffd700 1 -- 172.21.15.7:0/4270356701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b4011d20 con 0x7f78c00b5360 2023-06-09T11:15:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 -- 172.21.15.7:0/4270356701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 msgr2=0x7f78c015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c015cdc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009840 tx=0x7f78b4009b50 comp rx=0 tx=0).stop 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 -- 172.21.15.7:0/4270356701 shutdown_connections 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca580 0x7f78c00c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 --2- 172.21.15.7:0/4270356701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c015cdc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 -- 172.21.15.7:0/4270356701 >> 172.21.15.7:0/4270356701 conn(0x7f78c0072090 msgr2=0x7f78c0072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 -- 172.21.15.7:0/4270356701 shutdown_connections 2023-06-09T11:15:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.912+0000 7f78c67ff700 1 -- 172.21.15.7:0/4270356701 wait complete. 2023-06-09T11:15:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.914+0000 7f78c67ff700 1 Processor -- start 2023-06-09T11:15:24.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.915+0000 7f78c67ff700 1 -- start start 2023-06-09T11:15:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 0x7f78c01f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 0x7f78c01e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c01e4b40 con 0x7f78c00b5360 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c015ab90 con 0x7f78c00ca580 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78c67ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c00d01c0 con 0x7f78c00ca1b0 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58766/0 (socket says 172.21.15.7:58766) 2023-06-09T11:15:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.916+0000 7f78bffff700 1 -- 172.21.15.7:0/3559640796 learned_addr learned my addr 172.21.15.7:0/3559640796 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 0x7f78c01f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78c4d9c700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 0x7f78c01e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 -- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 msgr2=0x7f78c01f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 0x7f78c01f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 -- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 msgr2=0x7f78c01e61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 0x7f78c01e61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 -- 172.21.15.7:0/3559640796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b40094a0 con 0x7f78c00b5360 2023-06-09T11:15:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 0x7f78c01f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78c4d9c700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 0x7f78c01e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:15:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bffff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78b000c700 tx=0x7f78b000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0005d70 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b00070d0 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c006a570 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.917+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b0007460 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.918+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c015da10 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.919+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c006a850 con 0x7f78c00b5360 2023-06-09T11:15:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.920+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78b00075c0 con 0x7f78c00b5360 2023-06-09T11:15:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.921+0000 7f78bcff9700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.921+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f78b0088d00 con 0x7f78c00b5360 2023-06-09T11:15:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.922+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.922+0000 7f78bf7fe700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 0x7f78ac06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000c00 tx=0x7f78b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:24.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:24.925+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f78b0053b50 con 0x7f78c00b5360 2023-06-09T11:15:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.183+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78c0074cf0 con 0x7f78c00b5360 2023-06-09T11:15:25.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.184+0000 7f78bcff9700 1 -- 172.21.15.7:0/3559640796 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f78b0057170 con 0x7f78c00b5360 2023-06-09T11:15:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:25.189 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:19.166725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 msgr2=0x7f78ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 0x7f78ac06d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f78b4000c00 tx=0x7f78b4008030 comp rx=0 tx=0).stop 2023-06-09T11:15:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 msgr2=0x7f78c00cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:25.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f78b000c700 tx=0x7f78b000cac0 comp rx=0 tx=0).stop 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 shutdown_connections 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78c00ca580 0x7f78c01e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78c00b5360 0x7f78c00cfc80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78ac06ace0 0x7f78ac06d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 --2- 172.21.15.7:0/3559640796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78c00ca1b0 0x7f78c01f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.186+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 >> 172.21.15.7:0/3559640796 conn(0x7f78c0072090 msgr2=0x7f78c014f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:25.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.187+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 shutdown_connections 2023-06-09T11:15:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:25.187+0000 7f78c67ff700 1 -- 172.21.15.7:0/3559640796 wait complete. 2023-06-09T11:15:25.193 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 27 2023-06-09T11:15:25.210 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:27.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:27.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.417+0000 7fd01f9ff700 1 Processor -- start 2023-06-09T11:15:27.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.417+0000 7fd01f9ff700 1 -- start start 2023-06-09T11:15:27.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.418+0000 7fd01f9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd018153880 0x7fd01815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01f9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd01815b4a0 0x7fd0180c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01f9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01815a500 con 0x7fd0180c9f40 2023-06-09T11:15:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01f9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0180c5f10 con 0x7fd01815b4a0 2023-06-09T11:15:27.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01f9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01815da00 con 0x7fd018153880 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01df9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd01815b4a0 0x7fd0180c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01cf9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0180c9f40 0x7fd01815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56130/0 (socket says 172.21.15.7:56130) 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01cf9a700 1 -- 172.21.15.7:0/1484048330 learned_addr learned my addr 172.21.15.7:0/1484048330 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01d79b700 1 --2- 172.21.15.7:0/1484048330 >> v2:172.21.15.117:3300/0 conn(0x7fd018153880 0x7fd01815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01df9c700 1 -- 172.21.15.7:0/1484048330 >> v2:172.21.15.117:3300/0 conn(0x7fd018153880 msgr2=0x7fd01815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:27.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01df9c700 1 --2- 172.21.15.7:0/1484048330 >> v2:172.21.15.117:3300/0 conn(0x7fd018153880 0x7fd01815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01df9c700 1 -- 172.21.15.7:0/1484048330 >> v1:172.21.15.7:6789/0 conn(0x7fd0180c9f40 legacy=0x7fd01815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.419+0000 7fd01df9c700 1 -- 172.21.15.7:0/1484048330 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0181596e0 con 0x7fd01815b4a0 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd01df9c700 1 --2- 172.21.15.7:0/1484048330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0180c4e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd01808d340 tx=0x7fd01000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3136530506314ae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd00ffff700 1 -- 172.21.15.7:0/1484048330 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000e070 con 0x7fd01815b4a0 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd00ffff700 1 -- 172.21.15.7:0/1484048330 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000bb20 con 0x7fd01815b4a0 2023-06-09T11:15:27.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd00ffff700 1 -- 172.21.15.7:0/1484048330 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012430 con 0x7fd01815b4a0 2023-06-09T11:15:27.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd01f9ff700 1 -- 172.21.15.7:0/1484048330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 msgr2=0x7fd0180c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.420+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/1484048330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0180c4e30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd01808d340 tx=0x7fd01000b330 comp rx=0 tx=0).stop 2023-06-09T11:15:27.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 -- 172.21.15.7:0/1484048330 shutdown_connections 2023-06-09T11:15:27.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/1484048330 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0180c4e30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/1484048330 >> v2:172.21.15.117:3300/0 conn(0x7fd018153880 0x7fd01815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 -- 172.21.15.7:0/1484048330 >> 172.21.15.7:0/1484048330 conn(0x7fd0180737f0 msgr2=0x7fd0180720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 -- 172.21.15.7:0/1484048330 shutdown_connections 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.421+0000 7fd01f9ff700 1 -- 172.21.15.7:0/1484048330 wait complete. 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.422+0000 7fd01f9ff700 1 Processor -- start 2023-06-09T11:15:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.423+0000 7fd01f9ff700 1 -- start start 2023-06-09T11:15:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.423+0000 7fd01f9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0180c9f40 0x7fd0180cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.423+0000 7fd01f9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd018153880 0x7fd0180cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01f9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01f9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180cf390 con 0x7fd01815b4a0 2023-06-09T11:15:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01f9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180b54d0 con 0x7fd018153880 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01f9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180b5210 con 0x7fd0180c9f40 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58790/0 (socket says 172.21.15.7:58790) 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 -- 172.21.15.7:0/2166851937 learned_addr learned my addr 172.21.15.7:0/2166851937 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01d79b700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0180c9f40 0x7fd0180cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01cf9a700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd018153880 0x7fd0180cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 -- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0180c9f40 msgr2=0x7fd0180cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0180c9f40 0x7fd0180cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 -- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd018153880 msgr2=0x7fd0180cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd018153880 0x7fd0180cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.424+0000 7fd01df9c700 1 -- 172.21.15.7:0/2166851937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd010009a70 con 0x7fd01815b4a0 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd01df9c700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd010014040 tx=0x7fd010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012430 con 0x7fd01815b4a0 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd010020070 con 0x7fd01815b4a0 2023-06-09T11:15:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd018159c60 con 0x7fd01815b4a0 2023-06-09T11:15:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100098f0 con 0x7fd01815b4a0 2023-06-09T11:15:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.425+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd018159980 con 0x7fd01815b4a0 2023-06-09T11:15:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.426+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd018159f40 con 0x7fd01815b4a0 2023-06-09T11:15:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.427+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd010004040 con 0x7fd01815b4a0 2023-06-09T11:15:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.428+0000 7fd00dffb700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 0x7fcffc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.428+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fd010022070 con 0x7fd01815b4a0 2023-06-09T11:15:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.428+0000 7fd01d79b700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 0x7fcffc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.429+0000 7fd01d79b700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 0x7fcffc06d1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd014009be0 tx=0x7fd014009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.432+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd010056040 con 0x7fd01815b4a0 2023-06-09T11:15:27.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.685+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd018074cf0 con 0x7fd01815b4a0 2023-06-09T11:15:27.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.686+0000 7fd00dffb700 1 -- 172.21.15.7:0/2166851937 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5458 (secure 0 0 0) 0x7fd010059660 con 0x7fd01815b4a0 2023-06-09T11:15:27.690 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:27.690 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:25.200677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:resolve","state_seq":25,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 msgr2=0x7fcffc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 0x7fcffc06d1c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd014009be0 tx=0x7fd014009210 comp rx=0 tx=0).stop 2023-06-09T11:15:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 msgr2=0x7fd0181589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:27.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd010014040 tx=0x7fd010004ad0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 shutdown_connections 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd01815b4a0 0x7fd0181589a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcffc06ad10 0x7fcffc06d1c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0180c9f40 0x7fd0180cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 --2- 172.21.15.7:0/2166851937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd018153880 0x7fd0180cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 >> 172.21.15.7:0/2166851937 conn(0x7fd0180737f0 msgr2=0x7fd0181e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:27.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 shutdown_connections 2023-06-09T11:15:27.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:27.688+0000 7fd01f9ff700 1 -- 172.21.15.7:0/2166851937 wait complete. 2023-06-09T11:15:27.695 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2023-06-09T11:15:27.711 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:29.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:29.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.918+0000 7f24c93ff700 1 Processor -- start 2023-06-09T11:15:29.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.918+0000 7f24c93ff700 1 -- start start 2023-06-09T11:15:29.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.919+0000 7f24c93ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24c4153880 0x7f24c415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.919+0000 7f24c93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.919+0000 7f24c93ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c415a500 con 0x7f24c415b2d0 2023-06-09T11:15:29.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.919+0000 7f24c93ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c40c5f10 con 0x7f24c4153880 2023-06-09T11:15:29.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.919+0000 7f24c93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c415da00 con 0x7f24c40c9f40 2023-06-09T11:15:29.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24c4153880 0x7f24c415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c27fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c27fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:37518/0 (socket says 172.21.15.7:37518) 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c27fc700 1 -- 172.21.15.7:0/1279356171 learned_addr learned my addr 172.21.15.7:0/1279356171 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 -- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 msgr2=0x7f24c415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 -- 172.21.15.7:0/1279356171 >> v1:172.21.15.117:6789/0 conn(0x7f24c415b2d0 legacy=0x7f24c40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 -- 172.21.15.7:0/1279356171 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c415b750 con 0x7f24c4153880 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c1ffb700 1 -- 172.21.15.7:0/1279356171 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3048487395 0 0) 0x7f24bc001490 con 0x7f24c415b2d0 2023-06-09T11:15:29.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c27fc700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:15:29.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.920+0000 7f24c2ffd700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c4153880 0x7f24c415d4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009840 tx=0x7f24b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50e0e5d07705d9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:29.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c1ffb700 1 -- 172.21.15.7:0/1279356171 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4011830 con 0x7f24c4153880 2023-06-09T11:15:29.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c1ffb700 1 -- 172.21.15.7:0/1279356171 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4011990 con 0x7f24c4153880 2023-06-09T11:15:29.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c1ffb700 1 -- 172.21.15.7:0/1279356171 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4011d20 con 0x7f24c4153880 2023-06-09T11:15:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 -- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c4153880 msgr2=0x7f24c415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c4153880 0x7f24c415d4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009840 tx=0x7f24b4009b50 comp rx=0 tx=0).stop 2023-06-09T11:15:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 -- 172.21.15.7:0/1279356171 shutdown_connections 2023-06-09T11:15:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c4153880 0x7f24c415d4c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:29.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 --2- 172.21.15.7:0/1279356171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c40c9f40 0x7f24c415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:29.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 -- 172.21.15.7:0/1279356171 >> 172.21.15.7:0/1279356171 conn(0x7f24c40737f0 msgr2=0x7f24c40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:29.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.921+0000 7f24c93ff700 1 -- 172.21.15.7:0/1279356171 shutdown_connections 2023-06-09T11:15:29.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.922+0000 7f24c93ff700 1 -- 172.21.15.7:0/1279356171 wait complete. 2023-06-09T11:15:29.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.924+0000 7f24c93ff700 1 Processor -- start 2023-06-09T11:15:29.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.924+0000 7f24c93ff700 1 -- start start 2023-06-09T11:15:29.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c4153880 0x7f24c4149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c415b2d0 0x7f24c4148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c4158e50 con 0x7f24c40c9f40 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c4158050 con 0x7f24c415b2d0 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.925+0000 7f24c93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c414b570 con 0x7f24c4153880 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58798/0 (socket says 172.21.15.7:58798) 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 -- 172.21.15.7:0/3272428059 learned_addr learned my addr 172.21.15.7:0/3272428059 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c27fc700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c4153880 0x7f24c4149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c37fe700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c415b2d0 0x7f24c4148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 -- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c4153880 msgr2=0x7f24c4149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c4153880 0x7f24c4149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 -- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c415b2d0 msgr2=0x7f24c4148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:29.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c415b2d0 0x7f24c4148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:29.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 -- 172.21.15.7:0/3272428059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b8009550 con 0x7f24c40c9f40 2023-06-09T11:15:29.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.926+0000 7f24c2ffd700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009fd0 tx=0x7f24b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:29.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.927+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40036a0 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.927+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b401f070 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.927+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003a40 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.927+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b40094a0 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.927+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c4159980 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.928+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c41596e0 con 0x7f24c40c9f40 2023-06-09T11:15:29.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.929+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24b4003ba0 con 0x7f24c40c9f40 2023-06-09T11:15:29.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.930+0000 7f24ab7fe700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 0x7f24b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:29.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.930+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f24b4021070 con 0x7f24c40c9f40 2023-06-09T11:15:29.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.930+0000 7f24c27fc700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 0x7f24b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:29.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.931+0000 7f24c27fc700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 0x7f24b006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24c4157100 tx=0x7f24b80092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:29.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:29.935+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f24b4055d10 con 0x7f24c40c9f40 2023-06-09T11:15:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.183+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24c4159c60 con 0x7f24c40c9f40 2023-06-09T11:15:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.184+0000 7f24ab7fe700 1 -- 172.21.15.7:0/3272428059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+5458 (secure 0 0 0) 0x7f24b402b090 con 0x7f24c40c9f40 2023-06-09T11:15:30.189 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:30.189 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:25.200677+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:resolve","state_seq":25,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 msgr2=0x7f24b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 0x7f24b006d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24c4157100 tx=0x7f24b80092c0 comp rx=0 tx=0).stop 2023-06-09T11:15:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 msgr2=0x7f24c4158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009fd0 tx=0x7f24b4004750 comp rx=0 tx=0).stop 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 shutdown_connections 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24c415b2d0 0x7f24c4148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24c40c9f40 0x7f24c4158910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f24b006ace0 0x7f24b006d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 --2- 172.21.15.7:0/3272428059 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24c4153880 0x7f24c4149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:30.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.186+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 >> 172.21.15.7:0/3272428059 conn(0x7f24c40737f0 msgr2=0x7f24c414d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:30.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.187+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 shutdown_connections 2023-06-09T11:15:30.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:30.187+0000 7f24c93ff700 1 -- 172.21.15.7:0/3272428059 wait complete. 2023-06-09T11:15:30.193 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2023-06-09T11:15:30.209 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:32.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.416+0000 7f4dcfdff700 1 Processor -- start 2023-06-09T11:15:32.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.416+0000 7f4dcfdff700 1 -- start start 2023-06-09T11:15:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.417+0000 7f4dcfdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4dc80cb9a0 0x7f4dc80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcfdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc8159b20 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcfdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc80cac80 con 0x7f4dc815cd40 2023-06-09T11:15:32.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcfdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc81597d0 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcdb9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4dc80cb5d0 0x7f4dc8159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56138/0 (socket says 172.21.15.7:56138) 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcdb9b700 1 -- 172.21.15.7:0/3440429678 learned_addr learned my addr 172.21.15.7:0/3440429678 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/3440429678 >> v2:172.21.15.117:3300/0 conn(0x7f4dc80cb9a0 0x7f4dc80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dccb99700 1 -- 172.21.15.7:0/3440429678 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842048121 0 0) 0x7f4dc8159b20 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dccb99700 1 -- 172.21.15.7:0/3440429678 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db40035b0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcd39a700 1 -- 172.21.15.7:0/3440429678 >> v1:172.21.15.117:6789/0 conn(0x7f4dc815cd40 legacy=0x7f4dc80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcd39a700 1 -- 172.21.15.7:0/3440429678 >> v1:172.21.15.7:6789/0 conn(0x7f4dc80cb5d0 legacy=0x7f4dc8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.418+0000 7f4dcd39a700 1 -- 172.21.15.7:0/3440429678 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc8071ab0 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.419+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/3440429678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc80cb9a0 0x7f4dc80b52b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009840 tx=0x7f4db8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad843ada525991d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.419+0000 7f4dccb99700 1 -- 172.21.15.7:0/3440429678 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8011830 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.419+0000 7f4dccb99700 1 -- 172.21.15.7:0/3440429678 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8011990 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.419+0000 7f4dccb99700 1 -- 172.21.15.7:0/3440429678 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8011d20 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 -- 172.21.15.7:0/3440429678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc80cb9a0 msgr2=0x7f4dc80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/3440429678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc80cb9a0 0x7f4dc80b52b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009840 tx=0x7f4db8009b50 comp rx=0 tx=0).stop 2023-06-09T11:15:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 -- 172.21.15.7:0/3440429678 shutdown_connections 2023-06-09T11:15:32.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/3440429678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc80cb9a0 0x7f4dc80b52b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 -- 172.21.15.7:0/3440429678 >> 172.21.15.7:0/3440429678 conn(0x7f4dc8070af0 msgr2=0x7f4dc8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 -- 172.21.15.7:0/3440429678 shutdown_connections 2023-06-09T11:15:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.420+0000 7f4dcfdff700 1 -- 172.21.15.7:0/3440429678 wait complete. 2023-06-09T11:15:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.421+0000 7f4dcfdff700 1 Processor -- start 2023-06-09T11:15:32.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.421+0000 7f4dcfdff700 1 -- start start 2023-06-09T11:15:32.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:32.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 0x7f4dc80b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:32.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 0x7f4dc81f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:32.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81f2900 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81f2600 con 0x7f4dc815cd40 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.422+0000 7f4dcfdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80c5b80 con 0x7f4dc80cb9a0 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58802/0 (socket says 172.21.15.7:58802) 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 -- 172.21.15.7:0/997705 learned_addr learned my addr 172.21.15.7:0/997705 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dce39c700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 0x7f4dc81f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:32.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 0x7f4dc80b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 -- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 msgr2=0x7f4dc80b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 0x7f4dc80b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 -- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 msgr2=0x7f4dc81f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 0x7f4dc81f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 -- 172.21.15.7:0/997705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db80094a0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 0x7f4dc80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dce39c700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 0x7f4dc81f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:32.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcdb9b700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c730 tx=0x7f4dc400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4005d70 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc40070d0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc815d4f0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.423+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4007440 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.424+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc815d210 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.425+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc815d7d0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.426+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4dc40075a0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.426+0000 7f4dbe7fc700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 0x7f4dac06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.427+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f4dc4088b70 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.427+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 0x7f4dac06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:32.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.427+0000 7f4dcd39a700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 0x7f4dac06d1c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cbd70 tx=0x7f4db8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:32.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.431+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4dc4053af0 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.688+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc81f2740 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.689+0000 7f4dbe7fc700 1 -- 172.21.15.7:0/997705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f4dc81f2740 con 0x7f4dc80cb5d0 2023-06-09T11:15:32.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:32.695 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:31.213862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.691+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 msgr2=0x7f4dac06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.691+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 0x7f4dac06d1c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80cbd70 tx=0x7f4db8008030 comp rx=0 tx=0).stop 2023-06-09T11:15:32.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.691+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 msgr2=0x7f4dc80c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.691+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4dc400c730 tx=0x7f4dc400caf0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 shutdown_connections 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dc815cd40 0x7f4dc81f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dc80cb5d0 0x7f4dc80c4ff0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dac06ad10 0x7f4dac06d1c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 --2- 172.21.15.7:0/997705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dc80cb9a0 0x7f4dc80b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 >> 172.21.15.7:0/997705 conn(0x7f4dc8070af0 msgr2=0x7f4dc8158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 shutdown_connections 2023-06-09T11:15:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:32.692+0000 7f4dcfdff700 1 -- 172.21.15.7:0/997705 wait complete. 2023-06-09T11:15:32.699 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2023-06-09T11:15:32.716 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 29, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 29, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4291": { "addr": "172.21.15.117:6837/2733808306", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 2733808306, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 2733808306, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 4291, "incarnation": 4, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 4 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:reconnect", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 27, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:15:31.213862+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4291, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:15:34.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.922+0000 7f25627ff700 1 Processor -- start 2023-06-09T11:15:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.922+0000 7f25627ff700 1 -- start start 2023-06-09T11:15:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f255c1593d0 0x7f255c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f255c0c08d0 0x7f255c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c071d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0714f0 con 0x7f255c1593d0 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f255c0cc140 con 0x7f255c0c08d0 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f25627ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c0c1830 con 0x7f255c071910 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f255bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f255c1593d0 0x7f255c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f255bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f255c1593d0 0x7f255c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51828/0 (socket says 172.21.15.7:51828) 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f255bfff700 1 -- 172.21.15.7:0/3651054260 learned_addr learned my addr 172.21.15.7:0/3651054260 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f2560d9c700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.924+0000 7f255b7fe700 1 --2- 172.21.15.7:0/3651054260 >> v2:172.21.15.117:3300/0 conn(0x7f255c0c08d0 0x7f255c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 -- 172.21.15.7:0/3651054260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 msgr2=0x7f255c071d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c071d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 -- 172.21.15.7:0/3651054260 >> v2:172.21.15.117:3300/0 conn(0x7f255c0c08d0 msgr2=0x7f255c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 --2- 172.21.15.7:0/3651054260 >> v2:172.21.15.117:3300/0 conn(0x7f255c0c08d0 0x7f255c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 -- 172.21.15.7:0/3651054260 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c0c0560 con 0x7f255c1593d0 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255bfff700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c1593d0 0x7f255c1597a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f255c06b700 tx=0x7f254c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2d26e6c0011ae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:34.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255affd700 1 -- 172.21.15.7:0/3651054260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c00db60 con 0x7f255c1593d0 2023-06-09T11:15:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255affd700 1 -- 172.21.15.7:0/3651054260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254c00dcc0 con 0x7f255c1593d0 2023-06-09T11:15:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.925+0000 7f255affd700 1 -- 172.21.15.7:0/3651054260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254c017630 con 0x7f255c1593d0 2023-06-09T11:15:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 -- 172.21.15.7:0/3651054260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c1593d0 msgr2=0x7f255c1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c1593d0 0x7f255c1597a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f255c06b700 tx=0x7f254c00be70 comp rx=0 tx=0).stop 2023-06-09T11:15:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 -- 172.21.15.7:0/3651054260 shutdown_connections 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c071d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 --2- 172.21.15.7:0/3651054260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c1593d0 0x7f255c1597a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 --2- 172.21.15.7:0/3651054260 >> v2:172.21.15.117:3300/0 conn(0x7f255c0c08d0 0x7f255c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 -- 172.21.15.7:0/3651054260 >> 172.21.15.7:0/3651054260 conn(0x7f255c06a4d0 msgr2=0x7f255c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 -- 172.21.15.7:0/3651054260 shutdown_connections 2023-06-09T11:15:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.926+0000 7f25627ff700 1 -- 172.21.15.7:0/3651054260 wait complete. 2023-06-09T11:15:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.928+0000 7f25627ff700 1 Processor -- start 2023-06-09T11:15:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.928+0000 7f25627ff700 1 -- start start 2023-06-09T11:15:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c1df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 0x7f255c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1dff20 con 0x7f255c0c08d0 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1eb750 con 0x7f255c1593d0 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f25627ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255c1def90 con 0x7f255c071910 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f255b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51844/0 (socket says 172.21.15.7:51844) 2023-06-09T11:15:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f255b7fe700 1 -- 172.21.15.7:0/883871523 learned_addr learned my addr 172.21.15.7:0/883871523 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f255bfff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c1df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.929+0000 7f2560d9c700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 0x7f255c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 -- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 msgr2=0x7f255c1df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c1df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 -- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 msgr2=0x7f255c14efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 0x7f255c14efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 -- 172.21.15.7:0/883871523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c00b7f0 con 0x7f255c0c08d0 2023-06-09T11:15:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f2560d9c700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 0x7f255c14efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f255b7fe700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9b0 tx=0x7f255000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2550009710 con 0x7f255c0c08d0 2023-06-09T11:15:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255000c850 con 0x7f255c0c08d0 2023-06-09T11:15:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255000cbc0 con 0x7f255c0c08d0 2023-06-09T11:15:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255c0c1d60 con 0x7f255c0c08d0 2023-06-09T11:15:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.930+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255c0c1a80 con 0x7f255c0c08d0 2023-06-09T11:15:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.931+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f255c0c2040 con 0x7f255c0c08d0 2023-06-09T11:15:34.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.932+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f255000cd20 con 0x7f255c0c08d0 2023-06-09T11:15:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.933+0000 7f2558ff9700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 0x7f254806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.933+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f2550089710 con 0x7f255c0c08d0 2023-06-09T11:15:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.933+0000 7f255bfff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 0x7f254806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:34.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.934+0000 7f255bfff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 0x7f254806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f254c019040 tx=0x7f254c014000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:34.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:34.938+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2550058070 con 0x7f255c0c08d0 2023-06-09T11:15:35.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.190+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f255c14f500 con 0x7f255c0c08d0 2023-06-09T11:15:35.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.191+0000 7f2558ff9700 1 -- 172.21.15.7:0/883871523 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f2550057c00 con 0x7f255c0c08d0 2023-06-09T11:15:35.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:35.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:31.213862+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:35.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 msgr2=0x7f254806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:35.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 0x7f254806d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f254c019040 tx=0x7f254c014000 comp rx=0 tx=0).stop 2023-06-09T11:15:35.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 msgr2=0x7f255c1eb210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:35.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f255000e9b0 tx=0x7f255000ed70 comp rx=0 tx=0).stop 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 shutdown_connections 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f255c1593d0 0x7f255c14efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f254806ace0 0x7f254806d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f255c071910 0x7f255c1df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 --2- 172.21.15.7:0/883871523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f255c0c08d0 0x7f255c1eb210 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:35.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 >> 172.21.15.7:0/883871523 conn(0x7f255c06a4d0 msgr2=0x7f255c0c6310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 shutdown_connections 2023-06-09T11:15:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:35.193+0000 7f25627ff700 1 -- 172.21.15.7:0/883871523 wait complete. 2023-06-09T11:15:35.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 29 2023-06-09T11:15:35.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:37.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:37.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.426+0000 7f8ecadff700 1 Processor -- start 2023-06-09T11:15:37.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.426+0000 7f8ecadff700 1 -- start start 2023-06-09T11:15:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.427+0000 7f8ecadff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 0x7f8ec40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.427+0000 7f8ecadff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.427+0000 7f8ecadff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec415cac0 con 0x7f8ec40c6280 2023-06-09T11:15:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.427+0000 7f8ecadff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec40c8870 con 0x7f8ec406a4d0 2023-06-09T11:15:37.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.427+0000 7f8ecadff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec40c37d0 con 0x7f8ec406e460 2023-06-09T11:15:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 0x7f8ec40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 0x7f8ec40c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:39954/0 (socket says 172.21.15.7:39954) 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39306/0 (socket says 172.21.15.7:39306) 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec3fff700 1 -- 172.21.15.7:0/2365896676 learned_addr learned my addr 172.21.15.7:0/2365896676 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 -- 172.21.15.7:0/2365896676 >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 msgr2=0x7f8ec40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 --2- 172.21.15.7:0/2365896676 >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 0x7f8ec40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 -- 172.21.15.7:0/2365896676 >> v1:172.21.15.117:6789/0 conn(0x7f8ec40c6280 legacy=0x7f8ec40737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.428+0000 7f8ec939c700 1 -- 172.21.15.7:0/2365896676 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec415cd30 con 0x7f8ec406e460 2023-06-09T11:15:37.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ec939c700 1 --2- 172.21.15.7:0/2365896676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009d90 tx=0x7f8ebc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0a7ed773f123190 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:37.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ec37fe700 1 -- 172.21.15.7:0/2365896676 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc00e070 con 0x7f8ec406e460 2023-06-09T11:15:37.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ec37fe700 1 -- 172.21.15.7:0/2365896676 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc00bb20 con 0x7f8ec406e460 2023-06-09T11:15:37.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ec37fe700 1 -- 172.21.15.7:0/2365896676 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc012430 con 0x7f8ec406e460 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ecadff700 1 -- 172.21.15.7:0/2365896676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 msgr2=0x7f8ec406cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.429+0000 7f8ecadff700 1 --2- 172.21.15.7:0/2365896676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc009d90 tx=0x7f8ebc00b330 comp rx=0 tx=0).stop 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 -- 172.21.15.7:0/2365896676 shutdown_connections 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 --2- 172.21.15.7:0/2365896676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406e460 0x7f8ec406cf30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 --2- 172.21.15.7:0/2365896676 >> v2:172.21.15.117:3300/0 conn(0x7f8ec406a4d0 0x7f8ec40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 -- 172.21.15.7:0/2365896676 >> 172.21.15.7:0/2365896676 conn(0x7f8ec406f980 msgr2=0x7f8ec40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:37.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 -- 172.21.15.7:0/2365896676 shutdown_connections 2023-06-09T11:15:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.430+0000 7f8ecadff700 1 -- 172.21.15.7:0/2365896676 wait complete. 2023-06-09T11:15:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.431+0000 7f8ecadff700 1 Processor -- start 2023-06-09T11:15:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.431+0000 7f8ecadff700 1 -- start start 2023-06-09T11:15:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406a4d0 0x7f8ec413e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec40c6280 0x7f8ec4141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec4157500 con 0x7f8ec406e460 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec41efd50 con 0x7f8ec40c6280 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.432+0000 7f8ecadff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec41ef1d0 con 0x7f8ec406a4d0 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec8b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406a4d0 0x7f8ec413e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51860/0 (socket says 172.21.15.7:51860) 2023-06-09T11:15:37.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 -- 172.21.15.7:0/3372974465 learned_addr learned my addr 172.21.15.7:0/3372974465 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec939c700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec40c6280 0x7f8ec4141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 -- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406a4d0 msgr2=0x7f8ec413e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406a4d0 0x7f8ec413e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 -- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec40c6280 msgr2=0x7f8ec4141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec40c6280 0x7f8ec4141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 -- 172.21.15.7:0/3372974465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc009a70 con 0x7f8ec406e460 2023-06-09T11:15:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.433+0000 7f8ec3fff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000e910 tx=0x7f8eb000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.434+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb00096a0 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.434+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0013680 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.434+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0013a30 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.434+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec415d2f0 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.434+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec415d010 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.435+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec415d5d0 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.436+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8eb0013b90 con 0x7f8ec406e460 2023-06-09T11:15:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.437+0000 7f8ec17fa700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 0x7f8ea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.437+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8eb0089610 con 0x7f8ec406e460 2023-06-09T11:15:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.437+0000 7f8ec8b9b700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 0x7f8ea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.438+0000 7f8ec8b9b700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 0x7f8ea806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007fd0 tx=0x7f8eb8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:37.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.441+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8eb0054450 con 0x7f8ec406e460 2023-06-09T11:15:37.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.694+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ec4074cf0 con 0x7f8ec406e460 2023-06-09T11:15:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.694+0000 7f8ec17fa700 1 -- 172.21.15.7:0/3372974465 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f8eb0057a70 con 0x7f8ec406e460 2023-06-09T11:15:37.700 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:37.700 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:36.232937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:37.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 msgr2=0x7f8ea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 0x7f8ea806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8007fd0 tx=0x7f8eb8009040 comp rx=0 tx=0).stop 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 msgr2=0x7f8ec4159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8eb000e910 tx=0x7f8eb000ec20 comp rx=0 tx=0).stop 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 shutdown_connections 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ec40c6280 0x7f8ec4141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ea806ace0 0x7f8ea806d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ec406a4d0 0x7f8ec413e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 --2- 172.21.15.7:0/3372974465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ec406e460 0x7f8ec4159e70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 >> 172.21.15.7:0/3372974465 conn(0x7f8ec406f980 msgr2=0x7f8ec40c3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 shutdown_connections 2023-06-09T11:15:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:37.697+0000 7f8ecadff700 1 -- 172.21.15.7:0/3372974465 wait complete. 2023-06-09T11:15:37.704 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2023-06-09T11:15:37.721 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:39.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:39.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.928+0000 7f8c9ebff700 1 Processor -- start 2023-06-09T11:15:39.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.928+0000 7f8c9ebff700 1 -- start start 2023-06-09T11:15:39.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c9ebff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c9815b4a0 0x7f8c9815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c9ebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 0x7f8c980c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c9ebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9815a500 con 0x7f8c98153880 2023-06-09T11:15:39.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c9ebff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c980c5f10 con 0x7f8c9815b4a0 2023-06-09T11:15:39.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c9ebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9815da00 con 0x7f8c980c9f40 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c97fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c9815b4a0 0x7f8c9815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c97fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c9815b4a0 0x7f8c9815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51876/0 (socket says 172.21.15.7:51876) 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.929+0000 7f8c97fff700 1 -- 172.21.15.7:0/3044822957 learned_addr learned my addr 172.21.15.7:0/3044822957 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c9d19c700 1 --2- 172.21.15.7:0/3044822957 >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 0x7f8c980c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c97fff700 1 -- 172.21.15.7:0/3044822957 >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 msgr2=0x7f8c980c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c97fff700 1 --2- 172.21.15.7:0/3044822957 >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 0x7f8c980c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c97fff700 1 -- 172.21.15.7:0/3044822957 >> v1:172.21.15.117:6789/0 conn(0x7f8c98153880 legacy=0x7f8c9815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c97fff700 1 -- 172.21.15.7:0/3044822957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c981596e0 con 0x7f8c9815b4a0 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c9d19c700 1 --2- 172.21.15.7:0/3044822957 >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 0x7f8c980c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c97fff700 1 --2- 172.21.15.7:0/3044822957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9815d4c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c980749f0 tx=0x7f8c84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af2d4a7a5036f7ca server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c977fe700 1 -- 172.21.15.7:0/3044822957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84004230 con 0x7f8c9815b4a0 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c977fe700 1 -- 172.21.15.7:0/3044822957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c84004390 con 0x7f8c9815b4a0 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.930+0000 7f8c977fe700 1 -- 172.21.15.7:0/3044822957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c840036a0 con 0x7f8c9815b4a0 2023-06-09T11:15:39.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 -- 172.21.15.7:0/3044822957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 msgr2=0x7f8c9815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/3044822957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9815d4c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8c980749f0 tx=0x7f8c84009bd0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 -- 172.21.15.7:0/3044822957 shutdown_connections 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/3044822957 >> v2:172.21.15.117:3300/0 conn(0x7f8c980c9f40 0x7f8c980c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/3044822957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9815d4c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 -- 172.21.15.7:0/3044822957 >> 172.21.15.7:0/3044822957 conn(0x7f8c980737f0 msgr2=0x7f8c980720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 -- 172.21.15.7:0/3044822957 shutdown_connections 2023-06-09T11:15:39.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.931+0000 7f8c9ebff700 1 -- 172.21.15.7:0/3044822957 wait complete. 2023-06-09T11:15:39.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.933+0000 7f8c9ebff700 1 Processor -- start 2023-06-09T11:15:39.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.933+0000 7f8c9ebff700 1 -- start start 2023-06-09T11:15:39.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c980c9f40 0x7f8c98151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c98153880 0x7f8c981f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981f3350 con 0x7f8c9815b4a0 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98152150 con 0x7f8c98153880 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98152430 con 0x7f8c980c9f40 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9d19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51886/0 (socket says 172.21.15.7:51886) 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9d19c700 1 -- 172.21.15.7:0/298844387 learned_addr learned my addr 172.21.15.7:0/298844387 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:39.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c97fff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c98153880 0x7f8c981f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.934+0000 7f8c9c99b700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c980c9f40 0x7f8c98151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 -- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c980c9f40 msgr2=0x7f8c98151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c980c9f40 0x7f8c98151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 -- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c98153880 msgr2=0x7f8c981f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c98153880 0x7f8c981f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 -- 172.21.15.7:0/298844387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84009550 con 0x7f8c9815b4a0 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9d19c700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000cef0 tx=0x7f8c9000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9000ad00 con 0x7f8c9815b4a0 2023-06-09T11:15:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9000ae60 con 0x7f8c9815b4a0 2023-06-09T11:15:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98159c60 con 0x7f8c9815b4a0 2023-06-09T11:15:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90015680 con 0x7f8c9815b4a0 2023-06-09T11:15:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.935+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c98159980 con 0x7f8c9815b4a0 2023-06-09T11:15:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.936+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98159f40 con 0x7f8c9815b4a0 2023-06-09T11:15:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.937+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c9000d430 con 0x7f8c9815b4a0 2023-06-09T11:15:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.939+0000 7f8c957fa700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 0x7f8c7c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.939+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f8c90089b50 con 0x7f8c9815b4a0 2023-06-09T11:15:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.939+0000 7f8c9c99b700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 0x7f8c7c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:39.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.939+0000 7f8c9c99b700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 0x7f8c7c06d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c007fd0 tx=0x7f8c8c007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:39.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:39.943+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c90054990 con 0x7f8c9815b4a0 2023-06-09T11:15:40.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.196+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c980b2db0 con 0x7f8c9815b4a0 2023-06-09T11:15:40.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.196+0000 7f8c957fa700 1 -- 172.21.15.7:0/298844387 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f8c980b2db0 con 0x7f8c9815b4a0 2023-06-09T11:15:40.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:40.203 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:36.232937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:40.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 msgr2=0x7f8c7c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 0x7f8c7c06d1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c007fd0 tx=0x7f8c8c007ec0 comp rx=0 tx=0).stop 2023-06-09T11:15:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 msgr2=0x7f8c9806a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c9000cef0 tx=0x7f8c9000cf20 comp rx=0 tx=0).stop 2023-06-09T11:15:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 shutdown_connections 2023-06-09T11:15:40.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c9815b4a0 0x7f8c9806a740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c7c06ad10 0x7f8c7c06d1c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c980c9f40 0x7f8c98151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 --2- 172.21.15.7:0/298844387 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c98153880 0x7f8c981f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 >> 172.21.15.7:0/298844387 conn(0x7f8c980737f0 msgr2=0x7f8c980cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 shutdown_connections 2023-06-09T11:15:40.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:40.199+0000 7f8c9ebff700 1 -- 172.21.15.7:0/298844387 wait complete. 2023-06-09T11:15:40.207 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 30 2023-06-09T11:15:40.223 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:15:41.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:41.268 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:15:42.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.430+0000 7fdedc5ff700 1 Processor -- start 2023-06-09T11:15:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.430+0000 7fdedc5ff700 1 -- start start 2023-06-09T11:15:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdedc5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fded40c9f40 0x7fded40c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdedc5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40c5a80 con 0x7fded4153880 2023-06-09T11:15:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdedc5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded415a500 con 0x7fded415b4a0 2023-06-09T11:15:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdedc5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded40c5f10 con 0x7fded40c9f40 2023-06-09T11:15:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdeda39b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fded4153880 0x7fded415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57998/0 (socket says 172.21.15.7:57998) 2023-06-09T11:15:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdeda39b700 1 -- 172.21.15.7:0/624016359 learned_addr learned my addr 172.21.15.7:0/624016359 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:15:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.431+0000 7fdedab9c700 1 --2- 172.21.15.7:0/624016359 >> v2:172.21.15.7:3300/0 conn(0x7fded40c9f40 0x7fded40c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fdedab9c700 1 -- 172.21.15.7:0/624016359 >> v1:172.21.15.117:6789/0 conn(0x7fded415b4a0 legacy=0x7fded40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fdedab9c700 1 -- 172.21.15.7:0/624016359 >> v1:172.21.15.7:6789/0 conn(0x7fded4153880 legacy=0x7fded415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fdedab9c700 1 -- 172.21.15.7:0/624016359 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded41596e0 con 0x7fded40c9f40 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fded9399700 1 -- 172.21.15.7:0/624016359 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188727237 0 0) 0x7fded40c5a80 con 0x7fded4153880 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fdedab9c700 1 --2- 172.21.15.7:0/624016359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded40c9f40 0x7fded40c5540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009d90 tx=0x7fdecc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1472b4de9f3a95 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fded9399700 1 -- 172.21.15.7:0/624016359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00e070 con 0x7fded40c9f40 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fded9399700 1 -- 172.21.15.7:0/624016359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc00bb90 con 0x7fded40c9f40 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.432+0000 7fded9399700 1 -- 172.21.15.7:0/624016359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc003a20 con 0x7fded40c9f40 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 -- 172.21.15.7:0/624016359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded40c9f40 msgr2=0x7fded40c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/624016359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded40c9f40 0x7fded40c5540 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009d90 tx=0x7fdecc00b330 comp rx=0 tx=0).stop 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 -- 172.21.15.7:0/624016359 shutdown_connections 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/624016359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded40c9f40 0x7fded40c5540 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 -- 172.21.15.7:0/624016359 >> 172.21.15.7:0/624016359 conn(0x7fded40737f0 msgr2=0x7fded40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 -- 172.21.15.7:0/624016359 shutdown_connections 2023-06-09T11:15:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.433+0000 7fdedc5ff700 1 -- 172.21.15.7:0/624016359 wait complete. 2023-06-09T11:15:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.435+0000 7fdedc5ff700 1 Processor -- start 2023-06-09T11:15:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.436+0000 7fdedc5ff700 1 -- start start 2023-06-09T11:15:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 0x7fded41571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 0x7fded4158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4151d70 con 0x7fded4153880 2023-06-09T11:15:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded41f2f10 con 0x7fded415b4a0 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedc5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded41579e0 con 0x7fded40c9f40 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fded9b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fded9b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51904/0 (socket says 172.21.15.7:51904) 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fded9b9a700 1 -- 172.21.15.7:0/1815987575 learned_addr learned my addr 172.21.15.7:0/1815987575 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdeda39b700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 0x7fded41571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.437+0000 7fdedab9c700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 0x7fded4158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:42.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 -- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 msgr2=0x7fded41571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 0x7fded41571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 -- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 msgr2=0x7fded4158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 0x7fded4158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 -- 172.21.15.7:0/1815987575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc009a70 con 0x7fded4153880 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdeda39b700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 0x7fded41571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdedab9c700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 0x7fded4158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:15:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fded9b9a700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b4a0 tx=0x7fdec400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4011830 con 0x7fded4153880 2023-06-09T11:15:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec4011990 con 0x7fded4153880 2023-06-09T11:15:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4011d40 con 0x7fded4153880 2023-06-09T11:15:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded4159c60 con 0x7fded4153880 2023-06-09T11:15:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.438+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded4159980 con 0x7fded4153880 2023-06-09T11:15:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.439+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4159f40 con 0x7fded4153880 2023-06-09T11:15:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.440+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdec401a400 con 0x7fded4153880 2023-06-09T11:15:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.441+0000 7fdecaffd700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 0x7fdeb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.441+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7fdec4089560 con 0x7fded4153880 2023-06-09T11:15:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.442+0000 7fdeda39b700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 0x7fdeb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:42.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.442+0000 7fdeda39b700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 0x7fdeb806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fded0000f60 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:42.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.446+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdec4054320 con 0x7fded4153880 2023-06-09T11:15:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.700+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fded4074cf0 con 0x7fded4153880 2023-06-09T11:15:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.701+0000 7fdecaffd700 1 -- 172.21.15.7:0/1815987575 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fdec4057940 con 0x7fded4153880 2023-06-09T11:15:42.707 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:42.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:42.182070+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 msgr2=0x7fdeb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 0x7fdeb806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fded0000f60 tx=0x7fded0009040 comp rx=0 tx=0).stop 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 msgr2=0x7fded414f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdec400b4a0 tx=0x7fdec400b860 comp rx=0 tx=0).stop 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 shutdown_connections 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fded415b4a0 0x7fded4158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdeb806ace0 0x7fdeb806d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fded40c9f40 0x7fded41571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 --2- 172.21.15.7:0/1815987575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fded4153880 0x7fded414f240 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 >> 172.21.15.7:0/1815987575 conn(0x7fded40737f0 msgr2=0x7fded415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 shutdown_connections 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:42.703+0000 7fdedc5ff700 1 -- 172.21.15.7:0/1815987575 wait complete. 2023-06-09T11:15:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 31 2023-06-09T11:15:42.727 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=1, replacing gid=4288 2023-06-09T11:15:42.727 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.d 2023-06-09T11:15:44.808 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:15:44.803+0000 7f7e32bf1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:15:47.224 INFO:tasks.workunit.client.0.smithi007.stdout:./270/.snap/snap-subdir-test 2023-06-09T11:15:47.247 INFO:tasks.workunit.client.0.smithi007.stdout:./125/.snap/snap-subdir-test 2023-06-09T11:15:47.265 INFO:tasks.workunit.client.0.smithi007.stdout:./194/.snap/snap-subdir-test 2023-06-09T11:15:47.283 INFO:tasks.workunit.client.0.smithi007.stdout:./119/.snap/snap-subdir-test 2023-06-09T11:15:47.300 INFO:tasks.workunit.client.0.smithi007.stdout:./297/.snap/snap-subdir-test 2023-06-09T11:15:47.317 INFO:tasks.workunit.client.0.smithi007.stdout:./230/.snap/snap-subdir-test 2023-06-09T11:15:48.083 INFO:tasks.workunit.client.0.smithi007.stdout:./260/.snap/snap-subdir-test 2023-06-09T11:15:48.106 INFO:tasks.workunit.client.0.smithi007.stdout:./157/.snap/snap-subdir-test 2023-06-09T11:15:48.125 INFO:tasks.workunit.client.0.smithi007.stdout:./172/.snap/snap-subdir-test 2023-06-09T11:15:48.142 INFO:tasks.workunit.client.0.smithi007.stdout:./277/.snap/snap-subdir-test 2023-06-09T11:15:48.159 INFO:tasks.workunit.client.0.smithi007.stdout:./186/.snap/snap-subdir-test 2023-06-09T11:15:48.175 INFO:tasks.workunit.client.0.smithi007.stdout:./237/.snap/snap-subdir-test 2023-06-09T11:15:48.728 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:15:48.728 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:15:48.729 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:15:48.733 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:15:48.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:15:48.882 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:15:48.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.938+0000 7f1e533ff700 1 Processor -- start 2023-06-09T11:15:48.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.939+0000 7f1e533ff700 1 -- start start 2023-06-09T11:15:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e4c0c6280 0x7f1e4c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e4c06e460 0x7f1e4c073d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c0c8870 con 0x7f1e4c06e460 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c15cac0 con 0x7f1e4c0c6280 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e533ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0c37d0 con 0x7f1e4c06a4d0 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e5199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e4c06e460 0x7f1e4c073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e5199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e4c06e460 0x7f1e4c073d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53146/0 (socket says 172.21.15.7:53146) 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e5199c700 1 -- 172.21.15.7:0/3478679190 learned_addr learned my addr 172.21.15.7:0/3478679190 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e5099a700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.940+0000 7f1e5119b700 1 --2- 172.21.15.7:0/3478679190 >> v2:172.21.15.117:3300/0 conn(0x7f1e4c0c6280 0x7f1e4c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 -- 172.21.15.7:0/3478679190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 msgr2=0x7f1e4c0737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c0737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 -- 172.21.15.7:0/3478679190 >> v2:172.21.15.117:3300/0 conn(0x7f1e4c0c6280 msgr2=0x7f1e4c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:15:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 --2- 172.21.15.7:0/3478679190 >> v2:172.21.15.117:3300/0 conn(0x7f1e4c0c6280 0x7f1e4c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 -- 172.21.15.7:0/3478679190 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c15cd30 con 0x7f1e4c06e460 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e5199c700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c073d30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0743a0 tx=0x7f1e4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0746fe89702509 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e43fff700 1 -- 172.21.15.7:0/3478679190 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400e070 con 0x7f1e4c06e460 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e43fff700 1 -- 172.21.15.7:0/3478679190 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4400b910 con 0x7f1e4c06e460 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.941+0000 7f1e43fff700 1 -- 172.21.15.7:0/3478679190 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4400bcc0 con 0x7f1e4c06e460 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 -- 172.21.15.7:0/3478679190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 msgr2=0x7f1e4c073d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c073d30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c0743a0 tx=0x7f1e4400b330 comp rx=0 tx=0).stop 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 -- 172.21.15.7:0/3478679190 shutdown_connections 2023-06-09T11:15:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 --2- 172.21.15.7:0/3478679190 >> v2:172.21.15.117:3300/0 conn(0x7f1e4c0c6280 0x7f1e4c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c073d30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 --2- 172.21.15.7:0/3478679190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c0737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 -- 172.21.15.7:0/3478679190 >> 172.21.15.7:0/3478679190 conn(0x7f1e4c06f980 msgr2=0x7f1e4c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:48.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 -- 172.21.15.7:0/3478679190 shutdown_connections 2023-06-09T11:15:48.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.942+0000 7f1e533ff700 1 -- 172.21.15.7:0/3478679190 wait complete. 2023-06-09T11:15:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.945+0000 7f1e533ff700 1 Processor -- start 2023-06-09T11:15:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.945+0000 7f1e533ff700 1 -- start start 2023-06-09T11:15:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.946+0000 7f1e533ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c1ef640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e533ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e533ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c0c6280 0x7f1e4c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e533ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c153cd0 con 0x7f1e4c06e460 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e533ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c159960 con 0x7f1e4c0c6280 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e533ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c159c40 con 0x7f1e4c06a4d0 2023-06-09T11:15:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53150/0 (socket says 172.21.15.7:53150) 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 -- 172.21.15.7:0/323595562 learned_addr learned my addr 172.21.15.7:0/323595562 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5119b700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c1ef640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5199c700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c0c6280 0x7f1e4c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 -- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 msgr2=0x7f1e4c1ef640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c1ef640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 -- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c0c6280 msgr2=0x7f1e4c141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c0c6280 0x7f1e4c141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.947+0000 7f1e5099a700 1 -- 172.21.15.7:0/323595562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44009b30 con 0x7f1e4c06e460 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e5099a700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e930 tx=0x7f1e3c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c013680 con 0x7f1e4c06e460 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c0137e0 con 0x7f1e4c06e460 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c15d2f0 con 0x7f1e4c06e460 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c013b70 con 0x7f1e4c06e460 2023-06-09T11:15:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.948+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c15d010 con 0x7f1e4c06e460 2023-06-09T11:15:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.949+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c15d5d0 con 0x7f1e4c06e460 2023-06-09T11:15:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.950+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e3c013cd0 con 0x7f1e4c06e460 2023-06-09T11:15:48.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.952+0000 7f1e41ffb700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:15:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.952+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6378+0+0 (secure 0 0 0) 0x7f1e3c089880 con 0x7f1e4c06e460 2023-06-09T11:15:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.952+0000 7f1e5119b700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:15:48.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.952+0000 7f1e5119b700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 0x7f1e3006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009b40 tx=0x7f1e48009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:15:48.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:48.955+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1e3c0547e0 con 0x7f1e4c06e460 2023-06-09T11:15:49.068 INFO:tasks.workunit.client.0.smithi007.stdout:./190/.snap/snap-subdir-test 2023-06-09T11:15:49.084 INFO:tasks.workunit.client.0.smithi007.stdout:./213/.snap/snap-subdir-test 2023-06-09T11:15:49.101 INFO:tasks.workunit.client.0.smithi007.stdout:./104/.snap/snap-subdir-test 2023-06-09T11:15:49.118 INFO:tasks.workunit.client.0.smithi007.stdout:./164/.snap/snap-subdir-test 2023-06-09T11:15:49.135 INFO:tasks.workunit.client.0.smithi007.stdout:./112/.snap/snap-subdir-test 2023-06-09T11:15:49.153 INFO:tasks.workunit.client.0.smithi007.stdout:./295/.snap/snap-subdir-test 2023-06-09T11:15:49.171 INFO:tasks.workunit.client.0.smithi007.stdout:./272/.snap/snap-subdir-test 2023-06-09T11:15:49.190 INFO:tasks.workunit.client.0.smithi007.stdout:./100/.snap/snap-subdir-test 2023-06-09T11:15:49.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.230+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e4c074cf0 con 0x7f1e4c06e460 2023-06-09T11:15:49.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.231+0000 7f1e41ffb700 1 -- 172.21.15.7:0/323595562 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+6217 (secure 0 0 0) 0x7f1e3c057e00 con 0x7f1e4c06e460 2023-06-09T11:15:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:15:49.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:15:42.182070+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:15:49.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 msgr2=0x7f1e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:49.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 0x7f1e3006d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009b40 tx=0x7f1e48009210 comp rx=0 tx=0).stop 2023-06-09T11:15:49.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 msgr2=0x7f1e4c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:15:49.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00e930 tx=0x7f1e3c00ec40 comp rx=0 tx=0).stop 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 shutdown_connections 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e4c0c6280 0x7f1e4c141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e3006ace0 0x7f1e3006d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e4c06a4d0 0x7f1e4c1ef640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 --2- 172.21.15.7:0/323595562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e4c06e460 0x7f1e4c0cae20 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 >> 172.21.15.7:0/323595562 conn(0x7f1e4c06f980 msgr2=0x7f1e4c06dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:15:49.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 shutdown_connections 2023-06-09T11:15:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:15:49.233+0000 7f1e533ff700 1 -- 172.21.15.7:0/323595562 wait complete. 2023-06-09T11:15:49.241 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 32 2023-06-09T11:15:49.257 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:15:49.257 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 67 secs before thrashing 2023-06-09T11:15:49.715 INFO:tasks.workunit.client.0.smithi007.stdout:./134/.snap/snap-subdir-test 2023-06-09T11:15:49.732 INFO:tasks.workunit.client.0.smithi007.stdout:./200/.snap/snap-subdir-test 2023-06-09T11:15:50.079 INFO:tasks.workunit.client.0.smithi007.stdout:./115/.snap/snap-subdir-test 2023-06-09T11:15:50.097 INFO:tasks.workunit.client.0.smithi007.stdout:./284/.snap/snap-subdir-test 2023-06-09T11:15:50.114 INFO:tasks.workunit.client.0.smithi007.stdout:./280/.snap/snap-subdir-test 2023-06-09T11:15:50.133 INFO:tasks.workunit.client.0.smithi007.stdout:./116/.snap/snap-subdir-test 2023-06-09T11:15:50.150 INFO:tasks.workunit.client.0.smithi007.stdout:./245/.snap/snap-subdir-test 2023-06-09T11:15:50.167 INFO:tasks.workunit.client.0.smithi007.stdout:./195/.snap/snap-subdir-test 2023-06-09T11:15:50.184 INFO:tasks.workunit.client.0.smithi007.stdout:./138/.snap/snap-subdir-test 2023-06-09T11:15:50.207 INFO:tasks.workunit.client.0.smithi007.stdout:./135/.snap/snap-subdir-test 2023-06-09T11:15:50.227 INFO:tasks.workunit.client.0.smithi007.stdout:./117/.snap/snap-subdir-test 2023-06-09T11:15:50.244 INFO:tasks.workunit.client.0.smithi007.stdout:./241/.snap/snap-subdir-test 2023-06-09T11:15:50.266 INFO:tasks.workunit.client.0.smithi007.stdout:./143/.snap/snap-subdir-test 2023-06-09T11:15:50.700 INFO:tasks.workunit.client.0.smithi007.stdout:./180/.snap/snap-subdir-test 2023-06-09T11:15:50.725 INFO:tasks.workunit.client.0.smithi007.stdout:./126/.snap/snap-subdir-test 2023-06-09T11:15:50.748 INFO:tasks.workunit.client.0.smithi007.stdout:./161/.snap/snap-subdir-test 2023-06-09T11:15:50.851 INFO:tasks.workunit.client.0.smithi007.stdout:./204/.snap/snap-subdir-test 2023-06-09T11:15:50.874 INFO:tasks.workunit.client.0.smithi007.stdout:./176/.snap/snap-subdir-test 2023-06-09T11:15:50.890 INFO:tasks.workunit.client.0.smithi007.stdout:./255/.snap/snap-subdir-test 2023-06-09T11:15:50.908 INFO:tasks.workunit.client.0.smithi007.stdout:./215/.snap/snap-subdir-test 2023-06-09T11:15:50.927 INFO:tasks.workunit.client.0.smithi007.stdout:./111/.snap/snap-subdir-test 2023-06-09T11:15:50.946 INFO:tasks.workunit.client.0.smithi007.stdout:./238/.snap/snap-subdir-test 2023-06-09T11:15:50.967 INFO:tasks.workunit.client.0.smithi007.stdout:./191/.snap/snap-subdir-test 2023-06-09T11:15:50.988 INFO:tasks.workunit.client.0.smithi007.stdout:./184/.snap/snap-subdir-test 2023-06-09T11:15:51.009 INFO:tasks.workunit.client.0.smithi007.stdout:./192/.snap/snap-subdir-test 2023-06-09T11:15:51.028 INFO:tasks.workunit.client.0.smithi007.stdout:./103/.snap/snap-subdir-test 2023-06-09T11:15:51.048 INFO:tasks.workunit.client.0.smithi007.stdout:./128/.snap/snap-subdir-test 2023-06-09T11:15:51.069 INFO:tasks.workunit.client.0.smithi007.stdout:./229/.snap/snap-subdir-test 2023-06-09T11:15:51.091 INFO:tasks.workunit.client.0.smithi007.stdout:./144/.snap/snap-subdir-test 2023-06-09T11:15:51.112 INFO:tasks.workunit.client.0.smithi007.stdout:./146/.snap/snap-subdir-test 2023-06-09T11:15:51.131 INFO:tasks.workunit.client.0.smithi007.stdout:./227/.snap/snap-subdir-test 2023-06-09T11:15:51.146 INFO:tasks.workunit.client.0.smithi007.stdout:./278/.snap/snap-subdir-test 2023-06-09T11:15:52.050 INFO:tasks.workunit.client.0.smithi007.stdout:./141/.snap/snap-subdir-test 2023-06-09T11:15:52.074 INFO:tasks.workunit.client.0.smithi007.stdout:./234/.snap/snap-subdir-test 2023-06-09T11:15:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:./225/.snap/snap-subdir-test 2023-06-09T11:15:52.121 INFO:tasks.workunit.client.0.smithi007.stdout:./182/.snap/snap-subdir-test 2023-06-09T11:15:52.145 INFO:tasks.workunit.client.0.smithi007.stdout:./107/.snap/snap-subdir-test 2023-06-09T11:15:52.166 INFO:tasks.workunit.client.0.smithi007.stdout:./178/.snap/snap-subdir-test 2023-06-09T11:15:52.187 INFO:tasks.workunit.client.0.smithi007.stdout:./266/.snap/snap-subdir-test 2023-06-09T11:15:52.486 INFO:tasks.workunit.client.0.smithi007.stdout:./240/.snap/snap-subdir-test 2023-06-09T11:15:52.510 INFO:tasks.workunit.client.0.smithi007.stdout:./156/.snap/snap-subdir-test 2023-06-09T11:15:52.535 INFO:tasks.workunit.client.0.smithi007.stdout:./131/.snap/snap-subdir-test 2023-06-09T11:15:52.556 INFO:tasks.workunit.client.0.smithi007.stdout:./123/.snap/snap-subdir-test 2023-06-09T11:15:52.579 INFO:tasks.workunit.client.0.smithi007.stdout:./160/.snap/snap-subdir-test 2023-06-09T11:15:52.602 INFO:tasks.workunit.client.0.smithi007.stdout:./165/.snap/snap-subdir-test 2023-06-09T11:15:52.623 INFO:tasks.workunit.client.0.smithi007.stdout:./101/.snap/snap-subdir-test 2023-06-09T11:15:52.644 INFO:tasks.workunit.client.0.smithi007.stdout:./148/.snap/snap-subdir-test 2023-06-09T11:15:52.664 INFO:tasks.workunit.client.0.smithi007.stdout:./168/.snap/snap-subdir-test 2023-06-09T11:15:52.685 INFO:tasks.workunit.client.0.smithi007.stdout:./179/.snap/snap-subdir-test 2023-06-09T11:15:52.705 INFO:tasks.workunit.client.0.smithi007.stdout:./257/.snap/snap-subdir-test 2023-06-09T11:15:52.724 INFO:tasks.workunit.client.0.smithi007.stdout:./252/.snap/snap-subdir-test 2023-06-09T11:15:52.745 INFO:tasks.workunit.client.0.smithi007.stdout:./132/.snap/snap-subdir-test 2023-06-09T11:15:52.764 INFO:tasks.workunit.client.0.smithi007.stdout:./236/.snap/snap-subdir-test 2023-06-09T11:15:52.782 INFO:tasks.workunit.client.0.smithi007.stdout:./211/.snap/snap-subdir-test 2023-06-09T11:15:53.777 INFO:tasks.workunit.client.0.smithi007.stdout:./199/.snap/snap-subdir-test 2023-06-09T11:15:53.799 INFO:tasks.workunit.client.0.smithi007.stdout:./205/.snap/snap-subdir-test 2023-06-09T11:15:53.821 INFO:tasks.workunit.client.0.smithi007.stdout:./163/.snap/snap-subdir-test 2023-06-09T11:15:53.841 INFO:tasks.workunit.client.0.smithi007.stdout:./261/.snap/snap-subdir-test 2023-06-09T11:15:53.862 INFO:tasks.workunit.client.0.smithi007.stdout:./201/.snap/snap-subdir-test 2023-06-09T11:15:53.881 INFO:tasks.workunit.client.0.smithi007.stdout:./282/.snap/snap-subdir-test 2023-06-09T11:15:54.049 INFO:tasks.workunit.client.0.smithi007.stdout:./296/.snap/snap-subdir-test 2023-06-09T11:15:54.071 INFO:tasks.workunit.client.0.smithi007.stdout:./217/.snap/snap-subdir-test 2023-06-09T11:15:54.089 INFO:tasks.workunit.client.0.smithi007.stdout:./153/.snap/snap-subdir-test 2023-06-09T11:15:54.521 INFO:tasks.workunit.client.0.smithi007.stdout:./206/.snap/snap-subdir-test 2023-06-09T11:15:54.800 INFO:tasks.workunit.client.0.smithi007.stdout:./122/.snap/snap-subdir-test 2023-06-09T11:15:54.820 INFO:tasks.workunit.client.0.smithi007.stdout:./162/.snap/snap-subdir-test 2023-06-09T11:15:54.848 INFO:tasks.workunit.client.0.smithi007.stdout:./197/.snap/snap-subdir-test 2023-06-09T11:15:54.871 INFO:tasks.workunit.client.0.smithi007.stdout:./216/.snap/snap-subdir-test 2023-06-09T11:15:58.554 INFO:tasks.workunit.client.0.smithi007.stdout:./254/.snap/snap-subdir-test 2023-06-09T11:15:58.575 INFO:tasks.workunit.client.0.smithi007.stdout:./113/.snap/snap-subdir-test 2023-06-09T11:15:58.592 INFO:tasks.workunit.client.0.smithi007.stdout:./152/.snap/snap-subdir-test 2023-06-09T11:15:58.613 INFO:tasks.workunit.client.0.smithi007.stdout:./121/.snap/snap-subdir-test 2023-06-09T11:15:58.634 INFO:tasks.workunit.client.0.smithi007.stdout:./118/.snap/snap-subdir-test 2023-06-09T11:15:58.654 INFO:tasks.workunit.client.0.smithi007.stdout:./196/.snap/snap-subdir-test 2023-06-09T11:15:58.675 INFO:tasks.workunit.client.0.smithi007.stdout:./124/.snap/snap-subdir-test 2023-06-09T11:15:58.695 INFO:tasks.workunit.client.0.smithi007.stdout:./239/.snap/snap-subdir-test 2023-06-09T11:15:58.716 INFO:tasks.workunit.client.0.smithi007.stdout:./188/.snap/snap-subdir-test 2023-06-09T11:15:58.735 INFO:tasks.workunit.client.0.smithi007.stdout:./287/.snap/snap-subdir-test 2023-06-09T11:15:58.755 INFO:tasks.workunit.client.0.smithi007.stdout:./275/.snap/snap-subdir-test 2023-06-09T11:15:58.775 INFO:tasks.workunit.client.0.smithi007.stdout:./259/.snap/snap-subdir-test 2023-06-09T11:15:58.794 INFO:tasks.workunit.client.0.smithi007.stdout:./224/.snap/snap-subdir-test 2023-06-09T11:15:58.812 INFO:tasks.workunit.client.0.smithi007.stdout:./264/.snap/snap-subdir-test 2023-06-09T11:15:59.750 INFO:tasks.workunit.client.0.smithi007.stdout:./145/.snap/snap-subdir-test 2023-06-09T11:16:00.561 INFO:tasks.workunit.client.0.smithi007.stdout:./279/.snap/snap-subdir-test 2023-06-09T11:16:00.587 INFO:tasks.workunit.client.0.smithi007.stdout:./263/.snap/snap-subdir-test 2023-06-09T11:16:00.611 INFO:tasks.workunit.client.0.smithi007.stdout:./130/.snap/snap-subdir-test 2023-06-09T11:16:01.363 INFO:tasks.workunit.client.0.smithi007.stdout:./193/.snap/snap-subdir-test 2023-06-09T11:16:01.389 INFO:tasks.workunit.client.0.smithi007.stdout:./223/.snap/snap-subdir-test 2023-06-09T11:16:01.411 INFO:tasks.workunit.client.0.smithi007.stdout:./207/.snap/snap-subdir-test 2023-06-09T11:16:01.435 INFO:tasks.workunit.client.0.smithi007.stdout:./276/.snap/snap-subdir-test 2023-06-09T11:16:01.458 INFO:tasks.workunit.client.0.smithi007.stdout:./271/.snap/snap-subdir-test 2023-06-09T11:16:01.480 INFO:tasks.workunit.client.0.smithi007.stdout:./251/.snap/snap-subdir-test 2023-06-09T11:16:01.503 INFO:tasks.workunit.client.0.smithi007.stdout:./175/.snap/snap-subdir-test 2023-06-09T11:16:01.523 INFO:tasks.workunit.client.0.smithi007.stdout:./222/.snap/snap-subdir-test 2023-06-09T11:16:01.543 INFO:tasks.workunit.client.0.smithi007.stdout:./220/.snap/snap-subdir-test 2023-06-09T11:16:01.563 INFO:tasks.workunit.client.0.smithi007.stdout:./203/.snap/snap-subdir-test 2023-06-09T11:16:01.585 INFO:tasks.workunit.client.0.smithi007.stdout:./181/.snap/snap-subdir-test 2023-06-09T11:16:01.604 INFO:tasks.workunit.client.0.smithi007.stdout:./286/.snap/snap-subdir-test 2023-06-09T11:16:01.626 INFO:tasks.workunit.client.0.smithi007.stdout:./127/.snap/snap-subdir-test 2023-06-09T11:16:01.648 INFO:tasks.workunit.client.0.smithi007.stdout:./108/.snap/snap-subdir-test 2023-06-09T11:16:01.669 INFO:tasks.workunit.client.0.smithi007.stdout:./183/.snap/snap-subdir-test 2023-06-09T11:16:01.689 INFO:tasks.workunit.client.0.smithi007.stdout:./294/.snap/snap-subdir-test 2023-06-09T11:16:01.709 INFO:tasks.workunit.client.0.smithi007.stdout:./235/.snap/snap-subdir-test 2023-06-09T11:16:01.732 INFO:tasks.workunit.client.0.smithi007.stdout:./177/.snap/snap-subdir-test 2023-06-09T11:16:01.751 INFO:tasks.workunit.client.0.smithi007.stdout:./169/.snap/snap-subdir-test 2023-06-09T11:16:01.769 INFO:tasks.workunit.client.0.smithi007.stdout:./212/.snap/snap-subdir-test 2023-06-09T11:16:01.790 INFO:tasks.workunit.client.0.smithi007.stdout:./154/.snap/snap-subdir-test 2023-06-09T11:16:01.809 INFO:tasks.workunit.client.0.smithi007.stdout:./171/.snap/snap-subdir-test 2023-06-09T11:16:01.828 INFO:tasks.workunit.client.0.smithi007.stdout:./139/.snap/snap-subdir-test 2023-06-09T11:16:01.849 INFO:tasks.workunit.client.0.smithi007.stdout:./102/.snap/snap-subdir-test 2023-06-09T11:16:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:./129/.snap/snap-subdir-test 2023-06-09T11:16:11.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:11.374 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:20.586 INFO:tasks.workunit.client.0.smithi007.stdout:Delete all the files and directories ... 2023-06-09T11:16:22.499 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/19' 2023-06-09T11:16:22.499 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/15' 2023-06-09T11:16:22.499 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/31' 2023-06-09T11:16:22.500 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/36' 2023-06-09T11:16:22.500 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/20' 2023-06-09T11:16:22.500 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/12' 2023-06-09T11:16:22.500 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/39' 2023-06-09T11:16:22.501 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/10' 2023-06-09T11:16:22.501 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/34' 2023-06-09T11:16:22.501 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/38' 2023-06-09T11:16:22.501 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/11' 2023-06-09T11:16:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/16' 2023-06-09T11:16:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/32' 2023-06-09T11:16:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/30' 2023-06-09T11:16:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/14' 2023-06-09T11:16:22.502 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/18' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/13' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/33' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/17' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/35' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './100/37' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './100' 2023-06-09T11:16:22.503 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/19' 2023-06-09T11:16:22.504 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/15' 2023-06-09T11:16:22.504 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/31' 2023-06-09T11:16:22.504 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/36' 2023-06-09T11:16:22.504 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/20' 2023-06-09T11:16:22.504 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/12' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/39' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/10' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/34' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/38' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/11' 2023-06-09T11:16:22.505 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/16' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/32' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/30' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/14' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/18' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/13' 2023-06-09T11:16:22.506 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/33' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/17' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/35' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './101/37' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './101' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/19' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/15' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/31' 2023-06-09T11:16:22.507 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/36' 2023-06-09T11:16:22.508 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/20' 2023-06-09T11:16:22.508 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/12' 2023-06-09T11:16:22.508 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/39' 2023-06-09T11:16:22.508 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/10' 2023-06-09T11:16:22.508 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/34' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/38' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/11' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/16' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/32' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/30' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/14' 2023-06-09T11:16:22.509 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/18' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/13' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/33' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/17' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/35' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './102/37' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './102' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/19' 2023-06-09T11:16:22.510 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/15' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/31' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/36' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/20' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/12' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/39' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/10' 2023-06-09T11:16:22.511 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/34' 2023-06-09T11:16:22.512 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/38' 2023-06-09T11:16:22.512 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/11' 2023-06-09T11:16:22.512 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/16' 2023-06-09T11:16:22.512 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/32' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/30' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/14' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/18' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/13' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/33' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/17' 2023-06-09T11:16:22.513 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/35' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './103/37' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './103' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/19' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/15' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/31' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/36' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/20' 2023-06-09T11:16:22.514 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/12' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/39' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/10' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/34' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/38' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/11' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/16' 2023-06-09T11:16:22.515 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/32' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/30' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/14' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/18' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/13' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/33' 2023-06-09T11:16:22.516 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/17' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/35' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './104/37' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './104' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/19' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/15' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/31' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/36' 2023-06-09T11:16:22.517 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/20' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/12' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/39' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/10' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/34' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/38' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/11' 2023-06-09T11:16:22.518 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/16' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/32' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/30' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/14' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/18' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/13' 2023-06-09T11:16:22.519 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/33' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/17' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/35' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './105/37' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './105' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/19' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/15' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/31' 2023-06-09T11:16:22.520 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/36' 2023-06-09T11:16:22.521 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/20' 2023-06-09T11:16:22.521 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/12' 2023-06-09T11:16:22.521 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/39' 2023-06-09T11:16:22.521 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/10' 2023-06-09T11:16:22.521 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/34' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/38' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/11' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/16' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/32' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/30' 2023-06-09T11:16:22.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/14' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/18' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/13' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/33' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/17' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/35' 2023-06-09T11:16:22.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './106/37' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './106' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/19' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/15' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/31' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/36' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/20' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/12' 2023-06-09T11:16:22.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/39' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/10' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/34' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/38' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/11' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/16' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/32' 2023-06-09T11:16:22.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/30' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/14' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/18' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/13' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/33' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/17' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/35' 2023-06-09T11:16:22.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './107/37' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './107' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/19' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/15' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/31' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/36' 2023-06-09T11:16:22.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/20' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/12' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/39' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/10' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/34' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/38' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/11' 2023-06-09T11:16:22.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/16' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/32' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/30' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/14' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/18' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/13' 2023-06-09T11:16:22.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/33' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/17' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/35' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './108/37' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './108' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/19' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/15' 2023-06-09T11:16:22.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/31' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/36' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/20' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/12' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/39' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/10' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/34' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/38' 2023-06-09T11:16:22.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/11' 2023-06-09T11:16:22.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/16' 2023-06-09T11:16:22.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/32' 2023-06-09T11:16:22.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/30' 2023-06-09T11:16:24.080 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/14' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/18' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/13' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/33' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/17' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/35' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed './109/37' 2023-06-09T11:16:24.081 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './109' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/19' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/15' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/31' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/36' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/20' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/12' 2023-06-09T11:16:24.082 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/39' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/10' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/34' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/38' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/11' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/16' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/32' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/30' 2023-06-09T11:16:24.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/14' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/18' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/13' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/33' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/17' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/35' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './110/37' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './110' 2023-06-09T11:16:24.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/19' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/15' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/31' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/36' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/20' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/12' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/39' 2023-06-09T11:16:24.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/10' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/34' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/38' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/11' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/16' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/32' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/30' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/14' 2023-06-09T11:16:24.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/18' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/13' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/33' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/17' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/35' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './111/37' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './111' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/19' 2023-06-09T11:16:24.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/15' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/31' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/36' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/20' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/12' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/39' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/10' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/34' 2023-06-09T11:16:24.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/38' 2023-06-09T11:16:24.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/11' 2023-06-09T11:16:24.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/16' 2023-06-09T11:16:24.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/32' 2023-06-09T11:16:24.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/30' 2023-06-09T11:16:24.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/14' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/18' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/13' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/33' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/17' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/35' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './112/37' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './112' 2023-06-09T11:16:24.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/19' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/15' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/31' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/36' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/20' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/12' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/39' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/10' 2023-06-09T11:16:24.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/34' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/38' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/11' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/16' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/32' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/30' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/14' 2023-06-09T11:16:24.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/18' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/13' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/33' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/17' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/35' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './113/37' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './113' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/19' 2023-06-09T11:16:24.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/15' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/31' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/36' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/20' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/12' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/39' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/10' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/34' 2023-06-09T11:16:24.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/38' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/11' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/16' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/32' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/30' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/14' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/18' 2023-06-09T11:16:24.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/13' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/33' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/17' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/35' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './114/37' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './114' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/19' 2023-06-09T11:16:24.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/15' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/31' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/36' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/20' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/12' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/39' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/10' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/34' 2023-06-09T11:16:24.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/38' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/11' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/16' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/32' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/30' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/14' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/18' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/13' 2023-06-09T11:16:24.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/33' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/17' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/35' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './115/37' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './115' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/19' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/15' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/31' 2023-06-09T11:16:24.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/36' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/20' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/12' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/39' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/10' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/34' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/38' 2023-06-09T11:16:24.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/11' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/16' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/32' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/30' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/14' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/18' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/13' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/33' 2023-06-09T11:16:24.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/17' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/35' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './116/37' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './116' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/19' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/15' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/31' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/36' 2023-06-09T11:16:24.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/20' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/12' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/39' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/10' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/34' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/38' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/11' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/16' 2023-06-09T11:16:24.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/32' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/30' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/14' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/18' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/13' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/33' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/17' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/35' 2023-06-09T11:16:24.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './117/37' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './117' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/19' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/15' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/31' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/36' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/20' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/12' 2023-06-09T11:16:24.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/39' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/10' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/34' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/38' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/11' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/16' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/32' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/30' 2023-06-09T11:16:24.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/14' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/18' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/13' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/33' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/17' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/35' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './118/37' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './118' 2023-06-09T11:16:24.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/19' 2023-06-09T11:16:24.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/15' 2023-06-09T11:16:24.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/31' 2023-06-09T11:16:24.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/36' 2023-06-09T11:16:24.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/20' 2023-06-09T11:16:24.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/12' 2023-06-09T11:16:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/39' 2023-06-09T11:16:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/10' 2023-06-09T11:16:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/34' 2023-06-09T11:16:25.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/38' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/11' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/16' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/32' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/30' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/14' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/18' 2023-06-09T11:16:25.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/13' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/33' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/17' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/35' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './119/37' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './119' 2023-06-09T11:16:25.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/19' 2023-06-09T11:16:25.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/15' 2023-06-09T11:16:25.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/31' 2023-06-09T11:16:25.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/36' 2023-06-09T11:16:25.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/20' 2023-06-09T11:16:25.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/12' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/39' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/10' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/34' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/38' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/11' 2023-06-09T11:16:25.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/16' 2023-06-09T11:16:25.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/32' 2023-06-09T11:16:25.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/30' 2023-06-09T11:16:25.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/14' 2023-06-09T11:16:25.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/18' 2023-06-09T11:16:25.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/13' 2023-06-09T11:16:25.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/33' 2023-06-09T11:16:25.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/17' 2023-06-09T11:16:25.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/35' 2023-06-09T11:16:25.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './120/37' 2023-06-09T11:16:25.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './120' 2023-06-09T11:16:25.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/19' 2023-06-09T11:16:25.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/15' 2023-06-09T11:16:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/31' 2023-06-09T11:16:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/36' 2023-06-09T11:16:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/20' 2023-06-09T11:16:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/12' 2023-06-09T11:16:25.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/39' 2023-06-09T11:16:25.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/10' 2023-06-09T11:16:25.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/34' 2023-06-09T11:16:25.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/38' 2023-06-09T11:16:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/11' 2023-06-09T11:16:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/16' 2023-06-09T11:16:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/32' 2023-06-09T11:16:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/30' 2023-06-09T11:16:25.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/14' 2023-06-09T11:16:25.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/18' 2023-06-09T11:16:25.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/13' 2023-06-09T11:16:25.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/33' 2023-06-09T11:16:25.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/17' 2023-06-09T11:16:25.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/35' 2023-06-09T11:16:25.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './121/37' 2023-06-09T11:16:25.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './121' 2023-06-09T11:16:25.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/19' 2023-06-09T11:16:25.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/15' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/31' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/36' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/20' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/12' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/39' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/10' 2023-06-09T11:16:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/34' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/38' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/11' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/16' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/32' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/30' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/14' 2023-06-09T11:16:25.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/18' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/13' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/33' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/17' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/35' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './122/37' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './122' 2023-06-09T11:16:25.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/19' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/15' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/31' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/36' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/20' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/12' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/39' 2023-06-09T11:16:25.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/10' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/34' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/38' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/11' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/16' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/32' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/30' 2023-06-09T11:16:25.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/14' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/18' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/13' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/33' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/17' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/35' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './123/37' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './123' 2023-06-09T11:16:25.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/19' 2023-06-09T11:16:25.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/15' 2023-06-09T11:16:25.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/31' 2023-06-09T11:16:25.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/36' 2023-06-09T11:16:25.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/20' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/12' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/39' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/10' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/34' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/38' 2023-06-09T11:16:25.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/11' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/16' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/32' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/30' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/14' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/18' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/13' 2023-06-09T11:16:25.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/33' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/17' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/35' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './124/37' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './124' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/19' 2023-06-09T11:16:25.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/15' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/31' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/36' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/20' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/12' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/39' 2023-06-09T11:16:25.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/10' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/34' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/38' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/11' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/16' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/32' 2023-06-09T11:16:25.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/30' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/14' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/18' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/13' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/33' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/17' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/35' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './125/37' 2023-06-09T11:16:25.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './125' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/19' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/15' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/31' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/36' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/20' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/12' 2023-06-09T11:16:25.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/39' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/10' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/34' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/38' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/11' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/16' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/32' 2023-06-09T11:16:25.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/30' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/14' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/18' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/13' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/33' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/17' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/35' 2023-06-09T11:16:25.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './126/37' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './126' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/19' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/15' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/31' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/36' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/20' 2023-06-09T11:16:25.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/12' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/39' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/10' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/34' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/38' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/11' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/16' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/32' 2023-06-09T11:16:25.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/30' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/14' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/18' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/13' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/33' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/17' 2023-06-09T11:16:25.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/35' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './127/37' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './127' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/19' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/15' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/31' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/36' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/20' 2023-06-09T11:16:25.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/12' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/39' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/10' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/34' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/38' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/11' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/16' 2023-06-09T11:16:25.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/32' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/30' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/14' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/18' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/13' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/33' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/17' 2023-06-09T11:16:25.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/35' 2023-06-09T11:16:28.643 INFO:tasks.workunit.client.0.smithi007.stdout:removed './128/37' 2023-06-09T11:16:28.643 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './128' 2023-06-09T11:16:28.643 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/19' 2023-06-09T11:16:28.643 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/15' 2023-06-09T11:16:28.643 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/31' 2023-06-09T11:16:28.644 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/36' 2023-06-09T11:16:28.644 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/20' 2023-06-09T11:16:28.644 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/12' 2023-06-09T11:16:28.644 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/39' 2023-06-09T11:16:28.644 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/10' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/34' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/38' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/11' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/16' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/32' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/30' 2023-06-09T11:16:28.645 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/14' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/18' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/13' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/33' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/17' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/35' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './129/37' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './129' 2023-06-09T11:16:28.646 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/19' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/15' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/31' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/36' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/20' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/12' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/39' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/10' 2023-06-09T11:16:28.647 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/34' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/38' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/11' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/16' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/32' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/30' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/14' 2023-06-09T11:16:28.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/18' 2023-06-09T11:16:28.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/13' 2023-06-09T11:16:28.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/33' 2023-06-09T11:16:28.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/17' 2023-06-09T11:16:28.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/35' 2023-06-09T11:16:28.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './130/37' 2023-06-09T11:16:28.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './130' 2023-06-09T11:16:28.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/19' 2023-06-09T11:16:28.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/15' 2023-06-09T11:16:28.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/31' 2023-06-09T11:16:28.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/36' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/20' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/12' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/39' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/10' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/34' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/38' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/11' 2023-06-09T11:16:28.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/16' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/32' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/30' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/14' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/18' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/13' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/33' 2023-06-09T11:16:28.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/17' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/35' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './131/37' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './131' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/19' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/15' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/31' 2023-06-09T11:16:28.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/36' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/20' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/12' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/39' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/10' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/34' 2023-06-09T11:16:28.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/38' 2023-06-09T11:16:28.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/11' 2023-06-09T11:16:28.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/16' 2023-06-09T11:16:28.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/32' 2023-06-09T11:16:28.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/30' 2023-06-09T11:16:28.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/14' 2023-06-09T11:16:28.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/18' 2023-06-09T11:16:28.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/13' 2023-06-09T11:16:28.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/33' 2023-06-09T11:16:28.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/17' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/35' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './132/37' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './132' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/19' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/15' 2023-06-09T11:16:28.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/31' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/36' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/20' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/12' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/39' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/10' 2023-06-09T11:16:28.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/34' 2023-06-09T11:16:28.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/38' 2023-06-09T11:16:28.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/11' 2023-06-09T11:16:28.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/16' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/32' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/30' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/14' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/18' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/13' 2023-06-09T11:16:28.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/33' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/17' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/35' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './133/37' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './133' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/19' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/15' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/31' 2023-06-09T11:16:28.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/36' 2023-06-09T11:16:28.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/20' 2023-06-09T11:16:28.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/12' 2023-06-09T11:16:28.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/39' 2023-06-09T11:16:28.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/10' 2023-06-09T11:16:28.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/34' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/38' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/11' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/16' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/32' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/30' 2023-06-09T11:16:28.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/14' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/18' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/13' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/33' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/17' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/35' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './134/37' 2023-06-09T11:16:28.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './134' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/19' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/15' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/31' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/36' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/20' 2023-06-09T11:16:28.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/12' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/39' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/10' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/34' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/38' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/11' 2023-06-09T11:16:28.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/16' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/32' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/30' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/14' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/18' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/13' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/33' 2023-06-09T11:16:28.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/17' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/35' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './135/37' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './135' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/19' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/15' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/31' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/36' 2023-06-09T11:16:28.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/20' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/12' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/39' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/10' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/34' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/38' 2023-06-09T11:16:28.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/11' 2023-06-09T11:16:28.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/16' 2023-06-09T11:16:28.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/32' 2023-06-09T11:16:28.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/30' 2023-06-09T11:16:28.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/14' 2023-06-09T11:16:28.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/18' 2023-06-09T11:16:28.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/13' 2023-06-09T11:16:28.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/33' 2023-06-09T11:16:28.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/17' 2023-06-09T11:16:28.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/35' 2023-06-09T11:16:28.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './136/37' 2023-06-09T11:16:28.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './136' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/19' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/15' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/31' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/36' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/20' 2023-06-09T11:16:28.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/12' 2023-06-09T11:16:28.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/39' 2023-06-09T11:16:28.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/10' 2023-06-09T11:16:28.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/34' 2023-06-09T11:16:28.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/38' 2023-06-09T11:16:28.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/11' 2023-06-09T11:16:28.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/16' 2023-06-09T11:16:28.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/32' 2023-06-09T11:16:28.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/30' 2023-06-09T11:16:28.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/14' 2023-06-09T11:16:28.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/18' 2023-06-09T11:16:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/13' 2023-06-09T11:16:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/33' 2023-06-09T11:16:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/17' 2023-06-09T11:16:28.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/35' 2023-06-09T11:16:28.678 INFO:tasks.workunit.client.0.smithi007.stdout:removed './137/37' 2023-06-09T11:16:28.678 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './137' 2023-06-09T11:16:28.678 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/19' 2023-06-09T11:16:28.679 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/15' 2023-06-09T11:16:28.679 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/31' 2023-06-09T11:16:28.679 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/36' 2023-06-09T11:16:28.680 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/20' 2023-06-09T11:16:28.680 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/12' 2023-06-09T11:16:28.680 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/39' 2023-06-09T11:16:28.680 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/10' 2023-06-09T11:16:28.681 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/34' 2023-06-09T11:16:28.681 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/38' 2023-06-09T11:16:28.681 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/11' 2023-06-09T11:16:28.681 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/16' 2023-06-09T11:16:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/32' 2023-06-09T11:16:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/30' 2023-06-09T11:16:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/14' 2023-06-09T11:16:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/18' 2023-06-09T11:16:29.722 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/13' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/33' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/17' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/35' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './138/37' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './138' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/19' 2023-06-09T11:16:29.723 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/15' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/31' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/36' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/20' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/12' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/39' 2023-06-09T11:16:29.724 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/10' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/34' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/38' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/11' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/16' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/32' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/30' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/14' 2023-06-09T11:16:29.725 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/18' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/13' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/33' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/17' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/35' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './139/37' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './139' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/19' 2023-06-09T11:16:29.726 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/15' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/31' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/36' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/20' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/12' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/39' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/10' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/34' 2023-06-09T11:16:29.727 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/38' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/11' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/16' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/32' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/30' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/14' 2023-06-09T11:16:29.728 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/18' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/13' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/33' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/17' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/35' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed './140/37' 2023-06-09T11:16:29.729 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './140' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/19' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/15' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/31' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/36' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/20' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/12' 2023-06-09T11:16:29.730 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/39' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/10' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/34' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/38' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/11' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/16' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/32' 2023-06-09T11:16:29.731 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/30' 2023-06-09T11:16:29.732 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/14' 2023-06-09T11:16:29.732 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/18' 2023-06-09T11:16:29.732 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/13' 2023-06-09T11:16:29.732 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/33' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/17' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/35' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './141/37' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './141' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/19' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/15' 2023-06-09T11:16:29.733 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/31' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/36' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/20' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/12' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/39' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/10' 2023-06-09T11:16:29.734 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/34' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/38' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/11' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/16' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/32' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/30' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/14' 2023-06-09T11:16:29.735 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/18' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/13' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/33' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/17' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/35' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './142/37' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './142' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/19' 2023-06-09T11:16:29.736 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/15' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/31' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/36' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/20' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/12' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/39' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/10' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/34' 2023-06-09T11:16:29.737 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/38' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/11' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/16' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/32' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/30' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/14' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/18' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/13' 2023-06-09T11:16:29.738 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/33' 2023-06-09T11:16:29.739 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/17' 2023-06-09T11:16:29.739 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/35' 2023-06-09T11:16:29.739 INFO:tasks.workunit.client.0.smithi007.stdout:removed './143/37' 2023-06-09T11:16:29.739 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './143' 2023-06-09T11:16:29.739 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/19' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/15' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/31' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/36' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/20' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/12' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/39' 2023-06-09T11:16:29.740 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/10' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/34' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/38' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/11' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/16' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/32' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/30' 2023-06-09T11:16:29.741 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/14' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/18' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/13' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/33' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/17' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/35' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed './144/37' 2023-06-09T11:16:29.742 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './144' 2023-06-09T11:16:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/19' 2023-06-09T11:16:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/15' 2023-06-09T11:16:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/31' 2023-06-09T11:16:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/36' 2023-06-09T11:16:29.743 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/20' 2023-06-09T11:16:29.744 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/12' 2023-06-09T11:16:29.744 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/39' 2023-06-09T11:16:29.744 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/10' 2023-06-09T11:16:29.744 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/34' 2023-06-09T11:16:29.744 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/38' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/11' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/16' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/32' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/30' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/14' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/18' 2023-06-09T11:16:29.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/13' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/33' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/17' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/35' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './145/37' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './145' 2023-06-09T11:16:29.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/19' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/15' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/31' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/36' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/20' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/12' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/39' 2023-06-09T11:16:29.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/10' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/34' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/38' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/11' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/16' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/32' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/30' 2023-06-09T11:16:29.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/14' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/18' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/13' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/33' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/17' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/35' 2023-06-09T11:16:29.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './146/37' 2023-06-09T11:16:29.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './146' 2023-06-09T11:16:29.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/19' 2023-06-09T11:16:29.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/15' 2023-06-09T11:16:29.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/31' 2023-06-09T11:16:29.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/36' 2023-06-09T11:16:29.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/20' 2023-06-09T11:16:29.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/12' 2023-06-09T11:16:29.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/39' 2023-06-09T11:16:29.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/10' 2023-06-09T11:16:29.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/34' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/38' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/11' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/16' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/32' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/30' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/14' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/18' 2023-06-09T11:16:29.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/13' 2023-06-09T11:16:29.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/33' 2023-06-09T11:16:29.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/17' 2023-06-09T11:16:29.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/35' 2023-06-09T11:16:29.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './147/37' 2023-06-09T11:16:29.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './147' 2023-06-09T11:16:29.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/19' 2023-06-09T11:16:29.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/15' 2023-06-09T11:16:29.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/31' 2023-06-09T11:16:29.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/36' 2023-06-09T11:16:30.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/20' 2023-06-09T11:16:30.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/12' 2023-06-09T11:16:30.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/39' 2023-06-09T11:16:30.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/10' 2023-06-09T11:16:30.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/34' 2023-06-09T11:16:30.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/38' 2023-06-09T11:16:30.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/11' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/16' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/32' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/30' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/14' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/18' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/13' 2023-06-09T11:16:30.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/33' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/17' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/35' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './148/37' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './148' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/19' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/15' 2023-06-09T11:16:30.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/31' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/36' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/20' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/12' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/39' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/10' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/34' 2023-06-09T11:16:30.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/38' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/11' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/16' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/32' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/30' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/14' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/18' 2023-06-09T11:16:30.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/13' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/33' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/17' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/35' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './149/37' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './149' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/19' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/15' 2023-06-09T11:16:30.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/31' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/36' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/20' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/12' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/39' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/10' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/34' 2023-06-09T11:16:30.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/38' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/11' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/16' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/32' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/30' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/14' 2023-06-09T11:16:30.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/18' 2023-06-09T11:16:30.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/13' 2023-06-09T11:16:30.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/33' 2023-06-09T11:16:30.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/17' 2023-06-09T11:16:30.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/35' 2023-06-09T11:16:30.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './150/37' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './150' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/19' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/15' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/31' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/36' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/20' 2023-06-09T11:16:30.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/12' 2023-06-09T11:16:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/39' 2023-06-09T11:16:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/10' 2023-06-09T11:16:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/34' 2023-06-09T11:16:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/38' 2023-06-09T11:16:30.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/11' 2023-06-09T11:16:30.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/16' 2023-06-09T11:16:30.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/32' 2023-06-09T11:16:30.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/30' 2023-06-09T11:16:30.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/14' 2023-06-09T11:16:30.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/18' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/13' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/33' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/17' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/35' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './151/37' 2023-06-09T11:16:30.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './151' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/19' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/15' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/31' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/36' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/20' 2023-06-09T11:16:30.925 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/12' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/39' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/10' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/34' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/38' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/11' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/16' 2023-06-09T11:16:30.926 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/32' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/30' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/14' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/18' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/13' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/33' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/17' 2023-06-09T11:16:30.927 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/35' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed './152/37' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './152' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/19' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/15' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/31' 2023-06-09T11:16:30.928 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/36' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/20' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/12' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/39' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/10' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/34' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/38' 2023-06-09T11:16:30.929 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/11' 2023-06-09T11:16:30.930 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/16' 2023-06-09T11:16:30.930 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/32' 2023-06-09T11:16:30.930 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/30' 2023-06-09T11:16:30.930 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/14' 2023-06-09T11:16:30.930 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/18' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/13' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/33' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/17' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/35' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './153/37' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './153' 2023-06-09T11:16:30.931 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/19' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/15' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/31' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/36' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/20' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/12' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/39' 2023-06-09T11:16:30.932 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/10' 2023-06-09T11:16:30.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/34' 2023-06-09T11:16:30.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/38' 2023-06-09T11:16:30.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/11' 2023-06-09T11:16:30.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/16' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/32' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/30' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/14' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/18' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/13' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/33' 2023-06-09T11:16:30.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/17' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/35' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './154/37' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './154' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/19' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/15' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/31' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/36' 2023-06-09T11:16:30.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/20' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/12' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/39' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/10' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/34' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/38' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/11' 2023-06-09T11:16:30.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/16' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/32' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/30' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/14' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/18' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/13' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/33' 2023-06-09T11:16:30.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/17' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/35' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './155/37' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './155' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/19' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/15' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/31' 2023-06-09T11:16:30.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/36' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/20' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/12' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/39' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/10' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/34' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/38' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/11' 2023-06-09T11:16:30.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/16' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/32' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/30' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/14' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/18' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/13' 2023-06-09T11:16:30.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/33' 2023-06-09T11:16:30.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/17' 2023-06-09T11:16:30.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/35' 2023-06-09T11:16:30.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './156/37' 2023-06-09T11:16:30.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './156' 2023-06-09T11:16:30.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/19' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/15' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/31' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/36' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/20' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/12' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/39' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/10' 2023-06-09T11:16:30.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/34' 2023-06-09T11:16:30.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/38' 2023-06-09T11:16:30.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/11' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/16' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/32' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/30' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/14' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/18' 2023-06-09T11:16:30.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/13' 2023-06-09T11:16:30.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/33' 2023-06-09T11:16:32.648 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/17' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/35' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './157/37' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './157' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/19' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/15' 2023-06-09T11:16:32.649 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/31' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/36' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/20' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/12' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/39' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/10' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/34' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/38' 2023-06-09T11:16:32.650 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/11' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/16' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/32' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/30' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/14' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/18' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/13' 2023-06-09T11:16:32.651 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/33' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/17' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/35' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './158/37' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './158' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/19' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/15' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/31' 2023-06-09T11:16:32.652 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/36' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/20' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/12' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/39' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/10' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/34' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/38' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/11' 2023-06-09T11:16:32.653 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/16' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/32' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/30' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/14' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/18' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/13' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/33' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/17' 2023-06-09T11:16:32.654 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/35' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './159/37' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './159' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/19' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/15' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/31' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/36' 2023-06-09T11:16:32.655 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/20' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/12' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/39' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/10' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/34' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/38' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/11' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/16' 2023-06-09T11:16:32.656 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/32' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/30' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/14' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/18' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/13' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/33' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/17' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/35' 2023-06-09T11:16:32.657 INFO:tasks.workunit.client.0.smithi007.stdout:removed './160/37' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './160' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/19' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/15' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/31' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/36' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/20' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/12' 2023-06-09T11:16:32.658 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/39' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/10' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/34' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/38' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/11' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/16' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/32' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/30' 2023-06-09T11:16:32.659 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/14' 2023-06-09T11:16:32.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/18' 2023-06-09T11:16:32.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/13' 2023-06-09T11:16:32.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/33' 2023-06-09T11:16:32.660 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/17' 2023-06-09T11:16:32.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/35' 2023-06-09T11:16:32.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './161/37' 2023-06-09T11:16:32.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './161' 2023-06-09T11:16:32.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/19' 2023-06-09T11:16:32.661 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/15' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/31' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/36' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/20' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/12' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/39' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/10' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/34' 2023-06-09T11:16:32.662 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/38' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/11' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/16' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/32' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/30' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/14' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/18' 2023-06-09T11:16:32.663 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/13' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/33' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/17' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/35' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './162/37' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './162' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/19' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/15' 2023-06-09T11:16:32.664 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/31' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/36' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/20' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/12' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/39' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/10' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/34' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/38' 2023-06-09T11:16:32.665 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/11' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/16' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/32' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/30' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/14' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/18' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/13' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/33' 2023-06-09T11:16:32.666 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/17' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/35' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './163/37' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './163' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/19' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/15' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/31' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/36' 2023-06-09T11:16:32.667 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/20' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/12' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/39' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/10' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/34' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/38' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/11' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/16' 2023-06-09T11:16:32.668 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/32' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/30' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/14' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/18' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/13' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/33' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/17' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/35' 2023-06-09T11:16:32.669 INFO:tasks.workunit.client.0.smithi007.stdout:removed './164/37' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './164' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/19' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/15' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/31' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/36' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/20' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/12' 2023-06-09T11:16:32.670 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/39' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/10' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/34' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/38' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/11' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/16' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/32' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/30' 2023-06-09T11:16:32.671 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/14' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/18' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/13' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/33' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/17' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/35' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed './165/37' 2023-06-09T11:16:32.672 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './165' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/19' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/15' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/31' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/36' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/20' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/12' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/39' 2023-06-09T11:16:32.673 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/10' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/34' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/38' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/11' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/16' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/32' 2023-06-09T11:16:32.674 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/30' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/14' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/18' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/13' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/33' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/17' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/35' 2023-06-09T11:16:32.675 INFO:tasks.workunit.client.0.smithi007.stdout:removed './166/37' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './166' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/19' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/15' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/31' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/36' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/20' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/12' 2023-06-09T11:16:32.676 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/39' 2023-06-09T11:16:32.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/10' 2023-06-09T11:16:32.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/34' 2023-06-09T11:16:32.677 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/38' 2023-06-09T11:16:33.336 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/11' 2023-06-09T11:16:33.336 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/16' 2023-06-09T11:16:33.336 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/32' 2023-06-09T11:16:33.336 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/30' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/14' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/18' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/13' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/33' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/17' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/35' 2023-06-09T11:16:33.337 INFO:tasks.workunit.client.0.smithi007.stdout:removed './167/37' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './167' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/19' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/15' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/31' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/36' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/20' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/12' 2023-06-09T11:16:33.338 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/39' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/10' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/34' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/38' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/11' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/16' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/32' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/30' 2023-06-09T11:16:33.339 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/14' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/18' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/13' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/33' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/17' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/35' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './168/37' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './168' 2023-06-09T11:16:33.340 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/19' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/15' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/31' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/36' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/20' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/12' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/39' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/10' 2023-06-09T11:16:33.341 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/34' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/38' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/11' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/16' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/32' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/30' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/14' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/18' 2023-06-09T11:16:33.342 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/13' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/33' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/17' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/35' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './169/37' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './169' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/19' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/15' 2023-06-09T11:16:33.343 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/31' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/36' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/20' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/12' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/39' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/10' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/34' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/38' 2023-06-09T11:16:33.344 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/11' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/16' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/32' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/30' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/14' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/18' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/13' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/33' 2023-06-09T11:16:33.345 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/17' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/35' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './170/37' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './170' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/19' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/15' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/31' 2023-06-09T11:16:33.346 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/36' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/20' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/12' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/39' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/10' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/34' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/38' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/11' 2023-06-09T11:16:33.347 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/16' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/32' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/30' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/14' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/18' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/13' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/33' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/17' 2023-06-09T11:16:33.348 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/35' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './171/37' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './171' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/19' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/15' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/31' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/36' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/20' 2023-06-09T11:16:33.349 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/12' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/39' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/10' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/34' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/38' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/11' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/16' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/32' 2023-06-09T11:16:33.350 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/30' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/14' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/18' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/13' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/33' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/17' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/35' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed './172/37' 2023-06-09T11:16:33.351 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './172' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/19' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/15' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/31' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/36' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/20' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/12' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/39' 2023-06-09T11:16:33.352 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/10' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/34' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/38' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/11' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/16' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/32' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/30' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/14' 2023-06-09T11:16:33.353 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/18' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/13' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/33' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/17' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/35' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './173/37' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './173' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/19' 2023-06-09T11:16:33.354 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/15' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/31' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/36' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/20' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/12' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/39' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/10' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/34' 2023-06-09T11:16:33.355 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/38' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/11' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/16' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/32' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/30' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/14' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/18' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/13' 2023-06-09T11:16:33.356 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/33' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/17' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/35' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './174/37' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './174' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/19' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/15' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/31' 2023-06-09T11:16:33.357 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/36' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/20' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/12' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/39' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/10' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/34' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/38' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/11' 2023-06-09T11:16:33.358 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/16' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/32' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/30' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/14' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/18' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/13' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/33' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/17' 2023-06-09T11:16:33.359 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/35' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './175/37' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './175' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/19' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/15' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/31' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/36' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/20' 2023-06-09T11:16:33.360 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/12' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/39' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/10' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/34' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/38' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/11' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/16' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/32' 2023-06-09T11:16:33.361 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/30' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/14' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/18' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/13' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/33' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/17' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/35' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './176/37' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './176' 2023-06-09T11:16:33.362 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/19' 2023-06-09T11:16:33.363 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/15' 2023-06-09T11:16:35.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/31' 2023-06-09T11:16:35.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/36' 2023-06-09T11:16:35.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/20' 2023-06-09T11:16:35.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/12' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/39' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/10' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/34' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/38' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/11' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/16' 2023-06-09T11:16:35.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/32' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/30' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/14' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/18' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/13' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/33' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/17' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/35' 2023-06-09T11:16:35.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './177/37' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './177' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/19' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/15' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/31' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/36' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/20' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/12' 2023-06-09T11:16:35.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/39' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/10' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/34' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/38' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/11' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/16' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/32' 2023-06-09T11:16:35.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/30' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/14' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/18' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/13' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/33' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/17' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/35' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed './178/37' 2023-06-09T11:16:35.121 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './178' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/19' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/15' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/31' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/36' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/20' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/12' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/39' 2023-06-09T11:16:35.122 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/10' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/34' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/38' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/11' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/16' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/32' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/30' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/14' 2023-06-09T11:16:35.123 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/18' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/13' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/33' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/17' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/35' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './179/37' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './179' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/19' 2023-06-09T11:16:35.124 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/15' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/31' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/36' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/20' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/12' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/39' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/10' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/34' 2023-06-09T11:16:35.125 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/38' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/11' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/16' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/32' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/30' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/14' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/18' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/13' 2023-06-09T11:16:35.126 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/33' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/17' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/35' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './180/37' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './180' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/19' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/15' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/31' 2023-06-09T11:16:35.127 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/36' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/20' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/12' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/39' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/10' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/34' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/38' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/11' 2023-06-09T11:16:35.128 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/16' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/32' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/30' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/14' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/18' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/13' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/33' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/17' 2023-06-09T11:16:35.129 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/35' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './181/37' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './181' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/19' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/15' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/31' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/36' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/20' 2023-06-09T11:16:35.130 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/12' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/39' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/10' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/34' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/38' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/11' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/16' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/32' 2023-06-09T11:16:35.131 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/30' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/14' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/18' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/13' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/33' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/17' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/35' 2023-06-09T11:16:35.132 INFO:tasks.workunit.client.0.smithi007.stdout:removed './182/37' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './182' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/19' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/15' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/31' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/36' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/20' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/12' 2023-06-09T11:16:35.133 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/39' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/10' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/34' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/38' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/11' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/16' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/32' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/30' 2023-06-09T11:16:35.134 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/14' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/18' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/13' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/33' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/17' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/35' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './183/37' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './183' 2023-06-09T11:16:35.135 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/19' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/15' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/31' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/36' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/20' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/12' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/39' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/10' 2023-06-09T11:16:35.136 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/34' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/38' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/11' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/16' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/32' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/30' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/14' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/18' 2023-06-09T11:16:35.137 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/13' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/33' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/17' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/35' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './184/37' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './184' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/19' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/15' 2023-06-09T11:16:35.138 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/31' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/36' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/20' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/12' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/39' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/10' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/34' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/38' 2023-06-09T11:16:35.139 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/11' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/16' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/32' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/30' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/14' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/18' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/13' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/33' 2023-06-09T11:16:35.140 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/17' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/35' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './185/37' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './185' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/19' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/15' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/31' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/36' 2023-06-09T11:16:35.141 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/20' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/12' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/39' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/10' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/34' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/38' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/11' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/16' 2023-06-09T11:16:35.142 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/32' 2023-06-09T11:16:35.143 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/30' 2023-06-09T11:16:35.143 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/14' 2023-06-09T11:16:35.143 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/18' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/13' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/33' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/17' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/35' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed './186/37' 2023-06-09T11:16:35.933 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './186' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/19' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/15' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/31' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/36' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/20' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/12' 2023-06-09T11:16:35.934 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/39' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/10' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/34' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/38' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/11' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/16' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/32' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/30' 2023-06-09T11:16:35.935 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/14' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/18' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/13' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/33' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/17' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/35' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './187/37' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './187' 2023-06-09T11:16:35.936 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/19' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/15' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/31' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/36' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/20' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/12' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/39' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/10' 2023-06-09T11:16:35.937 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/34' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/38' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/11' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/16' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/32' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/30' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/14' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/18' 2023-06-09T11:16:35.938 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/13' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/33' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/17' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/35' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './188/37' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './188' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/19' 2023-06-09T11:16:35.939 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/15' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/31' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/36' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/20' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/12' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/39' 2023-06-09T11:16:35.940 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/10' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/34' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/38' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/11' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/16' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/32' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/30' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/14' 2023-06-09T11:16:35.941 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/18' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/13' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/33' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/17' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/35' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './189/37' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './189' 2023-06-09T11:16:35.942 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/19' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/15' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/31' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/36' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/20' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/12' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/39' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/10' 2023-06-09T11:16:35.943 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/34' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/38' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/11' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/16' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/32' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/30' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/14' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/18' 2023-06-09T11:16:35.944 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/13' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/33' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/17' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/35' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './190/37' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './190' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/19' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/15' 2023-06-09T11:16:35.945 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/31' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/36' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/20' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/12' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/39' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/10' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/34' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/38' 2023-06-09T11:16:35.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/11' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/16' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/32' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/30' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/14' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/18' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/13' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/33' 2023-06-09T11:16:35.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/17' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/35' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './191/37' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './191' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/19' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/15' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/31' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/36' 2023-06-09T11:16:35.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/20' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/12' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/39' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/10' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/34' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/38' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/11' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/16' 2023-06-09T11:16:35.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/32' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/30' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/14' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/18' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/13' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/33' 2023-06-09T11:16:35.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/17' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/35' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './192/37' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './192' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/19' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/15' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/31' 2023-06-09T11:16:35.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/36' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/20' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/12' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/39' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/10' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/34' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/38' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/11' 2023-06-09T11:16:35.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/16' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/32' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/30' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/14' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/18' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/13' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/33' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/17' 2023-06-09T11:16:35.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/35' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './193/37' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './193' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/19' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/15' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/31' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/36' 2023-06-09T11:16:35.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/20' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/12' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/39' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/10' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/34' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/38' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/11' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/16' 2023-06-09T11:16:35.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/32' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/30' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/14' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/18' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/13' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/33' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/17' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/35' 2023-06-09T11:16:35.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './194/37' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './194' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/19' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/15' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/31' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/36' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/20' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/12' 2023-06-09T11:16:35.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/39' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/10' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/34' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/38' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/11' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/16' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/32' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/30' 2023-06-09T11:16:35.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/14' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/18' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/13' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/33' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/17' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/35' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './195/37' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './195' 2023-06-09T11:16:35.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/19' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/15' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/31' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/36' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/20' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/12' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/39' 2023-06-09T11:16:35.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/10' 2023-06-09T11:16:41.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:41.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:16:43.261 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/34' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/38' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/11' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/16' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/32' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/30' 2023-06-09T11:16:43.262 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/14' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/18' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/13' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/33' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/17' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/35' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed './196/37' 2023-06-09T11:16:43.263 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './196' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/19' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/15' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/31' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/36' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/20' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/12' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/39' 2023-06-09T11:16:43.264 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/10' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/34' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/38' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/11' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/16' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/32' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/30' 2023-06-09T11:16:43.265 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/14' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/18' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/13' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/33' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/17' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/35' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './197/37' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './197' 2023-06-09T11:16:43.266 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/19' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/15' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/31' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/36' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/20' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/12' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/39' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/10' 2023-06-09T11:16:43.267 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/34' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/38' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/11' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/16' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/32' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/30' 2023-06-09T11:16:43.268 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/14' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/18' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/13' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/33' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/17' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/35' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './198/37' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './198' 2023-06-09T11:16:43.269 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/19' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/15' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/31' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/36' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/20' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/12' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/39' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/10' 2023-06-09T11:16:43.270 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/34' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/38' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/11' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/16' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/32' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/30' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/14' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/18' 2023-06-09T11:16:43.271 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/13' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/33' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/17' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/35' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './199/37' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './199' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/21' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/27' 2023-06-09T11:16:43.272 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/31' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/36' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/20' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/39' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/34' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/38' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/28' 2023-06-09T11:16:43.273 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/24' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/29' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/32' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/22' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/25' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/30' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/33' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/23' 2023-06-09T11:16:43.274 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/35' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/26' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './200/37' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './200' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/21' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/27' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/31' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/36' 2023-06-09T11:16:43.275 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/20' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/39' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/34' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/38' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/28' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/24' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/29' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/32' 2023-06-09T11:16:43.276 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/22' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/25' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/30' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/33' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/23' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/35' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/26' 2023-06-09T11:16:43.277 INFO:tasks.workunit.client.0.smithi007.stdout:removed './201/37' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './201' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/21' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/27' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/31' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/36' 2023-06-09T11:16:43.278 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/20' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/39' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/34' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/38' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/28' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/24' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/29' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/32' 2023-06-09T11:16:43.279 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/22' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/25' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/30' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/33' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/23' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/35' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/26' 2023-06-09T11:16:43.280 INFO:tasks.workunit.client.0.smithi007.stdout:removed './202/37' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './202' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/21' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/27' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/31' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/36' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/20' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/39' 2023-06-09T11:16:43.281 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/34' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/38' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/28' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/24' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/29' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/32' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/22' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/25' 2023-06-09T11:16:43.282 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/30' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/33' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/23' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/35' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/26' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './203/37' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './203' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/21' 2023-06-09T11:16:43.283 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/27' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/31' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/36' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/20' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/39' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/34' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/38' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/28' 2023-06-09T11:16:43.284 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/24' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/29' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/32' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/22' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/25' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/30' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/33' 2023-06-09T11:16:43.285 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/23' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/35' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/26' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './204/37' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './204' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/21' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/27' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/31' 2023-06-09T11:16:43.286 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/36' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/20' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/39' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/34' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/38' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/28' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/24' 2023-06-09T11:16:43.287 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/29' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/32' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/22' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/25' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/30' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/33' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/23' 2023-06-09T11:16:43.288 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/35' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/26' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './205/37' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './205' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/21' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/27' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/31' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/36' 2023-06-09T11:16:43.289 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/20' 2023-06-09T11:16:43.290 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/39' 2023-06-09T11:16:46.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/34' 2023-06-09T11:16:46.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/38' 2023-06-09T11:16:46.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/28' 2023-06-09T11:16:46.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/24' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/29' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/32' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/22' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/25' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/30' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/33' 2023-06-09T11:16:46.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/23' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/35' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/26' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './206/37' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './206' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/21' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/27' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/31' 2023-06-09T11:16:46.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/36' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/20' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/39' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/34' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/38' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/28' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/24' 2023-06-09T11:16:46.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/29' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/32' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/22' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/25' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/30' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/33' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/23' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/35' 2023-06-09T11:16:46.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/26' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './207/37' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './207' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/21' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/27' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/31' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/36' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/20' 2023-06-09T11:16:46.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/39' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/34' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/38' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/28' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/24' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/29' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/32' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/22' 2023-06-09T11:16:46.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/25' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/30' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/33' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/23' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/35' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/26' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './208/37' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './208' 2023-06-09T11:16:46.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/21' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/27' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/31' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/36' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/20' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/39' 2023-06-09T11:16:46.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/34' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/38' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/28' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/24' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/29' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/32' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/22' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/25' 2023-06-09T11:16:46.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/30' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/33' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/23' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/35' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/26' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './209/37' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './209' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/21' 2023-06-09T11:16:46.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/27' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/31' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/36' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/20' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/39' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/34' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/38' 2023-06-09T11:16:46.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/28' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/24' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/29' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/32' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/22' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/25' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/30' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/33' 2023-06-09T11:16:46.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/23' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/35' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/26' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './210/37' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './210' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/21' 2023-06-09T11:16:46.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/27' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/31' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/36' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/20' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/39' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/34' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/38' 2023-06-09T11:16:46.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/28' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/24' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/29' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/32' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/22' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/25' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/30' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/33' 2023-06-09T11:16:46.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/23' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/35' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/26' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './211/37' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './211' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/21' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/27' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/31' 2023-06-09T11:16:46.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/36' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/20' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/39' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/34' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/38' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/28' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/24' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/29' 2023-06-09T11:16:46.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/32' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/22' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/25' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/30' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/33' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/23' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/35' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/26' 2023-06-09T11:16:46.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './212/37' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './212' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/21' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/27' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/31' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/36' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/20' 2023-06-09T11:16:46.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/39' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/34' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/38' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/28' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/24' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/29' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/32' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/22' 2023-06-09T11:16:46.485 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/25' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/30' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/33' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/23' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/35' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/26' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './213/37' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './213' 2023-06-09T11:16:46.486 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/21' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/27' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/31' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/36' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/20' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/39' 2023-06-09T11:16:46.487 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/34' 2023-06-09T11:16:46.488 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/38' 2023-06-09T11:16:46.488 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/28' 2023-06-09T11:16:46.488 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/24' 2023-06-09T11:16:46.488 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/29' 2023-06-09T11:16:46.488 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/32' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/22' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/25' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/30' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/33' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/23' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/35' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/26' 2023-06-09T11:16:46.489 INFO:tasks.workunit.client.0.smithi007.stdout:removed './214/37' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './214' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/21' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/27' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/31' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/36' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/20' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/39' 2023-06-09T11:16:46.490 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/34' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/38' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/28' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/24' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/29' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/32' 2023-06-09T11:16:46.491 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/22' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/25' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/30' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/33' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/23' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/35' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/26' 2023-06-09T11:16:46.492 INFO:tasks.workunit.client.0.smithi007.stdout:removed './215/37' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './215' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/21' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/27' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/31' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/36' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/20' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/39' 2023-06-09T11:16:46.493 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/34' 2023-06-09T11:16:46.494 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/38' 2023-06-09T11:16:48.183 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/28' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/24' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/29' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/32' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/22' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/25' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/30' 2023-06-09T11:16:48.184 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/33' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/23' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/35' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/26' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './216/37' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './216' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/21' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/27' 2023-06-09T11:16:48.185 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/31' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/36' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/20' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/39' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/34' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/38' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/28' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/24' 2023-06-09T11:16:48.186 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/29' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/32' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/22' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/25' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/30' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/33' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/23' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/35' 2023-06-09T11:16:48.187 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/26' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './217/37' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './217' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/21' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/27' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/31' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/36' 2023-06-09T11:16:48.188 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/20' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/39' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/34' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/38' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/28' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/24' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/29' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/32' 2023-06-09T11:16:48.189 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/22' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/25' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/30' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/33' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/23' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/35' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/26' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed './218/37' 2023-06-09T11:16:48.190 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './218' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/21' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/27' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/31' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/36' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/20' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/39' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/34' 2023-06-09T11:16:48.191 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/38' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/28' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/24' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/29' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/32' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/22' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/25' 2023-06-09T11:16:48.192 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/30' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/33' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/23' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/35' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/26' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './219/37' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './219' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/21' 2023-06-09T11:16:48.193 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/27' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/31' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/36' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/20' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/39' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/34' 2023-06-09T11:16:48.194 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/38' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/28' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/24' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/29' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/32' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/22' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/25' 2023-06-09T11:16:48.195 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/30' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/33' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/23' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/35' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/26' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './220/37' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './220' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/21' 2023-06-09T11:16:48.196 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/27' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/31' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/36' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/20' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/39' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/34' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/38' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/28' 2023-06-09T11:16:48.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/24' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/29' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/32' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/22' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/25' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/30' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/33' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/23' 2023-06-09T11:16:48.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/35' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/26' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './221/37' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './221' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/21' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/27' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/31' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/36' 2023-06-09T11:16:48.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/20' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/39' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/34' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/38' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/28' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/24' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/29' 2023-06-09T11:16:48.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/32' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/22' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/25' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/30' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/33' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/23' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/35' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/26' 2023-06-09T11:16:48.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './222/37' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './222' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/21' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/27' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/31' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/36' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/20' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/39' 2023-06-09T11:16:48.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/34' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/38' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/28' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/24' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/29' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/32' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/22' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/25' 2023-06-09T11:16:48.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/30' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/33' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/23' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/35' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/26' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './223/37' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './223' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/21' 2023-06-09T11:16:48.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/27' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/31' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/36' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/20' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/39' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/34' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/38' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/28' 2023-06-09T11:16:48.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/24' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/29' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/32' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/22' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/25' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/30' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/33' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/23' 2023-06-09T11:16:48.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/35' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/26' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './224/37' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './224' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/21' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/27' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/31' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/36' 2023-06-09T11:16:48.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/20' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/39' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/34' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/38' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/28' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/24' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/29' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/32' 2023-06-09T11:16:48.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/22' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/25' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/30' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/33' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/23' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/35' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/26' 2023-06-09T11:16:48.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './225/37' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './225' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/21' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/27' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/31' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/36' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/20' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/39' 2023-06-09T11:16:48.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/34' 2023-06-09T11:16:48.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/38' 2023-06-09T11:16:48.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/28' 2023-06-09T11:16:48.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/24' 2023-06-09T11:16:49.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/29' 2023-06-09T11:16:49.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/32' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/22' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/25' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/30' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/33' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/23' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/35' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/26' 2023-06-09T11:16:49.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './226/37' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './226' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/21' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/27' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/31' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/36' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/20' 2023-06-09T11:16:49.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/39' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/34' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/38' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/28' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/24' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/29' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/32' 2023-06-09T11:16:49.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/22' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/25' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/30' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/33' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/23' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/35' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/26' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './227/37' 2023-06-09T11:16:49.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './227' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/21' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/27' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/31' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/36' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/20' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/39' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/34' 2023-06-09T11:16:49.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/38' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/28' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/24' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/29' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/32' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/22' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/25' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/30' 2023-06-09T11:16:49.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/33' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/23' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/35' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/26' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './228/37' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './228' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/21' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/27' 2023-06-09T11:16:49.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/31' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/36' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/20' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/39' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/34' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/38' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/28' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/24' 2023-06-09T11:16:49.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/29' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/32' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/22' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/25' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/30' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/33' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/23' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/35' 2023-06-09T11:16:49.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/26' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './229/37' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './229' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/21' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/27' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/31' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/36' 2023-06-09T11:16:49.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/20' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/39' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/34' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/38' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/28' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/24' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/29' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/32' 2023-06-09T11:16:49.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/22' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/25' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/30' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/33' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/23' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/35' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/26' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './230/37' 2023-06-09T11:16:49.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './230' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/21' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/27' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/31' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/36' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/20' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/39' 2023-06-09T11:16:49.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/34' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/38' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/28' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/24' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/29' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/32' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/22' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/25' 2023-06-09T11:16:49.235 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/30' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/33' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/23' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/35' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/26' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './231/37' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './231' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/21' 2023-06-09T11:16:49.236 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/27' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/31' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/36' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/20' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/39' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/34' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/38' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/28' 2023-06-09T11:16:49.237 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/24' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/29' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/32' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/22' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/25' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/30' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/33' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/23' 2023-06-09T11:16:49.238 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/35' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/26' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './232/37' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './232' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/21' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/27' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/31' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/36' 2023-06-09T11:16:49.239 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/20' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/39' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/34' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/38' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/28' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/24' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/29' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/32' 2023-06-09T11:16:49.240 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/22' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/25' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/30' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/33' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/23' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/35' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/26' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed './233/37' 2023-06-09T11:16:49.241 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './233' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/21' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/27' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/31' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/36' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/20' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/39' 2023-06-09T11:16:49.242 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/34' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/38' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/28' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/24' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/29' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/32' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/22' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/25' 2023-06-09T11:16:49.243 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/30' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/33' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/23' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/35' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/26' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './234/37' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './234' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/21' 2023-06-09T11:16:49.244 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/27' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/31' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/36' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/20' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/39' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/34' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/38' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/28' 2023-06-09T11:16:49.245 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/24' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/29' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/32' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/22' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/25' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/30' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/33' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/23' 2023-06-09T11:16:49.246 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/35' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/26' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './235/37' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './235' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/21' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/27' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/31' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/36' 2023-06-09T11:16:49.247 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/20' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/39' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/34' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/38' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/28' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/24' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/29' 2023-06-09T11:16:49.248 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/32' 2023-06-09T11:16:51.292 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/22' 2023-06-09T11:16:51.292 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/25' 2023-06-09T11:16:51.292 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/30' 2023-06-09T11:16:51.292 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/33' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/23' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/35' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/26' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './236/37' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './236' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/21' 2023-06-09T11:16:51.293 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/27' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/31' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/36' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/20' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/39' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/34' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/38' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/28' 2023-06-09T11:16:51.294 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/24' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/29' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/32' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/22' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/25' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/30' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/33' 2023-06-09T11:16:51.295 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/23' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/35' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/26' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './237/37' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './237' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/21' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/27' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/31' 2023-06-09T11:16:51.296 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/36' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/20' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/39' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/34' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/38' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/28' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/24' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/29' 2023-06-09T11:16:51.297 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/32' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/22' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/25' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/30' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/33' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/23' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/35' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/26' 2023-06-09T11:16:51.298 INFO:tasks.workunit.client.0.smithi007.stdout:removed './238/37' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './238' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/21' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/27' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/31' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/36' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/20' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/39' 2023-06-09T11:16:51.299 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/34' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/38' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/28' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/24' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/29' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/32' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/22' 2023-06-09T11:16:51.300 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/25' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/30' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/33' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/23' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/35' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/26' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './239/37' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './239' 2023-06-09T11:16:51.301 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/21' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/27' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/31' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/36' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/20' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/39' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/34' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/38' 2023-06-09T11:16:51.302 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/28' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/24' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/29' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/32' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/22' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/25' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/30' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/33' 2023-06-09T11:16:51.303 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/23' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/35' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/26' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './240/37' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './240' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/21' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/27' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/31' 2023-06-09T11:16:51.304 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/36' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/20' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/39' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/34' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/38' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/28' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/24' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/29' 2023-06-09T11:16:51.305 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/32' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/22' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/25' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/30' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/33' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/23' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/35' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/26' 2023-06-09T11:16:51.306 INFO:tasks.workunit.client.0.smithi007.stdout:removed './241/37' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './241' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/21' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/27' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/31' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/36' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/20' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/39' 2023-06-09T11:16:51.307 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/34' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/38' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/28' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/24' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/29' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/32' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/22' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/25' 2023-06-09T11:16:51.308 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/30' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/33' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/23' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/35' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/26' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './242/37' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './242' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/21' 2023-06-09T11:16:51.309 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/27' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/31' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/36' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/20' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/39' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/34' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/38' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/28' 2023-06-09T11:16:51.310 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/24' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/29' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/32' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/22' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/25' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/30' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/33' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/23' 2023-06-09T11:16:51.311 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/35' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/26' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './243/37' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './243' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/21' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/27' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/31' 2023-06-09T11:16:51.312 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/36' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/20' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/39' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/34' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/38' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/28' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/24' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/29' 2023-06-09T11:16:51.313 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/32' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/22' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/25' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/30' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/33' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/23' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/35' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/26' 2023-06-09T11:16:51.314 INFO:tasks.workunit.client.0.smithi007.stdout:removed './244/37' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './244' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/21' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/27' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/31' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/36' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/20' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/39' 2023-06-09T11:16:51.315 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/34' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/38' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/28' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/24' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/29' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/32' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/22' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/25' 2023-06-09T11:16:51.316 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/30' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/33' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/23' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/35' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/26' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './245/37' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './245' 2023-06-09T11:16:51.317 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/21' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/27' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/31' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/36' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/20' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/39' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/34' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/38' 2023-06-09T11:16:51.318 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/28' 2023-06-09T11:16:51.319 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/24' 2023-06-09T11:16:51.319 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/29' 2023-06-09T11:16:51.319 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/32' 2023-06-09T11:16:51.319 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/22' 2023-06-09T11:16:53.896 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/25' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/30' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/33' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/23' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/35' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/26' 2023-06-09T11:16:53.897 INFO:tasks.workunit.client.0.smithi007.stdout:removed './246/37' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './246' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/21' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/27' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/31' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/36' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/20' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/39' 2023-06-09T11:16:53.898 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/34' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/38' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/28' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/24' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/29' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/32' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/22' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/25' 2023-06-09T11:16:53.899 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/30' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/33' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/23' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/35' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/26' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './247/37' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './247' 2023-06-09T11:16:53.900 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/21' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/27' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/31' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/36' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/20' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/39' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/34' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/38' 2023-06-09T11:16:53.901 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/28' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/24' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/29' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/32' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/22' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/25' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/30' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/33' 2023-06-09T11:16:53.902 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/23' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/35' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/26' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './248/37' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './248' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/21' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/27' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/31' 2023-06-09T11:16:53.903 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/36' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/20' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/39' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/34' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/38' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/28' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/24' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/29' 2023-06-09T11:16:53.904 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/32' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/22' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/25' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/30' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/33' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/23' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/35' 2023-06-09T11:16:53.905 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/26' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './249/37' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './249' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/21' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/27' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/31' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/36' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/20' 2023-06-09T11:16:53.906 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/39' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/34' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/38' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/28' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/24' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/29' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/32' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/22' 2023-06-09T11:16:53.907 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/25' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/30' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/33' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/23' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/35' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/26' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './250/37' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './250' 2023-06-09T11:16:53.908 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/21' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/27' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/31' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/36' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/20' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/39' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/34' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/38' 2023-06-09T11:16:53.909 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/28' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/24' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/29' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/32' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/22' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/25' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/30' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/33' 2023-06-09T11:16:53.910 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/23' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/35' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/26' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './251/37' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './251' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/21' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/27' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/31' 2023-06-09T11:16:53.911 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/36' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/20' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/39' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/34' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/38' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/28' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/24' 2023-06-09T11:16:53.912 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/29' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/32' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/22' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/25' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/30' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/33' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/23' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/35' 2023-06-09T11:16:53.913 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/26' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './252/37' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './252' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/21' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/27' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/31' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/36' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/20' 2023-06-09T11:16:53.914 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/39' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/34' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/38' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/28' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/24' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/29' 2023-06-09T11:16:53.915 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/32' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/22' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/25' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/30' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/33' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/23' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/35' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/26' 2023-06-09T11:16:53.916 INFO:tasks.workunit.client.0.smithi007.stdout:removed './253/37' 2023-06-09T11:16:53.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './253' 2023-06-09T11:16:53.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/21' 2023-06-09T11:16:53.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/27' 2023-06-09T11:16:53.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/31' 2023-06-09T11:16:53.917 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/36' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/20' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/39' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/34' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/38' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/28' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/24' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/29' 2023-06-09T11:16:53.918 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/32' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/22' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/25' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/30' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/33' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/23' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/35' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/26' 2023-06-09T11:16:53.919 INFO:tasks.workunit.client.0.smithi007.stdout:removed './254/37' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './254' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/21' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/27' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/31' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/36' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/20' 2023-06-09T11:16:53.920 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/39' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/34' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/38' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/28' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/24' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/29' 2023-06-09T11:16:53.921 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/32' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/22' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/25' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/30' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/33' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/23' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/35' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/26' 2023-06-09T11:16:53.922 INFO:tasks.workunit.client.0.smithi007.stdout:removed './255/37' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './255' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/21' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/27' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/31' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/36' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/20' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/39' 2023-06-09T11:16:53.923 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/34' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/38' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/28' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/24' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/29' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/32' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/22' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/25' 2023-06-09T11:16:53.924 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/30' 2023-06-09T11:16:56.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/33' 2023-06-09T11:16:56.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/23' 2023-06-09T11:16:56.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/35' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/26' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './256/37' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './256' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/21' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/27' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/31' 2023-06-09T11:16:56.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/36' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/20' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/39' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/34' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/38' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/28' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/24' 2023-06-09T11:16:56.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/29' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/32' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/22' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/25' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/30' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/33' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/23' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/35' 2023-06-09T11:16:56.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/26' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './257/37' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './257' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/21' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/27' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/31' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/36' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/20' 2023-06-09T11:16:56.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/39' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/34' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/38' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/28' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/24' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/29' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/32' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/22' 2023-06-09T11:16:56.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/25' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/30' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/33' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/23' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/35' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/26' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './258/37' 2023-06-09T11:16:56.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './258' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/21' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/27' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/31' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/36' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/20' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/39' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/34' 2023-06-09T11:16:56.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/38' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/28' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/24' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/29' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/32' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/22' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/25' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/30' 2023-06-09T11:16:56.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/33' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/23' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/35' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/26' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './259/37' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './259' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/21' 2023-06-09T11:16:56.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/27' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/31' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/36' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/20' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/39' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/34' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/38' 2023-06-09T11:16:56.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/28' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/24' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/29' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/32' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/22' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/25' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/30' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/33' 2023-06-09T11:16:56.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/23' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/35' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/26' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './260/37' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './260' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/21' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/27' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/31' 2023-06-09T11:16:56.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/36' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/20' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/39' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/34' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/38' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/28' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/24' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/29' 2023-06-09T11:16:56.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/32' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/22' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/25' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/30' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/33' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/23' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/35' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/26' 2023-06-09T11:16:56.212 INFO:tasks.workunit.client.0.smithi007.stdout:removed './261/37' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './261' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/21' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/27' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/31' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/36' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/20' 2023-06-09T11:16:56.213 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/39' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/34' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/38' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/28' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/24' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/29' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/32' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/22' 2023-06-09T11:16:56.214 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/25' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/30' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/33' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/23' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/35' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/26' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './262/37' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './262' 2023-06-09T11:16:56.215 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/21' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/27' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/31' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/36' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/20' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/39' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/34' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/38' 2023-06-09T11:16:56.216 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/28' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/24' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/29' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/32' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/22' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/25' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/30' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/33' 2023-06-09T11:16:56.217 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/23' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/35' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/26' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './263/37' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './263' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/21' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/27' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/31' 2023-06-09T11:16:56.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/36' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/20' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/39' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/34' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/38' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/28' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/24' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/29' 2023-06-09T11:16:56.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/32' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/22' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/25' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/30' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/33' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/23' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/35' 2023-06-09T11:16:56.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/26' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './264/37' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './264' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/21' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/27' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/31' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/36' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/20' 2023-06-09T11:16:56.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/39' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/34' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/38' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/28' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/24' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/29' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/32' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/22' 2023-06-09T11:16:56.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/25' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/30' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/33' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/23' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/35' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/26' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './265/37' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './265' 2023-06-09T11:16:56.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/21' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/27' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/31' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/36' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/20' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/39' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/34' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/38' 2023-06-09T11:16:56.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/28' 2023-06-09T11:16:56.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/24' 2023-06-09T11:16:56.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/29' 2023-06-09T11:16:56.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/32' 2023-06-09T11:16:56.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/22' 2023-06-09T11:16:56.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/25' 2023-06-09T11:16:56.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/30' 2023-06-09T11:16:56.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/33' 2023-06-09T11:16:56.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/23' 2023-06-09T11:16:56.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:16:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.527+0000 7fde3b59e700 1 Processor -- start 2023-06-09T11:16:56.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.527+0000 7fde3b59e700 1 -- start start 2023-06-09T11:16:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.529+0000 7fde3b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.529+0000 7fde3b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c155f80 0x7fde3c1563e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.529+0000 7fde3b59e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c06bb20 con 0x7fde3c0b08e0 2023-06-09T11:16:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3b59e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c0c0c10 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0c1ab0 con 0x7fde3c155f80 2023-06-09T11:16:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3a59c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fde3c0b08e0 0x7fde3c06fe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:46496/0 (socket says 172.21.15.7:46496) 2023-06-09T11:16:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40160/0 (socket says 172.21.15.7:40160) 2023-06-09T11:16:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3a59c700 1 -- 172.21.15.7:0/1587961238 learned_addr learned my addr 172.21.15.7:0/1587961238 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:16:56.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3ad9d700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c155f80 0x7fde3c1563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3959a700 1 -- 172.21.15.7:0/1587961238 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044155818 0 0) 0x7fde2c001490 con 0x7fde3c0b08e0 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde3959a700 1 -- 172.21.15.7:0/1587961238 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde240035b0 con 0x7fde3c0b08e0 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 -- 172.21.15.7:0/1587961238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c155f80 msgr2=0x7fde3c1563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c155f80 0x7fde3c1563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 -- 172.21.15.7:0/1587961238 >> v1:172.21.15.117:6789/0 conn(0x7fde3c0b08e0 legacy=0x7fde3c06fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.530+0000 7fde39d9b700 1 -- 172.21.15.7:0/1587961238 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0c1f40 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.531+0000 7fde39d9b700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde30009840 tx=0x7fde30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf8347478981bb3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.531+0000 7fde3959a700 1 -- 172.21.15.7:0/1587961238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30011830 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.531+0000 7fde3959a700 1 -- 172.21.15.7:0/1587961238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde30011990 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.531+0000 7fde3959a700 1 -- 172.21.15.7:0/1587961238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30011d20 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 -- 172.21.15.7:0/1587961238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 msgr2=0x7fde3c06b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde30009840 tx=0x7fde30009b50 comp rx=0 tx=0).stop 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 -- 172.21.15.7:0/1587961238 shutdown_connections 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c155f80 0x7fde3c1563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 --2- 172.21.15.7:0/1587961238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06b5e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 -- 172.21.15.7:0/1587961238 >> 172.21.15.7:0/1587961238 conn(0x7fde3c0cbcb0 msgr2=0x7fde3c0c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:56.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 -- 172.21.15.7:0/1587961238 shutdown_connections 2023-06-09T11:16:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.532+0000 7fde3b59e700 1 -- 172.21.15.7:0/1587961238 wait complete. 2023-06-09T11:16:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.535+0000 7fde3b59e700 1 Processor -- start 2023-06-09T11:16:56.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.535+0000 7fde3b59e700 1 -- start start 2023-06-09T11:16:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c0b08e0 0x7fde3c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 0x7fde3c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c13c600 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c154080 con 0x7fde3c155f80 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde3b59e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c1f2420 con 0x7fde3c0b08e0 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.536+0000 7fde39d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40168/0 (socket says 172.21.15.7:40168) 2023-06-09T11:16:56.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 -- 172.21.15.7:0/2826433409 learned_addr learned my addr 172.21.15.7:0/2826433409 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:16:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde3ad9d700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 0x7fde3c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde3a59c700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c0b08e0 0x7fde3c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 -- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c0b08e0 msgr2=0x7fde3c153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c0b08e0 0x7fde3c153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 -- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 msgr2=0x7fde3c06ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 0x7fde3c06ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 -- 172.21.15.7:0/2826433409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde300094a0 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde3ad9d700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 0x7fde3c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:16:56.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.537+0000 7fde39d9b700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde30000c00 tx=0x7fde30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.538+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300036a0 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.538+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3001f070 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.538+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30003a60 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.538+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c0c0650 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.538+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0c10a0 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.539+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c0c0930 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.540+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde30003bc0 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.541+0000 7fde2affd700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 0x7fde1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:56.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.541+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fde30021070 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.541+0000 7fde3a59c700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 0x7fde1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:56.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.542+0000 7fde3a59c700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 0x7fde1c06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0015f0 tx=0x7fde2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:56.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.546+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde30055e70 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.815+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde3c04f760 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.816+0000 7fde2affd700 1 -- 172.21.15.7:0/2826433409 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7fde30059490 con 0x7fde3c0b0cb0 2023-06-09T11:16:56.822 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:16:56.822 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:16:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 msgr2=0x7fde1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 0x7fde1c06d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0015f0 tx=0x7fde2c009040 comp rx=0 tx=0).stop 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 msgr2=0x7fde3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde30000c00 tx=0x7fde30004750 comp rx=0 tx=0).stop 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 shutdown_connections 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde3c155f80 0x7fde3c06ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde1c06ace0 0x7fde1c06d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde3c0b08e0 0x7fde3c153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 --2- 172.21.15.7:0/2826433409 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde3c0b0cb0 0x7fde3c06d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 >> 172.21.15.7:0/2826433409 conn(0x7fde3c0cbcb0 msgr2=0x7fde3c1497c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 shutdown_connections 2023-06-09T11:16:56.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:56.818+0000 7fde3b59e700 1 -- 172.21.15.7:0/2826433409 wait complete. 2023-06-09T11:16:56.827 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:16:56.842 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2023-06-09T11:16:56.842 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:16:56.843 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:16:56.870 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:16:56.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:16:57.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.066+0000 7f83595ff700 1 Processor -- start 2023-06-09T11:16:57.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.066+0000 7f83595ff700 1 -- start start 2023-06-09T11:16:57.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.067+0000 7f83595ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8354074150 0x7f835406a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:57.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83595ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835406a4d0 con 0x7f8354073d80 2023-06-09T11:16:57.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83595ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83540cc280 con 0x7f83540bf5d0 2023-06-09T11:16:57.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83595ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8354073a10 con 0x7f8354074150 2023-06-09T11:16:57.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8354074150 0x7f835406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:57.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8354074150 0x7f835406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40176/0 (socket says 172.21.15.7:40176) 2023-06-09T11:16:57.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 -- 172.21.15.7:0/3100395457 learned_addr learned my addr 172.21.15.7:0/3100395457 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:16:57.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 -- 172.21.15.7:0/3100395457 >> v1:172.21.15.117:6789/0 conn(0x7f83540bf5d0 legacy=0x7f83541534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 -- 172.21.15.7:0/3100395457 >> v1:172.21.15.7:6789/0 conn(0x7f8354073d80 legacy=0x7f835406b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 -- 172.21.15.7:0/3100395457 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835406bdc0 con 0x7f8354074150 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.068+0000 7f83537fe700 1 --2- 172.21.15.7:0/3100395457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f835406a960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32360b37ab5d5b58 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f8351ffb700 1 -- 172.21.15.7:0/3100395457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00e070 con 0x7f8354074150 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f8351ffb700 1 -- 172.21.15.7:0/3100395457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c00b9c0 con 0x7f8354074150 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f8351ffb700 1 -- 172.21.15.7:0/3100395457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c00bd70 con 0x7f8354074150 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 -- 172.21.15.7:0/3100395457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 msgr2=0x7f835406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 --2- 172.21.15.7:0/3100395457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f835406a960 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f834c009e50 tx=0x7f834c00b330 comp rx=0 tx=0).stop 2023-06-09T11:16:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 -- 172.21.15.7:0/3100395457 shutdown_connections 2023-06-09T11:16:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 --2- 172.21.15.7:0/3100395457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f835406a960 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 -- 172.21.15.7:0/3100395457 >> 172.21.15.7:0/3100395457 conn(0x7f835406d150 msgr2=0x7f835406d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 -- 172.21.15.7:0/3100395457 shutdown_connections 2023-06-09T11:16:57.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.069+0000 7f83595ff700 1 -- 172.21.15.7:0/3100395457 wait complete. 2023-06-09T11:16:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.072+0000 7f83595ff700 1 Processor -- start 2023-06-09T11:16:57.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.072+0000 7f83595ff700 1 -- start start 2023-06-09T11:16:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354073d80 0x7f83541e2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:57.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83540bf5d0 0x7f835414c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541ee7b0 con 0x7f8354074150 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83541e26b0 con 0x7f8354073d80 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83595ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83541ef010 con 0x7f83540bf5d0 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83527fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:57.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83527fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40178/0 (socket says 172.21.15.7:40178) 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.073+0000 7f83527fc700 1 -- 172.21.15.7:0/2043730822 learned_addr learned my addr 172.21.15.7:0/2043730822 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f8352ffd700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354073d80 0x7f83541e2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83537fe700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83540bf5d0 0x7f835414c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 -- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83540bf5d0 msgr2=0x7f835414c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83540bf5d0 0x7f835414c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 -- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354073d80 msgr2=0x7f83541e2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354073d80 0x7f83541e2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 -- 172.21.15.7:0/2043730822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009b30 con 0x7f8354074150 2023-06-09T11:16:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83527fc700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f834800c400 tx=0x7f834800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480052d0 con 0x7f8354074150 2023-06-09T11:16:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8348005430 con 0x7f8354074150 2023-06-09T11:16:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83540cc4d0 con 0x7f8354074150 2023-06-09T11:16:57.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480038a0 con 0x7f8354074150 2023-06-09T11:16:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.074+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83540c8700 con 0x7f8354074150 2023-06-09T11:16:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.076+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83540cc7b0 con 0x7f8354074150 2023-06-09T11:16:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.076+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8348003a00 con 0x7f8354074150 2023-06-09T11:16:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.077+0000 7f833b7fe700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:57.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.077+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f8348088ac0 con 0x7f8354074150 2023-06-09T11:16:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.077+0000 7f8352ffd700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:57.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.078+0000 7f8352ffd700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 0x7f834006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f8344009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:57.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.082+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f83480538f0 con 0x7f8354074150 2023-06-09T11:16:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.343+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83540c73e0 con 0x7f8354074150 2023-06-09T11:16:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.343+0000 7f833b7fe700 1 -- 172.21.15.7:0/2043730822 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7f8348003cb0 con 0x7f8354074150 2023-06-09T11:16:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:16:57.349 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:16:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 msgr2=0x7f834006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 0x7f834006d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8344000f60 tx=0x7f8344009040 comp rx=0 tx=0).stop 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 msgr2=0x7f83541523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f834800c400 tx=0x7f834800c7c0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 shutdown_connections 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834006ace0 0x7f834006d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83540bf5d0 0x7f835414c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8354073d80 0x7f83541e2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 --2- 172.21.15.7:0/2043730822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8354074150 0x7f83541523f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 >> 172.21.15.7:0/2043730822 conn(0x7f835406d150 msgr2=0x7f83540cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 shutdown_connections 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:57.346+0000 7f83595ff700 1 -- 172.21.15.7:0/2043730822 wait complete. 2023-06-09T11:16:57.353 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:16:57.370 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:16:59.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:16:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.574+0000 7fd0c7fff700 1 Processor -- start 2023-06-09T11:16:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.575+0000 7fd0c7fff700 1 -- start start 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 0x7fd0c00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0c015cd40 0x7fd0c00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c7fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c0159b20 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00cac80 con 0x7fd0c015cd40 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c01597d0 con 0x7fd0c00cb5d0 2023-06-09T11:16:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0c015cd40 0x7fd0c00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0c015cd40 0x7fd0c00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40192/0 (socket says 172.21.15.7:40192) 2023-06-09T11:16:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c659c700 1 -- 172.21.15.7:0/2142093867 learned_addr learned my addr 172.21.15.7:0/2142093867 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:16:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c559a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0c00cb9a0 0x7fd0c0159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51954/0 (socket says 172.21.15.7:51954) 2023-06-09T11:16:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c659c700 1 -- 172.21.15.7:0/2142093867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 msgr2=0x7fd0c00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:16:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c5d9b700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 0x7fd0c00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.576+0000 7fd0c4d99700 1 -- 172.21.15.7:0/2142093867 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544585819 0 0) 0x7fd0c0159b20 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c659c700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 0x7fd0c00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c659c700 1 -- 172.21.15.7:0/2142093867 >> v1:172.21.15.7:6789/0 conn(0x7fd0c00cb9a0 legacy=0x7fd0c0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c659c700 1 -- 172.21.15.7:0/2142093867 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c0071ab0 con 0x7fd0c015cd40 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c5d9b700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 0x7fd0c00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c659c700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c015cd40 0x7fd0c00cbea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009e50 tx=0x7fd0b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b28af93eb9d435b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c4d99700 1 -- 172.21.15.7:0/2142093867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800e070 con 0x7fd0c015cd40 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c4d99700 1 -- 172.21.15.7:0/2142093867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b800b9c0 con 0x7fd0c015cd40 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.577+0000 7fd0c4d99700 1 -- 172.21.15.7:0/2142093867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b800bd70 con 0x7fd0c015cd40 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 -- 172.21.15.7:0/2142093867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c015cd40 msgr2=0x7fd0c00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c015cd40 0x7fd0c00cbea0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8009e50 tx=0x7fd0b800b330 comp rx=0 tx=0).stop 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 -- 172.21.15.7:0/2142093867 shutdown_connections 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c015cd40 0x7fd0c00cbea0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/2142093867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c00cb5d0 0x7fd0c00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 -- 172.21.15.7:0/2142093867 >> 172.21.15.7:0/2142093867 conn(0x7fd0c0070af0 msgr2=0x7fd0c0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 -- 172.21.15.7:0/2142093867 shutdown_connections 2023-06-09T11:16:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.578+0000 7fd0c7fff700 1 -- 172.21.15.7:0/2142093867 wait complete. 2023-06-09T11:16:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.579+0000 7fd0c7fff700 1 Processor -- start 2023-06-09T11:16:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 -- start start 2023-06-09T11:16:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c00cb5d0 0x7fd0c01ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c015cd40 0x7fd0c00cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c01ef980 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c015a930 con 0x7fd0c00cb5d0 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.580+0000 7fd0c7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00cf0b0 con 0x7fd0c015cd40 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40196/0 (socket says 172.21.15.7:40196) 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 -- 172.21.15.7:0/149829898 learned_addr learned my addr 172.21.15.7:0/149829898 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:16:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c659c700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c015cd40 0x7fd0c00cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c5d9b700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c00cb5d0 0x7fd0c01ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 -- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c015cd40 msgr2=0x7fd0c00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c015cd40 0x7fd0c00cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 -- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c00cb5d0 msgr2=0x7fd0c01ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c00cb5d0 0x7fd0c01ef440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 -- 172.21.15.7:0/149829898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8009b30 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.581+0000 7fd0c559a700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c540 tx=0x7fd0b000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.582+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004c80 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.582+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b0004de0 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.582+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c015d4f0 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.582+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0007310 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.582+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c015d210 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.583+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c015d7d0 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.584+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0b0007470 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.585+0000 7fd0b67fc700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 0x7fd0a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:16:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.585+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7fd0b0089440 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.585+0000 7fd0c5d9b700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 0x7fd0a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:16:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.586+0000 7fd0c5d9b700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 0x7fd0a406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc006fd0 tx=0x7fd0bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:16:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.590+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd0b0054300 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.840+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c0074cf0 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.841+0000 7fd0b67fc700 1 -- 172.21.15.7:0/149829898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7fd0b0057920 con 0x7fd0c00cb9a0 2023-06-09T11:16:59.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:16:59.847 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:16:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 msgr2=0x7fd0a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 0x7fd0a406d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc006fd0 tx=0x7fd0bc008040 comp rx=0 tx=0).stop 2023-06-09T11:16:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 msgr2=0x7fd0c00aae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c540 tx=0x7fd0b000c900 comp rx=0 tx=0).stop 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 shutdown_connections 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd0a406ace0 0x7fd0a406d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0c015cd40 0x7fd0c00cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0c00cb5d0 0x7fd0c01ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 --2- 172.21.15.7:0/149829898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0c00cb9a0 0x7fd0c00aae80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:16:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 >> 172.21.15.7:0/149829898 conn(0x7fd0c0070af0 msgr2=0x7fd0c0070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:16:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 shutdown_connections 2023-06-09T11:16:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:16:59.843+0000 7fd0c7fff700 1 -- 172.21.15.7:0/149829898 wait complete. 2023-06-09T11:16:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:16:59.867 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:01.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:02.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.072+0000 7f23fd3ff700 1 Processor -- start 2023-06-09T11:17:02.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.073+0000 7f23fd3ff700 1 -- start start 2023-06-09T11:17:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23fd3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23f80c9090 0x7f23f80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23fd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f81598b0 0x7f23f8159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23fd3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f815cec0 con 0x7f23f80cad00 2023-06-09T11:17:02.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23fd3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f815d160 con 0x7f23f80c9090 2023-06-09T11:17:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23fd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f815d440 con 0x7f23f81598b0 2023-06-09T11:17:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23f80c9090 0x7f23f80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23effff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f23f80cad00 0x7f23f80cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51970/0 (socket says 172.21.15.7:51970) 2023-06-09T11:17:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23effff700 1 -- 172.21.15.7:0/3330419495 learned_addr learned my addr 172.21.15.7:0/3330419495 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:02.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23f80c9090 0x7f23f80c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40202/0 (socket says 172.21.15.7:40202) 2023-06-09T11:17:02.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 -- 172.21.15.7:0/3330419495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f81598b0 msgr2=0x7f23f8159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 --2- 172.21.15.7:0/3330419495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f81598b0 0x7f23f8159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 -- 172.21.15.7:0/3330419495 >> v1:172.21.15.7:6789/0 conn(0x7f23f80cad00 legacy=0x7f23f80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.074+0000 7f23f6ffd700 1 -- 172.21.15.7:0/3330419495 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f815d720 con 0x7f23f80c9090 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.075+0000 7f23f6ffd700 1 --2- 172.21.15.7:0/3330419495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f80c9090 0x7f23f80c9fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009840 tx=0x7f23e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f62bd0bd9488dc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.075+0000 7f23f67fc700 1 -- 172.21.15.7:0/3330419495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8011830 con 0x7f23f80c9090 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.075+0000 7f23f67fc700 1 -- 172.21.15.7:0/3330419495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e8011990 con 0x7f23f80c9090 2023-06-09T11:17:02.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.075+0000 7f23f67fc700 1 -- 172.21.15.7:0/3330419495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e8011d20 con 0x7f23f80c9090 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 -- 172.21.15.7:0/3330419495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f80c9090 msgr2=0x7f23f80c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/3330419495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f80c9090 0x7f23f80c9fe0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009840 tx=0x7f23e8009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 -- 172.21.15.7:0/3330419495 shutdown_connections 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/3330419495 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f81598b0 0x7f23f8159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/3330419495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f80c9090 0x7f23f80c9fe0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 -- 172.21.15.7:0/3330419495 >> 172.21.15.7:0/3330419495 conn(0x7f23f8070af0 msgr2=0x7f23f8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:02.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 -- 172.21.15.7:0/3330419495 shutdown_connections 2023-06-09T11:17:02.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.076+0000 7f23fd3ff700 1 -- 172.21.15.7:0/3330419495 wait complete. 2023-06-09T11:17:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.078+0000 7f23fd3ff700 1 Processor -- start 2023-06-09T11:17:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.078+0000 7f23fd3ff700 1 -- start start 2023-06-09T11:17:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.079+0000 7f23fd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 0x7f23f8158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.079+0000 7f23fd3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f80cad00 0x7f23f80c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.079+0000 7f23fd3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 0x7f23f80c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23fd3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81f2f80 con 0x7f23f81598b0 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23fd3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f80c5e50 con 0x7f23f80cad00 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23fd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f81574a0 con 0x7f23f80c9090 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 0x7f23f8158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 0x7f23f80c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:02.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 0x7f23f8158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:58432/0 (socket says 172.21.15.7:58432) 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f6ffd700 1 -- 172.21.15.7:0/2164752922 learned_addr learned my addr 172.21.15.7:0/2164752922 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23effff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f80cad00 0x7f23f80c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 -- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 msgr2=0x7f23f8158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 0x7f23f8158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 -- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f80cad00 msgr2=0x7f23f80c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f80cad00 0x7f23f80c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.080+0000 7f23f77fe700 1 -- 172.21.15.7:0/2164752922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e80094a0 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23f77fe700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 0x7f23f80c36d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f23f000be20 tx=0x7f23f0009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f000cb90 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f000cd10 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f815a250 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f0014680 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.081+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f80ca620 con 0x7f23f81598b0 2023-06-09T11:17:02.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.082+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f80cb160 con 0x7f23f81598b0 2023-06-09T11:17:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.083+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f23f001c360 con 0x7f23f81598b0 2023-06-09T11:17:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.084+0000 7f23ef7fe700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 0x7f23d806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.084+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f23f0089430 con 0x7f23f81598b0 2023-06-09T11:17:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.084+0000 7f23f6ffd700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 0x7f23d806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:02.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.085+0000 7f23f6ffd700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 0x7f23d806d1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009b90 tx=0x7f23e801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:02.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.088+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f23f00541d0 con 0x7f23f81598b0 2023-06-09T11:17:02.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.340+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23f8074cf0 con 0x7f23f81598b0 2023-06-09T11:17:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.341+0000 7f23ef7fe700 1 -- 172.21.15.7:0/2164752922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7f23f00577f0 con 0x7f23f81598b0 2023-06-09T11:17:02.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:02.347 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 msgr2=0x7f23d806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 0x7f23d806d1c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f23e8009b90 tx=0x7f23e801c040 comp rx=0 tx=0).stop 2023-06-09T11:17:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 msgr2=0x7f23f80c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 0x7f23f80c36d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f23f000be20 tx=0x7f23f0009ed0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 shutdown_connections 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23f81598b0 0x7f23f80c36d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23d806ad10 0x7f23d806d1c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23f80c9090 0x7f23f8158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 --2- 172.21.15.7:0/2164752922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f23f80cad00 0x7f23f80c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 >> 172.21.15.7:0/2164752922 conn(0x7f23f8070af0 msgr2=0x7f23f8158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.343+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 shutdown_connections 2023-06-09T11:17:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:02.344+0000 7f23fd3ff700 1 -- 172.21.15.7:0/2164752922 wait complete. 2023-06-09T11:17:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:17:02.367 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:04.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.572+0000 7feb913ff700 1 Processor -- start 2023-06-09T11:17:04.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.573+0000 7feb913ff700 1 -- start start 2023-06-09T11:17:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb913ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 0x7feb8c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb913ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb8c0c9f40 0x7feb8c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb913ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c15a500 con 0x7feb8c153880 2023-06-09T11:17:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb913ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c0c5f10 con 0x7feb8c0c9f40 2023-06-09T11:17:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb913ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c15da00 con 0x7feb8c15b4a0 2023-06-09T11:17:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb8b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb8c0c9f40 0x7feb8c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb8b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb8c0c9f40 0x7feb8c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60222/0 (socket says 172.21.15.7:60222) 2023-06-09T11:17:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3192661228 learned_addr learned my addr 172.21.15.7:0/3192661228 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb8affd700 1 --1- 172.21.15.7:0/3192661228 >> v1:172.21.15.7:6789/0 conn(0x7feb8c153880 0x7feb8c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57364/0 (socket says 172.21.15.7:57364) 2023-06-09T11:17:04.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.574+0000 7feb8a7fc700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 0x7feb8c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3192661228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 msgr2=0x7feb8c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:04.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8b7fe700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 0x7feb8c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3192661228 >> v1:172.21.15.7:6789/0 conn(0x7feb8c153880 legacy=0x7feb8c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3192661228 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c1596e0 con 0x7feb8c0c9f40 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8a7fc700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 0x7feb8c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb8b7fe700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c0c9f40 0x7feb8c0c4e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb84009e50 tx=0x7feb8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aab3a9534b163 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb89ffb700 1 -- 172.21.15.7:0/3192661228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400e070 con 0x7feb8c0c9f40 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb89ffb700 1 -- 172.21.15.7:0/3192661228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8400b9c0 con 0x7feb8c0c9f40 2023-06-09T11:17:04.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.575+0000 7feb89ffb700 1 -- 172.21.15.7:0/3192661228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400bd70 con 0x7feb8c0c9f40 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 -- 172.21.15.7:0/3192661228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c0c9f40 msgr2=0x7feb8c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c0c9f40 0x7feb8c0c4e30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb84009e50 tx=0x7feb8400b330 comp rx=0 tx=0).stop 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 -- 172.21.15.7:0/3192661228 shutdown_connections 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c0c9f40 0x7feb8c0c4e30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 --2- 172.21.15.7:0/3192661228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c15b4a0 0x7feb8c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 -- 172.21.15.7:0/3192661228 >> 172.21.15.7:0/3192661228 conn(0x7feb8c0737f0 msgr2=0x7feb8c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 -- 172.21.15.7:0/3192661228 shutdown_connections 2023-06-09T11:17:04.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.576+0000 7feb913ff700 1 -- 172.21.15.7:0/3192661228 wait complete. 2023-06-09T11:17:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.578+0000 7feb913ff700 1 Processor -- start 2023-06-09T11:17:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.578+0000 7feb913ff700 1 -- start start 2023-06-09T11:17:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb8c0c9f40 0x7feb8c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c153880 0x7feb8c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0b31c0 con 0x7feb8c15b4a0 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1582c0 con 0x7feb8c0c9f40 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb913ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c158ba0 con 0x7feb8c153880 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60238/0 (socket says 172.21.15.7:60238) 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3860505287 learned_addr learned my addr 172.21.15.7:0/3860505287 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:04.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8a7fc700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c153880 0x7feb8c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8affd700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb8c0c9f40 0x7feb8c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c153880 msgr2=0x7feb8c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c153880 0x7feb8c0cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb8c0c9f40 msgr2=0x7feb8c158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.579+0000 7feb8b7fe700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb8c0c9f40 0x7feb8c158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb8b7fe700 1 -- 172.21.15.7:0/3860505287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb84009b30 con 0x7feb8c15b4a0 2023-06-09T11:17:04.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb8b7fe700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b730 tx=0x7feb84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840047b0 con 0x7feb8c15b4a0 2023-06-09T11:17:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84004930 con 0x7feb8c15b4a0 2023-06-09T11:17:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8401c680 con 0x7feb8c15b4a0 2023-06-09T11:17:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c159c60 con 0x7feb8c15b4a0 2023-06-09T11:17:04.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.580+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c159980 con 0x7feb8c15b4a0 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.581+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c159f40 con 0x7feb8c15b4a0 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.582+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb8401c7e0 con 0x7feb8c15b4a0 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.583+0000 7feb737fe700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 0x7feb7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.583+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7feb8408b220 con 0x7feb8c15b4a0 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.583+0000 7feb8affd700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 0x7feb7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:04.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.584+0000 7feb8affd700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 0x7feb7806d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb7c000f60 tx=0x7feb7c007ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:04.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.588+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feb84056200 con 0x7feb8c15b4a0 2023-06-09T11:17:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.838+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb8c157240 con 0x7feb8c15b4a0 2023-06-09T11:17:04.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.839+0000 7feb737fe700 1 -- 172.21.15.7:0/3860505287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7feb84059820 con 0x7feb8c15b4a0 2023-06-09T11:17:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:04.845 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 msgr2=0x7feb7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 0x7feb7806d190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feb7c000f60 tx=0x7feb7c007ed0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 msgr2=0x7feb8c0b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:04.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feb8400b730 tx=0x7feb84004ad0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 shutdown_connections 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb8c15b4a0 0x7feb8c0b35e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb8c0c9f40 0x7feb8c158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb7806ace0 0x7feb7806d190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 --2- 172.21.15.7:0/3860505287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb8c153880 0x7feb8c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 >> 172.21.15.7:0/3860505287 conn(0x7feb8c0737f0 msgr2=0x7feb8c14d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 shutdown_connections 2023-06-09T11:17:04.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:04.841+0000 7feb913ff700 1 -- 172.21.15.7:0/3860505287 wait complete. 2023-06-09T11:17:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:17:04.865 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:06.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:07.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.072+0000 7ff3429ff700 1 Processor -- start 2023-06-09T11:17:07.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.072+0000 7ff3429ff700 1 -- start start 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff3429ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c9f40 0x7ff33c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff3429ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c0c5a80 con 0x7ff33c15b4a0 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff3429ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33c15a500 con 0x7ff33c153880 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff3429ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33c0c5f10 con 0x7ff33c0c9f40 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff33b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff33c15b4a0 0x7ff33c0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57380/0 (socket says 172.21.15.7:57380) 2023-06-09T11:17:07.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.073+0000 7ff33b7fe700 1 -- 172.21.15.7:0/4059051768 learned_addr learned my addr 172.21.15.7:0/4059051768 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:07.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff340f9c700 1 --2- 172.21.15.7:0/4059051768 >> v2:172.21.15.117:3300/0 conn(0x7ff33c0c9f40 0x7ff33c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3978926850 0 0) 0x7ff33c0c5a80 con 0x7ff33c15b4a0 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3240035b0 con 0x7ff33c15b4a0 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872248115 0 0) 0x7ff32c001110 con 0x7ff33c153880 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff33c0c5a80 con 0x7ff33c153880 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3872268443 0 0) 0x7ff3240035b0 con 0x7ff33c15b4a0 2023-06-09T11:17:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff340f9c700 1 -- 172.21.15.7:0/4059051768 >> v1:172.21.15.117:6789/0 conn(0x7ff33c153880 legacy=0x7ff33c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff340f9c700 1 -- 172.21.15.7:0/4059051768 >> v1:172.21.15.7:6789/0 conn(0x7ff33c15b4a0 legacy=0x7ff33c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.074+0000 7ff340f9c700 1 -- 172.21.15.7:0/4059051768 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c1596e0 con 0x7ff33c0c9f40 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff340f9c700 1 --2- 172.21.15.7:0/4059051768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c9f40 0x7ff33c0c5540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d90 tx=0x7ff33400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78ec8167fad19a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400e070 con 0x7ff33c0c9f40 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff334003a20 con 0x7ff33c0c9f40 2023-06-09T11:17:07.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff33affd700 1 -- 172.21.15.7:0/4059051768 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330001670 con 0x7ff33c0c9f40 2023-06-09T11:17:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 -- 172.21.15.7:0/4059051768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c9f40 msgr2=0x7ff33c0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4059051768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c9f40 0x7ff33c0c5540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff334009d90 tx=0x7ff33400b330 comp rx=0 tx=0).stop 2023-06-09T11:17:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 -- 172.21.15.7:0/4059051768 shutdown_connections 2023-06-09T11:17:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4059051768 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c0c9f40 0x7ff33c0c5540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 -- 172.21.15.7:0/4059051768 >> 172.21.15.7:0/4059051768 conn(0x7ff33c0737f0 msgr2=0x7ff33c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.075+0000 7ff3429ff700 1 -- 172.21.15.7:0/4059051768 shutdown_connections 2023-06-09T11:17:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.076+0000 7ff3429ff700 1 -- 172.21.15.7:0/4059051768 wait complete. 2023-06-09T11:17:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.078+0000 7ff3429ff700 1 Processor -- start 2023-06-09T11:17:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.078+0000 7ff3429ff700 1 -- start start 2023-06-09T11:17:07.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c9f40 0x7ff33c1e34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c15b4a0 0x7ff33c0b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c1e3a00 con 0x7ff33c153880 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0c8df0 con 0x7ff33c15b4a0 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.079+0000 7ff3429ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33c0c9090 con 0x7ff33c0c9f40 2023-06-09T11:17:07.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60242/0 (socket says 172.21.15.7:60242) 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 -- 172.21.15.7:0/4173716017 learned_addr learned my addr 172.21.15.7:0/4173716017 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff340f9c700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c15b4a0 0x7ff33c0b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33bfff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c9f40 0x7ff33c1e34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 -- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c9f40 msgr2=0x7ff33c1e34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c9f40 0x7ff33c1e34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 -- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c15b4a0 msgr2=0x7ff33c0b4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c15b4a0 0x7ff33c0b4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 -- 172.21.15.7:0/4173716017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334009a70 con 0x7ff33c153880 2023-06-09T11:17:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff33b7fe700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c6b0 tx=0x7ff33000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.080+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330004c80 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.081+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c159c60 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.081+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330004de0 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.081+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3300072f0 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.081+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c159980 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.082+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c159f40 con 0x7ff33c153880 2023-06-09T11:17:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.083+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff330007450 con 0x7ff33c153880 2023-06-09T11:17:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.083+0000 7ff338ff9700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 0x7ff32806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.084+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7ff330089430 con 0x7ff33c153880 2023-06-09T11:17:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.084+0000 7ff33bfff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 0x7ff32806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:07.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.084+0000 7ff33bfff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 0x7ff32806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff32c001370 tx=0x7ff32c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.088+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff3300542f0 con 0x7ff33c153880 2023-06-09T11:17:07.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.343+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff33c074cf0 con 0x7ff33c153880 2023-06-09T11:17:07.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.344+0000 7ff338ff9700 1 -- 172.21.15.7:0/4173716017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7ff330007700 con 0x7ff33c153880 2023-06-09T11:17:07.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:07.350 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.346+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 msgr2=0x7ff32806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.346+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 0x7ff32806d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff32c001370 tx=0x7ff32c009040 comp rx=0 tx=0).stop 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.346+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 msgr2=0x7ff33c1e23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.346+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff33000c6b0 tx=0x7ff33000c9c0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.346+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 shutdown_connections 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff33c15b4a0 0x7ff33c0b4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff32806ace0 0x7ff32806d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff33c0c9f40 0x7ff33c1e34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 --2- 172.21.15.7:0/4173716017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff33c153880 0x7ff33c1e23d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 >> 172.21.15.7:0/4173716017 conn(0x7ff33c0737f0 msgr2=0x7ff33c0720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 shutdown_connections 2023-06-09T11:17:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:07.347+0000 7ff3429ff700 1 -- 172.21.15.7:0/4173716017 wait complete. 2023-06-09T11:17:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:17:07.370 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:09.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.576+0000 7f15d5bff700 1 Processor -- start 2023-06-09T11:17:09.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.576+0000 7f15d5bff700 1 -- start start 2023-06-09T11:17:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.577+0000 7f15d5bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00ca070 0x7f15d015cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.577+0000 7f15d5bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d015a390 con 0x7f15d0153760 2023-06-09T11:17:09.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.577+0000 7f15d5bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00ca720 con 0x7f15d0153b30 2023-06-09T11:17:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.577+0000 7f15d5bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00c5f40 con 0x7f15d00ca070 2023-06-09T11:17:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00ca070 0x7f15d015cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f15d0153760 0x7f15d00b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57396/0 (socket says 172.21.15.7:57396) 2023-06-09T11:17:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f15d00ca070 0x7f15d015cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60248/0 (socket says 172.21.15.7:60248) 2023-06-09T11:17:09.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cf7fe700 1 -- 172.21.15.7:0/106547484 learned_addr learned my addr 172.21.15.7:0/106547484 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:09.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 -- 172.21.15.7:0/106547484 >> v1:172.21.15.117:6789/0 conn(0x7f15d0153b30 legacy=0x7f15d0071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 -- 172.21.15.7:0/106547484 >> v1:172.21.15.7:6789/0 conn(0x7f15d0153760 legacy=0x7f15d00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 -- 172.21.15.7:0/106547484 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00c5a20 con 0x7f15d00ca070 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15ce7fc700 1 -- 172.21.15.7:0/106547484 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875451944 0 0) 0x7f15d015a390 con 0x7f15d0153760 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.578+0000 7f15cffff700 1 --2- 172.21.15.7:0/106547484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00ca070 0x7f15d015cc70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009d90 tx=0x7f15c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226d84127e3e4b76 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15ce7fc700 1 -- 172.21.15.7:0/106547484 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800e070 con 0x7f15d00ca070 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15ce7fc700 1 -- 172.21.15.7:0/106547484 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c800bbd0 con 0x7f15d00ca070 2023-06-09T11:17:09.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15ce7fc700 1 -- 172.21.15.7:0/106547484 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8012430 con 0x7f15d00ca070 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 -- 172.21.15.7:0/106547484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00ca070 msgr2=0x7f15d015cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 --2- 172.21.15.7:0/106547484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00ca070 0x7f15d015cc70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009d90 tx=0x7f15c800b330 comp rx=0 tx=0).stop 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 -- 172.21.15.7:0/106547484 shutdown_connections 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 --2- 172.21.15.7:0/106547484 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d00ca070 0x7f15d015cc70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 -- 172.21.15.7:0/106547484 >> 172.21.15.7:0/106547484 conn(0x7f15d00737f0 msgr2=0x7f15d0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 -- 172.21.15.7:0/106547484 shutdown_connections 2023-06-09T11:17:09.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.579+0000 7f15d5bff700 1 -- 172.21.15.7:0/106547484 wait complete. 2023-06-09T11:17:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.581+0000 7f15d5bff700 1 Processor -- start 2023-06-09T11:17:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.581+0000 7f15d5bff700 1 -- start start 2023-06-09T11:17:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d00ca070 0x7f15d01e2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 0x7f15d00b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:09.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d01e55f0 con 0x7f15d0153b30 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b49c0 con 0x7f15d00ca070 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15d5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00b3800 con 0x7f15d0153760 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.582+0000 7f15cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60256/0 (socket says 172.21.15.7:60256) 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 -- 172.21.15.7:0/330757501 learned_addr learned my addr 172.21.15.7:0/330757501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:09.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cf7fe700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d00ca070 0x7f15d01e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15ceffd700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 0x7f15d00b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 -- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 msgr2=0x7f15d00b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 0x7f15d00b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 -- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d00ca070 msgr2=0x7f15d01e2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d00ca070 0x7f15d01e2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 -- 172.21.15.7:0/330757501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c8009a70 con 0x7f15d0153b30 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15ceffd700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 0x7f15d00b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:09.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15cffff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15c8014040 tx=0x7f15c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c801b820 con 0x7f15d0153b30 2023-06-09T11:17:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8020070 con 0x7f15d0153b30 2023-06-09T11:17:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d0159a60 con 0x7f15d0153b30 2023-06-09T11:17:09.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.583+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c801bbc0 con 0x7f15d0153b30 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.584+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d0159780 con 0x7f15d0153b30 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.585+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d0159d40 con 0x7f15d0153b30 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.586+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15c801bd20 con 0x7f15d0153b30 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.587+0000 7f15b7fff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 0x7f15bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.587+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f15c808b580 con 0x7f15d0153b30 2023-06-09T11:17:09.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.587+0000 7f15cf7fe700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 0x7f15bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:09.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.587+0000 7f15cf7fe700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 0x7f15bc06d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15c0007b10 tx=0x7f15c0007a60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:09.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.591+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f15c8059e40 con 0x7f15d0153b30 2023-06-09T11:17:09.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.841+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15d00b4b30 con 0x7f15d0153b30 2023-06-09T11:17:09.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.842+0000 7f15b7fff700 1 -- 172.21.15.7:0/330757501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7f15c80599d0 con 0x7f15d0153b30 2023-06-09T11:17:09.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:09.848 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 msgr2=0x7f15bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 0x7f15bc06d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f15c0007b10 tx=0x7f15c0007a60 comp rx=0 tx=0).stop 2023-06-09T11:17:09.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 msgr2=0x7f15d014f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f15c8014040 tx=0x7f15c8004ad0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 shutdown_connections 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15d0153b30 0x7f15d014f460 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15d00ca070 0x7f15d01e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15bc06ace0 0x7f15bc06d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 --2- 172.21.15.7:0/330757501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15d0153760 0x7f15d00b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:09.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.844+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 >> 172.21.15.7:0/330757501 conn(0x7f15d00737f0 msgr2=0x7f15d00662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.845+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 shutdown_connections 2023-06-09T11:17:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:09.845+0000 7f15d5bff700 1 -- 172.21.15.7:0/330757501 wait complete. 2023-06-09T11:17:09.852 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:17:09.868 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:11.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:11.640 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:11.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:12.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.071+0000 7f39ab7ff700 1 Processor -- start 2023-06-09T11:17:12.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.072+0000 7f39ab7ff700 1 -- start start 2023-06-09T11:17:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39ab7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 0x7f39a40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:12.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39ab7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a4159b20 con 0x7f39a40cb9a0 2023-06-09T11:17:12.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39ab7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39a40cac80 con 0x7f39a415cd40 2023-06-09T11:17:12.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39ab7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41597d0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39a8d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39a40cb9a0 0x7f39a4159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57408/0 (socket says 172.21.15.7:57408) 2023-06-09T11:17:12.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39a8d9a700 1 -- 172.21.15.7:0/3135077992 learned_addr learned my addr 172.21.15.7:0/3135077992 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:12.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f39a959b700 1 --2- 172.21.15.7:0/3135077992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 0x7f39a40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:12.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238497128 0 0) 0x7f39a4159b20 con 0x7f39a40cb9a0 2023-06-09T11:17:12.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.073+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39900035b0 con 0x7f39a40cb9a0 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495950498 0 0) 0x7f39a40cac80 con 0x7f39a415cd40 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39a4159b20 con 0x7f39a415cd40 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f39a959b700 1 -- 172.21.15.7:0/3135077992 >> v1:172.21.15.117:6789/0 conn(0x7f39a415cd40 legacy=0x7f39a40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f39a959b700 1 -- 172.21.15.7:0/3135077992 >> v1:172.21.15.7:6789/0 conn(0x7f39a40cb9a0 legacy=0x7f39a4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f39a959b700 1 -- 172.21.15.7:0/3135077992 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4071ab0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f39a959b700 1 --2- 172.21.15.7:0/3135077992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 0x7f39a40b52b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=266689b2921acfc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011830 con 0x7f39a40cb5d0 2023-06-09T11:17:12.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0011990 con 0x7f39a40cb5d0 2023-06-09T11:17:12.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.074+0000 7f399bfff700 1 -- 172.21.15.7:0/3135077992 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011d20 con 0x7f39a40cb5d0 2023-06-09T11:17:12.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 -- 172.21.15.7:0/3135077992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 msgr2=0x7f39a40b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/3135077992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 0x7f39a40b52b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a0009b80 comp rx=0 tx=0).stop 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 -- 172.21.15.7:0/3135077992 shutdown_connections 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/3135077992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb5d0 0x7f39a40b52b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 -- 172.21.15.7:0/3135077992 >> 172.21.15.7:0/3135077992 conn(0x7f39a4070af0 msgr2=0x7f39a4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 -- 172.21.15.7:0/3135077992 shutdown_connections 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.075+0000 7f39ab7ff700 1 -- 172.21.15.7:0/3135077992 wait complete. 2023-06-09T11:17:12.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.077+0000 7f39ab7ff700 1 Processor -- start 2023-06-09T11:17:12.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.077+0000 7f39ab7ff700 1 -- start start 2023-06-09T11:17:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb9a0 0x7f39a4154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39a415cd40 0x7f39a40c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:12.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41ef290 con 0x7f39a40cb5d0 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39a41eeff0 con 0x7f39a415cd40 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39ab7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39a40c6a70 con 0x7f39a40cb9a0 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60260/0 (socket says 172.21.15.7:60260) 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 -- 172.21.15.7:0/2115060592 learned_addr learned my addr 172.21.15.7:0/2115060592 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a8d9a700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb9a0 0x7f39a4154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a9d9c700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39a415cd40 0x7f39a40c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:12.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 -- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb9a0 msgr2=0x7f39a4154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb9a0 0x7f39a4154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 -- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39a415cd40 msgr2=0x7f39a40c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39a415cd40 0x7f39a40c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.078+0000 7f39a959b700 1 -- 172.21.15.7:0/2115060592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a00094a0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f39a959b700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39a00090b0 tx=0x7f39a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a001a860 con 0x7f39a40cb5d0 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a415d4f0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a001a9c0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a001ad30 con 0x7f39a40cb5d0 2023-06-09T11:17:12.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.079+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a415d210 con 0x7f39a40cb5d0 2023-06-09T11:17:12.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.080+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39a415d7d0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.081+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39a0022070 con 0x7f39a40cb5d0 2023-06-09T11:17:12.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.082+0000 7f3999ffb700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:12.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.082+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6534+0+0 (secure 0 0 0) 0x7f39a008a420 con 0x7f39a40cb5d0 2023-06-09T11:17:12.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.082+0000 7f39a8d9a700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:12.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.083+0000 7f39a8d9a700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 0x7f398806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39a4156f30 tx=0x7f3994009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:12.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.086+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f39a00553a0 con 0x7f39a40cb5d0 2023-06-09T11:17:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.341+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39a40c7640 con 0x7f39a40cb5d0 2023-06-09T11:17:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.342+0000 7f3999ffb700 1 -- 172.21.15.7:0/2115060592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+6218 (secure 0 0 0) 0x7f39a0015090 con 0x7f39a40cb5d0 2023-06-09T11:17:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:12.348 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4889,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:16:36.192924+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":27,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4291,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"f","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.117:6837/2733808306","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":2733808306},{"type":"v1","addr":"172.21.15.117:6837","nonce":2733808306}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 msgr2=0x7f398806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 0x7f398806d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39a4156f30 tx=0x7f3994009040 comp rx=0 tx=0).stop 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 msgr2=0x7f39a4156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f39a00090b0 tx=0x7f39a0004750 comp rx=0 tx=0).stop 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 shutdown_connections 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39a415cd40 0x7f39a40c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39a40cb5d0 0x7f39a4156040 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f398806ace0 0x7f398806d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 --2- 172.21.15.7:0/2115060592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39a40cb9a0 0x7f39a4154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.344+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 >> 172.21.15.7:0/2115060592 conn(0x7f39a4070af0 msgr2=0x7f39a4158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.345+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 shutdown_connections 2023-06-09T11:17:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:12.345+0000 7f39ab7ff700 1 -- 172.21.15.7:0/2115060592 wait complete. 2023-06-09T11:17:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 33 2023-06-09T11:17:12.368 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:17:14.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.574+0000 7f5c117ff700 1 Processor -- start 2023-06-09T11:17:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.574+0000 7f5c117ff700 1 -- start start 2023-06-09T11:17:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.575+0000 7f5c117ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 0x7f5c0c1579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.575+0000 7f5c117ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 0x7f5c0c157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.575+0000 7f5c117ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c0c0810 con 0x7f5c0c0c5d20 2023-06-09T11:17:14.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.575+0000 7f5c117ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c15db60 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.575+0000 7f5c117ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1615a0 con 0x7f5c0c1610d0 2023-06-09T11:17:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0a7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 0x7f5c0c1579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0affd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f5c0c0c5d20 0x7f5c0c15e340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:53844/0 (socket says 172.21.15.7:53844) 2023-06-09T11:17:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0affd700 1 -- 172.21.15.7:0/3491940726 learned_addr learned my addr 172.21.15.7:0/3491940726 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0b7fe700 1 --2- 172.21.15.7:0/3491940726 >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 0x7f5c0c157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/3491940726 >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 msgr2=0x7f5c0c157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0a7fc700 1 --2- 172.21.15.7:0/3491940726 >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 0x7f5c0c157ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/3491940726 >> v1:172.21.15.117:6789/0 conn(0x7f5c0c0c5d20 legacy=0x7f5c0c15e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/3491940726 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c161880 con 0x7f5c0c1610d0 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.576+0000 7f5c0b7fe700 1 --2- 172.21.15.7:0/3491940726 >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 0x7f5c0c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c0a7fc700 1 --2- 172.21.15.7:0/3491940726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 0x7f5c0c1579b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009840 tx=0x7f5c00009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c4edb41a91941a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c09ffb700 1 -- 172.21.15.7:0/3491940726 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011830 con 0x7f5c0c1610d0 2023-06-09T11:17:14.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c09ffb700 1 -- 172.21.15.7:0/3491940726 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00011990 con 0x7f5c0c1610d0 2023-06-09T11:17:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c09ffb700 1 -- 172.21.15.7:0/3491940726 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00011d20 con 0x7f5c0c1610d0 2023-06-09T11:17:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c117ff700 1 -- 172.21.15.7:0/3491940726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 msgr2=0x7f5c0c1579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.577+0000 7f5c117ff700 1 --2- 172.21.15.7:0/3491940726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 0x7f5c0c1579b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009840 tx=0x7f5c00009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 -- 172.21.15.7:0/3491940726 shutdown_connections 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 --2- 172.21.15.7:0/3491940726 >> v2:172.21.15.117:3300/0 conn(0x7f5c0c0c60f0 0x7f5c0c157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 --2- 172.21.15.7:0/3491940726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c1610d0 0x7f5c0c1579b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 -- 172.21.15.7:0/3491940726 >> 172.21.15.7:0/3491940726 conn(0x7f5c0c06a4d0 msgr2=0x7f5c0c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 -- 172.21.15.7:0/3491940726 shutdown_connections 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.578+0000 7f5c117ff700 1 -- 172.21.15.7:0/3491940726 wait complete. 2023-06-09T11:17:14.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.579+0000 7f5c117ff700 1 Processor -- start 2023-06-09T11:17:14.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.580+0000 7f5c117ff700 1 -- start start 2023-06-09T11:17:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.580+0000 7f5c117ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c0c5d20 0x7f5c0c1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.580+0000 7f5c117ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.580+0000 7f5c117ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c0c1610d0 0x7f5c0c160660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c117ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c151d90 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c117ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c15c470 con 0x7f5c0c1610d0 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c117ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c1504b0 con 0x7f5c0c0c5d20 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34062/0 (socket says 172.21.15.7:34062) 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/2295002653 learned_addr learned my addr 172.21.15.7:0/2295002653 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0affd700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c0c5d20 0x7f5c0c1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0b7fe700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c0c1610d0 0x7f5c0c160660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c0c5d20 msgr2=0x7f5c0c1509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c0c5d20 0x7f5c0c1509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c0c1610d0 msgr2=0x7f5c0c160660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c0c1610d0 0x7f5c0c160660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.581+0000 7f5c0a7fc700 1 -- 172.21.15.7:0/2295002653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04009a70 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5c0a7fc700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009b90 tx=0x7f5c00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00004550 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c000036a0 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c000094a0 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00003a30 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.582+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c161e40 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.583+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c161b60 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.584+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c00003b90 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.585+0000 7f5bf37fe700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 0x7f5bf806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.585+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f5c0008a910 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.586+0000 7f5c0affd700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 0x7f5bf806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:14.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.586+0000 7f5c0affd700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 0x7f5bf806d1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc005910 tx=0x7f5bfc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:14.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.590+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5c00055960 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.846+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c0c15de20 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.846+0000 7f5bf37fe700 1 -- 172.21.15.7:0/2295002653 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f5c00058f80 con 0x7f5c0c0c60f0 2023-06-09T11:17:14.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:14.853 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:14.193720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 msgr2=0x7f5bf806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 0x7f5bf806d1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc005910 tx=0x7f5bfc009040 comp rx=0 tx=0).stop 2023-06-09T11:17:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 msgr2=0x7f5c0c160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009b90 tx=0x7f5c00004750 comp rx=0 tx=0).stop 2023-06-09T11:17:14.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 shutdown_connections 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c0c1610d0 0x7f5c0c160660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5bf806ad10 0x7f5bf806d1c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c0c0c5d20 0x7f5c0c1509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 --2- 172.21.15.7:0/2295002653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c0c0c60f0 0x7f5c0c160120 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 >> 172.21.15.7:0/2295002653 conn(0x7f5c0c06a4d0 msgr2=0x7f5c0c152630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 shutdown_connections 2023-06-09T11:17:14.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:14.849+0000 7f5c117ff700 1 -- 172.21.15.7:0/2295002653 wait complete. 2023-06-09T11:17:14.857 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2023-06-09T11:17:14.873 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:17:14.873 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:17:14.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:15.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.067+0000 7f0e2d1ff700 1 Processor -- start 2023-06-09T11:17:15.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.067+0000 7f0e2d1ff700 1 -- start start 2023-06-09T11:17:15.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.068+0000 7f0e2d1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 0x7f0e280b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.068+0000 7f0e2d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e28159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.068+0000 7f0e2d1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e28159b20 con 0x7f0e2815cd40 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.068+0000 7f0e2d1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e280cac80 con 0x7f0e280cb5d0 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.068+0000 7f0e2d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e281597d0 con 0x7f0e280cb9a0 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2759e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0e2815cd40 0x7f0e280cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55726/0 (socket says 172.21.15.7:55726) 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2759e700 1 -- 172.21.15.7:0/3635901836 learned_addr learned my addr 172.21.15.7:0/3635901836 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e26d9d700 1 --2- 172.21.15.7:0/3635901836 >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 0x7f0e280b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2659c700 1 --2- 172.21.15.7:0/3635901836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e28159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e25d9b700 1 -- 172.21.15.7:0/3635901836 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563552916 0 0) 0x7f0e28159b20 con 0x7f0e2815cd40 2023-06-09T11:17:15.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e25d9b700 1 -- 172.21.15.7:0/3635901836 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e100035b0 con 0x7f0e2815cd40 2023-06-09T11:17:15.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2659c700 1 -- 172.21.15.7:0/3635901836 >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 msgr2=0x7f0e280b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:15.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2659c700 1 --2- 172.21.15.7:0/3635901836 >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 0x7f0e280b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2659c700 1 -- 172.21.15.7:0/3635901836 >> v1:172.21.15.7:6789/0 conn(0x7f0e2815cd40 legacy=0x7f0e280cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e2659c700 1 -- 172.21.15.7:0/3635901836 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28071ab0 con 0x7f0e280cb9a0 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.069+0000 7f0e26d9d700 1 --2- 172.21.15.7:0/3635901836 >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 0x7f0e280b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2659c700 1 --2- 172.21.15.7:0/3635901836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e28159fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009840 tx=0x7f0e1c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=440fd458c63df77e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e25d9b700 1 -- 172.21.15.7:0/3635901836 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011830 con 0x7f0e280cb9a0 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e25d9b700 1 -- 172.21.15.7:0/3635901836 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c011990 con 0x7f0e280cb9a0 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e25d9b700 1 -- 172.21.15.7:0/3635901836 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011d20 con 0x7f0e280cb9a0 2023-06-09T11:17:15.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/3635901836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 msgr2=0x7f0e28159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/3635901836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e28159fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009840 tx=0x7f0e1c009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/3635901836 shutdown_connections 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/3635901836 >> v2:172.21.15.117:3300/0 conn(0x7f0e280cb5d0 0x7f0e280b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/3635901836 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e28159fb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/3635901836 >> 172.21.15.7:0/3635901836 conn(0x7f0e28070af0 msgr2=0x7f0e28070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:15.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/3635901836 shutdown_connections 2023-06-09T11:17:15.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.070+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/3635901836 wait complete. 2023-06-09T11:17:15.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.072+0000 7f0e2d1ff700 1 Processor -- start 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.072+0000 7f0e2d1ff700 1 -- start start 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e280cb5d0 0x7f0e280c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e280cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280c5440 con 0x7f0e2815cd40 2023-06-09T11:17:15.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280d04f0 con 0x7f0e280cb5d0 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e280c8a70 con 0x7f0e280cb9a0 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34078/0 (socket says 172.21.15.7:34078) 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.073+0000 7f0e2759e700 1 -- 172.21.15.7:0/462675962 learned_addr learned my addr 172.21.15.7:0/462675962 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2659c700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e280cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e26d9d700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e280cb5d0 0x7f0e280c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 -- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 msgr2=0x7f0e280cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e280cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 -- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e280cb5d0 msgr2=0x7f0e280c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e280cb5d0 0x7f0e280c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 -- 172.21.15.7:0/462675962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180094a0 con 0x7f0e2815cd40 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2759e700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000caa0 tx=0x7f0e2000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000a9e0 con 0x7f0e2815cd40 2023-06-09T11:17:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000ab40 con 0x7f0e2815cd40 2023-06-09T11:17:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c0094a0 con 0x7f0e2815cd40 2023-06-09T11:17:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.074+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20012400 con 0x7f0e2815cd40 2023-06-09T11:17:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.075+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2815d4f0 con 0x7f0e2815cd40 2023-06-09T11:17:15.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.076+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2815d210 con 0x7f0e2815cd40 2023-06-09T11:17:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.077+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e2001b3b0 con 0x7f0e2815cd40 2023-06-09T11:17:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.077+0000 7f0e0f7fe700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 0x7f0e1406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.078+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0e20089320 con 0x7f0e2815cd40 2023-06-09T11:17:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.078+0000 7f0e26d9d700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 0x7f0e1406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:15.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.078+0000 7f0e26d9d700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 0x7f0e1406d1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009d00 tx=0x7f0e18008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:15.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.082+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0e200541e0 con 0x7f0e2815cd40 2023-06-09T11:17:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.335+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e281efff0 con 0x7f0e2815cd40 2023-06-09T11:17:15.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.335+0000 7f0e0f7fe700 1 -- 172.21.15.7:0/462675962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f0e20057800 con 0x7f0e2815cd40 2023-06-09T11:17:15.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:15.342 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:14.193720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:15.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 msgr2=0x7f0e1406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 0x7f0e1406d1c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009d00 tx=0x7f0e18008040 comp rx=0 tx=0).stop 2023-06-09T11:17:15.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 msgr2=0x7f0e280b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:15.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000caa0 tx=0x7f0e2000ce60 comp rx=0 tx=0).stop 2023-06-09T11:17:15.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 shutdown_connections 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e2815cd40 0x7f0e280b0ea0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e280cb5d0 0x7f0e280c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e1406ad10 0x7f0e1406d1c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 --2- 172.21.15.7:0/462675962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e280cb9a0 0x7f0e280cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 >> 172.21.15.7:0/462675962 conn(0x7f0e28070af0 msgr2=0x7f0e280c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 shutdown_connections 2023-06-09T11:17:15.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:15.338+0000 7f0e2d1ff700 1 -- 172.21.15.7:0/462675962 wait complete. 2023-06-09T11:17:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2023-06-09T11:17:15.363 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 34, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 34, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4889": { "addr": "172.21.15.117:6835/1617501683", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1617501683, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1617501683, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4889, "incarnation": 34, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:17:14.193720+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4889, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:17:17.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.570+0000 7fc8713ff700 1 Processor -- start 2023-06-09T11:17:17.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.570+0000 7fc8713ff700 1 -- start start 2023-06-09T11:17:17.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc8713ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc86c153880 0x7fc86c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc8713ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc86c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc8713ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c15a500 con 0x7fc86c0c9f40 2023-06-09T11:17:17.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc8713ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0c5f10 con 0x7fc86c153880 2023-06-09T11:17:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc8713ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c15da00 con 0x7fc86c15b4a0 2023-06-09T11:17:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc86a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc86c0c9f40 0x7fc86c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55734/0 (socket says 172.21.15.7:55734) 2023-06-09T11:17:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc86a7fc700 1 -- 172.21.15.7:0/502568602 learned_addr learned my addr 172.21.15.7:0/502568602 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:17.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> v2:172.21.15.117:3300/0 conn(0x7fc86c153880 0x7fc86c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc86c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183869410 0 0) 0x7fc860001110 con 0x7fc86c0c9f40 2023-06-09T11:17:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.571+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8540035b0 con 0x7fc86c0c9f40 2023-06-09T11:17:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934148651 0 0) 0x7fc8540035b0 con 0x7fc86c0c9f40 2023-06-09T11:17:17.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 msgr2=0x7fc86c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:17.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc86c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 >> v2:172.21.15.117:3300/0 conn(0x7fc86c153880 msgr2=0x7fc86c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:17.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 --2- 172.21.15.7:0/502568602 >> v2:172.21.15.117:3300/0 conn(0x7fc86c153880 0x7fc86c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c1596e0 con 0x7fc86c0c9f40 2023-06-09T11:17:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fc860001800 con 0x7fc86c0c9f40 2023-06-09T11:17:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> v2:172.21.15.117:3300/0 conn(0x7fc86c153880 0x7fc86c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:17.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 >> v1:172.21.15.7:6789/0 conn(0x7fc86c0c9f40 legacy=0x7fc86c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854003a00 0x7fc854003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc854004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc854004d90 0x7fc854005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0c5f10 con 0x7fc854003a00 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c15da00 con 0x7fc854004d90 2023-06-09T11:17:17.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85400ba10 con 0x7fc86c15b4a0 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854003a00 0x7fc854003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc86a7fc700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc854004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.572+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc854004d90 0x7fc854005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 -- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 msgr2=0x7fc854004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc86c15b4a0 0x7fc854004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 -- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc854004d90 msgr2=0x7fc854005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:17.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc854004d90 0x7fc854005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 -- 172.21.15.7:0/502568602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540035b0 con 0x7fc854003a00 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc86affd700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854003a00 0x7fc854003e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009d30 tx=0x7fc854005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1617f0c2a34a09b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c009e80 con 0x7fc854003a00 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c00e9a0 con 0x7fc854003a00 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.573+0000 7fc869ffb700 1 -- 172.21.15.7:0/502568602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00ed30 con 0x7fc854003a00 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 -- 172.21.15.7:0/502568602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854003a00 msgr2=0x7fc854003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 --2- 172.21.15.7:0/502568602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc854003a00 0x7fc854003e10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009d30 tx=0x7fc854005880 comp rx=0 tx=0).stop 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc86b7fe700 1 -- 172.21.15.7:0/502568602 reap_dead start 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 -- 172.21.15.7:0/502568602 shutdown_connections 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 -- 172.21.15.7:0/502568602 >> 172.21.15.7:0/502568602 conn(0x7fc86c0737f0 msgr2=0x7fc86c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 -- 172.21.15.7:0/502568602 shutdown_connections 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.574+0000 7fc8713ff700 1 -- 172.21.15.7:0/502568602 wait complete. 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.575+0000 7fc8713ff700 1 Processor -- start 2023-06-09T11:17:17.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.576+0000 7fc8713ff700 1 -- start start 2023-06-09T11:17:17.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.576+0000 7fc8713ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc854003a00 0x7fc86c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.576+0000 7fc8713ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c1f0030 0x7fc86c0c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc8713ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc8713ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c151a10 con 0x7fc86c0caa80 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc8713ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1ef340 con 0x7fc86c1f0030 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc8713ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1efba0 con 0x7fc854003a00 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc854003a00 0x7fc86c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34090/0 (socket says 172.21.15.7:34090) 2023-06-09T11:17:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 -- 172.21.15.7:0/513022388 learned_addr learned my addr 172.21.15.7:0/513022388 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86a7fc700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c1f0030 0x7fc86c0c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 -- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc854003a00 msgr2=0x7fc86c150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc854003a00 0x7fc86c150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 -- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c1f0030 msgr2=0x7fc86c0c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c1f0030 0x7fc86c0c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.577+0000 7fc86b7fe700 1 -- 172.21.15.7:0/513022388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c159c60 con 0x7fc86c0caa80 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc86b7fe700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc86c072cb0 tx=0x7fc8640111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400e070 con 0x7fc86c0caa80 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864003cf0 con 0x7fc86c0caa80 2023-06-09T11:17:17.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864003ea0 con 0x7fc86c0caa80 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c159980 con 0x7fc86c0caa80 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.578+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c159f40 con 0x7fc86c0caa80 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.579+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c159420 con 0x7fc86c0caa80 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.580+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc864011740 con 0x7fc86c0caa80 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.581+0000 7fc8537fe700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 0x7fc85806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:17.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.581+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc864089bd0 con 0x7fc86c0caa80 2023-06-09T11:17:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.581+0000 7fc86affd700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 0x7fc85806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.582+0000 7fc86affd700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 0x7fc85806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc85c000c00 tx=0x7fc85c01f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:17.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.586+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc8640549f0 con 0x7fc86c0caa80 2023-06-09T11:17:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.841+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc86c074cf0 con 0x7fc86c0caa80 2023-06-09T11:17:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.841+0000 7fc8537fe700 1 -- 172.21.15.7:0/513022388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fc864058010 con 0x7fc86c0caa80 2023-06-09T11:17:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:17.848 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:14.193720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 msgr2=0x7fc85806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 0x7fc85806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc85c000c00 tx=0x7fc85c01f040 comp rx=0 tx=0).stop 2023-06-09T11:17:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 msgr2=0x7fc86c0c5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc86c072cb0 tx=0x7fc8640111d0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 shutdown_connections 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc86c0caa80 0x7fc86c0c5160 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc85806ace0 0x7fc85806d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc854003a00 0x7fc86c150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 --2- 172.21.15.7:0/513022388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc86c1f0030 0x7fc86c0c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 >> 172.21.15.7:0/513022388 conn(0x7fc86c0737f0 msgr2=0x7fc86c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 shutdown_connections 2023-06-09T11:17:17.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:17.844+0000 7fc8713ff700 1 -- 172.21.15.7:0/513022388 wait complete. 2023-06-09T11:17:17.852 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2023-06-09T11:17:17.868 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:19.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.083+0000 7f8e495ff700 1 Processor -- start 2023-06-09T11:17:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.083+0000 7f8e495ff700 1 -- start start 2023-06-09T11:17:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e441593d0 0x7f8e441597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e440c08d0 0x7f8e44159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8e44071910 0x7f8e44071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440714f0 con 0x7f8e440c08d0 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440cc140 con 0x7f8e44071910 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e495ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440c1830 con 0x7f8e441593d0 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e427fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e440c08d0 0x7f8e44159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e427fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e440c08d0 0x7f8e44159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34098/0 (socket says 172.21.15.7:34098) 2023-06-09T11:17:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e427fc700 1 -- 172.21.15.7:0/471772517 learned_addr learned my addr 172.21.15.7:0/471772517 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e42ffd700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e441593d0 0x7f8e441597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.085+0000 7f8e437fe700 1 --2- 172.21.15.7:0/471772517 >> v2:172.21.15.117:3300/0 conn(0x7f8e44071910 0x7f8e44071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 -- 172.21.15.7:0/471772517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e441593d0 msgr2=0x7f8e441597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e441593d0 0x7f8e441597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 -- 172.21.15.7:0/471772517 >> v2:172.21.15.117:3300/0 conn(0x7f8e44071910 msgr2=0x7f8e44071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 --2- 172.21.15.7:0/471772517 >> v2:172.21.15.117:3300/0 conn(0x7f8e44071910 0x7f8e44071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 -- 172.21.15.7:0/471772517 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440c0560 con 0x7f8e440c08d0 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e427fc700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e440c08d0 0x7f8e44159ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c9050 tx=0x7f8e3800ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a8d8ea51d015da server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e41ffb700 1 -- 172.21.15.7:0/471772517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004230 con 0x7f8e440c08d0 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e41ffb700 1 -- 172.21.15.7:0/471772517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e380043b0 con 0x7f8e440c08d0 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.086+0000 7f8e41ffb700 1 -- 172.21.15.7:0/471772517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380038e0 con 0x7f8e440c08d0 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 -- 172.21.15.7:0/471772517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e440c08d0 msgr2=0x7f8e44159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e440c08d0 0x7f8e44159ce0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c9050 tx=0x7f8e3800ac80 comp rx=0 tx=0).stop 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 -- 172.21.15.7:0/471772517 shutdown_connections 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 --2- 172.21.15.7:0/471772517 >> v2:172.21.15.117:3300/0 conn(0x7f8e44071910 0x7f8e44071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e441593d0 0x7f8e441597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 --2- 172.21.15.7:0/471772517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e440c08d0 0x7f8e44159ce0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 -- 172.21.15.7:0/471772517 >> 172.21.15.7:0/471772517 conn(0x7f8e4406a4d0 msgr2=0x7f8e4406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 -- 172.21.15.7:0/471772517 shutdown_connections 2023-06-09T11:17:20.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.087+0000 7f8e495ff700 1 -- 172.21.15.7:0/471772517 wait complete. 2023-06-09T11:17:20.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.089+0000 7f8e495ff700 1 Processor -- start 2023-06-09T11:17:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.089+0000 7f8e495ff700 1 -- start start 2023-06-09T11:17:20.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e44071910 0x7f8e4414db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e440c08d0 0x7f8e440c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441e1080 con 0x7f8e441593d0 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441e1520 con 0x7f8e440c08d0 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e495ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e441e1b40 con 0x7f8e44071910 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34110/0 (socket says 172.21.15.7:34110) 2023-06-09T11:17:20.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 -- 172.21.15.7:0/1743755021 learned_addr learned my addr 172.21.15.7:0/1743755021 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e42ffd700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e44071910 0x7f8e4414db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e427fc700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e440c08d0 0x7f8e440c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 -- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e44071910 msgr2=0x7f8e4414db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e44071910 0x7f8e4414db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 -- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e440c08d0 msgr2=0x7f8e440c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e440c08d0 0x7f8e440c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.090+0000 7f8e437fe700 1 -- 172.21.15.7:0/1743755021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3800a600 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e437fe700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c011e80 tx=0x7f8e3c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c009bf0 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00e040 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c015440 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440c1a80 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.091+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e44072290 con 0x7f8e441593d0 2023-06-09T11:17:20.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.092+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e440c1d60 con 0x7f8e441593d0 2023-06-09T11:17:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.093+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e3c0155c0 con 0x7f8e441593d0 2023-06-09T11:17:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.094+0000 7f8e2b7fe700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 0x7f8e3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:20.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.094+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f8e3c08bec0 con 0x7f8e441593d0 2023-06-09T11:17:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.094+0000 7f8e42ffd700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 0x7f8e3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:20.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.095+0000 7f8e42ffd700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 0x7f8e3006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009b40 tx=0x7f8e34009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:20.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.099+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8e3c05a760 con 0x7f8e441593d0 2023-06-09T11:17:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.352+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e441ea5b0 con 0x7f8e441593d0 2023-06-09T11:17:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.352+0000 7f8e2b7fe700 1 -- 172.21.15.7:0/1743755021 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f8e441ea5b0 con 0x7f8e441593d0 2023-06-09T11:17:20.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:20.358 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:14.193720+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 msgr2=0x7f8e3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 0x7f8e3006d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009b40 tx=0x7f8e34009210 comp rx=0 tx=0).stop 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 msgr2=0x7f8e441ea890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c011e80 tx=0x7f8e3c00f4a0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 shutdown_connections 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e441593d0 0x7f8e441ea890 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e3006ace0 0x7f8e3006d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e44071910 0x7f8e4414db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 --2- 172.21.15.7:0/1743755021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e440c08d0 0x7f8e440c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 >> 172.21.15.7:0/1743755021 conn(0x7f8e4406a4d0 msgr2=0x7f8e440c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 shutdown_connections 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:20.355+0000 7f8e495ff700 1 -- 172.21.15.7:0/1743755021 wait complete. 2023-06-09T11:17:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 34 2023-06-09T11:17:20.378 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:22.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:22.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.615+0000 7f2b03dff700 1 Processor -- start 2023-06-09T11:17:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.616+0000 7f2b03dff700 1 -- start start 2023-06-09T11:17:22.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b03dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b03dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2afc15cd40 0x7f2afc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b03dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2afc159b20 con 0x7f2afc0cb9a0 2023-06-09T11:17:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b03dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0cac80 con 0x7f2afc15cd40 2023-06-09T11:17:22.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b03dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1597d0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b01b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b01b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46180/0 (socket says 172.21.15.7:46180) 2023-06-09T11:17:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b01b9b700 1 -- 172.21.15.7:0/509893231 learned_addr learned my addr 172.21.15.7:0/509893231 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:22.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.617+0000 7f2b0239c700 1 --2- 172.21.15.7:0/509893231 >> v2:172.21.15.117:3300/0 conn(0x7f2afc15cd40 0x7f2afc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b01b9b700 1 -- 172.21.15.7:0/509893231 >> v2:172.21.15.117:3300/0 conn(0x7f2afc15cd40 msgr2=0x7f2afc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b01b9b700 1 --2- 172.21.15.7:0/509893231 >> v2:172.21.15.117:3300/0 conn(0x7f2afc15cd40 0x7f2afc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b01b9b700 1 -- 172.21.15.7:0/509893231 >> v1:172.21.15.117:6789/0 conn(0x7f2afc0cb9a0 legacy=0x7f2afc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b01b9b700 1 -- 172.21.15.7:0/509893231 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc071ab0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b00b99700 1 -- 172.21.15.7:0/509893231 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749369858 0 0) 0x7f2afc159b20 con 0x7f2afc0cb9a0 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b01b9b700 1 --2- 172.21.15.7:0/509893231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2af80098f0 tx=0x7f2af8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a8e67b92c79d3e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:22.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b00b99700 1 -- 172.21.15.7:0/509893231 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8011830 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b00b99700 1 -- 172.21.15.7:0/509893231 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8011990 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.618+0000 7f2b00b99700 1 -- 172.21.15.7:0/509893231 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8011d20 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 -- 172.21.15.7:0/509893231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 msgr2=0x7f2afc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 --2- 172.21.15.7:0/509893231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2af80098f0 tx=0x7f2af8009c00 comp rx=0 tx=0).stop 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 -- 172.21.15.7:0/509893231 shutdown_connections 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 --2- 172.21.15.7:0/509893231 >> v2:172.21.15.117:3300/0 conn(0x7f2afc15cd40 0x7f2afc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 --2- 172.21.15.7:0/509893231 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc0b52b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 -- 172.21.15.7:0/509893231 >> 172.21.15.7:0/509893231 conn(0x7f2afc070af0 msgr2=0x7f2afc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 -- 172.21.15.7:0/509893231 shutdown_connections 2023-06-09T11:17:22.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.619+0000 7f2b03dff700 1 -- 172.21.15.7:0/509893231 wait complete. 2023-06-09T11:17:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.621+0000 7f2b03dff700 1 Processor -- start 2023-06-09T11:17:22.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.622+0000 7f2b03dff700 1 -- start start 2023-06-09T11:17:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc1f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc0cb9a0 0x7f2afc1f0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ad340 con 0x7f2afc15cd40 2023-06-09T11:17:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc1f0610 con 0x7f2afc0cb9a0 2023-06-09T11:17:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b03dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0abf00 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b0239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b01b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc1f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b0239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34114/0 (socket says 172.21.15.7:34114) 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b0239c700 1 -- 172.21.15.7:0/2827795503 learned_addr learned my addr 172.21.15.7:0/2827795503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.623+0000 7f2b0139a700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc0cb9a0 0x7f2afc1f0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 -- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc0cb9a0 msgr2=0x7f2afc1f0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc0cb9a0 0x7f2afc1f0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 -- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 msgr2=0x7f2afc1539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 -- 172.21.15.7:0/2827795503 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af8009550 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b0239c700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b01b9b700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc1f0aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2af8016000 tx=0x7f2af8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8004400 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af801f070 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc15d4f0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.624+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af80038e0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.625+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc15d210 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.625+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc15d7d0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.627+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2af8021030 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.628+0000 7f2af27fc700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 0x7f2ae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.628+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f2af808add0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.628+0000 7f2b0139a700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 0x7f2ae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.628+0000 7f2b0139a700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 0x7f2ae006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0ca800 tx=0x7f2aec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.631+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2af8055d00 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.881+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2afc074cf0 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.882+0000 7f2af27fc700 1 -- 172.21.15.7:0/2827795503 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5458 (secure 0 0 0) 0x7f2af8059320 con 0x7f2afc0cb5d0 2023-06-09T11:17:22.887 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:22.888 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:21.220544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:22.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 msgr2=0x7f2ae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 0x7f2ae006d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0ca800 tx=0x7f2aec009040 comp rx=0 tx=0).stop 2023-06-09T11:17:22.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 msgr2=0x7f2afc1f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:22.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc1f0aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2af8016000 tx=0x7f2af8004750 comp rx=0 tx=0).stop 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 shutdown_connections 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2afc15cd40 0x7f2afc1539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ae006ace0 0x7f2ae006d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2afc0cb5d0 0x7f2afc1f0aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 --2- 172.21.15.7:0/2827795503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2afc0cb9a0 0x7f2afc1f0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 >> 172.21.15.7:0/2827795503 conn(0x7f2afc070af0 msgr2=0x7f2afc15b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 shutdown_connections 2023-06-09T11:17:22.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:22.885+0000 7f2b03dff700 1 -- 172.21.15.7:0/2827795503 wait complete. 2023-06-09T11:17:22.892 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2023-06-09T11:17:22.908 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:24.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.115+0000 7fc40bdff700 1 Processor -- start 2023-06-09T11:17:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.116+0000 7fc40bdff700 1 -- start start 2023-06-09T11:17:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 0x7fc4040c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40bdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4040c5a80 con 0x7fc40415b4a0 2023-06-09T11:17:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40bdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc40415a500 con 0x7fc404153880 2023-06-09T11:17:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4040c5f10 con 0x7fc4040c9f40 2023-06-09T11:17:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40939a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc40415b4a0 0x7fc4040c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60016/0 (socket says 172.21.15.7:60016) 2023-06-09T11:17:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40939a700 1 -- 172.21.15.7:0/3201488108 learned_addr learned my addr 172.21.15.7:0/3201488108 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc40a39c700 1 --2- 172.21.15.7:0/3201488108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 0x7fc4040c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.117+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3417645938 0 0) 0x7fc4040c5a80 con 0x7fc40415b4a0 2023-06-09T11:17:25.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f00035b0 con 0x7fc40415b4a0 2023-06-09T11:17:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4023057550 0 0) 0x7fc400001110 con 0x7fc404153880 2023-06-09T11:17:25.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4040c5a80 con 0x7fc404153880 2023-06-09T11:17:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc40a39c700 1 -- 172.21.15.7:0/3201488108 >> v1:172.21.15.117:6789/0 conn(0x7fc404153880 legacy=0x7fc40415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc40a39c700 1 -- 172.21.15.7:0/3201488108 >> v1:172.21.15.7:6789/0 conn(0x7fc40415b4a0 legacy=0x7fc4040c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc40a39c700 1 -- 172.21.15.7:0/3201488108 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4041596e0 con 0x7fc4040c9f40 2023-06-09T11:17:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc40a39c700 1 --2- 172.21.15.7:0/3201488108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 0x7fc4040c5540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009dc0 tx=0x7fc3fc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b36c708e5b18e5b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:25.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc00e070 con 0x7fc4040c9f40 2023-06-09T11:17:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc00bb20 con 0x7fc4040c9f40 2023-06-09T11:17:25.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.118+0000 7fc408b99700 1 -- 172.21.15.7:0/3201488108 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc012430 con 0x7fc4040c9f40 2023-06-09T11:17:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 -- 172.21.15.7:0/3201488108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 msgr2=0x7fc4040c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3201488108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 0x7fc4040c5540 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc009dc0 tx=0x7fc3fc00b330 comp rx=0 tx=0).stop 2023-06-09T11:17:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 -- 172.21.15.7:0/3201488108 shutdown_connections 2023-06-09T11:17:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3201488108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4040c9f40 0x7fc4040c5540 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 -- 172.21.15.7:0/3201488108 >> 172.21.15.7:0/3201488108 conn(0x7fc4040737f0 msgr2=0x7fc4040720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:25.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 -- 172.21.15.7:0/3201488108 shutdown_connections 2023-06-09T11:17:25.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.119+0000 7fc40bdff700 1 -- 172.21.15.7:0/3201488108 wait complete. 2023-06-09T11:17:25.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.121+0000 7fc40bdff700 1 Processor -- start 2023-06-09T11:17:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.121+0000 7fc40bdff700 1 -- start start 2023-06-09T11:17:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4040c9f40 0x7fc4040b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 0x7fc4040ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:25.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041ef520 con 0x7fc40415b4a0 2023-06-09T11:17:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041e4030 con 0x7fc4040c9f40 2023-06-09T11:17:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4041e3d50 con 0x7fc404153880 2023-06-09T11:17:25.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51438/0 (socket says 172.21.15.7:51438) 2023-06-09T11:17:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40a39c700 1 -- 172.21.15.7:0/3946423596 learned_addr learned my addr 172.21.15.7:0/3946423596 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc409b9b700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4040c9f40 0x7fc4040b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.122+0000 7fc40939a700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 0x7fc4040ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 -- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 msgr2=0x7fc4040ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 0x7fc4040ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 -- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4040c9f40 msgr2=0x7fc4040b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4040c9f40 0x7fc4040b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 -- 172.21.15.7:0/3946423596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc009a70 con 0x7fc40415b4a0 2023-06-09T11:17:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40939a700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 0x7fc4040ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40a39c700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b370 tx=0x7fc3fc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc01b810 con 0x7fc40415b4a0 2023-06-09T11:17:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3fc020070 con 0x7fc40415b4a0 2023-06-09T11:17:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc404159c60 con 0x7fc40415b4a0 2023-06-09T11:17:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc01bbf0 con 0x7fc40415b4a0 2023-06-09T11:17:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.123+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc404159980 con 0x7fc40415b4a0 2023-06-09T11:17:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.124+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc404159f40 con 0x7fc40415b4a0 2023-06-09T11:17:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.125+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3fc01bd50 con 0x7fc40415b4a0 2023-06-09T11:17:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.126+0000 7fc3fa7fc700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.126+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fc3fc022070 con 0x7fc40415b4a0 2023-06-09T11:17:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.127+0000 7fc409b9b700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:25.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.127+0000 7fc409b9b700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 0x7fc3e806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4000013b0 tx=0x7fc400009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.131+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc3fc056d00 con 0x7fc40415b4a0 2023-06-09T11:17:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.388+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4041ef6b0 con 0x7fc40415b4a0 2023-06-09T11:17:25.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.389+0000 7fc3fa7fc700 1 -- 172.21.15.7:0/3946423596 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5458 (secure 0 0 0) 0x7fc3fc05a320 con 0x7fc40415b4a0 2023-06-09T11:17:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:25.394 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:21.220544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 msgr2=0x7fc3e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 0x7fc3e806d190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4000013b0 tx=0x7fc400009040 comp rx=0 tx=0).stop 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 msgr2=0x7fc404151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc00b370 tx=0x7fc3fc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 shutdown_connections 2023-06-09T11:17:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc40415b4a0 0x7fc404151b10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4040c9f40 0x7fc4040b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3e806ace0 0x7fc3e806d190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 --2- 172.21.15.7:0/3946423596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc404153880 0x7fc4040ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 >> 172.21.15.7:0/3946423596 conn(0x7fc4040737f0 msgr2=0x7fc40414dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 shutdown_connections 2023-06-09T11:17:25.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:25.391+0000 7fc40bdff700 1 -- 172.21.15.7:0/3946423596 wait complete. 2023-06-09T11:17:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2023-06-09T11:17:25.414 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:27.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:27.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.620+0000 7f7a26bff700 1 Processor -- start 2023-06-09T11:17:27.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.620+0000 7f7a26bff700 1 -- start start 2023-06-09T11:17:27.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a26bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7a2015b4a0 0x7f7a2015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:27.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a26bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2015a500 con 0x7f7a200c9f40 2023-06-09T11:17:27.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a26bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a2015da00 con 0x7f7a20153880 2023-06-09T11:17:27.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a26bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a200c5f10 con 0x7f7a2015b4a0 2023-06-09T11:17:27.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a2519c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a200c9f40 0x7f7a200c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60032/0 (socket says 172.21.15.7:60032) 2023-06-09T11:17:27.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a2519c700 1 -- 172.21.15.7:0/1532200579 learned_addr learned my addr 172.21.15.7:0/1532200579 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:27.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a1ffff700 1 --2- 172.21.15.7:0/1532200579 >> v2:172.21.15.117:3300/0 conn(0x7f7a2015b4a0 0x7f7a2015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:27.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201333595 0 0) 0x7f7a18001490 con 0x7f7a200c9f40 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.622+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a100035b0 con 0x7f7a200c9f40 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921309927 0 0) 0x7f7a14001110 con 0x7f7a20153880 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a18001490 con 0x7f7a20153880 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1ffff700 1 -- 172.21.15.7:0/1532200579 >> v1:172.21.15.117:6789/0 conn(0x7f7a20153880 legacy=0x7f7a2015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1ffff700 1 -- 172.21.15.7:0/1532200579 >> v1:172.21.15.7:6789/0 conn(0x7f7a200c9f40 legacy=0x7f7a200c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1ffff700 1 -- 172.21.15.7:0/1532200579 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a201596e0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1ffff700 1 --2- 172.21.15.7:0/1532200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2015b4a0 0x7f7a2015d4c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009840 tx=0x7f7a0c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b6906870ab55291 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:27.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011830 con 0x7f7a2015b4a0 2023-06-09T11:17:27.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0c011990 con 0x7f7a2015b4a0 2023-06-09T11:17:27.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.623+0000 7f7a1f7fe700 1 -- 172.21.15.7:0/1532200579 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0c011d20 con 0x7f7a2015b4a0 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 -- 172.21.15.7:0/1532200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2015b4a0 msgr2=0x7f7a2015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1532200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2015b4a0 0x7f7a2015d4c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009840 tx=0x7f7a0c009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 -- 172.21.15.7:0/1532200579 shutdown_connections 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1532200579 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a2015b4a0 0x7f7a2015d4c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 -- 172.21.15.7:0/1532200579 >> 172.21.15.7:0/1532200579 conn(0x7f7a200737f0 msgr2=0x7f7a200720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:27.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 -- 172.21.15.7:0/1532200579 shutdown_connections 2023-06-09T11:17:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.624+0000 7f7a26bff700 1 -- 172.21.15.7:0/1532200579 wait complete. 2023-06-09T11:17:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.626+0000 7f7a26bff700 1 Processor -- start 2023-06-09T11:17:27.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.626+0000 7f7a26bff700 1 -- start start 2023-06-09T11:17:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a200c9f40 0x7f7a200b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:27.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a20153880 0x7f7a201f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a201542e0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20154e50 con 0x7f7a20153880 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a26bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a201f02f0 con 0x7f7a200c9f40 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51440/0 (socket says 172.21.15.7:51440) 2023-06-09T11:17:27.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 -- 172.21.15.7:0/1350938014 learned_addr learned my addr 172.21.15.7:0/1350938014 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a1ffff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a20153880 0x7f7a201f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2499b700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a200c9f40 0x7f7a200b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 -- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a200c9f40 msgr2=0x7f7a200b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a200c9f40 0x7f7a200b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 -- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a20153880 msgr2=0x7f7a201f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a20153880 0x7f7a201f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.627+0000 7f7a2519c700 1 -- 172.21.15.7:0/1350938014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0094a0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a2519c700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a18001f50 tx=0x7f7a1800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800ab70 con 0x7f7a2015b4a0 2023-06-09T11:17:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a1800acd0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18012600 con 0x7f7a2015b4a0 2023-06-09T11:17:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a20159c60 con 0x7f7a2015b4a0 2023-06-09T11:17:27.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.628+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a20159980 con 0x7f7a2015b4a0 2023-06-09T11:17:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.629+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20159f40 con 0x7f7a2015b4a0 2023-06-09T11:17:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.630+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a1801b3b0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.631+0000 7f7a1d7fa700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:27.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.631+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f7a180892d0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.632+0000 7f7a2499b700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:27.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.632+0000 7f7a2499b700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 0x7f7a0406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001370 tx=0x7f7a14009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.635+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7a18057ba0 con 0x7f7a2015b4a0 2023-06-09T11:17:27.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.888+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a2015a500 con 0x7f7a2015b4a0 2023-06-09T11:17:27.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.888+0000 7f7a1d7fa700 1 -- 172.21.15.7:0/1350938014 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5460 (secure 0 0 0) 0x7f7a2015a500 con 0x7f7a2015b4a0 2023-06-09T11:17:27.893 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:27.893 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:27.240670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:27.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.890+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 msgr2=0x7f7a0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.890+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 0x7f7a0406d190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7a14001370 tx=0x7f7a14009040 comp rx=0 tx=0).stop 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 msgr2=0x7f7a201f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f7a18001f50 tx=0x7f7a1800a060 comp rx=0 tx=0).stop 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 shutdown_connections 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a2015b4a0 0x7f7a201f2930 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7a0406ace0 0x7f7a0406d190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7a200c9f40 0x7f7a200b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 --2- 172.21.15.7:0/1350938014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a20153880 0x7f7a201f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 >> 172.21.15.7:0/1350938014 conn(0x7f7a200737f0 msgr2=0x7f7a2014d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 shutdown_connections 2023-06-09T11:17:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:27.891+0000 7f7a26bff700 1 -- 172.21.15.7:0/1350938014 wait complete. 2023-06-09T11:17:27.897 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2023-06-09T11:17:27.914 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 36, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 36, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4889": { "addr": "172.21.15.117:6835/1617501683", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1617501683, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1617501683, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4889, "incarnation": 34, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 27 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:17:27.240670+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4889, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:17:29.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:30.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.120+0000 7fe4201ff700 1 Processor -- start 2023-06-09T11:17:30.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.120+0000 7fe4201ff700 1 -- start start 2023-06-09T11:17:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe4201ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe4180c9090 0x7fe4180c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe4201ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4181598b0 0x7fe418159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe4201ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41815cec0 con 0x7fe4180cad00 2023-06-09T11:17:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe4201ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41815d160 con 0x7fe4180c9090 2023-06-09T11:17:30.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe4201ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41815d440 con 0x7fe4181598b0 2023-06-09T11:17:30.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe41d79a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe4180cad00 0x7fe4180cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60048/0 (socket says 172.21.15.7:60048) 2023-06-09T11:17:30.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.121+0000 7fe41d79a700 1 -- 172.21.15.7:0/3939270227 learned_addr learned my addr 172.21.15.7:0/3939270227 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:30.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 --2- 172.21.15.7:0/3939270227 >> v2:172.21.15.117:3300/0 conn(0x7fe4180c9090 0x7fe4180c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41e79c700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4181598b0 0x7fe418159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41cf99700 1 -- 172.21.15.7:0/3939270227 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763770132 0 0) 0x7fe41815cec0 con 0x7fe4180cad00 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41cf99700 1 -- 172.21.15.7:0/3939270227 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4040035b0 con 0x7fe4180cad00 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 -- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4181598b0 msgr2=0x7fe418159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4181598b0 0x7fe418159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 -- 172.21.15.7:0/3939270227 >> v1:172.21.15.7:6789/0 conn(0x7fe4180cad00 legacy=0x7fe4180cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 -- 172.21.15.7:0/3939270227 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41815d720 con 0x7fe4180c9090 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.122+0000 7fe41df9b700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4180c9fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe414009840 tx=0x7fe414009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53bbc003c788e9a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe41cf99700 1 -- 172.21.15.7:0/3939270227 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414011830 con 0x7fe4180c9090 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe41cf99700 1 -- 172.21.15.7:0/3939270227 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414011990 con 0x7fe4180c9090 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe41cf99700 1 -- 172.21.15.7:0/3939270227 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414011d00 con 0x7fe4180c9090 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 -- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 msgr2=0x7fe4180c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4180c9fe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe414009840 tx=0x7fe414009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 -- 172.21.15.7:0/3939270227 shutdown_connections 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4181598b0 0x7fe418159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 --2- 172.21.15.7:0/3939270227 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4180c9fe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 -- 172.21.15.7:0/3939270227 >> 172.21.15.7:0/3939270227 conn(0x7fe418070af0 msgr2=0x7fe418070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 -- 172.21.15.7:0/3939270227 shutdown_connections 2023-06-09T11:17:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.123+0000 7fe4201ff700 1 -- 172.21.15.7:0/3939270227 wait complete. 2023-06-09T11:17:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.125+0000 7fe4201ff700 1 Processor -- start 2023-06-09T11:17:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.125+0000 7fe4201ff700 1 -- start start 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4181efe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4180cad00 0x7fe4180c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ce690 con 0x7fe4181598b0 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181ef9d0 con 0x7fe4180c9090 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe4201ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41815a3e0 con 0x7fe4180cad00 2023-06-09T11:17:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe41e79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe41e79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51458/0 (socket says 172.21.15.7:51458) 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.126+0000 7fe41e79c700 1 -- 172.21.15.7:0/2283237674 learned_addr learned my addr 172.21.15.7:0/2283237674 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41df9b700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4181efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41d79a700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4180cad00 0x7fe4180c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 -- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4180cad00 msgr2=0x7fe4180c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4180cad00 0x7fe4180c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 -- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 msgr2=0x7fe4181efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4181efe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 -- 172.21.15.7:0/2283237674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140094a0 con 0x7fe4181598b0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe41e79c700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe41000efc0 tx=0x7fe41000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000cb80 con 0x7fe4181598b0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41000f040 con 0x7fe4181598b0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41815a250 con 0x7fe4181598b0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.127+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410014480 con 0x7fe4181598b0 2023-06-09T11:17:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.128+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4180ca620 con 0x7fe4181598b0 2023-06-09T11:17:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.128+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4180cb160 con 0x7fe4181598b0 2023-06-09T11:17:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.129+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4100145e0 con 0x7fe4181598b0 2023-06-09T11:17:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.130+0000 7fe40e7fc700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 0x7fe3fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.131+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fe41008a1c0 con 0x7fe4181598b0 2023-06-09T11:17:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.131+0000 7fe41df9b700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 0x7fe3fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.131+0000 7fe41df9b700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 0x7fe3fc06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe414000c00 tx=0x7fe41401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.135+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe410055060 con 0x7fe4181598b0 2023-06-09T11:17:30.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.393+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe41815a9e0 con 0x7fe4181598b0 2023-06-09T11:17:30.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.393+0000 7fe40e7fc700 1 -- 172.21.15.7:0/2283237674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5460 (secure 0 0 0) 0x7fe410058680 con 0x7fe4181598b0 2023-06-09T11:17:30.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:30.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:27.240670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 msgr2=0x7fe3fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 0x7fe3fc06d190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe414000c00 tx=0x7fe41401c040 comp rx=0 tx=0).stop 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 msgr2=0x7fe4180c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe41000efc0 tx=0x7fe41000c0c0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 shutdown_connections 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4181598b0 0x7fe4180c84f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4180c9090 0x7fe4181efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3fc06ace0 0x7fe3fc06d190 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 --2- 172.21.15.7:0/2283237674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4180cad00 0x7fe4180c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 >> 172.21.15.7:0/2283237674 conn(0x7fe418070af0 msgr2=0x7fe418157720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 shutdown_connections 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:30.396+0000 7fe4201ff700 1 -- 172.21.15.7:0/2283237674 wait complete. 2023-06-09T11:17:30.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2023-06-09T11:17:30.418 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:32.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:32.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.626+0000 7facc91ff700 1 Processor -- start 2023-06-09T11:17:32.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.626+0000 7facc91ff700 1 -- start start 2023-06-09T11:17:32.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.627+0000 7facc91ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facc415b4a0 0x7facc415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:32.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.627+0000 7facc91ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc415a500 con 0x7facc4153880 2023-06-09T11:17:32.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.627+0000 7facc91ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc415da00 con 0x7facc40c9f40 2023-06-09T11:17:32.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.627+0000 7facc91ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc40c5f10 con 0x7facc415b4a0 2023-06-09T11:17:32.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facc415b4a0 0x7facc415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:32.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7facc415b4a0 0x7facc415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51474/0 (socket says 172.21.15.7:51474) 2023-06-09T11:17:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 -- 172.21.15.7:0/763026749 learned_addr learned my addr 172.21.15.7:0/763026749 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:32.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc2d9d700 1 --1- 172.21.15.7:0/763026749 >> v1:172.21.15.7:6789/0 conn(0x7facc4153880 0x7facc415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60050/0 (socket says 172.21.15.7:60050) 2023-06-09T11:17:32.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 -- 172.21.15.7:0/763026749 >> v1:172.21.15.117:6789/0 conn(0x7facc40c9f40 legacy=0x7facc40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 -- 172.21.15.7:0/763026749 >> v1:172.21.15.7:6789/0 conn(0x7facc4153880 legacy=0x7facc415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 -- 172.21.15.7:0/763026749 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc41596e0 con 0x7facc415b4a0 2023-06-09T11:17:32.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.628+0000 7facc259c700 1 --2- 172.21.15.7:0/763026749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc415b4a0 0x7facc415d4c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7facb80098f0 tx=0x7facb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a00c46568b9122a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:32.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc1d9b700 1 -- 172.21.15.7:0/763026749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800b8a0 con 0x7facc415b4a0 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc1d9b700 1 -- 172.21.15.7:0/763026749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb800ba00 con 0x7facc415b4a0 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc1d9b700 1 -- 172.21.15.7:0/763026749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb800bd90 con 0x7facc415b4a0 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 -- 172.21.15.7:0/763026749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc415b4a0 msgr2=0x7facc415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 --2- 172.21.15.7:0/763026749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc415b4a0 0x7facc415d4c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7facb80098f0 tx=0x7facb8009c00 comp rx=0 tx=0).stop 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 -- 172.21.15.7:0/763026749 shutdown_connections 2023-06-09T11:17:32.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 --2- 172.21.15.7:0/763026749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc415b4a0 0x7facc415d4c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 -- 172.21.15.7:0/763026749 >> 172.21.15.7:0/763026749 conn(0x7facc40737f0 msgr2=0x7facc40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 -- 172.21.15.7:0/763026749 shutdown_connections 2023-06-09T11:17:32.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.629+0000 7facc91ff700 1 -- 172.21.15.7:0/763026749 wait complete. 2023-06-09T11:17:32.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.631+0000 7facc91ff700 1 Processor -- start 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.631+0000 7facc91ff700 1 -- start start 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc40c9f40 0x7facc4151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facc415b4a0 0x7facc41f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41f2650 con 0x7facc4153880 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc4152610 con 0x7facc40c9f40 2023-06-09T11:17:32.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc41f2370 con 0x7facc415b4a0 2023-06-09T11:17:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51478/0 (socket says 172.21.15.7:51478) 2023-06-09T11:17:32.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.632+0000 7facc259c700 1 -- 172.21.15.7:0/1500456624 learned_addr learned my addr 172.21.15.7:0/1500456624 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc2d9d700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc40c9f40 0x7facc4151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc359e700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facc415b4a0 0x7facc41f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 -- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facc415b4a0 msgr2=0x7facc41f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facc415b4a0 0x7facc41f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 -- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc40c9f40 msgr2=0x7facc4151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc40c9f40 0x7facc4151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 -- 172.21.15.7:0/1500456624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facb8009550 con 0x7facc4153880 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facc259c700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7facb8014040 tx=0x7facb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb80038a0 con 0x7facc4153880 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb801e070 con 0x7facc4153880 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.633+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb8003c60 con 0x7facc4153880 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.634+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc4159c60 con 0x7facc4153880 2023-06-09T11:17:32.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.634+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc4159980 con 0x7facc4153880 2023-06-09T11:17:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.635+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc4159f40 con 0x7facc4153880 2023-06-09T11:17:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.636+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7facb8011420 con 0x7facc4153880 2023-06-09T11:17:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.636+0000 7facab7fe700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 0x7facb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:32.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.637+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7facb8020070 con 0x7facc4153880 2023-06-09T11:17:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.637+0000 7facc2d9d700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 0x7facb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:32.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.637+0000 7facc2d9d700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 0x7facb006d1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7facb4000f60 tx=0x7facb4007d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:32.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.641+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7facb80559f0 con 0x7facc4153880 2023-06-09T11:17:32.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.890+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facc41f2800 con 0x7facc4153880 2023-06-09T11:17:32.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.891+0000 7facab7fe700 1 -- 172.21.15.7:0/1500456624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+5460 (secure 0 0 0) 0x7facc41f2800 con 0x7facc4153880 2023-06-09T11:17:32.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:32.896 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:27.240670+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 msgr2=0x7facb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 0x7facb006d1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7facb4000f60 tx=0x7facb4007d10 comp rx=0 tx=0).stop 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 msgr2=0x7facc41e6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7facb8014040 tx=0x7facb8004750 comp rx=0 tx=0).stop 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 shutdown_connections 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7facb006ad10 0x7facb006d1c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7facc415b4a0 0x7facc41f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7facc40c9f40 0x7facc4151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 --2- 172.21.15.7:0/1500456624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7facc4153880 0x7facc41e6550 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:32.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.893+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 >> 172.21.15.7:0/1500456624 conn(0x7facc40737f0 msgr2=0x7facc406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:32.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.894+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 shutdown_connections 2023-06-09T11:17:32.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:32.894+0000 7facc91ff700 1 -- 172.21.15.7:0/1500456624 wait complete. 2023-06-09T11:17:32.900 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2023-06-09T11:17:32.916 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:34.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.124+0000 7f43b8dff700 1 Processor -- start 2023-06-09T11:17:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.124+0000 7f43b8dff700 1 -- start start 2023-06-09T11:17:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b8dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43b40c61c0 0x7f43b40c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b8dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b4153330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b8dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b40bf000 con 0x7f43b40704b0 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b8dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b40c7c00 con 0x7f43b40c61c0 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b8dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4070eb0 con 0x7f43b40c8a50 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.126+0000 7f43b259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43b40c61c0 0x7f43b40c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43b40c61c0 0x7f43b40c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40290/0 (socket says 172.21.15.7:40290) 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 -- 172.21.15.7:0/1753367521 learned_addr learned my addr 172.21.15.7:0/1753367521 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b1d9b700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b4153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 -- 172.21.15.7:0/1753367521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 msgr2=0x7f43b4153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b4153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 -- 172.21.15.7:0/1753367521 >> v1:172.21.15.117:6789/0 conn(0x7f43b40704b0 legacy=0x7f43b40712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 -- 172.21.15.7:0/1753367521 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b415c9e0 con 0x7f43b40c61c0 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b1d9b700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.127+0000 7f43b259c700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b40c6590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43a40098f0 tx=0x7f43a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d082dc7b3caa12ec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b159a700 1 -- 172.21.15.7:0/1753367521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400b8a0 con 0x7f43b40c61c0 2023-06-09T11:17:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b159a700 1 -- 172.21.15.7:0/1753367521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400ba00 con 0x7f43b40c61c0 2023-06-09T11:17:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b159a700 1 -- 172.21.15.7:0/1753367521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400bd90 con 0x7f43b40c61c0 2023-06-09T11:17:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 -- 172.21.15.7:0/1753367521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 msgr2=0x7f43b40c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b40c6590 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43a40098f0 tx=0x7f43a4009c00 comp rx=0 tx=0).stop 2023-06-09T11:17:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 -- 172.21.15.7:0/1753367521 shutdown_connections 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b40c6590 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 --2- 172.21.15.7:0/1753367521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b4153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 -- 172.21.15.7:0/1753367521 >> 172.21.15.7:0/1753367521 conn(0x7f43b406a4d0 msgr2=0x7f43b406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 -- 172.21.15.7:0/1753367521 shutdown_connections 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.128+0000 7f43b8dff700 1 -- 172.21.15.7:0/1753367521 wait complete. 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.130+0000 7f43b8dff700 1 Processor -- start 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.130+0000 7f43b8dff700 1 -- start start 2023-06-09T11:17:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b40704b0 0x7f43b413cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b41eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b413d0b0 con 0x7f43b40c61c0 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b40cc5f0 con 0x7f43b40704b0 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b8dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b41ee500 con 0x7f43b40c8a50 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b1d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40306/0 (socket says 172.21.15.7:40306) 2023-06-09T11:17:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.131+0000 7f43b1d9b700 1 -- 172.21.15.7:0/2309529479 learned_addr learned my addr 172.21.15.7:0/2309529479 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b259c700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b40704b0 0x7f43b413cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b2d9d700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b41eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 -- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 msgr2=0x7f43b41eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b41eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 -- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b40704b0 msgr2=0x7f43b413cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b40704b0 0x7f43b413cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 -- 172.21.15.7:0/2309529479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a4009550 con 0x7f43b40c61c0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b1d9b700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43a800e810 tx=0x7f43a800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a800c020 con 0x7f43b40c61c0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8004250 con 0x7f43b40c61c0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a80036a0 con 0x7f43b40c61c0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.132+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b415cf60 con 0x7f43b40c61c0 2023-06-09T11:17:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.133+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b415cc80 con 0x7f43b40c61c0 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.134+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b415d240 con 0x7f43b40c61c0 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.134+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43a8003800 con 0x7f43b40c61c0 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.135+0000 7f43a2ffd700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 0x7f439406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.135+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f43a8088f40 con 0x7f43b40c61c0 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.135+0000 7f43b259c700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 0x7f439406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.136+0000 7f43b259c700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 0x7f439406d160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009c40 tx=0x7f43a4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.140+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f43a8053e70 con 0x7f43b40c61c0 2023-06-09T11:17:35.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.392+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43b41ee8f0 con 0x7f43b40c61c0 2023-06-09T11:17:35.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.393+0000 7f43a2ffd700 1 -- 172.21.15.7:0/2309529479 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f43a8057490 con 0x7f43b40c61c0 2023-06-09T11:17:35.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:35.398 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:33.210050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 msgr2=0x7f439406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 0x7f439406d160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009c40 tx=0x7f43a4008030 comp rx=0 tx=0).stop 2023-06-09T11:17:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 msgr2=0x7f43b4071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f43a800e810 tx=0x7f43a800eb20 comp rx=0 tx=0).stop 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 shutdown_connections 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f439406acb0 0x7f439406d160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43b40c8a50 0x7f43b41eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43b40704b0 0x7f43b413cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 --2- 172.21.15.7:0/2309529479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43b40c61c0 0x7f43b4071540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 >> 172.21.15.7:0/2309529479 conn(0x7f43b406a4d0 msgr2=0x7f43b40be160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 shutdown_connections 2023-06-09T11:17:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:35.395+0000 7f43b8dff700 1 -- 172.21.15.7:0/2309529479 wait complete. 2023-06-09T11:17:35.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2023-06-09T11:17:35.418 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:37.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.625+0000 7fb0c0fff700 1 Processor -- start 2023-06-09T11:17:37.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.625+0000 7fb0c0fff700 1 -- start start 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 0x7fb0bc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c5f10 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0bc15a500 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0c0fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc15da00 con 0x7fb0bc153880 2023-06-09T11:17:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40308/0 (socket says 172.21.15.7:40308) 2023-06-09T11:17:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 -- 172.21.15.7:0/843618912 learned_addr learned my addr 172.21.15.7:0/843618912 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0ba59c700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/843618912 >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 0x7fb0bc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 -- 172.21.15.7:0/843618912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 msgr2=0x7fb0bc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 -- 172.21.15.7:0/843618912 >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 msgr2=0x7fb0bc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 --2- 172.21.15.7:0/843618912 >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 0x7fb0bc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0bad9d700 1 -- 172.21.15.7:0/843618912 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc15b750 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.627+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/843618912 >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 0x7fb0bc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0bad9d700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0cd140 tx=0x7fb0b400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65733809c8fcbf8d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0b9d9b700 1 -- 172.21.15.7:0/843618912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400e070 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0b9d9b700 1 -- 172.21.15.7:0/843618912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b400b910 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0b9d9b700 1 -- 172.21.15.7:0/843618912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b400bcc0 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0c0fff700 1 -- 172.21.15.7:0/843618912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc15b2d0 msgr2=0x7fb0bc0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.628+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0cd140 tx=0x7fb0b400b330 comp rx=0 tx=0).stop 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 -- 172.21.15.7:0/843618912 shutdown_connections 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0c4e30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/843618912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/843618912 >> v2:172.21.15.117:3300/0 conn(0x7fb0bc0c9f40 0x7fb0bc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 -- 172.21.15.7:0/843618912 >> 172.21.15.7:0/843618912 conn(0x7fb0bc0737f0 msgr2=0x7fb0bc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 -- 172.21.15.7:0/843618912 shutdown_connections 2023-06-09T11:17:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.629+0000 7fb0c0fff700 1 -- 172.21.15.7:0/843618912 wait complete. 2023-06-09T11:17:37.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.630+0000 7fb0c0fff700 1 Processor -- start 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 -- start start 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc150ba0 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc1523c0 con 0x7fb0bc15b2d0 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.631+0000 7fb0c0fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0bc0c69a0 con 0x7fb0bc153880 2023-06-09T11:17:37.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40322/0 (socket says 172.21.15.7:40322) 2023-06-09T11:17:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 -- 172.21.15.7:0/2123247575 learned_addr learned my addr 172.21.15.7:0/2123247575 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0bad9d700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 -- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 msgr2=0x7fb0bc0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 -- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0bc15b2d0 msgr2=0x7fb0bc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 -- 172.21.15.7:0/2123247575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4009b30 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:17:37.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0ba59c700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9b0 tx=0x7fb0ac00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.632+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac009710 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.633+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac00c850 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.633+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0bc159980 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.633+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac00cbc0 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.633+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0bc1596e0 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.634+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0bc159c60 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.635+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0ac00cd20 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.635+0000 7fb0b2ffd700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.636+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fb0ac089700 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.636+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:37.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.636+0000 7fb0b3fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 0x7fb09c06d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0cf3c0 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:37.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.640+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb0ac0580c0 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.891+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0bc0cc1e0 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.891+0000 7fb0b2ffd700 1 -- 172.21.15.7:0/2123247575 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fb0ac057c50 con 0x7fb0bc0c9f40 2023-06-09T11:17:37.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:37.897 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:33.210050+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 msgr2=0x7fb09c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 0x7fb09c06d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0cf3c0 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2023-06-09T11:17:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 msgr2=0x7fb0bc14d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00e9b0 tx=0x7fb0ac00ed70 comp rx=0 tx=0).stop 2023-06-09T11:17:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 shutdown_connections 2023-06-09T11:17:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0bc15b2d0 0x7fb0bc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0bc0c9f40 0x7fb0bc14d260 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb09c06ace0 0x7fb09c06d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 --2- 172.21.15.7:0/2123247575 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0bc153880 0x7fb0bc0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 >> 172.21.15.7:0/2123247575 conn(0x7fb0bc0737f0 msgr2=0x7fb0bc06a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 shutdown_connections 2023-06-09T11:17:37.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:37.894+0000 7fb0c0fff700 1 -- 172.21.15.7:0/2123247575 wait complete. 2023-06-09T11:17:37.903 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2023-06-09T11:17:37.917 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:39.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.125+0000 7fac6ebff700 1 Processor -- start 2023-06-09T11:17:40.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.125+0000 7fac6ebff700 1 -- start start 2023-06-09T11:17:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.126+0000 7fac6ebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fac6806a4d0 0x7fac680c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.126+0000 7fac6ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 0x7fac6806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.126+0000 7fac6ebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6815cac0 con 0x7fac680c6280 2023-06-09T11:17:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.126+0000 7fac6ebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac680c8870 con 0x7fac6806a4d0 2023-06-09T11:17:40.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.126+0000 7fac6ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680c37d0 con 0x7fac6806e460 2023-06-09T11:17:40.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac6d19c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 0x7fac6806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac67fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fac6806a4d0 0x7fac680c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac6c99b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fac680c6280 0x7fac680737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34380/0 (socket says 172.21.15.7:34380) 2023-06-09T11:17:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac6c99b700 1 -- 172.21.15.7:0/853143939 learned_addr learned my addr 172.21.15.7:0/853143939 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac677fe700 1 -- 172.21.15.7:0/853143939 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826122988 0 0) 0x7fac6815cac0 con 0x7fac680c6280 2023-06-09T11:17:40.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac677fe700 1 -- 172.21.15.7:0/853143939 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac580035b0 con 0x7fac680c6280 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac67fff700 1 -- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 msgr2=0x7fac6806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac67fff700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 0x7fac6806cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac67fff700 1 -- 172.21.15.7:0/853143939 >> v1:172.21.15.117:6789/0 conn(0x7fac680c6280 legacy=0x7fac680737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac67fff700 1 -- 172.21.15.7:0/853143939 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6815cd30 con 0x7fac6806a4d0 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.127+0000 7fac6d19c700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 0x7fac6806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac67fff700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac680c76b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac54009840 tx=0x7fac54009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8639584aa4c3fa42 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:40.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac677fe700 1 -- 172.21.15.7:0/853143939 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54011830 con 0x7fac6806a4d0 2023-06-09T11:17:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac677fe700 1 -- 172.21.15.7:0/853143939 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac54011990 con 0x7fac6806a4d0 2023-06-09T11:17:40.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac677fe700 1 -- 172.21.15.7:0/853143939 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54011d00 con 0x7fac6806a4d0 2023-06-09T11:17:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac6ebff700 1 -- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 msgr2=0x7fac680c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.128+0000 7fac6ebff700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac680c76b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fac54009840 tx=0x7fac54009b50 comp rx=0 tx=0).stop 2023-06-09T11:17:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 -- 172.21.15.7:0/853143939 shutdown_connections 2023-06-09T11:17:40.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac6806e460 0x7fac6806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 --2- 172.21.15.7:0/853143939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac680c76b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 -- 172.21.15.7:0/853143939 >> 172.21.15.7:0/853143939 conn(0x7fac6806f980 msgr2=0x7fac680cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 -- 172.21.15.7:0/853143939 shutdown_connections 2023-06-09T11:17:40.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.129+0000 7fac6ebff700 1 -- 172.21.15.7:0/853143939 wait complete. 2023-06-09T11:17:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.130+0000 7fac6ebff700 1 Processor -- start 2023-06-09T11:17:40.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.131+0000 7fac6ebff700 1 -- start start 2023-06-09T11:17:40.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.131+0000 7fac6ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac68159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.131+0000 7fac6ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac680c6280 0x7fac681ef120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac681f3290 con 0x7fac6806e460 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac68157080 con 0x7fac6806a4d0 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6815a3c0 con 0x7fac680c6280 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6c99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac68159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40336/0 (socket says 172.21.15.7:40336) 2023-06-09T11:17:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6c99b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac68159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:54622/0 (socket says 172.21.15.7:54622) 2023-06-09T11:17:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac6d19c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac680c6280 0x7fac681ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 -- 172.21.15.7:0/1374429931 learned_addr learned my addr 172.21.15.7:0/1374429931 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 -- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac680c6280 msgr2=0x7fac681ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac680c6280 0x7fac681ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 -- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 msgr2=0x7fac68159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac68159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.132+0000 7fac67fff700 1 -- 172.21.15.7:0/1374429931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac60009a70 con 0x7fac6806e460 2023-06-09T11:17:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac67fff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac540090b0 tx=0x7fac54004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac54004550 con 0x7fac6806e460 2023-06-09T11:17:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5401f070 con 0x7fac6806e460 2023-06-09T11:17:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac540094a0 con 0x7fac6806e460 2023-06-09T11:17:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540038a0 con 0x7fac6806e460 2023-06-09T11:17:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.133+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6815d2f0 con 0x7fac6806e460 2023-06-09T11:17:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.134+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6815d010 con 0x7fac6806e460 2023-06-09T11:17:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.135+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac54003a00 con 0x7fac6806e460 2023-06-09T11:17:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.136+0000 7fac657fa700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 0x7fac4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.136+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7fac54021070 con 0x7fac6806e460 2023-06-09T11:17:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.136+0000 7fac6c99b700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 0x7fac4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.137+0000 7fac6c99b700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 0x7fac4c06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.141+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fac54055d00 con 0x7fac6806e460 2023-06-09T11:17:40.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.397+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac6815d5d0 con 0x7fac6806e460 2023-06-09T11:17:40.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.398+0000 7fac657fa700 1 -- 172.21.15.7:0/1374429931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5463 (secure 0 0 0) 0x7fac54003cb0 con 0x7fac6806e460 2023-06-09T11:17:40.403 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:40.403 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:39.223730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:clientreplay","state_seq":31,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 msgr2=0x7fac4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 0x7fac4c06d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c007dc0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 msgr2=0x7fac680cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fac540090b0 tx=0x7fac54004750 comp rx=0 tx=0).stop 2023-06-09T11:17:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 shutdown_connections 2023-06-09T11:17:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac4c06ace0 0x7fac4c06d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac680c6280 0x7fac681ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac6806a4d0 0x7fac68159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 --2- 172.21.15.7:0/1374429931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac6806e460 0x7fac680cb330 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 >> 172.21.15.7:0/1374429931 conn(0x7fac6806f980 msgr2=0x7fac6815b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 shutdown_connections 2023-06-09T11:17:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:40.400+0000 7fac6ebff700 1 -- 172.21.15.7:0/1374429931 wait complete. 2023-06-09T11:17:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2023-06-09T11:17:40.424 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 38, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 38, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4889": { "addr": "172.21.15.117:6835/1617501683", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1617501683, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1617501683, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4889, "incarnation": 34, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:clientreplay", "state_seq": 31 }, "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 41, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:17:39.223730+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 4889, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:17:41.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:41.818 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:17:42.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.631+0000 7f13da3ff700 1 Processor -- start 2023-06-09T11:17:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.631+0000 7f13da3ff700 1 -- start start 2023-06-09T11:17:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.632+0000 7f13da3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 0x7f13d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.632+0000 7f13da3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13d40c08d0 0x7f13d4071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.632+0000 7f13da3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d40cc140 con 0x7f13d41593d0 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.632+0000 7f13da3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40714f0 con 0x7f13d40c08d0 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.632+0000 7f13da3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c1830 con 0x7f13d4159eb0 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f13d41593d0 0x7f13d41597a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34224/0 (socket says 172.21.15.7:34224) 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d3fff700 1 -- 172.21.15.7:0/612672483 learned_addr learned my addr 172.21.15.7:0/612672483 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d37fe700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 0x7f13d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d899c700 1 --2- 172.21.15.7:0/612672483 >> v2:172.21.15.117:3300/0 conn(0x7f13d40c08d0 0x7f13d4071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d2ffd700 1 -- 172.21.15.7:0/612672483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1187833710 0 0) 0x7f13d40cc140 con 0x7f13d41593d0 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d2ffd700 1 -- 172.21.15.7:0/612672483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13bc0035b0 con 0x7f13d41593d0 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d899c700 1 -- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 msgr2=0x7f13d4071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d899c700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 0x7f13d4071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d899c700 1 -- 172.21.15.7:0/612672483 >> v1:172.21.15.7:6789/0 conn(0x7f13d41593d0 legacy=0x7f13d41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d899c700 1 -- 172.21.15.7:0/612672483 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40c0560 con 0x7f13d40c08d0 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.633+0000 7f13d37fe700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 0x7f13d4071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.634+0000 7f13d899c700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d4071e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009d90 tx=0x7f13cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22994cc9f29bf588 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.634+0000 7f13d2ffd700 1 -- 172.21.15.7:0/612672483 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00e070 con 0x7f13d40c08d0 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.634+0000 7f13d2ffd700 1 -- 172.21.15.7:0/612672483 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc00bb20 con 0x7f13d40c08d0 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.634+0000 7f13d2ffd700 1 -- 172.21.15.7:0/612672483 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012430 con 0x7f13d40c08d0 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 -- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 msgr2=0x7f13d4071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d4071e50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009d90 tx=0x7f13cc00b330 comp rx=0 tx=0).stop 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 -- 172.21.15.7:0/612672483 shutdown_connections 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d4071e50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 --2- 172.21.15.7:0/612672483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d4159eb0 0x7f13d4071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 -- 172.21.15.7:0/612672483 >> 172.21.15.7:0/612672483 conn(0x7f13d406a4d0 msgr2=0x7f13d406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 -- 172.21.15.7:0/612672483 shutdown_connections 2023-06-09T11:17:42.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.635+0000 7f13da3ff700 1 -- 172.21.15.7:0/612672483 wait complete. 2023-06-09T11:17:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.637+0000 7f13da3ff700 1 Processor -- start 2023-06-09T11:17:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.637+0000 7f13da3ff700 1 -- start start 2023-06-09T11:17:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d41def90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d41593d0 0x7f13d41eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40c5400 con 0x7f13d4159eb0 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d4154a70 con 0x7f13d40c08d0 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13da3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d41ef430 con 0x7f13d41593d0 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40344/0 (socket says 172.21.15.7:40344) 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 -- 172.21.15.7:0/823804980 learned_addr learned my addr 172.21.15.7:0/823804980 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d3fff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d41def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d37fe700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d41593d0 0x7f13d41eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 -- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d41593d0 msgr2=0x7f13d41eeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d41593d0 0x7f13d41eeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 -- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 msgr2=0x7f13d41def90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d41def90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.638+0000 7f13d899c700 1 -- 172.21.15.7:0/823804980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c800b740 con 0x7f13d4159eb0 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13d899c700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f13cc014040 tx=0x7f13cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:42.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0129f0 con 0x7f13d4159eb0 2023-06-09T11:17:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc020070 con 0x7f13d4159eb0 2023-06-09T11:17:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13cc009a70 con 0x7f13d4159eb0 2023-06-09T11:17:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012db0 con 0x7f13d4159eb0 2023-06-09T11:17:42.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.639+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40c1d60 con 0x7f13d4159eb0 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.640+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d40c1a80 con 0x7f13d4159eb0 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.641+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13cc004040 con 0x7f13d4159eb0 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.642+0000 7f13d0ff9700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.642+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f13cc022070 con 0x7f13d4159eb0 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.642+0000 7f13d3fff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:42.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.643+0000 7f13d3fff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 0x7f13c006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13c4007fd0 tx=0x7f13c4007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.647+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f13cc056260 con 0x7f13d4159eb0 2023-06-09T11:17:42.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.902+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13d40c2040 con 0x7f13d4159eb0 2023-06-09T11:17:42.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.902+0000 7f13d0ff9700 1 -- 172.21.15.7:0/823804980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5463 (secure 0 0 0) 0x7f13cc059880 con 0x7f13d4159eb0 2023-06-09T11:17:42.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:42.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:39.223730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:clientreplay","state_seq":31,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 msgr2=0x7f13c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 0x7f13c006d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f13c4007fd0 tx=0x7f13c4007dc0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 msgr2=0x7f13d41481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f13cc014040 tx=0x7f13cc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 shutdown_connections 2023-06-09T11:17:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13d4159eb0 0x7f13d41481e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13d40c08d0 0x7f13d41def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c006ace0 0x7f13c006d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 --2- 172.21.15.7:0/823804980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13d41593d0 0x7f13d41eeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 >> 172.21.15.7:0/823804980 conn(0x7f13d406a4d0 msgr2=0x7f13d40c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 shutdown_connections 2023-06-09T11:17:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:42.905+0000 7f13da3ff700 1 -- 172.21.15.7:0/823804980 wait complete. 2023-06-09T11:17:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 38 2023-06-09T11:17:42.928 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:17:44.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.134+0000 7f03a2bff700 1 Processor -- start 2023-06-09T11:17:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.134+0000 7f03a2bff700 1 -- start start 2023-06-09T11:17:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f039c151510 0x7f039c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f039c15b380 0x7f039c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c1518e0 con 0x7f039c151510 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0714c0 con 0x7f039c15b380 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.135+0000 7f03a2bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1580a0 con 0x7f039c071060 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f039c151510 0x7f039c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f039c151510 0x7f039c0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52230/0 (socket says 172.21.15.7:52230) 2023-06-09T11:17:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 -- 172.21.15.7:0/800973291 learned_addr learned my addr 172.21.15.7:0/800973291 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a119c700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f039bfff700 1 --2- 172.21.15.7:0/800973291 >> v2:172.21.15.117:3300/0 conn(0x7f039c15b380 0x7f039c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 -- 172.21.15.7:0/800973291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 msgr2=0x7f039c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 -- 172.21.15.7:0/800973291 >> v2:172.21.15.117:3300/0 conn(0x7f039c15b380 msgr2=0x7f039c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 --2- 172.21.15.7:0/800973291 >> v2:172.21.15.117:3300/0 conn(0x7f039c15b380 0x7f039c070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 -- 172.21.15.7:0/800973291 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0c6180 con 0x7f039c151510 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.136+0000 7f03a099b700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c151510 0x7f039c0c5b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f039c06b170 tx=0x7f0390009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242f634d3583c3ac server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f039b7fe700 1 -- 172.21.15.7:0/800973291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000b870 con 0x7f039c151510 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f039b7fe700 1 -- 172.21.15.7:0/800973291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039000b9d0 con 0x7f039c151510 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f039b7fe700 1 -- 172.21.15.7:0/800973291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039000bd60 con 0x7f039c151510 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 -- 172.21.15.7:0/800973291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c151510 msgr2=0x7f039c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c151510 0x7f039c0c5b40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f039c06b170 tx=0x7f0390009bd0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 -- 172.21.15.7:0/800973291 shutdown_connections 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 --2- 172.21.15.7:0/800973291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c151510 0x7f039c0c5b40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 --2- 172.21.15.7:0/800973291 >> v2:172.21.15.117:3300/0 conn(0x7f039c15b380 0x7f039c070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.137+0000 7f03a2bff700 1 -- 172.21.15.7:0/800973291 >> 172.21.15.7:0/800973291 conn(0x7f039c06a4d0 msgr2=0x7f039c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.138+0000 7f03a2bff700 1 -- 172.21.15.7:0/800973291 shutdown_connections 2023-06-09T11:17:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.138+0000 7f03a2bff700 1 -- 172.21.15.7:0/800973291 wait complete. 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.140+0000 7f03a2bff700 1 Processor -- start 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.140+0000 7f03a2bff700 1 -- start start 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039c151510 0x7f039c144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1eefa0 con 0x7f039c15b380 2023-06-09T11:17:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1ee390 con 0x7f039c151510 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.141+0000 7f03a2bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1ee600 con 0x7f039c071060 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52236/0 (socket says 172.21.15.7:52236) 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 -- 172.21.15.7:0/128028437 learned_addr learned my addr 172.21.15.7:0/128028437 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f039bfff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039c151510 0x7f039c144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a099b700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 -- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 msgr2=0x7f039c145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 -- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039c151510 msgr2=0x7f039c144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039c151510 0x7f039c144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 -- 172.21.15.7:0/128028437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009550 con 0x7f039c15b380 2023-06-09T11:17:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a119c700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0394007e60 tx=0x7f039400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400ed30 con 0x7f039c15b380 2023-06-09T11:17:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394014420 con 0x7f039c15b380 2023-06-09T11:17:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940147d0 con 0x7f039c15b380 2023-06-09T11:17:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.142+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c157770 con 0x7f039c15b380 2023-06-09T11:17:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.143+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c157490 con 0x7f039c15b380 2023-06-09T11:17:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.144+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c157a50 con 0x7f039c15b380 2023-06-09T11:17:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.144+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0394014930 con 0x7f039c15b380 2023-06-09T11:17:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.146+0000 7f03997fa700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 0x7f038006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.146+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f0394089cc0 con 0x7f039c15b380 2023-06-09T11:17:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.146+0000 7f03a099b700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 0x7f038006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.147+0000 7f03a099b700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 0x7f038006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f0390008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:45.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.150+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0394090080 con 0x7f039c15b380 2023-06-09T11:17:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.407+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f039c1ef0e0 con 0x7f039c15b380 2023-06-09T11:17:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.408+0000 7f03997fa700 1 -- 172.21.15.7:0/128028437 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f039c1ef0e0 con 0x7f039c15b380 2023-06-09T11:17:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:45.414 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:45.245021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:45.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 msgr2=0x7f038006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:45.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 0x7f038006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0390000c00 tx=0x7f0390008030 comp rx=0 tx=0).stop 2023-06-09T11:17:45.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 msgr2=0x7f039c1ef3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0394007e60 tx=0x7f039400e3c0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 shutdown_connections 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f039c15b380 0x7f039c1ef3c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038006ace0 0x7f038006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f039c071060 0x7f039c145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 --2- 172.21.15.7:0/128028437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f039c151510 0x7f039c144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 >> 172.21.15.7:0/128028437 conn(0x7f039c06a4d0 msgr2=0x7f039c15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.410+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 shutdown_connections 2023-06-09T11:17:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:45.411+0000 7f03a2bff700 1 -- 172.21.15.7:0/128028437 wait complete. 2023-06-09T11:17:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 39 2023-06-09T11:17:45.434 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=4291 2023-06-09T11:17:45.434 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.f 2023-06-09T11:17:49.435 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:17:49.436 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:17:49.436 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:17:49.441 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:17:49.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:49.589 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:17:49.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.646+0000 7f5746fff700 1 Processor -- start 2023-06-09T11:17:49.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.646+0000 7f5746fff700 1 -- start start 2023-06-09T11:17:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.647+0000 7f5746fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015a0c0 0x7f57401501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.647+0000 7f5746fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400c0bf0 con 0x7f57400c0140 2023-06-09T11:17:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.647+0000 7f5746fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400702f0 con 0x7f57400c1bc0 2023-06-09T11:17:49.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.647+0000 7f5746fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5740070110 con 0x7f574015a0c0 2023-06-09T11:17:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.647+0000 7f573ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015a0c0 0x7f57401501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f574015a0c0 0x7f57401501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52246/0 (socket says 172.21.15.7:52246) 2023-06-09T11:17:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 -- 172.21.15.7:0/1574229258 learned_addr learned my addr 172.21.15.7:0/1574229258 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:49.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f574559c700 1 --1- 172.21.15.7:0/1574229258 >> v1:172.21.15.7:6789/0 conn(0x7f57400c0140 0x7f5740160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56766/0 (socket says 172.21.15.7:56766) 2023-06-09T11:17:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 -- 172.21.15.7:0/1574229258 >> v1:172.21.15.117:6789/0 conn(0x7f57400c1bc0 legacy=0x7f57401599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 -- 172.21.15.7:0/1574229258 >> v1:172.21.15.7:6789/0 conn(0x7f57400c0140 legacy=0x7f5740160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 -- 172.21.15.7:0/1574229258 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400c0e90 con 0x7f574015a0c0 2023-06-09T11:17:49.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573ffff700 1 --2- 172.21.15.7:0/1574229258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015a0c0 0x7f57401501b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f572c0098f0 tx=0x7f572c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62709cf21508ade server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.648+0000 7f573f7fe700 1 -- 172.21.15.7:0/1574229258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00f8a0 con 0x7f574015a0c0 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f573f7fe700 1 -- 172.21.15.7:0/1574229258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c00fa00 con 0x7f574015a0c0 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f573f7fe700 1 -- 172.21.15.7:0/1574229258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00fd90 con 0x7f574015a0c0 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 -- 172.21.15.7:0/1574229258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015a0c0 msgr2=0x7f57401501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 --2- 172.21.15.7:0/1574229258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015a0c0 0x7f57401501b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f572c0098f0 tx=0x7f572c009c00 comp rx=0 tx=0).stop 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 -- 172.21.15.7:0/1574229258 shutdown_connections 2023-06-09T11:17:49.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 --2- 172.21.15.7:0/1574229258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f574015a0c0 0x7f57401501b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 -- 172.21.15.7:0/1574229258 >> 172.21.15.7:0/1574229258 conn(0x7f574006a4d0 msgr2=0x7f574006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 -- 172.21.15.7:0/1574229258 shutdown_connections 2023-06-09T11:17:49.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.649+0000 7f5746fff700 1 -- 172.21.15.7:0/1574229258 wait complete. 2023-06-09T11:17:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.651+0000 7f5746fff700 1 Processor -- start 2023-06-09T11:17:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.651+0000 7f5746fff700 1 -- start start 2023-06-09T11:17:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400c0140 0x7f574014c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:49.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574015a0c0 0x7f5740155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401efab0 con 0x7f57400c1bc0 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574014c540 con 0x7f574015a0c0 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5746fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57401ef530 con 0x7f57400c0140 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f573ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f573ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52252/0 (socket says 172.21.15.7:52252) 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f573ffff700 1 -- 172.21.15.7:0/930122688 learned_addr learned my addr 172.21.15.7:0/930122688 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f574559c700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574015a0c0 0x7f5740155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:49.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.652+0000 7f5744d9b700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400c0140 0x7f574014c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:49.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 -- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400c0140 msgr2=0x7f574014c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400c0140 0x7f574014c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 -- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574015a0c0 msgr2=0x7f5740155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574015a0c0 0x7f5740155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 -- 172.21.15.7:0/930122688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c009550 con 0x7f57400c1bc0 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573ffff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f572c015000 tx=0x7f572c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c0038a0 con 0x7f57400c1bc0 2023-06-09T11:17:49.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c003a00 con 0x7f57400c1bc0 2023-06-09T11:17:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57400c1450 con 0x7f57400c1bc0 2023-06-09T11:17:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c003db0 con 0x7f57400c1bc0 2023-06-09T11:17:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.653+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57400c1170 con 0x7f57400c1bc0 2023-06-09T11:17:49.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.654+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57400705d0 con 0x7f57400c1bc0 2023-06-09T11:17:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.655+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f572c024450 con 0x7f57400c1bc0 2023-06-09T11:17:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.656+0000 7f573d7fa700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.657+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f572c08aaa0 con 0x7f57400c1bc0 2023-06-09T11:17:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.657+0000 7f5744d9b700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:49.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.657+0000 7f5744d9b700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5734007fd0 tx=0x7f5734009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:49.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.661+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f572c0559d0 con 0x7f57400c1bc0 2023-06-09T11:17:49.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.914+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5740074cf0 con 0x7f57400c1bc0 2023-06-09T11:17:49.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.914+0000 7f573d7fa700 1 -- 172.21.15.7:0/930122688 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f572c058ff0 con 0x7f57400c1bc0 2023-06-09T11:17:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:49.920 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:45.245021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 msgr2=0x7f572406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5734007fd0 tx=0x7f5734009040 comp rx=0 tx=0).stop 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 msgr2=0x7f57400ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f572c015000 tx=0x7f572c004750 comp rx=0 tx=0).stop 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 shutdown_connections 2023-06-09T11:17:49.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f574015a0c0 0x7f5740155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572406ace0 0x7f572406d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400c0140 0x7f574014c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 --2- 172.21.15.7:0/930122688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400c1bc0 0x7f57400ca990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 >> 172.21.15.7:0/930122688 conn(0x7f574006a4d0 msgr2=0x7f574014ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 shutdown_connections 2023-06-09T11:17:49.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:49.917+0000 7f5746fff700 1 -- 172.21.15.7:0/930122688 wait complete. 2023-06-09T11:17:49.925 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 39 2023-06-09T11:17:49.940 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-06-09T11:17:50.433 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/35' 2023-06-09T11:17:50.433 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/26' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './266/37' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './266' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/21' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/27' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/31' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/36' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/20' 2023-06-09T11:17:50.434 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/39' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/34' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/38' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/28' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/24' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/29' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/32' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/22' 2023-06-09T11:17:50.435 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/25' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/30' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/33' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/23' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/35' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/26' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed './267/37' 2023-06-09T11:17:50.436 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './267' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/21' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/27' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/31' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/36' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/20' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/39' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/34' 2023-06-09T11:17:50.437 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/38' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/28' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/24' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/29' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/32' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/22' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/25' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/30' 2023-06-09T11:17:50.438 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/33' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/23' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/35' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/26' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './268/37' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './268' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/21' 2023-06-09T11:17:50.439 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/27' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/31' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/36' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/20' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/39' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/34' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/38' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/28' 2023-06-09T11:17:50.440 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/24' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/29' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/32' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/22' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/25' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/30' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/33' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/23' 2023-06-09T11:17:50.441 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/35' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/26' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './269/37' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './269' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/21' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/27' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/31' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/36' 2023-06-09T11:17:50.442 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/20' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/39' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/34' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/38' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/28' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/24' 2023-06-09T11:17:50.443 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/29' 2023-06-09T11:17:50.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/32' 2023-06-09T11:17:50.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/22' 2023-06-09T11:17:50.444 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/25' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/30' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/33' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/23' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/35' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/26' 2023-06-09T11:17:50.445 INFO:tasks.workunit.client.0.smithi007.stdout:removed './270/37' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './270' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/21' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/27' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/31' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/36' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/20' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/39' 2023-06-09T11:17:50.446 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/34' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/38' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/28' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/24' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/29' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/32' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/22' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/25' 2023-06-09T11:17:50.447 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/30' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/33' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/23' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/35' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/26' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './271/37' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './271' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/21' 2023-06-09T11:17:50.448 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/27' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/31' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/36' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/20' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/39' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/34' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/38' 2023-06-09T11:17:50.449 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/28' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/24' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/29' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/32' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/22' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/25' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/30' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/33' 2023-06-09T11:17:50.450 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/23' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/35' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/26' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './272/37' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './272' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/21' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/27' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/31' 2023-06-09T11:17:50.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/36' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/20' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/39' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/34' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/38' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/28' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/24' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/29' 2023-06-09T11:17:50.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/32' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/22' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/25' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/30' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/33' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/23' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/35' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/26' 2023-06-09T11:17:50.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './273/37' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './273' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/21' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/27' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/31' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/36' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/20' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/39' 2023-06-09T11:17:50.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/34' 2023-06-09T11:17:50.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/38' 2023-06-09T11:17:50.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/28' 2023-06-09T11:17:50.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/24' 2023-06-09T11:17:50.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/29' 2023-06-09T11:17:50.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/32' 2023-06-09T11:17:50.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/22' 2023-06-09T11:17:50.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/25' 2023-06-09T11:17:50.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/30' 2023-06-09T11:17:50.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/33' 2023-06-09T11:17:50.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/23' 2023-06-09T11:17:50.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/35' 2023-06-09T11:17:50.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/26' 2023-06-09T11:17:50.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './274/37' 2023-06-09T11:17:50.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './274' 2023-06-09T11:17:50.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/21' 2023-06-09T11:17:50.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/27' 2023-06-09T11:17:50.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/31' 2023-06-09T11:17:50.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/36' 2023-06-09T11:17:50.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/20' 2023-06-09T11:17:50.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/39' 2023-06-09T11:17:50.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/34' 2023-06-09T11:17:50.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/38' 2023-06-09T11:17:50.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/28' 2023-06-09T11:17:50.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/24' 2023-06-09T11:17:50.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/29' 2023-06-09T11:17:50.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/32' 2023-06-09T11:17:50.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/22' 2023-06-09T11:17:50.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/25' 2023-06-09T11:17:50.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/30' 2023-06-09T11:17:50.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/33' 2023-06-09T11:17:50.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/23' 2023-06-09T11:17:50.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/35' 2023-06-09T11:17:50.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/26' 2023-06-09T11:17:50.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './275/37' 2023-06-09T11:17:50.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './275' 2023-06-09T11:17:50.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/21' 2023-06-09T11:17:50.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/27' 2023-06-09T11:17:50.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/31' 2023-06-09T11:17:50.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/36' 2023-06-09T11:17:50.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/20' 2023-06-09T11:17:50.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/39' 2023-06-09T11:17:50.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/34' 2023-06-09T11:17:50.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/38' 2023-06-09T11:17:50.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/28' 2023-06-09T11:17:50.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/24' 2023-06-09T11:17:50.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/29' 2023-06-09T11:17:50.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/32' 2023-06-09T11:17:50.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/22' 2023-06-09T11:17:50.466 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/25' 2023-06-09T11:17:50.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/30' 2023-06-09T11:17:50.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/33' 2023-06-09T11:17:50.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/23' 2023-06-09T11:17:50.467 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/35' 2023-06-09T11:17:50.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/26' 2023-06-09T11:17:51.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:17:52.083 INFO:tasks.workunit.client.0.smithi007.stdout:removed './276/37' 2023-06-09T11:17:52.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './276' 2023-06-09T11:17:52.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/21' 2023-06-09T11:17:52.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/27' 2023-06-09T11:17:52.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/31' 2023-06-09T11:17:52.084 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/36' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/20' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/39' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/34' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/38' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/28' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/24' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/29' 2023-06-09T11:17:52.085 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/32' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/22' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/25' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/30' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/33' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/23' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/35' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/26' 2023-06-09T11:17:52.086 INFO:tasks.workunit.client.0.smithi007.stdout:removed './277/37' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './277' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/21' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/27' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/31' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/36' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/20' 2023-06-09T11:17:52.087 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/39' 2023-06-09T11:17:52.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/34' 2023-06-09T11:17:52.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/38' 2023-06-09T11:17:52.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/28' 2023-06-09T11:17:52.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/24' 2023-06-09T11:17:52.088 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/29' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/32' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/22' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/25' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/30' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/33' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/23' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/35' 2023-06-09T11:17:52.089 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/26' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './278/37' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './278' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/21' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/27' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/31' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/36' 2023-06-09T11:17:52.090 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/20' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/39' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/34' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/38' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/28' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/24' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/29' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/32' 2023-06-09T11:17:52.091 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/22' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/25' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/30' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/33' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/23' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/35' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/26' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed './279/37' 2023-06-09T11:17:52.092 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './279' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/21' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/27' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/31' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/36' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/20' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/39' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/34' 2023-06-09T11:17:52.093 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/38' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/28' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/24' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/29' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/32' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/22' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/25' 2023-06-09T11:17:52.094 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/30' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/33' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/23' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/35' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/26' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './280/37' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './280' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/21' 2023-06-09T11:17:52.095 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/27' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/31' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/36' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/20' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/39' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/34' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/38' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/28' 2023-06-09T11:17:52.096 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/24' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/29' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/32' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/22' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/25' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/30' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/33' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/23' 2023-06-09T11:17:52.097 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/35' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/26' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './281/37' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './281' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/21' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/27' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/31' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/36' 2023-06-09T11:17:52.098 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/20' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/39' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/34' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/38' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/28' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/24' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/29' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/32' 2023-06-09T11:17:52.099 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/22' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/25' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/30' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/33' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/23' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/35' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/26' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed './282/37' 2023-06-09T11:17:52.100 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './282' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/21' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/27' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/31' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/36' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/20' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/39' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/34' 2023-06-09T11:17:52.101 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/38' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/28' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/24' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/29' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/32' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/22' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/25' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/30' 2023-06-09T11:17:52.102 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/33' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/23' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/35' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/26' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './283/37' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './283' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/21' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/27' 2023-06-09T11:17:52.103 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/31' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/36' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/20' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/39' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/34' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/38' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/28' 2023-06-09T11:17:52.104 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/24' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/29' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/32' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/22' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/25' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/30' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/33' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/23' 2023-06-09T11:17:52.105 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/35' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/26' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './284/37' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './284' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/21' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/27' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/31' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/36' 2023-06-09T11:17:52.106 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/20' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/39' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/34' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/38' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/28' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/24' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/29' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/32' 2023-06-09T11:17:52.107 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/22' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/25' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/30' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/33' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/23' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/35' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/26' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed './285/37' 2023-06-09T11:17:52.108 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './285' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/21' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/27' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/31' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/36' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/20' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/39' 2023-06-09T11:17:52.109 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/34' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/38' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/28' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/24' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/29' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/32' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/22' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/25' 2023-06-09T11:17:52.110 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/30' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/33' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/23' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/35' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/26' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed './286/37' 2023-06-09T11:17:52.111 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './286' 2023-06-09T11:17:52.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.154+0000 7f2f5e5ff700 1 Processor -- start 2023-06-09T11:17:52.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.154+0000 7f2f5e5ff700 1 -- start start 2023-06-09T11:17:52.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.155+0000 7f2f5e5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2f58153880 0x7f2f5815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.155+0000 7f2f5e5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f580c9f40 0x7f2f5815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.155+0000 7f2f5e5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5815a500 con 0x7f2f5815b2d0 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.155+0000 7f2f5e5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f580c5f10 con 0x7f2f580c9f40 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.155+0000 7f2f5e5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5815da00 con 0x7f2f58153880 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f580c9f40 0x7f2f5815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2f580c9f40 0x7f2f5815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52260/0 (socket says 172.21.15.7:52260) 2023-06-09T11:17:52.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 -- 172.21.15.7:0/171988471 learned_addr learned my addr 172.21.15.7:0/171988471 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:52.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f57fff700 1 --2- 172.21.15.7:0/171988471 >> v2:172.21.15.117:3300/0 conn(0x7f2f58153880 0x7f2f5815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 -- 172.21.15.7:0/171988471 >> v2:172.21.15.117:3300/0 conn(0x7f2f58153880 msgr2=0x7f2f5815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 --2- 172.21.15.7:0/171988471 >> v2:172.21.15.117:3300/0 conn(0x7f2f58153880 0x7f2f5815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 -- 172.21.15.7:0/171988471 >> v1:172.21.15.117:6789/0 conn(0x7f2f5815b2d0 legacy=0x7f2f580c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.156+0000 7f2f577fe700 1 -- 172.21.15.7:0/171988471 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f581596e0 con 0x7f2f580c9f40 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f577fe700 1 --2- 172.21.15.7:0/171988471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f580c9f40 0x7f2f5815ad90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f5808d340 tx=0x7f2f4c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca65d59a9ded8b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f56ffd700 1 -- 172.21.15.7:0/171988471 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c011830 con 0x7f2f580c9f40 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f56ffd700 1 -- 172.21.15.7:0/171988471 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c011990 con 0x7f2f580c9f40 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f56ffd700 1 -- 172.21.15.7:0/171988471 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c011d20 con 0x7f2f580c9f40 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/171988471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f580c9f40 msgr2=0x7f2f5815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.157+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/171988471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f580c9f40 0x7f2f5815ad90 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f5808d340 tx=0x7f2f4c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/171988471 shutdown_connections 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/171988471 >> v2:172.21.15.117:3300/0 conn(0x7f2f58153880 0x7f2f5815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/171988471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f580c9f40 0x7f2f5815ad90 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/171988471 >> 172.21.15.7:0/171988471 conn(0x7f2f580737f0 msgr2=0x7f2f580720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/171988471 shutdown_connections 2023-06-09T11:17:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.158+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/171988471 wait complete. 2023-06-09T11:17:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.160+0000 7f2f5e5ff700 1 Processor -- start 2023-06-09T11:17:52.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.160+0000 7f2f5e5ff700 1 -- start start 2023-06-09T11:17:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f580c9f40 0x7f2f58152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 0x7f2f5814c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 0x7f2f581f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f581f23b0 con 0x7f2f5815b2d0 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f580ce8c0 con 0x7f2f58153880 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f580ceba0 con 0x7f2f580c9f40 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.161+0000 7f2f5cb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 0x7f2f581f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f5cb9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 0x7f2f581f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52268/0 (socket says 172.21.15.7:52268) 2023-06-09T11:17:52.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f5cb9c700 1 -- 172.21.15.7:0/3146732748 learned_addr learned my addr 172.21.15.7:0/3146732748 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:17:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 0x7f2f5814c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f57fff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f580c9f40 0x7f2f58152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 -- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f580c9f40 msgr2=0x7f2f58152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f580c9f40 0x7f2f58152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 -- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 msgr2=0x7f2f581f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 0x7f2f581f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 -- 172.21.15.7:0/3146732748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c009550 con 0x7f2f58153880 2023-06-09T11:17:52.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.162+0000 7f2f577fe700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 0x7f2f5814c9e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c016000 tx=0x7f2f4c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.163+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c004380 con 0x7f2f58153880 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.163+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4c01f070 con 0x7f2f58153880 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.163+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f58159c60 con 0x7f2f58153880 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.163+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4c0036a0 con 0x7f2f58153880 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.163+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f58159980 con 0x7f2f58153880 2023-06-09T11:17:52.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.164+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f58159f40 con 0x7f2f58153880 2023-06-09T11:17:52.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.166+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f4c021030 con 0x7f2f58153880 2023-06-09T11:17:52.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.167+0000 7f2f54ff9700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 0x7f2f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:17:52.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.167+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6464+0+0 (secure 0 0 0) 0x7f2f4c08a910 con 0x7f2f58153880 2023-06-09T11:17:52.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.168+0000 7f2f57fff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 0x7f2f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:17:52.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.168+0000 7f2f57fff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 0x7f2f4406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009be0 tx=0x7f2f48009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:17:52.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.172+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2f4c055840 con 0x7f2f58153880 2023-06-09T11:17:52.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.473+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f58153450 con 0x7f2f58153880 2023-06-09T11:17:52.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.474+0000 7f2f54ff9700 1 -- 172.21.15.7:0/3146732748 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+6217 (secure 0 0 0) 0x7f2f4c058e60 con 0x7f2f58153880 2023-06-09T11:17:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:17:52.481 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:17:45.245021+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 msgr2=0x7f2f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 0x7f2f4406d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009be0 tx=0x7f2f48009210 comp rx=0 tx=0).stop 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 msgr2=0x7f2f5814c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 0x7f2f5814c9e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c016000 tx=0x7f2f4c004750 comp rx=0 tx=0).stop 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 shutdown_connections 2023-06-09T11:17:52.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5815b2d0 0x7f2f581f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2f4406ace0 0x7f2f4406d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f580c9f40 0x7f2f58152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 --2- 172.21.15.7:0/3146732748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f58153880 0x7f2f5814c9e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 >> 172.21.15.7:0/3146732748 conn(0x7f2f580737f0 msgr2=0x7f2f5814e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 shutdown_connections 2023-06-09T11:17:52.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:17:52.477+0000 7f2f5e5ff700 1 -- 172.21.15.7:0/3146732748 wait complete. 2023-06-09T11:17:52.485 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 40 2023-06-09T11:17:52.501 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:17:52.501 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 114 secs before thrashing 2023-06-09T11:17:53.569 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/21' 2023-06-09T11:17:53.569 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/27' 2023-06-09T11:17:53.569 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/31' 2023-06-09T11:17:53.570 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/36' 2023-06-09T11:17:53.570 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/20' 2023-06-09T11:17:53.570 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/39' 2023-06-09T11:17:53.571 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/34' 2023-06-09T11:17:53.571 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/38' 2023-06-09T11:17:53.571 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/28' 2023-06-09T11:17:53.572 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/24' 2023-06-09T11:17:53.572 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/29' 2023-06-09T11:17:53.572 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/32' 2023-06-09T11:17:53.572 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/22' 2023-06-09T11:17:53.573 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/25' 2023-06-09T11:17:53.573 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/30' 2023-06-09T11:17:53.573 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/33' 2023-06-09T11:17:53.574 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/23' 2023-06-09T11:17:53.574 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/35' 2023-06-09T11:17:53.574 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/26' 2023-06-09T11:17:53.574 INFO:tasks.workunit.client.0.smithi007.stdout:removed './287/37' 2023-06-09T11:17:53.575 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './287' 2023-06-09T11:17:53.575 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/21' 2023-06-09T11:17:53.575 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/27' 2023-06-09T11:17:53.575 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/31' 2023-06-09T11:17:53.576 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/36' 2023-06-09T11:17:53.576 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/20' 2023-06-09T11:17:53.576 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/39' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/34' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/38' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/28' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/24' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/29' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/32' 2023-06-09T11:17:53.577 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/22' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/25' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/30' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/33' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/23' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/35' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/26' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed './288/37' 2023-06-09T11:17:53.578 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './288' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/21' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/27' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/31' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/36' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/20' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/39' 2023-06-09T11:17:53.579 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/34' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/38' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/28' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/24' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/29' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/32' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/22' 2023-06-09T11:17:53.580 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/25' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/30' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/33' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/23' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/35' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/26' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './289/37' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './289' 2023-06-09T11:17:53.581 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/21' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/27' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/31' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/36' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/20' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/39' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/34' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/38' 2023-06-09T11:17:53.582 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/28' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/24' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/29' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/32' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/22' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/25' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/30' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/33' 2023-06-09T11:17:53.583 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/23' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/35' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/26' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './290/37' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './290' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/21' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/27' 2023-06-09T11:17:53.584 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/31' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/36' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/20' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/39' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/34' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/38' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/28' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/24' 2023-06-09T11:17:53.585 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/29' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/32' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/22' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/25' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/30' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/33' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/23' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/35' 2023-06-09T11:17:53.586 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/26' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './291/37' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './291' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/21' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/27' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/31' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/36' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/20' 2023-06-09T11:17:53.587 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/39' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/34' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/38' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/28' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/24' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/29' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/32' 2023-06-09T11:17:53.588 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/22' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/25' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/30' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/33' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/23' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/35' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/26' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed './292/37' 2023-06-09T11:17:53.589 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './292' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/21' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/27' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/31' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/36' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/20' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/39' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/34' 2023-06-09T11:17:53.590 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/38' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/28' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/24' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/29' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/32' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/22' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/25' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/30' 2023-06-09T11:17:53.591 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/33' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/23' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/35' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/26' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './293/37' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './293' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/21' 2023-06-09T11:17:53.592 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/27' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/31' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/36' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/20' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/39' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/34' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/38' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/28' 2023-06-09T11:17:53.593 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/24' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/29' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/32' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/22' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/25' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/30' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/33' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/23' 2023-06-09T11:17:53.594 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/35' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/26' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './294/37' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './294' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/21' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/27' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/31' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/36' 2023-06-09T11:17:53.595 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/20' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/39' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/34' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/38' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/28' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/24' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/29' 2023-06-09T11:17:53.596 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/32' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/22' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/25' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/30' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/33' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/23' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/35' 2023-06-09T11:17:53.597 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/26' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './295/37' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './295' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/21' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/27' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/31' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/36' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/20' 2023-06-09T11:17:53.598 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/39' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/34' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/38' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/28' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/24' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/29' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/32' 2023-06-09T11:17:53.599 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/22' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/25' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/30' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/33' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/23' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/35' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/26' 2023-06-09T11:17:53.600 INFO:tasks.workunit.client.0.smithi007.stdout:removed './296/37' 2023-06-09T11:17:53.601 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './296' 2023-06-09T11:17:53.601 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/21' 2023-06-09T11:17:53.601 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/27' 2023-06-09T11:17:54.222 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:17:54.218+0000 7efd679f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:17:54.222 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:17:54.218+0000 7efd679f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:17:54.223 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:17:54.218+0000 7efd679f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:17:55.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/31' 2023-06-09T11:17:55.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/36' 2023-06-09T11:17:55.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/20' 2023-06-09T11:17:55.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/39' 2023-06-09T11:17:55.946 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/34' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/38' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/28' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/24' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/29' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/32' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/22' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/25' 2023-06-09T11:17:55.947 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/30' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/33' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/23' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/35' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/26' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './297/37' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './297' 2023-06-09T11:17:55.948 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/21' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/27' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/31' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/36' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/20' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/39' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/34' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/38' 2023-06-09T11:17:55.949 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/28' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/24' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/29' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/32' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/22' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/25' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/30' 2023-06-09T11:17:55.950 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/33' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/23' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/35' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/26' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './298/37' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './298' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/21' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/27' 2023-06-09T11:17:55.951 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/31' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/36' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/20' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/39' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/34' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/38' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/28' 2023-06-09T11:17:55.952 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/24' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/29' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/32' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/22' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/25' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/30' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/33' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/23' 2023-06-09T11:17:55.953 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/35' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/26' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './299/37' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './299' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/19' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/15' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/20' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/12' 2023-06-09T11:17:55.954 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/10' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/11' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/16' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/14' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/18' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/13' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/166/17' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/166' 2023-06-09T11:17:55.955 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/19' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/15' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/20' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/12' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/10' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/11' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/16' 2023-06-09T11:17:55.956 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/14' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/18' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/13' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/147/17' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/147' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/19' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/15' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/20' 2023-06-09T11:17:55.957 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/12' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/10' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/11' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/16' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/14' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/18' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/13' 2023-06-09T11:17:55.958 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/151/17' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/151' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/19' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/15' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/20' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/12' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/10' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/11' 2023-06-09T11:17:55.959 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/16' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/14' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/18' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/13' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/170/17' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/170' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/19' 2023-06-09T11:17:55.960 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/15' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/20' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/12' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/10' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/11' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/16' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/14' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/18' 2023-06-09T11:17:55.961 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/13' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/167/17' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/167' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/19' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/15' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/20' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/12' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/10' 2023-06-09T11:17:55.962 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/11' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/16' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/14' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/18' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/13' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/140/17' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/140' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/19' 2023-06-09T11:17:55.963 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/15' 2023-06-09T11:17:55.964 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/20' 2023-06-09T11:17:55.964 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/12' 2023-06-09T11:17:55.964 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/10' 2023-06-09T11:17:57.002 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/11' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/16' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/14' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/18' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/13' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/142/17' 2023-06-09T11:17:57.003 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/142' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/19' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/15' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/20' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/12' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/10' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/11' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/16' 2023-06-09T11:17:57.004 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/14' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/18' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/13' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/150/17' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/150' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/19' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/15' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/20' 2023-06-09T11:17:57.005 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/12' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/10' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/11' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/16' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/14' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/18' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/13' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/105/17' 2023-06-09T11:17:57.006 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/105' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/19' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/15' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/20' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/12' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/10' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/11' 2023-06-09T11:17:57.007 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/16' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/14' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/18' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/13' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/149/17' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/149' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/19' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/15' 2023-06-09T11:17:57.008 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/20' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/12' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/10' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/11' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/16' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/14' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/18' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/13' 2023-06-09T11:17:57.009 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/109/17' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/109' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/19' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/15' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/20' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/12' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/10' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/11' 2023-06-09T11:17:57.010 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/16' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/14' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/18' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/13' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/158/17' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/158' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/19' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/15' 2023-06-09T11:17:57.011 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/20' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/12' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/10' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/11' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/16' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/14' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/18' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/13' 2023-06-09T11:17:57.012 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/106/17' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/106' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/19' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/15' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/20' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/12' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/10' 2023-06-09T11:17:57.013 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/11' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/16' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/14' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/18' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/13' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/189/17' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/189' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/19' 2023-06-09T11:17:57.014 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/15' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/20' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/12' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/10' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/11' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/16' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/14' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/18' 2023-06-09T11:17:57.015 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/13' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/133/17' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/133' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/19' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/15' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/20' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/12' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/10' 2023-06-09T11:17:57.016 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/11' 2023-06-09T11:17:57.468 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/16' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/14' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/18' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/13' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/173/17' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/173' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/19' 2023-06-09T11:17:57.469 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/15' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/20' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/12' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/10' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/11' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/16' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/14' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/18' 2023-06-09T11:17:57.470 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/13' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/174/17' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/174' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/19' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/15' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/20' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/12' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/10' 2023-06-09T11:17:57.471 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/11' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/16' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/14' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/18' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/13' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/155/17' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/155' 2023-06-09T11:17:57.472 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/19' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/15' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/20' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/12' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/10' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/11' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/16' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/14' 2023-06-09T11:17:57.473 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/18' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/13' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/114/17' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/114' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/19' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/15' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/20' 2023-06-09T11:17:57.474 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/12' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/10' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/11' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/16' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/14' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/18' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/13' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/159/17' 2023-06-09T11:17:57.475 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/159' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/19' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/15' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/20' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/12' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/10' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/11' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/16' 2023-06-09T11:17:57.476 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/14' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/18' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/13' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/187/17' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/187' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/19' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/15' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/20' 2023-06-09T11:17:57.477 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/12' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/10' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/11' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/16' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/14' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/18' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/13' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/185/17' 2023-06-09T11:17:57.478 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/185' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/19' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/15' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/20' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/12' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/10' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/11' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/16' 2023-06-09T11:17:57.479 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/14' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/18' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/13' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/120/17' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/120' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/19' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/15' 2023-06-09T11:17:57.480 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/20' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/12' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/10' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/11' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/16' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/14' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/18' 2023-06-09T11:17:57.481 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/13' 2023-06-09T11:17:57.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/110/17' 2023-06-09T11:17:57.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/110' 2023-06-09T11:17:57.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/19' 2023-06-09T11:17:57.482 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/15' 2023-06-09T11:17:57.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/20' 2023-06-09T11:17:57.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/12' 2023-06-09T11:17:57.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/10' 2023-06-09T11:17:57.483 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/11' 2023-06-09T11:17:57.484 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/16' 2023-06-09T11:18:00.522 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/14' 2023-06-09T11:18:00.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/18' 2023-06-09T11:18:00.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/13' 2023-06-09T11:18:00.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/137/17' 2023-06-09T11:18:00.523 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/137' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/19' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/15' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/20' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/12' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/10' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/11' 2023-06-09T11:18:00.524 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/16' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/14' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/18' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/13' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/198/17' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/198' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/19' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/15' 2023-06-09T11:18:00.525 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/20' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/12' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/10' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/11' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/16' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/14' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/18' 2023-06-09T11:18:00.526 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/13' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/136/17' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/136' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/19' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/15' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/20' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/12' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/10' 2023-06-09T11:18:00.527 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/11' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/16' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/14' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/18' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/13' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/125/17' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/125' 2023-06-09T11:18:00.528 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/19' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/15' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/20' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/12' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/10' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/11' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/16' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/14' 2023-06-09T11:18:00.529 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/18' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/13' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/194/17' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/194' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/19' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/15' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/20' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/12' 2023-06-09T11:18:00.530 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/10' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/11' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/16' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/14' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/18' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/13' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/119/17' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/119' 2023-06-09T11:18:00.531 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/19' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/15' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/20' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/12' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/10' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/11' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/16' 2023-06-09T11:18:00.532 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/14' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/18' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/13' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/157/17' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/157' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/19' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/15' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/20' 2023-06-09T11:18:00.533 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/12' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/10' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/11' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/16' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/14' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/18' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/13' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/172/17' 2023-06-09T11:18:00.534 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/172' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/19' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/15' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/20' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/12' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/10' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/11' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/16' 2023-06-09T11:18:00.535 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/14' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/18' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/13' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/186/17' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/186' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/19' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/15' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/20' 2023-06-09T11:18:00.536 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/12' 2023-06-09T11:18:00.537 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/10' 2023-06-09T11:18:00.537 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/11' 2023-06-09T11:18:00.537 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/16' 2023-06-09T11:18:00.537 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/14' 2023-06-09T11:18:01.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/18' 2023-06-09T11:18:01.197 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/13' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/190/17' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/190' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/19' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/15' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/20' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/12' 2023-06-09T11:18:01.198 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/10' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/11' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/16' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/14' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/18' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/13' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/104/17' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/104' 2023-06-09T11:18:01.199 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/19' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/15' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/20' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/12' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/10' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/11' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/16' 2023-06-09T11:18:01.200 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/14' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/18' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/13' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/164/17' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/164' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/19' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/15' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/20' 2023-06-09T11:18:01.201 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/12' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/10' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/11' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/16' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/14' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/18' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/13' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/112/17' 2023-06-09T11:18:01.202 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/112' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/19' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/15' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/20' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/12' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/10' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/11' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/16' 2023-06-09T11:18:01.203 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/14' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/18' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/13' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/100/17' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/100' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/19' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/15' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/20' 2023-06-09T11:18:01.204 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/12' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/10' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/11' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/16' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/14' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/18' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/13' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/134/17' 2023-06-09T11:18:01.205 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/134' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/19' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/15' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/20' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/12' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/10' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/11' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/16' 2023-06-09T11:18:01.206 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/14' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/18' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/13' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/115/17' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/115' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/19' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/15' 2023-06-09T11:18:01.207 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/20' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/12' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/10' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/11' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/16' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/14' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/18' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/13' 2023-06-09T11:18:01.208 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/116/17' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/116' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/19' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/15' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/20' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/12' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/10' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/11' 2023-06-09T11:18:01.209 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/16' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/14' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/18' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/13' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/195/17' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/195' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/19' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/15' 2023-06-09T11:18:01.210 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/20' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/12' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/10' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/11' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/16' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/14' 2023-06-09T11:18:01.211 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/18' 2023-06-09T11:18:02.218 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/13' 2023-06-09T11:18:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/138/17' 2023-06-09T11:18:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/138' 2023-06-09T11:18:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/19' 2023-06-09T11:18:02.219 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/15' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/20' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/12' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/10' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/11' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/16' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/14' 2023-06-09T11:18:02.220 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/18' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/13' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/135/17' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/135' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/19' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/15' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/20' 2023-06-09T11:18:02.221 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/12' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/10' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/11' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/16' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/14' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/18' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/13' 2023-06-09T11:18:02.222 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/117/17' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/117' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/19' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/15' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/20' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/12' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/10' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/11' 2023-06-09T11:18:02.223 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/16' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/14' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/18' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/13' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/143/17' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/143' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/19' 2023-06-09T11:18:02.224 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/15' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/20' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/12' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/10' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/11' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/16' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/14' 2023-06-09T11:18:02.225 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/18' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/13' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/180/17' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/180' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/19' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/15' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/20' 2023-06-09T11:18:02.226 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/12' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/10' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/11' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/16' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/14' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/18' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/13' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/126/17' 2023-06-09T11:18:02.227 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/126' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/19' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/15' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/20' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/12' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/10' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/11' 2023-06-09T11:18:02.228 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/16' 2023-06-09T11:18:02.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/14' 2023-06-09T11:18:02.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/18' 2023-06-09T11:18:02.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/13' 2023-06-09T11:18:02.229 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/161/17' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/161' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/19' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/15' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/20' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/12' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/10' 2023-06-09T11:18:02.230 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/11' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/16' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/14' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/18' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/13' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/176/17' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/176' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/19' 2023-06-09T11:18:02.231 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/15' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/20' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/12' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/10' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/11' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/16' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/14' 2023-06-09T11:18:02.232 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/18' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/13' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/111/17' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/111' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/19' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/15' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/20' 2023-06-09T11:18:02.233 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/12' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/10' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/11' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/16' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/14' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/18' 2023-06-09T11:18:02.234 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/13' 2023-06-09T11:18:03.451 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/191/17' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/191' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/19' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/15' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/20' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/12' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/10' 2023-06-09T11:18:03.452 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/11' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/16' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/14' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/18' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/13' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/184/17' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/184' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/19' 2023-06-09T11:18:03.453 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/15' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/20' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/12' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/10' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/11' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/16' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/14' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/18' 2023-06-09T11:18:03.454 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/13' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/192/17' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/192' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/19' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/15' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/20' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/12' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/10' 2023-06-09T11:18:03.455 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/11' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/16' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/14' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/18' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/13' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/103/17' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/103' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/19' 2023-06-09T11:18:03.456 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/15' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/20' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/12' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/10' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/11' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/16' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/14' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/18' 2023-06-09T11:18:03.457 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/13' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/128/17' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/128' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/19' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/15' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/20' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/12' 2023-06-09T11:18:03.458 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/10' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/11' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/16' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/14' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/18' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/13' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/144/17' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/144' 2023-06-09T11:18:03.459 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/19' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/15' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/20' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/12' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/10' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/11' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/16' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/14' 2023-06-09T11:18:03.460 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/18' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/13' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/146/17' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/146' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/19' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/15' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/20' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/12' 2023-06-09T11:18:03.461 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/10' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/11' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/16' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/14' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/18' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/13' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/141/17' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/141' 2023-06-09T11:18:03.462 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/19' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/15' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/20' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/12' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/10' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/11' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/16' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/14' 2023-06-09T11:18:03.463 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/18' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/13' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/182/17' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/182' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/19' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/15' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/20' 2023-06-09T11:18:03.464 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/12' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/10' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/11' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/16' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/14' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/18' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/13' 2023-06-09T11:18:03.465 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/107/17' 2023-06-09T11:18:04.798 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/107' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/19' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/15' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/20' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/12' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/10' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/11' 2023-06-09T11:18:04.799 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/16' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/14' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/18' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/13' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/178/17' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/178' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/19' 2023-06-09T11:18:04.800 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/15' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/20' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/12' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/10' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/11' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/16' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/14' 2023-06-09T11:18:04.801 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/18' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/13' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/156/17' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/156' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/19' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/15' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/20' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/12' 2023-06-09T11:18:04.802 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/10' 2023-06-09T11:18:04.803 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/11' 2023-06-09T11:18:04.803 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/16' 2023-06-09T11:18:04.803 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/14' 2023-06-09T11:18:04.803 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/18' 2023-06-09T11:18:04.803 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/13' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/131/17' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/131' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/19' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/15' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/20' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/12' 2023-06-09T11:18:04.804 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/10' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/11' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/16' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/14' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/18' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/13' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/123/17' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/123' 2023-06-09T11:18:04.805 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/19' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/15' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/20' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/12' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/10' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/11' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/16' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/14' 2023-06-09T11:18:04.806 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/18' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/13' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/160/17' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/160' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/19' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/15' 2023-06-09T11:18:04.807 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/20' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/12' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/10' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/11' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/16' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/14' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/18' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/13' 2023-06-09T11:18:04.808 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/165/17' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/165' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/19' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/15' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/20' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/12' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/10' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/11' 2023-06-09T11:18:04.809 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/16' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/14' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/18' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/13' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/101/17' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/101' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/19' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/15' 2023-06-09T11:18:04.810 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/20' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/12' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/10' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/11' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/16' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/14' 2023-06-09T11:18:04.811 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/18' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/13' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/148/17' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/148' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/19' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/15' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/20' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/12' 2023-06-09T11:18:04.812 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/10' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/11' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/16' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/14' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/18' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/13' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/168/17' 2023-06-09T11:18:04.813 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/168' 2023-06-09T11:18:05.863 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/19' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/15' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/20' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/12' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/10' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/11' 2023-06-09T11:18:05.864 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/16' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/14' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/18' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/13' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/179/17' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/179' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/19' 2023-06-09T11:18:05.865 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/15' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/20' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/12' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/10' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/11' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/16' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/14' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/18' 2023-06-09T11:18:05.866 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/13' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/132/17' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/132' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/19' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/15' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/20' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/12' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/10' 2023-06-09T11:18:05.867 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/11' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/16' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/14' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/18' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/13' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/199/17' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/199' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/19' 2023-06-09T11:18:05.868 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/15' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/20' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/12' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/10' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/11' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/16' 2023-06-09T11:18:05.869 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/14' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/18' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/13' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/163/17' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/163' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/19' 2023-06-09T11:18:05.870 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/15' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/20' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/12' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/10' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/11' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/16' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/14' 2023-06-09T11:18:05.871 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/18' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/13' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/153/17' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/153' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/19' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/15' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/20' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/12' 2023-06-09T11:18:05.872 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/10' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/11' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/16' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/14' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/18' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/13' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/122/17' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/122' 2023-06-09T11:18:05.873 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/19' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/15' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/20' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/12' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/10' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/11' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/16' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/14' 2023-06-09T11:18:05.874 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/18' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/13' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/162/17' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/162' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/19' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/15' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/20' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/12' 2023-06-09T11:18:05.875 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/10' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/11' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/16' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/14' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/18' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/13' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/197/17' 2023-06-09T11:18:05.876 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/197' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/19' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/15' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/20' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/12' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/10' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/11' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/16' 2023-06-09T11:18:05.877 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/14' 2023-06-09T11:18:05.878 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/18' 2023-06-09T11:18:05.878 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/13' 2023-06-09T11:18:05.878 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/113/17' 2023-06-09T11:18:05.878 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/113' 2023-06-09T11:18:05.878 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/19' 2023-06-09T11:18:07.023 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/15' 2023-06-09T11:18:07.023 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/20' 2023-06-09T11:18:07.023 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/12' 2023-06-09T11:18:07.023 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/10' 2023-06-09T11:18:07.023 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/11' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/16' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/14' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/18' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/13' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/152/17' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/152' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/19' 2023-06-09T11:18:07.024 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/15' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/20' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/12' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/10' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/11' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/16' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/14' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/18' 2023-06-09T11:18:07.025 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/13' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/121/17' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/121' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/19' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/15' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/20' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/12' 2023-06-09T11:18:07.026 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/10' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/11' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/16' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/14' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/18' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/13' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/118/17' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/118' 2023-06-09T11:18:07.027 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/19' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/15' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/20' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/12' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/10' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/11' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/16' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/14' 2023-06-09T11:18:07.028 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/18' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/13' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/196/17' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/196' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/19' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/15' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/20' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/12' 2023-06-09T11:18:07.029 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/10' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/11' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/16' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/14' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/18' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/13' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/124/17' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/124' 2023-06-09T11:18:07.030 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/19' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/15' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/20' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/12' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/10' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/11' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/16' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/14' 2023-06-09T11:18:07.031 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/18' 2023-06-09T11:18:07.032 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/13' 2023-06-09T11:18:07.032 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/188/17' 2023-06-09T11:18:07.032 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/188' 2023-06-09T11:18:07.032 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/19' 2023-06-09T11:18:07.032 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/15' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/20' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/12' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/10' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/11' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/16' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/14' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/18' 2023-06-09T11:18:07.033 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/13' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/145/17' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/145' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/19' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/15' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/20' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/12' 2023-06-09T11:18:07.034 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/10' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/11' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/16' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/14' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/18' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/13' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/130/17' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/130' 2023-06-09T11:18:07.035 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/19' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/15' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/20' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/12' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/10' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/11' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/16' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/14' 2023-06-09T11:18:07.036 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/18' 2023-06-09T11:18:07.037 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/13' 2023-06-09T11:18:07.037 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/193/17' 2023-06-09T11:18:07.037 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/193' 2023-06-09T11:18:07.037 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/19' 2023-06-09T11:18:07.037 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/15' 2023-06-09T11:18:07.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/20' 2023-06-09T11:18:07.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/12' 2023-06-09T11:18:07.745 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/10' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/11' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/16' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/14' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/18' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/13' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/175/17' 2023-06-09T11:18:07.746 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/175' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/19' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/15' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/20' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/12' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/10' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/11' 2023-06-09T11:18:07.747 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/16' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/14' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/18' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/13' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/181/17' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/181' 2023-06-09T11:18:07.748 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/19' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/15' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/20' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/12' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/10' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/11' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/16' 2023-06-09T11:18:07.749 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/14' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/18' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/13' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/127/17' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/127' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/19' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/15' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/20' 2023-06-09T11:18:07.750 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/12' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/10' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/11' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/16' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/14' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/18' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/13' 2023-06-09T11:18:07.751 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/108/17' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/108' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/19' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/15' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/20' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/12' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/10' 2023-06-09T11:18:07.752 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/11' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/16' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/14' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/18' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/13' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/183/17' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/183' 2023-06-09T11:18:07.753 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/19' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/15' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/20' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/12' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/10' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/11' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/16' 2023-06-09T11:18:07.754 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/14' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/18' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/13' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/177/17' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/177' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/19' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/15' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/20' 2023-06-09T11:18:07.755 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/12' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/10' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/11' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/16' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/14' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/18' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/13' 2023-06-09T11:18:07.756 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/169/17' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/169' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/19' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/15' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/20' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/12' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/10' 2023-06-09T11:18:07.757 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/11' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/16' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/14' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/18' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/13' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/154/17' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/154' 2023-06-09T11:18:07.758 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/19' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/15' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/20' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/12' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/10' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/11' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/16' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/14' 2023-06-09T11:18:07.759 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/18' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/13' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/171/17' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/171' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/19' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/15' 2023-06-09T11:18:07.760 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/20' 2023-06-09T11:18:08.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/12' 2023-06-09T11:18:08.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/10' 2023-06-09T11:18:08.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/11' 2023-06-09T11:18:08.116 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/16' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/14' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/18' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/13' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/139/17' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/139' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/19' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/15' 2023-06-09T11:18:08.117 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/20' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/12' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/10' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/11' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/16' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/14' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/18' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/13' 2023-06-09T11:18:08.118 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/102/17' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/102' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/19' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/15' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/20' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/12' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/10' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/11' 2023-06-09T11:18:08.119 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/16' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/14' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/18' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/13' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed './copyofsnap1/test1/129/17' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1/129' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1/test1' 2023-06-09T11:18:08.120 INFO:tasks.workunit.client.0.smithi007.stdout:removed directory './copyofsnap1' 2023-06-09T11:18:08.121 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:08.122 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:08.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:08.212 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-authwb.sh... 2023-06-09T11:18:08.212 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-authwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-authwb.sh 2023-06-09T11:18:08.276 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2023-06-09T11:18:08.277 INFO:tasks.workunit.client.0.smithi007.stderr:+ touch foo 2023-06-09T11:18:08.281 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod +x foo 2023-06-09T11:18:08.284 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/s 2023-06-09T11:18:08.300 INFO:tasks.workunit.client.0.smithi007.stderr:+ find .snap/s/foo -executable 2023-06-09T11:18:08.300 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep foo 2023-06-09T11:18:08.309 INFO:tasks.workunit.client.0.smithi007.stdout:.snap/s/foo 2023-06-09T11:18:08.309 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir .snap/s 2023-06-09T11:18:08.365 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm foo 2023-06-09T11:18:08.371 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:18:08.372 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:08.372 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:08.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:08.461 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-capwb.sh... 2023-06-09T11:18:08.461 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-capwb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-capwb.sh 2023-06-09T11:18:08.534 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2023-06-09T11:18:08.535 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir foo 2023-06-09T11:18:08.539 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo x 2023-06-09T11:18:08.542 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:09.664 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir foo/.snap/ss 2023-06-09T11:18:09.686 INFO:tasks.workunit.client.0.smithi007.stderr:+ ln foo/x foo/xx 2023-06-09T11:18:09.706 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat foo/.snap/ss/x 2023-06-09T11:18:09.716 INFO:tasks.workunit.client.0.smithi007.stdout:x 2023-06-09T11:18:09.716 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir foo/.snap/ss 2023-06-09T11:18:09.730 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo a 2023-06-09T11:18:09.732 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo b 2023-06-09T11:18:09.735 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir foo/.snap/s 2023-06-09T11:18:09.751 INFO:tasks.workunit.client.0.smithi007.stderr:++ cat foo/.snap/s/a 2023-06-09T11:18:09.764 INFO:tasks.workunit.client.0.smithi007.stderr:+ r=a 2023-06-09T11:18:09.765 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' -z a ']' 2023-06-09T11:18:09.765 INFO:tasks.workunit.client.0.smithi007.stderr:+ ln foo/b foo/b2 2023-06-09T11:18:09.771 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat foo/.snap/s/b 2023-06-09T11:18:09.781 INFO:tasks.workunit.client.0.smithi007.stdout:b 2023-06-09T11:18:09.781 INFO:tasks.workunit.client.0.smithi007.stdout:this used to hang: 2023-06-09T11:18:09.782 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 'this used to hang:' 2023-06-09T11:18:09.782 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo more 2023-06-09T11:18:09.785 INFO:tasks.workunit.client.0.smithi007.stdout:oh, it didn't hang! good job. 2023-06-09T11:18:09.786 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 'oh, it didn'\''t hang! good job.' 2023-06-09T11:18:09.786 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat foo/b 2023-06-09T11:18:09.789 INFO:tasks.workunit.client.0.smithi007.stdout:b 2023-06-09T11:18:09.789 INFO:tasks.workunit.client.0.smithi007.stdout:more 2023-06-09T11:18:09.789 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir foo/.snap/s 2023-06-09T11:18:10.262 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm -r foo 2023-06-09T11:18:10.297 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:10.297 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:18:10.298 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:10.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:10.604 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-dir-rename.sh... 2023-06-09T11:18:10.605 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-dir-rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-dir-rename.sh 2023-06-09T11:18:10.671 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2023-06-09T11:18:10.672 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:10.676 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/bar 2023-06-09T11:18:10.694 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/foo 2023-06-09T11:18:10.705 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/foo 2023-06-09T11:18:10.717 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:10.724 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/bar/a 2023-06-09T11:18:10.736 INFO:tasks.workunit.client.0.smithi007.stdout: File: .snap/bar/a 2023-06-09T11:18:10.736 INFO:tasks.workunit.client.0.smithi007.stdout: Size: 0 Blocks: 1 IO Block: 4096 directory 2023-06-09T11:18:10.736 INFO:tasks.workunit.client.0.smithi007.stdout:Device: 2bh/43d Inode: 845524441765078 Links: 2 2023-06-09T11:18:10.736 INFO:tasks.workunit.client.0.smithi007.stdout:Access: (0775/drwxrwxr-x) Uid: ( 1000/ ubuntu) Gid: ( 1265/ ubuntu) 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stdout:Context: system_u:object_r:fusefs_t:s0 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stdout:Access: 2023-06-09 11:18:10.670690461 +0000 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stdout:Modify: 2023-06-09 11:18:10.670690461 +0000 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stdout:Change: 2023-06-09 11:18:10.670690461 +0000 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stdout: Birth: - 2023-06-09T11:18:10.737 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir .snap/bar 2023-06-09T11:18:10.877 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:18:10.878 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:10.878 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:10.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:10.956 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-double-null.sh... 2023-06-09T11:18:10.956 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-double-null.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-double-null.sh 2023-06-09T11:18:11.020 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2023-06-09T11:18:11.021 INFO:tasks.workunit.client.0.smithi007.stderr:++ seq 1 20 2023-06-09T11:18:11.022 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:11.022 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:11.027 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:11.028 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:11.044 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:11.056 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:11.059 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:11.229 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:11.258 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:11.258 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:11.258 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:11.259 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:11.259 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:11.259 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:11.259 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:11.259 INFO:tasks.workunit.client.0.smithi007.stdout: 145047 ? 00:00:00 ps 2023-06-09T11:18:11.263 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:11.288 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:11.295 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:11.309 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:11.310 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:11.315 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:11.315 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:11.332 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:11.346 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:11.350 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:11.488 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:11.506 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:11.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:11.506 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:11.507 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:11.507 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:11.507 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:11.507 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:11.507 INFO:tasks.workunit.client.0.smithi007.stdout: 145057 ? 00:00:00 ps 2023-06-09T11:18:11.510 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:11.529 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:11.770 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:11.780 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:11.781 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:11.785 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:11.786 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:11.801 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:11.811 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:11.814 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:11.911 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:11.928 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:11.928 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:11.929 INFO:tasks.workunit.client.0.smithi007.stdout: 145067 ? 00:00:00 ps 2023-06-09T11:18:11.933 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:11.951 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:11.956 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:11.966 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:11.966 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:11.971 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:11.971 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:11.985 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:12.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:12.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:12.697 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:12.700 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:12.864 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:12.882 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:12.883 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:12.883 INFO:tasks.workunit.client.0.smithi007.stdout: 145098 ? 00:00:00 ps 2023-06-09T11:18:12.886 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:13.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:13.340 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:14.273 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:14.273 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:14.278 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:14.278 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:14.295 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:14.307 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:14.311 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:14.513 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:14.531 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:14.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:14.531 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:14.531 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:14.532 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:14.532 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:14.532 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:14.532 INFO:tasks.workunit.client.0.smithi007.stdout: 145108 ? 00:00:00 ps 2023-06-09T11:18:14.536 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:14.557 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:14.563 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:14.572 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:14.572 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:14.577 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:14.578 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:14.613 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:14.625 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:14.628 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:14.704 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:14.722 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:14.722 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:14.722 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:14.722 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:14.722 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:14.723 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:14.723 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:14.723 INFO:tasks.workunit.client.0.smithi007.stdout: 145123 ? 00:00:00 ps 2023-06-09T11:18:14.727 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:14.746 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:14.751 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:14.757 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:14.758 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:14.762 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:14.763 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:14.777 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:14.788 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:14.791 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:14.862 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:14.879 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:14.879 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:14.879 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:14.879 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:14.879 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:14.880 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:14.880 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:14.880 INFO:tasks.workunit.client.0.smithi007.stdout: 145136 ? 00:00:00 ps 2023-06-09T11:18:14.884 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:14.903 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:14.908 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:14.914 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:14.915 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:14.919 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:14.920 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:14.935 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:14.946 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:14.949 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:15.029 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:15.046 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:15.046 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:15.046 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:15.047 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:15.047 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:15.047 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:15.047 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:15.047 INFO:tasks.workunit.client.0.smithi007.stdout: 145146 ? 00:00:00 ps 2023-06-09T11:18:15.051 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:15.071 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:15.076 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:15.385 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:15.386 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:15.390 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:15.390 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:15.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:15.416 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:15.419 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:15.506 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:15.523 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:15.524 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:15.524 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:15.524 INFO:tasks.workunit.client.0.smithi007.stdout: 145156 ? 00:00:00 ps 2023-06-09T11:18:15.527 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:15.547 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:15.552 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:15.558 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:15.558 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:15.563 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:15.563 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:15.579 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:15.858 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:15.862 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:15.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:15.972 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:15.973 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:15.974 INFO:tasks.workunit.client.0.smithi007.stdout: 145166 ? 00:00:00 ps 2023-06-09T11:18:15.978 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:15.998 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:16.004 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:16.010 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:16.010 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:16.015 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:16.015 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:16.031 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:16.043 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:16.046 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:16.113 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:16.131 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:16.132 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:16.132 INFO:tasks.workunit.client.0.smithi007.stdout: 145176 ? 00:00:00 ps 2023-06-09T11:18:16.136 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:16.155 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:16.161 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:16.167 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:16.167 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:16.172 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:16.172 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:16.189 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:16.199 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:16.202 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:16.272 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:16.296 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:16.296 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:16.296 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:16.297 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:16.297 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:16.297 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:16.298 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:16.298 INFO:tasks.workunit.client.0.smithi007.stdout: 145186 ? 00:00:00 ps 2023-06-09T11:18:16.302 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:16.327 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:16.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:16.342 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:16.342 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:16.349 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:16.349 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:16.369 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:16.384 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:16.388 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:16.556 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:16.578 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:16.578 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:16.578 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:16.579 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:16.579 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:16.579 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:16.579 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:16.579 INFO:tasks.workunit.client.0.smithi007.stdout: 145196 ? 00:00:00 ps 2023-06-09T11:18:16.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:16.608 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:16.614 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:16.621 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:16.622 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:16.627 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:16.627 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:16.644 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:16.656 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:16.660 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:16.848 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:16.866 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:16.866 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:16.866 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:16.866 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:16.866 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:16.867 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:16.867 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:16.867 INFO:tasks.workunit.client.0.smithi007.stdout: 145206 ? 00:00:00 ps 2023-06-09T11:18:16.871 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:16.892 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:16.897 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:17.237 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:17.238 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:17.243 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:17.243 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:17.257 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:17.270 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:17.273 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:17.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:17.412 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:17.413 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:17.413 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:17.413 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:17.413 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:17.413 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:17.414 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:17.414 INFO:tasks.workunit.client.0.smithi007.stdout: 145216 ? 00:00:00 ps 2023-06-09T11:18:17.417 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:17.444 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:17.450 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:17.460 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:17.460 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:17.467 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:17.468 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:17.486 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:17.498 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:17.503 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:17.657 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:17.675 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:17.676 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:17.676 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:17.676 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:17.676 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:17.676 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:17.677 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:17.677 INFO:tasks.workunit.client.0.smithi007.stdout: 145226 ? 00:00:00 ps 2023-06-09T11:18:17.681 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:17.702 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:17.707 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:17.713 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:17.713 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:17.718 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:17.718 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:17.733 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:17.744 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:17.747 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:17.817 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:17.834 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:17.834 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:17.835 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:17.835 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:17.835 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:17.835 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:17.835 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:17.836 INFO:tasks.workunit.client.0.smithi007.stdout: 145236 ? 00:00:00 ps 2023-06-09T11:18:17.838 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:18.526 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:18.534 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:18.545 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:18.546 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:18.551 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:18.552 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:18.847 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:18.860 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:18.863 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:19.041 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:19.058 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:19.058 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:19.059 INFO:tasks.workunit.client.0.smithi007.stdout: 145246 ? 00:00:00 ps 2023-06-09T11:18:19.063 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:19.083 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:19.088 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:19.094 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:19.095 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:19.100 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:19.100 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:19.115 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:19.315 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:19.318 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:19.451 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:19.471 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:19.472 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:19.473 INFO:tasks.workunit.client.0.smithi007.stdout: 145256 ? 00:00:00 ps 2023-06-09T11:18:19.477 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:19.498 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:19.504 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:19.512 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 20` 2023-06-09T11:18:19.512 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a 2023-06-09T11:18:19.518 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/one 2023-06-09T11:18:19.518 INFO:tasks.workunit.client.0.smithi007.stderr:+ cat 2023-06-09T11:18:19.534 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir a/.snap/two 2023-06-09T11:18:19.546 INFO:tasks.workunit.client.0.smithi007.stderr:+ chmod 777 a/foo 2023-06-09T11:18:19.550 INFO:tasks.workunit.client.0.smithi007.stderr:+ sync 2023-06-09T11:18:19.675 INFO:tasks.workunit.client.0.smithi007.stderr:+ ps 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: PID TTY TIME CMD 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: 1087 ? 00:00:00 systemd 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: 1090 ? 00:00:00 (sd-pam) 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: 85285 ? 00:00:01 sshd 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: 137471 ? 00:00:00 bash 2023-06-09T11:18:19.693 INFO:tasks.workunit.client.0.smithi007.stdout: 145018 ? 00:00:00 timeout 2023-06-09T11:18:19.694 INFO:tasks.workunit.client.0.smithi007.stdout: 145039 ? 00:00:00 snaptest-double 2023-06-09T11:18:19.694 INFO:tasks.workunit.client.0.smithi007.stdout: 145266 ? 00:00:00 ps 2023-06-09T11:18:19.697 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a/.snap/one a/.snap/two 2023-06-09T11:18:20.059 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm a/foo 2023-06-09T11:18:20.064 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir a 2023-06-09T11:18:20.070 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:18:20.071 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:20.071 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:20.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:20.152 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-estale.sh... 2023-06-09T11:18:20.153 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-estale.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-estale.sh 2023-06-09T11:18:20.217 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/foo 2023-06-09T11:18:20.236 INFO:tasks.workunit.client.0.smithi007.stdout:We want ENOENT, not ESTALE, here. 2023-06-09T11:18:20.236 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo 'We want ENOENT, not ESTALE, here.' 2023-06-09T11:18:20.236 INFO:tasks.workunit.client.0.smithi007.stderr:++ seq 1 100 2023-06-09T11:18:20.239 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.239 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/1 2023-06-09T11:18:20.240 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.246 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/1': No such file or directory 2023-06-09T11:18:20.247 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.247 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/2 2023-06-09T11:18:20.247 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.252 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/2': No such file or directory 2023-06-09T11:18:20.253 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.254 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/3 2023-06-09T11:18:20.255 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.262 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/3': No such file or directory 2023-06-09T11:18:20.263 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.264 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/4 2023-06-09T11:18:20.265 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.270 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/4': No such file or directory 2023-06-09T11:18:20.270 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.271 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/5 2023-06-09T11:18:20.271 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.277 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/5': No such file or directory 2023-06-09T11:18:20.277 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.278 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/6 2023-06-09T11:18:20.279 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.286 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/6': No such file or directory 2023-06-09T11:18:20.287 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.288 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/7 2023-06-09T11:18:20.288 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.294 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/7': No such file or directory 2023-06-09T11:18:20.294 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.295 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/8 2023-06-09T11:18:20.295 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.301 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/8': No such file or directory 2023-06-09T11:18:20.302 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.302 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/9 2023-06-09T11:18:20.303 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.309 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/9': No such file or directory 2023-06-09T11:18:20.309 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.310 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/10 2023-06-09T11:18:20.311 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.316 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/10': No such file or directory 2023-06-09T11:18:20.316 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.317 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/11 2023-06-09T11:18:20.317 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.323 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/11': No such file or directory 2023-06-09T11:18:20.324 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.324 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/12 2023-06-09T11:18:20.325 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.330 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/12': No such file or directory 2023-06-09T11:18:20.330 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.331 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/13 2023-06-09T11:18:20.332 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.336 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/13': No such file or directory 2023-06-09T11:18:20.337 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.338 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/14 2023-06-09T11:18:20.338 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.343 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/14': No such file or directory 2023-06-09T11:18:20.344 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.344 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/15 2023-06-09T11:18:20.344 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.349 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/15': No such file or directory 2023-06-09T11:18:20.349 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.350 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/16 2023-06-09T11:18:20.351 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.356 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/16': No such file or directory 2023-06-09T11:18:20.356 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.357 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/17 2023-06-09T11:18:20.357 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.362 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/17': No such file or directory 2023-06-09T11:18:20.363 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.363 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/18 2023-06-09T11:18:20.364 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.370 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/18': No such file or directory 2023-06-09T11:18:20.370 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.371 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/19 2023-06-09T11:18:20.371 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.376 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/19': No such file or directory 2023-06-09T11:18:20.376 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.377 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/20 2023-06-09T11:18:20.377 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.382 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/20': No such file or directory 2023-06-09T11:18:20.383 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.383 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/21 2023-06-09T11:18:20.384 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.388 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/21': No such file or directory 2023-06-09T11:18:20.389 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.389 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/22 2023-06-09T11:18:20.389 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.394 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/22': No such file or directory 2023-06-09T11:18:20.394 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.394 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/23 2023-06-09T11:18:20.395 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.399 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/23': No such file or directory 2023-06-09T11:18:20.400 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.400 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/24 2023-06-09T11:18:20.401 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.406 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/24': No such file or directory 2023-06-09T11:18:20.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.407 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/25 2023-06-09T11:18:20.407 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.413 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/25': No such file or directory 2023-06-09T11:18:20.413 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.414 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/26 2023-06-09T11:18:20.414 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.419 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/26': No such file or directory 2023-06-09T11:18:20.419 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.420 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/27 2023-06-09T11:18:20.420 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.425 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/27': No such file or directory 2023-06-09T11:18:20.425 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.426 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/28 2023-06-09T11:18:20.426 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.431 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/28': No such file or directory 2023-06-09T11:18:20.431 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.432 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/29 2023-06-09T11:18:20.432 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.437 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/29': No such file or directory 2023-06-09T11:18:20.438 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.438 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/30 2023-06-09T11:18:20.438 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.443 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/30': No such file or directory 2023-06-09T11:18:20.443 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.444 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/31 2023-06-09T11:18:20.444 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.448 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/31': No such file or directory 2023-06-09T11:18:20.449 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.449 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/32 2023-06-09T11:18:20.450 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.456 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/32': No such file or directory 2023-06-09T11:18:20.456 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/33 2023-06-09T11:18:20.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.461 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/33': No such file or directory 2023-06-09T11:18:20.461 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.462 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/34 2023-06-09T11:18:20.462 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.467 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/34': No such file or directory 2023-06-09T11:18:20.467 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.468 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/35 2023-06-09T11:18:20.468 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.472 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/35': No such file or directory 2023-06-09T11:18:20.473 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.473 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/36 2023-06-09T11:18:20.474 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.478 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/36': No such file or directory 2023-06-09T11:18:20.478 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.479 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/37 2023-06-09T11:18:20.479 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.484 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/37': No such file or directory 2023-06-09T11:18:20.484 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.484 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/38 2023-06-09T11:18:20.485 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.490 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/38': No such file or directory 2023-06-09T11:18:20.490 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.491 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/39 2023-06-09T11:18:20.491 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.495 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/39': No such file or directory 2023-06-09T11:18:20.496 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.496 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/40 2023-06-09T11:18:20.497 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.501 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/40': No such file or directory 2023-06-09T11:18:20.502 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.502 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/41 2023-06-09T11:18:20.502 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.507 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/41': No such file or directory 2023-06-09T11:18:20.508 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.508 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/42 2023-06-09T11:18:20.509 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.513 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/42': No such file or directory 2023-06-09T11:18:20.513 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.514 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/43 2023-06-09T11:18:20.514 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.519 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/43': No such file or directory 2023-06-09T11:18:20.519 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.520 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/44 2023-06-09T11:18:20.520 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.524 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/44': No such file or directory 2023-06-09T11:18:20.525 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.525 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/45 2023-06-09T11:18:20.525 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.530 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/45': No such file or directory 2023-06-09T11:18:20.530 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.530 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/46 2023-06-09T11:18:20.531 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.537 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/46': No such file or directory 2023-06-09T11:18:20.537 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.538 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/47 2023-06-09T11:18:20.538 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.542 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/47': No such file or directory 2023-06-09T11:18:20.543 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.543 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/48 2023-06-09T11:18:20.544 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.549 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/48': No such file or directory 2023-06-09T11:18:20.550 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.550 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/49 2023-06-09T11:18:20.551 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.555 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/49': No such file or directory 2023-06-09T11:18:20.556 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.556 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/50 2023-06-09T11:18:20.557 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.562 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/50': No such file or directory 2023-06-09T11:18:20.562 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.562 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/51 2023-06-09T11:18:20.563 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.567 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/51': No such file or directory 2023-06-09T11:18:20.568 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.568 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/52 2023-06-09T11:18:20.569 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:20.573 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/52': No such file or directory 2023-06-09T11:18:20.574 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:20.574 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/53 2023-06-09T11:18:20.575 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.423 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/53': No such file or directory 2023-06-09T11:18:21.423 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.424 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/54 2023-06-09T11:18:21.424 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.430 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/54': No such file or directory 2023-06-09T11:18:21.430 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.431 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/55 2023-06-09T11:18:21.431 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.437 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/55': No such file or directory 2023-06-09T11:18:21.437 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.437 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/56 2023-06-09T11:18:21.438 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.443 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/56': No such file or directory 2023-06-09T11:18:21.444 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.444 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/57 2023-06-09T11:18:21.445 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.450 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/57': No such file or directory 2023-06-09T11:18:21.450 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.451 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/58 2023-06-09T11:18:21.451 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.456 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/58': No such file or directory 2023-06-09T11:18:21.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/59 2023-06-09T11:18:21.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.463 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/59': No such file or directory 2023-06-09T11:18:21.463 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.464 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/60 2023-06-09T11:18:21.464 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.469 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/60': No such file or directory 2023-06-09T11:18:21.469 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.470 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/61 2023-06-09T11:18:21.470 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.474 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/61': No such file or directory 2023-06-09T11:18:21.475 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.475 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/62 2023-06-09T11:18:21.476 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.480 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/62': No such file or directory 2023-06-09T11:18:21.480 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.481 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/63 2023-06-09T11:18:21.481 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.486 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/63': No such file or directory 2023-06-09T11:18:21.487 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.487 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/64 2023-06-09T11:18:21.488 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:21.493 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/64': No such file or directory 2023-06-09T11:18:21.493 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:21.493 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/65 2023-06-09T11:18:21.494 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.021 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/65': No such file or directory 2023-06-09T11:18:22.023 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.023 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/66 2023-06-09T11:18:22.023 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.352 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/66': No such file or directory 2023-06-09T11:18:22.353 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.353 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/67 2023-06-09T11:18:22.354 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.361 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/67': No such file or directory 2023-06-09T11:18:22.361 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.362 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/68 2023-06-09T11:18:22.362 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.368 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/68': No such file or directory 2023-06-09T11:18:22.368 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.369 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/69 2023-06-09T11:18:22.369 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.375 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/69': No such file or directory 2023-06-09T11:18:22.376 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.376 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/70 2023-06-09T11:18:22.377 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.383 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/70': No such file or directory 2023-06-09T11:18:22.383 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.384 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/71 2023-06-09T11:18:22.384 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.390 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/71': No such file or directory 2023-06-09T11:18:22.390 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/72 2023-06-09T11:18:22.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.954 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/72': No such file or directory 2023-06-09T11:18:22.954 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/73 2023-06-09T11:18:22.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.962 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/73': No such file or directory 2023-06-09T11:18:22.963 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.963 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/74 2023-06-09T11:18:22.964 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.969 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/74': No such file or directory 2023-06-09T11:18:22.970 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.970 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/75 2023-06-09T11:18:22.971 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.976 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/75': No such file or directory 2023-06-09T11:18:22.977 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.977 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/76 2023-06-09T11:18:22.978 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.983 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/76': No such file or directory 2023-06-09T11:18:22.984 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.984 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/77 2023-06-09T11:18:22.985 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.991 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/77': No such file or directory 2023-06-09T11:18:22.991 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.992 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/78 2023-06-09T11:18:22.992 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:22.998 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/78': No such file or directory 2023-06-09T11:18:22.998 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:22.999 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/79 2023-06-09T11:18:23.000 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.006 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/79': No such file or directory 2023-06-09T11:18:23.006 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.007 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/80 2023-06-09T11:18:23.007 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.013 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/80': No such file or directory 2023-06-09T11:18:23.013 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.014 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/81 2023-06-09T11:18:23.014 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.020 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/81': No such file or directory 2023-06-09T11:18:23.020 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.021 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/82 2023-06-09T11:18:23.021 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.027 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/82': No such file or directory 2023-06-09T11:18:23.027 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.028 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/83 2023-06-09T11:18:23.028 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.035 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/83': No such file or directory 2023-06-09T11:18:23.035 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.036 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/84 2023-06-09T11:18:23.036 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.042 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/84': No such file or directory 2023-06-09T11:18:23.043 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.043 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/85 2023-06-09T11:18:23.043 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.049 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/85': No such file or directory 2023-06-09T11:18:23.049 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.050 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/86 2023-06-09T11:18:23.050 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.056 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/86': No such file or directory 2023-06-09T11:18:23.056 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.057 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/87 2023-06-09T11:18:23.057 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.062 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/87': No such file or directory 2023-06-09T11:18:23.063 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.063 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/88 2023-06-09T11:18:23.064 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.069 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/88': No such file or directory 2023-06-09T11:18:23.069 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.070 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/89 2023-06-09T11:18:23.070 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.076 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/89': No such file or directory 2023-06-09T11:18:23.076 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.077 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/90 2023-06-09T11:18:23.077 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.082 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/90': No such file or directory 2023-06-09T11:18:23.082 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.083 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/91 2023-06-09T11:18:23.083 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.089 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/91': No such file or directory 2023-06-09T11:18:23.089 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.090 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/92 2023-06-09T11:18:23.090 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.095 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/92': No such file or directory 2023-06-09T11:18:23.096 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.096 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/93 2023-06-09T11:18:23.096 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.103 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/93': No such file or directory 2023-06-09T11:18:23.103 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.103 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/94 2023-06-09T11:18:23.104 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.109 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/94': No such file or directory 2023-06-09T11:18:23.109 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.109 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/95 2023-06-09T11:18:23.110 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.115 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/95': No such file or directory 2023-06-09T11:18:23.116 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.116 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/96 2023-06-09T11:18:23.116 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.122 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/96': No such file or directory 2023-06-09T11:18:23.122 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/97 2023-06-09T11:18:23.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.128 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/97': No such file or directory 2023-06-09T11:18:23.129 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.129 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/98 2023-06-09T11:18:23.130 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.134 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/98': No such file or directory 2023-06-09T11:18:23.135 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.135 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/99 2023-06-09T11:18:23.136 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.141 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/99': No such file or directory 2023-06-09T11:18:23.141 INFO:tasks.workunit.client.0.smithi007.stderr:+ for f in `seq 1 100` 2023-06-09T11:18:23.142 INFO:tasks.workunit.client.0.smithi007.stderr:+ stat .snap/foo/100 2023-06-09T11:18:23.142 INFO:tasks.workunit.client.0.smithi007.stderr:+ grep 'No such file' 2023-06-09T11:18:23.358 INFO:tasks.workunit.client.0.smithi007.stdout:stat: cannot statx '.snap/foo/100': No such file or directory 2023-06-09T11:18:23.358 INFO:tasks.workunit.client.0.smithi007.stderr:+ rmdir .snap/foo 2023-06-09T11:18:23.373 INFO:tasks.workunit.client.0.smithi007.stdout:OK 2023-06-09T11:18:23.374 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo OK 2023-06-09T11:18:23.374 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-06-09T11:18:23.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-06-09T11:18:23.451 INFO:tasks.workunit:Running workunit fs/snaps/snaptest-git-ceph.sh... 2023-06-09T11:18:23.451 DEBUG:teuthology.orchestra.run.smithi007:workunit test fs/snaps/snaptest-git-ceph.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh 2023-06-09T11:18:23.516 INFO:tasks.workunit.client.0.smithi007.stderr:+ set -e 2023-06-09T11:18:23.516 INFO:tasks.workunit.client.0.smithi007.stderr:+ retried=false 2023-06-09T11:18:23.516 INFO:tasks.workunit.client.0.smithi007.stderr:+ trap -- retry EXIT 2023-06-09T11:18:23.516 INFO:tasks.workunit.client.0.smithi007.stderr:+ rm -rf ceph 2023-06-09T11:18:23.518 INFO:tasks.workunit.client.0.smithi007.stderr:+ timeout 1800 git clone https://git.ceph.com/ceph.git 2023-06-09T11:18:23.528 INFO:tasks.workunit.client.0.smithi007.stderr:Cloning into 'ceph'... 2023-06-09T11:18:42.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:18:42.149 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:12.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:12.320 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:42.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:42.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:19:46.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:46.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.701+0000 7f97d43ff700 1 Processor -- start 2023-06-09T11:19:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.701+0000 7f97d43ff700 1 -- start start 2023-06-09T11:19:46.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d43ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f97cc0cb5d0 0x7f97cc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d43ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97cc15cd40 0x7f97cc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d43ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc159b20 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d43ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc0cac80 con 0x7f97cc15cd40 2023-06-09T11:19:46.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d43ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc1597d0 con 0x7f97cc0cb5d0 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d299c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97cc15cd40 0x7f97cc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d199a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f97cc0cb9a0 0x7f97cc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51362/0 (socket says 172.21.15.7:51362) 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d299c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97cc15cd40 0x7f97cc0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38988/0 (socket says 172.21.15.7:38988) 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d199a700 1 -- 172.21.15.7:0/612390583 learned_addr learned my addr 172.21.15.7:0/612390583 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.702+0000 7f97d219b700 1 --2- 172.21.15.7:0/612390583 >> v2:172.21.15.117:3300/0 conn(0x7f97cc0cb5d0 0x7f97cc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d299c700 1 -- 172.21.15.7:0/612390583 >> v2:172.21.15.117:3300/0 conn(0x7f97cc0cb5d0 msgr2=0x7f97cc0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d299c700 1 --2- 172.21.15.7:0/612390583 >> v2:172.21.15.117:3300/0 conn(0x7f97cc0cb5d0 0x7f97cc0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d299c700 1 -- 172.21.15.7:0/612390583 >> v1:172.21.15.7:6789/0 conn(0x7f97cc0cb9a0 legacy=0x7f97cc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d299c700 1 -- 172.21.15.7:0/612390583 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc071ab0 con 0x7f97cc15cd40 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d1199700 1 -- 172.21.15.7:0/612390583 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991544249 0 0) 0x7f97cc159b20 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d299c700 1 --2- 172.21.15.7:0/612390583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc15cd40 0x7f97cc0cbea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f97cc08d420 tx=0x7f97c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f4a1c2bf7e93f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d1199700 1 -- 172.21.15.7:0/612390583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400e070 con 0x7f97cc15cd40 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d1199700 1 -- 172.21.15.7:0/612390583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c400bb20 con 0x7f97cc15cd40 2023-06-09T11:19:46.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.703+0000 7f97d1199700 1 -- 172.21.15.7:0/612390583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4012430 con 0x7f97cc15cd40 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 -- 172.21.15.7:0/612390583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc15cd40 msgr2=0x7f97cc0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 --2- 172.21.15.7:0/612390583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc15cd40 0x7f97cc0cbea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f97cc08d420 tx=0x7f97c400b330 comp rx=0 tx=0).stop 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 -- 172.21.15.7:0/612390583 shutdown_connections 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 --2- 172.21.15.7:0/612390583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc15cd40 0x7f97cc0cbea0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 --2- 172.21.15.7:0/612390583 >> v2:172.21.15.117:3300/0 conn(0x7f97cc0cb5d0 0x7f97cc0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 -- 172.21.15.7:0/612390583 >> 172.21.15.7:0/612390583 conn(0x7f97cc070af0 msgr2=0x7f97cc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 -- 172.21.15.7:0/612390583 shutdown_connections 2023-06-09T11:19:46.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.704+0000 7f97d43ff700 1 -- 172.21.15.7:0/612390583 wait complete. 2023-06-09T11:19:46.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.706+0000 7f97d43ff700 1 Processor -- start 2023-06-09T11:19:46.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.707+0000 7f97d43ff700 1 -- start start 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97cc0cb5d0 0x7f97cc0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 0x7f97cc1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc1573f0 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc1591a0 con 0x7f97cc0cb5d0 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d43ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc1528e0 con 0x7f97cc15cd40 2023-06-09T11:19:46.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39000/0 (socket says 172.21.15.7:39000) 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d199a700 1 -- 172.21.15.7:0/474306159 learned_addr learned my addr 172.21.15.7:0/474306159 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d219b700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97cc0cb5d0 0x7f97cc0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.708+0000 7f97d299c700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 0x7f97cc1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 -- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 msgr2=0x7f97cc1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 0x7f97cc1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:46.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 -- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97cc0cb5d0 msgr2=0x7f97cc0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97cc0cb5d0 0x7f97cc0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 -- 172.21.15.7:0/474306159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009a70 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d299c700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 0x7f97cc1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d199a700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c540 tx=0x7f97bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc004c80 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc004de0 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc15d4f0 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc007310 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.709+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc15d210 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.710+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc15d7d0 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.711+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97bc007470 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.712+0000 7f97c2ffd700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 0x7f97b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.712+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7f97bc089370 con 0x7f97cc0cb9a0 2023-06-09T11:19:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.712+0000 7f97d219b700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 0x7f97b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:46.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.713+0000 7f97d219b700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 0x7f97b006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009be0 tx=0x7f97c8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:46.717+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f97bc054260 con 0x7f97cc0cb9a0 2023-06-09T11:19:47.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.015+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97cc1534f0 con 0x7f97cc0cb9a0 2023-06-09T11:19:47.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.016+0000 7f97c2ffd700 1 -- 172.21.15.7:0/474306159 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7f97bc057880 con 0x7f97cc0cb9a0 2023-06-09T11:19:47.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:47.021 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:47.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 msgr2=0x7f97b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 0x7f97b006d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f97c8009be0 tx=0x7f97c8009210 comp rx=0 tx=0).stop 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 msgr2=0x7f97cc1f1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00c540 tx=0x7f97bc00c850 comp rx=0 tx=0).stop 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 shutdown_connections 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f97b006ace0 0x7f97b006d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97cc15cd40 0x7f97cc1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97cc0cb5d0 0x7f97cc0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 --2- 172.21.15.7:0/474306159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97cc0cb9a0 0x7f97cc1f1ce0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 >> 172.21.15.7:0/474306159 conn(0x7f97cc070af0 msgr2=0x7f97cc0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 shutdown_connections 2023-06-09T11:19:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.019+0000 7f97d43ff700 1 -- 172.21.15.7:0/474306159 wait complete. 2023-06-09T11:19:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:47.043 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2023-06-09T11:19:47.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:19:47.044 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:19:47.063 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:19:47.064 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.259+0000 7f5f215ff700 1 Processor -- start 2023-06-09T11:19:47.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.259+0000 7f5f215ff700 1 -- start start 2023-06-09T11:19:47.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.260+0000 7f5f215ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f1c1535e0 0x7f5f1c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.260+0000 7f5f215ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5f1c1539b0 0x7f5f1c0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.260+0000 7f5f215ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1c15a420 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.260+0000 7f5f215ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c5a00 con 0x7f5f1c1535e0 2023-06-09T11:19:47.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.260+0000 7f5f215ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1c06b460 con 0x7f5f1c1539b0 2023-06-09T11:19:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f1c1535e0 0x7f5f1c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5f1c0c4ce0 0x7f5f1c15abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51366/0 (socket says 172.21.15.7:51366) 2023-06-09T11:19:47.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1b7fe700 1 -- 172.21.15.7:0/4207955930 learned_addr learned my addr 172.21.15.7:0/4207955930 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:19:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1a7fc700 1 --2- 172.21.15.7:0/4207955930 >> v2:172.21.15.117:3300/0 conn(0x7f5f1c1539b0 0x7f5f1c0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 -- 172.21.15.7:0/4207955930 >> v2:172.21.15.117:3300/0 conn(0x7f5f1c1539b0 msgr2=0x7f5f1c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 --2- 172.21.15.7:0/4207955930 >> v2:172.21.15.117:3300/0 conn(0x7f5f1c1539b0 0x7f5f1c0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 -- 172.21.15.7:0/4207955930 >> v1:172.21.15.7:6789/0 conn(0x7f5f1c0c4ce0 legacy=0x7f5f1c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f19ffb700 1 -- 172.21.15.7:0/4207955930 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531625897 0 0) 0x7f5f14001490 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 -- 172.21.15.7:0/4207955930 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c06a570 con 0x7f5f1c1535e0 2023-06-09T11:19:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.261+0000 7f5f1affd700 1 --2- 172.21.15.7:0/4207955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c159ee0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c08d1e0 tx=0x7f5f0c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb50661cc9a6db1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:47.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f19ffb700 1 -- 172.21.15.7:0/4207955930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011830 con 0x7f5f1c1535e0 2023-06-09T11:19:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f19ffb700 1 -- 172.21.15.7:0/4207955930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c011990 con 0x7f5f1c1535e0 2023-06-09T11:19:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f19ffb700 1 -- 172.21.15.7:0/4207955930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c011d20 con 0x7f5f1c1535e0 2023-06-09T11:19:47.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 -- 172.21.15.7:0/4207955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c1535e0 msgr2=0x7f5f1c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 --2- 172.21.15.7:0/4207955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c159ee0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c08d1e0 tx=0x7f5f0c009b20 comp rx=0 tx=0).stop 2023-06-09T11:19:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 -- 172.21.15.7:0/4207955930 shutdown_connections 2023-06-09T11:19:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 --2- 172.21.15.7:0/4207955930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c159ee0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 --2- 172.21.15.7:0/4207955930 >> v2:172.21.15.117:3300/0 conn(0x7f5f1c1539b0 0x7f5f1c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 -- 172.21.15.7:0/4207955930 >> 172.21.15.7:0/4207955930 conn(0x7f5f1c0737f0 msgr2=0x7f5f1c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 -- 172.21.15.7:0/4207955930 shutdown_connections 2023-06-09T11:19:47.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.262+0000 7f5f215ff700 1 -- 172.21.15.7:0/4207955930 wait complete. 2023-06-09T11:19:47.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.265+0000 7f5f215ff700 1 Processor -- start 2023-06-09T11:19:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.265+0000 7f5f215ff700 1 -- start start 2023-06-09T11:19:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c0b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c1539b0 0x7f5f1c14e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c6c90 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c6f70 con 0x7f5f1c1535e0 2023-06-09T11:19:47.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f215ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1c0c7250 con 0x7f5f1c1539b0 2023-06-09T11:19:47.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:47.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f1affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39024/0 (socket says 172.21.15.7:39024) 2023-06-09T11:19:47.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.266+0000 7f5f1affd700 1 -- 172.21.15.7:0/2058346208 learned_addr learned my addr 172.21.15.7:0/2058346208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:47.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1a7fc700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c0b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 -- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c1539b0 msgr2=0x7f5f1c14e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c1539b0 0x7f5f1c14e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 -- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 msgr2=0x7f5f1c0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c0b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 -- 172.21.15.7:0/2058346208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0094a0 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1a7fc700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:19:47.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f1affd700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0090b0 tx=0x7f5f0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c0038a0 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c01f070 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1c06ab30 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.267+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c003c20 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.268+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1c06a850 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.268+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c06ae10 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.269+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f0c026480 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.270+0000 7f5f037fe700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 0x7f5f0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.270+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7f5f0c021070 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.271+0000 7f5f1a7fc700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 0x7f5f0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:47.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.271+0000 7f5f1a7fc700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 0x7f5f0806d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c14f490 tx=0x7f5f10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:47.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.275+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5f0c055fa0 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.528+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f1c074cf0 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.529+0000 7f5f037fe700 1 -- 172.21.15.7:0/2058346208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7f5f0c015090 con 0x7f5f1c0c4ce0 2023-06-09T11:19:47.534 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:47.534 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 msgr2=0x7f5f0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 0x7f5f0806d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c14f490 tx=0x7f5f10008040 comp rx=0 tx=0).stop 2023-06-09T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 msgr2=0x7f5f1c1ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0090b0 tx=0x7f5f0c004750 comp rx=0 tx=0).stop 2023-06-09T11:19:47.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 shutdown_connections 2023-06-09T11:19:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5f0806ace0 0x7f5f0806d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f1c1539b0 0x7f5f1c14e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f1c0c4ce0 0x7f5f1c1ef620 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 --2- 172.21.15.7:0/2058346208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f1c1535e0 0x7f5f1c0b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 >> 172.21.15.7:0/2058346208 conn(0x7f5f1c0737f0 msgr2=0x7f5f1c152240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 shutdown_connections 2023-06-09T11:19:47.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:47.532+0000 7f5f215ff700 1 -- 172.21.15.7:0/2058346208 wait complete. 2023-06-09T11:19:47.540 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:47.555 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:19:49.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:49.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.761+0000 7efc675ff700 1 Processor -- start 2023-06-09T11:19:49.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.761+0000 7efc675ff700 1 -- start start 2023-06-09T11:19:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc60153880 0x7efc6015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc600c9f40 0x7efc6015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc6015b2d0 0x7efc600c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc600c5f10 con 0x7efc60153880 2023-06-09T11:19:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6015a500 con 0x7efc6015b2d0 2023-06-09T11:19:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc675ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6015da00 con 0x7efc600c9f40 2023-06-09T11:19:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc6539b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc60153880 0x7efc6015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc6539b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc60153880 0x7efc6015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39036/0 (socket says 172.21.15.7:39036) 2023-06-09T11:19:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.762+0000 7efc6539b700 1 -- 172.21.15.7:0/123751956 learned_addr learned my addr 172.21.15.7:0/123751956 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc65b9c700 1 --2- 172.21.15.7:0/123751956 >> v2:172.21.15.117:3300/0 conn(0x7efc6015b2d0 0x7efc600c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc64b9a700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc600c9f40 0x7efc6015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 -- 172.21.15.7:0/123751956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc600c9f40 msgr2=0x7efc6015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc600c9f40 0x7efc6015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 -- 172.21.15.7:0/123751956 >> v2:172.21.15.117:3300/0 conn(0x7efc6015b2d0 msgr2=0x7efc600c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 --2- 172.21.15.7:0/123751956 >> v2:172.21.15.117:3300/0 conn(0x7efc6015b2d0 0x7efc600c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 -- 172.21.15.7:0/123751956 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6015b750 con 0x7efc60153880 2023-06-09T11:19:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc6539b700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6015d4c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc60071db0 tx=0x7efc5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63c516e0b1ea282 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.763+0000 7efc57fff700 1 -- 172.21.15.7:0/123751956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00b870 con 0x7efc60153880 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc57fff700 1 -- 172.21.15.7:0/123751956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5c00b9d0 con 0x7efc60153880 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc57fff700 1 -- 172.21.15.7:0/123751956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00bd60 con 0x7efc60153880 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 -- 172.21.15.7:0/123751956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 msgr2=0x7efc6015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6015d4c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7efc60071db0 tx=0x7efc5c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 -- 172.21.15.7:0/123751956 shutdown_connections 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 --2- 172.21.15.7:0/123751956 >> v2:172.21.15.117:3300/0 conn(0x7efc6015b2d0 0x7efc600c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6015d4c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 --2- 172.21.15.7:0/123751956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc600c9f40 0x7efc6015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 -- 172.21.15.7:0/123751956 >> 172.21.15.7:0/123751956 conn(0x7efc600737f0 msgr2=0x7efc600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 -- 172.21.15.7:0/123751956 shutdown_connections 2023-06-09T11:19:49.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.764+0000 7efc675ff700 1 -- 172.21.15.7:0/123751956 wait complete. 2023-06-09T11:19:49.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.766+0000 7efc675ff700 1 Processor -- start 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.766+0000 7efc675ff700 1 -- start start 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc600c9f40 0x7efc600cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 0x7efc601eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc601545a0 con 0x7efc60153880 2023-06-09T11:19:49.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600cb6e0 con 0x7efc600c9f40 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc675ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc600cbc10 con 0x7efc6015b2d0 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc64b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc64b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39052/0 (socket says 172.21.15.7:39052) 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.767+0000 7efc64b9a700 1 -- 172.21.15.7:0/1830537460 learned_addr learned my addr 172.21.15.7:0/1830537460 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc6539b700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc600c9f40 0x7efc600cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc65b9c700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 0x7efc601eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 -- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 msgr2=0x7efc601eeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:49.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 0x7efc601eeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 -- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc600c9f40 msgr2=0x7efc600cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc600c9f40 0x7efc600cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 -- 172.21.15.7:0/1830537460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c009550 con 0x7efc60153880 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc65b9c700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 0x7efc601eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc64b9a700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e840 tx=0x7efc5000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50013610 con 0x7efc60153880 2023-06-09T11:19:49.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc50013770 con 0x7efc60153880 2023-06-09T11:19:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc60159980 con 0x7efc60153880 2023-06-09T11:19:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50013b00 con 0x7efc60153880 2023-06-09T11:19:49.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.768+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc601596e0 con 0x7efc60153880 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.769+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc60159c60 con 0x7efc60153880 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.770+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc50013c60 con 0x7efc60153880 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.771+0000 7efc55ffb700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 0x7efc4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.771+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7efc50089360 con 0x7efc60153880 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.772+0000 7efc6539b700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 0x7efc4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:49.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.772+0000 7efc6539b700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 0x7efc4406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000c00 tx=0x7efc5c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:49.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:49.776+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc50054130 con 0x7efc60153880 2023-06-09T11:19:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.025+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc60074cf0 con 0x7efc60153880 2023-06-09T11:19:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.025+0000 7efc55ffb700 1 -- 172.21.15.7:0/1830537460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7efc50009440 con 0x7efc60153880 2023-06-09T11:19:50.030 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:50.030 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:50.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.027+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 msgr2=0x7efc4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.027+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 0x7efc4406d190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7efc5c000c00 tx=0x7efc5c008030 comp rx=0 tx=0).stop 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.027+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 msgr2=0x7efc6014ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.027+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc5000e840 tx=0x7efc5000ec00 comp rx=0 tx=0).stop 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 shutdown_connections 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc4406ace0 0x7efc4406d190 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc6015b2d0 0x7efc601eeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc600c9f40 0x7efc600cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 --2- 172.21.15.7:0/1830537460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc60153880 0x7efc6014ed30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 >> 172.21.15.7:0/1830537460 conn(0x7efc600737f0 msgr2=0x7efc6014dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 shutdown_connections 2023-06-09T11:19:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:50.028+0000 7efc675ff700 1 -- 172.21.15.7:0/1830537460 wait complete. 2023-06-09T11:19:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:50.050 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:19:52.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:52.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.259+0000 7f0099bff700 1 Processor -- start 2023-06-09T11:19:52.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.259+0000 7f0099bff700 1 -- start start 2023-06-09T11:19:52.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0099bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 0x7f009415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0099bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0099bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009415a500 con 0x7f00940c9f40 2023-06-09T11:19:52.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0099bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00940c5f10 con 0x7f0094153880 2023-06-09T11:19:52.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0099bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009415da00 con 0x7f009415b4a0 2023-06-09T11:19:52.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.260+0000 7f0093fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54814/0 (socket says 172.21.15.7:54814) 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0092ffd700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f00940c9f40 0x7f009415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:55376/0 (socket says 172.21.15.7:55376) 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 -- 172.21.15.7:0/2159037597 learned_addr learned my addr 172.21.15.7:0/2159037597 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f00937fe700 1 --2- 172.21.15.7:0/2159037597 >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 0x7f009415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f00927fc700 1 -- 172.21.15.7:0/2159037597 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581031457 0 0) 0x7f0088001110 con 0x7f00940c9f40 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f00927fc700 1 -- 172.21.15.7:0/2159037597 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f007c0035b0 con 0x7f00940c9f40 2023-06-09T11:19:52.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 -- 172.21.15.7:0/2159037597 >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 msgr2=0x7f009415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 --2- 172.21.15.7:0/2159037597 >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 0x7f009415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 -- 172.21.15.7:0/2159037597 >> v1:172.21.15.117:6789/0 conn(0x7f00940c9f40 legacy=0x7f009415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 -- 172.21.15.7:0/2159037597 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00941596e0 con 0x7f009415b4a0 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f00937fe700 1 --2- 172.21.15.7:0/2159037597 >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 0x7f009415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.261+0000 7f0093fff700 1 --2- 172.21.15.7:0/2159037597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f008c009e50 tx=0x7f008c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf047e9584162a7d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f00927fc700 1 -- 172.21.15.7:0/2159037597 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c00e070 con 0x7f009415b4a0 2023-06-09T11:19:52.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f00927fc700 1 -- 172.21.15.7:0/2159037597 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c00b910 con 0x7f009415b4a0 2023-06-09T11:19:52.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f00927fc700 1 -- 172.21.15.7:0/2159037597 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c00bcc0 con 0x7f009415b4a0 2023-06-09T11:19:52.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 -- 172.21.15.7:0/2159037597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 msgr2=0x7f00940c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 --2- 172.21.15.7:0/2159037597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f008c009e50 tx=0x7f008c00b330 comp rx=0 tx=0).stop 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 -- 172.21.15.7:0/2159037597 shutdown_connections 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 --2- 172.21.15.7:0/2159037597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940c4e30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 --2- 172.21.15.7:0/2159037597 >> v2:172.21.15.117:3300/0 conn(0x7f0094153880 0x7f009415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 -- 172.21.15.7:0/2159037597 >> 172.21.15.7:0/2159037597 conn(0x7f00940737f0 msgr2=0x7f00940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 -- 172.21.15.7:0/2159037597 shutdown_connections 2023-06-09T11:19:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.262+0000 7f0099bff700 1 -- 172.21.15.7:0/2159037597 wait complete. 2023-06-09T11:19:52.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.265+0000 7f0099bff700 1 Processor -- start 2023-06-09T11:19:52.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.265+0000 7f0099bff700 1 -- start start 2023-06-09T11:19:52.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 0x7f0094148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094157460 con 0x7f0094153880 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00940ceee0 con 0x7f00940c9f40 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0099bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00940cd3a0 con 0x7f009415b4a0 2023-06-09T11:19:52.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.266+0000 7f0092ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39056/0 (socket says 172.21.15.7:39056) 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 -- 172.21.15.7:0/3867009809 learned_addr learned my addr 172.21.15.7:0/3867009809 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f00937fe700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 0x7f0094148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0093fff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 -- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 msgr2=0x7f00940ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 -- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 msgr2=0x7f0094148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 0x7f0094148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 -- 172.21.15.7:0/3867009809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00840094a0 con 0x7f0094153880 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0093fff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f00937fe700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 0x7f0094148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0092ffd700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f008800c700 tx=0x7f008800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088005cb0 con 0x7f0094153880 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0088005e10 con 0x7f0094153880 2023-06-09T11:19:52.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.267+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008c009b30 con 0x7f0094153880 2023-06-09T11:19:52.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.268+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880072f0 con 0x7f0094153880 2023-06-09T11:19:52.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.268+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0094159c60 con 0x7f0094153880 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.269+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0094159980 con 0x7f0094153880 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.270+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0088007450 con 0x7f0094153880 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.270+0000 7f007bfff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 0x7f008006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.271+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7f0088088bd0 con 0x7f0094153880 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.271+0000 7f00937fe700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 0x7f008006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:52.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.271+0000 7f00937fe700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 0x7f008006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0084009dc0 tx=0x7f0084008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:52.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.275+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00880539a0 con 0x7f0094153880 2023-06-09T11:19:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.527+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00941ef0e0 con 0x7f0094153880 2023-06-09T11:19:52.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.528+0000 7f007bfff700 1 -- 172.21.15.7:0/3867009809 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7f0088017070 con 0x7f0094153880 2023-06-09T11:19:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:52.533 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:52.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 msgr2=0x7f008006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 0x7f008006d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0084009dc0 tx=0x7f0084008040 comp rx=0 tx=0).stop 2023-06-09T11:19:52.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 msgr2=0x7f0094149430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:52.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f008800c700 tx=0x7f008800cac0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 shutdown_connections 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008006ace0 0x7f008006d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f009415b4a0 0x7f00940ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00940c9f40 0x7f0094148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 --2- 172.21.15.7:0/3867009809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0094153880 0x7f0094149430 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:52.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 >> 172.21.15.7:0/3867009809 conn(0x7f00940737f0 msgr2=0x7f009414c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:52.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 shutdown_connections 2023-06-09T11:19:52.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:52.530+0000 7f0099bff700 1 -- 172.21.15.7:0/3867009809 wait complete. 2023-06-09T11:19:52.537 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:52.553 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:19:54.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:54.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.760+0000 7fc3c7fff700 1 Processor -- start 2023-06-09T11:19:54.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.760+0000 7fc3c7fff700 1 -- start start 2023-06-09T11:19:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 0x7fc3c01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3c00c0140 0x7fc3c0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c7fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c00c0bf0 con 0x7fc3c00c1f90 2023-06-09T11:19:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c0070110 con 0x7fc3c00c0140 2023-06-09T11:19:54.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c0071300 con 0x7fc3c00c1bc0 2023-06-09T11:19:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c559a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3c00c0140 0x7fc3c0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c559a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3c00c0140 0x7fc3c0159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41788/0 (socket says 172.21.15.7:41788) 2023-06-09T11:19:54.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c559a700 1 -- 172.21.15.7:0/1545342268 learned_addr learned my addr 172.21.15.7:0/1545342268 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:54.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.762+0000 7fc3c5d9b700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 0x7fc3c01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c559a700 1 -- 172.21.15.7:0/1545342268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 msgr2=0x7fc3c01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c559a700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 0x7fc3c01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:54.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c559a700 1 -- 172.21.15.7:0/1545342268 >> v1:172.21.15.117:6789/0 conn(0x7fc3c00c1f90 legacy=0x7fc3c01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c559a700 1 -- 172.21.15.7:0/1545342268 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c00702f0 con 0x7fc3c00c0140 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c5d9b700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 0x7fc3c01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c559a700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c0140 0x7fc3c0159ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00098f0 tx=0x7fc3b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238c797f28c7c379 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c4d99700 1 -- 172.21.15.7:0/1545342268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000b8a0 con 0x7fc3c00c0140 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c4d99700 1 -- 172.21.15.7:0/1545342268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b000ba00 con 0x7fc3c00c0140 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c4d99700 1 -- 172.21.15.7:0/1545342268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b000bd90 con 0x7fc3c00c0140 2023-06-09T11:19:54.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1545342268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c0140 msgr2=0x7fc3c0159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.763+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c0140 0x7fc3c0159ef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00098f0 tx=0x7fc3b0009c00 comp rx=0 tx=0).stop 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1545342268 shutdown_connections 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c1bc0 0x7fc3c01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1545342268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c0140 0x7fc3c0159ef0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1545342268 >> 172.21.15.7:0/1545342268 conn(0x7fc3c006a4d0 msgr2=0x7fc3c006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1545342268 shutdown_connections 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.764+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1545342268 wait complete. 2023-06-09T11:19:54.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.765+0000 7fc3c7fff700 1 Processor -- start 2023-06-09T11:19:54.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 -- start start 2023-06-09T11:19:54.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c0140 0x7fc3c00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c00c1bc0 0x7fc3c0156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00ca430 con 0x7fc3c00c1f90 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c01595c0 con 0x7fc3c00c1bc0 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.766+0000 7fc3c7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c00c9370 con 0x7fc3c00c0140 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41792/0 (socket says 172.21.15.7:41792) 2023-06-09T11:19:54.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 -- 172.21.15.7:0/1578977887 learned_addr learned my addr 172.21.15.7:0/1578977887 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c559a700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c00c1bc0 0x7fc3c0156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c5d9b700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c0140 0x7fc3c00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 -- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c0140 msgr2=0x7fc3c00c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c0140 0x7fc3c00c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 -- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c00c1bc0 msgr2=0x7fc3c0156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c00c1bc0 0x7fc3c0156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:54.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 -- 172.21.15.7:0/1578977887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b0009550 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.767+0000 7fc3c659c700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800cb50 tx=0x7fc3b800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.768+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8003c00 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.768+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b8003d60 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.768+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c00c1170 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.768+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b800ab70 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.768+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c00c0e90 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.769+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c00c1450 con 0x7fc3c00c1f90 2023-06-09T11:19:54.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.770+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc3b800acd0 con 0x7fc3c00c1f90 2023-06-09T11:19:54.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.771+0000 7fc3b67fc700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 0x7fc3a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:54.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.771+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7fc3b8089670 con 0x7fc3c00c1f90 2023-06-09T11:19:54.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.771+0000 7fc3c5d9b700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 0x7fc3a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:54.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.772+0000 7fc3c5d9b700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 0x7fc3a406d1c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009930 tx=0x7fc3bc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:54.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:54.775+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc3b8054560 con 0x7fc3c00c1f90 2023-06-09T11:19:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.029+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c0074cf0 con 0x7fc3c00c1f90 2023-06-09T11:19:55.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.030+0000 7fc3b67fc700 1 -- 172.21.15.7:0/1578977887 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7fc3b8057b80 con 0x7fc3c00c1f90 2023-06-09T11:19:55.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:55.035 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 msgr2=0x7fc3a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 0x7fc3a406d1c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009930 tx=0x7fc3bc008040 comp rx=0 tx=0).stop 2023-06-09T11:19:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 msgr2=0x7fc3c01f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:55.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc3b800cb50 tx=0x7fc3b800cf10 comp rx=0 tx=0).stop 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 shutdown_connections 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3c00c1f90 0x7fc3c01f1f50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc3a406ad10 0x7fc3a406d1c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3c00c0140 0x7fc3c00c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 --2- 172.21.15.7:0/1578977887 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3c00c1bc0 0x7fc3c0156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 >> 172.21.15.7:0/1578977887 conn(0x7fc3c006a4d0 msgr2=0x7fc3c014cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:55.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 shutdown_connections 2023-06-09T11:19:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:55.032+0000 7fc3c7fff700 1 -- 172.21.15.7:0/1578977887 wait complete. 2023-06-09T11:19:55.039 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:55.055 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:19:57.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.262+0000 7fa2d79ff700 1 Processor -- start 2023-06-09T11:19:57.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.262+0000 7fa2d79ff700 1 -- start start 2023-06-09T11:19:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa2d01593d0 0x7fa2d01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d0159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa2d0071910 0x7fa2d0071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00714f0 con 0x7fa2d01593d0 2023-06-09T11:19:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d00cc140 con 0x7fa2d0071910 2023-06-09T11:19:57.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.264+0000 7fa2d79ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d00c1830 con 0x7fa2d00c08d0 2023-06-09T11:19:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa2d01593d0 0x7fa2d01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa2d01593d0 0x7fa2d01597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41794/0 (socket says 172.21.15.7:41794) 2023-06-09T11:19:57.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 -- 172.21.15.7:0/2050539058 learned_addr learned my addr 172.21.15.7:0/2050539058 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d5f9c700 1 --2- 172.21.15.7:0/2050539058 >> v2:172.21.15.117:3300/0 conn(0x7fa2d0071910 0x7fa2d0071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d4f9a700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d0159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 -- 172.21.15.7:0/2050539058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 msgr2=0x7fa2d0159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d0159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 -- 172.21.15.7:0/2050539058 >> v2:172.21.15.117:3300/0 conn(0x7fa2d0071910 msgr2=0x7fa2d0071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2050539058 >> v2:172.21.15.117:3300/0 conn(0x7fa2d0071910 0x7fa2d0071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 -- 172.21.15.7:0/2050539058 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00c0560 con 0x7fa2d01593d0 2023-06-09T11:19:57.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.265+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d01597a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00c9050 tx=0x7fa2cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3419f9845df518 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2c7fff700 1 -- 172.21.15.7:0/2050539058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00b870 con 0x7fa2d01593d0 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2c7fff700 1 -- 172.21.15.7:0/2050539058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc00b9d0 con 0x7fa2d01593d0 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2c7fff700 1 -- 172.21.15.7:0/2050539058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00bd60 con 0x7fa2d01593d0 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2050539058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 msgr2=0x7fa2d01597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d01597a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00c9050 tx=0x7fa2cc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2050539058 shutdown_connections 2023-06-09T11:19:57.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2050539058 >> v2:172.21.15.117:3300/0 conn(0x7fa2d0071910 0x7fa2d0071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d01597a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2050539058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d0159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2050539058 >> 172.21.15.7:0/2050539058 conn(0x7fa2d006a4d0 msgr2=0x7fa2d006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2050539058 shutdown_connections 2023-06-09T11:19:57.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.266+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2050539058 wait complete. 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.268+0000 7fa2d79ff700 1 Processor -- start 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.268+0000 7fa2d79ff700 1 -- start start 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 0x7fa2d01e15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d01eec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d01e2660 con 0x7fa2d01593d0 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d014db90 con 0x7fa2d0071910 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d79ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d01e2c60 con 0x7fa2d00c08d0 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d5f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d5f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41806/0 (socket says 172.21.15.7:41806) 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.269+0000 7fa2d5f9c700 1 -- 172.21.15.7:0/2228093116 learned_addr learned my addr 172.21.15.7:0/2228093116 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 0x7fa2d01e15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d4f9a700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d01eec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 -- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 msgr2=0x7fa2d01eec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d01eec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 -- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 msgr2=0x7fa2d01e15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 0x7fa2d01e15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 -- 172.21.15.7:0/2228093116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc009550 con 0x7fa2d01593d0 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 0x7fa2d01e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:19:57.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2d5f9c700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8007e60 tx=0x7fa2c800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.270+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800ede0 con 0x7fa2d01593d0 2023-06-09T11:19:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.271+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d00c1a80 con 0x7fa2d01593d0 2023-06-09T11:19:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.271+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8014420 con 0x7fa2d01593d0 2023-06-09T11:19:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.271+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c80147d0 con 0x7fa2d01593d0 2023-06-09T11:19:57.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.271+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d0072290 con 0x7fa2d01593d0 2023-06-09T11:19:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.272+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d00c1d60 con 0x7fa2d01593d0 2023-06-09T11:19:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.273+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2c8014930 con 0x7fa2d01593d0 2023-06-09T11:19:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.274+0000 7fa2c5ffb700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 0x7fa2b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.274+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7fa2c8089e10 con 0x7fa2d01593d0 2023-06-09T11:19:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.274+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 0x7fa2b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:57.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.274+0000 7fa2d579b700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 0x7fa2b406d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc014040 tx=0x7fa2cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:57.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.278+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa2c808d030 con 0x7fa2d01593d0 2023-06-09T11:19:57.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.529+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2d01e2da0 con 0x7fa2d01593d0 2023-06-09T11:19:57.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.530+0000 7fa2c5ffb700 1 -- 172.21.15.7:0/2228093116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7fa2d01e2da0 con 0x7fa2d01593d0 2023-06-09T11:19:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:19:57.536 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.532+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 msgr2=0x7fa2b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.532+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 0x7fa2b406d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc014040 tx=0x7fa2cc008030 comp rx=0 tx=0).stop 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 msgr2=0x7fa2d00ca470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8007e60 tx=0x7fa2c800e3c0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 shutdown_connections 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2d01593d0 0x7fa2d00ca470 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa2d0071910 0x7fa2d01e15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2b406ace0 0x7fa2b406d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 --2- 172.21.15.7:0/2228093116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa2d00c08d0 0x7fa2d01eec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 >> 172.21.15.7:0/2228093116 conn(0x7fa2d006a4d0 msgr2=0x7fa2d00c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 shutdown_connections 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:57.533+0000 7fa2d79ff700 1 -- 172.21.15.7:0/2228093116 wait complete. 2023-06-09T11:19:57.539 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:19:57.555 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:19:59.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:19:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.762+0000 7f4af15ff700 1 Processor -- start 2023-06-09T11:19:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.762+0000 7f4af15ff700 1 -- start start 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c1bc0 0x7f4aec1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aec0c0140 0x7f4aec159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4aec0c1f90 0x7f4aec1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec070110 con 0x7f4aec0c0140 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec071300 con 0x7f4aec0c1f90 2023-06-09T11:19:59.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4af15ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0cbd20 con 0x7f4aec0c1bc0 2023-06-09T11:19:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4ae3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aec0c0140 0x7f4aec159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4ae3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aec0c0140 0x7f4aec159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41822/0 (socket says 172.21.15.7:41822) 2023-06-09T11:19:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4ae3fff700 1 -- 172.21.15.7:0/1367333596 learned_addr learned my addr 172.21.15.7:0/1367333596 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4aeb7fe700 1 --2- 172.21.15.7:0/1367333596 >> v2:172.21.15.117:3300/0 conn(0x7f4aec0c1f90 0x7f4aec1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4aeaffd700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c1bc0 0x7f4aec1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4ae3fff700 1 -- 172.21.15.7:0/1367333596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c1bc0 msgr2=0x7f4aec1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.764+0000 7f4ae3fff700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c1bc0 0x7f4aec1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4ae3fff700 1 -- 172.21.15.7:0/1367333596 >> v2:172.21.15.117:3300/0 conn(0x7f4aec0c1f90 msgr2=0x7f4aec1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4ae3fff700 1 --2- 172.21.15.7:0/1367333596 >> v2:172.21.15.117:3300/0 conn(0x7f4aec0c1f90 0x7f4aec1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4ae3fff700 1 -- 172.21.15.7:0/1367333596 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0c0bf0 con 0x7f4aec0c0140 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4ae3fff700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c0140 0x7f4aec159ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0cc250 tx=0x7f4ad4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb57b61189896982 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4aea7fc700 1 -- 172.21.15.7:0/1367333596 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011830 con 0x7f4aec0c0140 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4aea7fc700 1 -- 172.21.15.7:0/1367333596 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4011990 con 0x7f4aec0c0140 2023-06-09T11:19:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4aea7fc700 1 -- 172.21.15.7:0/1367333596 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011d20 con 0x7f4aec0c0140 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4af15ff700 1 -- 172.21.15.7:0/1367333596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c0140 msgr2=0x7f4aec159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.765+0000 7f4af15ff700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c0140 0x7f4aec159ef0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0cc250 tx=0x7f4ad4009c00 comp rx=0 tx=0).stop 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 -- 172.21.15.7:0/1367333596 shutdown_connections 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 --2- 172.21.15.7:0/1367333596 >> v2:172.21.15.117:3300/0 conn(0x7f4aec0c1f90 0x7f4aec1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c1bc0 0x7f4aec1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 --2- 172.21.15.7:0/1367333596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c0140 0x7f4aec159ef0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 -- 172.21.15.7:0/1367333596 >> 172.21.15.7:0/1367333596 conn(0x7f4aec06a4d0 msgr2=0x7f4aec06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:19:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 -- 172.21.15.7:0/1367333596 shutdown_connections 2023-06-09T11:19:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.766+0000 7f4af15ff700 1 -- 172.21.15.7:0/1367333596 wait complete. 2023-06-09T11:19:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.767+0000 7f4af15ff700 1 Processor -- start 2023-06-09T11:19:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.768+0000 7f4af15ff700 1 -- start start 2023-06-09T11:19:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c0140 0x7f4aec0c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aec0c1bc0 0x7f4aec157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1574f0 con 0x7f4aec0c1f90 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec157770 con 0x7f4aec0c1bc0 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4af15ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0c37d0 con 0x7f4aec0c0140 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41830/0 (socket says 172.21.15.7:41830) 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 -- 172.21.15.7:0/3431825057 learned_addr learned my addr 172.21.15.7:0/3431825057 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeaffd700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c0140 0x7f4aec0c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4ae3fff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aec0c1bc0 0x7f4aec157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 -- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c0140 msgr2=0x7f4aec0c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c0140 0x7f4aec0c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 -- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aec0c1bc0 msgr2=0x7f4aec157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aec0c1bc0 0x7f4aec157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:19:59.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.769+0000 7f4aeb7fe700 1 -- 172.21.15.7:0/3431825057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad4009550 con 0x7f4aec0c1f90 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4aeb7fe700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400be10 tx=0x7f4ae400e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4010040 con 0x7f4aec0c1f90 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4003a20 con 0x7f4aec0c1f90 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4003dd0 con 0x7f4aec0c1f90 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec0c0e90 con 0x7f4aec0c1f90 2023-06-09T11:19:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.770+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec0702f0 con 0x7f4aec0c1f90 2023-06-09T11:19:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.771+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0c1170 con 0x7f4aec0c1f90 2023-06-09T11:19:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.772+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ae400e880 con 0x7f4aec0c1f90 2023-06-09T11:19:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.773+0000 7f4ae37fe700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 0x7f4acc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:19:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.773+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6492+0+0 (secure 0 0 0) 0x7f4ae4089640 con 0x7f4aec0c1f90 2023-06-09T11:19:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.773+0000 7f4aeaffd700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 0x7f4acc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:19:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.774+0000 7f4aeaffd700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 0x7f4acc06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009950 tx=0x7f4adc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:19:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:19:59.777+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4ae4054410 con 0x7f4aec0c1f90 2023-06-09T11:20:00.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.029+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aec1578b0 con 0x7f4aec0c1f90 2023-06-09T11:20:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.029+0000 7f4ae37fe700 1 -- 172.21.15.7:0/3431825057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+6225 (secure 0 0 0) 0x7f4aec1578b0 con 0x7f4aec0c1f90 2023-06-09T11:20:00.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:00.034 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5012,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:18:41.219496+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":41,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4889,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4889":{"gid":4889,"name":"d","rank":0,"incarnation":34,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6835/1617501683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1617501683},{"type":"v1","addr":"172.21.15.117:6835","nonce":1617501683}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.031+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 msgr2=0x7f4acc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:00.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.031+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 0x7f4acc06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009950 tx=0x7f4adc008040 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.031+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 msgr2=0x7f4aec158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.031+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400be10 tx=0x7f4ae400e3a0 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 shutdown_connections 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aec0c1f90 0x7f4aec158140 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4acc06ad10 0x7f4acc06d1c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aec0c0140 0x7f4aec0c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 --2- 172.21.15.7:0/3431825057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aec0c1bc0 0x7f4aec157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:00.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 >> 172.21.15.7:0/3431825057 conn(0x7f4aec06a4d0 msgr2=0x7f4aec0c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 shutdown_connections 2023-06-09T11:20:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:00.032+0000 7f4af15ff700 1 -- 172.21.15.7:0/3431825057 wait complete. 2023-06-09T11:20:00.038 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 44 2023-06-09T11:20:00.054 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:20:02.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:02.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.266+0000 7f8d209ff700 1 Processor -- start 2023-06-09T11:20:02.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.266+0000 7f8d209ff700 1 -- start start 2023-06-09T11:20:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d209ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8d1c1593d0 0x7f8d1c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d209ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0cc140 con 0x7f8d1c072020 2023-06-09T11:20:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d209ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1c0c1830 con 0x7f8d1c0c08d0 2023-06-09T11:20:02.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d209ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1c0714f0 con 0x7f8d1c1593d0 2023-06-09T11:20:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d1ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8d1c072020 0x7f8d1c14f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55364/0 (socket says 172.21.15.7:55364) 2023-06-09T11:20:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.267+0000 7f8d1ad9d700 1 -- 172.21.15.7:0/2507735265 learned_addr learned my addr 172.21.15.7:0/2507735265 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:02.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1a59c700 1 --2- 172.21.15.7:0/2507735265 >> v2:172.21.15.117:3300/0 conn(0x7f8d1c1593d0 0x7f8d1c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532118183 0 0) 0x7f8d1c0cc140 con 0x7f8d1c072020 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d040035b0 con 0x7f8d1c072020 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206549893 0 0) 0x7f8d1c0c1830 con 0x7f8d1c0c08d0 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d1c0cc140 con 0x7f8d1c0c08d0 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2531074357 0 0) 0x7f8d040035b0 con 0x7f8d1c072020 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 >> v2:172.21.15.117:3300/0 conn(0x7f8d1c1593d0 msgr2=0x7f8d1c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 --2- 172.21.15.7:0/2507735265 >> v2:172.21.15.117:3300/0 conn(0x7f8d1c1593d0 0x7f8d1c1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 >> v1:172.21.15.117:6789/0 conn(0x7f8d1c0c08d0 legacy=0x7f8d1c071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0c0560 con 0x7f8d1c072020 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f8d140019f0 con 0x7f8d1c072020 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 >> v1:172.21.15.7:6789/0 conn(0x7f8d1c072020 legacy=0x7f8d1c14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d04003b90 0x7f8d04003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 0x7f8d04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04005200 0x7f8d0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0714f0 con 0x7f8d04004850 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0400e320 con 0x7f8d04005200 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.268+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0400e4e0 con 0x7f8d04003b90 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 0x7f8d04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1a59c700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d04003b90 0x7f8d04003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1ad9d700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04005200 0x7f8d0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d04003b90 msgr2=0x7f8d04003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d04003b90 0x7f8d04003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04005200 msgr2=0x7f8d0400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04005200 0x7f8d0400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2507735265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d040035b0 con 0x7f8d04004850 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1ad9d700 1 -- 172.21.15.7:0/2507735265 reap_dead start 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 0x7f8d04004cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dd9956cb8738e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10001430 con 0x7f8d04004850 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d100052f0 con 0x7f8d04004850 2023-06-09T11:20:02.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.269+0000 7f8d1959a700 1 -- 172.21.15.7:0/2507735265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10001d40 con 0x7f8d04004850 2023-06-09T11:20:02.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 -- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 msgr2=0x7f8d04004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 0x7f8d04004cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8d10007fd0 tx=0x7f8d04005880 comp rx=0 tx=0).stop 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 -- 172.21.15.7:0/2507735265 shutdown_connections 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2507735265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d04004850 0x7f8d04004cc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 -- 172.21.15.7:0/2507735265 >> 172.21.15.7:0/2507735265 conn(0x7f8d1c06a4d0 msgr2=0x7f8d1c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 -- 172.21.15.7:0/2507735265 shutdown_connections 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.270+0000 7f8d209ff700 1 -- 172.21.15.7:0/2507735265 wait complete. 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.272+0000 7f8d209ff700 1 Processor -- start 2023-06-09T11:20:02.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.272+0000 7f8d209ff700 1 -- start start 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04004850 0x7f8d1c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 0x7f8d1c14b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c0afb90 con 0x7f8d1c152010 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c152f40 con 0x7f8d04004850 2023-06-09T11:20:02.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d209ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c1f3100 con 0x7f8d1c14f0f0 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d19d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41858/0 (socket says 172.21.15.7:41858) 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2816848983 learned_addr learned my addr 172.21.15.7:0/2816848983 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.273+0000 7f8d1a59c700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04004850 0x7f8d1c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d1ad9d700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 0x7f8d1c14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 msgr2=0x7f8d1c14b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 0x7f8d1c14b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04004850 msgr2=0x7f8d1c0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04004850 0x7f8d1c0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 -- 172.21.15.7:0/2816848983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c0c1d60 con 0x7f8d1c152010 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d1ad9d700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 0x7f8d1c14b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d19d9b700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005040 tx=0x7f8d100012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100013d0 con 0x7f8d1c152010 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10006a70 con 0x7f8d1c152010 2023-06-09T11:20:02.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10006e20 con 0x7f8d1c152010 2023-06-09T11:20:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.274+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c0c1a80 con 0x7f8d1c152010 2023-06-09T11:20:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.275+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c0c2040 con 0x7f8d1c152010 2023-06-09T11:20:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.275+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c0671a0 con 0x7f8d1c152010 2023-06-09T11:20:02.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.276+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d10007040 con 0x7f8d1c152010 2023-06-09T11:20:02.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.277+0000 7f8d0affd700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 0x7f8cfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.277+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f8d10089fe0 con 0x7f8d1c152010 2023-06-09T11:20:02.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.278+0000 7f8d1a59c700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 0x7f8cfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.278+0000 7f8d1a59c700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 0x7f8cfc06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c064570 tx=0x7f8d0c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.282+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8d10058a80 con 0x7f8d1c152010 2023-06-09T11:20:02.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.536+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1c074cf0 con 0x7f8d1c152010 2023-06-09T11:20:02.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.537+0000 7f8d0affd700 1 -- 172.21.15.7:0/2816848983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f8d1002b090 con 0x7f8d1c152010 2023-06-09T11:20:02.544 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:02.544 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:01.247570+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 msgr2=0x7f8cfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 0x7f8cfc06d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c064570 tx=0x7f8d0c009210 comp rx=0 tx=0).stop 2023-06-09T11:20:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 msgr2=0x7f8d1c14afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8d10005040 tx=0x7f8d100012d0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 shutdown_connections 2023-06-09T11:20:02.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cfc06ace0 0x7f8cfc06d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8d1c14f0f0 0x7f8d1c14b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8d04004850 0x7f8d1c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 --2- 172.21.15.7:0/2816848983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d1c152010 0x7f8d1c14afc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.539+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 >> 172.21.15.7:0/2816848983 conn(0x7f8d1c06a4d0 msgr2=0x7f8d1c1586a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.540+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 shutdown_connections 2023-06-09T11:20:02.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.540+0000 7f8d209ff700 1 -- 172.21.15.7:0/2816848983 wait complete. 2023-06-09T11:20:02.549 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2023-06-09T11:20:02.564 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:20:02.564 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:20:02.564 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.760+0000 7f44d9dff700 1 Processor -- start 2023-06-09T11:20:02.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.760+0000 7f44d9dff700 1 -- start start 2023-06-09T11:20:02.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.761+0000 7f44d9dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f44d40ca1b0 0x7f44d415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.761+0000 7f44d9dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.761+0000 7f44d9dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d415dcf0 con 0x7f44d40b5360 2023-06-09T11:20:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.761+0000 7f44d9dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40c5e00 con 0x7f44d40ca1b0 2023-06-09T11:20:02.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.761+0000 7f44d9dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d415d450 con 0x7f44d40ca580 2023-06-09T11:20:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:34592/0 (socket says 172.21.15.7:34592) 2023-06-09T11:20:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 -- 172.21.15.7:0/768140281 learned_addr learned my addr 172.21.15.7:0/768140281 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:02.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d37fe700 1 --2- 172.21.15.7:0/768140281 >> v2:172.21.15.117:3300/0 conn(0x7f44d40ca1b0 0x7f44d415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d2ffd700 1 --1- 172.21.15.7:0/768140281 >> v1:172.21.15.117:6789/0 conn(0x7f44d40b5360 0x7f44d41536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:52624/0 (socket says 172.21.15.7:52624) 2023-06-09T11:20:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 -- 172.21.15.7:0/768140281 >> v2:172.21.15.117:3300/0 conn(0x7f44d40ca1b0 msgr2=0x7f44d415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 --2- 172.21.15.7:0/768140281 >> v2:172.21.15.117:3300/0 conn(0x7f44d40ca1b0 0x7f44d415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d27fc700 1 -- 172.21.15.7:0/768140281 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304941960 0 0) 0x7f44d415dcf0 con 0x7f44d40b5360 2023-06-09T11:20:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 -- 172.21.15.7:0/768140281 >> v1:172.21.15.117:6789/0 conn(0x7f44d40b5360 legacy=0x7f44d41536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.762+0000 7f44d3fff700 1 -- 172.21.15.7:0/768140281 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d415d730 con 0x7f44d40ca580 2023-06-09T11:20:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d3fff700 1 --2- 172.21.15.7:0/768140281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009e50 tx=0x7f44cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e633ae56fd2aa86a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d27fc700 1 -- 172.21.15.7:0/768140281 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00e070 con 0x7f44d40ca580 2023-06-09T11:20:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d27fc700 1 -- 172.21.15.7:0/768140281 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00b910 con 0x7f44d40ca580 2023-06-09T11:20:02.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d27fc700 1 -- 172.21.15.7:0/768140281 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00bcc0 con 0x7f44d40ca580 2023-06-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 -- 172.21.15.7:0/768140281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 msgr2=0x7f44d40c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 --2- 172.21.15.7:0/768140281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009e50 tx=0x7f44cc00b330 comp rx=0 tx=0).stop 2023-06-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 -- 172.21.15.7:0/768140281 shutdown_connections 2023-06-09T11:20:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 --2- 172.21.15.7:0/768140281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40ca580 0x7f44d40c58c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 --2- 172.21.15.7:0/768140281 >> v2:172.21.15.117:3300/0 conn(0x7f44d40ca1b0 0x7f44d415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 -- 172.21.15.7:0/768140281 >> 172.21.15.7:0/768140281 conn(0x7f44d4072090 msgr2=0x7f44d4072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:02.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 -- 172.21.15.7:0/768140281 shutdown_connections 2023-06-09T11:20:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.763+0000 7f44d9dff700 1 -- 172.21.15.7:0/768140281 wait complete. 2023-06-09T11:20:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.765+0000 7f44d9dff700 1 Processor -- start 2023-06-09T11:20:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.765+0000 7f44d9dff700 1 -- start start 2023-06-09T11:20:02.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 0x7f44d4156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d40ca1b0 0x7f44d41ef970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d4157240 con 0x7f44d40ca580 2023-06-09T11:20:02.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d41efeb0 con 0x7f44d40ca1b0 2023-06-09T11:20:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.766+0000 7f44d9dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d415c950 con 0x7f44d40b5360 2023-06-09T11:20:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41874/0 (socket says 172.21.15.7:41874) 2023-06-09T11:20:02.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 -- 172.21.15.7:0/2682577297 learned_addr learned my addr 172.21.15.7:0/2682577297 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d2ffd700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d40ca1b0 0x7f44d41ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d37fe700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 0x7f44d4156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 -- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 msgr2=0x7f44d4156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 0x7f44d4156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 -- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d40ca1b0 msgr2=0x7f44d41ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d40ca1b0 0x7f44d41ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 -- 172.21.15.7:0/2682577297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc009b30 con 0x7f44d40ca580 2023-06-09T11:20:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d37fe700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 0x7f44d4156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:02.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44d3fff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b680 tx=0x7f44cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.767+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc004960 con 0x7f44d40ca580 2023-06-09T11:20:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.768+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc0099b0 con 0x7f44d40ca580 2023-06-09T11:20:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.768+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc01c660 con 0x7f44d40ca580 2023-06-09T11:20:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.768+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d406a570 con 0x7f44d40ca580 2023-06-09T11:20:02.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.768+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d415da10 con 0x7f44d40ca580 2023-06-09T11:20:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.769+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d406a850 con 0x7f44d40ca580 2023-06-09T11:20:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.770+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44cc01c7c0 con 0x7f44d40ca580 2023-06-09T11:20:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.771+0000 7f44bbfff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 0x7f44c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:02.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.771+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f44cc08af90 con 0x7f44d40ca580 2023-06-09T11:20:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.771+0000 7f44d37fe700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 0x7f44c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:02.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.772+0000 7f44d37fe700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 0x7f44c006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009c30 tx=0x7f44c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:02.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:02.775+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44cc055db0 con 0x7f44d40ca580 2023-06-09T11:20:03.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.028+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44d4074cf0 con 0x7f44d40ca580 2023-06-09T11:20:03.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.029+0000 7f44bbfff700 1 -- 172.21.15.7:0/2682577297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f44cc05ae60 con 0x7f44d40ca580 2023-06-09T11:20:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:03.035 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:01.247570+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:03.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 msgr2=0x7f44c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 0x7f44c006d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44c4009c30 tx=0x7f44c4008040 comp rx=0 tx=0).stop 2023-06-09T11:20:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 msgr2=0x7f44d40b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f44cc00b680 tx=0x7f44cc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:20:03.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 shutdown_connections 2023-06-09T11:20:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44d40ca580 0x7f44d40b4340 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44c006ace0 0x7f44c006d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44d40b5360 0x7f44d4156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 --2- 172.21.15.7:0/2682577297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44d40ca1b0 0x7f44d41ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:03.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 >> 172.21.15.7:0/2682577297 conn(0x7f44d4072090 msgr2=0x7f44d40cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.031+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 shutdown_connections 2023-06-09T11:20:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:03.032+0000 7f44d9dff700 1 -- 172.21.15.7:0/2682577297 wait complete. 2023-06-09T11:20:03.040 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2023-06-09T11:20:03.056 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 46, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 46, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5012": { "addr": "172.21.15.117:6837/3174501100", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3174501100, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3174501100, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5012, "incarnation": 46, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:20:01.247570+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5012, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:20:05.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.263+0000 7f8ae31ff700 1 Processor -- start 2023-06-09T11:20:05.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.263+0000 7f8ae31ff700 1 -- start start 2023-06-09T11:20:05.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.264+0000 7f8ae31ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0c1bc0 0x7f8adc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.264+0000 7f8ae31ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0c0bf0 con 0x7f8adc0c1f90 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae31ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0702f0 con 0x7f8adc0c0140 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae31ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc070110 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0c1bc0 0x7f8adc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8adc0c1bc0 0x7f8adc1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60500/0 (socket says 172.21.15.7:60500) 2023-06-09T11:20:05.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae179c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8adc0c1f90 0x7f8adc160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58756/0 (socket says 172.21.15.7:58756) 2023-06-09T11:20:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 -- 172.21.15.7:0/1624720132 learned_addr learned my addr 172.21.15.7:0/1624720132 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 -- 172.21.15.7:0/1624720132 >> v1:172.21.15.117:6789/0 conn(0x7f8adc0c0140 legacy=0x7f8adc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 -- 172.21.15.7:0/1624720132 >> v1:172.21.15.7:6789/0 conn(0x7f8adc0c1f90 legacy=0x7f8adc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8adb7fe700 1 -- 172.21.15.7:0/1624720132 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087014745 0 0) 0x7f8adc0c0bf0 con 0x7f8adc0c1f90 2023-06-09T11:20:05.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.265+0000 7f8ae0f9b700 1 -- 172.21.15.7:0/1624720132 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0c0e90 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae0f9b700 1 --2- 172.21.15.7:0/1624720132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc1599b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00098f0 tx=0x7f8ad0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63601258e35f18e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8adb7fe700 1 -- 172.21.15.7:0/1624720132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000b8a0 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8adb7fe700 1 -- 172.21.15.7:0/1624720132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad000ba00 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8adb7fe700 1 -- 172.21.15.7:0/1624720132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000bd90 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 -- 172.21.15.7:0/1624720132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 msgr2=0x7f8adc1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/1624720132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc1599b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00098f0 tx=0x7f8ad0009c00 comp rx=0 tx=0).stop 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 -- 172.21.15.7:0/1624720132 shutdown_connections 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/1624720132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc1599b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 -- 172.21.15.7:0/1624720132 >> 172.21.15.7:0/1624720132 conn(0x7f8adc06a4d0 msgr2=0x7f8adc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.266+0000 7f8ae31ff700 1 -- 172.21.15.7:0/1624720132 shutdown_connections 2023-06-09T11:20:05.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.267+0000 7f8ae31ff700 1 -- 172.21.15.7:0/1624720132 wait complete. 2023-06-09T11:20:05.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.268+0000 7f8ae31ff700 1 Processor -- start 2023-06-09T11:20:05.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.268+0000 7f8ae31ff700 1 -- start start 2023-06-09T11:20:05.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8adc0c0140 0x7f8adc1e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8adc0c1f90 0x7f8adc148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc149360 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1e22b0 con 0x7f8adc0c0140 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.269+0000 7f8ae31ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1e1cf0 con 0x7f8adc0c1f90 2023-06-09T11:20:05.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60502/0 (socket says 172.21.15.7:60502) 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 -- 172.21.15.7:0/2658890576 learned_addr learned my addr 172.21.15.7:0/2658890576 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8ae0f9b700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8adc0c0140 0x7f8adc1e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 -- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8adc0c1f90 msgr2=0x7f8adc148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8adc0c1f90 0x7f8adc148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 -- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8adc0c0140 msgr2=0x7f8adc1e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8adc0c0140 0x7f8adc1e08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 -- 172.21.15.7:0/2658890576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009550 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8adbfff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800c4e0 tx=0x7f8ac800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80052d0 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac8005430 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.270+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc0c1450 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.271+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac80038c0 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.271+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc0c1170 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.272+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0705d0 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.273+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ac8003a20 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.274+0000 7f8ad97fa700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 0x7f8ac006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:05.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.274+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f8ac8088c20 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.274+0000 7f8ae0f9b700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 0x7f8ac006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:05.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.275+0000 7f8ae0f9b700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 0x7f8ac006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009130 tx=0x7f8ad0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:05.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.278+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8ac8053a40 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.531+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8adc074cf0 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.532+0000 7f8ad97fa700 1 -- 172.21.15.7:0/2658890576 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f8ac8057060 con 0x7f8adc0c1bc0 2023-06-09T11:20:05.538 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:05.538 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:01.247570+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 msgr2=0x7f8ac006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 0x7f8ac006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009130 tx=0x7f8ad0008030 comp rx=0 tx=0).stop 2023-06-09T11:20:05.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 msgr2=0x7f8adc147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800c4e0 tx=0x7f8ac800c8a0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 shutdown_connections 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ac006ace0 0x7f8ac006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8adc0c1f90 0x7f8adc148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8adc0c0140 0x7f8adc1e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 --2- 172.21.15.7:0/2658890576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8adc0c1bc0 0x7f8adc147cc0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:05.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.534+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 >> 172.21.15.7:0/2658890576 conn(0x7f8adc06a4d0 msgr2=0x7f8adc0c5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.535+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 shutdown_connections 2023-06-09T11:20:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:05.535+0000 7f8ae31ff700 1 -- 172.21.15.7:0/2658890576 wait complete. 2023-06-09T11:20:05.542 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2023-06-09T11:20:05.558 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:07.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:07.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.763+0000 7f10f83ff700 1 Processor -- start 2023-06-09T11:20:07.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.763+0000 7f10f83ff700 1 -- start start 2023-06-09T11:20:07.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.764+0000 7f10f83ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 0x7f10f015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f83ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10f00c9f40 0x7f10f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f83ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f015a500 con 0x7f10f015b2d0 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f83ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00c5f10 con 0x7f10f00c9f40 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f83ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f015da00 con 0x7f10f0153880 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f599a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10f00c9f40 0x7f10f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f699c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f10f015b2d0 0x7f10f00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58766/0 (socket says 172.21.15.7:58766) 2023-06-09T11:20:07.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f699c700 1 -- 172.21.15.7:0/1177699445 learned_addr learned my addr 172.21.15.7:0/1177699445 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f619b700 1 --2- 172.21.15.7:0/1177699445 >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 0x7f10f015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f599a700 1 -- 172.21.15.7:0/1177699445 >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 msgr2=0x7f10f015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f599a700 1 --2- 172.21.15.7:0/1177699445 >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 0x7f10f015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f599a700 1 -- 172.21.15.7:0/1177699445 >> v1:172.21.15.7:6789/0 conn(0x7f10f015b2d0 legacy=0x7f10f00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f599a700 1 -- 172.21.15.7:0/1177699445 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f01596e0 con 0x7f10f00c9f40 2023-06-09T11:20:07.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.765+0000 7f10f619b700 1 --2- 172.21.15.7:0/1177699445 >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 0x7f10f015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f599a700 1 --2- 172.21.15.7:0/1177699445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f00c9f40 0x7f10f015ad90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f10f008d340 tx=0x7f10e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa21a1cf838b512 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f5199700 1 -- 172.21.15.7:0/1177699445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011830 con 0x7f10f00c9f40 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f5199700 1 -- 172.21.15.7:0/1177699445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e0011990 con 0x7f10f00c9f40 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f5199700 1 -- 172.21.15.7:0/1177699445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0011d20 con 0x7f10f00c9f40 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 -- 172.21.15.7:0/1177699445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f00c9f40 msgr2=0x7f10f015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 --2- 172.21.15.7:0/1177699445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f00c9f40 0x7f10f015ad90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f10f008d340 tx=0x7f10e0009b20 comp rx=0 tx=0).stop 2023-06-09T11:20:07.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 -- 172.21.15.7:0/1177699445 shutdown_connections 2023-06-09T11:20:07.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 --2- 172.21.15.7:0/1177699445 >> v2:172.21.15.117:3300/0 conn(0x7f10f0153880 0x7f10f015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:07.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 --2- 172.21.15.7:0/1177699445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f00c9f40 0x7f10f015ad90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:07.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.766+0000 7f10f83ff700 1 -- 172.21.15.7:0/1177699445 >> 172.21.15.7:0/1177699445 conn(0x7f10f00737f0 msgr2=0x7f10f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:07.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.767+0000 7f10f83ff700 1 -- 172.21.15.7:0/1177699445 shutdown_connections 2023-06-09T11:20:07.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.767+0000 7f10f83ff700 1 -- 172.21.15.7:0/1177699445 wait complete. 2023-06-09T11:20:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.768+0000 7f10f83ff700 1 Processor -- start 2023-06-09T11:20:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.768+0000 7f10f83ff700 1 -- start start 2023-06-09T11:20:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f00c9f40 0x7f10f0152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f015b2d0 0x7f10f01f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f01f23b0 con 0x7f10f0153880 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ce8c0 con 0x7f10f00c9f40 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.769+0000 7f10f83ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00ceba0 con 0x7f10f015b2d0 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60528/0 (socket says 172.21.15.7:60528) 2023-06-09T11:20:07.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 -- 172.21.15.7:0/2216382638 learned_addr learned my addr 172.21.15.7:0/2216382638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f619b700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f00c9f40 0x7f10f0152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f699c700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f015b2d0 0x7f10f01f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 -- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f015b2d0 msgr2=0x7f10f01f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f015b2d0 0x7f10f01f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 -- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f00c9f40 msgr2=0x7f10f0152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f00c9f40 0x7f10f0152b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:07.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 -- 172.21.15.7:0/2216382638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00094a0 con 0x7f10f0153880 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.770+0000 7f10f599a700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.771+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00036a0 con 0x7f10f0153880 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.771+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e001f070 con 0x7f10f0153880 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.771+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e0003a60 con 0x7f10f0153880 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.771+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f0159c60 con 0x7f10f0153880 2023-06-09T11:20:07.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.771+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f0159980 con 0x7f10f0153880 2023-06-09T11:20:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.772+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f0159f40 con 0x7f10f0153880 2023-06-09T11:20:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.773+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f10e0003bc0 con 0x7f10f0153880 2023-06-09T11:20:07.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.774+0000 7f10e6ffd700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 0x7f10d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:07.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.774+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f10e0021070 con 0x7f10f0153880 2023-06-09T11:20:07.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.774+0000 7f10f619b700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 0x7f10d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:07.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.775+0000 7f10f619b700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 0x7f10d406d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009930 tx=0x7f10ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:07.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:07.778+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f10e0055f80 con 0x7f10f0153880 2023-06-09T11:20:08.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.027+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10f01f2970 con 0x7f10f0153880 2023-06-09T11:20:08.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.028+0000 7f10e6ffd700 1 -- 172.21.15.7:0/2216382638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f10e005de60 con 0x7f10f0153880 2023-06-09T11:20:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:08.034 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":47,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:07.257628+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 msgr2=0x7f10d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:08.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 0x7f10d406d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10ec009930 tx=0x7f10ec008040 comp rx=0 tx=0).stop 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 msgr2=0x7f10f014c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f10e0000c00 tx=0x7f10e0004750 comp rx=0 tx=0).stop 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 shutdown_connections 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d406ace0 0x7f10d406d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f015b2d0 0x7f10f01f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:08.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f00c9f40 0x7f10f0152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 --2- 172.21.15.7:0/2216382638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f0153880 0x7f10f014c9e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 >> 172.21.15.7:0/2216382638 conn(0x7f10f00737f0 msgr2=0x7f10f014e580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 shutdown_connections 2023-06-09T11:20:08.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:08.030+0000 7f10f83ff700 1 -- 172.21.15.7:0/2216382638 wait complete. 2023-06-09T11:20:08.039 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 47 2023-06-09T11:20:08.054 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:10.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:10.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.263+0000 7f9ec23ff700 1 Processor -- start 2023-06-09T11:20:10.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.263+0000 7f9ec23ff700 1 -- start start 2023-06-09T11:20:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ec23ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ebc0cc5b0 0x7f9ebc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ec23ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc159b20 con 0x7f9ebc0cb5d0 2023-06-09T11:20:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ec23ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ebc0cac80 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ec23ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ebc1597d0 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ebbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9ebc0cb5d0 0x7f9ebc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58782/0 (socket says 172.21.15.7:58782) 2023-06-09T11:20:10.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ebbfff700 1 -- 172.21.15.7:0/4059404479 learned_addr learned my addr 172.21.15.7:0/4059404479 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:10.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.264+0000 7f9ec099c700 1 --2- 172.21.15.7:0/4059404479 >> v2:172.21.15.7:3300/0 conn(0x7f9ebc0cc5b0 0x7f9ebc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ebaffd700 1 -- 172.21.15.7:0/4059404479 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052512056 0 0) 0x7f9ebc159b20 con 0x7f9ebc0cb5d0 2023-06-09T11:20:10.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ebaffd700 1 -- 172.21.15.7:0/4059404479 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea40035b0 con 0x7f9ebc0cb5d0 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ec099c700 1 -- 172.21.15.7:0/4059404479 >> v1:172.21.15.117:6789/0 conn(0x7f9ebc0cb9a0 legacy=0x7f9ebc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ec099c700 1 -- 172.21.15.7:0/4059404479 >> v1:172.21.15.7:6789/0 conn(0x7f9ebc0cb5d0 legacy=0x7f9ebc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ec099c700 1 -- 172.21.15.7:0/4059404479 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ebc071ab0 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ec099c700 1 --2- 172.21.15.7:0/4059404479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc0b52b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009d90 tx=0x7f9eb400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04d9d2ecde190af server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ebaffd700 1 -- 172.21.15.7:0/4059404479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400e070 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ebaffd700 1 -- 172.21.15.7:0/4059404479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb400bb20 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.265+0000 7f9ebaffd700 1 -- 172.21.15.7:0/4059404479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4012430 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4059404479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cc5b0 msgr2=0x7f9ebc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4059404479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc0b52b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009d90 tx=0x7f9eb400b330 comp rx=0 tx=0).stop 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4059404479 shutdown_connections 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4059404479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc0b52b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4059404479 >> 172.21.15.7:0/4059404479 conn(0x7f9ebc070af0 msgr2=0x7f9ebc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4059404479 shutdown_connections 2023-06-09T11:20:10.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.266+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4059404479 wait complete. 2023-06-09T11:20:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.268+0000 7f9ec23ff700 1 Processor -- start 2023-06-09T11:20:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.268+0000 7f9ec23ff700 1 -- start start 2023-06-09T11:20:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ebc0cb5d0 0x7f9ebc1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc0cd920 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc152690 con 0x7f9ebc0cc5b0 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec23ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ebc150760 con 0x7f9ebc0cb5d0 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ebb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ebb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60536/0 (socket says 172.21.15.7:60536) 2023-06-09T11:20:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ebb7fe700 1 -- 172.21.15.7:0/4263869038 learned_addr learned my addr 172.21.15.7:0/4263869038 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ebbfff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ebc0cb5d0 0x7f9ebc1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.269+0000 7f9ec099c700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 -- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ebc0cb5d0 msgr2=0x7f9ebc1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ebc0cb5d0 0x7f9ebc1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 -- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 msgr2=0x7f9ebc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 -- 172.21.15.7:0/4263869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb4009a70 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ec099c700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ebb7fe700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c4b0 tx=0x7f9eb000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00052d0 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb0005430 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ebc15d4f0 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00038c0 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.270+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ebc15d210 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.271+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ebc15d7d0 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.272+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9eb0003a20 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.273+0000 7f9eb8ff9700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 0x7f9ea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.273+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f9eb0088c90 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.273+0000 7f9ebbfff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 0x7f9ea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.274+0000 7f9ebbfff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 0x7f9ea806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0011e0 tx=0x7f9eac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.278+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9eb0053a20 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.532+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ebc0cda60 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.533+0000 7f9eb8ff9700 1 -- 172.21.15.7:0/4263869038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f9ebc0cda60 con 0x7f9ebc0cb9a0 2023-06-09T11:20:10.539 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:10.539 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:08.264861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:resolve","state_seq":36,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 msgr2=0x7f9ea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 0x7f9ea806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0011e0 tx=0x7f9eac009040 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 msgr2=0x7f9ebc0c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9eb000c4b0 tx=0x7f9eb000c870 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 shutdown_connections 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ebc0cc5b0 0x7f9ebc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ea806ace0 0x7f9ea806d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ebc0cb5d0 0x7f9ebc1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 --2- 172.21.15.7:0/4263869038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ebc0cb9a0 0x7f9ebc0c7c00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:10.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.535+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 >> 172.21.15.7:0/4263869038 conn(0x7f9ebc070af0 msgr2=0x7f9ebc0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:10.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.536+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 shutdown_connections 2023-06-09T11:20:10.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:10.536+0000 7f9ec23ff700 1 -- 172.21.15.7:0/4263869038 wait complete. 2023-06-09T11:20:10.543 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 48 2023-06-09T11:20:10.559 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:12.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:12.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:12.652 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.769+0000 7f878b5ff700 1 Processor -- start 2023-06-09T11:20:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.769+0000 7f878b5ff700 1 -- start start 2023-06-09T11:20:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.770+0000 7f878b5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8784153880 0x7f878415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.770+0000 7f878b5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87840c9f40 0x7f878415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.770+0000 7f878b5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f878415b2d0 0x7f87840c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.770+0000 7f878b5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87840c5f10 con 0x7f87840c9f40 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f878b5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878415a500 con 0x7f878415b2d0 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f878b5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878415da00 con 0x7f8784153880 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87840c9f40 0x7f878415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f87840c9f40 0x7f878415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60550/0 (socket says 172.21.15.7:60550) 2023-06-09T11:20:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 -- 172.21.15.7:0/2774028463 learned_addr learned my addr 172.21.15.7:0/2774028463 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 -- 172.21.15.7:0/2774028463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8784153880 msgr2=0x7f878415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2774028463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8784153880 0x7f878415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 -- 172.21.15.7:0/2774028463 >> v2:172.21.15.117:3300/0 conn(0x7f878415b2d0 msgr2=0x7f87840c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2774028463 >> v2:172.21.15.117:3300/0 conn(0x7f878415b2d0 0x7f87840c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 -- 172.21.15.7:0/2774028463 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878415b750 con 0x7f87840c9f40 2023-06-09T11:20:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.771+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2774028463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87840c9f40 0x7f878415ad90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8784071db0 tx=0x7f8774009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28988451c800afb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f877bfff700 1 -- 172.21.15.7:0/2774028463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400b870 con 0x7f87840c9f40 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f877bfff700 1 -- 172.21.15.7:0/2774028463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877400b9d0 con 0x7f87840c9f40 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f877bfff700 1 -- 172.21.15.7:0/2774028463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877400bd60 con 0x7f87840c9f40 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 -- 172.21.15.7:0/2774028463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87840c9f40 msgr2=0x7f878415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2774028463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87840c9f40 0x7f878415ad90 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8784071db0 tx=0x7f8774009bd0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 -- 172.21.15.7:0/2774028463 shutdown_connections 2023-06-09T11:20:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2774028463 >> v2:172.21.15.117:3300/0 conn(0x7f878415b2d0 0x7f87840c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2774028463 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8784153880 0x7f878415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2774028463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f87840c9f40 0x7f878415ad90 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 -- 172.21.15.7:0/2774028463 >> 172.21.15.7:0/2774028463 conn(0x7f87840737f0 msgr2=0x7f87840720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.772+0000 7f878b5ff700 1 -- 172.21.15.7:0/2774028463 shutdown_connections 2023-06-09T11:20:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.773+0000 7f878b5ff700 1 -- 172.21.15.7:0/2774028463 wait complete. 2023-06-09T11:20:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.774+0000 7f878b5ff700 1 Processor -- start 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.774+0000 7f878b5ff700 1 -- start start 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87840c9f40 0x7f87841eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f878415b2d0 0x7f8784148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8784151e10 con 0x7f8784153880 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87841590b0 con 0x7f878415b2d0 2023-06-09T11:20:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.775+0000 7f878b5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87840ccb80 con 0x7f87840c9f40 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60552/0 (socket says 172.21.15.7:60552) 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 -- 172.21.15.7:0/2736045683 learned_addr learned my addr 172.21.15.7:0/2736045683 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f878939b700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87840c9f40 0x7f87841eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8789b9c700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f878415b2d0 0x7f8784148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 -- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87840c9f40 msgr2=0x7f87841eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87840c9f40 0x7f87841eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 -- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f878415b2d0 msgr2=0x7f8784148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f878415b2d0 0x7f8784148fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 -- 172.21.15.7:0/2736045683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8774009550 con 0x7f8784153880 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.776+0000 7f8788b9a700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.777+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8774004310 con 0x7f8784153880 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.777+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877401f070 con 0x7f8784153880 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.777+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87740036a0 con 0x7f8784153880 2023-06-09T11:20:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.777+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8784159980 con 0x7f8784153880 2023-06-09T11:20:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.777+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87841596e0 con 0x7f8784153880 2023-06-09T11:20:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.778+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8784159c60 con 0x7f8784153880 2023-06-09T11:20:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.779+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8774021030 con 0x7f8784153880 2023-06-09T11:20:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.780+0000 7f8779ffb700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 0x7f876806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.780+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f877408aa20 con 0x7f8784153880 2023-06-09T11:20:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.780+0000 7f878939b700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 0x7f876806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.781+0000 7f878939b700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 0x7f876806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8780009a10 tx=0x7f8780009190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:12.784+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87740558d0 con 0x7f8784153880 2023-06-09T11:20:13.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.037+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8784074cf0 con 0x7f8784153880 2023-06-09T11:20:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.037+0000 7f8779ffb700 1 -- 172.21.15.7:0/2736045683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f8774058ef0 con 0x7f8784153880 2023-06-09T11:20:13.043 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:13.044 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:08.264861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:resolve","state_seq":36,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 msgr2=0x7f876806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 0x7f876806d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8780009a10 tx=0x7f8780009190 comp rx=0 tx=0).stop 2023-06-09T11:20:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 msgr2=0x7f87840ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774004750 comp rx=0 tx=0).stop 2023-06-09T11:20:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 shutdown_connections 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f878415b2d0 0x7f8784148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f876806ace0 0x7f876806d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87840c9f40 0x7f87841eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 --2- 172.21.15.7:0/2736045683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8784153880 0x7f87840ce660 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 >> 172.21.15.7:0/2736045683 conn(0x7f87840737f0 msgr2=0x7f878414c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 shutdown_connections 2023-06-09T11:20:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:13.040+0000 7f878b5ff700 1 -- 172.21.15.7:0/2736045683 wait complete. 2023-06-09T11:20:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 48 2023-06-09T11:20:13.064 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:15.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:15.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.271+0000 7f94e41ff700 1 Processor -- start 2023-06-09T11:20:15.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.271+0000 7f94e41ff700 1 -- start start 2023-06-09T11:20:15.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.272+0000 7f94e41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 0x7f94dc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.272+0000 7f94e41ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f94dc071060 0x7f94dc158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.272+0000 7f94e41ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94dc1580a0 con 0x7f94dc151510 2023-06-09T11:20:15.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.272+0000 7f94e41ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94dc0714e0 con 0x7f94dc071060 2023-06-09T11:20:15.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.272+0000 7f94e41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1518e0 con 0x7f94dc0c5b40 2023-06-09T11:20:15.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e1f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f94dc151510 0x7f94dc15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44702/0 (socket says 172.21.15.7:44702) 2023-06-09T11:20:15.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e1f9b700 1 -- 172.21.15.7:0/1614606323 learned_addr learned my addr 172.21.15.7:0/1614606323 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:15.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e279c700 1 --2- 172.21.15.7:0/1614606323 >> v2:172.21.15.117:3300/0 conn(0x7f94dc071060 0x7f94dc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e179a700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 0x7f94dc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447976391 0 0) 0x7f94dc1580a0 con 0x7f94dc151510 2023-06-09T11:20:15.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94c80035b0 con 0x7f94dc151510 2023-06-09T11:20:15.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706279663 0 0) 0x7f94c80035b0 con 0x7f94dc151510 2023-06-09T11:20:15.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e279c700 1 -- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 msgr2=0x7f94dc070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:15.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e279c700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 0x7f94dc070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e279c700 1 -- 172.21.15.7:0/1614606323 >> v1:172.21.15.7:6789/0 conn(0x7f94dc151510 legacy=0x7f94dc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e279c700 1 -- 172.21.15.7:0/1614606323 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0c6180 con 0x7f94dc071060 2023-06-09T11:20:15.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.273+0000 7f94e179a700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 0x7f94dc070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:20:15.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e279c700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc071060 0x7f94dc158870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009dc0 tx=0x7f94d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97b66d7634c1aeeb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:15.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d400e070 con 0x7f94dc071060 2023-06-09T11:20:15.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d4003a20 con 0x7f94dc071060 2023-06-09T11:20:15.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e0f99700 1 -- 172.21.15.7:0/1614606323 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8001670 con 0x7f94dc071060 2023-06-09T11:20:15.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e41ff700 1 -- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc071060 msgr2=0x7f94dc158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.274+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc071060 0x7f94dc158870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f94d4009dc0 tx=0x7f94d400b330 comp rx=0 tx=0).stop 2023-06-09T11:20:15.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 -- 172.21.15.7:0/1614606323 shutdown_connections 2023-06-09T11:20:15.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc071060 0x7f94dc158870 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1614606323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc0c5b40 0x7f94dc070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 -- 172.21.15.7:0/1614606323 >> 172.21.15.7:0/1614606323 conn(0x7f94dc06a4d0 msgr2=0x7f94dc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:15.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 -- 172.21.15.7:0/1614606323 shutdown_connections 2023-06-09T11:20:15.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.275+0000 7f94e41ff700 1 -- 172.21.15.7:0/1614606323 wait complete. 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.276+0000 7f94e41ff700 1 Processor -- start 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.277+0000 7f94e41ff700 1 -- start start 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.277+0000 7f94e41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.277+0000 7f94e41ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 0x7f94dc0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc151510 0x7f94dc0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc152390 con 0x7f94dc071060 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e41ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1efc50 con 0x7f94dc0c5b40 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94dc1ef060 con 0x7f94dc151510 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46886/0 (socket says 172.21.15.7:46886) 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 -- 172.21.15.7:0/1378119546 learned_addr learned my addr 172.21.15.7:0/1378119546 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e179a700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 0x7f94dc0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e279c700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc151510 0x7f94dc0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 -- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc151510 msgr2=0x7f94dc0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc151510 0x7f94dc0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 -- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 msgr2=0x7f94dc0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 0x7f94dc0c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e1f9b700 1 -- 172.21.15.7:0/1378119546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0094a0 con 0x7f94dc071060 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.278+0000 7f94e179a700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 0x7f94dc0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94e1f9b700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c750 tx=0x7f94d800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:15.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d8004c80 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d8004de0 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d80072f0 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d4009a70 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.279+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94dc157770 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.280+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94dc157490 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.281+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94d8007450 con 0x7f94dc071060 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.282+0000 7f94d27fc700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:15.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.282+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f94d8089430 con 0x7f94dc071060 2023-06-09T11:20:15.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.282+0000 7f94e179a700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:15.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.283+0000 7f94e179a700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94dc159590 tx=0x7f94cc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:15.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.287+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f94d8054250 con 0x7f94dc071060 2023-06-09T11:20:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.541+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94dc157a50 con 0x7f94dc071060 2023-06-09T11:20:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.542+0000 7f94d27fc700 1 -- 172.21.15.7:0/1378119546 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5461 (secure 0 0 0) 0x7f94d8007700 con 0x7f94dc071060 2023-06-09T11:20:15.548 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:15.548 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:13.377488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 msgr2=0x7f94c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 0x7f94c006d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94dc159590 tx=0x7f94cc009210 comp rx=0 tx=0).stop 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 msgr2=0x7f94dc14c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f94d800c750 tx=0x7f94d800ca60 comp rx=0 tx=0).stop 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 shutdown_connections 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.544+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f94c006ace0 0x7f94c006d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f94dc151510 0x7f94dc0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94dc071060 0x7f94dc14c180 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 --2- 172.21.15.7:0/1378119546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f94dc0c5b40 0x7f94dc0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 >> 172.21.15.7:0/1378119546 conn(0x7f94dc06a4d0 msgr2=0x7f94dc155e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 shutdown_connections 2023-06-09T11:20:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:15.545+0000 7f94e41ff700 1 -- 172.21.15.7:0/1378119546 wait complete. 2023-06-09T11:20:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2023-06-09T11:20:15.569 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 49, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 49, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_4936": { "addr": "172.21.15.117:6833/1239341654", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1239341654, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1239341654, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 4936, "incarnation": 21, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 18 }, "gid_5012": { "addr": "172.21.15.117:6837/3174501100", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3174501100, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3174501100, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5012, "incarnation": 46, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 38 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 54, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:20:13.377488+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5012, "mds_1": 5071, "mds_2": 4936 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:20:17.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.776+0000 7ffb667ff700 1 Processor -- start 2023-06-09T11:20:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.776+0000 7ffb667ff700 1 -- start start 2023-06-09T11:20:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb667ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb600c1bc0 0x7ffb601599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb667ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb60159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb667ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600c0bf0 con 0x7ffb600c1f90 2023-06-09T11:20:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb667ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb60070110 con 0x7ffb600c1bc0 2023-06-09T11:20:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb667ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60071300 con 0x7ffb600c0140 2023-06-09T11:20:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb5ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb600c1bc0 0x7ffb601599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb5ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb600c1bc0 0x7ffb601599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46890/0 (socket says 172.21.15.7:46890) 2023-06-09T11:20:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.777+0000 7ffb5ffff700 1 -- 172.21.15.7:0/3565402197 learned_addr learned my addr 172.21.15.7:0/3565402197 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5f7fe700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb60159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5ffff700 1 -- 172.21.15.7:0/3565402197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 msgr2=0x7ffb60159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb60159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5ffff700 1 -- 172.21.15.7:0/3565402197 >> v1:172.21.15.117:6789/0 conn(0x7ffb600c1f90 legacy=0x7ffb601501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5ffff700 1 -- 172.21.15.7:0/3565402197 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600702f0 con 0x7ffb600c1bc0 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5f7fe700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb60159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1bc0 0x7ffb601599b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000bb90 tx=0x7ffb5000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c38188f7cc938a3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5effd700 1 -- 172.21.15.7:0/3565402197 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000dd10 con 0x7ffb600c1bc0 2023-06-09T11:20:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5effd700 1 -- 172.21.15.7:0/3565402197 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5000de70 con 0x7ffb600c1bc0 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.778+0000 7ffb5effd700 1 -- 172.21.15.7:0/3565402197 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50004230 con 0x7ffb600c1bc0 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 -- 172.21.15.7:0/3565402197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1bc0 msgr2=0x7ffb601599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1bc0 0x7ffb601599b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000bb90 tx=0x7ffb5000bea0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 -- 172.21.15.7:0/3565402197 shutdown_connections 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1bc0 0x7ffb601599b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3565402197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb60159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 -- 172.21.15.7:0/3565402197 >> 172.21.15.7:0/3565402197 conn(0x7ffb6006a4d0 msgr2=0x7ffb6006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 -- 172.21.15.7:0/3565402197 shutdown_connections 2023-06-09T11:20:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.779+0000 7ffb667ff700 1 -- 172.21.15.7:0/3565402197 wait complete. 2023-06-09T11:20:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.781+0000 7ffb667ff700 1 Processor -- start 2023-06-09T11:20:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.782+0000 7ffb667ff700 1 -- start start 2023-06-09T11:20:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb6014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600c1bc0 0x7ffb600c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600c3af0 con 0x7ffb600c1f90 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601f42c0 con 0x7ffb600c1bc0 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb667ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601f38f0 con 0x7ffb600c0140 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb64d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb64d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46898/0 (socket says 172.21.15.7:46898) 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb64d9c700 1 -- 172.21.15.7:0/3795591820 learned_addr learned my addr 172.21.15.7:0/3795591820 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.783+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb6014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb5f7fe700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600c1bc0 0x7ffb600c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 -- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 msgr2=0x7ffb6014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb6014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 -- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600c1bc0 msgr2=0x7ffb600c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600c1bc0 0x7ffb600c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 -- 172.21.15.7:0/3795591820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5000b7f0 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb64d9c700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b970 tx=0x7ffb5800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800c930 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5800cad0 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb600c1170 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800ce80 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.784+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb600c0e90 con 0x7ffb600c1f90 2023-06-09T11:20:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.786+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb600c1450 con 0x7ffb600c1f90 2023-06-09T11:20:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.786+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb58014440 con 0x7ffb600c1f90 2023-06-09T11:20:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.787+0000 7ffb5cff9700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 0x7ffb4c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.787+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7ffb58089130 con 0x7ffb600c1f90 2023-06-09T11:20:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.788+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 0x7ffb4c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.788+0000 7ffb5ffff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 0x7ffb4c06d1c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000b340 tx=0x7ffb5001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:17.792+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffb580540a0 con 0x7ffb600c1f90 2023-06-09T11:20:18.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.048+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb60074cf0 con 0x7ffb600c1f90 2023-06-09T11:20:18.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.049+0000 7ffb5cff9700 1 -- 172.21.15.7:0/3795591820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+5461 (secure 0 0 0) 0x7ffb580576c0 con 0x7ffb600c1f90 2023-06-09T11:20:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:18.055 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:13.377488+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:reconnect","state_seq":38,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 msgr2=0x7ffb4c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 0x7ffb4c06d1c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000b340 tx=0x7ffb5001c040 comp rx=0 tx=0).stop 2023-06-09T11:20:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 msgr2=0x7ffb600c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b970 tx=0x7ffb5800bc80 comp rx=0 tx=0).stop 2023-06-09T11:20:18.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 shutdown_connections 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb600c1f90 0x7ffb600c2520 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb4c06ad10 0x7ffb4c06d1c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb600c0140 0x7ffb6014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 --2- 172.21.15.7:0/3795591820 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb600c1bc0 0x7ffb600c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 >> 172.21.15.7:0/3795591820 conn(0x7ffb6006a4d0 msgr2=0x7ffb6014daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 shutdown_connections 2023-06-09T11:20:18.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:18.051+0000 7ffb667ff700 1 -- 172.21.15.7:0/3795591820 wait complete. 2023-06-09T11:20:18.059 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2023-06-09T11:20:18.075 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:20.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.282+0000 7fe8defff700 1 Processor -- start 2023-06-09T11:20:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.282+0000 7fe8defff700 1 -- start start 2023-06-09T11:20:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8defff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8d80cadf0 0x7fe8d80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8defff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8d80b52b0 0x7fe8d80cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8defff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d815ccd0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8defff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80cfac0 con 0x7fe8d80b52b0 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8defff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80cfce0 con 0x7fe8d80cadf0 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.283+0000 7fe8d7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8d80b52b0 0x7fe8d80cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8dd59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8d80cb1c0 0x7fe8d80c8d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44716/0 (socket says 172.21.15.7:44716) 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8dd59c700 1 -- 172.21.15.7:0/3594550141 learned_addr learned my addr 172.21.15.7:0/3594550141 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8dcd9b700 1 --2- 172.21.15.7:0/3594550141 >> v2:172.21.15.117:3300/0 conn(0x7fe8d80cadf0 0x7fe8d80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d7fff700 1 -- 172.21.15.7:0/3594550141 >> v2:172.21.15.117:3300/0 conn(0x7fe8d80cadf0 msgr2=0x7fe8d80cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d7fff700 1 --2- 172.21.15.7:0/3594550141 >> v2:172.21.15.117:3300/0 conn(0x7fe8d80cadf0 0x7fe8d80cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d7fff700 1 -- 172.21.15.7:0/3594550141 >> v1:172.21.15.7:6789/0 conn(0x7fe8d80cb1c0 legacy=0x7fe8d80c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d7fff700 1 -- 172.21.15.7:0/3594550141 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d815cf20 con 0x7fe8d80b52b0 2023-06-09T11:20:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d77fe700 1 -- 172.21.15.7:0/3594550141 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627179131 0 0) 0x7fe8d815ccd0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.284+0000 7fe8d7fff700 1 --2- 172.21.15.7:0/3594550141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80b52b0 0x7fe8d80cf580 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe8d808d090 tx=0x7fe8c4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de1ca9c0df4c22c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8d77fe700 1 -- 172.21.15.7:0/3594550141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011830 con 0x7fe8d80b52b0 2023-06-09T11:20:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8d77fe700 1 -- 172.21.15.7:0/3594550141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4011990 con 0x7fe8d80b52b0 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8d77fe700 1 -- 172.21.15.7:0/3594550141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4011d20 con 0x7fe8d80b52b0 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 -- 172.21.15.7:0/3594550141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80b52b0 msgr2=0x7fe8d80cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 --2- 172.21.15.7:0/3594550141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80b52b0 0x7fe8d80cf580 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe8d808d090 tx=0x7fe8c4009b20 comp rx=0 tx=0).stop 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 -- 172.21.15.7:0/3594550141 shutdown_connections 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 --2- 172.21.15.7:0/3594550141 >> v2:172.21.15.117:3300/0 conn(0x7fe8d80cadf0 0x7fe8d80cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 --2- 172.21.15.7:0/3594550141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80b52b0 0x7fe8d80cf580 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 -- 172.21.15.7:0/3594550141 >> 172.21.15.7:0/3594550141 conn(0x7fe8d8070af0 msgr2=0x7fe8d8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 -- 172.21.15.7:0/3594550141 shutdown_connections 2023-06-09T11:20:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.285+0000 7fe8defff700 1 -- 172.21.15.7:0/3594550141 wait complete. 2023-06-09T11:20:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.287+0000 7fe8defff700 1 Processor -- start 2023-06-09T11:20:20.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.287+0000 7fe8defff700 1 -- start start 2023-06-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 0x7fe8d81efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 0x7fe8d80ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80c6170 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80ac260 con 0x7fe8d80cadf0 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8defff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b2ae0 con 0x7fe8d80b52b0 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8dd59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8dd59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46916/0 (socket says 172.21.15.7:46916) 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.288+0000 7fe8dd59c700 1 -- 172.21.15.7:0/3783418728 learned_addr learned my addr 172.21.15.7:0/3783418728 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dcd9b700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 0x7fe8d81efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8d7fff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 0x7fe8d80ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 -- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 msgr2=0x7fe8d81efc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 0x7fe8d81efc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 -- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 msgr2=0x7fe8d80ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 0x7fe8d80ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 -- 172.21.15.7:0/3783418728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40094a0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dcd9b700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 0x7fe8d81efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:20:20.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8d7fff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 0x7fe8d80ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8dd59c700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000cba0 tx=0x7fe8d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d000ab80 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d000ad00 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d815d4e0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0007310 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.289+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d815d200 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.291+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d815d7c0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.291+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8d0007470 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.293+0000 7fe8d57fa700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:20.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.294+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fe8d0089130 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.294+0000 7fe8dcd9b700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:20.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.294+0000 7fe8dcd9b700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 0x7fe8bc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009a00 tx=0x7fe8cc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:20.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.298+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe8d0053fe0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.550+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8d80ac3d0 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.551+0000 7fe8d57fa700 1 -- 172.21.15.7:0/3783418728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+5458 (secure 0 0 0) 0x7fe8d0057600 con 0x7fe8d80cb1c0 2023-06-09T11:20:20.557 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:20.557 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:19.248470+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 msgr2=0x7fe8bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 0x7fe8bc06d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009a00 tx=0x7fe8cc008040 comp rx=0 tx=0).stop 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 msgr2=0x7fe8d80b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000cba0 tx=0x7fe8d000cf60 comp rx=0 tx=0).stop 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 shutdown_connections 2023-06-09T11:20:20.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8d80cb1c0 0x7fe8d80b25a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8d80b52b0 0x7fe8d81efc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 --2- 172.21.15.7:0/3783418728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8d80cadf0 0x7fe8d80ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 >> 172.21.15.7:0/3783418728 conn(0x7fe8d8070af0 msgr2=0x7fe8d80c7930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 shutdown_connections 2023-06-09T11:20:20.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:20.554+0000 7fe8defff700 1 -- 172.21.15.7:0/3783418728 wait complete. 2023-06-09T11:20:20.562 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 50 2023-06-09T11:20:20.578 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:22.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:22.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.783+0000 7f8e763ff700 1 Processor -- start 2023-06-09T11:20:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.784+0000 7f8e763ff700 1 -- start start 2023-06-09T11:20:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e763ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e700c6280 0x7f8e700c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e763ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e7015cac0 con 0x7f8e7006a4d0 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e763ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e700c37d0 con 0x7f8e7006e460 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e763ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e700c8870 con 0x7f8e700c6280 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e6ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e700c6280 0x7f8e700c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e6f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e7006a4d0 0x7f8e700737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44726/0 (socket says 172.21.15.7:44726) 2023-06-09T11:20:22.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e6ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8e700c6280 0x7f8e700c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46922/0 (socket says 172.21.15.7:46922) 2023-06-09T11:20:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.785+0000 7f8e6f7fe700 1 -- 172.21.15.7:0/1246394024 learned_addr learned my addr 172.21.15.7:0/1246394024 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6ffff700 1 -- 172.21.15.7:0/1246394024 >> v1:172.21.15.117:6789/0 conn(0x7f8e7006e460 legacy=0x7f8e7006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6ffff700 1 -- 172.21.15.7:0/1246394024 >> v1:172.21.15.7:6789/0 conn(0x7f8e7006a4d0 legacy=0x7f8e700737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6ffff700 1 -- 172.21.15.7:0/1246394024 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7015cd30 con 0x7f8e700c6280 2023-06-09T11:20:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6ffff700 1 --2- 172.21.15.7:0/1246394024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e700c76b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009840 tx=0x7f8e60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca47b9312c9a92d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6effd700 1 -- 172.21.15.7:0/1246394024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e60004230 con 0x7f8e700c6280 2023-06-09T11:20:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6effd700 1 -- 172.21.15.7:0/1246394024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e60004390 con 0x7f8e700c6280 2023-06-09T11:20:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.786+0000 7f8e6effd700 1 -- 172.21.15.7:0/1246394024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e600036a0 con 0x7f8e700c6280 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 -- 172.21.15.7:0/1246394024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 msgr2=0x7f8e700c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 --2- 172.21.15.7:0/1246394024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e700c76b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e60009840 tx=0x7f8e60009b50 comp rx=0 tx=0).stop 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 -- 172.21.15.7:0/1246394024 shutdown_connections 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 --2- 172.21.15.7:0/1246394024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e700c76b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 -- 172.21.15.7:0/1246394024 >> 172.21.15.7:0/1246394024 conn(0x7f8e7006f980 msgr2=0x7f8e700cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 -- 172.21.15.7:0/1246394024 shutdown_connections 2023-06-09T11:20:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.787+0000 7f8e763ff700 1 -- 172.21.15.7:0/1246394024 wait complete. 2023-06-09T11:20:22.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.789+0000 7f8e763ff700 1 Processor -- start 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.789+0000 7f8e763ff700 1 -- start start 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e7006a4d0 0x7f8e700cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e7006e460 0x7f8e701e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70142770 con 0x7f8e700c6280 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e70141f60 con 0x7f8e7006e460 2023-06-09T11:20:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e763ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7015a210 con 0x7f8e7006a4d0 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46926/0 (socket says 172.21.15.7:46926) 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 -- 172.21.15.7:0/3334724385 learned_addr learned my addr 172.21.15.7:0/3334724385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e6ffff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e7006a4d0 0x7f8e700cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e6f7fe700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e7006e460 0x7f8e701e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:22.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 -- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e7006a4d0 msgr2=0x7f8e700cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e7006a4d0 0x7f8e700cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 -- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e7006e460 msgr2=0x7f8e701e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e7006e460 0x7f8e701e3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.790+0000 7f8e7499c700 1 -- 172.21.15.7:0/3334724385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e600094a0 con 0x7f8e700c6280 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e7499c700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800cc10 tx=0x7f8e6800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800aa60 con 0x7f8e700c6280 2023-06-09T11:20:22.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e6800abc0 con 0x7f8e700c6280 2023-06-09T11:20:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7015d2f0 con 0x7f8e700c6280 2023-06-09T11:20:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68012420 con 0x7f8e700c6280 2023-06-09T11:20:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.791+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7015d010 con 0x7f8e700c6280 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.792+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7015d5d0 con 0x7f8e700c6280 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.793+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e68012580 con 0x7f8e700c6280 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.794+0000 7f8e6cff9700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 0x7f8e5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.795+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f8e68089a30 con 0x7f8e700c6280 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.795+0000 7f8e6ffff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 0x7f8e5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:22.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.795+0000 7f8e6ffff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 0x7f8e5c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e600090b0 tx=0x7f8e60008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:22.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:22.799+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8e68054850 con 0x7f8e700c6280 2023-06-09T11:20:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.052+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e70074cf0 con 0x7f8e700c6280 2023-06-09T11:20:23.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.053+0000 7f8e6cff9700 1 -- 172.21.15.7:0/3334724385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f8e68057e70 con 0x7f8e700c6280 2023-06-09T11:20:23.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:23.059 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:21.255535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:rejoin","state_seq":40,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:23.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 msgr2=0x7f8e5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:23.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 0x7f8e5c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8e600090b0 tx=0x7f8e60008030 comp rx=0 tx=0).stop 2023-06-09T11:20:23.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 msgr2=0x7f8e701e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:23.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800cc10 tx=0x7f8e6800cf20 comp rx=0 tx=0).stop 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 shutdown_connections 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e700c6280 0x7f8e701e44d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e5c06ace0 0x7f8e5c06d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e7006a4d0 0x7f8e700cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 --2- 172.21.15.7:0/3334724385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e7006e460 0x7f8e701e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:23.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 >> 172.21.15.7:0/3334724385 conn(0x7f8e7006f980 msgr2=0x7f8e700c45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:23.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 shutdown_connections 2023-06-09T11:20:23.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:23.055+0000 7f8e763ff700 1 -- 172.21.15.7:0/3334724385 wait complete. 2023-06-09T11:20:23.063 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 51 2023-06-09T11:20:23.079 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:20:25.080 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.285+0000 7fcd7fdff700 1 Processor -- start 2023-06-09T11:20:25.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.285+0000 7fcd7fdff700 1 -- start start 2023-06-09T11:20:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.286+0000 7fcd7fdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd780cb9a0 0x7fcd780b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.286+0000 7fcd7fdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd7815cd40 0x7fcd780cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.286+0000 7fcd7fdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd78159b20 con 0x7fcd780cb5d0 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.286+0000 7fcd7fdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780cac80 con 0x7fcd780cb9a0 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.286+0000 7fcd7fdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781597d0 con 0x7fcd7815cd40 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd780cb9a0 0x7fcd780b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7db9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcd780cb5d0 0x7fcd78159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56568/0 (socket says 172.21.15.7:56568) 2023-06-09T11:20:25.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7db9b700 1 -- 172.21.15.7:0/1970208467 learned_addr learned my addr 172.21.15.7:0/1970208467 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7e39c700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd7815cd40 0x7fcd780cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 -- 172.21.15.7:0/1970208467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd7815cd40 msgr2=0x7fcd780cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd7815cd40 0x7fcd780cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 -- 172.21.15.7:0/1970208467 >> v1:172.21.15.7:6789/0 conn(0x7fcd780cb5d0 legacy=0x7fcd78159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 -- 172.21.15.7:0/1970208467 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78071ab0 con 0x7fcd780cb9a0 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7cb99700 1 -- 172.21.15.7:0/1970208467 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358193685 0 0) 0x7fcd78159b20 con 0x7fcd780cb5d0 2023-06-09T11:20:25.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.287+0000 7fcd7d39a700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780cb9a0 0x7fcd780b52b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009840 tx=0x7fcd68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b750477d11cc6a5b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7cb99700 1 -- 172.21.15.7:0/1970208467 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011830 con 0x7fcd780cb9a0 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7cb99700 1 -- 172.21.15.7:0/1970208467 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd68011990 con 0x7fcd780cb9a0 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7cb99700 1 -- 172.21.15.7:0/1970208467 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68011d20 con 0x7fcd780cb9a0 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 -- 172.21.15.7:0/1970208467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780cb9a0 msgr2=0x7fcd780b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780cb9a0 0x7fcd780b52b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009840 tx=0x7fcd68009b50 comp rx=0 tx=0).stop 2023-06-09T11:20:25.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 -- 172.21.15.7:0/1970208467 shutdown_connections 2023-06-09T11:20:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd7815cd40 0x7fcd780cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/1970208467 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd780cb9a0 0x7fcd780b52b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 -- 172.21.15.7:0/1970208467 >> 172.21.15.7:0/1970208467 conn(0x7fcd78070af0 msgr2=0x7fcd78070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 -- 172.21.15.7:0/1970208467 shutdown_connections 2023-06-09T11:20:25.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.288+0000 7fcd7fdff700 1 -- 172.21.15.7:0/1970208467 wait complete. 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.290+0000 7fcd7fdff700 1 Processor -- start 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.290+0000 7fcd7fdff700 1 -- start start 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd780cb5d0 0x7fcd7815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 0x7fcd781528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781f30d0 con 0x7fcd7815cd40 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781f2450 con 0x7fcd780cb5d0 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7fdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd781f2730 con 0x7fcd780cb9a0 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.291+0000 7fcd7e39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44018/0 (socket says 172.21.15.7:44018) 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 -- 172.21.15.7:0/353986178 learned_addr learned my addr 172.21.15.7:0/353986178 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7d39a700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 0x7fcd781528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7db9b700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd780cb5d0 0x7fcd7815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 -- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 msgr2=0x7fcd781528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 0x7fcd781528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 -- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd780cb5d0 msgr2=0x7fcd7815b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd780cb5d0 0x7fcd7815b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 -- 172.21.15.7:0/353986178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680094a0 con 0x7fcd7815cd40 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7d39a700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 0x7fcd781528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7e39c700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007d40 tx=0x7fcd7000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd7000ede0 con 0x7fcd7815cd40 2023-06-09T11:20:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70014420 con 0x7fcd7815cd40 2023-06-09T11:20:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700147d0 con 0x7fcd7815cd40 2023-06-09T11:20:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.292+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7815d4f0 con 0x7fcd7815cd40 2023-06-09T11:20:25.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.293+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7815d210 con 0x7fcd7815cd40 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.294+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd7815d7d0 con 0x7fcd7815cd40 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.295+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd70014930 con 0x7fcd7815cd40 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.296+0000 7fcd6e7fc700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.296+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7fcd70089e40 con 0x7fcd7815cd40 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.296+0000 7fcd7db9b700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.296+0000 7fcd7db9b700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 0x7fcd5c06d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcd74000f60 tx=0x7fcd74007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.300+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcd70054cf0 con 0x7fcd7815cd40 2023-06-09T11:20:25.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.558+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd780c9c90 con 0x7fcd7815cd40 2023-06-09T11:20:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.559+0000 7fcd6e7fc700 1 -- 172.21.15.7:0/353986178 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fcd70058310 con 0x7fcd7815cd40 2023-06-09T11:20:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:25.565 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:25.269650+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.561+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 msgr2=0x7fcd5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.561+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 0x7fcd5c06d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcd74000f60 tx=0x7fcd74007be0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.561+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 msgr2=0x7fcd78152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.561+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fcd70007d40 tx=0x7fcd7000e3c0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 shutdown_connections 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd7815cd40 0x7fcd78152de0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd780cb5d0 0x7fcd7815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd5c06ace0 0x7fcd5c06d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 --2- 172.21.15.7:0/353986178 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd780cb9a0 0x7fcd781528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 >> 172.21.15.7:0/353986178 conn(0x7fcd78070af0 msgr2=0x7fcd780cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 shutdown_connections 2023-06-09T11:20:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.562+0000 7fcd7fdff700 1 -- 172.21.15.7:0/353986178 wait complete. 2023-06-09T11:20:25.569 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2023-06-09T11:20:25.585 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=4889 2023-06-09T11:20:25.586 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.d 2023-06-09T11:20:25.586 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:20:25.586 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:20:25.586 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:20:25.590 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:20:25.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:25.741 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:20:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.787+0000 7f4a53bff700 1 Processor -- start 2023-06-09T11:20:25.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.787+0000 7f4a53bff700 1 -- start start 2023-06-09T11:20:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.788+0000 7f4a53bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4a4c0b5360 0x7f4a4c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.788+0000 7f4a53bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.788+0000 7f4a53bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c15dcf0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.788+0000 7f4a53bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0c5e00 con 0x7f4a4c0b5360 2023-06-09T11:20:25.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.788+0000 7f4a53bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15d450 con 0x7f4a4c0ca580 2023-06-09T11:20:25.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5199b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4a4c0ca1b0 0x7f4a4c1536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56572/0 (socket says 172.21.15.7:56572) 2023-06-09T11:20:25.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5199b700 1 -- 172.21.15.7:0/4213096751 learned_addr learned my addr 172.21.15.7:0/4213096751 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:20:25.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 --2- 172.21.15.7:0/4213096751 >> v2:172.21.15.117:3300/0 conn(0x7f4a4c0b5360 0x7f4a4c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5219c700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 -- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 msgr2=0x7f4a4c0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 -- 172.21.15.7:0/4213096751 >> v1:172.21.15.7:6789/0 conn(0x7f4a4c0ca1b0 legacy=0x7f4a4c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 -- 172.21.15.7:0/4213096751 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c15d730 con 0x7f4a4c0b5360 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a50999700 1 -- 172.21.15.7:0/4213096751 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364777706 0 0) 0x7f4a4c15dcf0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.789+0000 7f4a5119a700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c15cdc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009840 tx=0x7f4a3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d2642a5812a53e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a50999700 1 -- 172.21.15.7:0/4213096751 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c011830 con 0x7f4a4c0b5360 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a50999700 1 -- 172.21.15.7:0/4213096751 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c011990 con 0x7f4a4c0b5360 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a50999700 1 -- 172.21.15.7:0/4213096751 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c011d20 con 0x7f4a4c0b5360 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 -- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 msgr2=0x7f4a4c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c15cdc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009840 tx=0x7f4a3c009b50 comp rx=0 tx=0).stop 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 -- 172.21.15.7:0/4213096751 shutdown_connections 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 --2- 172.21.15.7:0/4213096751 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c15cdc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 -- 172.21.15.7:0/4213096751 >> 172.21.15.7:0/4213096751 conn(0x7f4a4c072090 msgr2=0x7f4a4c072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.790+0000 7f4a53bff700 1 -- 172.21.15.7:0/4213096751 shutdown_connections 2023-06-09T11:20:25.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.791+0000 7f4a53bff700 1 -- 172.21.15.7:0/4213096751 wait complete. 2023-06-09T11:20:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.792+0000 7f4a53bff700 1 Processor -- start 2023-06-09T11:20:25.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.792+0000 7f4a53bff700 1 -- start start 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cd920 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c15c280 con 0x7f4a4c0b5360 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a53bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1f3070 con 0x7f4a4c0ca580 2023-06-09T11:20:25.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.793+0000 7f4a5119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44022/0 (socket says 172.21.15.7:44022) 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 -- 172.21.15.7:0/3675174992 learned_addr learned my addr 172.21.15.7:0/3675174992 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5199b700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5219c700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 -- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 msgr2=0x7f4a4c14fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c14fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 -- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 msgr2=0x7f4a4c1f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:25.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c1f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 -- 172.21.15.7:0/3675174992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0094a0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5219c700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a5119a700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0090b0 tx=0x7f4a3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0036a0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c01f070 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.794+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c06a570 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.795+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c003a40 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.795+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c15da10 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.796+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c06a850 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.797+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a3c003ba0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.797+0000 7f4a427fc700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 0x7f4a3006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.798+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f4a3c021070 con 0x7f4a4c0ca1b0 2023-06-09T11:20:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.798+0000 7f4a5199b700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 0x7f4a3006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:25.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.798+0000 7f4a5199b700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 0x7f4a3006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007fd0 tx=0x7f4a48007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:25.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:25.802+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4a3c055e50 con 0x7f4a4c0ca1b0 2023-06-09T11:20:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.053+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c1f31e0 con 0x7f4a4c0ca1b0 2023-06-09T11:20:26.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.054+0000 7f4a427fc700 1 -- 172.21.15.7:0/3675174992 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f4a3c059470 con 0x7f4a4c0ca1b0 2023-06-09T11:20:26.060 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:26.060 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:25.269650+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 msgr2=0x7f4a3006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 0x7f4a3006d1c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007fd0 tx=0x7f4a48007de0 comp rx=0 tx=0).stop 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 msgr2=0x7f4a4c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0090b0 tx=0x7f4a3c004750 comp rx=0 tx=0).stop 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 shutdown_connections 2023-06-09T11:20:26.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3006ad10 0x7f4a3006d1c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a4c0ca580 0x7f4a4c14fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a4c0b5360 0x7f4a4c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 --2- 172.21.15.7:0/3675174992 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a4c0ca1b0 0x7f4a4c0b43e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 >> 172.21.15.7:0/3675174992 conn(0x7f4a4c072090 msgr2=0x7f4a4c1534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 shutdown_connections 2023-06-09T11:20:26.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:26.056+0000 7f4a53bff700 1 -- 172.21.15.7:0/3675174992 wait complete. 2023-06-09T11:20:26.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2023-06-09T11:20:26.080 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:20:28.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:20:28.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.285+0000 7f80d21ff700 1 Processor -- start 2023-06-09T11:20:28.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.286+0000 7f80d21ff700 1 -- start start 2023-06-09T11:20:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.287+0000 7f80d21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc153880 0x7f80cc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.287+0000 7f80d21ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80cc0c9f40 0x7f80cc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80d21ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc15a500 con 0x7f80cc15b2d0 2023-06-09T11:20:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80d21ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80cc0c5f10 con 0x7f80cc0c9f40 2023-06-09T11:20:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80d21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15da00 con 0x7f80cc153880 2023-06-09T11:20:28.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80cc0c9f40 0x7f80cc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f80cc0c9f40 0x7f80cc15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44032/0 (socket says 172.21.15.7:44032) 2023-06-09T11:20:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 -- 172.21.15.7:0/807027384 learned_addr learned my addr 172.21.15.7:0/807027384 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:28.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80cb7fe700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc153880 0x7f80cc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 -- 172.21.15.7:0/807027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc153880 msgr2=0x7f80cc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc153880 0x7f80cc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 -- 172.21.15.7:0/807027384 >> v1:172.21.15.117:6789/0 conn(0x7f80cc15b2d0 legacy=0x7f80cc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.288+0000 7f80c3fff700 1 -- 172.21.15.7:0/807027384 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc15b750 con 0x7f80cc0c9f40 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80c3fff700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc0c9f40 0x7f80cc15ad90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f80b40098f0 tx=0x7f80b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de5cb65db82afdd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80caffd700 1 -- 172.21.15.7:0/807027384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4011830 con 0x7f80cc0c9f40 2023-06-09T11:20:28.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80caffd700 1 -- 172.21.15.7:0/807027384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4011990 con 0x7f80cc0c9f40 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80caffd700 1 -- 172.21.15.7:0/807027384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4011d20 con 0x7f80cc0c9f40 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 -- 172.21.15.7:0/807027384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc0c9f40 msgr2=0x7f80cc15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc0c9f40 0x7f80cc15ad90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f80b40098f0 tx=0x7f80b4009c00 comp rx=0 tx=0).stop 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 -- 172.21.15.7:0/807027384 shutdown_connections 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc153880 0x7f80cc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 --2- 172.21.15.7:0/807027384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc0c9f40 0x7f80cc15ad90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.289+0000 7f80d21ff700 1 -- 172.21.15.7:0/807027384 >> 172.21.15.7:0/807027384 conn(0x7f80cc0737f0 msgr2=0x7f80cc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.290+0000 7f80d21ff700 1 -- 172.21.15.7:0/807027384 shutdown_connections 2023-06-09T11:20:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.290+0000 7f80d21ff700 1 -- 172.21.15.7:0/807027384 wait complete. 2023-06-09T11:20:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.291+0000 7f80d21ff700 1 Processor -- start 2023-06-09T11:20:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.291+0000 7f80d21ff700 1 -- start start 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc0c9f40 0x7f80cc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80cc15b2d0 0x7f80cc1ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc154540 con 0x7f80cc153880 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1f28b0 con 0x7f80cc15b2d0 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.292+0000 7f80d21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc0b53f0 con 0x7f80cc0c9f40 2023-06-09T11:20:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44044/0 (socket says 172.21.15.7:44044) 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 -- 172.21.15.7:0/1995141792 learned_addr learned my addr 172.21.15.7:0/1995141792 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80cb7fe700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc0c9f40 0x7f80cc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80cbfff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80cc15b2d0 0x7f80cc1ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 -- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc0c9f40 msgr2=0x7f80cc0b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc0c9f40 0x7f80cc0b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 -- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80cc15b2d0 msgr2=0x7f80cc1ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80cc15b2d0 0x7f80cc1ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 -- 172.21.15.7:0/1995141792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4009550 con 0x7f80cc153880 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c3fff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.293+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4004480 con 0x7f80cc153880 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.294+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b401f070 con 0x7f80cc153880 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.294+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc159980 con 0x7f80cc153880 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.294+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40038c0 con 0x7f80cc153880 2023-06-09T11:20:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.294+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1596e0 con 0x7f80cc153880 2023-06-09T11:20:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.295+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc159c60 con 0x7f80cc153880 2023-06-09T11:20:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.296+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f80b4021030 con 0x7f80cc153880 2023-06-09T11:20:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.296+0000 7f80c8ff9700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 0x7f80ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:20:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.297+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f80b408ac70 con 0x7f80cc153880 2023-06-09T11:20:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.297+0000 7f80cb7fe700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 0x7f80ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:20:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.297+0000 7f80cb7fe700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 0x7f80ac06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009be0 tx=0x7f80bc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:20:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.301+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f80b4055c40 con 0x7f80cc153880 2023-06-09T11:20:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.555+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80cc074cf0 con 0x7f80cc153880 2023-06-09T11:20:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.555+0000 7f80c8ff9700 1 -- 172.21.15.7:0/1995141792 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+6217 (secure 0 0 0) 0x7f80b4059260 con 0x7f80cc153880 2023-06-09T11:20:28.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:20:28.562 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:25.269650+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 msgr2=0x7f80ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 0x7f80ac06d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009be0 tx=0x7f80bc009210 comp rx=0 tx=0).stop 2023-06-09T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 msgr2=0x7f80cc0cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f80b4000c00 tx=0x7f80b4004750 comp rx=0 tx=0).stop 2023-06-09T11:20:28.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 shutdown_connections 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f80cc15b2d0 0x7f80cc1ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f80ac06ace0 0x7f80ac06d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80cc0c9f40 0x7f80cc0b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 --2- 172.21.15.7:0/1995141792 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80cc153880 0x7f80cc0cb4c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 >> 172.21.15.7:0/1995141792 conn(0x7f80cc0737f0 msgr2=0x7f80cc0720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 shutdown_connections 2023-06-09T11:20:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:20:28.558+0000 7f80d21ff700 1 -- 172.21.15.7:0/1995141792 wait complete. 2023-06-09T11:20:28.566 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 53 2023-06-09T11:20:28.581 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:20:28.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 85 secs before thrashing 2023-06-09T11:20:35.276 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:20:35.272+0000 7fa6f8ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:20:35.277 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:20:35.272+0000 7fa6f8ff1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:20:35.277 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:20:35.272+0000 7fa6f8ff1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:20:42.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:20:42.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:12.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:12.958 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:43.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:43.135 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:21:53.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:21:53.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.783+0000 7f4e1fdff700 1 Processor -- start 2023-06-09T11:21:53.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.783+0000 7f4e1fdff700 1 -- start start 2023-06-09T11:21:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.784+0000 7f4e1fdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e180cb5d0 0x7f4e180b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.784+0000 7f4e1fdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e18159b20 con 0x7f4e1815cd40 2023-06-09T11:21:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.784+0000 7f4e1fdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e180cac80 con 0x7f4e180cb9a0 2023-06-09T11:21:53.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.784+0000 7f4e1fdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e181597d0 con 0x7f4e180cb5d0 2023-06-09T11:21:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1e39c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e1815cd40 0x7f4e180cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50128/0 (socket says 172.21.15.7:50128) 2023-06-09T11:21:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1e39c700 1 -- 172.21.15.7:0/2994291040 learned_addr learned my addr 172.21.15.7:0/2994291040 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:21:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/2994291040 >> v2:172.21.15.117:3300/0 conn(0x7f4e180cb5d0 0x7f4e180b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2603191106 0 0) 0x7f4e18159b20 con 0x7f4e1815cd40 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e040035b0 con 0x7f4e1815cd40 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668534665 0 0) 0x7f4e040035b0 con 0x7f4e1815cd40 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 >> v2:172.21.15.117:3300/0 conn(0x7f4e180cb5d0 msgr2=0x7f4e180b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 --2- 172.21.15.7:0/2994291040 >> v2:172.21.15.117:3300/0 conn(0x7f4e180cb5d0 0x7f4e180b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 >> v1:172.21.15.117:6789/0 conn(0x7f4e180cb9a0 legacy=0x7f4e18159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e18071ab0 con 0x7f4e1815cd40 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f4e100019f0 con 0x7f4e1815cd40 2023-06-09T11:21:53.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 >> v1:172.21.15.7:6789/0 conn(0x7f4e1815cd40 legacy=0x7f4e180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04003a50 0x7f4e04003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 0x7f4e04004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e04005200 0x7f4e0400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181597d0 con 0x7f4e04004850 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0400c0c0 con 0x7f4e04005200 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.785+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0400c280 con 0x7f4e04003a50 2023-06-09T11:21:53.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 0x7f4e04004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04003a50 0x7f4e04003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1e39c700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e04005200 0x7f4e0400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 -- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04003a50 msgr2=0x7f4e04003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04003a50 0x7f4e04003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 -- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e04005200 msgr2=0x7f4e0400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e04005200 0x7f4e0400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 -- 172.21.15.7:0/2994291040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040035b0 con 0x7f4e04004850 2023-06-09T11:21:53.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04003a50 0x7f4e04003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:21:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1e39c700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e04005200 0x7f4e0400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:21:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1e39c700 1 -- 172.21.15.7:0/2994291040 reap_dead start 2023-06-09T11:21:53.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.786+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 0x7f4e04004cc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4e180cbd70 tx=0x7f4e04005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3162ed93e62da8b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080052d0 con 0x7f4e04004850 2023-06-09T11:21:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08005430 con 0x7f4e04004850 2023-06-09T11:21:53.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1cb99700 1 -- 172.21.15.7:0/2994291040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080038c0 con 0x7f4e04004850 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 -- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 msgr2=0x7f4e04004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 0x7f4e04004cc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4e180cbd70 tx=0x7f4e04005880 comp rx=0 tx=0).stop 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 -- 172.21.15.7:0/2994291040 shutdown_connections 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/2994291040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e04004850 0x7f4e04004cc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 -- 172.21.15.7:0/2994291040 >> 172.21.15.7:0/2994291040 conn(0x7f4e18070af0 msgr2=0x7f4e18070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 -- 172.21.15.7:0/2994291040 shutdown_connections 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.787+0000 7f4e1fdff700 1 -- 172.21.15.7:0/2994291040 wait complete. 2023-06-09T11:21:53.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.789+0000 7f4e1fdff700 1 Processor -- start 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.789+0000 7f4e1fdff700 1 -- start start 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04004850 0x7f4e180c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 0x7f4e1815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181ef080 con 0x7f4e18151540 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e180c5c00 con 0x7f4e18158770 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1fdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e181ee770 con 0x7f4e04004850 2023-06-09T11:21:53.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60202/0 (socket says 172.21.15.7:60202) 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1d39a700 1 -- 172.21.15.7:0/3250625561 learned_addr learned my addr 172.21.15.7:0/3250625561 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1e39c700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 0x7f4e1815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.790+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04004850 0x7f4e180c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 -- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04004850 msgr2=0x7f4e180c56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04004850 0x7f4e180c56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 -- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 msgr2=0x7f4e1815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 0x7f4e1815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 -- 172.21.15.7:0/3250625561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1815d4f0 con 0x7f4e18151540 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1e39c700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 0x7f4e1815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1d39a700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015040 tx=0x7f4e08001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08001290 con 0x7f4e18151540 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0801d480 con 0x7f4e18151540 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1815d210 con 0x7f4e18151540 2023-06-09T11:21:53.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0801d810 con 0x7f4e18151540 2023-06-09T11:21:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.791+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1815d7d0 con 0x7f4e18151540 2023-06-09T11:21:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.793+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e18074cf0 con 0x7f4e18151540 2023-06-09T11:21:53.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.794+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e08019040 con 0x7f4e18151540 2023-06-09T11:21:53.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.794+0000 7f4e0e7fc700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 0x7f4dfc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:53.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.795+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f4e08089f40 con 0x7f4e18151540 2023-06-09T11:21:53.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.795+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 0x7f4dfc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:53.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.795+0000 7f4e1db9b700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 0x7f4dfc06d1c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e18064570 tx=0x7f4e14008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:53.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:53.799+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4e08054df0 con 0x7f4e18151540 2023-06-09T11:21:54.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.049+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e180caa60 con 0x7f4e18151540 2023-06-09T11:21:54.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.050+0000 7f4e0e7fc700 1 -- 172.21.15.7:0/3250625561 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f4e08058410 con 0x7f4e18151540 2023-06-09T11:21:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:21:54.055 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:21:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.052+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 msgr2=0x7f4dfc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.052+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 0x7f4dfc06d1c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4e18064570 tx=0x7f4e14008040 comp rx=0 tx=0).stop 2023-06-09T11:21:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.052+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 msgr2=0x7f4e18152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.052+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4e08015040 tx=0x7f4e08001120 comp rx=0 tx=0).stop 2023-06-09T11:21:54.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 shutdown_connections 2023-06-09T11:21:54.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e18158770 0x7f4e1815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dfc06ad10 0x7f4dfc06d1c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e04004850 0x7f4e180c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 --2- 172.21.15.7:0/3250625561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e18151540 0x7f4e18152170 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 >> 172.21.15.7:0/3250625561 conn(0x7f4e18070af0 msgr2=0x7f4e180c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:54.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 shutdown_connections 2023-06-09T11:21:54.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.053+0000 7f4e1fdff700 1 -- 172.21.15.7:0/3250625561 wait complete. 2023-06-09T11:21:54.059 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:21:54.075 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2023-06-09T11:21:54.075 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:21:54.076 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:21:54.096 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:21:54.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:21:54.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.301+0000 7f62e51ff700 1 Processor -- start 2023-06-09T11:21:54.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.302+0000 7f62e51ff700 1 -- start start 2023-06-09T11:21:54.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.303+0000 7f62e51ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e0153880 0x7f62e015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.303+0000 7f62e51ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62e015b4a0 0x7f62e00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.303+0000 7f62e51ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e015a500 con 0x7f62e00c9f40 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.303+0000 7f62e51ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00c5f10 con 0x7f62e0153880 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62e51ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e015da00 con 0x7f62e015b4a0 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e0153880 0x7f62e015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62e0153880 0x7f62e015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60214/0 (socket says 172.21.15.7:60214) 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 -- 172.21.15.7:0/4152845387 learned_addr learned my addr 172.21.15.7:0/4152845387 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:54.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62df59e700 1 --2- 172.21.15.7:0/4152845387 >> v2:172.21.15.117:3300/0 conn(0x7f62e015b4a0 0x7f62e00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 -- 172.21.15.7:0/4152845387 >> v2:172.21.15.117:3300/0 conn(0x7f62e015b4a0 msgr2=0x7f62e00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 --2- 172.21.15.7:0/4152845387 >> v2:172.21.15.117:3300/0 conn(0x7f62e015b4a0 0x7f62e00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 -- 172.21.15.7:0/4152845387 >> v1:172.21.15.117:6789/0 conn(0x7f62e00c9f40 legacy=0x7f62e015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.304+0000 7f62ded9d700 1 -- 172.21.15.7:0/4152845387 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e01596e0 con 0x7f62e0153880 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.305+0000 7f62ded9d700 1 --2- 172.21.15.7:0/4152845387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0153880 0x7f62e015d4c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f62e00c4640 tx=0x7f62d0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e9cdfba200eedc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.305+0000 7f62ddd9b700 1 -- 172.21.15.7:0/4152845387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0011830 con 0x7f62e0153880 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.305+0000 7f62ddd9b700 1 -- 172.21.15.7:0/4152845387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0011990 con 0x7f62e0153880 2023-06-09T11:21:54.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.305+0000 7f62ddd9b700 1 -- 172.21.15.7:0/4152845387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0011d20 con 0x7f62e0153880 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 -- 172.21.15.7:0/4152845387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0153880 msgr2=0x7f62e015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 --2- 172.21.15.7:0/4152845387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0153880 0x7f62e015d4c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f62e00c4640 tx=0x7f62d0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 -- 172.21.15.7:0/4152845387 shutdown_connections 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 --2- 172.21.15.7:0/4152845387 >> v2:172.21.15.117:3300/0 conn(0x7f62e015b4a0 0x7f62e00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 --2- 172.21.15.7:0/4152845387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e0153880 0x7f62e015d4c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 -- 172.21.15.7:0/4152845387 >> 172.21.15.7:0/4152845387 conn(0x7f62e00737f0 msgr2=0x7f62e00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:54.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 -- 172.21.15.7:0/4152845387 shutdown_connections 2023-06-09T11:21:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.306+0000 7f62e51ff700 1 -- 172.21.15.7:0/4152845387 wait complete. 2023-06-09T11:21:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.308+0000 7f62e51ff700 1 Processor -- start 2023-06-09T11:21:54.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.308+0000 7f62e51ff700 1 -- start start 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 0x7f62e01e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e015b4a0 0x7f62e0157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e0157700 con 0x7f62e00c9f40 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00b4210 con 0x7f62e0153880 2023-06-09T11:21:54.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62e51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e01591f0 con 0x7f62e015b4a0 2023-06-09T11:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62ded9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62ded9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60218/0 (socket says 172.21.15.7:60218) 2023-06-09T11:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62ded9d700 1 -- 172.21.15.7:0/866459169 learned_addr learned my addr 172.21.15.7:0/866459169 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62de59c700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 0x7f62e01e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62df59e700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e015b4a0 0x7f62e0157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.310+0000 7f62ded9d700 1 -- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e015b4a0 msgr2=0x7f62e0157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62ded9d700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e015b4a0 0x7f62e0157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62ded9d700 1 -- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 msgr2=0x7f62e01e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62ded9d700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 0x7f62e01e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62ded9d700 1 -- 172.21.15.7:0/866459169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0009550 con 0x7f62e00c9f40 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62de59c700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 0x7f62e01e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62ded9d700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0004480 con 0x7f62e00c9f40 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d001f070 con 0x7f62e00c9f40 2023-06-09T11:21:54.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e0159c60 con 0x7f62e00c9f40 2023-06-09T11:21:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00038e0 con 0x7f62e00c9f40 2023-06-09T11:21:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.311+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e0159980 con 0x7f62e00c9f40 2023-06-09T11:21:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.312+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e0159f40 con 0x7f62e00c9f40 2023-06-09T11:21:54.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.313+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62d0021030 con 0x7f62e00c9f40 2023-06-09T11:21:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.315+0000 7f62c77fe700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 0x7f62cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.315+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f62d008aa20 con 0x7f62e00c9f40 2023-06-09T11:21:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.315+0000 7f62de59c700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 0x7f62cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:54.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.316+0000 7f62de59c700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 0x7f62cc06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d0410 tx=0x7f62d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:54.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.319+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62d00558d0 con 0x7f62e00c9f40 2023-06-09T11:21:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.597+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62e0074cf0 con 0x7f62e00c9f40 2023-06-09T11:21:54.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.598+0000 7f62c77fe700 1 -- 172.21.15.7:0/866459169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f62d0058ef0 con 0x7f62e00c9f40 2023-06-09T11:21:54.603 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:21:54.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:21:54.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 msgr2=0x7f62cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 0x7f62cc06d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d0410 tx=0x7f62d4009040 comp rx=0 tx=0).stop 2023-06-09T11:21:54.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 msgr2=0x7f62e01e4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:54.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f62d0000c00 tx=0x7f62d0004750 comp rx=0 tx=0).stop 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 shutdown_connections 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62cc06ace0 0x7f62cc06d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62e015b4a0 0x7f62e0157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62e00c9f40 0x7f62e01e4e30 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 --2- 172.21.15.7:0/866459169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62e0153880 0x7f62e01e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 >> 172.21.15.7:0/866459169 conn(0x7f62e00737f0 msgr2=0x7f62e00661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 shutdown_connections 2023-06-09T11:21:54.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:54.600+0000 7f62e51ff700 1 -- 172.21.15.7:0/866459169 wait complete. 2023-06-09T11:21:54.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:21:54.623 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:21:56.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:21:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.843+0000 7f12e27ff700 1 Processor -- start 2023-06-09T11:21:56.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.843+0000 7f12e27ff700 1 -- start start 2023-06-09T11:21:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc153880 0x7f12dc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f12dc0c9f40 0x7f12dc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12dc15b2d0 0x7f12dc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc0c5f10 con 0x7f12dc15b2d0 2023-06-09T11:21:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc15a500 con 0x7f12dc0c9f40 2023-06-09T11:21:56.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.844+0000 7f12e27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc15da00 con 0x7f12dc153880 2023-06-09T11:21:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12dc15b2d0 0x7f12dc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12dc15b2d0 0x7f12dc0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60228/0 (socket says 172.21.15.7:60228) 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 -- 172.21.15.7:0/2163587597 learned_addr learned my addr 172.21.15.7:0/2163587597 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12dbfff700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc153880 0x7f12dc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12db7fe700 1 --2- 172.21.15.7:0/2163587597 >> v2:172.21.15.117:3300/0 conn(0x7f12dc0c9f40 0x7f12dc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 -- 172.21.15.7:0/2163587597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc153880 msgr2=0x7f12dc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc153880 0x7f12dc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 -- 172.21.15.7:0/2163587597 >> v2:172.21.15.117:3300/0 conn(0x7f12dc0c9f40 msgr2=0x7f12dc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:56.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 --2- 172.21.15.7:0/2163587597 >> v2:172.21.15.117:3300/0 conn(0x7f12dc0c9f40 0x7f12dc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 -- 172.21.15.7:0/2163587597 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc15b750 con 0x7f12dc15b2d0 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.845+0000 7f12e0d9c700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc15b2d0 0x7f12dc0c4e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0cd140 tx=0x7f12d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93582e6bb74111e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12daffd700 1 -- 172.21.15.7:0/2163587597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d400e070 con 0x7f12dc15b2d0 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12daffd700 1 -- 172.21.15.7:0/2163587597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d400b910 con 0x7f12dc15b2d0 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12daffd700 1 -- 172.21.15.7:0/2163587597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d400bcc0 con 0x7f12dc15b2d0 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 -- 172.21.15.7:0/2163587597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc15b2d0 msgr2=0x7f12dc0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:56.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc15b2d0 0x7f12dc0c4e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0cd140 tx=0x7f12d400b330 comp rx=0 tx=0).stop 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 -- 172.21.15.7:0/2163587597 shutdown_connections 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc15b2d0 0x7f12dc0c4e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 --2- 172.21.15.7:0/2163587597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc153880 0x7f12dc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 --2- 172.21.15.7:0/2163587597 >> v2:172.21.15.117:3300/0 conn(0x7f12dc0c9f40 0x7f12dc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 -- 172.21.15.7:0/2163587597 >> 172.21.15.7:0/2163587597 conn(0x7f12dc0737f0 msgr2=0x7f12dc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.846+0000 7f12e27ff700 1 -- 172.21.15.7:0/2163587597 shutdown_connections 2023-06-09T11:21:56.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.847+0000 7f12e27ff700 1 -- 172.21.15.7:0/2163587597 wait complete. 2023-06-09T11:21:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.848+0000 7f12e27ff700 1 Processor -- start 2023-06-09T11:21:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.848+0000 7f12e27ff700 1 -- start start 2023-06-09T11:21:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 0x7f12dc14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 0x7f12dc0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc150ba0 con 0x7f12dc153880 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1523c0 con 0x7f12dc0c9f40 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12e27ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0c69a0 con 0x7f12dc15b2d0 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.849+0000 7f12db7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60240/0 (socket says 172.21.15.7:60240) 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 -- 172.21.15.7:0/3757856290 learned_addr learned my addr 172.21.15.7:0/3757856290 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12dbfff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 0x7f12dc14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12e0d9c700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 0x7f12dc0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 -- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 msgr2=0x7f12dc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 0x7f12dc0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 -- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 msgr2=0x7f12dc14d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 0x7f12dc14d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 -- 172.21.15.7:0/3757856290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4009b30 con 0x7f12dc153880 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12e0d9c700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 0x7f12dc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:21:56.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12dbfff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 0x7f12dc14d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12db7fe700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e930 tx=0x7f12d000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0009690 con 0x7f12dc153880 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000c850 con 0x7f12dc153880 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000cbe0 con 0x7f12dc153880 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.850+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc159980 con 0x7f12dc153880 2023-06-09T11:21:56.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.851+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc1596e0 con 0x7f12dc153880 2023-06-09T11:21:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.852+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc159c60 con 0x7f12dc153880 2023-06-09T11:21:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.853+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f12d000cd40 con 0x7f12dc153880 2023-06-09T11:21:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.853+0000 7f12d8ff9700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 0x7f12c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:56.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.854+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f12d0089710 con 0x7f12dc153880 2023-06-09T11:21:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.854+0000 7f12dbfff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 0x7f12c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:56.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.854+0000 7f12dbfff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 0x7f12c806d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009c30 tx=0x7f12cc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:56.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:56.858+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f12d00545c0 con 0x7f12dc153880 2023-06-09T11:21:57.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.112+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12dc074cf0 con 0x7f12dc153880 2023-06-09T11:21:57.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.113+0000 7f12d8ff9700 1 -- 172.21.15.7:0/3757856290 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f12d0057be0 con 0x7f12dc153880 2023-06-09T11:21:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:21:57.118 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:21:57.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 msgr2=0x7f12c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:57.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 0x7f12c806d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009c30 tx=0x7f12cc008040 comp rx=0 tx=0).stop 2023-06-09T11:21:57.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 msgr2=0x7f12dc0cc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:57.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f12d000e930 tx=0x7f12d000ecf0 comp rx=0 tx=0).stop 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 shutdown_connections 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f12c806ace0 0x7f12c806d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12dc15b2d0 0x7f12dc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f12dc0c9f40 0x7f12dc14d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 --2- 172.21.15.7:0/3757856290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12dc153880 0x7f12dc0cc4c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 >> 172.21.15.7:0/3757856290 conn(0x7f12dc0737f0 msgr2=0x7f12dc06a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 shutdown_connections 2023-06-09T11:21:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:57.115+0000 7f12e27ff700 1 -- 172.21.15.7:0/3757856290 wait complete. 2023-06-09T11:21:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:21:57.138 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:21:59.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:21:59.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.340+0000 7f38339ff700 1 Processor -- start 2023-06-09T11:21:59.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.341+0000 7f38339ff700 1 -- start start 2023-06-09T11:21:59.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f38339ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f382c0ca1b0 0x7f382c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f38339ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f38339ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f382c15dcf0 con 0x7f382c0b5360 2023-06-09T11:21:59.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f38339ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382c0c5e00 con 0x7f382c0ca1b0 2023-06-09T11:21:59.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f38339ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c15d450 con 0x7f382c0ca580 2023-06-09T11:21:59.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f383179b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f382c0ca1b0 0x7f382c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f383179b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f382c0ca1b0 0x7f382c15cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60252/0 (socket says 172.21.15.7:60252) 2023-06-09T11:21:59.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f383179b700 1 -- 172.21.15.7:0/1309753959 learned_addr learned my addr 172.21.15.7:0/1309753959 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:59.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.342+0000 7f3831f9c700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f383179b700 1 -- 172.21.15.7:0/1309753959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 msgr2=0x7f382c0c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:21:59.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f383179b700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c0c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f383179b700 1 -- 172.21.15.7:0/1309753959 >> v1:172.21.15.117:6789/0 conn(0x7f382c0b5360 legacy=0x7f382c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f383179b700 1 -- 172.21.15.7:0/1309753959 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c15d730 con 0x7f382c0ca1b0 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f3831f9c700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f383179b700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15cdc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38280098f0 tx=0x7f3828009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11037a53b2ceba1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f3823fff700 1 -- 172.21.15.7:0/1309753959 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800b8a0 con 0x7f382c0ca1b0 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f3823fff700 1 -- 172.21.15.7:0/1309753959 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382800ba00 con 0x7f382c0ca1b0 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.343+0000 7f3823fff700 1 -- 172.21.15.7:0/1309753959 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382800bd90 con 0x7f382c0ca1b0 2023-06-09T11:21:59.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 -- 172.21.15.7:0/1309753959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 msgr2=0x7f382c15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15cdc0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f38280098f0 tx=0x7f3828009c00 comp rx=0 tx=0).stop 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 -- 172.21.15.7:0/1309753959 shutdown_connections 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c0c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 --2- 172.21.15.7:0/1309753959 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15cdc0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 -- 172.21.15.7:0/1309753959 >> 172.21.15.7:0/1309753959 conn(0x7f382c072090 msgr2=0x7f382c072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 -- 172.21.15.7:0/1309753959 shutdown_connections 2023-06-09T11:21:59.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.344+0000 7f38339ff700 1 -- 172.21.15.7:0/1309753959 wait complete. 2023-06-09T11:21:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.345+0000 7f38339ff700 1 Processor -- start 2023-06-09T11:21:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.346+0000 7f38339ff700 1 -- start start 2023-06-09T11:21:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.346+0000 7f38339ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.346+0000 7f38339ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f38339ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f38339ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0cd510 con 0x7f382c0ca1b0 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f38339ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0cd760 con 0x7f382c0b5360 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f38339ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382c0cda40 con 0x7f382c0ca580 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f383179b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f383179b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:49292/0 (socket says 172.21.15.7:49292) 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f383179b700 1 -- 172.21.15.7:0/1305543001 learned_addr learned my addr 172.21.15.7:0/1305543001 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3831f9c700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 -- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 msgr2=0x7f382c148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 -- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 msgr2=0x7f382c1efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f3830f9a700 1 -- 172.21.15.7:0/1305543001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828009550 con 0x7f382c0ca1b0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.347+0000 7f383179b700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f3830f9a700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15bf40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f381c000f60 tx=0x7f381c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c0052d0 con 0x7f382c0ca1b0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c005430 con 0x7f382c0ca1b0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f382c06a570 con 0x7f382c0ca1b0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c0038c0 con 0x7f382c0ca1b0 2023-06-09T11:21:59.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.348+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f382c15da10 con 0x7f382c0ca1b0 2023-06-09T11:21:59.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.349+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382c06a850 con 0x7f382c0ca1b0 2023-06-09T11:21:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.350+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f381c003a20 con 0x7f382c0ca1b0 2023-06-09T11:21:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.351+0000 7f3821ffb700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 0x7f381006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:21:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.352+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f381c088a40 con 0x7f382c0ca1b0 2023-06-09T11:21:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.352+0000 7f383179b700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 0x7f381006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:21:59.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.352+0000 7f383179b700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 0x7f381006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3828009130 tx=0x7f3828008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:21:59.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.356+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f381c053860 con 0x7f382c0ca1b0 2023-06-09T11:21:59.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.609+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f382c074cf0 con 0x7f382c0ca1b0 2023-06-09T11:21:59.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.610+0000 7f3821ffb700 1 -- 172.21.15.7:0/1305543001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f381c056e80 con 0x7f382c0ca1b0 2023-06-09T11:21:59.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:21:59.615 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:21:59.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 msgr2=0x7f381006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 0x7f381006d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3828009130 tx=0x7f3828008030 comp rx=0 tx=0).stop 2023-06-09T11:21:59.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 msgr2=0x7f382c15bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15bf40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f381c000f60 tx=0x7f381c00c730 comp rx=0 tx=0).stop 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 shutdown_connections 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f381006ace0 0x7f381006d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f382c0ca580 0x7f382c148630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f382c0b5360 0x7f382c1efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 --2- 172.21.15.7:0/1305543001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f382c0ca1b0 0x7f382c15bf40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:21:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 >> 172.21.15.7:0/1305543001 conn(0x7f382c072090 msgr2=0x7f382c14e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:21:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.612+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 shutdown_connections 2023-06-09T11:21:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:21:59.613+0000 7f38339ff700 1 -- 172.21.15.7:0/1305543001 wait complete. 2023-06-09T11:21:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:21:59.635 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:22:01.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:01.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.852+0000 7f4c69fff700 1 Processor -- start 2023-06-09T11:22:01.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.852+0000 7f4c69fff700 1 -- start start 2023-06-09T11:22:01.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c69fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4c6415b4a0 0x7f4c6415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c69fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c640c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c69fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6415a500 con 0x7f4c64153880 2023-06-09T11:22:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c69fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c640c5f10 con 0x7f4c6415b4a0 2023-06-09T11:22:01.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c69fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6415da00 con 0x7f4c640c9f40 2023-06-09T11:22:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c637fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4c64153880 0x7f4c6415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50140/0 (socket says 172.21.15.7:50140) 2023-06-09T11:22:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c637fe700 1 -- 172.21.15.7:0/3011244075 learned_addr learned my addr 172.21.15.7:0/3011244075 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:01.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/3011244075 >> v2:172.21.15.117:3300/0 conn(0x7f4c6415b4a0 0x7f4c6415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c63fff700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c640c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.854+0000 7f4c627fc700 1 -- 172.21.15.7:0/3011244075 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021262309 0 0) 0x7f4c54001110 con 0x7f4c64153880 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c627fc700 1 -- 172.21.15.7:0/3011244075 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c4c0035b0 con 0x7f4c64153880 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c62ffd700 1 -- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 msgr2=0x7f4c640c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c640c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c62ffd700 1 -- 172.21.15.7:0/3011244075 >> v1:172.21.15.7:6789/0 conn(0x7f4c64153880 legacy=0x7f4c6415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:01.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c62ffd700 1 -- 172.21.15.7:0/3011244075 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c641596e0 con 0x7f4c6415b4a0 2023-06-09T11:22:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c63fff700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c640c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:22:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6415b4a0 0x7f4c6415d4c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009840 tx=0x7f4c58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f618dceddc64974 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c627fc700 1 -- 172.21.15.7:0/3011244075 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011830 con 0x7f4c6415b4a0 2023-06-09T11:22:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c627fc700 1 -- 172.21.15.7:0/3011244075 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58011990 con 0x7f4c6415b4a0 2023-06-09T11:22:01.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.855+0000 7f4c627fc700 1 -- 172.21.15.7:0/3011244075 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58011d20 con 0x7f4c6415b4a0 2023-06-09T11:22:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 -- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6415b4a0 msgr2=0x7f4c6415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6415b4a0 0x7f4c6415d4c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009840 tx=0x7f4c58009b50 comp rx=0 tx=0).stop 2023-06-09T11:22:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 -- 172.21.15.7:0/3011244075 shutdown_connections 2023-06-09T11:22:01.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c640c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 --2- 172.21.15.7:0/3011244075 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c6415b4a0 0x7f4c6415d4c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 -- 172.21.15.7:0/3011244075 >> 172.21.15.7:0/3011244075 conn(0x7f4c640737f0 msgr2=0x7f4c640720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 -- 172.21.15.7:0/3011244075 shutdown_connections 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.856+0000 7f4c69fff700 1 -- 172.21.15.7:0/3011244075 wait complete. 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.858+0000 7f4c69fff700 1 Processor -- start 2023-06-09T11:22:01.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.858+0000 7f4c69fff700 1 -- start start 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c641e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c64153880 0x7f4c64158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640cb7a0 con 0x7f4c6415b4a0 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c640b50d0 con 0x7f4c64153880 2023-06-09T11:22:01.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c69fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c641e6320 con 0x7f4c640c9f40 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c63fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c63fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60258/0 (socket says 172.21.15.7:60258) 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.859+0000 7f4c63fff700 1 -- 172.21.15.7:0/2548206795 learned_addr learned my addr 172.21.15.7:0/2548206795 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c637fe700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c641e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c62ffd700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c64153880 0x7f4c64158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 -- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 msgr2=0x7f4c641e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c641e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 -- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c64153880 msgr2=0x7f4c64158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c64153880 0x7f4c64158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 -- 172.21.15.7:0/2548206795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5c009a70 con 0x7f4c6415b4a0 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.860+0000 7f4c63fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00efc0 tx=0x7f4c5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.861+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00ca90 con 0x7f4c6415b4a0 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.861+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c00f040 con 0x7f4c6415b4a0 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.861+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c00ce10 con 0x7f4c6415b4a0 2023-06-09T11:22:01.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.861+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c580094a0 con 0x7f4c6415b4a0 2023-06-09T11:22:01.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.861+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c64159c60 con 0x7f4c6415b4a0 2023-06-09T11:22:01.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.862+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c64159980 con 0x7f4c6415b4a0 2023-06-09T11:22:01.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.863+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c5c014480 con 0x7f4c6415b4a0 2023-06-09T11:22:01.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.864+0000 7f4c4bfff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 0x7f4c5006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:01.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.864+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f4c5c089dc0 con 0x7f4c6415b4a0 2023-06-09T11:22:01.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.865+0000 7f4c637fe700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 0x7f4c5006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:01.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.865+0000 7f4c637fe700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 0x7f4c5006d1c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c54007fd0 tx=0x7f4c54009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:01.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:01.869+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c5c0585e0 con 0x7f4c6415b4a0 2023-06-09T11:22:02.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.154+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c64159f40 con 0x7f4c6415b4a0 2023-06-09T11:22:02.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.155+0000 7f4c4bfff700 1 -- 172.21.15.7:0/2548206795 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f4c5c018090 con 0x7f4c6415b4a0 2023-06-09T11:22:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:02.160 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.157+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 msgr2=0x7f4c5006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.157+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 0x7f4c5006d1c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c54007fd0 tx=0x7f4c54009040 comp rx=0 tx=0).stop 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.157+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 msgr2=0x7f4c64148df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.157+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00efc0 tx=0x7f4c5c00c3a0 comp rx=0 tx=0).stop 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 shutdown_connections 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c6415b4a0 0x7f4c64148df0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c5006ad10 0x7f4c5006d1c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c640c9f40 0x7f4c641e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:02.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 --2- 172.21.15.7:0/2548206795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c64153880 0x7f4c64158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 >> 172.21.15.7:0/2548206795 conn(0x7f4c640737f0 msgr2=0x7f4c6414de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 shutdown_connections 2023-06-09T11:22:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:02.158+0000 7f4c69fff700 1 -- 172.21.15.7:0/2548206795 wait complete. 2023-06-09T11:22:02.164 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:22:02.181 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:22:04.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:04.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.379+0000 7f379c1ff700 1 Processor -- start 2023-06-09T11:22:04.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.379+0000 7f379c1ff700 1 -- start start 2023-06-09T11:22:04.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379c1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f379415d4e0 0x7f3794066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379c1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37941592e0 0x7f379406e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379c1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3794071730 con 0x7f37941534c0 2023-06-09T11:22:04.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379c1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3794071ec0 con 0x7f379415d4e0 2023-06-09T11:22:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379c1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f379415d050 con 0x7f37941592e0 2023-06-09T11:22:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.380+0000 7f379979a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f379415d4e0 0x7f3794066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f379415d4e0 0x7f3794066a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55912/0 (socket says 172.21.15.7:55912) 2023-06-09T11:22:04.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 -- 172.21.15.7:0/3118342776 learned_addr learned my addr 172.21.15.7:0/3118342776 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379a79c700 1 --2- 172.21.15.7:0/3118342776 >> v2:172.21.15.117:3300/0 conn(0x7f37941592e0 0x7f379406e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 -- 172.21.15.7:0/3118342776 >> v2:172.21.15.117:3300/0 conn(0x7f37941592e0 msgr2=0x7f379406e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:04.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 --2- 172.21.15.7:0/3118342776 >> v2:172.21.15.117:3300/0 conn(0x7f37941592e0 0x7f379406e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 -- 172.21.15.7:0/3118342776 >> v1:172.21.15.117:6789/0 conn(0x7f37941534c0 legacy=0x7f3794070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 -- 172.21.15.7:0/3118342776 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3794071170 con 0x7f379415d4e0 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f379979a700 1 --2- 172.21.15.7:0/3118342776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f379415d4e0 0x7f3794066a40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f37940ccb00 tx=0x7f3784009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4199532f6be46278 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.381+0000 7f3798f99700 1 -- 172.21.15.7:0/3118342776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784011830 con 0x7f379415d4e0 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f3798f99700 1 -- 172.21.15.7:0/3118342776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3784011990 con 0x7f379415d4e0 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f3798f99700 1 -- 172.21.15.7:0/3118342776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784011d20 con 0x7f379415d4e0 2023-06-09T11:22:04.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 -- 172.21.15.7:0/3118342776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f379415d4e0 msgr2=0x7f3794066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3118342776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f379415d4e0 0x7f3794066a40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f37940ccb00 tx=0x7f3784009bd0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 -- 172.21.15.7:0/3118342776 shutdown_connections 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3118342776 >> v2:172.21.15.117:3300/0 conn(0x7f37941592e0 0x7f379406e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3118342776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f379415d4e0 0x7f3794066a40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 -- 172.21.15.7:0/3118342776 >> 172.21.15.7:0/3118342776 conn(0x7f37940cba20 msgr2=0x7f37940cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 -- 172.21.15.7:0/3118342776 shutdown_connections 2023-06-09T11:22:04.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.382+0000 7f379c1ff700 1 -- 172.21.15.7:0/3118342776 wait complete. 2023-06-09T11:22:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.384+0000 7f379c1ff700 1 Processor -- start 2023-06-09T11:22:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.384+0000 7f379c1ff700 1 -- start start 2023-06-09T11:22:04.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 0x7f37940c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379415d4e0 0x7f3794155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37941f2ed0 con 0x7f37941592e0 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940c5e10 con 0x7f37941534c0 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379c1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37941e3f50 con 0x7f379415d4e0 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55926/0 (socket says 172.21.15.7:55926) 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.385+0000 7f379979a700 1 -- 172.21.15.7:0/3666223526 learned_addr learned my addr 172.21.15.7:0/3666223526 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f3799f9b700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 0x7f37940c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379a79c700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379415d4e0 0x7f3794155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 -- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379415d4e0 msgr2=0x7f3794155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379415d4e0 0x7f3794155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 -- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 msgr2=0x7f37940c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 0x7f37940c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 -- 172.21.15.7:0/3666223526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3784009550 con 0x7f37941592e0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f3799f9b700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 0x7f37940c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379979a700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3784004530 con 0x7f37941592e0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378401f070 con 0x7f37941592e0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37840038e0 con 0x7f37941592e0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.386+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379415ca00 con 0x7f37941592e0 2023-06-09T11:22:04.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.387+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3794071450 con 0x7f37941592e0 2023-06-09T11:22:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.388+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379415cce0 con 0x7f37941592e0 2023-06-09T11:22:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.389+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3784021030 con 0x7f37941592e0 2023-06-09T11:22:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.390+0000 7f378a7fc700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.390+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f378408ab40 con 0x7f37941592e0 2023-06-09T11:22:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.390+0000 7f3799f9b700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.391+0000 7f3799f9b700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 0x7f377806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3790000f60 tx=0x7f3790009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.395+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f37840559f0 con 0x7f37941592e0 2023-06-09T11:22:04.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.645+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37941e4320 con 0x7f37941592e0 2023-06-09T11:22:04.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.646+0000 7f378a7fc700 1 -- 172.21.15.7:0/3666223526 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f37941e4320 con 0x7f37941592e0 2023-06-09T11:22:04.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:04.651 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 msgr2=0x7f377806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 0x7f377806d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3790000f60 tx=0x7f3790009040 comp rx=0 tx=0).stop 2023-06-09T11:22:04.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 msgr2=0x7f37940c86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3784000c00 tx=0x7f3784004750 comp rx=0 tx=0).stop 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 shutdown_connections 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f377806ace0 0x7f377806d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f379415d4e0 0x7f3794155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37941534c0 0x7f37940c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 --2- 172.21.15.7:0/3666223526 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37941592e0 0x7f37940c86f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:04.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 >> 172.21.15.7:0/3666223526 conn(0x7f37940cba20 msgr2=0x7f37940661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.648+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 shutdown_connections 2023-06-09T11:22:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:04.649+0000 7f379c1ff700 1 -- 172.21.15.7:0/3666223526 wait complete. 2023-06-09T11:22:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:22:04.671 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:22:06.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.870+0000 7f19411ff700 1 Processor -- start 2023-06-09T11:22:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.870+0000 7f19411ff700 1 -- start start 2023-06-09T11:22:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.871+0000 7f19411ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.871+0000 7f19411ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 0x7f193c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.871+0000 7f19411ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193c15cec0 con 0x7f193c0c9090 2023-06-09T11:22:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.871+0000 7f19411ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c0ca410 con 0x7f193c0cfaf0 2023-06-09T11:22:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.871+0000 7f19411ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c15d160 con 0x7f193c0c9fe0 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193b59e700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 0x7f193c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:41724/0 (socket says 172.21.15.7:41724) 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 -- 172.21.15.7:0/1136648504 learned_addr learned my addr 172.21.15.7:0/1136648504 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 -- 172.21.15.7:0/1136648504 >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 msgr2=0x7f193c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 --2- 172.21.15.7:0/1136648504 >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 0x7f193c1598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 -- 172.21.15.7:0/1136648504 >> v1:172.21.15.117:6789/0 conn(0x7f193c0c9090 legacy=0x7f193c0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193a59c700 1 -- 172.21.15.7:0/1136648504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c15d440 con 0x7f193c0c9fe0 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.872+0000 7f193b59e700 1 --2- 172.21.15.7:0/1136648504 >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 0x7f193c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f193a59c700 1 --2- 172.21.15.7:0/1136648504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19300098f0 tx=0x7f1930009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ef096b92eacefec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f1939d9b700 1 -- 172.21.15.7:0/1136648504 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930004230 con 0x7f193c0c9fe0 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f1939d9b700 1 -- 172.21.15.7:0/1136648504 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1930004390 con 0x7f193c0c9fe0 2023-06-09T11:22:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f1939d9b700 1 -- 172.21.15.7:0/1136648504 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19300036a0 con 0x7f193c0c9fe0 2023-06-09T11:22:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f19411ff700 1 -- 172.21.15.7:0/1136648504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 msgr2=0x7f193c0cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.873+0000 7f19411ff700 1 --2- 172.21.15.7:0/1136648504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f19300098f0 tx=0x7f1930009c00 comp rx=0 tx=0).stop 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 -- 172.21.15.7:0/1136648504 shutdown_connections 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 --2- 172.21.15.7:0/1136648504 >> v2:172.21.15.117:3300/0 conn(0x7f193c0cfaf0 0x7f193c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 --2- 172.21.15.7:0/1136648504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9fe0 0x7f193c0cf5b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 -- 172.21.15.7:0/1136648504 >> 172.21.15.7:0/1136648504 conn(0x7f193c070af0 msgr2=0x7f193c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 -- 172.21.15.7:0/1136648504 shutdown_connections 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.874+0000 7f19411ff700 1 -- 172.21.15.7:0/1136648504 wait complete. 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.875+0000 7f19411ff700 1 Processor -- start 2023-06-09T11:22:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.876+0000 7f19411ff700 1 -- start start 2023-06-09T11:22:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.876+0000 7f19411ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9090 0x7f193c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.876+0000 7f19411ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f193c0c9fe0 0x7f193c1e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f19411ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f19411ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1e4970 con 0x7f193c0cfaf0 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f19411ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1f2980 con 0x7f193c0c9fe0 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f19411ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1e3810 con 0x7f193c0c9090 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55928/0 (socket says 172.21.15.7:55928) 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 -- 172.21.15.7:0/1665940445 learned_addr learned my addr 172.21.15.7:0/1665940445 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193ad9d700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9090 0x7f193c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193a59c700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f193c0c9fe0 0x7f193c1e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 -- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9090 msgr2=0x7f193c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9090 0x7f193c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 -- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f193c0c9fe0 msgr2=0x7f193c1e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f193c0c9fe0 0x7f193c1e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.877+0000 7f193b59e700 1 -- 172.21.15.7:0/1665940445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009a70 con 0x7f193c0cfaf0 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f193b59e700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1934009e60 tx=0x7f193400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:06.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934009cd0 con 0x7f193c0cfaf0 2023-06-09T11:22:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400f040 con 0x7f193c0cfaf0 2023-06-09T11:22:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400ccb0 con 0x7f193c0cfaf0 2023-06-09T11:22:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1930009550 con 0x7f193c0cfaf0 2023-06-09T11:22:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.878+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c0ca620 con 0x7f193c0cfaf0 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.879+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c15d720 con 0x7f193c0cfaf0 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.880+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1934014480 con 0x7f193c0cfaf0 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.881+0000 7f19237fe700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 0x7f192806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.881+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6550+0+0 (secure 0 0 0) 0x7f1934089ce0 con 0x7f193c0cfaf0 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.881+0000 7f193ad9d700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 0x7f192806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:06.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.882+0000 7f193ad9d700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 0x7f192806d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f192c007fd0 tx=0x7f192c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:06.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:06.885+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1934058590 con 0x7f193c0cfaf0 2023-06-09T11:22:07.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.153+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f193c0b53f0 con 0x7f193c0cfaf0 2023-06-09T11:22:07.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.154+0000 7f19237fe700 1 -- 172.21.15.7:0/1665940445 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f1934058120 con 0x7f193c0cfaf0 2023-06-09T11:22:07.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:07.159 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5455,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:20:57.283725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":54,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":4936},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4936":{"gid":4936,"name":"b","rank":2,"incarnation":21,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6833/1239341654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1239341654},{"type":"v1","addr":"172.21.15.117:6833","nonce":1239341654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.156+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 msgr2=0x7f192806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.156+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 0x7f192806d1c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f192c007fd0 tx=0x7f192c009040 comp rx=0 tx=0).stop 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.156+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 msgr2=0x7f193c0c4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.156+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1934009e60 tx=0x7f193400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 shutdown_connections 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f193c0cfaf0 0x7f193c0c4bb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f192806ad10 0x7f192806d1c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f193c0c9090 0x7f193c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:07.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 --2- 172.21.15.7:0/1665940445 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f193c0c9fe0 0x7f193c1e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 >> 172.21.15.7:0/1665940445 conn(0x7f193c070af0 msgr2=0x7f193c0c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 shutdown_connections 2023-06-09T11:22:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:07.157+0000 7f19411ff700 1 -- 172.21.15.7:0/1665940445 wait complete. 2023-06-09T11:22:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2023-06-09T11:22:07.181 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:22:09.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:09.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.384+0000 7fb801fff700 1 Processor -- start 2023-06-09T11:22:09.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.384+0000 7fb801fff700 1 -- start start 2023-06-09T11:22:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.385+0000 7fb801fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7fc0c5b40 0x7fb7fc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.385+0000 7fb801fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb7fc071060 0x7fb7fc158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.385+0000 7fb801fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc1580a0 con 0x7fb7fc151510 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.385+0000 7fb801fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc1518e0 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.385+0000 7fb801fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc0c5f70 con 0x7fb7fc071060 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7fc0c5b40 0x7fb7fc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7fc0c5b40 0x7fb7fc070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55930/0 (socket says 172.21.15.7:55930) 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 -- 172.21.15.7:0/3637635658 learned_addr learned my addr 172.21.15.7:0/3637635658 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:09.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7fbfff700 1 --2- 172.21.15.7:0/3637635658 >> v2:172.21.15.117:3300/0 conn(0x7fb7fc071060 0x7fb7fc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 -- 172.21.15.7:0/3637635658 >> v2:172.21.15.117:3300/0 conn(0x7fb7fc071060 msgr2=0x7fb7fc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 --2- 172.21.15.7:0/3637635658 >> v2:172.21.15.117:3300/0 conn(0x7fb7fc071060 0x7fb7fc158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 -- 172.21.15.7:0/3637635658 >> v1:172.21.15.117:6789/0 conn(0x7fb7fc151510 legacy=0x7fb7fc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 -- 172.21.15.7:0/3637635658 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc0c6180 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7faffd700 1 --2- 172.21.15.7:0/3637635658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc070b20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06ff60 tx=0x7fb7f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbb95cd69e23a93 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.386+0000 7fb7fa7fc700 1 -- 172.21.15.7:0/3637635658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0011830 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb7fa7fc700 1 -- 172.21.15.7:0/3637635658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f0011990 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb7fa7fc700 1 -- 172.21.15.7:0/3637635658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0011d20 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 -- 172.21.15.7:0/3637635658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc0c5b40 msgr2=0x7fb7fc070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 --2- 172.21.15.7:0/3637635658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc070b20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06ff60 tx=0x7fb7f0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 -- 172.21.15.7:0/3637635658 shutdown_connections 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 --2- 172.21.15.7:0/3637635658 >> v2:172.21.15.117:3300/0 conn(0x7fb7fc071060 0x7fb7fc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 --2- 172.21.15.7:0/3637635658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc070b20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 -- 172.21.15.7:0/3637635658 >> 172.21.15.7:0/3637635658 conn(0x7fb7fc06a4d0 msgr2=0x7fb7fc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 -- 172.21.15.7:0/3637635658 shutdown_connections 2023-06-09T11:22:09.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.387+0000 7fb801fff700 1 -- 172.21.15.7:0/3637635658 wait complete. 2023-06-09T11:22:09.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.389+0000 7fb801fff700 1 Processor -- start 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.389+0000 7fb801fff700 1 -- start start 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7fc151510 0x7fb7fc0c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1efaa0 con 0x7fb7fc071060 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1f0520 con 0x7fb7fc0c5b40 2023-06-09T11:22:09.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb801fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1efc80 con 0x7fb7fc151510 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55946/0 (socket says 172.21.15.7:55946) 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb7fb7fe700 1 -- 172.21.15.7:0/1345090758 learned_addr learned my addr 172.21.15.7:0/1345090758 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb7fbfff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7fc151510 0x7fb7fc0c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.390+0000 7fb7faffd700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 -- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7fc151510 msgr2=0x7fb7fc0c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7fc151510 0x7fb7fc0c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 -- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc0c5b40 msgr2=0x7fb7fc155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 -- 172.21.15.7:0/1345090758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0009550 con 0x7fb7fc071060 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7fb7fe700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c4e0 tx=0x7fb7ec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0052d0 con 0x7fb7fc071060 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec005430 con 0x7fb7fc071060 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc157770 con 0x7fb7fc071060 2023-06-09T11:22:09.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec0038c0 con 0x7fb7fc071060 2023-06-09T11:22:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.391+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc157490 con 0x7fb7fc071060 2023-06-09T11:22:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.392+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc157a50 con 0x7fb7fc071060 2023-06-09T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.393+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7ec003a20 con 0x7fb7fc071060 2023-06-09T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.394+0000 7fb7e3fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 0x7fb7e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.394+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fb7ec088a70 con 0x7fb7fc071060 2023-06-09T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.394+0000 7fb7faffd700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 0x7fb7e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.395+0000 7fb7faffd700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 0x7fb7e806d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009c10 tx=0x7fb7f0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.398+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb7ec0538d0 con 0x7fb7fc071060 2023-06-09T11:22:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.668+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7fc074cf0 con 0x7fb7fc071060 2023-06-09T11:22:09.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.669+0000 7fb7e3fff700 1 -- 172.21.15.7:0/1345090758 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5455 (secure 0 0 0) 0x7fb7ec056ef0 con 0x7fb7fc071060 2023-06-09T11:22:09.674 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:09.674 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:09.265992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 msgr2=0x7fb7e806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 0x7fb7e806d1c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009c10 tx=0x7fb7f0008030 comp rx=0 tx=0).stop 2023-06-09T11:22:09.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 msgr2=0x7fb7fc1ef560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00c4e0 tx=0x7fb7ec00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 shutdown_connections 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7e806ad10 0x7fb7e806d1c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7fc151510 0x7fb7fc0c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7fc071060 0x7fb7fc1ef560 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 --2- 172.21.15.7:0/1345090758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7fc0c5b40 0x7fb7fc155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 >> 172.21.15.7:0/1345090758 conn(0x7fb7fc06a4d0 msgr2=0x7fb7fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:09.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.671+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 shutdown_connections 2023-06-09T11:22:09.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.672+0000 7fb801fff700 1 -- 172.21.15.7:0/1345090758 wait complete. 2023-06-09T11:22:09.678 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2023-06-09T11:22:09.696 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:22:09.697 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:22:09.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:09.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.912+0000 7f22c29ff700 1 Processor -- start 2023-06-09T11:22:09.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.912+0000 7f22c29ff700 1 -- start start 2023-06-09T11:22:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.913+0000 7f22c29ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.913+0000 7f22c29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.913+0000 7f22c29ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0c0bf0 con 0x7f22bc0c1bc0 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.913+0000 7f22c29ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc070110 con 0x7f22bc15a0c0 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.913+0000 7f22c29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc071300 con 0x7f22bc0c0140 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22bb7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22bb7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:54198/0 (socket says 172.21.15.7:54198) 2023-06-09T11:22:09.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22bb7fe700 1 -- 172.21.15.7:0/2477857572 learned_addr learned my addr 172.21.15.7:0/2477857572 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:41756/0 (socket says 172.21.15.7:41756) 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 -- 172.21.15.7:0/2477857572 >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 msgr2=0x7f22bc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 --2- 172.21.15.7:0/2477857572 >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 -- 172.21.15.7:0/2477857572 >> v1:172.21.15.117:6789/0 conn(0x7f22bc0c1bc0 legacy=0x7f22bc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 -- 172.21.15.7:0/2477857572 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0702f0 con 0x7f22bc0c0140 2023-06-09T11:22:09.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/2477857572 >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:22:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.914+0000 7f22c0f9c700 1 --2- 172.21.15.7:0/2477857572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009d90 tx=0x7f22b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2190716f1dcd3cd0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22baffd700 1 -- 172.21.15.7:0/2477857572 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b400bd20 con 0x7f22bc0c0140 2023-06-09T11:22:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22baffd700 1 -- 172.21.15.7:0/2477857572 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b400be80 con 0x7f22bc0c0140 2023-06-09T11:22:09.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22baffd700 1 -- 172.21.15.7:0/2477857572 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40116a0 con 0x7f22bc0c0140 2023-06-09T11:22:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 -- 172.21.15.7:0/2477857572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 msgr2=0x7f22bc1506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 --2- 172.21.15.7:0/2477857572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009d90 tx=0x7f22b400b330 comp rx=0 tx=0).stop 2023-06-09T11:22:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 -- 172.21.15.7:0/2477857572 shutdown_connections 2023-06-09T11:22:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 --2- 172.21.15.7:0/2477857572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c0140 0x7f22bc1506f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 --2- 172.21.15.7:0/2477857572 >> v2:172.21.15.117:3300/0 conn(0x7f22bc15a0c0 0x7f22bc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 -- 172.21.15.7:0/2477857572 >> 172.21.15.7:0/2477857572 conn(0x7f22bc06a4d0 msgr2=0x7f22bc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 -- 172.21.15.7:0/2477857572 shutdown_connections 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.915+0000 7f22c29ff700 1 -- 172.21.15.7:0/2477857572 wait complete. 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.917+0000 7f22c29ff700 1 Processor -- start 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.917+0000 7f22c29ff700 1 -- start start 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c1bc0 0x7f22bc14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22bc15a0c0 0x7f22bc14d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c3a60 con 0x7f22bc0c0140 2023-06-09T11:22:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c3d40 con 0x7f22bc15a0c0 2023-06-09T11:22:09.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22c29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0c4020 con 0x7f22bc0c1bc0 2023-06-09T11:22:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.918+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55954/0 (socket says 172.21.15.7:55954) 2023-06-09T11:22:09.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 -- 172.21.15.7:0/1525535692 learned_addr learned my addr 172.21.15.7:0/1525535692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c1bc0 0x7f22bc14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22c0f9c700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22bc15a0c0 0x7f22bc14d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 -- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c1bc0 msgr2=0x7f22bc14cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c1bc0 0x7f22bc14cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 -- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22bc15a0c0 msgr2=0x7f22bc14d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22bc15a0c0 0x7f22bc14d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:09.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 -- 172.21.15.7:0/1525535692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0009550 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22bbfff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c4e0 tx=0x7f22ac00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.919+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0052d0 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.920+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac005430 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.920+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0038c0 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.920+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b4009a70 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.920+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc0c1170 con 0x7f22bc0c0140 2023-06-09T11:22:09.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.921+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22bc0c0e90 con 0x7f22bc0c0140 2023-06-09T11:22:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.922+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22ac003a20 con 0x7f22bc0c0140 2023-06-09T11:22:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.922+0000 7f22b8ff9700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 0x7f22a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.923+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f22ac088b10 con 0x7f22bc0c0140 2023-06-09T11:22:09.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.923+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 0x7f22a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:09.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.923+0000 7f22bb7fe700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 0x7f22a806d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f22bc159ba0 tx=0x7f22b0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:09.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:09.927+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f22ac053a10 con 0x7f22bc0c0140 2023-06-09T11:22:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.181+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22bc0c1450 con 0x7f22bc0c0140 2023-06-09T11:22:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.182+0000 7f22b8ff9700 1 -- 172.21.15.7:0/1525535692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f22ac057030 con 0x7f22bc0c0140 2023-06-09T11:22:10.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:10.187 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:09.265992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 msgr2=0x7f22a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 0x7f22a806d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f22bc159ba0 tx=0x7f22b0008040 comp rx=0 tx=0).stop 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 msgr2=0x7f22bc1ef710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00c4e0 tx=0x7f22ac00c8a0 comp rx=0 tx=0).stop 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 shutdown_connections 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22bc15a0c0 0x7f22bc14d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22bc0c0140 0x7f22bc1ef710 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22a806ace0 0x7f22a806d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 --2- 172.21.15.7:0/1525535692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22bc0c1bc0 0x7f22bc14cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:10.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 >> 172.21.15.7:0/1525535692 conn(0x7f22bc06a4d0 msgr2=0x7f22bc0c6880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:10.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.184+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 shutdown_connections 2023-06-09T11:22:10.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:10.185+0000 7f22c29ff700 1 -- 172.21.15.7:0/1525535692 wait complete. 2023-06-09T11:22:10.191 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2023-06-09T11:22:10.207 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 55, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 55, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5012": { "addr": "172.21.15.117:6837/3174501100", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3174501100, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3174501100, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5012, "incarnation": 46, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 }, "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:22:09.265992+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5012, "mds_1": 5071, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:22:12.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.412+0000 7f74aa1ff700 1 Processor -- start 2023-06-09T11:22:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.413+0000 7f74aa1ff700 1 -- start start 2023-06-09T11:22:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f74aa1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74a406d640 0x7f74a40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f74aa1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a415cac0 con 0x7f74a406a4d0 2023-06-09T11:22:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f74aa1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a40c37d0 con 0x7f74a40c6280 2023-06-09T11:22:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f74aa1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a40c8870 con 0x7f74a406d640 2023-06-09T11:22:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f74a3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74a406d640 0x7f74a40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f749bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f74a406a4d0 0x7f74a406cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56022/0 (socket says 172.21.15.7:56022) 2023-06-09T11:22:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.414+0000 7f749bfff700 1 -- 172.21.15.7:0/3556508043 learned_addr learned my addr 172.21.15.7:0/3556508043 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a3fff700 1 -- 172.21.15.7:0/3556508043 >> v1:172.21.15.117:6789/0 conn(0x7f74a40c6280 legacy=0x7f74a40737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a3fff700 1 -- 172.21.15.7:0/3556508043 >> v1:172.21.15.7:6789/0 conn(0x7f74a406a4d0 legacy=0x7f74a406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a3fff700 1 -- 172.21.15.7:0/3556508043 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a415cd30 con 0x7f74a406d640 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a2ffd700 1 -- 172.21.15.7:0/3556508043 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772497024 0 0) 0x7f74a415cac0 con 0x7f74a406a4d0 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a3fff700 1 --2- 172.21.15.7:0/3556508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a406d640 0x7f74a40c76b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f749c009d90 tx=0x7f749c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485b2e1b92f4b744 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a2ffd700 1 -- 172.21.15.7:0/3556508043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c00e070 con 0x7f74a406d640 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a2ffd700 1 -- 172.21.15.7:0/3556508043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c00bbd0 con 0x7f74a406d640 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.415+0000 7f74a2ffd700 1 -- 172.21.15.7:0/3556508043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c012430 con 0x7f74a406d640 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3556508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a406d640 msgr2=0x7f74a40c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3556508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a406d640 0x7f74a40c76b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f749c009d90 tx=0x7f749c00b330 comp rx=0 tx=0).stop 2023-06-09T11:22:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3556508043 shutdown_connections 2023-06-09T11:22:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3556508043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a406d640 0x7f74a40c76b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3556508043 >> 172.21.15.7:0/3556508043 conn(0x7f74a406f980 msgr2=0x7f74a40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3556508043 shutdown_connections 2023-06-09T11:22:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.416+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3556508043 wait complete. 2023-06-09T11:22:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.418+0000 7f74aa1ff700 1 Processor -- start 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.418+0000 7f74aa1ff700 1 -- start start 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 0x7f74a4141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 0x7f74a4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4142020 con 0x7f74a40c6280 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a41f0590 con 0x7f74a406d640 2023-06-09T11:22:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74aa1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a41553b0 con 0x7f74a406a4d0 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55972/0 (socket says 172.21.15.7:55972) 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74a3fff700 1 -- 172.21.15.7:0/3549743705 learned_addr learned my addr 172.21.15.7:0/3549743705 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74a37fe700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 0x7f74a4141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f749bfff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 0x7f74a4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.419+0000 7f74a3fff700 1 -- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 msgr2=0x7f74a4141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a3fff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 0x7f74a4141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a3fff700 1 -- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 msgr2=0x7f74a4159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a3fff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 0x7f74a4159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a3fff700 1 -- 172.21.15.7:0/3549743705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c009a70 con 0x7f74a40c6280 2023-06-09T11:22:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a37fe700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 0x7f74a4141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f749bfff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 0x7f74a4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a3fff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f749c014040 tx=0x7f749c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c012430 con 0x7f74a40c6280 2023-06-09T11:22:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c020070 con 0x7f74a40c6280 2023-06-09T11:22:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a415d2f0 con 0x7f74a40c6280 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c004280 con 0x7f74a40c6280 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.420+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a415d010 con 0x7f74a40c6280 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.421+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a415d5d0 con 0x7f74a40c6280 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.422+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f749c0043e0 con 0x7f74a40c6280 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.423+0000 7f74a0ff9700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 0x7f748406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.423+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f749c022070 con 0x7f74a40c6280 2023-06-09T11:22:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.423+0000 7f74a37fe700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 0x7f748406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.424+0000 7f74a37fe700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 0x7f748406d1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7494000f60 tx=0x7f7494009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.427+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f749c0567f0 con 0x7f74a40c6280 2023-06-09T11:22:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.684+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74a4156510 con 0x7f74a40c6280 2023-06-09T11:22:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.684+0000 7f74a0ff9700 1 -- 172.21.15.7:0/3549743705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f749c0263e0 con 0x7f74a40c6280 2023-06-09T11:22:12.689 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:12.689 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:09.265992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 msgr2=0x7f748406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 0x7f748406d1c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7494000f60 tx=0x7f7494009040 comp rx=0 tx=0).stop 2023-06-09T11:22:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 msgr2=0x7f74a41ef850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f749c014040 tx=0x7f749c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 shutdown_connections 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74a40c6280 0x7f74a41ef850 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f748406ad10 0x7f748406d1c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74a406a4d0 0x7f74a4141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 --2- 172.21.15.7:0/3549743705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f74a406d640 0x7f74a4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 >> 172.21.15.7:0/3549743705 conn(0x7f74a406f980 msgr2=0x7f74a40737f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 shutdown_connections 2023-06-09T11:22:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:12.687+0000 7f74aa1ff700 1 -- 172.21.15.7:0/3549743705 wait complete. 2023-06-09T11:22:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2023-06-09T11:22:12.709 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:13.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:13.245 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:14.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:14.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.926+0000 7f14859ff700 1 Processor -- start 2023-06-09T11:22:14.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.926+0000 7f14859ff700 1 -- start start 2023-06-09T11:22:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f14859ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14800c5b40 0x7f1480070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f14859ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1480071060 0x7f1480158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f14859ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14801580a0 con 0x7f1480151510 2023-06-09T11:22:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f14859ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14800714e0 con 0x7f1480071060 2023-06-09T11:22:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f14859ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14801518e0 con 0x7f14800c5b40 2023-06-09T11:22:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1480151510 0x7f148015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36440/0 (socket says 172.21.15.7:36440) 2023-06-09T11:22:14.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147effd700 1 -- 172.21.15.7:0/3447671772 learned_addr learned my addr 172.21.15.7:0/3447671772 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147f7fe700 1 --2- 172.21.15.7:0/3447671772 >> v2:172.21.15.7:3300/0 conn(0x7f1480071060 0x7f1480158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147e7fc700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14800c5b40 0x7f1480070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147dffb700 1 -- 172.21.15.7:0/3447671772 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325137513 0 0) 0x7f14801580a0 con 0x7f1480151510 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147dffb700 1 -- 172.21.15.7:0/3447671772 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14680035b0 con 0x7f1480151510 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147f7fe700 1 -- 172.21.15.7:0/3447671772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14800c5b40 msgr2=0x7f1480070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147f7fe700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14800c5b40 0x7f1480070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:14.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147f7fe700 1 -- 172.21.15.7:0/3447671772 >> v1:172.21.15.7:6789/0 conn(0x7f1480151510 legacy=0x7f148015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.928+0000 7f147f7fe700 1 -- 172.21.15.7:0/3447671772 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800c6180 con 0x7f1480071060 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f147f7fe700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1480071060 0x7f1480158870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1478009dc0 tx=0x7f147800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaedad00083ec51d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f147dffb700 1 -- 172.21.15.7:0/3447671772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800e070 con 0x7f1480071060 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f147dffb700 1 -- 172.21.15.7:0/3447671772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147800bae0 con 0x7f1480071060 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f147dffb700 1 -- 172.21.15.7:0/3447671772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147800be90 con 0x7f1480071060 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f14859ff700 1 -- 172.21.15.7:0/3447671772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1480071060 msgr2=0x7f1480158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:14.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.929+0000 7f14859ff700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1480071060 0x7f1480158870 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1478009dc0 tx=0x7f147800b330 comp rx=0 tx=0).stop 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 -- 172.21.15.7:0/3447671772 shutdown_connections 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1480071060 0x7f1480158870 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 --2- 172.21.15.7:0/3447671772 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14800c5b40 0x7f1480070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 -- 172.21.15.7:0/3447671772 >> 172.21.15.7:0/3447671772 conn(0x7f148006a4d0 msgr2=0x7f148006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 -- 172.21.15.7:0/3447671772 shutdown_connections 2023-06-09T11:22:14.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.930+0000 7f14859ff700 1 -- 172.21.15.7:0/3447671772 wait complete. 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.931+0000 7f14859ff700 1 Processor -- start 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.932+0000 7f14859ff700 1 -- start start 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.932+0000 7f14859ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1480071060 0x7f14800c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.932+0000 7f14859ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f14859ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1480151510 0x7f1480145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f14859ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14800c11b0 con 0x7f14800c5b40 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f14859ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480159560 con 0x7f1480071060 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f14859ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148014d0a0 con 0x7f1480151510 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44026/0 (socket says 172.21.15.7:44026) 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 -- 172.21.15.7:0/3325867884 learned_addr learned my addr 172.21.15.7:0/3325867884 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147f7fe700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1480151510 0x7f1480145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:14.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 -- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1480151510 msgr2=0x7f1480145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1480151510 0x7f1480145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 -- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1480071060 msgr2=0x7f14800c0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1480071060 0x7f14800c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 -- 172.21.15.7:0/3325867884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1478009a70 con 0x7f14800c5b40 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.933+0000 7f147e7fc700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9b0 tx=0x7f147400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.934+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474009710 con 0x7f14800c5b40 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.934+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147400c850 con 0x7f14800c5b40 2023-06-09T11:22:14.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.934+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1480157770 con 0x7f14800c5b40 2023-06-09T11:22:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.934+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147400cbc0 con 0x7f14800c5b40 2023-06-09T11:22:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.934+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1480157490 con 0x7f14800c5b40 2023-06-09T11:22:14.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.935+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1480157a50 con 0x7f14800c5b40 2023-06-09T11:22:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.936+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f147400cd20 con 0x7f14800c5b40 2023-06-09T11:22:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.937+0000 7f1467fff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 0x7f146c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.937+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f1474089bf0 con 0x7f14800c5b40 2023-06-09T11:22:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.937+0000 7f147effd700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 0x7f146c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:14.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.937+0000 7f147effd700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 0x7f146c06d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1470007fd0 tx=0x7f1470007980 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:14.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:14.941+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14740549c0 con 0x7f14800c5b40 2023-06-09T11:22:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.190+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1480074cf0 con 0x7f14800c5b40 2023-06-09T11:22:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.191+0000 7f1467fff700 1 -- 172.21.15.7:0/3325867884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f1474057fe0 con 0x7f14800c5b40 2023-06-09T11:22:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:15.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":55,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:09.265992+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.193+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 msgr2=0x7f146c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.193+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 0x7f146c06d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1470007fd0 tx=0x7f1470007980 comp rx=0 tx=0).stop 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.193+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 msgr2=0x7f14800c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.193+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9b0 tx=0x7f147400ed70 comp rx=0 tx=0).stop 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 shutdown_connections 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f146c06ace0 0x7f146c06d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1480151510 0x7f1480145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1480071060 0x7f14800c0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 --2- 172.21.15.7:0/3325867884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14800c5b40 0x7f14800c1da0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 >> 172.21.15.7:0/3325867884 conn(0x7f148006a4d0 msgr2=0x7f148014ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 shutdown_connections 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:15.194+0000 7f14859ff700 1 -- 172.21.15.7:0/3325867884 wait complete. 2023-06-09T11:22:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 55 2023-06-09T11:22:15.217 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:17.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.417+0000 7f4559dff700 1 Processor -- start 2023-06-09T11:22:17.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.417+0000 7f4559dff700 1 -- start start 2023-06-09T11:22:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.418+0000 7f4559dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455415b4a0 0x7f455415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.418+0000 7f4559dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45540c9f40 0x7f45540c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.418+0000 7f4559dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455415a500 con 0x7f4554153880 2023-06-09T11:22:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4559dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45540c5f10 con 0x7f45540c9f40 2023-06-09T11:22:17.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4559dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455415da00 con 0x7f455415b4a0 2023-06-09T11:22:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45540c9f40 0x7f45540c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455415b4a0 0x7f455415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45540c9f40 0x7f45540c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:60310/0 (socket says 172.21.15.7:60310) 2023-06-09T11:22:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 -- 172.21.15.7:0/1276981074 learned_addr learned my addr 172.21.15.7:0/1276981074 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 -- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455415b4a0 msgr2=0x7f455415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:17.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 --2- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455415b4a0 0x7f455415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 -- 172.21.15.7:0/1276981074 >> v1:172.21.15.117:6789/0 conn(0x7f4554153880 legacy=0x7f455415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.419+0000 7f4553fff700 1 -- 172.21.15.7:0/1276981074 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45541596e0 con 0x7f45540c9f40 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f4553fff700 1 --2- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45540c9f40 0x7f45540c4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f454c009e50 tx=0x7f454c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db36a00951b29098 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f45527fc700 1 -- 172.21.15.7:0/1276981074 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c00e070 con 0x7f45540c9f40 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f45527fc700 1 -- 172.21.15.7:0/1276981074 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c00b910 con 0x7f45540c9f40 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f45527fc700 1 -- 172.21.15.7:0/1276981074 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c00bcc0 con 0x7f45540c9f40 2023-06-09T11:22:17.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f4559dff700 1 -- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45540c9f40 msgr2=0x7f45540c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f4559dff700 1 --2- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45540c9f40 0x7f45540c4e30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f454c009e50 tx=0x7f454c00b330 comp rx=0 tx=0).stop 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f4559dff700 1 -- 172.21.15.7:0/1276981074 shutdown_connections 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.420+0000 7f4559dff700 1 --2- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45540c9f40 0x7f45540c4e30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.421+0000 7f4559dff700 1 --2- 172.21.15.7:0/1276981074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455415b4a0 0x7f455415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.421+0000 7f4559dff700 1 -- 172.21.15.7:0/1276981074 >> 172.21.15.7:0/1276981074 conn(0x7f45540737f0 msgr2=0x7f45540720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.421+0000 7f4559dff700 1 -- 172.21.15.7:0/1276981074 shutdown_connections 2023-06-09T11:22:17.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.421+0000 7f4559dff700 1 -- 172.21.15.7:0/1276981074 wait complete. 2023-06-09T11:22:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.422+0000 7f4559dff700 1 Processor -- start 2023-06-09T11:22:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.422+0000 7f4559dff700 1 -- start start 2023-06-09T11:22:17.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4554153880 0x7f45540cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455415b4a0 0x7f45540b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540b31c0 con 0x7f45540c9f40 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45541582c0 con 0x7f455415b4a0 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.423+0000 7f4559dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4554158ba0 con 0x7f4554153880 2023-06-09T11:22:17.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f4552ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4554153880 0x7f45540cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44032/0 (socket says 172.21.15.7:44032) 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 -- 172.21.15.7:0/741996248 learned_addr learned my addr 172.21.15.7:0/741996248 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f4553fff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455415b4a0 0x7f45540b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 -- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4554153880 msgr2=0x7f45540cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4554153880 0x7f45540cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 -- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455415b4a0 msgr2=0x7f45540b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455415b4a0 0x7f45540b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 -- 172.21.15.7:0/741996248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c009b30 con 0x7f45540c9f40 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.424+0000 7f45537fe700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f454400c510 tx=0x7f454400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.425+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440052d0 con 0x7f45540c9f40 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.425+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544005430 con 0x7f45540c9f40 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.425+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4554159c60 con 0x7f45540c9f40 2023-06-09T11:22:17.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.425+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440038c0 con 0x7f45540c9f40 2023-06-09T11:22:17.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.425+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4554159980 con 0x7f45540c9f40 2023-06-09T11:22:17.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.426+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4554159f40 con 0x7f45540c9f40 2023-06-09T11:22:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.427+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4544003a20 con 0x7f45540c9f40 2023-06-09T11:22:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.428+0000 7f453bfff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 0x7f454006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.428+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f45440888b0 con 0x7f45540c9f40 2023-06-09T11:22:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.428+0000 7f4552ffd700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 0x7f454006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:17.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.429+0000 7f4552ffd700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 0x7f454006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45540b1f00 tx=0x7f4548009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:17.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.432+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4544053680 con 0x7f45540c9f40 2023-06-09T11:22:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.721+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f455415a500 con 0x7f45540c9f40 2023-06-09T11:22:17.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.722+0000 7f453bfff700 1 -- 172.21.15.7:0/741996248 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f455415a500 con 0x7f45540c9f40 2023-06-09T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:17.728 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:15.269597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:resolve","state_seq":29,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 msgr2=0x7f454006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 0x7f454006d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45540b1f00 tx=0x7f4548009210 comp rx=0 tx=0).stop 2023-06-09T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 msgr2=0x7f4554158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:17.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f454400c510 tx=0x7f454400c820 comp rx=0 tx=0).stop 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 shutdown_connections 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455415b4a0 0x7f45540b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45540c9f40 0x7f4554158e60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f454006ace0 0x7f454006d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 --2- 172.21.15.7:0/741996248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4554153880 0x7f45540cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 >> 172.21.15.7:0/741996248 conn(0x7f45540737f0 msgr2=0x7f455414d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:17.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 shutdown_connections 2023-06-09T11:22:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:17.724+0000 7f4559dff700 1 -- 172.21.15.7:0/741996248 wait complete. 2023-06-09T11:22:17.732 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 56 2023-06-09T11:22:17.749 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:19.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:19.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.945+0000 7f3eb6bff700 1 Processor -- start 2023-06-09T11:22:19.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.945+0000 7f3eb6bff700 1 -- start start 2023-06-09T11:22:19.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb6bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3eb00c1bc0 0x7f3eb01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:19.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb6bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00c0bf0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb6bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00702f0 con 0x7f3eb00c0140 2023-06-09T11:22:19.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb6bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb0070110 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb499b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3eb00c1bc0 0x7f3eb01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:19.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb519c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3eb00c1f90 0x7f3eb0160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36454/0 (socket says 172.21.15.7:36454) 2023-06-09T11:22:19.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.946+0000 7f3eb519c700 1 -- 172.21.15.7:0/1356667960 learned_addr learned my addr 172.21.15.7:0/1356667960 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:19.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eb499b700 1 -- 172.21.15.7:0/1356667960 >> v1:172.21.15.117:6789/0 conn(0x7f3eb00c0140 legacy=0x7f3eb01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eb499b700 1 -- 172.21.15.7:0/1356667960 >> v1:172.21.15.7:6789/0 conn(0x7f3eb00c1f90 legacy=0x7f3eb0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eb499b700 1 -- 172.21.15.7:0/1356667960 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00c0e90 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eaf7fe700 1 -- 172.21.15.7:0/1356667960 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3311470470 0 0) 0x7f3eb00c0bf0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eb499b700 1 --2- 172.21.15.7:0/1356667960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01599b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009840 tx=0x7f3ea4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708736f938390aa5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eaf7fe700 1 -- 172.21.15.7:0/1356667960 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011830 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eaf7fe700 1 -- 172.21.15.7:0/1356667960 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea4011990 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.947+0000 7f3eaf7fe700 1 -- 172.21.15.7:0/1356667960 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4011d20 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 -- 172.21.15.7:0/1356667960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1bc0 msgr2=0x7f3eb01599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:19.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/1356667960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01599b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009840 tx=0x7f3ea4009b50 comp rx=0 tx=0).stop 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 -- 172.21.15.7:0/1356667960 shutdown_connections 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/1356667960 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01599b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 -- 172.21.15.7:0/1356667960 >> 172.21.15.7:0/1356667960 conn(0x7f3eb006a4d0 msgr2=0x7f3eb006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 -- 172.21.15.7:0/1356667960 shutdown_connections 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.948+0000 7f3eb6bff700 1 -- 172.21.15.7:0/1356667960 wait complete. 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.950+0000 7f3eb6bff700 1 Processor -- start 2023-06-09T11:22:19.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.950+0000 7f3eb6bff700 1 -- start start 2023-06-09T11:22:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eb00c0140 0x7f3eb014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00aef80 con 0x7f3eb00c1f90 2023-06-09T11:22:19.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb01ef530 con 0x7f3eb00c1bc0 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb6bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb01eef00 con 0x7f3eb00c0140 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44044/0 (socket says 172.21.15.7:44044) 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb519c700 1 -- 172.21.15.7:0/589191528 learned_addr learned my addr 172.21.15.7:0/589191528 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eaffff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.951+0000 7f3eb499b700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eb00c0140 0x7f3eb014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:19.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 -- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eb00c0140 msgr2=0x7f3eb014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eb00c0140 0x7f3eb014dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 -- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb00c1bc0 msgr2=0x7f3eb01ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 -- 172.21.15.7:0/589191528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea40094a0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb519c700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800cba0 tx=0x7f3ea800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800aaa0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea800ac20 con 0x7f3eb00c1f90 2023-06-09T11:22:19.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea80070d0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c1450 con 0x7f3eb00c1f90 2023-06-09T11:22:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.952+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb00c1170 con 0x7f3eb00c1f90 2023-06-09T11:22:19.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.953+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb00705d0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.954+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ea8007230 con 0x7f3eb00c1f90 2023-06-09T11:22:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.956+0000 7f3ead7fa700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 0x7f3e9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.956+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f3ea80891a0 con 0x7f3eb00c1f90 2023-06-09T11:22:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.956+0000 7f3eb499b700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 0x7f3e9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:19.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.956+0000 7f3eb499b700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 0x7f3e9406d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40090b0 tx=0x7f3ea401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:19.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:19.960+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3ea8053f80 con 0x7f3eb00c1f90 2023-06-09T11:22:20.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.212+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eb01ef6a0 con 0x7f3eb00c1f90 2023-06-09T11:22:20.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.213+0000 7f3ead7fa700 1 -- 172.21.15.7:0/589191528 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f3ea8018020 con 0x7f3eb00c1f90 2023-06-09T11:22:20.320 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:20.320 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:15.269597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:resolve","state_seq":29,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 msgr2=0x7f3e9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:20.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 0x7f3e9406d190 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40090b0 tx=0x7f3ea401c040 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 msgr2=0x7f3eb00adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800cba0 tx=0x7f3ea800ceb0 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 shutdown_connections 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00c1f90 0x7f3eb00adb00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3e9406ace0 0x7f3e9406d190 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eb00c0140 0x7f3eb014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 --2- 172.21.15.7:0/589191528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eb00c1bc0 0x7f3eb01ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:20.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.215+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 >> 172.21.15.7:0/589191528 conn(0x7f3eb006a4d0 msgr2=0x7f3eb00c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:20.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.216+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 shutdown_connections 2023-06-09T11:22:20.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:20.216+0000 7f3eb6bff700 1 -- 172.21.15.7:0/589191528 wait complete. 2023-06-09T11:22:20.323 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 56 2023-06-09T11:22:20.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:22.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:22.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.544+0000 7f9aa63ff700 1 Processor -- start 2023-06-09T11:22:22.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.545+0000 7f9aa63ff700 1 -- start start 2023-06-09T11:22:22.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.546+0000 7f9aa63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 0x7f9aa00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9aa63ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa015a390 con 0x7f9aa0071fa0 2023-06-09T11:22:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9aa63ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa00ca720 con 0x7f9aa0153b30 2023-06-09T11:22:22.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9aa63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00c5f40 con 0x7f9aa0153760 2023-06-09T11:22:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9aa499c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9aa0071fa0 0x7f9aa015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36456/0 (socket says 172.21.15.7:36456) 2023-06-09T11:22:22.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9aa499c700 1 -- 172.21.15.7:0/923870196 learned_addr learned my addr 172.21.15.7:0/923870196 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9a9ffff700 1 --2- 172.21.15.7:0/923870196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 0x7f9aa00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025506806 0 0) 0x7f9aa015a390 con 0x7f9aa0071fa0 2023-06-09T11:22:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a880035b0 con 0x7f9aa0071fa0 2023-06-09T11:22:22.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.547+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141319269 0 0) 0x7f9aa00ca720 con 0x7f9aa0153b30 2023-06-09T11:22:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa015a390 con 0x7f9aa0153b30 2023-06-09T11:22:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9ffff700 1 -- 172.21.15.7:0/923870196 >> v1:172.21.15.117:6789/0 conn(0x7f9aa0153b30 legacy=0x7f9aa00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9ffff700 1 -- 172.21.15.7:0/923870196 >> v1:172.21.15.7:6789/0 conn(0x7f9aa0071fa0 legacy=0x7f9aa015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9ffff700 1 -- 172.21.15.7:0/923870196 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00c5a20 con 0x7f9aa0153760 2023-06-09T11:22:22.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9ffff700 1 --2- 172.21.15.7:0/923870196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 0x7f9aa00ca070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009870 tx=0x7f9a90009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ed998de0431bc7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:22.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011830 con 0x7f9aa0153760 2023-06-09T11:22:22.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a90011990 con 0x7f9aa0153760 2023-06-09T11:22:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.548+0000 7f9a9effd700 1 -- 172.21.15.7:0/923870196 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a90011d20 con 0x7f9aa0153760 2023-06-09T11:22:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 -- 172.21.15.7:0/923870196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 msgr2=0x7f9aa00ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/923870196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 0x7f9aa00ca070 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9a90009870 tx=0x7f9a90009b80 comp rx=0 tx=0).stop 2023-06-09T11:22:22.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 -- 172.21.15.7:0/923870196 shutdown_connections 2023-06-09T11:22:22.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/923870196 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153760 0x7f9aa00ca070 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 -- 172.21.15.7:0/923870196 >> 172.21.15.7:0/923870196 conn(0x7f9aa00737f0 msgr2=0x7f9aa0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:22.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 -- 172.21.15.7:0/923870196 shutdown_connections 2023-06-09T11:22:22.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.549+0000 7f9aa63ff700 1 -- 172.21.15.7:0/923870196 wait complete. 2023-06-09T11:22:22.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.551+0000 7f9aa63ff700 1 Processor -- start 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.552+0000 7f9aa63ff700 1 -- start start 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0153760 0x7f9aa015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 0x7f9aa0150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa01f2340 con 0x7f9aa0071fa0 2023-06-09T11:22:22.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00cfdc0 con 0x7f9aa0153760 2023-06-09T11:22:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00d00a0 con 0x7f9aa0153b30 2023-06-09T11:22:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9a9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9a9ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44046/0 (socket says 172.21.15.7:44046) 2023-06-09T11:22:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9a9ffff700 1 -- 172.21.15.7:0/1821393705 learned_addr learned my addr 172.21.15.7:0/1821393705 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:22.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9a9f7fe700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0153760 0x7f9aa015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.553+0000 7f9aa499c700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 0x7f9aa0150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 -- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 msgr2=0x7f9aa0150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 0x7f9aa0150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 -- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0153760 msgr2=0x7f9aa015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0153760 0x7f9aa015cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 -- 172.21.15.7:0/1821393705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900094a0 con 0x7f9aa0071fa0 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9aa499c700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 0x7f9aa0150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9ffff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a900090b0 tx=0x7f9a90004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9001a760 con 0x7f9aa0071fa0 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9001a8c0 con 0x7f9aa0071fa0 2023-06-09T11:22:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9001ac30 con 0x7f9aa0071fa0 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.554+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa0159a60 con 0x7f9aa0071fa0 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.555+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa0159780 con 0x7f9aa0071fa0 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.556+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa0159d40 con 0x7f9aa0071fa0 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.556+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a90022070 con 0x7f9aa0071fa0 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.557+0000 7f9a9cff9700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 0x7f9a8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:22.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.557+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f9a9008a750 con 0x7f9aa0071fa0 2023-06-09T11:22:22.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.558+0000 7f9a9f7fe700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 0x7f9a8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:22.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.558+0000 7f9a9f7fe700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 0x7f9a8c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0157220 tx=0x7f9a94009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:22.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.562+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9a90055540 con 0x7f9aa0071fa0 2023-06-09T11:22:22.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.862+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9aa0074cf0 con 0x7f9aa0071fa0 2023-06-09T11:22:22.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.863+0000 7f9a9cff9700 1 -- 172.21.15.7:0/1821393705 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f9a90015090 con 0x7f9aa0071fa0 2023-06-09T11:22:22.869 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:22.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:21.282520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.865+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 msgr2=0x7f9a8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.865+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 0x7f9a8c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0157220 tx=0x7f9a94009040 comp rx=0 tx=0).stop 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.865+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 msgr2=0x7f9aa01f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.865+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a900090b0 tx=0x7f9a90004750 comp rx=0 tx=0).stop 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 shutdown_connections 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a8c06ace0 0x7f9a8c06d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa0153b30 0x7f9aa0150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa0071fa0 0x7f9aa01f1e00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 --2- 172.21.15.7:0/1821393705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa0153760 0x7f9aa015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 >> 172.21.15.7:0/1821393705 conn(0x7f9aa00737f0 msgr2=0x7f9aa014e110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 shutdown_connections 2023-06-09T11:22:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:22.866+0000 7f9aa63ff700 1 -- 172.21.15.7:0/1821393705 wait complete. 2023-06-09T11:22:22.873 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2023-06-09T11:22:22.892 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 57, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 57, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5012": { "addr": "172.21.15.117:6837/3174501100", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3174501100, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3174501100, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5012, "incarnation": 46, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 42 }, "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 }, "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:reconnect", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 55, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:22:21.282520+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5012, "mds_1": 5071, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:22:24.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.101+0000 7f8e41bff700 1 Processor -- start 2023-06-09T11:22:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.101+0000 7f8e41bff700 1 -- start start 2023-06-09T11:22:25.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.102+0000 7f8e41bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e3c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.102+0000 7f8e41bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c159b20 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e41bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0cac80 con 0x7f8e3c15cd40 2023-06-09T11:22:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e41bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1597d0 con 0x7f8e3c0cb9a0 2023-06-09T11:22:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e3c0cb5d0 0x7f8e3c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42826/0 (socket says 172.21.15.7:42826) 2023-06-09T11:22:25.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/2087223483 learned_addr learned my addr 172.21.15.7:0/2087223483 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:25.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3affd700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e3c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400939187 0 0) 0x7f8e3c159b20 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e240035b0 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1291436027 0 0) 0x7f8e240035b0 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 msgr2=0x7f8e3c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e3c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 >> v1:172.21.15.117:6789/0 conn(0x7f8e3c15cd40 legacy=0x7f8e3c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c071ab0 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f8e2c001670 con 0x7f8e3c0cb5d0 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.103+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 >> v1:172.21.15.7:6789/0 conn(0x7f8e3c0cb5d0 legacy=0x7f8e3c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e24003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e24004d90 0x7f8e24005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1597d0 con 0x7f8e24003a30 2023-06-09T11:22:25.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2400bb50 con 0x7f8e24004d90 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2400bce0 con 0x7f8e3c0cb9a0 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e24003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3affd700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3bfff700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e24004d90 0x7f8e24005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 msgr2=0x7f8e24004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0cb9a0 0x7f8e24004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e24004d90 msgr2=0x7f8e24005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:25.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e24004d90 0x7f8e24005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/2087223483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240035b0 con 0x7f8e24003a30 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.104+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e24003e40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c2a0 tx=0x7f8e24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90aeacbb3516008 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c007c10 con 0x7f8e24003a30 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00cde0 con 0x7f8e24003a30 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e3a7fc700 1 -- 172.21.15.7:0/2087223483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c005ba0 con 0x7f8e24003a30 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 -- 172.21.15.7:0/2087223483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 msgr2=0x7f8e24003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 --2- 172.21.15.7:0/2087223483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e24003e40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c2a0 tx=0x7f8e24005880 comp rx=0 tx=0).stop 2023-06-09T11:22:25.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e3bfff700 1 -- 172.21.15.7:0/2087223483 reap_dead start 2023-06-09T11:22:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 -- 172.21.15.7:0/2087223483 shutdown_connections 2023-06-09T11:22:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 -- 172.21.15.7:0/2087223483 >> 172.21.15.7:0/2087223483 conn(0x7f8e3c070af0 msgr2=0x7f8e3c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 -- 172.21.15.7:0/2087223483 shutdown_connections 2023-06-09T11:22:25.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.105+0000 7f8e41bff700 1 -- 172.21.15.7:0/2087223483 wait complete. 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.108+0000 7f8e41bff700 1 Processor -- start 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.108+0000 7f8e41bff700 1 -- start start 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0c7430 0x7f8e3c1f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3c1e56c0 0x7f8e3c1e6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1e52a0 con 0x7f8e24003a30 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c1e4c90 con 0x7f8e3c1e56c0 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e41bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0c6940 con 0x7f8e3c0c7430 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.109+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46058/0 (socket says 172.21.15.7:46058) 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/1785895783 learned_addr learned my addr 172.21.15.7:0/1785895783 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3affd700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0c7430 0x7f8e3c1f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3bfff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3c1e56c0 0x7f8e3c1e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0c7430 msgr2=0x7f8e3c1f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0c7430 0x7f8e3c1f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3c1e56c0 msgr2=0x7f8e3c1e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3c1e56c0 0x7f8e3c1e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 -- 172.21.15.7:0/1785895783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c15d4f0 con 0x7f8e24003a30 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e3b7fe700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8e240047a0 tx=0x7f8e3c0b51d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.110+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c001630 con 0x7f8e24003a30 2023-06-09T11:22:25.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.111+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c0070d0 con 0x7f8e24003a30 2023-06-09T11:22:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.111+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c007480 con 0x7f8e24003a30 2023-06-09T11:22:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.111+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c15d210 con 0x7f8e24003a30 2023-06-09T11:22:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.111+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c15d7d0 con 0x7f8e24003a30 2023-06-09T11:22:25.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.112+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c074cf0 con 0x7f8e24003a30 2023-06-09T11:22:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.113+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8e2c019030 con 0x7f8e24003a30 2023-06-09T11:22:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.114+0000 7f8e23fff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 0x7f8e2806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.114+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f8e2c08a840 con 0x7f8e24003a30 2023-06-09T11:22:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.114+0000 7f8e3affd700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 0x7f8e2806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:25.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.114+0000 7f8e3affd700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 0x7f8e2806d1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c15dc90 tx=0x7f8e30009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.118+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8e2c0556a0 con 0x7f8e24003a30 2023-06-09T11:22:25.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.389+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e3c0c9370 con 0x7f8e24003a30 2023-06-09T11:22:25.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.390+0000 7f8e23fff700 1 -- 172.21.15.7:0/1785895783 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f8e2c058cc0 con 0x7f8e24003a30 2023-06-09T11:22:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:25.396 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:21.282520+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.392+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 msgr2=0x7f8e2806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.392+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 0x7f8e2806d1c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c15dc90 tx=0x7f8e30009210 comp rx=0 tx=0).stop 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.392+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 msgr2=0x7f8e3c0cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.392+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8e240047a0 tx=0x7f8e3c0b51d0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.392+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 shutdown_connections 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8e3c1e56c0 0x7f8e3c1e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e24003a30 0x7f8e3c0cd700 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8e2806ad10 0x7f8e2806d1c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 --2- 172.21.15.7:0/1785895783 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8e3c0c7430 0x7f8e3c1f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 >> 172.21.15.7:0/1785895783 conn(0x7f8e3c070af0 msgr2=0x7f8e3c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 shutdown_connections 2023-06-09T11:22:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:25.393+0000 7f8e41bff700 1 -- 172.21.15.7:0/1785895783 wait complete. 2023-06-09T11:22:25.400 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 57 2023-06-09T11:22:25.418 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:27.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:27.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.633+0000 7fee811ff700 1 Processor -- start 2023-06-09T11:22:27.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.633+0000 7fee811ff700 1 -- start start 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee811ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee7c153880 0x7fee7c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee811ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c15b4a0 0x7fee7c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee811ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c15a500 con 0x7fee7c0c9f40 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee811ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c0c5f10 con 0x7fee7c153880 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee811ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c15da00 con 0x7fee7c15b4a0 2023-06-09T11:22:27.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee7c153880 0x7fee7c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee7c153880 0x7fee7c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46070/0 (socket says 172.21.15.7:46070) 2023-06-09T11:22:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7ad9d700 1 -- 172.21.15.7:0/3915619301 learned_addr learned my addr 172.21.15.7:0/3915619301 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:27.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7b59e700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c15b4a0 0x7fee7c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7ad9d700 1 -- 172.21.15.7:0/3915619301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c15b4a0 msgr2=0x7fee7c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:27.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.635+0000 7fee7ad9d700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c15b4a0 0x7fee7c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee7ad9d700 1 -- 172.21.15.7:0/3915619301 >> v1:172.21.15.117:6789/0 conn(0x7fee7c0c9f40 legacy=0x7fee7c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee7ad9d700 1 -- 172.21.15.7:0/3915619301 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c1596e0 con 0x7fee7c153880 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee7ad9d700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c15d4c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0098f0 tx=0x7fee6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad2aaf3ee6199e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee79d9b700 1 -- 172.21.15.7:0/3915619301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00b8a0 con 0x7fee7c153880 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee79d9b700 1 -- 172.21.15.7:0/3915619301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c00ba00 con 0x7fee7c153880 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee79d9b700 1 -- 172.21.15.7:0/3915619301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c00bd90 con 0x7fee7c153880 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee811ff700 1 -- 172.21.15.7:0/3915619301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 msgr2=0x7fee7c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.636+0000 7fee811ff700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c15d4c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0098f0 tx=0x7fee6c009c00 comp rx=0 tx=0).stop 2023-06-09T11:22:27.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 -- 172.21.15.7:0/3915619301 shutdown_connections 2023-06-09T11:22:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c15b4a0 0x7fee7c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 --2- 172.21.15.7:0/3915619301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c15d4c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 -- 172.21.15.7:0/3915619301 >> 172.21.15.7:0/3915619301 conn(0x7fee7c0737f0 msgr2=0x7fee7c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 -- 172.21.15.7:0/3915619301 shutdown_connections 2023-06-09T11:22:27.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.637+0000 7fee811ff700 1 -- 172.21.15.7:0/3915619301 wait complete. 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.638+0000 7fee811ff700 1 Processor -- start 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.638+0000 7fee811ff700 1 -- start start 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c0c9f40 0x7fee7c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 0x7fee7c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1efc90 con 0x7fee7c153880 2023-06-09T11:22:27.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c14b5c0 con 0x7fee7c15b4a0 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.639+0000 7fee811ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1f00b0 con 0x7fee7c0c9f40 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46076/0 (socket says 172.21.15.7:46076) 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 -- 172.21.15.7:0/2702350110 learned_addr learned my addr 172.21.15.7:0/2702350110 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7ad9d700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c0c9f40 0x7fee7c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7b59e700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 0x7fee7c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 -- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c0c9f40 msgr2=0x7fee7c14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c0c9f40 0x7fee7c14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 -- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 msgr2=0x7fee7c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 0x7fee7c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 -- 172.21.15.7:0/2702350110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c009550 con 0x7fee7c153880 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7b59e700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 0x7fee7c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee7a59c700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c4e0 tx=0x7fee7000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.640+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700052d0 con 0x7fee7c153880 2023-06-09T11:22:27.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.641+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70005430 con 0x7fee7c153880 2023-06-09T11:22:27.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.641+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c159c60 con 0x7fee7c153880 2023-06-09T11:22:27.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.641+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700038c0 con 0x7fee7c153880 2023-06-09T11:22:27.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.641+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c159980 con 0x7fee7c153880 2023-06-09T11:22:27.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.642+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c159f40 con 0x7fee7c153880 2023-06-09T11:22:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.643+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee70003a20 con 0x7fee7c153880 2023-06-09T11:22:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.644+0000 7fee637fe700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 0x7fee6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.644+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fee70088960 con 0x7fee7c153880 2023-06-09T11:22:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.644+0000 7fee7ad9d700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 0x7fee6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.645+0000 7fee7ad9d700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 0x7fee6806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009c40 tx=0x7fee6c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:27.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.649+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fee70053880 con 0x7fee7c153880 2023-06-09T11:22:27.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.900+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee7c15a500 con 0x7fee7c153880 2023-06-09T11:22:27.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.900+0000 7fee637fe700 1 -- 172.21.15.7:0/2702350110 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fee7c15a500 con 0x7fee7c153880 2023-06-09T11:22:27.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:27.906 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:26.304164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 msgr2=0x7fee6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 0x7fee6806d190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fee6c009c40 tx=0x7fee6c008030 comp rx=0 tx=0).stop 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 msgr2=0x7fee7c0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c4e0 tx=0x7fee7000c7f0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 shutdown_connections 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee7c15b4a0 0x7fee7c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee6806ace0 0x7fee6806d190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee7c0c9f40 0x7fee7c14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 --2- 172.21.15.7:0/2702350110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee7c153880 0x7fee7c0ce850 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:27.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.902+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 >> 172.21.15.7:0/2702350110 conn(0x7fee7c0737f0 msgr2=0x7fee7c14dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:27.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.903+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 shutdown_connections 2023-06-09T11:22:27.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:27.903+0000 7fee811ff700 1 -- 172.21.15.7:0/2702350110 wait complete. 2023-06-09T11:22:27.910 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2023-06-09T11:22:27.926 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:29.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:30.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.121+0000 7f1d7edff700 1 Processor -- start 2023-06-09T11:22:30.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.121+0000 7f1d7edff700 1 -- start start 2023-06-09T11:22:30.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.122+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7edff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1d7815cd40 0x7f1d780cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7edff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d78159b20 con 0x7f1d780cb9a0 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7edff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d780cac80 con 0x7f1d7815cd40 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d781597d0 con 0x7f1d780cb5d0 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d77fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d780cb9a0 0x7f1d78159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42836/0 (socket says 172.21.15.7:42836) 2023-06-09T11:22:30.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d77fff700 1 -- 172.21.15.7:0/1940694486 learned_addr learned my addr 172.21.15.7:0/1940694486 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7d39c700 1 --2- 172.21.15.7:0/1940694486 >> v2:172.21.15.117:3300/0 conn(0x7f1d7815cd40 0x7f1d780cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d7cb9b700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d777fe700 1 -- 172.21.15.7:0/1940694486 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353665834 0 0) 0x7f1d78159b20 con 0x7f1d780cb9a0 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.123+0000 7f1d777fe700 1 -- 172.21.15.7:0/1940694486 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d680035b0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d7d39c700 1 -- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 msgr2=0x7f1d780b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d7d39c700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d7d39c700 1 -- 172.21.15.7:0/1940694486 >> v1:172.21.15.7:6789/0 conn(0x7f1d780cb9a0 legacy=0x7f1d78159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d7d39c700 1 -- 172.21.15.7:0/1940694486 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d78071ab0 con 0x7f1d7815cd40 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d7d39c700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780cbea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009d90 tx=0x7f1d7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed915d6e0ed8119e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d777fe700 1 -- 172.21.15.7:0/1940694486 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000e070 con 0x7f1d7815cd40 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d777fe700 1 -- 172.21.15.7:0/1940694486 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000bb20 con 0x7f1d7815cd40 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.124+0000 7f1d777fe700 1 -- 172.21.15.7:0/1940694486 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70012430 con 0x7f1d7815cd40 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 -- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 msgr2=0x7f1d780cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780cbea0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009d90 tx=0x7f1d7000b330 comp rx=0 tx=0).stop 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 -- 172.21.15.7:0/1940694486 shutdown_connections 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780cbea0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 --2- 172.21.15.7:0/1940694486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 -- 172.21.15.7:0/1940694486 >> 172.21.15.7:0/1940694486 conn(0x7f1d78070af0 msgr2=0x7f1d78070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 -- 172.21.15.7:0/1940694486 shutdown_connections 2023-06-09T11:22:30.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.125+0000 7f1d7edff700 1 -- 172.21.15.7:0/1940694486 wait complete. 2023-06-09T11:22:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.127+0000 7f1d7edff700 1 Processor -- start 2023-06-09T11:22:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.127+0000 7f1d7edff700 1 -- start start 2023-06-09T11:22:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d780aac10 con 0x7f1d780cb9a0 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d78071710 con 0x7f1d7815cd40 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d781ef550 con 0x7f1d780cb5d0 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46090/0 (socket says 172.21.15.7:46090) 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 -- 172.21.15.7:0/234802501 learned_addr learned my addr 172.21.15.7:0/234802501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7cb9b700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d7d39c700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 -- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 msgr2=0x7f1d780cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 -- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 msgr2=0x7f1d780ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.128+0000 7f1d77fff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d77fff700 1 -- 172.21.15.7:0/234802501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70009a70 con 0x7f1d780cb9a0 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d7d39c700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:30.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d77fff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c700 tx=0x7f1d6400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:30.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64005d70 con 0x7f1d780cb9a0 2023-06-09T11:22:30.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d640070d0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7815d4f0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d64007460 con 0x7f1d780cb9a0 2023-06-09T11:22:30.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.129+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7815d210 con 0x7f1d780cb9a0 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.130+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7815d7d0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.131+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d640075c0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.132+0000 7f1d757fa700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 0x7f1d5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.133+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f1d64088c60 con 0x7f1d780cb9a0 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.133+0000 7f1d7cb9b700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 0x7f1d5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:30.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.133+0000 7f1d7cb9b700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 0x7f1d5c06d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009be0 tx=0x7f1d6c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:30.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.137+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1d64053a30 con 0x7f1d780cb9a0 2023-06-09T11:22:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.390+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d780cdaa0 con 0x7f1d780cb9a0 2023-06-09T11:22:30.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.390+0000 7f1d757fa700 1 -- 172.21.15.7:0/234802501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f1d64088070 con 0x7f1d780cb9a0 2023-06-09T11:22:30.397 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:30.397 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:26.304164+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:30.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.392+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 msgr2=0x7f1d5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.392+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 0x7f1d5c06d190 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009be0 tx=0x7f1d6c009210 comp rx=0 tx=0).stop 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.392+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 msgr2=0x7f1d780cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.392+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400c700 tx=0x7f1d6400ca10 comp rx=0 tx=0).stop 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 shutdown_connections 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d7815cd40 0x7f1d780ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5c06ace0 0x7f1d5c06d190 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d780cb5d0 0x7f1d780cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 --2- 172.21.15.7:0/234802501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d780cb9a0 0x7f1d780cdfa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 >> 172.21.15.7:0/234802501 conn(0x7f1d78070af0 msgr2=0x7f1d78158920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 shutdown_connections 2023-06-09T11:22:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:30.393+0000 7f1d7edff700 1 -- 172.21.15.7:0/234802501 wait complete. 2023-06-09T11:22:30.400 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 58 2023-06-09T11:22:30.417 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:22:32.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.611+0000 7f5290dff700 1 Processor -- start 2023-06-09T11:22:32.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.611+0000 7f5290dff700 1 -- start start 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5290dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c15b4a0 0x7f528c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5290dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f528c0c9f40 0x7f528c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5290dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528c15a500 con 0x7f528c153880 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5290dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528c0c5f10 con 0x7f528c0c9f40 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5290dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c15da00 con 0x7f528c15b4a0 2023-06-09T11:22:32.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f528c0c9f40 0x7f528c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f528c0c9f40 0x7f528c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46106/0 (socket says 172.21.15.7:46106) 2023-06-09T11:22:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 -- 172.21.15.7:0/2361011674 learned_addr learned my addr 172.21.15.7:0/2361011674 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528a59c700 1 --1- 172.21.15.7:0/2361011674 >> v1:172.21.15.7:6789/0 conn(0x7f528c153880 0x7f528c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42846/0 (socket says 172.21.15.7:42846) 2023-06-09T11:22:32.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f5289d9b700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c15b4a0 0x7f528c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 -- 172.21.15.7:0/2361011674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c15b4a0 msgr2=0x7f528c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:22:32.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c15b4a0 0x7f528c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 -- 172.21.15.7:0/2361011674 >> v1:172.21.15.7:6789/0 conn(0x7f528c153880 legacy=0x7f528c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528ad9d700 1 -- 172.21.15.7:0/2361011674 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c1596e0 con 0x7f528c0c9f40 2023-06-09T11:22:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.613+0000 7f528959a700 1 -- 172.21.15.7:0/2361011674 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879248345 0 0) 0x7f527c001110 con 0x7f528c153880 2023-06-09T11:22:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f528ad9d700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0c4e30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5284009e50 tx=0x7f528400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc43e5cdaa3ba5da server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:32.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f528959a700 1 -- 172.21.15.7:0/2361011674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400e070 con 0x7f528c0c9f40 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f528959a700 1 -- 172.21.15.7:0/2361011674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528400b9c0 con 0x7f528c0c9f40 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f528959a700 1 -- 172.21.15.7:0/2361011674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528400bd70 con 0x7f528c0c9f40 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f5290dff700 1 -- 172.21.15.7:0/2361011674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 msgr2=0x7f528c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.614+0000 7f5290dff700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0c4e30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5284009e50 tx=0x7f528400b330 comp rx=0 tx=0).stop 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 -- 172.21.15.7:0/2361011674 shutdown_connections 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0c4e30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 --2- 172.21.15.7:0/2361011674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c15b4a0 0x7f528c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 -- 172.21.15.7:0/2361011674 >> 172.21.15.7:0/2361011674 conn(0x7f528c0737f0 msgr2=0x7f528c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 -- 172.21.15.7:0/2361011674 shutdown_connections 2023-06-09T11:22:32.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.615+0000 7f5290dff700 1 -- 172.21.15.7:0/2361011674 wait complete. 2023-06-09T11:22:32.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.616+0000 7f5290dff700 1 Processor -- start 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.616+0000 7f5290dff700 1 -- start start 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c153880 0x7f528c152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 0x7f528c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c150b60 con 0x7f528c0c9f40 2023-06-09T11:22:32.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528c0cd720 con 0x7f528c15b4a0 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.617+0000 7f5290dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528c158bb0 con 0x7f528c153880 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46120/0 (socket says 172.21.15.7:46120) 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 -- 172.21.15.7:0/702970638 learned_addr learned my addr 172.21.15.7:0/702970638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f5289d9b700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c153880 0x7f528c152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528ad9d700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 0x7f528c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 -- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c153880 msgr2=0x7f528c152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c153880 0x7f528c152450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 -- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 msgr2=0x7f528c152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 0x7f528c152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 -- 172.21.15.7:0/702970638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284009b30 con 0x7f528c0c9f40 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528ad9d700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 0x7f528c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.618+0000 7f528a59c700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c760 tx=0x7f527c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.619+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c004c80 con 0x7f528c0c9f40 2023-06-09T11:22:32.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.619+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527c004de0 con 0x7f528c0c9f40 2023-06-09T11:22:32.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.619+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527c007310 con 0x7f528c0c9f40 2023-06-09T11:22:32.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.619+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528c159c60 con 0x7f528c0c9f40 2023-06-09T11:22:32.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.619+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528c159980 con 0x7f528c0c9f40 2023-06-09T11:22:32.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.620+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528c159f40 con 0x7f528c0c9f40 2023-06-09T11:22:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.621+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f527c007470 con 0x7f528c0c9f40 2023-06-09T11:22:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.622+0000 7f527affd700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 0x7f526c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.622+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f527c089300 con 0x7f528c0c9f40 2023-06-09T11:22:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.622+0000 7f5289d9b700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 0x7f526c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:32.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.622+0000 7f5289d9b700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 0x7f526c06d1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f528c0b1f00 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:32.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.626+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f527c054060 con 0x7f528c0c9f40 2023-06-09T11:22:32.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.879+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f528c15a500 con 0x7f528c0c9f40 2023-06-09T11:22:32.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.880+0000 7f527affd700 1 -- 172.21.15.7:0/702970638 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f528c15a500 con 0x7f528c0c9f40 2023-06-09T11:22:32.886 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:32.886 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:32.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.882+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 msgr2=0x7f526c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.882+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 0x7f526c06d1c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f528c0b1f00 tx=0x7f5280008040 comp rx=0 tx=0).stop 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.882+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 msgr2=0x7f528c0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.882+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f527c00c760 tx=0x7f527c00ca70 comp rx=0 tx=0).stop 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 shutdown_connections 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f528c15b4a0 0x7f528c152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f528c0c9f40 0x7f528c0ce830 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f526c06ad10 0x7f526c06d1c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 --2- 172.21.15.7:0/702970638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f528c153880 0x7f528c152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 >> 172.21.15.7:0/702970638 conn(0x7f528c0737f0 msgr2=0x7f528c14bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 shutdown_connections 2023-06-09T11:22:32.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:32.883+0000 7f5290dff700 1 -- 172.21.15.7:0/702970638 wait complete. 2023-06-09T11:22:32.891 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 59 2023-06-09T11:22:32.906 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=2, replacing gid=4936 2023-06-09T11:22:32.906 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.b 2023-06-09T11:22:32.907 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:22:32.907 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:22:32.907 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:22:32.911 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:22:32.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:22:33.060 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:22:33.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.109+0000 7faea0fff700 1 Processor -- start 2023-06-09T11:22:33.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.109+0000 7faea0fff700 1 -- start start 2023-06-09T11:22:33.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7faea0fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 0x7fae9c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7faea0fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0c5a80 con 0x7fae9c153880 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7faea0fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c15a500 con 0x7fae9c15b4a0 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7faea0fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0c5f10 con 0x7fae9c0c9f40 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fae9c153880 0x7fae9c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34424/0 (socket says 172.21.15.7:34424) 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9a59c700 1 -- 172.21.15.7:0/1375123122 learned_addr learned my addr 172.21.15.7:0/1375123122 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:22:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9ad9d700 1 --2- 172.21.15.7:0/1375123122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 0x7fae9c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944062350 0 0) 0x7fae9c0c5a80 con 0x7fae9c153880 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae880035b0 con 0x7fae9c153880 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2538089382 0 0) 0x7fae84001110 con 0x7fae9c15b4a0 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae9c0c5a80 con 0x7fae9c15b4a0 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9ad9d700 1 -- 172.21.15.7:0/1375123122 >> v1:172.21.15.117:6789/0 conn(0x7fae9c15b4a0 legacy=0x7fae9c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9ad9d700 1 -- 172.21.15.7:0/1375123122 >> v1:172.21.15.7:6789/0 conn(0x7fae9c153880 legacy=0x7fae9c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.111+0000 7fae9ad9d700 1 -- 172.21.15.7:0/1375123122 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c1596e0 con 0x7fae9c0c9f40 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.112+0000 7fae9ad9d700 1 --2- 172.21.15.7:0/1375123122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 0x7fae9c0c5540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae94009dc0 tx=0x7fae9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0e828b330032d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.112+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400e070 con 0x7fae9c0c9f40 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.112+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400bb20 con 0x7fae9c0c9f40 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.112+0000 7fae99d9b700 1 -- 172.21.15.7:0/1375123122 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400be90 con 0x7fae9c0c9f40 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 -- 172.21.15.7:0/1375123122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 msgr2=0x7fae9c0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 --2- 172.21.15.7:0/1375123122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 0x7fae9c0c5540 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae94009dc0 tx=0x7fae9400b330 comp rx=0 tx=0).stop 2023-06-09T11:22:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 -- 172.21.15.7:0/1375123122 shutdown_connections 2023-06-09T11:22:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 --2- 172.21.15.7:0/1375123122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c0c9f40 0x7fae9c0c5540 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 -- 172.21.15.7:0/1375123122 >> 172.21.15.7:0/1375123122 conn(0x7fae9c0737f0 msgr2=0x7fae9c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 -- 172.21.15.7:0/1375123122 shutdown_connections 2023-06-09T11:22:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.113+0000 7faea0fff700 1 -- 172.21.15.7:0/1375123122 wait complete. 2023-06-09T11:22:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.114+0000 7faea0fff700 1 Processor -- start 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.115+0000 7faea0fff700 1 -- start start 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.115+0000 7faea0fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.115+0000 7faea0fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae9c153880 0x7fae9c1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7faea0fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c15b4a0 0x7fae9c14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7faea0fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b5440 con 0x7fae9c0c9f40 2023-06-09T11:22:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7faea0fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1e39d0 con 0x7fae9c153880 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7faea0fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c152ac0 con 0x7fae9c15b4a0 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53670/0 (socket says 172.21.15.7:53670) 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 -- 172.21.15.7:0/837162183 learned_addr learned my addr 172.21.15.7:0/837162183 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae93fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae9c153880 0x7fae9c1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9ad9d700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c15b4a0 0x7fae9c14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 -- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c15b4a0 msgr2=0x7fae9c14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c15b4a0 0x7fae9c14ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 -- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae9c153880 msgr2=0x7fae9c1f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae9c153880 0x7fae9c1f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 -- 172.21.15.7:0/837162183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94009a70 con 0x7fae9c0c9f40 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.116+0000 7fae9a59c700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00c700 tx=0x7fae8c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.117+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c005d70 con 0x7fae9c0c9f40 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.117+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c0070d0 con 0x7fae9c0c9f40 2023-06-09T11:22:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.117+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c007460 con 0x7fae9c0c9f40 2023-06-09T11:22:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.117+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c159c60 con 0x7fae9c0c9f40 2023-06-09T11:22:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.117+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c159980 con 0x7fae9c0c9f40 2023-06-09T11:22:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.118+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c159f40 con 0x7fae9c0c9f40 2023-06-09T11:22:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.119+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fae8c0075c0 con 0x7fae9c0c9f40 2023-06-09T11:22:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.120+0000 7fae92ffd700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 0x7fae7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:22:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.120+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fae8c088bc0 con 0x7fae9c0c9f40 2023-06-09T11:22:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.120+0000 7fae93fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 0x7fae7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:22:33.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.121+0000 7fae93fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 0x7fae7c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae9c1f35e0 tx=0x7fae84009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:22:33.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.124+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fae8c053990 con 0x7fae9c0c9f40 2023-06-09T11:22:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.377+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae9c074cf0 con 0x7fae9c0c9f40 2023-06-09T11:22:33.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.378+0000 7fae92ffd700 1 -- 172.21.15.7:0/837162183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fae8c017090 con 0x7fae9c0c9f40 2023-06-09T11:22:33.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:22:33.384 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:22:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 msgr2=0x7fae7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 0x7fae7c06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae9c1f35e0 tx=0x7fae84009040 comp rx=0 tx=0).stop 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 msgr2=0x7fae9c1579c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00c700 tx=0x7fae8c00ca10 comp rx=0 tx=0).stop 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 shutdown_connections 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fae7c06ace0 0x7fae7c06d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fae9c15b4a0 0x7fae9c14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae9c0c9f40 0x7fae9c1579c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 --2- 172.21.15.7:0/837162183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fae9c153880 0x7fae9c1f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:22:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.380+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 >> 172.21.15.7:0/837162183 conn(0x7fae9c0737f0 msgr2=0x7fae9c14d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:22:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.381+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 shutdown_connections 2023-06-09T11:22:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:22:33.381+0000 7faea0fff700 1 -- 172.21.15.7:0/837162183 wait complete. 2023-06-09T11:22:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:22:33.404 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:22:33.405 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 90 secs before thrashing 2023-06-09T11:22:35.284 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:22:35.279+0000 7fa6f8ff1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:22:43.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:22:43.450 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:13.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:13.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:43.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:23:43.815 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:03.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.627+0000 7fb8d01ff700 1 Processor -- start 2023-06-09T11:24:03.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.627+0000 7fb8d01ff700 1 -- start start 2023-06-09T11:24:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8d01ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c80cb5d0 0x7fb8c80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8d01ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8c815cd40 0x7fb8c80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8d01ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c8159b20 con 0x7fb8c80cb9a0 2023-06-09T11:24:03.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8d01ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80cac80 con 0x7fb8c815cd40 2023-06-09T11:24:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8d01ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81597d0 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8ce79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8c815cd40 0x7fb8c80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8ce79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb8c815cd40 0x7fb8c80cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41332/0 (socket says 172.21.15.7:41332) 2023-06-09T11:24:03.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.628+0000 7fb8ce79c700 1 -- 172.21.15.7:0/2796727310 learned_addr learned my addr 172.21.15.7:0/2796727310 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8cdf9b700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c80cb5d0 0x7fb8c80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ce79c700 1 -- 172.21.15.7:0/2796727310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c80cb5d0 msgr2=0x7fb8c80b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ce79c700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c80cb5d0 0x7fb8c80b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ce79c700 1 -- 172.21.15.7:0/2796727310 >> v1:172.21.15.7:6789/0 conn(0x7fb8c80cb9a0 legacy=0x7fb8c8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ce79c700 1 -- 172.21.15.7:0/2796727310 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8071ab0 con 0x7fb8c815cd40 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ce79c700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c815cd40 0x7fb8c80cbea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009e50 tx=0x7fb8c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bd3251218866 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ccf99700 1 -- 172.21.15.7:0/2796727310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000e070 con 0x7fb8c815cd40 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ccf99700 1 -- 172.21.15.7:0/2796727310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0003a20 con 0x7fb8c815cd40 2023-06-09T11:24:03.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.629+0000 7fb8ccf99700 1 -- 172.21.15.7:0/2796727310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003dd0 con 0x7fb8c815cd40 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 -- 172.21.15.7:0/2796727310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c815cd40 msgr2=0x7fb8c80cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c815cd40 0x7fb8c80cbea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009e50 tx=0x7fb8c000b330 comp rx=0 tx=0).stop 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 -- 172.21.15.7:0/2796727310 shutdown_connections 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c815cd40 0x7fb8c80cbea0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/2796727310 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c80cb5d0 0x7fb8c80b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 -- 172.21.15.7:0/2796727310 >> 172.21.15.7:0/2796727310 conn(0x7fb8c8070af0 msgr2=0x7fb8c8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:03.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 -- 172.21.15.7:0/2796727310 shutdown_connections 2023-06-09T11:24:03.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.630+0000 7fb8d01ff700 1 -- 172.21.15.7:0/2796727310 wait complete. 2023-06-09T11:24:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.632+0000 7fb8d01ff700 1 Processor -- start 2023-06-09T11:24:03.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.632+0000 7fb8d01ff700 1 -- start start 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c80cb9a0 0x7fb8c81f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c815cd40 0x7fb8c81f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81f2a10 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c80b1c70 con 0x7fb8c80cb9a0 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8d01ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c8153c40 con 0x7fb8c815cd40 2023-06-09T11:24:03.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41342/0 (socket says 172.21.15.7:41342) 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 -- 172.21.15.7:0/1175240421 learned_addr learned my addr 172.21.15.7:0/1175240421 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8ce79c700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c815cd40 0x7fb8c81f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cd79a700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c80cb9a0 0x7fb8c81f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 -- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c815cd40 msgr2=0x7fb8c81f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c815cd40 0x7fb8c81f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 -- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c80cb9a0 msgr2=0x7fb8c81f1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c80cb9a0 0x7fb8c81f1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.633+0000 7fb8cdf9b700 1 -- 172.21.15.7:0/1175240421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c0009b30 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8cdf9b700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400e840 tx=0x7fb8c400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4013610 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4013770 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4013b20 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c815d4f0 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.634+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c815d210 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.635+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c815d7d0 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.636+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8c4013c80 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.637+0000 7fb8be7fc700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 0x7fb8ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.637+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fb8c40893d0 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.637+0000 7fb8cd79a700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 0x7fb8ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:03.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.638+0000 7fb8cd79a700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 0x7fb8ac06d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8071290 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:03.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.642+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb8c4054120 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.916+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8c8074cf0 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.917+0000 7fb8be7fc700 1 -- 172.21.15.7:0/1175240421 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fb8c4057740 con 0x7fb8c80cb5d0 2023-06-09T11:24:03.922 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 msgr2=0x7fb8ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 0x7fb8ac06d190 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8071290 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 msgr2=0x7fb8c8153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400e840 tx=0x7fb8c400eb50 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 shutdown_connections 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb8ac06ace0 0x7fb8ac06d190 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8c815cd40 0x7fb8c81f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8c80cb5d0 0x7fb8c8153700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 --2- 172.21.15.7:0/1175240421 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb8c80cb9a0 0x7fb8c81f1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 >> 172.21.15.7:0/1175240421 conn(0x7fb8c8070af0 msgr2=0x7fb8c80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.919+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 shutdown_connections 2023-06-09T11:24:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:03.920+0000 7fb8d01ff700 1 -- 172.21.15.7:0/1175240421 wait complete. 2023-06-09T11:24:03.926 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:03.942 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2023-06-09T11:24:03.943 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:24:03.943 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:24:03.962 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:24:03.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:04.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.157+0000 7fcd015ff700 1 Processor -- start 2023-06-09T11:24:04.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.157+0000 7fcd015ff700 1 -- start start 2023-06-09T11:24:04.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fcd015ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fccfc15b4a0 0x7fccfc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:04.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fcd015ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccfc15a500 con 0x7fccfc153880 2023-06-09T11:24:04.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fcd015ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccfc15da00 con 0x7fccfc0c9f40 2023-06-09T11:24:04.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fcd015ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccfc0c5f10 con 0x7fccfc15b4a0 2023-06-09T11:24:04.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fccfaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fccfc153880 0x7fccfc15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42430/0 (socket says 172.21.15.7:42430) 2023-06-09T11:24:04.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fccfaffd700 1 -- 172.21.15.7:0/3305162780 learned_addr learned my addr 172.21.15.7:0/3305162780 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:04.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.158+0000 7fccfa7fc700 1 --2- 172.21.15.7:0/3305162780 >> v2:172.21.15.7:3300/0 conn(0x7fccfc15b4a0 0x7fccfc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:04.164 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2023-06-09T11:24:04.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccf9ffb700 1 -- 172.21.15.7:0/3305162780 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037820897 0 0) 0x7fccec001110 con 0x7fccfc153880 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccf9ffb700 1 -- 172.21.15.7:0/3305162780 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce40035b0 con 0x7fccfc153880 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccfa7fc700 1 -- 172.21.15.7:0/3305162780 >> v1:172.21.15.117:6789/0 conn(0x7fccfc0c9f40 legacy=0x7fccfc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccfa7fc700 1 -- 172.21.15.7:0/3305162780 >> v1:172.21.15.7:6789/0 conn(0x7fccfc153880 legacy=0x7fccfc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccfa7fc700 1 -- 172.21.15.7:0/3305162780 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc1596e0 con 0x7fccfc15b4a0 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccfa7fc700 1 --2- 172.21.15.7:0/3305162780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc15d4c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009840 tx=0x7fccf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f235ee0c9c6affb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccf9ffb700 1 -- 172.21.15.7:0/3305162780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0011830 con 0x7fccfc15b4a0 2023-06-09T11:24:04.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.159+0000 7fccf9ffb700 1 -- 172.21.15.7:0/3305162780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0011990 con 0x7fccfc15b4a0 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fccf9ffb700 1 -- 172.21.15.7:0/3305162780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0011d20 con 0x7fccfc15b4a0 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 -- 172.21.15.7:0/3305162780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 msgr2=0x7fccfc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 --2- 172.21.15.7:0/3305162780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc15d4c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009840 tx=0x7fccf0009b50 comp rx=0 tx=0).stop 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 -- 172.21.15.7:0/3305162780 shutdown_connections 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 --2- 172.21.15.7:0/3305162780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc15d4c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 -- 172.21.15.7:0/3305162780 >> 172.21.15.7:0/3305162780 conn(0x7fccfc0737f0 msgr2=0x7fccfc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 -- 172.21.15.7:0/3305162780 shutdown_connections 2023-06-09T11:24:04.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.160+0000 7fcd015ff700 1 -- 172.21.15.7:0/3305162780 wait complete. 2023-06-09T11:24:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.162+0000 7fcd015ff700 1 Processor -- start 2023-06-09T11:24:04.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.162+0000 7fcd015ff700 1 -- start start 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 0x7fccfc150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccfc153880 0x7fccfc14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc15c820 con 0x7fccfc15b4a0 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc0cea10 con 0x7fccfc153880 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fcd015ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc0ce120 con 0x7fccfc0c9f40 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:04.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fccfaffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 0x7fccfc150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fccfaffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 0x7fccfc150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:58940/0 (socket says 172.21.15.7:58940) 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fccfaffd700 1 -- 172.21.15.7:0/2575362410 learned_addr learned my addr 172.21.15.7:0/2575362410 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.163+0000 7fccfb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41354/0 (socket says 172.21.15.7:41354) 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfa7fc700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccfc153880 0x7fccfc14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 -- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 msgr2=0x7fccfc150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 0x7fccfc150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 -- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccfc153880 msgr2=0x7fccfc14eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccfc153880 0x7fccfc14eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 -- 172.21.15.7:0/2575362410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00094a0 con 0x7fccfc15b4a0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fccfb7fe700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fccf400cb90 tx=0x7fccf400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4003c00 con 0x7fccfc15b4a0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4003d60 con 0x7fccfc15b4a0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf400aa80 con 0x7fccfc15b4a0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.164+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccfc159c60 con 0x7fccfc15b4a0 2023-06-09T11:24:04.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.165+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccfc159980 con 0x7fccfc15b4a0 2023-06-09T11:24:04.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.166+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccfc159f40 con 0x7fccfc15b4a0 2023-06-09T11:24:04.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.166+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fccf400abe0 con 0x7fccfc15b4a0 2023-06-09T11:24:04.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.167+0000 7fcce37fe700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 0x7fcce806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.167+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fccf40894c0 con 0x7fccfc15b4a0 2023-06-09T11:24:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.168+0000 7fccfaffd700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 0x7fcce806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:04.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.168+0000 7fccfaffd700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 0x7fcce806d1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccec007fd0 tx=0x7fccec007d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:04.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.171+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fccf4054290 con 0x7fccfc15b4a0 2023-06-09T11:24:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.426+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccfc15da00 con 0x7fccfc15b4a0 2023-06-09T11:24:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.427+0000 7fcce37fe700 1 -- 172.21.15.7:0/2575362410 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fccfc15da00 con 0x7fccfc15b4a0 2023-06-09T11:24:04.432 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:04.432 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 msgr2=0x7fcce806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 0x7fcce806d1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fccec007fd0 tx=0x7fccec007d00 comp rx=0 tx=0).stop 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 msgr2=0x7fccfc0ceea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fccf400cb90 tx=0x7fccf400cea0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 shutdown_connections 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fccfc15b4a0 0x7fccfc0ceea0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcce806ad10 0x7fcce806d1c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fccfc0c9f40 0x7fccfc150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 --2- 172.21.15.7:0/2575362410 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fccfc153880 0x7fccfc14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.429+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 >> 172.21.15.7:0/2575362410 conn(0x7fccfc0737f0 msgr2=0x7fccfc14b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.430+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 shutdown_connections 2023-06-09T11:24:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:04.430+0000 7fcd015ff700 1 -- 172.21.15.7:0/2575362410 wait complete. 2023-06-09T11:24:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:04.452 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:06.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:06.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.658+0000 7fc72abff700 1 Processor -- start 2023-06-09T11:24:06.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.659+0000 7fc72abff700 1 -- start start 2023-06-09T11:24:06.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72abff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7240c1bc0 0x7fc7241599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:06.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72abff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240c0bf0 con 0x7fc7240c0140 2023-06-09T11:24:06.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72abff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240702f0 con 0x7fc7240c1f90 2023-06-09T11:24:06.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72abff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc724070110 con 0x7fc7240c1bc0 2023-06-09T11:24:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72899b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7240c1bc0 0x7fc7241599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72899b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc7240c1bc0 0x7fc7241599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41362/0 (socket says 172.21.15.7:41362) 2023-06-09T11:24:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc723fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc7240c0140 0x7fc7241501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42440/0 (socket says 172.21.15.7:42440) 2023-06-09T11:24:06.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.660+0000 7fc72899b700 1 -- 172.21.15.7:0/2339928560 learned_addr learned my addr 172.21.15.7:0/2339928560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc72899b700 1 -- 172.21.15.7:0/2339928560 >> v1:172.21.15.117:6789/0 conn(0x7fc7240c1f90 legacy=0x7fc724160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc72899b700 1 -- 172.21.15.7:0/2339928560 >> v1:172.21.15.7:6789/0 conn(0x7fc7240c0140 legacy=0x7fc7241501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc72899b700 1 -- 172.21.15.7:0/2339928560 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240c0e90 con 0x7fc7240c1bc0 2023-06-09T11:24:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc72899b700 1 --2- 172.21.15.7:0/2339928560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc7241599b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc71800bb90 tx=0x7fc71800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27b98023686b107 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:06.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc7237fe700 1 -- 172.21.15.7:0/2339928560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71800dd10 con 0x7fc7240c1bc0 2023-06-09T11:24:06.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc7237fe700 1 -- 172.21.15.7:0/2339928560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71800de70 con 0x7fc7240c1bc0 2023-06-09T11:24:06.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.661+0000 7fc7237fe700 1 -- 172.21.15.7:0/2339928560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718004230 con 0x7fc7240c1bc0 2023-06-09T11:24:06.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 -- 172.21.15.7:0/2339928560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 msgr2=0x7fc7241599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 --2- 172.21.15.7:0/2339928560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc7241599b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc71800bb90 tx=0x7fc71800bea0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 -- 172.21.15.7:0/2339928560 shutdown_connections 2023-06-09T11:24:06.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 --2- 172.21.15.7:0/2339928560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc7241599b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 -- 172.21.15.7:0/2339928560 >> 172.21.15.7:0/2339928560 conn(0x7fc72406a4d0 msgr2=0x7fc72406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:06.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 -- 172.21.15.7:0/2339928560 shutdown_connections 2023-06-09T11:24:06.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.662+0000 7fc72abff700 1 -- 172.21.15.7:0/2339928560 wait complete. 2023-06-09T11:24:06.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.664+0000 7fc72abff700 1 Processor -- start 2023-06-09T11:24:06.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.665+0000 7fc72abff700 1 -- start start 2023-06-09T11:24:06.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7240c0140 0x7fc7241eb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:06.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7240c1f90 0x7fc7240c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240ad650 con 0x7fc7240c1bc0 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7241ead80 con 0x7fc7240c0140 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72abff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240adb80 con 0x7fc7240c1f90 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc723fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc723fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41368/0 (socket says 172.21.15.7:41368) 2023-06-09T11:24:06.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc723fff700 1 -- 172.21.15.7:0/3011792099 learned_addr learned my addr 172.21.15.7:0/3011792099 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72899b700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7240c0140 0x7fc7241eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.666+0000 7fc72919c700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7240c1f90 0x7fc7240c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 -- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7240c1f90 msgr2=0x7fc7240c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7240c1f90 0x7fc7240c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 -- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7240c0140 msgr2=0x7fc7241eb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7240c0140 0x7fc7241eb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 -- 172.21.15.7:0/3011792099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71800b7f0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc723fff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c4c0 tx=0x7fc71000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710005cb0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710005e10 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7100072d0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240c1450 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.667+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240c1170 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.668+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240705d0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.669+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc710007430 con 0x7fc7240c1bc0 2023-06-09T11:24:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.671+0000 7fc7217fa700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 0x7fc70806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.671+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fc710088a40 con 0x7fc7240c1bc0 2023-06-09T11:24:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.671+0000 7fc72899b700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 0x7fc70806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:06.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.671+0000 7fc72899b700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 0x7fc70806d1c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b340 tx=0x7fc71801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:06.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.675+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc710053980 con 0x7fc7240c1bc0 2023-06-09T11:24:06.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.927+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc724074cf0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.928+0000 7fc7217fa700 1 -- 172.21.15.7:0/3011792099 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fc710056fa0 con 0x7fc7240c1bc0 2023-06-09T11:24:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 msgr2=0x7fc70806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 0x7fc70806d1c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b340 tx=0x7fc71801c040 comp rx=0 tx=0).stop 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 msgr2=0x7fc724155b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc71000c4c0 tx=0x7fc71000c7d0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 shutdown_connections 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc70806ad10 0x7fc70806d1c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc7240c1f90 0x7fc7240c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc7240c0140 0x7fc7241eb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 --2- 172.21.15.7:0/3011792099 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7240c1bc0 0x7fc724155b90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 >> 172.21.15.7:0/3011792099 conn(0x7fc72406a4d0 msgr2=0x7fc7240c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 shutdown_connections 2023-06-09T11:24:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:06.930+0000 7fc72abff700 1 -- 172.21.15.7:0/3011792099 wait complete. 2023-06-09T11:24:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:06.953 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:08.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.159+0000 7fa0babff700 1 Processor -- start 2023-06-09T11:24:09.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.159+0000 7fa0babff700 1 -- start start 2023-06-09T11:24:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0babff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0babff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0b40cb9a0 0x7fa0b4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0babff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0b4159b20 con 0x7fa0b415cd40 2023-06-09T11:24:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0babff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0b40cac80 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0babff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41597d0 con 0x7fa0b40cb5d0 2023-06-09T11:24:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0b3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0b40cb9a0 0x7fa0b4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0b919c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0b415cd40 0x7fa0b40cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42454/0 (socket says 172.21.15.7:42454) 2023-06-09T11:24:09.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.160+0000 7fa0b919c700 1 -- 172.21.15.7:0/1609167864 learned_addr learned my addr 172.21.15.7:0/1609167864 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:09.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b899b700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b3fff700 1 -- 172.21.15.7:0/1609167864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 msgr2=0x7fa0b40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b3fff700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b3fff700 1 -- 172.21.15.7:0/1609167864 >> v1:172.21.15.7:6789/0 conn(0x7fa0b415cd40 legacy=0x7fa0b40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b3fff700 1 -- 172.21.15.7:0/1609167864 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4071ab0 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b899b700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b3fff700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b4159fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009840 tx=0x7fa0a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf758efd202a316c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:09.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b37fe700 1 -- 172.21.15.7:0/1609167864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011830 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b37fe700 1 -- 172.21.15.7:0/1609167864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0011990 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.161+0000 7fa0b37fe700 1 -- 172.21.15.7:0/1609167864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0011d20 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 -- 172.21.15.7:0/1609167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b40cb9a0 msgr2=0x7fa0b4159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b4159fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009840 tx=0x7fa0a0009b50 comp rx=0 tx=0).stop 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 -- 172.21.15.7:0/1609167864 shutdown_connections 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 --2- 172.21.15.7:0/1609167864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b4159fb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 -- 172.21.15.7:0/1609167864 >> 172.21.15.7:0/1609167864 conn(0x7fa0b4070af0 msgr2=0x7fa0b4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 -- 172.21.15.7:0/1609167864 shutdown_connections 2023-06-09T11:24:09.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.162+0000 7fa0babff700 1 -- 172.21.15.7:0/1609167864 wait complete. 2023-06-09T11:24:09.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.164+0000 7fa0babff700 1 Processor -- start 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.164+0000 7fa0babff700 1 -- start start 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b41e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b41e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 0x7fa0b41f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41e5230 con 0x7fa0b415cd40 2023-06-09T11:24:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b41e5590 con 0x7fa0b40cb9a0 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0babff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b40c8b10 con 0x7fa0b40cb5d0 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0b919c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 0x7fa0b41f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0b899b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b41e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0b899b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b41e4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:58966/0 (socket says 172.21.15.7:58966) 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0b899b700 1 -- 172.21.15.7:0/116771915 learned_addr learned my addr 172.21.15.7:0/116771915 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.165+0000 7fa0b3fff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b41e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 -- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 msgr2=0x7fa0b41e4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b41e4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 -- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b40cb9a0 msgr2=0x7fa0b41e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b41e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 -- 172.21.15.7:0/116771915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a00094a0 con 0x7fa0b415cd40 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b919c700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 0x7fa0b41f0490 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4074a10 tx=0x7fa0ac00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00a9a0 con 0x7fa0b415cd40 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac00ab00 con 0x7fa0b415cd40 2023-06-09T11:24:09.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac00ae70 con 0x7fa0b415cd40 2023-06-09T11:24:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b415d4f0 con 0x7fa0b415cd40 2023-06-09T11:24:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.166+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b415d210 con 0x7fa0b415cd40 2023-06-09T11:24:09.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.167+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b415d7d0 con 0x7fa0b415cd40 2023-06-09T11:24:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.168+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa0ac01b3b0 con 0x7fa0b415cd40 2023-06-09T11:24:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.169+0000 7fa0b17fa700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 0x7fa09806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.169+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fa0ac088fb0 con 0x7fa0b415cd40 2023-06-09T11:24:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.170+0000 7fa0b899b700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 0x7fa09806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:09.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.170+0000 7fa0b899b700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 0x7fa09806d1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:09.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.173+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa0ac053ec0 con 0x7fa0b415cd40 2023-06-09T11:24:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.428+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0b4074cf0 con 0x7fa0b415cd40 2023-06-09T11:24:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.429+0000 7fa0b17fa700 1 -- 172.21.15.7:0/116771915 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fa0ac0574e0 con 0x7fa0b415cd40 2023-06-09T11:24:09.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:09.434 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 msgr2=0x7fa09806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 0x7fa09806d1c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8006fd0 tx=0x7fa0a8008040 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 msgr2=0x7fa0b41f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 0x7fa0b41f0490 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4074a10 tx=0x7fa0ac00cd50 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 shutdown_connections 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0b415cd40 0x7fa0b41f0490 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa09806ad10 0x7fa09806d1c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0b40cb5d0 0x7fa0b41e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 --2- 172.21.15.7:0/116771915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0b40cb9a0 0x7fa0b41e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.431+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 >> 172.21.15.7:0/116771915 conn(0x7fa0b4070af0 msgr2=0x7fa0b40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.432+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 shutdown_connections 2023-06-09T11:24:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:09.432+0000 7fa0babff700 1 -- 172.21.15.7:0/116771915 wait complete. 2023-06-09T11:24:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:09.454 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:10.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2023-06-09T11:24:11.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.660+0000 7f4a57dff700 1 Processor -- start 2023-06-09T11:24:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.660+0000 7f4a57dff700 1 -- start start 2023-06-09T11:24:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.661+0000 7f4a57dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c1bc0 0x7f4a501599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.661+0000 7f4a57dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a500c1f90 0x7f4a50159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.661+0000 7f4a57dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a500c0bf0 con 0x7f4a500c0140 2023-06-09T11:24:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.661+0000 7f4a57dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a50070110 con 0x7f4a500c1f90 2023-06-09T11:24:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.661+0000 7f4a57dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a50071300 con 0x7f4a500c1bc0 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a500c1f90 0x7f4a50159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4a500c1f90 0x7f4a50159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41400/0 (socket says 172.21.15.7:41400) 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 -- 172.21.15.7:0/3498705574 learned_addr learned my addr 172.21.15.7:0/3498705574 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a55b9b700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c1bc0 0x7f4a501599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a54b99700 1 -- 172.21.15.7:0/3498705574 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151409097 0 0) 0x7f4a500c0bf0 con 0x7f4a500c0140 2023-06-09T11:24:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 -- 172.21.15.7:0/3498705574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c1bc0 msgr2=0x7f4a501599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c1bc0 0x7f4a501599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 -- 172.21.15.7:0/3498705574 >> v1:172.21.15.117:6789/0 conn(0x7f4a500c0140 legacy=0x7f4a501501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 -- 172.21.15.7:0/3498705574 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500702f0 con 0x7f4a500c1f90 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.662+0000 7f4a5639c700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1f90 0x7f4a50159ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009e50 tx=0x7f4a4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebc7d6f56adb881 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a54b99700 1 -- 172.21.15.7:0/3498705574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800e070 con 0x7f4a500c1f90 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a54b99700 1 -- 172.21.15.7:0/3498705574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4800b910 con 0x7f4a500c1f90 2023-06-09T11:24:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a54b99700 1 -- 172.21.15.7:0/3498705574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800bcc0 con 0x7f4a500c1f90 2023-06-09T11:24:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 -- 172.21.15.7:0/3498705574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1f90 msgr2=0x7f4a50159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1f90 0x7f4a50159ef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009e50 tx=0x7f4a4800b330 comp rx=0 tx=0).stop 2023-06-09T11:24:11.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 -- 172.21.15.7:0/3498705574 shutdown_connections 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1f90 0x7f4a50159ef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 --2- 172.21.15.7:0/3498705574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c1bc0 0x7f4a501599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 -- 172.21.15.7:0/3498705574 >> 172.21.15.7:0/3498705574 conn(0x7f4a5006a4d0 msgr2=0x7f4a5006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 -- 172.21.15.7:0/3498705574 shutdown_connections 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.663+0000 7f4a57dff700 1 -- 172.21.15.7:0/3498705574 wait complete. 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.665+0000 7f4a57dff700 1 Processor -- start 2023-06-09T11:24:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.665+0000 7f4a57dff700 1 -- start start 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 0x7f4a500c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a500c1f90 0x7f4a501ee4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500c2ee0 con 0x7f4a500c1bc0 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a501547b0 con 0x7f4a500c1f90 2023-06-09T11:24:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a57dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a500c4b70 con 0x7f4a500c0140 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a5539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a5539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41404/0 (socket says 172.21.15.7:41404) 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.666+0000 7f4a5539a700 1 -- 172.21.15.7:0/2616905140 learned_addr learned my addr 172.21.15.7:0/2616905140 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a55b9b700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 0x7f4a500c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5639c700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a500c1f90 0x7f4a501ee4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 -- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 msgr2=0x7f4a500c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 0x7f4a500c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 -- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a500c1f90 msgr2=0x7f4a501ee4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a500c1f90 0x7f4a501ee4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 -- 172.21.15.7:0/2616905140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48009b30 con 0x7f4a500c1bc0 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a55b9b700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 0x7f4a500c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a5539a700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a400011e0 tx=0x7f4a4000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a400054e0 con 0x7f4a500c1bc0 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40003680 con 0x7f4a500c1bc0 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40003a30 con 0x7f4a500c1bc0 2023-06-09T11:24:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.667+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a500c1170 con 0x7f4a500c1bc0 2023-06-09T11:24:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.668+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a500c0e90 con 0x7f4a500c1bc0 2023-06-09T11:24:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.669+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a500c1450 con 0x7f4a500c1bc0 2023-06-09T11:24:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.670+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a40003b90 con 0x7f4a500c1bc0 2023-06-09T11:24:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.671+0000 7f4a467fc700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.671+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f4a40088c60 con 0x7f4a500c1bc0 2023-06-09T11:24:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.671+0000 7f4a55b9b700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.672+0000 7f4a55b9b700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 0x7f4a3406d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009c30 tx=0x7f4a4c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.676+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4a40053a30 con 0x7f4a500c1bc0 2023-06-09T11:24:11.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.927+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a50074cf0 con 0x7f4a500c1bc0 2023-06-09T11:24:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.928+0000 7f4a467fc700 1 -- 172.21.15.7:0/2616905140 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f4a40017090 con 0x7f4a500c1bc0 2023-06-09T11:24:11.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:11.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:11.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 msgr2=0x7f4a3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 0x7f4a3406d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009c30 tx=0x7f4a4c008040 comp rx=0 tx=0).stop 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 msgr2=0x7f4a5014bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4a400011e0 tx=0x7f4a4000ca30 comp rx=0 tx=0).stop 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 shutdown_connections 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a500c1f90 0x7f4a501ee4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a3406ace0 0x7f4a3406d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a500c0140 0x7f4a500c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 --2- 172.21.15.7:0/2616905140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a500c1bc0 0x7f4a5014bf00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:11.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 >> 172.21.15.7:0/2616905140 conn(0x7f4a5006a4d0 msgr2=0x7f4a500c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:11.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.930+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 shutdown_connections 2023-06-09T11:24:11.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:11.931+0000 7f4a57dff700 1 -- 172.21.15.7:0/2616905140 wait complete. 2023-06-09T11:24:11.937 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:11.953 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:13.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:13.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:14.004 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.163+0000 7ff96bbff700 1 Processor -- start 2023-06-09T11:24:14.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.164+0000 7ff96bbff700 1 -- start start 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96bbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 0x7ff9640b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96bbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff96415cd40 0x7ff9640cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96bbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff964159b20 con 0x7ff9640cb5d0 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96bbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9640cac80 con 0x7ff96415cd40 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96bbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9641597d0 con 0x7ff9640cb9a0 2023-06-09T11:24:14.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96999b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff9640cb5d0 0x7ff964159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43578/0 (socket says 172.21.15.7:43578) 2023-06-09T11:24:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96999b700 1 -- 172.21.15.7:0/278076893 learned_addr learned my addr 172.21.15.7:0/278076893 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96a19c700 1 --2- 172.21.15.7:0/278076893 >> v2:172.21.15.7:3300/0 conn(0x7ff96415cd40 0x7ff9640cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff96919a700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 0x7ff9640b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff968999700 1 -- 172.21.15.7:0/278076893 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139688503 0 0) 0x7ff964159b20 con 0x7ff9640cb5d0 2023-06-09T11:24:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.165+0000 7ff968999700 1 -- 172.21.15.7:0/278076893 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9500035b0 con 0x7ff9640cb5d0 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96a19c700 1 -- 172.21.15.7:0/278076893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 msgr2=0x7ff9640b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96a19c700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 0x7ff9640b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96a19c700 1 -- 172.21.15.7:0/278076893 >> v1:172.21.15.7:6789/0 conn(0x7ff9640cb5d0 legacy=0x7ff964159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96a19c700 1 -- 172.21.15.7:0/278076893 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964071ab0 con 0x7ff96415cd40 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96919a700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 0x7ff9640b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff96a19c700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff9640cbea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009d90 tx=0x7ff95c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1988fd067dccd4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff968999700 1 -- 172.21.15.7:0/278076893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00e070 con 0x7ff96415cd40 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff968999700 1 -- 172.21.15.7:0/278076893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c00bb20 con 0x7ff96415cd40 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.166+0000 7ff968999700 1 -- 172.21.15.7:0/278076893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c012430 con 0x7ff96415cd40 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 -- 172.21.15.7:0/278076893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 msgr2=0x7ff9640cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff9640cbea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009d90 tx=0x7ff95c00b330 comp rx=0 tx=0).stop 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 -- 172.21.15.7:0/278076893 shutdown_connections 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff9640cbea0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 --2- 172.21.15.7:0/278076893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb9a0 0x7ff9640b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 -- 172.21.15.7:0/278076893 >> 172.21.15.7:0/278076893 conn(0x7ff964070af0 msgr2=0x7ff964070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 -- 172.21.15.7:0/278076893 shutdown_connections 2023-06-09T11:24:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.167+0000 7ff96bbff700 1 -- 172.21.15.7:0/278076893 wait complete. 2023-06-09T11:24:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.168+0000 7ff96bbff700 1 Processor -- start 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 -- start start 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb5d0 0x7ff9640cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9640cb9a0 0x7ff9640cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640cf8b0 con 0x7ff96415cd40 2023-06-09T11:24:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff964150740 con 0x7ff9640cb9a0 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.169+0000 7ff96bbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9640d0550 con 0x7ff9640cb5d0 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39664/0 (socket says 172.21.15.7:39664) 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 -- 172.21.15.7:0/1157498291 learned_addr learned my addr 172.21.15.7:0/1157498291 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96999b700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb5d0 0x7ff9640cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96919a700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9640cb9a0 0x7ff9640cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 -- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb5d0 msgr2=0x7ff9640cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb5d0 0x7ff9640cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 -- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9640cb9a0 msgr2=0x7ff9640cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9640cb9a0 0x7ff9640cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.170+0000 7ff96a19c700 1 -- 172.21.15.7:0/1157498291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c009a70 con 0x7ff96415cd40 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff96a19c700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff95c013000 tx=0x7ff95c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c012430 con 0x7ff96415cd40 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c020070 con 0x7ff96415cd40 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96415d4f0 con 0x7ff96415cd40 2023-06-09T11:24:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c003ea0 con 0x7ff96415cd40 2023-06-09T11:24:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.171+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96415d210 con 0x7ff96415cd40 2023-06-09T11:24:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.172+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96415d7d0 con 0x7ff96415cd40 2023-06-09T11:24:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.173+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff95c004040 con 0x7ff96415cd40 2023-06-09T11:24:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.174+0000 7ff95a7fc700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 0x7ff94806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.174+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7ff95c022070 con 0x7ff96415cd40 2023-06-09T11:24:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.174+0000 7ff96999b700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 0x7ff94806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.175+0000 7ff96999b700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 0x7ff94806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff960007fd0 tx=0x7ff960007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.178+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff95c0561d0 con 0x7ff96415cd40 2023-06-09T11:24:14.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.435+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff964074cf0 con 0x7ff96415cd40 2023-06-09T11:24:14.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.436+0000 7ff95a7fc700 1 -- 172.21.15.7:0/1157498291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7ff95c0597f0 con 0x7ff96415cd40 2023-06-09T11:24:14.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:14.441 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:14.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 msgr2=0x7ff94806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 0x7ff94806d190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff960007fd0 tx=0x7ff960007dc0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 msgr2=0x7ff964152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:14.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff95c013000 tx=0x7ff95c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 shutdown_connections 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff96415cd40 0x7ff964152200 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff94806ace0 0x7ff94806d190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9640cb5d0 0x7ff9640cfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 --2- 172.21.15.7:0/1157498291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9640cb9a0 0x7ff9640cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:14.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 >> 172.21.15.7:0/1157498291 conn(0x7ff964070af0 msgr2=0x7ff964158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 shutdown_connections 2023-06-09T11:24:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:14.438+0000 7ff96bbff700 1 -- 172.21.15.7:0/1157498291 wait complete. 2023-06-09T11:24:14.445 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:14.461 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:16.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.668+0000 7f41d97ff700 1 Processor -- start 2023-06-09T11:24:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.669+0000 7f41d97ff700 1 -- start start 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d97ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 0x7f41d40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d97ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41d415cd40 0x7f41d40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d97ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d4159b20 con 0x7f41d40cb5d0 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d97ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d40cac80 con 0x7f41d415cd40 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d97ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41597d0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41d415cd40 0x7f41d40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f41d40cb5d0 0x7f41d4159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43584/0 (socket says 172.21.15.7:43584) 2023-06-09T11:24:16.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f41d415cd40 0x7f41d40cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39666/0 (socket says 172.21.15.7:39666) 2023-06-09T11:24:16.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d2ffd700 1 -- 172.21.15.7:0/2719487392 learned_addr learned my addr 172.21.15.7:0/2719487392 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:16.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d27fc700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 0x7f41d40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d37fe700 1 -- 172.21.15.7:0/2719487392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 msgr2=0x7f41d40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:16.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.670+0000 7f41d37fe700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 0x7f41d40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d37fe700 1 -- 172.21.15.7:0/2719487392 >> v1:172.21.15.7:6789/0 conn(0x7f41d40cb5d0 legacy=0x7f41d4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d37fe700 1 -- 172.21.15.7:0/2719487392 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d4071ab0 con 0x7f41d415cd40 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d27fc700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 0x7f41d40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d37fe700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d415cd40 0x7f41d40cbea0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009d90 tx=0x7f41cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8e19d0384cfcf7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d1ffb700 1 -- 172.21.15.7:0/2719487392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00e070 con 0x7f41d415cd40 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d1ffb700 1 -- 172.21.15.7:0/2719487392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc00bbd0 con 0x7f41d415cd40 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.671+0000 7f41d1ffb700 1 -- 172.21.15.7:0/2719487392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc012430 con 0x7f41d415cd40 2023-06-09T11:24:16.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 -- 172.21.15.7:0/2719487392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d415cd40 msgr2=0x7f41d40cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d415cd40 0x7f41d40cbea0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009d90 tx=0x7f41cc00b330 comp rx=0 tx=0).stop 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 -- 172.21.15.7:0/2719487392 shutdown_connections 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d415cd40 0x7f41d40cbea0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 --2- 172.21.15.7:0/2719487392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb9a0 0x7f41d40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 -- 172.21.15.7:0/2719487392 >> 172.21.15.7:0/2719487392 conn(0x7f41d4070af0 msgr2=0x7f41d4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 -- 172.21.15.7:0/2719487392 shutdown_connections 2023-06-09T11:24:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.672+0000 7f41d97ff700 1 -- 172.21.15.7:0/2719487392 wait complete. 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.674+0000 7f41d97ff700 1 Processor -- start 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.674+0000 7f41d97ff700 1 -- start start 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb5d0 0x7f41d41ef0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d415cd40 0x7f41d40ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41ef5e0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4159620 con 0x7f41d415cd40 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d97ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40c3230 con 0x7f41d40cb5d0 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39682/0 (socket says 172.21.15.7:39682) 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 -- 172.21.15.7:0/3133213256 learned_addr learned my addr 172.21.15.7:0/3133213256 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d2ffd700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb5d0 0x7f41d41ef0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d37fe700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d415cd40 0x7f41d40ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 -- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb5d0 msgr2=0x7f41d41ef0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb5d0 0x7f41d41ef0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 -- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d415cd40 msgr2=0x7f41d40ce7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d415cd40 0x7f41d40ce7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.676+0000 7f41d27fc700 1 -- 172.21.15.7:0/3133213256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc009a70 con 0x7f41d40cb9a0 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41d27fc700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e7e0 tx=0x7f41c800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8004230 con 0x7f41d40cb9a0 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8004390 con 0x7f41d40cb9a0 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d415d4f0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80036a0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.677+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d415d210 con 0x7f41d40cb9a0 2023-06-09T11:24:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.678+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d415d7d0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.679+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41c8003800 con 0x7f41d40cb9a0 2023-06-09T11:24:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.680+0000 7f41bb7fe700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 0x7f41c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.680+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7f41c80891e0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.680+0000 7f41d2ffd700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 0x7f41c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:16.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.681+0000 7f41d2ffd700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 0x7f41c006d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005910 tx=0x7f41c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:16.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.684+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f41c8057a50 con 0x7f41d40cb9a0 2023-06-09T11:24:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.937+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41d4074cf0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.938+0000 7f41bb7fe700 1 -- 172.21.15.7:0/3133213256 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f41c80575e0 con 0x7f41d40cb9a0 2023-06-09T11:24:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:16.943 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5587,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:22:32.285048+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":55,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5012,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5012":{"gid":5012,"name":"f","rank":0,"incarnation":46,"state":"up:active","state_seq":42,"addr":"172.21.15.117:6837/3174501100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3174501100},{"type":"v1","addr":"172.21.15.117:6837","nonce":3174501100}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 msgr2=0x7f41c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 0x7f41c006d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005910 tx=0x7f41c4009040 comp rx=0 tx=0).stop 2023-06-09T11:24:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 msgr2=0x7f41d40c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f41c800e7e0 tx=0x7f41c800eaf0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 shutdown_connections 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41d415cd40 0x7f41d40ce7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41c006ace0 0x7f41c006d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41d40cb5d0 0x7f41d41ef0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 --2- 172.21.15.7:0/3133213256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41d40cb9a0 0x7f41d40c7d30 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 >> 172.21.15.7:0/3133213256 conn(0x7f41d4070af0 msgr2=0x7f41d40b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 shutdown_connections 2023-06-09T11:24:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:16.940+0000 7f41d97ff700 1 -- 172.21.15.7:0/3133213256 wait complete. 2023-06-09T11:24:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 60 2023-06-09T11:24:16.962 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:24:18.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.168+0000 7fe011bff700 1 Processor -- start 2023-06-09T11:24:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.169+0000 7fe011bff700 1 -- start start 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 0x7fe00c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe00c071910 0x7fe00c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0714f0 con 0x7fe00c071910 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c0cc140 con 0x7fe00c1593d0 2023-06-09T11:24:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe011bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0c1830 con 0x7fe00c0c08d0 2023-06-09T11:24:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe00bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe00c071910 0x7fe00c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe00bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe00c071910 0x7fe00c071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39698/0 (socket says 172.21.15.7:39698) 2023-06-09T11:24:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.170+0000 7fe00bfff700 1 -- 172.21.15.7:0/3952909777 learned_addr learned my addr 172.21.15.7:0/3952909777 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00b7fe700 1 --2- 172.21.15.7:0/3952909777 >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 0x7fe00c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00affd700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 -- 172.21.15.7:0/3952909777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 msgr2=0x7fe00c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 -- 172.21.15.7:0/3952909777 >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 msgr2=0x7fe00c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 --2- 172.21.15.7:0/3952909777 >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 0x7fe00c1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 -- 172.21.15.7:0/3952909777 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0c0560 con 0x7fe00c071910 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00b7fe700 1 --2- 172.21.15.7:0/3952909777 >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 0x7fe00c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00affd700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00bfff700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c071d50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06ff10 tx=0x7fe00400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5505620754b98a7a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00a7fc700 1 -- 172.21.15.7:0/3952909777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400f070 con 0x7fe00c071910 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00a7fc700 1 -- 172.21.15.7:0/3952909777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00400a2e0 con 0x7fe00c071910 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.171+0000 7fe00a7fc700 1 -- 172.21.15.7:0/3952909777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00400ccc0 con 0x7fe00c071910 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 -- 172.21.15.7:0/3952909777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 msgr2=0x7fe00c071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c071d50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe00c06ff10 tx=0x7fe00400c330 comp rx=0 tx=0).stop 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 -- 172.21.15.7:0/3952909777 shutdown_connections 2023-06-09T11:24:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 --2- 172.21.15.7:0/3952909777 >> v2:172.21.15.117:3300/0 conn(0x7fe00c1593d0 0x7fe00c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c071d50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 --2- 172.21.15.7:0/3952909777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 -- 172.21.15.7:0/3952909777 >> 172.21.15.7:0/3952909777 conn(0x7fe00c06a4d0 msgr2=0x7fe00c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 -- 172.21.15.7:0/3952909777 shutdown_connections 2023-06-09T11:24:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.172+0000 7fe011bff700 1 -- 172.21.15.7:0/3952909777 wait complete. 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.174+0000 7fe011bff700 1 Processor -- start 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.174+0000 7fe011bff700 1 -- start start 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c1ed070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 0x7fe00c0ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1f0090 con 0x7fe00c071910 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c1578c0 con 0x7fe00c1593d0 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe011bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c157160 con 0x7fe00c0c08d0 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39704/0 (socket says 172.21.15.7:39704) 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 -- 172.21.15.7:0/1268557422 learned_addr learned my addr 172.21.15.7:0/1268557422 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00affd700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c1ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00bfff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 0x7fe00c0ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 -- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 msgr2=0x7fe00c1ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c1ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 -- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 msgr2=0x7fe00c0ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 0x7fe00c0ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00b7fe700 1 -- 172.21.15.7:0/1268557422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00400ac40 con 0x7fe00c071910 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00bfff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 0x7fe00c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.175+0000 7fe00affd700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c1ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fe00b7fe700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c780 tx=0x7fdffc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc011800 con 0x7fe00c071910 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc011960 con 0x7fe00c071910 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c0c1a80 con 0x7fe00c071910 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc011cd0 con 0x7fe00c071910 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.176+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c072290 con 0x7fe00c071910 2023-06-09T11:24:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.177+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00c0c1d60 con 0x7fe00c071910 2023-06-09T11:24:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.178+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdffc021350 con 0x7fe00c071910 2023-06-09T11:24:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.179+0000 7fdff3fff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 0x7fdff806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.179+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6636+0+0 (secure 0 0 0) 0x7fdffc089780 con 0x7fe00c071910 2023-06-09T11:24:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.179+0000 7fe00affd700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 0x7fdff806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.180+0000 7fe00affd700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 0x7fdff806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe00c147d30 tx=0x7fe000008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.183+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdffc057f60 con 0x7fe00c071910 2023-06-09T11:24:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.438+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe00c074cf0 con 0x7fe00c071910 2023-06-09T11:24:19.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.438+0000 7fdff3fff700 1 -- 172.21.15.7:0/1268557422 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5455 (secure 0 0 0) 0x7fdffc057af0 con 0x7fe00c071910 2023-06-09T11:24:19.444 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:19.444 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:19.295510+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 msgr2=0x7fdff806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 0x7fdff806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe00c147d30 tx=0x7fe000008040 comp rx=0 tx=0).stop 2023-06-09T11:24:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 msgr2=0x7fe00c1f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c780 tx=0x7fdffc00cb40 comp rx=0 tx=0).stop 2023-06-09T11:24:19.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 shutdown_connections 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe00c1593d0 0x7fe00c0ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe00c071910 0x7fe00c1f0520 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdff806ace0 0x7fdff806d190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 --2- 172.21.15.7:0/1268557422 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe00c0c08d0 0x7fe00c1ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 >> 172.21.15.7:0/1268557422 conn(0x7fe00c06a4d0 msgr2=0x7fe00c0c78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 shutdown_connections 2023-06-09T11:24:19.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.441+0000 7fe011bff700 1 -- 172.21.15.7:0/1268557422 wait complete. 2023-06-09T11:24:19.448 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2023-06-09T11:24:19.463 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:24:19.463 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:24:19.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.660+0000 7fe35d5ff700 1 Processor -- start 2023-06-09T11:24:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.661+0000 7fe35d5ff700 1 -- start start 2023-06-09T11:24:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3580cb5d0 0x7fe3580b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe358159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe35815cd40 0x7fe3580cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe358159b20 con 0x7fe35815cd40 2023-06-09T11:24:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580cac80 con 0x7fe3580cb5d0 2023-06-09T11:24:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe35d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3581597d0 con 0x7fe3580cb9a0 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe3577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe35815cd40 0x7fe3580cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe3577fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe35815cd40 0x7fe3580cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39708/0 (socket says 172.21.15.7:39708) 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe3577fe700 1 -- 172.21.15.7:0/1608757078 learned_addr learned my addr 172.21.15.7:0/1608757078 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe3567fc700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe358159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.662+0000 7fe356ffd700 1 --2- 172.21.15.7:0/1608757078 >> v2:172.21.15.117:3300/0 conn(0x7fe3580cb5d0 0x7fe3580b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 -- 172.21.15.7:0/1608757078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 msgr2=0x7fe358159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe358159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 -- 172.21.15.7:0/1608757078 >> v2:172.21.15.117:3300/0 conn(0x7fe3580cb5d0 msgr2=0x7fe3580b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 --2- 172.21.15.7:0/1608757078 >> v2:172.21.15.117:3300/0 conn(0x7fe3580cb5d0 0x7fe3580b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 -- 172.21.15.7:0/1608757078 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe358071ab0 con 0x7fe35815cd40 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe3577fe700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580cbea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe35808d420 tx=0x7fe35000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e15cac4b9118796 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe355ffb700 1 -- 172.21.15.7:0/1608757078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000e070 con 0x7fe35815cd40 2023-06-09T11:24:19.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe355ffb700 1 -- 172.21.15.7:0/1608757078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000b910 con 0x7fe35815cd40 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.663+0000 7fe355ffb700 1 -- 172.21.15.7:0/1608757078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35000bcc0 con 0x7fe35815cd40 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 -- 172.21.15.7:0/1608757078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 msgr2=0x7fe3580cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580cbea0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe35808d420 tx=0x7fe35000b330 comp rx=0 tx=0).stop 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 -- 172.21.15.7:0/1608757078 shutdown_connections 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/1608757078 >> v2:172.21.15.117:3300/0 conn(0x7fe3580cb5d0 0x7fe3580b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580cbea0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/1608757078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe358159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 -- 172.21.15.7:0/1608757078 >> 172.21.15.7:0/1608757078 conn(0x7fe358070af0 msgr2=0x7fe358070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:19.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 -- 172.21.15.7:0/1608757078 shutdown_connections 2023-06-09T11:24:19.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.664+0000 7fe35d5ff700 1 -- 172.21.15.7:0/1608757078 wait complete. 2023-06-09T11:24:19.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.666+0000 7fe35d5ff700 1 Processor -- start 2023-06-09T11:24:19.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.666+0000 7fe35d5ff700 1 -- start start 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580cb5d0 0x7fe3580c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe3581528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c47f0 con 0x7fe35815cd40 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c6050 con 0x7fe3580cb5d0 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe35d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c8e20 con 0x7fe3580cb9a0 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe356ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580cb5d0 0x7fe3580c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39714/0 (socket says 172.21.15.7:39714) 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3567fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe3581528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 -- 172.21.15.7:0/3746327222 learned_addr learned my addr 172.21.15.7:0/3746327222 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 -- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 msgr2=0x7fe3581528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe3581528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 -- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580cb5d0 msgr2=0x7fe3580c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580cb5d0 0x7fe3580c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.668+0000 7fe3577fe700 1 -- 172.21.15.7:0/3746327222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350009b30 con 0x7fe35815cd40 2023-06-09T11:24:19.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe3577fe700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe350012000 tx=0x7fe350004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500092e0 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35000ce80 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35815d4f0 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3500044b0 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.669+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35815d210 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.670+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35815d7d0 con 0x7fe35815cd40 2023-06-09T11:24:19.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.671+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe350004610 con 0x7fe35815cd40 2023-06-09T11:24:19.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.672+0000 7fe33f7fe700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 0x7fe34406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:19.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.672+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fe35008abe0 con 0x7fe35815cd40 2023-06-09T11:24:19.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.672+0000 7fe356ffd700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 0x7fe34406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:19.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.673+0000 7fe356ffd700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 0x7fe34406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe348009ac0 tx=0x7fe348009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:19.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.676+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe350059350 con 0x7fe35815cd40 2023-06-09T11:24:19.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.928+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe358074cf0 con 0x7fe35815cd40 2023-06-09T11:24:19.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.929+0000 7fe33f7fe700 1 -- 172.21.15.7:0/3746327222 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5455 (secure 0 0 0) 0x7fe35002a090 con 0x7fe35815cd40 2023-06-09T11:24:19.934 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:19.934 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:19.295510+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 msgr2=0x7fe34406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 0x7fe34406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe348009ac0 tx=0x7fe348009210 comp rx=0 tx=0).stop 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 msgr2=0x7fe3580c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe350012000 tx=0x7fe350004ad0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 shutdown_connections 2023-06-09T11:24:19.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe35815cd40 0x7fe3580c7990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3580cb5d0 0x7fe3580c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe34406ace0 0x7fe34406d190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 --2- 172.21.15.7:0/3746327222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3580cb9a0 0x7fe3581528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.931+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 >> 172.21.15.7:0/3746327222 conn(0x7fe358070af0 msgr2=0x7fe3580cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.932+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 shutdown_connections 2023-06-09T11:24:19.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:19.932+0000 7fe35d5ff700 1 -- 172.21.15.7:0/3746327222 wait complete. 2023-06-09T11:24:19.939 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2023-06-09T11:24:19.955 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 61, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 61, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 }, "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:24:19.295510+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5071, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:24:21.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:22.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.163+0000 7f96acdff700 1 Processor -- start 2023-06-09T11:24:22.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.164+0000 7f96acdff700 1 -- start start 2023-06-09T11:24:22.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96acdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 0x7f96a815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96acdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96a806aa80 0x7f96a806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96acdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96a80b51c0 con 0x7f96a8156960 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96acdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a80b4180 con 0x7f96a806aa80 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96acdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a80b54a0 con 0x7f96a806a6b0 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f969ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f96a806aa80 0x7f96a806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:22.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96a6d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96a8156960 0x7f96a80b2a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43592/0 (socket says 172.21.15.7:43592) 2023-06-09T11:24:22.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96a6d9d700 1 -- 172.21.15.7:0/823704043 learned_addr learned my addr 172.21.15.7:0/823704043 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96a659c700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 0x7f96a815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f969ffff700 1 -- 172.21.15.7:0/823704043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 msgr2=0x7f96a815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f969ffff700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 0x7f96a815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f969ffff700 1 -- 172.21.15.7:0/823704043 >> v1:172.21.15.7:6789/0 conn(0x7f96a8156960 legacy=0x7f96a80b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f969ffff700 1 -- 172.21.15.7:0/823704043 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80c9f60 con 0x7f96a806aa80 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.165+0000 7f96a5d9b700 1 -- 172.21.15.7:0/823704043 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613825298 0 0) 0x7f96a80b51c0 con 0x7f96a8156960 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.166+0000 7f96a659c700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 0x7f96a815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.166+0000 7f969ffff700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806aa80 0x7f96a806aed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9690009820 tx=0x7f9690009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c69d27fdd4eb7cc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.166+0000 7f96a5d9b700 1 -- 172.21.15.7:0/823704043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690005e30 con 0x7f96a806aa80 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.166+0000 7f96a5d9b700 1 -- 172.21.15.7:0/823704043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9690006030 con 0x7f96a806aa80 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.166+0000 7f96a5d9b700 1 -- 172.21.15.7:0/823704043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900063e0 con 0x7f96a806aa80 2023-06-09T11:24:22.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 -- 172.21.15.7:0/823704043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806aa80 msgr2=0x7f96a806aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806aa80 0x7f96a806aed0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9690009820 tx=0x7f9690009b30 comp rx=0 tx=0).stop 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 -- 172.21.15.7:0/823704043 shutdown_connections 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a806a6b0 0x7f96a815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 --2- 172.21.15.7:0/823704043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806aa80 0x7f96a806aed0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 -- 172.21.15.7:0/823704043 >> 172.21.15.7:0/823704043 conn(0x7f96a8070af0 msgr2=0x7f96a8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 -- 172.21.15.7:0/823704043 shutdown_connections 2023-06-09T11:24:22.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.167+0000 7f96acdff700 1 -- 172.21.15.7:0/823704043 wait complete. 2023-06-09T11:24:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.169+0000 7f96acdff700 1 Processor -- start 2023-06-09T11:24:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.169+0000 7f96acdff700 1 -- start start 2023-06-09T11:24:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a806aa80 0x7f96a8154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a8156960 0x7f96a8154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a815c0a0 con 0x7f96a806a6b0 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a81563f0 con 0x7f96a806aa80 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96acdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a8155e50 con 0x7f96a8156960 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39738/0 (socket says 172.21.15.7:39738) 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 -- 172.21.15.7:0/2252197493 learned_addr learned my addr 172.21.15.7:0/2252197493 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 -- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a8156960 msgr2=0x7f96a8154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a8156960 0x7f96a8154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 -- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a806aa80 msgr2=0x7f96a8154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a806aa80 0x7f96a8154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.170+0000 7f96a659c700 1 -- 172.21.15.7:0/2252197493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009430 con 0x7f96a806a6b0 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f96a659c700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f969800c750 tx=0x7f969800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698011800 con 0x7f96a806a6b0 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9698011960 con 0x7f96a806a6b0 2023-06-09T11:24:22.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a80ca520 con 0x7f96a806a6b0 2023-06-09T11:24:22.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698011cd0 con 0x7f96a806a6b0 2023-06-09T11:24:22.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.171+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a80ca240 con 0x7f96a806a6b0 2023-06-09T11:24:22.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.172+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a80b2570 con 0x7f96a806a6b0 2023-06-09T11:24:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.173+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9698021350 con 0x7f96a806a6b0 2023-06-09T11:24:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.174+0000 7f969effd700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 0x7f968806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.174+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f96980897c0 con 0x7f96a806a6b0 2023-06-09T11:24:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.174+0000 7f969ffff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 0x7f968806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:22.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.174+0000 7f969ffff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 0x7f968806d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9690009820 tx=0x7f9690010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:22.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.178+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9698054530 con 0x7f96a806a6b0 2023-06-09T11:24:22.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.432+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96a8074cf0 con 0x7f96a806a6b0 2023-06-09T11:24:22.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.433+0000 7f969effd700 1 -- 172.21.15.7:0/2252197493 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f9698057b50 con 0x7f96a806a6b0 2023-06-09T11:24:22.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:22.438 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:19.295510+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.435+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 msgr2=0x7f968806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.435+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 0x7f968806d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9690009820 tx=0x7f9690010000 comp rx=0 tx=0).stop 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.435+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 msgr2=0x7f96a815a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.435+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f969800c750 tx=0x7f969800cb10 comp rx=0 tx=0).stop 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 shutdown_connections 2023-06-09T11:24:22.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968806ace0 0x7f968806d190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f96a8156960 0x7f96a8154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96a806a6b0 0x7f96a815a980 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 --2- 172.21.15.7:0/2252197493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f96a806aa80 0x7f96a8154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 >> 172.21.15.7:0/2252197493 conn(0x7f96a8070af0 msgr2=0x7f96a80cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 shutdown_connections 2023-06-09T11:24:22.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:22.436+0000 7f96acdff700 1 -- 172.21.15.7:0/2252197493 wait complete. 2023-06-09T11:24:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2023-06-09T11:24:22.459 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:24.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:24.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.660+0000 7ffa6adff700 1 Processor -- start 2023-06-09T11:24:24.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.661+0000 7ffa6adff700 1 -- start start 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 0x7ffa6415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6adff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffa640ca580 0x7ffa640c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6adff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6415dcf0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6adff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa640c5e00 con 0x7ffa640ca580 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6415d450 con 0x7ffa640b5360 2023-06-09T11:24:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa68b9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffa640ca1b0 0x7ffa641536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43862/0 (socket says 172.21.15.7:43862) 2023-06-09T11:24:24.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa68b9b700 1 -- 172.21.15.7:0/2914327603 learned_addr learned my addr 172.21.15.7:0/2914327603 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa63fff700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 0x7ffa6415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.662+0000 7ffa6939c700 1 --2- 172.21.15.7:0/2914327603 >> v2:172.21.15.117:3300/0 conn(0x7ffa640ca580 0x7ffa640c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa637fe700 1 -- 172.21.15.7:0/2914327603 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038477456 0 0) 0x7ffa6415dcf0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa637fe700 1 -- 172.21.15.7:0/2914327603 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa540035b0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa6939c700 1 -- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 msgr2=0x7ffa6415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa6939c700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 0x7ffa6415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa6939c700 1 -- 172.21.15.7:0/2914327603 >> v1:172.21.15.7:6789/0 conn(0x7ffa640ca1b0 legacy=0x7ffa641536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa6939c700 1 -- 172.21.15.7:0/2914327603 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6415d730 con 0x7ffa640ca580 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa63fff700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 0x7ffa6415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa6939c700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640ca580 0x7ffa640c58c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009d90 tx=0x7ffa5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf91f443b03249d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.663+0000 7ffa637fe700 1 -- 172.21.15.7:0/2914327603 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00e070 con 0x7ffa640ca580 2023-06-09T11:24:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa637fe700 1 -- 172.21.15.7:0/2914327603 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5c00bb20 con 0x7ffa640ca580 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa637fe700 1 -- 172.21.15.7:0/2914327603 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00be90 con 0x7ffa640ca580 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 -- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640ca580 msgr2=0x7ffa640c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640ca580 0x7ffa640c58c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c009d90 tx=0x7ffa5c00b330 comp rx=0 tx=0).stop 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 -- 172.21.15.7:0/2914327603 shutdown_connections 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640ca580 0x7ffa640c58c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2914327603 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640b5360 0x7ffa6415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 -- 172.21.15.7:0/2914327603 >> 172.21.15.7:0/2914327603 conn(0x7ffa64072090 msgr2=0x7ffa64072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:24.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 -- 172.21.15.7:0/2914327603 shutdown_connections 2023-06-09T11:24:24.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.664+0000 7ffa6adff700 1 -- 172.21.15.7:0/2914327603 wait complete. 2023-06-09T11:24:24.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.667+0000 7ffa6adff700 1 Processor -- start 2023-06-09T11:24:24.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.667+0000 7ffa6adff700 1 -- start start 2023-06-09T11:24:24.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640b5360 0x7ffa6415c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 0x7ffa641f00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 0x7ffa641f0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640d0210 con 0x7ffa640ca580 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa64066900 con 0x7ffa640b5360 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa640cf2d0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa68b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640b5360 0x7ffa6415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 0x7ffa641f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa63fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 0x7ffa641f00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 0x7ffa641f0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48756/0 (socket says 172.21.15.7:48756) 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.668+0000 7ffa6939c700 1 -- 172.21.15.7:0/2389320493 learned_addr learned my addr 172.21.15.7:0/2389320493 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 -- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640b5360 msgr2=0x7ffa6415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640b5360 0x7ffa6415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 -- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 msgr2=0x7ffa641f0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 0x7ffa641f0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 -- 172.21.15.7:0/2389320493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa500094a0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa63fff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 0x7ffa641f00e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e980 tx=0x7ffa5000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000c690 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5000c7f0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c009a70 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.669+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5000cba0 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.670+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6406a570 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.671+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6415da10 con 0x7ffa640ca1b0 2023-06-09T11:24:24.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.672+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffa5000cd00 con 0x7ffa640ca1b0 2023-06-09T11:24:24.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.673+0000 7ffa617fa700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 0x7ffa4806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:24.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.673+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7ffa50089b50 con 0x7ffa640ca1b0 2023-06-09T11:24:24.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.673+0000 7ffa68b9b700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 0x7ffa4806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:24.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.674+0000 7ffa68b9b700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 0x7ffa4806d1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa58007fd0 tx=0x7ffa58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:24.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.678+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffa50054830 con 0x7ffa640ca1b0 2023-06-09T11:24:24.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.929+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa6406a850 con 0x7ffa640ca1b0 2023-06-09T11:24:24.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.930+0000 7ffa617fa700 1 -- 172.21.15.7:0/2389320493 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+5455 (secure 0 0 0) 0x7ffa50017090 con 0x7ffa640ca1b0 2023-06-09T11:24:24.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:24.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:19.295510+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:24.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.932+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 msgr2=0x7ffa4806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.932+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 0x7ffa4806d1c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffa58007fd0 tx=0x7ffa58009040 comp rx=0 tx=0).stop 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.932+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 msgr2=0x7ffa641f00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.932+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 0x7ffa641f00e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000e980 tx=0x7ffa5000ec90 comp rx=0 tx=0).stop 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 shutdown_connections 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa640ca580 0x7ffa641f0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffa640b5360 0x7ffa6415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffa4806ad10 0x7ffa4806d1c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 --2- 172.21.15.7:0/2389320493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffa640ca1b0 0x7ffa641f00e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:24.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 >> 172.21.15.7:0/2389320493 conn(0x7ffa64072090 msgr2=0x7ffa6414f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:24.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 shutdown_connections 2023-06-09T11:24:24.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:24.933+0000 7ffa6adff700 1 -- 172.21.15.7:0/2389320493 wait complete. 2023-06-09T11:24:24.939 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 61 2023-06-09T11:24:24.955 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:26.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.162+0000 7fe960bff700 1 Processor -- start 2023-06-09T11:24:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.162+0000 7fe960bff700 1 -- start start 2023-06-09T11:24:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe960bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe95c15b4a0 0x7fe95c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe960bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe95c0c9f40 0x7fe95c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe960bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c15a500 con 0x7fe95c153880 2023-06-09T11:24:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe960bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c0c5f10 con 0x7fe95c0c9f40 2023-06-09T11:24:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe960bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c15da00 con 0x7fe95c15b4a0 2023-06-09T11:24:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe95ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe95c0c9f40 0x7fe95c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe95a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe95c153880 0x7fe95c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43868/0 (socket says 172.21.15.7:43868) 2023-06-09T11:24:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.163+0000 7fe95a59c700 1 -- 172.21.15.7:0/848665509 learned_addr learned my addr 172.21.15.7:0/848665509 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe953fff700 1 --2- 172.21.15.7:0/848665509 >> v2:172.21.15.117:3300/0 conn(0x7fe95c15b4a0 0x7fe95c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe95ad9d700 1 -- 172.21.15.7:0/848665509 >> v2:172.21.15.117:3300/0 conn(0x7fe95c15b4a0 msgr2=0x7fe95c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe95ad9d700 1 --2- 172.21.15.7:0/848665509 >> v2:172.21.15.117:3300/0 conn(0x7fe95c15b4a0 0x7fe95c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe95ad9d700 1 -- 172.21.15.7:0/848665509 >> v1:172.21.15.7:6789/0 conn(0x7fe95c153880 legacy=0x7fe95c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe95ad9d700 1 -- 172.21.15.7:0/848665509 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c1596e0 con 0x7fe95c0c9f40 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe959d9b700 1 -- 172.21.15.7:0/848665509 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562027324 0 0) 0x7fe94c001110 con 0x7fe95c153880 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe95ad9d700 1 --2- 172.21.15.7:0/848665509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c0c4e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cce30 tx=0x7fe95400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a7d8cf7e793b6e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe959d9b700 1 -- 172.21.15.7:0/848665509 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400e070 con 0x7fe95c0c9f40 2023-06-09T11:24:27.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe959d9b700 1 -- 172.21.15.7:0/848665509 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400bbd0 con 0x7fe95c0c9f40 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.164+0000 7fe959d9b700 1 -- 172.21.15.7:0/848665509 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954012430 con 0x7fe95c0c9f40 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 -- 172.21.15.7:0/848665509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 msgr2=0x7fe95c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 --2- 172.21.15.7:0/848665509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c0c4e30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cce30 tx=0x7fe95400b330 comp rx=0 tx=0).stop 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 -- 172.21.15.7:0/848665509 shutdown_connections 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 --2- 172.21.15.7:0/848665509 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c0c4e30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 --2- 172.21.15.7:0/848665509 >> v2:172.21.15.117:3300/0 conn(0x7fe95c15b4a0 0x7fe95c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 -- 172.21.15.7:0/848665509 >> 172.21.15.7:0/848665509 conn(0x7fe95c0737f0 msgr2=0x7fe95c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:27.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 -- 172.21.15.7:0/848665509 shutdown_connections 2023-06-09T11:24:27.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.165+0000 7fe960bff700 1 -- 172.21.15.7:0/848665509 wait complete. 2023-06-09T11:24:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.167+0000 7fe960bff700 1 Processor -- start 2023-06-09T11:24:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.167+0000 7fe960bff700 1 -- start start 2023-06-09T11:24:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c153880 0x7fe95c0b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c15b4a0 0x7fe95c158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c150d30 con 0x7fe95c0c9f40 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c150510 con 0x7fe95c153880 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe960bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c1507b0 con 0x7fe95c15b4a0 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48778/0 (socket says 172.21.15.7:48778) 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 -- 172.21.15.7:0/2355662660 learned_addr learned my addr 172.21.15.7:0/2355662660 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe953fff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c153880 0x7fe95c0b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95ad9d700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c15b4a0 0x7fe95c158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 -- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c15b4a0 msgr2=0x7fe95c158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c15b4a0 0x7fe95c158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 -- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c153880 msgr2=0x7fe95c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c153880 0x7fe95c0b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.168+0000 7fe95a59c700 1 -- 172.21.15.7:0/2355662660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe954009a70 con 0x7fe95c0c9f40 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe95a59c700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00c730 tx=0x7fe94c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c004c80 con 0x7fe95c0c9f40 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe94c004e00 con 0x7fe95c0c9f40 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c159c60 con 0x7fe95c0c9f40 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94c007310 con 0x7fe95c0c9f40 2023-06-09T11:24:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.169+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c159980 con 0x7fe95c0c9f40 2023-06-09T11:24:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.170+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c159f40 con 0x7fe95c0c9f40 2023-06-09T11:24:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.171+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe94c007470 con 0x7fe95c0c9f40 2023-06-09T11:24:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.172+0000 7fe952ffd700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 0x7fe93c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.172+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fe94c089290 con 0x7fe95c0c9f40 2023-06-09T11:24:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.172+0000 7fe953fff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 0x7fe93c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.173+0000 7fe953fff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 0x7fe93c06d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cad20 tx=0x7fe944008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.176+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe94c054000 con 0x7fe95c0c9f40 2023-06-09T11:24:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.432+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe95c15a500 con 0x7fe95c0c9f40 2023-06-09T11:24:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.432+0000 7fe952ffd700 1 -- 172.21.15.7:0/2355662660 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fe95c15a500 con 0x7fe95c0c9f40 2023-06-09T11:24:27.437 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:27.437 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:26.304289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:resolve","state_seq":30,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.434+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 msgr2=0x7fe93c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.434+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 0x7fe93c06d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0cad20 tx=0x7fe944008040 comp rx=0 tx=0).stop 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.434+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 msgr2=0x7fe95c1f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.434+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00c730 tx=0x7fe94c00ca40 comp rx=0 tx=0).stop 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 shutdown_connections 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe93c06ace0 0x7fe93c06d190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe95c15b4a0 0x7fe95c158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe95c0c9f40 0x7fe95c1f3120 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 --2- 172.21.15.7:0/2355662660 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe95c153880 0x7fe95c0b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 >> 172.21.15.7:0/2355662660 conn(0x7fe95c0737f0 msgr2=0x7fe95c0661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 shutdown_connections 2023-06-09T11:24:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:27.435+0000 7fe960bff700 1 -- 172.21.15.7:0/2355662660 wait complete. 2023-06-09T11:24:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 62 2023-06-09T11:24:27.457 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:29.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.663+0000 7fbe31bff700 1 Processor -- start 2023-06-09T11:24:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.663+0000 7fbe31bff700 1 -- start start 2023-06-09T11:24:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe31bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe2c1593d0 0x7fbe2c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe31bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2c0cc140 con 0x7fbe2c072020 2023-06-09T11:24:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe31bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe2c0c1830 con 0x7fbe2c0c08d0 2023-06-09T11:24:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe31bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2c0714f0 con 0x7fbe2c1593d0 2023-06-09T11:24:29.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe2bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbe2c072020 0x7fbe2c14f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43870/0 (socket says 172.21.15.7:43870) 2023-06-09T11:24:29.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.664+0000 7fbe2bfff700 1 -- 172.21.15.7:0/3688886113 learned_addr learned my addr 172.21.15.7:0/3688886113 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/3688886113 >> v2:172.21.15.117:3300/0 conn(0x7fbe2c1593d0 0x7fbe2c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514705983 0 0) 0x7fbe2c0cc140 con 0x7fbe2c072020 2023-06-09T11:24:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe180034d0 con 0x7fbe2c072020 2023-06-09T11:24:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108180876 0 0) 0x7fbe2c0c1830 con 0x7fbe2c0c08d0 2023-06-09T11:24:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe2c0cc140 con 0x7fbe2c0c08d0 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641334622 0 0) 0x7fbe180034d0 con 0x7fbe2c072020 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 >> v2:172.21.15.117:3300/0 conn(0x7fbe2c1593d0 msgr2=0x7fbe2c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 --2- 172.21.15.7:0/3688886113 >> v2:172.21.15.117:3300/0 conn(0x7fbe2c1593d0 0x7fbe2c1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 >> v1:172.21.15.117:6789/0 conn(0x7fbe2c0c08d0 legacy=0x7fbe2c071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0c0560 con 0x7fbe2c072020 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fbe240019f0 con 0x7fbe2c072020 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/3688886113 >> v2:172.21.15.117:3300/0 conn(0x7fbe2c1593d0 0x7fbe2c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:24:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 >> v1:172.21.15.7:6789/0 conn(0x7fbe2c072020 legacy=0x7fbe2c14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18003c80 0x7fbe18004070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 0x7fbe1800ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe1800b580 0x7fbe1800da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.665+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c0714f0 con 0x7fbe180089f0 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1800e310 con 0x7fbe18003c80 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1800e4d0 con 0x7fbe1800b580 2023-06-09T11:24:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 0x7fbe1800ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18003c80 0x7fbe18004070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe2bfff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe1800b580 0x7fbe1800da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 -- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe1800b580 msgr2=0x7fbe1800da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe1800b580 0x7fbe1800da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 -- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18003c80 msgr2=0x7fbe18004070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18003c80 0x7fbe18004070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 -- 172.21.15.7:0/3688886113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180034d0 con 0x7fbe180089f0 2023-06-09T11:24:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe2bfff700 1 -- 172.21.15.7:0/3688886113 reap_dead start 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.666+0000 7fbe23fff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 0x7fbe1800ae70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe14001940 tx=0x7fbe18003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b23de60f360439 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14001430 con 0x7fbe180089f0 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14001d70 con 0x7fbe180089f0 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe2affd700 1 -- 172.21.15.7:0/3688886113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14005530 con 0x7fbe180089f0 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 -- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 msgr2=0x7fbe1800ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 0x7fbe1800ae70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbe14001940 tx=0x7fbe18003780 comp rx=0 tx=0).stop 2023-06-09T11:24:29.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 -- 172.21.15.7:0/3688886113 shutdown_connections 2023-06-09T11:24:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 --2- 172.21.15.7:0/3688886113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180089f0 0x7fbe1800ae70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 -- 172.21.15.7:0/3688886113 >> 172.21.15.7:0/3688886113 conn(0x7fbe2c06a4d0 msgr2=0x7fbe2c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 -- 172.21.15.7:0/3688886113 shutdown_connections 2023-06-09T11:24:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.667+0000 7fbe31bff700 1 -- 172.21.15.7:0/3688886113 wait complete. 2023-06-09T11:24:29.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.669+0000 7fbe31bff700 1 Processor -- start 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.669+0000 7fbe31bff700 1 -- start start 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180089f0 0x7fbe2c1f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe2c0c6fa0 0x7fbe2c14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c1f2be0 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c14a0a0 con 0x7fbe2c0c6fa0 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe31bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2c1ef360 con 0x7fbe180089f0 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48796/0 (socket says 172.21.15.7:48796) 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe2bfff700 1 -- 172.21.15.7:0/1476035924 learned_addr learned my addr 172.21.15.7:0/1476035924 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180089f0 0x7fbe2c1f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.670+0000 7fbe23fff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe2c0c6fa0 0x7fbe2c14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 -- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180089f0 msgr2=0x7fbe2c1f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180089f0 0x7fbe2c1f26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 -- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe2c0c6fa0 msgr2=0x7fbe2c14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe2c0c6fa0 0x7fbe2c14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 -- 172.21.15.7:0/1476035924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0c1d60 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe2bfff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000f80 tx=0x7fbe2400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400ebe0 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2400ed40 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe2c0c1a80 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24013660 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.671+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe2c0c2040 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.672+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2c074cf0 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.673+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe240137c0 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.674+0000 7fbe28ff9700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 0x7fbe0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:29.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.674+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fbe24089c10 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.675+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 0x7fbe0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:29.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.675+0000 7fbe2b7fe700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 0x7fbe0c06d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c064570 tx=0x7fbe1c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:29.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.679+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbe24054a10 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.931+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe2c074cf0 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.932+0000 7fbe28ff9700 1 -- 172.21.15.7:0/1476035924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fbe24058030 con 0x7fbe2c1ee9b0 2023-06-09T11:24:29.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:29.938 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":62,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:26.304289+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:resolve","state_seq":30,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 msgr2=0x7fbe0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 0x7fbe0c06d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c064570 tx=0x7fbe1c009210 comp rx=0 tx=0).stop 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 msgr2=0x7fbe2c1eee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000f80 tx=0x7fbe2400e440 comp rx=0 tx=0).stop 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 shutdown_connections 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2c1ee9b0 0x7fbe2c1eee20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0c06ace0 0x7fbe0c06d190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180089f0 0x7fbe2c1f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 --2- 172.21.15.7:0/1476035924 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe2c0c6fa0 0x7fbe2c14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:29.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 >> 172.21.15.7:0/1476035924 conn(0x7fbe2c06a4d0 msgr2=0x7fbe2c156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:29.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.934+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 shutdown_connections 2023-06-09T11:24:29.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:29.935+0000 7fbe31bff700 1 -- 172.21.15.7:0/1476035924 wait complete. 2023-06-09T11:24:29.942 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 62 2023-06-09T11:24:29.958 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:31.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.164+0000 7f00c0dff700 1 Processor -- start 2023-06-09T11:24:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.165+0000 7f00c0dff700 1 -- start start 2023-06-09T11:24:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00c0dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f00bc0c9090 0x7f00bc0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00c0dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 0x7f00bc0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00c0dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00bc15cec0 con 0x7f00bc0cfaf0 2023-06-09T11:24:32.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00c0dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00bc15d160 con 0x7f00bc0c9090 2023-06-09T11:24:32.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00c0dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc15d440 con 0x7f00bc0cad00 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00bad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00bc0cfaf0 0x7f00bc1598b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43876/0 (socket says 172.21.15.7:43876) 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00bad9d700 1 -- 172.21.15.7:0/2142150182 learned_addr learned my addr 172.21.15.7:0/2142150182 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00ba59c700 1 --2- 172.21.15.7:0/2142150182 >> v2:172.21.15.117:3300/0 conn(0x7f00bc0c9090 0x7f00bc0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.166+0000 7f00b3fff700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 0x7f00bc0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b9d9b700 1 -- 172.21.15.7:0/2142150182 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860235856 0 0) 0x7f00bc15cec0 con 0x7f00bc0cfaf0 2023-06-09T11:24:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b9d9b700 1 -- 172.21.15.7:0/2142150182 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00a80035b0 con 0x7f00bc0cfaf0 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00ba59c700 1 -- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 msgr2=0x7f00bc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00ba59c700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 0x7f00bc0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00ba59c700 1 -- 172.21.15.7:0/2142150182 >> v1:172.21.15.7:6789/0 conn(0x7f00bc0cfaf0 legacy=0x7f00bc1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00ba59c700 1 -- 172.21.15.7:0/2142150182 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc15d720 con 0x7f00bc0c9090 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b3fff700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 0x7f00bc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00ba59c700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c9fe0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e131408b4c62f2b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b9d9b700 1 -- 172.21.15.7:0/2142150182 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011830 con 0x7f00bc0c9090 2023-06-09T11:24:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b9d9b700 1 -- 172.21.15.7:0/2142150182 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac011990 con 0x7f00bc0c9090 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.167+0000 7f00b9d9b700 1 -- 172.21.15.7:0/2142150182 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011d20 con 0x7f00bc0c9090 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 -- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 msgr2=0x7f00bc0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c9fe0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).stop 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 -- 172.21.15.7:0/2142150182 shutdown_connections 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c9fe0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 --2- 172.21.15.7:0/2142150182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cad00 0x7f00bc0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 -- 172.21.15.7:0/2142150182 >> 172.21.15.7:0/2142150182 conn(0x7f00bc070af0 msgr2=0x7f00bc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 -- 172.21.15.7:0/2142150182 shutdown_connections 2023-06-09T11:24:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.168+0000 7f00c0dff700 1 -- 172.21.15.7:0/2142150182 wait complete. 2023-06-09T11:24:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.170+0000 7f00c0dff700 1 Processor -- start 2023-06-09T11:24:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.170+0000 7f00c0dff700 1 -- start start 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cfaf0 0x7f00bc0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c5700 con 0x7f00bc0cad00 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c5c80 con 0x7f00bc0c9090 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00c0dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc0c3190 con 0x7f00bc0cfaf0 2023-06-09T11:24:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48806/0 (socket says 172.21.15.7:48806) 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00b3fff700 1 -- 172.21.15.7:0/319117195 learned_addr learned my addr 172.21.15.7:0/319117195 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00bad9d700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cfaf0 0x7f00bc0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.171+0000 7f00ba59c700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 -- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cfaf0 msgr2=0x7f00bc0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cfaf0 0x7f00bc0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 -- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 msgr2=0x7f00bc0c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 -- 172.21.15.7:0/319117195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40094a0 con 0x7f00bc0cad00 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b3fff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ebe0 tx=0x7f00a400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4009760 con 0x7f00bc0cad00 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a400c7f0 con 0x7f00bc0cad00 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400cb80 con 0x7f00bc0cad00 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0094a0 con 0x7f00bc0cad00 2023-06-09T11:24:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.172+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc0cb130 con 0x7f00bc0cad00 2023-06-09T11:24:32.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.173+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc0ca620 con 0x7f00bc0cad00 2023-06-09T11:24:32.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.174+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00a400cce0 con 0x7f00bc0cad00 2023-06-09T11:24:32.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.175+0000 7f00b2ffd700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 0x7f009c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:32.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.175+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f00a4089a50 con 0x7f00bc0cad00 2023-06-09T11:24:32.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.176+0000 7f00ba59c700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 0x7f009c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:32.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.176+0000 7f00ba59c700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 0x7f009c06d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:32.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.180+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00a4054850 con 0x7f00bc0cad00 2023-06-09T11:24:32.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.440+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00bc0cb2c0 con 0x7f00bc0cad00 2023-06-09T11:24:32.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.441+0000 7f00b2ffd700 1 -- 172.21.15.7:0/319117195 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f00a4057e70 con 0x7f00bc0cad00 2023-06-09T11:24:32.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:32.447 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:32.303797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:32.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 msgr2=0x7f009c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 0x7f009c06d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00ac000c00 tx=0x7f00ac01c040 comp rx=0 tx=0).stop 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 msgr2=0x7f00bc1f2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ebe0 tx=0x7f00a400eef0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 shutdown_connections 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f009c06ace0 0x7f009c06d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00bc0cfaf0 0x7f00bc0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00bc0c9090 0x7f00bc0c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 --2- 172.21.15.7:0/319117195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00bc0cad00 0x7f00bc1f2290 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.443+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 >> 172.21.15.7:0/319117195 conn(0x7f00bc070af0 msgr2=0x7f00bc0c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.444+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 shutdown_connections 2023-06-09T11:24:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:32.444+0000 7f00c0dff700 1 -- 172.21.15.7:0/319117195 wait complete. 2023-06-09T11:24:32.451 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 63 2023-06-09T11:24:32.470 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 63, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 63, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5071": { "addr": "172.21.15.7:6839/2631794280", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2631794280, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2631794280, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5071, "incarnation": 27, "join_fscid": -1, "name": "e", "rank": 1, "state": "up:active", "state_seq": 31 }, "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 56, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:24:32.303797+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5071, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:24:34.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.677+0000 7fbfcfbff700 1 Processor -- start 2023-06-09T11:24:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.678+0000 7fbfcfbff700 1 -- start start 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcfbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcfbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbfc815cd40 0x7fbfc80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcfbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc8159b20 con 0x7fbfc80cb5d0 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcfbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80cac80 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcfbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc81597d0 con 0x7fbfc815cd40 2023-06-09T11:24:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36032/0 (socket says 172.21.15.7:36032) 2023-06-09T11:24:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 -- 172.21.15.7:0/553235599 learned_addr learned my addr 172.21.15.7:0/553235599 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd99b700 1 --1- 172.21.15.7:0/553235599 >> v1:172.21.15.7:6789/0 conn(0x7fbfc80cb5d0 0x7fbfc8159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58398/0 (socket says 172.21.15.7:58398) 2023-06-09T11:24:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfce19c700 1 --2- 172.21.15.7:0/553235599 >> v2:172.21.15.117:3300/0 conn(0x7fbfc815cd40 0x7fbfc80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 -- 172.21.15.7:0/553235599 >> v2:172.21.15.117:3300/0 conn(0x7fbfc815cd40 msgr2=0x7fbfc80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:34.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 --2- 172.21.15.7:0/553235599 >> v2:172.21.15.117:3300/0 conn(0x7fbfc815cd40 0x7fbfc80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 -- 172.21.15.7:0/553235599 >> v1:172.21.15.7:6789/0 conn(0x7fbfc80cb5d0 legacy=0x7fbfc8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.679+0000 7fbfcd19a700 1 -- 172.21.15.7:0/553235599 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc8071ab0 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.680+0000 7fbfcd19a700 1 --2- 172.21.15.7:0/553235599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c9b80 tx=0x7fbfb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95732b4722a751a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.680+0000 7fbfcc999700 1 -- 172.21.15.7:0/553235599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8011830 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.680+0000 7fbfcc999700 1 -- 172.21.15.7:0/553235599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8011990 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.680+0000 7fbfcc999700 1 -- 172.21.15.7:0/553235599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8011d20 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 -- 172.21.15.7:0/553235599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 msgr2=0x7fbfc80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/553235599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80c9b80 tx=0x7fbfb8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 -- 172.21.15.7:0/553235599 shutdown_connections 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/553235599 >> v2:172.21.15.117:3300/0 conn(0x7fbfc815cd40 0x7fbfc80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/553235599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc80b52b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 -- 172.21.15.7:0/553235599 >> 172.21.15.7:0/553235599 conn(0x7fbfc8070af0 msgr2=0x7fbfc8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 -- 172.21.15.7:0/553235599 shutdown_connections 2023-06-09T11:24:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.681+0000 7fbfcfbff700 1 -- 172.21.15.7:0/553235599 wait complete. 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.683+0000 7fbfcfbff700 1 Processor -- start 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.683+0000 7fbfcfbff700 1 -- start start 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 0x7fbfc80cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc815cd40 0x7fbfc8150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc8153670 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc81f2c60 con 0x7fbfc80cb5d0 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcfbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80cdaa0 con 0x7fbfc815cd40 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36046/0 (socket says 172.21.15.7:36046) 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 -- 172.21.15.7:0/3218058520 learned_addr learned my addr 172.21.15.7:0/3218058520 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfce19c700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc815cd40 0x7fbfc8150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd99b700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 0x7fbfc80cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 -- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc815cd40 msgr2=0x7fbfc8150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc815cd40 0x7fbfc8150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 -- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 msgr2=0x7fbfc80cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 0x7fbfc80cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd19a700 1 -- 172.21.15.7:0/3218058520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb8009550 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.684+0000 7fbfcd99b700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 0x7fbfc80cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfcd19a700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8000c00 tx=0x7fbfb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8004480 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb801f070 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb80038e0 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc815d4f0 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.685+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc815d210 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.686+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc815d7d0 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.687+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfb8021030 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.688+0000 7fbfbe7fc700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 0x7fbfac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.688+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fbfb808ab70 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.688+0000 7fbfcd99b700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 0x7fbfac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:34.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.689+0000 7fbfcd99b700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 0x7fbfac06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000f60 tx=0x7fbfc4007dd0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.693+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbfb8055a00 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.943+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfc8074cf0 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.943+0000 7fbfbe7fc700 1 -- 172.21.15.7:0/3218058520 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+5459 (secure 0 0 0) 0x7fbfb8015090 con 0x7fbfc80cb9a0 2023-06-09T11:24:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:34.950 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:32.303797+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:reconnect","state_seq":32,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:34.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.945+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 msgr2=0x7fbfac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.945+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 0x7fbfac06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000f60 tx=0x7fbfc4007dd0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.945+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 msgr2=0x7fbfc81f1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8000c00 tx=0x7fbfb8004750 comp rx=0 tx=0).stop 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 shutdown_connections 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbfac06ace0 0x7fbfac06d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbfc815cd40 0x7fbfc8150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbfc80cb5d0 0x7fbfc80cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 --2- 172.21.15.7:0/3218058520 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbfc80cb9a0 0x7fbfc81f1f50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 >> 172.21.15.7:0/3218058520 conn(0x7fbfc8070af0 msgr2=0x7fbfc80cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 shutdown_connections 2023-06-09T11:24:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:34.946+0000 7fbfcfbff700 1 -- 172.21.15.7:0/3218058520 wait complete. 2023-06-09T11:24:34.953 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 63 2023-06-09T11:24:34.969 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:36.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.175+0000 7f4e081ff700 1 Processor -- start 2023-06-09T11:24:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.176+0000 7f4e081ff700 1 -- start start 2023-06-09T11:24:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e081ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e00153880 0x7f4e0015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e081ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0015a500 con 0x7f4e000c9f40 2023-06-09T11:24:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e081ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0015da00 con 0x7f4e0015b4a0 2023-06-09T11:24:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e081ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e000c5f10 con 0x7f4e00153880 2023-06-09T11:24:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e0579a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e000c9f40 0x7f4e0015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58400/0 (socket says 172.21.15.7:58400) 2023-06-09T11:24:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e0579a700 1 -- 172.21.15.7:0/2361344647 learned_addr learned my addr 172.21.15.7:0/2361344647 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.177+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/2361344647 >> v2:172.21.15.117:3300/0 conn(0x7f4e00153880 0x7f4e0015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837215045 0 0) 0x7f4df0001110 con 0x7f4e000c9f40 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dec0035b0 con 0x7f4e000c9f40 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e05f9b700 1 -- 172.21.15.7:0/2361344647 >> v1:172.21.15.117:6789/0 conn(0x7f4e0015b4a0 legacy=0x7f4e000c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 11189924 0 0) 0x7f4df8001490 con 0x7f4e0015b4a0 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e05f9b700 1 -- 172.21.15.7:0/2361344647 >> v1:172.21.15.7:6789/0 conn(0x7f4e000c9f40 legacy=0x7f4e0015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e05f9b700 1 -- 172.21.15.7:0/2361344647 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e001596e0 con 0x7f4e00153880 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/2361344647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0015d4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009840 tx=0x7f4dfc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1391f50fd8985c5a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.178+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc011830 con 0x7f4e00153880 2023-06-09T11:24:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc011990 con 0x7f4e00153880 2023-06-09T11:24:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e04f99700 1 -- 172.21.15.7:0/2361344647 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc011d20 con 0x7f4e00153880 2023-06-09T11:24:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 -- 172.21.15.7:0/2361344647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 msgr2=0x7f4e0015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 --2- 172.21.15.7:0/2361344647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0015d4c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009840 tx=0x7f4dfc009b50 comp rx=0 tx=0).stop 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 -- 172.21.15.7:0/2361344647 shutdown_connections 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 --2- 172.21.15.7:0/2361344647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0015d4c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 -- 172.21.15.7:0/2361344647 >> 172.21.15.7:0/2361344647 conn(0x7f4e000737f0 msgr2=0x7f4e000720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 -- 172.21.15.7:0/2361344647 shutdown_connections 2023-06-09T11:24:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.179+0000 7f4e081ff700 1 -- 172.21.15.7:0/2361344647 wait complete. 2023-06-09T11:24:37.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.181+0000 7f4e081ff700 1 Processor -- start 2023-06-09T11:24:37.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.181+0000 7f4e081ff700 1 -- start start 2023-06-09T11:24:37.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 0x7f4e000cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:37.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:37.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000cbff0 con 0x7f4e0015b4a0 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000cc9c0 con 0x7f4e00153880 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e081ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e000cbca0 con 0x7f4e000c9f40 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e0679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e0679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36050/0 (socket says 172.21.15.7:36050) 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e0679c700 1 -- 172.21.15.7:0/3265710055 learned_addr learned my addr 172.21.15.7:0/3265710055 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e0579a700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:37.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.182+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 0x7f4e000cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 -- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 msgr2=0x7f4e000cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 0x7f4e000cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 -- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 msgr2=0x7f4e0014ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0014ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 -- 172.21.15.7:0/3265710055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc0094a0 con 0x7f4e0015b4a0 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 0x7f4e000cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e0679c700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c7a0 tx=0x7f4df800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:37.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8003e30 con 0x7f4e0015b4a0 2023-06-09T11:24:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df8005070 con 0x7f4e0015b4a0 2023-06-09T11:24:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00159c60 con 0x7f4e0015b4a0 2023-06-09T11:24:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8005420 con 0x7f4e0015b4a0 2023-06-09T11:24:37.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.183+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e00159980 con 0x7f4e0015b4a0 2023-06-09T11:24:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.184+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00159f40 con 0x7f4e0015b4a0 2023-06-09T11:24:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.185+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4df8005580 con 0x7f4e0015b4a0 2023-06-09T11:24:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.187+0000 7f4df67fc700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.187+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4df8088c90 con 0x7f4e0015b4a0 2023-06-09T11:24:37.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.187+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:37.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.188+0000 7f4e05f9b700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 0x7f4de406d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009fd0 tx=0x7f4dfc01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.191+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4df8053a00 con 0x7f4e0015b4a0 2023-06-09T11:24:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.443+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e0015da00 con 0x7f4e0015b4a0 2023-06-09T11:24:37.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.444+0000 7f4df67fc700 1 -- 172.21.15.7:0/3265710055 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f4e0015da00 con 0x7f4e0015b4a0 2023-06-09T11:24:37.449 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:37.450 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:37.324314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 msgr2=0x7f4de406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 0x7f4de406d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009fd0 tx=0x7f4dfc01c040 comp rx=0 tx=0).stop 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 msgr2=0x7f4e00148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c7a0 tx=0x7f4df800cab0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 shutdown_connections 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e0015b4a0 0x7f4e00148ef0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4de406ace0 0x7f4de406d190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e000c9f40 0x7f4e000cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 --2- 172.21.15.7:0/3265710055 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e00153880 0x7f4e0014ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 >> 172.21.15.7:0/3265710055 conn(0x7f4e000737f0 msgr2=0x7f4e0014d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 shutdown_connections 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:37.446+0000 7f4e081ff700 1 -- 172.21.15.7:0/3265710055 wait complete. 2023-06-09T11:24:37.453 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 64 2023-06-09T11:24:37.469 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:39.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.675+0000 7f4d5a9ff700 1 Processor -- start 2023-06-09T11:24:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.675+0000 7f4d5a9ff700 1 -- start start 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.676+0000 7f4d5a9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1bc0 0x7f4d541599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.676+0000 7f4d5a9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d540c1f90 0x7f4d54159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.676+0000 7f4d5a9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d540c0bf0 con 0x7f4d540c0140 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.676+0000 7f4d5a9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d54070110 con 0x7f4d540c1f90 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.676+0000 7f4d5a9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d54071300 con 0x7f4d540c1bc0 2023-06-09T11:24:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d540c1f90 0x7f4d54159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d537fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d540c0140 0x7f4d541501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58408/0 (socket says 172.21.15.7:58408) 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d537fe700 1 -- 172.21.15.7:0/1878243869 learned_addr learned my addr 172.21.15.7:0/1878243869 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d53fff700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1bc0 0x7f4d541599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 -- 172.21.15.7:0/1878243869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1bc0 msgr2=0x7f4d541599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1bc0 0x7f4d541599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 -- 172.21.15.7:0/1878243869 >> v1:172.21.15.7:6789/0 conn(0x7f4d540c0140 legacy=0x7f4d541501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 -- 172.21.15.7:0/1878243869 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d540702f0 con 0x7f4d540c1f90 2023-06-09T11:24:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d52ffd700 1 -- 172.21.15.7:0/1878243869 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993276812 0 0) 0x7f4d540c0bf0 con 0x7f4d540c0140 2023-06-09T11:24:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.677+0000 7f4d58f9c700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1f90 0x7f4d54159ef0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009d90 tx=0x7f4d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b7fe515055fc92 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d52ffd700 1 -- 172.21.15.7:0/1878243869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00e070 con 0x7f4d540c1f90 2023-06-09T11:24:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d52ffd700 1 -- 172.21.15.7:0/1878243869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c00bbd0 con 0x7f4d540c1f90 2023-06-09T11:24:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d52ffd700 1 -- 172.21.15.7:0/1878243869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c012430 con 0x7f4d540c1f90 2023-06-09T11:24:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/1878243869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1f90 msgr2=0x7f4d54159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1f90 0x7f4d54159ef0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009d90 tx=0x7f4d4c00b330 comp rx=0 tx=0).stop 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/1878243869 shutdown_connections 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1f90 0x7f4d54159ef0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/1878243869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1bc0 0x7f4d541599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/1878243869 >> 172.21.15.7:0/1878243869 conn(0x7f4d5406a4d0 msgr2=0x7f4d5406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/1878243869 shutdown_connections 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.678+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/1878243869 wait complete. 2023-06-09T11:24:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.680+0000 7f4d5a9ff700 1 Processor -- start 2023-06-09T11:24:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.680+0000 7f4d5a9ff700 1 -- start start 2023-06-09T11:24:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d540c0140 0x7f4d541e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1f90 0x7f4d54152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5414bad0 con 0x7f4d540c1bc0 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5414bd40 con 0x7f4d540c0140 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d5a9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5414c020 con 0x7f4d540c1f90 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d537fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36064/0 (socket says 172.21.15.7:36064) 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.681+0000 7f4d537fe700 1 -- 172.21.15.7:0/2760332828 learned_addr learned my addr 172.21.15.7:0/2760332828 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d58f9c700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1f90 0x7f4d54152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d53fff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d540c0140 0x7f4d541e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 -- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1f90 msgr2=0x7f4d54152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1f90 0x7f4d54152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 -- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d540c0140 msgr2=0x7f4d541e46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d540c0140 0x7f4d541e46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 -- 172.21.15.7:0/2760332828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c009a70 con 0x7f4d540c1bc0 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d537fe700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4d480011e0 tx=0x7f4d4800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48004c80 con 0x7f4d540c1bc0 2023-06-09T11:24:39.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d48004e00 con 0x7f4d540c1bc0 2023-06-09T11:24:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d540c1170 con 0x7f4d540c1bc0 2023-06-09T11:24:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48007310 con 0x7f4d540c1bc0 2023-06-09T11:24:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.682+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d540c0e90 con 0x7f4d540c1bc0 2023-06-09T11:24:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.683+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d540c1450 con 0x7f4d540c1bc0 2023-06-09T11:24:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.684+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d48007470 con 0x7f4d540c1bc0 2023-06-09T11:24:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.685+0000 7f4d50ff9700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 0x7f4d4006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.685+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f4d480895e0 con 0x7f4d540c1bc0 2023-06-09T11:24:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.685+0000 7f4d53fff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 0x7f4d4006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:39.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.686+0000 7f4d53fff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 0x7f4d4006d1c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009930 tx=0x7f4d44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.690+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4d480542c0 con 0x7f4d540c1bc0 2023-06-09T11:24:39.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.942+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d54074cf0 con 0x7f4d540c1bc0 2023-06-09T11:24:39.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.943+0000 7f4d50ff9700 1 -- 172.21.15.7:0/2760332828 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f4d480578e0 con 0x7f4d540c1bc0 2023-06-09T11:24:39.949 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:39.949 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:37.324314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:39.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 msgr2=0x7f4d4006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 0x7f4d4006d1c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009930 tx=0x7f4d44008040 comp rx=0 tx=0).stop 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 msgr2=0x7f4d54152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4d480011e0 tx=0x7f4d4800cae0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 shutdown_connections 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d4006ad10 0x7f4d4006d1c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d540c1f90 0x7f4d54152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d540c0140 0x7f4d541e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 --2- 172.21.15.7:0/2760332828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d540c1bc0 0x7f4d54152230 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 >> 172.21.15.7:0/2760332828 conn(0x7f4d5406a4d0 msgr2=0x7f4d540c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 shutdown_connections 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:39.946+0000 7f4d5a9ff700 1 -- 172.21.15.7:0/2760332828 wait complete. 2023-06-09T11:24:39.953 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 64 2023-06-09T11:24:39.969 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:41.970 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.174+0000 7f6b317ff700 1 Processor -- start 2023-06-09T11:24:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.174+0000 7f6b317ff700 1 -- start start 2023-06-09T11:24:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b317ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b317ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c1580a0 con 0x7f6b2c071060 2023-06-09T11:24:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b317ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b2c0c6180 con 0x7f6b2c151510 2023-06-09T11:24:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b317ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c0714e0 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b2b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b2c071060 0x7f6b2c158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58420/0 (socket says 172.21.15.7:58420) 2023-06-09T11:24:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b2b7fe700 1 -- 172.21.15.7:0/4227129542 learned_addr learned my addr 172.21.15.7:0/4227129542 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.176+0000 7f6b2a7fc700 1 --2- 172.21.15.7:0/4227129542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308034437 0 0) 0x7f6b2c1580a0 con 0x7f6b2c071060 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b140035b0 con 0x7f6b2c071060 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1911686793 0 0) 0x7f6b2c0c6180 con 0x7f6b2c151510 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c1580a0 con 0x7f6b2c151510 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b2a7fc700 1 -- 172.21.15.7:0/4227129542 >> v1:172.21.15.117:6789/0 conn(0x7f6b2c151510 legacy=0x7f6b2c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b2a7fc700 1 -- 172.21.15.7:0/4227129542 >> v1:172.21.15.7:6789/0 conn(0x7f6b2c071060 legacy=0x7f6b2c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b2a7fc700 1 -- 172.21.15.7:0/4227129542 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c157490 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b2a7fc700 1 --2- 172.21.15.7:0/4227129542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c070b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9e9a7d0ee75033 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20011830 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.177+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20011990 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b29ffb700 1 -- 172.21.15.7:0/4227129542 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20011d20 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 -- 172.21.15.7:0/4227129542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 msgr2=0x7f6b2c070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 --2- 172.21.15.7:0/4227129542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c070b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b20009870 tx=0x7f6b20009b80 comp rx=0 tx=0).stop 2023-06-09T11:24:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 -- 172.21.15.7:0/4227129542 shutdown_connections 2023-06-09T11:24:42.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 --2- 172.21.15.7:0/4227129542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c070b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 -- 172.21.15.7:0/4227129542 >> 172.21.15.7:0/4227129542 conn(0x7f6b2c06a4d0 msgr2=0x7f6b2c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:42.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 -- 172.21.15.7:0/4227129542 shutdown_connections 2023-06-09T11:24:42.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.178+0000 7f6b317ff700 1 -- 172.21.15.7:0/4227129542 wait complete. 2023-06-09T11:24:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.180+0000 7f6b317ff700 1 Processor -- start 2023-06-09T11:24:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.180+0000 7f6b317ff700 1 -- start start 2023-06-09T11:24:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c071060 0x7f6b2c1f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1e17e0 con 0x7f6b2c151510 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c159c80 con 0x7f6b2c071060 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b317ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2c1e1500 con 0x7f6b2c0c5b40 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36072/0 (socket says 172.21.15.7:36072) 2023-06-09T11:24:42.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b2a7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b2b7fe700 1 -- 172.21.15.7:0/20232104 learned_addr learned my addr 172.21.15.7:0/20232104 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.181+0000 7f6b2affd700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c071060 0x7f6b2c1f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 -- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 msgr2=0x7f6b2c156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 -- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c071060 msgr2=0x7f6b2c1f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c071060 0x7f6b2c1f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 -- 172.21.15.7:0/20232104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b200094a0 con 0x7f6b2c151510 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b2b7fe700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400cb40 tx=0x7f6b2400ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400a9e0 con 0x7f6b2c151510 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400ab40 con 0x7f6b2c151510 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b2c157a50 con 0x7f6b2c151510 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.182+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24012400 con 0x7f6b2c151510 2023-06-09T11:24:42.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.183+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b2c157770 con 0x7f6b2c151510 2023-06-09T11:24:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.184+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b2c157d30 con 0x7f6b2c151510 2023-06-09T11:24:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.185+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b2401b3b0 con 0x7f6b2c151510 2023-06-09T11:24:42.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.186+0000 7f6b137fe700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 0x7f6b1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:42.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.186+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f6b240893d0 con 0x7f6b2c151510 2023-06-09T11:24:42.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.186+0000 7f6b2affd700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 0x7f6b1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:42.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.187+0000 7f6b2affd700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 0x7f6b1806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0011e0 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:42.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.190+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6b240541d0 con 0x7f6b2c151510 2023-06-09T11:24:42.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.449+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b2c159df0 con 0x7f6b2c151510 2023-06-09T11:24:42.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.449+0000 7f6b137fe700 1 -- 172.21.15.7:0/20232104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f6b2c159df0 con 0x7f6b2c151510 2023-06-09T11:24:42.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:42.455 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":64,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:37.324314+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:rejoin","state_seq":34,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 msgr2=0x7f6b1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 0x7f6b1806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0011e0 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 msgr2=0x7f6b2c13d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400cb40 tx=0x7f6b2400ce50 comp rx=0 tx=0).stop 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 shutdown_connections 2023-06-09T11:24:42.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b2c151510 0x7f6b2c13d980 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b2c071060 0x7f6b2c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b1806ace0 0x7f6b1806d190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 --2- 172.21.15.7:0/20232104 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b2c0c5b40 0x7f6b2c156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 >> 172.21.15.7:0/20232104 conn(0x7f6b2c06a4d0 msgr2=0x7f6b2c1e4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 shutdown_connections 2023-06-09T11:24:42.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:42.452+0000 7f6b317ff700 1 -- 172.21.15.7:0/20232104 wait complete. 2023-06-09T11:24:42.460 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 64 2023-06-09T11:24:42.476 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:24:44.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:44.372 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:24:44.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.674+0000 7f97af3ff700 1 Processor -- start 2023-06-09T11:24:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.674+0000 7f97af3ff700 1 -- start start 2023-06-09T11:24:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.675+0000 7f97af3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f97a8071230 0x7f97a8158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.675+0000 7f97af3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a81580a0 con 0x7f97a80c5b40 2023-06-09T11:24:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.675+0000 7f97af3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80c6180 con 0x7f97a8151510 2023-06-09T11:24:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.675+0000 7f97af3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a81518e0 con 0x7f97a8071230 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ac99a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f97a80c5b40 0x7f97a8070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35154/0 (socket says 172.21.15.7:35154) 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ac99a700 1 -- 172.21.15.7:0/1916149221 learned_addr learned my addr 172.21.15.7:0/1916149221 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ad99c700 1 --2- 172.21.15.7:0/1916149221 >> v2:172.21.15.7:3300/0 conn(0x7f97a8071230 0x7f97a8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839225865 0 0) 0x7f97a81580a0 con 0x7f97a80c5b40 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97940035b0 con 0x7f97a80c5b40 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084733009 0 0) 0x7f97a80c6180 con 0x7f97a8151510 2023-06-09T11:24:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ad99c700 1 -- 172.21.15.7:0/1916149221 >> v1:172.21.15.117:6789/0 conn(0x7f97a8151510 legacy=0x7f97a815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ad99c700 1 -- 172.21.15.7:0/1916149221 >> v1:172.21.15.7:6789/0 conn(0x7f97a80c5b40 legacy=0x7f97a8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.676+0000 7f97ad99c700 1 -- 172.21.15.7:0/1916149221 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8157490 con 0x7f97a8071230 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f97ad99c700 1 --2- 172.21.15.7:0/1916149221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8071230 0x7f97a8158870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94deba2901a72f19 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000e070 con 0x7f97a8071230 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a000bb20 con 0x7f97a8071230 2023-06-09T11:24:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f979ffff700 1 -- 172.21.15.7:0/1916149221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a000bed0 con 0x7f97a8071230 2023-06-09T11:24:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f97af3ff700 1 -- 172.21.15.7:0/1916149221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8071230 msgr2=0x7f97a8158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f97af3ff700 1 --2- 172.21.15.7:0/1916149221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8071230 0x7f97a8158870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f97a0009d90 tx=0x7f97a000b330 comp rx=0 tx=0).stop 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.677+0000 7f97af3ff700 1 -- 172.21.15.7:0/1916149221 shutdown_connections 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.678+0000 7f97af3ff700 1 --2- 172.21.15.7:0/1916149221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8071230 0x7f97a8158870 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.678+0000 7f97af3ff700 1 -- 172.21.15.7:0/1916149221 >> 172.21.15.7:0/1916149221 conn(0x7f97a806a4d0 msgr2=0x7f97a806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.678+0000 7f97af3ff700 1 -- 172.21.15.7:0/1916149221 shutdown_connections 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.678+0000 7f97af3ff700 1 -- 172.21.15.7:0/1916149221 wait complete. 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.679+0000 7f97af3ff700 1 Processor -- start 2023-06-09T11:24:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.679+0000 7f97af3ff700 1 -- start start 2023-06-09T11:24:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97a8071230 0x7f97a80c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a80c5b40 0x7f97a81ef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80c4490 con 0x7f97a8151510 2023-06-09T11:24:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80c23f0 con 0x7f97a80c5b40 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.680+0000 7f97af3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a81efd10 con 0x7f97a8071230 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47642/0 (socket says 172.21.15.7:47642) 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 -- 172.21.15.7:0/4286927728 learned_addr learned my addr 172.21.15.7:0/4286927728 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ac99a700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a80c5b40 0x7f97a81ef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad19b700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97a8071230 0x7f97a80c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 -- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97a8071230 msgr2=0x7f97a80c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97a8071230 0x7f97a80c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 -- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a80c5b40 msgr2=0x7f97a81ef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a80c5b40 0x7f97a81ef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 -- 172.21.15.7:0/4286927728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009a70 con 0x7f97a8151510 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.681+0000 7f97ad99c700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f97a0014040 tx=0x7f97a0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.682+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a001b810 con 0x7f97a8151510 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.682+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0020070 con 0x7f97a8151510 2023-06-09T11:24:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.682+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a001bbb0 con 0x7f97a8151510 2023-06-09T11:24:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.682+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a8157a50 con 0x7f97a8151510 2023-06-09T11:24:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.682+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a8157770 con 0x7f97a8151510 2023-06-09T11:24:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.683+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a8157d30 con 0x7f97a8151510 2023-06-09T11:24:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.684+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97a001bd10 con 0x7f97a8151510 2023-06-09T11:24:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.685+0000 7f979dffb700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 0x7f978c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.686+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f97a0022070 con 0x7f97a8151510 2023-06-09T11:24:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.686+0000 7f97ad19b700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 0x7f978c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.686+0000 7f97ad19b700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 0x7f978c06d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.690+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f97a005a110 con 0x7f97a8151510 2023-06-09T11:24:44.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.941+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97a8074cf0 con 0x7f97a8151510 2023-06-09T11:24:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.942+0000 7f979dffb700 1 -- 172.21.15.7:0/4286927728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f97a0059ca0 con 0x7f97a8151510 2023-06-09T11:24:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:44.948 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":65,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 msgr2=0x7f978c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 0x7f978c06d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f97a4000f60 tx=0x7f97a4009040 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 msgr2=0x7f97a81f1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f97a0014040 tx=0x7f97a0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 shutdown_connections 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97a8151510 0x7f97a81f1b90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f978c06ace0 0x7f978c06d190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f97a8071230 0x7f97a80c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 --2- 172.21.15.7:0/4286927728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f97a80c5b40 0x7f97a81ef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:44.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 >> 172.21.15.7:0/4286927728 conn(0x7f97a806a4d0 msgr2=0x7f97a814c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.944+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 shutdown_connections 2023-06-09T11:24:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:44.945+0000 7f97af3ff700 1 -- 172.21.15.7:0/4286927728 wait complete. 2023-06-09T11:24:44.952 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 65 2023-06-09T11:24:44.968 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=5012 2023-06-09T11:24:44.969 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2023-06-09T11:24:45.969 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:24:45.970 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:24:45.970 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:24:45.975 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:24:45.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:46.123 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:24:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.181+0000 7f93365ff700 1 Processor -- start 2023-06-09T11:24:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.181+0000 7f93365ff700 1 -- start start 2023-06-09T11:24:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f93365ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9330153880 0x7f933015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f93365ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933015a500 con 0x7f93300c9f40 2023-06-09T11:24:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f93365ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f933015da00 con 0x7f933015b4a0 2023-06-09T11:24:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f93365ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93300c5f10 con 0x7f9330153880 2023-06-09T11:24:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f932ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9330153880 0x7f933015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f932ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9330153880 0x7f933015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47650/0 (socket says 172.21.15.7:47650) 2023-06-09T11:24:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.182+0000 7f932ffff700 1 -- 172.21.15.7:0/2004253470 learned_addr learned my addr 172.21.15.7:0/2004253470 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932ffff700 1 -- 172.21.15.7:0/2004253470 >> v1:172.21.15.117:6789/0 conn(0x7f933015b4a0 legacy=0x7f93300c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932ffff700 1 -- 172.21.15.7:0/2004253470 >> v1:172.21.15.7:6789/0 conn(0x7f93300c9f40 legacy=0x7f933015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932ffff700 1 -- 172.21.15.7:0/2004253470 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93301596e0 con 0x7f9330153880 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932ffff700 1 --2- 172.21.15.7:0/2004253470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9330153880 0x7f933015d4c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f93200098f0 tx=0x7f9320009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89fc89f21dc672d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932effd700 1 -- 172.21.15.7:0/2004253470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932000b8a0 con 0x7f9330153880 2023-06-09T11:24:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932effd700 1 -- 172.21.15.7:0/2004253470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932000ba00 con 0x7f9330153880 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.183+0000 7f932effd700 1 -- 172.21.15.7:0/2004253470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932000bd90 con 0x7f9330153880 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 -- 172.21.15.7:0/2004253470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9330153880 msgr2=0x7f933015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 --2- 172.21.15.7:0/2004253470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9330153880 0x7f933015d4c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f93200098f0 tx=0x7f9320009c00 comp rx=0 tx=0).stop 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 -- 172.21.15.7:0/2004253470 shutdown_connections 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 --2- 172.21.15.7:0/2004253470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9330153880 0x7f933015d4c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 -- 172.21.15.7:0/2004253470 >> 172.21.15.7:0/2004253470 conn(0x7f93300737f0 msgr2=0x7f93300720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 -- 172.21.15.7:0/2004253470 shutdown_connections 2023-06-09T11:24:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.184+0000 7f93365ff700 1 -- 172.21.15.7:0/2004253470 wait complete. 2023-06-09T11:24:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.186+0000 7f93365ff700 1 Processor -- start 2023-06-09T11:24:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.186+0000 7f93365ff700 1 -- start start 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93300c9f40 0x7f933014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 0x7f9330157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9330158010 con 0x7f933015b4a0 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93300ce050 con 0x7f9330153880 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f93365ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9330151b40 con 0x7f93300c9f40 2023-06-09T11:24:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47658/0 (socket says 172.21.15.7:47658) 2023-06-09T11:24:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 -- 172.21.15.7:0/3110958024 learned_addr learned my addr 172.21.15.7:0/3110958024 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f932ffff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93300c9f40 0x7f933014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f932f7fe700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 0x7f9330157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 -- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93300c9f40 msgr2=0x7f933014b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93300c9f40 0x7f933014b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 -- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 msgr2=0x7f9330157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 0x7f9330157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f9334b9c700 1 -- 172.21.15.7:0/3110958024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320009550 con 0x7f933015b4a0 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.188+0000 7f932f7fe700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 0x7f9330157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f9334b9c700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f932800cb80 tx=0x7f932800ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328003c00 con 0x7f933015b4a0 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9328003d60 con 0x7f933015b4a0 2023-06-09T11:24:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9330159c60 con 0x7f933015b4a0 2023-06-09T11:24:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932800ab00 con 0x7f933015b4a0 2023-06-09T11:24:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.189+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9330159980 con 0x7f933015b4a0 2023-06-09T11:24:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.190+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9330159f40 con 0x7f933015b4a0 2023-06-09T11:24:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.191+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f932800ac60 con 0x7f933015b4a0 2023-06-09T11:24:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.192+0000 7f932cff9700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.192+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f93280894b0 con 0x7f933015b4a0 2023-06-09T11:24:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.192+0000 7f932ffff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.193+0000 7f932ffff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 0x7f931c06d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9320009c40 tx=0x7f9320008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.196+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9328054350 con 0x7f933015b4a0 2023-06-09T11:24:46.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.457+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933015da00 con 0x7f933015b4a0 2023-06-09T11:24:46.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.458+0000 7f932cff9700 1 -- 172.21.15.7:0/3110958024 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f933015da00 con 0x7f933015b4a0 2023-06-09T11:24:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:46.464 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 msgr2=0x7f931c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 0x7f931c06d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9320009c40 tx=0x7f9320008030 comp rx=0 tx=0).stop 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 msgr2=0x7f9330157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f932800cb80 tx=0x7f932800ce90 comp rx=0 tx=0).stop 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 shutdown_connections 2023-06-09T11:24:46.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f933015b4a0 0x7f9330157ad0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f931c06ace0 0x7f931c06d190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f93300c9f40 0x7f933014b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 --2- 172.21.15.7:0/3110958024 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9330153880 0x7f9330157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 >> 172.21.15.7:0/3110958024 conn(0x7f93300737f0 msgr2=0x7f93300cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 shutdown_connections 2023-06-09T11:24:46.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:46.461+0000 7f93365ff700 1 -- 172.21.15.7:0/3110958024 wait complete. 2023-06-09T11:24:46.469 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:46.484 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:24:46.485 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before thrashing 2023-06-09T11:24:48.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:48.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.680+0000 7f53043ff700 1 Processor -- start 2023-06-09T11:24:48.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.680+0000 7f53043ff700 1 -- start start 2023-06-09T11:24:48.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f53043ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:48.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f53043ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc0c5a80 con 0x7f52fc15b4a0 2023-06-09T11:24:48.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f53043ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52fc15a500 con 0x7f52fc153880 2023-06-09T11:24:48.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f53043ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc0c5f10 con 0x7f52fc0c9f40 2023-06-09T11:24:48.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f530199a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f52fc15b4a0 0x7f52fc0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35180/0 (socket says 172.21.15.7:35180) 2023-06-09T11:24:48.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f530199a700 1 -- 172.21.15.7:0/3110335556 learned_addr learned my addr 172.21.15.7:0/3110335556 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:48.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f530299c700 1 --2- 172.21.15.7:0/3110335556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:48.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739292262 0 0) 0x7f52fc0c5a80 con 0x7f52fc15b4a0 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.682+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e80035b0 con 0x7f52fc15b4a0 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474615228 0 0) 0x7f52f8001110 con 0x7f52fc153880 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52fc0c5a80 con 0x7f52fc153880 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f530299c700 1 -- 172.21.15.7:0/3110335556 >> v1:172.21.15.117:6789/0 conn(0x7f52fc153880 legacy=0x7f52fc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276873818 0 0) 0x7f52e80035b0 con 0x7f52fc15b4a0 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f530299c700 1 -- 172.21.15.7:0/3110335556 >> v1:172.21.15.7:6789/0 conn(0x7f52fc15b4a0 legacy=0x7f52fc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f530299c700 1 -- 172.21.15.7:0/3110335556 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52fc1596e0 con 0x7f52fc0c9f40 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f530299c700 1 --2- 172.21.15.7:0/3110335556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc0c5540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009dc0 tx=0x7f52f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a8b5dfabd626180 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f400e070 con 0x7f52fc0c9f40 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4003a20 con 0x7f52fc0c9f40 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.683+0000 7f5301199700 1 -- 172.21.15.7:0/3110335556 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec001670 con 0x7f52fc0c9f40 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 -- 172.21.15.7:0/3110335556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 msgr2=0x7f52fc0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 --2- 172.21.15.7:0/3110335556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc0c5540 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009dc0 tx=0x7f52f400b330 comp rx=0 tx=0).stop 2023-06-09T11:24:48.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 -- 172.21.15.7:0/3110335556 shutdown_connections 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 --2- 172.21.15.7:0/3110335556 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc0c5540 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 -- 172.21.15.7:0/3110335556 >> 172.21.15.7:0/3110335556 conn(0x7f52fc0737f0 msgr2=0x7f52fc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 -- 172.21.15.7:0/3110335556 shutdown_connections 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.684+0000 7f53043ff700 1 -- 172.21.15.7:0/3110335556 wait complete. 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.686+0000 7f53043ff700 1 Processor -- start 2023-06-09T11:24:48.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.686+0000 7f53043ff700 1 -- start start 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc1e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 0x7f52fc14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc157660 con 0x7f52fc153880 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1e40e0 con 0x7f52fc15b4a0 2023-06-09T11:24:48.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f53043ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52fc1ef410 con 0x7f52fc0c9f40 2023-06-09T11:24:48.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f530199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f530199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47672/0 (socket says 172.21.15.7:47672) 2023-06-09T11:24:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f530199a700 1 -- 172.21.15.7:0/4088006834 learned_addr learned my addr 172.21.15.7:0/4088006834 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f530219b700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc1e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.687+0000 7f530299c700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 0x7f52fc14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:48.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 -- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 msgr2=0x7f52fc1e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc1e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 -- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 msgr2=0x7f52fc14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 0x7f52fc14aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 -- 172.21.15.7:0/4088006834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4009a70 con 0x7f52fc153880 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530299c700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 0x7f52fc14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f530199a700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c6d0 tx=0x7f52ec00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:48.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec004c80 con 0x7f52fc153880 2023-06-09T11:24:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec004de0 con 0x7f52fc153880 2023-06-09T11:24:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52fc159c60 con 0x7f52fc153880 2023-06-09T11:24:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec007310 con 0x7f52fc153880 2023-06-09T11:24:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.688+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52fc159980 con 0x7f52fc153880 2023-06-09T11:24:48.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.689+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52fc159f40 con 0x7f52fc153880 2023-06-09T11:24:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.691+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f52ec007470 con 0x7f52fc153880 2023-06-09T11:24:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.692+0000 7f52f2ffd700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 0x7f52e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.692+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f52ec0895e0 con 0x7f52fc153880 2023-06-09T11:24:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.692+0000 7f530219b700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 0x7f52e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:48.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.693+0000 7f530219b700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 0x7f52e006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007fd0 tx=0x7f52f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:48.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.697+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52ec054350 con 0x7f52fc153880 2023-06-09T11:24:48.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.986+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52fc1e3bb0 con 0x7f52fc153880 2023-06-09T11:24:48.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.987+0000 7f52f2ffd700 1 -- 172.21.15.7:0/4088006834 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f52ec057970 con 0x7f52fc153880 2023-06-09T11:24:48.993 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:48.993 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:48.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.989+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 msgr2=0x7f52e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.989+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 0x7f52e006d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52f8007fd0 tx=0x7f52f8009040 comp rx=0 tx=0).stop 2023-06-09T11:24:48.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 msgr2=0x7f52fc1e4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:48.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f52ec00c6d0 tx=0x7f52ec00ca90 comp rx=0 tx=0).stop 2023-06-09T11:24:48.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 shutdown_connections 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f52fc15b4a0 0x7f52fc14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f52e006ace0 0x7f52e006d190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f52fc0c9f40 0x7f52fc1e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 --2- 172.21.15.7:0/4088006834 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52fc153880 0x7f52fc1e4660 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 >> 172.21.15.7:0/4088006834 conn(0x7f52fc0737f0 msgr2=0x7f52fc14d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:48.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 shutdown_connections 2023-06-09T11:24:48.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:48.990+0000 7f53043ff700 1 -- 172.21.15.7:0/4088006834 wait complete. 2023-06-09T11:24:48.998 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:49.014 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=1) 2023-06-09T11:24:49.015 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-06-09T11:24:49.015 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:24:49.033 INFO:tasks.ceph.mds.e:Stopped 2023-06-09T11:24:49.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.267+0000 7fc15d9ff700 1 Processor -- start 2023-06-09T11:24:49.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.267+0000 7fc15d9ff700 1 -- start start 2023-06-09T11:24:49.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.268+0000 7fc15d9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1580c1bc0 0x7fc1581599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.268+0000 7fc15d9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 0x7fc158159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc15d9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1581501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc15d9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc158070110 con 0x7fc1580c1bc0 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc15d9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc158071300 con 0x7fc1580c0140 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc15d9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1580cbd20 con 0x7fc1580c1f90 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1580c1bc0 0x7fc1581599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1580c1bc0 0x7fc1581599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47682/0 (socket says 172.21.15.7:47682) 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 -- 172.21.15.7:0/769885844 learned_addr learned my addr 172.21.15.7:0/769885844 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc1567fc700 1 --2- 172.21.15.7:0/769885844 >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 0x7fc158159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc1577fe700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1581501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 -- 172.21.15.7:0/769885844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 msgr2=0x7fc1581501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1581501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 -- 172.21.15.7:0/769885844 >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 msgr2=0x7fc158159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 --2- 172.21.15.7:0/769885844 >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 0x7fc158159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc156ffd700 1 -- 172.21.15.7:0/769885844 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1580c0bf0 con 0x7fc1580c1bc0 2023-06-09T11:24:49.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.269+0000 7fc1567fc700 1 --2- 172.21.15.7:0/769885844 >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 0x7fc158159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:49.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc156ffd700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c1bc0 0x7fc1581599b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc15806d950 tx=0x7fc148009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe40765f7b38ed0a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc155ffb700 1 -- 172.21.15.7:0/769885844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800b860 con 0x7fc1580c1bc0 2023-06-09T11:24:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc155ffb700 1 -- 172.21.15.7:0/769885844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14800b9c0 con 0x7fc1580c1bc0 2023-06-09T11:24:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc155ffb700 1 -- 172.21.15.7:0/769885844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800bd50 con 0x7fc1580c1bc0 2023-06-09T11:24:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 -- 172.21.15.7:0/769885844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c1bc0 msgr2=0x7fc1581599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:49.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c1bc0 0x7fc1581599b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fc15806d950 tx=0x7fc148009c00 comp rx=0 tx=0).stop 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 -- 172.21.15.7:0/769885844 shutdown_connections 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1581501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/769885844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c1bc0 0x7fc1581599b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/769885844 >> v2:172.21.15.117:3300/0 conn(0x7fc1580c0140 0x7fc158159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.270+0000 7fc15d9ff700 1 -- 172.21.15.7:0/769885844 >> 172.21.15.7:0/769885844 conn(0x7fc15806a4d0 msgr2=0x7fc15806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.271+0000 7fc15d9ff700 1 -- 172.21.15.7:0/769885844 shutdown_connections 2023-06-09T11:24:49.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.271+0000 7fc15d9ff700 1 -- 172.21.15.7:0/769885844 wait complete. 2023-06-09T11:24:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.273+0000 7fc15d9ff700 1 Processor -- start 2023-06-09T11:24:49.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.273+0000 7fc15d9ff700 1 -- start start 2023-06-09T11:24:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1580c1bc0 0x7fc1580c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1580cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581f30c0 con 0x7fc1580c0140 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581f2820 con 0x7fc1580c1bc0 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.274+0000 7fc15d9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1581f2b00 con 0x7fc1580c1f90 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47684/0 (socket says 172.21.15.7:47684) 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 -- 172.21.15.7:0/973690068 learned_addr learned my addr 172.21.15.7:0/973690068 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:49.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc1567fc700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1580c1bc0 0x7fc1580c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc1577fe700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1580cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 -- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 msgr2=0x7fc1580cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1580cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 -- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1580c1bc0 msgr2=0x7fc1580c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1580c1bc0 0x7fc1580c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 -- 172.21.15.7:0/973690068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009550 con 0x7fc1580c0140 2023-06-09T11:24:49.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.275+0000 7fc156ffd700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc148009c40 tx=0x7fc148004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.276+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148004380 con 0x7fc1580c0140 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.276+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14801f070 con 0x7fc1580c0140 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.276+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1480036a0 con 0x7fc1580c0140 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.276+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1580c0e90 con 0x7fc1580c0140 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.276+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1580702f0 con 0x7fc1580c0140 2023-06-09T11:24:49.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.277+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1580c1170 con 0x7fc1580c0140 2023-06-09T11:24:49.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.278+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc148021030 con 0x7fc1580c0140 2023-06-09T11:24:49.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.279+0000 7fc13ffff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:49.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.279+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fc14808ab90 con 0x7fc1580c0140 2023-06-09T11:24:49.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.279+0000 7fc1567fc700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:49.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.280+0000 7fc1567fc700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 0x7fc14406d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc15815ea10 tx=0x7fc14c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:49.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.283+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc148055900 con 0x7fc1580c0140 2023-06-09T11:24:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.543+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1580c4cd0 con 0x7fc1580c0140 2023-06-09T11:24:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.543+0000 7fc13ffff700 1 -- 172.21.15.7:0/973690068 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fc148058f20 con 0x7fc1580c0140 2023-06-09T11:24:49.549 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:49.550 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 msgr2=0x7fc14406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 0x7fc14406d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc15815ea10 tx=0x7fc14c008040 comp rx=0 tx=0).stop 2023-06-09T11:24:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 msgr2=0x7fc1581f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fc148009c40 tx=0x7fc148004750 comp rx=0 tx=0).stop 2023-06-09T11:24:49.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 shutdown_connections 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc14406ace0 0x7fc14406d190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1580c1f90 0x7fc1580cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1580c0140 0x7fc1581f2170 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 --2- 172.21.15.7:0/973690068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1580c1bc0 0x7fc1580c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 >> 172.21.15.7:0/973690068 conn(0x7fc15806a4d0 msgr2=0x7fc15814ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 shutdown_connections 2023-06-09T11:24:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:49.546+0000 7fc15d9ff700 1 -- 172.21.15.7:0/973690068 wait complete. 2023-06-09T11:24:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:49.570 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:24:51.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:51.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.774+0000 7fe6ecbff700 1 Processor -- start 2023-06-09T11:24:51.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.774+0000 7fe6ecbff700 1 -- start start 2023-06-09T11:24:51.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.775+0000 7fe6ecbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe6e80c5d20 0x7fe6e815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.775+0000 7fe6ecbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6e81610d0 0x7fe6e81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.775+0000 7fe6ecbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e80c0810 con 0x7fe6e815e880 2023-06-09T11:24:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6ecbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e815db60 con 0x7fe6e81610d0 2023-06-09T11:24:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6ecbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e81615a0 con 0x7fe6e80c5d20 2023-06-09T11:24:51.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6e81610d0 0x7fe6e81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6e81610d0 0x7fe6e81579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47698/0 (socket says 172.21.15.7:47698) 2023-06-09T11:24:51.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 -- 172.21.15.7:0/3676727700 learned_addr learned my addr 172.21.15.7:0/3676727700 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e659c700 1 --2- 172.21.15.7:0/3676727700 >> v2:172.21.15.117:3300/0 conn(0x7fe6e80c5d20 0x7fe6e815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 -- 172.21.15.7:0/3676727700 >> v2:172.21.15.117:3300/0 conn(0x7fe6e80c5d20 msgr2=0x7fe6e815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/3676727700 >> v2:172.21.15.117:3300/0 conn(0x7fe6e80c5d20 0x7fe6e815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 -- 172.21.15.7:0/3676727700 >> v1:172.21.15.117:6789/0 conn(0x7fe6e815e880 legacy=0x7fe6e80c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.776+0000 7fe6e5d9b700 1 -- 172.21.15.7:0/3676727700 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8161880 con 0x7fe6e81610d0 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/3676727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e81610d0 0x7fe6e81579b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c8d40 tx=0x7fe6dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45728bacb096f332 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6e559a700 1 -- 172.21.15.7:0/3676727700 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc004230 con 0x7fe6e81610d0 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6e559a700 1 -- 172.21.15.7:0/3676727700 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6dc004390 con 0x7fe6e81610d0 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6e559a700 1 -- 172.21.15.7:0/3676727700 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6dc0036a0 con 0x7fe6e81610d0 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 -- 172.21.15.7:0/3676727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e81610d0 msgr2=0x7fe6e81579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/3676727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e81610d0 0x7fe6e81579b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c8d40 tx=0x7fe6dc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 -- 172.21.15.7:0/3676727700 shutdown_connections 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/3676727700 >> v2:172.21.15.117:3300/0 conn(0x7fe6e80c5d20 0x7fe6e815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/3676727700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e81610d0 0x7fe6e81579b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 -- 172.21.15.7:0/3676727700 >> 172.21.15.7:0/3676727700 conn(0x7fe6e806a4d0 msgr2=0x7fe6e806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.777+0000 7fe6ecbff700 1 -- 172.21.15.7:0/3676727700 shutdown_connections 2023-06-09T11:24:51.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.778+0000 7fe6ecbff700 1 -- 172.21.15.7:0/3676727700 wait complete. 2023-06-09T11:24:51.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.779+0000 7fe6ecbff700 1 Processor -- start 2023-06-09T11:24:51.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.779+0000 7fe6ecbff700 1 -- start start 2023-06-09T11:24:51.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 0x7fe6e81f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 0x7fe6e8160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8155210 con 0x7fe6e80c5d20 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8156b90 con 0x7fe6e81610d0 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.780+0000 7fe6ecbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8156f10 con 0x7fe6e815e880 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47704/0 (socket says 172.21.15.7:47704) 2023-06-09T11:24:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 -- 172.21.15.7:0/816611620 learned_addr learned my addr 172.21.15.7:0/816611620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 0x7fe6e81f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e6d9d700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 0x7fe6e8160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 -- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 msgr2=0x7fe6e81f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 0x7fe6e81f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 -- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 msgr2=0x7fe6e8160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 0x7fe6e8160810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 -- 172.21.15.7:0/816611620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc009550 con 0x7fe6e80c5d20 2023-06-09T11:24:51.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 0x7fe6e81f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:51.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e6d9d700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 0x7fe6e8160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:51.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.781+0000 7fe6e659c700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e810 tx=0x7fe6d800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.782+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8013610 con 0x7fe6e80c5d20 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.782+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d8013770 con 0x7fe6e80c5d20 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.782+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8013b20 con 0x7fe6e80c5d20 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.782+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e8161e40 con 0x7fe6e80c5d20 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.782+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e8161b60 con 0x7fe6e80c5d20 2023-06-09T11:24:51.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.783+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e815de20 con 0x7fe6e80c5d20 2023-06-09T11:24:51.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.784+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe6d8013c80 con 0x7fe6e80c5d20 2023-06-09T11:24:51.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.784+0000 7fe6d6ffd700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 0x7fe6c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:51.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.785+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fe6d8089490 con 0x7fe6e80c5d20 2023-06-09T11:24:51.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.785+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 0x7fe6c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:51.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.786+0000 7fe6e5d9b700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 0x7fe6c806d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c8d40 tx=0x7fe6dc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:51.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:51.789+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe6d8054210 con 0x7fe6e80c5d20 2023-06-09T11:24:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.043+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6e8155350 con 0x7fe6e80c5d20 2023-06-09T11:24:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.044+0000 7fe6d6ffd700 1 -- 172.21.15.7:0/816611620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fe6d8057830 con 0x7fe6e80c5d20 2023-06-09T11:24:52.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:52.050 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:52.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 msgr2=0x7fe6c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:52.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 0x7fe6c806d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80c8d40 tx=0x7fe6dc008030 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 msgr2=0x7fe6e81562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800e810 tx=0x7fe6d800ebd0 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 shutdown_connections 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6e81610d0 0x7fe6e8160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e80c5d20 0x7fe6e81562f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe6c806ace0 0x7fe6c806d190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 --2- 172.21.15.7:0/816611620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6e815e880 0x7fe6e81f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.046+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 >> 172.21.15.7:0/816611620 conn(0x7fe6e806a4d0 msgr2=0x7fe6e806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:52.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.047+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 shutdown_connections 2023-06-09T11:24:52.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:52.047+0000 7fe6ecbff700 1 -- 172.21.15.7:0/816611620 wait complete. 2023-06-09T11:24:52.054 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:52.070 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:24:53.321 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:24:53.315+0000 7f7c8b3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:24:53.321 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:24:53.315+0000 7f7c8b3f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:24:53.321 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:24:53.315+0000 7f7c8b3f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:24:54.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.275+0000 7f836bbff700 1 Processor -- start 2023-06-09T11:24:54.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.275+0000 7f836bbff700 1 -- start start 2023-06-09T11:24:54.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836bbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 0x7f836414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:54.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836bbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640cc140 con 0x7f8364159eb0 2023-06-09T11:24:54.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836bbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640c1830 con 0x7f83641593d0 2023-06-09T11:24:54.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836bbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640714f0 con 0x7f8364072020 2023-06-09T11:24:54.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836999b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f83641593d0 0x7f83641597a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:58482/0 (socket says 172.21.15.7:58482) 2023-06-09T11:24:54.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.277+0000 7f836999b700 1 -- 172.21.15.7:0/2239655846 learned_addr learned my addr 172.21.15.7:0/2239655846 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f836a19c700 1 --2- 172.21.15.7:0/2239655846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 0x7f836414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858252264 0 0) 0x7f83640c1830 con 0x7f83641593d0 2023-06-09T11:24:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83500035b0 con 0x7f83641593d0 2023-06-09T11:24:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223500013 0 0) 0x7f83640cc140 con 0x7f8364159eb0 2023-06-09T11:24:54.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83640c1830 con 0x7f8364159eb0 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f836a19c700 1 -- 172.21.15.7:0/2239655846 >> v1:172.21.15.117:6789/0 conn(0x7f83641593d0 legacy=0x7f83641597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f836a19c700 1 -- 172.21.15.7:0/2239655846 >> v1:172.21.15.7:6789/0 conn(0x7f8364159eb0 legacy=0x7f8364071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823567316 0 0) 0x7f83640c1830 con 0x7f8364159eb0 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.278+0000 7f836a19c700 1 -- 172.21.15.7:0/2239655846 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640c0560 con 0x7f8364072020 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f836a19c700 1 --2- 172.21.15.7:0/2239655846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 0x7f836414f9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f835c00aed0 tx=0x7f835c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=175ff9a7de7f7c31 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c00f070 con 0x7f8364072020 2023-06-09T11:24:54.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c00a2e0 con 0x7f8364072020 2023-06-09T11:24:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f8368999700 1 -- 172.21.15.7:0/2239655846 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354001670 con 0x7f8364072020 2023-06-09T11:24:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f836bbff700 1 -- 172.21.15.7:0/2239655846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 msgr2=0x7f836414f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f836bbff700 1 --2- 172.21.15.7:0/2239655846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 0x7f836414f9a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f835c00aed0 tx=0x7f835c00c330 comp rx=0 tx=0).stop 2023-06-09T11:24:54.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.279+0000 7f836bbff700 1 -- 172.21.15.7:0/2239655846 shutdown_connections 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.280+0000 7f836bbff700 1 --2- 172.21.15.7:0/2239655846 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364072020 0x7f836414f9a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.280+0000 7f836bbff700 1 -- 172.21.15.7:0/2239655846 >> 172.21.15.7:0/2239655846 conn(0x7f836406a4d0 msgr2=0x7f836406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.280+0000 7f836bbff700 1 -- 172.21.15.7:0/2239655846 shutdown_connections 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.280+0000 7f836bbff700 1 -- 172.21.15.7:0/2239655846 wait complete. 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.281+0000 7f836bbff700 1 Processor -- start 2023-06-09T11:24:54.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.281+0000 7f836bbff700 1 -- start start 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 0x7f836414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364159eb0 0x7f83640afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641eeb00 con 0x7f8364072020 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641ef030 con 0x7f83641593d0 2023-06-09T11:24:54.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836bbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83641eed50 con 0x7f8364159eb0 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.282+0000 7f836999b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52646/0 (socket says 172.21.15.7:52646) 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 -- 172.21.15.7:0/1313420493 learned_addr learned my addr 172.21.15.7:0/1313420493 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836a19c700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364159eb0 0x7f83640afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836919a700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 0x7f836414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 -- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364159eb0 msgr2=0x7f83640afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364159eb0 0x7f83640afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 -- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 msgr2=0x7f836414e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 0x7f836414e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 -- 172.21.15.7:0/1313420493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c00ab80 con 0x7f8364072020 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836919a700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 0x7f836414e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f836999b700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f836000c4d0 tx=0x7f836000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360005d70 con 0x7f8364072020 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83600070d0 con 0x7f8364072020 2023-06-09T11:24:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.283+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8360007440 con 0x7f8364072020 2023-06-09T11:24:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.284+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83640c1d60 con 0x7f8364072020 2023-06-09T11:24:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.284+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83640c1a80 con 0x7f8364072020 2023-06-09T11:24:54.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.285+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83640c2040 con 0x7f8364072020 2023-06-09T11:24:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.286+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83600075a0 con 0x7f8364072020 2023-06-09T11:24:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.286+0000 7f835a7fc700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 0x7f834806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.287+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f8360088bd0 con 0x7f8364072020 2023-06-09T11:24:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.287+0000 7f836919a700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 0x7f834806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:54.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.287+0000 7f836919a700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 0x7f834806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83640caec0 tx=0x7f8354009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:54.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.291+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8360053a00 con 0x7f8364072020 2023-06-09T11:24:54.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.548+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8364074cf0 con 0x7f8364072020 2023-06-09T11:24:54.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.548+0000 7f835a7fc700 1 -- 172.21.15.7:0/1313420493 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f8360057020 con 0x7f8364072020 2023-06-09T11:24:54.554 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:54.555 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:54.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.550+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 msgr2=0x7f834806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.550+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 0x7f834806d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83640caec0 tx=0x7f8354009040 comp rx=0 tx=0).stop 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 msgr2=0x7f836414dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f836000c4d0 tx=0x7f836000c890 comp rx=0 tx=0).stop 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 shutdown_connections 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f834806ace0 0x7f834806d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8364159eb0 0x7f83640afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8364072020 0x7f836414dec0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 --2- 172.21.15.7:0/1313420493 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83641593d0 0x7f836414e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 >> 172.21.15.7:0/1313420493 conn(0x7f836406a4d0 msgr2=0x7f8364157870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 shutdown_connections 2023-06-09T11:24:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:54.551+0000 7f836bbff700 1 -- 172.21.15.7:0/1313420493 wait complete. 2023-06-09T11:24:54.558 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:54.574 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:24:56.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.781+0000 7f49d0dff700 1 Processor -- start 2023-06-09T11:24:56.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.782+0000 7f49d0dff700 1 -- start start 2023-06-09T11:24:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49cc0c1bc0 0x7f49cc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 0x7f49cc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc070110 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc071300 con 0x7f49cc0c0140 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49d0dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0cbd20 con 0x7f49cc0c1f90 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49ca59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49cc0c1bc0 0x7f49cc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49ca59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49cc0c1bc0 0x7f49cc1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52658/0 (socket says 172.21.15.7:52658) 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49ca59c700 1 -- 172.21.15.7:0/189361831 learned_addr learned my addr 172.21.15.7:0/189361831 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49cad9d700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.783+0000 7f49c3fff700 1 --2- 172.21.15.7:0/189361831 >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 0x7f49cc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 -- 172.21.15.7:0/189361831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 msgr2=0x7f49cc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 -- 172.21.15.7:0/189361831 >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 msgr2=0x7f49cc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 --2- 172.21.15.7:0/189361831 >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 0x7f49cc159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 -- 172.21.15.7:0/189361831 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0c0bf0 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49c3fff700 1 --2- 172.21.15.7:0/189361831 >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 0x7f49cc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49ca59c700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc1599b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06d950 tx=0x7f49bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c155b530bf0b3e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:56.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49c9d9b700 1 -- 172.21.15.7:0/189361831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc011830 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49c9d9b700 1 -- 172.21.15.7:0/189361831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc011990 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.784+0000 7f49c9d9b700 1 -- 172.21.15.7:0/189361831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc011d20 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 -- 172.21.15.7:0/189361831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 msgr2=0x7f49cc1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc1599b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06d950 tx=0x7f49bc009c00 comp rx=0 tx=0).stop 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 -- 172.21.15.7:0/189361831 shutdown_connections 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 --2- 172.21.15.7:0/189361831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc1599b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 --2- 172.21.15.7:0/189361831 >> v2:172.21.15.117:3300/0 conn(0x7f49cc0c0140 0x7f49cc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 -- 172.21.15.7:0/189361831 >> 172.21.15.7:0/189361831 conn(0x7f49cc06a4d0 msgr2=0x7f49cc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 -- 172.21.15.7:0/189361831 shutdown_connections 2023-06-09T11:24:56.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.785+0000 7f49d0dff700 1 -- 172.21.15.7:0/189361831 wait complete. 2023-06-09T11:24:56.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.787+0000 7f49d0dff700 1 Processor -- start 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.787+0000 7f49d0dff700 1 -- start start 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49cc0c0140 0x7f49cc158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc157d00 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc15e980 con 0x7f49cc0c0140 2023-06-09T11:24:56.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49d0dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc158660 con 0x7f49cc0c1f90 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52672/0 (socket says 172.21.15.7:52672) 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 -- 172.21.15.7:0/3893357339 learned_addr learned my addr 172.21.15.7:0/3893357339 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49ca59c700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49cc0c0140 0x7f49cc158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49cad9d700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 -- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 msgr2=0x7f49cc1f2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:56.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1f2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 -- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49cc0c0140 msgr2=0x7f49cc158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49cc0c0140 0x7f49cc158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.788+0000 7f49c3fff700 1 -- 172.21.15.7:0/3893357339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009550 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49c3fff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f49b400d800 tx=0x7f49b400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4004030 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4004190 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc0c0e90 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4004520 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.789+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc0702f0 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.790+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc0c1170 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.791+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49b40036a0 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.792+0000 7f49c2ffd700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.792+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f49b4088f90 con 0x7f49cc0c1bc0 2023-06-09T11:24:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.793+0000 7f49ca59c700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:56.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.793+0000 7f49ca59c700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009870 tx=0x7f49bc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:56.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:56.797+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f49b4057840 con 0x7f49cc0c1bc0 2023-06-09T11:24:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.048+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49cc074cf0 con 0x7f49cc0c1bc0 2023-06-09T11:24:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.049+0000 7f49c2ffd700 1 -- 172.21.15.7:0/3893357339 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f49b4019090 con 0x7f49cc0c1bc0 2023-06-09T11:24:57.055 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:57.055 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:57.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 msgr2=0x7f49ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009870 tx=0x7f49bc008030 comp rx=0 tx=0).stop 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 msgr2=0x7f49cc14ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f49b400d800 tx=0x7f49b400db10 comp rx=0 tx=0).stop 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 shutdown_connections 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49cc0c1f90 0x7f49cc1f2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49cc0c0140 0x7f49cc158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 --2- 172.21.15.7:0/3893357339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49cc0c1bc0 0x7f49cc14ce60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:57.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.051+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 >> 172.21.15.7:0/3893357339 conn(0x7f49cc06a4d0 msgr2=0x7f49cc0c7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:57.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.052+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 shutdown_connections 2023-06-09T11:24:57.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:57.052+0000 7f49d0dff700 1 -- 172.21.15.7:0/3893357339 wait complete. 2023-06-09T11:24:57.059 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:57.076 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:24:59.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:24:59.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.281+0000 7fb527bff700 1 Processor -- start 2023-06-09T11:24:59.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.281+0000 7fb527bff700 1 -- start start 2023-06-09T11:24:59.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.282+0000 7fb527bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb52015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.282+0000 7fb527bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52015dcf0 con 0x7fb5200ca1b0 2023-06-09T11:24:59.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.282+0000 7fb527bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52015d450 con 0x7fb5200b5360 2023-06-09T11:24:59.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.282+0000 7fb527bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015d730 con 0x7fb52006b480 2023-06-09T11:24:59.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb52599b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb5200ca1b0 0x7fb5201536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58802/0 (socket says 172.21.15.7:58802) 2023-06-09T11:24:59.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb52599b700 1 -- 172.21.15.7:0/2167787805 learned_addr learned my addr 172.21.15.7:0/2167787805 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:24:59.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb52619c700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb52015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011601160 0 0) 0x7fb52015dcf0 con 0x7fb5200ca1b0 2023-06-09T11:24:59.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb50c0035b0 con 0x7fb5200ca1b0 2023-06-09T11:24:59.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 552176853 0 0) 0x7fb50c0035b0 con 0x7fb5200ca1b0 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 msgr2=0x7fb52015cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb52015cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 >> v1:172.21.15.117:6789/0 conn(0x7fb5200b5360 legacy=0x7fb52006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52015da10 con 0x7fb5200ca1b0 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb51c001670 con 0x7fb5200ca1b0 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 >> v1:172.21.15.7:6789/0 conn(0x7fb5200ca1b0 legacy=0x7fb5201536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c003a30 0x7fb50c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.283+0000 7fb524999700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c0043d0 0x7fb50c004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb524999700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb50c0050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015d730 con 0x7fb50c003a30 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c00ba10 con 0x7fb50c0043d0 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50c00bba0 con 0x7fb52006b480 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c003a30 0x7fb50c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52619c700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb50c0050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52519a700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c0043d0 0x7fb50c004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 -- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 msgr2=0x7fb50c0050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52006b480 0x7fb50c0050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 -- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c0043d0 msgr2=0x7fb50c004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb50c0043d0 0x7fb50c004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 -- 172.21.15.7:0/2167787805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0035b0 con 0x7fb50c003a30 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.284+0000 7fb52599b700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c003a30 0x7fb50c003e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c2f0 tx=0x7fb50c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62107708c5d8007 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c00ce30 con 0x7fb50c003a30 2023-06-09T11:24:59.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c005960 con 0x7fb50c003a30 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb524999700 1 -- 172.21.15.7:0/2167787805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c005cf0 con 0x7fb50c003a30 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 -- 172.21.15.7:0/2167787805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c003a30 msgr2=0x7fb50c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 --2- 172.21.15.7:0/2167787805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb50c003a30 0x7fb50c003e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00c2f0 tx=0x7fb50c005880 comp rx=0 tx=0).stop 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb52619c700 1 -- 172.21.15.7:0/2167787805 reap_dead start 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 -- 172.21.15.7:0/2167787805 shutdown_connections 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 -- 172.21.15.7:0/2167787805 >> 172.21.15.7:0/2167787805 conn(0x7fb520072090 msgr2=0x7fb520072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 -- 172.21.15.7:0/2167787805 shutdown_connections 2023-06-09T11:24:59.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.285+0000 7fb527bff700 1 -- 172.21.15.7:0/2167787805 wait complete. 2023-06-09T11:24:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.287+0000 7fb527bff700 1 Processor -- start 2023-06-09T11:24:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.287+0000 7fb527bff700 1 -- start start 2023-06-09T11:24:59.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 0x7fb520150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 0x7fb52014cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52015a020 con 0x7fb520157520 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201f0010 con 0x7fb520159620 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb527bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201513b0 con 0x7fb50c003a30 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb52519a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb52519a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52680/0 (socket says 172.21.15.7:52680) 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.288+0000 7fb52519a700 1 -- 172.21.15.7:0/625853271 learned_addr learned my addr 172.21.15.7:0/625853271 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52599b700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 0x7fb520150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52619c700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 0x7fb52014cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 -- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 msgr2=0x7fb520150e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 0x7fb520150e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 -- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 msgr2=0x7fb52014cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 0x7fb52014cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 -- 172.21.15.7:0/625853271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52006a850 con 0x7fb520157520 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52599b700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 0x7fb520150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52619c700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 0x7fb52014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb52519a700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb51000f7c0 tx=0x7fb51000fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100052e0 con 0x7fb520157520 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510005460 con 0x7fb520157520 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52006a570 con 0x7fb520157520 2023-06-09T11:24:59.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.289+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100038c0 con 0x7fb520157520 2023-06-09T11:24:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.290+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb520153e00 con 0x7fb520157520 2023-06-09T11:24:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.291+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb520154a60 con 0x7fb520157520 2023-06-09T11:24:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.292+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb510003a20 con 0x7fb520157520 2023-06-09T11:24:59.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.292+0000 7fb5167fc700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 0x7fb50406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:24:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.293+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7fb510089630 con 0x7fb520157520 2023-06-09T11:24:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.293+0000 7fb52599b700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 0x7fb50406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:24:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.293+0000 7fb52599b700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 0x7fb50406d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb51c015040 tx=0x7fb51c007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:24:59.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.297+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb5100543a0 con 0x7fb520157520 2023-06-09T11:24:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.555+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb520074cf0 con 0x7fb520157520 2023-06-09T11:24:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.555+0000 7fb5167fc700 1 -- 172.21.15.7:0/625853271 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fb5100579c0 con 0x7fb520157520 2023-06-09T11:24:59.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:24:59.561 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 msgr2=0x7fb50406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 0x7fb50406d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb51c015040 tx=0x7fb51c007000 comp rx=0 tx=0).stop 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 msgr2=0x7fb5201590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb51000f7c0 tx=0x7fb51000fb80 comp rx=0 tx=0).stop 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 shutdown_connections 2023-06-09T11:24:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb520159620 0x7fb52014cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb50406ace0 0x7fb50406d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb50c003a30 0x7fb520150e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 --2- 172.21.15.7:0/625853271 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb520157520 0x7fb5201590e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 >> 172.21.15.7:0/625853271 conn(0x7fb520072090 msgr2=0x7fb5200cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 shutdown_connections 2023-06-09T11:24:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:24:59.558+0000 7fb527bff700 1 -- 172.21.15.7:0/625853271 wait complete. 2023-06-09T11:24:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:24:59.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:25:01.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:01.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.786+0000 7f45b4fff700 1 Processor -- start 2023-06-09T11:25:01.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.787+0000 7f45b4fff700 1 -- start start 2023-06-09T11:25:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01535e0 0x7f45b0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b01539b0 0x7f45b00c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45b00c4ce0 0x7f45b00c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00c5a00 con 0x7f45b01539b0 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b015a420 con 0x7f45b00c4ce0 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45b4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b006b460 con 0x7f45b01535e0 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b01539b0 0x7f45b00c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f45b01539b0 0x7f45b00c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52694/0 (socket says 172.21.15.7:52694) 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.788+0000 7f45a7fff700 1 -- 172.21.15.7:0/1820478445 learned_addr learned my addr 172.21.15.7:0/1820478445 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45ae59c700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01535e0 0x7f45b0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45aed9d700 1 --2- 172.21.15.7:0/1820478445 >> v2:172.21.15.117:3300/0 conn(0x7f45b00c4ce0 0x7f45b00c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 -- 172.21.15.7:0/1820478445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01535e0 msgr2=0x7f45b0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01535e0 0x7f45b0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 -- 172.21.15.7:0/1820478445 >> v2:172.21.15.117:3300/0 conn(0x7f45b00c4ce0 msgr2=0x7f45b00c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 --2- 172.21.15.7:0/1820478445 >> v2:172.21.15.117:3300/0 conn(0x7f45b00c4ce0 0x7f45b00c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 -- 172.21.15.7:0/1820478445 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b006a570 con 0x7f45b01539b0 2023-06-09T11:25:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45a7fff700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01539b0 0x7f45b00c9f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45b015c910 tx=0x7f4598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7ecc0a7f13bcc9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45add9b700 1 -- 172.21.15.7:0/1820478445 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800b870 con 0x7f45b01539b0 2023-06-09T11:25:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45add9b700 1 -- 172.21.15.7:0/1820478445 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459800b9d0 con 0x7f45b01539b0 2023-06-09T11:25:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.789+0000 7f45add9b700 1 -- 172.21.15.7:0/1820478445 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459800bd60 con 0x7f45b01539b0 2023-06-09T11:25:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 -- 172.21.15.7:0/1820478445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01539b0 msgr2=0x7f45b00c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01539b0 0x7f45b00c9f60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45b015c910 tx=0x7f4598009bd0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 -- 172.21.15.7:0/1820478445 shutdown_connections 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 --2- 172.21.15.7:0/1820478445 >> v2:172.21.15.117:3300/0 conn(0x7f45b00c4ce0 0x7f45b00c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01535e0 0x7f45b0159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 --2- 172.21.15.7:0/1820478445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01539b0 0x7f45b00c9f60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 -- 172.21.15.7:0/1820478445 >> 172.21.15.7:0/1820478445 conn(0x7f45b00737f0 msgr2=0x7f45b0072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 -- 172.21.15.7:0/1820478445 shutdown_connections 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.790+0000 7f45b4fff700 1 -- 172.21.15.7:0/1820478445 wait complete. 2023-06-09T11:25:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.792+0000 7f45b4fff700 1 Processor -- start 2023-06-09T11:25:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.792+0000 7f45b4fff700 1 -- start start 2023-06-09T11:25:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00c4ce0 0x7f45b015bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01539b0 0x7f45b014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0154bd0 con 0x7f45b01535e0 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01e5760 con 0x7f45b00c4ce0 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45b4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01e4f70 con 0x7f45b01539b0 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52708/0 (socket says 172.21.15.7:52708) 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45a7fff700 1 -- 172.21.15.7:0/3541755122 learned_addr learned my addr 172.21.15.7:0/3541755122 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45ae59c700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00c4ce0 0x7f45b015bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.793+0000 7f45aed9d700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01539b0 0x7f45b014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 -- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01539b0 msgr2=0x7f45b014e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01539b0 0x7f45b014e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 -- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00c4ce0 msgr2=0x7f45b015bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00c4ce0 0x7f45b015bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 -- 172.21.15.7:0/3541755122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598009550 con 0x7f45b01535e0 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a7fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4598013000 tx=0x7f4598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598004480 con 0x7f45b01535e0 2023-06-09T11:25:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45980045e0 con 0x7f45b01535e0 2023-06-09T11:25:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45980038e0 con 0x7f45b01535e0 2023-06-09T11:25:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b006ab30 con 0x7f45b01535e0 2023-06-09T11:25:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.794+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b006a850 con 0x7f45b01535e0 2023-06-09T11:25:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.795+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b006ae10 con 0x7f45b01535e0 2023-06-09T11:25:01.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.796+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4598021070 con 0x7f45b01535e0 2023-06-09T11:25:01.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.797+0000 7f45a6ffd700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 0x7f459006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:01.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.797+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f459808ac90 con 0x7f45b01535e0 2023-06-09T11:25:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.797+0000 7f45ae59c700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 0x7f459006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.798+0000 7f45ae59c700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 0x7f459006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45a0006b20 tx=0x7f45a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:01.802+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4598055b30 con 0x7f45b01535e0 2023-06-09T11:25:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.055+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45b0074cf0 con 0x7f45b01535e0 2023-06-09T11:25:02.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.056+0000 7f45a6ffd700 1 -- 172.21.15.7:0/3541755122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f4598015090 con 0x7f45b01535e0 2023-06-09T11:25:02.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:02.062 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5692,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:24:43.308490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5071,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5071":{"gid":5071,"name":"e","rank":1,"incarnation":27,"state":"up:active","state_seq":31,"addr":"172.21.15.7:6839/2631794280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2631794280},{"type":"v1","addr":"172.21.15.7:6839","nonce":2631794280}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.058+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 msgr2=0x7f459006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.058+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 0x7f459006d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f45a0006b20 tx=0x7f45a0008040 comp rx=0 tx=0).stop 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.058+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 msgr2=0x7f45b01e3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.058+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4598013000 tx=0x7f4598004750 comp rx=0 tx=0).stop 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.058+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 shutdown_connections 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459006ace0 0x7f459006d190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b01539b0 0x7f45b014e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00c4ce0 0x7f45b015bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 --2- 172.21.15.7:0/3541755122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b01535e0 0x7f45b01e3880 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 >> 172.21.15.7:0/3541755122 conn(0x7f45b00737f0 msgr2=0x7f45b00caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 shutdown_connections 2023-06-09T11:25:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:02.059+0000 7f45b4fff700 1 -- 172.21.15.7:0/3541755122 wait complete. 2023-06-09T11:25:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2023-06-09T11:25:02.083 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:25:04.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.287+0000 7f2411dff700 1 Processor -- start 2023-06-09T11:25:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.287+0000 7f2411dff700 1 -- start start 2023-06-09T11:25:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.288+0000 7f2411dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c0af980 0x7f240c165010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f2411dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c15ce00 con 0x7f240c153960 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f2411dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c15d0e0 con 0x7f240c071c80 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f2411dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c0adab0 con 0x7f240c0af980 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c0af980 0x7f240c165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f240c0af980 0x7f240c165010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49372/0 (socket says 172.21.15.7:49372) 2023-06-09T11:25:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 -- 172.21.15.7:0/3621498852 learned_addr learned my addr 172.21.15.7:0/3621498852 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240b7fe700 1 --1- 172.21.15.7:0/3621498852 >> v1:172.21.15.7:6789/0 conn(0x7f240c153960 0x7f240c06fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60092/0 (socket says 172.21.15.7:60092) 2023-06-09T11:25:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 -- 172.21.15.7:0/3621498852 >> v1:172.21.15.117:6789/0 conn(0x7f240c071c80 legacy=0x7f240c159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 -- 172.21.15.7:0/3621498852 >> v1:172.21.15.7:6789/0 conn(0x7f240c153960 legacy=0x7f240c06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.289+0000 7f240bfff700 1 -- 172.21.15.7:0/3621498852 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c15d3c0 con 0x7f240c0af980 2023-06-09T11:25:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f240bfff700 1 --2- 172.21.15.7:0/3621498852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c0af980 0x7f240c165010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2404009e50 tx=0x7f240400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ebc50e09ff803e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f240a7fc700 1 -- 172.21.15.7:0/3621498852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400e070 con 0x7f240c0af980 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f240a7fc700 1 -- 172.21.15.7:0/3621498852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240400b9c0 con 0x7f240c0af980 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f240a7fc700 1 -- 172.21.15.7:0/3621498852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240400bd70 con 0x7f240c0af980 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 -- 172.21.15.7:0/3621498852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c0af980 msgr2=0x7f240c165010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 --2- 172.21.15.7:0/3621498852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c0af980 0x7f240c165010 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2404009e50 tx=0x7f240400b330 comp rx=0 tx=0).stop 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 -- 172.21.15.7:0/3621498852 shutdown_connections 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 --2- 172.21.15.7:0/3621498852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c0af980 0x7f240c165010 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 -- 172.21.15.7:0/3621498852 >> 172.21.15.7:0/3621498852 conn(0x7f240c0c9f00 msgr2=0x7f240c0ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:04.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.290+0000 7f2411dff700 1 -- 172.21.15.7:0/3621498852 shutdown_connections 2023-06-09T11:25:04.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.291+0000 7f2411dff700 1 -- 172.21.15.7:0/3621498852 wait complete. 2023-06-09T11:25:04.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.292+0000 7f2411dff700 1 Processor -- start 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.292+0000 7f2411dff700 1 -- start start 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 0x7f240c147150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 0x7f240c0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c149130 con 0x7f240c153960 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0c2de0 con 0x7f240c0af980 2023-06-09T11:25:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f2411dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0cd080 con 0x7f240c071c80 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f240bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f240bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49384/0 (socket says 172.21.15.7:49384) 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.293+0000 7f240bfff700 1 -- 172.21.15.7:0/307296177 learned_addr learned my addr 172.21.15.7:0/307296177 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240affd700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 0x7f240c0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240b7fe700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 0x7f240c147150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 -- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 msgr2=0x7f240c147150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 0x7f240c147150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 -- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 msgr2=0x7f240c0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 0x7f240c0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 -- 172.21.15.7:0/307296177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2404009b30 con 0x7f240c153960 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240b7fe700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 0x7f240c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240affd700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 0x7f240c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f240bfff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f240400b730 tx=0x7f2404004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404004960 con 0x7f240c153960 2023-06-09T11:25:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040092e0 con 0x7f240c153960 2023-06-09T11:25:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240401c660 con 0x7f240c153960 2023-06-09T11:25:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.294+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c15d980 con 0x7f240c153960 2023-06-09T11:25:04.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.295+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c15d6a0 con 0x7f240c153960 2023-06-09T11:25:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.296+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c0c99c0 con 0x7f240c153960 2023-06-09T11:25:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.297+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f240401c7c0 con 0x7f240c153960 2023-06-09T11:25:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.298+0000 7f23f3fff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 0x7f23f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.298+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6722+0+0 (secure 0 0 0) 0x7f240408b5a0 con 0x7f240c153960 2023-06-09T11:25:04.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.298+0000 7f240b7fe700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 0x7f23f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.298+0000 7f240b7fe700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 0x7f23f806d1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000f60 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.302+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f24040563a0 con 0x7f240c153960 2023-06-09T11:25:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.559+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f240c0aded0 con 0x7f240c153960 2023-06-09T11:25:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.559+0000 7f23f3fff700 1 -- 172.21.15.7:0/307296177 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5461 (secure 0 0 0) 0x7f24040599c0 con 0x7f240c153960 2023-06-09T11:25:04.565 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:04.565 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:04.306848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 msgr2=0x7f23f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 0x7f23f806d1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000f60 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2023-06-09T11:25:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 msgr2=0x7f240c0cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f240400b730 tx=0x7f2404004ad0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 shutdown_connections 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f240c153960 0x7f240c0cca60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f23f806ad10 0x7f23f806d1c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f240c071c80 0x7f240c147150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 --2- 172.21.15.7:0/307296177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f240c0af980 0x7f240c0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 >> 172.21.15.7:0/307296177 conn(0x7f240c0c9f00 msgr2=0x7f240c1527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 shutdown_connections 2023-06-09T11:25:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.562+0000 7f2411dff700 1 -- 172.21.15.7:0/307296177 wait complete. 2023-06-09T11:25:04.570 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2023-06-09T11:25:04.586 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-06-09T11:25:04.586 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:25:04.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.783+0000 7f02b29ff700 1 Processor -- start 2023-06-09T11:25:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.783+0000 7f02b29ff700 1 -- start start 2023-06-09T11:25:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac0c1bc0 0x7f02ac1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 0x7f02ac159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac070110 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac071300 con 0x7f02ac0c0140 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0cbd20 con 0x7f02ac0c1f90 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02abfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac0c1bc0 0x7f02ac1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02abfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02ac0c1bc0 0x7f02ac1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49392/0 (socket says 172.21.15.7:49392) 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02abfff700 1 -- 172.21.15.7:0/393135635 learned_addr learned my addr 172.21.15.7:0/393135635 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02b0f9c700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.785+0000 7f02ab7fe700 1 --2- 172.21.15.7:0/393135635 >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 0x7f02ac159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 -- 172.21.15.7:0/393135635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 msgr2=0x7f02ac1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac1501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 -- 172.21.15.7:0/393135635 >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 msgr2=0x7f02ac159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 --2- 172.21.15.7:0/393135635 >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 0x7f02ac159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 -- 172.21.15.7:0/393135635 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0c0bf0 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02ab7fe700 1 --2- 172.21.15.7:0/393135635 >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 0x7f02ac159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02b0f9c700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02abfff700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac1599b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02ac06d950 tx=0x7f029c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03d59e2d6bde696 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02aaffd700 1 -- 172.21.15.7:0/393135635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00b860 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02aaffd700 1 -- 172.21.15.7:0/393135635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00b9c0 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.786+0000 7f02aaffd700 1 -- 172.21.15.7:0/393135635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00bd50 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 -- 172.21.15.7:0/393135635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 msgr2=0x7f02ac1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac1599b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f02ac06d950 tx=0x7f029c009c00 comp rx=0 tx=0).stop 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 -- 172.21.15.7:0/393135635 shutdown_connections 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac1501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 --2- 172.21.15.7:0/393135635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac1599b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 --2- 172.21.15.7:0/393135635 >> v2:172.21.15.117:3300/0 conn(0x7f02ac0c0140 0x7f02ac159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 -- 172.21.15.7:0/393135635 >> 172.21.15.7:0/393135635 conn(0x7f02ac06a4d0 msgr2=0x7f02ac06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 -- 172.21.15.7:0/393135635 shutdown_connections 2023-06-09T11:25:04.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.787+0000 7f02b29ff700 1 -- 172.21.15.7:0/393135635 wait complete. 2023-06-09T11:25:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.789+0000 7f02b29ff700 1 Processor -- start 2023-06-09T11:25:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.789+0000 7f02b29ff700 1 -- start start 2023-06-09T11:25:04.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02ac0c0140 0x7f02ac0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac14adf0 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac14b3b0 con 0x7f02ac0c0140 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b29ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac14ccb0 con 0x7f02ac0c1f90 2023-06-09T11:25:04.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49396/0 (socket says 172.21.15.7:49396) 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 -- 172.21.15.7:0/2947415669 learned_addr learned my addr 172.21.15.7:0/2947415669 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02abfff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02ac0c0140 0x7f02ac0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02b0f9c700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 -- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 msgr2=0x7f02ac14dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac14dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 -- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02ac0c0140 msgr2=0x7f02ac0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02ac0c0140 0x7f02ac0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.790+0000 7f02ab7fe700 1 -- 172.21.15.7:0/2947415669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c009550 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02ab7fe700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d800 tx=0x7f02a000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0004030 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0004190 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac0c0e90 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0004520 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.791+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac0702f0 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.792+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac0c1170 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.793+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f02a00036a0 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.794+0000 7f02a8ff9700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 0x7f029806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:04.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.794+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f02a0089d80 con 0x7f02ac0c1bc0 2023-06-09T11:25:04.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.794+0000 7f02abfff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 0x7f029806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:04.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.795+0000 7f02abfff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 0x7f029806d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f029c000c00 tx=0x7f029c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:04.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:04.798+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f02a0058530 con 0x7f02ac0c1bc0 2023-06-09T11:25:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.049+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02ac074cf0 con 0x7f02ac0c1bc0 2023-06-09T11:25:05.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.050+0000 7f02a8ff9700 1 -- 172.21.15.7:0/2947415669 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5461 (secure 0 0 0) 0x7f02a00580c0 con 0x7f02ac0c1bc0 2023-06-09T11:25:05.056 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:05.056 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:04.306848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 msgr2=0x7f029806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 0x7f029806d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f029c000c00 tx=0x7f029c008030 comp rx=0 tx=0).stop 2023-06-09T11:25:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 msgr2=0x7f02ac14d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d800 tx=0x7f02a000db10 comp rx=0 tx=0).stop 2023-06-09T11:25:05.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 shutdown_connections 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f029806ace0 0x7f029806d190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02ac0c1f90 0x7f02ac14dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f02ac0c0140 0x7f02ac0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 --2- 172.21.15.7:0/2947415669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02ac0c1bc0 0x7f02ac14d840 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 >> 172.21.15.7:0/2947415669 conn(0x7f02ac06a4d0 msgr2=0x7f02ac14ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 shutdown_connections 2023-06-09T11:25:05.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:05.052+0000 7f02b29ff700 1 -- 172.21.15.7:0/2947415669 wait complete. 2023-06-09T11:25:05.060 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2023-06-09T11:25:05.077 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 67, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 67, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 36 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:25:04.306848+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5692, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:25:07.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.283+0000 7f883cdff700 1 Processor -- start 2023-06-09T11:25:07.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.284+0000 7f883cdff700 1 -- start start 2023-06-09T11:25:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f883cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88380c5b40 0x7f8838070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f883cdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8838071060 0x7f8838158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f883cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88381580a0 con 0x7f8838151510 2023-06-09T11:25:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f883cdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88380714e0 con 0x7f8838071060 2023-06-09T11:25:07.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f883cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88381518e0 con 0x7f88380c5b40 2023-06-09T11:25:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f8836d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8838071060 0x7f8838158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f8836d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8838071060 0x7f8838158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49404/0 (socket says 172.21.15.7:49404) 2023-06-09T11:25:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f8836d9d700 1 -- 172.21.15.7:0/1766105436 learned_addr learned my addr 172.21.15.7:0/1766105436 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.285+0000 7f882ffff700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88380c5b40 0x7f8838070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8836d9d700 1 -- 172.21.15.7:0/1766105436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88380c5b40 msgr2=0x7f8838070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8836d9d700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88380c5b40 0x7f8838070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8836d9d700 1 -- 172.21.15.7:0/1766105436 >> v1:172.21.15.117:6789/0 conn(0x7f8838151510 legacy=0x7f883815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8836d9d700 1 -- 172.21.15.7:0/1766105436 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88380c6180 con 0x7f8838071060 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8836d9d700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838071060 0x7f8838158870 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8830009e80 tx=0x7f883000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa59362b6484a700 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8835d9b700 1 -- 172.21.15.7:0/1766105436 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000e070 con 0x7f8838071060 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8835d9b700 1 -- 172.21.15.7:0/1766105436 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883000b8d0 con 0x7f8838071060 2023-06-09T11:25:07.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.286+0000 7f8835d9b700 1 -- 172.21.15.7:0/1766105436 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000bc80 con 0x7f8838071060 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 -- 172.21.15.7:0/1766105436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838071060 msgr2=0x7f8838158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838071060 0x7f8838158870 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8830009e80 tx=0x7f883000b330 comp rx=0 tx=0).stop 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 -- 172.21.15.7:0/1766105436 shutdown_connections 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8838071060 0x7f8838158870 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 --2- 172.21.15.7:0/1766105436 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88380c5b40 0x7f8838070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 -- 172.21.15.7:0/1766105436 >> 172.21.15.7:0/1766105436 conn(0x7f883806a4d0 msgr2=0x7f883806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:07.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 -- 172.21.15.7:0/1766105436 shutdown_connections 2023-06-09T11:25:07.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.287+0000 7f883cdff700 1 -- 172.21.15.7:0/1766105436 wait complete. 2023-06-09T11:25:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.289+0000 7f883cdff700 1 Processor -- start 2023-06-09T11:25:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.289+0000 7f883cdff700 1 -- start start 2023-06-09T11:25:07.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8838071060 0x7f88380c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8838151510 0x7f8838146d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88380c7400 con 0x7f88380c5b40 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88381ef6b0 con 0x7f8838151510 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8838149110 con 0x7f8838071060 2023-06-09T11:25:07.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49416/0 (socket says 172.21.15.7:49416) 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 -- 172.21.15.7:0/3147239725 learned_addr learned my addr 172.21.15.7:0/3147239725 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f883659c700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8838071060 0x7f88380c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f8836d9d700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8838151510 0x7f8838146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 -- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8838071060 msgr2=0x7f88380c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8838071060 0x7f88380c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 -- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8838151510 msgr2=0x7f8838146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8838151510 0x7f8838146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.290+0000 7f882ffff700 1 -- 172.21.15.7:0/3147239725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8830009b30 con 0x7f88380c5b40 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f882ffff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9b0 tx=0x7f882000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820009710 con 0x7f88380c5b40 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882000c850 con 0x7f88380c5b40 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8838157770 con 0x7f88380c5b40 2023-06-09T11:25:07.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882000cbc0 con 0x7f88380c5b40 2023-06-09T11:25:07.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.291+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8838157490 con 0x7f88380c5b40 2023-06-09T11:25:07.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.292+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8838157a50 con 0x7f88380c5b40 2023-06-09T11:25:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.293+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f882000cd20 con 0x7f88380c5b40 2023-06-09T11:25:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.294+0000 7f882effd700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 0x7f881806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.294+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f8820089950 con 0x7f88380c5b40 2023-06-09T11:25:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.294+0000 7f883659c700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 0x7f881806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.295+0000 7f883659c700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 0x7f881806d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f60 tx=0x7f8828009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:07.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.298+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8820058100 con 0x7f88380c5b40 2023-06-09T11:25:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.556+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8838074cf0 con 0x7f88380c5b40 2023-06-09T11:25:07.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.557+0000 7f882effd700 1 -- 172.21.15.7:0/3147239725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5461 (secure 0 0 0) 0x7f8820057c90 con 0x7f88380c5b40 2023-06-09T11:25:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:07.563 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:04.306848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 msgr2=0x7f881806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 0x7f881806d190 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8828000f60 tx=0x7f8828009040 comp rx=0 tx=0).stop 2023-06-09T11:25:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 msgr2=0x7f8838146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:07.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f882000e9b0 tx=0x7f882000ed70 comp rx=0 tx=0).stop 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 shutdown_connections 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8838151510 0x7f8838146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f881806ace0 0x7f881806d190 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8838071060 0x7f88380c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 --2- 172.21.15.7:0/3147239725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88380c5b40 0x7f8838146830 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:07.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 >> 172.21.15.7:0/3147239725 conn(0x7f883806a4d0 msgr2=0x7f883814e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 shutdown_connections 2023-06-09T11:25:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:07.559+0000 7f883cdff700 1 -- 172.21.15.7:0/3147239725 wait complete. 2023-06-09T11:25:07.567 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2023-06-09T11:25:07.583 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:09.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 Processor -- start 2023-06-09T11:25:09.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 -- start start 2023-06-09T11:25:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7fc153880 0x7fd7fc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd7fc15b2d0 0x7fd7fc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7fc0c5f10 con 0x7fd7fc153880 2023-06-09T11:25:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7fc15a500 con 0x7fd7fc15b2d0 2023-06-09T11:25:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.788+0000 7fd8045ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc15da00 con 0x7fd7fc0c9f40 2023-06-09T11:25:09.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7fc153880 0x7fd7fc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7fc153880 0x7fd7fc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49424/0 (socket says 172.21.15.7:49424) 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 -- 172.21.15.7:0/4142203722 learned_addr learned my addr 172.21.15.7:0/4142203722 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd802b9c700 1 --2- 172.21.15.7:0/4142203722 >> v2:172.21.15.117:3300/0 conn(0x7fd7fc15b2d0 0x7fd7fc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd801b9a700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 -- 172.21.15.7:0/4142203722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 msgr2=0x7fd7fc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 -- 172.21.15.7:0/4142203722 >> v2:172.21.15.117:3300/0 conn(0x7fd7fc15b2d0 msgr2=0x7fd7fc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 --2- 172.21.15.7:0/4142203722 >> v2:172.21.15.117:3300/0 conn(0x7fd7fc15b2d0 0x7fd7fc0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.789+0000 7fd80239b700 1 -- 172.21.15.7:0/4142203722 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7fc15b750 con 0x7fd7fc153880 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd80239b700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc15d4c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc071db0 tx=0x7fd7f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f275c338d8f46451 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd801399700 1 -- 172.21.15.7:0/4142203722 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8011830 con 0x7fd7fc153880 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd801399700 1 -- 172.21.15.7:0/4142203722 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f8011990 con 0x7fd7fc153880 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd801399700 1 -- 172.21.15.7:0/4142203722 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f8011d20 con 0x7fd7fc153880 2023-06-09T11:25:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 -- 172.21.15.7:0/4142203722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 msgr2=0x7fd7fc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc15d4c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc071db0 tx=0x7fd7f8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 -- 172.21.15.7:0/4142203722 shutdown_connections 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 --2- 172.21.15.7:0/4142203722 >> v2:172.21.15.117:3300/0 conn(0x7fd7fc15b2d0 0x7fd7fc0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc15d4c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 --2- 172.21.15.7:0/4142203722 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.790+0000 7fd8045ff700 1 -- 172.21.15.7:0/4142203722 >> 172.21.15.7:0/4142203722 conn(0x7fd7fc0737f0 msgr2=0x7fd7fc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.791+0000 7fd8045ff700 1 -- 172.21.15.7:0/4142203722 shutdown_connections 2023-06-09T11:25:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.791+0000 7fd8045ff700 1 -- 172.21.15.7:0/4142203722 wait complete. 2023-06-09T11:25:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.792+0000 7fd8045ff700 1 Processor -- start 2023-06-09T11:25:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.792+0000 7fd8045ff700 1 -- start start 2023-06-09T11:25:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7fc15b2d0 0x7fd7fc1ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc0b4c30 con 0x7fd7fc153880 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc1eef70 con 0x7fd7fc15b2d0 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.793+0000 7fd8045ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc1f25f0 con 0x7fd7fc0c9f40 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49430/0 (socket says 172.21.15.7:49430) 2023-06-09T11:25:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 -- 172.21.15.7:0/1867431793 learned_addr learned my addr 172.21.15.7:0/1867431793 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd80239b700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd802b9c700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7fc15b2d0 0x7fd7fc1ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 -- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 msgr2=0x7fd7fc14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 -- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7fc15b2d0 msgr2=0x7fd7fc1ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7fc15b2d0 0x7fd7fc1ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:09.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 -- 172.21.15.7:0/1867431793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8009550 con 0x7fd7fc153880 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.794+0000 7fd801b9a700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e930 tx=0x7fd7ec00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.795+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec013680 con 0x7fd7fc153880 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.795+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ec0137e0 con 0x7fd7fc153880 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.795+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7fc159980 con 0x7fd7fc153880 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.795+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec013b70 con 0x7fd7fc153880 2023-06-09T11:25:09.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.795+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7fc1596e0 con 0x7fd7fc153880 2023-06-09T11:25:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.796+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7fc159c60 con 0x7fd7fc153880 2023-06-09T11:25:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.797+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7ec013cd0 con 0x7fd7fc153880 2023-06-09T11:25:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.798+0000 7fd7f2ffd700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 0x7fd7e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.798+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fd7ec089b20 con 0x7fd7fc153880 2023-06-09T11:25:09.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.798+0000 7fd80239b700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 0x7fd7e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:09.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.799+0000 7fd80239b700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 0x7fd7e006d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009c10 tx=0x7fd7f8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:09.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:09.802+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd7ec054840 con 0x7fd7fc153880 2023-06-09T11:25:10.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.054+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7fc074cf0 con 0x7fd7fc153880 2023-06-09T11:25:10.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.055+0000 7fd7f2ffd700 1 -- 172.21.15.7:0/1867431793 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+5461 (secure 0 0 0) 0x7fd7ec057e60 con 0x7fd7fc153880 2023-06-09T11:25:10.060 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:10.060 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:04.306848+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:10.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 msgr2=0x7fd7e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 0x7fd7e006d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009c10 tx=0x7fd7f8008030 comp rx=0 tx=0).stop 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 msgr2=0x7fd7fc1ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00e930 tx=0x7fd7ec00ec40 comp rx=0 tx=0).stop 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 shutdown_connections 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7fc15b2d0 0x7fd7fc1ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7e006ace0 0x7fd7e006d190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:10.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7fc0c9f40 0x7fd7fc14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 --2- 172.21.15.7:0/1867431793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7fc153880 0x7fd7fc1ef380 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.057+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 >> 172.21.15.7:0/1867431793 conn(0x7fd7fc0737f0 msgr2=0x7fd7fc15b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.058+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 shutdown_connections 2023-06-09T11:25:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:10.058+0000 7fd8045ff700 1 -- 172.21.15.7:0/1867431793 wait complete. 2023-06-09T11:25:10.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 67 2023-06-09T11:25:10.081 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:12.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.286+0000 7f9ababff700 1 Processor -- start 2023-06-09T11:25:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.286+0000 7f9ababff700 1 -- start start 2023-06-09T11:25:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.287+0000 7f9ababff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ab40cc5b0 0x7f9ab40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.287+0000 7f9ababff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab4159b20 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.287+0000 7f9ababff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab40cac80 con 0x7f9ab40cb5d0 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.287+0000 7f9ababff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab41597d0 con 0x7f9ab40cc5b0 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9ab40cb9a0 0x7f9ab40cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60100/0 (socket says 172.21.15.7:60100) 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab3fff700 1 -- 172.21.15.7:0/182038074 learned_addr learned my addr 172.21.15.7:0/182038074 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:12.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab919c700 1 --2- 172.21.15.7:0/182038074 >> v2:172.21.15.117:3300/0 conn(0x7f9ab40cc5b0 0x7f9ab40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744022616 0 0) 0x7f9ab4159b20 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa40035b0 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162978563 0 0) 0x7f9ab40cac80 con 0x7f9ab40cb5d0 2023-06-09T11:25:12.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ab4159b20 con 0x7f9ab40cb5d0 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2072506691 0 0) 0x7f9aa40035b0 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 >> v2:172.21.15.117:3300/0 conn(0x7f9ab40cc5b0 msgr2=0x7f9ab40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 --2- 172.21.15.7:0/182038074 >> v2:172.21.15.117:3300/0 conn(0x7f9ab40cc5b0 0x7f9ab40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 >> v1:172.21.15.117:6789/0 conn(0x7f9ab40cb5d0 legacy=0x7f9ab4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.288+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab4071ab0 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab919c700 1 --2- 172.21.15.7:0/182038074 >> v2:172.21.15.117:3300/0 conn(0x7f9ab40cc5b0 0x7f9ab40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:25:12.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f9aa0001670 con 0x7f9ab40cb9a0 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 >> v1:172.21.15.7:6789/0 conn(0x7f9ab40cb9a0 legacy=0x7f9ab40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 0x7f9aa4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4004850 0x7f9aa4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa4005200 0x7f9aa400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab41597d0 con 0x7f9aa4003b90 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa400c000 con 0x7f9aa4005200 2023-06-09T11:25:12.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa400c190 con 0x7f9aa4004850 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 0x7f9aa4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab919c700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa4005200 0x7f9aa400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab3fff700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4004850 0x7f9aa4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 -- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4004850 msgr2=0x7f9aa4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4004850 0x7f9aa4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 -- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa4005200 msgr2=0x7f9aa400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:12.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9aa4005200 0x7f9aa400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab899b700 1 -- 172.21.15.7:0/182038074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa40035b0 con 0x7f9aa4003b90 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.289+0000 7f9ab919c700 1 -- 172.21.15.7:0/182038074 reap_dead start 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ab899b700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 0x7f9aa4003f80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8001940 tx=0x7f9aa4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d44ef321c55104f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8001430 con 0x7f9aa4003b90 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa80052d0 con 0x7f9aa4003b90 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ab37fe700 1 -- 172.21.15.7:0/182038074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8001d10 con 0x7f9aa4003b90 2023-06-09T11:25:12.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ababff700 1 -- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 msgr2=0x7f9aa4003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.290+0000 7f9ababff700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 0x7f9aa4003f80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8001940 tx=0x7f9aa4005880 comp rx=0 tx=0).stop 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.291+0000 7f9ababff700 1 -- 172.21.15.7:0/182038074 shutdown_connections 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.291+0000 7f9ababff700 1 --2- 172.21.15.7:0/182038074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9aa4003b90 0x7f9aa4003f80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.291+0000 7f9ababff700 1 -- 172.21.15.7:0/182038074 >> 172.21.15.7:0/182038074 conn(0x7f9ab4070af0 msgr2=0x7f9ab4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.291+0000 7f9ababff700 1 -- 172.21.15.7:0/182038074 shutdown_connections 2023-06-09T11:25:12.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.291+0000 7f9ababff700 1 -- 172.21.15.7:0/182038074 wait complete. 2023-06-09T11:25:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.293+0000 7f9ababff700 1 Processor -- start 2023-06-09T11:25:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.293+0000 7f9ababff700 1 -- start start 2023-06-09T11:25:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.294+0000 7f9ababff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 0x7f9ab41506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.294+0000 7f9ababff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 0x7f9ab415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.294+0000 7f9ababff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab415b830 0x7f9ab4155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ababff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40c36d0 con 0x7f9ab4151920 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ababff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab41f04d0 con 0x7f9ab415b830 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ababff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab415bf10 con 0x7f9aa4003b90 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 0x7f9ab415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab899b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 0x7f9ab41506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab899b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 0x7f9ab41506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:33446/0 (socket says 172.21.15.7:33446) 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab899b700 1 -- 172.21.15.7:0/2275133014 learned_addr learned my addr 172.21.15.7:0/2275133014 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab919c700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab415b830 0x7f9ab4155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 -- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 msgr2=0x7f9ab41506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 0x7f9ab41506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 -- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab415b830 msgr2=0x7f9ab4155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab415b830 0x7f9ab4155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 -- 172.21.15.7:0/2275133014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab415d4f0 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.295+0000 7f9ab3fff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 0x7f9ab415b2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000c00 tx=0x7f9aa0007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.296+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0011580 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.296+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa00116e0 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.296+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0011a90 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.296+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab415d210 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.296+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab415d7d0 con 0x7f9ab4151920 2023-06-09T11:25:12.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.297+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab41f0800 con 0x7f9ab4151920 2023-06-09T11:25:12.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.298+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9aa0011bf0 con 0x7f9ab4151920 2023-06-09T11:25:12.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.299+0000 7f9ab17fa700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 0x7f9a9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:12.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.299+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f9aa0089820 con 0x7f9ab4151920 2023-06-09T11:25:12.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.299+0000 7f9ab899b700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 0x7f9a9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:12.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.300+0000 7f9ab899b700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 0x7f9a9806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8015040 tx=0x7f9aa800a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:12.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.303+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9aa00544b0 con 0x7f9ab4151920 2023-06-09T11:25:12.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.557+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab4074cf0 con 0x7f9ab4151920 2023-06-09T11:25:12.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.558+0000 7f9ab17fa700 1 -- 172.21.15.7:0/2275133014 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f9aa0057ad0 con 0x7f9ab4151920 2023-06-09T11:25:12.563 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:12.563 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:10.339507+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:resolve","state_seq":7,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:12.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.560+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 msgr2=0x7f9a9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.560+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 0x7f9a9806d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8015040 tx=0x7f9aa800a000 comp rx=0 tx=0).stop 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.560+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 msgr2=0x7f9ab415b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.560+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 0x7f9ab415b2f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0000c00 tx=0x7f9aa0007570 comp rx=0 tx=0).stop 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 shutdown_connections 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ab415b830 0x7f9ab4155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a9806ace0 0x7f9a9806d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9aa4003b90 0x7f9ab41506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 --2- 172.21.15.7:0/2275133014 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ab4151920 0x7f9ab415b2f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 >> 172.21.15.7:0/2275133014 conn(0x7f9ab4070af0 msgr2=0x7f9ab4157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 shutdown_connections 2023-06-09T11:25:12.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:12.561+0000 7f9ababff700 1 -- 172.21.15.7:0/2275133014 wait complete. 2023-06-09T11:25:12.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2023-06-09T11:25:12.584 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:14.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:14.515 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:14.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.782+0000 7f79b45ff700 1 Processor -- start 2023-06-09T11:25:14.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.782+0000 7f79b45ff700 1 -- start start 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b45ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79ac15b4a0 0x7f79ac15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b45ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79ac0c9f40 0x7f79ac0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b45ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac15a500 con 0x7f79ac153880 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b45ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac0c5f10 con 0x7f79ac0c9f40 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b45ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ac15da00 con 0x7f79ac15b4a0 2023-06-09T11:25:14.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b239b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f79ac153880 0x7f79ac15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43406/0 (socket says 172.21.15.7:43406) 2023-06-09T11:25:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b239b700 1 -- 172.21.15.7:0/3922482434 learned_addr learned my addr 172.21.15.7:0/3922482434 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:14.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/3922482434 >> v2:172.21.15.7:3300/0 conn(0x7f79ac0c9f40 0x7f79ac0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b1b9a700 1 --2- 172.21.15.7:0/3922482434 >> v2:172.21.15.117:3300/0 conn(0x7f79ac15b4a0 0x7f79ac15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b1399700 1 -- 172.21.15.7:0/3922482434 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647495066 0 0) 0x7f79a8001110 con 0x7f79ac153880 2023-06-09T11:25:14.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b2b9c700 1 -- 172.21.15.7:0/3922482434 >> v2:172.21.15.117:3300/0 conn(0x7f79ac15b4a0 msgr2=0x7f79ac15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.784+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/3922482434 >> v2:172.21.15.117:3300/0 conn(0x7f79ac15b4a0 0x7f79ac15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b2b9c700 1 -- 172.21.15.7:0/3922482434 >> v1:172.21.15.7:6789/0 conn(0x7f79ac153880 legacy=0x7f79ac15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b2b9c700 1 -- 172.21.15.7:0/3922482434 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac1596e0 con 0x7f79ac0c9f40 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/3922482434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac0c9f40 0x7f79ac0c4e30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0cce30 tx=0x7f79a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f2c1bbc7529f44 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b1399700 1 -- 172.21.15.7:0/3922482434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400e070 con 0x7f79ac0c9f40 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b1399700 1 -- 172.21.15.7:0/3922482434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a400bb20 con 0x7f79ac0c9f40 2023-06-09T11:25:14.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.785+0000 7f79b1399700 1 -- 172.21.15.7:0/3922482434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4012430 con 0x7f79ac0c9f40 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 -- 172.21.15.7:0/3922482434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac0c9f40 msgr2=0x7f79ac0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 --2- 172.21.15.7:0/3922482434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac0c9f40 0x7f79ac0c4e30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0cce30 tx=0x7f79a400b330 comp rx=0 tx=0).stop 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 -- 172.21.15.7:0/3922482434 shutdown_connections 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 --2- 172.21.15.7:0/3922482434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac0c9f40 0x7f79ac0c4e30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 --2- 172.21.15.7:0/3922482434 >> v2:172.21.15.117:3300/0 conn(0x7f79ac15b4a0 0x7f79ac15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 -- 172.21.15.7:0/3922482434 >> 172.21.15.7:0/3922482434 conn(0x7f79ac0737f0 msgr2=0x7f79ac0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:14.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 -- 172.21.15.7:0/3922482434 shutdown_connections 2023-06-09T11:25:14.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.786+0000 7f79b45ff700 1 -- 172.21.15.7:0/3922482434 wait complete. 2023-06-09T11:25:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.788+0000 7f79b45ff700 1 Processor -- start 2023-06-09T11:25:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.788+0000 7f79b45ff700 1 -- start start 2023-06-09T11:25:14.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.789+0000 7f79b45ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 0x7f79ac14b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ac153880 0x7f79ac0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b45ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b45ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac150da0 con 0x7f79ac15b4a0 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac14bd70 con 0x7f79ac153880 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b45ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac152ed0 con 0x7f79ac0c9f40 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42872/0 (socket says 172.21.15.7:42872) 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 -- 172.21.15.7:0/600574494 learned_addr learned my addr 172.21.15.7:0/600574494 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b1b9a700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ac153880 0x7f79ac0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b239b700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 0x7f79ac14b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 -- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 msgr2=0x7f79ac14b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 0x7f79ac14b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 -- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ac153880 msgr2=0x7f79ac0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:14.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ac153880 0x7f79ac0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b2b9c700 1 -- 172.21.15.7:0/600574494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a4009a70 con 0x7f79ac15b4a0 2023-06-09T11:25:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.790+0000 7f79b239b700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 0x7f79ac14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:14.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79b2b9c700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f79a4012810 tx=0x7f79a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4012430 con 0x7f79ac15b4a0 2023-06-09T11:25:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4020070 con 0x7f79ac15b4a0 2023-06-09T11:25:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac159c60 con 0x7f79ac15b4a0 2023-06-09T11:25:14.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4004280 con 0x7f79ac15b4a0 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.791+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac159980 con 0x7f79ac15b4a0 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.792+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac159f40 con 0x7f79ac15b4a0 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.793+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79a40043e0 con 0x7f79ac15b4a0 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.794+0000 7f79a2ffd700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 0x7f799006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.794+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f79a4022070 con 0x7f79ac15b4a0 2023-06-09T11:25:14.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.794+0000 7f79b239b700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 0x7f799006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:14.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.795+0000 7f79b239b700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 0x7f799006d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f79a8007fd0 tx=0x7f79a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:14.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:14.798+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f79a40562a0 con 0x7f79ac15b4a0 2023-06-09T11:25:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.055+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79ac074cf0 con 0x7f79ac15b4a0 2023-06-09T11:25:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.055+0000 7f79a2ffd700 1 -- 172.21.15.7:0/600574494 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f79a40598c0 con 0x7f79ac15b4a0 2023-06-09T11:25:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:15.061 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:10.339507+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:resolve","state_seq":7,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 msgr2=0x7f799006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 0x7f799006d190 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f79a8007fd0 tx=0x7f79a8009040 comp rx=0 tx=0).stop 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 msgr2=0x7f79ac15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f79a4012810 tx=0x7f79a4004ad0 comp rx=0 tx=0).stop 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 shutdown_connections 2023-06-09T11:25:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79ac15b4a0 0x7f79ac15bb60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f799006ace0 0x7f799006d190 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79ac0c9f40 0x7f79ac14b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 --2- 172.21.15.7:0/600574494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79ac153880 0x7f79ac0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 >> 172.21.15.7:0/600574494 conn(0x7f79ac0737f0 msgr2=0x7f79ac1e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 shutdown_connections 2023-06-09T11:25:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:15.058+0000 7f79b45ff700 1 -- 172.21.15.7:0/600574494 wait complete. 2023-06-09T11:25:15.065 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2023-06-09T11:25:15.080 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:17.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:17.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.287+0000 7fc0513ff700 1 Processor -- start 2023-06-09T11:25:17.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.287+0000 7fc0513ff700 1 -- start start 2023-06-09T11:25:17.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.288+0000 7fc0513ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:17.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.288+0000 7fc0513ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15a500 con 0x7fc04c0c9f40 2023-06-09T11:25:17.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.288+0000 7fc0513ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15da00 con 0x7fc04c15b4a0 2023-06-09T11:25:17.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.288+0000 7fc0513ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0c5f10 con 0x7fc04c153880 2023-06-09T11:25:17.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:17.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc04c0c9f40 0x7fc04c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43422/0 (socket says 172.21.15.7:43422) 2023-06-09T11:25:17.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04a7fc700 1 -- 172.21.15.7:0/2577196553 learned_addr learned my addr 172.21.15.7:0/2577196553 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:17.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04affd700 1 -- 172.21.15.7:0/2577196553 >> v1:172.21.15.117:6789/0 conn(0x7fc04c15b4a0 legacy=0x7fc04c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04affd700 1 -- 172.21.15.7:0/2577196553 >> v1:172.21.15.7:6789/0 conn(0x7fc04c0c9f40 legacy=0x7fc04c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04affd700 1 -- 172.21.15.7:0/2577196553 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c1596e0 con 0x7fc04c153880 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc049ffb700 1 -- 172.21.15.7:0/2577196553 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679776344 0 0) 0x7fc040001110 con 0x7fc04c0c9f40 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.289+0000 7fc04affd700 1 --2- 172.21.15.7:0/2577196553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009840 tx=0x7fc03c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5ff46579cf6234 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc049ffb700 1 -- 172.21.15.7:0/2577196553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c004230 con 0x7fc04c153880 2023-06-09T11:25:17.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc049ffb700 1 -- 172.21.15.7:0/2577196553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c004390 con 0x7fc04c153880 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc049ffb700 1 -- 172.21.15.7:0/2577196553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0036a0 con 0x7fc04c153880 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 -- 172.21.15.7:0/2577196553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 msgr2=0x7fc04c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 --2- 172.21.15.7:0/2577196553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc03c009840 tx=0x7fc03c009b50 comp rx=0 tx=0).stop 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 -- 172.21.15.7:0/2577196553 shutdown_connections 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 --2- 172.21.15.7:0/2577196553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 -- 172.21.15.7:0/2577196553 >> 172.21.15.7:0/2577196553 conn(0x7fc04c0737f0 msgr2=0x7fc04c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:17.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 -- 172.21.15.7:0/2577196553 shutdown_connections 2023-06-09T11:25:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.290+0000 7fc0513ff700 1 -- 172.21.15.7:0/2577196553 wait complete. 2023-06-09T11:25:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.292+0000 7fc0513ff700 1 Processor -- start 2023-06-09T11:25:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.292+0000 7fc0513ff700 1 -- start start 2023-06-09T11:25:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c153880 0x7fc04c0ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:17.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c14bdd0 con 0x7fc04c0c9f40 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c0cd830 con 0x7fc04c153880 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc0513ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c14b810 con 0x7fc04c15b4a0 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc04affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc04affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42892/0 (socket says 172.21.15.7:42892) 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.293+0000 7fc04affd700 1 -- 172.21.15.7:0/977994536 learned_addr learned my addr 172.21.15.7:0/977994536 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:17.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04a7fc700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c153880 0x7fc04c0ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04b7fe700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 -- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 msgr2=0x7fc04c0cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 -- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c153880 msgr2=0x7fc04c0ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c153880 0x7fc04c0ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 -- 172.21.15.7:0/977994536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc03c0094a0 con 0x7fc04c0c9f40 2023-06-09T11:25:17.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.294+0000 7fc04affd700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0090b0 tx=0x7fc03c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.295+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011830 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.295+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c01e070 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.295+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c159c60 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.295+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c011bb0 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.295+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c159980 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.296+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c159f40 con 0x7fc04c0c9f40 2023-06-09T11:25:17.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.297+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc03c011d10 con 0x7fc04c0c9f40 2023-06-09T11:25:17.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.298+0000 7fc0337fe700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 0x7fc03806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:17.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.298+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fc03c08a9e0 con 0x7fc04c0c9f40 2023-06-09T11:25:17.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.298+0000 7fc04a7fc700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 0x7fc03806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:17.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.298+0000 7fc04a7fc700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 0x7fc03806d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc040000f60 tx=0x7fc040007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:17.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.302+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc03c055790 con 0x7fc04c0c9f40 2023-06-09T11:25:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.559+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc04c15da00 con 0x7fc04c0c9f40 2023-06-09T11:25:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.560+0000 7fc0337fe700 1 -- 172.21.15.7:0/977994536 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5464 (secure 0 0 0) 0x7fc04c15da00 con 0x7fc04c0c9f40 2023-06-09T11:25:17.565 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:17.565 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:16.316693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 msgr2=0x7fc03806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 0x7fc03806d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc040000f60 tx=0x7fc040007a40 comp rx=0 tx=0).stop 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 msgr2=0x7fc04c0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0090b0 tx=0x7fc03c004750 comp rx=0 tx=0).stop 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 shutdown_connections 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03806ace0 0x7fc03806d190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c0c9f40 0x7fc04c0cdd80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 --2- 172.21.15.7:0/977994536 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c153880 0x7fc04c0ce2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 >> 172.21.15.7:0/977994536 conn(0x7fc04c0737f0 msgr2=0x7fc04c1e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.562+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 shutdown_connections 2023-06-09T11:25:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:17.563+0000 7fc0513ff700 1 -- 172.21.15.7:0/977994536 wait complete. 2023-06-09T11:25:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 69 2023-06-09T11:25:17.586 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 69, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 69, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5455": { "addr": "172.21.15.117:6835/2166536228", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 2166536228, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 2166536228, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5455, "incarnation": 55, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 35 }, "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 36 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 57, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:25:16.316693+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5692, "mds_2": 5455 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:25:19.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.794+0000 7f400adff700 1 Processor -- start 2023-06-09T11:25:19.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.794+0000 7f400adff700 1 -- start start 2023-06-09T11:25:19.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f400adff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40040c6280 0x7f40040c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f400adff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f400406e460 0x7f400406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f400adff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f400415cac0 con 0x7f400406a4d0 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f400adff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40040c8870 con 0x7f40040c6280 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f400adff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40040c37d0 con 0x7f400406e460 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.795+0000 7f4008b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40040c6280 0x7f40040c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f40040c6280 0x7f40040c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42904/0 (socket says 172.21.15.7:42904) 2023-06-09T11:25:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 -- 172.21.15.7:0/2051671064 learned_addr learned my addr 172.21.15.7:0/2051671064 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f400939c700 1 --2- 172.21.15.7:0/2051671064 >> v2:172.21.15.117:3300/0 conn(0x7f400406e460 0x7f400406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 -- 172.21.15.7:0/2051671064 >> v2:172.21.15.117:3300/0 conn(0x7f400406e460 msgr2=0x7f400406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 --2- 172.21.15.7:0/2051671064 >> v2:172.21.15.117:3300/0 conn(0x7f400406e460 0x7f400406cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 -- 172.21.15.7:0/2051671064 >> v1:172.21.15.7:6789/0 conn(0x7f400406a4d0 legacy=0x7f40040737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 -- 172.21.15.7:0/2051671064 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f400415cd30 con 0x7f40040c6280 2023-06-09T11:25:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f4008b9b700 1 --2- 172.21.15.7:0/2051671064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40040c6280 0x7f40040c76b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f400413f730 tx=0x7f3ff8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770caa2764bd2847 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.796+0000 7f40037fe700 1 -- 172.21.15.7:0/2051671064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8011830 con 0x7f40040c6280 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f40037fe700 1 -- 172.21.15.7:0/2051671064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff8011990 con 0x7f40040c6280 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f40037fe700 1 -- 172.21.15.7:0/2051671064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8011d20 con 0x7f40040c6280 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 -- 172.21.15.7:0/2051671064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40040c6280 msgr2=0x7f40040c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 --2- 172.21.15.7:0/2051671064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40040c6280 0x7f40040c76b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f400413f730 tx=0x7f3ff8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 -- 172.21.15.7:0/2051671064 shutdown_connections 2023-06-09T11:25:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 --2- 172.21.15.7:0/2051671064 >> v2:172.21.15.117:3300/0 conn(0x7f400406e460 0x7f400406cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 --2- 172.21.15.7:0/2051671064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40040c6280 0x7f40040c76b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 -- 172.21.15.7:0/2051671064 >> 172.21.15.7:0/2051671064 conn(0x7f400406f980 msgr2=0x7f40040cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 -- 172.21.15.7:0/2051671064 shutdown_connections 2023-06-09T11:25:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.797+0000 7f400adff700 1 -- 172.21.15.7:0/2051671064 wait complete. 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.799+0000 7f400adff700 1 Processor -- start 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.799+0000 7f400adff700 1 -- start start 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 0x7f4004156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40040c6280 0x7f40041eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400413b230 con 0x7f400406e460 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400413c690 con 0x7f40040c6280 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f400adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400413c930 con 0x7f400406a4d0 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f4003fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f4003fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42912/0 (socket says 172.21.15.7:42912) 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.800+0000 7f4003fff700 1 -- 172.21.15.7:0/38368684 learned_addr learned my addr 172.21.15.7:0/38368684 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f400939c700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40040c6280 0x7f40041eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4008b9b700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 0x7f4004156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 -- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 msgr2=0x7f4004156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 0x7f4004156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 -- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40040c6280 msgr2=0x7f40041eff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40040c6280 0x7f40041eff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 -- 172.21.15.7:0/38368684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8009550 con 0x7f400406e460 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4008b9b700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 0x7f4004156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f4003fff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c4e0 tx=0x7f3ff000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00052d0 con 0x7f400406e460 2023-06-09T11:25:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff0005430 con 0x7f400406e460 2023-06-09T11:25:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff00038c0 con 0x7f400406e460 2023-06-09T11:25:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400415d2f0 con 0x7f400406e460 2023-06-09T11:25:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.801+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400415d010 con 0x7f400406e460 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.802+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400415d5d0 con 0x7f400406e460 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.803+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ff0003a20 con 0x7f400406e460 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.804+0000 7f40017fa700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 0x7f3fe806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.805+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f3ff0088d30 con 0x7f400406e460 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.805+0000 7f4008b9b700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 0x7f3fe806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.805+0000 7f4008b9b700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 0x7f3fe806d1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8014040 tx=0x7f3ff8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:19.809+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3ff0053a50 con 0x7f400406e460 2023-06-09T11:25:20.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.059+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4004074cf0 con 0x7f400406e460 2023-06-09T11:25:20.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.059+0000 7f40017fa700 1 -- 172.21.15.7:0/38368684 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+5464 (secure 0 0 0) 0x7f3ff0057070 con 0x7f400406e460 2023-06-09T11:25:20.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:20.065 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:16.316693+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:reconnect","state_seq":9,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:20.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.061+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 msgr2=0x7f3fe806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.061+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 0x7f3fe806d1c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8014040 tx=0x7f3ff8008030 comp rx=0 tx=0).stop 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.061+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 msgr2=0x7f400413b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.061+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000c4e0 tx=0x7f3ff000c8a0 comp rx=0 tx=0).stop 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 shutdown_connections 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f40040c6280 0x7f40041eff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3fe806ad10 0x7f3fe806d1c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:20.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f400406a4d0 0x7f4004156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 --2- 172.21.15.7:0/38368684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f400406e460 0x7f400413b9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 >> 172.21.15.7:0/38368684 conn(0x7f400406f980 msgr2=0x7f400415b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 shutdown_connections 2023-06-09T11:25:20.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:20.062+0000 7f400adff700 1 -- 172.21.15.7:0/38368684 wait complete. 2023-06-09T11:25:20.069 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 69 2023-06-09T11:25:20.085 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:22.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.293+0000 7f87051ff700 1 Processor -- start 2023-06-09T11:25:22.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.293+0000 7f87051ff700 1 -- start start 2023-06-09T11:25:22.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.294+0000 7f87051ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8700151510 0x7f87000c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:22.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.294+0000 7f87051ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87001580a0 con 0x7f8700071230 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.294+0000 7f87051ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000c6180 con 0x7f870015b380 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.294+0000 7f87051ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87001518e0 con 0x7f8700151510 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8700151510 0x7f87000c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8700151510 0x7f87000c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42918/0 (socket says 172.21.15.7:42918) 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 -- 172.21.15.7:0/1590837283 learned_addr learned my addr 172.21.15.7:0/1590837283 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:22.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86ff59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8700071230 0x7f8700158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43444/0 (socket says 172.21.15.7:43444) 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 -- 172.21.15.7:0/1590837283 >> v1:172.21.15.117:6789/0 conn(0x7f870015b380 legacy=0x7f8700070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 -- 172.21.15.7:0/1590837283 >> v1:172.21.15.7:6789/0 conn(0x7f8700071230 legacy=0x7f8700158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 -- 172.21.15.7:0/1590837283 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700157490 con 0x7f8700151510 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fdd9b700 1 -- 172.21.15.7:0/1590837283 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685409474 0 0) 0x7f87001580a0 con 0x7f8700071230 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.295+0000 7f86fed9d700 1 --2- 172.21.15.7:0/1590837283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f87000c5b40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f86f00098f0 tx=0x7f86f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdb8834ae2bf15b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:22.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f86fdd9b700 1 -- 172.21.15.7:0/1590837283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000b8a0 con 0x7f8700151510 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f86fdd9b700 1 -- 172.21.15.7:0/1590837283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f000ba00 con 0x7f8700151510 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f86fdd9b700 1 -- 172.21.15.7:0/1590837283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000bd90 con 0x7f8700151510 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 -- 172.21.15.7:0/1590837283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 msgr2=0x7f87000c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 --2- 172.21.15.7:0/1590837283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f87000c5b40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f86f00098f0 tx=0x7f86f0009c00 comp rx=0 tx=0).stop 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 -- 172.21.15.7:0/1590837283 shutdown_connections 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 --2- 172.21.15.7:0/1590837283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f87000c5b40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 -- 172.21.15.7:0/1590837283 >> 172.21.15.7:0/1590837283 conn(0x7f870006a4d0 msgr2=0x7f870006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:22.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 -- 172.21.15.7:0/1590837283 shutdown_connections 2023-06-09T11:25:22.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.296+0000 7f87051ff700 1 -- 172.21.15.7:0/1590837283 wait complete. 2023-06-09T11:25:22.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.298+0000 7f87051ff700 1 Processor -- start 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.298+0000 7f87051ff700 1 -- start start 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8700071230 0x7f87000c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 0x7f87001eed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001f22b0 con 0x7f8700151510 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87001f3250 con 0x7f870015b380 2023-06-09T11:25:22.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f87051ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000ca5f0 con 0x7f8700071230 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f86fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f86fe59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42926/0 (socket says 172.21.15.7:42926) 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f86fe59c700 1 -- 172.21.15.7:0/531404120 learned_addr learned my addr 172.21.15.7:0/531404120 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f86fed9d700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8700071230 0x7f87000c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.299+0000 7f86ff59e700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 0x7f87001eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 -- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8700071230 msgr2=0x7f87000c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8700071230 0x7f87000c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 -- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 msgr2=0x7f87001eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 0x7f87001eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 -- 172.21.15.7:0/531404120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009550 con 0x7f8700151510 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86ff59e700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 0x7f87001eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86fe59c700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c450 tx=0x7f86f400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40052d0 con 0x7f8700151510 2023-06-09T11:25:22.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4005430 con 0x7f8700151510 2023-06-09T11:25:22.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40038c0 con 0x7f8700151510 2023-06-09T11:25:22.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8700157a50 con 0x7f8700151510 2023-06-09T11:25:22.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.300+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8700157770 con 0x7f8700151510 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.301+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700157d30 con 0x7f8700151510 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.302+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86f4003a20 con 0x7f8700151510 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.303+0000 7f86e77fe700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.303+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f86f4088b50 con 0x7f8700151510 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.304+0000 7f86fed9d700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:22.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.304+0000 7f86fed9d700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 0x7f86ec06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.308+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f86f4053900 con 0x7f8700151510 2023-06-09T11:25:22.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.561+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8700074cf0 con 0x7f8700151510 2023-06-09T11:25:22.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.562+0000 7f86e77fe700 1 -- 172.21.15.7:0/531404120 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f86f4056f20 con 0x7f8700151510 2023-06-09T11:25:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:22.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:22.315529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 msgr2=0x7f86ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 0x7f86ec06d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f86f0000c00 tx=0x7f86f0008030 comp rx=0 tx=0).stop 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 msgr2=0x7f8700159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c450 tx=0x7f86f400c760 comp rx=0 tx=0).stop 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 shutdown_connections 2023-06-09T11:25:22.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f870015b380 0x7f87001eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8700071230 0x7f87000c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 --2- 172.21.15.7:0/531404120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700151510 0x7f8700159c00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 >> 172.21.15.7:0/531404120 conn(0x7f870006a4d0 msgr2=0x7f870014bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 shutdown_connections 2023-06-09T11:25:22.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:22.565+0000 7f87051ff700 1 -- 172.21.15.7:0/531404120 wait complete. 2023-06-09T11:25:22.572 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2023-06-09T11:25:22.588 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:24.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:24.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.795+0000 7f9d387ff700 1 Processor -- start 2023-06-09T11:25:24.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.796+0000 7f9d387ff700 1 -- start start 2023-06-09T11:25:24.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d387ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9d301539b0 0x7f9d30159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d387ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d300c4ce0 0x7f9d300c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d387ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3015a420 con 0x7f9d301535e0 2023-06-09T11:25:24.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d387ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300ca670 con 0x7f9d300c4ce0 2023-06-09T11:25:24.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d387ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d300c5a00 con 0x7f9d301539b0 2023-06-09T11:25:24.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d36d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d300c4ce0 0x7f9d300c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d36d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9d300c4ce0 0x7f9d300c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53052/0 (socket says 172.21.15.7:53052) 2023-06-09T11:25:24.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3510477723 learned_addr learned my addr 172.21.15.7:0/3510477723 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.797+0000 7f9d35d9a700 1 --2- 172.21.15.7:0/3510477723 >> v2:172.21.15.117:3300/0 conn(0x7f9d301539b0 0x7f9d30159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3510477723 >> v2:172.21.15.117:3300/0 conn(0x7f9d301539b0 msgr2=0x7f9d30159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d36d9c700 1 --2- 172.21.15.7:0/3510477723 >> v2:172.21.15.117:3300/0 conn(0x7f9d301539b0 0x7f9d30159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3510477723 >> v1:172.21.15.117:6789/0 conn(0x7f9d301535e0 legacy=0x7f9d300c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3510477723 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3006b460 con 0x7f9d300c4ce0 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d36d9c700 1 --2- 172.21.15.7:0/3510477723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d300c4ce0 0x7f9d300c5140 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d3008d1e0 tx=0x7f9d2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb34b194741e15d3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d35599700 1 -- 172.21.15.7:0/3510477723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800e070 con 0x7f9d300c4ce0 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d35599700 1 -- 172.21.15.7:0/3510477723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28003a20 con 0x7f9d300c4ce0 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.798+0000 7f9d35599700 1 -- 172.21.15.7:0/3510477723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28003dd0 con 0x7f9d300c4ce0 2023-06-09T11:25:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 -- 172.21.15.7:0/3510477723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d300c4ce0 msgr2=0x7f9d300c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3510477723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d300c4ce0 0x7f9d300c5140 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f9d3008d1e0 tx=0x7f9d2800b330 comp rx=0 tx=0).stop 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 -- 172.21.15.7:0/3510477723 shutdown_connections 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3510477723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d300c4ce0 0x7f9d300c5140 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3510477723 >> v2:172.21.15.117:3300/0 conn(0x7f9d301539b0 0x7f9d30159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 -- 172.21.15.7:0/3510477723 >> 172.21.15.7:0/3510477723 conn(0x7f9d300737f0 msgr2=0x7f9d30072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 -- 172.21.15.7:0/3510477723 shutdown_connections 2023-06-09T11:25:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.799+0000 7f9d387ff700 1 -- 172.21.15.7:0/3510477723 wait complete. 2023-06-09T11:25:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.800+0000 7f9d387ff700 1 Processor -- start 2023-06-09T11:25:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.801+0000 7f9d387ff700 1 -- start start 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.801+0000 7f9d387ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 0x7f9d300cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.801+0000 7f9d387ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 0x7f9d300ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d387ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d387ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300cd890 con 0x7f9d301539b0 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d387ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300b3ec0 con 0x7f9d300c4ce0 2023-06-09T11:25:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d387ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d300b4760 con 0x7f9d301535e0 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53054/0 (socket says 172.21.15.7:53054) 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3605808965 learned_addr learned my addr 172.21.15.7:0/3605808965 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d35d9a700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 0x7f9d300ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d3659b700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 0x7f9d300cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 msgr2=0x7f9d300ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 0x7f9d300ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 msgr2=0x7f9d300cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 0x7f9d300cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d36d9c700 1 -- 172.21.15.7:0/3605808965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28009b30 con 0x7f9d301539b0 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d35d9a700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 0x7f9d300ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.802+0000 7f9d3659b700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 0x7f9d300cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d36d9c700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9d28015040 tx=0x7f9d28004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28004440 con 0x7f9d301539b0 2023-06-09T11:25:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d280045c0 con 0x7f9d301539b0 2023-06-09T11:25:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28004970 con 0x7f9d301539b0 2023-06-09T11:25:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3006a850 con 0x7f9d301539b0 2023-06-09T11:25:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.803+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3006a570 con 0x7f9d301539b0 2023-06-09T11:25:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.804+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3006ab30 con 0x7f9d301539b0 2023-06-09T11:25:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.805+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d28025450 con 0x7f9d301539b0 2023-06-09T11:25:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.806+0000 7f9d26ffd700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 0x7f9d1406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.806+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f9d2808ad50 con 0x7f9d301539b0 2023-06-09T11:25:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.806+0000 7f9d3659b700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 0x7f9d1406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.807+0000 7f9d3659b700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 0x7f9d1406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0017d0 tx=0x7f9d2c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:24.811+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9d28059590 con 0x7f9d301539b0 2023-06-09T11:25:25.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.063+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d3015a420 con 0x7f9d301539b0 2023-06-09T11:25:25.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.064+0000 7f9d26ffd700 1 -- 172.21.15.7:0/3605808965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f9d3015a420 con 0x7f9d301539b0 2023-06-09T11:25:25.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:25.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:22.315529+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:rejoin","state_seq":11,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 msgr2=0x7f9d1406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:25.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 0x7f9d1406d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c0017d0 tx=0x7f9d2c009040 comp rx=0 tx=0).stop 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 msgr2=0x7f9d30151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9d28015040 tx=0x7f9d28004ad0 comp rx=0 tx=0).stop 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 shutdown_connections 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9d301539b0 0x7f9d30151570 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9d300c4ce0 0x7f9d300cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9d1406ace0 0x7f9d1406d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:25.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 --2- 172.21.15.7:0/3605808965 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9d301535e0 0x7f9d300ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 >> 172.21.15.7:0/3605808965 conn(0x7f9d300737f0 msgr2=0x7f9d30072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 shutdown_connections 2023-06-09T11:25:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:25.066+0000 7f9d387ff700 1 -- 172.21.15.7:0/3605808965 wait complete. 2023-06-09T11:25:25.073 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2023-06-09T11:25:25.089 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:25:27.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:27.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.298+0000 7f84319ff700 1 Processor -- start 2023-06-09T11:25:27.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.298+0000 7f84319ff700 1 -- start start 2023-06-09T11:25:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f84319ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f842c0cb9a0 0x7f842c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f84319ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c159b20 con 0x7f842c0cb5d0 2023-06-09T11:25:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f84319ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f842c0cac80 con 0x7f842c15cd40 2023-06-09T11:25:27.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f84319ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842c1597d0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f842a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f842c0cb9a0 0x7f842c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f842affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f842c0cb5d0 0x7f842c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41030/0 (socket says 172.21.15.7:41030) 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.299+0000 7f842affd700 1 -- 172.21.15.7:0/4289846575 learned_addr learned my addr 172.21.15.7:0/4289846575 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f842a7fc700 1 -- 172.21.15.7:0/4289846575 >> v1:172.21.15.117:6789/0 conn(0x7f842c15cd40 legacy=0x7f842c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f842a7fc700 1 -- 172.21.15.7:0/4289846575 >> v1:172.21.15.7:6789/0 conn(0x7f842c0cb5d0 legacy=0x7f842c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f842a7fc700 1 -- 172.21.15.7:0/4289846575 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c071ab0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f8429ffb700 1 -- 172.21.15.7:0/4289846575 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771782097 0 0) 0x7f842c159b20 con 0x7f842c0cb5d0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f842a7fc700 1 --2- 172.21.15.7:0/4289846575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c0b52b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8420009840 tx=0x7f8420009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ae6a466ed08aa3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f8429ffb700 1 -- 172.21.15.7:0/4289846575 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420004230 con 0x7f842c0cb9a0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f8429ffb700 1 -- 172.21.15.7:0/4289846575 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8420004390 con 0x7f842c0cb9a0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.300+0000 7f8429ffb700 1 -- 172.21.15.7:0/4289846575 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84200036a0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 -- 172.21.15.7:0/4289846575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 msgr2=0x7f842c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 --2- 172.21.15.7:0/4289846575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c0b52b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8420009840 tx=0x7f8420009b50 comp rx=0 tx=0).stop 2023-06-09T11:25:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 -- 172.21.15.7:0/4289846575 shutdown_connections 2023-06-09T11:25:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 --2- 172.21.15.7:0/4289846575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c0b52b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 -- 172.21.15.7:0/4289846575 >> 172.21.15.7:0/4289846575 conn(0x7f842c070af0 msgr2=0x7f842c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:27.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 -- 172.21.15.7:0/4289846575 shutdown_connections 2023-06-09T11:25:27.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.301+0000 7f84319ff700 1 -- 172.21.15.7:0/4289846575 wait complete. 2023-06-09T11:25:27.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.303+0000 7f84319ff700 1 Processor -- start 2023-06-09T11:25:27.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.303+0000 7f84319ff700 1 -- start start 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 0x7f842c1f0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 0x7f842c1e57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c1e34f0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0714e0 con 0x7f842c0cb5d0 2023-06-09T11:25:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f84319ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842c1f2e00 con 0x7f842c15cd40 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53072/0 (socket says 172.21.15.7:53072) 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 -- 172.21.15.7:0/2584107459 learned_addr learned my addr 172.21.15.7:0/2584107459 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 -- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 msgr2=0x7f842c1e57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842b7fe700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 0x7f842c1e57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842affd700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 0x7f842c1f0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 0x7f842c1e57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 -- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 msgr2=0x7f842c1f0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 0x7f842c1f0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842a7fc700 1 -- 172.21.15.7:0/2584107459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200094a0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842affd700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 0x7f842c1f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.304+0000 7f842b7fe700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 0x7f842c1e57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f842a7fc700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8420009b90 tx=0x7f8420004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8420011630 con 0x7f842c0cb9a0 2023-06-09T11:25:27.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842001e070 con 0x7f842c0cb9a0 2023-06-09T11:25:27.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c15d4f0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84200119b0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.305+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c15d210 con 0x7f842c0cb9a0 2023-06-09T11:25:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.306+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c15d7d0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.307+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8420011b10 con 0x7f842c0cb9a0 2023-06-09T11:25:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.308+0000 7f8413fff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 0x7f841806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.308+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f842008ace0 con 0x7f842c0cb9a0 2023-06-09T11:25:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.308+0000 7f842affd700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 0x7f841806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.309+0000 7f842affd700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 0x7f841806d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f841c000f60 tx=0x7f841c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.313+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8420055970 con 0x7f842c0cb9a0 2023-06-09T11:25:27.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.569+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f842c1f0090 con 0x7f842c0cb9a0 2023-06-09T11:25:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.569+0000 7f8413fff700 1 -- 172.21.15.7:0/2584107459 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f842c1f0090 con 0x7f842c0cb9a0 2023-06-09T11:25:27.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:27.575 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 msgr2=0x7f841806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 0x7f841806d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f841c000f60 tx=0x7f841c009040 comp rx=0 tx=0).stop 2023-06-09T11:25:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 msgr2=0x7f842c1f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8420009b90 tx=0x7f8420004750 comp rx=0 tx=0).stop 2023-06-09T11:25:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 shutdown_connections 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f841806ace0 0x7f841806d190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f842c15cd40 0x7f842c1e57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f842c0cb5d0 0x7f842c1f0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 --2- 172.21.15.7:0/2584107459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f842c0cb9a0 0x7f842c1f0f70 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 >> 172.21.15.7:0/2584107459 conn(0x7f842c070af0 msgr2=0x7f842c156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 shutdown_connections 2023-06-09T11:25:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:27.572+0000 7f84319ff700 1 -- 172.21.15.7:0/2584107459 wait complete. 2023-06-09T11:25:27.579 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 71 2023-06-09T11:25:27.595 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=5071 2023-06-09T11:25:27.595 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 8 secs before reviving mds.e 2023-06-09T11:25:28.323 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:25:28.318+0000 7f7c8b3f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:25:35.596 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-06-09T11:25:35.597 INFO:tasks.ceph.mds.e:Restarting daemon 2023-06-09T11:25:35.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-06-09T11:25:35.602 INFO:tasks.ceph.mds.e:Started 2023-06-09T11:25:35.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:35.757 INFO:tasks.ceph.mds.e.smithi007.stdout:starting mds.e at 2023-06-09T11:25:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.812+0000 7f89c35ff700 1 Processor -- start 2023-06-09T11:25:35.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.812+0000 7f89c35ff700 1 -- start start 2023-06-09T11:25:35.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.813+0000 7f89c35ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89bc0b5360 0x7f89bc15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c35ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89bc15dcf0 con 0x7f89bc0ca580 2023-06-09T11:25:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c35ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89bc15d450 con 0x7f89bc0ca1b0 2023-06-09T11:25:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c35ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89bc15d730 con 0x7f89bc0b5360 2023-06-09T11:25:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c1b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f89bc0ca580 0x7f89bc06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42006/0 (socket says 172.21.15.7:42006) 2023-06-09T11:25:35.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c1b9c700 1 -- 172.21.15.7:0/4149483054 learned_addr learned my addr 172.21.15.7:0/4149483054 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89c0b9a700 1 --2- 172.21.15.7:0/4149483054 >> v2:172.21.15.117:3300/0 conn(0x7f89bc0b5360 0x7f89bc15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674327451 0 0) 0x7f89bc15dcf0 con 0x7f89bc0ca580 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.814+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89a80035b0 con 0x7f89bc0ca580 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89c0b9a700 1 -- 172.21.15.7:0/4149483054 >> v1:172.21.15.117:6789/0 conn(0x7f89bc0ca1b0 legacy=0x7f89bc1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89c0b9a700 1 -- 172.21.15.7:0/4149483054 >> v1:172.21.15.7:6789/0 conn(0x7f89bc0ca580 legacy=0x7f89bc06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1535694834 0 0) 0x7f89a80035b0 con 0x7f89bc0ca580 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89c0b9a700 1 -- 172.21.15.7:0/4149483054 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc15da10 con 0x7f89bc0b5360 2023-06-09T11:25:35.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89c0b9a700 1 --2- 172.21.15.7:0/4149483054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0b5360 0x7f89bc15cdc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009840 tx=0x7f89ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de340f96df0a3b10 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40019f0 con 0x7f89bc0b5360 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ac004030 con 0x7f89bc0b5360 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.815+0000 7f89b3fff700 1 -- 172.21.15.7:0/4149483054 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ac0043c0 con 0x7f89bc0b5360 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 -- 172.21.15.7:0/4149483054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0b5360 msgr2=0x7f89bc15cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 --2- 172.21.15.7:0/4149483054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0b5360 0x7f89bc15cdc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f89ac009840 tx=0x7f89ac009b50 comp rx=0 tx=0).stop 2023-06-09T11:25:35.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 -- 172.21.15.7:0/4149483054 shutdown_connections 2023-06-09T11:25:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 --2- 172.21.15.7:0/4149483054 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0b5360 0x7f89bc15cdc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 -- 172.21.15.7:0/4149483054 >> 172.21.15.7:0/4149483054 conn(0x7f89bc072090 msgr2=0x7f89bc072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 -- 172.21.15.7:0/4149483054 shutdown_connections 2023-06-09T11:25:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.816+0000 7f89c35ff700 1 -- 172.21.15.7:0/4149483054 wait complete. 2023-06-09T11:25:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.817+0000 7f89c35ff700 1 Processor -- start 2023-06-09T11:25:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.818+0000 7f89c35ff700 1 -- start start 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89bc0b5360 0x7f89bc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 0x7f89bc14c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0c9a30 con 0x7f89bc0ca580 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc0c97e0 con 0x7f89bc0ca1b0 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c35ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89bc15bd30 con 0x7f89bc0b5360 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46444/0 (socket says 172.21.15.7:46444) 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 -- 172.21.15.7:0/1441526764 learned_addr learned my addr 172.21.15.7:0/1441526764 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c139b700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89bc0b5360 0x7f89bc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c0b9a700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 0x7f89bc14c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 -- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89bc0b5360 msgr2=0x7f89bc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89bc0b5360 0x7f89bc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 -- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 msgr2=0x7f89bc14c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:35.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 0x7f89bc14c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c1b9c700 1 -- 172.21.15.7:0/1441526764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ac0094a0 con 0x7f89bc0ca580 2023-06-09T11:25:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.819+0000 7f89c0b9a700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 0x7f89bc14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:25:35.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89c1b9c700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f89b400cc00 tx=0x7f89b400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b400aa20 con 0x7f89bc0ca580 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89bc06a850 con 0x7f89bc0ca580 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89bc06a570 con 0x7f89bc0ca580 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89b400ab80 con 0x7f89bc0ca580 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.820+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89b40070d0 con 0x7f89bc0ca580 2023-06-09T11:25:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.821+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c0052e0 con 0x7f89bc0ca580 2023-06-09T11:25:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.822+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89b4007230 con 0x7f89bc0ca580 2023-06-09T11:25:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.823+0000 7f89b1ffb700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 0x7f89a006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.823+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f89b4089540 con 0x7f89bc0ca580 2023-06-09T11:25:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.823+0000 7f89c139b700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 0x7f89a006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:35.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.824+0000 7f89c139b700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 0x7f89a006d1c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000f60 tx=0x7f89b8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:35.828+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f89b4054290 con 0x7f89bc0ca580 2023-06-09T11:25:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.077+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f899c005b10 con 0x7f89bc0ca580 2023-06-09T11:25:36.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.077+0000 7f89b1ffb700 1 -- 172.21.15.7:0/1441526764 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f89b40578b0 con 0x7f89bc0ca580 2023-06-09T11:25:36.083 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:36.084 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":71,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 msgr2=0x7f89a006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 0x7f89a006d1c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f89b8000f60 tx=0x7f89b8009040 comp rx=0 tx=0).stop 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 msgr2=0x7f89bc0b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f89b400cc00 tx=0x7f89b400cf10 comp rx=0 tx=0).stop 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 shutdown_connections 2023-06-09T11:25:36.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89bc0ca580 0x7f89bc0b40d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f89a006ad10 0x7f89a006d1c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89bc0b5360 0x7f89bc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 --2- 172.21.15.7:0/1441526764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89bc0ca1b0 0x7f89bc14c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 >> 172.21.15.7:0/1441526764 conn(0x7f89bc072090 msgr2=0x7f89bc06ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 shutdown_connections 2023-06-09T11:25:36.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:36.080+0000 7f89c35ff700 1 -- 172.21.15.7:0/1441526764 wait complete. 2023-06-09T11:25:36.088 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 71 2023-06-09T11:25:36.104 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-06-09T11:25:38.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:25:38.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.312+0000 7fe0f17ff700 1 Processor -- start 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.312+0000 7fe0f17ff700 1 -- start start 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.313+0000 7fe0f17ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0ec0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.313+0000 7fe0f17ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ec169bc0 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.313+0000 7fe0f17ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ec169ea0 con 0x7fe0ec166b00 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.313+0000 7fe0f17ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec16a180 con 0x7fe0ec0c5d20 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.314+0000 7fe0ea7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe0ec0c60f0 0x7fe0ec0cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42018/0 (socket says 172.21.15.7:42018) 2023-06-09T11:25:38.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.314+0000 7fe0ea7fc700 1 -- 172.21.15.7:0/1576401552 learned_addr learned my addr 172.21.15.7:0/1576401552 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.314+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0ec0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.314+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1840318811 0 0) 0x7fe0ec169bc0 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.320+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0d4003590 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.320+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647648479 0 0) 0x7fe0ec169ea0 con 0x7fe0ec166b00 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.320+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0ec169bc0 con 0x7fe0ec166b00 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3454989549 0 0) 0x7fe0d4003590 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 msgr2=0x7fe0ec0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0ec0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 >> v1:172.21.15.117:6789/0 conn(0x7fe0ec166b00 legacy=0x7fe0ec166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec16a460 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fe0e0001670 con 0x7fe0ec0c60f0 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 >> v1:172.21.15.7:6789/0 conn(0x7fe0ec0c60f0 legacy=0x7fe0ec0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0d4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0d4004710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d400d4a0 0x7fe0d400f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec16a180 con 0x7fe0d40042a0 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4010020 con 0x7fe0d400d4a0 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d40101e0 con 0x7fe0ec0c5d20 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0d4004710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0d4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0eb7fe700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d400d4a0 0x7fe0d400f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.321+0000 7fe0ea7fc700 1 -- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 msgr2=0x7fe0d4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0d4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0ea7fc700 1 -- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d400d4a0 msgr2=0x7fe0d400f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0d400d4a0 0x7fe0d400f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0ea7fc700 1 -- 172.21.15.7:0/1576401552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d4003590 con 0x7fe0d40042a0 2023-06-09T11:25:38.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c5d20 0x7fe0d4003d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:25:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0d4004710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c160 tx=0x7fe0d4004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d696b0bd24033a3c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e000cca0 con 0x7fe0d40042a0 2023-06-09T11:25:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e000ce20 con 0x7fe0d40042a0 2023-06-09T11:25:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.322+0000 7fe0e9ffb700 1 -- 172.21.15.7:0/1576401552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e0005ba0 con 0x7fe0d40042a0 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 -- 172.21.15.7:0/1576401552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 msgr2=0x7fe0d4004710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/1576401552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0d4004710 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe0e000c160 tx=0x7fe0d4004de0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0eb7fe700 1 -- 172.21.15.7:0/1576401552 reap_dead start 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 -- 172.21.15.7:0/1576401552 shutdown_connections 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 -- 172.21.15.7:0/1576401552 >> 172.21.15.7:0/1576401552 conn(0x7fe0ec06a4d0 msgr2=0x7fe0ec06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 -- 172.21.15.7:0/1576401552 shutdown_connections 2023-06-09T11:25:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.323+0000 7fe0f17ff700 1 -- 172.21.15.7:0/1576401552 wait complete. 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.324+0000 7fe0f17ff700 1 Processor -- start 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.325+0000 7fe0f17ff700 1 -- start start 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.325+0000 7fe0f17ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0f17ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec1ffad0 0x7fe0ec159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0f17ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c8fa0 0x7fe0ec164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0f17ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec160ff0 con 0x7fe0d40042a0 2023-06-09T11:25:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0f17ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1feda0 con 0x7fe0ec1ffad0 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0f17ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1ff640 con 0x7fe0ec0c8fa0 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0ea7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec1ffad0 0x7fe0ec159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46470/0 (socket says 172.21.15.7:46470) 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 -- 172.21.15.7:0/3832917374 learned_addr learned my addr 172.21.15.7:0/3832917374 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eb7fe700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c8fa0 0x7fe0ec164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 -- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c8fa0 msgr2=0x7fe0ec164ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c8fa0 0x7fe0ec164ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 -- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec1ffad0 msgr2=0x7fe0ec159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec1ffad0 0x7fe0ec159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.326+0000 7fe0eaffd700 1 -- 172.21.15.7:0/3832917374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0cb9b0 con 0x7fe0d40042a0 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0eaffd700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc011bc0 tx=0x7fe0dc011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:38.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc00e040 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc00f040 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc004240 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec0cb820 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.327+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec0c74e0 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.328+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ec1ff920 con 0x7fe0d40042a0 2023-06-09T11:25:38.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.329+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0dc0036a0 con 0x7fe0d40042a0 2023-06-09T11:25:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.330+0000 7fe0d37fe700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 0x7fe0d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:25:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.330+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fe0dc089a80 con 0x7fe0d40042a0 2023-06-09T11:25:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.330+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 0x7fe0d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:25:38.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.331+0000 7fe0ea7fc700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 0x7fe0d806d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0015040 tx=0x7fe0e0007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:25:38.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.334+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe0dc054710 con 0x7fe0d40042a0 2023-06-09T11:25:38.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.587+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0ec074cf0 con 0x7fe0d40042a0 2023-06-09T11:25:38.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.587+0000 7fe0d37fe700 1 -- 172.21.15.7:0/3832917374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fe0dc022090 con 0x7fe0d40042a0 2023-06-09T11:25:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:25:38.594 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:25:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 msgr2=0x7fe0d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 0x7fe0d806d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0015040 tx=0x7fe0e0007000 comp rx=0 tx=0).stop 2023-06-09T11:25:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 msgr2=0x7fe0ec162ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:25:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc011bc0 tx=0x7fe0dc011ed0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 shutdown_connections 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe0d806ace0 0x7fe0d806d190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe0ec0c8fa0 0x7fe0ec164ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0d40042a0 0x7fe0ec162ae0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 --2- 172.21.15.7:0/3832917374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe0ec1ffad0 0x7fe0ec159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 >> 172.21.15.7:0/3832917374 conn(0x7fe0ec06a4d0 msgr2=0x7fe0ec0c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:25:38.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 shutdown_connections 2023-06-09T11:25:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:25:38.590+0000 7fe0f17ff700 1 -- 172.21.15.7:0/3832917374 wait complete. 2023-06-09T11:25:38.598 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:25:38.613 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-06-09T11:25:38.614 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 112 secs before thrashing 2023-06-09T11:25:44.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:25:44.670 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:14.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:14.788 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:44.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:26:44.893 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:15.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:15.009 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:30.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:30.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.833+0000 7fc395f9d700 1 Processor -- start 2023-06-09T11:27:30.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.833+0000 7fc395f9d700 1 -- start start 2023-06-09T11:27:30.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.834+0000 7fc395f9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc390156f20 0x7fc3900cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc395f9d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3900d01a0 con 0x7fc390156b50 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc395f9d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3900b54a0 con 0x7fc390071d50 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc395f9d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3900cb2b0 con 0x7fc390156f20 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc390156f20 0x7fc3900cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc394f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc390156b50 0x7fc3900caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47278/0 (socket says 172.21.15.7:47278) 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc394f9b700 1 -- 172.21.15.7:0/2023019688 learned_addr learned my addr 172.21.15.7:0/2023019688 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:30.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc390156f20 0x7fc3900cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57422/0 (socket says 172.21.15.7:57422) 2023-06-09T11:27:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718023907 0 0) 0x7fc3900d01a0 con 0x7fc390156b50 2023-06-09T11:27:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3800035b0 con 0x7fc390156b50 2023-06-09T11:27:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547909956 0 0) 0x7fc3900b54a0 con 0x7fc390071d50 2023-06-09T11:27:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.835+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3900d01a0 con 0x7fc390071d50 2023-06-09T11:27:30.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38ffff700 1 -- 172.21.15.7:0/2023019688 >> v1:172.21.15.117:6789/0 conn(0x7fc390071d50 legacy=0x7fc3900b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38ffff700 1 -- 172.21.15.7:0/2023019688 >> v1:172.21.15.7:6789/0 conn(0x7fc390156b50 legacy=0x7fc3900caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38ffff700 1 -- 172.21.15.7:0/2023019688 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3900cba50 con 0x7fc390156f20 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38ffff700 1 --2- 172.21.15.7:0/2023019688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390156f20 0x7fc3900cb510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009840 tx=0x7fc37c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ce003f37adef78 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c004230 con 0x7fc390156f20 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c004390 con 0x7fc390156f20 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.836+0000 7fc38f7fe700 1 -- 172.21.15.7:0/2023019688 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c0036a0 con 0x7fc390156f20 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 -- 172.21.15.7:0/2023019688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390156f20 msgr2=0x7fc3900cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:30.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 --2- 172.21.15.7:0/2023019688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390156f20 0x7fc3900cb510 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009840 tx=0x7fc37c009b50 comp rx=0 tx=0).stop 2023-06-09T11:27:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 -- 172.21.15.7:0/2023019688 shutdown_connections 2023-06-09T11:27:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 --2- 172.21.15.7:0/2023019688 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390156f20 0x7fc3900cb510 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 -- 172.21.15.7:0/2023019688 >> 172.21.15.7:0/2023019688 conn(0x7fc3900737f0 msgr2=0x7fc390073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 -- 172.21.15.7:0/2023019688 shutdown_connections 2023-06-09T11:27:30.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.837+0000 7fc395f9d700 1 -- 172.21.15.7:0/2023019688 wait complete. 2023-06-09T11:27:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.839+0000 7fc395f9d700 1 Processor -- start 2023-06-09T11:27:30.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.839+0000 7fc395f9d700 1 -- start start 2023-06-09T11:27:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc390156b50 0x7fc3901f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390156f20 0x7fc3900c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:30.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900ab5a0 con 0x7fc390071d50 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3901ee750 con 0x7fc390156f20 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.840+0000 7fc395f9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3900aafe0 con 0x7fc390156b50 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57438/0 (socket says 172.21.15.7:57438) 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 -- 172.21.15.7:0/1817795634 learned_addr learned my addr 172.21.15.7:0/1817795634 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc38ffff700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc390156b50 0x7fc3901f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc39579c700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390156f20 0x7fc3900c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:30.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 -- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc390156b50 msgr2=0x7fc3901f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:30.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc390156b50 0x7fc3901f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:30.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 -- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390156f20 msgr2=0x7fc3900c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:30.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390156f20 0x7fc3900c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:30.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.841+0000 7fc394f9b700 1 -- 172.21.15.7:0/1817795634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0094a0 con 0x7fc390071d50 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc394f9b700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc384001260 tx=0x7fc38400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3840074e0 con 0x7fc390071d50 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc384007640 con 0x7fc390071d50 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3900cf870 con 0x7fc390071d50 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc384005a50 con 0x7fc390071d50 2023-06-09T11:27:30.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.842+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3900cf590 con 0x7fc390071d50 2023-06-09T11:27:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.843+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3900cfb50 con 0x7fc390071d50 2023-06-09T11:27:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.844+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc384005bb0 con 0x7fc390071d50 2023-06-09T11:27:30.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.846+0000 7fc38d7fa700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 0x7fc37406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.846+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fc384088c80 con 0x7fc390071d50 2023-06-09T11:27:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.846+0000 7fc38ffff700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 0x7fc37406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:30.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.847+0000 7fc38ffff700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 0x7fc37406d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0090b0 tx=0x7fc37c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:30.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:30.851+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc384057440 con 0x7fc390071d50 2023-06-09T11:27:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.140+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc39004f760 con 0x7fc390071d50 2023-06-09T11:27:31.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.141+0000 7fc38d7fa700 1 -- 172.21.15.7:0/1817795634 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fc384017070 con 0x7fc390071d50 2023-06-09T11:27:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:31.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 msgr2=0x7fc37406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 0x7fc37406d190 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0090b0 tx=0x7fc37c010000 comp rx=0 tx=0).stop 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 msgr2=0x7fc3900c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc384001260 tx=0x7fc38400c9e0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 shutdown_connections 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc390156f20 0x7fc3900c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc390071d50 0x7fc3900c7700 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc37406ace0 0x7fc37406d190 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 --2- 172.21.15.7:0/1817795634 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc390156b50 0x7fc3901f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.143+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 >> 172.21.15.7:0/1817795634 conn(0x7fc3900737f0 msgr2=0x7fc3900caae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.144+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 shutdown_connections 2023-06-09T11:27:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.144+0000 7fc395f9d700 1 -- 172.21.15.7:0/1817795634 wait complete. 2023-06-09T11:27:31.150 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:31.169 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=2) 2023-06-09T11:27:31.169 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:27:31.170 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:27:31.189 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:27:31.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.419+0000 7f420edff700 1 Processor -- start 2023-06-09T11:27:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.419+0000 7f420edff700 1 -- start start 2023-06-09T11:27:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.420+0000 7f420edff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42080cb5d0 0x7f42080b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.420+0000 7f420edff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42080cb9a0 0x7f4208159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.420+0000 7f420edff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4208159b20 con 0x7f420815cd40 2023-06-09T11:27:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.420+0000 7f420edff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42080cac80 con 0x7f42080cb9a0 2023-06-09T11:27:31.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.420+0000 7f420edff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42081597d0 con 0x7f42080cb5d0 2023-06-09T11:27:31.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42080cb9a0 0x7f4208159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f42080cb9a0 0x7f4208159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57446/0 (socket says 172.21.15.7:57446) 2023-06-09T11:27:31.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 -- 172.21.15.7:0/1305793020 learned_addr learned my addr 172.21.15.7:0/1305793020 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:31.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f420cb9b700 1 --2- 172.21.15.7:0/1305793020 >> v2:172.21.15.117:3300/0 conn(0x7f42080cb5d0 0x7f42080b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 -- 172.21.15.7:0/1305793020 >> v2:172.21.15.117:3300/0 conn(0x7f42080cb5d0 msgr2=0x7f42080b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 --2- 172.21.15.7:0/1305793020 >> v2:172.21.15.117:3300/0 conn(0x7f42080cb5d0 0x7f42080b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 -- 172.21.15.7:0/1305793020 >> v1:172.21.15.117:6789/0 conn(0x7f420815cd40 legacy=0x7f42080cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.421+0000 7f4207fff700 1 -- 172.21.15.7:0/1305793020 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208071ab0 con 0x7f42080cb9a0 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.422+0000 7f4207fff700 1 --2- 172.21.15.7:0/1305793020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42080cb9a0 0x7f4208159fb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f420808d420 tx=0x7f41f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4cfbb0710d7adf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:31.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.422+0000 7f42077fe700 1 -- 172.21.15.7:0/1305793020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011830 con 0x7f42080cb9a0 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.422+0000 7f42077fe700 1 -- 172.21.15.7:0/1305793020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f4011990 con 0x7f42080cb9a0 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.422+0000 7f42077fe700 1 -- 172.21.15.7:0/1305793020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4011d20 con 0x7f42080cb9a0 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 -- 172.21.15.7:0/1305793020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42080cb9a0 msgr2=0x7f4208159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 --2- 172.21.15.7:0/1305793020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42080cb9a0 0x7f4208159fb0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f420808d420 tx=0x7f41f4009bd0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 -- 172.21.15.7:0/1305793020 shutdown_connections 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 --2- 172.21.15.7:0/1305793020 >> v2:172.21.15.117:3300/0 conn(0x7f42080cb5d0 0x7f42080b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 --2- 172.21.15.7:0/1305793020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42080cb9a0 0x7f4208159fb0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 -- 172.21.15.7:0/1305793020 >> 172.21.15.7:0/1305793020 conn(0x7f4208070af0 msgr2=0x7f4208070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 -- 172.21.15.7:0/1305793020 shutdown_connections 2023-06-09T11:27:31.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.423+0000 7f420edff700 1 -- 172.21.15.7:0/1305793020 wait complete. 2023-06-09T11:27:31.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.425+0000 7f420edff700 1 Processor -- start 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.425+0000 7f420edff700 1 -- start start 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080cb5d0 0x7f4208153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 0x7f42081f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081546b0 con 0x7f420815cd40 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081f2c40 con 0x7f42080cb9a0 2023-06-09T11:27:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080cfb10 con 0x7f42080cb5d0 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420d39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420d39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57462/0 (socket says 172.21.15.7:57462) 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 -- 172.21.15.7:0/1223681404 learned_addr learned my addr 172.21.15.7:0/1223681404 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f4207fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 0x7f42081f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.426+0000 7f420cb9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080cb5d0 0x7f4208153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 -- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080cb5d0 msgr2=0x7f4208153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080cb5d0 0x7f4208153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 -- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 msgr2=0x7f42081f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 0x7f42081f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 -- 172.21.15.7:0/1223681404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009550 con 0x7f420815cd40 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f4207fff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 0x7f42081f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f420d39c700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f420000cbc0 tx=0x7f420000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200003c00 con 0x7f420815cd40 2023-06-09T11:27:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.427+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200003d60 con 0x7f420815cd40 2023-06-09T11:27:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.428+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420000aa20 con 0x7f420815cd40 2023-06-09T11:27:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.428+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420815d4f0 con 0x7f420815cd40 2023-06-09T11:27:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.428+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420815d210 con 0x7f420815cd40 2023-06-09T11:27:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.429+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420815d7d0 con 0x7f420815cd40 2023-06-09T11:27:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.430+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f420000ab80 con 0x7f420815cd40 2023-06-09T11:27:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.432+0000 7f42057fa700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 0x7f41ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.432+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f4200089590 con 0x7f420815cd40 2023-06-09T11:27:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.432+0000 7f420cb9b700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 0x7f41ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.432+0000 7f420cb9b700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 0x7f41ec06d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009be0 tx=0x7f41fc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.437+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4200054490 con 0x7f420815cd40 2023-06-09T11:27:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.786+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4208074cf0 con 0x7f420815cd40 2023-06-09T11:27:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.787+0000 7f42057fa700 1 -- 172.21.15.7:0/1223681404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f4200057ab0 con 0x7f420815cd40 2023-06-09T11:27:31.792 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:31.792 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 msgr2=0x7f41ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 0x7f41ec06d190 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009be0 tx=0x7f41fc009210 comp rx=0 tx=0).stop 2023-06-09T11:27:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 msgr2=0x7f4208152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f420000cbc0 tx=0x7f420000ced0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 shutdown_connections 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f420815cd40 0x7f4208152560 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f41ec06ace0 0x7f41ec06d190 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42080cb5d0 0x7f4208153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 --2- 172.21.15.7:0/1223681404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42080cb9a0 0x7f42081f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.789+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 >> 172.21.15.7:0/1223681404 conn(0x7f4208070af0 msgr2=0x7f42080b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.790+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 shutdown_connections 2023-06-09T11:27:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:31.790+0000 7f420edff700 1 -- 172.21.15.7:0/1223681404 wait complete. 2023-06-09T11:27:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:31.814 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:27:33.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:34.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.045+0000 7f13a9fff700 1 Processor -- start 2023-06-09T11:27:34.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.045+0000 7f13a9fff700 1 -- start start 2023-06-09T11:27:34.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a9fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13a41593d0 0x7f13a41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a9fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f13a40c08d0 0x7f13a4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a9fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a40cc140 con 0x7f13a4071910 2023-06-09T11:27:34.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a9fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a40714f0 con 0x7f13a41593d0 2023-06-09T11:27:34.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a9fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a40c1830 con 0x7f13a40c08d0 2023-06-09T11:27:34.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13a41593d0 0x7f13a41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13a41593d0 0x7f13a41597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43894/0 (socket says 172.21.15.7:43894) 2023-06-09T11:27:34.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.047+0000 7f13a37fe700 1 -- 172.21.15.7:0/78740458 learned_addr learned my addr 172.21.15.7:0/78740458 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:34.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a2ffd700 1 --2- 172.21.15.7:0/78740458 >> v2:172.21.15.117:3300/0 conn(0x7f13a40c08d0 0x7f13a4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a27fc700 1 -- 172.21.15.7:0/78740458 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401172948 0 0) 0x7f13a40cc140 con 0x7f13a4071910 2023-06-09T11:27:34.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a27fc700 1 -- 172.21.15.7:0/78740458 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f138c0035b0 con 0x7f13a4071910 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a37fe700 1 -- 172.21.15.7:0/78740458 >> v2:172.21.15.117:3300/0 conn(0x7f13a40c08d0 msgr2=0x7f13a4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a37fe700 1 --2- 172.21.15.7:0/78740458 >> v2:172.21.15.117:3300/0 conn(0x7f13a40c08d0 0x7f13a4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a37fe700 1 -- 172.21.15.7:0/78740458 >> v1:172.21.15.7:6789/0 conn(0x7f13a4071910 legacy=0x7f13a4071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a37fe700 1 -- 172.21.15.7:0/78740458 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40c0560 con 0x7f13a41593d0 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.048+0000 7f13a37fe700 1 --2- 172.21.15.7:0/78740458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a41593d0 0x7f13a41597a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f13a406b700 tx=0x7f139400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18114d664f9c2d78 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.049+0000 7f13a27fc700 1 -- 172.21.15.7:0/78740458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139400db60 con 0x7f13a41593d0 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.049+0000 7f13a27fc700 1 -- 172.21.15.7:0/78740458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139400dcc0 con 0x7f13a41593d0 2023-06-09T11:27:34.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.049+0000 7f13a27fc700 1 -- 172.21.15.7:0/78740458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1394017610 con 0x7f13a41593d0 2023-06-09T11:27:34.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.049+0000 7f13a9fff700 1 -- 172.21.15.7:0/78740458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a41593d0 msgr2=0x7f13a41597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.049+0000 7f13a9fff700 1 --2- 172.21.15.7:0/78740458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a41593d0 0x7f13a41597a0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f13a406b700 tx=0x7f139400be70 comp rx=0 tx=0).stop 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 -- 172.21.15.7:0/78740458 shutdown_connections 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 --2- 172.21.15.7:0/78740458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a41593d0 0x7f13a41597a0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 --2- 172.21.15.7:0/78740458 >> v2:172.21.15.117:3300/0 conn(0x7f13a40c08d0 0x7f13a4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 -- 172.21.15.7:0/78740458 >> 172.21.15.7:0/78740458 conn(0x7f13a406a4d0 msgr2=0x7f13a406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 -- 172.21.15.7:0/78740458 shutdown_connections 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.050+0000 7f13a9fff700 1 -- 172.21.15.7:0/78740458 wait complete. 2023-06-09T11:27:34.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.051+0000 7f13a9fff700 1 Processor -- start 2023-06-09T11:27:34.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.052+0000 7f13a9fff700 1 -- start start 2023-06-09T11:27:34.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.052+0000 7f13a9fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4071910 0x7f13a40ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a9fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a9fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13a41593d0 0x7f13a41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a9fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40b0400 con 0x7f13a40c08d0 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a9fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40af860 con 0x7f13a4071910 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a9fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4158e20 con 0x7f13a41593d0 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43902/0 (socket says 172.21.15.7:43902) 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 -- 172.21.15.7:0/3442937169 learned_addr learned my addr 172.21.15.7:0/3442937169 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a37fe700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4071910 0x7f13a40ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a3fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13a41593d0 0x7f13a41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 -- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13a41593d0 msgr2=0x7f13a41573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13a41593d0 0x7f13a41573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 -- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4071910 msgr2=0x7f13a40ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4071910 0x7f13a40ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.053+0000 7f13a2ffd700 1 -- 172.21.15.7:0/3442937169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139400b7f0 con 0x7f13a40c08d0 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f13a2ffd700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9b0 tx=0x7f139800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398009710 con 0x7f13a40c08d0 2023-06-09T11:27:34.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139800c850 con 0x7f13a40c08d0 2023-06-09T11:27:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139800cbc0 con 0x7f13a40c08d0 2023-06-09T11:27:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a40c1d60 con 0x7f13a40c08d0 2023-06-09T11:27:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.054+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a40c1a80 con 0x7f13a40c08d0 2023-06-09T11:27:34.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.055+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a40c2040 con 0x7f13a40c08d0 2023-06-09T11:27:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.057+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f139800cd20 con 0x7f13a40c08d0 2023-06-09T11:27:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.058+0000 7f138bfff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 0x7f139006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.058+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f1398089870 con 0x7f13a40c08d0 2023-06-09T11:27:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.058+0000 7f13a37fe700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 0x7f139006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:34.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.059+0000 7f13a37fe700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 0x7f139006d1c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1394019040 tx=0x7f1394014000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:34.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.063+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1398054500 con 0x7f13a40c08d0 2023-06-09T11:27:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.353+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13a4074cf0 con 0x7f13a40c08d0 2023-06-09T11:27:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.353+0000 7f138bfff700 1 -- 172.21.15.7:0/3442937169 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f1398057b20 con 0x7f13a40c08d0 2023-06-09T11:27:34.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:34.359 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.356+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 msgr2=0x7f139006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.356+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 0x7f139006d1c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1394019040 tx=0x7f1394014000 comp rx=0 tx=0).stop 2023-06-09T11:27:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.356+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 msgr2=0x7f13a4156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.356+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f139800e9b0 tx=0x7f139800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 shutdown_connections 2023-06-09T11:27:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f139006ad10 0x7f139006d1c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13a41593d0 0x7f13a41573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13a4071910 0x7f13a40ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 --2- 172.21.15.7:0/3442937169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13a40c08d0 0x7f13a4156e70 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 >> 172.21.15.7:0/3442937169 conn(0x7f13a406a4d0 msgr2=0x7f13a41597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 shutdown_connections 2023-06-09T11:27:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:34.357+0000 7f13a9fff700 1 -- 172.21.15.7:0/3442937169 wait complete. 2023-06-09T11:27:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:34.381 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:27:36.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.605+0000 7f8bb95ff700 1 Processor -- start 2023-06-09T11:27:36.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.605+0000 7f8bb95ff700 1 -- start start 2023-06-09T11:27:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb95ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8bb4159ee0 0x7f8bb415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb95ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb406b460 con 0x7f8bb41539b0 2023-06-09T11:27:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb95ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb406a570 con 0x7f8bb41535e0 2023-06-09T11:27:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb95ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40ca670 con 0x7f8bb4159ee0 2023-06-09T11:27:36.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb27fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bb41539b0 0x7f8bb40c4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45026/0 (socket says 172.21.15.7:45026) 2023-06-09T11:27:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb27fc700 1 -- 172.21.15.7:0/967444876 learned_addr learned my addr 172.21.15.7:0/967444876 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.607+0000 7f8bb37fe700 1 --2- 172.21.15.7:0/967444876 >> v2:172.21.15.117:3300/0 conn(0x7f8bb4159ee0 0x7f8bb415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488526716 0 0) 0x7f8bb406b460 con 0x7f8bb41539b0 2023-06-09T11:27:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b9c0035b0 con 0x7f8bb41539b0 2023-06-09T11:27:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215636695 0 0) 0x7f8bb406a570 con 0x7f8bb41535e0 2023-06-09T11:27:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bb406b460 con 0x7f8bb41535e0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb37fe700 1 -- 172.21.15.7:0/967444876 >> v1:172.21.15.117:6789/0 conn(0x7f8bb41535e0 legacy=0x7f8bb40c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb37fe700 1 -- 172.21.15.7:0/967444876 >> v1:172.21.15.7:6789/0 conn(0x7f8bb41539b0 legacy=0x7f8bb40c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb37fe700 1 -- 172.21.15.7:0/967444876 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb406a850 con 0x7f8bb4159ee0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb37fe700 1 --2- 172.21.15.7:0/967444876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb4159ee0 0x7f8bb415abb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2f41b59e4b8d63 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.608+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00e070 con 0x7f8bb4159ee0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac00bb20 con 0x7f8bb4159ee0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb1ffb700 1 -- 172.21.15.7:0/967444876 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac012430 con 0x7f8bb4159ee0 2023-06-09T11:27:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 -- 172.21.15.7:0/967444876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb4159ee0 msgr2=0x7f8bb415abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/967444876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb4159ee0 0x7f8bb415abb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8bac009d90 tx=0x7f8bac00b330 comp rx=0 tx=0).stop 2023-06-09T11:27:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 -- 172.21.15.7:0/967444876 shutdown_connections 2023-06-09T11:27:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/967444876 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb4159ee0 0x7f8bb415abb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 -- 172.21.15.7:0/967444876 >> 172.21.15.7:0/967444876 conn(0x7f8bb40737f0 msgr2=0x7f8bb4072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 -- 172.21.15.7:0/967444876 shutdown_connections 2023-06-09T11:27:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.609+0000 7f8bb95ff700 1 -- 172.21.15.7:0/967444876 wait complete. 2023-06-09T11:27:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.611+0000 7f8bb95ff700 1 Processor -- start 2023-06-09T11:27:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.611+0000 7f8bb95ff700 1 -- start start 2023-06-09T11:27:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.612+0000 7f8bb95ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb41535e0 0x7f8bb4156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.612+0000 7f8bb95ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.612+0000 7f8bb95ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 0x7f8bb4152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb95ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41562a0 con 0x7f8bb41539b0 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb95ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb41553f0 con 0x7f8bb41535e0 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb95ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4155690 con 0x7f8bb4159ee0 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43914/0 (socket says 172.21.15.7:43914) 2023-06-09T11:27:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 -- 172.21.15.7:0/417517810 learned_addr learned my addr 172.21.15.7:0/417517810 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb2ffd700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb41535e0 0x7f8bb4156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb37fe700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 0x7f8bb4152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 -- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 msgr2=0x7f8bb4152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 0x7f8bb4152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 -- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb41535e0 msgr2=0x7f8bb4156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb41535e0 0x7f8bb4156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb27fc700 1 -- 172.21.15.7:0/417517810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac009a70 con 0x7f8bb41539b0 2023-06-09T11:27:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.613+0000 7f8bb37fe700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 0x7f8bb4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:27:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8bb27fc700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005910 tx=0x7f8ba800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8005d70 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80070d0 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8007480 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb406ae10 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.614+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb406ab30 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.615+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb406b0f0 con 0x7f8bb41539b0 2023-06-09T11:27:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.617+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ba80075e0 con 0x7f8bb41539b0 2023-06-09T11:27:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.618+0000 7f8b9b7fe700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 0x7f8ba006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.618+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f8ba8088cc0 con 0x7f8bb41539b0 2023-06-09T11:27:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.618+0000 7f8bb2ffd700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 0x7f8ba006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.619+0000 7f8bb2ffd700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 0x7f8ba006d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40011e0 tx=0x7f8ba4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.622+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8ba808f080 con 0x7f8bb41539b0 2023-06-09T11:27:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.947+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb40c9c30 con 0x7f8bb41539b0 2023-06-09T11:27:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.947+0000 7f8b9b7fe700 1 -- 172.21.15.7:0/417517810 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f8ba8017090 con 0x7f8bb41539b0 2023-06-09T11:27:36.952 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:36.953 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 msgr2=0x7f8ba006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 0x7f8ba006d190 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40011e0 tx=0x7f8ba4009040 comp rx=0 tx=0).stop 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 msgr2=0x7f8bb41547e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8005910 tx=0x7f8ba800c9e0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 shutdown_connections 2023-06-09T11:27:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ba006ace0 0x7f8ba006d190 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bb4159ee0 0x7f8bb4152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bb41535e0 0x7f8bb4156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 --2- 172.21.15.7:0/417517810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bb41539b0 0x7f8bb41547e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 >> 172.21.15.7:0/417517810 conn(0x7f8bb40737f0 msgr2=0x7f8bb4159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 shutdown_connections 2023-06-09T11:27:36.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:36.950+0000 7f8bb95ff700 1 -- 172.21.15.7:0/417517810 wait complete. 2023-06-09T11:27:36.957 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:36.974 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:27:38.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.202+0000 7fa646bff700 1 Processor -- start 2023-06-09T11:27:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.202+0000 7fa646bff700 1 -- start start 2023-06-09T11:27:39.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6400cb5d0 0x7fa6400b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6400cb9a0 0x7fa640159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa64015cd40 0x7fa6400cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa640159b20 con 0x7fa64015cd40 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6400cac80 con 0x7fa6400cb5d0 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa646bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401597d0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa64519c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa64015cd40 0x7fa6400cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa64519c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa64015cd40 0x7fa6400cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43930/0 (socket says 172.21.15.7:43930) 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa64519c700 1 -- 172.21.15.7:0/3138067098 learned_addr learned my addr 172.21.15.7:0/3138067098 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.203+0000 7fa64499b700 1 --2- 172.21.15.7:0/3138067098 >> v2:172.21.15.117:3300/0 conn(0x7fa6400cb5d0 0x7fa6400b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa63ffff700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6400cb9a0 0x7fa640159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 -- 172.21.15.7:0/3138067098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6400cb9a0 msgr2=0x7fa640159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6400cb9a0 0x7fa640159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 -- 172.21.15.7:0/3138067098 >> v2:172.21.15.117:3300/0 conn(0x7fa6400cb5d0 msgr2=0x7fa6400b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 --2- 172.21.15.7:0/3138067098 >> v2:172.21.15.117:3300/0 conn(0x7fa6400cb5d0 0x7fa6400b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 -- 172.21.15.7:0/3138067098 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640071ab0 con 0x7fa64015cd40 2023-06-09T11:27:39.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa64519c700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64015cd40 0x7fa6400cbea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa64008d420 tx=0x7fa63800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913cc61594f0a33c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.204+0000 7fa63f7fe700 1 -- 172.21.15.7:0/3138067098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800e070 con 0x7fa64015cd40 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa63f7fe700 1 -- 172.21.15.7:0/3138067098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63800b910 con 0x7fa64015cd40 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa63f7fe700 1 -- 172.21.15.7:0/3138067098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800bcc0 con 0x7fa64015cd40 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 -- 172.21.15.7:0/3138067098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64015cd40 msgr2=0x7fa6400cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64015cd40 0x7fa6400cbea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa64008d420 tx=0x7fa63800b330 comp rx=0 tx=0).stop 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 -- 172.21.15.7:0/3138067098 shutdown_connections 2023-06-09T11:27:39.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 --2- 172.21.15.7:0/3138067098 >> v2:172.21.15.117:3300/0 conn(0x7fa6400cb5d0 0x7fa6400b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa64015cd40 0x7fa6400cbea0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 --2- 172.21.15.7:0/3138067098 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6400cb9a0 0x7fa640159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 -- 172.21.15.7:0/3138067098 >> 172.21.15.7:0/3138067098 conn(0x7fa640070af0 msgr2=0x7fa640070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 -- 172.21.15.7:0/3138067098 shutdown_connections 2023-06-09T11:27:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.205+0000 7fa646bff700 1 -- 172.21.15.7:0/3138067098 wait complete. 2023-06-09T11:27:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.207+0000 7fa646bff700 1 Processor -- start 2023-06-09T11:27:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.208+0000 7fa646bff700 1 -- start start 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6400cb5d0 0x7fa6400c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa64015cd40 0x7fa6400c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400c47f0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400c6050 con 0x7fa6400cb5d0 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa646bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400c8e20 con 0x7fa64015cd40 2023-06-09T11:27:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43942/0 (socket says 172.21.15.7:43942) 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa64499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6400cb5d0 0x7fa6400c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 -- 172.21.15.7:0/1211691351 learned_addr learned my addr 172.21.15.7:0/1211691351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa64519c700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa64015cd40 0x7fa6400c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 -- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa64015cd40 msgr2=0x7fa6400c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa64015cd40 0x7fa6400c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 -- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6400cb5d0 msgr2=0x7fa6400c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6400cb5d0 0x7fa6400c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.209+0000 7fa63ffff700 1 -- 172.21.15.7:0/1211691351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638009b30 con 0x7fa6400cb9a0 2023-06-09T11:27:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa63ffff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00e980 tx=0x7fa62c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c009710 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c00c690 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00ca40 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64015d4f0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.210+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64015d210 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.212+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64015d7d0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.212+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa62c00cba0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.214+0000 7fa63d7fa700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 0x7fa62406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.214+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fa62c089b30 con 0x7fa6400cb9a0 2023-06-09T11:27:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.214+0000 7fa64499b700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 0x7fa62406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.215+0000 7fa64499b700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 0x7fa62406d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa634009ac0 tx=0x7fa634009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.219+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa62c054860 con 0x7fa6400cb9a0 2023-06-09T11:27:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.506+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa640074cf0 con 0x7fa6400cb9a0 2023-06-09T11:27:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.507+0000 7fa63d7fa700 1 -- 172.21.15.7:0/1211691351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fa62c021090 con 0x7fa6400cb9a0 2023-06-09T11:27:39.512 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:39.512 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.509+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 msgr2=0x7fa62406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.509+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 0x7fa62406d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa634009ac0 tx=0x7fa634009210 comp rx=0 tx=0).stop 2023-06-09T11:27:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.509+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 msgr2=0x7fa6401528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.509+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00e980 tx=0x7fa62c00ec90 comp rx=0 tx=0).stop 2023-06-09T11:27:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 shutdown_connections 2023-06-09T11:27:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa62406ace0 0x7fa62406d190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa64015cd40 0x7fa6400c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6400cb5d0 0x7fa6400c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 --2- 172.21.15.7:0/1211691351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6400cb9a0 0x7fa6401528f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 >> 172.21.15.7:0/1211691351 conn(0x7fa640070af0 msgr2=0x7fa6400cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 shutdown_connections 2023-06-09T11:27:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:39.510+0000 7fa646bff700 1 -- 172.21.15.7:0/1211691351 wait complete. 2023-06-09T11:27:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:39.534 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:27:41.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.754+0000 7feb0d3ff700 1 Processor -- start 2023-06-09T11:27:41.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.754+0000 7feb0d3ff700 1 -- start start 2023-06-09T11:27:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca1b0 0x7feb0815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb080b5360 0x7feb080c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7feb080ca580 0x7feb081536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb080c5e00 con 0x7feb080b5360 2023-06-09T11:27:41.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0815dcf0 con 0x7feb080ca580 2023-06-09T11:27:41.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.756+0000 7feb0d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0815d450 con 0x7feb080ca1b0 2023-06-09T11:27:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb080b5360 0x7feb080c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb080b5360 0x7feb080c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43954/0 (socket says 172.21.15.7:43954) 2023-06-09T11:27:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 -- 172.21.15.7:0/3480929004 learned_addr learned my addr 172.21.15.7:0/3480929004 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb077fe700 1 --2- 172.21.15.7:0/3480929004 >> v2:172.21.15.117:3300/0 conn(0x7feb080ca580 0x7feb081536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb06ffd700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca1b0 0x7feb0815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 -- 172.21.15.7:0/3480929004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca1b0 msgr2=0x7feb0815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca1b0 0x7feb0815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 -- 172.21.15.7:0/3480929004 >> v2:172.21.15.117:3300/0 conn(0x7feb080ca580 msgr2=0x7feb081536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 --2- 172.21.15.7:0/3480929004 >> v2:172.21.15.117:3300/0 conn(0x7feb080ca580 0x7feb081536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 -- 172.21.15.7:0/3480929004 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0815d730 con 0x7feb080b5360 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.757+0000 7feb067fc700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb080c58c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7feb0808d4a0 tx=0x7feafc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e881464f5c8025a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:41.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb05ffb700 1 -- 172.21.15.7:0/3480929004 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011830 con 0x7feb080b5360 2023-06-09T11:27:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb05ffb700 1 -- 172.21.15.7:0/3480929004 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc011990 con 0x7feb080b5360 2023-06-09T11:27:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb05ffb700 1 -- 172.21.15.7:0/3480929004 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc011d20 con 0x7feb080b5360 2023-06-09T11:27:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3480929004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 msgr2=0x7feb080c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:41.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb080c58c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7feb0808d4a0 tx=0x7feafc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3480929004 shutdown_connections 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3480929004 >> v2:172.21.15.117:3300/0 conn(0x7feb080ca580 0x7feb081536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca1b0 0x7feb0815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3480929004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb080c58c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.758+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3480929004 >> 172.21.15.7:0/3480929004 conn(0x7feb08072090 msgr2=0x7feb08072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:41.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.759+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3480929004 shutdown_connections 2023-06-09T11:27:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.759+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3480929004 wait complete. 2023-06-09T11:27:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.761+0000 7feb0d3ff700 1 Processor -- start 2023-06-09T11:27:41.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.761+0000 7feb0d3ff700 1 -- start start 2023-06-09T11:27:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 0x7feb0815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca580 0x7feb0814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb081f0230 con 0x7feb080b5360 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0815aa30 con 0x7feb080ca1b0 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb0d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0815c7d0 con 0x7feb080ca580 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb06ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43962/0 (socket says 172.21.15.7:43962) 2023-06-09T11:27:41.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb06ffd700 1 -- 172.21.15.7:0/3641631676 learned_addr learned my addr 172.21.15.7:0/3641631676 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb077fe700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca580 0x7feb0814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.762+0000 7feb067fc700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 0x7feb0815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 -- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca580 msgr2=0x7feb0814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca580 0x7feb0814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 -- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 msgr2=0x7feb0815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 0x7feb0815b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:41.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 -- 172.21.15.7:0/3641631676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc009550 con 0x7feb080b5360 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb067fc700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 0x7feb0815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb06ffd700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e930 tx=0x7feaf800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf800ee80 con 0x7feb080b5360 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf800c850 con 0x7feb080b5360 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf800cbe0 con 0x7feb080b5360 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.763+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0806a570 con 0x7feb080b5360 2023-06-09T11:27:41.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.764+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0815da10 con 0x7feb080b5360 2023-06-09T11:27:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.765+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0806a850 con 0x7feb080b5360 2023-06-09T11:27:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.766+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feaf800cd40 con 0x7feb080b5360 2023-06-09T11:27:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.767+0000 7feaef7fe700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 0x7feaf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:41.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.767+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7feaf8089920 con 0x7feb080b5360 2023-06-09T11:27:41.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.767+0000 7feb067fc700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 0x7feaf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:41.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.768+0000 7feb067fc700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 0x7feaf406d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feb080b5750 tx=0x7feafc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:41.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:41.772+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feaf80545b0 con 0x7feb080b5360 2023-06-09T11:27:42.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.061+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb08074cf0 con 0x7feb080b5360 2023-06-09T11:27:42.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.062+0000 7feaef7fe700 1 -- 172.21.15.7:0/3641631676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+6216 (secure 0 0 0) 0x7feaf8021090 con 0x7feb080b5360 2023-06-09T11:27:42.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:42.067 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":72,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5994,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:25:27.452456+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":57,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5455},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5455":{"gid":5455,"name":"d","rank":2,"incarnation":55,"state":"up:active","state_seq":35,"addr":"172.21.15.117:6835/2166536228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":2166536228},{"type":"v1","addr":"172.21.15.117:6835","nonce":2166536228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:42.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.064+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 msgr2=0x7feaf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.064+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 0x7feaf406d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7feb080b5750 tx=0x7feafc008030 comp rx=0 tx=0).stop 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.064+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 msgr2=0x7feb081efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.064+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e930 tx=0x7feaf800ec40 comp rx=0 tx=0).stop 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 shutdown_connections 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feaf406ace0 0x7feaf406d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb080ca580 0x7feb0814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb080b5360 0x7feb081efcf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 --2- 172.21.15.7:0/3641631676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb080ca1b0 0x7feb0815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 >> 172.21.15.7:0/3641631676 conn(0x7feb08072090 msgr2=0x7feb0814f490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 shutdown_connections 2023-06-09T11:27:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:42.065+0000 7feb0d3ff700 1 -- 172.21.15.7:0/3641631676 wait complete. 2023-06-09T11:27:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 72 2023-06-09T11:27:42.092 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:27:44.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:44.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.307+0000 7f340ef9d700 1 Processor -- start 2023-06-09T11:27:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.308+0000 7f340ef9d700 1 -- start start 2023-06-09T11:27:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.309+0000 7f340ef9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.309+0000 7f340ef9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3408071d50 0x7f3408072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.309+0000 7f340ef9d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34080d01a0 con 0x7f3408156f20 2023-06-09T11:27:44.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.309+0000 7f340ef9d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34080cb2b0 con 0x7f3408071d50 2023-06-09T11:27:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.309+0000 7f340ef9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34080cb020 con 0x7f3408156b50 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3408071d50 0x7f3408072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340df9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340df9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:59462/0 (socket says 172.21.15.7:59462) 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3408071d50 0x7f3408072170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:47680/0 (socket says 172.21.15.7:47680) 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340df9b700 1 -- 172.21.15.7:0/4003807393 learned_addr learned my addr 172.21.15.7:0/4003807393 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:44.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 -- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 msgr2=0x7f34080caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 -- 172.21.15.7:0/4003807393 >> v1:172.21.15.117:6789/0 conn(0x7f3408156f20 legacy=0x7f34080cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340e79c700 1 -- 172.21.15.7:0/4003807393 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080b54a0 con 0x7f3408071d50 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.310+0000 7f340df9b700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340e79c700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408071d50 0x7f3408072170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3400009d90 tx=0x7f340000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c71c3ea10241087 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340cf99700 1 -- 172.21.15.7:0/4003807393 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000e070 con 0x7f3408071d50 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340cf99700 1 -- 172.21.15.7:0/4003807393 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340000bb20 con 0x7f3408071d50 2023-06-09T11:27:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340cf99700 1 -- 172.21.15.7:0/4003807393 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400012430 con 0x7f3408071d50 2023-06-09T11:27:44.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340ef9d700 1 -- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408071d50 msgr2=0x7f3408072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.311+0000 7f340ef9d700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408071d50 0x7f3408072170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3400009d90 tx=0x7f340000b330 comp rx=0 tx=0).stop 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 -- 172.21.15.7:0/4003807393 shutdown_connections 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408071d50 0x7f3408072170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 --2- 172.21.15.7:0/4003807393 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34080caae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 -- 172.21.15.7:0/4003807393 >> 172.21.15.7:0/4003807393 conn(0x7f34080737f0 msgr2=0x7f3408073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 -- 172.21.15.7:0/4003807393 shutdown_connections 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.312+0000 7f340ef9d700 1 -- 172.21.15.7:0/4003807393 wait complete. 2023-06-09T11:27:44.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.314+0000 7f340ef9d700 1 Processor -- start 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.314+0000 7f340ef9d700 1 -- start start 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34081540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408156f20 0x7f340814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3408153750 con 0x7f3408071d50 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3408153400 con 0x7f3408156f20 2023-06-09T11:27:44.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340ef9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3408152310 con 0x7f3408156b50 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340df9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340df9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47624/0 (socket says 172.21.15.7:47624) 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.315+0000 7f340df9b700 1 -- 172.21.15.7:0/225348074 learned_addr learned my addr 172.21.15.7:0/225348074 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340e79c700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408156f20 0x7f340814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340d79a700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34081540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 -- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 msgr2=0x7f34081540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34081540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 -- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408156f20 msgr2=0x7f340814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408156f20 0x7f340814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 -- 172.21.15.7:0/225348074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3404009500 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f340df9b700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f340400ebb0 tx=0x7f340400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.316+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3404005a20 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.317+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3404005b80 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.317+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340400c850 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.317+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3400009a70 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.317+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34080cf870 con 0x7f3408071d50 2023-06-09T11:27:44.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.318+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34080cf590 con 0x7f3408071d50 2023-06-09T11:27:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.319+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f340400c9b0 con 0x7f3408071d50 2023-06-09T11:27:44.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.320+0000 7f33fe7fc700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 0x7f33ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.321+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f3404089a90 con 0x7f3408071d50 2023-06-09T11:27:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.321+0000 7f340d79a700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 0x7f33ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.321+0000 7f340d79a700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 0x7f33ec06d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34080b1e30 tx=0x7f33f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.325+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f34040547b0 con 0x7f3408071d50 2023-06-09T11:27:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.632+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34080cfb50 con 0x7f3408071d50 2023-06-09T11:27:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.632+0000 7f33fe7fc700 1 -- 172.21.15.7:0/225348074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f3404057dd0 con 0x7f3408071d50 2023-06-09T11:27:44.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:44.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:44.343614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 msgr2=0x7f33ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 0x7f33ec06d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f34080b1e30 tx=0x7f33f8009040 comp rx=0 tx=0).stop 2023-06-09T11:27:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 msgr2=0x7f3408153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f340400ebb0 tx=0x7f340400eec0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 shutdown_connections 2023-06-09T11:27:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3408156f20 0x7f340814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3408071d50 0x7f3408153b70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33ec06ace0 0x7f33ec06d190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 --2- 172.21.15.7:0/225348074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3408156b50 0x7f34081540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 >> 172.21.15.7:0/225348074 conn(0x7f34080737f0 msgr2=0x7f34080ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:44.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.635+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 shutdown_connections 2023-06-09T11:27:44.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.636+0000 7f340ef9d700 1 -- 172.21.15.7:0/225348074 wait complete. 2023-06-09T11:27:44.642 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 73 2023-06-09T11:27:44.669 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:27:44.669 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:27:44.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.886+0000 7ff073dff700 1 Processor -- start 2023-06-09T11:27:44.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.886+0000 7ff073dff700 1 -- start start 2023-06-09T11:27:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.888+0000 7ff073dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff06c0ca070 0x7ff06c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.888+0000 7ff073dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c15a390 con 0x7ff06c153b30 2023-06-09T11:27:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.888+0000 7ff073dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0ca720 con 0x7ff06c153760 2023-06-09T11:27:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.888+0000 7ff073dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0c5f40 con 0x7ff06c0ca070 2023-06-09T11:27:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07139a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff06c153b30 0x7ff06c071fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41098/0 (socket says 172.21.15.7:41098) 2023-06-09T11:27:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07139a700 1 -- 172.21.15.7:0/3218273229 learned_addr learned my addr 172.21.15.7:0/3218273229 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:44.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07239c700 1 --2- 172.21.15.7:0/3218273229 >> v2:172.21.15.7:3300/0 conn(0x7ff06c0ca070 0x7ff06c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303569212 0 0) 0x7ff06c15a390 con 0x7ff06c153b30 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0580035b0 con 0x7ff06c153b30 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639478033 0 0) 0x7ff06c0ca720 con 0x7ff06c153760 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07239c700 1 -- 172.21.15.7:0/3218273229 >> v1:172.21.15.117:6789/0 conn(0x7ff06c153760 legacy=0x7ff06c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07239c700 1 -- 172.21.15.7:0/3218273229 >> v1:172.21.15.7:6789/0 conn(0x7ff06c153b30 legacy=0x7ff06c071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.889+0000 7ff07239c700 1 -- 172.21.15.7:0/3218273229 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0c5a20 con 0x7ff06c0ca070 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff07239c700 1 --2- 172.21.15.7:0/3218273229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca070 0x7ff06c15cc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff064009d90 tx=0x7ff06400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1f0e37a5413319 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400e070 con 0x7ff06c0ca070 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400bb20 con 0x7ff06c0ca070 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff070b99700 1 -- 172.21.15.7:0/3218273229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400bed0 con 0x7ff06c0ca070 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff073dff700 1 -- 172.21.15.7:0/3218273229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca070 msgr2=0x7ff06c15cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.890+0000 7ff073dff700 1 --2- 172.21.15.7:0/3218273229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca070 0x7ff06c15cc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff064009d90 tx=0x7ff06400b330 comp rx=0 tx=0).stop 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.891+0000 7ff073dff700 1 -- 172.21.15.7:0/3218273229 shutdown_connections 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.891+0000 7ff073dff700 1 --2- 172.21.15.7:0/3218273229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c0ca070 0x7ff06c15cc70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.891+0000 7ff073dff700 1 -- 172.21.15.7:0/3218273229 >> 172.21.15.7:0/3218273229 conn(0x7ff06c0737f0 msgr2=0x7ff06c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.891+0000 7ff073dff700 1 -- 172.21.15.7:0/3218273229 shutdown_connections 2023-06-09T11:27:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.891+0000 7ff073dff700 1 -- 172.21.15.7:0/3218273229 wait complete. 2023-06-09T11:27:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.893+0000 7ff073dff700 1 Processor -- start 2023-06-09T11:27:44.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.893+0000 7ff073dff700 1 -- start start 2023-06-09T11:27:44.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff06c0ca070 0x7ff06c0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff06c153760 0x7ff06c0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0ce600 con 0x7ff06c153b30 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0b2400 con 0x7ff06c0ca070 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff073dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0ce370 con 0x7ff06c153760 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff07239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff07239c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47652/0 (socket says 172.21.15.7:47652) 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff07239c700 1 -- 172.21.15.7:0/889421687 learned_addr learned my addr 172.21.15.7:0/889421687 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff07139a700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff06c153760 0x7ff06c0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.894+0000 7ff071b9b700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff06c0ca070 0x7ff06c0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 -- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff06c153760 msgr2=0x7ff06c0721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff06c153760 0x7ff06c0721f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 -- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff06c0ca070 msgr2=0x7ff06c0b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff06c0ca070 0x7ff06c0b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:44.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 -- 172.21.15.7:0/889421687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009a70 con 0x7ff06c153b30 2023-06-09T11:27:44.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff07239c700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0d07f0 tx=0x7ff064004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06401b810 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff064020070 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06401bbd0 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.895+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c159a60 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.896+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c159780 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.897+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c159d40 con 0x7ff06c153b30 2023-06-09T11:27:44.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.898+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff06401bd30 con 0x7ff06c153b30 2023-06-09T11:27:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.899+0000 7ff0627fc700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 0x7ff05006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.899+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7ff064022070 con 0x7ff06c153b30 2023-06-09T11:27:44.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.899+0000 7ff071b9b700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 0x7ff05006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:44.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.900+0000 7ff071b9b700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 0x7ff05006d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff068007fd0 tx=0x7ff068009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:44.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:44.904+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff064056590 con 0x7ff06c153b30 2023-06-09T11:27:45.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:45.185 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:27:45.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.216+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff06c1546a0 con 0x7ff06c153b30 2023-06-09T11:27:45.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.217+0000 7ff0627fc700 1 -- 172.21.15.7:0/889421687 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5455 (secure 0 0 0) 0x7ff06c1546a0 con 0x7ff06c153b30 2023-06-09T11:27:45.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:45.222 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:44.343614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 msgr2=0x7ff05006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 0x7ff05006d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff068007fd0 tx=0x7ff068009040 comp rx=0 tx=0).stop 2023-06-09T11:27:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 msgr2=0x7ff06c1f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0d07f0 tx=0x7ff064004ad0 comp rx=0 tx=0).stop 2023-06-09T11:27:45.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 shutdown_connections 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff06c153b30 0x7ff06c1f2a00 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff06c0ca070 0x7ff06c0b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff05006ace0 0x7ff05006d190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 --2- 172.21.15.7:0/889421687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff06c153760 0x7ff06c0721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 >> 172.21.15.7:0/889421687 conn(0x7ff06c0737f0 msgr2=0x7ff06c150440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 shutdown_connections 2023-06-09T11:27:45.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:45.220+0000 7ff073dff700 1 -- 172.21.15.7:0/889421687 wait complete. 2023-06-09T11:27:45.227 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 73 2023-06-09T11:27:45.246 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 73, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 73, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 36 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:27:44.343614+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:27:47.247 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:47.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.486+0000 7f14f45ff700 1 Processor -- start 2023-06-09T11:27:47.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.487+0000 7f14f45ff700 1 -- start start 2023-06-09T11:27:47.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f45ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14ec06fce0 0x7f14ec151310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f45ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f14ec0c9230 0x7f14ec151850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f45ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec15b700 con 0x7f14ec0c2e30 2023-06-09T11:27:47.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f45ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0cacb0 con 0x7f14ec06fce0 2023-06-09T11:27:47.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f45ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0c8730 con 0x7f14ec0c9230 2023-06-09T11:27:47.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f1b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f14ec0c2e30 0x7f14ec0c8b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41106/0 (socket says 172.21.15.7:41106) 2023-06-09T11:27:47.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f1b9a700 1 -- 172.21.15.7:0/1877893679 learned_addr learned my addr 172.21.15.7:0/1877893679 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f239b700 1 --2- 172.21.15.7:0/1877893679 >> v2:172.21.15.7:3300/0 conn(0x7f14ec06fce0 0x7f14ec151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.488+0000 7f14f2b9c700 1 --2- 172.21.15.7:0/1877893679 >> v2:172.21.15.117:3300/0 conn(0x7f14ec0c9230 0x7f14ec151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f239b700 1 -- 172.21.15.7:0/1877893679 >> v2:172.21.15.117:3300/0 conn(0x7f14ec0c9230 msgr2=0x7f14ec151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f239b700 1 --2- 172.21.15.7:0/1877893679 >> v2:172.21.15.117:3300/0 conn(0x7f14ec0c9230 0x7f14ec151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f239b700 1 -- 172.21.15.7:0/1877893679 >> v1:172.21.15.7:6789/0 conn(0x7f14ec0c2e30 legacy=0x7f14ec0c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f239b700 1 -- 172.21.15.7:0/1877893679 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec15ab30 con 0x7f14ec06fce0 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f239b700 1 --2- 172.21.15.7:0/1877893679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec06fce0 0x7f14ec151310 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b3750 tx=0x7f14e8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a62dec7989af01 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:47.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.489+0000 7f14f1399700 1 -- 172.21.15.7:0/1877893679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011830 con 0x7f14ec06fce0 2023-06-09T11:27:47.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f1399700 1 -- 172.21.15.7:0/1877893679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e8011990 con 0x7f14ec06fce0 2023-06-09T11:27:47.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f1399700 1 -- 172.21.15.7:0/1877893679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e8011d20 con 0x7f14ec06fce0 2023-06-09T11:27:47.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 -- 172.21.15.7:0/1877893679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec06fce0 msgr2=0x7f14ec151310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1877893679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec06fce0 0x7f14ec151310 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0b3750 tx=0x7f14e8009b20 comp rx=0 tx=0).stop 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 -- 172.21.15.7:0/1877893679 shutdown_connections 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1877893679 >> v2:172.21.15.117:3300/0 conn(0x7f14ec0c9230 0x7f14ec151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1877893679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec06fce0 0x7f14ec151310 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 -- 172.21.15.7:0/1877893679 >> 172.21.15.7:0/1877893679 conn(0x7f14ec0c9f00 msgr2=0x7f14ec0ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.490+0000 7f14f45ff700 1 -- 172.21.15.7:0/1877893679 shutdown_connections 2023-06-09T11:27:47.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.491+0000 7f14f45ff700 1 -- 172.21.15.7:0/1877893679 wait complete. 2023-06-09T11:27:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.493+0000 7f14f45ff700 1 Processor -- start 2023-06-09T11:27:47.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.493+0000 7f14f45ff700 1 -- start start 2023-06-09T11:27:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 0x7f14ec0cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14ec0c9230 0x7f14ec14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec1efdc0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec06ce30 con 0x7f14ec0c9230 2023-06-09T11:27:47.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.494+0000 7f14f45ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec150070 con 0x7f14ec06fce0 2023-06-09T11:27:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47670/0 (socket says 172.21.15.7:47670) 2023-06-09T11:27:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 -- 172.21.15.7:0/1841683395 learned_addr learned my addr 172.21.15.7:0/1841683395 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f239b700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 0x7f14ec0cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f2b9c700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14ec0c9230 0x7f14ec14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 -- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 msgr2=0x7f14ec0cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 0x7f14ec0cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 -- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14ec0c9230 msgr2=0x7f14ec14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14ec0c9230 0x7f14ec14f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 -- 172.21.15.7:0/1841683395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80094a0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f239b700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 0x7f14ec0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.495+0000 7f14f1b9a700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c520 tx=0x7f14dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:47.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.496+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc005d70 con 0x7f14ec0c2e30 2023-06-09T11:27:47.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.496+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc0070d0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.496+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc007440 con 0x7f14ec0c2e30 2023-06-09T11:27:47.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.496+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec15b0b0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.496+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec15add0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.497+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec14efd0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.498+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14dc0075a0 con 0x7f14ec0c2e30 2023-06-09T11:27:47.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.500+0000 7f14e2ffd700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 0x7f14d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:47.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.500+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f14dc088a70 con 0x7f14ec0c2e30 2023-06-09T11:27:47.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.500+0000 7f14f239b700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 0x7f14d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:47.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.501+0000 7f14f239b700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 0x7f14d006d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14e80090b0 tx=0x7f14e8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.504+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14dc053800 con 0x7f14ec0c2e30 2023-06-09T11:27:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.825+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14ec15b390 con 0x7f14ec0c2e30 2023-06-09T11:27:47.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.826+0000 7f14e2ffd700 1 -- 172.21.15.7:0/1841683395 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f14dc017090 con 0x7f14ec0c2e30 2023-06-09T11:27:47.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:47.831 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:44.343614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 msgr2=0x7f14d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 0x7f14d006d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f14e80090b0 tx=0x7f14e8008030 comp rx=0 tx=0).stop 2023-06-09T11:27:47.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 msgr2=0x7f14ec14f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c520 tx=0x7f14dc00c830 comp rx=0 tx=0).stop 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 shutdown_connections 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14ec0c9230 0x7f14ec14f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14d006ace0 0x7f14d006d190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14ec06fce0 0x7f14ec0cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 --2- 172.21.15.7:0/1841683395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14ec0c2e30 0x7f14ec14f460 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:47.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 >> 172.21.15.7:0/1841683395 conn(0x7f14ec0c9f00 msgr2=0x7f14ec14b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:47.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.828+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 shutdown_connections 2023-06-09T11:27:47.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:47.829+0000 7f14f45ff700 1 -- 172.21.15.7:0/1841683395 wait complete. 2023-06-09T11:27:47.835 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 73 2023-06-09T11:27:47.854 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:27:49.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:50.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.103+0000 7f9456dff700 1 Processor -- start 2023-06-09T11:27:50.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.103+0000 7f9456dff700 1 -- start start 2023-06-09T11:27:50.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.104+0000 7f9456dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9450153760 0x7f94500ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9456dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9450153b30 0x7f94500b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9456dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450071fa0 0x7f94500723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9456dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500c5f40 con 0x7f9450153b30 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9456dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945015a390 con 0x7f9450153760 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9456dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500ca720 con 0x7f9450071fa0 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f944ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9450153b30 0x7f94500b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f9454b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9450153760 0x7f94500ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f944ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9450153b30 0x7f94500b5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47686/0 (socket says 172.21.15.7:47686) 2023-06-09T11:27:50.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f944ffff700 1 -- 172.21.15.7:0/43070283 learned_addr learned my addr 172.21.15.7:0/43070283 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:50.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.105+0000 7f945539c700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450071fa0 0x7f94500723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 -- 172.21.15.7:0/43070283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450071fa0 msgr2=0x7f94500723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450071fa0 0x7f94500723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 -- 172.21.15.7:0/43070283 >> v2:172.21.15.117:3300/0 conn(0x7f9450153760 msgr2=0x7f94500ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 --2- 172.21.15.7:0/43070283 >> v2:172.21.15.117:3300/0 conn(0x7f9450153760 0x7f94500ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 -- 172.21.15.7:0/43070283 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500c5a20 con 0x7f9450153b30 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944ffff700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153b30 0x7f94500b5100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f94500743c0 tx=0x7f943c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbe61fb452ed505 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944f7fe700 1 -- 172.21.15.7:0/43070283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c011830 con 0x7f9450153b30 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944f7fe700 1 -- 172.21.15.7:0/43070283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c011990 con 0x7f9450153b30 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.106+0000 7f944f7fe700 1 -- 172.21.15.7:0/43070283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c011d20 con 0x7f9450153b30 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 -- 172.21.15.7:0/43070283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153b30 msgr2=0x7f94500b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153b30 0x7f94500b5100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f94500743c0 tx=0x7f943c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 -- 172.21.15.7:0/43070283 shutdown_connections 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 --2- 172.21.15.7:0/43070283 >> v2:172.21.15.117:3300/0 conn(0x7f9450153760 0x7f94500ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450071fa0 0x7f94500723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 --2- 172.21.15.7:0/43070283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153b30 0x7f94500b5100 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.107+0000 7f9456dff700 1 -- 172.21.15.7:0/43070283 >> 172.21.15.7:0/43070283 conn(0x7f94500737f0 msgr2=0x7f9450073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.108+0000 7f9456dff700 1 -- 172.21.15.7:0/43070283 shutdown_connections 2023-06-09T11:27:50.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.108+0000 7f9456dff700 1 -- 172.21.15.7:0/43070283 wait complete. 2023-06-09T11:27:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.110+0000 7f9456dff700 1 Processor -- start 2023-06-09T11:27:50.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.110+0000 7f9456dff700 1 -- start start 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9450071fa0 0x7f945015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450153b30 0x7f94501f0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b40f0 con 0x7f9450153760 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945014b930 con 0x7f9450071fa0 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9456dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500c77d0 con 0x7f9450153b30 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f944ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f944ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47694/0 (socket says 172.21.15.7:47694) 2023-06-09T11:27:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f944ffff700 1 -- 172.21.15.7:0/612504551 learned_addr learned my addr 172.21.15.7:0/612504551 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f945539c700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450153b30 0x7f94501f0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.111+0000 7f9454b9b700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9450071fa0 0x7f945015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 -- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450153b30 msgr2=0x7f94501f0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450153b30 0x7f94501f0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 -- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9450071fa0 msgr2=0x7f945015c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9450071fa0 0x7f945015c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 -- 172.21.15.7:0/612504551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c009550 con 0x7f9450153760 2023-06-09T11:27:50.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.112+0000 7f944ffff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f943c009c10 tx=0x7f943c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:50.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.113+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c004450 con 0x7f9450153760 2023-06-09T11:27:50.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.113+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c01e070 con 0x7f9450153760 2023-06-09T11:27:50.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.113+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c0038e0 con 0x7f9450153760 2023-06-09T11:27:50.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.113+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9450159a60 con 0x7f9450153760 2023-06-09T11:27:50.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.113+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450159780 con 0x7f9450153760 2023-06-09T11:27:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.114+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450159d40 con 0x7f9450153760 2023-06-09T11:27:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.115+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f943c003a40 con 0x7f9450153760 2023-06-09T11:27:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.117+0000 7f944d7fa700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 0x7f943406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.117+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f943c08a570 con 0x7f9450153760 2023-06-09T11:27:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.118+0000 7f9454b9b700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 0x7f943406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:50.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.119+0000 7f9454b9b700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 0x7f943406d1e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9444009be0 tx=0x7f9444009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.122+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f943c055230 con 0x7f9450153760 2023-06-09T11:27:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.496+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9450074cf0 con 0x7f9450153760 2023-06-09T11:27:50.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.496+0000 7f944d7fa700 1 -- 172.21.15.7:0/612504551 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f943c02c3b0 con 0x7f9450153760 2023-06-09T11:27:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:44.343614+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:50.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 msgr2=0x7f943406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:50.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 0x7f943406d1e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9444009be0 tx=0x7f9444009210 comp rx=0 tx=0).stop 2023-06-09T11:27:50.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 msgr2=0x7f94501f0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:50.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f943c009c10 tx=0x7f943c004750 comp rx=0 tx=0).stop 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 shutdown_connections 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f943406ad30 0x7f943406d1e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9450153b30 0x7f94501f0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9450071fa0 0x7f945015c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 --2- 172.21.15.7:0/612504551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9450153760 0x7f94501f0010 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 >> 172.21.15.7:0/612504551 conn(0x7f94500737f0 msgr2=0x7f94500cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:50.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 shutdown_connections 2023-06-09T11:27:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:50.499+0000 7f9456dff700 1 -- 172.21.15.7:0/612504551 wait complete. 2023-06-09T11:27:50.507 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 73 2023-06-09T11:27:50.525 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:27:52.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.744+0000 7f0dbafff700 1 Processor -- start 2023-06-09T11:27:52.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.744+0000 7f0dbafff700 1 -- start start 2023-06-09T11:27:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.745+0000 7f0dbafff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 0x7f0db40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0dbafff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0db4071230 0x7f0db4158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0dbafff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db41580a0 con 0x7f0db415b380 2023-06-09T11:27:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0dbafff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db41518e0 con 0x7f0db4071230 2023-06-09T11:27:52.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0dbafff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c6180 con 0x7f0db4151510 2023-06-09T11:27:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0db3fff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0db415b380 0x7f0db4070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:43066/0 (socket says 172.21.15.7:43066) 2023-06-09T11:27:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0db3fff700 1 -- 172.21.15.7:0/1947720499 learned_addr learned my addr 172.21.15.7:0/1947720499 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1947720499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 0x7f0db40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0db959c700 1 --2- 172.21.15.7:0/1947720499 >> v2:172.21.15.117:3300/0 conn(0x7f0db4071230 0x7f0db4158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.746+0000 7f0db37fe700 1 -- 172.21.15.7:0/1947720499 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014892915 0 0) 0x7f0db41580a0 con 0x7f0db415b380 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db37fe700 1 -- 172.21.15.7:0/1947720499 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da40035b0 con 0x7f0db415b380 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1947720499 >> v2:172.21.15.117:3300/0 conn(0x7f0db4071230 msgr2=0x7f0db4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1947720499 >> v2:172.21.15.117:3300/0 conn(0x7f0db4071230 0x7f0db4158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1947720499 >> v1:172.21.15.117:6789/0 conn(0x7f0db415b380 legacy=0x7f0db4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1947720499 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4157490 con 0x7f0db4151510 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1947720499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 0x7f0db40c5b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009840 tx=0x7f0da8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d046a6105f16b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:52.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db37fe700 1 -- 172.21.15.7:0/1947720499 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8011830 con 0x7f0db4151510 2023-06-09T11:27:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db37fe700 1 -- 172.21.15.7:0/1947720499 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8011990 con 0x7f0db4151510 2023-06-09T11:27:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.747+0000 7f0db37fe700 1 -- 172.21.15.7:0/1947720499 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8011d20 con 0x7f0db4151510 2023-06-09T11:27:52.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 -- 172.21.15.7:0/1947720499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 msgr2=0x7f0db40c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1947720499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 0x7f0db40c5b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0da8009840 tx=0x7f0da8009b50 comp rx=0 tx=0).stop 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 -- 172.21.15.7:0/1947720499 shutdown_connections 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1947720499 >> v2:172.21.15.117:3300/0 conn(0x7f0db4071230 0x7f0db4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1947720499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db4151510 0x7f0db40c5b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 -- 172.21.15.7:0/1947720499 >> 172.21.15.7:0/1947720499 conn(0x7f0db406a4d0 msgr2=0x7f0db406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 -- 172.21.15.7:0/1947720499 shutdown_connections 2023-06-09T11:27:52.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.748+0000 7f0dbafff700 1 -- 172.21.15.7:0/1947720499 wait complete. 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.750+0000 7f0dbafff700 1 Processor -- start 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.750+0000 7f0dbafff700 1 -- start start 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4151510 0x7f0db40c0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db415b380 0x7f0db40c1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c1d00 con 0x7f0db4071230 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c1fc0 con 0x7f0db4151510 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.751+0000 7f0dbafff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db41f2e90 con 0x7f0db415b380 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47704/0 (socket says 172.21.15.7:47704) 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1940312983 learned_addr learned my addr 172.21.15.7:0/1940312983 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db3fff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4151510 0x7f0db40c0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db959c700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db415b380 0x7f0db40c1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db415b380 msgr2=0x7f0db40c1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db415b380 0x7f0db40c1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4151510 msgr2=0x7f0db40c0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4151510 0x7f0db40c0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:52.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 -- 172.21.15.7:0/1940312983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da80094a0 con 0x7f0db4071230 2023-06-09T11:27:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.752+0000 7f0db8d9b700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0da80090b0 tx=0x7f0da8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.753+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da80036a0 con 0x7f0db4071230 2023-06-09T11:27:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.753+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da801f070 con 0x7f0db4071230 2023-06-09T11:27:52.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.753+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db4157a50 con 0x7f0db4071230 2023-06-09T11:27:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.753+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8003a40 con 0x7f0db4071230 2023-06-09T11:27:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.753+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db4157770 con 0x7f0db4071230 2023-06-09T11:27:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.754+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db4157d30 con 0x7f0db4071230 2023-06-09T11:27:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.755+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0da8003ba0 con 0x7f0db4071230 2023-06-09T11:27:52.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.756+0000 7f0db17fa700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 0x7f0d9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:52.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.757+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f0da8021070 con 0x7f0db4071230 2023-06-09T11:27:52.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.757+0000 7f0db3fff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 0x7f0d9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:52.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.757+0000 7f0db3fff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 0x7f0d9806d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0db4156b40 tx=0x7f0da0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:52.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:52.762+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0da8055e60 con 0x7f0db4071230 2023-06-09T11:27:53.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.044+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0db40c8170 con 0x7f0db4071230 2023-06-09T11:27:53.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.045+0000 7f0db17fa700 1 -- 172.21.15.7:0/1940312983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f0db40c8170 con 0x7f0db4071230 2023-06-09T11:27:53.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:53.051 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:51.348223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:resolve","state_seq":35,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.047+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 msgr2=0x7f0d9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.047+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 0x7f0d9806d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0db4156b40 tx=0x7f0da0009040 comp rx=0 tx=0).stop 2023-06-09T11:27:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.047+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 msgr2=0x7f0db41f2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:53.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.047+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0da80090b0 tx=0x7f0da8004750 comp rx=0 tx=0).stop 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 shutdown_connections 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0d9806ace0 0x7f0d9806d190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0db415b380 0x7f0db40c1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0db4071230 0x7f0db41f2420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 --2- 172.21.15.7:0/1940312983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0db4151510 0x7f0db40c0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 >> 172.21.15.7:0/1940312983 conn(0x7f0db406a4d0 msgr2=0x7f0db4158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:53.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 shutdown_connections 2023-06-09T11:27:53.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:53.048+0000 7f0dbafff700 1 -- 172.21.15.7:0/1940312983 wait complete. 2023-06-09T11:27:53.056 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2023-06-09T11:27:53.074 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:27:55.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:55.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.298+0000 7f986a5ff700 1 Processor -- start 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.298+0000 7f986a5ff700 1 -- start start 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f986a5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f98640c5d20 0x7f98640c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f986a5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9864166b00 0x7f9864166f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f986a5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9864167440 con 0x7f98640c60f0 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f986a5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9864169bc0 con 0x7f98640c5d20 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f986a5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9864169ea0 con 0x7f9864166b00 2023-06-09T11:27:55.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f98637fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f98640c60f0 0x7f98640cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56420/0 (socket says 172.21.15.7:56420) 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f98637fe700 1 -- 172.21.15.7:0/3751830820 learned_addr learned my addr 172.21.15.7:0/3751830820 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f9863fff700 1 --2- 172.21.15.7:0/3751830820 >> v2:172.21.15.7:3300/0 conn(0x7f98640c5d20 0x7f98640c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.301+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720678365 0 0) 0x7f9864167440 con 0x7f98640c60f0 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.609+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c0057c0 con 0x7f98640c60f0 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.300+0000 7f9868b9c700 1 --2- 172.21.15.7:0/3751830820 >> v2:172.21.15.117:3300/0 conn(0x7f9864166b00 0x7f9864166f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9863fff700 1 -- 172.21.15.7:0/3751830820 >> v2:172.21.15.117:3300/0 conn(0x7f9864166b00 msgr2=0x7f9864166f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:55.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9863fff700 1 --2- 172.21.15.7:0/3751830820 >> v2:172.21.15.117:3300/0 conn(0x7f9864166b00 0x7f9864166f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9863fff700 1 -- 172.21.15.7:0/3751830820 >> v1:172.21.15.7:6789/0 conn(0x7f98640c60f0 legacy=0x7f98640cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9863fff700 1 -- 172.21.15.7:0/3751830820 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986416a180 con 0x7f98640c5d20 2023-06-09T11:27:55.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3699214766 0 0) 0x7f984c0057c0 con 0x7f98640c60f0 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9863fff700 1 --2- 172.21.15.7:0/3751830820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98640c5d20 0x7f98640c74e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f98640c02f0 tx=0x7f9854008200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb27ba2dedc52f0e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.610+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858001670 con 0x7f98640c5d20 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.611+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9854004030 con 0x7f98640c5d20 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.611+0000 7f9862ffd700 1 -- 172.21.15.7:0/3751830820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540043c0 con 0x7f98640c5d20 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 -- 172.21.15.7:0/3751830820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98640c5d20 msgr2=0x7f98640c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 --2- 172.21.15.7:0/3751830820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98640c5d20 0x7f98640c74e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f98640c02f0 tx=0x7f9854008200 comp rx=0 tx=0).stop 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 -- 172.21.15.7:0/3751830820 shutdown_connections 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 --2- 172.21.15.7:0/3751830820 >> v2:172.21.15.117:3300/0 conn(0x7f9864166b00 0x7f9864166f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 --2- 172.21.15.7:0/3751830820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f98640c5d20 0x7f98640c74e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 -- 172.21.15.7:0/3751830820 >> 172.21.15.7:0/3751830820 conn(0x7f986406a4d0 msgr2=0x7f986406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 -- 172.21.15.7:0/3751830820 shutdown_connections 2023-06-09T11:27:55.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.612+0000 7f986a5ff700 1 -- 172.21.15.7:0/3751830820 wait complete. 2023-06-09T11:27:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.615+0000 7f986a5ff700 1 Processor -- start 2023-06-09T11:27:55.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.615+0000 7f986a5ff700 1 -- start start 2023-06-09T11:27:55.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.616+0000 7f986a5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640c5d20 0x7f98640ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.616+0000 7f986a5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640c60f0 0x7f9864165040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f986a5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f986a5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986415feb0 con 0x7f9864166b00 2023-06-09T11:27:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f986a5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640cb670 con 0x7f98640c60f0 2023-06-09T11:27:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f986a5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98641fb610 con 0x7f98640c5d20 2023-06-09T11:27:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f9868b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f9868b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48454/0 (socket says 172.21.15.7:48454) 2023-06-09T11:27:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f9868b9c700 1 -- 172.21.15.7:0/2696589149 learned_addr learned my addr 172.21.15.7:0/2696589149 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f9863fff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640c5d20 0x7f98640ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.617+0000 7f98637fe700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640c60f0 0x7f9864165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 -- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640c5d20 msgr2=0x7f98640ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640c5d20 0x7f98640ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 -- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640c60f0 msgr2=0x7f9864165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640c60f0 0x7f9864165040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 -- 172.21.15.7:0/2696589149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9854008520 con 0x7f9864166b00 2023-06-09T11:27:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.618+0000 7f9868b9c700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f985c007e90 tx=0x7f985c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.619+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00ed30 con 0x7f9864166b00 2023-06-09T11:27:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.619+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00ee90 con 0x7f9864166b00 2023-06-09T11:27:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.619+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98640cb820 con 0x7f9864166b00 2023-06-09T11:27:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.619+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c014660 con 0x7f9864166b00 2023-06-09T11:27:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.619+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986416a460 con 0x7f9864166b00 2023-06-09T11:27:55.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.621+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98640cb9b0 con 0x7f9864166b00 2023-06-09T11:27:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.622+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f985c0147c0 con 0x7f9864166b00 2023-06-09T11:27:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.624+0000 7f9860ff9700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 0x7f985006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.624+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f985c089e10 con 0x7f9864166b00 2023-06-09T11:27:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.624+0000 7f9863fff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 0x7f985006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.625+0000 7f9863fff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 0x7f985006d1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9854015040 tx=0x7f985400a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.631+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f985c08d030 con 0x7f9864166b00 2023-06-09T11:27:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.959+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98641fb750 con 0x7f9864166b00 2023-06-09T11:27:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.960+0000 7f9860ff9700 1 -- 172.21.15.7:0/2696589149 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f985c054b90 con 0x7f9864166b00 2023-06-09T11:27:55.966 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:55.966 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":74,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:51.348223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:resolve","state_seq":35,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 msgr2=0x7f985006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 0x7f985006d1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9854015040 tx=0x7f985400a000 comp rx=0 tx=0).stop 2023-06-09T11:27:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 msgr2=0x7f98641fbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f985c007e90 tx=0x7f985c00e3c0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 shutdown_connections 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9864166b00 0x7f98641fbaa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f985006ad10 0x7f985006d1c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f98640c5d20 0x7f98640ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 --2- 172.21.15.7:0/2696589149 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f98640c60f0 0x7f9864165040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 >> 172.21.15.7:0/2696589149 conn(0x7f986406a4d0 msgr2=0x7f98640c7360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 shutdown_connections 2023-06-09T11:27:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:55.963+0000 7f986a5ff700 1 -- 172.21.15.7:0/2696589149 wait complete. 2023-06-09T11:27:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 74 2023-06-09T11:27:55.992 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:27:57.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:27:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.231+0000 7efc403ff700 1 Processor -- start 2023-06-09T11:27:59.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.231+0000 7efc403ff700 1 -- start start 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc403ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc380c5d20 0x7efc380c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc403ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 0x7efc38166f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc403ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc38167440 con 0x7efc380c60f0 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc403ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc38169bc0 con 0x7efc380c5d20 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc403ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc38169ea0 con 0x7efc38166b00 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc3d99a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efc380c60f0 0x7efc380cb110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56432/0 (socket says 172.21.15.7:56432) 2023-06-09T11:27:59.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc3e19b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc380c5d20 0x7efc380c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc3d99a700 1 -- 172.21.15.7:0/1197624876 learned_addr learned my addr 172.21.15.7:0/1197624876 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.234+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122200704 0 0) 0x7efc38167440 con 0x7efc380c60f0 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:58.233+0000 7efc3e99c700 1 --2- 172.21.15.7:0/1197624876 >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 0x7efc38166f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.150+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc24003590 con 0x7efc380c60f0 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e19b700 1 -- 172.21.15.7:0/1197624876 >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 msgr2=0x7efc38166f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e19b700 1 --2- 172.21.15.7:0/1197624876 >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 0x7efc38166f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255903692 0 0) 0x7efc24003590 con 0x7efc380c60f0 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e19b700 1 -- 172.21.15.7:0/1197624876 >> v1:172.21.15.7:6789/0 conn(0x7efc380c60f0 legacy=0x7efc380cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e19b700 1 -- 172.21.15.7:0/1197624876 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3816a180 con 0x7efc380c5d20 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e99c700 1 --2- 172.21.15.7:0/1197624876 >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 0x7efc38166f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3e19b700 1 --2- 172.21.15.7:0/1197624876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380c5d20 0x7efc380c74e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efc380c02f0 tx=0x7efc3400b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20072fc16f8434f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28001670 con 0x7efc380c5d20 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc34004030 con 0x7efc380c5d20 2023-06-09T11:27:59.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.151+0000 7efc3d199700 1 -- 172.21.15.7:0/1197624876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc340043a0 con 0x7efc380c5d20 2023-06-09T11:27:59.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 -- 172.21.15.7:0/1197624876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380c5d20 msgr2=0x7efc380c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 --2- 172.21.15.7:0/1197624876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380c5d20 0x7efc380c74e0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efc380c02f0 tx=0x7efc3400b710 comp rx=0 tx=0).stop 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 -- 172.21.15.7:0/1197624876 shutdown_connections 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 --2- 172.21.15.7:0/1197624876 >> v2:172.21.15.117:3300/0 conn(0x7efc38166b00 0x7efc38166f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 --2- 172.21.15.7:0/1197624876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380c5d20 0x7efc380c74e0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 -- 172.21.15.7:0/1197624876 >> 172.21.15.7:0/1197624876 conn(0x7efc3806a4d0 msgr2=0x7efc3806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 -- 172.21.15.7:0/1197624876 shutdown_connections 2023-06-09T11:27:59.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.152+0000 7efc403ff700 1 -- 172.21.15.7:0/1197624876 wait complete. 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.154+0000 7efc403ff700 1 Processor -- start 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.155+0000 7efc403ff700 1 -- start start 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c5d20 0x7efc38168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c60f0 0x7efc380c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381fcae0 con 0x7efc38166b00 2023-06-09T11:27:59.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380caff0 con 0x7efc380c60f0 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc403ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380ca3e0 con 0x7efc380c5d20 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48478/0 (socket says 172.21.15.7:48478) 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 -- 172.21.15.7:0/1691263800 learned_addr learned my addr 172.21.15.7:0/1691263800 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3d99a700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c60f0 0x7efc380c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e19b700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c5d20 0x7efc38168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 -- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c5d20 msgr2=0x7efc38168b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c5d20 0x7efc38168b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 -- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c60f0 msgr2=0x7efc380c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.156+0000 7efc3e99c700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c60f0 0x7efc380c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc3e99c700 1 -- 172.21.15.7:0/1691263800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc34014c10 con 0x7efc38166b00 2023-06-09T11:27:59.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc3e99c700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efc3000ec10 tx=0x7efc3000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30003c20 con 0x7efc38166b00 2023-06-09T11:27:59.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc30003d80 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000caa0 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc380cb820 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.157+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3816a460 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.158+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc380cb9b0 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.159+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc3000cc00 con 0x7efc38166b00 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.161+0000 7efc2effd700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 0x7efc1c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:27:59.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.161+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7efc3008a3e0 con 0x7efc38166b00 2023-06-09T11:27:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.161+0000 7efc3e19b700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 0x7efc1c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:27:59.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.162+0000 7efc3e19b700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 0x7efc1c06d1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc340090b0 tx=0x7efc34018000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:27:59.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.166+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc300550b0 con 0x7efc38166b00 2023-06-09T11:27:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.437+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc380c90a0 con 0x7efc38166b00 2023-06-09T11:27:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.437+0000 7efc2effd700 1 -- 172.21.15.7:0/1691263800 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5459 (secure 0 0 0) 0x7efc380c90a0 con 0x7efc38166b00 2023-06-09T11:27:59.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:27:59.444 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:57.350653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:27:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 msgr2=0x7efc1c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 0x7efc1c06d1c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efc340090b0 tx=0x7efc34018000 comp rx=0 tx=0).stop 2023-06-09T11:27:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 msgr2=0x7efc380c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:27:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7efc3000ec10 tx=0x7efc3000ef20 comp rx=0 tx=0).stop 2023-06-09T11:27:59.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 shutdown_connections 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc38166b00 0x7efc380c98c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1c06ad10 0x7efc1c06d1c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c5d20 0x7efc38168b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 --2- 172.21.15.7:0/1691263800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c60f0 0x7efc380c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 >> 172.21.15.7:0/1691263800 conn(0x7efc3806a4d0 msgr2=0x7efc38164830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 shutdown_connections 2023-06-09T11:27:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:27:59.440+0000 7efc403ff700 1 -- 172.21.15.7:0/1691263800 wait complete. 2023-06-09T11:27:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 75 2023-06-09T11:27:59.465 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 75, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 75, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5587": { "addr": "172.21.15.117:6833/2112619043", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2112619043, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2112619043, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5587, "incarnation": 61, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 36 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:reconnect", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 58, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:27:57.350653+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5587, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:28:01.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.693+0000 7f02467ff700 1 Processor -- start 2023-06-09T11:28:01.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.693+0000 7f02467ff700 1 -- start start 2023-06-09T11:28:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f02467ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f024015b4a0 0x7f024015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f02467ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f02400c9f40 0x7f02400c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f02467ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024015a500 con 0x7f0240153880 2023-06-09T11:28:01.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f02467ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02400c5f10 con 0x7f02400c9f40 2023-06-09T11:28:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f02467ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024015da00 con 0x7f024015b4a0 2023-06-09T11:28:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f023ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0240153880 0x7f024015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56436/0 (socket says 172.21.15.7:56436) 2023-06-09T11:28:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f023ffff700 1 -- 172.21.15.7:0/1979265672 learned_addr learned my addr 172.21.15.7:0/1979265672 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:28:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f0244d9c700 1 --2- 172.21.15.7:0/1979265672 >> v2:172.21.15.7:3300/0 conn(0x7f02400c9f40 0x7f02400c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.695+0000 7f023f7fe700 1 --2- 172.21.15.7:0/1979265672 >> v2:172.21.15.117:3300/0 conn(0x7f024015b4a0 0x7f024015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f0244d9c700 1 -- 172.21.15.7:0/1979265672 >> v2:172.21.15.117:3300/0 conn(0x7f024015b4a0 msgr2=0x7f024015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f0244d9c700 1 --2- 172.21.15.7:0/1979265672 >> v2:172.21.15.117:3300/0 conn(0x7f024015b4a0 0x7f024015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:01.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f0244d9c700 1 -- 172.21.15.7:0/1979265672 >> v1:172.21.15.7:6789/0 conn(0x7f0240153880 legacy=0x7f024015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f0244d9c700 1 -- 172.21.15.7:0/1979265672 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02401596e0 con 0x7f02400c9f40 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f0244d9c700 1 --2- 172.21.15.7:0/1979265672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02400c9f40 0x7f02400c4e30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f02400cce30 tx=0x7f023800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badcffef1adc5699 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f023effd700 1 -- 172.21.15.7:0/1979265672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023800e070 con 0x7f02400c9f40 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f023effd700 1 -- 172.21.15.7:0/1979265672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023800bb20 con 0x7f02400c9f40 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.696+0000 7f023effd700 1 -- 172.21.15.7:0/1979265672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238012430 con 0x7f02400c9f40 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 -- 172.21.15.7:0/1979265672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02400c9f40 msgr2=0x7f02400c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:01.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 --2- 172.21.15.7:0/1979265672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02400c9f40 0x7f02400c4e30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f02400cce30 tx=0x7f023800b330 comp rx=0 tx=0).stop 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 -- 172.21.15.7:0/1979265672 shutdown_connections 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 --2- 172.21.15.7:0/1979265672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02400c9f40 0x7f02400c4e30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 --2- 172.21.15.7:0/1979265672 >> v2:172.21.15.117:3300/0 conn(0x7f024015b4a0 0x7f024015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 -- 172.21.15.7:0/1979265672 >> 172.21.15.7:0/1979265672 conn(0x7f02400737f0 msgr2=0x7f02400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 -- 172.21.15.7:0/1979265672 shutdown_connections 2023-06-09T11:28:01.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.697+0000 7f02467ff700 1 -- 172.21.15.7:0/1979265672 wait complete. 2023-06-09T11:28:01.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.699+0000 7f02467ff700 1 Processor -- start 2023-06-09T11:28:01.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.700+0000 7f02467ff700 1 -- start start 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02400c9f40 0x7f02401519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0240153880 0x7f02400caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02400ced00 con 0x7f024015b4a0 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0240151540 con 0x7f0240153880 2023-06-09T11:28:01.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f02467ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02400cd970 con 0x7f02400c9f40 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f0244d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f0244d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48502/0 (socket says 172.21.15.7:48502) 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f0244d9c700 1 -- 172.21.15.7:0/2544063956 learned_addr learned my addr 172.21.15.7:0/2544063956 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f023f7fe700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0240153880 0x7f02400caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.701+0000 7f023ffff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02400c9f40 0x7f02401519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 -- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02400c9f40 msgr2=0x7f02401519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02400c9f40 0x7f02401519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 -- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0240153880 msgr2=0x7f02400caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0240153880 0x7f02400caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 -- 172.21.15.7:0/2544063956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238009a70 con 0x7f024015b4a0 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f0244d9c700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0238014040 tx=0x7f0238004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238012430 con 0x7f024015b4a0 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0238020070 con 0x7f024015b4a0 2023-06-09T11:28:01.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380098f0 con 0x7f024015b4a0 2023-06-09T11:28:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.702+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0240159c60 con 0x7f024015b4a0 2023-06-09T11:28:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.703+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0240159980 con 0x7f024015b4a0 2023-06-09T11:28:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.704+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0240159f40 con 0x7f024015b4a0 2023-06-09T11:28:01.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.705+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0238004040 con 0x7f024015b4a0 2023-06-09T11:28:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.706+0000 7f023cff9700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 0x7f022c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.706+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f0238022070 con 0x7f024015b4a0 2023-06-09T11:28:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.706+0000 7f023ffff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 0x7f022c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:01.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.707+0000 7f023ffff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 0x7f022c06d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0230005910 tx=0x7f0230009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:01.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:01.712+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0238056130 con 0x7f024015b4a0 2023-06-09T11:28:02.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.019+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f024014e350 con 0x7f024015b4a0 2023-06-09T11:28:02.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.020+0000 7f023cff9700 1 -- 172.21.15.7:0/2544063956 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f0238059750 con 0x7f024015b4a0 2023-06-09T11:28:02.026 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:02.026 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:27:57.350653+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:reconnect","state_seq":37,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:02.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.022+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 msgr2=0x7f022c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.022+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 0x7f022c06d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0230005910 tx=0x7f0230009040 comp rx=0 tx=0).stop 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 msgr2=0x7f024006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0238014040 tx=0x7f0238004ad0 comp rx=0 tx=0).stop 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 shutdown_connections 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f024015b4a0 0x7f024006a740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f022c06ace0 0x7f022c06d190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:02.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f02400c9f40 0x7f02401519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 --2- 172.21.15.7:0/2544063956 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0240153880 0x7f02400caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 >> 172.21.15.7:0/2544063956 conn(0x7f02400737f0 msgr2=0x7f024014d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 shutdown_connections 2023-06-09T11:28:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:02.023+0000 7f02467ff700 1 -- 172.21.15.7:0/2544063956 wait complete. 2023-06-09T11:28:02.030 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 75 2023-06-09T11:28:02.051 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:28:04.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.269+0000 7f53525ff700 1 Processor -- start 2023-06-09T11:28:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.269+0000 7f53525ff700 1 -- start start 2023-06-09T11:28:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.270+0000 7f53525ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f534c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f53525ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534c15a390 con 0x7f534c153760 2023-06-09T11:28:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f53525ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534c0ca720 con 0x7f534c071fa0 2023-06-09T11:28:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f53525ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c0c5f40 con 0x7f534c153b30 2023-06-09T11:28:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f534bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f534c153760 0x7f534c0b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40372/0 (socket says 172.21.15.7:40372) 2023-06-09T11:28:04.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f534bfff700 1 -- 172.21.15.7:0/2368049704 learned_addr learned my addr 172.21.15.7:0/2368049704 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:28:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f534b7fe700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f534c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1209430490 0 0) 0x7f534c15a390 con 0x7f534c153760 2023-06-09T11:28:04.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.271+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53340035b0 con 0x7f534c153760 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782738459 0 0) 0x7f534c0ca720 con 0x7f534c071fa0 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f534c15a390 con 0x7f534c071fa0 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 17325441 0 0) 0x7f53340035b0 con 0x7f534c153760 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 msgr2=0x7f534c0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f534c0ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 >> v1:172.21.15.117:6789/0 conn(0x7f534c071fa0 legacy=0x7f534c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c0c5a20 con 0x7f534c153760 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f533c001670 con 0x7f534c153760 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534b7fe700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f534c0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 >> v1:172.21.15.7:6789/0 conn(0x7f534c153760 legacy=0x7f534c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f5334003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004100 0x7f5334004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5334004d90 0x7f5334005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c0c5f40 con 0x7f5334004d90 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533400bac0 con 0x7f5334004100 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533400bc80 con 0x7f534c153b30 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f5350b9c700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5334004d90 0x7f5334005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534bfff700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f5334003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.272+0000 7f534b7fe700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004100 0x7f5334004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 -- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 msgr2=0x7f5334003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c153b30 0x7f5334003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 -- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004100 msgr2=0x7f5334004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004100 0x7f5334004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 -- 172.21.15.7:0/2368049704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400094a0 con 0x7f5334004d90 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f534b7fe700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004100 0x7f5334004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f5350b9c700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5334004d90 0x7f5334005200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5344002230 tx=0x7f5334005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a1879471123c689 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344001f30 con 0x7f5334004d90 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534400ce80 con 0x7f5334004d90 2023-06-09T11:28:04.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.273+0000 7f534affd700 1 -- 172.21.15.7:0/2368049704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344003c40 con 0x7f5334004d90 2023-06-09T11:28:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 -- 172.21.15.7:0/2368049704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5334004d90 msgr2=0x7f5334005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 --2- 172.21.15.7:0/2368049704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5334004d90 0x7f5334005200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5344002230 tx=0x7f5334005880 comp rx=0 tx=0).stop 2023-06-09T11:28:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f5350b9c700 1 -- 172.21.15.7:0/2368049704 reap_dead start 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 -- 172.21.15.7:0/2368049704 shutdown_connections 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 -- 172.21.15.7:0/2368049704 >> 172.21.15.7:0/2368049704 conn(0x7f534c0737f0 msgr2=0x7f534c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 -- 172.21.15.7:0/2368049704 shutdown_connections 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.274+0000 7f53525ff700 1 -- 172.21.15.7:0/2368049704 wait complete. 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.276+0000 7f53525ff700 1 Processor -- start 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.276+0000 7f53525ff700 1 -- start start 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004d90 0x7f534c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c157830 0x7f534c158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c157cd0 con 0x7f534c1f2380 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1eefb0 con 0x7f5334004d90 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.277+0000 7f53525ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534c1ef290 con 0x7f534c157830 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58226/0 (socket says 172.21.15.7:58226) 2023-06-09T11:28:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 -- 172.21.15.7:0/115024426 learned_addr learned my addr 172.21.15.7:0/115024426 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f534b7fe700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c157830 0x7f534c158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f534bfff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004d90 0x7f534c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 -- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c157830 msgr2=0x7f534c158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c157830 0x7f534c158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 -- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004d90 msgr2=0x7f534c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004d90 0x7f534c0ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 -- 172.21.15.7:0/115024426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c159a60 con 0x7f534c1f2380 2023-06-09T11:28:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.278+0000 7f5350b9c700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5344004000 tx=0x7f5344001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.279+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344003c10 con 0x7f534c1f2380 2023-06-09T11:28:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.279+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344003d70 con 0x7f534c1f2380 2023-06-09T11:28:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.279+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344005310 con 0x7f534c1f2380 2023-06-09T11:28:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.279+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c159780 con 0x7f534c1f2380 2023-06-09T11:28:04.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.279+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c159d40 con 0x7f534c1f2380 2023-06-09T11:28:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.280+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c157e20 con 0x7f534c1f2380 2023-06-09T11:28:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.281+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5344019070 con 0x7f534c1f2380 2023-06-09T11:28:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.282+0000 7f5348ff9700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 0x7f533806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.282+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f534408a160 con 0x7f534c1f2380 2023-06-09T11:28:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.282+0000 7f534bfff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 0x7f533806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.283+0000 7f534bfff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 0x7f533806d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f533c001c20 tx=0x7f533c001e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.289+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5344055130 con 0x7f534c1f2380 2023-06-09T11:28:04.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.600+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f534c074cf0 con 0x7f534c1f2380 2023-06-09T11:28:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.600+0000 7f5348ff9700 1 -- 172.21.15.7:0/115024426 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f5344022090 con 0x7f534c1f2380 2023-06-09T11:28:04.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:04.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:03.352966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:04.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.602+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 msgr2=0x7f533806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.602+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 0x7f533806d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f533c001c20 tx=0x7f533c001e00 comp rx=0 tx=0).stop 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.602+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 msgr2=0x7f534c158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.602+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5344004000 tx=0x7f5344001650 comp rx=0 tx=0).stop 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 shutdown_connections 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f534c1f2380 0x7f534c158640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5334004d90 0x7f534c0ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f533806ace0 0x7f533806d190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 --2- 172.21.15.7:0/115024426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f534c157830 0x7f534c158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 >> 172.21.15.7:0/115024426 conn(0x7f534c0737f0 msgr2=0x7f534c0662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 shutdown_connections 2023-06-09T11:28:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:04.603+0000 7f53525ff700 1 -- 172.21.15.7:0/115024426 wait complete. 2023-06-09T11:28:04.610 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2023-06-09T11:28:04.627 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:28:06.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:06.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.863+0000 7f9b2cdff700 1 Processor -- start 2023-06-09T11:28:06.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.863+0000 7f9b2cdff700 1 -- start start 2023-06-09T11:28:06.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2cdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9b28153880 0x7f9b2815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2cdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9b280c9f40 0x7f9b2815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2815a500 con 0x7f9b2815b2d0 2023-06-09T11:28:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2cdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b280c5f10 con 0x7f9b280c9f40 2023-06-09T11:28:06.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2cdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2815da00 con 0x7f9b28153880 2023-06-09T11:28:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b25d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9b280c9f40 0x7f9b2815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b25d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9b280c9f40 0x7f9b2815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58232/0 (socket says 172.21.15.7:58232) 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b25d9b700 1 -- 172.21.15.7:0/2071631421 learned_addr learned my addr 172.21.15.7:0/2071631421 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.865+0000 7f9b2659c700 1 --2- 172.21.15.7:0/2071631421 >> v2:172.21.15.117:3300/0 conn(0x7f9b28153880 0x7f9b2815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b25d9b700 1 -- 172.21.15.7:0/2071631421 >> v2:172.21.15.117:3300/0 conn(0x7f9b28153880 msgr2=0x7f9b2815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b25d9b700 1 --2- 172.21.15.7:0/2071631421 >> v2:172.21.15.117:3300/0 conn(0x7f9b28153880 0x7f9b2815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b25d9b700 1 -- 172.21.15.7:0/2071631421 >> v1:172.21.15.117:6789/0 conn(0x7f9b2815b2d0 legacy=0x7f9b280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b25d9b700 1 -- 172.21.15.7:0/2071631421 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b281596e0 con 0x7f9b280c9f40 2023-06-09T11:28:06.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b25d9b700 1 --2- 172.21.15.7:0/2071631421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b280c9f40 0x7f9b2815ad90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9b2808d340 tx=0x7f9b1c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d52ca10f759233 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b2559a700 1 -- 172.21.15.7:0/2071631421 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c011830 con 0x7f9b280c9f40 2023-06-09T11:28:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b2559a700 1 -- 172.21.15.7:0/2071631421 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c011990 con 0x7f9b280c9f40 2023-06-09T11:28:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.866+0000 7f9b2559a700 1 -- 172.21.15.7:0/2071631421 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c011d20 con 0x7f9b280c9f40 2023-06-09T11:28:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 -- 172.21.15.7:0/2071631421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b280c9f40 msgr2=0x7f9b2815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:06.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/2071631421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b280c9f40 0x7f9b2815ad90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9b2808d340 tx=0x7f9b1c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 -- 172.21.15.7:0/2071631421 shutdown_connections 2023-06-09T11:28:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/2071631421 >> v2:172.21.15.117:3300/0 conn(0x7f9b28153880 0x7f9b2815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/2071631421 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b280c9f40 0x7f9b2815ad90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 -- 172.21.15.7:0/2071631421 >> 172.21.15.7:0/2071631421 conn(0x7f9b280737f0 msgr2=0x7f9b280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:06.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 -- 172.21.15.7:0/2071631421 shutdown_connections 2023-06-09T11:28:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.867+0000 7f9b2cdff700 1 -- 172.21.15.7:0/2071631421 wait complete. 2023-06-09T11:28:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.869+0000 7f9b2cdff700 1 Processor -- start 2023-06-09T11:28:06.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.869+0000 7f9b2cdff700 1 -- start start 2023-06-09T11:28:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b280c9f40 0x7f9b281efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 0x7f9b281f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b281effc0 con 0x7f9b28153880 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280b3640 con 0x7f9b2815b2d0 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b2cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b280b3970 con 0x7f9b280c9f40 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b25d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b25d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58244/0 (socket says 172.21.15.7:58244) 2023-06-09T11:28:06.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.870+0000 7f9b25d9b700 1 -- 172.21.15.7:0/48105747 learned_addr learned my addr 172.21.15.7:0/48105747 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b26d9d700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 0x7f9b281f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b2659c700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b280c9f40 0x7f9b281efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 -- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b280c9f40 msgr2=0x7f9b281efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b280c9f40 0x7f9b281efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 -- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 msgr2=0x7f9b281f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 0x7f9b281f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 -- 172.21.15.7:0/48105747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c009550 con 0x7f9b28153880 2023-06-09T11:28:06.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b26d9d700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 0x7f9b281f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b25d9b700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c000c00 tx=0x7f9b1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.871+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c004570 con 0x7f9b28153880 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.872+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c01f070 con 0x7f9b28153880 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.872+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b28159c60 con 0x7f9b28153880 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.872+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c0038e0 con 0x7f9b28153880 2023-06-09T11:28:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.872+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b28159980 con 0x7f9b28153880 2023-06-09T11:28:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.873+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b28159f40 con 0x7f9b28153880 2023-06-09T11:28:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.874+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b1c021030 con 0x7f9b28153880 2023-06-09T11:28:06.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.876+0000 7f9b16ffd700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 0x7f9b0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.876+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f9b1c08ac60 con 0x7f9b28153880 2023-06-09T11:28:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.876+0000 7f9b2659c700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 0x7f9b0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:06.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.877+0000 7f9b2659c700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 0x7f9b0806d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009b70 tx=0x7f9b18008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:06.881+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9b1c0559c0 con 0x7f9b28153880 2023-06-09T11:28:07.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.154+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b28074cf0 con 0x7f9b28153880 2023-06-09T11:28:07.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.154+0000 7f9b16ffd700 1 -- 172.21.15.7:0/48105747 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f9b1c015090 con 0x7f9b28153880 2023-06-09T11:28:07.161 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:07.161 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:03.352966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:rejoin","state_seq":39,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 msgr2=0x7f9b0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 0x7f9b0806d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9b18009b70 tx=0x7f9b18008040 comp rx=0 tx=0).stop 2023-06-09T11:28:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 msgr2=0x7f9b28152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c000c00 tx=0x7f9b1c004750 comp rx=0 tx=0).stop 2023-06-09T11:28:07.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 shutdown_connections 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9b2815b2d0 0x7f9b281f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9b0806ace0 0x7f9b0806d190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9b280c9f40 0x7f9b281efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 --2- 172.21.15.7:0/48105747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9b28153880 0x7f9b28152350 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 >> 172.21.15.7:0/48105747 conn(0x7f9b280737f0 msgr2=0x7f9b2806ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 shutdown_connections 2023-06-09T11:28:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:07.157+0000 7f9b2cdff700 1 -- 172.21.15.7:0/48105747 wait complete. 2023-06-09T11:28:07.165 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2023-06-09T11:28:07.184 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:28:09.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:09.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.418+0000 7fa8affff700 1 Processor -- start 2023-06-09T11:28:09.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.419+0000 7fa8affff700 1 -- start start 2023-06-09T11:28:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8affff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 0x7fa8a81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8affff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80c0bf0 con 0x7fa8a80c1bc0 2023-06-09T11:28:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8affff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a80702f0 con 0x7fa8a80c0140 2023-06-09T11:28:09.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8affff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8070110 con 0x7fa8a815a0c0 2023-06-09T11:28:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8add9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8a80c1bc0 0x7fa8a81599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40374/0 (socket says 172.21.15.7:40374) 2023-06-09T11:28:09.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8add9b700 1 -- 172.21.15.7:0/1380899778 learned_addr learned my addr 172.21.15.7:0/1380899778 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:28:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.420+0000 7fa8ad59a700 1 --2- 172.21.15.7:0/1380899778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 0x7fa8a81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977099427 0 0) 0x7fa8a80c0bf0 con 0x7fa8a80c1bc0 2023-06-09T11:28:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8940057c0 con 0x7fa8a80c1bc0 2023-06-09T11:28:09.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300247950 0 0) 0x7fa8a80702f0 con 0x7fa8a80c0140 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a80c0bf0 con 0x7fa8a80c0140 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8ad59a700 1 -- 172.21.15.7:0/1380899778 >> v1:172.21.15.117:6789/0 conn(0x7fa8a80c0140 legacy=0x7fa8a8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8ad59a700 1 -- 172.21.15.7:0/1380899778 >> v1:172.21.15.7:6789/0 conn(0x7fa8a80c1bc0 legacy=0x7fa8a81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8ad59a700 1 -- 172.21.15.7:0/1380899778 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a80c0e90 con 0x7fa8a815a0c0 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.421+0000 7fa8ad59a700 1 --2- 172.21.15.7:0/1380899778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 0x7fa8a81501b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa898009870 tx=0x7fa898009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb30a3c7e015fe9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898011830 con 0x7fa8a815a0c0 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa898011990 con 0x7fa8a815a0c0 2023-06-09T11:28:09.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8acd99700 1 -- 172.21.15.7:0/1380899778 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa898011d20 con 0x7fa8a815a0c0 2023-06-09T11:28:09.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 -- 172.21.15.7:0/1380899778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 msgr2=0x7fa8a81501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 --2- 172.21.15.7:0/1380899778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 0x7fa8a81501b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa898009870 tx=0x7fa898009b80 comp rx=0 tx=0).stop 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 -- 172.21.15.7:0/1380899778 shutdown_connections 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 --2- 172.21.15.7:0/1380899778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a815a0c0 0x7fa8a81501b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 -- 172.21.15.7:0/1380899778 >> 172.21.15.7:0/1380899778 conn(0x7fa8a806a4d0 msgr2=0x7fa8a806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 -- 172.21.15.7:0/1380899778 shutdown_connections 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.422+0000 7fa8affff700 1 -- 172.21.15.7:0/1380899778 wait complete. 2023-06-09T11:28:09.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.425+0000 7fa8affff700 1 Processor -- start 2023-06-09T11:28:09.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.425+0000 7fa8affff700 1 -- start start 2023-06-09T11:28:09.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8a80c0140 0x7fa8a80c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a80c1bc0 0x7fa8a81575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80c40d0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a81ef860 con 0x7fa8a80c0140 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.426+0000 7fa8affff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a80cac80 con 0x7fa8a80c1bc0 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:09.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58250/0 (socket says 172.21.15.7:58250) 2023-06-09T11:28:09.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 -- 172.21.15.7:0/876009896 learned_addr learned my addr 172.21.15.7:0/876009896 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:09.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8add9b700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8a80c0140 0x7fa8a80c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ad59a700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a80c1bc0 0x7fa8a81575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 -- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a80c1bc0 msgr2=0x7fa8a81575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a80c1bc0 0x7fa8a81575a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 -- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8a80c0140 msgr2=0x7fa8a80c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8a80c0140 0x7fa8a80c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.427+0000 7fa8ae59c700 1 -- 172.21.15.7:0/876009896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980094a0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa8ae59c700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000cbf0 tx=0x7fa8a000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0003c00 con 0x7fa8a815a0c0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a0003d60 con 0x7fa8a815a0c0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000aa60 con 0x7fa8a815a0c0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a80c1450 con 0x7fa8a815a0c0 2023-06-09T11:28:09.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.428+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a80c1170 con 0x7fa8a815a0c0 2023-06-09T11:28:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.429+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a80705d0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.430+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa8a000abc0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.432+0000 7fa89e7fc700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 0x7fa88c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.432+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7fa8a0089790 con 0x7fa8a815a0c0 2023-06-09T11:28:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.432+0000 7fa8add9b700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 0x7fa88c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.433+0000 7fa8add9b700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 0x7fa88c06d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4007fd0 tx=0x7fa8a4007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:09.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.436+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa8a00544e0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.736+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8a8074cf0 con 0x7fa8a815a0c0 2023-06-09T11:28:09.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.737+0000 7fa89e7fc700 1 -- 172.21.15.7:0/876009896 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fa8a0057b00 con 0x7fa8a815a0c0 2023-06-09T11:28:09.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:09.743 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 msgr2=0x7fa88c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 0x7fa88c06d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4007fd0 tx=0x7fa8a4007dc0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 msgr2=0x7fa8a81efcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000cbf0 tx=0x7fa8a000cf00 comp rx=0 tx=0).stop 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 shutdown_connections 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa8a815a0c0 0x7fa8a81efcf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8a80c0140 0x7fa8a80c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa88c06ace0 0x7fa88c06d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 --2- 172.21.15.7:0/876009896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8a80c1bc0 0x7fa8a81575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.739+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 >> 172.21.15.7:0/876009896 conn(0x7fa8a806a4d0 msgr2=0x7fa8a80c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.740+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 shutdown_connections 2023-06-09T11:28:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:09.740+0000 7fa8affff700 1 -- 172.21.15.7:0/876009896 wait complete. 2023-06-09T11:28:09.747 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2023-06-09T11:28:09.765 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=2, replacing gid=5455 2023-06-09T11:28:09.765 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.d 2023-06-09T11:28:10.765 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:28:10.766 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:28:10.766 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:28:10.771 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:28:10.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:10.919 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:28:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.002+0000 7f39b3dff700 1 Processor -- start 2023-06-09T11:28:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.002+0000 7f39b3dff700 1 -- start start 2023-06-09T11:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b3dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f39ac0ca1b0 0x7f39ac15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b3dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39ac0b5360 0x7f39ac0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b3dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39ac15dcf0 con 0x7f39ac0ca580 2023-06-09T11:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b3dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac0c5e00 con 0x7f39ac0b5360 2023-06-09T11:28:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b3dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39ac15d450 con 0x7f39ac0ca1b0 2023-06-09T11:28:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39ac0b5360 0x7f39ac0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39ac0b5360 0x7f39ac0c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58264/0 (socket says 172.21.15.7:58264) 2023-06-09T11:28:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 -- 172.21.15.7:0/1156682567 learned_addr learned my addr 172.21.15.7:0/1156682567 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b1b9b700 1 --2- 172.21.15.7:0/1156682567 >> v2:172.21.15.117:3300/0 conn(0x7f39ac0ca1b0 0x7f39ac15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 -- 172.21.15.7:0/1156682567 >> v2:172.21.15.117:3300/0 conn(0x7f39ac0ca1b0 msgr2=0x7f39ac15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 --2- 172.21.15.7:0/1156682567 >> v2:172.21.15.117:3300/0 conn(0x7f39ac0ca1b0 0x7f39ac15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 -- 172.21.15.7:0/1156682567 >> v1:172.21.15.117:6789/0 conn(0x7f39ac0ca580 legacy=0x7f39ac1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.004+0000 7f39b139a700 1 -- 172.21.15.7:0/1156682567 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac15d730 con 0x7f39ac0b5360 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.005+0000 7f39b139a700 1 --2- 172.21.15.7:0/1156682567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac0c58c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06bb80 tx=0x7f399c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218bd7b241e73d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.005+0000 7f39b0b99700 1 -- 172.21.15.7:0/1156682567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c011830 con 0x7f39ac0b5360 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.005+0000 7f39b0b99700 1 -- 172.21.15.7:0/1156682567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f399c011990 con 0x7f39ac0b5360 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.005+0000 7f39b0b99700 1 -- 172.21.15.7:0/1156682567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f399c011d20 con 0x7f39ac0b5360 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 -- 172.21.15.7:0/1156682567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 msgr2=0x7f39ac0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 --2- 172.21.15.7:0/1156682567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac0c58c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f39ac06bb80 tx=0x7f399c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 -- 172.21.15.7:0/1156682567 shutdown_connections 2023-06-09T11:28:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 --2- 172.21.15.7:0/1156682567 >> v2:172.21.15.117:3300/0 conn(0x7f39ac0ca1b0 0x7f39ac15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 --2- 172.21.15.7:0/1156682567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac0c58c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 -- 172.21.15.7:0/1156682567 >> 172.21.15.7:0/1156682567 conn(0x7f39ac072090 msgr2=0x7f39ac072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 -- 172.21.15.7:0/1156682567 shutdown_connections 2023-06-09T11:28:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.006+0000 7f39b3dff700 1 -- 172.21.15.7:0/1156682567 wait complete. 2023-06-09T11:28:11.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.008+0000 7f39b3dff700 1 Processor -- start 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.008+0000 7f39b3dff700 1 -- start start 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39ac0ca1b0 0x7f39ac0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 0x7f39ac1f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac150f80 con 0x7f39ac0b5360 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac158c50 con 0x7f39ac0ca1b0 2023-06-09T11:28:11.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b3dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac0b4840 con 0x7f39ac0ca580 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b1b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b1b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58276/0 (socket says 172.21.15.7:58276) 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b1b9b700 1 -- 172.21.15.7:0/3014539980 learned_addr learned my addr 172.21.15.7:0/3014539980 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b239c700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 0x7f39ac1f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.009+0000 7f39b139a700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39ac0ca1b0 0x7f39ac0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 -- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 msgr2=0x7f39ac1f00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 0x7f39ac1f00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 -- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39ac0ca1b0 msgr2=0x7f39ac0b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39ac0ca1b0 0x7f39ac0b4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 -- 172.21.15.7:0/3014539980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c009550 con 0x7f39ac0b5360 2023-06-09T11:28:11.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b239c700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 0x7f39ac1f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b1b9b700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9b0 tx=0x7f39a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8009710 con 0x7f39ac0b5360 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a800c850 con 0x7f39ac0b5360 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800cbc0 con 0x7f39ac0b5360 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.010+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac06a570 con 0x7f39ac0b5360 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.011+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac15da10 con 0x7f39ac0b5360 2023-06-09T11:28:11.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.012+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac06a850 con 0x7f39ac0b5360 2023-06-09T11:28:11.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.013+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39a800cd20 con 0x7f39ac0b5360 2023-06-09T11:28:11.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.014+0000 7f39a27fc700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 0x7f399006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:11.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.014+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f39a8089980 con 0x7f39ac0b5360 2023-06-09T11:28:11.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.014+0000 7f39b139a700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 0x7f399006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:11.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.015+0000 7f39b139a700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 0x7f399006d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f399c009c10 tx=0x7f399c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:11.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.019+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f39a80545c0 con 0x7f39ac0b5360 2023-06-09T11:28:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.303+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39ac074cf0 con 0x7f39ac0b5360 2023-06-09T11:28:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.304+0000 7f39a27fc700 1 -- 172.21.15.7:0/3014539980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f39a8021090 con 0x7f39ac0b5360 2023-06-09T11:28:11.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:11.310 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 msgr2=0x7f399006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 0x7f399006d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f399c009c10 tx=0x7f399c008030 comp rx=0 tx=0).stop 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 msgr2=0x7f39ac1590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f39a800e9b0 tx=0x7f39a800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 shutdown_connections 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399006ace0 0x7f399006d190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39ac0ca580 0x7f39ac1f00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39ac0b5360 0x7f39ac1590e0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 --2- 172.21.15.7:0/3014539980 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39ac0ca1b0 0x7f39ac0b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 >> 172.21.15.7:0/3014539980 conn(0x7f39ac072090 msgr2=0x7f39ac151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 shutdown_connections 2023-06-09T11:28:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:11.307+0000 7f39b3dff700 1 -- 172.21.15.7:0/3014539980 wait complete. 2023-06-09T11:28:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 77 2023-06-09T11:28:11.333 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:28:13.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:28:13.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.571+0000 7f125c3ff700 1 Processor -- start 2023-06-09T11:28:13.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.571+0000 7f125c3ff700 1 -- start start 2023-06-09T11:28:13.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125c3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540cb9a0 0x7f12540b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125c3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f125415cd40 0x7f12540cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125c3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1254159b20 con 0x7f12540cb5d0 2023-06-09T11:28:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125c3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12540cac80 con 0x7f12540cb9a0 2023-06-09T11:28:13.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125c3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12541597d0 con 0x7f125415cd40 2023-06-09T11:28:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540cb9a0 0x7f12540b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125a99c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f125415cd40 0x7f12540cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f12540cb9a0 0x7f12540b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55970/0 (socket says 172.21.15.7:55970) 2023-06-09T11:28:13.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 -- 172.21.15.7:0/2113255664 learned_addr learned my addr 172.21.15.7:0/2113255664 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 -- 172.21.15.7:0/2113255664 >> v2:172.21.15.117:3300/0 conn(0x7f125415cd40 msgr2=0x7f12540cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 --2- 172.21.15.7:0/2113255664 >> v2:172.21.15.117:3300/0 conn(0x7f125415cd40 0x7f12540cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 -- 172.21.15.7:0/2113255664 >> v1:172.21.15.117:6789/0 conn(0x7f12540cb5d0 legacy=0x7f1254159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.573+0000 7f125999a700 1 -- 172.21.15.7:0/2113255664 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1254071ab0 con 0x7f12540cb9a0 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.574+0000 7f125999a700 1 --2- 172.21.15.7:0/2113255664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb9a0 0x7f12540b52b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f12540c9b80 tx=0x7f1244009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca39f5cb1d8b4d9a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.574+0000 7f1259199700 1 -- 172.21.15.7:0/2113255664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244011830 con 0x7f12540cb9a0 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.574+0000 7f1259199700 1 -- 172.21.15.7:0/2113255664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1244011990 con 0x7f12540cb9a0 2023-06-09T11:28:13.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.574+0000 7f1259199700 1 -- 172.21.15.7:0/2113255664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1244011d20 con 0x7f12540cb9a0 2023-06-09T11:28:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 -- 172.21.15.7:0/2113255664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb9a0 msgr2=0x7f12540b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 --2- 172.21.15.7:0/2113255664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb9a0 0x7f12540b52b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f12540c9b80 tx=0x7f1244009bd0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 -- 172.21.15.7:0/2113255664 shutdown_connections 2023-06-09T11:28:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 --2- 172.21.15.7:0/2113255664 >> v2:172.21.15.117:3300/0 conn(0x7f125415cd40 0x7f12540cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 --2- 172.21.15.7:0/2113255664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb9a0 0x7f12540b52b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 -- 172.21.15.7:0/2113255664 >> 172.21.15.7:0/2113255664 conn(0x7f1254070af0 msgr2=0x7f1254070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 -- 172.21.15.7:0/2113255664 shutdown_connections 2023-06-09T11:28:13.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.575+0000 7f125c3ff700 1 -- 172.21.15.7:0/2113255664 wait complete. 2023-06-09T11:28:13.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.577+0000 7f125c3ff700 1 Processor -- start 2023-06-09T11:28:13.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.577+0000 7f125c3ff700 1 -- start start 2023-06-09T11:28:13.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12540cb9a0 0x7f12540cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125415cd40 0x7f12540cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12540cf510 con 0x7f12540cb5d0 2023-06-09T11:28:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12541ecf60 con 0x7f125415cd40 2023-06-09T11:28:13.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.578+0000 7f125c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1254156bb0 con 0x7f12540cb9a0 2023-06-09T11:28:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55984/0 (socket says 172.21.15.7:55984) 2023-06-09T11:28:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 -- 172.21.15.7:0/3410121633 learned_addr learned my addr 172.21.15.7:0/3410121633 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:28:13.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a99c700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125415cd40 0x7f12540cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125999a700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12540cb9a0 0x7f12540cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 -- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12540cb9a0 msgr2=0x7f12540cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12540cb9a0 0x7f12540cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 -- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125415cd40 msgr2=0x7f12540cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125415cd40 0x7f12540cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 -- 172.21.15.7:0/3410121633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244009550 con 0x7f12540cb5d0 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.579+0000 7f125a19b700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1250001f00 tx=0x7f125000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:13.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.580+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250003c00 con 0x7f12540cb5d0 2023-06-09T11:28:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.580+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1250003d60 con 0x7f12540cb5d0 2023-06-09T11:28:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.580+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125415d4f0 con 0x7f12540cb5d0 2023-06-09T11:28:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.580+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125000ab70 con 0x7f12540cb5d0 2023-06-09T11:28:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.580+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125415d210 con 0x7f12540cb5d0 2023-06-09T11:28:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.581+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f125415d7d0 con 0x7f12540cb5d0 2023-06-09T11:28:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.582+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f125000acd0 con 0x7f12540cb5d0 2023-06-09T11:28:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.583+0000 7f124affd700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 0x7f123806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:28:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.583+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f1250089940 con 0x7f12540cb5d0 2023-06-09T11:28:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.583+0000 7f125999a700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 0x7f123806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:28:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.584+0000 7f125999a700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 0x7f123806d1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f12540cbd70 tx=0x7f1244008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:28:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.588+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1250054580 con 0x7f12540cb5d0 2023-06-09T11:28:13.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.900+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1254074cf0 con 0x7f12540cb5d0 2023-06-09T11:28:13.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.901+0000 7f124affd700 1 -- 172.21.15.7:0/3410121633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f1250057ba0 con 0x7f12540cb5d0 2023-06-09T11:28:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:28:13.907 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:28:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.903+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 msgr2=0x7f123806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.903+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 0x7f123806d1c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f12540cbd70 tx=0x7f1244008030 comp rx=0 tx=0).stop 2023-06-09T11:28:13.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.903+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 msgr2=0x7f12541efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:28:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.903+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f1250001f00 tx=0x7f125000a060 comp rx=0 tx=0).stop 2023-06-09T11:28:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 shutdown_connections 2023-06-09T11:28:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f125415cd40 0x7f12540cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12540cb5d0 0x7f12541efa00 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f123806ad10 0x7f123806d1c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 --2- 172.21.15.7:0/3410121633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f12540cb9a0 0x7f12540cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 >> 172.21.15.7:0/3410121633 conn(0x7f1254070af0 msgr2=0x7f1254158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 shutdown_connections 2023-06-09T11:28:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:28:13.904+0000 7f125c3ff700 1 -- 172.21.15.7:0/3410121633 wait complete. 2023-06-09T11:28:13.912 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:28:13.930 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:28:13.930 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 62 secs before thrashing 2023-06-09T11:28:15.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:15.328 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:45.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:28:45.500 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:15.624 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:15.631 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:15.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.126+0000 7f14959ff700 1 Processor -- start 2023-06-09T11:29:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.126+0000 7f14959ff700 1 -- start start 2023-06-09T11:29:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.127+0000 7f14959ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f149006b480 0x7f149015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.127+0000 7f14959ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149015dcf0 con 0x7f14900b5360 2023-06-09T11:29:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.127+0000 7f14959ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149015d450 con 0x7f14900ca1b0 2023-06-09T11:29:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.127+0000 7f14959ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149015d730 con 0x7f149006b480 2023-06-09T11:29:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f149006b480 0x7f149015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f14900b5360 0x7f149006ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34342/0 (socket says 172.21.15.7:34342) 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148e7fc700 1 -- 172.21.15.7:0/2826769976 learned_addr learned my addr 172.21.15.7:0/2826769976 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148dffb700 1 -- 172.21.15.7:0/2826769976 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895120115 0 0) 0x7f149015dcf0 con 0x7f14900b5360 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148f7fe700 1 -- 172.21.15.7:0/2826769976 >> v1:172.21.15.117:6789/0 conn(0x7f14900ca1b0 legacy=0x7f14901536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148f7fe700 1 -- 172.21.15.7:0/2826769976 >> v1:172.21.15.7:6789/0 conn(0x7f14900b5360 legacy=0x7f149006ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148f7fe700 1 -- 172.21.15.7:0/2826769976 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149015da10 con 0x7f149006b480 2023-06-09T11:29:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.128+0000 7f148f7fe700 1 --2- 172.21.15.7:0/2826769976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f149006b480 0x7f149015cdc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1488009d90 tx=0x7f148800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f2073edbae701f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f148dffb700 1 -- 172.21.15.7:0/2826769976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148800e070 con 0x7f149006b480 2023-06-09T11:29:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f148dffb700 1 -- 172.21.15.7:0/2826769976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148800bbd0 con 0x7f149006b480 2023-06-09T11:29:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f148dffb700 1 -- 172.21.15.7:0/2826769976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488012430 con 0x7f149006b480 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 -- 172.21.15.7:0/2826769976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f149006b480 msgr2=0x7f149015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 --2- 172.21.15.7:0/2826769976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f149006b480 0x7f149015cdc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1488009d90 tx=0x7f148800b330 comp rx=0 tx=0).stop 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 -- 172.21.15.7:0/2826769976 shutdown_connections 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 --2- 172.21.15.7:0/2826769976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f149006b480 0x7f149015cdc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 -- 172.21.15.7:0/2826769976 >> 172.21.15.7:0/2826769976 conn(0x7f1490072090 msgr2=0x7f1490072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:16.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 -- 172.21.15.7:0/2826769976 shutdown_connections 2023-06-09T11:29:16.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.129+0000 7f14959ff700 1 -- 172.21.15.7:0/2826769976 wait complete. 2023-06-09T11:29:16.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.131+0000 7f14959ff700 1 Processor -- start 2023-06-09T11:29:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.131+0000 7f14959ff700 1 -- start start 2023-06-09T11:29:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f149006b480 0x7f14901e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14900b5360 0x7f14901f0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149015ce20 con 0x7f14900ca1b0 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149015bf30 con 0x7f14900b5360 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.132+0000 7f14959ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149015c210 con 0x7f149006b480 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44602/0 (socket says 172.21.15.7:44602) 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 -- 172.21.15.7:0/424104588 learned_addr learned my addr 172.21.15.7:0/424104588 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148e7fc700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14900b5360 0x7f14901f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148effd700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f149006b480 0x7f14901e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 -- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f149006b480 msgr2=0x7f14901e66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f149006b480 0x7f14901e66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 -- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14900b5360 msgr2=0x7f14901f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14900b5360 0x7f14901f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 -- 172.21.15.7:0/424104588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488009a70 con 0x7f14900ca1b0 2023-06-09T11:29:16.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f148f7fe700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1488014040 tx=0x7f1488004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.133+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148801b810 con 0x7f14900ca1b0 2023-06-09T11:29:16.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.134+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488020070 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.134+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148801bbb0 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.134+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149006a850 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.134+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149006a570 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.135+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1490153e00 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.136+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f148801bd10 con 0x7f14900ca1b0 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.137+0000 7f1477fff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 0x7f147c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.137+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f1488022070 con 0x7f14900ca1b0 2023-06-09T11:29:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.137+0000 7f148effd700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 0x7f147c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.137+0000 7f148effd700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 0x7f147c06d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1480007fd0 tx=0x7f1480009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.141+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14880565c0 con 0x7f14900ca1b0 2023-06-09T11:29:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.395+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1490074cf0 con 0x7f14900ca1b0 2023-06-09T11:29:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.396+0000 7f1477fff700 1 -- 172.21.15.7:0/424104588 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f1488059be0 con 0x7f14900ca1b0 2023-06-09T11:29:16.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:16.402 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 msgr2=0x7f147c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 0x7f147c06d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1480007fd0 tx=0x7f1480009040 comp rx=0 tx=0).stop 2023-06-09T11:29:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 msgr2=0x7f14901f09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1488014040 tx=0x7f1488004ad0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 shutdown_connections 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14900ca1b0 0x7f14901f09b0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f147c06ace0 0x7f147c06d190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f149006b480 0x7f14901e66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 --2- 172.21.15.7:0/424104588 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14900b5360 0x7f14901f0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 >> 172.21.15.7:0/424104588 conn(0x7f1490072090 msgr2=0x7f14901519f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 shutdown_connections 2023-06-09T11:29:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.398+0000 7f14959ff700 1 -- 172.21.15.7:0/424104588 wait complete. 2023-06-09T11:29:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:16.422 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2023-06-09T11:29:16.422 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:29:16.423 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:29:16.439 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:29:16.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:16.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.646+0000 7f05665ff700 1 Processor -- start 2023-06-09T11:29:16.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.647+0000 7f05665ff700 1 -- start start 2023-06-09T11:29:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f05665ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05600c1bc0 0x7f05601599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f05665ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05600c0140 0x7f0560159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f05665ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600c0bf0 con 0x7f05600c1f90 2023-06-09T11:29:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f05665ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0560070110 con 0x7f05600c0140 2023-06-09T11:29:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f05665ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0560071300 con 0x7f05600c1bc0 2023-06-09T11:29:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f0557fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05600c0140 0x7f0560159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f0557fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05600c0140 0x7f0560159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44606/0 (socket says 172.21.15.7:44606) 2023-06-09T11:29:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f0557fff700 1 -- 172.21.15.7:0/2304595415 learned_addr learned my addr 172.21.15.7:0/2304595415 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:16.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.648+0000 7f055ffff700 1 --2- 172.21.15.7:0/2304595415 >> v2:172.21.15.117:3300/0 conn(0x7f05600c1bc0 0x7f05601599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f0557fff700 1 -- 172.21.15.7:0/2304595415 >> v2:172.21.15.117:3300/0 conn(0x7f05600c1bc0 msgr2=0x7f05601599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f0557fff700 1 --2- 172.21.15.7:0/2304595415 >> v2:172.21.15.117:3300/0 conn(0x7f05600c1bc0 0x7f05601599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f0557fff700 1 -- 172.21.15.7:0/2304595415 >> v1:172.21.15.117:6789/0 conn(0x7f05600c1f90 legacy=0x7f05601501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f0557fff700 1 -- 172.21.15.7:0/2304595415 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600702f0 con 0x7f05600c0140 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f0557fff700 1 --2- 172.21.15.7:0/2304595415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560159ef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f05600cc0c0 tx=0x7f054800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2b4c24187dce45 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f055f7fe700 1 -- 172.21.15.7:0/2304595415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054800dc70 con 0x7f05600c0140 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f055f7fe700 1 -- 172.21.15.7:0/2304595415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054800ddd0 con 0x7f05600c0140 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.649+0000 7f055f7fe700 1 -- 172.21.15.7:0/2304595415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548010690 con 0x7f05600c0140 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 -- 172.21.15.7:0/2304595415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 msgr2=0x7f0560159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 --2- 172.21.15.7:0/2304595415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560159ef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f05600cc0c0 tx=0x7f054800be70 comp rx=0 tx=0).stop 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 -- 172.21.15.7:0/2304595415 shutdown_connections 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 --2- 172.21.15.7:0/2304595415 >> v2:172.21.15.117:3300/0 conn(0x7f05600c1bc0 0x7f05601599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 --2- 172.21.15.7:0/2304595415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560159ef0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 -- 172.21.15.7:0/2304595415 >> 172.21.15.7:0/2304595415 conn(0x7f056006a4d0 msgr2=0x7f056006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 -- 172.21.15.7:0/2304595415 shutdown_connections 2023-06-09T11:29:16.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.650+0000 7f05665ff700 1 -- 172.21.15.7:0/2304595415 wait complete. 2023-06-09T11:29:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.652+0000 7f05665ff700 1 Processor -- start 2023-06-09T11:29:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.652+0000 7f05665ff700 1 -- start start 2023-06-09T11:29:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 0x7f05601e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05600c1f90 0x7f05601e8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c5510 con 0x7f05600c0140 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c3270 con 0x7f05600c1bc0 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f05665ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c4910 con 0x7f05600c1f90 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44618/0 (socket says 172.21.15.7:44618) 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 -- 172.21.15.7:0/3018713962 learned_addr learned my addr 172.21.15.7:0/3018713962 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:16.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f0564b9c700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05600c1f90 0x7f05601e8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f0557fff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 0x7f05601e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 -- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05600c1f90 msgr2=0x7f05601e8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05600c1f90 0x7f05601e8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 -- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 msgr2=0x7f05601e8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 0x7f05601e8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f055ffff700 1 -- 172.21.15.7:0/3018713962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054800b7f0 con 0x7f05600c0140 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.653+0000 7f0557fff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 0x7f05601e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:16.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f055ffff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9b0 tx=0x7f055000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550009710 con 0x7f05600c0140 2023-06-09T11:29:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f055000c850 con 0x7f05600c0140 2023-06-09T11:29:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000cbc0 con 0x7f05600c0140 2023-06-09T11:29:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05600c1170 con 0x7f05600c0140 2023-06-09T11:29:16.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.654+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05600c0e90 con 0x7f05600c0140 2023-06-09T11:29:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.655+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05600c1450 con 0x7f05600c0140 2023-06-09T11:29:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.656+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f055000cd20 con 0x7f05600c0140 2023-06-09T11:29:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.657+0000 7f055d7fa700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 0x7f054006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.657+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f05500899a0 con 0x7f05600c0140 2023-06-09T11:29:16.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.657+0000 7f0557fff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 0x7f054006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:16.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.658+0000 7f0557fff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 0x7f054006d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f054800beb0 tx=0x7f0548013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:16.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.661+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f05500545e0 con 0x7f05600c0140 2023-06-09T11:29:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.948+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0560074cf0 con 0x7f05600c0140 2023-06-09T11:29:16.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.949+0000 7f055d7fa700 1 -- 172.21.15.7:0/3018713962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f0550017090 con 0x7f05600c0140 2023-06-09T11:29:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:16.955 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 msgr2=0x7f054006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 0x7f054006d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f054800beb0 tx=0x7f0548013000 comp rx=0 tx=0).stop 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 msgr2=0x7f0560158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9b0 tx=0x7f055000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 shutdown_connections 2023-06-09T11:29:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f054006ace0 0x7f054006d190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05600c1f90 0x7f05601e8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05600c0140 0x7f0560158ba0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 --2- 172.21.15.7:0/3018713962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05600c1bc0 0x7f05601e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 >> 172.21.15.7:0/3018713962 conn(0x7f056006a4d0 msgr2=0x7f056014d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 shutdown_connections 2023-06-09T11:29:16.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:16.951+0000 7f05665ff700 1 -- 172.21.15.7:0/3018713962 wait complete. 2023-06-09T11:29:16.959 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:16.975 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:18.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.171+0000 7f18cffff700 1 Processor -- start 2023-06-09T11:29:19.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.171+0000 7f18cffff700 1 -- start start 2023-06-09T11:29:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18c80cb5d0 0x7f18c80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18c80cb9a0 0x7f18c8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c8159b20 con 0x7f18c80cb9a0 2023-06-09T11:29:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80cac80 con 0x7f18c80cb5d0 2023-06-09T11:29:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.172+0000 7f18cffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81597d0 con 0x7f18c815cd40 2023-06-09T11:29:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18c80cb9a0 0x7f18c8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f18c80cb9a0 0x7f18c8159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44626/0 (socket says 172.21.15.7:44626) 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 -- 172.21.15.7:0/3633193641 learned_addr learned my addr 172.21.15.7:0/3633193641 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cdd9b700 1 --2- 172.21.15.7:0/3633193641 >> v2:172.21.15.117:3300/0 conn(0x7f18c80cb5d0 0x7f18c80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18ce59c700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 -- 172.21.15.7:0/3633193641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 msgr2=0x7f18c80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 -- 172.21.15.7:0/3633193641 >> v2:172.21.15.117:3300/0 conn(0x7f18c80cb5d0 msgr2=0x7f18c80b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3633193641 >> v2:172.21.15.117:3300/0 conn(0x7f18c80cb5d0 0x7f18c80b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.173+0000 7f18cd59a700 1 -- 172.21.15.7:0/3633193641 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8071ab0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c8159fb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18c808d420 tx=0x7f18b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e1879bb199d46b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18ccd99700 1 -- 172.21.15.7:0/3633193641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011830 con 0x7f18c80cb9a0 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18ccd99700 1 -- 172.21.15.7:0/3633193641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8011990 con 0x7f18c80cb9a0 2023-06-09T11:29:19.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18ccd99700 1 -- 172.21.15.7:0/3633193641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8011d20 con 0x7f18c80cb9a0 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18cffff700 1 -- 172.21.15.7:0/3633193641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 msgr2=0x7f18c8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.174+0000 7f18cffff700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c8159fb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f18c808d420 tx=0x7f18b8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 -- 172.21.15.7:0/3633193641 shutdown_connections 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 --2- 172.21.15.7:0/3633193641 >> v2:172.21.15.117:3300/0 conn(0x7f18c80cb5d0 0x7f18c80b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 --2- 172.21.15.7:0/3633193641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c8159fb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 -- 172.21.15.7:0/3633193641 >> 172.21.15.7:0/3633193641 conn(0x7f18c8070af0 msgr2=0x7f18c8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:19.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 -- 172.21.15.7:0/3633193641 shutdown_connections 2023-06-09T11:29:19.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.175+0000 7f18cffff700 1 -- 172.21.15.7:0/3633193641 wait complete. 2023-06-09T11:29:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.176+0000 7f18cffff700 1 Processor -- start 2023-06-09T11:29:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 -- start start 2023-06-09T11:29:19.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18c80cb5d0 0x7f18c81f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c81ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81efe70 con 0x7f18c80cb9a0 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c81542f0 con 0x7f18c80cb5d0 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.177+0000 7f18cffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c8152170 con 0x7f18c815cd40 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cdd9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18c80cb5d0 0x7f18c81f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44628/0 (socket says 172.21.15.7:44628) 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 -- 172.21.15.7:0/3564625947 learned_addr learned my addr 172.21.15.7:0/3564625947 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18ce59c700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c81ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 -- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 msgr2=0x7f18c81ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c81ef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 -- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18c80cb5d0 msgr2=0x7f18c81f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:19.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18c80cb5d0 0x7f18c81f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 -- 172.21.15.7:0/3564625947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b8009550 con 0x7f18c80cb9a0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.178+0000 7f18cd59a700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18b8000c00 tx=0x7f18b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.179+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8004240 con 0x7f18c80cb9a0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.179+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b801f070 con 0x7f18c80cb9a0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.179+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c815d4f0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.179+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80036a0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.179+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c815d210 con 0x7f18c80cb9a0 2023-06-09T11:29:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.180+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c815d7d0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.181+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18b8021030 con 0x7f18c80cb9a0 2023-06-09T11:29:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.182+0000 7f18be7fc700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 0x7f18ac06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.182+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f18b808a8c0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.182+0000 7f18cdd9b700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 0x7f18ac06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:19.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.183+0000 7f18cdd9b700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 0x7f18ac06d1e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009b60 tx=0x7f18c4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.186+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f18b8055580 con 0x7f18c80cb9a0 2023-06-09T11:29:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.450+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18c8074cf0 con 0x7f18c80cb9a0 2023-06-09T11:29:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.450+0000 7f18be7fc700 1 -- 172.21.15.7:0/3564625947 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f18b8015070 con 0x7f18c80cb9a0 2023-06-09T11:29:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:19.457 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 msgr2=0x7f18ac06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 0x7f18ac06d1e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009b60 tx=0x7f18c4009210 comp rx=0 tx=0).stop 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 msgr2=0x7f18c81ef310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f18b8000c00 tx=0x7f18b8004750 comp rx=0 tx=0).stop 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 shutdown_connections 2023-06-09T11:29:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18ac06ad30 0x7f18ac06d1e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18c815cd40 0x7f18c81ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18c80cb5d0 0x7f18c81f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 --2- 172.21.15.7:0/3564625947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18c80cb9a0 0x7f18c81ef310 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 >> 172.21.15.7:0/3564625947 conn(0x7f18c8070af0 msgr2=0x7f18c815b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 shutdown_connections 2023-06-09T11:29:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:19.453+0000 7f18cffff700 1 -- 172.21.15.7:0/3564625947 wait complete. 2023-06-09T11:29:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:19.477 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:21.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.687+0000 7ff5587ff700 1 Processor -- start 2023-06-09T11:29:21.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.687+0000 7ff5587ff700 1 -- start start 2023-06-09T11:29:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff5587ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 0x7ff5500b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff5587ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 0x7ff5500cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff5587ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff550159b20 con 0x7ff5500cb5d0 2023-06-09T11:29:21.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff5587ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5500cac80 con 0x7ff5500cb9a0 2023-06-09T11:29:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff5587ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501597d0 con 0x7ff55015cd40 2023-06-09T11:29:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff55659b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff5500cb5d0 0x7ff550159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34350/0 (socket says 172.21.15.7:34350) 2023-06-09T11:29:21.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.689+0000 7ff55659b700 1 -- 172.21.15.7:0/269291500 learned_addr learned my addr 172.21.15.7:0/269291500 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff555d9a700 1 --2- 172.21.15.7:0/269291500 >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 0x7ff5500b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff556d9c700 1 --2- 172.21.15.7:0/269291500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 0x7ff5500cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff555599700 1 -- 172.21.15.7:0/269291500 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345238585 0 0) 0x7ff550159b20 con 0x7ff5500cb5d0 2023-06-09T11:29:21.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff555599700 1 -- 172.21.15.7:0/269291500 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff53c0035b0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff556d9c700 1 -- 172.21.15.7:0/269291500 >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 msgr2=0x7ff5500b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff556d9c700 1 --2- 172.21.15.7:0/269291500 >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 0x7ff5500b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff556d9c700 1 -- 172.21.15.7:0/269291500 >> v1:172.21.15.7:6789/0 conn(0x7ff5500cb5d0 legacy=0x7ff550159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff556d9c700 1 -- 172.21.15.7:0/269291500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff550071ab0 con 0x7ff55015cd40 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.690+0000 7ff555d9a700 1 --2- 172.21.15.7:0/269291500 >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 0x7ff5500b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.691+0000 7ff556d9c700 1 --2- 172.21.15.7:0/269291500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 0x7ff5500cbea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d90 tx=0x7ff54800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b4b43b4c78ebad server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:21.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.691+0000 7ff555599700 1 -- 172.21.15.7:0/269291500 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800e070 con 0x7ff55015cd40 2023-06-09T11:29:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.691+0000 7ff555599700 1 -- 172.21.15.7:0/269291500 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54800bb20 con 0x7ff55015cd40 2023-06-09T11:29:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.691+0000 7ff555599700 1 -- 172.21.15.7:0/269291500 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548012430 con 0x7ff55015cd40 2023-06-09T11:29:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 -- 172.21.15.7:0/269291500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 msgr2=0x7ff5500cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:21.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 --2- 172.21.15.7:0/269291500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 0x7ff5500cbea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff548009d90 tx=0x7ff54800b330 comp rx=0 tx=0).stop 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 -- 172.21.15.7:0/269291500 shutdown_connections 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 --2- 172.21.15.7:0/269291500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff55015cd40 0x7ff5500cbea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 --2- 172.21.15.7:0/269291500 >> v2:172.21.15.117:3300/0 conn(0x7ff5500cb9a0 0x7ff5500b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 -- 172.21.15.7:0/269291500 >> 172.21.15.7:0/269291500 conn(0x7ff550070af0 msgr2=0x7ff550070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 -- 172.21.15.7:0/269291500 shutdown_connections 2023-06-09T11:29:21.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.692+0000 7ff5587ff700 1 -- 172.21.15.7:0/269291500 wait complete. 2023-06-09T11:29:21.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.694+0000 7ff5587ff700 1 Processor -- start 2023-06-09T11:29:21.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.695+0000 7ff5587ff700 1 -- start start 2023-06-09T11:29:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 0x7ff5500c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55015cd40 0x7ff5500c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5501ef830 con 0x7ff5500cb5d0 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff550150530 con 0x7ff55015cd40 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.696+0000 7ff5587ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff550150810 con 0x7ff5500cb9a0 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44642/0 (socket says 172.21.15.7:44642) 2023-06-09T11:29:21.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 -- 172.21.15.7:0/93220720 learned_addr learned my addr 172.21.15.7:0/93220720 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:21.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff555d9a700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 0x7ff5500c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff556d9c700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55015cd40 0x7ff5500c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 -- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 msgr2=0x7ff5500c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 0x7ff5500c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 -- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55015cd40 msgr2=0x7ff5500c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55015cd40 0x7ff5500c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff55659b700 1 -- 172.21.15.7:0/93220720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400094a0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.697+0000 7ff555d9a700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 0x7ff5500c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.698+0000 7ff55659b700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c700 tx=0x7ff54c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:21.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.698+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c005d70 con 0x7ff5500cb5d0 2023-06-09T11:29:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.698+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c0070d0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.698+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c007460 con 0x7ff5500cb5d0 2023-06-09T11:29:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.698+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff548009a70 con 0x7ff5500cb5d0 2023-06-09T11:29:21.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.699+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55015d4f0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.700+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55015d210 con 0x7ff5500cb5d0 2023-06-09T11:29:21.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.702+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff54c0075c0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.704+0000 7ff546ffd700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 0x7ff53406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.704+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7ff54c088ff0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.705+0000 7ff555d9a700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 0x7ff53406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:21.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.705+0000 7ff555d9a700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 0x7ff53406d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5500c6480 tx=0x7ff540009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:21.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.709+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff54c053e30 con 0x7ff5500cb5d0 2023-06-09T11:29:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.991+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff55015d7d0 con 0x7ff5500cb5d0 2023-06-09T11:29:21.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.992+0000 7ff546ffd700 1 -- 172.21.15.7:0/93220720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7ff54c017090 con 0x7ff5500cb5d0 2023-06-09T11:29:21.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:21.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 msgr2=0x7ff53406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:22.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 0x7ff53406d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff5500c6480 tx=0x7ff540009210 comp rx=0 tx=0).stop 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 msgr2=0x7ff55014fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff54c00c700 tx=0x7ff54c00ca10 comp rx=0 tx=0).stop 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 shutdown_connections 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff55015cd40 0x7ff5500c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5500cb5d0 0x7ff55014fca0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff53406ace0 0x7ff53406d190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 --2- 172.21.15.7:0/93220720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5500cb9a0 0x7ff5500c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 >> 172.21.15.7:0/93220720 conn(0x7ff550070af0 msgr2=0x7ff5500cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 shutdown_connections 2023-06-09T11:29:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:21.994+0000 7ff5587ff700 1 -- 172.21.15.7:0/93220720 wait complete. 2023-06-09T11:29:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:22.018 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:24.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.269+0000 7f5eaefff700 1 Processor -- start 2023-06-09T11:29:24.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.269+0000 7f5eaefff700 1 -- start start 2023-06-09T11:29:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.270+0000 7f5eaefff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ea80cb5d0 0x7f5ea80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.270+0000 7f5eaefff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ea80cb9a0 0x7f5ea8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.270+0000 7f5eaefff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea8159b20 con 0x7f5ea815cd40 2023-06-09T11:29:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.270+0000 7f5eaefff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea80cac80 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.270+0000 7f5eaefff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea81597d0 con 0x7f5ea80cb9a0 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5ead59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5ea815cd40 0x7f5ea80cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38348/0 (socket says 172.21.15.7:38348) 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5eacd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ea80cb5d0 0x7f5ea80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5ead59c700 1 -- 172.21.15.7:0/2364142206 learned_addr learned my addr 172.21.15.7:0/2364142206 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5ea7fff700 1 --2- 172.21.15.7:0/2364142206 >> v2:172.21.15.117:3300/0 conn(0x7f5ea80cb9a0 0x7f5ea8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5ea77fe700 1 -- 172.21.15.7:0/2364142206 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247170764 0 0) 0x7f5ea8159b20 con 0x7f5ea815cd40 2023-06-09T11:29:24.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5eacd9b700 1 -- 172.21.15.7:0/2364142206 >> v2:172.21.15.117:3300/0 conn(0x7f5ea80cb9a0 msgr2=0x7f5ea8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5eacd9b700 1 --2- 172.21.15.7:0/2364142206 >> v2:172.21.15.117:3300/0 conn(0x7f5ea80cb9a0 0x7f5ea8159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5eacd9b700 1 -- 172.21.15.7:0/2364142206 >> v1:172.21.15.7:6789/0 conn(0x7f5ea815cd40 legacy=0x7f5ea80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.271+0000 7f5eacd9b700 1 -- 172.21.15.7:0/2364142206 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8071ab0 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5eacd9b700 1 --2- 172.21.15.7:0/2364142206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea80cb5d0 0x7f5ea80b52b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80c9c60 tx=0x7f5e9c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f6d085ee0d8714 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5ea77fe700 1 -- 172.21.15.7:0/2364142206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011830 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5ea77fe700 1 -- 172.21.15.7:0/2364142206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c011990 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5ea77fe700 1 -- 172.21.15.7:0/2364142206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c011d20 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5eaefff700 1 -- 172.21.15.7:0/2364142206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea80cb5d0 msgr2=0x7f5ea80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.272+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2364142206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea80cb5d0 0x7f5ea80b52b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80c9c60 tx=0x7f5e9c009b20 comp rx=0 tx=0).stop 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 -- 172.21.15.7:0/2364142206 shutdown_connections 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2364142206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea80cb5d0 0x7f5ea80b52b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2364142206 >> v2:172.21.15.117:3300/0 conn(0x7f5ea80cb9a0 0x7f5ea8159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 -- 172.21.15.7:0/2364142206 >> 172.21.15.7:0/2364142206 conn(0x7f5ea8070af0 msgr2=0x7f5ea8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 -- 172.21.15.7:0/2364142206 shutdown_connections 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.273+0000 7f5eaefff700 1 -- 172.21.15.7:0/2364142206 wait complete. 2023-06-09T11:29:24.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.274+0000 7f5eaefff700 1 Processor -- start 2023-06-09T11:29:24.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.275+0000 7f5eaefff700 1 -- start start 2023-06-09T11:29:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ea80cb5d0 0x7f5ea80cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea80cb9a0 0x7f5ea80c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80cd560 con 0x7f5ea815cd40 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80cdd70 con 0x7f5ea80cb9a0 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eaefff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea81e2aa0 con 0x7f5ea80cb5d0 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5eacd9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ea80cb5d0 0x7f5ea80cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52008/0 (socket says 172.21.15.7:52008) 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 -- 172.21.15.7:0/2078437922 learned_addr learned my addr 172.21.15.7:0/2078437922 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:24.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ea7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea80cb9a0 0x7f5ea80c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 -- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ea80cb5d0 msgr2=0x7f5ea80cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ea80cb5d0 0x7f5ea80cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 -- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea80cb9a0 msgr2=0x7f5ea80c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea80cb9a0 0x7f5ea80c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.276+0000 7f5ead59c700 1 -- 172.21.15.7:0/2078437922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c0094a0 con 0x7f5ea815cd40 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5ead59c700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000cc10 tx=0x7f5ea000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea000aaa0 con 0x7f5ea815cd40 2023-06-09T11:29:24.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea000ac20 con 0x7f5ea815cd40 2023-06-09T11:29:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea815d4f0 con 0x7f5ea815cd40 2023-06-09T11:29:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea00070d0 con 0x7f5ea815cd40 2023-06-09T11:29:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.277+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea815d210 con 0x7f5ea815cd40 2023-06-09T11:29:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.278+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea815d7d0 con 0x7f5ea815cd40 2023-06-09T11:29:24.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.280+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5ea0007230 con 0x7f5ea815cd40 2023-06-09T11:29:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.282+0000 7f5ea57fa700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 0x7f5e8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:24.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.282+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f5ea0089550 con 0x7f5ea815cd40 2023-06-09T11:29:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.282+0000 7f5eacd9b700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 0x7f5e8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:24.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.283+0000 7f5eacd9b700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 0x7f5e8c06d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0090b0 tx=0x7f5e9c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:24.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.286+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5ea0054220 con 0x7f5ea815cd40 2023-06-09T11:29:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.540+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ea808c690 con 0x7f5ea815cd40 2023-06-09T11:29:24.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.541+0000 7f5ea57fa700 1 -- 172.21.15.7:0/2078437922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f5ea0057840 con 0x7f5ea815cd40 2023-06-09T11:29:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:24.547 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:24.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 msgr2=0x7f5e8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 0x7f5e8c06d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0090b0 tx=0x7f5e9c008030 comp rx=0 tx=0).stop 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 msgr2=0x7f5ea814fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5ea000cc10 tx=0x7f5ea000cf20 comp rx=0 tx=0).stop 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 shutdown_connections 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ea815cd40 0x7f5ea814fd50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5e8c06ace0 0x7f5e8c06d190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ea80cb5d0 0x7f5ea80cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 --2- 172.21.15.7:0/2078437922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ea80cb9a0 0x7f5ea80c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.543+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 >> 172.21.15.7:0/2078437922 conn(0x7f5ea8070af0 msgr2=0x7f5ea80cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.544+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 shutdown_connections 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:24.544+0000 7f5eaefff700 1 -- 172.21.15.7:0/2078437922 wait complete. 2023-06-09T11:29:24.551 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:24.575 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:26.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.771+0000 7f239c9ff700 1 Processor -- start 2023-06-09T11:29:26.777 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.771+0000 7f239c9ff700 1 -- start start 2023-06-09T11:29:26.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.772+0000 7f239c9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f239815b4a0 0x7f239815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.772+0000 7f239c9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23980c9f40 0x7f23980c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.772+0000 7f239c9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239815a500 con 0x7f2398153880 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.772+0000 7f239c9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23980c5f10 con 0x7f23980c9f40 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.772+0000 7f239c9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239815da00 con 0x7f239815b4a0 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23980c9f40 0x7f23980c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f239659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2398153880 0x7f239815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38350/0 (socket says 172.21.15.7:38350) 2023-06-09T11:29:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f239659c700 1 -- 172.21.15.7:0/3029286152 learned_addr learned my addr 172.21.15.7:0/3029286152 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2395d9b700 1 --2- 172.21.15.7:0/3029286152 >> v2:172.21.15.117:3300/0 conn(0x7f239815b4a0 0x7f239815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 -- 172.21.15.7:0/3029286152 >> v2:172.21.15.117:3300/0 conn(0x7f239815b4a0 msgr2=0x7f239815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 --2- 172.21.15.7:0/3029286152 >> v2:172.21.15.117:3300/0 conn(0x7f239815b4a0 0x7f239815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 -- 172.21.15.7:0/3029286152 >> v1:172.21.15.7:6789/0 conn(0x7f2398153880 legacy=0x7f239815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 -- 172.21.15.7:0/3029286152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23981596e0 con 0x7f23980c9f40 2023-06-09T11:29:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.773+0000 7f2396d9d700 1 --2- 172.21.15.7:0/3029286152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f40 0x7f23980c4e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f23980cce30 tx=0x7f239000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb4a34fd1cc8691 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239559a700 1 -- 172.21.15.7:0/3029286152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000e070 con 0x7f23980c9f40 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239559a700 1 -- 172.21.15.7:0/3029286152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000bbd0 con 0x7f23980c9f40 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239559a700 1 -- 172.21.15.7:0/3029286152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390012430 con 0x7f23980c9f40 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 -- 172.21.15.7:0/3029286152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f40 msgr2=0x7f23980c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 --2- 172.21.15.7:0/3029286152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f40 0x7f23980c4e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f23980cce30 tx=0x7f239000b330 comp rx=0 tx=0).stop 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 -- 172.21.15.7:0/3029286152 shutdown_connections 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 --2- 172.21.15.7:0/3029286152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23980c9f40 0x7f23980c4e30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 --2- 172.21.15.7:0/3029286152 >> v2:172.21.15.117:3300/0 conn(0x7f239815b4a0 0x7f239815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 -- 172.21.15.7:0/3029286152 >> 172.21.15.7:0/3029286152 conn(0x7f23980737f0 msgr2=0x7f23980720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 -- 172.21.15.7:0/3029286152 shutdown_connections 2023-06-09T11:29:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.774+0000 7f239c9ff700 1 -- 172.21.15.7:0/3029286152 wait complete. 2023-06-09T11:29:26.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.776+0000 7f239c9ff700 1 Processor -- start 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.776+0000 7f239c9ff700 1 -- start start 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c9f40 0x7f23981494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239815b4a0 0x7f239814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2398158f00 con 0x7f2398153880 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980b3c10 con 0x7f239815b4a0 2023-06-09T11:29:26.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f239c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980b34f0 con 0x7f23980c9f40 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f2395d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f2395d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52038/0 (socket says 172.21.15.7:52038) 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.777+0000 7f2395d9b700 1 -- 172.21.15.7:0/1842690922 learned_addr learned my addr 172.21.15.7:0/1842690922 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2396d9d700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239815b4a0 0x7f239814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f239659c700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c9f40 0x7f23981494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 -- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c9f40 msgr2=0x7f23981494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c9f40 0x7f23981494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:26.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 -- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239815b4a0 msgr2=0x7f239814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239815b4a0 0x7f239814ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 -- 172.21.15.7:0/1842690922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009a70 con 0x7f2398153880 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2395d9b700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f238c00d830 tx=0x7f238c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0125f0 con 0x7f2398153880 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c012750 con 0x7f2398153880 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c012b00 con 0x7f2398153880 2023-06-09T11:29:26.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.778+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2398159c60 con 0x7f2398153880 2023-06-09T11:29:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.779+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2398159980 con 0x7f2398153880 2023-06-09T11:29:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.780+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2398159f40 con 0x7f2398153880 2023-06-09T11:29:26.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.781+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f238c012c60 con 0x7f2398153880 2023-06-09T11:29:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.782+0000 7f2386ffd700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 0x7f237806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.782+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f238c089800 con 0x7f2398153880 2023-06-09T11:29:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.782+0000 7f239659c700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 0x7f237806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:26.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.783+0000 7f239659c700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 0x7f237806d1c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2388000f60 tx=0x7f2388007f10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:26.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:26.787+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f238c054610 con 0x7f2398153880 2023-06-09T11:29:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.038+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23980b38c0 con 0x7f2398153880 2023-06-09T11:29:27.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.039+0000 7f2386ffd700 1 -- 172.21.15.7:0/1842690922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f23980b38c0 con 0x7f2398153880 2023-06-09T11:29:27.045 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:27.045 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 msgr2=0x7f237806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:27.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 0x7f237806d1c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2388000f60 tx=0x7f2388007f10 comp rx=0 tx=0).stop 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 msgr2=0x7f239814b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f238c00d830 tx=0x7f238c00db40 comp rx=0 tx=0).stop 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 shutdown_connections 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f239815b4a0 0x7f239814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f237806ad10 0x7f237806d1c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f23980c9f40 0x7f23981494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:27.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 --2- 172.21.15.7:0/1842690922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2398153880 0x7f239814b4d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 >> 172.21.15.7:0/1842690922 conn(0x7f23980737f0 msgr2=0x7f239815b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 shutdown_connections 2023-06-09T11:29:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:27.041+0000 7f239c9ff700 1 -- 172.21.15.7:0/1842690922 wait complete. 2023-06-09T11:29:27.049 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:27.065 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:29.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:29.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.294+0000 7f85ed7ff700 1 Processor -- start 2023-06-09T11:29:29.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.294+0000 7f85ed7ff700 1 -- start start 2023-06-09T11:29:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85e81514f0 0x7f85e806d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85e80c8250 0x7f85e80c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e80c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80c28b0 con 0x7f85e81514f0 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85e80c95d0 con 0x7f85e80c8250 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.295+0000 7f85ed7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80c7e40 con 0x7f85e80c9150 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85e81514f0 0x7f85e806d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85e81514f0 0x7f85e806d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52044/0 (socket says 172.21.15.7:52044) 2023-06-09T11:29:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 -- 172.21.15.7:0/625484336 learned_addr learned my addr 172.21.15.7:0/625484336 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:29.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e67fc700 1 --2- 172.21.15.7:0/625484336 >> v2:172.21.15.117:3300/0 conn(0x7f85e80c8250 0x7f85e80c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e77fe700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e80c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 -- 172.21.15.7:0/625484336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 msgr2=0x7f85e80c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e80c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 -- 172.21.15.7:0/625484336 >> v2:172.21.15.117:3300/0 conn(0x7f85e80c8250 msgr2=0x7f85e80c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 --2- 172.21.15.7:0/625484336 >> v2:172.21.15.117:3300/0 conn(0x7f85e80c8250 0x7f85e80c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.296+0000 7f85e6ffd700 1 -- 172.21.15.7:0/625484336 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e8158010 con 0x7f85e81514f0 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.297+0000 7f85e6ffd700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e806d0f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f85e8071720 tx=0x7f85d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96c883903174674 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.297+0000 7f85e5ffb700 1 -- 172.21.15.7:0/625484336 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8011830 con 0x7f85e81514f0 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.297+0000 7f85e5ffb700 1 -- 172.21.15.7:0/625484336 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d8011990 con 0x7f85e81514f0 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.297+0000 7f85e5ffb700 1 -- 172.21.15.7:0/625484336 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d8011d20 con 0x7f85e81514f0 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 -- 172.21.15.7:0/625484336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 msgr2=0x7f85e806d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e806d0f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f85e8071720 tx=0x7f85d8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 -- 172.21.15.7:0/625484336 shutdown_connections 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e80c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/625484336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e806d0f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/625484336 >> v2:172.21.15.117:3300/0 conn(0x7f85e80c8250 0x7f85e80c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 -- 172.21.15.7:0/625484336 >> 172.21.15.7:0/625484336 conn(0x7f85e80cd330 msgr2=0x7f85e80cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 -- 172.21.15.7:0/625484336 shutdown_connections 2023-06-09T11:29:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.298+0000 7f85ed7ff700 1 -- 172.21.15.7:0/625484336 wait complete. 2023-06-09T11:29:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.301+0000 7f85ed7ff700 1 Processor -- start 2023-06-09T11:29:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.301+0000 7f85ed7ff700 1 -- start start 2023-06-09T11:29:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e80c8250 0x7f85e806bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e81f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e8148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80cc970 con 0x7f85e81514f0 2023-06-09T11:29:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e80cbf80 con 0x7f85e80c8250 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85ed7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e806aba0 con 0x7f85e80c9150 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85e6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e80c8250 0x7f85e806bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85e67fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e81f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85e67fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e81f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:35388/0 (socket says 172.21.15.7:35388) 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85e67fc700 1 -- 172.21.15.7:0/3597407064 learned_addr learned my addr 172.21.15.7:0/3597407064 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.303+0000 7f85e77fe700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e8148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 -- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 msgr2=0x7f85e81f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e81f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 -- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e80c8250 msgr2=0x7f85e806bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e80c8250 0x7f85e806bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 -- 172.21.15.7:0/3597407064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8009550 con 0x7f85e81514f0 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85e77fe700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e8148aa0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f85e0007dc0 tx=0x7f85e000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.304+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0018070 con 0x7f85e81514f0 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.305+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e000eb30 con 0x7f85e81514f0 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.305+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e8157490 con 0x7f85e81514f0 2023-06-09T11:29:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.305+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e81571b0 con 0x7f85e81514f0 2023-06-09T11:29:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.305+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000eec0 con 0x7f85e81514f0 2023-06-09T11:29:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.306+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c40052e0 con 0x7f85e81514f0 2023-06-09T11:29:29.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.309+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85e0003a20 con 0x7f85e81514f0 2023-06-09T11:29:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.310+0000 7f85df7fe700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 0x7f85d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:29.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.310+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f85e008a190 con 0x7f85e81514f0 2023-06-09T11:29:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.311+0000 7f85e6ffd700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 0x7f85d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:29.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.312+0000 7f85e6ffd700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 0x7f85d006d1c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009c10 tx=0x7f85d8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:29.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.315+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f85e0054f70 con 0x7f85e81514f0 2023-06-09T11:29:29.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.654+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c4005860 con 0x7f85e81514f0 2023-06-09T11:29:29.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.654+0000 7f85df7fe700 1 -- 172.21.15.7:0/3597407064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f85e0009950 con 0x7f85e81514f0 2023-06-09T11:29:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:29.661 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 msgr2=0x7f85d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 0x7f85d006d1c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f85d8009c10 tx=0x7f85d8008030 comp rx=0 tx=0).stop 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 msgr2=0x7f85e8148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e8148aa0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f85e0007dc0 tx=0x7f85e000e3c0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 shutdown_connections 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85e81514f0 0x7f85e8148aa0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85e80c8250 0x7f85e806bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f85d006ad10 0x7f85d006d1c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 --2- 172.21.15.7:0/3597407064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85e80c9150 0x7f85e81f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 >> 172.21.15.7:0/3597407064 conn(0x7f85e80cd330 msgr2=0x7f85e814b140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 shutdown_connections 2023-06-09T11:29:29.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:29.657+0000 7f85ed7ff700 1 -- 172.21.15.7:0/3597407064 wait complete. 2023-06-09T11:29:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:29.680 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:31.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:31.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.917+0000 7f7982d9d700 1 Processor -- start 2023-06-09T11:29:31.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.918+0000 7f7982d9d700 1 -- start start 2023-06-09T11:29:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.919+0000 7f7982d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f797c0c0d10 0x7f797c06f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.919+0000 7f7982d9d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06b000 con 0x7f797c071c60 2023-06-09T11:29:31.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.919+0000 7f7982d9d700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f797c06b2a0 con 0x7f797c0c8730 2023-06-09T11:29:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.919+0000 7f7982d9d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f797c157e20 con 0x7f797c0c0d10 2023-06-09T11:29:31.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f7981d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f797c071c60 0x7f797c06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38364/0 (socket says 172.21.15.7:38364) 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f7981d9b700 1 -- 172.21.15.7:0/4089085464 learned_addr learned my addr 172.21.15.7:0/4089085464 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f798259c700 1 --2- 172.21.15.7:0/4089085464 >> v2:172.21.15.7:3300/0 conn(0x7f797c0c0d10 0x7f797c06f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f7980d99700 1 -- 172.21.15.7:0/4089085464 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116068863 0 0) 0x7f797c06b000 con 0x7f797c071c60 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f798259c700 1 -- 172.21.15.7:0/4089085464 >> v1:172.21.15.117:6789/0 conn(0x7f797c0c8730 legacy=0x7f797c0c0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.920+0000 7f798259c700 1 -- 172.21.15.7:0/4089085464 >> v1:172.21.15.7:6789/0 conn(0x7f797c071c60 legacy=0x7f797c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:31.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.921+0000 7f798259c700 1 -- 172.21.15.7:0/4089085464 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797c15e0b0 con 0x7f797c0c0d10 2023-06-09T11:29:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.921+0000 7f798259c700 1 --2- 172.21.15.7:0/4089085464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c0d10 0x7f797c06f660 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7974009d90 tx=0x7f797400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf86209d9f8ca71 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:31.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.921+0000 7f7980d99700 1 -- 172.21.15.7:0/4089085464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400e070 con 0x7f797c0c0d10 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.921+0000 7f7980d99700 1 -- 172.21.15.7:0/4089085464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797400bbd0 con 0x7f797c0c0d10 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.921+0000 7f7980d99700 1 -- 172.21.15.7:0/4089085464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974012430 con 0x7f797c0c0d10 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 -- 172.21.15.7:0/4089085464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c0d10 msgr2=0x7f797c06f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 --2- 172.21.15.7:0/4089085464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c0d10 0x7f797c06f660 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7974009d90 tx=0x7f797400b330 comp rx=0 tx=0).stop 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 -- 172.21.15.7:0/4089085464 shutdown_connections 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 --2- 172.21.15.7:0/4089085464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c0d10 0x7f797c06f660 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 -- 172.21.15.7:0/4089085464 >> 172.21.15.7:0/4089085464 conn(0x7f797c0cbcb0 msgr2=0x7f797c0c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:31.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.922+0000 7f7982d9d700 1 -- 172.21.15.7:0/4089085464 shutdown_connections 2023-06-09T11:29:31.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.923+0000 7f7982d9d700 1 -- 172.21.15.7:0/4089085464 wait complete. 2023-06-09T11:29:31.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.925+0000 7f7982d9d700 1 Processor -- start 2023-06-09T11:29:31.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.925+0000 7f7982d9d700 1 -- start start 2023-06-09T11:29:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c071c60 0x7f797c0c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c0d10 0x7f797c0c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:31.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0c44c0 con 0x7f797c0c8730 2023-06-09T11:29:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797c1f2e80 con 0x7f797c0c0d10 2023-06-09T11:29:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.926+0000 7f7982d9d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f797c0c8e60 con 0x7f797c071c60 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52068/0 (socket says 172.21.15.7:52068) 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 -- 172.21.15.7:0/2618877116 learned_addr learned my addr 172.21.15.7:0/2618877116 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f7981d9b700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c071c60 0x7f797c0c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798159a700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c0d10 0x7f797c0c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 -- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c071c60 msgr2=0x7f797c0c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c071c60 0x7f797c0c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 -- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c0d10 msgr2=0x7f797c0c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c0d10 0x7f797c0c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.927+0000 7f798259c700 1 -- 172.21.15.7:0/2618877116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7974009a70 con 0x7f797c0c8730 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f798259c700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f797c068c30 tx=0x7f7974004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797401b810 con 0x7f797c0c8730 2023-06-09T11:29:31.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974020070 con 0x7f797c0c8730 2023-06-09T11:29:31.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797c0c00e0 con 0x7f797c0c8730 2023-06-09T11:29:31.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797401bbb0 con 0x7f797c0c8730 2023-06-09T11:29:31.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.928+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797c15e390 con 0x7f797c0c8730 2023-06-09T11:29:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.929+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797c06abe0 con 0x7f797c0c8730 2023-06-09T11:29:31.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.931+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f797401bd10 con 0x7f797c0c8730 2023-06-09T11:29:31.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.933+0000 7f79727fc700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 0x7f796006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.933+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f7974022070 con 0x7f797c0c8730 2023-06-09T11:29:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.933+0000 7f7981d9b700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 0x7f796006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:31.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.934+0000 7f7981d9b700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 0x7f796006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978004a50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:31.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:31.938+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7974056dd0 con 0x7f797c0c8730 2023-06-09T11:29:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.194+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f797c04f760 con 0x7f797c0c8730 2023-06-09T11:29:32.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.195+0000 7f79727fc700 1 -- 172.21.15.7:0/2618877116 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f797405a3f0 con 0x7f797c0c8730 2023-06-09T11:29:32.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:32.201 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5435,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:28:08.492020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5587,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5587":{"gid":5587,"name":"b","rank":0,"incarnation":61,"state":"up:active","state_seq":36,"addr":"172.21.15.117:6833/2112619043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2112619043},{"type":"v1","addr":"172.21.15.117:6833","nonce":2112619043}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 msgr2=0x7f796006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 0x7f796006d190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978004a50 comp rx=0 tx=0).stop 2023-06-09T11:29:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 msgr2=0x7f797c0ca3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f797c068c30 tx=0x7f7974004ad0 comp rx=0 tx=0).stop 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 shutdown_connections 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f797c0c8730 0x7f797c0ca3b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f796006ace0 0x7f796006d190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f797c071c60 0x7f797c0c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 --2- 172.21.15.7:0/2618877116 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f797c0c0d10 0x7f797c0c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 >> 172.21.15.7:0/2618877116 conn(0x7f797c0cbcb0 msgr2=0x7f797c0c0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:32.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 shutdown_connections 2023-06-09T11:29:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:32.197+0000 7f7982d9d700 1 -- 172.21.15.7:0/2618877116 wait complete. 2023-06-09T11:29:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2023-06-09T11:29:32.221 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:29:34.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.435+0000 7f4d7adff700 1 Processor -- start 2023-06-09T11:29:34.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.435+0000 7f4d7adff700 1 -- start start 2023-06-09T11:29:34.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7adff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d740c2d90 0x7f4d740c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7adff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d7415a0f0 0x7f4d740cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7adff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d740c88d0 con 0x7f4d74153a70 2023-06-09T11:29:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7adff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d740c2bf0 con 0x7f4d740c2d90 2023-06-09T11:29:34.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7adff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d74153e40 con 0x7f4d7415a0f0 2023-06-09T11:29:34.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d73fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d740c2d90 0x7f4d740c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d73fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d740c2d90 0x7f4d740c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35208/0 (socket says 172.21.15.7:35208) 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d73fff700 1 -- 172.21.15.7:0/2467218635 learned_addr learned my addr 172.21.15.7:0/2467218635 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.437+0000 7f4d7939c700 1 --2- 172.21.15.7:0/2467218635 >> v2:172.21.15.117:3300/0 conn(0x7f4d7415a0f0 0x7f4d740cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d73fff700 1 -- 172.21.15.7:0/2467218635 >> v2:172.21.15.117:3300/0 conn(0x7f4d7415a0f0 msgr2=0x7f4d740cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d73fff700 1 --2- 172.21.15.7:0/2467218635 >> v2:172.21.15.117:3300/0 conn(0x7f4d7415a0f0 0x7f4d740cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d73fff700 1 -- 172.21.15.7:0/2467218635 >> v1:172.21.15.117:6789/0 conn(0x7f4d74153a70 legacy=0x7f4d740ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d73fff700 1 -- 172.21.15.7:0/2467218635 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740ca350 con 0x7f4d740c2d90 2023-06-09T11:29:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d73fff700 1 --2- 172.21.15.7:0/2467218635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d740c2d90 0x7f4d740c26b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4d74071c00 tx=0x7f4d60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b01439773f46da5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d737fe700 1 -- 172.21.15.7:0/2467218635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000b870 con 0x7f4d740c2d90 2023-06-09T11:29:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d737fe700 1 -- 172.21.15.7:0/2467218635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6000b9d0 con 0x7f4d740c2d90 2023-06-09T11:29:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.438+0000 7f4d737fe700 1 -- 172.21.15.7:0/2467218635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000bd60 con 0x7f4d740c2d90 2023-06-09T11:29:34.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.439+0000 7f4d7adff700 1 -- 172.21.15.7:0/2467218635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d740c2d90 msgr2=0x7f4d740c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.439+0000 7f4d7adff700 1 --2- 172.21.15.7:0/2467218635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d740c2d90 0x7f4d740c26b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4d74071c00 tx=0x7f4d60009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 -- 172.21.15.7:0/2467218635 shutdown_connections 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 --2- 172.21.15.7:0/2467218635 >> v2:172.21.15.117:3300/0 conn(0x7f4d7415a0f0 0x7f4d740cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 --2- 172.21.15.7:0/2467218635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d740c2d90 0x7f4d740c26b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 -- 172.21.15.7:0/2467218635 >> 172.21.15.7:0/2467218635 conn(0x7f4d7406e410 msgr2=0x7f4d7406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 -- 172.21.15.7:0/2467218635 shutdown_connections 2023-06-09T11:29:34.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.440+0000 7f4d7adff700 1 -- 172.21.15.7:0/2467218635 wait complete. 2023-06-09T11:29:34.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.443+0000 7f4d7adff700 1 Processor -- start 2023-06-09T11:29:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.444+0000 7f4d7adff700 1 -- start start 2023-06-09T11:29:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d740c2d90 0x7f4d741587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 0x7f4d7406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 0x7f4d741516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d741f6840 con 0x7f4d7415a0f0 2023-06-09T11:29:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d741f76a0 con 0x7f4d740c2d90 2023-06-09T11:29:34.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d741f6a90 con 0x7f4d74153a70 2023-06-09T11:29:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 0x7f4d741516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7939c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 0x7f4d741516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35222/0 (socket says 172.21.15.7:35222) 2023-06-09T11:29:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d7939c700 1 -- 172.21.15.7:0/3392857755 learned_addr learned my addr 172.21.15.7:0/3392857755 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d73fff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 0x7f4d7406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.446+0000 7f4d78b9b700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d740c2d90 0x7f4d741587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 -- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d740c2d90 msgr2=0x7f4d741587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d740c2d90 0x7f4d741587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 -- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 msgr2=0x7f4d741516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 0x7f4d741516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 -- 172.21.15.7:0/3392857755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d60009550 con 0x7f4d74153a70 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.447+0000 7f4d73fff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 0x7f4d7406b3b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d60016000 tx=0x7f4d60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.448+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60004250 con 0x7f4d74153a70 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.448+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6001f070 con 0x7f4d74153a70 2023-06-09T11:29:34.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.448+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d740c85f0 con 0x7f4d74153a70 2023-06-09T11:29:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.448+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d600045d0 con 0x7f4d74153a70 2023-06-09T11:29:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.448+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d740c8350 con 0x7f4d74153a70 2023-06-09T11:29:34.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.450+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d740c9d00 con 0x7f4d74153a70 2023-06-09T11:29:34.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.451+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d60021030 con 0x7f4d74153a70 2023-06-09T11:29:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.453+0000 7f4d717fa700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 0x7f4d5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.453+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f4d6008ac90 con 0x7f4d74153a70 2023-06-09T11:29:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.454+0000 7f4d78b9b700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 0x7f4d5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.455+0000 7f4d78b9b700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 0x7f4d5806d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4d68007fd0 tx=0x7f4d68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.458+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4d60055a30 con 0x7f4d74153a70 2023-06-09T11:29:34.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.762+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d74074cf0 con 0x7f4d74153a70 2023-06-09T11:29:34.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.763+0000 7f4d717fa700 1 -- 172.21.15.7:0/3392857755 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f4d60059050 con 0x7f4d74153a70 2023-06-09T11:29:34.769 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:34.769 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:34.368189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:34.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 msgr2=0x7f4d5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 0x7f4d5806d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4d68007fd0 tx=0x7f4d68009040 comp rx=0 tx=0).stop 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 msgr2=0x7f4d7406b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 0x7f4d7406b3b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4d60016000 tx=0x7f4d60004750 comp rx=0 tx=0).stop 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 shutdown_connections 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d7415a0f0 0x7f4d741516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d740c2d90 0x7f4d741587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4d5806ace0 0x7f4d5806d190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 --2- 172.21.15.7:0/3392857755 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d74153a70 0x7f4d7406b3b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 >> 172.21.15.7:0/3392857755 conn(0x7f4d7406e410 msgr2=0x7f4d74155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.765+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 shutdown_connections 2023-06-09T11:29:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.766+0000 7f4d7adff700 1 -- 172.21.15.7:0/3392857755 wait complete. 2023-06-09T11:29:34.773 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2023-06-09T11:29:34.789 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:29:34.789 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:29:34.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:34.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.982+0000 7f4d115ff700 1 Processor -- start 2023-06-09T11:29:34.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.982+0000 7f4d115ff700 1 -- start start 2023-06-09T11:29:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.983+0000 7f4d115ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4d0c151510 0x7f4d0c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.983+0000 7f4d115ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4d0c15b380 0x7f4d0c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d115ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0c1580a0 con 0x7f4d0c071060 2023-06-09T11:29:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d115ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0c1518e0 con 0x7f4d0c151510 2023-06-09T11:29:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d115ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d0c071480 con 0x7f4d0c15b380 2023-06-09T11:29:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d0c071060 0x7f4d0c158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42656/0 (socket says 172.21.15.7:42656) 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0b7fe700 1 -- 172.21.15.7:0/800919493 learned_addr learned my addr 172.21.15.7:0/800919493 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0affd700 1 --2- 172.21.15.7:0/800919493 >> v2:172.21.15.7:3300/0 conn(0x7f4d0c151510 0x7f4d0c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0a7fc700 1 --2- 172.21.15.7:0/800919493 >> v2:172.21.15.117:3300/0 conn(0x7f4d0c15b380 0x7f4d0c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d09ffb700 1 -- 172.21.15.7:0/800919493 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1109928108 0 0) 0x7f4d0c1580a0 con 0x7f4d0c071060 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d09ffb700 1 -- 172.21.15.7:0/800919493 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cf40035b0 con 0x7f4d0c071060 2023-06-09T11:29:34.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0affd700 1 -- 172.21.15.7:0/800919493 >> v2:172.21.15.117:3300/0 conn(0x7f4d0c15b380 msgr2=0x7f4d0c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0affd700 1 --2- 172.21.15.7:0/800919493 >> v2:172.21.15.117:3300/0 conn(0x7f4d0c15b380 0x7f4d0c070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0affd700 1 -- 172.21.15.7:0/800919493 >> v1:172.21.15.7:6789/0 conn(0x7f4d0c071060 legacy=0x7f4d0c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.984+0000 7f4d0affd700 1 -- 172.21.15.7:0/800919493 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0c0c6180 con 0x7f4d0c151510 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d0affd700 1 --2- 172.21.15.7:0/800919493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c151510 0x7f4d0c0c5b40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c06b170 tx=0x7f4cfc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ec5abe1ec42b48 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d09ffb700 1 -- 172.21.15.7:0/800919493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc011830 con 0x7f4d0c151510 2023-06-09T11:29:34.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d09ffb700 1 -- 172.21.15.7:0/800919493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc011990 con 0x7f4d0c151510 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d09ffb700 1 -- 172.21.15.7:0/800919493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc011d20 con 0x7f4d0c151510 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d115ff700 1 -- 172.21.15.7:0/800919493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c151510 msgr2=0x7f4d0c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.985+0000 7f4d115ff700 1 --2- 172.21.15.7:0/800919493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c151510 0x7f4d0c0c5b40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c06b170 tx=0x7f4cfc009b20 comp rx=0 tx=0).stop 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 -- 172.21.15.7:0/800919493 shutdown_connections 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 --2- 172.21.15.7:0/800919493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c151510 0x7f4d0c0c5b40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 --2- 172.21.15.7:0/800919493 >> v2:172.21.15.117:3300/0 conn(0x7f4d0c15b380 0x7f4d0c070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 -- 172.21.15.7:0/800919493 >> 172.21.15.7:0/800919493 conn(0x7f4d0c06a4d0 msgr2=0x7f4d0c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:34.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 -- 172.21.15.7:0/800919493 shutdown_connections 2023-06-09T11:29:34.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.986+0000 7f4d115ff700 1 -- 172.21.15.7:0/800919493 wait complete. 2023-06-09T11:29:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.987+0000 7f4d115ff700 1 Processor -- start 2023-06-09T11:29:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.988+0000 7f4d115ff700 1 -- start start 2023-06-09T11:29:34.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.988+0000 7f4d115ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 0x7f4d0c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.988+0000 7f4d115ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0c151510 0x7f4d0c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.988+0000 7f4d115ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 0x7f4d0c14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d115ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c14f9f0 con 0x7f4d0c15b380 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d115ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c14fcd0 con 0x7f4d0c071060 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d115ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c0c0c60 con 0x7f4d0c151510 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 0x7f4d0c14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 0x7f4d0c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0affd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 0x7f4d0c1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:34982/0 (socket says 172.21.15.7:34982) 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0affd700 1 -- 172.21.15.7:0/105950447 learned_addr learned my addr 172.21.15.7:0/105950447 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0a7fc700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0c151510 0x7f4d0c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 -- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0c151510 msgr2=0x7f4d0c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0c151510 0x7f4d0c14a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 -- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 msgr2=0x7f4d0c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 0x7f4d0c1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:34.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.989+0000 7f4d0b7fe700 1 -- 172.21.15.7:0/105950447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0094a0 con 0x7f4d0c15b380 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4d0b7fe700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 0x7f4d0c14ad70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400ca80 tx=0x7f4d0400cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0400a9e0 con 0x7f4d0c15b380 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0400ab40 con 0x7f4d0c15b380 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c157770 con 0x7f4d0c15b380 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04012400 con 0x7f4d0c15b380 2023-06-09T11:29:34.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.990+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c157490 con 0x7f4d0c15b380 2023-06-09T11:29:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.991+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c157a50 con 0x7f4d0c15b380 2023-06-09T11:29:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.992+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d0401b3b0 con 0x7f4d0c15b380 2023-06-09T11:29:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.993+0000 7f4cf37fe700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 0x7f4cf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.993+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f4d040893d0 con 0x7f4d0c15b380 2023-06-09T11:29:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.993+0000 7f4d0affd700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 0x7f4cf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.994+0000 7f4d0affd700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 0x7f4cf806d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc000c00 tx=0x7f4cfc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:34.997+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4d040540f0 con 0x7f4d0c15b380 2023-06-09T11:29:35.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.276+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0c0c0ac0 con 0x7f4d0c15b380 2023-06-09T11:29:35.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.276+0000 7f4cf37fe700 1 -- 172.21.15.7:0/105950447 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f4d0c0c0ac0 con 0x7f4d0c15b380 2023-06-09T11:29:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:35.283 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:34.368189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 msgr2=0x7f4cf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 0x7f4cf806d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc000c00 tx=0x7f4cfc008030 comp rx=0 tx=0).stop 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 msgr2=0x7f4d0c14ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 0x7f4d0c14ad70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400ca80 tx=0x7f4d0400cd90 comp rx=0 tx=0).stop 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 shutdown_connections 2023-06-09T11:29:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d0c15b380 0x7f4d0c14ad70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d0c071060 0x7f4d0c1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cf806ace0 0x7f4cf806d190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 --2- 172.21.15.7:0/105950447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d0c151510 0x7f4d0c14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 >> 172.21.15.7:0/105950447 conn(0x7f4d0c06a4d0 msgr2=0x7f4d0c14d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 shutdown_connections 2023-06-09T11:29:35.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:35.279+0000 7f4d115ff700 1 -- 172.21.15.7:0/105950447 wait complete. 2023-06-09T11:29:35.287 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2023-06-09T11:29:35.303 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 79, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 79, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5435": { "addr": "172.21.15.117:6835/3983837537", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3983837537, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3983837537, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5435, "incarnation": 79, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:29:34.368189+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5435, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:29:37.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:37.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.513+0000 7f570b7ff700 1 Processor -- start 2023-06-09T11:29:37.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.513+0000 7f570b7ff700 1 -- start start 2023-06-09T11:29:37.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.514+0000 7f570b7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57040c5b40 0x7f57040c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.514+0000 7f570b7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5704142850 0x7f570415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.514+0000 7f570b7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570415dab0 con 0x7f57040c6980 2023-06-09T11:29:37.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.514+0000 7f570b7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5704159aa0 con 0x7f5704142850 2023-06-09T11:29:37.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.514+0000 7f570b7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570415d210 con 0x7f57040c5b40 2023-06-09T11:29:37.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5704142850 0x7f570415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5704142850 0x7f570415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35244/0 (socket says 172.21.15.7:35244) 2023-06-09T11:29:37.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 -- 172.21.15.7:0/2545281620 learned_addr learned my addr 172.21.15.7:0/2545281620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:37.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f570959b700 1 --2- 172.21.15.7:0/2545281620 >> v2:172.21.15.117:3300/0 conn(0x7f57040c5b40 0x7f57040c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 -- 172.21.15.7:0/2545281620 >> v2:172.21.15.117:3300/0 conn(0x7f57040c5b40 msgr2=0x7f57040c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:37.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 --2- 172.21.15.7:0/2545281620 >> v2:172.21.15.117:3300/0 conn(0x7f57040c5b40 0x7f57040c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 -- 172.21.15.7:0/2545281620 >> v1:172.21.15.117:6789/0 conn(0x7f57040c6980 legacy=0x7f5704159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.515+0000 7f5709d9c700 1 -- 172.21.15.7:0/2545281620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570415d4f0 con 0x7f5704142850 2023-06-09T11:29:37.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.516+0000 7f5709d9c700 1 --2- 172.21.15.7:0/2545281620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5704142850 0x7f570415cb60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f57040c8d40 tx=0x7f56fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d73d16567ae374 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:37.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.516+0000 7f56fbfff700 1 -- 172.21.15.7:0/2545281620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00e070 con 0x7f5704142850 2023-06-09T11:29:37.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.516+0000 7f56fbfff700 1 -- 172.21.15.7:0/2545281620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00b910 con 0x7f5704142850 2023-06-09T11:29:37.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.516+0000 7f56fbfff700 1 -- 172.21.15.7:0/2545281620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00bcc0 con 0x7f5704142850 2023-06-09T11:29:37.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.517+0000 7f570b7ff700 1 -- 172.21.15.7:0/2545281620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5704142850 msgr2=0x7f570415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.517+0000 7f570b7ff700 1 --2- 172.21.15.7:0/2545281620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5704142850 0x7f570415cb60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f57040c8d40 tx=0x7f56fc00b330 comp rx=0 tx=0).stop 2023-06-09T11:29:37.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 -- 172.21.15.7:0/2545281620 shutdown_connections 2023-06-09T11:29:37.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 --2- 172.21.15.7:0/2545281620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5704142850 0x7f570415cb60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 --2- 172.21.15.7:0/2545281620 >> v2:172.21.15.117:3300/0 conn(0x7f57040c5b40 0x7f57040c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 -- 172.21.15.7:0/2545281620 >> 172.21.15.7:0/2545281620 conn(0x7f570406a4d0 msgr2=0x7f570406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:37.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 -- 172.21.15.7:0/2545281620 shutdown_connections 2023-06-09T11:29:37.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.518+0000 7f570b7ff700 1 -- 172.21.15.7:0/2545281620 wait complete. 2023-06-09T11:29:37.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.521+0000 7f570b7ff700 1 Processor -- start 2023-06-09T11:29:37.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.521+0000 7f570b7ff700 1 -- start start 2023-06-09T11:29:37.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57040c6980 0x7f570415c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 0x7f5704156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570413c9e0 con 0x7f57040c5b40 2023-06-09T11:29:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570413d2b0 con 0x7f57040c6980 2023-06-09T11:29:37.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.522+0000 7f570b7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570415a730 con 0x7f5704142850 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.523+0000 7f570959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.523+0000 7f570959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35260/0 (socket says 172.21.15.7:35260) 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.523+0000 7f570959b700 1 -- 172.21.15.7:0/1708749118 learned_addr learned my addr 172.21.15.7:0/1708749118 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.523+0000 7f5709d9c700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 0x7f5704156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.523+0000 7f5708d9a700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57040c6980 0x7f570415c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 -- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57040c6980 msgr2=0x7f570415c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57040c6980 0x7f570415c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 -- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 msgr2=0x7f57040c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 -- 172.21.15.7:0/1708749118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc009b30 con 0x7f5704142850 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f570959b700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:37.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f5709d9c700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 0x7f5704156440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56fc015000 tx=0x7f56fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc004880 con 0x7f5704142850 2023-06-09T11:29:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc0092e0 con 0x7f5704142850 2023-06-09T11:29:37.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc01b650 con 0x7f5704142850 2023-06-09T11:29:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.524+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5704159730 con 0x7f5704142850 2023-06-09T11:29:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.525+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570415d7d0 con 0x7f5704142850 2023-06-09T11:29:37.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.526+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57040c5f10 con 0x7f5704142850 2023-06-09T11:29:37.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.528+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56fc01b7b0 con 0x7f5704142850 2023-06-09T11:29:37.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.530+0000 7f56f9ffb700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 0x7f56e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:37.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.530+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f56fc08b160 con 0x7f5704142850 2023-06-09T11:29:37.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.530+0000 7f570959b700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 0x7f56e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:37.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.531+0000 7f570959b700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 0x7f56e806d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57000113d0 tx=0x7f5700009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:37.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.532+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f56fc055e50 con 0x7f5704142850 2023-06-09T11:29:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.877+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5704074cf0 con 0x7f5704142850 2023-06-09T11:29:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.877+0000 7f56f9ffb700 1 -- 172.21.15.7:0/1708749118 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f56fc059470 con 0x7f5704142850 2023-06-09T11:29:37.883 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:37.883 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:34.368189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 msgr2=0x7f56e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 0x7f56e806d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f57000113d0 tx=0x7f5700009210 comp rx=0 tx=0).stop 2023-06-09T11:29:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 msgr2=0x7f5704156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 0x7f5704156440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f56fc015000 tx=0x7f56fc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 shutdown_connections 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f56e806ace0 0x7f56e806d190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5704142850 0x7f5704156440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57040c5b40 0x7f57040c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 --2- 172.21.15.7:0/1708749118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57040c6980 0x7f570415c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 >> 172.21.15.7:0/1708749118 conn(0x7f570406a4d0 msgr2=0x7f5704157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 shutdown_connections 2023-06-09T11:29:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:37.880+0000 7f570b7ff700 1 -- 172.21.15.7:0/1708749118 wait complete. 2023-06-09T11:29:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2023-06-09T11:29:37.904 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:39.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.099+0000 7f86a25ff700 1 Processor -- start 2023-06-09T11:29:40.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.099+0000 7f86a25ff700 1 -- start start 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.100+0000 7f86a25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb5d0 0x7f869c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.100+0000 7f86a25ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f869c0cb9a0 0x7f869c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a25ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 0x7f869c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a25ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c159b20 con 0x7f869c0cb9a0 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a25ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f869c0cac80 con 0x7f869c15cd40 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1597d0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f869c0cb9a0 0x7f869c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f869c0cb9a0 0x7f869c159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35262/0 (socket says 172.21.15.7:35262) 2023-06-09T11:29:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 -- 172.21.15.7:0/1627031296 learned_addr learned my addr 172.21.15.7:0/1627031296 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f869bfff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb5d0 0x7f869c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a0b9c700 1 --2- 172.21.15.7:0/1627031296 >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 0x7f869c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 -- 172.21.15.7:0/1627031296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb5d0 msgr2=0x7f869c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb5d0 0x7f869c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 -- 172.21.15.7:0/1627031296 >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 msgr2=0x7f869c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 --2- 172.21.15.7:0/1627031296 >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 0x7f869c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 -- 172.21.15.7:0/1627031296 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c071ab0 con 0x7f869c0cb9a0 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f86a0b9c700 1 --2- 172.21.15.7:0/1627031296 >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 0x7f869c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.101+0000 7f8693fff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c0cb9a0 0x7f869c159fb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f869c0b4ed0 tx=0x7f8684009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82714c6baf1a734 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f869b7fe700 1 -- 172.21.15.7:0/1627031296 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011830 con 0x7f869c0cb9a0 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f869b7fe700 1 -- 172.21.15.7:0/1627031296 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684011990 con 0x7f869c0cb9a0 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f869b7fe700 1 -- 172.21.15.7:0/1627031296 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684011d20 con 0x7f869c0cb9a0 2023-06-09T11:29:40.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 -- 172.21.15.7:0/1627031296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c0cb9a0 msgr2=0x7f869c159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c0cb9a0 0x7f869c159fb0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f869c0b4ed0 tx=0x7f8684009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 -- 172.21.15.7:0/1627031296 shutdown_connections 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 --2- 172.21.15.7:0/1627031296 >> v2:172.21.15.117:3300/0 conn(0x7f869c15cd40 0x7f869c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb5d0 0x7f869c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 --2- 172.21.15.7:0/1627031296 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c0cb9a0 0x7f869c159fb0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.102+0000 7f86a25ff700 1 -- 172.21.15.7:0/1627031296 >> 172.21.15.7:0/1627031296 conn(0x7f869c070af0 msgr2=0x7f869c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:40.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.103+0000 7f86a25ff700 1 -- 172.21.15.7:0/1627031296 shutdown_connections 2023-06-09T11:29:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.103+0000 7f86a25ff700 1 -- 172.21.15.7:0/1627031296 wait complete. 2023-06-09T11:29:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.105+0000 7f86a25ff700 1 Processor -- start 2023-06-09T11:29:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.105+0000 7f86a25ff700 1 -- start start 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 0x7f869c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 0x7f869c1efaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c1ef160 con 0x7f869c15cd40 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0cf550 con 0x7f869c0cb5d0 2023-06-09T11:29:40.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.106+0000 7f86a25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869c0cf810 con 0x7f869c0cb9a0 2023-06-09T11:29:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 0x7f869c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f8693fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f86a0b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 0x7f869c1efaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f8693fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:33188/0 (socket says 172.21.15.7:33188) 2023-06-09T11:29:40.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f8693fff700 1 -- 172.21.15.7:0/2318056686 learned_addr learned my addr 172.21.15.7:0/2318056686 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 -- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 msgr2=0x7f869c1ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 -- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 msgr2=0x7f869c1efaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 0x7f869c1efaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 -- 172.21.15.7:0/2318056686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8684009550 con 0x7f869c0cb5d0 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f8693fff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f86a0b9c700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 0x7f869c1efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.107+0000 7f869bfff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 0x7f869c0b1670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e870 tx=0x7f868c00eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.108+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c013680 con 0x7f869c0cb5d0 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.108+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c0137e0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.108+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c013b50 con 0x7f869c0cb5d0 2023-06-09T11:29:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.108+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869c15d4f0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.108+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f869c15d210 con 0x7f869c0cb5d0 2023-06-09T11:29:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.109+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869c15d7d0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.110+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f868c013cb0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.111+0000 7f86997fa700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 0x7f867c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.111+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f868c089660 con 0x7f869c0cb5d0 2023-06-09T11:29:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.111+0000 7f8693fff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 0x7f867c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.111+0000 7f8693fff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 0x7f867c06d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f8684008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:40.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.121+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f868c057ce0 con 0x7f869c0cb5d0 2023-06-09T11:29:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.378+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f869c15ae20 con 0x7f869c0cb5d0 2023-06-09T11:29:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.378+0000 7f86997fa700 1 -- 172.21.15.7:0/2318056686 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f868c057870 con 0x7f869c0cb5d0 2023-06-09T11:29:40.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:40.384 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":79,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:34.368189+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:40.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 msgr2=0x7f867c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 0x7f867c06d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f8684009fd0 tx=0x7f8684008030 comp rx=0 tx=0).stop 2023-06-09T11:29:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 msgr2=0x7f869c0b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 0x7f869c0b1670 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f868c00e870 tx=0x7f868c00eb80 comp rx=0 tx=0).stop 2023-06-09T11:29:40.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 shutdown_connections 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f869c15cd40 0x7f869c1efaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f869c0cb5d0 0x7f869c0b1670 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867c06ace0 0x7f867c06d190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 --2- 172.21.15.7:0/2318056686 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f869c0cb9a0 0x7f869c1ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 >> 172.21.15.7:0/2318056686 conn(0x7f869c070af0 msgr2=0x7f869c15b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 shutdown_connections 2023-06-09T11:29:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:40.382+0000 7f86a25ff700 1 -- 172.21.15.7:0/2318056686 wait complete. 2023-06-09T11:29:40.389 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 79 2023-06-09T11:29:40.414 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:42.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.619+0000 7fa5c03ff700 1 Processor -- start 2023-06-09T11:29:42.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.619+0000 7fa5c03ff700 1 -- start start 2023-06-09T11:29:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.621+0000 7fa5c03ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa5b80cb5d0 0x7fa5b80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.621+0000 7fa5c03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b80cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.621+0000 7fa5c03ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b8159b20 con 0x7fa5b80cb9a0 2023-06-09T11:29:42.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.621+0000 7fa5c03ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b80cac80 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.621+0000 7fa5c03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b81597d0 con 0x7fa5b815cd40 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5bd99a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa5b80cb9a0 0x7fa5b8159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42664/0 (socket says 172.21.15.7:42664) 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5bd99a700 1 -- 172.21.15.7:0/3657890210 learned_addr learned my addr 172.21.15.7:0/3657890210 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be19b700 1 --2- 172.21.15.7:0/3657890210 >> v2:172.21.15.7:3300/0 conn(0x7fa5b80cb5d0 0x7fa5b80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be99c700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b80cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be19b700 1 -- 172.21.15.7:0/3657890210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 msgr2=0x7fa5b80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:42.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be19b700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b80cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be19b700 1 -- 172.21.15.7:0/3657890210 >> v1:172.21.15.7:6789/0 conn(0x7fa5b80cb9a0 legacy=0x7fa5b8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5be19b700 1 -- 172.21.15.7:0/3657890210 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8071ab0 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.622+0000 7fa5bd199700 1 -- 172.21.15.7:0/3657890210 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254617124 0 0) 0x7fa5b8159b20 con 0x7fa5b80cb9a0 2023-06-09T11:29:42.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.623+0000 7fa5be19b700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80b52b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009840 tx=0x7fa5b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842cc977a00cb62a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.623+0000 7fa5bd199700 1 -- 172.21.15.7:0/3657890210 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4011830 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.623+0000 7fa5bd199700 1 -- 172.21.15.7:0/3657890210 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b4011990 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.623+0000 7fa5bd199700 1 -- 172.21.15.7:0/3657890210 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4011d20 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.624+0000 7fa5c03ff700 1 -- 172.21.15.7:0/3657890210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 msgr2=0x7fa5b80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:42.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.624+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80b52b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009840 tx=0x7fa5b4009b50 comp rx=0 tx=0).stop 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 -- 172.21.15.7:0/3657890210 shutdown_connections 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b80cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/3657890210 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80b52b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 -- 172.21.15.7:0/3657890210 >> 172.21.15.7:0/3657890210 conn(0x7fa5b8070af0 msgr2=0x7fa5b8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 -- 172.21.15.7:0/3657890210 shutdown_connections 2023-06-09T11:29:42.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.625+0000 7fa5c03ff700 1 -- 172.21.15.7:0/3657890210 wait complete. 2023-06-09T11:29:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.627+0000 7fa5c03ff700 1 Processor -- start 2023-06-09T11:29:42.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.627+0000 7fa5c03ff700 1 -- start start 2023-06-09T11:29:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b81f06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cfce0 con 0x7fa5b80cb5d0 2023-06-09T11:29:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80d04a0 con 0x7fa5b80cb9a0 2023-06-09T11:29:42.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.628+0000 7fa5c03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b80cf0d0 con 0x7fa5b815cd40 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.629+0000 7fa5bd99a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.629+0000 7fa5be19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.629+0000 7fa5bd99a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:35032/0 (socket says 172.21.15.7:35032) 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.629+0000 7fa5bd99a700 1 -- 172.21.15.7:0/2940869787 learned_addr learned my addr 172.21.15.7:0/2940869787 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.629+0000 7fa5be99c700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b81f06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 -- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 msgr2=0x7fa5b81f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:42.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 -- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 msgr2=0x7fa5b80c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 -- 172.21.15.7:0/2940869787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b40094a0 con 0x7fa5b815cd40 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5bd99a700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be19b700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:42.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5be99c700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b81f06f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5b815d140 tx=0x7fa5b000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000c940 con 0x7fa5b815cd40 2023-06-09T11:29:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000f040 con 0x7fa5b815cd40 2023-06-09T11:29:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000cce0 con 0x7fa5b815cd40 2023-06-09T11:29:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.630+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b815d4f0 con 0x7fa5b815cd40 2023-06-09T11:29:42.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.631+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b815d210 con 0x7fa5b815cd40 2023-06-09T11:29:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.632+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b815d7d0 con 0x7fa5b815cd40 2023-06-09T11:29:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.634+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5b0014480 con 0x7fa5b815cd40 2023-06-09T11:29:42.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.635+0000 7fa5aeffd700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.635+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fa5b008a140 con 0x7fa5b815cd40 2023-06-09T11:29:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.635+0000 7fa5be19b700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:42.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.636+0000 7fa5be19b700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 0x7fa59c06d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009fd0 tx=0x7fa5b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:42.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:42.640+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa5b00588d0 con 0x7fa5b815cd40 2023-06-09T11:29:43.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.063+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5b8074cf0 con 0x7fa5b815cd40 2023-06-09T11:29:43.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.064+0000 7fa5aeffd700 1 -- 172.21.15.7:0/2940869787 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fa5b0058460 con 0x7fa5b815cd40 2023-06-09T11:29:43.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:43.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:40.552819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:resolve","state_seq":24,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:43.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 msgr2=0x7fa59c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 0x7fa59c06d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009fd0 tx=0x7fa5b4008030 comp rx=0 tx=0).stop 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 msgr2=0x7fa5b81f06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b81f06f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5b815d140 tx=0x7fa5b000ee50 comp rx=0 tx=0).stop 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 shutdown_connections 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa59c06ace0 0x7fa59c06d190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5b815cd40 0x7fa5b81f06f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:43.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5b80cb5d0 0x7fa5b80c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:43.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.067+0000 7fa5c03ff700 1 --2- 172.21.15.7:0/2940869787 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5b80cb9a0 0x7fa5b81f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:43.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.068+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 >> 172.21.15.7:0/2940869787 conn(0x7fa5b8070af0 msgr2=0x7fa5b80b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:43.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.068+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 shutdown_connections 2023-06-09T11:29:43.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:43.068+0000 7fa5c03ff700 1 -- 172.21.15.7:0/2940869787 wait complete. 2023-06-09T11:29:43.074 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 80 2023-06-09T11:29:43.096 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:45.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.374+0000 7f08c31ff700 1 Processor -- start 2023-06-09T11:29:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.374+0000 7f08c31ff700 1 -- start start 2023-06-09T11:29:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c31ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08bc151510 0x7f08bc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c31ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c31ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08bc1580a0 con 0x7f08bc15b380 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c31ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08bc1518e0 con 0x7f08bc151510 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c31ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc0c6180 con 0x7f08bc071230 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c0f9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08bc151510 0x7f08bc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08bbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08bc15b380 0x7f08bc070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:32834/0 (socket says 172.21.15.7:32834) 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08bbfff700 1 -- 172.21.15.7:0/2806202574 learned_addr learned my addr 172.21.15.7:0/2806202574 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.376+0000 7f08c179c700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c0f9b700 1 -- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 msgr2=0x7f08bc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c0f9b700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c0f9b700 1 -- 172.21.15.7:0/2806202574 >> v1:172.21.15.7:6789/0 conn(0x7f08bc15b380 legacy=0x7f08bc070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c0f9b700 1 -- 172.21.15.7:0/2806202574 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc157490 con 0x7f08bc151510 2023-06-09T11:29:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c179c700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.377+0000 7f08c0f9b700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc0c5b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009840 tx=0x7f08b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a1b3f2d1d34f87f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.378+0000 7f08bb7fe700 1 -- 172.21.15.7:0/2806202574 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0011830 con 0x7f08bc151510 2023-06-09T11:29:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.379+0000 7f08bb7fe700 1 -- 172.21.15.7:0/2806202574 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b0011990 con 0x7f08bc151510 2023-06-09T11:29:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.379+0000 7f08bb7fe700 1 -- 172.21.15.7:0/2806202574 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0011d20 con 0x7f08bc151510 2023-06-09T11:29:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.379+0000 7f08c31ff700 1 -- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 msgr2=0x7f08bc0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.379+0000 7f08c31ff700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc0c5b40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009840 tx=0x7f08b0009b50 comp rx=0 tx=0).stop 2023-06-09T11:29:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 -- 172.21.15.7:0/2806202574 shutdown_connections 2023-06-09T11:29:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 --2- 172.21.15.7:0/2806202574 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc0c5b40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 -- 172.21.15.7:0/2806202574 >> 172.21.15.7:0/2806202574 conn(0x7f08bc06a4d0 msgr2=0x7f08bc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 -- 172.21.15.7:0/2806202574 shutdown_connections 2023-06-09T11:29:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.380+0000 7f08c31ff700 1 -- 172.21.15.7:0/2806202574 wait complete. 2023-06-09T11:29:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.383+0000 7f08c31ff700 1 Processor -- start 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.383+0000 7f08c31ff700 1 -- start start 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1f68e0 con 0x7f08bc15b380 2023-06-09T11:29:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc1f7420 con 0x7f08bc151510 2023-06-09T11:29:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c31ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc143770 con 0x7f08bc071230 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c179c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08bbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c0f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c179c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42090/0 (socket says 172.21.15.7:42090) 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08c179c700 1 -- 172.21.15.7:0/3351343500 learned_addr learned my addr 172.21.15.7:0/3351343500 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.385+0000 7f08bbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:37250/0 (socket says 172.21.15.7:37250) 2023-06-09T11:29:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 -- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 msgr2=0x7f08bc153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 -- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 msgr2=0x7f08bc143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 -- 172.21.15.7:0/3351343500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4009a70 con 0x7f08bc071230 2023-06-09T11:29:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08bbfff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c179c700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08c0f9b700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08b0016000 tx=0x7f08b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b00036a0 con 0x7f08bc071230 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b001f070 con 0x7f08bc071230 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.386+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b0003a60 con 0x7f08bc071230 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.387+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b00094a0 con 0x7f08bc071230 2023-06-09T11:29:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.387+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc157a50 con 0x7f08bc071230 2023-06-09T11:29:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.388+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc157770 con 0x7f08bc071230 2023-06-09T11:29:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.390+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08b0003bc0 con 0x7f08bc071230 2023-06-09T11:29:45.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.392+0000 7f08b97fa700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:45.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.392+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f08b0021070 con 0x7f08bc071230 2023-06-09T11:29:45.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.392+0000 7f08bbfff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.393+0000 7f08bbfff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 0x7f08a006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08bc14cb80 tx=0x7f08a8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:45.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.396+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08b008e030 con 0x7f08bc071230 2023-06-09T11:29:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.717+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08bc157d30 con 0x7f08bc071230 2023-06-09T11:29:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.718+0000 7f08b97fa700 1 -- 172.21.15.7:0/3351343500 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f08b0055cd0 con 0x7f08bc071230 2023-06-09T11:29:45.723 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:45.723 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":80,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:40.552819+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:resolve","state_seq":24,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 msgr2=0x7f08a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 0x7f08a006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08bc14cb80 tx=0x7f08a8009040 comp rx=0 tx=0).stop 2023-06-09T11:29:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 msgr2=0x7f08bc158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08b0016000 tx=0x7f08b0004750 comp rx=0 tx=0).stop 2023-06-09T11:29:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 shutdown_connections 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08bc15b380 0x7f08bc143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08a006ace0 0x7f08a006d190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08bc071230 0x7f08bc158de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 --2- 172.21.15.7:0/3351343500 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08bc151510 0x7f08bc153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 >> 172.21.15.7:0/3351343500 conn(0x7f08bc06a4d0 msgr2=0x7f08bc1569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 shutdown_connections 2023-06-09T11:29:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:45.720+0000 7f08c31ff700 1 -- 172.21.15.7:0/3351343500 wait complete. 2023-06-09T11:29:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 80 2023-06-09T11:29:45.743 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:45.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:45.822 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:29:47.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:47.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.954+0000 7fbab5dff700 1 Processor -- start 2023-06-09T11:29:47.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.954+0000 7fbab5dff700 1 -- start start 2023-06-09T11:29:47.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.955+0000 7fbab5dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 0x7fbab00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:47.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.955+0000 7fbab5dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab015a390 con 0x7fbab0153b30 2023-06-09T11:29:47.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.955+0000 7fbab5dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbab00ca720 con 0x7fbab0071fa0 2023-06-09T11:29:47.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.955+0000 7fbab5dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab00c5f40 con 0x7fbab0153760 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaeffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbab0153b30 0x7fbab00b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:32842/0 (socket says 172.21.15.7:32842) 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaeffd700 1 -- 172.21.15.7:0/1540285120 learned_addr learned my addr 172.21.15.7:0/1540285120 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaf7fe700 1 --2- 172.21.15.7:0/1540285120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 0x7fbab00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415650337 0 0) 0x7fbab015a390 con 0x7fbab0153b30 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba980035b0 con 0x7fbab0153b30 2023-06-09T11:29:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 986366469 0 0) 0x7fbab00ca720 con 0x7fbab0071fa0 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbab015a390 con 0x7fbab0071fa0 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaf7fe700 1 -- 172.21.15.7:0/1540285120 >> v1:172.21.15.117:6789/0 conn(0x7fbab0071fa0 legacy=0x7fbab015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaf7fe700 1 -- 172.21.15.7:0/1540285120 >> v1:172.21.15.7:6789/0 conn(0x7fbab0153b30 legacy=0x7fbab00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.956+0000 7fbaaf7fe700 1 -- 172.21.15.7:0/1540285120 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab00c5a20 con 0x7fbab0153760 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.957+0000 7fbaaf7fe700 1 --2- 172.21.15.7:0/1540285120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 0x7fbab00ca070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009870 tx=0x7fbaa0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb15c2d0d736085 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.957+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa0004230 con 0x7fbab0153760 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.957+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa0004390 con 0x7fbab0153760 2023-06-09T11:29:47.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.957+0000 7fbaae7fc700 1 -- 172.21.15.7:0/1540285120 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa00036a0 con 0x7fbab0153760 2023-06-09T11:29:47.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 -- 172.21.15.7:0/1540285120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 msgr2=0x7fbab00ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:47.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 --2- 172.21.15.7:0/1540285120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 0x7fbab00ca070 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbaa0009870 tx=0x7fbaa0009b80 comp rx=0 tx=0).stop 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 -- 172.21.15.7:0/1540285120 shutdown_connections 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 --2- 172.21.15.7:0/1540285120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0153760 0x7fbab00ca070 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 -- 172.21.15.7:0/1540285120 >> 172.21.15.7:0/1540285120 conn(0x7fbab00737f0 msgr2=0x7fbab0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 -- 172.21.15.7:0/1540285120 shutdown_connections 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.958+0000 7fbab5dff700 1 -- 172.21.15.7:0/1540285120 wait complete. 2023-06-09T11:29:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.960+0000 7fbab5dff700 1 Processor -- start 2023-06-09T11:29:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.960+0000 7fbab5dff700 1 -- start start 2023-06-09T11:29:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0071fa0 0x7fbab015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0153760 0x7fbab0157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0157b10 con 0x7fbab0153b30 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab0158990 con 0x7fbab0153760 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbab5dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbab01f2e50 con 0x7fbab0071fa0 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbaaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:47.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbaaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42096/0 (socket says 172.21.15.7:42096) 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbaaffff700 1 -- 172.21.15.7:0/2224225242 learned_addr learned my addr 172.21.15.7:0/2224225242 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbaaf7fe700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0071fa0 0x7fbab015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.961+0000 7fbaaeffd700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0153760 0x7fbab0157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 -- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0071fa0 msgr2=0x7fbab015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0071fa0 0x7fbab015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 -- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0153760 msgr2=0x7fbab0157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0153760 0x7fbab0157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:47.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 -- 172.21.15.7:0/2224225242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa00094a0 con 0x7fbab0153b30 2023-06-09T11:29:47.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbaaffff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c840 tx=0x7fbaa800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:47.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8003e30 con 0x7fbab0153b30 2023-06-09T11:29:47.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab0159a60 con 0x7fbab0153b30 2023-06-09T11:29:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaa8005070 con 0x7fbab0153b30 2023-06-09T11:29:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8005420 con 0x7fbab0153b30 2023-06-09T11:29:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.962+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab0159780 con 0x7fbab0153b30 2023-06-09T11:29:47.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.964+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab0159d40 con 0x7fbab0153b30 2023-06-09T11:29:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.965+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbaa8005580 con 0x7fbab0153b30 2023-06-09T11:29:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.966+0000 7fba97fff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 0x7fba9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.966+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fbaa80894e0 con 0x7fbab0153b30 2023-06-09T11:29:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.966+0000 7fbaaf7fe700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 0x7fba9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:47.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.967+0000 7fbaaf7fe700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 0x7fba9c06d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097c0 tx=0x7fbaa001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:47.972+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbaa8054170 con 0x7fbab0153b30 2023-06-09T11:29:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.286+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbab0074cf0 con 0x7fbab0153b30 2023-06-09T11:29:48.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.286+0000 7fba97fff700 1 -- 172.21.15.7:0/2224225242 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5459 (secure 0 0 0) 0x7fbaa8057790 con 0x7fbab0153b30 2023-06-09T11:29:48.291 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:48.292 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:46.380331+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:48.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 msgr2=0x7fba9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:48.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 0x7fba9c06d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbaa00097c0 tx=0x7fbaa001c040 comp rx=0 tx=0).stop 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 msgr2=0x7fbab0158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800c840 tx=0x7fbaa800cb50 comp rx=0 tx=0).stop 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 shutdown_connections 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbab0153b30 0x7fbab0158450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fba9c06ace0 0x7fba9c06d190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbab0071fa0 0x7fbab015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:48.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 --2- 172.21.15.7:0/2224225242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbab0153760 0x7fbab0157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.289+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 >> 172.21.15.7:0/2224225242 conn(0x7fbab00737f0 msgr2=0x7fbab014d590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.290+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 shutdown_connections 2023-06-09T11:29:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:48.290+0000 7fbab5dff700 1 -- 172.21.15.7:0/2224225242 wait complete. 2023-06-09T11:29:48.296 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2023-06-09T11:29:48.313 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 81, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 81, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5435": { "addr": "172.21.15.117:6835/3983837537", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3983837537, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3983837537, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5435, "incarnation": 79, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 26 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 59, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:29:46.380331+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5435, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:29:50.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.522+0000 7f0137bff700 1 Processor -- start 2023-06-09T11:29:50.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.522+0000 7f0137bff700 1 -- start start 2023-06-09T11:29:50.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.523+0000 7f0137bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01300c1bc0 0x7f01301599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.523+0000 7f0137bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01300c0140 0x7f0130159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.523+0000 7f0137bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300c0bf0 con 0x7f01300c1f90 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.523+0000 7f0137bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0130070110 con 0x7f01300c1bc0 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.523+0000 7f0137bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0130071300 con 0x7f01300c0140 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01300c1bc0 0x7f01301599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01300c1bc0 0x7f01301599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42100/0 (socket says 172.21.15.7:42100) 2023-06-09T11:29:50.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 -- 172.21.15.7:0/970339182 learned_addr learned my addr 172.21.15.7:0/970339182 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:50.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013519a700 1 --2- 172.21.15.7:0/970339182 >> v2:172.21.15.117:3300/0 conn(0x7f01300c0140 0x7f0130159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 -- 172.21.15.7:0/970339182 >> v2:172.21.15.117:3300/0 conn(0x7f01300c0140 msgr2=0x7f0130159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 --2- 172.21.15.7:0/970339182 >> v2:172.21.15.117:3300/0 conn(0x7f01300c0140 0x7f0130159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 -- 172.21.15.7:0/970339182 >> v1:172.21.15.117:6789/0 conn(0x7f01300c1f90 legacy=0x7f01301501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 -- 172.21.15.7:0/970339182 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300702f0 con 0x7f01300c1bc0 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.524+0000 7f013599b700 1 --2- 172.21.15.7:0/970339182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c1bc0 0x7f01301599b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f013006d950 tx=0x7f012c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4189a2a2600332a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:50.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0134999700 1 -- 172.21.15.7:0/970339182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011830 con 0x7f01300c1bc0 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0134999700 1 -- 172.21.15.7:0/970339182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c011990 con 0x7f01300c1bc0 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0134999700 1 -- 172.21.15.7:0/970339182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011d20 con 0x7f01300c1bc0 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 -- 172.21.15.7:0/970339182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c1bc0 msgr2=0x7f01301599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 --2- 172.21.15.7:0/970339182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c1bc0 0x7f01301599b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f013006d950 tx=0x7f012c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 -- 172.21.15.7:0/970339182 shutdown_connections 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 --2- 172.21.15.7:0/970339182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c1bc0 0x7f01301599b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 --2- 172.21.15.7:0/970339182 >> v2:172.21.15.117:3300/0 conn(0x7f01300c0140 0x7f0130159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 -- 172.21.15.7:0/970339182 >> 172.21.15.7:0/970339182 conn(0x7f013006a4d0 msgr2=0x7f013006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 -- 172.21.15.7:0/970339182 shutdown_connections 2023-06-09T11:29:50.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.525+0000 7f0137bff700 1 -- 172.21.15.7:0/970339182 wait complete. 2023-06-09T11:29:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.527+0000 7f0137bff700 1 Processor -- start 2023-06-09T11:29:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.527+0000 7f0137bff700 1 -- start start 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01300c1bc0 0x7f01301e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01300c1f90 0x7f013014b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301ea750 con 0x7f01300c0140 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301ee6d0 con 0x7f01300c1bc0 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f0137bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01301eec00 con 0x7f01300c1f90 2023-06-09T11:29:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f013599b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f013599b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42102/0 (socket says 172.21.15.7:42102) 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.528+0000 7f013599b700 1 -- 172.21.15.7:0/4218383162 learned_addr learned my addr 172.21.15.7:0/4218383162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013519a700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01300c1bc0 0x7f01301e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013619c700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01300c1f90 0x7f013014b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 -- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01300c1f90 msgr2=0x7f013014b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01300c1f90 0x7f013014b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 -- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01300c1bc0 msgr2=0x7f01301e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01300c1bc0 0x7f01301e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 -- 172.21.15.7:0/4218383162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009550 con 0x7f01300c0140 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f013599b700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f012c000c00 tx=0x7f012c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c004570 con 0x7f01300c0140 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c01f070 con 0x7f01300c0140 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01300c1170 con 0x7f01300c0140 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.529+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0038e0 con 0x7f01300c0140 2023-06-09T11:29:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.530+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01300c0e90 con 0x7f01300c0140 2023-06-09T11:29:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.530+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01300c1450 con 0x7f01300c0140 2023-06-09T11:29:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.531+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f012c021030 con 0x7f01300c0140 2023-06-09T11:29:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.533+0000 7f01267fc700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 0x7f011406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.533+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f012c08ac70 con 0x7f01300c0140 2023-06-09T11:29:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.533+0000 7f013519a700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 0x7f011406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:50.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.534+0000 7f013519a700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 0x7f011406d1c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01300c6190 tx=0x7f0120008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:50.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.538+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f012c055a90 con 0x7f01300c0140 2023-06-09T11:29:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.789+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01301ea9f0 con 0x7f01300c0140 2023-06-09T11:29:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.789+0000 7f01267fc700 1 -- 172.21.15.7:0/4218383162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f012c015090 con 0x7f01300c0140 2023-06-09T11:29:50.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:50.795 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:46.380331+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:reconnect","state_seq":26,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 msgr2=0x7f011406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 0x7f011406d1c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f01300c6190 tx=0x7f0120008040 comp rx=0 tx=0).stop 2023-06-09T11:29:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 msgr2=0x7f0130159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:50.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f012c000c00 tx=0x7f012c004750 comp rx=0 tx=0).stop 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 shutdown_connections 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f011406ad10 0x7f011406d1c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01300c1f90 0x7f013014b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01300c0140 0x7f0130159810 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 --2- 172.21.15.7:0/4218383162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01300c1bc0 0x7f01301e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 >> 172.21.15.7:0/4218383162 conn(0x7f013006a4d0 msgr2=0x7f01300c79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 shutdown_connections 2023-06-09T11:29:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:50.792+0000 7f0137bff700 1 -- 172.21.15.7:0/4218383162 wait complete. 2023-06-09T11:29:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 81 2023-06-09T11:29:50.815 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:52.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:53.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.021+0000 7fa19c9ff700 1 Processor -- start 2023-06-09T11:29:53.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.022+0000 7fa19c9ff700 1 -- start start 2023-06-09T11:29:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 0x7fa1980b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1980cb9a0 0x7fa198159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 0x7fa1980cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa198159b20 con 0x7fa1980cb9a0 2023-06-09T11:29:53.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980cac80 con 0x7fa1980cb5d0 2023-06-09T11:29:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa19c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981597d0 con 0x7fa19815cd40 2023-06-09T11:29:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa195d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1980cb9a0 0x7fa198159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa195d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1980cb9a0 0x7fa198159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42750/0 (socket says 172.21.15.7:42750) 2023-06-09T11:29:53.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.023+0000 7fa195d9b700 1 -- 172.21.15.7:0/2644270094 learned_addr learned my addr 172.21.15.7:0/2644270094 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 -- 172.21.15.7:0/2644270094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 msgr2=0x7fa1980cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa196d9d700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 0x7fa1980cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa19659c700 1 --2- 172.21.15.7:0/2644270094 >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 0x7fa1980b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 0x7fa1980cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 -- 172.21.15.7:0/2644270094 >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 msgr2=0x7fa1980b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 --2- 172.21.15.7:0/2644270094 >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 0x7fa1980b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 -- 172.21.15.7:0/2644270094 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198071ab0 con 0x7fa1980cb9a0 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa196d9d700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 0x7fa1980cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa19659c700 1 --2- 172.21.15.7:0/2644270094 >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 0x7fa1980b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:29:53.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa195d9b700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb9a0 0x7fa198159fb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa19808d420 tx=0x7fa18c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb7d1771b54d05c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa19559a700 1 -- 172.21.15.7:0/2644270094 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00b870 con 0x7fa1980cb9a0 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa19559a700 1 -- 172.21.15.7:0/2644270094 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c00b9d0 con 0x7fa1980cb9a0 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.024+0000 7fa19559a700 1 -- 172.21.15.7:0/2644270094 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00bd60 con 0x7fa1980cb9a0 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 -- 172.21.15.7:0/2644270094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb9a0 msgr2=0x7fa198159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb9a0 0x7fa198159fb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa19808d420 tx=0x7fa18c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 -- 172.21.15.7:0/2644270094 shutdown_connections 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/2644270094 >> v2:172.21.15.117:3300/0 conn(0x7fa1980cb5d0 0x7fa1980b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa19815cd40 0x7fa1980cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/2644270094 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb9a0 0x7fa198159fb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 -- 172.21.15.7:0/2644270094 >> 172.21.15.7:0/2644270094 conn(0x7fa198070af0 msgr2=0x7fa198070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 -- 172.21.15.7:0/2644270094 shutdown_connections 2023-06-09T11:29:53.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.025+0000 7fa19c9ff700 1 -- 172.21.15.7:0/2644270094 wait complete. 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.027+0000 7fa19c9ff700 1 Processor -- start 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.027+0000 7fa19c9ff700 1 -- start start 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 0x7fa1980c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 0x7fa1980c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981eff40 con 0x7fa1980cb5d0 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c4800 con 0x7fa19815cd40 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c4dc0 con 0x7fa1980cb9a0 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42764/0 (socket says 172.21.15.7:42764) 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 -- 172.21.15.7:0/28950157 learned_addr learned my addr 172.21.15.7:0/28950157 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa196d9d700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 0x7fa1980c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa195d9b700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 0x7fa1980c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 -- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 msgr2=0x7fa1980c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 0x7fa1980c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 -- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 msgr2=0x7fa1980c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 0x7fa1980c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa19659c700 1 -- 172.21.15.7:0/28950157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c009550 con 0x7fa1980cb5d0 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa195d9b700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 0x7fa1980c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.029+0000 7fa196d9d700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 0x7fa1980c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:53.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa19659c700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c740 tx=0x7fa18800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:53.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188011800 con 0x7fa1980cb5d0 2023-06-09T11:29:53.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188011960 con 0x7fa1980cb5d0 2023-06-09T11:29:53.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188011cd0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19815d4f0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.030+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19815d210 con 0x7fa1980cb5d0 2023-06-09T11:29:53.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.031+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19815d7d0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.032+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa188021350 con 0x7fa1980cb5d0 2023-06-09T11:29:53.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.033+0000 7fa186ffd700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 0x7fa17806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:53.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.033+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fa1880897c0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.034+0000 7fa195d9b700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 0x7fa17806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:53.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.034+0000 7fa195d9b700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 0x7fa17806d1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1980cbd70 tx=0x7fa18c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:53.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.038+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa188057ed0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.288+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1980c50a0 con 0x7fa1980cb5d0 2023-06-09T11:29:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.289+0000 7fa186ffd700 1 -- 172.21.15.7:0/28950157 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fa188057a60 con 0x7fa1980cb5d0 2023-06-09T11:29:53.294 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:53.294 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:51.515511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 msgr2=0x7fa17806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 0x7fa17806d1c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa1980cbd70 tx=0x7fa18c008030 comp rx=0 tx=0).stop 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 msgr2=0x7fa1981efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c740 tx=0x7fa18800cb00 comp rx=0 tx=0).stop 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 shutdown_connections 2023-06-09T11:29:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa19815cd40 0x7fa1980c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1980cb5d0 0x7fa1981efa00 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa17806ad10 0x7fa17806d1c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 --2- 172.21.15.7:0/28950157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1980cb9a0 0x7fa1980c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 >> 172.21.15.7:0/28950157 conn(0x7fa198070af0 msgr2=0x7fa1980cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 shutdown_connections 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:53.292+0000 7fa19c9ff700 1 -- 172.21.15.7:0/28950157 wait complete. 2023-06-09T11:29:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 82 2023-06-09T11:29:53.315 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:55.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.525+0000 7f62acbff700 1 Processor -- start 2023-06-09T11:29:55.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.525+0000 7f62acbff700 1 -- start start 2023-06-09T11:29:55.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.526+0000 7f62acbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 0x7f62a815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.526+0000 7f62acbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62a80c9f40 0x7f62a80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.526+0000 7f62acbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a815a500 con 0x7f62a8153880 2023-06-09T11:29:55.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.526+0000 7f62acbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80c5f10 con 0x7f62a80c9f40 2023-06-09T11:29:55.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.526+0000 7f62acbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a815da00 con 0x7f62a815b4a0 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62a8153880 0x7f62a815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40952/0 (socket says 172.21.15.7:40952) 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a659c700 1 -- 172.21.15.7:0/2672675849 learned_addr learned my addr 172.21.15.7:0/2672675849 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/2672675849 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 0x7f62a815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a6d9d700 1 --2- 172.21.15.7:0/2672675849 >> v2:172.21.15.117:3300/0 conn(0x7f62a80c9f40 0x7f62a80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a559a700 1 -- 172.21.15.7:0/2672675849 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284909621 0 0) 0x7f6298001110 con 0x7f62a8153880 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a559a700 1 -- 172.21.15.7:0/2672675849 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62900035b0 con 0x7f62a8153880 2023-06-09T11:29:55.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a5d9b700 1 -- 172.21.15.7:0/2672675849 >> v2:172.21.15.117:3300/0 conn(0x7f62a80c9f40 msgr2=0x7f62a80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/2672675849 >> v2:172.21.15.117:3300/0 conn(0x7f62a80c9f40 0x7f62a80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a5d9b700 1 -- 172.21.15.7:0/2672675849 >> v1:172.21.15.7:6789/0 conn(0x7f62a8153880 legacy=0x7f62a815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.527+0000 7f62a5d9b700 1 -- 172.21.15.7:0/2672675849 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a81596e0 con 0x7f62a815b4a0 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/2672675849 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 0x7f62a815d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f629c009840 tx=0x7f629c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b2fbe2a6a56556 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62a559a700 1 -- 172.21.15.7:0/2672675849 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c011830 con 0x7f62a815b4a0 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62a559a700 1 -- 172.21.15.7:0/2672675849 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c011990 con 0x7f62a815b4a0 2023-06-09T11:29:55.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62a559a700 1 -- 172.21.15.7:0/2672675849 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c011d00 con 0x7f62a815b4a0 2023-06-09T11:29:55.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62acbff700 1 -- 172.21.15.7:0/2672675849 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 msgr2=0x7f62a815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.528+0000 7f62acbff700 1 --2- 172.21.15.7:0/2672675849 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 0x7f62a815d4c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f629c009840 tx=0x7f629c009b50 comp rx=0 tx=0).stop 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 -- 172.21.15.7:0/2672675849 shutdown_connections 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 --2- 172.21.15.7:0/2672675849 >> v2:172.21.15.117:3300/0 conn(0x7f62a80c9f40 0x7f62a80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 --2- 172.21.15.7:0/2672675849 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a815b4a0 0x7f62a815d4c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 -- 172.21.15.7:0/2672675849 >> 172.21.15.7:0/2672675849 conn(0x7f62a80737f0 msgr2=0x7f62a80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 -- 172.21.15.7:0/2672675849 shutdown_connections 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.529+0000 7f62acbff700 1 -- 172.21.15.7:0/2672675849 wait complete. 2023-06-09T11:29:55.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.530+0000 7f62acbff700 1 Processor -- start 2023-06-09T11:29:55.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 -- start start 2023-06-09T11:29:55.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 0x7f62a81f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 0x7f62a81e6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80caa80 con 0x7f62a8153880 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80ce000 con 0x7f62a815b4a0 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.531+0000 7f62acbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a81e5ec0 con 0x7f62a80c9f40 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42778/0 (socket says 172.21.15.7:42778) 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 -- 172.21.15.7:0/967389321 learned_addr learned my addr 172.21.15.7:0/967389321 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a6d9d700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 0x7f62a81e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a659c700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 0x7f62a81f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 -- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 msgr2=0x7f62a81f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 0x7f62a81f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 -- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 msgr2=0x7f62a81e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 0x7f62a81e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 -- 172.21.15.7:0/967389321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c0094a0 con 0x7f62a8153880 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a659c700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 0x7f62a81f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:55.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a6d9d700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 0x7f62a81e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.532+0000 7f62a5d9b700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.533+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0036a0 con 0x7f62a8153880 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.533+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c01e070 con 0x7f62a8153880 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.533+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c003a40 con 0x7f62a8153880 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.533+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a8159c60 con 0x7f62a8153880 2023-06-09T11:29:55.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.533+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a8159980 con 0x7f62a8153880 2023-06-09T11:29:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.534+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8159f40 con 0x7f62a8153880 2023-06-09T11:29:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.535+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f629c003ba0 con 0x7f62a8153880 2023-06-09T11:29:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.535+0000 7f6296ffd700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 0x7f628806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:55.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.536+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f629c08a7a0 con 0x7f62a8153880 2023-06-09T11:29:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.536+0000 7f62a659c700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 0x7f628806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:55.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.536+0000 7f62a659c700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 0x7f628806d1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6298005910 tx=0x7f6298009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:55.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.540+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f629c0554a0 con 0x7f62a8153880 2023-06-09T11:29:55.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.791+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62a8074cf0 con 0x7f62a8153880 2023-06-09T11:29:55.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.792+0000 7f6296ffd700 1 -- 172.21.15.7:0/967389321 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f629c006cc0 con 0x7f62a8153880 2023-06-09T11:29:55.797 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:55.797 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":82,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:51.515511+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:rejoin","state_seq":28,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.794+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 msgr2=0x7f628806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.794+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 0x7f628806d1c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6298005910 tx=0x7f6298009040 comp rx=0 tx=0).stop 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.794+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 msgr2=0x7f62a81e5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f629c000c00 tx=0x7f629c004750 comp rx=0 tx=0).stop 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 shutdown_connections 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62a815b4a0 0x7f62a81e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f628806ad10 0x7f628806d1c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62a80c9f40 0x7f62a81f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 --2- 172.21.15.7:0/967389321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62a8153880 0x7f62a81e5030 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 >> 172.21.15.7:0/967389321 conn(0x7f62a80737f0 msgr2=0x7f62a814ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 shutdown_connections 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:55.795+0000 7f62acbff700 1 -- 172.21.15.7:0/967389321 wait complete. 2023-06-09T11:29:55.801 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 82 2023-06-09T11:29:55.818 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:29:57.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:29:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.025+0000 7f07eb9ff700 1 Processor -- start 2023-06-09T11:29:58.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.025+0000 7f07eb9ff700 1 -- start start 2023-06-09T11:29:58.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.026+0000 7f07eb9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.026+0000 7f07eb9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07eb9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e415a500 con 0x7f07e40c9f40 2023-06-09T11:29:58.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07eb9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e40c5f10 con 0x7f07e4153880 2023-06-09T11:29:58.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07eb9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e415da00 con 0x7f07e415b4a0 2023-06-09T11:29:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e979b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e979b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41394/0 (socket says 172.21.15.7:41394) 2023-06-09T11:29:58.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e979b700 1 -- 172.21.15.7:0/1947946549 learned_addr learned my addr 172.21.15.7:0/1947946549 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54582/0 (socket says 172.21.15.7:54582) 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 -- 172.21.15.7:0/1947946549 >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 msgr2=0x7f07e415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 --2- 172.21.15.7:0/1947946549 >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 -- 172.21.15.7:0/1947946549 >> v1:172.21.15.117:6789/0 conn(0x7f07e40c9f40 legacy=0x7f07e415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e9f9c700 1 -- 172.21.15.7:0/1947946549 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e41596e0 con 0x7f07e415b4a0 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.027+0000 7f07e979b700 1 --2- 172.21.15.7:0/1947946549 >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.028+0000 7f07e9f9c700 1 --2- 172.21.15.7:0/1947946549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009d90 tx=0x7f07dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f162b4c749263ce server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:58.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.028+0000 7f07dbfff700 1 -- 172.21.15.7:0/1947946549 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc00e070 con 0x7f07e415b4a0 2023-06-09T11:29:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.028+0000 7f07dbfff700 1 -- 172.21.15.7:0/1947946549 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07dc00bb20 con 0x7f07e415b4a0 2023-06-09T11:29:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.028+0000 7f07dbfff700 1 -- 172.21.15.7:0/1947946549 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07dc012430 con 0x7f07e415b4a0 2023-06-09T11:29:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.028+0000 7f07eb9ff700 1 -- 172.21.15.7:0/1947946549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 msgr2=0x7f07e40c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/1947946549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07dc009d90 tx=0x7f07dc00b330 comp rx=0 tx=0).stop 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 -- 172.21.15.7:0/1947946549 shutdown_connections 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/1947946549 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e40c4e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/1947946549 >> v2:172.21.15.117:3300/0 conn(0x7f07e4153880 0x7f07e415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 -- 172.21.15.7:0/1947946549 >> 172.21.15.7:0/1947946549 conn(0x7f07e40737f0 msgr2=0x7f07e40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 -- 172.21.15.7:0/1947946549 shutdown_connections 2023-06-09T11:29:58.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.029+0000 7f07eb9ff700 1 -- 172.21.15.7:0/1947946549 wait complete. 2023-06-09T11:29:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.031+0000 7f07eb9ff700 1 Processor -- start 2023-06-09T11:29:58.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.031+0000 7f07eb9ff700 1 -- start start 2023-06-09T11:29:58.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.032+0000 7f07eb9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07eb9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07e4153880 0x7f07e40cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07eb9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e4150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07eb9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41f2f60 con 0x7f07e40c9f40 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07eb9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41f3490 con 0x7f07e4153880 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07eb9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e41eefe0 con 0x7f07e415b4a0 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42784/0 (socket says 172.21.15.7:42784) 2023-06-09T11:29:58.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 -- 172.21.15.7:0/568304566 learned_addr learned my addr 172.21.15.7:0/568304566 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e9f9c700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e4150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e8f9a700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07e4153880 0x7f07e40cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 -- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 msgr2=0x7f07e4150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e4150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 -- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07e4153880 msgr2=0x7f07e40cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07e4153880 0x7f07e40cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e979b700 1 -- 172.21.15.7:0/568304566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0009550 con 0x7f07e40c9f40 2023-06-09T11:29:58.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.033+0000 7f07e9f9c700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e4150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:29:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07e979b700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ebe0 tx=0x7f07e0009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000c8b0 con 0x7f07e40c9f40 2023-06-09T11:29:58.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07e000ca10 con 0x7f07e40c9f40 2023-06-09T11:29:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc009a70 con 0x7f07e40c9f40 2023-06-09T11:29:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e000cd80 con 0x7f07e40c9f40 2023-06-09T11:29:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.034+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e4159c60 con 0x7f07e40c9f40 2023-06-09T11:29:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.035+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e4159980 con 0x7f07e40c9f40 2023-06-09T11:29:58.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.036+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07e0013470 con 0x7f07e40c9f40 2023-06-09T11:29:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.037+0000 7f07d9ffb700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 0x7f07c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:29:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.037+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f07e0089e40 con 0x7f07e40c9f40 2023-06-09T11:29:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.037+0000 7f07e8f9a700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 0x7f07c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:29:58.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.038+0000 7f07e8f9a700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 0x7f07c806d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f07e41e69a0 tx=0x7f07d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:29:58.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.041+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f07e0054a30 con 0x7f07e40c9f40 2023-06-09T11:29:58.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.291+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07e4159f40 con 0x7f07e40c9f40 2023-06-09T11:29:58.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.291+0000 7f07d9ffb700 1 -- 172.21.15.7:0/568304566 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f07e0058050 con 0x7f07e40c9f40 2023-06-09T11:29:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:29:58.297 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 msgr2=0x7f07c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 0x7f07c806d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f07e41e69a0 tx=0x7f07d4009040 comp rx=0 tx=0).stop 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 msgr2=0x7f07e40ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f07e000ebe0 tx=0x7f07e0009e70 comp rx=0 tx=0).stop 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 shutdown_connections 2023-06-09T11:29:58.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f07c806ace0 0x7f07c806d190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07e415b4a0 0x7f07e4150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07e40c9f40 0x7f07e40ce6d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 --2- 172.21.15.7:0/568304566 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07e4153880 0x7f07e40cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 >> 172.21.15.7:0/568304566 conn(0x7f07e40737f0 msgr2=0x7f07e40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 shutdown_connections 2023-06-09T11:29:58.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:29:58.294+0000 7f07eb9ff700 1 -- 172.21.15.7:0/568304566 wait complete. 2023-06-09T11:29:58.301 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 83 2023-06-09T11:29:58.317 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=5587 2023-06-09T11:29:58.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.b 2023-06-09T11:30:00.318 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:30:00.319 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:30:00.319 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:30:00.323 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:30:00.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:00.479 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:30:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.535+0000 7f6b049ff700 1 Processor -- start 2023-06-09T11:30:00.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.535+0000 7f6b049ff700 1 -- start start 2023-06-09T11:30:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.536+0000 7f6b049ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6b000c1bc0 0x7f6b001599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.536+0000 7f6b049ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.536+0000 7f6b049ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b000c0bf0 con 0x7f6b000c0140 2023-06-09T11:30:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.536+0000 7f6b049ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b00070110 con 0x7f6b000c1bc0 2023-06-09T11:30:00.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.536+0000 7f6b049ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b00071300 con 0x7f6b000c1f90 2023-06-09T11:30:00.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6b000c1bc0 0x7f6b001599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afed9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afed9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54590/0 (socket says 172.21.15.7:54590) 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6b000c1bc0 0x7f6b001599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41404/0 (socket says 172.21.15.7:41404) 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afed9d700 1 -- 172.21.15.7:0/331503330 learned_addr learned my addr 172.21.15.7:0/331503330 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afdd9b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f6b000c0140 0x7f6b001501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:49694/0 (socket says 172.21.15.7:49694) 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 -- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 msgr2=0x7f6b00159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 -- 172.21.15.7:0/331503330 >> v1:172.21.15.117:6789/0 conn(0x7f6b000c0140 legacy=0x7f6b001501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 -- 172.21.15.7:0/331503330 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b000702f0 con 0x7f6b000c1bc0 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afed9d700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afd59a700 1 -- 172.21.15.7:0/331503330 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719534686 0 0) 0x7f6b000c0bf0 con 0x7f6b000c0140 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.537+0000 7f6afe59c700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001599b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009840 tx=0x7f6af0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4d18df2c076511 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6afd59a700 1 -- 172.21.15.7:0/331503330 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0011830 con 0x7f6b000c1bc0 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6afd59a700 1 -- 172.21.15.7:0/331503330 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0011990 con 0x7f6b000c1bc0 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6afd59a700 1 -- 172.21.15.7:0/331503330 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af0011d20 con 0x7f6b000c1bc0 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 -- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 msgr2=0x7f6b001599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001599b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6af0009840 tx=0x7f6af0009b50 comp rx=0 tx=0).stop 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 -- 172.21.15.7:0/331503330 shutdown_connections 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c1f90 0x7f6b00159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 --2- 172.21.15.7:0/331503330 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001599b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 -- 172.21.15.7:0/331503330 >> 172.21.15.7:0/331503330 conn(0x7f6b0006a4d0 msgr2=0x7f6b0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.538+0000 7f6b049ff700 1 -- 172.21.15.7:0/331503330 shutdown_connections 2023-06-09T11:30:00.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.539+0000 7f6b049ff700 1 -- 172.21.15.7:0/331503330 wait complete. 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.540+0000 7f6b049ff700 1 Processor -- start 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.540+0000 7f6b049ff700 1 -- start start 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c0140 0x7f6b0014d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b001ee980 con 0x7f6b000c1f90 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000cb3f0 con 0x7f6b000c1bc0 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6b049ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b000c9d20 con 0x7f6b000c0140 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6afed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.541+0000 7f6afed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42796/0 (socket says 172.21.15.7:42796) 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 -- 172.21.15.7:0/2066235292 learned_addr learned my addr 172.21.15.7:0/2066235292 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afe59c700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c0140 0x7f6b0014d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afdd9b700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 -- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c0140 msgr2=0x7f6b0014d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c0140 0x7f6b0014d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 -- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 msgr2=0x7f6b001ebc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001ebc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 -- 172.21.15.7:0/2066235292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af800bd40 con 0x7f6b000c1f90 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6afed9d700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e0f0 tx=0x7f6af800e400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:00.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af800b520 con 0x7f6b000c1f90 2023-06-09T11:30:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af8010030 con 0x7f6b000c1f90 2023-06-09T11:30:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af8003c20 con 0x7f6b000c1f90 2023-06-09T11:30:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.542+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af00094a0 con 0x7f6b000c1f90 2023-06-09T11:30:00.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.543+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b000c1170 con 0x7f6b000c1f90 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.544+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b000c0e90 con 0x7f6b000c1f90 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.545+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6af800ea00 con 0x7f6b000c1f90 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.546+0000 7f6aeeffd700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 0x7f6ae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.546+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f6af808bd10 con 0x7f6b000c1f90 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.546+0000 7f6afe59c700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 0x7f6ae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:00.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.547+0000 7f6afe59c700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 0x7f6ae006d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af00090b0 tx=0x7f6af0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:00.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.551+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6af8091020 con 0x7f6b000c1f90 2023-06-09T11:30:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.802+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b000c1450 con 0x7f6b000c1f90 2023-06-09T11:30:00.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.803+0000 7f6aeeffd700 1 -- 172.21.15.7:0/2066235292 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f6af805a4a0 con 0x7f6b000c1f90 2023-06-09T11:30:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:00.808 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:00.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 msgr2=0x7f6ae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 0x7f6ae006d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6af00090b0 tx=0x7f6af0008030 comp rx=0 tx=0).stop 2023-06-09T11:30:00.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 msgr2=0x7f6b001ec1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:00.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6af800e0f0 tx=0x7f6af800e400 comp rx=0 tx=0).stop 2023-06-09T11:30:00.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 shutdown_connections 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b000c1f90 0x7f6b001ec1c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6ae006ace0 0x7f6ae006d190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b000c0140 0x7f6b0014d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 --2- 172.21.15.7:0/2066235292 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b000c1bc0 0x7f6b001ebc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 >> 172.21.15.7:0/2066235292 conn(0x7f6b0006a4d0 msgr2=0x7f6b0014e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 shutdown_connections 2023-06-09T11:30:00.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:00.805+0000 7f6b049ff700 1 -- 172.21.15.7:0/2066235292 wait complete. 2023-06-09T11:30:00.812 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 83 2023-06-09T11:30:00.828 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:30:02.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.045+0000 7fa57c3ff700 1 Processor -- start 2023-06-09T11:30:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.045+0000 7fa57c3ff700 1 -- start start 2023-06-09T11:30:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 0x7fa57415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57c3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57415a390 con 0x7fa574153b30 2023-06-09T11:30:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57c3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5740ca490 con 0x7fa574153760 2023-06-09T11:30:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5740c5f40 con 0x7fa5740ca070 2023-06-09T11:30:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57999a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa574153b30 0x7fa574071fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38614/0 (socket says 172.21.15.7:38614) 2023-06-09T11:30:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57999a700 1 -- 172.21.15.7:0/197737542 learned_addr learned my addr 172.21.15.7:0/197737542 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.047+0000 7fa57a99c700 1 --2- 172.21.15.7:0/197737542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 0x7fa57415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:03.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041511543 0 0) 0x7fa57415a390 con 0x7fa574153b30 2023-06-09T11:30:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5600035b0 con 0x7fa574153b30 2023-06-09T11:30:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396272435 0 0) 0x7fa5740ca490 con 0x7fa574153760 2023-06-09T11:30:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57415a390 con 0x7fa574153760 2023-06-09T11:30:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa57a99c700 1 -- 172.21.15.7:0/197737542 >> v1:172.21.15.117:6789/0 conn(0x7fa574153760 legacy=0x7fa5740b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa57a99c700 1 -- 172.21.15.7:0/197737542 >> v1:172.21.15.7:6789/0 conn(0x7fa574153b30 legacy=0x7fa574071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa57a99c700 1 -- 172.21.15.7:0/197737542 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5740ca720 con 0x7fa5740ca070 2023-06-09T11:30:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa57a99c700 1 --2- 172.21.15.7:0/197737542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 0x7fa57415cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5740ca910 tx=0x7fa56c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7ea8fb5718fee3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c016070 con 0x7fa5740ca070 2023-06-09T11:30:03.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00bb20 con 0x7fa5740ca070 2023-06-09T11:30:03.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.048+0000 7fa579199700 1 -- 172.21.15.7:0/197737542 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00e070 con 0x7fa5740ca070 2023-06-09T11:30:03.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 -- 172.21.15.7:0/197737542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 msgr2=0x7fa57415cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/197737542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 0x7fa57415cc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5740ca910 tx=0x7fa56c00b330 comp rx=0 tx=0).stop 2023-06-09T11:30:03.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 -- 172.21.15.7:0/197737542 shutdown_connections 2023-06-09T11:30:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/197737542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5740ca070 0x7fa57415cc70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 -- 172.21.15.7:0/197737542 >> 172.21.15.7:0/197737542 conn(0x7fa5740737f0 msgr2=0x7fa574073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 -- 172.21.15.7:0/197737542 shutdown_connections 2023-06-09T11:30:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.049+0000 7fa57c3ff700 1 -- 172.21.15.7:0/197737542 wait complete. 2023-06-09T11:30:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.051+0000 7fa57c3ff700 1 Processor -- start 2023-06-09T11:30:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.051+0000 7fa57c3ff700 1 -- start start 2023-06-09T11:30:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5740ca070 0x7fa57415b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa574153760 0x7fa5741574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:03.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57415c2f0 con 0x7fa574153b30 2023-06-09T11:30:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414c950 con 0x7fa5740ca070 2023-06-09T11:30:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.052+0000 7fa57c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414cc30 con 0x7fa574153760 2023-06-09T11:30:03.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57999a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa574153760 0x7fa5741574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49494/0 (socket says 172.21.15.7:49494) 2023-06-09T11:30:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 -- 172.21.15.7:0/1238906289 learned_addr learned my addr 172.21.15.7:0/1238906289 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a19b700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5740ca070 0x7fa57415b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 -- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa574153760 msgr2=0x7fa5741574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa574153760 0x7fa5741574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 -- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5740ca070 msgr2=0x7fa57415b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5740ca070 0x7fa57415b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 -- 172.21.15.7:0/1238906289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c009a70 con 0x7fa574153b30 2023-06-09T11:30:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.053+0000 7fa57a99c700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa56c014040 tx=0x7fa56c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.054+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c016040 con 0x7fa574153b30 2023-06-09T11:30:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.054+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c004860 con 0x7fa574153b30 2023-06-09T11:30:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.054+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c01b680 con 0x7fa574153b30 2023-06-09T11:30:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.054+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa574159780 con 0x7fa574153b30 2023-06-09T11:30:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.054+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5740c5a20 con 0x7fa574153b30 2023-06-09T11:30:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.055+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa574159a60 con 0x7fa574153b30 2023-06-09T11:30:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.056+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa56c01b7e0 con 0x7fa574153b30 2023-06-09T11:30:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.057+0000 7fa56affd700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.057+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fa56c08adc0 con 0x7fa574153b30 2023-06-09T11:30:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.058+0000 7fa57a19b700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.058+0000 7fa57a19b700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 0x7fa55806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa5700011e0 tx=0x7fa570009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.062+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa56c055b50 con 0x7fa574153b30 2023-06-09T11:30:03.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.393+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa57415c430 con 0x7fa574153b30 2023-06-09T11:30:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.394+0000 7fa56affd700 1 -- 172.21.15.7:0/1238906289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fa56c08a020 con 0x7fa574153b30 2023-06-09T11:30:03.400 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:03.400 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 msgr2=0x7fa55806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 0x7fa55806d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa5700011e0 tx=0x7fa570009040 comp rx=0 tx=0).stop 2023-06-09T11:30:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 msgr2=0x7fa5740b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa56c014040 tx=0x7fa56c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 shutdown_connections 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa574153b30 0x7fa5740b46b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa5740ca070 0x7fa57415b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55806ace0 0x7fa55806d190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 --2- 172.21.15.7:0/1238906289 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa574153760 0x7fa5741574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 >> 172.21.15.7:0/1238906289 conn(0x7fa5740737f0 msgr2=0x7fa57414f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 shutdown_connections 2023-06-09T11:30:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:03.397+0000 7fa57c3ff700 1 -- 172.21.15.7:0/1238906289 wait complete. 2023-06-09T11:30:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:03.425 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:30:03.426 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 15 secs before thrashing 2023-06-09T11:30:06.518 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:30:06.512+0000 7f0f311f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:30:06.518 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:30:06.512+0000 7f0f311f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:30:06.518 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:30:06.512+0000 7f0f311f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:30:15.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:15.942 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:18.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.675+0000 7f59711ff700 1 Processor -- start 2023-06-09T11:30:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.675+0000 7f59711ff700 1 -- start start 2023-06-09T11:30:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f59711ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 0x7f596c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f59711ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c06a4d0 con 0x7f596c0bf5d0 2023-06-09T11:30:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f59711ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0cc280 con 0x7f596c073d80 2023-06-09T11:30:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f59711ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c073a10 con 0x7f596c074150 2023-06-09T11:30:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f596a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f596c0bf5d0 0x7f596c1534b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51512/0 (socket says 172.21.15.7:51512) 2023-06-09T11:30:18.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f596a59c700 1 -- 172.21.15.7:0/2771749561 learned_addr learned my addr 172.21.15.7:0/2771749561 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f596ad9d700 1 --1- 172.21.15.7:0/2771749561 >> v1:172.21.15.117:6789/0 conn(0x7f596c073d80 0x7f596c06b6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:35352/0 (socket says 172.21.15.7:35352) 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.677+0000 7f596b59e700 1 --2- 172.21.15.7:0/2771749561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 0x7f596c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.678+0000 7f5969d9b700 1 -- 172.21.15.7:0/2771749561 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930662076 0 0) 0x7f596c0cc280 con 0x7f596c073d80 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.678+0000 7f596b59e700 1 -- 172.21.15.7:0/2771749561 >> v1:172.21.15.117:6789/0 conn(0x7f596c073d80 legacy=0x7f596c06b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.678+0000 7f596b59e700 1 -- 172.21.15.7:0/2771749561 >> v1:172.21.15.7:6789/0 conn(0x7f596c0bf5d0 legacy=0x7f596c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:18.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.678+0000 7f596b59e700 1 -- 172.21.15.7:0/2771749561 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c06bdc0 con 0x7f596c074150 2023-06-09T11:30:18.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.678+0000 7f596b59e700 1 --2- 172.21.15.7:0/2771749561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 0x7f596c06a960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5964009dc0 tx=0x7f596400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473fc00137af52f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:18.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f5969d9b700 1 -- 172.21.15.7:0/2771749561 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400e070 con 0x7f596c074150 2023-06-09T11:30:18.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f5969d9b700 1 -- 172.21.15.7:0/2771749561 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596400bad0 con 0x7f596c074150 2023-06-09T11:30:18.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f5969d9b700 1 -- 172.21.15.7:0/2771749561 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400be80 con 0x7f596c074150 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f59711ff700 1 -- 172.21.15.7:0/2771749561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 msgr2=0x7f596c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f59711ff700 1 --2- 172.21.15.7:0/2771749561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 0x7f596c06a960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5964009dc0 tx=0x7f596400b330 comp rx=0 tx=0).stop 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f59711ff700 1 -- 172.21.15.7:0/2771749561 shutdown_connections 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f59711ff700 1 --2- 172.21.15.7:0/2771749561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c074150 0x7f596c06a960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.679+0000 7f59711ff700 1 -- 172.21.15.7:0/2771749561 >> 172.21.15.7:0/2771749561 conn(0x7f596c06d150 msgr2=0x7f596c06d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.680+0000 7f59711ff700 1 -- 172.21.15.7:0/2771749561 shutdown_connections 2023-06-09T11:30:18.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.680+0000 7f59711ff700 1 -- 172.21.15.7:0/2771749561 wait complete. 2023-06-09T11:30:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.682+0000 7f59711ff700 1 Processor -- start 2023-06-09T11:30:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.682+0000 7f59711ff700 1 -- start start 2023-06-09T11:30:18.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596c074150 0x7f596c158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c0bf5d0 0x7f596c14ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1efaa0 con 0x7f596c074150 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0c1e40 con 0x7f596c073d80 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.683+0000 7f59711ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0c22d0 con 0x7f596c0bf5d0 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:18.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:48800/0 (socket says 172.21.15.7:48800) 2023-06-09T11:30:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 -- 172.21.15.7:0/82687413 learned_addr learned my addr 172.21.15.7:0/82687413 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596a59c700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596c074150 0x7f596c158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596b59e700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c0bf5d0 0x7f596c14ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 -- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c0bf5d0 msgr2=0x7f596c14ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c0bf5d0 0x7f596c14ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 -- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596c074150 msgr2=0x7f596c158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596c074150 0x7f596c158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:18.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 -- 172.21.15.7:0/82687413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009a70 con 0x7f596c073d80 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.684+0000 7f596ad9d700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c520 tx=0x7f595c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.685+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c004c80 con 0x7f596c073d80 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.685+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c004e00 con 0x7f596c073d80 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.685+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c0cc4d0 con 0x7f596c073d80 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.685+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c007310 con 0x7f596c073d80 2023-06-09T11:30:18.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.685+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c0c8700 con 0x7f596c073d80 2023-06-09T11:30:18.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.686+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c0cc7b0 con 0x7f596c073d80 2023-06-09T11:30:18.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.687+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f595c007470 con 0x7f596c073d80 2023-06-09T11:30:18.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.688+0000 7f59537fe700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 0x7f595806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:18.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.688+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f595c0894b0 con 0x7f596c073d80 2023-06-09T11:30:18.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.689+0000 7f596a59c700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 0x7f595806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:18.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.689+0000 7f596a59c700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 0x7f595806d1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5960007c20 tx=0x7f5960007b70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:18.693+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f595c054120 con 0x7f596c073d80 2023-06-09T11:30:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.020+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f596c074cf0 con 0x7f596c073d80 2023-06-09T11:30:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.021+0000 7f59537fe700 1 -- 172.21.15.7:0/82687413 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f595c057740 con 0x7f596c073d80 2023-06-09T11:30:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:19.027 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 msgr2=0x7f595806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 0x7f595806d1c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5960007c20 tx=0x7f5960007b70 comp rx=0 tx=0).stop 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 msgr2=0x7f596c0c4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f595c00c520 tx=0x7f595c00c830 comp rx=0 tx=0).stop 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 shutdown_connections 2023-06-09T11:30:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f595806ad10 0x7f595806d1c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f596c0bf5d0 0x7f596c14ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f596c073d80 0x7f596c0c4d00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 --2- 172.21.15.7:0/82687413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f596c074150 0x7f596c158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 >> 172.21.15.7:0/82687413 conn(0x7f596c06d150 msgr2=0x7f596c06a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 shutdown_connections 2023-06-09T11:30:19.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.024+0000 7f59711ff700 1 -- 172.21.15.7:0/82687413 wait complete. 2023-06-09T11:30:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:19.053 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2023-06-09T11:30:19.053 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:30:19.053 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:30:19.071 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:30:19.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.269+0000 7f322e5ff700 1 Processor -- start 2023-06-09T11:30:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.270+0000 7f322e5ff700 1 -- start start 2023-06-09T11:30:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322e5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3228153880 0x7f322815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 0x7f322815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322e5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322815a500 con 0x7f322815b2d0 2023-06-09T11:30:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322e5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32280c5f10 con 0x7f3228153880 2023-06-09T11:30:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322815da00 con 0x7f32280c9f40 2023-06-09T11:30:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322cb9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f322815b2d0 0x7f32280c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51516/0 (socket says 172.21.15.7:51516) 2023-06-09T11:30:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f322cb9c700 1 -- 172.21.15.7:0/1867755478 learned_addr learned my addr 172.21.15.7:0/1867755478 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f3227fff700 1 --2- 172.21.15.7:0/1867755478 >> v2:172.21.15.117:3300/0 conn(0x7f3228153880 0x7f322815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.271+0000 7f32277fe700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 0x7f322815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3226ffd700 1 -- 172.21.15.7:0/1867755478 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204130230 0 0) 0x7f3220001490 con 0x7f322815b2d0 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3227fff700 1 -- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 msgr2=0x7f322815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3227fff700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 0x7f322815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3227fff700 1 -- 172.21.15.7:0/1867755478 >> v1:172.21.15.7:6789/0 conn(0x7f322815b2d0 legacy=0x7f32280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3227fff700 1 -- 172.21.15.7:0/1867755478 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322815b750 con 0x7f3228153880 2023-06-09T11:30:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f32277fe700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 0x7f322815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:30:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3227fff700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f322815d4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3218009840 tx=0x7f3218009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21d480b7054b7fe6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3226ffd700 1 -- 172.21.15.7:0/1867755478 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011830 con 0x7f3228153880 2023-06-09T11:30:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3226ffd700 1 -- 172.21.15.7:0/1867755478 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3218011990 con 0x7f3228153880 2023-06-09T11:30:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.272+0000 7f3226ffd700 1 -- 172.21.15.7:0/1867755478 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218011d20 con 0x7f3228153880 2023-06-09T11:30:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 -- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 msgr2=0x7f322815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f322815d4c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3218009840 tx=0x7f3218009b50 comp rx=0 tx=0).stop 2023-06-09T11:30:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 -- 172.21.15.7:0/1867755478 shutdown_connections 2023-06-09T11:30:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f322815d4c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1867755478 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32280c9f40 0x7f322815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 -- 172.21.15.7:0/1867755478 >> 172.21.15.7:0/1867755478 conn(0x7f32280737f0 msgr2=0x7f32280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 -- 172.21.15.7:0/1867755478 shutdown_connections 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.273+0000 7f322e5ff700 1 -- 172.21.15.7:0/1867755478 wait complete. 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.275+0000 7f322e5ff700 1 Processor -- start 2023-06-09T11:30:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.275+0000 7f322e5ff700 1 -- start start 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f32281e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322815b2d0 0x7f32281e47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322814cc00 con 0x7f32280c9f40 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32281ef930 con 0x7f3228153880 2023-06-09T11:30:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322815c500 con 0x7f322815b2d0 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f3227fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f3227fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45168/0 (socket says 172.21.15.7:45168) 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f3227fff700 1 -- 172.21.15.7:0/1820708006 learned_addr learned my addr 172.21.15.7:0/1820708006 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f32277fe700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f32281e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.276+0000 7f322cb9c700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322815b2d0 0x7f32281e47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 -- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322815b2d0 msgr2=0x7f32281e47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322815b2d0 0x7f32281e47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 -- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 msgr2=0x7f32281e4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f32281e4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 -- 172.21.15.7:0/1820708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0094a0 con 0x7f32280c9f40 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3227fff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3218009b90 tx=0x7f3218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32180036a0 con 0x7f32280c9f40 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321801f070 con 0x7f32280c9f40 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32180094a0 con 0x7f32280c9f40 2023-06-09T11:30:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3218003a40 con 0x7f32280c9f40 2023-06-09T11:30:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.277+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3228159980 con 0x7f32280c9f40 2023-06-09T11:30:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.278+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32281596e0 con 0x7f32280c9f40 2023-06-09T11:30:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.279+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3218003ba0 con 0x7f32280c9f40 2023-06-09T11:30:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.281+0000 7f3224ff9700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 0x7f321406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.281+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f3218021070 con 0x7f32280c9f40 2023-06-09T11:30:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.281+0000 7f32277fe700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 0x7f321406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.282+0000 7f32277fe700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 0x7f321406d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f322814fae0 tx=0x7f321c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.286+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3218055c80 con 0x7f32280c9f40 2023-06-09T11:30:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.555+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32281e40b0 con 0x7f32280c9f40 2023-06-09T11:30:19.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.555+0000 7f3224ff9700 1 -- 172.21.15.7:0/1820708006 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f32180592a0 con 0x7f32280c9f40 2023-06-09T11:30:19.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:19.562 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:19.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.557+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 msgr2=0x7f321406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.557+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 0x7f321406d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f322814fae0 tx=0x7f321c009210 comp rx=0 tx=0).stop 2023-06-09T11:30:19.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.557+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 msgr2=0x7f32280b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:19.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.557+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3218009b90 tx=0x7f3218004750 comp rx=0 tx=0).stop 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 shutdown_connections 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f321406ace0 0x7f321406d190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f322815b2d0 0x7f32281e47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32280c9f40 0x7f32280b3120 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 --2- 172.21.15.7:0/1820708006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3228153880 0x7f32281e4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 >> 172.21.15.7:0/1820708006 conn(0x7f32280737f0 msgr2=0x7f32280cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 shutdown_connections 2023-06-09T11:30:19.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:19.558+0000 7f322e5ff700 1 -- 172.21.15.7:0/1820708006 wait complete. 2023-06-09T11:30:19.566 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:19.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:21.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.788+0000 7f160e3ff700 1 Processor -- start 2023-06-09T11:30:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.788+0000 7f160e3ff700 1 -- start start 2023-06-09T11:30:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.789+0000 7f160e3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16080cb5d0 0x7f16080b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.789+0000 7f160e3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1608159b20 con 0x7f160815cd40 2023-06-09T11:30:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.789+0000 7f160e3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16080cac80 con 0x7f16080cb9a0 2023-06-09T11:30:21.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.789+0000 7f160e3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16081597d0 con 0x7f16080cb5d0 2023-06-09T11:30:21.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f160c99c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f160815cd40 0x7f16080cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51522/0 (socket says 172.21.15.7:51522) 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f160c99c700 1 -- 172.21.15.7:0/3494293295 learned_addr learned my addr 172.21.15.7:0/3494293295 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1607fff700 1 --2- 172.21.15.7:0/3494293295 >> v2:172.21.15.117:3300/0 conn(0x7f16080cb5d0 0x7f16080b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601835929 0 0) 0x7f1608159b20 con 0x7f160815cd40 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15f00035b0 con 0x7f160815cd40 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1336245271 0 0) 0x7f15f00035b0 con 0x7f160815cd40 2023-06-09T11:30:21.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 >> v2:172.21.15.117:3300/0 conn(0x7f16080cb5d0 msgr2=0x7f16080b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 --2- 172.21.15.7:0/3494293295 >> v2:172.21.15.117:3300/0 conn(0x7f16080cb5d0 0x7f16080b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 >> v1:172.21.15.117:6789/0 conn(0x7f16080cb9a0 legacy=0x7f1608159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608071ab0 con 0x7f160815cd40 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.790+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f16000019f0 con 0x7f160815cd40 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 >> v1:172.21.15.7:6789/0 conn(0x7f160815cd40 legacy=0x7f16080cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f0003a00 0x7f15f0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f15f0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f0005200 0x7f15f000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16081597d0 con 0x7f15f0004850 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f000c060 con 0x7f15f0005200 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f000c220 con 0x7f15f0003a00 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f15f0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f1607fff700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f0003a00 0x7f15f0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f160c99c700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f0005200 0x7f15f000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 -- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f0003a00 msgr2=0x7f15f0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:21.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f15f0003a00 0x7f15f0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 -- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f0005200 msgr2=0x7f15f000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f15f0005200 0x7f15f000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 -- 172.21.15.7:0/3494293295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00035b0 con 0x7f15f0004850 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f160c99c700 1 -- 172.21.15.7:0/3494293295 reap_dead start 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.791+0000 7f16077fe700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f15f0004cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16080cbd70 tx=0x7f15f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94e3142f3d24139 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0052d0 con 0x7f15f0004850 2023-06-09T11:30:21.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15fc005430 con 0x7f15f0004850 2023-06-09T11:30:21.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f1606ffd700 1 -- 172.21.15.7:0/3494293295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0038c0 con 0x7f15f0004850 2023-06-09T11:30:21.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 -- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 msgr2=0x7f15f0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f15f0004cc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16080cbd70 tx=0x7f15f0005880 comp rx=0 tx=0).stop 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 -- 172.21.15.7:0/3494293295 shutdown_connections 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 --2- 172.21.15.7:0/3494293295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f15f0004cc0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 -- 172.21.15.7:0/3494293295 >> 172.21.15.7:0/3494293295 conn(0x7f1608070af0 msgr2=0x7f1608070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 -- 172.21.15.7:0/3494293295 shutdown_connections 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.792+0000 7f160e3ff700 1 -- 172.21.15.7:0/3494293295 wait complete. 2023-06-09T11:30:21.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.794+0000 7f160e3ff700 1 Processor -- start 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.794+0000 7f160e3ff700 1 -- start start 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 0x7f16081523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 0x7f1608152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1608153910 con 0x7f15f0004850 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1608151f50 con 0x7f160815c250 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.795+0000 7f160e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16081ef830 con 0x7f160815be00 2023-06-09T11:30:21.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45194/0 (socket says 172.21.15.7:45194) 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 -- 172.21.15.7:0/1456722894 learned_addr learned my addr 172.21.15.7:0/1456722894 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f16077fe700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 0x7f16081523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 -- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 msgr2=0x7f16081523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f160c99c700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 0x7f1608152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 0x7f16081523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 -- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 msgr2=0x7f1608152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 0x7f1608152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 -- 172.21.15.7:0/1456722894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f160815d4f0 con 0x7f15f0004850 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f16077fe700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 0x7f16081523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f160c99c700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 0x7f1608152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1607fff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f15f8011900 tx=0x7f15f8011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.796+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8004240 con 0x7f15f0004850 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.797+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f80043a0 con 0x7f15f0004850 2023-06-09T11:30:21.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.797+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800f730 con 0x7f15f0004850 2023-06-09T11:30:21.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.797+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160815d210 con 0x7f15f0004850 2023-06-09T11:30:21.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.797+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160815d7d0 con 0x7f15f0004850 2023-06-09T11:30:21.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.798+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1608074cf0 con 0x7f15f0004850 2023-06-09T11:30:21.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.799+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15f800f890 con 0x7f15f0004850 2023-06-09T11:30:21.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.799+0000 7f1604ff9700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:21.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.800+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f15f8089d10 con 0x7f15f0004850 2023-06-09T11:30:21.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.800+0000 7f16077fe700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:21.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.800+0000 7f16077fe700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 0x7f15f406d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f160815dc90 tx=0x7f15fc009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:21.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:21.804+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f15f808f020 con 0x7f15f0004850 2023-06-09T11:30:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.057+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1608152090 con 0x7f15f0004850 2023-06-09T11:30:22.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.057+0000 7f1604ff9700 1 -- 172.21.15.7:0/1456722894 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f15f8019090 con 0x7f15f0004850 2023-06-09T11:30:22.064 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:22.064 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:22.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 msgr2=0x7f15f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:22.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 0x7f15f406d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f160815dc90 tx=0x7f15fc009030 comp rx=0 tx=0).stop 2023-06-09T11:30:22.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 msgr2=0x7f16081ef2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:22.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f15f8011900 tx=0x7f15f8011cc0 comp rx=0 tx=0).stop 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 shutdown_connections 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f160815c250 0x7f1608152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f15f0004850 0x7f16081ef2f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f15f406ace0 0x7f15f406d190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 --2- 172.21.15.7:0/1456722894 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f160815be00 0x7f16081523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 >> 172.21.15.7:0/1456722894 conn(0x7f1608070af0 msgr2=0x7f16080c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:22.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 shutdown_connections 2023-06-09T11:30:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:22.060+0000 7f160e3ff700 1 -- 172.21.15.7:0/1456722894 wait complete. 2023-06-09T11:30:22.068 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:22.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:24.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:24.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.289+0000 7f6b7e1ff700 1 Processor -- start 2023-06-09T11:30:24.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.289+0000 7f6b7e1ff700 1 -- start start 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b7e1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b7815b4a0 0x7f6b7815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b7e1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6b780c9f40 0x7f6b780c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b7e1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7815a500 con 0x7f6b78153880 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b7e1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b780c5f10 con 0x7f6b7815b4a0 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b7e1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7815da00 con 0x7f6b780c9f40 2023-06-09T11:30:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b76ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6b7815b4a0 0x7f6b7815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:24.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6b78153880 0x7f6b7815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57926/0 (socket says 172.21.15.7:57926) 2023-06-09T11:30:24.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b777fe700 1 -- 172.21.15.7:0/2668289572 learned_addr learned my addr 172.21.15.7:0/2668289572 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:24.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b77fff700 1 --2- 172.21.15.7:0/2668289572 >> v2:172.21.15.117:3300/0 conn(0x7f6b780c9f40 0x7f6b780c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:24.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.291+0000 7f6b767fc700 1 -- 172.21.15.7:0/2668289572 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764046355 0 0) 0x7f6b68001110 con 0x7f6b78153880 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b76ffd700 1 -- 172.21.15.7:0/2668289572 >> v2:172.21.15.117:3300/0 conn(0x7f6b780c9f40 msgr2=0x7f6b780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b76ffd700 1 --2- 172.21.15.7:0/2668289572 >> v2:172.21.15.117:3300/0 conn(0x7f6b780c9f40 0x7f6b780c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b76ffd700 1 -- 172.21.15.7:0/2668289572 >> v1:172.21.15.7:6789/0 conn(0x7f6b78153880 legacy=0x7f6b7815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b76ffd700 1 -- 172.21.15.7:0/2668289572 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b781596e0 con 0x7f6b7815b4a0 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b76ffd700 1 --2- 172.21.15.7:0/2668289572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b7815b4a0 0x7f6b7815d4c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6b780749f0 tx=0x7f6b6c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb11bd2f67298c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b767fc700 1 -- 172.21.15.7:0/2668289572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c011830 con 0x7f6b7815b4a0 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b767fc700 1 -- 172.21.15.7:0/2668289572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c011990 con 0x7f6b7815b4a0 2023-06-09T11:30:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.292+0000 7f6b767fc700 1 -- 172.21.15.7:0/2668289572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c011d20 con 0x7f6b7815b4a0 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/2668289572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b7815b4a0 msgr2=0x7f6b7815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/2668289572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b7815b4a0 0x7f6b7815d4c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6b780749f0 tx=0x7f6b6c009b20 comp rx=0 tx=0).stop 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/2668289572 shutdown_connections 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/2668289572 >> v2:172.21.15.117:3300/0 conn(0x7f6b780c9f40 0x7f6b780c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/2668289572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b7815b4a0 0x7f6b7815d4c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/2668289572 >> 172.21.15.7:0/2668289572 conn(0x7f6b780737f0 msgr2=0x7f6b780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/2668289572 shutdown_connections 2023-06-09T11:30:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.293+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/2668289572 wait complete. 2023-06-09T11:30:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.295+0000 7f6b7e1ff700 1 Processor -- start 2023-06-09T11:30:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.295+0000 7f6b7e1ff700 1 -- start start 2023-06-09T11:30:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b780c9f40 0x7f6b780ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b7815b4a0 0x7f6b781f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780b4a80 con 0x7f6b78153880 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78151770 con 0x7f6b7815b4a0 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b7e1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781f2d30 con 0x7f6b780c9f40 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50726/0 (socket says 172.21.15.7:50726) 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 -- 172.21.15.7:0/3063659181 learned_addr learned my addr 172.21.15.7:0/3063659181 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 -- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b780c9f40 msgr2=0x7f6b780ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b780c9f40 0x7f6b780ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 -- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b7815b4a0 msgr2=0x7f6b781f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b7815b4a0 0x7f6b781f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.296+0000 7f6b76ffd700 1 -- 172.21.15.7:0/3063659181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0094a0 con 0x7f6b78153880 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b76ffd700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0090b0 tx=0x7f6b6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c0036a0 con 0x7f6b78153880 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c01e070 con 0x7f6b78153880 2023-06-09T11:30:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c003a20 con 0x7f6b78153880 2023-06-09T11:30:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b78159c60 con 0x7f6b78153880 2023-06-09T11:30:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.297+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78159980 con 0x7f6b78153880 2023-06-09T11:30:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.298+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b78159f40 con 0x7f6b78153880 2023-06-09T11:30:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.299+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b6c003b80 con 0x7f6b78153880 2023-06-09T11:30:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.301+0000 7f6b5ffff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 0x7f6b6406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.301+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f6b6c08a840 con 0x7f6b78153880 2023-06-09T11:30:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.301+0000 7f6b777fe700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 0x7f6b6406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.302+0000 7f6b777fe700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 0x7f6b6406d1c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6b680016b0 tx=0x7f6b68001310 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:24.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.305+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6b6c055540 con 0x7f6b78153880 2023-06-09T11:30:24.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.560+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b780ce510 con 0x7f6b78153880 2023-06-09T11:30:24.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.561+0000 7f6b5ffff700 1 -- 172.21.15.7:0/3063659181 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f6b6c058b60 con 0x7f6b78153880 2023-06-09T11:30:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:24.567 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:24.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 msgr2=0x7f6b6406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 0x7f6b6406d1c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6b680016b0 tx=0x7f6b68001310 comp rx=0 tx=0).stop 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 msgr2=0x7f6b78151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0090b0 tx=0x7f6b6c004750 comp rx=0 tx=0).stop 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 shutdown_connections 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b7815b4a0 0x7f6b781f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6b6406ad10 0x7f6b6406d1c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b780c9f40 0x7f6b780ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 --2- 172.21.15.7:0/3063659181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6b78153880 0x7f6b78151230 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 >> 172.21.15.7:0/3063659181 conn(0x7f6b780737f0 msgr2=0x7f6b780cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 shutdown_connections 2023-06-09T11:30:24.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:24.564+0000 7f6b7e1ff700 1 -- 172.21.15.7:0/3063659181 wait complete. 2023-06-09T11:30:24.572 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:24.588 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:26.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:26.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.794+0000 7f1e7dfff700 1 Processor -- start 2023-06-09T11:30:26.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.794+0000 7f1e7dfff700 1 -- start start 2023-06-09T11:30:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e7dfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e780c1bc0 0x7f1e781599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e7dfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 0x7f1e78159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e7dfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e780c0bf0 con 0x7f1e780c0140 2023-06-09T11:30:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e7dfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78070110 con 0x7f1e780c1bc0 2023-06-09T11:30:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e7dfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e78071300 con 0x7f1e780c1f90 2023-06-09T11:30:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e777fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1e780c1bc0 0x7f1e781599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e76ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1e780c0140 0x7f1e781501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57942/0 (socket says 172.21.15.7:57942) 2023-06-09T11:30:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.795+0000 7f1e76ffd700 1 -- 172.21.15.7:0/4261041967 learned_addr learned my addr 172.21.15.7:0/4261041967 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e777fe700 1 -- 172.21.15.7:0/4261041967 >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 msgr2=0x7f1e78159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e77fff700 1 --2- 172.21.15.7:0/4261041967 >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 0x7f1e78159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e777fe700 1 --2- 172.21.15.7:0/4261041967 >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 0x7f1e78159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e767fc700 1 -- 172.21.15.7:0/4261041967 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889352248 0 0) 0x7f1e780c0bf0 con 0x7f1e780c0140 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e777fe700 1 -- 172.21.15.7:0/4261041967 >> v1:172.21.15.7:6789/0 conn(0x7f1e780c0140 legacy=0x7f1e781501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e777fe700 1 -- 172.21.15.7:0/4261041967 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e780702f0 con 0x7f1e780c1bc0 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e77fff700 1 --2- 172.21.15.7:0/4261041967 >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 0x7f1e78159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e777fe700 1 --2- 172.21.15.7:0/4261041967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1bc0 0x7f1e781599b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1e78071ae0 tx=0x7f1e68009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2ffc839419aa72 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.796+0000 7f1e767fc700 1 -- 172.21.15.7:0/4261041967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68011830 con 0x7f1e780c1bc0 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e767fc700 1 -- 172.21.15.7:0/4261041967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68011990 con 0x7f1e780c1bc0 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e767fc700 1 -- 172.21.15.7:0/4261041967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e68011d20 con 0x7f1e780c1bc0 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 -- 172.21.15.7:0/4261041967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1bc0 msgr2=0x7f1e781599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/4261041967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1bc0 0x7f1e781599b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1e78071ae0 tx=0x7f1e68009b20 comp rx=0 tx=0).stop 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 -- 172.21.15.7:0/4261041967 shutdown_connections 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/4261041967 >> v2:172.21.15.117:3300/0 conn(0x7f1e780c1f90 0x7f1e78159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:26.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/4261041967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1bc0 0x7f1e781599b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:26.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 -- 172.21.15.7:0/4261041967 >> 172.21.15.7:0/4261041967 conn(0x7f1e7806a4d0 msgr2=0x7f1e7806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:26.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 -- 172.21.15.7:0/4261041967 shutdown_connections 2023-06-09T11:30:26.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.797+0000 7f1e7dfff700 1 -- 172.21.15.7:0/4261041967 wait complete. 2023-06-09T11:30:26.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.799+0000 7f1e7dfff700 1 Processor -- start 2023-06-09T11:30:26.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.799+0000 7f1e7dfff700 1 -- start start 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e780c0140 0x7f1e781566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e780c1bc0 0x7f1e7814bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781eea30 con 0x7f1e780c1f90 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78150150 con 0x7f1e780c1bc0 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e7dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e781eec40 con 0x7f1e780c0140 2023-06-09T11:30:26.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50742/0 (socket says 172.21.15.7:50742) 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e77fff700 1 -- 172.21.15.7:0/2351016084 learned_addr learned my addr 172.21.15.7:0/2351016084 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e777fe700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e780c0140 0x7f1e781566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.800+0000 7f1e76ffd700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e780c1bc0 0x7f1e7814bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 -- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e780c0140 msgr2=0x7f1e781566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e780c0140 0x7f1e781566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 -- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e780c1bc0 msgr2=0x7f1e7814bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:26.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e780c1bc0 0x7f1e7814bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 -- 172.21.15.7:0/2351016084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680094a0 con 0x7f1e780c1f90 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e77fff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cd50 tx=0x7f1e7000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000cea0 con 0x7f1e780c1f90 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7000f480 con 0x7f1e780c1f90 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e780c1170 con 0x7f1e780c1f90 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.801+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000f830 con 0x7f1e780c1f90 2023-06-09T11:30:26.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.802+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e780c0e90 con 0x7f1e780c1f90 2023-06-09T11:30:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.803+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e780c1450 con 0x7f1e780c1f90 2023-06-09T11:30:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.803+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e7000b040 con 0x7f1e780c1f90 2023-06-09T11:30:26.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.805+0000 7f1e5ffff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 0x7f1e6406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.805+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f1e70089f50 con 0x7f1e780c1f90 2023-06-09T11:30:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.805+0000 7f1e777fe700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 0x7f1e6406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:26.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.806+0000 7f1e777fe700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 0x7f1e6406d1e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:26.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:26.810+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1e70054bb0 con 0x7f1e780c1f90 2023-06-09T11:30:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.060+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e78074cf0 con 0x7f1e780c1f90 2023-06-09T11:30:27.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.060+0000 7f1e5ffff700 1 -- 172.21.15.7:0/2351016084 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f1e700581d0 con 0x7f1e780c1f90 2023-06-09T11:30:27.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:27.067 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 msgr2=0x7f1e6406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 0x7f1e6406d1e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1e680090b0 tx=0x7f1e68008030 comp rx=0 tx=0).stop 2023-06-09T11:30:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 msgr2=0x7f1e7814c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:27.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cd50 tx=0x7f1e7000a560 comp rx=0 tx=0).stop 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 shutdown_connections 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e780c1f90 0x7f1e7814c3e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1e6406ad30 0x7f1e6406d1e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1e780c0140 0x7f1e781566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 --2- 172.21.15.7:0/2351016084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1e780c1bc0 0x7f1e7814bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 >> 172.21.15.7:0/2351016084 conn(0x7f1e7806a4d0 msgr2=0x7f1e7814d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:27.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 shutdown_connections 2023-06-09T11:30:27.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:27.063+0000 7f1e7dfff700 1 -- 172.21.15.7:0/2351016084 wait complete. 2023-06-09T11:30:27.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:27.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:29.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:29.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.294+0000 7f49a33ff700 1 Processor -- start 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.294+0000 7f49a33ff700 1 -- start start 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.295+0000 7f49a33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c15b4a0 0x7f499c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.295+0000 7f49a33ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0c9f40 0x7f499c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.295+0000 7f49a33ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c15a500 con 0x7f499c153880 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a33ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0c5f10 con 0x7f499c0c9f40 2023-06-09T11:30:29.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15da00 con 0x7f499c15b4a0 2023-06-09T11:30:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0c9f40 0x7f499c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f499c0c9f40 0x7f499c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50744/0 (socket says 172.21.15.7:50744) 2023-06-09T11:30:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 -- 172.21.15.7:0/3939411079 learned_addr learned my addr 172.21.15.7:0/3939411079 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:29.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a099a700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c15b4a0 0x7f499c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 -- 172.21.15.7:0/3939411079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c15b4a0 msgr2=0x7f499c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c15b4a0 0x7f499c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 -- 172.21.15.7:0/3939411079 >> v1:172.21.15.117:6789/0 conn(0x7f499c153880 legacy=0x7f499c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.296+0000 7f49a199c700 1 -- 172.21.15.7:0/3939411079 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c1596e0 con 0x7f499c0c9f40 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a199c700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0c9f40 0x7f499c0c4e30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4994009e50 tx=0x7f499400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69460215414f4f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f4993fff700 1 -- 172.21.15.7:0/3939411079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400e070 con 0x7f499c0c9f40 2023-06-09T11:30:29.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f4993fff700 1 -- 172.21.15.7:0/3939411079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499400b9c0 con 0x7f499c0c9f40 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f4993fff700 1 -- 172.21.15.7:0/3939411079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400bd70 con 0x7f499c0c9f40 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 -- 172.21.15.7:0/3939411079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0c9f40 msgr2=0x7f499c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0c9f40 0x7f499c0c4e30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4994009e50 tx=0x7f499400b330 comp rx=0 tx=0).stop 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 -- 172.21.15.7:0/3939411079 shutdown_connections 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c0c9f40 0x7f499c0c4e30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3939411079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c15b4a0 0x7f499c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.297+0000 7f49a33ff700 1 -- 172.21.15.7:0/3939411079 >> 172.21.15.7:0/3939411079 conn(0x7f499c0737f0 msgr2=0x7f499c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:29.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.298+0000 7f49a33ff700 1 -- 172.21.15.7:0/3939411079 shutdown_connections 2023-06-09T11:30:29.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.298+0000 7f49a33ff700 1 -- 172.21.15.7:0/3939411079 wait complete. 2023-06-09T11:30:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.299+0000 7f49a33ff700 1 Processor -- start 2023-06-09T11:30:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.299+0000 7f49a33ff700 1 -- start start 2023-06-09T11:30:29.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c9f40 0x7f499c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c15b4a0 0x7f499c15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c154a10 con 0x7f499c153880 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c154690 con 0x7f499c15b4a0 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.300+0000 7f49a33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c14c410 con 0x7f499c0c9f40 2023-06-09T11:30:29.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50760/0 (socket says 172.21.15.7:50760) 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 -- 172.21.15.7:0/3257288898 learned_addr learned my addr 172.21.15.7:0/3257288898 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a119b700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c9f40 0x7f499c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a199c700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c15b4a0 0x7f499c15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 -- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c9f40 msgr2=0x7f499c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c9f40 0x7f499c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 -- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c15b4a0 msgr2=0x7f499c15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c15b4a0 0x7f499c15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 -- 172.21.15.7:0/3257288898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009b30 con 0x7f499c153880 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.301+0000 7f49a099a700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f498c00d800 tx=0x7f498c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.302+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0126b0 con 0x7f499c153880 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.302+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c012810 con 0x7f499c153880 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.302+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c012bc0 con 0x7f499c153880 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.302+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c159c60 con 0x7f499c153880 2023-06-09T11:30:29.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.302+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c159980 con 0x7f499c153880 2023-06-09T11:30:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.303+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c159f40 con 0x7f499c153880 2023-06-09T11:30:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.304+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f498c012d20 con 0x7f499c153880 2023-06-09T11:30:29.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.305+0000 7f4991ffb700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 0x7f498006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.305+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f498c0898a0 con 0x7f499c153880 2023-06-09T11:30:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.305+0000 7f49a119b700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 0x7f498006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:29.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.306+0000 7f49a119b700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 0x7f498006d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4998007fd0 tx=0x7f4998009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:29.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.309+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f498c0580d0 con 0x7f499c153880 2023-06-09T11:30:29.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.563+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f499c0ca380 con 0x7f499c153880 2023-06-09T11:30:29.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.564+0000 7f4991ffb700 1 -- 172.21.15.7:0/3257288898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f498c057c60 con 0x7f499c153880 2023-06-09T11:30:29.570 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:29.570 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 msgr2=0x7f498006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 0x7f498006d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4998007fd0 tx=0x7f4998009040 comp rx=0 tx=0).stop 2023-06-09T11:30:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 msgr2=0x7f499c1f2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f498c00d800 tx=0x7f498c00db10 comp rx=0 tx=0).stop 2023-06-09T11:30:29.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 shutdown_connections 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f499c15b4a0 0x7f499c15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f498006ace0 0x7f498006d190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f499c0c9f40 0x7f499c14c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 --2- 172.21.15.7:0/3257288898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f499c153880 0x7f499c1f2380 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 >> 172.21.15.7:0/3257288898 conn(0x7f499c0737f0 msgr2=0x7f499c0cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 shutdown_connections 2023-06-09T11:30:29.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:29.566+0000 7f49a33ff700 1 -- 172.21.15.7:0/3257288898 wait complete. 2023-06-09T11:30:29.574 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:29.590 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:31.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.797+0000 7fe4ba9ff700 1 Processor -- start 2023-06-09T11:30:31.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.797+0000 7fe4ba9ff700 1 -- start start 2023-06-09T11:30:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4ba9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe4b40cb9a0 0x7fe4b40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4ba9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe4b415cd40 0x7fe4b40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4ba9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b4159b20 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4ba9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b40cac80 con 0x7fe4b415cd40 2023-06-09T11:30:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4ba9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b41597d0 con 0x7fe4b40cb9a0 2023-06-09T11:30:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4b3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe4b40cb5d0 0x7fe4b4159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57946/0 (socket says 172.21.15.7:57946) 2023-06-09T11:30:31.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4b3fff700 1 -- 172.21.15.7:0/546766218 learned_addr learned my addr 172.21.15.7:0/546766218 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:31.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.798+0000 7fe4b8f9c700 1 --2- 172.21.15.7:0/546766218 >> v2:172.21.15.7:3300/0 conn(0x7fe4b415cd40 0x7fe4b40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b37fe700 1 --2- 172.21.15.7:0/546766218 >> v2:172.21.15.117:3300/0 conn(0x7fe4b40cb9a0 0x7fe4b40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b2ffd700 1 -- 172.21.15.7:0/546766218 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357393432 0 0) 0x7fe4b4159b20 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b2ffd700 1 -- 172.21.15.7:0/546766218 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe49c0035b0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b8f9c700 1 -- 172.21.15.7:0/546766218 >> v2:172.21.15.117:3300/0 conn(0x7fe4b40cb9a0 msgr2=0x7fe4b40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b8f9c700 1 --2- 172.21.15.7:0/546766218 >> v2:172.21.15.117:3300/0 conn(0x7fe4b40cb9a0 0x7fe4b40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b8f9c700 1 -- 172.21.15.7:0/546766218 >> v1:172.21.15.7:6789/0 conn(0x7fe4b40cb5d0 legacy=0x7fe4b4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:31.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b8f9c700 1 -- 172.21.15.7:0/546766218 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4071ab0 con 0x7fe4b415cd40 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b8f9c700 1 --2- 172.21.15.7:0/546766218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415cd40 0x7fe4b40cbea0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c9c60 tx=0x7fe4ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15d32674dbb754dc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.799+0000 7fe4b2ffd700 1 -- 172.21.15.7:0/546766218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac00e070 con 0x7fe4b415cd40 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4b2ffd700 1 -- 172.21.15.7:0/546766218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac00bb20 con 0x7fe4b415cd40 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4b2ffd700 1 -- 172.21.15.7:0/546766218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac012430 con 0x7fe4b415cd40 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/546766218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415cd40 msgr2=0x7fe4b40cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/546766218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415cd40 0x7fe4b40cbea0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40c9c60 tx=0x7fe4ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:30:31.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/546766218 shutdown_connections 2023-06-09T11:30:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/546766218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b415cd40 0x7fe4b40cbea0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/546766218 >> v2:172.21.15.117:3300/0 conn(0x7fe4b40cb9a0 0x7fe4b40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/546766218 >> 172.21.15.7:0/546766218 conn(0x7fe4b4070af0 msgr2=0x7fe4b4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/546766218 shutdown_connections 2023-06-09T11:30:31.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.800+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/546766218 wait complete. 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.802+0000 7fe4ba9ff700 1 Processor -- start 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.802+0000 7fe4ba9ff700 1 -- start start 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 0x7fe4b40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 0x7fe4b40c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40cf4c0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40cf1e0 con 0x7fe4b415cd40 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4ba9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40cfd40 con 0x7fe4b40cb9a0 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4b3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50770/0 (socket says 172.21.15.7:50770) 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.803+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1109823592 learned_addr learned my addr 172.21.15.7:0/1109823592 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b8f9c700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 0x7fe4b40c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b37fe700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 0x7fe4b40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 msgr2=0x7fe4b40cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 0x7fe4b40cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 msgr2=0x7fe4b40c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 0x7fe4b40c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 -- 172.21.15.7:0/1109823592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac009a70 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b8f9c700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 0x7fe4b40c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b37fe700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 0x7fe4b40cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:31.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b3fff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c700 tx=0x7fe4a400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:31.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4005d70 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a40070d0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4007460 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b415d4f0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.804+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b415d210 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.806+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b415d7d0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.807+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4a40075c0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.807+0000 7fe4b0ff9700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 0x7fe4a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.808+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7fe4a4088df0 con 0x7fe4b40cb5d0 2023-06-09T11:30:31.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.808+0000 7fe4b37fe700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 0x7fe4a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:31.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.808+0000 7fe4b37fe700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 0x7fe4a006d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4b415b290 tx=0x7fe4a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:31.812+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe4a4053b10 con 0x7fe4b40cb5d0 2023-06-09T11:30:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.064+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4b4074cf0 con 0x7fe4b40cb5d0 2023-06-09T11:30:32.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.065+0000 7fe4b0ff9700 1 -- 172.21.15.7:0/1109823592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+6216 (secure 0 0 0) 0x7fe4a4057130 con 0x7fe4b40cb5d0 2023-06-09T11:30:32.071 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:32.071 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5531,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:29:57.384815+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":59,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5435,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5435":{"gid":5435,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/3983837537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3983837537},{"type":"v1","addr":"172.21.15.117:6835","nonce":3983837537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.067+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 msgr2=0x7fe4a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.067+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 0x7fe4a006d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe4b415b290 tx=0x7fe4a8008040 comp rx=0 tx=0).stop 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.067+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 msgr2=0x7fe4b40c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.067+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c700 tx=0x7fe4a400cac0 comp rx=0 tx=0).stop 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 shutdown_connections 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe4b415cd40 0x7fe4b40c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4b40cb5d0 0x7fe4b40c50e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe4a006ace0 0x7fe4a006d190 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 --2- 172.21.15.7:0/1109823592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe4b40cb9a0 0x7fe4b40cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 >> 172.21.15.7:0/1109823592 conn(0x7fe4b4070af0 msgr2=0x7fe4b40cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 shutdown_connections 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:32.068+0000 7fe4ba9ff700 1 -- 172.21.15.7:0/1109823592 wait complete. 2023-06-09T11:30:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 84 2023-06-09T11:30:32.091 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:30:34.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.301+0000 7f00ccfff700 1 Processor -- start 2023-06-09T11:30:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.301+0000 7f00ccfff700 1 -- start start 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.302+0000 7f00ccfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f00c815b4a0 0x7f00c815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.302+0000 7f00ccfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f00c80c9f40 0x7f00c80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.302+0000 7f00ccfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c815a500 con 0x7f00c8153880 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.302+0000 7f00ccfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c80c5f10 con 0x7f00c80c9f40 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.302+0000 7f00ccfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c815da00 con 0x7f00c815b4a0 2023-06-09T11:30:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00c8153880 0x7f00c815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43964/0 (socket says 172.21.15.7:43964) 2023-06-09T11:30:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c659c700 1 -- 172.21.15.7:0/2931492284 learned_addr learned my addr 172.21.15.7:0/2931492284 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2931492284 >> v2:172.21.15.7:3300/0 conn(0x7f00c80c9f40 0x7f00c80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/2931492284 >> v2:172.21.15.117:3300/0 conn(0x7f00c815b4a0 0x7f00c815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c559a700 1 -- 172.21.15.7:0/2931492284 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970073828 0 0) 0x7f00b8001110 con 0x7f00c8153880 2023-06-09T11:30:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c559a700 1 -- 172.21.15.7:0/2931492284 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b00035b0 con 0x7f00c8153880 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 -- 172.21.15.7:0/2931492284 >> v2:172.21.15.117:3300/0 conn(0x7f00c815b4a0 msgr2=0x7f00c815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2931492284 >> v2:172.21.15.117:3300/0 conn(0x7f00c815b4a0 0x7f00c815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 -- 172.21.15.7:0/2931492284 >> v1:172.21.15.7:6789/0 conn(0x7f00c8153880 legacy=0x7f00c815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 -- 172.21.15.7:0/2931492284 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c81596e0 con 0x7f00c80c9f40 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.303+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/2931492284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c80c9f40 0x7f00c80c4e30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f00c80cce30 tx=0x7f00c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4b4576bc4ebb92 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00c559a700 1 -- 172.21.15.7:0/2931492284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c000e070 con 0x7f00c80c9f40 2023-06-09T11:30:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00c559a700 1 -- 172.21.15.7:0/2931492284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c000bb20 con 0x7f00c80c9f40 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00c559a700 1 -- 172.21.15.7:0/2931492284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0012430 con 0x7f00c80c9f40 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 -- 172.21.15.7:0/2931492284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c80c9f40 msgr2=0x7f00c80c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 --2- 172.21.15.7:0/2931492284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c80c9f40 0x7f00c80c4e30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f00c80cce30 tx=0x7f00c000b330 comp rx=0 tx=0).stop 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 -- 172.21.15.7:0/2931492284 shutdown_connections 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 --2- 172.21.15.7:0/2931492284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c80c9f40 0x7f00c80c4e30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 --2- 172.21.15.7:0/2931492284 >> v2:172.21.15.117:3300/0 conn(0x7f00c815b4a0 0x7f00c815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 -- 172.21.15.7:0/2931492284 >> 172.21.15.7:0/2931492284 conn(0x7f00c80737f0 msgr2=0x7f00c80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 -- 172.21.15.7:0/2931492284 shutdown_connections 2023-06-09T11:30:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.304+0000 7f00ccfff700 1 -- 172.21.15.7:0/2931492284 wait complete. 2023-06-09T11:30:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.306+0000 7f00ccfff700 1 Processor -- start 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.306+0000 7f00ccfff700 1 -- start start 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 0x7f00c80b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 0x7f00c80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80cda50 con 0x7f00c815b4a0 2023-06-09T11:30:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80ce5a0 con 0x7f00c80c9f40 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00ccfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8151340 con 0x7f00c8153880 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00c6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33748/0 (socket says 172.21.15.7:33748) 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.307+0000 7f00c6d9d700 1 -- 172.21.15.7:0/3070888776 learned_addr learned my addr 172.21.15.7:0/3070888776 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c659c700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 0x7f00c80b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 0x7f00c80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 -- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 msgr2=0x7f00c80b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 0x7f00c80b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 -- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 msgr2=0x7f00c80b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 0x7f00c80b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 -- 172.21.15.7:0/3070888776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0009a70 con 0x7f00c815b4a0 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c659c700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 0x7f00c80b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c5d9b700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 0x7f00c80b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00c6d9d700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f00c806b580 tx=0x7f00c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0012430 con 0x7f00c815b4a0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c0020070 con 0x7f00c815b4a0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c8159c60 con 0x7f00c815b4a0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.308+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0004280 con 0x7f00c815b4a0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.309+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c8159980 con 0x7f00c815b4a0 2023-06-09T11:30:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.310+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c8159f40 con 0x7f00c815b4a0 2023-06-09T11:30:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.311+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00c00043e0 con 0x7f00c815b4a0 2023-06-09T11:30:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.312+0000 7f00b6ffd700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.312+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6980+0+0 (secure 0 0 0) 0x7f00c0022070 con 0x7f00c815b4a0 2023-06-09T11:30:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.312+0000 7f00c659c700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.312+0000 7f00c659c700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 0x7f00a806d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007fd0 tx=0x7f00b8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.316+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f00c00561b0 con 0x7f00c815b4a0 2023-06-09T11:30:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.582+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00c81514b0 con 0x7f00c815b4a0 2023-06-09T11:30:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.583+0000 7f00b6ffd700 1 -- 172.21.15.7:0/3070888776 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f00c81514b0 con 0x7f00c815b4a0 2023-06-09T11:30:34.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:34.590 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:34.383204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:34.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.588+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 msgr2=0x7f00a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.588+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 0x7f00a806d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007fd0 tx=0x7f00b8009040 comp rx=0 tx=0).stop 2023-06-09T11:30:34.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 msgr2=0x7f00c8151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f00c806b580 tx=0x7f00c0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 shutdown_connections 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00c815b4a0 0x7f00c8151790 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00c80c9f40 0x7f00c80b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f00a806ace0 0x7f00a806d190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 --2- 172.21.15.7:0/3070888776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00c8153880 0x7f00c80b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 >> 172.21.15.7:0/3070888776 conn(0x7f00c80737f0 msgr2=0x7f00c814d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 shutdown_connections 2023-06-09T11:30:34.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.589+0000 7f00ccfff700 1 -- 172.21.15.7:0/3070888776 wait complete. 2023-06-09T11:30:34.597 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2023-06-09T11:30:34.613 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:30:34.613 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:30:34.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:34.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.806+0000 7fdce4fff700 1 Processor -- start 2023-06-09T11:30:34.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.806+0000 7fdce4fff700 1 -- start start 2023-06-09T11:30:34.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.807+0000 7fdce4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 0x7fdce015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.807+0000 7fdce4fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 0x7fdce015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdce4fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce015a500 con 0x7fdce015b2d0 2023-06-09T11:30:34.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdce4fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce00c5f10 con 0x7fdce00c9f40 2023-06-09T11:30:34.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdce4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce015da00 con 0x7fdce0153880 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcded9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdce015b2d0 0x7fdce00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43974/0 (socket says 172.21.15.7:43974) 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcded9d700 1 -- 172.21.15.7:0/548676080 learned_addr learned my addr 172.21.15.7:0/548676080 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcde59c700 1 --2- 172.21.15.7:0/548676080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 0x7fdce015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcddd9b700 1 --2- 172.21.15.7:0/548676080 >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 0x7fdce015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcde59c700 1 -- 172.21.15.7:0/548676080 >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 msgr2=0x7fdce015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:34.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcdd59a700 1 -- 172.21.15.7:0/548676080 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883042851 0 0) 0x7fdcd8001490 con 0x7fdce015b2d0 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcde59c700 1 --2- 172.21.15.7:0/548676080 >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 0x7fdce015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcde59c700 1 -- 172.21.15.7:0/548676080 >> v1:172.21.15.7:6789/0 conn(0x7fdce015b2d0 legacy=0x7fdce00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcde59c700 1 -- 172.21.15.7:0/548676080 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce015b750 con 0x7fdce0153880 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.808+0000 7fdcddd9b700 1 --2- 172.21.15.7:0/548676080 >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 0x7fdce015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.809+0000 7fdcde59c700 1 --2- 172.21.15.7:0/548676080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 0x7fdce015d4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009840 tx=0x7fdcd0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3150bb85914c3af9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.809+0000 7fdcdd59a700 1 -- 172.21.15.7:0/548676080 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011830 con 0x7fdce0153880 2023-06-09T11:30:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.809+0000 7fdcdd59a700 1 -- 172.21.15.7:0/548676080 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd0011990 con 0x7fdce0153880 2023-06-09T11:30:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.809+0000 7fdcdd59a700 1 -- 172.21.15.7:0/548676080 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0011d20 con 0x7fdce0153880 2023-06-09T11:30:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 -- 172.21.15.7:0/548676080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 msgr2=0x7fdce015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 --2- 172.21.15.7:0/548676080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 0x7fdce015d4c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0009840 tx=0x7fdcd0009b50 comp rx=0 tx=0).stop 2023-06-09T11:30:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 -- 172.21.15.7:0/548676080 shutdown_connections 2023-06-09T11:30:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 --2- 172.21.15.7:0/548676080 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce0153880 0x7fdce015d4c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 --2- 172.21.15.7:0/548676080 >> v2:172.21.15.117:3300/0 conn(0x7fdce00c9f40 0x7fdce015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 -- 172.21.15.7:0/548676080 >> 172.21.15.7:0/548676080 conn(0x7fdce00737f0 msgr2=0x7fdce00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 -- 172.21.15.7:0/548676080 shutdown_connections 2023-06-09T11:30:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.810+0000 7fdce4fff700 1 -- 172.21.15.7:0/548676080 wait complete. 2023-06-09T11:30:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.811+0000 7fdce4fff700 1 Processor -- start 2023-06-09T11:30:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 -- start start 2023-06-09T11:30:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce00c9f40 0x7fdce01f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce015b2d0 0x7fdce0157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce01f31c0 con 0x7fdce0153880 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce00cbe30 con 0x7fdce015b2d0 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.812+0000 7fdce4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce01f2ee0 con 0x7fdce00c9f40 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33750/0 (socket says 172.21.15.7:33750) 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 -- 172.21.15.7:0/4122890667 learned_addr learned my addr 172.21.15.7:0/4122890667 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcde59c700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce00c9f40 0x7fdce01f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcded9d700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce015b2d0 0x7fdce0157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 -- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce00c9f40 msgr2=0x7fdce01f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce00c9f40 0x7fdce01f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 -- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce015b2d0 msgr2=0x7fdce0157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce015b2d0 0x7fdce0157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 -- 172.21.15.7:0/4122890667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40094a0 con 0x7fdce0153880 2023-06-09T11:30:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.813+0000 7fdcddd9b700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400ebe0 tx=0x7fdcd400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.814+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4009800 con 0x7fdce0153880 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.814+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd400c910 con 0x7fdce0153880 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.814+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400cc80 con 0x7fdce0153880 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.814+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd00094a0 con 0x7fdce0153880 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.814+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce0159980 con 0x7fdce0153880 2023-06-09T11:30:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.815+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce01596e0 con 0x7fdce0153880 2023-06-09T11:30:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.816+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcd4013470 con 0x7fdce0153880 2023-06-09T11:30:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.817+0000 7fdcceffd700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 0x7fdcc006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.817+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fdcd4089d20 con 0x7fdce0153880 2023-06-09T11:30:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.817+0000 7fdcde59c700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 0x7fdcc006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.818+0000 7fdcde59c700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 0x7fdcc006d1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0000c00 tx=0x7fdcd0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:34.821+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdcd40549d0 con 0x7fdce0153880 2023-06-09T11:30:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.070+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdce0159c60 con 0x7fdce0153880 2023-06-09T11:30:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.071+0000 7fdcceffd700 1 -- 172.21.15.7:0/4122890667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5455 (secure 0 0 0) 0x7fdcd4057ff0 con 0x7fdce0153880 2023-06-09T11:30:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:35.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:34.383204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 msgr2=0x7fdcc006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 0x7fdcc006d1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0000c00 tx=0x7fdcd0008030 comp rx=0 tx=0).stop 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 msgr2=0x7fdce0157580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400ebe0 tx=0x7fdcd400eef0 comp rx=0 tx=0).stop 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 shutdown_connections 2023-06-09T11:30:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdce015b2d0 0x7fdce0157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcc006ad10 0x7fdcc006d1c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdce00c9f40 0x7fdce01f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 --2- 172.21.15.7:0/4122890667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdce0153880 0x7fdce0157580 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 >> 172.21.15.7:0/4122890667 conn(0x7fdce00737f0 msgr2=0x7fdce00cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 shutdown_connections 2023-06-09T11:30:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:35.073+0000 7fdce4fff700 1 -- 172.21.15.7:0/4122890667 wait complete. 2023-06-09T11:30:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2023-06-09T11:30:35.097 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 85, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 85, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5531": { "addr": "172.21.15.117:6833/1018690909", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1018690909, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1018690909, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5531, "incarnation": 85, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:30:34.383204+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5531, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:30:37.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.305+0000 7f1cd75ff700 1 Processor -- start 2023-06-09T11:30:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.306+0000 7f1cd75ff700 1 -- start start 2023-06-09T11:30:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd75ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 0x7f1cd0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd75ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd01580a0 con 0x7f1cd0071060 2023-06-09T11:30:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd75ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd00c6180 con 0x7f1cd0151510 2023-06-09T11:30:37.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd75ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd00714e0 con 0x7f1cd00c5b40 2023-06-09T11:30:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd5b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1cd0071060 0x7f1cd0158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43988/0 (socket says 172.21.15.7:43988) 2023-06-09T11:30:37.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd5b9c700 1 -- 172.21.15.7:0/2851505792 learned_addr learned my addr 172.21.15.7:0/2851505792 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.307+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/2851505792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 0x7f1cd0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008224004 0 0) 0x7f1cd01580a0 con 0x7f1cd0071060 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cbc0035b0 con 0x7f1cd0071060 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082547494 0 0) 0x7f1cd00c6180 con 0x7f1cd0151510 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd01580a0 con 0x7f1cd0151510 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3333586208 0 0) 0x7f1cbc0035b0 con 0x7f1cd0071060 2023-06-09T11:30:37.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cd4b9a700 1 -- 172.21.15.7:0/2851505792 >> v1:172.21.15.117:6789/0 conn(0x7f1cd0151510 legacy=0x7f1cd015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cd4b9a700 1 -- 172.21.15.7:0/2851505792 >> v1:172.21.15.7:6789/0 conn(0x7f1cd0071060 legacy=0x7f1cd0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cd4b9a700 1 -- 172.21.15.7:0/2851505792 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0157490 con 0x7f1cd00c5b40 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.308+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/2851505792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 0x7f1cd0070b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009870 tx=0x7f1cc0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ffcb3d74f2eb33 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc80019f0 con 0x7f1cd00c5b40 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc0004030 con 0x7f1cd00c5b40 2023-06-09T11:30:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cc7fff700 1 -- 172.21.15.7:0/2851505792 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc00043a0 con 0x7f1cd00c5b40 2023-06-09T11:30:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 -- 172.21.15.7:0/2851505792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 msgr2=0x7f1cd0070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/2851505792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 0x7f1cd0070b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009870 tx=0x7f1cc0009b80 comp rx=0 tx=0).stop 2023-06-09T11:30:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 -- 172.21.15.7:0/2851505792 shutdown_connections 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/2851505792 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd00c5b40 0x7f1cd0070b20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 -- 172.21.15.7:0/2851505792 >> 172.21.15.7:0/2851505792 conn(0x7f1cd006a4d0 msgr2=0x7f1cd006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 -- 172.21.15.7:0/2851505792 shutdown_connections 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.309+0000 7f1cd75ff700 1 -- 172.21.15.7:0/2851505792 wait complete. 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.311+0000 7f1cd75ff700 1 Processor -- start 2023-06-09T11:30:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.311+0000 7f1cd75ff700 1 -- start start 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 0x7f1cd0158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 0x7f1cd0149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd01594f0 con 0x7f1cd0071060 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd014f970 con 0x7f1cd00c5b40 2023-06-09T11:30:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd75ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd014fc50 con 0x7f1cd0151510 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd539b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33766/0 (socket says 172.21.15.7:33766) 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.312+0000 7f1cd539b700 1 -- 172.21.15.7:0/3040678257 learned_addr learned my addr 172.21.15.7:0/3040678257 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 0x7f1cd0158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 -- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 msgr2=0x7f1cd0149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd5b9c700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 0x7f1cd0149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 0x7f1cd0149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 -- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 msgr2=0x7f1cd0158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 0x7f1cd0158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 -- 172.21.15.7:0/3040678257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc00094a0 con 0x7f1cd0071060 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 0x7f1cd0158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd5b9c700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 0x7f1cd0149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd539b700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c500 tx=0x7f1ccc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0052d0 con 0x7f1cd0071060 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc005450 con 0x7f1cd0071060 2023-06-09T11:30:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd0157a50 con 0x7f1cd0071060 2023-06-09T11:30:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0038c0 con 0x7f1cd0071060 2023-06-09T11:30:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.313+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd0157770 con 0x7f1cd0071060 2023-06-09T11:30:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.314+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd0157d30 con 0x7f1cd0071060 2023-06-09T11:30:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.315+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ccc003a20 con 0x7f1cd0071060 2023-06-09T11:30:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.316+0000 7f1cc5ffb700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 0x7f1cb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.316+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f1ccc088eb0 con 0x7f1cd0071060 2023-06-09T11:30:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.317+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 0x7f1cb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.317+0000 7f1cd4b9a700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 0x7f1cb406d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00097c0 tx=0x7f1cc0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.320+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1ccc053a40 con 0x7f1cd0071060 2023-06-09T11:30:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.579+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cd0074cf0 con 0x7f1cd0071060 2023-06-09T11:30:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.580+0000 7f1cc5ffb700 1 -- 172.21.15.7:0/3040678257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f1ccc017020 con 0x7f1cd0071060 2023-06-09T11:30:37.586 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:37.586 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:34.383204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:37.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 msgr2=0x7f1cb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 0x7f1cb406d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1cc00097c0 tx=0x7f1cc0008030 comp rx=0 tx=0).stop 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 msgr2=0x7f1cd00c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00c500 tx=0x7f1ccc00c8c0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 shutdown_connections 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb406ace0 0x7f1cb406d190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cd0151510 0x7f1cd0149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cd0071060 0x7f1cd00c0a10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 --2- 172.21.15.7:0/3040678257 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cd00c5b40 0x7f1cd0158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 >> 172.21.15.7:0/3040678257 conn(0x7f1cd006a4d0 msgr2=0x7f1cd014d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 shutdown_connections 2023-06-09T11:30:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:37.582+0000 7f1cd75ff700 1 -- 172.21.15.7:0/3040678257 wait complete. 2023-06-09T11:30:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2023-06-09T11:30:37.607 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:39.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:39.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.811+0000 7efc7e9ff700 1 Processor -- start 2023-06-09T11:30:39.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.811+0000 7efc7e9ff700 1 -- start start 2023-06-09T11:30:39.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.812+0000 7efc7e9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc78151510 0x7efc780c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.812+0000 7efc7e9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc78071230 0x7efc78158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.812+0000 7efc7e9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc781580a0 con 0x7efc7815b380 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.812+0000 7efc7e9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc781518e0 con 0x7efc78071230 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.812+0000 7efc7e9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc780c6180 con 0x7efc78151510 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc78071230 0x7efc78158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc77fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc78151510 0x7efc780c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc78071230 0x7efc78158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33770/0 (socket says 172.21.15.7:33770) 2023-06-09T11:30:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 -- 172.21.15.7:0/2584477009 learned_addr learned my addr 172.21.15.7:0/2584477009 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 -- 172.21.15.7:0/2584477009 >> v2:172.21.15.117:3300/0 conn(0x7efc78151510 msgr2=0x7efc780c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 --2- 172.21.15.7:0/2584477009 >> v2:172.21.15.117:3300/0 conn(0x7efc78151510 0x7efc780c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 -- 172.21.15.7:0/2584477009 >> v1:172.21.15.117:6789/0 conn(0x7efc7815b380 legacy=0x7efc78070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 -- 172.21.15.7:0/2584477009 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc78157490 con 0x7efc78071230 2023-06-09T11:30:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.813+0000 7efc7cf9c700 1 --2- 172.21.15.7:0/2584477009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78071230 0x7efc78158870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc7806ff10 tx=0x7efc7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3b0508dca474ce server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc76ffd700 1 -- 172.21.15.7:0/2584477009 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000e070 con 0x7efc78071230 2023-06-09T11:30:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc76ffd700 1 -- 172.21.15.7:0/2584477009 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7000b910 con 0x7efc78071230 2023-06-09T11:30:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc76ffd700 1 -- 172.21.15.7:0/2584477009 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7000bcc0 con 0x7efc78071230 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 -- 172.21.15.7:0/2584477009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78071230 msgr2=0x7efc78158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/2584477009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78071230 0x7efc78158870 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efc7806ff10 tx=0x7efc7000b330 comp rx=0 tx=0).stop 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 -- 172.21.15.7:0/2584477009 shutdown_connections 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/2584477009 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78071230 0x7efc78158870 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/2584477009 >> v2:172.21.15.117:3300/0 conn(0x7efc78151510 0x7efc780c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 -- 172.21.15.7:0/2584477009 >> 172.21.15.7:0/2584477009 conn(0x7efc7806a4d0 msgr2=0x7efc7806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 -- 172.21.15.7:0/2584477009 shutdown_connections 2023-06-09T11:30:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.814+0000 7efc7e9ff700 1 -- 172.21.15.7:0/2584477009 wait complete. 2023-06-09T11:30:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.816+0000 7efc7e9ff700 1 Processor -- start 2023-06-09T11:30:39.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.816+0000 7efc7e9ff700 1 -- start start 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc78071230 0x7efc780c11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7815b380 0x7efc781f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780c9640 con 0x7efc78151510 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780c7fa0 con 0x7efc78071230 2023-06-09T11:30:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.817+0000 7efc7e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc780c0d10 con 0x7efc7815b380 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33780/0 (socket says 172.21.15.7:33780) 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 -- 172.21.15.7:0/1096989086 learned_addr learned my addr 172.21.15.7:0/1096989086 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc77fff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc78071230 0x7efc780c11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc7cf9c700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7815b380 0x7efc781f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 -- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7815b380 msgr2=0x7efc781f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7815b380 0x7efc781f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 -- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc78071230 msgr2=0x7efc780c11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc78071230 0x7efc780c11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 -- 172.21.15.7:0/1096989086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc70009b30 con 0x7efc78151510 2023-06-09T11:30:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc777fe700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c450 tx=0x7efc6c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.818+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c0050d0 con 0x7efc78151510 2023-06-09T11:30:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.819+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6c005230 con 0x7efc78151510 2023-06-09T11:30:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.819+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c003680 con 0x7efc78151510 2023-06-09T11:30:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.819+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc78157a50 con 0x7efc78151510 2023-06-09T11:30:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.819+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc78157770 con 0x7efc78151510 2023-06-09T11:30:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.820+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc78157d30 con 0x7efc78151510 2023-06-09T11:30:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.821+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc6c0037e0 con 0x7efc78151510 2023-06-09T11:30:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.822+0000 7efc74ff9700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 0x7efc6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.822+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7efc6c088c10 con 0x7efc78151510 2023-06-09T11:30:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.822+0000 7efc77fff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 0x7efc6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.822+0000 7efc77fff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 0x7efc6406d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efc68009be0 tx=0x7efc68009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:39.826+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc6c053830 con 0x7efc78151510 2023-06-09T11:30:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.077+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc78074cf0 con 0x7efc78151510 2023-06-09T11:30:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.077+0000 7efc74ff9700 1 -- 172.21.15.7:0/1096989086 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+5455 (secure 0 0 0) 0x7efc6c056e50 con 0x7efc78151510 2023-06-09T11:30:40.083 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:40.084 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:34.383204+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.079+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 msgr2=0x7efc6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.079+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 0x7efc6406d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efc68009be0 tx=0x7efc68009210 comp rx=0 tx=0).stop 2023-06-09T11:30:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 msgr2=0x7efc781f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c450 tx=0x7efc6c00c760 comp rx=0 tx=0).stop 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 shutdown_connections 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc6406ace0 0x7efc6406d190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc7815b380 0x7efc781f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc78071230 0x7efc780c11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 --2- 172.21.15.7:0/1096989086 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc78151510 0x7efc781f2cc0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 >> 172.21.15.7:0/1096989086 conn(0x7efc7806a4d0 msgr2=0x7efc78070010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 shutdown_connections 2023-06-09T11:30:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:40.080+0000 7efc7e9ff700 1 -- 172.21.15.7:0/1096989086 wait complete. 2023-06-09T11:30:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2023-06-09T11:30:40.104 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:42.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.312+0000 7f3466fff700 1 Processor -- start 2023-06-09T11:30:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.312+0000 7f3466fff700 1 -- start start 2023-06-09T11:30:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3466fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3460151510 0x7f34600c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3466fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3460071230 0x7f3460158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3466fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34601580a0 con 0x7f346015b380 2023-06-09T11:30:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3466fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34601518e0 con 0x7f3460151510 2023-06-09T11:30:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3466fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34600c6180 con 0x7f3460071230 2023-06-09T11:30:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f345ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f346015b380 0x7f3460070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44002/0 (socket says 172.21.15.7:44002) 2023-06-09T11:30:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f345ffff700 1 -- 172.21.15.7:0/3162983181 learned_addr learned my addr 172.21.15.7:0/3162983181 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3464d9b700 1 --2- 172.21.15.7:0/3162983181 >> v2:172.21.15.7:3300/0 conn(0x7f3460151510 0x7f34600c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f346559c700 1 --2- 172.21.15.7:0/3162983181 >> v2:172.21.15.117:3300/0 conn(0x7f3460071230 0x7f3460158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3464d9b700 1 -- 172.21.15.7:0/3162983181 >> v2:172.21.15.117:3300/0 conn(0x7f3460071230 msgr2=0x7f3460158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3464d9b700 1 --2- 172.21.15.7:0/3162983181 >> v2:172.21.15.117:3300/0 conn(0x7f3460071230 0x7f3460158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.314+0000 7f3464d9b700 1 -- 172.21.15.7:0/3162983181 >> v1:172.21.15.7:6789/0 conn(0x7f346015b380 legacy=0x7f3460070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f3464d9b700 1 -- 172.21.15.7:0/3162983181 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3460157490 con 0x7f3460151510 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f345f7fe700 1 -- 172.21.15.7:0/3162983181 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794413510 0 0) 0x7f34601580a0 con 0x7f346015b380 2023-06-09T11:30:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f3464d9b700 1 --2- 172.21.15.7:0/3162983181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f34600c5b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f346006fec0 tx=0x7f3454009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fc27586d076b60 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f345f7fe700 1 -- 172.21.15.7:0/3162983181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011830 con 0x7f3460151510 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f345f7fe700 1 -- 172.21.15.7:0/3162983181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454011990 con 0x7f3460151510 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.315+0000 7f345f7fe700 1 -- 172.21.15.7:0/3162983181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454011d20 con 0x7f3460151510 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 -- 172.21.15.7:0/3162983181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 msgr2=0x7f34600c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 --2- 172.21.15.7:0/3162983181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f34600c5b40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f346006fec0 tx=0x7f3454009b20 comp rx=0 tx=0).stop 2023-06-09T11:30:42.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 -- 172.21.15.7:0/3162983181 shutdown_connections 2023-06-09T11:30:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 --2- 172.21.15.7:0/3162983181 >> v2:172.21.15.117:3300/0 conn(0x7f3460071230 0x7f3460158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 --2- 172.21.15.7:0/3162983181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f34600c5b40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 -- 172.21.15.7:0/3162983181 >> 172.21.15.7:0/3162983181 conn(0x7f346006a4d0 msgr2=0x7f346006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 -- 172.21.15.7:0/3162983181 shutdown_connections 2023-06-09T11:30:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.316+0000 7f3466fff700 1 -- 172.21.15.7:0/3162983181 wait complete. 2023-06-09T11:30:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.317+0000 7f3466fff700 1 Processor -- start 2023-06-09T11:30:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 -- start start 2023-06-09T11:30:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3460071230 0x7f34601e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f346015b380 0x7f34600c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3460147920 con 0x7f3460151510 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34600c6d60 con 0x7f346015b380 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.318+0000 7f3466fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3460155660 con 0x7f3460071230 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33800/0 (socket says 172.21.15.7:33800) 2023-06-09T11:30:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 -- 172.21.15.7:0/2765398120 learned_addr learned my addr 172.21.15.7:0/2765398120 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f346559c700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f346015b380 0x7f34600c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f3464d9b700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3460071230 0x7f34601e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 -- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3460071230 msgr2=0x7f34601e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3460071230 0x7f34601e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 -- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f346015b380 msgr2=0x7f34600c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f346015b380 0x7f34600c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 -- 172.21.15.7:0/2765398120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34540094a0 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.319+0000 7f345ffff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f344c00c580 tx=0x7f344c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.320+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c004c80 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.320+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c004e00 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.320+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3460157a50 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.320+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c007310 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.320+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3460157770 con 0x7f3460151510 2023-06-09T11:30:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.321+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3460157d30 con 0x7f3460151510 2023-06-09T11:30:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.322+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f344c007470 con 0x7f3460151510 2023-06-09T11:30:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.323+0000 7f345d7fa700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 0x7f344406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:42.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.323+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f344c089580 con 0x7f3460151510 2023-06-09T11:30:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.323+0000 7f3464d9b700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 0x7f344406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.324+0000 7f3464d9b700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 0x7f344406d1c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34540090b0 tx=0x7f3454008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.327+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f344c0541a0 con 0x7f3460151510 2023-06-09T11:30:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.582+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34600b10b0 con 0x7f3460151510 2023-06-09T11:30:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.583+0000 7f345d7fa700 1 -- 172.21.15.7:0/2765398120 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f344c0577c0 con 0x7f3460151510 2023-06-09T11:30:42.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:42.590 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:40.557872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 msgr2=0x7f344406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 0x7f344406d1c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f34540090b0 tx=0x7f3454008030 comp rx=0 tx=0).stop 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 msgr2=0x7f3460155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f344c00c580 tx=0x7f344c00c890 comp rx=0 tx=0).stop 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 shutdown_connections 2023-06-09T11:30:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f346015b380 0x7f34600c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f344406ad10 0x7f344406d1c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3460071230 0x7f34601e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 --2- 172.21.15.7:0/2765398120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3460151510 0x7f3460155b40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 >> 172.21.15.7:0/2765398120 conn(0x7f346006a4d0 msgr2=0x7f346014e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.585+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 shutdown_connections 2023-06-09T11:30:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:42.586+0000 7f3466fff700 1 -- 172.21.15.7:0/2765398120 wait complete. 2023-06-09T11:30:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2023-06-09T11:30:42.610 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:44.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.816+0000 7fde555ff700 1 Processor -- start 2023-06-09T11:30:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.816+0000 7fde555ff700 1 -- start start 2023-06-09T11:30:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.817+0000 7fde555ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.817+0000 7fde555ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde5015cd40 0x7fde500cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.817+0000 7fde555ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde50159b20 con 0x7fde500cb9a0 2023-06-09T11:30:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.817+0000 7fde555ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde500cac80 con 0x7fde5015cd40 2023-06-09T11:30:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.817+0000 7fde555ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde501597d0 con 0x7fde500cb5d0 2023-06-09T11:30:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde5015cd40 0x7fde500cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde5015cd40 0x7fde500cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:60830/0 (socket says 172.21.15.7:60830) 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 -- 172.21.15.7:0/3827690494 learned_addr learned my addr 172.21.15.7:0/3827690494 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 -- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 msgr2=0x7fde500b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 --2- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 -- 172.21.15.7:0/3827690494 >> v1:172.21.15.117:6789/0 conn(0x7fde500cb9a0 legacy=0x7fde50159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.818+0000 7fde4f7fe700 1 -- 172.21.15.7:0/3827690494 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde50071ab0 con 0x7fde5015cd40 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde4f7fe700 1 --2- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cbea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a159881c748a5ec9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde4dffb700 1 -- 172.21.15.7:0/3827690494 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800e070 con 0x7fde5015cd40 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde4dffb700 1 -- 172.21.15.7:0/3827690494 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800b910 con 0x7fde5015cd40 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde4dffb700 1 -- 172.21.15.7:0/3827690494 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800bcc0 con 0x7fde5015cd40 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde555ff700 1 -- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 msgr2=0x7fde500cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.819+0000 7fde555ff700 1 --2- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cbea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fde48009e50 tx=0x7fde4800b330 comp rx=0 tx=0).stop 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 -- 172.21.15.7:0/3827690494 shutdown_connections 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 --2- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cbea0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 --2- 172.21.15.7:0/3827690494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 -- 172.21.15.7:0/3827690494 >> 172.21.15.7:0/3827690494 conn(0x7fde50070af0 msgr2=0x7fde50070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 -- 172.21.15.7:0/3827690494 shutdown_connections 2023-06-09T11:30:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.820+0000 7fde555ff700 1 -- 172.21.15.7:0/3827690494 wait complete. 2023-06-09T11:30:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.821+0000 7fde555ff700 1 Processor -- start 2023-06-09T11:30:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.822+0000 7fde555ff700 1 -- start start 2023-06-09T11:30:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 0x7fde50151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde500cd560 con 0x7fde500cb9a0 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50154310 con 0x7fde5015cd40 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde555ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5015aba0 con 0x7fde500cb5d0 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 0x7fde50151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:52842/0 (socket says 172.21.15.7:52842) 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4effd700 1 -- 172.21.15.7:0/1390830823 learned_addr learned my addr 172.21.15.7:0/1390830823 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4f7fe700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 -- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 msgr2=0x7fde500c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 -- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 msgr2=0x7fde500cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4e7fc700 1 -- 172.21.15.7:0/1390830823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde48009b30 con 0x7fde500cb9a0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.823+0000 7fde4effd700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde4e7fc700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 0x7fde50151870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fde44000f60 tx=0x7fde4400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440050d0 con 0x7fde500cb9a0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44005230 con 0x7fde500cb9a0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5015d4f0 con 0x7fde500cb9a0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44003680 con 0x7fde500cb9a0 2023-06-09T11:30:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.824+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5015d210 con 0x7fde500cb9a0 2023-06-09T11:30:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.825+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5015d7d0 con 0x7fde500cb9a0 2023-06-09T11:30:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.826+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde440037e0 con 0x7fde500cb9a0 2023-06-09T11:30:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.827+0000 7fde377fe700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 0x7fde3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.827+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fde44088b90 con 0x7fde500cb9a0 2023-06-09T11:30:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.827+0000 7fde4effd700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 0x7fde3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.828+0000 7fde4effd700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 0x7fde3c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde400113d0 tx=0x7fde40009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:44.831+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde440537b0 con 0x7fde500cb9a0 2023-06-09T11:30:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.091+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde50074cf0 con 0x7fde500cb9a0 2023-06-09T11:30:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.091+0000 7fde377fe700 1 -- 172.21.15.7:0/1390830823 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+5457 (secure 0 0 0) 0x7fde44056dd0 con 0x7fde500cb9a0 2023-06-09T11:30:45.096 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:45.097 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":86,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:40.557872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:45.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 msgr2=0x7fde3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 0x7fde3c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fde400113d0 tx=0x7fde40009210 comp rx=0 tx=0).stop 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 msgr2=0x7fde50151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 0x7fde50151870 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fde44000f60 tx=0x7fde4400c730 comp rx=0 tx=0).stop 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 shutdown_connections 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde5015cd40 0x7fde500cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:45.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.093+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde3c06ace0 0x7fde3c06d190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.094+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde500cb5d0 0x7fde500c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.094+0000 7fde555ff700 1 --2- 172.21.15.7:0/1390830823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde500cb9a0 0x7fde50151870 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.094+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 >> 172.21.15.7:0/1390830823 conn(0x7fde50070af0 msgr2=0x7fde50158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.094+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 shutdown_connections 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:45.094+0000 7fde555ff700 1 -- 172.21.15.7:0/1390830823 wait complete. 2023-06-09T11:30:45.100 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 86 2023-06-09T11:30:45.116 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:46.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:46.078 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:30:47.118 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:47.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.323+0000 7f1c9b9ff700 1 Processor -- start 2023-06-09T11:30:47.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.324+0000 7f1c9b9ff700 1 -- start start 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9b9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1c9415b4a0 0x7f1c9415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9b9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c940c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9b9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9415a500 con 0x7f1c94153880 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9b9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c940c5f10 con 0x7f1c9415b4a0 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9b9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9415da00 con 0x7f1c940c9f40 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9979b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1c94153880 0x7f1c9415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55746/0 (socket says 172.21.15.7:55746) 2023-06-09T11:30:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c9979b700 1 -- 172.21.15.7:0/3030978785 learned_addr learned my addr 172.21.15.7:0/3030978785 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c98f9a700 1 --2- 172.21.15.7:0/3030978785 >> v2:172.21.15.117:3300/0 conn(0x7f1c9415b4a0 0x7f1c9415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c940c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.325+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079369554 0 0) 0x7f1c90001110 con 0x7f1c94153880 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c800035b0 con 0x7f1c94153880 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342631366 0 0) 0x7f1c800035b0 con 0x7f1c94153880 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 msgr2=0x7f1c940c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c940c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 >> v2:172.21.15.117:3300/0 conn(0x7f1c9415b4a0 msgr2=0x7f1c9415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 --2- 172.21.15.7:0/3030978785 >> v2:172.21.15.117:3300/0 conn(0x7f1c9415b4a0 0x7f1c9415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c941596e0 con 0x7f1c94153880 2023-06-09T11:30:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f1c90001800 con 0x7f1c94153880 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 >> v1:172.21.15.7:6789/0 conn(0x7f1c94153880 legacy=0x7f1c9415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c80003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80003f80 0x7f1c80004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c80004d90 0x7f1c80005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940c5f10 con 0x7f1c80004d90 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9415da00 con 0x7f1c80003f80 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8000ba40 con 0x7f1c940c9f40 2023-06-09T11:30:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c80004d90 0x7f1c80005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c98f9a700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80003f80 0x7f1c80004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.326+0000 7f1c9979b700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c80003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 -- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 msgr2=0x7f1c80003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c940c9f40 0x7f1c80003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 -- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80003f80 msgr2=0x7f1c80004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c80003f80 0x7f1c80004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 -- 172.21.15.7:0/3030978785 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800035b0 con 0x7f1c80004d90 2023-06-09T11:30:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c80004d90 0x7f1c80005200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fc0 tx=0x7f1c80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ffbf2dd2c49112 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c00ebd0 con 0x7f1c80004d90 2023-06-09T11:30:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c00f040 con 0x7f1c80004d90 2023-06-09T11:30:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.327+0000 7f1c8bfff700 1 -- 172.21.15.7:0/3030978785 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c014480 con 0x7f1c80004d90 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/3030978785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c80004d90 msgr2=0x7f1c80005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/3030978785 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c80004d90 0x7f1c80005200 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009fc0 tx=0x7f1c80005880 comp rx=0 tx=0).stop 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c99f9c700 1 -- 172.21.15.7:0/3030978785 reap_dead start 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/3030978785 shutdown_connections 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/3030978785 >> 172.21.15.7:0/3030978785 conn(0x7f1c940737f0 msgr2=0x7f1c940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/3030978785 shutdown_connections 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.328+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/3030978785 wait complete. 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.330+0000 7f1c9b9ff700 1 Processor -- start 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.330+0000 7f1c9b9ff700 1 -- start start 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c80004d90 0x7f1c940cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c940cb8d0 0x7f1c9414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c941594a0 con 0x7f1c940cb460 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940b4b40 con 0x7f1c940cb8d0 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9b9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c940b3430 con 0x7f1c80004d90 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37678/0 (socket says 172.21.15.7:37678) 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 -- 172.21.15.7:0/727935720 learned_addr learned my addr 172.21.15.7:0/727935720 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c99f9c700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c940cb8d0 0x7f1c9414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c9979b700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c80004d90 0x7f1c940cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 -- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c80004d90 msgr2=0x7f1c940cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c80004d90 0x7f1c940cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 -- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c940cb8d0 msgr2=0x7f1c9414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c940cb8d0 0x7f1c9414c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.331+0000 7f1c98f9a700 1 -- 172.21.15.7:0/727935720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94159c60 con 0x7f1c940cb460 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c98f9a700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1c84011ad0 tx=0x7f1c84011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400f8e0 con 0x7f1c940cb460 2023-06-09T11:30:47.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8400fa60 con 0x7f1c940cb460 2023-06-09T11:30:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c94159980 con 0x7f1c940cb460 2023-06-09T11:30:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400fe10 con 0x7f1c940cb460 2023-06-09T11:30:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.332+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c94159f40 con 0x7f1c940cb460 2023-06-09T11:30:47.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.333+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c940b4880 con 0x7f1c940cb460 2023-06-09T11:30:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.334+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1c84014440 con 0x7f1c940cb460 2023-06-09T11:30:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.335+0000 7f1c89ffb700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 0x7f1c7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.335+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f1c84089f00 con 0x7f1c940cb460 2023-06-09T11:30:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.335+0000 7f1c9979b700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 0x7f1c7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:47.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.336+0000 7f1c9979b700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 0x7f1c7806d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c90001050 tx=0x7f1c90007b00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:47.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.339+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1c84054be0 con 0x7f1c940cb460 2023-06-09T11:30:47.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.594+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c940b4c80 con 0x7f1c940cb460 2023-06-09T11:30:47.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.595+0000 7f1c89ffb700 1 -- 172.21.15.7:0/727935720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f1c940b4c80 con 0x7f1c940cb460 2023-06-09T11:30:47.600 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:47.600 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:46.393718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:47.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 msgr2=0x7f1c7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 0x7f1c7806d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1c90001050 tx=0x7f1c90007b00 comp rx=0 tx=0).stop 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 msgr2=0x7f1c9414c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1c84011ad0 tx=0x7f1c84011de0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 shutdown_connections 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c940cb8d0 0x7f1c9414c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1c7806ace0 0x7f1c7806d190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c80004d90 0x7f1c940cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 --2- 172.21.15.7:0/727935720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c940cb460 0x7f1c9414c250 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.597+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 >> 172.21.15.7:0/727935720 conn(0x7f1c940737f0 msgr2=0x7f1c9415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.598+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 shutdown_connections 2023-06-09T11:30:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:47.598+0000 7f1c9b9ff700 1 -- 172.21.15.7:0/727935720 wait complete. 2023-06-09T11:30:47.604 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 87 2023-06-09T11:30:47.621 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 87, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 87, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5531": { "addr": "172.21.15.117:6833/1018690909", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1018690909, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1018690909, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5531, "incarnation": 85, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 14 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:30:46.393718+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5531, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:30:49.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.829+0000 7f5221bff700 1 Processor -- start 2023-06-09T11:30:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.829+0000 7f5221bff700 1 -- start start 2023-06-09T11:30:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.830+0000 7f5221bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c153880 0x7f521c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.830+0000 7f5221bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f521c15b4a0 0x7f521c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f5221bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f521c15a500 con 0x7f521c0c9f40 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f5221bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f521c0c5f10 con 0x7f521c15b4a0 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f5221bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c15da00 con 0x7f521c153880 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f521c0c9f40 0x7f521c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55752/0 (socket says 172.21.15.7:55752) 2023-06-09T11:30:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521affd700 1 -- 172.21.15.7:0/466344358 learned_addr learned my addr 172.21.15.7:0/466344358 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:49.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521bfff700 1 --2- 172.21.15.7:0/466344358 >> v2:172.21.15.7:3300/0 conn(0x7f521c15b4a0 0x7f521c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521b7fe700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c153880 0x7f521c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521a7fc700 1 -- 172.21.15.7:0/466344358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852243681 0 0) 0x7f5210001110 con 0x7f521c0c9f40 2023-06-09T11:30:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521a7fc700 1 -- 172.21.15.7:0/466344358 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52040035b0 con 0x7f521c0c9f40 2023-06-09T11:30:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521bfff700 1 -- 172.21.15.7:0/466344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c153880 msgr2=0x7f521c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:49.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521bfff700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c153880 0x7f521c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521bfff700 1 -- 172.21.15.7:0/466344358 >> v1:172.21.15.7:6789/0 conn(0x7f521c0c9f40 legacy=0x7f521c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.831+0000 7f521bfff700 1 -- 172.21.15.7:0/466344358 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521c1596e0 con 0x7f521c15b4a0 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f521bfff700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c0c4e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5214009d90 tx=0x7f521400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4626b65e17e70a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f521a7fc700 1 -- 172.21.15.7:0/466344358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400e070 con 0x7f521c15b4a0 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f521a7fc700 1 -- 172.21.15.7:0/466344358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521400bb20 con 0x7f521c15b4a0 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f521a7fc700 1 -- 172.21.15.7:0/466344358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521400be90 con 0x7f521c15b4a0 2023-06-09T11:30:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f5221bff700 1 -- 172.21.15.7:0/466344358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 msgr2=0x7f521c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.832+0000 7f5221bff700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c0c4e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5214009d90 tx=0x7f521400b330 comp rx=0 tx=0).stop 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 -- 172.21.15.7:0/466344358 shutdown_connections 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c0c4e30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 --2- 172.21.15.7:0/466344358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c153880 0x7f521c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 -- 172.21.15.7:0/466344358 >> 172.21.15.7:0/466344358 conn(0x7f521c0737f0 msgr2=0x7f521c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 -- 172.21.15.7:0/466344358 shutdown_connections 2023-06-09T11:30:49.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.833+0000 7f5221bff700 1 -- 172.21.15.7:0/466344358 wait complete. 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.834+0000 7f5221bff700 1 Processor -- start 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.834+0000 7f5221bff700 1 -- start start 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c0c9f40 0x7f521c1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f521c153880 0x7f521c0cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c1552a0 con 0x7f521c15b4a0 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c14b320 con 0x7f521c153880 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.835+0000 7f5221bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f521c155620 con 0x7f521c0c9f40 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37696/0 (socket says 172.21.15.7:37696) 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 -- 172.21.15.7:0/268050204 learned_addr learned my addr 172.21.15.7:0/268050204 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521affd700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f521c153880 0x7f521c0cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521b7fe700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c0c9f40 0x7f521c1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 -- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c0c9f40 msgr2=0x7f521c1f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c0c9f40 0x7f521c1f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 -- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f521c153880 msgr2=0x7f521c0cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f521c153880 0x7f521c0cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 -- 172.21.15.7:0/268050204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009a70 con 0x7f521c15b4a0 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.836+0000 7f521bfff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5214014040 tx=0x7f5214004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.837+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214007000 con 0x7f521c15b4a0 2023-06-09T11:30:49.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.837+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5214012860 con 0x7f521c15b4a0 2023-06-09T11:30:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.837+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c159c60 con 0x7f521c15b4a0 2023-06-09T11:30:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.837+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214012c10 con 0x7f521c15b4a0 2023-06-09T11:30:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.837+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c159980 con 0x7f521c15b4a0 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.838+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c159f40 con 0x7f521c15b4a0 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.839+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f521401c400 con 0x7f521c15b4a0 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.840+0000 7f5203fff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 0x7f520806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.840+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f521408b840 con 0x7f521c15b4a0 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.840+0000 7f521b7fe700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 0x7f520806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:49.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.840+0000 7f521b7fe700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 0x7f520806d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f520c009930 tx=0x7f520c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:49.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:49.844+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f52140566c0 con 0x7f521c15b4a0 2023-06-09T11:30:50.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.096+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f521c1530e0 con 0x7f521c15b4a0 2023-06-09T11:30:50.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.096+0000 7f5203fff700 1 -- 172.21.15.7:0/268050204 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f5214059ce0 con 0x7f521c15b4a0 2023-06-09T11:30:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:50.102 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":87,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:46.393718+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.098+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 msgr2=0x7f520806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 0x7f520806d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f520c009930 tx=0x7f520c008040 comp rx=0 tx=0).stop 2023-06-09T11:30:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 msgr2=0x7f521c155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:50.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5214014040 tx=0x7f5214004ad0 comp rx=0 tx=0).stop 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 shutdown_connections 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f521c15b4a0 0x7f521c155dc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f520806ace0 0x7f520806d190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f521c0c9f40 0x7f521c1f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 --2- 172.21.15.7:0/268050204 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f521c153880 0x7f521c0cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 >> 172.21.15.7:0/268050204 conn(0x7f521c0737f0 msgr2=0x7f521c14bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:50.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 shutdown_connections 2023-06-09T11:30:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:50.099+0000 7f5221bff700 1 -- 172.21.15.7:0/268050204 wait complete. 2023-06-09T11:30:50.106 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 87 2023-06-09T11:30:50.121 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:52.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:52.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.328+0000 7fdad41ff700 1 Processor -- start 2023-06-09T11:30:52.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.328+0000 7fdad41ff700 1 -- start start 2023-06-09T11:30:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad41ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdacc0c9090 0x7fdacc0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad41ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdacc1598b0 0x7fdacc159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad41ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdacc15cec0 con 0x7fdacc0cad00 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad41ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc15d160 con 0x7fdacc0c9090 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad41ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc15d440 con 0x7fdacc1598b0 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad1f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdacc0c9090 0x7fdacc0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad179a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdacc0cad00 0x7fdacc0cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55760/0 (socket says 172.21.15.7:55760) 2023-06-09T11:30:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.329+0000 7fdad179a700 1 -- 172.21.15.7:0/1254756936 learned_addr learned my addr 172.21.15.7:0/1254756936 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad279c700 1 --2- 172.21.15.7:0/1254756936 >> v2:172.21.15.117:3300/0 conn(0x7fdacc1598b0 0x7fdacc159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad1f9b700 1 -- 172.21.15.7:0/1254756936 >> v2:172.21.15.117:3300/0 conn(0x7fdacc1598b0 msgr2=0x7fdacc159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad1f9b700 1 --2- 172.21.15.7:0/1254756936 >> v2:172.21.15.117:3300/0 conn(0x7fdacc1598b0 0x7fdacc159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad1f9b700 1 -- 172.21.15.7:0/1254756936 >> v1:172.21.15.7:6789/0 conn(0x7fdacc0cad00 legacy=0x7fdacc0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad1f9b700 1 -- 172.21.15.7:0/1254756936 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc15d720 con 0x7fdacc0c9090 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad0f99700 1 -- 172.21.15.7:0/1254756936 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4004784243 0 0) 0x7fdacc15cec0 con 0x7fdacc0cad00 2023-06-09T11:30:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad1f9b700 1 --2- 172.21.15.7:0/1254756936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc0c9090 0x7fdacc0c9fe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cd510 tx=0x7fdac8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3385b0f44e65812 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.330+0000 7fdad0f99700 1 -- 172.21.15.7:0/1254756936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8011830 con 0x7fdacc0c9090 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad0f99700 1 -- 172.21.15.7:0/1254756936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac8011990 con 0x7fdacc0c9090 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad0f99700 1 -- 172.21.15.7:0/1254756936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8011d20 con 0x7fdacc0c9090 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 -- 172.21.15.7:0/1254756936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc0c9090 msgr2=0x7fdacc0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 --2- 172.21.15.7:0/1254756936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc0c9090 0x7fdacc0c9fe0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdacc0cd510 tx=0x7fdac8009b20 comp rx=0 tx=0).stop 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 -- 172.21.15.7:0/1254756936 shutdown_connections 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 --2- 172.21.15.7:0/1254756936 >> v2:172.21.15.117:3300/0 conn(0x7fdacc1598b0 0x7fdacc159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 --2- 172.21.15.7:0/1254756936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc0c9090 0x7fdacc0c9fe0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 -- 172.21.15.7:0/1254756936 >> 172.21.15.7:0/1254756936 conn(0x7fdacc070af0 msgr2=0x7fdacc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 -- 172.21.15.7:0/1254756936 shutdown_connections 2023-06-09T11:30:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.331+0000 7fdad41ff700 1 -- 172.21.15.7:0/1254756936 wait complete. 2023-06-09T11:30:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.333+0000 7fdad41ff700 1 Processor -- start 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.333+0000 7fdad41ff700 1 -- start start 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdacc0c9090 0x7fdacc155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdacc0cad00 0x7fdacc0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc1e2e60 con 0x7fdacc1598b0 2023-06-09T11:30:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0c4ee0 con 0x7fdacc0cad00 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0c3500 con 0x7fdacc0c9090 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad279c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad279c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37700/0 (socket says 172.21.15.7:37700) 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.334+0000 7fdad279c700 1 -- 172.21.15.7:0/171505570 learned_addr learned my addr 172.21.15.7:0/171505570 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad1f9b700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdacc0c9090 0x7fdacc155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad179a700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdacc0cad00 0x7fdacc0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 -- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdacc0c9090 msgr2=0x7fdacc155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdacc0c9090 0x7fdacc155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 -- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdacc0cad00 msgr2=0x7fdacc0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdacc0cad00 0x7fdacc0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 -- 172.21.15.7:0/171505570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80094a0 con 0x7fdacc1598b0 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad279c700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007e60 tx=0x7fdac400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400ede0 con 0x7fdacc1598b0 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac4014420 con 0x7fdacc1598b0 2023-06-09T11:30:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40147d0 con 0x7fdacc1598b0 2023-06-09T11:30:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdacc15a210 con 0x7fdacc1598b0 2023-06-09T11:30:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.335+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc0ca620 con 0x7fdacc1598b0 2023-06-09T11:30:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.336+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdacc0cb160 con 0x7fdacc1598b0 2023-06-09T11:30:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.337+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdac4014930 con 0x7fdacc1598b0 2023-06-09T11:30:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.338+0000 7fdac27fc700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 0x7fdab006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.339+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fdac408a070 con 0x7fdacc1598b0 2023-06-09T11:30:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.339+0000 7fdad1f9b700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 0x7fdab006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.339+0000 7fdad1f9b700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 0x7fdab006d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdac80090b0 tx=0x7fdac801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.343+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdac40bc2d0 con 0x7fdacc1598b0 2023-06-09T11:30:52.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.595+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdacc074cf0 con 0x7fdacc1598b0 2023-06-09T11:30:52.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.596+0000 7fdac27fc700 1 -- 172.21.15.7:0/171505570 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fdac4058720 con 0x7fdacc1598b0 2023-06-09T11:30:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:52.601 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:52.412544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 msgr2=0x7fdab006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 0x7fdab006d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdac80090b0 tx=0x7fdac801c040 comp rx=0 tx=0).stop 2023-06-09T11:30:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 msgr2=0x7fdacc0cdaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007e60 tx=0x7fdac400e3c0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 shutdown_connections 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdacc1598b0 0x7fdacc0cdaa0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdab006ace0 0x7fdab006d190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdacc0c9090 0x7fdacc155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 --2- 172.21.15.7:0/171505570 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdacc0cad00 0x7fdacc0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 >> 172.21.15.7:0/171505570 conn(0x7fdacc070af0 msgr2=0x7fdacc0b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 shutdown_connections 2023-06-09T11:30:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:52.598+0000 7fdad41ff700 1 -- 172.21.15.7:0/171505570 wait complete. 2023-06-09T11:30:52.605 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 88 2023-06-09T11:30:52.622 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:54.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:54.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.829+0000 7f65b51ff700 1 Processor -- start 2023-06-09T11:30:54.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.829+0000 7f65b51ff700 1 -- start start 2023-06-09T11:30:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.830+0000 7f65b51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f65b0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.830+0000 7f65b51ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b01580a0 con 0x7f65b0151510 2023-06-09T11:30:54.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.830+0000 7f65b51ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b00c6180 con 0x7f65b0071060 2023-06-09T11:30:54.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.830+0000 7f65b51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00714e0 con 0x7f65b00c5b40 2023-06-09T11:30:54.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65aed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65b0151510 0x7f65b015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37714/0 (socket says 172.21.15.7:37714) 2023-06-09T11:30:54.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65aed9d700 1 -- 172.21.15.7:0/3555151595 learned_addr learned my addr 172.21.15.7:0/3555151595 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65ae59c700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f65b0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805364210 0 0) 0x7f65b01580a0 con 0x7f65b0151510 2023-06-09T11:30:54.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65980035b0 con 0x7f65b0151510 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307931177 0 0) 0x7f65b00c6180 con 0x7f65b0071060 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b01580a0 con 0x7f65b0071060 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3833551683 0 0) 0x7f65980035b0 con 0x7f65b0151510 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 msgr2=0x7f65b0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f65b0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 >> v1:172.21.15.117:6789/0 conn(0x7f65b0071060 legacy=0x7f65b0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:54.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b0157490 con 0x7f65b0151510 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f65a00015f0 con 0x7f65b0151510 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 >> v1:172.21.15.7:6789/0 conn(0x7f65b0151510 legacy=0x7f65b015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598003bc0 0x7f6598003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598004850 0x7f6598004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f6598005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.831+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b00714e0 con 0x7f6598003bc0 2023-06-09T11:30:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659800bac0 con 0x7f6598004850 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659800bc80 con 0x7f65b00c5b40 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598003bc0 0x7f6598003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65ae59c700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598004850 0x7f6598004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65af59e700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f6598005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 -- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 msgr2=0x7f6598005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b00c5b40 0x7f6598005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 -- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598004850 msgr2=0x7f6598004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:30:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598004850 0x7f6598004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 -- 172.21.15.7:0/3555151595 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65980035b0 con 0x7f6598003bc0 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65ae59c700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598004850 0x7f6598004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.832+0000 7f65aed9d700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598003bc0 0x7f6598003fb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007f70 tx=0x7f6598005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7512d59ffac46095 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0007a60 con 0x7f6598003bc0 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a000cbf0 con 0x7f6598003bc0 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65add9b700 1 -- 172.21.15.7:0/3555151595 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0005960 con 0x7f6598003bc0 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 -- 172.21.15.7:0/3555151595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598003bc0 msgr2=0x7f6598003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 --2- 172.21.15.7:0/3555151595 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6598003bc0 0x7f6598003fb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f65a0007f70 tx=0x7f6598005880 comp rx=0 tx=0).stop 2023-06-09T11:30:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65af59e700 1 -- 172.21.15.7:0/3555151595 reap_dead start 2023-06-09T11:30:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 -- 172.21.15.7:0/3555151595 shutdown_connections 2023-06-09T11:30:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 -- 172.21.15.7:0/3555151595 >> 172.21.15.7:0/3555151595 conn(0x7f65b006a4d0 msgr2=0x7f65b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 -- 172.21.15.7:0/3555151595 shutdown_connections 2023-06-09T11:30:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.833+0000 7f65b51ff700 1 -- 172.21.15.7:0/3555151595 wait complete. 2023-06-09T11:30:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.835+0000 7f65b51ff700 1 Processor -- start 2023-06-09T11:30:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.835+0000 7f65b51ff700 1 -- start start 2023-06-09T11:30:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 0x7f65b0149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 0x7f65b00c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b0158e00 con 0x7f65b0158430 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b01599c0 con 0x7f6598003bc0 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.836+0000 7f65b51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b015a650 con 0x7f65b015a1c0 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39864/0 (socket says 172.21.15.7:39864) 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 -- 172.21.15.7:0/2687179297 learned_addr learned my addr 172.21.15.7:0/2687179297 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65aed9d700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 0x7f65b0149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65af59e700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 0x7f65b00c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 -- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 msgr2=0x7f65b00c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 0x7f65b00c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 -- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 msgr2=0x7f65b0149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 0x7f65b0149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 -- 172.21.15.7:0/2687179297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b0157a50 con 0x7f65b0158430 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65af59e700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 0x7f65b00c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:30:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65aed9d700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 0x7f65b0149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65ae59c700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f65a4011cd0 tx=0x7f65a400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.837+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a400fad0 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.838+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a400fc30 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.838+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a4004020 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.838+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b0157770 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.838+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b0157d30 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.839+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b0074cf0 con 0x7f65b0158430 2023-06-09T11:30:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.840+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65a4004180 con 0x7f65b0158430 2023-06-09T11:30:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.841+0000 7f65977fe700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 0x7f659c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.841+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f65a4089df0 con 0x7f65b0158430 2023-06-09T11:30:54.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.841+0000 7f65aed9d700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 0x7f659c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.842+0000 7f65aed9d700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 0x7f659c06d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65a0015040 tx=0x7f65a0007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:54.845+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f65a4054ab0 con 0x7f65b0158430 2023-06-09T11:30:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.099+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65b00b10b0 con 0x7f65b0158430 2023-06-09T11:30:55.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.100+0000 7f65977fe700 1 -- 172.21.15.7:0/2687179297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f65a4022020 con 0x7f65b0158430 2023-06-09T11:30:55.105 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:55.106 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:52.412544+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:55.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 msgr2=0x7f659c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 0x7f659c06d190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f65a0015040 tx=0x7f65a0007000 comp rx=0 tx=0).stop 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 msgr2=0x7f65b0159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f65a4011cd0 tx=0x7f65a400f520 comp rx=0 tx=0).stop 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 shutdown_connections 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f659c06ace0 0x7f659c06d190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:55.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.102+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65b015a1c0 0x7f65b00c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.103+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6598003bc0 0x7f65b0149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.103+0000 7f65b51ff700 1 --2- 172.21.15.7:0/2687179297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f65b0158430 0x7f65b0159c80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.103+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 >> 172.21.15.7:0/2687179297 conn(0x7f65b006a4d0 msgr2=0x7f65b014ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.103+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 shutdown_connections 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:55.103+0000 7f65b51ff700 1 -- 172.21.15.7:0/2687179297 wait complete. 2023-06-09T11:30:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 88 2023-06-09T11:30:55.126 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:57.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:57.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.334+0000 7f3ef25ff700 1 Processor -- start 2023-06-09T11:30:57.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.334+0000 7f3ef25ff700 1 -- start start 2023-06-09T11:30:57.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3ef25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 0x7f3eec0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:57.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3ef25ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec0c5a80 con 0x7f3eec153880 2023-06-09T11:30:57.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3ef25ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec15a500 con 0x7f3eec15b4a0 2023-06-09T11:30:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3ef25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0c5f10 con 0x7f3eec0c9f40 2023-06-09T11:30:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3eebfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3eec153880 0x7f3eec15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37730/0 (socket says 172.21.15.7:37730) 2023-06-09T11:30:57.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.335+0000 7f3eebfff700 1 -- 172.21.15.7:0/3087968239 learned_addr learned my addr 172.21.15.7:0/3087968239 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:57.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3ef0b9c700 1 --2- 172.21.15.7:0/3087968239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 0x7f3eec0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:57.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581528227 0 0) 0x7f3eec0c5a80 con 0x7f3eec153880 2023-06-09T11:30:57.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed40035b0 con 0x7f3eec153880 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3ef0b9c700 1 -- 172.21.15.7:0/3087968239 >> v1:172.21.15.117:6789/0 conn(0x7f3eec15b4a0 legacy=0x7f3eec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330340712 0 0) 0x7f3ee0001110 con 0x7f3eec15b4a0 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3ef0b9c700 1 -- 172.21.15.7:0/3087968239 >> v1:172.21.15.7:6789/0 conn(0x7f3eec153880 legacy=0x7f3eec15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3ef0b9c700 1 -- 172.21.15.7:0/3087968239 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec1596e0 con 0x7f3eec0c9f40 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.336+0000 7f3ef0b9c700 1 --2- 172.21.15.7:0/3087968239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 0x7f3eec0c5540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009dc0 tx=0x7f3ee400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d2f522006d99b8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400e070 con 0x7f3eec0c9f40 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee400bb20 con 0x7f3eec0c9f40 2023-06-09T11:30:57.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3eeaffd700 1 -- 172.21.15.7:0/3087968239 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400bed0 con 0x7f3eec0c9f40 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 -- 172.21.15.7:0/3087968239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 msgr2=0x7f3eec0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/3087968239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 0x7f3eec0c5540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009dc0 tx=0x7f3ee400b330 comp rx=0 tx=0).stop 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 -- 172.21.15.7:0/3087968239 shutdown_connections 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/3087968239 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec0c9f40 0x7f3eec0c5540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 -- 172.21.15.7:0/3087968239 >> 172.21.15.7:0/3087968239 conn(0x7f3eec0737f0 msgr2=0x7f3eec0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 -- 172.21.15.7:0/3087968239 shutdown_connections 2023-06-09T11:30:57.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.337+0000 7f3ef25ff700 1 -- 172.21.15.7:0/3087968239 wait complete. 2023-06-09T11:30:57.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.340+0000 7f3ef25ff700 1 Processor -- start 2023-06-09T11:30:57.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.340+0000 7f3ef25ff700 1 -- start start 2023-06-09T11:30:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec153880 0x7f3eec1ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:57.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 0x7f3eec14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec157750 con 0x7f3eec0c9f40 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0ce2e0 con 0x7f3eec15b4a0 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3ef25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1eea70 con 0x7f3eec153880 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3eebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39870/0 (socket says 172.21.15.7:39870) 2023-06-09T11:30:57.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.341+0000 7f3eebfff700 1 -- 172.21.15.7:0/194047716 learned_addr learned my addr 172.21.15.7:0/194047716 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3ef0b9c700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 0x7f3eec14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:57.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eeb7fe700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec153880 0x7f3eec1ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 -- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec153880 msgr2=0x7f3eec1ef250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec153880 0x7f3eec1ef250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 -- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 msgr2=0x7f3eec14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 0x7f3eec14ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 -- 172.21.15.7:0/194047716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee4009a70 con 0x7f3eec0c9f40 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3ef0b9c700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 0x7f3eec14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:30:57.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3eebfff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00c4d0 tx=0x7f3edc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc005d70 con 0x7f3eec0c9f40 2023-06-09T11:30:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.342+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc0070d0 con 0x7f3eec0c9f40 2023-06-09T11:30:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.343+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec159c60 con 0x7f3eec0c9f40 2023-06-09T11:30:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.343+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc007440 con 0x7f3eec0c9f40 2023-06-09T11:30:57.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.343+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec159980 con 0x7f3eec0c9f40 2023-06-09T11:30:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.344+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec159f40 con 0x7f3eec0c9f40 2023-06-09T11:30:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.344+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3edc0075a0 con 0x7f3eec0c9f40 2023-06-09T11:30:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.345+0000 7f3ee8ff9700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 0x7f3ed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.345+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f3edc088d40 con 0x7f3eec0c9f40 2023-06-09T11:30:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.346+0000 7f3eeb7fe700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 0x7f3ed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:57.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.346+0000 7f3eeb7fe700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 0x7f3ed806d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3eec14bcb0 tx=0x7f3ee0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:57.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.350+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3edc053a10 con 0x7f3eec0c9f40 2023-06-09T11:30:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.603+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eec074cf0 con 0x7f3eec0c9f40 2023-06-09T11:30:57.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.604+0000 7f3ee8ff9700 1 -- 172.21.15.7:0/194047716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f3edc017090 con 0x7f3eec0c9f40 2023-06-09T11:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:30:57.609 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:57.533244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:clientreplay","state_seq":18,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:30:57.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 msgr2=0x7f3ed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 0x7f3ed806d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3eec14bcb0 tx=0x7f3ee0009040 comp rx=0 tx=0).stop 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 msgr2=0x7f3eec14f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3edc00c4d0 tx=0x7f3edc00c890 comp rx=0 tx=0).stop 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 shutdown_connections 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3eec15b4a0 0x7f3eec14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eec0c9f40 0x7f3eec14f260 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ed806ace0 0x7f3ed806d190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 --2- 172.21.15.7:0/194047716 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3eec153880 0x7f3eec1ef250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 >> 172.21.15.7:0/194047716 conn(0x7f3eec0737f0 msgr2=0x7f3eec14d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 shutdown_connections 2023-06-09T11:30:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:57.606+0000 7f3ef25ff700 1 -- 172.21.15.7:0/194047716 wait complete. 2023-06-09T11:30:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2023-06-09T11:30:57.629 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:30:59.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:30:59.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.836+0000 7f7ce8fff700 1 Processor -- start 2023-06-09T11:30:59.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.836+0000 7f7ce8fff700 1 -- start start 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce8fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7ce40c2d90 0x7f7ce40c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce8fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce40c88d0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce8fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ce4153e40 con 0x7f7ce4153a70 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce8fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ce40c2bf0 con 0x7f7ce40c2d90 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7ce415a0f0 0x7f7ce40cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37734/0 (socket says 172.21.15.7:37734) 2023-06-09T11:30:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce2d9d700 1 -- 172.21.15.7:0/4182406569 learned_addr learned my addr 172.21.15.7:0/4182406569 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce1d9b700 1 --2- 172.21.15.7:0/4182406569 >> v2:172.21.15.117:3300/0 conn(0x7f7ce40c2d90 0x7f7ce40c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39943104 0 0) 0x7f7ce40c88d0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.838+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ccc0035b0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce1d9b700 1 -- 172.21.15.7:0/4182406569 >> v1:172.21.15.117:6789/0 conn(0x7f7ce4153a70 legacy=0x7f7ce40ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968061953 0 0) 0x7f7cd4001110 con 0x7f7ce4153a70 2023-06-09T11:30:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce1d9b700 1 -- 172.21.15.7:0/4182406569 >> v1:172.21.15.7:6789/0 conn(0x7f7ce415a0f0 legacy=0x7f7ce40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce1d9b700 1 -- 172.21.15.7:0/4182406569 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40ca350 con 0x7f7ce40c2d90 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce1d9b700 1 --2- 172.21.15.7:0/4182406569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce40c26b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009840 tx=0x7f7cd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723d70dcf685744f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8011830 con 0x7f7ce40c2d90 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd8011990 con 0x7f7ce40c2d90 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.839+0000 7f7ce159a700 1 -- 172.21.15.7:0/4182406569 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8011d20 con 0x7f7ce40c2d90 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 -- 172.21.15.7:0/4182406569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 msgr2=0x7f7ce40c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/4182406569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce40c26b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009840 tx=0x7f7cd8009b50 comp rx=0 tx=0).stop 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 -- 172.21.15.7:0/4182406569 shutdown_connections 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/4182406569 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce40c26b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 -- 172.21.15.7:0/4182406569 >> 172.21.15.7:0/4182406569 conn(0x7f7ce406e410 msgr2=0x7f7ce406e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 -- 172.21.15.7:0/4182406569 shutdown_connections 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.840+0000 7f7ce8fff700 1 -- 172.21.15.7:0/4182406569 wait complete. 2023-06-09T11:30:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.842+0000 7f7ce8fff700 1 Processor -- start 2023-06-09T11:30:59.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.843+0000 7f7ce8fff700 1 -- start start 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce41e3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ce4153a70 0x7f7ce41e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41e3540 con 0x7f7ce415a0f0 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce406d550 con 0x7f7ce40c2d90 2023-06-09T11:30:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce8fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce41f2b50 con 0x7f7ce4153a70 2023-06-09T11:30:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39882/0 (socket says 172.21.15.7:39882) 2023-06-09T11:30:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce2d9d700 1 -- 172.21.15.7:0/112967532 learned_addr learned my addr 172.21.15.7:0/112967532 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:30:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce259c700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce41e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.844+0000 7f7ce1d9b700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ce4153a70 0x7f7ce41e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 -- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ce4153a70 msgr2=0x7f7ce41e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ce4153a70 0x7f7ce41e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 -- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 msgr2=0x7f7ce41e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce41e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 -- 172.21.15.7:0/112967532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd80094a0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce2d9d700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00caa0 tx=0x7f7cdc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00a9e0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cdc00ab40 con 0x7f7ce415a0f0 2023-06-09T11:30:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc012400 con 0x7f7ce415a0f0 2023-06-09T11:30:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce40c85f0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.845+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce40c8350 con 0x7f7ce415a0f0 2023-06-09T11:30:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.846+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce40c9d00 con 0x7f7ce415a0f0 2023-06-09T11:30:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.847+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cdc01b3b0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.848+0000 7f7cd2ffd700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:30:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.848+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f7cdc0895c0 con 0x7f7ce415a0f0 2023-06-09T11:30:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.849+0000 7f7ce259c700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:30:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.849+0000 7f7ce259c700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4000f60 tx=0x7f7cd4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:30:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:30:59.853+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7cdc054210 con 0x7f7ce415a0f0 2023-06-09T11:31:00.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.106+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ce41f3070 con 0x7f7ce415a0f0 2023-06-09T11:31:00.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.106+0000 7f7cd2ffd700 1 -- 172.21.15.7:0/112967532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f7cdc057830 con 0x7f7ce415a0f0 2023-06-09T11:31:00.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:31:00.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:57.533244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:clientreplay","state_seq":18,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:31:00.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 msgr2=0x7f7cc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:00.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 0x7f7cc406d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4000f60 tx=0x7f7cd4009040 comp rx=0 tx=0).stop 2023-06-09T11:31:00.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 msgr2=0x7f7ce41f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:00.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc00caa0 tx=0x7f7cdc00cdb0 comp rx=0 tx=0).stop 2023-06-09T11:31:00.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 shutdown_connections 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ce415a0f0 0x7f7ce41f02e0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7ce40c2d90 0x7f7ce41e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7cc406ace0 0x7f7cc406d190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 --2- 172.21.15.7:0/112967532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7ce4153a70 0x7f7ce41e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 >> 172.21.15.7:0/112967532 conn(0x7f7ce406e410 msgr2=0x7f7ce4159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 shutdown_connections 2023-06-09T11:31:00.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:00.109+0000 7f7ce8fff700 1 -- 172.21.15.7:0/112967532 wait complete. 2023-06-09T11:31:00.116 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2023-06-09T11:31:00.132 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 89, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 89, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5531": { "addr": "172.21.15.117:6833/1018690909", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1018690909, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1018690909, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5531, "incarnation": 85, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:clientreplay", "state_seq": 18 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 60, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:30:57.533244+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5531, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:31:02.134 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:31:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.338+0000 7f843dfff700 1 Processor -- start 2023-06-09T11:31:02.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.338+0000 7f843dfff700 1 -- start start 2023-06-09T11:31:02.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 0x7f843815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84380c9f40 0x7f843815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f84380c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84380c5f10 con 0x7f84380c9f40 2023-06-09T11:31:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843815a500 con 0x7f8438153880 2023-06-09T11:31:02.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f843dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843815da00 con 0x7f843815b2d0 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84380c9f40 0x7f843815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f84380c9f40 0x7f843815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39892/0 (socket says 172.21.15.7:39892) 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 -- 172.21.15.7:0/1717534997 learned_addr learned my addr 172.21.15.7:0/1717534997 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8437fff700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f84380c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f84377fe700 1 --2- 172.21.15.7:0/1717534997 >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 0x7f843815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 -- 172.21.15.7:0/1717534997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 msgr2=0x7f84380c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f84380c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 -- 172.21.15.7:0/1717534997 >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 msgr2=0x7f843815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 --2- 172.21.15.7:0/1717534997 >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 0x7f843815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f8436ffd700 1 -- 172.21.15.7:0/1717534997 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84381596e0 con 0x7f84380c9f40 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.340+0000 7f84377fe700 1 --2- 172.21.15.7:0/1717534997 >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 0x7f843815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f8436ffd700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f843815ad90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f843808d340 tx=0x7f842c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5adc0d0cde82b7f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f84367fc700 1 -- 172.21.15.7:0/1717534997 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00b870 con 0x7f84380c9f40 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f84367fc700 1 -- 172.21.15.7:0/1717534997 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c00b9d0 con 0x7f84380c9f40 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f84367fc700 1 -- 172.21.15.7:0/1717534997 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00bd60 con 0x7f84380c9f40 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f843dfff700 1 -- 172.21.15.7:0/1717534997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 msgr2=0x7f843815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.341+0000 7f843dfff700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f843815ad90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f843808d340 tx=0x7f842c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 -- 172.21.15.7:0/1717534997 shutdown_connections 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 --2- 172.21.15.7:0/1717534997 >> v2:172.21.15.117:3300/0 conn(0x7f8438153880 0x7f843815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f84380c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 --2- 172.21.15.7:0/1717534997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f843815ad90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 -- 172.21.15.7:0/1717534997 >> 172.21.15.7:0/1717534997 conn(0x7f84380737f0 msgr2=0x7f84380720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 -- 172.21.15.7:0/1717534997 shutdown_connections 2023-06-09T11:31:02.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.342+0000 7f843dfff700 1 -- 172.21.15.7:0/1717534997 wait complete. 2023-06-09T11:31:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.343+0000 7f843dfff700 1 Processor -- start 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.343+0000 7f843dfff700 1 -- start start 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8438153880 0x7f84381efa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f843815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438152a20 con 0x7f84380c9f40 2023-06-09T11:31:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8438152c30 con 0x7f8438153880 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.344+0000 7f843dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8438152f10 con 0x7f843815b2d0 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39902/0 (socket says 172.21.15.7:39902) 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 -- 172.21.15.7:0/2600508034 learned_addr learned my addr 172.21.15.7:0/2600508034 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f8437fff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f843815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f8436ffd700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8438153880 0x7f84381efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 -- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 msgr2=0x7f843815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f843815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 -- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8438153880 msgr2=0x7f84381efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8438153880 0x7f84381efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 -- 172.21.15.7:0/2600508034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842c009550 con 0x7f84380c9f40 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f8437fff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f843815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f84377fe700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f842800c740 tx=0x7f842800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428011800 con 0x7f84380c9f40 2023-06-09T11:31:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8428011960 con 0x7f84380c9f40 2023-06-09T11:31:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8428011cd0 con 0x7f84380c9f40 2023-06-09T11:31:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.345+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8438159980 con 0x7f84380c9f40 2023-06-09T11:31:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.346+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843815b730 con 0x7f84380c9f40 2023-06-09T11:31:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.347+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8438159c60 con 0x7f84380c9f40 2023-06-09T11:31:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.347+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8428021350 con 0x7f84380c9f40 2023-06-09T11:31:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.348+0000 7f841ffff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 0x7f842406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.349+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f8428089840 con 0x7f84380c9f40 2023-06-09T11:31:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.349+0000 7f8436ffd700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 0x7f842406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.349+0000 7f8436ffd700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 0x7f842406d1c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:02.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.353+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8428057f00 con 0x7f84380c9f40 2023-06-09T11:31:02.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.608+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8438153080 con 0x7f84380c9f40 2023-06-09T11:31:02.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.610+0000 7f841ffff700 1 -- 172.21.15.7:0/2600508034 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f8428057a90 con 0x7f84380c9f40 2023-06-09T11:31:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:31:02.615 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:30:57.533244+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:clientreplay","state_seq":18,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:31:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.612+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 msgr2=0x7f842406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:02.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.612+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 0x7f842406d1c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f842c000c00 tx=0x7f842c008030 comp rx=0 tx=0).stop 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.612+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 msgr2=0x7f8438151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.612+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f842800c740 tx=0x7f842800cb00 comp rx=0 tx=0).stop 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 shutdown_connections 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f842406ad10 0x7f842406d1c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f843815b2d0 0x7f843815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84380c9f40 0x7f8438151130 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 --2- 172.21.15.7:0/2600508034 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8438153880 0x7f84381efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 >> 172.21.15.7:0/2600508034 conn(0x7f84380737f0 msgr2=0x7f843806ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 shutdown_connections 2023-06-09T11:31:02.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:02.613+0000 7f843dfff700 1 -- 172.21.15.7:0/2600508034 wait complete. 2023-06-09T11:31:02.620 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 89 2023-06-09T11:31:02.637 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:31:04.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:31:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.844+0000 7f0abfdff700 1 Processor -- start 2023-06-09T11:31:04.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.844+0000 7f0abfdff700 1 -- start start 2023-06-09T11:31:04.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.845+0000 7f0abfdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ab8072020 0x7f0ab814f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.845+0000 7f0abfdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80cc140 con 0x7f0ab8159eb0 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.845+0000 7f0abfdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80c1830 con 0x7f0ab81593d0 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.845+0000 7f0abfdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80714f0 con 0x7f0ab8072020 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abd39a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0ab8159eb0 0x7f0ab8071910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52860/0 (socket says 172.21.15.7:52860) 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abd39a700 1 -- 172.21.15.7:0/936896819 learned_addr learned my addr 172.21.15.7:0/936896819 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:31:04.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abe39c700 1 --2- 172.21.15.7:0/936896819 >> v2:172.21.15.7:3300/0 conn(0x7f0ab8072020 0x7f0ab814f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:04.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abcb99700 1 -- 172.21.15.7:0/936896819 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492039469 0 0) 0x7f0ab80cc140 con 0x7f0ab8159eb0 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abcb99700 1 -- 172.21.15.7:0/936896819 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa40035b0 con 0x7f0ab8159eb0 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abe39c700 1 -- 172.21.15.7:0/936896819 >> v1:172.21.15.117:6789/0 conn(0x7f0ab81593d0 legacy=0x7f0ab81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abe39c700 1 -- 172.21.15.7:0/936896819 >> v1:172.21.15.7:6789/0 conn(0x7f0ab8159eb0 legacy=0x7f0ab8071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.846+0000 7f0abe39c700 1 -- 172.21.15.7:0/936896819 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80c0560 con 0x7f0ab8072020 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abe39c700 1 --2- 172.21.15.7:0/936896819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8072020 0x7f0ab814f9a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000aea0 tx=0x7f0ab000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f561296ba64bc968 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abcb99700 1 -- 172.21.15.7:0/936896819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000f070 con 0x7f0ab8072020 2023-06-09T11:31:04.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abcb99700 1 -- 172.21.15.7:0/936896819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000c910 con 0x7f0ab8072020 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abcb99700 1 -- 172.21.15.7:0/936896819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000ccc0 con 0x7f0ab8072020 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 -- 172.21.15.7:0/936896819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8072020 msgr2=0x7f0ab814f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 --2- 172.21.15.7:0/936896819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8072020 0x7f0ab814f9a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000aea0 tx=0x7f0ab000c330 comp rx=0 tx=0).stop 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 -- 172.21.15.7:0/936896819 shutdown_connections 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 --2- 172.21.15.7:0/936896819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab8072020 0x7f0ab814f9a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 -- 172.21.15.7:0/936896819 >> 172.21.15.7:0/936896819 conn(0x7f0ab806a4d0 msgr2=0x7f0ab806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:04.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 -- 172.21.15.7:0/936896819 shutdown_connections 2023-06-09T11:31:04.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.847+0000 7f0abfdff700 1 -- 172.21.15.7:0/936896819 wait complete. 2023-06-09T11:31:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.849+0000 7f0abfdff700 1 Processor -- start 2023-06-09T11:31:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.849+0000 7f0abfdff700 1 -- start start 2023-06-09T11:31:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ab8072020 0x7f0ab81ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:04.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ab8159eb0 0x7f0ab814d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab81ed4c0 con 0x7f0ab81593d0 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8156a70 con 0x7f0ab8159eb0 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abfdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8156d10 con 0x7f0ab8072020 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47352/0 (socket says 172.21.15.7:47352) 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.850+0000 7f0abd39a700 1 -- 172.21.15.7:0/12820734 learned_addr learned my addr 172.21.15.7:0/12820734 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:31:04.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abe39c700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ab8159eb0 0x7f0ab814d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abdb9b700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ab8072020 0x7f0ab81ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 -- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ab8072020 msgr2=0x7f0ab81ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ab8072020 0x7f0ab81ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 -- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ab8159eb0 msgr2=0x7f0ab814d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ab8159eb0 0x7f0ab814d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 -- 172.21.15.7:0/12820734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab000ab80 con 0x7f0ab81593d0 2023-06-09T11:31:04.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abd39a700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005910 tx=0x7f0aa800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:04.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8005d70 con 0x7f0ab81593d0 2023-06-09T11:31:04.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800ce80 con 0x7f0ab81593d0 2023-06-09T11:31:04.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab80c1d60 con 0x7f0ab81593d0 2023-06-09T11:31:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.851+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80072f0 con 0x7f0ab81593d0 2023-06-09T11:31:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.852+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab80c1a80 con 0x7f0ab81593d0 2023-06-09T11:31:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.853+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80c2040 con 0x7f0ab81593d0 2023-06-09T11:31:04.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.854+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0aa8007450 con 0x7f0ab81593d0 2023-06-09T11:31:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.855+0000 7f0aae7fc700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 0x7f0a9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.855+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f0aa8088cf0 con 0x7f0ab81593d0 2023-06-09T11:31:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.855+0000 7f0abdb9b700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 0x7f0a9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:04.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.856+0000 7f0abdb9b700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 0x7f0a9c06d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007fd0 tx=0x7f0ab4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:04.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:04.860+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0aa808d070 con 0x7f0ab81593d0 2023-06-09T11:31:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.110+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab8074cf0 con 0x7f0ab81593d0 2023-06-09T11:31:05.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.111+0000 7f0aae7fc700 1 -- 172.21.15.7:0/12820734 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f0aa8017020 con 0x7f0ab81593d0 2023-06-09T11:31:05.116 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:31:05.116 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:04.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:31:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 msgr2=0x7f0a9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:05.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 0x7f0a9c06d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4007fd0 tx=0x7f0ab4009040 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 msgr2=0x7f0ab80cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8005910 tx=0x7f0aa800c7b0 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 shutdown_connections 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ab8159eb0 0x7f0ab814d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a9c06ace0 0x7f0a9c06d190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ab8072020 0x7f0ab81ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 --2- 172.21.15.7:0/12820734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ab81593d0 0x7f0ab80cb560 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:05.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 >> 172.21.15.7:0/12820734 conn(0x7f0ab806a4d0 msgr2=0x7f0ab806dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 shutdown_connections 2023-06-09T11:31:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:05.113+0000 7f0abfdff700 1 -- 172.21.15.7:0/12820734 wait complete. 2023-06-09T11:31:05.120 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2023-06-09T11:31:05.137 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=5435 2023-06-09T11:31:05.137 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.d 2023-06-09T11:31:11.419 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:31:11.414+0000 7fe7fb3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:31:11.419 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:31:11.414+0000 7fe7fb3f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:31:11.419 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:31:11.414+0000 7fe7fb3f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:31:12.138 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:31:12.138 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:31:12.138 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:31:12.143 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:31:12.143 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:31:12.288 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:31:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.363+0000 7f7f0219d700 1 Processor -- start 2023-06-09T11:31:12.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.364+0000 7f7f0219d700 1 -- start start 2023-06-09T11:31:12.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0219d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 0x7f7efc0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0219d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7efc071d50 0x7f7efc072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0219d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7efc0d01a0 con 0x7f7efc156b50 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0219d700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc0cb2b0 con 0x7f7efc071d50 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0219d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0b54a0 con 0x7f7efc156f20 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0119b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7efc156b50 0x7f7efc0caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52866/0 (socket says 172.21.15.7:52866) 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0119b700 1 -- 172.21.15.7:0/2838382486 learned_addr learned my addr 172.21.15.7:0/2838382486 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0199c700 1 --2- 172.21.15.7:0/2838382486 >> v2:172.21.15.7:3300/0 conn(0x7f7efc071d50 0x7f7efc072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.365+0000 7f7f0099a700 1 --2- 172.21.15.7:0/2838382486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 0x7f7efc0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7f0099a700 1 -- 172.21.15.7:0/2838382486 >> v2:172.21.15.7:3300/0 conn(0x7f7efc071d50 msgr2=0x7f7efc072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7f0099a700 1 --2- 172.21.15.7:0/2838382486 >> v2:172.21.15.7:3300/0 conn(0x7f7efc071d50 0x7f7efc072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7f0099a700 1 -- 172.21.15.7:0/2838382486 >> v1:172.21.15.7:6789/0 conn(0x7f7efc156b50 legacy=0x7f7efc0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7ef3fff700 1 -- 172.21.15.7:0/2838382486 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30728653 0 0) 0x7f7efc0d01a0 con 0x7f7efc156b50 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7f0099a700 1 -- 172.21.15.7:0/2838382486 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0cba50 con 0x7f7efc156f20 2023-06-09T11:31:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.366+0000 7f7f0099a700 1 --2- 172.21.15.7:0/2838382486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 0x7f7efc0cb510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009840 tx=0x7f7eec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75bfc7c92ce815a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7ef3fff700 1 -- 172.21.15.7:0/2838382486 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011830 con 0x7f7efc156f20 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7ef3fff700 1 -- 172.21.15.7:0/2838382486 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec011990 con 0x7f7efc156f20 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7ef3fff700 1 -- 172.21.15.7:0/2838382486 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011d20 con 0x7f7efc156f20 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 -- 172.21.15.7:0/2838382486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 msgr2=0x7f7efc0cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 --2- 172.21.15.7:0/2838382486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 0x7f7efc0cb510 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009840 tx=0x7f7eec009b50 comp rx=0 tx=0).stop 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 -- 172.21.15.7:0/2838382486 shutdown_connections 2023-06-09T11:31:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 --2- 172.21.15.7:0/2838382486 >> v2:172.21.15.7:3300/0 conn(0x7f7efc071d50 0x7f7efc072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 --2- 172.21.15.7:0/2838382486 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc156f20 0x7f7efc0cb510 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.367+0000 7f7f0219d700 1 -- 172.21.15.7:0/2838382486 >> 172.21.15.7:0/2838382486 conn(0x7f7efc0737f0 msgr2=0x7f7efc073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.368+0000 7f7f0219d700 1 -- 172.21.15.7:0/2838382486 shutdown_connections 2023-06-09T11:31:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.368+0000 7f7f0219d700 1 -- 172.21.15.7:0/2838382486 wait complete. 2023-06-09T11:31:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.370+0000 7f7f0219d700 1 Processor -- start 2023-06-09T11:31:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.370+0000 7f7f0219d700 1 -- start start 2023-06-09T11:31:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 0x7f7efc1f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 0x7f7efc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f7640 con 0x7f7efc156b50 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc0cf0b0 con 0x7f7efc156f20 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0219d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc1f3180 con 0x7f7efc071d50 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0099a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47358/0 (socket says 172.21.15.7:47358) 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0099a700 1 -- 172.21.15.7:0/3392924140 learned_addr learned my addr 172.21.15.7:0/3392924140 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:31:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0119b700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 0x7f7efc1f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.371+0000 7f7f0199c700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 0x7f7efc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 -- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 msgr2=0x7f7efc1f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 0x7f7efc1f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 -- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 msgr2=0x7f7efc152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 0x7f7efc152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 -- 172.21.15.7:0/3392924140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0094a0 con 0x7f7efc156b50 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0119b700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 0x7f7efc1f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0199c700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 0x7f7efc152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:31:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0099a700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009fd0 tx=0x7f7eec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec004590 con 0x7f7efc156b50 2023-06-09T11:31:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0036a0 con 0x7f7efc156b50 2023-06-09T11:31:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec003a30 con 0x7f7efc156b50 2023-06-09T11:31:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.372+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0cf870 con 0x7f7efc156b50 2023-06-09T11:31:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.373+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc0cf590 con 0x7f7efc156b50 2023-06-09T11:31:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.374+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc0cfb50 con 0x7f7efc156b50 2023-06-09T11:31:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.375+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7eec003b90 con 0x7f7efc156b50 2023-06-09T11:31:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.376+0000 7f7ef1ffb700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:31:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.376+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f7eec08ab50 con 0x7f7efc156b50 2023-06-09T11:31:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.376+0000 7f7f0119b700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:31:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.377+0000 7f7f0119b700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 0x7f7ee006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80010e0 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:31:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.381+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7eec055800 con 0x7f7efc156b50 2023-06-09T11:31:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.703+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7efc0b3440 con 0x7f7efc156b50 2023-06-09T11:31:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.704+0000 7f7ef1ffb700 1 -- 172.21.15.7:0/3392924140 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+6216 (secure 0 0 0) 0x7f7eec058e20 con 0x7f7efc156b50 2023-06-09T11:31:12.710 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:31:12.710 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:04.556417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:31:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 msgr2=0x7f7ee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 0x7f7ee006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80010e0 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 msgr2=0x7f7efc0ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009fd0 tx=0x7f7eec004750 comp rx=0 tx=0).stop 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 shutdown_connections 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7efc156f20 0x7f7efc152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee006ace0 0x7f7ee006d190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7efc071d50 0x7f7efc1f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 --2- 172.21.15.7:0/3392924140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7efc156b50 0x7f7efc0ceb70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.706+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 >> 172.21.15.7:0/3392924140 conn(0x7f7efc0737f0 msgr2=0x7f7efc0c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.707+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 shutdown_connections 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:31:12.707+0000 7f7f0219d700 1 -- 172.21.15.7:0/3392924140 wait complete. 2023-06-09T11:31:12.714 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 91 2023-06-09T11:31:12.732 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:31:12.733 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 95 secs before thrashing 2023-06-09T11:31:16.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:16.257 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:46.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:31:46.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:16.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:16.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:46.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:46.595 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:32:47.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:47.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.968+0000 7f47eddff700 1 Processor -- start 2023-06-09T11:32:47.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.969+0000 7f47eddff700 1 -- start start 2023-06-09T11:32:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47eddff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47e80c6280 0x7f47e80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47eddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47eddff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e815cac0 con 0x7f47e806a4d0 2023-06-09T11:32:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47eddff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e80c8870 con 0x7f47e80c6280 2023-06-09T11:32:47.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47eddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e80c37d0 con 0x7f47e806e460 2023-06-09T11:32:47.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47e77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47e80c6280 0x7f47e80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.970+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39832/0 (socket says 172.21.15.7:39832) 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e7fff700 1 -- 172.21.15.7:0/4164097740 learned_addr learned my addr 172.21.15.7:0/4164097740 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47e80c6280 0x7f47e80c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:58710/0 (socket says 172.21.15.7:58710) 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e6ffd700 1 --1- 172.21.15.7:0/4164097740 >> v1:172.21.15.117:6789/0 conn(0x7f47e806a4d0 0x7f47e80737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34192/0 (socket says 172.21.15.7:34192) 2023-06-09T11:32:47.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 -- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 msgr2=0x7f47e806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 -- 172.21.15.7:0/4164097740 >> v1:172.21.15.117:6789/0 conn(0x7f47e806a4d0 legacy=0x7f47e80737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 -- 172.21.15.7:0/4164097740 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e815cd30 con 0x7f47e80c6280 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e7fff700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.971+0000 7f47e77fe700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e80c6280 0x7f47e80c76b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009840 tx=0x7f47d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad35a44284bf221 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47e67fc700 1 -- 172.21.15.7:0/4164097740 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8011830 con 0x7f47e80c6280 2023-06-09T11:32:47.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47e67fc700 1 -- 172.21.15.7:0/4164097740 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d8011990 con 0x7f47e80c6280 2023-06-09T11:32:47.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47e67fc700 1 -- 172.21.15.7:0/4164097740 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8011d20 con 0x7f47e80c6280 2023-06-09T11:32:47.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 -- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e80c6280 msgr2=0x7f47e80c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e80c6280 0x7f47e80c76b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009840 tx=0x7f47d8009b50 comp rx=0 tx=0).stop 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 -- 172.21.15.7:0/4164097740 shutdown_connections 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e806cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 --2- 172.21.15.7:0/4164097740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e80c6280 0x7f47e80c76b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.972+0000 7f47eddff700 1 -- 172.21.15.7:0/4164097740 >> 172.21.15.7:0/4164097740 conn(0x7f47e806f980 msgr2=0x7f47e80cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:47.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.973+0000 7f47eddff700 1 -- 172.21.15.7:0/4164097740 shutdown_connections 2023-06-09T11:32:47.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.973+0000 7f47eddff700 1 -- 172.21.15.7:0/4164097740 wait complete. 2023-06-09T11:32:47.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.974+0000 7f47eddff700 1 Processor -- start 2023-06-09T11:32:47.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.975+0000 7f47eddff700 1 -- start start 2023-06-09T11:32:47.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e806a4d0 0x7f47e8139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e815a0b0 con 0x7f47e80c6280 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e81f3f20 con 0x7f47e806a4d0 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47eddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e806bd50 con 0x7f47e806e460 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.976+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38214/0 (socket says 172.21.15.7:38214) 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 -- 172.21.15.7:0/322616198 learned_addr learned my addr 172.21.15.7:0/322616198 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e6ffd700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e77fe700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e806a4d0 0x7f47e8139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 -- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 msgr2=0x7f47e80cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e80cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 -- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e806a4d0 msgr2=0x7f47e8139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e806a4d0 0x7f47e8139120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 -- 172.21.15.7:0/322616198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0009b30 con 0x7f47e80c6280 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e6ffd700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:32:47.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47e7fff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f47e000efc0 tx=0x7f47e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:47.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0018070 con 0x7f47e80c6280 2023-06-09T11:32:47.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e000f040 con 0x7f47e80c6280 2023-06-09T11:32:47.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.977+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0003c20 con 0x7f47e80c6280 2023-06-09T11:32:47.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.978+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d80094a0 con 0x7f47e80c6280 2023-06-09T11:32:47.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.978+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e815d2f0 con 0x7f47e80c6280 2023-06-09T11:32:47.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.979+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e815d010 con 0x7f47e80c6280 2023-06-09T11:32:47.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.980+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f47e000c930 con 0x7f47e80c6280 2023-06-09T11:32:47.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.981+0000 7f47cffff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 0x7f47d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:47.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.981+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f47e0089ad0 con 0x7f47e80c6280 2023-06-09T11:32:47.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.981+0000 7f47e77fe700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 0x7f47d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:47.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.982+0000 7f47e77fe700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 0x7f47d406d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:47.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:47.986+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f47e0054780 con 0x7f47e80c6280 2023-06-09T11:32:48.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.295+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47e815d5d0 con 0x7f47e80c6280 2023-06-09T11:32:48.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.296+0000 7f47cffff700 1 -- 172.21.15.7:0/322616198 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7f47e0057da0 con 0x7f47e80c6280 2023-06-09T11:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:48.302 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:48.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 msgr2=0x7f47d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 0x7f47d406d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f47d8000c00 tx=0x7f47d801c040 comp rx=0 tx=0).stop 2023-06-09T11:32:48.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 msgr2=0x7f47e80cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f47e000efc0 tx=0x7f47e000c3a0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 shutdown_connections 2023-06-09T11:32:48.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47e80c6280 0x7f47e80cc270 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47e806a4d0 0x7f47e8139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f47d406ace0 0x7f47d406d190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 --2- 172.21.15.7:0/322616198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47e806e460 0x7f47e80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 >> 172.21.15.7:0/322616198 conn(0x7f47e806f980 msgr2=0x7f47e815ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:48.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 shutdown_connections 2023-06-09T11:32:48.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.299+0000 7f47eddff700 1 -- 172.21.15.7:0/322616198 wait complete. 2023-06-09T11:32:48.307 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:48.324 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2023-06-09T11:32:48.325 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:32:48.325 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:32:48.345 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:32:48.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:48.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.558+0000 7f13e93ff700 1 Processor -- start 2023-06-09T11:32:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.558+0000 7f13e93ff700 1 -- start start 2023-06-09T11:32:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e4153880 0x7f13e415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e93ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13e40c9f40 0x7f13e415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e93ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e415a500 con 0x7f13e415b2d0 2023-06-09T11:32:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e93ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e40c5f10 con 0x7f13e40c9f40 2023-06-09T11:32:48.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e415da00 con 0x7f13e4153880 2023-06-09T11:32:48.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13dbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f13e40c9f40 0x7f13e415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e2ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e4153880 0x7f13e415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e37fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f13e415b2d0 0x7f13e40c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:34202/0 (socket says 172.21.15.7:34202) 2023-06-09T11:32:48.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.560+0000 7f13e37fe700 1 -- 172.21.15.7:0/1278117561 learned_addr learned my addr 172.21.15.7:0/1278117561 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:32:48.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.561+0000 7f13dbfff700 1 -- 172.21.15.7:0/1278117561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e4153880 msgr2=0x7f13e415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:48.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.561+0000 7f13dbfff700 1 --2- 172.21.15.7:0/1278117561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e4153880 0x7f13e415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.561+0000 7f13dbfff700 1 -- 172.21.15.7:0/1278117561 >> v1:172.21.15.117:6789/0 conn(0x7f13e415b2d0 legacy=0x7f13e40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.561+0000 7f13dbfff700 1 -- 172.21.15.7:0/1278117561 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e415b750 con 0x7f13e40c9f40 2023-06-09T11:32:48.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.561+0000 7f13dbfff700 1 --2- 172.21.15.7:0/1278117561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e415ad90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009840 tx=0x7f13cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f0a89d05dc7403 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:48.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.563+0000 7f13e27fc700 1 -- 172.21.15.7:0/1278117561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc011830 con 0x7f13e40c9f40 2023-06-09T11:32:48.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.563+0000 7f13e27fc700 1 -- 172.21.15.7:0/1278117561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc011990 con 0x7f13e40c9f40 2023-06-09T11:32:48.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.563+0000 7f13e27fc700 1 -- 172.21.15.7:0/1278117561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc011d20 con 0x7f13e40c9f40 2023-06-09T11:32:48.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 -- 172.21.15.7:0/1278117561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 msgr2=0x7f13e415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 --2- 172.21.15.7:0/1278117561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e415ad90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009840 tx=0x7f13cc009b50 comp rx=0 tx=0).stop 2023-06-09T11:32:48.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 -- 172.21.15.7:0/1278117561 shutdown_connections 2023-06-09T11:32:48.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 --2- 172.21.15.7:0/1278117561 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e4153880 0x7f13e415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 --2- 172.21.15.7:0/1278117561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e415ad90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 -- 172.21.15.7:0/1278117561 >> 172.21.15.7:0/1278117561 conn(0x7f13e40737f0 msgr2=0x7f13e40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:48.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 -- 172.21.15.7:0/1278117561 shutdown_connections 2023-06-09T11:32:48.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.564+0000 7f13e93ff700 1 -- 172.21.15.7:0/1278117561 wait complete. 2023-06-09T11:32:48.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.566+0000 7f13e93ff700 1 Processor -- start 2023-06-09T11:32:48.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.566+0000 7f13e93ff700 1 -- start start 2023-06-09T11:32:48.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 0x7f13e40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e415b2d0 0x7f13e41ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e4154540 con 0x7f13e40c9f40 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e41f28b0 con 0x7f13e4153880 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40b53f0 con 0x7f13e415b2d0 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 0x7f13e40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38232/0 (socket says 172.21.15.7:38232) 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e2ffd700 1 -- 172.21.15.7:0/2037753907 learned_addr learned my addr 172.21.15.7:0/2037753907 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13e37fe700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e415b2d0 0x7f13e41ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 -- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e415b2d0 msgr2=0x7f13e41ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e415b2d0 0x7f13e41ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 -- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 msgr2=0x7f13e40b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.567+0000 7f13dbfff700 1 -- 172.21.15.7:0/2037753907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13cc0094a0 con 0x7f13e4153880 2023-06-09T11:32:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.568+0000 7f13e2ffd700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:32:48.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.568+0000 7f13dbfff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 0x7f13e40cb4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f13e4071db0 tx=0x7f13cc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.571+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0036a0 con 0x7f13e4153880 2023-06-09T11:32:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.572+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc01f070 con 0x7f13e4153880 2023-06-09T11:32:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.572+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc003a80 con 0x7f13e4153880 2023-06-09T11:32:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.572+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e4159980 con 0x7f13e4153880 2023-06-09T11:32:48.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.572+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e41596e0 con 0x7f13e4153880 2023-06-09T11:32:48.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.573+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4159c60 con 0x7f13e4153880 2023-06-09T11:32:48.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.574+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13cc003be0 con 0x7f13e4153880 2023-06-09T11:32:48.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.575+0000 7f13db7fe700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 0x7f13c406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:48.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.575+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f13cc021070 con 0x7f13e4153880 2023-06-09T11:32:48.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.575+0000 7f13e2ffd700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 0x7f13c406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:48.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.576+0000 7f13e2ffd700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 0x7f13c406d1e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f13d40113d0 tx=0x7f13d4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:48.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.580+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f13cc055cd0 con 0x7f13e4153880 2023-06-09T11:32:48.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.834+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e4074cf0 con 0x7f13e4153880 2023-06-09T11:32:48.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.835+0000 7f13db7fe700 1 -- 172.21.15.7:0/2037753907 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7f13cc0592f0 con 0x7f13e4153880 2023-06-09T11:32:48.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:48.841 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:48.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 msgr2=0x7f13c406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 0x7f13c406d1e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f13d40113d0 tx=0x7f13d4009210 comp rx=0 tx=0).stop 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 msgr2=0x7f13e40cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 0x7f13e40cb4c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f13e4071db0 tx=0x7f13cc004750 comp rx=0 tx=0).stop 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 shutdown_connections 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f13c406ad30 0x7f13c406d1e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13e415b2d0 0x7f13e41ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f13e40c9f40 0x7f13e40b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 --2- 172.21.15.7:0/2037753907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13e4153880 0x7f13e40cb4c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 >> 172.21.15.7:0/2037753907 conn(0x7f13e40737f0 msgr2=0x7f13e40720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.837+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 shutdown_connections 2023-06-09T11:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:48.838+0000 7f13e93ff700 1 -- 172.21.15.7:0/2037753907 wait complete. 2023-06-09T11:32:48.845 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:48.861 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:32:50.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:51.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.067+0000 7efe4efff700 1 Processor -- start 2023-06-09T11:32:51.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.068+0000 7efe4efff700 1 -- start start 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe48151510 0x7efe480c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 0x7efe48070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe48071060 0x7efe48158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe480714e0 con 0x7efe48151510 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe481518e0 con 0x7efe48071060 2023-06-09T11:32:51.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4efff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe481580a0 con 0x7efe4815b380 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4cd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe48151510 0x7efe480c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4cd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe48151510 0x7efe480c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38236/0 (socket says 172.21.15.7:38236) 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2199439669 learned_addr learned my addr 172.21.15.7:0/2199439669 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe47fff700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 0x7efe48070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.069+0000 7efe4d59c700 1 --2- 172.21.15.7:0/2199439669 >> v2:172.21.15.117:3300/0 conn(0x7efe48071060 0x7efe48158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2199439669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 msgr2=0x7efe48070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 0x7efe48070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2199439669 >> v2:172.21.15.117:3300/0 conn(0x7efe48071060 msgr2=0x7efe48158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:51.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2199439669 >> v2:172.21.15.117:3300/0 conn(0x7efe48071060 0x7efe48158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2199439669 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480c6180 con 0x7efe48151510 2023-06-09T11:32:51.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe47fff700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 0x7efe48070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:32:51.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48151510 0x7efe480c5b40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efe480adac0 tx=0x7efe3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3397139a38b47261 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:51.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe477fe700 1 -- 172.21.15.7:0/2199439669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c011830 con 0x7efe48151510 2023-06-09T11:32:51.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe477fe700 1 -- 172.21.15.7:0/2199439669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c011990 con 0x7efe48151510 2023-06-09T11:32:51.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.070+0000 7efe477fe700 1 -- 172.21.15.7:0/2199439669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c011d20 con 0x7efe48151510 2023-06-09T11:32:51.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 -- 172.21.15.7:0/2199439669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48151510 msgr2=0x7efe480c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48151510 0x7efe480c5b40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7efe480adac0 tx=0x7efe3c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 -- 172.21.15.7:0/2199439669 shutdown_connections 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 --2- 172.21.15.7:0/2199439669 >> v2:172.21.15.117:3300/0 conn(0x7efe48071060 0x7efe48158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48151510 0x7efe480c5b40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 --2- 172.21.15.7:0/2199439669 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe4815b380 0x7efe48070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 -- 172.21.15.7:0/2199439669 >> 172.21.15.7:0/2199439669 conn(0x7efe4806a4d0 msgr2=0x7efe4806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:51.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 -- 172.21.15.7:0/2199439669 shutdown_connections 2023-06-09T11:32:51.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.071+0000 7efe4efff700 1 -- 172.21.15.7:0/2199439669 wait complete. 2023-06-09T11:32:51.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.073+0000 7efe4efff700 1 Processor -- start 2023-06-09T11:32:51.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.073+0000 7efe4efff700 1 -- start start 2023-06-09T11:32:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe48151510 0x7efe48147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4815b380 0x7efe480c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480add00 con 0x7efe48071060 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480cb590 con 0x7efe4815b380 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4efff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe480cb7a0 con 0x7efe48151510 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38246/0 (socket says 172.21.15.7:38246) 2023-06-09T11:32:51.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2501405554 learned_addr learned my addr 172.21.15.7:0/2501405554 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe47fff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe48151510 0x7efe48147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4d59c700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4815b380 0x7efe480c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe48151510 msgr2=0x7efe48147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe48151510 0x7efe48147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4815b380 msgr2=0x7efe480c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.074+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4815b380 0x7efe480c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe4cd9b700 1 -- 172.21.15.7:0/2501405554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c009550 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe4cd9b700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000c00 tx=0x7efe3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c004240 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c01f070 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe48157770 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c0036a0 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.075+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe48157490 con 0x7efe48071060 2023-06-09T11:32:51.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.076+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48157a50 con 0x7efe48071060 2023-06-09T11:32:51.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.077+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe3c021030 con 0x7efe48071060 2023-06-09T11:32:51.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.078+0000 7efe457fa700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 0x7efe2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:51.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.078+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7efe3c08ab20 con 0x7efe48071060 2023-06-09T11:32:51.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.079+0000 7efe47fff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 0x7efe2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:51.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.079+0000 7efe47fff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 0x7efe2c06d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe480c5580 tx=0x7efe34008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:51.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.083+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efe3c055740 con 0x7efe48071060 2023-06-09T11:32:51.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.337+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe48074cf0 con 0x7efe48071060 2023-06-09T11:32:51.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.337+0000 7efe457fa700 1 -- 172.21.15.7:0/2501405554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7efe3c058d60 con 0x7efe48071060 2023-06-09T11:32:51.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:51.344 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 msgr2=0x7efe2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 0x7efe2c06d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efe480c5580 tx=0x7efe34008040 comp rx=0 tx=0).stop 2023-06-09T11:32:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 msgr2=0x7efe48146af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000c00 tx=0x7efe3c004750 comp rx=0 tx=0).stop 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 shutdown_connections 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe4815b380 0x7efe480c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe48071060 0x7efe48146af0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe2c06ace0 0x7efe2c06d190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 --2- 172.21.15.7:0/2501405554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe48151510 0x7efe48147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 >> 172.21.15.7:0/2501405554 conn(0x7efe4806a4d0 msgr2=0x7efe48149610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 shutdown_connections 2023-06-09T11:32:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:51.340+0000 7efe4efff700 1 -- 172.21.15.7:0/2501405554 wait complete. 2023-06-09T11:32:51.348 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:51.364 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:32:53.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.572+0000 7f4fdf9ff700 1 Processor -- start 2023-06-09T11:32:53.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.572+0000 7f4fdf9ff700 1 -- start start 2023-06-09T11:32:53.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.573+0000 7f4fdf9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.573+0000 7f4fdf9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 0x7f4fd80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.573+0000 7f4fdf9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd815a500 con 0x7f4fd8153880 2023-06-09T11:32:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.573+0000 7f4fdf9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80c5f10 con 0x7f4fd80c9f40 2023-06-09T11:32:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.573+0000 7f4fdf9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd815da00 con 0x7f4fd815b4a0 2023-06-09T11:32:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdd79b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4fd8153880 0x7f4fd815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42380/0 (socket says 172.21.15.7:42380) 2023-06-09T11:32:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdd79b700 1 -- 172.21.15.7:0/2732343308 learned_addr learned my addr 172.21.15.7:0/2732343308 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:32:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2732343308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fddf9c700 1 --2- 172.21.15.7:0/2732343308 >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 0x7f4fd80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdcf9a700 1 -- 172.21.15.7:0/2732343308 >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 msgr2=0x7f4fd80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2732343308 >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 0x7f4fd80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdcf9a700 1 -- 172.21.15.7:0/2732343308 >> v1:172.21.15.7:6789/0 conn(0x7f4fd8153880 legacy=0x7f4fd815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fcffff700 1 -- 172.21.15.7:0/2732343308 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28505887 0 0) 0x7f4fd4001110 con 0x7f4fd8153880 2023-06-09T11:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fdcf9a700 1 -- 172.21.15.7:0/2732343308 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd81596e0 con 0x7f4fd815b4a0 2023-06-09T11:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.574+0000 7f4fddf9c700 1 --2- 172.21.15.7:0/2732343308 >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 0x7f4fd80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:32:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.575+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2732343308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd815d4c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009840 tx=0x7f4fc8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=609a794278b57e69 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fcffff700 1 -- 172.21.15.7:0/2732343308 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8011830 con 0x7f4fd815b4a0 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fcffff700 1 -- 172.21.15.7:0/2732343308 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc8011990 con 0x7f4fd815b4a0 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fcffff700 1 -- 172.21.15.7:0/2732343308 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8011d20 con 0x7f4fd815b4a0 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2732343308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 msgr2=0x7f4fd815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2732343308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd815d4c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009840 tx=0x7f4fc8009b50 comp rx=0 tx=0).stop 2023-06-09T11:32:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2732343308 shutdown_connections 2023-06-09T11:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2732343308 >> v2:172.21.15.117:3300/0 conn(0x7f4fd80c9f40 0x7f4fd80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2732343308 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd815d4c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2732343308 >> 172.21.15.7:0/2732343308 conn(0x7f4fd80737f0 msgr2=0x7f4fd80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2732343308 shutdown_connections 2023-06-09T11:32:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.576+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2732343308 wait complete. 2023-06-09T11:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.578+0000 7f4fdf9ff700 1 Processor -- start 2023-06-09T11:32:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.578+0000 7f4fdf9ff700 1 -- start start 2023-06-09T11:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd8153880 0x7f4fd806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd806aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd814bae0 con 0x7f4fd80c9f40 2023-06-09T11:32:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd814b4b0 con 0x7f4fd8153880 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.579+0000 7f4fdf9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80b5350 con 0x7f4fd815b4a0 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51682/0 (socket says 172.21.15.7:51682) 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 -- 172.21.15.7:0/2169620767 learned_addr learned my addr 172.21.15.7:0/2169620767 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd8153880 0x7f4fd806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fddf9c700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd806aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 -- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 msgr2=0x7f4fd806aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd806aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 -- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd8153880 msgr2=0x7f4fd806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd8153880 0x7f4fd806a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 -- 172.21.15.7:0/2169620767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0009a70 con 0x7f4fd80c9f40 2023-06-09T11:32:53.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fdd79b700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c760 tx=0x7f4fd400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.580+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004c80 con 0x7f4fd80c9f40 2023-06-09T11:32:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.581+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4004de0 con 0x7f4fd80c9f40 2023-06-09T11:32:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.581+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4007310 con 0x7f4fd80c9f40 2023-06-09T11:32:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.581+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc80094a0 con 0x7f4fd80c9f40 2023-06-09T11:32:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.581+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd8159c60 con 0x7f4fd80c9f40 2023-06-09T11:32:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.582+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd8159980 con 0x7f4fd80c9f40 2023-06-09T11:32:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.583+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4fd4007470 con 0x7f4fd80c9f40 2023-06-09T11:32:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.584+0000 7f4fcdffb700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 0x7f4fbc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.584+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f4fd4089300 con 0x7f4fd80c9f40 2023-06-09T11:32:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.584+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 0x7f4fbc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.584+0000 7f4fdcf9a700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 0x7f4fbc06d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.588+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4fd4053fb0 con 0x7f4fd80c9f40 2023-06-09T11:32:53.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.840+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fd8159f40 con 0x7f4fd80c9f40 2023-06-09T11:32:53.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.841+0000 7f4fcdffb700 1 -- 172.21.15.7:0/2169620767 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7f4fd40575d0 con 0x7f4fd80c9f40 2023-06-09T11:32:53.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:53.847 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:53.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 msgr2=0x7f4fbc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 0x7f4fbc06d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc801c040 comp rx=0 tx=0).stop 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 msgr2=0x7f4fd80cd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400c760 tx=0x7f4fd400ca70 comp rx=0 tx=0).stop 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 shutdown_connections 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4fbc06ace0 0x7f4fbc06d190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4fd815b4a0 0x7f4fd806aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd80c9f40 0x7f4fd80cd510 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 --2- 172.21.15.7:0/2169620767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4fd8153880 0x7f4fd806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:53.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 >> 172.21.15.7:0/2169620767 conn(0x7f4fd80737f0 msgr2=0x7f4fd814da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:53.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 shutdown_connections 2023-06-09T11:32:53.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:53.843+0000 7f4fdf9ff700 1 -- 172.21.15.7:0/2169620767 wait complete. 2023-06-09T11:32:53.851 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:53.867 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:32:55.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:56.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 Processor -- start 2023-06-09T11:32:56.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 -- start start 2023-06-09T11:32:56.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe740c2ae0 0x7efe740c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efe740c88e0 0x7efe7415a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe740c9bd0 con 0x7efe740c7cb0 2023-06-09T11:32:56.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.075+0000 7efe7c7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe74159360 con 0x7efe740c2ae0 2023-06-09T11:32:56.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7c7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740c8e20 con 0x7efe740c88e0 2023-06-09T11:32:56.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7a59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efe740c2ae0 0x7efe740c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7ad9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efe740c7cb0 0x7efe7415c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42382/0 (socket says 172.21.15.7:42382) 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7ad9c700 1 -- 172.21.15.7:0/3955201756 learned_addr learned my addr 172.21.15.7:0/3955201756 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe79d9a700 1 --2- 172.21.15.7:0/3955201756 >> v2:172.21.15.117:3300/0 conn(0x7efe740c88e0 0x7efe7415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7a59b700 1 -- 172.21.15.7:0/3955201756 >> v2:172.21.15.117:3300/0 conn(0x7efe740c88e0 msgr2=0x7efe7415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7a59b700 1 --2- 172.21.15.7:0/3955201756 >> v2:172.21.15.117:3300/0 conn(0x7efe740c88e0 0x7efe7415a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7a59b700 1 -- 172.21.15.7:0/3955201756 >> v1:172.21.15.7:6789/0 conn(0x7efe740c7cb0 legacy=0x7efe7415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe7a59b700 1 -- 172.21.15.7:0/3955201756 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74159570 con 0x7efe740c2ae0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.076+0000 7efe79599700 1 -- 172.21.15.7:0/3955201756 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751553131 0 0) 0x7efe740c9bd0 con 0x7efe740c7cb0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7a59b700 1 --2- 172.21.15.7:0/3955201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe740c2eb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efe740c4600 tx=0x7efe70009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba4de32db5fc588 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe79599700 1 -- 172.21.15.7:0/3955201756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70011830 con 0x7efe740c2ae0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe79599700 1 -- 172.21.15.7:0/3955201756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70011990 con 0x7efe740c2ae0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe79599700 1 -- 172.21.15.7:0/3955201756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70011d20 con 0x7efe740c2ae0 2023-06-09T11:32:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 -- 172.21.15.7:0/3955201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 msgr2=0x7efe740c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/3955201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe740c2eb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7efe740c4600 tx=0x7efe70009b20 comp rx=0 tx=0).stop 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 -- 172.21.15.7:0/3955201756 shutdown_connections 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/3955201756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe740c2eb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/3955201756 >> v2:172.21.15.117:3300/0 conn(0x7efe740c88e0 0x7efe7415a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.077+0000 7efe7c7ff700 1 -- 172.21.15.7:0/3955201756 >> 172.21.15.7:0/3955201756 conn(0x7efe740cd330 msgr2=0x7efe740cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.078+0000 7efe7c7ff700 1 -- 172.21.15.7:0/3955201756 shutdown_connections 2023-06-09T11:32:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.078+0000 7efe7c7ff700 1 -- 172.21.15.7:0/3955201756 wait complete. 2023-06-09T11:32:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.079+0000 7efe7c7ff700 1 Processor -- start 2023-06-09T11:32:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.079+0000 7efe7c7ff700 1 -- start start 2023-06-09T11:32:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 0x7efe74150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe740c88e0 0x7efe74151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741f2bc0 con 0x7efe740c2ae0 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741530c0 con 0x7efe740c7cb0 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.080+0000 7efe7c7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe741f2920 con 0x7efe740c88e0 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51702/0 (socket says 172.21.15.7:51702) 2023-06-09T11:32:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 -- 172.21.15.7:0/357169280 learned_addr learned my addr 172.21.15.7:0/357169280 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe79d9a700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 0x7efe74150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7ad9c700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe740c88e0 0x7efe74151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 -- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe740c88e0 msgr2=0x7efe74151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe740c88e0 0x7efe74151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 -- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 msgr2=0x7efe74150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 0x7efe74150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 -- 172.21.15.7:0/357169280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe700094a0 con 0x7efe740c2ae0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe79d9a700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 0x7efe74150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe7a59b700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efe74071280 tx=0x7efe70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.081+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700038a0 con 0x7efe740c2ae0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.082+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7001f070 con 0x7efe740c2ae0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.082+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70003c40 con 0x7efe740c2ae0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.082+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe74159af0 con 0x7efe740c2ae0 2023-06-09T11:32:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.082+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe74159810 con 0x7efe740c2ae0 2023-06-09T11:32:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.083+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74159dd0 con 0x7efe740c2ae0 2023-06-09T11:32:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.084+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe70026480 con 0x7efe740c2ae0 2023-06-09T11:32:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.086+0000 7efe6affd700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 0x7efe5806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.086+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7efe70021070 con 0x7efe740c2ae0 2023-06-09T11:32:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.086+0000 7efe79d9a700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 0x7efe5806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:56.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.086+0000 7efe79d9a700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 0x7efe5806d160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe64009b90 tx=0x7efe64008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.090+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efe70055b40 con 0x7efe740c2ae0 2023-06-09T11:32:56.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.337+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe74074cf0 con 0x7efe740c2ae0 2023-06-09T11:32:56.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.337+0000 7efe6affd700 1 -- 172.21.15.7:0/357169280 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7efe70059160 con 0x7efe740c2ae0 2023-06-09T11:32:56.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:56.344 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 msgr2=0x7efe5806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 0x7efe5806d160 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7efe64009b90 tx=0x7efe64008040 comp rx=0 tx=0).stop 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 msgr2=0x7efe741f1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7efe74071280 tx=0x7efe70004750 comp rx=0 tx=0).stop 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 shutdown_connections 2023-06-09T11:32:56.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe5806acb0 0x7efe5806d160 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe740c88e0 0x7efe74151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe740c2ae0 0x7efe741f1fd0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 --2- 172.21.15.7:0/357169280 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe740c7cb0 0x7efe74150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 >> 172.21.15.7:0/357169280 conn(0x7efe740cd330 msgr2=0x7efe7415bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 shutdown_connections 2023-06-09T11:32:56.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:56.340+0000 7efe7c7ff700 1 -- 172.21.15.7:0/357169280 wait complete. 2023-06-09T11:32:56.348 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:56.363 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:32:58.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:32:58.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.571+0000 7fc4e43ff700 1 Processor -- start 2023-06-09T11:32:58.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.572+0000 7fc4e43ff700 1 -- start start 2023-06-09T11:32:58.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e43ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4dc0cb9a0 0x7fc4dc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:58.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e43ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc159b20 con 0x7fc4dc15cd40 2023-06-09T11:32:58.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e43ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc0cac80 con 0x7fc4dc0cb5d0 2023-06-09T11:32:58.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e43ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc1597d0 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e199a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4dc0cb9a0 0x7fc4dc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:58.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e299c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc4dc15cd40 0x7fc4dc0cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42392/0 (socket says 172.21.15.7:42392) 2023-06-09T11:32:58.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e299c700 1 -- 172.21.15.7:0/3120854649 learned_addr learned my addr 172.21.15.7:0/3120854649 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:32:58.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e199a700 1 -- 172.21.15.7:0/3120854649 >> v1:172.21.15.117:6789/0 conn(0x7fc4dc0cb5d0 legacy=0x7fc4dc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:32:58.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e199a700 1 -- 172.21.15.7:0/3120854649 >> v1:172.21.15.7:6789/0 conn(0x7fc4dc15cd40 legacy=0x7fc4dc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.573+0000 7fc4e199a700 1 -- 172.21.15.7:0/3120854649 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc071ab0 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.574+0000 7fc4e199a700 1 --2- 172.21.15.7:0/3120854649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc0b52b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009840 tx=0x7fc4cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3aa2599807036f2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:58.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.574+0000 7fc4e1199700 1 -- 172.21.15.7:0/3120854649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc004230 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.574+0000 7fc4e1199700 1 -- 172.21.15.7:0/3120854649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4cc004390 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.574+0000 7fc4e1199700 1 -- 172.21.15.7:0/3120854649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0036a0 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 -- 172.21.15.7:0/3120854649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0cb9a0 msgr2=0x7fc4dc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/3120854649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc0b52b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009840 tx=0x7fc4cc009b50 comp rx=0 tx=0).stop 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 -- 172.21.15.7:0/3120854649 shutdown_connections 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/3120854649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc0b52b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 -- 172.21.15.7:0/3120854649 >> 172.21.15.7:0/3120854649 conn(0x7fc4dc070af0 msgr2=0x7fc4dc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:58.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 -- 172.21.15.7:0/3120854649 shutdown_connections 2023-06-09T11:32:58.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.575+0000 7fc4e43ff700 1 -- 172.21.15.7:0/3120854649 wait complete. 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.576+0000 7fc4e43ff700 1 Processor -- start 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 -- start start 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 0x7fc4dc1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc154310 con 0x7fc4dc15cd40 2023-06-09T11:32:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.577+0000 7fc4e43ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc0c2f70 con 0x7fc4dc0cb9a0 2023-06-09T11:32:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e43ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1f1070 con 0x7fc4dc0cb5d0 2023-06-09T11:32:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51726/0 (socket says 172.21.15.7:51726) 2023-06-09T11:32:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 -- 172.21.15.7:0/2726990258 learned_addr learned my addr 172.21.15.7:0/2726990258 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:32:58.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e199a700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:58.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e219b700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 0x7fc4dc1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:58.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 -- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 msgr2=0x7fc4dc1f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 0x7fc4dc1f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 -- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc0cb9a0 msgr2=0x7fc4dc1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 -- 172.21.15.7:0/2726990258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0094a0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e219b700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 0x7fc4dc1f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.578+0000 7fc4e299c700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400caa0 tx=0x7fc4d400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.579+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400aa80 con 0x7fc4dc15cd40 2023-06-09T11:32:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.579+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d400abe0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.579+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc15d4f0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.579+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4012400 con 0x7fc4dc15cd40 2023-06-09T11:32:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.579+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc15d210 con 0x7fc4dc15cd40 2023-06-09T11:32:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.580+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc15d7d0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.581+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4d401b3b0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.582+0000 7fc4d2ffd700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 0x7fc4c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:32:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.582+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fc4d40893f0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.582+0000 7fc4e219b700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 0x7fc4c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:32:58.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.583+0000 7fc4e219b700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 0x7fc4c006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003f40 tx=0x7fc4d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:32:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.586+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4d4054160 con 0x7fc4dc15cd40 2023-06-09T11:32:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.839+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4dc0b44d0 con 0x7fc4dc15cd40 2023-06-09T11:32:58.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.839+0000 7fc4d2ffd700 1 -- 172.21.15.7:0/2726990258 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7fc4d4057780 con 0x7fc4dc15cd40 2023-06-09T11:32:58.845 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:32:58.845 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:32:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 msgr2=0x7fc4c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 0x7fc4c006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003f40 tx=0x7fc4d8009040 comp rx=0 tx=0).stop 2023-06-09T11:32:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 msgr2=0x7fc4dc1507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:32:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400caa0 tx=0x7fc4d400ce60 comp rx=0 tx=0).stop 2023-06-09T11:32:58.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 shutdown_connections 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4dc15cd40 0x7fc4dc1507e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4c006ace0 0x7fc4c006d190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4dc0cb5d0 0x7fc4dc1f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 --2- 172.21.15.7:0/2726990258 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4dc0cb9a0 0x7fc4dc1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 >> 172.21.15.7:0/2726990258 conn(0x7fc4dc070af0 msgr2=0x7fc4dc0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 shutdown_connections 2023-06-09T11:32:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:32:58.842+0000 7fc4e43ff700 1 -- 172.21.15.7:0/2726990258 wait complete. 2023-06-09T11:32:58.849 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:32:58.865 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:33:00.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.074+0000 7f3c4d1ff700 1 Processor -- start 2023-06-09T11:33:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.074+0000 7f3c4d1ff700 1 -- start start 2023-06-09T11:33:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca1b0 0x7f3c4815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4d1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c480b5360 0x7f3c480c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4d1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4815dcf0 con 0x7f3c480ca580 2023-06-09T11:33:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4d1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c480c5e00 con 0x7f3c480b5360 2023-06-09T11:33:01.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4815d450 con 0x7f3c480ca1b0 2023-06-09T11:33:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4759e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3c480ca580 0x7f3c481536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42408/0 (socket says 172.21.15.7:42408) 2023-06-09T11:33:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.075+0000 7f3c4659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c480b5360 0x7f3c480c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4759e700 1 -- 172.21.15.7:0/3188240085 learned_addr learned my addr 172.21.15.7:0/3188240085 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c46d9d700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca1b0 0x7f3c4815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c45d9b700 1 -- 172.21.15.7:0/3188240085 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729246320 0 0) 0x7f3c4815dcf0 con 0x7f3c480ca580 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c45d9b700 1 -- 172.21.15.7:0/3188240085 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c300035b0 con 0x7f3c480ca580 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4659c700 1 -- 172.21.15.7:0/3188240085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca1b0 msgr2=0x7f3c4815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4659c700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca1b0 0x7f3c4815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4659c700 1 -- 172.21.15.7:0/3188240085 >> v1:172.21.15.7:6789/0 conn(0x7f3c480ca580 legacy=0x7f3c481536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4659c700 1 -- 172.21.15.7:0/3188240085 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4815d730 con 0x7f3c480b5360 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.076+0000 7f3c4659c700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c480c58c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009840 tx=0x7f3c3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a9caae5008f12 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c45d9b700 1 -- 172.21.15.7:0/3188240085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c011830 con 0x7f3c480b5360 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c45d9b700 1 -- 172.21.15.7:0/3188240085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c011990 con 0x7f3c480b5360 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c45d9b700 1 -- 172.21.15.7:0/3188240085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c011d20 con 0x7f3c480b5360 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/3188240085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 msgr2=0x7f3c480c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c480c58c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009840 tx=0x7f3c3c009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/3188240085 shutdown_connections 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca1b0 0x7f3c4815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/3188240085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c480c58c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/3188240085 >> 172.21.15.7:0/3188240085 conn(0x7f3c48072090 msgr2=0x7f3c48072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/3188240085 shutdown_connections 2023-06-09T11:33:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.077+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/3188240085 wait complete. 2023-06-09T11:33:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.079+0000 7f3c4d1ff700 1 Processor -- start 2023-06-09T11:33:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.079+0000 7f3c4d1ff700 1 -- start start 2023-06-09T11:33:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 0x7f3c4814cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 0x7f3c4814d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480c68d0 con 0x7f3c480b5360 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480c7780 con 0x7f3c480ca1b0 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c4d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c480cf9b0 con 0x7f3c480ca580 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c46d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c46d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51736/0 (socket says 172.21.15.7:51736) 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.080+0000 7f3c46d9d700 1 -- 172.21.15.7:0/799525066 learned_addr learned my addr 172.21.15.7:0/799525066 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4759e700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 0x7f3c4814d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4659c700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 0x7f3c4814cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 -- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 msgr2=0x7f3c4814d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 0x7f3c4814d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 -- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 msgr2=0x7f3c4814cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 0x7f3c4814cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 -- 172.21.15.7:0/799525066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c0094a0 con 0x7f3c480b5360 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4759e700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 0x7f3c4814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4659c700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 0x7f3c4814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c46d9d700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e9d0 tx=0x7f3c3800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38009730 con 0x7f3c480b5360 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3800c7f0 con 0x7f3c480b5360 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800cb80 con 0x7f3c480b5360 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4806a570 con 0x7f3c480b5360 2023-06-09T11:33:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.081+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4815da10 con 0x7f3c480b5360 2023-06-09T11:33:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.082+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4806a850 con 0x7f3c480b5360 2023-06-09T11:33:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.083+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c3800cce0 con 0x7f3c480b5360 2023-06-09T11:33:01.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.084+0000 7f3c2f7fe700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.084+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7f3c38089d40 con 0x7f3c480b5360 2023-06-09T11:33:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.085+0000 7f3c4659c700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.085+0000 7f3c4659c700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 0x7f3c3406d190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c480b5750 tx=0x7f3c3c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.089+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3c380549f0 con 0x7f3c480b5360 2023-06-09T11:33:01.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.338+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c4815d160 con 0x7f3c480b5360 2023-06-09T11:33:01.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.339+0000 7f3c2f7fe700 1 -- 172.21.15.7:0/799525066 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7f3c38058010 con 0x7f3c480b5360 2023-06-09T11:33:01.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:01.344 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 msgr2=0x7f3c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 0x7f3c3406d190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c480b5750 tx=0x7f3c3c008030 comp rx=0 tx=0).stop 2023-06-09T11:33:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 msgr2=0x7f3c48154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:01.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800e9d0 tx=0x7f3c3800ed90 comp rx=0 tx=0).stop 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 shutdown_connections 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c3406ace0 0x7f3c3406d190 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c480ca580 0x7f3c4814d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c480b5360 0x7f3c48154180 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 --2- 172.21.15.7:0/799525066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c480ca1b0 0x7f3c4814cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 >> 172.21.15.7:0/799525066 conn(0x7f3c48072090 msgr2=0x7f3c481534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:01.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 shutdown_connections 2023-06-09T11:33:01.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:01.341+0000 7f3c4d1ff700 1 -- 172.21.15.7:0/799525066 wait complete. 2023-06-09T11:33:01.348 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:33:01.364 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:33:03.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.571+0000 7fc6f5bff700 1 Processor -- start 2023-06-09T11:33:03.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.571+0000 7fc6f5bff700 1 -- start start 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.572+0000 7fc6f5bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc6f0153880 0x7fc6f015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.572+0000 7fc6f5bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6f00c9f40 0x7fc6f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6f5bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015a500 con 0x7fc6f015b2d0 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6f5bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00c5f10 con 0x7fc6f00c9f40 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6f5bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f015da00 con 0x7fc6f0153880 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6f00c9f40 0x7fc6f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc6f00c9f40 0x7fc6f015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49366/0 (socket says 172.21.15.7:49366) 2023-06-09T11:33:03.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 -- 172.21.15.7:0/3966437900 learned_addr learned my addr 172.21.15.7:0/3966437900 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:03.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6ef7fe700 1 --2- 172.21.15.7:0/3966437900 >> v2:172.21.15.117:3300/0 conn(0x7fc6f0153880 0x7fc6f015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 -- 172.21.15.7:0/3966437900 >> v2:172.21.15.117:3300/0 conn(0x7fc6f0153880 msgr2=0x7fc6f015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:03.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 --2- 172.21.15.7:0/3966437900 >> v2:172.21.15.117:3300/0 conn(0x7fc6f0153880 0x7fc6f015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 -- 172.21.15.7:0/3966437900 >> v1:172.21.15.117:6789/0 conn(0x7fc6f015b2d0 legacy=0x7fc6f00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:03.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.573+0000 7fc6eeffd700 1 -- 172.21.15.7:0/3966437900 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01596e0 con 0x7fc6f00c9f40 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6eeffd700 1 --2- 172.21.15.7:0/3966437900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f00c9f40 0x7fc6f015ad90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc6f008d340 tx=0x7fc6e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55940d247f86d86e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6ee7fc700 1 -- 172.21.15.7:0/3966437900 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4004230 con 0x7fc6f00c9f40 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6ee7fc700 1 -- 172.21.15.7:0/3966437900 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4004390 con 0x7fc6f00c9f40 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6ee7fc700 1 -- 172.21.15.7:0/3966437900 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40036a0 con 0x7fc6f00c9f40 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 -- 172.21.15.7:0/3966437900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f00c9f40 msgr2=0x7fc6f015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/3966437900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f00c9f40 0x7fc6f015ad90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc6f008d340 tx=0x7fc6e4009bd0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 -- 172.21.15.7:0/3966437900 shutdown_connections 2023-06-09T11:33:03.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/3966437900 >> v2:172.21.15.117:3300/0 conn(0x7fc6f0153880 0x7fc6f015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/3966437900 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f00c9f40 0x7fc6f015ad90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.574+0000 7fc6f5bff700 1 -- 172.21.15.7:0/3966437900 >> 172.21.15.7:0/3966437900 conn(0x7fc6f00737f0 msgr2=0x7fc6f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:03.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.575+0000 7fc6f5bff700 1 -- 172.21.15.7:0/3966437900 shutdown_connections 2023-06-09T11:33:03.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.575+0000 7fc6f5bff700 1 -- 172.21.15.7:0/3966437900 wait complete. 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.576+0000 7fc6f5bff700 1 Processor -- start 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.576+0000 7fc6f5bff700 1 -- start start 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 0x7fc6f00b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 0x7fc6f00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b4e00 con 0x7fc6f0153880 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00cbe20 con 0x7fc6f00c9f40 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.577+0000 7fc6f5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b30c0 con 0x7fc6f015b2d0 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49368/0 (socket says 172.21.15.7:49368) 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 -- 172.21.15.7:0/706150863 learned_addr learned my addr 172.21.15.7:0/706150863 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6ef7fe700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 0x7fc6f00b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6effff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 0x7fc6f00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 -- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 msgr2=0x7fc6f00caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 0x7fc6f00caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 -- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 msgr2=0x7fc6f00b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 0x7fc6f00b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 -- 172.21.15.7:0/706150863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e4009550 con 0x7fc6f0153880 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6ef7fe700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 0x7fc6f00b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6effff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 0x7fc6f00caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.578+0000 7fc6eeffd700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009130 tx=0x7fc6e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.579+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4003d40 con 0x7fc6f0153880 2023-06-09T11:33:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.579+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e401f070 con 0x7fc6f0153880 2023-06-09T11:33:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.579+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4011870 con 0x7fc6f0153880 2023-06-09T11:33:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.579+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f0159c60 con 0x7fc6f0153880 2023-06-09T11:33:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.579+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f0159980 con 0x7fc6f0153880 2023-06-09T11:33:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.580+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f0159f40 con 0x7fc6f0153880 2023-06-09T11:33:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.581+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6e4021030 con 0x7fc6f0153880 2023-06-09T11:33:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.582+0000 7fc6d7fff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 0x7fc6dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:03.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.582+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7066+0+0 (secure 0 0 0) 0x7fc6e408ad00 con 0x7fc6f0153880 2023-06-09T11:33:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.582+0000 7fc6ef7fe700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 0x7fc6dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.583+0000 7fc6ef7fe700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 0x7fc6dc06d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009a00 tx=0x7fc6e0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.586+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc6e40559b0 con 0x7fc6f0153880 2023-06-09T11:33:03.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.837+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f00b3230 con 0x7fc6f0153880 2023-06-09T11:33:03.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.838+0000 7fc6d7fff700 1 -- 172.21.15.7:0/706150863 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+6219 (secure 0 0 0) 0x7fc6f00b3230 con 0x7fc6f0153880 2023-06-09T11:33:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:03.843 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5630,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":91}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:31:20.541006+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":60,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5531,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5531":{"gid":5531,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":20,"addr":"172.21.15.117:6833/1018690909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1018690909},{"type":"v1","addr":"172.21.15.117:6833","nonce":1018690909}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 msgr2=0x7fc6dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 0x7fc6dc06d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009a00 tx=0x7fc6e0008040 comp rx=0 tx=0).stop 2023-06-09T11:33:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 msgr2=0x7fc6f01501c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:03.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009130 tx=0x7fc6e4004750 comp rx=0 tx=0).stop 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 shutdown_connections 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc6dc06ace0 0x7fc6dc06d190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc6f015b2d0 0x7fc6f00caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc6f00c9f40 0x7fc6f00b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 --2- 172.21.15.7:0/706150863 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6f0153880 0x7fc6f01501c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 >> 172.21.15.7:0/706150863 conn(0x7fc6f00737f0 msgr2=0x7fc6f015c820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 shutdown_connections 2023-06-09T11:33:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:03.840+0000 7fc6f5bff700 1 -- 172.21.15.7:0/706150863 wait complete. 2023-06-09T11:33:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2023-06-09T11:33:03.863 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:33:05.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:06.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.069+0000 7f74983ff700 1 Processor -- start 2023-06-09T11:33:06.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.069+0000 7f74983ff700 1 -- start start 2023-06-09T11:33:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f74983ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7490153760 0x7f74900ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f74983ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 0x7f74900b5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f74983ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749015a390 con 0x7f7490071fa0 2023-06-09T11:33:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f74983ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74900c5f40 con 0x7f7490153760 2023-06-09T11:33:06.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f74983ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74900ca720 con 0x7f7490153b30 2023-06-09T11:33:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f749619b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7490153760 0x7f74900ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f749699c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7490071fa0 0x7f749015cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59542/0 (socket says 172.21.15.7:59542) 2023-06-09T11:33:06.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f749699c700 1 -- 172.21.15.7:0/1471457915 learned_addr learned my addr 172.21.15.7:0/1471457915 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f749599a700 1 --2- 172.21.15.7:0/1471457915 >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 0x7f74900b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.071+0000 7f749619b700 1 -- 172.21.15.7:0/1471457915 >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 msgr2=0x7f74900b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f749619b700 1 --2- 172.21.15.7:0/1471457915 >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 0x7f74900b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f749619b700 1 -- 172.21.15.7:0/1471457915 >> v1:172.21.15.7:6789/0 conn(0x7f7490071fa0 legacy=0x7f749015cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f749619b700 1 -- 172.21.15.7:0/1471457915 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74900c5a20 con 0x7f7490153760 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f749599a700 1 --2- 172.21.15.7:0/1471457915 >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 0x7f74900b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f7495199700 1 -- 172.21.15.7:0/1471457915 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799270335 0 0) 0x7f749015a390 con 0x7f7490071fa0 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f749619b700 1 --2- 172.21.15.7:0/1471457915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153760 0x7f74900ca070 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f74900b5090 tx=0x7f748c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb4702d3fa78eb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f7495199700 1 -- 172.21.15.7:0/1471457915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011830 con 0x7f7490153760 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f7495199700 1 -- 172.21.15.7:0/1471457915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c011990 con 0x7f7490153760 2023-06-09T11:33:06.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.072+0000 7f7495199700 1 -- 172.21.15.7:0/1471457915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c011d20 con 0x7f7490153760 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 -- 172.21.15.7:0/1471457915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153760 msgr2=0x7f74900ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 --2- 172.21.15.7:0/1471457915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153760 0x7f74900ca070 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f74900b5090 tx=0x7f748c009b20 comp rx=0 tx=0).stop 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 -- 172.21.15.7:0/1471457915 shutdown_connections 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 --2- 172.21.15.7:0/1471457915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153760 0x7f74900ca070 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 --2- 172.21.15.7:0/1471457915 >> v2:172.21.15.117:3300/0 conn(0x7f7490153b30 0x7f74900b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 -- 172.21.15.7:0/1471457915 >> 172.21.15.7:0/1471457915 conn(0x7f74900737f0 msgr2=0x7f7490073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 -- 172.21.15.7:0/1471457915 shutdown_connections 2023-06-09T11:33:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.073+0000 7f74983ff700 1 -- 172.21.15.7:0/1471457915 wait complete. 2023-06-09T11:33:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.074+0000 7f74983ff700 1 Processor -- start 2023-06-09T11:33:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.075+0000 7f74983ff700 1 -- start start 2023-06-09T11:33:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.075+0000 7f74983ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 0x7f74900c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.075+0000 7f74983ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7490153760 0x7f74901ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f74983ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f74983ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74901ef380 con 0x7f7490153b30 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f74983ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74900c8eb0 con 0x7f7490071fa0 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f74983ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74900c9180 con 0x7f7490153760 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49390/0 (socket says 172.21.15.7:49390) 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 -- 172.21.15.7:0/2966818044 learned_addr learned my addr 172.21.15.7:0/2966818044 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749619b700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 0x7f74900c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749599a700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7490153760 0x7f74901ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 -- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7490153760 msgr2=0x7f74901ef7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7490153760 0x7f74901ef7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 -- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 msgr2=0x7f74900c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 0x7f74900c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749699c700 1 -- 172.21.15.7:0/2966818044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0094a0 con 0x7f7490153b30 2023-06-09T11:33:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.076+0000 7f749619b700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 0x7f74900c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f749699c700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f748800ccd0 tx=0x7f748800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800ada0 con 0x7f7490153b30 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880070d0 con 0x7f7490153b30 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488007480 con 0x7f7490153b30 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7490159a60 con 0x7f7490153b30 2023-06-09T11:33:06.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.077+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7490159780 con 0x7f7490153b30 2023-06-09T11:33:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.078+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490159d40 con 0x7f7490153b30 2023-06-09T11:33:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.079+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f74880075e0 con 0x7f7490153b30 2023-06-09T11:33:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.080+0000 7f7486ffd700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 0x7f747406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.080+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f7488089550 con 0x7f7490153b30 2023-06-09T11:33:06.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.080+0000 7f749619b700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 0x7f747406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.081+0000 7f749619b700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 0x7f747406d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f748c0090b0 tx=0x7f748c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.084+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f74880541d0 con 0x7f7490153b30 2023-06-09T11:33:06.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.339+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74901ef4c0 con 0x7f7490153b30 2023-06-09T11:33:06.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.340+0000 7f7486ffd700 1 -- 172.21.15.7:0/2966818044 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f74901ef4c0 con 0x7f7490153b30 2023-06-09T11:33:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:06.345 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:04.416639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:06.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 msgr2=0x7f747406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 0x7f747406d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f748c0090b0 tx=0x7f748c008030 comp rx=0 tx=0).stop 2023-06-09T11:33:06.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 msgr2=0x7f74900b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f748800ccd0 tx=0x7f748800a340 comp rx=0 tx=0).stop 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 shutdown_connections 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7490153b30 0x7f74900b32e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7490071fa0 0x7f74900c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747406ace0 0x7f747406d190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 --2- 172.21.15.7:0/2966818044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7490153760 0x7f74901ef7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 >> 172.21.15.7:0/2966818044 conn(0x7f74900737f0 msgr2=0x7f74900b50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:06.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 shutdown_connections 2023-06-09T11:33:06.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.342+0000 7f74983ff700 1 -- 172.21.15.7:0/2966818044 wait complete. 2023-06-09T11:33:06.349 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2023-06-09T11:33:06.366 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:33:06.366 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:33:06.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:06.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.561+0000 7f36337ff700 1 Processor -- start 2023-06-09T11:33:06.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.561+0000 7f36337ff700 1 -- start start 2023-06-09T11:33:06.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.562+0000 7f36337ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 0x7f362c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f36337ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c15a500 con 0x7f362c0c9f40 2023-06-09T11:33:06.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f36337ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c15da00 con 0x7f362c15b4a0 2023-06-09T11:33:06.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f36337ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0c5f10 con 0x7f362c153880 2023-06-09T11:33:06.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f3630d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f362c0c9f40 0x7f362c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59546/0 (socket says 172.21.15.7:59546) 2023-06-09T11:33:06.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f3630d9a700 1 -- 172.21.15.7:0/1805938268 learned_addr learned my addr 172.21.15.7:0/1805938268 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:06.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f363159b700 1 --2- 172.21.15.7:0/1805938268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 0x7f362c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937734104 0 0) 0x7f361c001110 con 0x7f362c0c9f40 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.563+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36180035b0 con 0x7f362c0c9f40 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164379507 0 0) 0x7f3624001490 con 0x7f362c15b4a0 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f363159b700 1 -- 172.21.15.7:0/1805938268 >> v1:172.21.15.117:6789/0 conn(0x7f362c15b4a0 legacy=0x7f362c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f363159b700 1 -- 172.21.15.7:0/1805938268 >> v1:172.21.15.7:6789/0 conn(0x7f362c0c9f40 legacy=0x7f362c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f363159b700 1 -- 172.21.15.7:0/1805938268 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c1596e0 con 0x7f362c153880 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f363159b700 1 --2- 172.21.15.7:0/1805938268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 0x7f362c15d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f3628009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b4ed15ef9f2454 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c001800 con 0x7f362c153880 2023-06-09T11:33:06.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628004030 con 0x7f362c153880 2023-06-09T11:33:06.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.564+0000 7f3623fff700 1 -- 172.21.15.7:0/1805938268 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280043a0 con 0x7f362c153880 2023-06-09T11:33:06.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 -- 172.21.15.7:0/1805938268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 msgr2=0x7f362c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 --2- 172.21.15.7:0/1805938268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 0x7f362c15d4c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f3628009b80 comp rx=0 tx=0).stop 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 -- 172.21.15.7:0/1805938268 shutdown_connections 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 --2- 172.21.15.7:0/1805938268 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c153880 0x7f362c15d4c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 -- 172.21.15.7:0/1805938268 >> 172.21.15.7:0/1805938268 conn(0x7f362c0737f0 msgr2=0x7f362c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 -- 172.21.15.7:0/1805938268 shutdown_connections 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.565+0000 7f36337ff700 1 -- 172.21.15.7:0/1805938268 wait complete. 2023-06-09T11:33:06.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.567+0000 7f36337ff700 1 Processor -- start 2023-06-09T11:33:06.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.567+0000 7f36337ff700 1 -- start start 2023-06-09T11:33:06.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362c0c9f40 0x7f362c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c15b4a0 0x7f362c14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1e5b90 con 0x7f362c153880 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1ef670 con 0x7f362c0c9f40 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f36337ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0ce8c0 con 0x7f362c15b4a0 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49396/0 (socket says 172.21.15.7:49396) 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 -- 172.21.15.7:0/1117843337 learned_addr learned my addr 172.21.15.7:0/1117843337 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3631d9c700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c15b4a0 0x7f362c14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f363159b700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362c0c9f40 0x7f362c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 -- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c15b4a0 msgr2=0x7f362c14cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c15b4a0 0x7f362c14cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 -- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362c0c9f40 msgr2=0x7f362c152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362c0c9f40 0x7f362c152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.568+0000 7f3630d9a700 1 -- 172.21.15.7:0/1117843337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280094a0 con 0x7f362c153880 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f3630d9a700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c750 tx=0x7f361c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c004c80 con 0x7f362c153880 2023-06-09T11:33:06.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c004de0 con 0x7f362c153880 2023-06-09T11:33:06.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c007310 con 0x7f362c153880 2023-06-09T11:33:06.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c159c60 con 0x7f362c153880 2023-06-09T11:33:06.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.569+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c159980 con 0x7f362c153880 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.570+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c159f40 con 0x7f362c153880 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.571+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f361c007470 con 0x7f362c153880 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.572+0000 7f3621ffb700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 0x7f361006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.572+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f361c0895e0 con 0x7f362c153880 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.572+0000 7f363159b700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 0x7f361006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:06.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.573+0000 7f363159b700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 0x7f361006d1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3628000c00 tx=0x7f3628008000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:06.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.576+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f361c054200 con 0x7f362c153880 2023-06-09T11:33:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.828+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f362c15a500 con 0x7f362c153880 2023-06-09T11:33:06.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.828+0000 7f3621ffb700 1 -- 172.21.15.7:0/1117843337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f362c15a500 con 0x7f362c153880 2023-06-09T11:33:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:06.834 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:04.416639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 msgr2=0x7f361006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 0x7f361006d1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3628000c00 tx=0x7f3628008000 comp rx=0 tx=0).stop 2023-06-09T11:33:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 msgr2=0x7f362c0cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f361c00c750 tx=0x7f361c00ca60 comp rx=0 tx=0).stop 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 shutdown_connections 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f361006ad10 0x7f361006d1c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f362c15b4a0 0x7f362c14cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f362c0c9f40 0x7f362c152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 --2- 172.21.15.7:0/1117843337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f362c153880 0x7f362c0cd3a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 >> 172.21.15.7:0/1117843337 conn(0x7f362c0737f0 msgr2=0x7f362c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 shutdown_connections 2023-06-09T11:33:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:06.831+0000 7f36337ff700 1 -- 172.21.15.7:0/1117843337 wait complete. 2023-06-09T11:33:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2023-06-09T11:33:06.855 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 94, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 94, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5630": { "addr": "172.21.15.117:6835/3162263716", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3162263716, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3162263716, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5630, "incarnation": 94, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:33:04.416639+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5630, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:33:08.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:09.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.061+0000 7f37c5bff700 1 Processor -- start 2023-06-09T11:33:09.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.062+0000 7f37c5bff700 1 -- start start 2023-06-09T11:33:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37c5bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37c00ca1b0 0x7f37c015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37c5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00b5360 0x7f37c00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37c5bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c015dcf0 con 0x7f37c00ca580 2023-06-09T11:33:09.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37c5bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c00c5e00 con 0x7f37c00ca1b0 2023-06-09T11:33:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37c5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c015d450 con 0x7f37c00b5360 2023-06-09T11:33:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37bf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f37c00ca1b0 0x7f37c015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37bffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f37c00ca580 0x7f37c01536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59548/0 (socket says 172.21.15.7:59548) 2023-06-09T11:33:09.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37bffff700 1 -- 172.21.15.7:0/4197389534 learned_addr learned my addr 172.21.15.7:0/4197389534 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.063+0000 7f37b7fff700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00b5360 0x7f37c00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37bf7fe700 1 -- 172.21.15.7:0/4197389534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00b5360 msgr2=0x7f37c00c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37bf7fe700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00b5360 0x7f37c00c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37bf7fe700 1 -- 172.21.15.7:0/4197389534 >> v1:172.21.15.7:6789/0 conn(0x7f37c00ca580 legacy=0x7f37c01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37bf7fe700 1 -- 172.21.15.7:0/4197389534 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c015d730 con 0x7f37c00ca1b0 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37beffd700 1 -- 172.21.15.7:0/4197389534 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922574975 0 0) 0x7f37c015dcf0 con 0x7f37c00ca580 2023-06-09T11:33:09.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37bf7fe700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca1b0 0x7f37c015cdc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009840 tx=0x7f37b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8eedbc1af88be9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37beffd700 1 -- 172.21.15.7:0/4197389534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011830 con 0x7f37c00ca1b0 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37beffd700 1 -- 172.21.15.7:0/4197389534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b0011990 con 0x7f37c00ca1b0 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.064+0000 7f37beffd700 1 -- 172.21.15.7:0/4197389534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0011d20 con 0x7f37c00ca1b0 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 -- 172.21.15.7:0/4197389534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca1b0 msgr2=0x7f37c015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca1b0 0x7f37c015cdc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009840 tx=0x7f37b0009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 -- 172.21.15.7:0/4197389534 shutdown_connections 2023-06-09T11:33:09.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca1b0 0x7f37c015cdc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 --2- 172.21.15.7:0/4197389534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00b5360 0x7f37c00c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 -- 172.21.15.7:0/4197389534 >> 172.21.15.7:0/4197389534 conn(0x7f37c0072090 msgr2=0x7f37c0072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:09.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 -- 172.21.15.7:0/4197389534 shutdown_connections 2023-06-09T11:33:09.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.065+0000 7f37c5bff700 1 -- 172.21.15.7:0/4197389534 wait complete. 2023-06-09T11:33:09.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.067+0000 7f37c5bff700 1 Processor -- start 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.067+0000 7f37c5bff700 1 -- start start 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37c00b5360 0x7f37c0155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00ca1b0 0x7f37c014f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c01553d0 con 0x7f37c00ca580 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c0154180 con 0x7f37c00b5360 2023-06-09T11:33:09.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37c5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c01523f0 con 0x7f37c00ca1b0 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49410/0 (socket says 172.21.15.7:49410) 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37bffff700 1 -- 172.21.15.7:0/2419022698 learned_addr learned my addr 172.21.15.7:0/2419022698 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37b7fff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00ca1b0 0x7f37c014f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.068+0000 7f37bf7fe700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37c00b5360 0x7f37c0155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 -- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00ca1b0 msgr2=0x7f37c014f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00ca1b0 0x7f37c014f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 -- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37c00b5360 msgr2=0x7f37c0155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37c00b5360 0x7f37c0155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 -- 172.21.15.7:0/2419022698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00094a0 con 0x7f37c00ca580 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bffff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f37b800cb60 tx=0x7f37b800ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b800a9e0 con 0x7f37c00ca580 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b800ab40 con 0x7f37c00ca580 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c006a570 con 0x7f37c00ca580 2023-06-09T11:33:09.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b80070d0 con 0x7f37c00ca580 2023-06-09T11:33:09.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.069+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c015da10 con 0x7f37c00ca580 2023-06-09T11:33:09.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.071+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c006a850 con 0x7f37c00ca580 2023-06-09T11:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.071+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f37b8007230 con 0x7f37c00ca580 2023-06-09T11:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.072+0000 7f37bcff9700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 0x7f37a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.072+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f37b8089690 con 0x7f37c00ca580 2023-06-09T11:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.073+0000 7f37bf7fe700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 0x7f37a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.073+0000 7f37bf7fe700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 0x7f37a006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000c00 tx=0x7f37b0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.076+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f37b80541f0 con 0x7f37c00ca580 2023-06-09T11:33:09.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.329+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37c0155580 con 0x7f37c00ca580 2023-06-09T11:33:09.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.330+0000 7f37bcff9700 1 -- 172.21.15.7:0/2419022698 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5455 (secure 0 0 0) 0x7f37c0155580 con 0x7f37c00ca580 2023-06-09T11:33:09.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:09.335 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:04.416639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:09.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 msgr2=0x7f37a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 0x7f37a006d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f37b0000c00 tx=0x7f37b0008030 comp rx=0 tx=0).stop 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 msgr2=0x7f37c01f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f37b800cb60 tx=0x7f37b800ce70 comp rx=0 tx=0).stop 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 shutdown_connections 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37c00ca580 0x7f37c01f2560 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37c00b5360 0x7f37c0155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37a006ace0 0x7f37a006d190 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 --2- 172.21.15.7:0/2419022698 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37c00ca1b0 0x7f37c014f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:09.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 >> 172.21.15.7:0/2419022698 conn(0x7f37c0072090 msgr2=0x7f37c0151ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:09.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.332+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 shutdown_connections 2023-06-09T11:33:09.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:09.333+0000 7f37c5bff700 1 -- 172.21.15.7:0/2419022698 wait complete. 2023-06-09T11:33:09.339 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2023-06-09T11:33:09.356 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:11.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:11.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.564+0000 7fd6977ff700 1 Processor -- start 2023-06-09T11:33:11.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.565+0000 7fd6977ff700 1 -- start start 2023-06-09T11:33:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd6977ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd6900c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd6977ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 0x7fd690159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd6977ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69015cec0 con 0x7fd6900cad00 2023-06-09T11:33:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd6977ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69015d160 con 0x7fd6901598b0 2023-06-09T11:33:11.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd6977ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69015d440 con 0x7fd6900c9090 2023-06-09T11:33:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd694d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd6900cad00 0x7fd6900cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59562/0 (socket says 172.21.15.7:59562) 2023-06-09T11:33:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd694d9a700 1 -- 172.21.15.7:0/4093245645 learned_addr learned my addr 172.21.15.7:0/4093245645 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd69559b700 1 --2- 172.21.15.7:0/4093245645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd6900c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd695d9c700 1 --2- 172.21.15.7:0/4093245645 >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 0x7fd690159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.566+0000 7fd687fff700 1 -- 172.21.15.7:0/4093245645 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116126636 0 0) 0x7fd69015cec0 con 0x7fd6900cad00 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd687fff700 1 -- 172.21.15.7:0/4093245645 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd67c0035b0 con 0x7fd6900cad00 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd69559b700 1 -- 172.21.15.7:0/4093245645 >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 msgr2=0x7fd690159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd69559b700 1 --2- 172.21.15.7:0/4093245645 >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 0x7fd690159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd69559b700 1 -- 172.21.15.7:0/4093245645 >> v1:172.21.15.7:6789/0 conn(0x7fd6900cad00 legacy=0x7fd6900cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd69559b700 1 -- 172.21.15.7:0/4093245645 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69015d720 con 0x7fd6900c9090 2023-06-09T11:33:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd695d9c700 1 --2- 172.21.15.7:0/4093245645 >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 0x7fd690159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd69559b700 1 --2- 172.21.15.7:0/4093245645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd6900c9fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009840 tx=0x7fd68c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b4a4848e67af9d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd687fff700 1 -- 172.21.15.7:0/4093245645 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c011830 con 0x7fd6900c9090 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd687fff700 1 -- 172.21.15.7:0/4093245645 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c011990 con 0x7fd6900c9090 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.567+0000 7fd687fff700 1 -- 172.21.15.7:0/4093245645 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c011d20 con 0x7fd6900c9090 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 -- 172.21.15.7:0/4093245645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 msgr2=0x7fd6900c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 --2- 172.21.15.7:0/4093245645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd6900c9fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009840 tx=0x7fd68c009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 -- 172.21.15.7:0/4093245645 shutdown_connections 2023-06-09T11:33:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 --2- 172.21.15.7:0/4093245645 >> v2:172.21.15.117:3300/0 conn(0x7fd6901598b0 0x7fd690159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 --2- 172.21.15.7:0/4093245645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd6900c9fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 -- 172.21.15.7:0/4093245645 >> 172.21.15.7:0/4093245645 conn(0x7fd690070af0 msgr2=0x7fd690070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 -- 172.21.15.7:0/4093245645 shutdown_connections 2023-06-09T11:33:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.568+0000 7fd6977ff700 1 -- 172.21.15.7:0/4093245645 wait complete. 2023-06-09T11:33:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.570+0000 7fd6977ff700 1 Processor -- start 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.571+0000 7fd6977ff700 1 -- start start 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6900cad00 0x7fd6900ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 0x7fd690154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901ef6c0 con 0x7fd6901598b0 2023-06-09T11:33:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6901f0690 con 0x7fd6900cad00 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd6977ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd690156440 con 0x7fd6900c9090 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd695d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 0x7fd690154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd69559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd69559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:48234/0 (socket says 172.21.15.7:48234) 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd69559b700 1 -- 172.21.15.7:0/1584286789 learned_addr learned my addr 172.21.15.7:0/1584286789 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.572+0000 7fd694d9a700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6900cad00 0x7fd6900ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 -- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6900cad00 msgr2=0x7fd6900ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6900cad00 0x7fd6900ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 -- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 msgr2=0x7fd690154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 0x7fd690154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 -- 172.21.15.7:0/1584286789 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009a70 con 0x7fd6900c9090 2023-06-09T11:33:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd695d9c700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 0x7fd690154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd69559b700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009fd0 tx=0x7fd68c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c004550 con 0x7fd6900c9090 2023-06-09T11:33:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c01e070 con 0x7fd6900c9090 2023-06-09T11:33:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c0038e0 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68c0094a0 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.573+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69015a250 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.574+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6900ca620 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.575+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd68c003a40 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.576+0000 7fd685ffb700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 0x7fd67406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.576+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fd68c08a740 con 0x7fd6900c9090 2023-06-09T11:33:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.577+0000 7fd694d9a700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 0x7fd67406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.577+0000 7fd694d9a700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 0x7fd67406d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6900b55d0 tx=0x7fd680007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.581+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd68c055330 con 0x7fd6900c9090 2023-06-09T11:33:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.831+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6900cb130 con 0x7fd6900c9090 2023-06-09T11:33:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.832+0000 7fd685ffb700 1 -- 172.21.15.7:0/1584286789 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+5455 (secure 0 0 0) 0x7fd68c058950 con 0x7fd6900c9090 2023-06-09T11:33:11.838 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:11.838 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":94,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:04.416639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 msgr2=0x7fd67406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 0x7fd67406d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd6900b55d0 tx=0x7fd680007dc0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 msgr2=0x7fd690156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:11.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd68c009fd0 tx=0x7fd68c004750 comp rx=0 tx=0).stop 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 shutdown_connections 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6901598b0 0x7fd690154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd67406ace0 0x7fd67406d190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd6900c9090 0x7fd690156720 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 --2- 172.21.15.7:0/1584286789 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd6900cad00 0x7fd6900ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 >> 172.21.15.7:0/1584286789 conn(0x7fd690070af0 msgr2=0x7fd6901585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 shutdown_connections 2023-06-09T11:33:11.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:11.835+0000 7fd6977ff700 1 -- 172.21.15.7:0/1584286789 wait complete. 2023-06-09T11:33:11.842 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 94 2023-06-09T11:33:11.859 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:13.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.066+0000 7f08ed5ff700 1 Processor -- start 2023-06-09T11:33:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.066+0000 7f08ed5ff700 1 -- start start 2023-06-09T11:33:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.067+0000 7f08ed5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca1b0 0x7f08e815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.067+0000 7f08ed5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08e80b5360 0x7f08e80c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.067+0000 7f08ed5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e815dcf0 con 0x7f08e80ca580 2023-06-09T11:33:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.067+0000 7f08ed5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e80c5e00 con 0x7f08e80b5360 2023-06-09T11:33:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08ed5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e815d450 con 0x7f08e80ca1b0 2023-06-09T11:33:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e67fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08e80b5360 0x7f08e80c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f08e80ca580 0x7f08e81536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39588/0 (socket says 172.21.15.7:39588) 2023-06-09T11:33:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e77fe700 1 -- 172.21.15.7:0/1809206269 learned_addr learned my addr 172.21.15.7:0/1809206269 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e6ffd700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca1b0 0x7f08e815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e5ffb700 1 -- 172.21.15.7:0/1809206269 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091064475 0 0) 0x7f08e815dcf0 con 0x7f08e80ca580 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e67fc700 1 -- 172.21.15.7:0/1809206269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca1b0 msgr2=0x7f08e815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e67fc700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca1b0 0x7f08e815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e67fc700 1 -- 172.21.15.7:0/1809206269 >> v1:172.21.15.7:6789/0 conn(0x7f08e80ca580 legacy=0x7f08e81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.068+0000 7f08e67fc700 1 -- 172.21.15.7:0/1809206269 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e815d730 con 0x7f08e80b5360 2023-06-09T11:33:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08e67fc700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80b5360 0x7f08e80c58c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009840 tx=0x7f08dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df90670a847bdffe server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08e5ffb700 1 -- 172.21.15.7:0/1809206269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011830 con 0x7f08e80b5360 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08e5ffb700 1 -- 172.21.15.7:0/1809206269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc011990 con 0x7f08e80b5360 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08e5ffb700 1 -- 172.21.15.7:0/1809206269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011d20 con 0x7f08e80b5360 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08ed5ff700 1 -- 172.21.15.7:0/1809206269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80b5360 msgr2=0x7f08e80c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.069+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80b5360 0x7f08e80c58c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009840 tx=0x7f08dc009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 -- 172.21.15.7:0/1809206269 shutdown_connections 2023-06-09T11:33:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca1b0 0x7f08e815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/1809206269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80b5360 0x7f08e80c58c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 -- 172.21.15.7:0/1809206269 >> 172.21.15.7:0/1809206269 conn(0x7f08e8072090 msgr2=0x7f08e8072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 -- 172.21.15.7:0/1809206269 shutdown_connections 2023-06-09T11:33:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.070+0000 7f08ed5ff700 1 -- 172.21.15.7:0/1809206269 wait complete. 2023-06-09T11:33:14.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.071+0000 7f08ed5ff700 1 Processor -- start 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.072+0000 7f08ed5ff700 1 -- start start 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.072+0000 7f08ed5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e80b5360 0x7f08e81e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.072+0000 7f08ed5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08ed5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 0x7f08e814c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08ed5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80b45b0 con 0x7f08e80ca1b0 2023-06-09T11:33:14.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08ed5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81e5a50 con 0x7f08e80b5360 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08ed5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e81e2cc0 con 0x7f08e80ca580 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38748/0 (socket says 172.21.15.7:38748) 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 -- 172.21.15.7:0/3303265778 learned_addr learned my addr 172.21.15.7:0/3303265778 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e6ffd700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e80b5360 0x7f08e81e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e77fe700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 0x7f08e814c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 -- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 msgr2=0x7f08e814c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 0x7f08e814c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 -- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e80b5360 msgr2=0x7f08e81e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e80b5360 0x7f08e81e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e67fc700 1 -- 172.21.15.7:0/3303265778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0094a0 con 0x7f08e80ca1b0 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.073+0000 7f08e77fe700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 0x7f08e814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08e67fc700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fd0 tx=0x7f08dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc0036a0 con 0x7f08e80ca1b0 2023-06-09T11:33:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc01f070 con 0x7f08e80ca1b0 2023-06-09T11:33:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc003a20 con 0x7f08e80ca1b0 2023-06-09T11:33:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e806a570 con 0x7f08e80ca1b0 2023-06-09T11:33:14.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.074+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e815da10 con 0x7f08e80ca1b0 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.075+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e806a850 con 0x7f08e80ca1b0 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.076+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08dc003b80 con 0x7f08e80ca1b0 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.077+0000 7f08cf7fe700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 0x7f08d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.077+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f08dc021070 con 0x7f08e80ca1b0 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.077+0000 7f08e6ffd700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 0x7f08d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:14.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.078+0000 7f08e6ffd700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 0x7f08d406d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009be0 tx=0x7f08d8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:14.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.081+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08dc0557b0 con 0x7f08e80ca1b0 2023-06-09T11:33:14.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.341+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08e81e38d0 con 0x7f08e80ca1b0 2023-06-09T11:33:14.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.342+0000 7f08cf7fe700 1 -- 172.21.15.7:0/3303265778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f08dc015020 con 0x7f08e80ca1b0 2023-06-09T11:33:14.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:14.347 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:12.625559+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:resolve","state_seq":31,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:14.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 msgr2=0x7f08d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 0x7f08d406d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009be0 tx=0x7f08d8009210 comp rx=0 tx=0).stop 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 msgr2=0x7f08e8153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08dc009fd0 tx=0x7f08dc004750 comp rx=0 tx=0).stop 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 shutdown_connections 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08d406ace0 0x7f08d406d190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08e80ca580 0x7f08e814c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08e80b5360 0x7f08e81e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 --2- 172.21.15.7:0/3303265778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08e80ca1b0 0x7f08e8153730 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 >> 172.21.15.7:0/3303265778 conn(0x7f08e8072090 msgr2=0x7f08e815cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 shutdown_connections 2023-06-09T11:33:14.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:14.344+0000 7f08ed5ff700 1 -- 172.21.15.7:0/3303265778 wait complete. 2023-06-09T11:33:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 95 2023-06-09T11:33:14.367 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:16.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:16.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.574+0000 7f01b03ff700 1 Processor -- start 2023-06-09T11:33:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.574+0000 7f01b03ff700 1 -- start start 2023-06-09T11:33:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.575+0000 7f01b03ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f01a8153760 0x7f01a80ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.575+0000 7f01b03ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01a8071fa0 0x7f01a8072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01b03ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a815a390 con 0x7f01a8153b30 2023-06-09T11:33:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01b03ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80c5f40 con 0x7f01a8071fa0 2023-06-09T11:33:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01b03ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a80728b0 con 0x7f01a8153760 2023-06-09T11:33:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01a8071fa0 0x7f01a8072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f01a8071fa0 0x7f01a8072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38752/0 (socket says 172.21.15.7:38752) 2023-06-09T11:33:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 -- 172.21.15.7:0/1665288828 learned_addr learned my addr 172.21.15.7:0/1665288828 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae19b700 1 --2- 172.21.15.7:0/1665288828 >> v2:172.21.15.117:3300/0 conn(0x7f01a8153760 0x7f01a80ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 -- 172.21.15.7:0/1665288828 >> v2:172.21.15.117:3300/0 conn(0x7f01a8153760 msgr2=0x7f01a80ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 --2- 172.21.15.7:0/1665288828 >> v2:172.21.15.117:3300/0 conn(0x7f01a8153760 0x7f01a80ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 -- 172.21.15.7:0/1665288828 >> v1:172.21.15.117:6789/0 conn(0x7f01a8153b30 legacy=0x7f01a80b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.576+0000 7f01ae99c700 1 -- 172.21.15.7:0/1665288828 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80ca720 con 0x7f01a8071fa0 2023-06-09T11:33:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01ae99c700 1 --2- 172.21.15.7:0/1665288828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a8072370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01a80743c0 tx=0x7f01a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390ac2dffed5b28f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01ad199700 1 -- 172.21.15.7:0/1665288828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000e070 con 0x7f01a8071fa0 2023-06-09T11:33:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01ad199700 1 -- 172.21.15.7:0/1665288828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a000b8d0 con 0x7f01a8071fa0 2023-06-09T11:33:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01ad199700 1 -- 172.21.15.7:0/1665288828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a000bc80 con 0x7f01a8071fa0 2023-06-09T11:33:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01b03ff700 1 -- 172.21.15.7:0/1665288828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 msgr2=0x7f01a8072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.577+0000 7f01b03ff700 1 --2- 172.21.15.7:0/1665288828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a8072370 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01a80743c0 tx=0x7f01a000b330 comp rx=0 tx=0).stop 2023-06-09T11:33:16.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 -- 172.21.15.7:0/1665288828 shutdown_connections 2023-06-09T11:33:16.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 --2- 172.21.15.7:0/1665288828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a8072370 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 --2- 172.21.15.7:0/1665288828 >> v2:172.21.15.117:3300/0 conn(0x7f01a8153760 0x7f01a80ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 -- 172.21.15.7:0/1665288828 >> 172.21.15.7:0/1665288828 conn(0x7f01a80737f0 msgr2=0x7f01a8073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 -- 172.21.15.7:0/1665288828 shutdown_connections 2023-06-09T11:33:16.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.578+0000 7f01b03ff700 1 -- 172.21.15.7:0/1665288828 wait complete. 2023-06-09T11:33:16.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.579+0000 7f01b03ff700 1 Processor -- start 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.579+0000 7f01b03ff700 1 -- start start 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 0x7f01a8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01a8153b30 0x7f01a81e5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81510b0 con 0x7f01a8071fa0 2023-06-09T11:33:16.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a81571a0 con 0x7f01a8153760 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.580+0000 7f01b03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a8151300 con 0x7f01a8153b30 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38754/0 (socket says 172.21.15.7:38754) 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 -- 172.21.15.7:0/2398891128 learned_addr learned my addr 172.21.15.7:0/2398891128 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ad99a700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 0x7f01a8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae99c700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01a8153b30 0x7f01a81e5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 -- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01a8153b30 msgr2=0x7f01a81e5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01a8153b30 0x7f01a81e5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 -- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 msgr2=0x7f01a8157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 0x7f01a8157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 -- 172.21.15.7:0/2398891128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0009b30 con 0x7f01a8071fa0 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ad99a700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 0x7f01a8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01ae19b700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f01a400ea00 tx=0x7f01a400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4009730 con 0x7f01a8071fa0 2023-06-09T11:33:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400c850 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400cbe0 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.581+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a8159780 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.582+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80c5a20 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.583+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8159a60 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.584+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01a400cd40 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.584+0000 7f019effd700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 0x7f018c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.585+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f01a40899f0 con 0x7f01a8071fa0 2023-06-09T11:33:16.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.585+0000 7f01ad99a700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 0x7f018c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.585+0000 7f01ad99a700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 0x7f018c06d1c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01a80b22d0 tx=0x7f0198009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.589+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f01a4054670 con 0x7f01a8071fa0 2023-06-09T11:33:16.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:16.703 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:16.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.845+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01a8074cf0 con 0x7f01a8071fa0 2023-06-09T11:33:16.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.846+0000 7f019effd700 1 -- 172.21.15.7:0/2398891128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+5457 (secure 0 0 0) 0x7f01a4057c90 con 0x7f01a8071fa0 2023-06-09T11:33:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:16.851 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:12.625559+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:resolve","state_seq":31,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:16.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 msgr2=0x7f018c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 0x7f018c06d1c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f01a80b22d0 tx=0x7f0198009040 comp rx=0 tx=0).stop 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 msgr2=0x7f01a81f2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f01a400ea00 tx=0x7f01a400edc0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 shutdown_connections 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f018c06ad10 0x7f018c06d1c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01a8153b30 0x7f01a81e5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01a8071fa0 0x7f01a81f2a70 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 --2- 172.21.15.7:0/2398891128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f01a8153760 0x7f01a8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 >> 172.21.15.7:0/2398891128 conn(0x7f01a80737f0 msgr2=0x7f01a80cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 shutdown_connections 2023-06-09T11:33:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:16.848+0000 7f01b03ff700 1 -- 172.21.15.7:0/2398891128 wait complete. 2023-06-09T11:33:16.855 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 95 2023-06-09T11:33:16.872 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:18.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.078+0000 7f0c3cfff700 1 Processor -- start 2023-06-09T11:33:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.078+0000 7f0c3cfff700 1 -- start start 2023-06-09T11:33:19.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.079+0000 7f0c3cfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c380c9f40 0x7f0c380c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.079+0000 7f0c3cfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c380c5a80 con 0x7f0c38153880 2023-06-09T11:33:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.079+0000 7f0c3cfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3815a500 con 0x7f0c3815b4a0 2023-06-09T11:33:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.079+0000 7f0c3cfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c380c5f10 con 0x7f0c380c9f40 2023-06-09T11:33:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c38153880 0x7f0c3815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39594/0 (socket says 172.21.15.7:39594) 2023-06-09T11:33:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3659c700 1 -- 172.21.15.7:0/3773589403 learned_addr learned my addr 172.21.15.7:0/3773589403 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c36d9d700 1 --2- 172.21.15.7:0/3773589403 >> v2:172.21.15.117:3300/0 conn(0x7f0c380c9f40 0x7f0c380c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097110642 0 0) 0x7f0c380c5a80 con 0x7f0c38153880 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c200035b0 con 0x7f0c38153880 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 972207962 0 0) 0x7f0c200035b0 con 0x7f0c38153880 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 >> v2:172.21.15.117:3300/0 conn(0x7f0c380c9f40 msgr2=0x7f0c380c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 --2- 172.21.15.7:0/3773589403 >> v2:172.21.15.117:3300/0 conn(0x7f0c380c9f40 0x7f0c380c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 >> v1:172.21.15.117:6789/0 conn(0x7f0c3815b4a0 legacy=0x7f0c380c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c381596e0 con 0x7f0c38153880 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.080+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f0c28001670 con 0x7f0c38153880 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 >> v1:172.21.15.7:6789/0 conn(0x7f0c38153880 legacy=0x7f0c3815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20003a00 0x7f0c20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 0x7f0c20004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c20005200 0x7f0c2000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c380c5f10 con 0x7f0c20004850 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2000c110 con 0x7f0c20003a00 2023-06-09T11:33:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2000c2d0 con 0x7f0c20005200 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 0x7f0c20004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c3659c700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20003a00 0x7f0c20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c36d9d700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c20005200 0x7f0c2000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c20005200 msgr2=0x7f0c2000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c20005200 0x7f0c2000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20003a00 msgr2=0x7f0c20003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20003a00 0x7f0c20003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3773589403 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200035b0 con 0x7f0c20004850 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.081+0000 7f0c36d9d700 1 -- 172.21.15.7:0/3773589403 reap_dead start 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 0x7f0c20004cc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c001620 tx=0x7f0c20005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4db5738652fbcf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0052d0 con 0x7f0c20004850 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c005470 con 0x7f0c20004850 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3559a700 1 -- 172.21.15.7:0/3773589403 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0038c0 con 0x7f0c20004850 2023-06-09T11:33:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 msgr2=0x7f0c20004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 0x7f0c20004cc0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c001620 tx=0x7f0c20005880 comp rx=0 tx=0).stop 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3773589403 shutdown_connections 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3773589403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c20004850 0x7f0c20004cc0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3773589403 >> 172.21.15.7:0/3773589403 conn(0x7f0c380737f0 msgr2=0x7f0c380720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.082+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3773589403 shutdown_connections 2023-06-09T11:33:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.083+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3773589403 wait complete. 2023-06-09T11:33:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.084+0000 7f0c3cfff700 1 Processor -- start 2023-06-09T11:33:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.084+0000 7f0c3cfff700 1 -- start start 2023-06-09T11:33:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 0x7f0c380ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c3814b790 0x7f0c381ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c38151210 con 0x7f0c380b38e0 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381eebb0 con 0x7f0c20004850 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.085+0000 7f0c3cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381f3070 con 0x7f0c3814b790 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38772/0 (socket says 172.21.15.7:38772) 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3475303331 learned_addr learned my addr 172.21.15.7:0/3475303331 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:19.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c3659c700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 0x7f0c380ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c36d9d700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c3814b790 0x7f0c381ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c3814b790 msgr2=0x7f0c381ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c3814b790 0x7f0c381ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 msgr2=0x7f0c380ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 0x7f0c380ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 -- 172.21.15.7:0/3475303331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c38159c60 con 0x7f0c380b38e0 2023-06-09T11:33:19.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c3659c700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 0x7f0c380ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.086+0000 7f0c35d9b700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004000 tx=0x7f0c2c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.087+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c007240 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.087+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c0073c0 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.087+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c38159980 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.087+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c007770 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.087+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c38159f40 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.088+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c381ef130 con 0x7f0c380b38e0 2023-06-09T11:33:19.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.089+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c2c003680 con 0x7f0c380b38e0 2023-06-09T11:33:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.090+0000 7f0c26ffd700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 0x7f0c1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.090+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f0c2c08acd0 con 0x7f0c380b38e0 2023-06-09T11:33:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.090+0000 7f0c3659c700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 0x7f0c1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:19.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.091+0000 7f0c3659c700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 0x7f0c1806d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0c28001200 tx=0x7f0c28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:19.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.095+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0c2c055950 con 0x7f0c380b38e0 2023-06-09T11:33:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.346+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c3815a500 con 0x7f0c380b38e0 2023-06-09T11:33:19.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.346+0000 7f0c26ffd700 1 -- 172.21.15.7:0/3475303331 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f0c3815a500 con 0x7f0c380b38e0 2023-06-09T11:33:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:19.352 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:18.639647+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 msgr2=0x7f0c1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 0x7f0c1806d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0c28001200 tx=0x7f0c28009040 comp rx=0 tx=0).stop 2023-06-09T11:33:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 msgr2=0x7f0c3814b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c004000 tx=0x7f0c2c001120 comp rx=0 tx=0).stop 2023-06-09T11:33:19.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 shutdown_connections 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0c1806ace0 0x7f0c1806d190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c3814b790 0x7f0c381ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c20004850 0x7f0c380ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 --2- 172.21.15.7:0/3475303331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c380b38e0 0x7f0c3814b250 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 >> 172.21.15.7:0/3475303331 conn(0x7f0c380737f0 msgr2=0x7f0c380cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 shutdown_connections 2023-06-09T11:33:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:19.349+0000 7f0c3cfff700 1 -- 172.21.15.7:0/3475303331 wait complete. 2023-06-09T11:33:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 96 2023-06-09T11:33:19.373 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 96, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 96, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5630": { "addr": "172.21.15.117:6835/3162263716", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3162263716, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3162263716, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5630, "incarnation": 94, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:reconnect", "state_seq": 33 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:33:18.639647+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5630, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:33:21.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:21.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.579+0000 7f562edff700 1 Processor -- start 2023-06-09T11:33:21.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.579+0000 7f562edff700 1 -- start start 2023-06-09T11:33:21.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562edff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f562815b4a0 0x7f562815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562edff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56280c9f40 0x7f56280c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562edff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562815a500 con 0x7f5628153880 2023-06-09T11:33:21.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562edff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56280c5f10 con 0x7f562815b4a0 2023-06-09T11:33:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562edff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562815da00 con 0x7f56280c9f40 2023-06-09T11:33:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f5627fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f562815b4a0 0x7f562815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562cb9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5628153880 0x7f562815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39606/0 (socket says 172.21.15.7:39606) 2023-06-09T11:33:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562cb9b700 1 -- 172.21.15.7:0/2460595027 learned_addr learned my addr 172.21.15.7:0/2460595027 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:21.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f562d39c700 1 --2- 172.21.15.7:0/2460595027 >> v2:172.21.15.117:3300/0 conn(0x7f56280c9f40 0x7f56280c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f5627fff700 1 -- 172.21.15.7:0/2460595027 >> v2:172.21.15.117:3300/0 conn(0x7f56280c9f40 msgr2=0x7f56280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f5627fff700 1 --2- 172.21.15.7:0/2460595027 >> v2:172.21.15.117:3300/0 conn(0x7f56280c9f40 0x7f56280c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f5627fff700 1 -- 172.21.15.7:0/2460595027 >> v1:172.21.15.7:6789/0 conn(0x7f5628153880 legacy=0x7f562815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f5627fff700 1 -- 172.21.15.7:0/2460595027 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56281596e0 con 0x7f562815b4a0 2023-06-09T11:33:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.581+0000 7f56277fe700 1 -- 172.21.15.7:0/2460595027 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423009959 0 0) 0x7f561c001110 con 0x7f5628153880 2023-06-09T11:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f5627fff700 1 --2- 172.21.15.7:0/2460595027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562815b4a0 0x7f562815d4c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56280749f0 tx=0x7f5614009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a012eec9f5b33d6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f56277fe700 1 -- 172.21.15.7:0/2460595027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614011830 con 0x7f562815b4a0 2023-06-09T11:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f56277fe700 1 -- 172.21.15.7:0/2460595027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614011990 con 0x7f562815b4a0 2023-06-09T11:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f56277fe700 1 -- 172.21.15.7:0/2460595027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614011d20 con 0x7f562815b4a0 2023-06-09T11:33:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f562edff700 1 -- 172.21.15.7:0/2460595027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562815b4a0 msgr2=0x7f562815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.582+0000 7f562edff700 1 --2- 172.21.15.7:0/2460595027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562815b4a0 0x7f562815d4c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f56280749f0 tx=0x7f5614009b20 comp rx=0 tx=0).stop 2023-06-09T11:33:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 -- 172.21.15.7:0/2460595027 shutdown_connections 2023-06-09T11:33:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 --2- 172.21.15.7:0/2460595027 >> v2:172.21.15.117:3300/0 conn(0x7f56280c9f40 0x7f56280c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 --2- 172.21.15.7:0/2460595027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562815b4a0 0x7f562815d4c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 -- 172.21.15.7:0/2460595027 >> 172.21.15.7:0/2460595027 conn(0x7f56280737f0 msgr2=0x7f56280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 -- 172.21.15.7:0/2460595027 shutdown_connections 2023-06-09T11:33:21.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.583+0000 7f562edff700 1 -- 172.21.15.7:0/2460595027 wait complete. 2023-06-09T11:33:21.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.584+0000 7f562edff700 1 Processor -- start 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 -- start start 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 0x7f56280ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562815b4a0 0x7f56281f27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56280b4a80 con 0x7f5628153880 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5628151770 con 0x7f562815b4a0 2023-06-09T11:33:21.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.585+0000 7f562edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56281f2d30 con 0x7f56280c9f40 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38784/0 (socket says 172.21.15.7:38784) 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 -- 172.21.15.7:0/1838617391 learned_addr learned my addr 172.21.15.7:0/1838617391 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f562cb9b700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 0x7f56280ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f562d39c700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562815b4a0 0x7f56281f27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 -- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 msgr2=0x7f56280ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 0x7f56280ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 -- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562815b4a0 msgr2=0x7f56281f27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562815b4a0 0x7f56281f27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 -- 172.21.15.7:0/1838617391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140094a0 con 0x7f5628153880 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f562cb9b700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 0x7f56280ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.586+0000 7f5627fff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f56140090b0 tx=0x7f5614004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.587+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140036a0 con 0x7f5628153880 2023-06-09T11:33:21.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.587+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561401e070 con 0x7f5628153880 2023-06-09T11:33:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.587+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5628159c60 con 0x7f5628153880 2023-06-09T11:33:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.587+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614003a20 con 0x7f5628153880 2023-06-09T11:33:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.587+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5628159980 con 0x7f5628153880 2023-06-09T11:33:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.588+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5628159f40 con 0x7f5628153880 2023-06-09T11:33:21.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.589+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5614003b80 con 0x7f5628153880 2023-06-09T11:33:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.590+0000 7f56257fa700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 0x7f560c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.590+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f561408aa10 con 0x7f5628153880 2023-06-09T11:33:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.590+0000 7f562cb9b700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 0x7f560c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:21.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.591+0000 7f562cb9b700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 0x7f560c06d1c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f561c000f60 tx=0x7f561c007ae0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:21.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.594+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5614055690 con 0x7f5628153880 2023-06-09T11:33:21.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.849+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5628074cf0 con 0x7f5628153880 2023-06-09T11:33:21.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.850+0000 7f56257fa700 1 -- 172.21.15.7:0/1838617391 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f5614058cb0 con 0x7f5628153880 2023-06-09T11:33:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:21.855 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:18.639647+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:21.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 msgr2=0x7f560c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 0x7f560c06d1c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f561c000f60 tx=0x7f561c007ae0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 msgr2=0x7f5628151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f56140090b0 tx=0x7f5614004750 comp rx=0 tx=0).stop 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 shutdown_connections 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562815b4a0 0x7f56281f27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f560c06ad10 0x7f560c06d1c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f56280c9f40 0x7f56280ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 --2- 172.21.15.7:0/1838617391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5628153880 0x7f5628151230 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:21.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 >> 172.21.15.7:0/1838617391 conn(0x7f56280737f0 msgr2=0x7f56280cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:21.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 shutdown_connections 2023-06-09T11:33:21.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:21.852+0000 7f562edff700 1 -- 172.21.15.7:0/1838617391 wait complete. 2023-06-09T11:33:21.859 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 96 2023-06-09T11:33:21.875 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:23.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.082+0000 7f7e6f5ff700 1 Processor -- start 2023-06-09T11:33:24.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.082+0000 7f7e6f5ff700 1 -- start start 2023-06-09T11:33:24.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.083+0000 7f7e6f5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e680b5360 0x7f7e6815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.083+0000 7f7e6f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e680c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6f5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6815dcf0 con 0x7f7e680ca1b0 2023-06-09T11:33:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6f5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680c5e00 con 0x7f7e680b5360 2023-06-09T11:33:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815d450 con 0x7f7e680ca580 2023-06-09T11:33:24.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e680b5360 0x7f7e6815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e680b5360 0x7f7e6815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40004/0 (socket says 172.21.15.7:40004) 2023-06-09T11:33:24.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/3504968176 learned_addr learned my addr 172.21.15.7:0/3504968176 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:24.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6db9c700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e680c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/3504968176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 msgr2=0x7f7e680c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:24.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e680c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/3504968176 >> v1:172.21.15.117:6789/0 conn(0x7f7e680ca1b0 legacy=0x7f7e681536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/3504968176 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6815d730 con 0x7f7e680b5360 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.084+0000 7f7e6cb9a700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680b5360 0x7f7e6815cdc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7e580098f0 tx=0x7f7e58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2558ef754e7c4c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e5ffff700 1 -- 172.21.15.7:0/3504968176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800b8a0 con 0x7f7e680b5360 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e5ffff700 1 -- 172.21.15.7:0/3504968176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5800ba00 con 0x7f7e680b5360 2023-06-09T11:33:24.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e5ffff700 1 -- 172.21.15.7:0/3504968176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5800bd90 con 0x7f7e680b5360 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/3504968176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680b5360 msgr2=0x7f7e6815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680b5360 0x7f7e6815cdc0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f7e580098f0 tx=0x7f7e58009c00 comp rx=0 tx=0).stop 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/3504968176 shutdown_connections 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e680c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/3504968176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680b5360 0x7f7e6815cdc0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/3504968176 >> 172.21.15.7:0/3504968176 conn(0x7f7e68072090 msgr2=0x7f7e68072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:24.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.085+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/3504968176 shutdown_connections 2023-06-09T11:33:24.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.086+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/3504968176 wait complete. 2023-06-09T11:33:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.088+0000 7f7e6f5ff700 1 Processor -- start 2023-06-09T11:33:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.088+0000 7f7e6f5ff700 1 -- start start 2023-06-09T11:33:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e680b5360 0x7f7e680c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e6815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f3440 con 0x7f7e680ca1b0 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681f38d0 con 0x7f7e680b5360 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.089+0000 7f7e6f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815b9b0 con 0x7f7e680ca580 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6d39b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e680b5360 0x7f7e680c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40008/0 (socket says 172.21.15.7:40008) 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/1579565005 learned_addr learned my addr 172.21.15.7:0/1579565005 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:24.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6db9c700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e6815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 msgr2=0x7f7e6815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e6815b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e680b5360 msgr2=0x7f7e680c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e680b5360 0x7f7e680c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 -- 172.21.15.7:0/1579565005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009550 con 0x7f7e680ca1b0 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e6cb9a700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009c40 tx=0x7f7e58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:24.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.090+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58004300 con 0x7f7e680ca1b0 2023-06-09T11:33:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.091+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5801f070 con 0x7f7e680ca1b0 2023-06-09T11:33:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.091+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580036a0 con 0x7f7e680ca1b0 2023-06-09T11:33:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.091+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6806a570 con 0x7f7e680ca1b0 2023-06-09T11:33:24.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.091+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6815da10 con 0x7f7e680ca1b0 2023-06-09T11:33:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.092+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6806a850 con 0x7f7e680ca1b0 2023-06-09T11:33:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.093+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e58021030 con 0x7f7e680ca1b0 2023-06-09T11:33:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.094+0000 7f7e5dffb700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 0x7f7e4c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:24.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.094+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f7e5808acd0 con 0x7f7e680ca1b0 2023-06-09T11:33:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.094+0000 7f7e6d39b700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 0x7f7e4c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:24.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.094+0000 7f7e6d39b700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 0x7f7e4c06d1c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007fd0 tx=0x7f7e64009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:24.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.098+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7e58055a70 con 0x7f7e680ca1b0 2023-06-09T11:33:24.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.348+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e6815d030 con 0x7f7e680ca1b0 2023-06-09T11:33:24.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.349+0000 7f7e5dffb700 1 -- 172.21.15.7:0/1579565005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+5459 (secure 0 0 0) 0x7f7e58059090 con 0x7f7e680ca1b0 2023-06-09T11:33:24.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:24.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":96,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:18.639647+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 msgr2=0x7f7e4c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 0x7f7e4c06d1c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007fd0 tx=0x7f7e64009040 comp rx=0 tx=0).stop 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 msgr2=0x7f7e6815af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009c40 tx=0x7f7e58004750 comp rx=0 tx=0).stop 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 shutdown_connections 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e4c06ad10 0x7f7e4c06d1c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e680ca580 0x7f7e6815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e680b5360 0x7f7e680c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 --2- 172.21.15.7:0/1579565005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e680ca1b0 0x7f7e6815af30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 >> 172.21.15.7:0/1579565005 conn(0x7f7e68072090 msgr2=0x7f7e6814f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 shutdown_connections 2023-06-09T11:33:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:24.351+0000 7f7e6f5ff700 1 -- 172.21.15.7:0/1579565005 wait complete. 2023-06-09T11:33:24.359 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 96 2023-06-09T11:33:24.376 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:26.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.582+0000 7fbcb6dff700 1 Processor -- start 2023-06-09T11:33:26.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.582+0000 7fbcb6dff700 1 -- start start 2023-06-09T11:33:26.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcb6dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbcb0153880 0x7fbcb015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcb6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbcb00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcb6dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb015a500 con 0x7fbcb00c9f40 2023-06-09T11:33:26.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcb6dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb00c5f10 con 0x7fbcb0153880 2023-06-09T11:33:26.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcb6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb015da00 con 0x7fbcb015b4a0 2023-06-09T11:33:26.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcaffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbcb00c9f40 0x7fbcb015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37778/0 (socket says 172.21.15.7:37778) 2023-06-09T11:33:26.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.583+0000 7fbcaffff700 1 -- 172.21.15.7:0/1664254750 learned_addr learned my addr 172.21.15.7:0/1664254750 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcb539c700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbcb00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcb4b9b700 1 --2- 172.21.15.7:0/1664254750 >> v2:172.21.15.117:3300/0 conn(0x7fbcb0153880 0x7fbcb015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547897949 0 0) 0x7fbc9c001110 con 0x7fbcb00c9f40 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca00035b0 con 0x7fbcb00c9f40 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3481841711 0 0) 0x7fbca00035b0 con 0x7fbcb00c9f40 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 msgr2=0x7fbcb00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbcb00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 >> v2:172.21.15.117:3300/0 conn(0x7fbcb0153880 msgr2=0x7fbcb015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:26.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 --2- 172.21.15.7:0/1664254750 >> v2:172.21.15.117:3300/0 conn(0x7fbcb0153880 0x7fbcb015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb01596e0 con 0x7fbcb00c9f40 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fbc9c001800 con 0x7fbcb00c9f40 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 >> v1:172.21.15.7:6789/0 conn(0x7fbcb00c9f40 legacy=0x7fbcb015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca0003a00 0x7fbca0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbca0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.584+0000 7fbcaf7fe700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00c5f10 con 0x7fbca0004d90 2023-06-09T11:33:26.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb015da00 con 0x7fbca0003a00 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca000ba40 con 0x7fbcb015b4a0 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb4b9b700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca0003a00 0x7fbca0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcaffff700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbca0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 -- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 msgr2=0x7fbca0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbcb015b4a0 0x7fbca0004850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 -- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca0003a00 msgr2=0x7fbca0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:26.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbca0003a00 0x7fbca0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 -- 172.21.15.7:0/1664254750 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca00035b0 con 0x7fbca0004d90 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.585+0000 7fbcb539c700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009f90 tx=0x7fbca0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dd230a02dc97d9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca800eb70 con 0x7fbca0004d90 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800f040 con 0x7fbca0004d90 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcaf7fe700 1 -- 172.21.15.7:0/1664254750 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8014480 con 0x7fbca0004d90 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 -- 172.21.15.7:0/1664254750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca0004d90 msgr2=0x7fbca0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/1664254750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbca0004d90 0x7fbca0005200 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009f90 tx=0x7fbca0005880 comp rx=0 tx=0).stop 2023-06-09T11:33:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb539c700 1 -- 172.21.15.7:0/1664254750 reap_dead start 2023-06-09T11:33:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 -- 172.21.15.7:0/1664254750 shutdown_connections 2023-06-09T11:33:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 -- 172.21.15.7:0/1664254750 >> 172.21.15.7:0/1664254750 conn(0x7fbcb00737f0 msgr2=0x7fbcb00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 -- 172.21.15.7:0/1664254750 shutdown_connections 2023-06-09T11:33:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.586+0000 7fbcb6dff700 1 -- 172.21.15.7:0/1664254750 wait complete. 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.588+0000 7fbcb6dff700 1 Processor -- start 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.588+0000 7fbcb6dff700 1 -- start start 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbca0004d90 0x7fbcb00cb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcb00cb910 0x7fbcb014cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb0158460 con 0x7fbcb014d510 2023-06-09T11:33:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00b4c90 con 0x7fbcb00cb910 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00cc4d0 con 0x7fbca0004d90 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.589+0000 7fbcb539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40014/0 (socket says 172.21.15.7:40014) 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 -- 172.21.15.7:0/117537247 learned_addr learned my addr 172.21.15.7:0/117537247 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcaffff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcb00cb910 0x7fbcb014cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb4b9b700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbca0004d90 0x7fbcb00cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 -- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbca0004d90 msgr2=0x7fbcb00cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbca0004d90 0x7fbcb00cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 -- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcb00cb910 msgr2=0x7fbcb014cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcb00cb910 0x7fbcb014cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 -- 172.21.15.7:0/117537247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0159c60 con 0x7fbcb014d510 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcb539c700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbca0004770 tx=0x7fbca8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8007000 con 0x7fbcb014d510 2023-06-09T11:33:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.590+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca8003a20 con 0x7fbcb014d510 2023-06-09T11:33:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.591+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb0159980 con 0x7fbcb014d510 2023-06-09T11:33:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.591+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8003db0 con 0x7fbcb014d510 2023-06-09T11:33:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.591+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb0159f40 con 0x7fbcb014d510 2023-06-09T11:33:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.592+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb00b4e40 con 0x7fbcb014d510 2023-06-09T11:33:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.593+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbca8023030 con 0x7fbcb014d510 2023-06-09T11:33:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.594+0000 7fbcad7fa700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 0x7fbc9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.594+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fbca808aac0 con 0x7fbcb014d510 2023-06-09T11:33:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.594+0000 7fbcb4b9b700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 0x7fbc9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.595+0000 7fbcb4b9b700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 0x7fbc9406d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbca40045f0 tx=0x7fbca4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:26.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.598+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbca80591d0 con 0x7fbcb014d510 2023-06-09T11:33:26.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.851+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbcb0074cf0 con 0x7fbcb014d510 2023-06-09T11:33:26.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.852+0000 7fbcad7fa700 1 -- 172.21.15.7:0/117537247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fbca8058d60 con 0x7fbcb014d510 2023-06-09T11:33:26.858 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:26.858 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:24.660145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 msgr2=0x7fbc9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 0x7fbc9406d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbca40045f0 tx=0x7fbca4008040 comp rx=0 tx=0).stop 2023-06-09T11:33:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 msgr2=0x7fbcb0157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fbca0004770 tx=0x7fbca8009750 comp rx=0 tx=0).stop 2023-06-09T11:33:26.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 shutdown_connections 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbcb014d510 0x7fbcb0157350 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc9406ace0 0x7fbc9406d190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbca0004d90 0x7fbcb00cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 --2- 172.21.15.7:0/117537247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbcb00cb910 0x7fbcb014cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 >> 172.21.15.7:0/117537247 conn(0x7fbcb00737f0 msgr2=0x7fbcb015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 shutdown_connections 2023-06-09T11:33:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:26.854+0000 7fbcb6dff700 1 -- 172.21.15.7:0/117537247 wait complete. 2023-06-09T11:33:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 97 2023-06-09T11:33:26.878 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:28.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:29.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.083+0000 7f0e7e9ff700 1 Processor -- start 2023-06-09T11:33:29.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.083+0000 7f0e7e9ff700 1 -- start start 2023-06-09T11:33:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e780ca1b0 0x7f0e7815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0e780b5360 0x7f0e780c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e780c5e00 con 0x7f0e780ca1b0 2023-06-09T11:33:29.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7815dcf0 con 0x7f0e780b5360 2023-06-09T11:33:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7815d450 con 0x7f0e780ca580 2023-06-09T11:33:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e780ca1b0 0x7f0e7815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0e780ca1b0 0x7f0e7815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40018/0 (socket says 172.21.15.7:40018) 2023-06-09T11:33:29.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 -- 172.21.15.7:0/4278907730 learned_addr learned my addr 172.21.15.7:0/4278907730 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e7cf9c700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e777fe700 1 --2- 172.21.15.7:0/4278907730 >> v2:172.21.15.117:3300/0 conn(0x7f0e780b5360 0x7f0e780c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 -- 172.21.15.7:0/4278907730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 msgr2=0x7f0e781536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 -- 172.21.15.7:0/4278907730 >> v2:172.21.15.117:3300/0 conn(0x7f0e780b5360 msgr2=0x7f0e780c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.085+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4278907730 >> v2:172.21.15.117:3300/0 conn(0x7f0e780b5360 0x7f0e780c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.086+0000 7f0e77fff700 1 -- 172.21.15.7:0/4278907730 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7815d730 con 0x7f0e780ca1b0 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.086+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815cdc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0e78157440 tx=0x7f0e68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e22bd287d11c2f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.086+0000 7f0e76ffd700 1 -- 172.21.15.7:0/4278907730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6800b870 con 0x7f0e780ca1b0 2023-06-09T11:33:29.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.086+0000 7f0e76ffd700 1 -- 172.21.15.7:0/4278907730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6800b9d0 con 0x7f0e780ca1b0 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.086+0000 7f0e76ffd700 1 -- 172.21.15.7:0/4278907730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6800bd60 con 0x7f0e780ca1b0 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4278907730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 msgr2=0x7f0e7815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815cdc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0e78157440 tx=0x7f0e68009bd0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4278907730 shutdown_connections 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4278907730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815cdc0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4278907730 >> v2:172.21.15.117:3300/0 conn(0x7f0e780b5360 0x7f0e780c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4278907730 >> 172.21.15.7:0/4278907730 conn(0x7f0e78072090 msgr2=0x7f0e78072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4278907730 shutdown_connections 2023-06-09T11:33:29.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.087+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4278907730 wait complete. 2023-06-09T11:33:29.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.089+0000 7f0e7e9ff700 1 Processor -- start 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.089+0000 7f0e7e9ff700 1 -- start start 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e780b5360 0x7f0e78157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780cdc40 con 0x7f0e780ca1b0 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7815cdd0 con 0x7f0e780b5360 2023-06-09T11:33:29.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78158170 con 0x7f0e780ca580 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40028/0 (socket says 172.21.15.7:40028) 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 -- 172.21.15.7:0/4181251503 learned_addr learned my addr 172.21.15.7:0/4181251503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e780b5360 0x7f0e78157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e7cf9c700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 -- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 msgr2=0x7f0e781587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 -- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e780b5360 msgr2=0x7f0e78157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e780b5360 0x7f0e78157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.090+0000 7f0e777fe700 1 -- 172.21.15.7:0/4181251503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68009550 con 0x7f0e780ca1b0 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e777fe700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e810 tx=0x7f0e6c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00c070 con 0x7f0e780ca1b0 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c013610 con 0x7f0e780ca1b0 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0139a0 con 0x7f0e780ca1b0 2023-06-09T11:33:29.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7806a570 con 0x7f0e780ca1b0 2023-06-09T11:33:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.091+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7815da10 con 0x7f0e780ca1b0 2023-06-09T11:33:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.092+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7806a850 con 0x7f0e780ca1b0 2023-06-09T11:33:29.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.093+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e6c013b00 con 0x7f0e780ca1b0 2023-06-09T11:33:29.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.094+0000 7f0e74ff9700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 0x7f0e6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:29.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.094+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f0e6c089610 con 0x7f0e780ca1b0 2023-06-09T11:33:29.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.094+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 0x7f0e6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:29.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.095+0000 7f0e77fff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 0x7f0e6406d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:29.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.098+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0e6c054200 con 0x7f0e780ca1b0 2023-06-09T11:33:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.352+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e780c4d50 con 0x7f0e780ca1b0 2023-06-09T11:33:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.352+0000 7f0e74ff9700 1 -- 172.21.15.7:0/4181251503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+5456 (secure 0 0 0) 0x7f0e6c057820 con 0x7f0e780ca1b0 2023-06-09T11:33:29.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:29.359 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":97,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:24.660145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 msgr2=0x7f0e6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 0x7f0e6406d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e68000c00 tx=0x7f0e68008030 comp rx=0 tx=0).stop 2023-06-09T11:33:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 msgr2=0x7f0e7815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00e810 tx=0x7f0e6c00eb20 comp rx=0 tx=0).stop 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 shutdown_connections 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0e6406ace0 0x7f0e6406d190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0e780ca580 0x7f0e781587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0e780b5360 0x7f0e78157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 --2- 172.21.15.7:0/4181251503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e780ca1b0 0x7f0e7815c890 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 >> 172.21.15.7:0/4181251503 conn(0x7f0e78072090 msgr2=0x7f0e7814e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:29.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 shutdown_connections 2023-06-09T11:33:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:29.355+0000 7f0e7e9ff700 1 -- 172.21.15.7:0/4181251503 wait complete. 2023-06-09T11:33:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 97 2023-06-09T11:33:29.379 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:31.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:31.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.584+0000 7f7f079ff700 1 Processor -- start 2023-06-09T11:33:31.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.584+0000 7f7f079ff700 1 -- start start 2023-06-09T11:33:31.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.585+0000 7f7f079ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7f000c1bc0 0x7f7f001599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.585+0000 7f7f079ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f000c0bf0 con 0x7f7f000c1f90 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.585+0000 7f7f079ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f000702f0 con 0x7f7f000c0140 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.585+0000 7f7f079ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f00070110 con 0x7f7f000c1bc0 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f05f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f000c1f90 0x7f7f00160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37782/0 (socket says 172.21.15.7:37782) 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f05f9c700 1 -- 172.21.15.7:0/546185133 learned_addr learned my addr 172.21.15.7:0/546185133 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:31.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f0579b700 1 --2- 172.21.15.7:0/546185133 >> v2:172.21.15.117:3300/0 conn(0x7f7f000c1bc0 0x7f7f001599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7ef7fff700 1 -- 172.21.15.7:0/546185133 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129410419 0 0) 0x7f7f000c0bf0 con 0x7f7f000c1f90 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7ef7fff700 1 -- 172.21.15.7:0/546185133 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eec0035b0 con 0x7f7f000c1f90 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f0579b700 1 -- 172.21.15.7:0/546185133 >> v1:172.21.15.117:6789/0 conn(0x7f7f000c0140 legacy=0x7f7f001501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f0579b700 1 -- 172.21.15.7:0/546185133 >> v1:172.21.15.7:6789/0 conn(0x7f7f000c1f90 legacy=0x7f7f00160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.586+0000 7f7f0579b700 1 -- 172.21.15.7:0/546185133 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f000c0e90 con 0x7f7f000c1bc0 2023-06-09T11:33:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.587+0000 7f7f0579b700 1 --2- 172.21.15.7:0/546185133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001599b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009840 tx=0x7f7efc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a169fdb1e59f18d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.587+0000 7f7ef7fff700 1 -- 172.21.15.7:0/546185133 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc011830 con 0x7f7f000c1bc0 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.587+0000 7f7ef7fff700 1 -- 172.21.15.7:0/546185133 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc011990 con 0x7f7f000c1bc0 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.587+0000 7f7ef7fff700 1 -- 172.21.15.7:0/546185133 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc011d20 con 0x7f7f000c1bc0 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 -- 172.21.15.7:0/546185133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 msgr2=0x7f7f001599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 --2- 172.21.15.7:0/546185133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001599b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009840 tx=0x7f7efc009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 -- 172.21.15.7:0/546185133 shutdown_connections 2023-06-09T11:33:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 --2- 172.21.15.7:0/546185133 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001599b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 -- 172.21.15.7:0/546185133 >> 172.21.15.7:0/546185133 conn(0x7f7f0006a4d0 msgr2=0x7f7f0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 -- 172.21.15.7:0/546185133 shutdown_connections 2023-06-09T11:33:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.588+0000 7f7f079ff700 1 -- 172.21.15.7:0/546185133 wait complete. 2023-06-09T11:33:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.590+0000 7f7f079ff700 1 Processor -- start 2023-06-09T11:33:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.590+0000 7f7f079ff700 1 -- start start 2023-06-09T11:33:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 0x7f7f001e08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00153770 con 0x7f7f000c0140 2023-06-09T11:33:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f001f00e0 con 0x7f7f000c1bc0 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f079ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00152e10 con 0x7f7f000c1f90 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40032/0 (socket says 172.21.15.7:40032) 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 -- 172.21.15.7:0/2354363315 learned_addr learned my addr 172.21.15.7:0/2354363315 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f04f9a700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f05f9c700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 0x7f7f001e08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 -- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 msgr2=0x7f7f001e08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 0x7f7f001e08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 -- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 msgr2=0x7f7f001efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f0579b700 1 -- 172.21.15.7:0/2354363315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0094a0 con 0x7f7f000c0140 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f04f9a700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.592+0000 7f7f05f9c700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 0x7f7f001e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7f0579b700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009fd0 tx=0x7f7efc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0038a0 con 0x7f7f000c0140 2023-06-09T11:33:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc01f070 con 0x7f7f000c0140 2023-06-09T11:33:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f000c1450 con 0x7f7f000c0140 2023-06-09T11:33:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc003c40 con 0x7f7f000c0140 2023-06-09T11:33:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.593+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f000c1170 con 0x7f7f000c0140 2023-06-09T11:33:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.594+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f000705d0 con 0x7f7f000c0140 2023-06-09T11:33:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.595+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7efc026480 con 0x7f7f000c0140 2023-06-09T11:33:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.596+0000 7f7ef5ffb700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.596+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f7efc021070 con 0x7f7f000c0140 2023-06-09T11:33:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.596+0000 7f7f04f9a700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.597+0000 7f7f04f9a700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 0x7f7ee406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f00155660 tx=0x7f7ef0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.601+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7efc055e10 con 0x7f7f000c0140 2023-06-09T11:33:31.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.855+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f00074cf0 con 0x7f7f000c0140 2023-06-09T11:33:31.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.856+0000 7f7ef5ffb700 1 -- 172.21.15.7:0/2354363315 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f7efc059430 con 0x7f7f000c0140 2023-06-09T11:33:31.862 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:31.862 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:30.675453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:31.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 msgr2=0x7f7ee406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 0x7f7ee406d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f00155660 tx=0x7f7ef0009040 comp rx=0 tx=0).stop 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 msgr2=0x7f7f00150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009fd0 tx=0x7f7efc004750 comp rx=0 tx=0).stop 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 shutdown_connections 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ee406ace0 0x7f7ee406d190 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7f000c1f90 0x7f7f001e08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f000c0140 0x7f7f00150db0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 --2- 172.21.15.7:0/2354363315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7f000c1bc0 0x7f7f001efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 >> 172.21.15.7:0/2354363315 conn(0x7f7f0006a4d0 msgr2=0x7f7f00154180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 shutdown_connections 2023-06-09T11:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:31.858+0000 7f7f079ff700 1 -- 172.21.15.7:0/2354363315 wait complete. 2023-06-09T11:33:31.866 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 98 2023-06-09T11:33:31.883 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 98, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 98, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5630": { "addr": "172.21.15.117:6835/3162263716", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3162263716, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3162263716, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5630, "incarnation": 94, "join_fscid": -1, "name": "d", "rank": 0, "state": "up:clientreplay", "state_seq": 37 }, "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 61, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:33:30.675453+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5630, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:33:33.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:34.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.089+0000 7f70b89ff700 1 Processor -- start 2023-06-09T11:33:34.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.089+0000 7f70b89ff700 1 -- start start 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.090+0000 7f70b89ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70b415b4a0 0x7f70b415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.090+0000 7f70b89ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f70b40c9f40 0x7f70b40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.090+0000 7f70b89ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b415a500 con 0x7f70b4153880 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.090+0000 7f70b89ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40c5f10 con 0x7f70b415b4a0 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.090+0000 7f70b89ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b415da00 con 0x7f70b40c9f40 2023-06-09T11:33:34.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70b415b4a0 0x7f70b415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f70b415b4a0 0x7f70b415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58206/0 (socket says 172.21.15.7:58206) 2023-06-09T11:33:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 -- 172.21.15.7:0/3705290579 learned_addr learned my addr 172.21.15.7:0/3705290579 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:34.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b2d9d700 1 --2- 172.21.15.7:0/3705290579 >> v2:172.21.15.117:3300/0 conn(0x7f70b40c9f40 0x7f70b40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 -- 172.21.15.7:0/3705290579 >> v2:172.21.15.117:3300/0 conn(0x7f70b40c9f40 msgr2=0x7f70b40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:34.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/3705290579 >> v2:172.21.15.117:3300/0 conn(0x7f70b40c9f40 0x7f70b40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 -- 172.21.15.7:0/3705290579 >> v1:172.21.15.117:6789/0 conn(0x7f70b4153880 legacy=0x7f70b415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.091+0000 7f70b1d9b700 1 -- 172.21.15.7:0/3705290579 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b41596e0 con 0x7f70b415b4a0 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/3705290579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b415b4a0 0x7f70b415d4c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f70b40749f0 tx=0x7f70a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af01627ecbd0366a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b159a700 1 -- 172.21.15.7:0/3705290579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011830 con 0x7f70b415b4a0 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b159a700 1 -- 172.21.15.7:0/3705290579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a8011990 con 0x7f70b415b4a0 2023-06-09T11:33:34.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b159a700 1 -- 172.21.15.7:0/3705290579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011d20 con 0x7f70b415b4a0 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 -- 172.21.15.7:0/3705290579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b415b4a0 msgr2=0x7f70b415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 --2- 172.21.15.7:0/3705290579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b415b4a0 0x7f70b415d4c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f70b40749f0 tx=0x7f70a8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 -- 172.21.15.7:0/3705290579 shutdown_connections 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 --2- 172.21.15.7:0/3705290579 >> v2:172.21.15.117:3300/0 conn(0x7f70b40c9f40 0x7f70b40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 --2- 172.21.15.7:0/3705290579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b415b4a0 0x7f70b415d4c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.092+0000 7f70b89ff700 1 -- 172.21.15.7:0/3705290579 >> 172.21.15.7:0/3705290579 conn(0x7f70b40737f0 msgr2=0x7f70b40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:34.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.093+0000 7f70b89ff700 1 -- 172.21.15.7:0/3705290579 shutdown_connections 2023-06-09T11:33:34.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.093+0000 7f70b89ff700 1 -- 172.21.15.7:0/3705290579 wait complete. 2023-06-09T11:33:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.094+0000 7f70b89ff700 1 Processor -- start 2023-06-09T11:33:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.094+0000 7f70b89ff700 1 -- start start 2023-06-09T11:33:34.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 0x7f70b414a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b415b4a0 0x7f70b414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41594b0 con 0x7f70b40c9f40 2023-06-09T11:33:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40cea30 con 0x7f70b4153880 2023-06-09T11:33:34.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.095+0000 7f70b89ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b414b630 con 0x7f70b415b4a0 2023-06-09T11:33:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58220/0 (socket says 172.21.15.7:58220) 2023-06-09T11:33:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 -- 172.21.15.7:0/939387492 learned_addr learned my addr 172.21.15.7:0/939387492 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 0x7f70b414a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b2d9d700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b415b4a0 0x7f70b414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 -- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b415b4a0 msgr2=0x7f70b414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b415b4a0 0x7f70b414b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 -- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 msgr2=0x7f70b414a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 0x7f70b414a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 -- 172.21.15.7:0/939387492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8009550 con 0x7f70b40c9f40 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 0x7f70b414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70b259c700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c510 tx=0x7f70a400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:34.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.096+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40052d0 con 0x7f70b40c9f40 2023-06-09T11:33:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.097+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4005430 con 0x7f70b40c9f40 2023-06-09T11:33:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.097+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b4159c60 con 0x7f70b40c9f40 2023-06-09T11:33:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.097+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40038c0 con 0x7f70b40c9f40 2023-06-09T11:33:34.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.097+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b4159980 con 0x7f70b40c9f40 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.098+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4159f40 con 0x7f70b40c9f40 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.099+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70a4003a20 con 0x7f70b40c9f40 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.100+0000 7f70a2ffd700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 0x7f709406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.100+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f70a4088b30 con 0x7f70b40c9f40 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.100+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 0x7f709406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:34.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.101+0000 7f70b1d9b700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 0x7f709406d1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f70a8014040 tx=0x7f70a8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:34.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.104+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f70a4053830 con 0x7f70b40c9f40 2023-06-09T11:33:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.356+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70b40cf030 con 0x7f70b40c9f40 2023-06-09T11:33:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.357+0000 7f70a2ffd700 1 -- 172.21.15.7:0/939387492 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5462 (secure 0 0 0) 0x7f70b40cf030 con 0x7f70b40c9f40 2023-06-09T11:33:34.363 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:34.363 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:30.675453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:34.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 msgr2=0x7f709406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 0x7f709406d1c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f70a8014040 tx=0x7f70a8008030 comp rx=0 tx=0).stop 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 msgr2=0x7f70b40caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c510 tx=0x7f70a400c8d0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 shutdown_connections 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f709406ad10 0x7f709406d1c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70b415b4a0 0x7f70b414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70b40c9f40 0x7f70b40caa80 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 --2- 172.21.15.7:0/939387492 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f70b4153880 0x7f70b414a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.359+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 >> 172.21.15.7:0/939387492 conn(0x7f70b40737f0 msgr2=0x7f70b414c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.360+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 shutdown_connections 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:34.360+0000 7f70b89ff700 1 -- 172.21.15.7:0/939387492 wait complete. 2023-06-09T11:33:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 98 2023-06-09T11:33:34.384 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:36.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.661+0000 7ff998bff700 1 Processor -- start 2023-06-09T11:33:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.661+0000 7ff998bff700 1 -- start start 2023-06-09T11:33:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff998bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 0x7ff9940c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff998bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff994142850 0x7ff99415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff998bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99415dab0 con 0x7ff9940c6980 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff998bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff994159aa0 con 0x7ff9940c5b40 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff998bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99415d210 con 0x7ff994142850 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff99259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 0x7ff9940c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.663+0000 7ff992d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff994142850 0x7ff99415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff99259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 0x7ff9940c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58226/0 (socket says 172.21.15.7:58226) 2023-06-09T11:33:36.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff992d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff994142850 0x7ff99415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:34138/0 (socket says 172.21.15.7:34138) 2023-06-09T11:33:36.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff99259c700 1 -- 172.21.15.7:0/376651222 learned_addr learned my addr 172.21.15.7:0/376651222 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff991d9b700 1 -- 172.21.15.7:0/376651222 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704298650 0 0) 0x7ff99415dab0 con 0x7ff9940c6980 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff991d9b700 1 -- 172.21.15.7:0/376651222 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9800035b0 con 0x7ff9940c6980 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff992d9d700 1 -- 172.21.15.7:0/376651222 >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 msgr2=0x7ff9940c0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff992d9d700 1 --2- 172.21.15.7:0/376651222 >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 0x7ff9940c0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff992d9d700 1 -- 172.21.15.7:0/376651222 >> v1:172.21.15.117:6789/0 conn(0x7ff9940c6980 legacy=0x7ff994159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.664+0000 7ff992d9d700 1 -- 172.21.15.7:0/376651222 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99415d4f0 con 0x7ff994142850 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.665+0000 7ff992d9d700 1 --2- 172.21.15.7:0/376651222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff994142850 0x7ff99415cb60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff99406f740 tx=0x7ff98c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=689ce18004bf628d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.665+0000 7ff991d9b700 1 -- 172.21.15.7:0/376651222 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004230 con 0x7ff994142850 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.665+0000 7ff991d9b700 1 -- 172.21.15.7:0/376651222 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c004390 con 0x7ff994142850 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.665+0000 7ff991d9b700 1 -- 172.21.15.7:0/376651222 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0036a0 con 0x7ff994142850 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 -- 172.21.15.7:0/376651222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff994142850 msgr2=0x7ff99415cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 --2- 172.21.15.7:0/376651222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff994142850 0x7ff99415cb60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff99406f740 tx=0x7ff98c009b20 comp rx=0 tx=0).stop 2023-06-09T11:33:36.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 -- 172.21.15.7:0/376651222 shutdown_connections 2023-06-09T11:33:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 --2- 172.21.15.7:0/376651222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff994142850 0x7ff99415cb60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 --2- 172.21.15.7:0/376651222 >> v2:172.21.15.7:3300/0 conn(0x7ff9940c5b40 0x7ff9940c0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 -- 172.21.15.7:0/376651222 >> 172.21.15.7:0/376651222 conn(0x7ff99406a4d0 msgr2=0x7ff99406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 -- 172.21.15.7:0/376651222 shutdown_connections 2023-06-09T11:33:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.666+0000 7ff998bff700 1 -- 172.21.15.7:0/376651222 wait complete. 2023-06-09T11:33:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.668+0000 7ff998bff700 1 Processor -- start 2023-06-09T11:33:36.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.668+0000 7ff998bff700 1 -- start start 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 0x7ff99413d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 0x7ff9940cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9940c9bb0 con 0x7ff9940c6980 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99413f0f0 con 0x7ff9940c5b40 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff998bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99413eb30 con 0x7ff994142850 2023-06-09T11:33:36.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff98bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 0x7ff99413d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff99259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff99259c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:34140/0 (socket says 172.21.15.7:34140) 2023-06-09T11:33:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff99259c700 1 -- 172.21.15.7:0/427043594 learned_addr learned my addr 172.21.15.7:0/427043594 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:36.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.669+0000 7ff992d9d700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 0x7ff9940cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 -- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 msgr2=0x7ff9940cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 0x7ff9940cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 -- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 msgr2=0x7ff99413d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 0x7ff99413d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 -- 172.21.15.7:0/427043594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0094a0 con 0x7ff9940c5b40 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff98bfff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 0x7ff99413d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff992d9d700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 0x7ff9940cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff99259c700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff98400efc0 tx=0x7ff98400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:36.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400cb00 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98400f040 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.670+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98400ce60 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.671+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff994159730 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.671+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99415d7d0 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.672+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9940c5f10 con 0x7ff9940c5b40 2023-06-09T11:33:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.673+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff984014480 con 0x7ff9940c5b40 2023-06-09T11:33:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.673+0000 7ff98affd700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 0x7ff97406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.674+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7ff984089d20 con 0x7ff9940c5b40 2023-06-09T11:33:36.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.674+0000 7ff98bfff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 0x7ff97406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:36.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.674+0000 7ff98bfff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 0x7ff97406d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff994070600 tx=0x7ff97c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:36.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.678+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff984054910 con 0x7ff9940c5b40 2023-06-09T11:33:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.940+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff994074cf0 con 0x7ff9940c5b40 2023-06-09T11:33:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.941+0000 7ff98affd700 1 -- 172.21.15.7:0/427043594 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) v1 ==== 94+0+5462 (secure 0 0 0) 0x7ff984009e60 con 0x7ff9940c5b40 2023-06-09T11:33:36.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:36.948 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":98,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":98,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:30.675453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:clientreplay","state_seq":37,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 msgr2=0x7ff97406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 0x7ff97406d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff994070600 tx=0x7ff97c009040 comp rx=0 tx=0).stop 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 msgr2=0x7ff99413c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff98400efc0 tx=0x7ff98400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 shutdown_connections 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff97406ace0 0x7ff97406d190 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff994142850 0x7ff9940cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff9940c5b40 0x7ff99413c940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 --2- 172.21.15.7:0/427043594 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9940c6980 0x7ff99413d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.943+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 >> 172.21.15.7:0/427043594 conn(0x7ff99406a4d0 msgr2=0x7ff9940c0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.944+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 shutdown_connections 2023-06-09T11:33:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:36.944+0000 7ff998bff700 1 -- 172.21.15.7:0/427043594 wait complete. 2023-06-09T11:33:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 98 2023-06-09T11:33:36.967 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:33:38.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.172+0000 7fe9dd3ff700 1 Processor -- start 2023-06-09T11:33:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.172+0000 7fe9dd3ff700 1 -- start start 2023-06-09T11:33:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.173+0000 7fe9dd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.173+0000 7fe9dd3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe9d80ca580 0x7fe9d80c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9dd3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d815dcf0 con 0x7fe9d80b5360 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9dd3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d80c5e00 con 0x7fe9d80ca580 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9dd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d815d450 con 0x7fe9d80ca1b0 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d67fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe9d80b5360 0x7fe9d81536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39124/0 (socket says 172.21.15.7:39124) 2023-06-09T11:33:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d67fc700 1 -- 172.21.15.7:0/3579555372 learned_addr learned my addr 172.21.15.7:0/3579555372 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/3579555372 >> v2:172.21.15.7:3300/0 conn(0x7fe9d80ca580 0x7fe9d80c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d6ffd700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d5ffb700 1 -- 172.21.15.7:0/3579555372 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958374245 0 0) 0x7fe9d815dcf0 con 0x7fe9d80b5360 2023-06-09T11:33:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d5ffb700 1 -- 172.21.15.7:0/3579555372 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c00035b0 con 0x7fe9d80b5360 2023-06-09T11:33:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d77fe700 1 -- 172.21.15.7:0/3579555372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 msgr2=0x7fe9d815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d77fe700 1 -- 172.21.15.7:0/3579555372 >> v1:172.21.15.7:6789/0 conn(0x7fe9d80b5360 legacy=0x7fe9d81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.174+0000 7fe9d77fe700 1 -- 172.21.15.7:0/3579555372 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d815d730 con 0x7fe9d80ca580 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c58c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e965332740717fd0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9d5ffb700 1 -- 172.21.15.7:0/3579555372 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d000e070 con 0x7fe9d80ca580 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9d5ffb700 1 -- 172.21.15.7:0/3579555372 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d000bb20 con 0x7fe9d80ca580 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9d5ffb700 1 -- 172.21.15.7:0/3579555372 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0012430 con 0x7fe9d80ca580 2023-06-09T11:33:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/3579555372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 msgr2=0x7fe9d80c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.175+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c58c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0009d90 tx=0x7fe9d000b330 comp rx=0 tx=0).stop 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/3579555372 shutdown_connections 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c58c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/3579555372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/3579555372 >> 172.21.15.7:0/3579555372 conn(0x7fe9d8072090 msgr2=0x7fe9d8072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/3579555372 shutdown_connections 2023-06-09T11:33:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.176+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/3579555372 wait complete. 2023-06-09T11:33:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.177+0000 7fe9dd3ff700 1 Processor -- start 2023-06-09T11:33:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.177+0000 7fe9dd3ff700 1 -- start start 2023-06-09T11:33:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d80b5360 0x7fe9d80cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d80c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d815cd20 con 0x7fe9d80ca580 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80c94b0 con 0x7fe9d80b5360 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.178+0000 7fe9dd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d80c9790 con 0x7fe9d80ca1b0 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58246/0 (socket says 172.21.15.7:58246) 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 -- 172.21.15.7:0/279558225 learned_addr learned my addr 172.21.15.7:0/279558225 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d6ffd700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d80b5360 0x7fe9d80cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d67fc700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d80c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 -- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 msgr2=0x7fe9d80c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d80c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 -- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d80b5360 msgr2=0x7fe9d80cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d80b5360 0x7fe9d80cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 -- 172.21.15.7:0/279558225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d0009a70 con 0x7fe9d80ca580 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d67fc700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d80c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.179+0000 7fe9d77fe700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0014040 tx=0x7fe9d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.180+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00129f0 con 0x7fe9d80ca580 2023-06-09T11:33:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.180+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d0020070 con 0x7fe9d80ca580 2023-06-09T11:33:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.180+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d806a570 con 0x7fe9d80ca580 2023-06-09T11:33:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.180+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0012db0 con 0x7fe9d80ca580 2023-06-09T11:33:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.180+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d815da10 con 0x7fe9d80ca580 2023-06-09T11:33:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.181+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d806a850 con 0x7fe9d80ca580 2023-06-09T11:33:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.182+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9d0004040 con 0x7fe9d80ca580 2023-06-09T11:33:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.183+0000 7fe9bf7fe700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 0x7fe9c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.183+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fe9d0022070 con 0x7fe9d80ca580 2023-06-09T11:33:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.183+0000 7fe9d6ffd700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 0x7fe9c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.183+0000 7fe9d6ffd700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 0x7fe9c406d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009be0 tx=0x7fe9c8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.187+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe9d0056170 con 0x7fe9d80ca580 2023-06-09T11:33:39.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.441+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9d80c67b0 con 0x7fe9d80ca580 2023-06-09T11:33:39.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.441+0000 7fe9bf7fe700 1 -- 172.21.15.7:0/279558225 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5456 (secure 0 0 0) 0x7fe9d80c67b0 con 0x7fe9d80ca580 2023-06-09T11:33:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:39.447 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:38.700473+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.443+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 msgr2=0x7fe9c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.443+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 0x7fe9c406d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009be0 tx=0x7fe9c8009210 comp rx=0 tx=0).stop 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.443+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 msgr2=0x7fe9d80c7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.443+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0014040 tx=0x7fe9d0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 shutdown_connections 2023-06-09T11:33:39.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9d80ca580 0x7fe9d80c7690 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe9d80b5360 0x7fe9d80cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe9c406ace0 0x7fe9c406d190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 --2- 172.21.15.7:0/279558225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe9d80ca1b0 0x7fe9d80c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 >> 172.21.15.7:0/279558225 conn(0x7fe9d8072090 msgr2=0x7fe9d814dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 shutdown_connections 2023-06-09T11:33:39.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:39.444+0000 7fe9dd3ff700 1 -- 172.21.15.7:0/279558225 wait complete. 2023-06-09T11:33:39.452 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 99 2023-06-09T11:33:39.468 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=0, replacing gid=5531 2023-06-09T11:33:39.468 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.b 2023-06-09T11:33:44.468 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:33:44.469 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:33:44.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:33:44.474 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:33:44.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:44.625 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:33:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.694+0000 7ffbbedff700 1 Processor -- start 2023-06-09T11:33:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.694+0000 7ffbbedff700 1 -- start start 2023-06-09T11:33:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbedff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 0x7ffbb815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbedff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 0x7ffbb815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbedff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb815a500 con 0x7ffbb815b2d0 2023-06-09T11:33:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbedff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80c5f10 con 0x7ffbb8153880 2023-06-09T11:33:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbedff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb815da00 con 0x7ffbb80c9f40 2023-06-09T11:33:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbd39c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffbb815b2d0 0x7ffbb80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49252/0 (socket says 172.21.15.7:49252) 2023-06-09T11:33:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbd39c700 1 -- 172.21.15.7:0/1627115245 learned_addr learned my addr 172.21.15.7:0/1627115245 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1627115245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 0x7ffbb815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.696+0000 7ffbbcb9b700 1 --2- 172.21.15.7:0/1627115245 >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 0x7ffbb815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1627115245 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713436090 0 0) 0x7ffbb0001490 con 0x7ffbb815b2d0 2023-06-09T11:33:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1627115245 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba80035b0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1627115245 >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 msgr2=0x7ffbb815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1627115245 >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 0x7ffbb815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1627115245 >> v1:172.21.15.7:6789/0 conn(0x7ffbb815b2d0 legacy=0x7ffbb80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb7fff700 1 -- 172.21.15.7:0/1627115245 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb815b750 con 0x7ffbb80c9f40 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbbcb9b700 1 --2- 172.21.15.7:0/1627115245 >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 0x7ffbb815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1627115245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 0x7ffbb815ad90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009840 tx=0x7ffba4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a33f2cde85aa80 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1627115245 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011830 con 0x7ffbb80c9f40 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1627115245 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4011990 con 0x7ffbb80c9f40 2023-06-09T11:33:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.697+0000 7ffbb77fe700 1 -- 172.21.15.7:0/1627115245 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011d20 con 0x7ffbb80c9f40 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 -- 172.21.15.7:0/1627115245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 msgr2=0x7ffbb815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1627115245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 0x7ffbb815ad90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffba4009840 tx=0x7ffba4009b50 comp rx=0 tx=0).stop 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 -- 172.21.15.7:0/1627115245 shutdown_connections 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1627115245 >> v2:172.21.15.117:3300/0 conn(0x7ffbb8153880 0x7ffbb815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1627115245 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb80c9f40 0x7ffbb815ad90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 -- 172.21.15.7:0/1627115245 >> 172.21.15.7:0/1627115245 conn(0x7ffbb80737f0 msgr2=0x7ffbb80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 -- 172.21.15.7:0/1627115245 shutdown_connections 2023-06-09T11:33:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.698+0000 7ffbbedff700 1 -- 172.21.15.7:0/1627115245 wait complete. 2023-06-09T11:33:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.700+0000 7ffbbedff700 1 Processor -- start 2023-06-09T11:33:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.700+0000 7ffbbedff700 1 -- start start 2023-06-09T11:33:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb80c9f40 0x7ffbb80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 0x7ffbb81e6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80b4c10 con 0x7ffbb815b2d0 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80b5510 con 0x7ffbb80c9f40 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbedff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80b5250 con 0x7ffbb8153880 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47112/0 (socket says 172.21.15.7:47112) 2023-06-09T11:33:44.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 -- 172.21.15.7:0/1315539036 learned_addr learned my addr 172.21.15.7:0/1315539036 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbcb9b700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb80c9f40 0x7ffbb80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 -- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 msgr2=0x7ffbb81e6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 0x7ffbb81e6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 0x7ffbb81e6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 -- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb80c9f40 msgr2=0x7ffbb80b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb80c9f40 0x7ffbb80b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbbd39c700 1 -- 172.21.15.7:0/1315539036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0094a0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.701+0000 7ffbb7fff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 0x7ffbb81e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbbd39c700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cae0 tx=0x7ffbb000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000aa80 con 0x7ffbb815b2d0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000abe0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0012400 con 0x7ffbb815b2d0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffba40094a0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.702+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb8159980 con 0x7ffbb815b2d0 2023-06-09T11:33:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.703+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb81596e0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.704+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffbb001b3b0 con 0x7ffbb815b2d0 2023-06-09T11:33:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.705+0000 7ffbb57fa700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 0x7ffb9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:44.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.705+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7ffbb0089510 con 0x7ffbb815b2d0 2023-06-09T11:33:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.705+0000 7ffbbcb9b700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 0x7ffb9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.706+0000 7ffbbcb9b700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 0x7ffb9c06d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d00 tx=0x7ffbac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:44.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.709+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffbb0054100 con 0x7ffbb815b2d0 2023-06-09T11:33:44.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.984+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbb8159c60 con 0x7ffbb815b2d0 2023-06-09T11:33:44.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.984+0000 7ffbb57fa700 1 -- 172.21.15.7:0/1315539036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+5456 (secure 0 0 0) 0x7ffbb0001b70 con 0x7ffbb815b2d0 2023-06-09T11:33:44.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:44.991 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":99,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:38.700473+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:44.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 msgr2=0x7ffb9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 0x7ffb9c06d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009d00 tx=0x7ffbac008040 comp rx=0 tx=0).stop 2023-06-09T11:33:44.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 msgr2=0x7ffbb81f2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:44.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000cae0 tx=0x7ffbb000cea0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 shutdown_connections 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbb815b2d0 0x7ffbb81f2590 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffbb80c9f40 0x7ffbb80b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb9c06ace0 0x7ffb9c06d190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 --2- 172.21.15.7:0/1315539036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffbb8153880 0x7ffbb81e6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 >> 172.21.15.7:0/1315539036 conn(0x7ffbb80737f0 msgr2=0x7ffbb814e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:44.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 shutdown_connections 2023-06-09T11:33:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:44.987+0000 7ffbbedff700 1 -- 172.21.15.7:0/1315539036 wait complete. 2023-06-09T11:33:44.995 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 99 2023-06-09T11:33:45.011 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:33:46.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:46.883 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:33:47.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:33:47.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.212+0000 7f39b65ff700 1 Processor -- start 2023-06-09T11:33:47.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.212+0000 7f39b65ff700 1 -- start start 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.213+0000 7f39b65ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b00ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.213+0000 7f39b65ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39b0071fa0 0x7f39b00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.213+0000 7f39b65ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b015a390 con 0x7f39b0153b30 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.213+0000 7f39b65ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b00c5f40 con 0x7f39b0071fa0 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.213+0000 7f39b65ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00ca720 con 0x7f39b0153760 2023-06-09T11:33:47.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f39b0071fa0 0x7f39b00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39af7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f39b0153b30 0x7f39b00b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49254/0 (socket says 172.21.15.7:49254) 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39af7fe700 1 -- 172.21.15.7:0/2359221577 learned_addr learned my addr 172.21.15.7:0/2359221577 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39affff700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b00ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 -- 172.21.15.7:0/2359221577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 msgr2=0x7f39b00ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b00ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 -- 172.21.15.7:0/2359221577 >> v1:172.21.15.7:6789/0 conn(0x7f39b0153b30 legacy=0x7f39b00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 -- 172.21.15.7:0/2359221577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b00c5a20 con 0x7f39b0071fa0 2023-06-09T11:33:47.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39affff700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b00ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:33:47.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39aeffd700 1 -- 172.21.15.7:0/2359221577 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118916483 0 0) 0x7f39b015a390 con 0x7f39b0153b30 2023-06-09T11:33:47.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.214+0000 7f39b4b9c700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0071fa0 0x7f39b00723c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009d90 tx=0x7f39a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30208b7f10cb0460 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:47.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39aeffd700 1 -- 172.21.15.7:0/2359221577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a800e070 con 0x7f39b0071fa0 2023-06-09T11:33:47.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39aeffd700 1 -- 172.21.15.7:0/2359221577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a800bbd0 con 0x7f39b0071fa0 2023-06-09T11:33:47.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39aeffd700 1 -- 172.21.15.7:0/2359221577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8012430 con 0x7f39b0071fa0 2023-06-09T11:33:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 -- 172.21.15.7:0/2359221577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0071fa0 msgr2=0x7f39b00723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0071fa0 0x7f39b00723c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009d90 tx=0x7f39a800b330 comp rx=0 tx=0).stop 2023-06-09T11:33:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 -- 172.21.15.7:0/2359221577 shutdown_connections 2023-06-09T11:33:47.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0071fa0 0x7f39b00723c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 --2- 172.21.15.7:0/2359221577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b00ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 -- 172.21.15.7:0/2359221577 >> 172.21.15.7:0/2359221577 conn(0x7f39b00737f0 msgr2=0x7f39b0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 -- 172.21.15.7:0/2359221577 shutdown_connections 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.215+0000 7f39b65ff700 1 -- 172.21.15.7:0/2359221577 wait complete. 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.218+0000 7f39b65ff700 1 Processor -- start 2023-06-09T11:33:47.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.218+0000 7f39b65ff700 1 -- start start 2023-06-09T11:33:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 0x7f39b00c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b0159530 con 0x7f39b0153b30 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b01f2bb0 con 0x7f39b0071fa0 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.219+0000 7f39b65ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00c8190 con 0x7f39b0153760 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:60208/0 (socket says 172.21.15.7:60208) 2023-06-09T11:33:47.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 -- 172.21.15.7:0/563727717 learned_addr learned my addr 172.21.15.7:0/563727717 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39af7fe700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39b4b9c700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 0x7f39b00c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 -- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 msgr2=0x7f39b0158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b0158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 -- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 msgr2=0x7f39b00c69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 0x7f39b00c69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39affff700 1 -- 172.21.15.7:0/563727717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a8009a70 con 0x7f39b0071fa0 2023-06-09T11:33:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.220+0000 7f39b4b9c700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 0x7f39b00c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:33:47.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39affff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e810 tx=0x7f39a000eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a000c070 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0004230 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00036a0 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b0159a60 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.221+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b0159780 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.222+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b0159d40 con 0x7f39b0071fa0 2023-06-09T11:33:47.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.224+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f39a0003800 con 0x7f39b0071fa0 2023-06-09T11:33:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.224+0000 7f39acff9700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:33:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.225+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f39a0089590 con 0x7f39b0071fa0 2023-06-09T11:33:47.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.225+0000 7f39af7fe700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:33:47.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.225+0000 7f39af7fe700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 0x7f399c06d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f39b00b2890 tx=0x7f39a4007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:33:47.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.230+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f39a0054170 con 0x7f39b0071fa0 2023-06-09T11:33:47.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.563+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f39b0074cf0 con 0x7f39b0071fa0 2023-06-09T11:33:47.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.564+0000 7f39acff9700 1 -- 172.21.15.7:0/563727717 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+6215 (secure 0 0 0) 0x7f39a0057790 con 0x7f39b0071fa0 2023-06-09T11:33:47.571 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:33:47.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:33:38.700473+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:33:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 msgr2=0x7f399c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 0x7f399c06d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f39b00b2890 tx=0x7f39a4007e70 comp rx=0 tx=0).stop 2023-06-09T11:33:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 msgr2=0x7f39b01424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:33:47.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f39a000e810 tx=0x7f39a000eb20 comp rx=0 tx=0).stop 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 shutdown_connections 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39b0153b30 0x7f39b00c69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f39b0071fa0 0x7f39b01424e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f399c06ace0 0x7f399c06d190 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 --2- 172.21.15.7:0/563727717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f39b0153760 0x7f39b0158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 >> 172.21.15.7:0/563727717 conn(0x7f39b00737f0 msgr2=0x7f39b014d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:33:47.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 shutdown_connections 2023-06-09T11:33:47.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:33:47.567+0000 7f39b65ff700 1 -- 172.21.15.7:0/563727717 wait complete. 2023-06-09T11:33:47.575 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 100 2023-06-09T11:33:47.591 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:33:47.586+0000 7f0e9a7f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:33:47.591 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:33:47.586+0000 7f0e9a7f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:33:47.591 INFO:tasks.ceph.mds.d.smithi117.stderr:2023-06-09T11:33:47.586+0000 7f0e9a7f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:33:47.593 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:33:47.594 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 47 secs before thrashing 2023-06-09T11:33:55.394 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 1% (176/11800) Updating files: 2% (236/11800) Updating files: 2% (305/11800) Updating files: 3% (354/11800) Updating files: 3% (436/11800) Updating files: 4% (472/11800) Updating files: 4% (498/11800) Updating files: 5% (590/11800) Updating files: 6% (708/11800) Updating files: 6% (727/11800) Updating files: 7% (826/11800) Updating files: 7% (872/11800) Updating files: 8% (944/11800) Updating files: 8% (1037/11800) Updating files: 9% (1062/11800) Updating files: 9% (1089/11800) Updating files: 9% (1099/11800) Updating files: 9% (1135/11800) Updating files: 10% (1180/11800) Updating files: 10% (1280/11800) Updating files: 11% (1298/11800) Updating files: 11% (1369/11800) Updating files: 12% (1416/11800) Updating files: 13% (1534/11800) Updating files: 13% (1564/11800) Updating files: 13% (1600/11800) Updating files: 14% (1652/11800) Updating files: 14% (1738/11800) Updating files: 15% (1770/11800) Updating files: 15% (1847/11800) Updating files: 16% (1888/11800) Updating files: 16% (1966/11800) Updating files: 17% (2006/11800) Updating files: 17% (2021/11800) Updating files: 18% (2124/11800) Updating files: 18% (2159/11800) Updating files: 18% (2202/11800) Updating files: 18% (2219/11800) Updating files: 19% (2242/11800) Updating files: 19% (2262/11800) Updating files: 19% (2276/11800) Updating files: 19% (2332/11800) Updating files: 20% (2360/11800) Updating files: 20% (2403/11800) Updating files: 20% (2474/11800) Updating files: 21% (2478/11800) Updating files: 21% (2558/11800) Updating files: 22% (2596/11800) Updating files: 22% (2657/11800) Updating files: 22% (2701/11800) Updating files: 23% (2714/11800) Updating files: 24% (2832/11800) Updating files: 24% (2865/11800) Updating files: 25% (2950/11800) Updating files: 25% (2969/11800) Updating files: 26% (3068/11800) Updating files: 26% (3126/11800) Updating files: 27% (3186/11800) Updating files: 28% (3304/11800) Updating files: 28% (3306/11800) Updating files: 28% (3404/11800) Updating files: 29% (3422/11800) Updating files: 29% (3449/11800) Updating files: 30% (3540/11800) Updating files: 30% (3612/11800) Updating files: 31% (3658/11800) Updating files: 31% (3737/11800) Updating files: 32% (3776/11800) Updating files: 32% (3827/11800) Updating files: 32% (3854/11800) Updating files: 33% (3894/11800) Updating files: 33% (4006/11800) Updating files: 34% (4012/11800) Updating files: 35% (4130/11800) Updating files: 35% (4178/11800) Updating files: 35% (4220/11800) Updating files: 35% (4241/11800) Updating files: 36% (4248/11800) Updating files: 36% (4338/11800) Updating files: 37% (4366/11800) Updating files: 38% (4484/11800) Updating files: 38% (4557/11800) Updating files: 39% (4602/11800) Updating files: 39% (4681/11800) Updating files: 40% (4720/11800) Updating files: 40% (4735/11800) Updating files: 41% (4838/11800) Updating files: 42% (4956/11800) Updating files: 42% (4975/11800) Updating files: 42% (5060/11800) Updating files: 43% (5074/11800) Updating files: 43% (5109/11800) Updating files: 44% (5192/11800) Updating files: 44% (5254/11800) Updating files: 44% (5298/11800) Updating files: 45% (5310/11800) Updating files: 45% (5413/11800) Updating files: 46% (5428/11800) Updating files: 46% (5517/11800) Updating files: 47% (5546/11800) Updating files: 48% (5664/11800) Updating files: 48% (5781/11800) Updating files: 49% (5782/11800) Updating files: 50% (5900/11800) Updating files: 50% (6005/11800) Updating files: 51% (6018/11800) Updating files: 51% (6056/11800) Updating files: 51% (6130/11800) Updating files: 52% (6136/11800) Updating files: 52% (6236/11800) Updating files: 53% (6254/11800) Updating files: 53% (6270/11800) Updating files: 53% (6367/11800) Updating files: 54% (6372/11800) Updating files: 54% (6464/11800) Updating files: 55% (6490/11800) Updating files: 55% (6557/11800) Updating files: 56% (6608/11800) Updating files: 57% (6726/11800) Updating files: 57% (6734/11800) Updating files: 57% (6755/11800) Updating files: 58% (6844/11800) Updating files: 58% (6928/11800) Updating files: 59% (6962/11800) Updating files: 59% (7035/11800) Updating files: 60% (7080/11800) Updating files: 60% (7136/11800) Updating files: 61% (7198/11800) Updating files: 61% (7238/11800) Updating files: 62% (7316/11800) Updating files: 63% (7434/11800) Updating files: 63% (7474/11800) Updating files: 64% (7552/11800) Updating files: 65% (7670/11800) Updating files: 65% (7698/11800) Updating files: 66% (7788/11800) Updating files: 66% (7868/11800) Updating files: 67% (7906/11800) Updating files: 67% (7966/11800) Updating files: 67% (8015/11800) Updating files: 68% (8024/11800) Updating files: 68% (8130/11800) Updating files: 69% (8142/11800) Updating files: 69% (8235/11800) Updating files: 70% (8260/11800) Updating files: 70% (8289/11800) Updating files: 70% (8319/11800) Updating files: 71% (8378/11800) Updating files: 71% (8403/11800) Updating files: 72% (8496/11800) Updating files: 72% (8517/11800) Updating files: 72% (8600/11800) Updating files: 73% (8614/11800) Updating files: 73% (8717/11800) Updating files: 74% (8732/11800) Updating files: 74% (8764/11800) Updating files: 74% (8840/11800) Updating files: 75% (8850/11800) Updating files: 75% (8867/11800) Updating files: 76% (8968/11800) Updating files: 76% (8997/11800) Updating files: 77% (9086/11800) Updating files: 77% (9092/11800) Updating files: 77% (9093/11800) Updating files: 77% (9114/11800) Updating files: 78% (9204/11800) Updating files: 78% (9247/11800) Updating files: 79% (9322/11800) Updating files: 79% (9357/11800) Updating files: 79% (9372/11800) Updating files: 80% (9440/11800) Updating files: 81% (9558/11800) Updating files: 81% (9623/11800) Updating files: 82% (9676/11800) Updating files: 82% (9716/11800) Updating files: 83% (9794/11800) Updating files: 83% (9841/11800) Updating files: 84% (9912/11800) Updating files: 84% (9946/11800) Updating files: 85% (10030/11800) Updating files: 85% (10100/11800) Updating files: 86% (10148/11800) Updating files: 87% (10266/11800) Updating files: 87% (10327/11800) Updating files: 88% (10384/11800) Updating files: 88% (10430/11800) Updating files: 88% (10431/11800) Updating files: 89% (10502/11800) Updating files: 89% (10544/11800) Updating files: 89% (10619/11800) Updating files: 90% (10620/11800) Updating files: 90% (10696/11800) Updating files: 91% (10738/11800) Updating files: 91% (10805/11800) Updating files: 92% (10856/11800) Updating files: 92% (10965/11800) Updating files: 93% (10974/11800) Updating files: 93% (11060/11800) Updating files: 93% (11070/11800) Updating files: 94% (11092/11800) Updating files: 94% (11148/11800) Updating files: 95% (11210/11800) Updating files: 95% (11285/11800) Updating files: 96% (11328/11800) Updating files: 96% (11350/11800) Updating files: 97% (11446/11800) Updating files: 97% (11468/11800) Updating files: 98% (11564/11800) Updating files: 98% (11606/11800) Updating files: 98% (11630/11800) Updating files: 98% (11643/11800) Updating files: 99% (11682/11800) Updating files: 99% (11757/11800) Updating files: 100% (11800/11800) Updating files: 100% (11800/11800), done. 2023-06-09T11:33:55.457 INFO:tasks.workunit.client.0.smithi007.stderr:+ trap - EXIT 2023-06-09T11:33:55.458 INFO:tasks.workunit.client.0.smithi007.stderr:+ cd ceph 2023-06-09T11:33:55.461 INFO:tasks.workunit.client.0.smithi007.stderr:++ seq 1 90 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:+ versions='1 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:2 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:3 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:4 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:5 2023-06-09T11:33:55.463 INFO:tasks.workunit.client.0.smithi007.stderr:6 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:7 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:8 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:9 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:10 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:11 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:12 2023-06-09T11:33:55.464 INFO:tasks.workunit.client.0.smithi007.stderr:13 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:14 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:15 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:16 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:17 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:18 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:19 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:20 2023-06-09T11:33:55.465 INFO:tasks.workunit.client.0.smithi007.stderr:21 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:22 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:23 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:24 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:25 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:26 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:27 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:28 2023-06-09T11:33:55.466 INFO:tasks.workunit.client.0.smithi007.stderr:29 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:30 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:31 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:32 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:33 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:34 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:35 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:36 2023-06-09T11:33:55.467 INFO:tasks.workunit.client.0.smithi007.stderr:37 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:38 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:39 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:40 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:41 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:42 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:43 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:44 2023-06-09T11:33:55.468 INFO:tasks.workunit.client.0.smithi007.stderr:45 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:46 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:47 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:48 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:49 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:50 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:51 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:52 2023-06-09T11:33:55.469 INFO:tasks.workunit.client.0.smithi007.stderr:53 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:54 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:55 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:56 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:57 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:58 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:59 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:60 2023-06-09T11:33:55.470 INFO:tasks.workunit.client.0.smithi007.stderr:61 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:62 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:63 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:64 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:65 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:66 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:67 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:68 2023-06-09T11:33:55.471 INFO:tasks.workunit.client.0.smithi007.stderr:69 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:70 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:71 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:72 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:73 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:74 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:75 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:76 2023-06-09T11:33:55.472 INFO:tasks.workunit.client.0.smithi007.stderr:77 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:78 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:79 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:80 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:81 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:82 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:83 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:84 2023-06-09T11:33:55.473 INFO:tasks.workunit.client.0.smithi007.stderr:85 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:86 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:87 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:88 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:89 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:90' 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 1 -eq 48 ']' 2023-06-09T11:33:55.474 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.1 2023-06-09T11:33:55.475 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.1 2023-06-09T11:33:55.475 INFO:tasks.workunit.client.0.smithi007.stdout:v0.1 2023-06-09T11:33:55.475 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.1 2023-06-09T11:34:17.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:17.091 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:34.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.817+0000 7fc4d5fff700 1 Processor -- start 2023-06-09T11:34:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.817+0000 7fc4d5fff700 1 -- start start 2023-06-09T11:34:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4d00cb9a0 0x7fc4d0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d015cd40 0x7fc4d00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d0159b20 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d00cac80 con 0x7fc4d00cb9a0 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.818+0000 7fc4d5fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01597d0 con 0x7fc4d015cd40 2023-06-09T11:34:34.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50048/0 (socket says 172.21.15.7:50048) 2023-06-09T11:34:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 -- 172.21.15.7:0/2860978557 learned_addr learned my addr 172.21.15.7:0/2860978557 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cffff700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d015cd40 0x7fc4d00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4ceffd700 1 --2- 172.21.15.7:0/2860978557 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00cb9a0 0x7fc4d0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 -- 172.21.15.7:0/2860978557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d015cd40 msgr2=0x7fc4d00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d015cd40 0x7fc4d00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 -- 172.21.15.7:0/2860978557 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00cb9a0 msgr2=0x7fc4d0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/2860978557 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00cb9a0 0x7fc4d0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 -- 172.21.15.7:0/2860978557 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d0071ab0 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.819+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00c9c60 tx=0x7fc4c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c85648781273be server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4ce7fc700 1 -- 172.21.15.7:0/2860978557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0011830 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4ce7fc700 1 -- 172.21.15.7:0/2860978557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c0011990 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4ce7fc700 1 -- 172.21.15.7:0/2860978557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0011d20 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 -- 172.21.15.7:0/2860978557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d00cb5d0 msgr2=0x7fc4d00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00c9c60 tx=0x7fc4c0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 -- 172.21.15.7:0/2860978557 shutdown_connections 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d015cd40 0x7fc4d00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/2860978557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d00cb5d0 0x7fc4d00b52b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/2860978557 >> v2:172.21.15.117:3300/0 conn(0x7fc4d00cb9a0 0x7fc4d0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 -- 172.21.15.7:0/2860978557 >> 172.21.15.7:0/2860978557 conn(0x7fc4d0070af0 msgr2=0x7fc4d0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 -- 172.21.15.7:0/2860978557 shutdown_connections 2023-06-09T11:34:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.820+0000 7fc4d5fff700 1 -- 172.21.15.7:0/2860978557 wait complete. 2023-06-09T11:34:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.822+0000 7fc4d5fff700 1 Processor -- start 2023-06-09T11:34:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.822+0000 7fc4d5fff700 1 -- start start 2023-06-09T11:34:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d00cb5d0 0x7fc4d01506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00cb9a0 0x7fc4d0155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0153be0 con 0x7fc4d015cd40 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d0150be0 con 0x7fc4d00cb9a0 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4d5fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d01f02a0 con 0x7fc4d00cb5d0 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50054/0 (socket says 172.21.15.7:50054) 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4cffff700 1 -- 172.21.15.7:0/93217316 learned_addr learned my addr 172.21.15.7:0/93217316 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d00cb5d0 0x7fc4d01506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.823+0000 7fc4ceffd700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00cb9a0 0x7fc4d0155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 -- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d00cb5d0 msgr2=0x7fc4d01506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d00cb5d0 0x7fc4d01506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 -- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00cb9a0 msgr2=0x7fc4d0155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00cb9a0 0x7fc4d0155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 -- 172.21.15.7:0/93217316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0009550 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4cffff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800efc0 tx=0x7fc4c8009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8009b20 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c800f040 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c800cc70 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d015d4f0 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.824+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d015d210 con 0x7fc4d015cd40 2023-06-09T11:34:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.825+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d015d7d0 con 0x7fc4d015cd40 2023-06-09T11:34:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.826+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4c8014480 con 0x7fc4d015cd40 2023-06-09T11:34:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.827+0000 7fc4b7fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 0x7fc4bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.827+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fc4c808a3c0 con 0x7fc4d015cd40 2023-06-09T11:34:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.828+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 0x7fc4bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.828+0000 7fc4cf7fe700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 0x7fc4bc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009c10 tx=0x7fc4c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:34.832+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4c8054f20 con 0x7fc4d015cd40 2023-06-09T11:34:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.101+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4d0074cf0 con 0x7fc4d015cd40 2023-06-09T11:34:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.102+0000 7fc4b7fff700 1 -- 172.21.15.7:0/93217316 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fc4c8018090 con 0x7fc4d015cd40 2023-06-09T11:34:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:35.109 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 msgr2=0x7fc4bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 0x7fc4bc06d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009c10 tx=0x7fc4c0008030 comp rx=0 tx=0).stop 2023-06-09T11:34:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 msgr2=0x7fc4d0156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800efc0 tx=0x7fc4c8009f60 comp rx=0 tx=0).stop 2023-06-09T11:34:35.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 shutdown_connections 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4d015cd40 0x7fc4d0156440 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4bc06ace0 0x7fc4bc06d190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4d00cb5d0 0x7fc4d01506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 --2- 172.21.15.7:0/93217316 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4d00cb9a0 0x7fc4d0155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 >> 172.21.15.7:0/93217316 conn(0x7fc4d0070af0 msgr2=0x7fc4d00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 shutdown_connections 2023-06-09T11:34:35.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.105+0000 7fc4d5fff700 1 -- 172.21.15.7:0/93217316 wait complete. 2023-06-09T11:34:35.113 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:35.131 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=0) 2023-06-09T11:34:35.132 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:34:35.132 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:34:35.151 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:34:35.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.365+0000 7fb8989ff700 1 Processor -- start 2023-06-09T11:34:35.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.365+0000 7fb8989ff700 1 -- start start 2023-06-09T11:34:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.366+0000 7fb8989ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb8941599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.366+0000 7fb8989ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940c0bf0 con 0x7fb8940c0140 2023-06-09T11:34:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.366+0000 7fb8989ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940702f0 con 0x7fb8940c1f90 2023-06-09T11:34:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.366+0000 7fb8989ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb894070110 con 0x7fb8940c1bc0 2023-06-09T11:34:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb891d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb8940c0140 0x7fb8941501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59566/0 (socket says 172.21.15.7:59566) 2023-06-09T11:34:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb891d9b700 1 -- 172.21.15.7:0/222404942 learned_addr learned my addr 172.21.15.7:0/222404942 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:35.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140681183 0 0) 0x7fb8940c0bf0 con 0x7fb8940c0140 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89259c700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb8941599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb87c0057c0 con 0x7fb8940c0140 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337601142 0 0) 0x7fb8940702f0 con 0x7fb8940c1f90 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8940c0bf0 con 0x7fb8940c1f90 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4049159309 0 0) 0x7fb87c0057c0 con 0x7fb8940c0140 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 msgr2=0x7fb8941599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb8941599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 >> v1:172.21.15.117:6789/0 conn(0x7fb8940c1f90 legacy=0x7fb894160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940c0e90 con 0x7fb8940c0140 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb888001670 con 0x7fb8940c0140 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 >> v1:172.21.15.7:6789/0 conn(0x7fb8940c0140 legacy=0x7fb8941501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.367+0000 7fb89159a700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb87c005dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89159a700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c006330 0x7fb87c006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89159a700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c006fa0 0x7fb87c007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb894070110 con 0x7fb87c006fa0 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c00dcd0 con 0x7fb87c006330 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87c00de90 con 0x7fb8940c1bc0 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb892d9d700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c006fa0 0x7fb87c007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb891d9b700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c006330 0x7fb87c006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb87c005dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 -- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c006330 msgr2=0x7fb87c006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb87c006330 0x7fb87c006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 -- 172.21.15.7:0/222404942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c006fa0 msgr2=0x7fb87c007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb87c006fa0 0x7fb87c007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 -- 172.21.15.7:0/222404942 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87c0057c0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.368+0000 7fb89259c700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb87c005dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb884009b30 tx=0x7fb87c007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d6036cd39dc6648 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400e6c0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88400e820 con 0x7fb8940c1bc0 2023-06-09T11:34:35.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb89159a700 1 -- 172.21.15.7:0/222404942 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400eb90 con 0x7fb8940c1bc0 2023-06-09T11:34:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 -- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 msgr2=0x7fb87c005dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 --2- 172.21.15.7:0/222404942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c1bc0 0x7fb87c005dd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb884009b30 tx=0x7fb87c007a90 comp rx=0 tx=0).stop 2023-06-09T11:34:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb892d9d700 1 -- 172.21.15.7:0/222404942 reap_dead start 2023-06-09T11:34:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 -- 172.21.15.7:0/222404942 shutdown_connections 2023-06-09T11:34:35.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 -- 172.21.15.7:0/222404942 >> 172.21.15.7:0/222404942 conn(0x7fb89406a4d0 msgr2=0x7fb89406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 -- 172.21.15.7:0/222404942 shutdown_connections 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.369+0000 7fb8989ff700 1 -- 172.21.15.7:0/222404942 wait complete. 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.372+0000 7fb8989ff700 1 Processor -- start 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.372+0000 7fb8989ff700 1 -- start start 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb894155d50 0x7fb8941ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c4720 0x7fb8941de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941ddb20 con 0x7fb8940c1bc0 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941dd7f0 con 0x7fb894155d50 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb8989ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b0f60 con 0x7fb8940c4720 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb89259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.373+0000 7fb89259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50086/0 (socket says 172.21.15.7:50086) 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 -- 172.21.15.7:0/219198715 learned_addr learned my addr 172.21.15.7:0/219198715 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb891d9b700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb894155d50 0x7fb8941ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb892d9d700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c4720 0x7fb8941de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 -- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c4720 msgr2=0x7fb8941de4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c4720 0x7fb8941de4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 -- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb894155d50 msgr2=0x7fb8941ddfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb894155d50 0x7fb8941ddfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 -- 172.21.15.7:0/219198715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940c1450 con 0x7fb8940c1bc0 2023-06-09T11:34:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb89259c700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8840090b0 tx=0x7fb8840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.374+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884021070 con 0x7fb8940c1bc0 2023-06-09T11:34:35.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.375+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8840044a0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.375+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8940c1170 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.375+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88400e6c0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.375+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8940705d0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.376+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8940c7450 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.377+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb884025070 con 0x7fb8940c1bc0 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.377+0000 7fb882ffd700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 0x7fb87406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:35.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.378+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fb88408a820 con 0x7fb8940c1bc0 2023-06-09T11:34:35.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.378+0000 7fb891d9b700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 0x7fb87406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:35.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.378+0000 7fb891d9b700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 0x7fb87406d1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb8940c4df0 tx=0x7fb888012040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:35.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.382+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb884055550 con 0x7fb8940c1bc0 2023-06-09T11:34:35.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.635+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb894074cf0 con 0x7fb8940c1bc0 2023-06-09T11:34:35.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.636+0000 7fb882ffd700 1 -- 172.21.15.7:0/219198715 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fb884058b70 con 0x7fb8940c1bc0 2023-06-09T11:34:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:35.642 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:35.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 msgr2=0x7fb87406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 0x7fb87406d1c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb8940c4df0 tx=0x7fb888012040 comp rx=0 tx=0).stop 2023-06-09T11:34:35.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 msgr2=0x7fb8940c2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb8840090b0 tx=0x7fb8840047c0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 shutdown_connections 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb87406ad10 0x7fb87406d1c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb8940c4720 0x7fb8941de4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb8940c1bc0 0x7fb8940c2520 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 --2- 172.21.15.7:0/219198715 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb894155d50 0x7fb8941ddfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:35.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 >> 172.21.15.7:0/219198715 conn(0x7fb89406a4d0 msgr2=0x7fb8940c66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:35.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 shutdown_connections 2023-06-09T11:34:35.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:35.638+0000 7fb8989ff700 1 -- 172.21.15.7:0/219198715 wait complete. 2023-06-09T11:34:35.646 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:35.662 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:37.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:37.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.867+0000 7ff6ec3ff700 1 Processor -- start 2023-06-09T11:34:37.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.867+0000 7ff6ec3ff700 1 -- start start 2023-06-09T11:34:37.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.868+0000 7ff6ec3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9fe0 0x7ff6e40cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.868+0000 7ff6ec3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6e40cfaf0 0x7ff6e41598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.868+0000 7ff6ec3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e415cec0 con 0x7ff6e40c9090 2023-06-09T11:34:37.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ec3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40ca410 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ec3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e415d160 con 0x7ff6e40c9fe0 2023-06-09T11:34:37.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea99c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff6e40cfaf0 0x7ff6e41598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea19b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6e40c9090 0x7ff6e40cad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59574/0 (socket says 172.21.15.7:59574) 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea19b700 1 -- 172.21.15.7:0/2053667473 learned_addr learned my addr 172.21.15.7:0/2053667473 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6e999a700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9fe0 0x7ff6e40cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea99c700 1 -- 172.21.15.7:0/2053667473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9fe0 msgr2=0x7ff6e40cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea99c700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9fe0 0x7ff6e40cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea99c700 1 -- 172.21.15.7:0/2053667473 >> v1:172.21.15.7:6789/0 conn(0x7ff6e40c9090 legacy=0x7ff6e40cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6ea99c700 1 -- 172.21.15.7:0/2053667473 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e415d440 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.869+0000 7ff6e9199700 1 -- 172.21.15.7:0/2053667473 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 503508702 0 0) 0x7ff6e415cec0 con 0x7ff6e40c9090 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ea99c700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e41598b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009d90 tx=0x7ff6dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21220e1e8112559a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6e9199700 1 -- 172.21.15.7:0/2053667473 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00e070 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6e9199700 1 -- 172.21.15.7:0/2053667473 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00bb20 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6e9199700 1 -- 172.21.15.7:0/2053667473 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc012430 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/2053667473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 msgr2=0x7ff6e41598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e41598b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc009d90 tx=0x7ff6dc00b330 comp rx=0 tx=0).stop 2023-06-09T11:34:37.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/2053667473 shutdown_connections 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e41598b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/2053667473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9fe0 0x7ff6e40cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.870+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/2053667473 >> 172.21.15.7:0/2053667473 conn(0x7ff6e4070af0 msgr2=0x7ff6e4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.871+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/2053667473 shutdown_connections 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.871+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/2053667473 wait complete. 2023-06-09T11:34:37.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.873+0000 7ff6ec3ff700 1 Processor -- start 2023-06-09T11:34:37.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.873+0000 7ff6ec3ff700 1 -- start start 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9090 0x7ff6e4152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e40c9fe0 0x7ff6e40c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41ef380 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e41ef870 con 0x7ff6e40c9fe0 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.874+0000 7ff6ec3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40c48b0 con 0x7ff6e40c9090 2023-06-09T11:34:37.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50106/0 (socket says 172.21.15.7:50106) 2023-06-09T11:34:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea19b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9090 0x7ff6e4152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 -- 172.21.15.7:0/705206678 learned_addr learned my addr 172.21.15.7:0/705206678 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:37.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6e999a700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e40c9fe0 0x7ff6e40c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 -- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9090 msgr2=0x7ff6e4152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9090 0x7ff6e4152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 -- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e40c9fe0 msgr2=0x7ff6e40c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:37.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e40c9fe0 0x7ff6e40c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 -- 172.21.15.7:0/705206678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6dc009a70 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.875+0000 7ff6ea99c700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff6e408cda0 tx=0x7ff6dc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.876+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc012430 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.876+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc020070 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.876+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc004280 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.876+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40ca620 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.876+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e415d720 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.877+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e40b53f0 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.878+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6dc0043e0 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.879+0000 7ff6daffd700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 0x7ff6c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:37.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.879+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7ff6dc022070 con 0x7ff6e40cfaf0 2023-06-09T11:34:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.879+0000 7ff6ea19b700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 0x7ff6c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:37.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.880+0000 7ff6ea19b700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 0x7ff6c806d1c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0000f60 tx=0x7ff6e0007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:37.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:37.883+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff6dc056140 con 0x7ff6e40cfaf0 2023-06-09T11:34:38.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.134+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e4074cf0 con 0x7ff6e40cfaf0 2023-06-09T11:34:38.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.135+0000 7ff6daffd700 1 -- 172.21.15.7:0/705206678 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7ff6dc059760 con 0x7ff6e40cfaf0 2023-06-09T11:34:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:38.141 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:38.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 msgr2=0x7ff6c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 0x7ff6c806d1c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0000f60 tx=0x7ff6e0007e70 comp rx=0 tx=0).stop 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 msgr2=0x7ff6e415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff6e408cda0 tx=0x7ff6dc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 shutdown_connections 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6e40cfaf0 0x7ff6e415c140 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:38.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6c806ad10 0x7ff6c806d1c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6e40c9090 0x7ff6e4152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 --2- 172.21.15.7:0/705206678 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6e40c9fe0 0x7ff6e40c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.137+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 >> 172.21.15.7:0/705206678 conn(0x7ff6e4070af0 msgr2=0x7ff6e40c7180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.138+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 shutdown_connections 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:38.138+0000 7ff6ec3ff700 1 -- 172.21.15.7:0/705206678 wait complete. 2023-06-09T11:34:38.145 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:38.161 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:40.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:40.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.366+0000 7f17463ff700 1 Processor -- start 2023-06-09T11:34:40.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.366+0000 7f17463ff700 1 -- start start 2023-06-09T11:34:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.367+0000 7f17463ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1740073d80 0x7f174006b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.367+0000 7f17463ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740074150 0x7f174006a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.367+0000 7f17463ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174006a4d0 con 0x7f17400bf5d0 2023-06-09T11:34:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.367+0000 7f17463ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1740073a10 con 0x7f1740073d80 2023-06-09T11:34:40.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.367+0000 7f17463ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17400cc280 con 0x7f1740074150 2023-06-09T11:34:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1740073d80 0x7f174006b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1740073d80 0x7f174006b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50118/0 (socket says 172.21.15.7:50118) 2023-06-09T11:34:40.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 -- 172.21.15.7:0/1877980142 learned_addr learned my addr 172.21.15.7:0/1877980142 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f174499c700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740074150 0x7f174006a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 -- 172.21.15.7:0/1877980142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740074150 msgr2=0x7f174006a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740074150 0x7f174006a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 -- 172.21.15.7:0/1877980142 >> v1:172.21.15.117:6789/0 conn(0x7f17400bf5d0 legacy=0x7f17401534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 -- 172.21.15.7:0/1877980142 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400c8700 con 0x7f1740073d80 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.368+0000 7f173ffff700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1740073d80 0x7f174006b6b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f17300098f0 tx=0x7f1730009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239e7486e16f2a36 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:40.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f173effd700 1 -- 172.21.15.7:0/1877980142 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730011830 con 0x7f1740073d80 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f173effd700 1 -- 172.21.15.7:0/1877980142 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1730011990 con 0x7f1740073d80 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f173effd700 1 -- 172.21.15.7:0/1877980142 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1730011d20 con 0x7f1740073d80 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 -- 172.21.15.7:0/1877980142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1740073d80 msgr2=0x7f174006b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1740073d80 0x7f174006b6b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f17300098f0 tx=0x7f1730009c00 comp rx=0 tx=0).stop 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 -- 172.21.15.7:0/1877980142 shutdown_connections 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740074150 0x7f174006a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 --2- 172.21.15.7:0/1877980142 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1740073d80 0x7f174006b6b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 -- 172.21.15.7:0/1877980142 >> 172.21.15.7:0/1877980142 conn(0x7f174006d150 msgr2=0x7f174006d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 -- 172.21.15.7:0/1877980142 shutdown_connections 2023-06-09T11:34:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.369+0000 7f17463ff700 1 -- 172.21.15.7:0/1877980142 wait complete. 2023-06-09T11:34:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.372+0000 7f17463ff700 1 Processor -- start 2023-06-09T11:34:40.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.372+0000 7f17463ff700 1 -- start start 2023-06-09T11:34:40.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740073d80 0x7f1740151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 0x7f1740151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401f2d20 con 0x7f17400bf5d0 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17401f20a0 con 0x7f1740074150 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f17463ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17401f2380 con 0x7f1740073d80 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f174499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.373+0000 7f174499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50126/0 (socket says 172.21.15.7:50126) 2023-06-09T11:34:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 -- 172.21.15.7:0/2013704916 learned_addr learned my addr 172.21.15.7:0/2013704916 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f173ffff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740073d80 0x7f1740151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f173f7fe700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 0x7f1740151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 -- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740073d80 msgr2=0x7f1740151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740073d80 0x7f1740151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 -- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 msgr2=0x7f1740151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 0x7f1740151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 -- 172.21.15.7:0/2013704916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1730009550 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f173f7fe700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 0x7f1740151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f174499c700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f173800efc0 tx=0x7f173800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738009ea0 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173800f040 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.374+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17400cc7b0 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.375+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1738014480 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.375+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17400cc4d0 con 0x7f17400bf5d0 2023-06-09T11:34:40.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.376+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17400cca90 con 0x7f17400bf5d0 2023-06-09T11:34:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.377+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f17380145e0 con 0x7f17400bf5d0 2023-06-09T11:34:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.378+0000 7f173cff9700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:40.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.378+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f173808a200 con 0x7f17400bf5d0 2023-06-09T11:34:40.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.378+0000 7f173ffff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:40.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.378+0000 7f173ffff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 0x7f172c06d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1730009c40 tx=0x7f1730008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:40.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.383+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1738054df0 con 0x7f17400bf5d0 2023-06-09T11:34:40.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.635+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1740066c40 con 0x7f17400bf5d0 2023-06-09T11:34:40.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.636+0000 7f173cff9700 1 -- 172.21.15.7:0/2013704916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f1740066c40 con 0x7f17400bf5d0 2023-06-09T11:34:40.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:40.642 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.638+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 msgr2=0x7f172c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.638+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 0x7f172c06d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1730009c40 tx=0x7f1730008030 comp rx=0 tx=0).stop 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.638+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 msgr2=0x7f17400c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.638+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f173800efc0 tx=0x7f173800c3a0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 shutdown_connections 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17400bf5d0 0x7f17400c4760 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f172c06ace0 0x7f172c06d190 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1740073d80 0x7f1740151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 --2- 172.21.15.7:0/2013704916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1740074150 0x7f1740151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 >> 172.21.15.7:0/2013704916 conn(0x7f174006d150 msgr2=0x7f17400cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 shutdown_connections 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:40.639+0000 7f17463ff700 1 -- 172.21.15.7:0/2013704916 wait complete. 2023-06-09T11:34:40.646 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:40.662 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:42.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:42.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.868+0000 7f8ce47ff700 1 Processor -- start 2023-06-09T11:34:42.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.868+0000 7f8ce47ff700 1 -- start start 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.869+0000 7f8ce47ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce47ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc0c5a80 con 0x7f8cdc153880 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce47ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cdc15a500 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce47ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0c5f10 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce259b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8cdc153880 0x7f8cdc15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59582/0 (socket says 172.21.15.7:59582) 2023-06-09T11:34:42.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce259b700 1 -- 172.21.15.7:0/1835030383 learned_addr learned my addr 172.21.15.7:0/1835030383 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:42.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce2d9c700 1 --2- 172.21.15.7:0/1835030383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357153362 0 0) 0x7f8cdc0c5a80 con 0x7f8cdc153880 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc80035b0 con 0x7f8cdc153880 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306512099 0 0) 0x7f8ccc001110 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/1835030383 >> v1:172.21.15.117:6789/0 conn(0x7f8cdc15b4a0 legacy=0x7f8cdc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/1835030383 >> v1:172.21.15.7:6789/0 conn(0x7f8cdc153880 legacy=0x7f8cdc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:42.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.870+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/1835030383 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc1596e0 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce2d9c700 1 --2- 172.21.15.7:0/1835030383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc0c5540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009dc0 tx=0x7f8cd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2227b1daaf7cdefe server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400e070 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd400bb20 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce1599700 1 -- 172.21.15.7:0/1835030383 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400bed0 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce47ff700 1 -- 172.21.15.7:0/1835030383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 msgr2=0x7f8cdc0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.871+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/1835030383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc0c5540 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009dc0 tx=0x7f8cd400b330 comp rx=0 tx=0).stop 2023-06-09T11:34:42.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.872+0000 7f8ce47ff700 1 -- 172.21.15.7:0/1835030383 shutdown_connections 2023-06-09T11:34:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.872+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/1835030383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc0c5540 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.872+0000 7f8ce47ff700 1 -- 172.21.15.7:0/1835030383 >> 172.21.15.7:0/1835030383 conn(0x7f8cdc0737f0 msgr2=0x7f8cdc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.872+0000 7f8ce47ff700 1 -- 172.21.15.7:0/1835030383 shutdown_connections 2023-06-09T11:34:42.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.872+0000 7f8ce47ff700 1 -- 172.21.15.7:0/1835030383 wait complete. 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.873+0000 7f8ce47ff700 1 Processor -- start 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.873+0000 7f8ce47ff700 1 -- start start 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc1e3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cdc153880 0x7f8cdc1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b43c0 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc0b5570 con 0x7f8cdc153880 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.874+0000 7f8ce47ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1e3980 con 0x7f8cdc0c9f40 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50130/0 (socket says 172.21.15.7:50130) 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/538499334 learned_addr learned my addr 172.21.15.7:0/538499334 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce259b700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc1e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce1d9a700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cdc153880 0x7f8cdc1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:42.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 msgr2=0x7f8cdc1e3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc1e3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cdc153880 msgr2=0x7f8cdc1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cdc153880 0x7f8cdc1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 -- 172.21.15.7:0/538499334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4009a70 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.875+0000 7f8ce2d9c700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009f30 tx=0x7f8cd4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.876+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd401b810 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.876+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd4020070 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.876+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cdc159c60 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.876+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd401bbb0 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.876+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc159980 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.877+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc159f40 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.878+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cd401bd10 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.879+0000 7f8cd2ffd700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 0x7f8cc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.879+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f8cd4022070 con 0x7f8cdc15b4a0 2023-06-09T11:34:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.879+0000 7f8ce259b700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 0x7f8cc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:42.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.880+0000 7f8ce259b700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 0x7f8cc006d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8007fd0 tx=0x7f8cd8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:42.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:42.883+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8cd40568d0 con 0x7f8cdc15b4a0 2023-06-09T11:34:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.139+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cdc1e3ff0 con 0x7f8cdc15b4a0 2023-06-09T11:34:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.139+0000 7f8cd2ffd700 1 -- 172.21.15.7:0/538499334 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f8cdc1e3ff0 con 0x7f8cdc15b4a0 2023-06-09T11:34:43.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:43.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.141+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 msgr2=0x7f8cc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.141+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 0x7f8cc006d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8007fd0 tx=0x7f8cd8009040 comp rx=0 tx=0).stop 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 msgr2=0x7f8cdc14cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4009f30 tx=0x7f8cd4004ad0 comp rx=0 tx=0).stop 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 shutdown_connections 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cdc15b4a0 0x7f8cdc14cfd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:43.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cc006ace0 0x7f8cc006d190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cdc0c9f40 0x7f8cdc1e3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 --2- 172.21.15.7:0/538499334 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cdc153880 0x7f8cdc1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 >> 172.21.15.7:0/538499334 conn(0x7f8cdc0737f0 msgr2=0x7f8cdc0cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 shutdown_connections 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:43.142+0000 7f8ce47ff700 1 -- 172.21.15.7:0/538499334 wait complete. 2023-06-09T11:34:43.149 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:43.165 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:45.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.372+0000 7f63759ff700 1 Processor -- start 2023-06-09T11:34:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.372+0000 7f63759ff700 1 -- start start 2023-06-09T11:34:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.373+0000 7f63759ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 0x7f63701599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f63759ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f63759ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63700c0bf0 con 0x7f63700c0140 2023-06-09T11:34:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f63759ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6370070110 con 0x7f63700c1bc0 2023-06-09T11:34:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f63759ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6370071300 con 0x7f63700c1f90 2023-06-09T11:34:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f636f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f636effd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 0x7f63701599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f636f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:56016/0 (socket says 172.21.15.7:56016) 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f636f7fe700 1 -- 172.21.15.7:0/3990109347 learned_addr learned my addr 172.21.15.7:0/3990109347 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.374+0000 7f636effd700 1 --2- 172.21.15.7:0/3990109347 >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 0x7f63701599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:53214/0 (socket says 172.21.15.7:53214) 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636f7fe700 1 -- 172.21.15.7:0/3990109347 >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 msgr2=0x7f63701599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636f7fe700 1 --2- 172.21.15.7:0/3990109347 >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 0x7f63701599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636f7fe700 1 -- 172.21.15.7:0/3990109347 >> v1:172.21.15.117:6789/0 conn(0x7f63700c0140 legacy=0x7f63701501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636f7fe700 1 -- 172.21.15.7:0/3990109347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700702f0 con 0x7f63700c1f90 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636f7fe700 1 --2- 172.21.15.7:0/3990109347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e50 tx=0x7f636800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8258d4f922ab4c39 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636dffb700 1 -- 172.21.15.7:0/3990109347 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800e070 con 0x7f63700c1f90 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636dffb700 1 -- 172.21.15.7:0/3990109347 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636800b910 con 0x7f63700c1f90 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.375+0000 7f636dffb700 1 -- 172.21.15.7:0/3990109347 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636800bcc0 con 0x7f63700c1f90 2023-06-09T11:34:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 -- 172.21.15.7:0/3990109347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 msgr2=0x7f6370159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 --2- 172.21.15.7:0/3990109347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6368009e50 tx=0x7f636800b330 comp rx=0 tx=0).stop 2023-06-09T11:34:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 -- 172.21.15.7:0/3990109347 shutdown_connections 2023-06-09T11:34:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 --2- 172.21.15.7:0/3990109347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f6370159ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 --2- 172.21.15.7:0/3990109347 >> v2:172.21.15.117:3300/0 conn(0x7f63700c1bc0 0x7f63701599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 -- 172.21.15.7:0/3990109347 >> 172.21.15.7:0/3990109347 conn(0x7f637006a4d0 msgr2=0x7f637006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 -- 172.21.15.7:0/3990109347 shutdown_connections 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.376+0000 7f63759ff700 1 -- 172.21.15.7:0/3990109347 wait complete. 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.378+0000 7f63759ff700 1 Processor -- start 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.378+0000 7f63759ff700 1 -- start start 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63700c1bc0 0x7f63701ed060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f63701ed5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63701f05d0 con 0x7f63700c0140 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c54d0 con 0x7f63700c1bc0 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f63759ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63700c2f20 con 0x7f63700c1f90 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51868/0 (socket says 172.21.15.7:51868) 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 -- 172.21.15.7:0/1526704869 learned_addr learned my addr 172.21.15.7:0/1526704869 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636f7fe700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f63701ed5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636e7fc700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63700c1bc0 0x7f63701ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 -- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 msgr2=0x7f63701ed5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f63701ed5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 -- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63700c1bc0 msgr2=0x7f63701ed060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.379+0000 7f636effd700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63700c1bc0 0x7f63701ed060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f636effd700 1 -- 172.21.15.7:0/1526704869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009b30 con 0x7f63700c0140 2023-06-09T11:34:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f636effd700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f636000e9b0 tx=0x7f636000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360009710 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636000ee50 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636000ca50 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63700c1170 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.380+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63700c0e90 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.381+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63700c1450 con 0x7f63700c0140 2023-06-09T11:34:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.382+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f636000cbb0 con 0x7f63700c0140 2023-06-09T11:34:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.384+0000 7f6357fff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 0x7f635c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.384+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7f6360089920 con 0x7f63700c0140 2023-06-09T11:34:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.384+0000 7f636e7fc700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 0x7f635c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.384+0000 7f636e7fc700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 0x7f635c06d1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f63701e3340 tx=0x7f6364009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.388+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6360057fa0 con 0x7f63700c0140 2023-06-09T11:34:45.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.641+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63700c6610 con 0x7f63700c0140 2023-06-09T11:34:45.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.642+0000 7f6357fff700 1 -- 172.21.15.7:0/1526704869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f636000ce60 con 0x7f63700c0140 2023-06-09T11:34:45.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:45.648 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.644+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 msgr2=0x7f635c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.644+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 0x7f635c06d1c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f63701e3340 tx=0x7f6364009040 comp rx=0 tx=0).stop 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.644+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 msgr2=0x7f6370158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.644+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f636000e9b0 tx=0x7f636000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 shutdown_connections 2023-06-09T11:34:45.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f635c06ad10 0x7f635c06d1c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63700c1f90 0x7f63701ed5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63700c0140 0x7f6370158c30 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 --2- 172.21.15.7:0/1526704869 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63700c1bc0 0x7f63701ed060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 >> 172.21.15.7:0/1526704869 conn(0x7f637006a4d0 msgr2=0x7f637014ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 shutdown_connections 2023-06-09T11:34:45.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:45.645+0000 7f63759ff700 1 -- 172.21.15.7:0/1526704869 wait complete. 2023-06-09T11:34:45.653 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:45.669 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:47.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:47.206 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:34:47.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:47.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.873+0000 7fb041bff700 1 Processor -- start 2023-06-09T11:34:47.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.874+0000 7fb041bff700 1 -- start start 2023-06-09T11:34:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.875+0000 7fb041bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb03c1593d0 0x7fb03c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.875+0000 7fb041bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 0x7fb03c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.875+0000 7fb041bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03c0cc140 con 0x7fb03c071910 2023-06-09T11:34:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.875+0000 7fb041bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb03c0714f0 con 0x7fb03c1593d0 2023-06-09T11:34:47.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.875+0000 7fb041bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c0c1830 con 0x7fb03c0c08d0 2023-06-09T11:34:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb03c071910 0x7fb03c071d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58364/0 (socket says 172.21.15.7:58364) 2023-06-09T11:34:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03bfff700 1 -- 172.21.15.7:0/3860552614 learned_addr learned my addr 172.21.15.7:0/3860552614 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03affd700 1 --2- 172.21.15.7:0/3860552614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 0x7fb03c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03b7fe700 1 --2- 172.21.15.7:0/3860552614 >> v2:172.21.15.117:3300/0 conn(0x7fb03c1593d0 0x7fb03c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2865020256 0 0) 0x7fb03c0cc140 con 0x7fb03c071910 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0240035b0 con 0x7fb03c071910 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03affd700 1 -- 172.21.15.7:0/3860552614 >> v2:172.21.15.117:3300/0 conn(0x7fb03c1593d0 msgr2=0x7fb03c1597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03affd700 1 --2- 172.21.15.7:0/3860552614 >> v2:172.21.15.117:3300/0 conn(0x7fb03c1593d0 0x7fb03c1597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2590887063 0 0) 0x7fb0240035b0 con 0x7fb03c071910 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03affd700 1 -- 172.21.15.7:0/3860552614 >> v1:172.21.15.7:6789/0 conn(0x7fb03c071910 legacy=0x7fb03c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:47.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.876+0000 7fb03affd700 1 -- 172.21.15.7:0/3860552614 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0c0560 con 0x7fb03c0c08d0 2023-06-09T11:34:47.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb03affd700 1 --2- 172.21.15.7:0/3860552614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 0x7fb03c159ce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb030009840 tx=0x7fb030009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2e92a486845366 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:47.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340019f0 con 0x7fb03c0c08d0 2023-06-09T11:34:47.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb030004030 con 0x7fb03c0c08d0 2023-06-09T11:34:47.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb03a7fc700 1 -- 172.21.15.7:0/3860552614 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0300043c0 con 0x7fb03c0c08d0 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb041bff700 1 -- 172.21.15.7:0/3860552614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 msgr2=0x7fb03c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.877+0000 7fb041bff700 1 --2- 172.21.15.7:0/3860552614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 0x7fb03c159ce0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb030009840 tx=0x7fb030009b50 comp rx=0 tx=0).stop 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 -- 172.21.15.7:0/3860552614 shutdown_connections 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 --2- 172.21.15.7:0/3860552614 >> v2:172.21.15.117:3300/0 conn(0x7fb03c1593d0 0x7fb03c1597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 --2- 172.21.15.7:0/3860552614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c0c08d0 0x7fb03c159ce0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:47.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 -- 172.21.15.7:0/3860552614 >> 172.21.15.7:0/3860552614 conn(0x7fb03c06a4d0 msgr2=0x7fb03c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:47.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 -- 172.21.15.7:0/3860552614 shutdown_connections 2023-06-09T11:34:47.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.878+0000 7fb041bff700 1 -- 172.21.15.7:0/3860552614 wait complete. 2023-06-09T11:34:47.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.879+0000 7fb041bff700 1 Processor -- start 2023-06-09T11:34:47.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.879+0000 7fb041bff700 1 -- start start 2023-06-09T11:34:47.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0c08d0 0x7fb03c0c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 0x7fb03c0c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c1ef870 con 0x7fb03c071910 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c1eea10 con 0x7fb03c0c08d0 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.880+0000 7fb041bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c1eecf0 con 0x7fb03c1593d0 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51870/0 (socket says 172.21.15.7:51870) 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 -- 172.21.15.7:0/1423057084 learned_addr learned my addr 172.21.15.7:0/1423057084 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03affd700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0c08d0 0x7fb03c0c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03bfff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 0x7fb03c0c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 -- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 msgr2=0x7fb03c0c62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 0x7fb03c0c62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 -- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0c08d0 msgr2=0x7fb03c0c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:47.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0c08d0 0x7fb03c0c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 -- 172.21.15.7:0/1423057084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0300094a0 con 0x7fb03c071910 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03bfff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 0x7fb03c0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.881+0000 7fb03b7fe700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e850 tx=0x7fb02c00eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.882+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c013680 con 0x7fb03c071910 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.882+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c013800 con 0x7fb03c071910 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.882+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c0c1d60 con 0x7fb03c071910 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.882+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c013bb0 con 0x7fb03c071910 2023-06-09T11:34:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.882+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c0c1a80 con 0x7fb03c071910 2023-06-09T11:34:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.883+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c0c2040 con 0x7fb03c071910 2023-06-09T11:34:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.884+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb02c013d10 con 0x7fb03c071910 2023-06-09T11:34:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.885+0000 7fb023fff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 0x7fb02806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.885+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7152+0+0 (secure 0 0 0) 0x7fb02c089be0 con 0x7fb03c071910 2023-06-09T11:34:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.885+0000 7fb03affd700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 0x7fb02806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:47.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.886+0000 7fb03affd700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 0x7fb02806d1c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb0300090b0 tx=0x7fb030008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:47.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:47.889+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb02c0547d0 con 0x7fb03c071910 2023-06-09T11:34:48.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.139+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb03c074cf0 con 0x7fb03c071910 2023-06-09T11:34:48.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.140+0000 7fb023fff700 1 -- 172.21.15.7:0/1423057084 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fb02c057df0 con 0x7fb03c071910 2023-06-09T11:34:48.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:48.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":106,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5741,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":100}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:18.619634+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":61,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5630,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5630":{"gid":5630,"name":"d","rank":0,"incarnation":94,"state":"up:active","state_seq":40,"addr":"172.21.15.117:6835/3162263716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3162263716},{"type":"v1","addr":"172.21.15.117:6835","nonce":3162263716}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:48.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.142+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 msgr2=0x7fb02806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.142+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 0x7fb02806d1c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb0300090b0 tx=0x7fb030008030 comp rx=0 tx=0).stop 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.142+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 msgr2=0x7fb03c0c2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.142+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e850 tx=0x7fb02c00eb60 comp rx=0 tx=0).stop 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.142+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 shutdown_connections 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb02806ad10 0x7fb02806d1c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:48.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb03c1593d0 0x7fb03c0c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb03c071910 0x7fb03c0c2790 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 --2- 172.21.15.7:0/1423057084 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb03c0c08d0 0x7fb03c0c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 >> 172.21.15.7:0/1423057084 conn(0x7fb03c06a4d0 msgr2=0x7fb03c146730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 shutdown_connections 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:48.143+0000 7fb041bff700 1 -- 172.21.15.7:0/1423057084 wait complete. 2023-06-09T11:34:48.150 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 106 2023-06-09T11:34:48.166 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:34:50.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.375+0000 7f37b3bff700 1 Processor -- start 2023-06-09T11:34:50.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.375+0000 7f37b3bff700 1 -- start start 2023-06-09T11:34:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.377+0000 7f37b3bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 0x7f37ac1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.377+0000 7f37b3bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 0x7f37ac159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.377+0000 7f37b3bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac0cc140 con 0x7f37ac071910 2023-06-09T11:34:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.377+0000 7f37b3bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac0714f0 con 0x7f37ac0c08d0 2023-06-09T11:34:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.377+0000 7f37b3bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0c1830 con 0x7f37ac1593d0 2023-06-09T11:34:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b219c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f37ac071910 0x7f37ac071d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58366/0 (socket says 172.21.15.7:58366) 2023-06-09T11:34:50.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b219c700 1 -- 172.21.15.7:0/1548883392 learned_addr learned my addr 172.21.15.7:0/1548883392 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b119a700 1 --2- 172.21.15.7:0/1548883392 >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 0x7f37ac159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b199b700 1 --2- 172.21.15.7:0/1548883392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 0x7f37ac1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337299912 0 0) 0x7f37ac0cc140 con 0x7f37ac071910 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37980035b0 con 0x7f37ac071910 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3804410263 0 0) 0x7f37980035b0 con 0x7f37ac071910 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b199b700 1 -- 172.21.15.7:0/1548883392 >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 msgr2=0x7f37ac159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:50.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b199b700 1 --2- 172.21.15.7:0/1548883392 >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 0x7f37ac159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b199b700 1 -- 172.21.15.7:0/1548883392 >> v1:172.21.15.7:6789/0 conn(0x7f37ac071910 legacy=0x7f37ac071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b199b700 1 -- 172.21.15.7:0/1548883392 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac0c0560 con 0x7f37ac1593d0 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.378+0000 7f37b119a700 1 --2- 172.21.15.7:0/1548883392 >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 0x7f37ac159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b199b700 1 --2- 172.21.15.7:0/1548883392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 0x7f37ac1597a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009840 tx=0x7f37a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f774d1504f065c6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40019f0 con 0x7f37ac1593d0 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a8004030 con 0x7f37ac1593d0 2023-06-09T11:34:50.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b0999700 1 -- 172.21.15.7:0/1548883392 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a80043c0 con 0x7f37ac1593d0 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b3bff700 1 -- 172.21.15.7:0/1548883392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 msgr2=0x7f37ac1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.379+0000 7f37b3bff700 1 --2- 172.21.15.7:0/1548883392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 0x7f37ac1597a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009840 tx=0x7f37a8009b50 comp rx=0 tx=0).stop 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 -- 172.21.15.7:0/1548883392 shutdown_connections 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 --2- 172.21.15.7:0/1548883392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac1593d0 0x7f37ac1597a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 --2- 172.21.15.7:0/1548883392 >> v2:172.21.15.117:3300/0 conn(0x7f37ac0c08d0 0x7f37ac159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 -- 172.21.15.7:0/1548883392 >> 172.21.15.7:0/1548883392 conn(0x7f37ac06a4d0 msgr2=0x7f37ac06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 -- 172.21.15.7:0/1548883392 shutdown_connections 2023-06-09T11:34:50.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.380+0000 7f37b3bff700 1 -- 172.21.15.7:0/1548883392 wait complete. 2023-06-09T11:34:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.382+0000 7f37b3bff700 1 Processor -- start 2023-06-09T11:34:50.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.382+0000 7f37b3bff700 1 -- start start 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.383+0000 7f37b3bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac071910 0x7f37ac0c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.383+0000 7f37b3bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b3bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ac1593d0 0x7f37ac157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b3bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac156de0 con 0x7f37ac0c08d0 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b3bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac1557c0 con 0x7f37ac1593d0 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b3bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac155a80 con 0x7f37ac071910 2023-06-09T11:34:50.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51874/0 (socket says 172.21.15.7:51874) 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 -- 172.21.15.7:0/4173602916 learned_addr learned my addr 172.21.15.7:0/4173602916 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b199b700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac071910 0x7f37ac0c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b219c700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ac1593d0 0x7f37ac157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 -- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac071910 msgr2=0x7f37ac0c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac071910 0x7f37ac0c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 -- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ac1593d0 msgr2=0x7f37ac157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ac1593d0 0x7f37ac157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 -- 172.21.15.7:0/4173602916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c0094a0 con 0x7f37ac0c08d0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.384+0000 7f37b119a700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f379c00ebe0 tx=0x7f379c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.385+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c009800 con 0x7f37ac0c08d0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.385+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c00c910 con 0x7f37ac0c08d0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.385+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a80094a0 con 0x7f37ac0c08d0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.385+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c00cc80 con 0x7f37ac0c08d0 2023-06-09T11:34:50.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.385+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac0c1d60 con 0x7f37ac0c08d0 2023-06-09T11:34:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.386+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac0c1a80 con 0x7f37ac0c08d0 2023-06-09T11:34:50.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.387+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f379c013470 con 0x7f37ac0c08d0 2023-06-09T11:34:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.388+0000 7f37a27fc700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 0x7f379006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.388+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f379c089b20 con 0x7f37ac0c08d0 2023-06-09T11:34:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.388+0000 7f37b199b700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 0x7f379006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.389+0000 7f37b199b700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 0x7f379006d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f37a800a010 tx=0x7f37a8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.392+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f379c058120 con 0x7f37ac0c08d0 2023-06-09T11:34:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.650+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37ac0c2040 con 0x7f37ac0c08d0 2023-06-09T11:34:50.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.650+0000 7f37a27fc700 1 -- 172.21.15.7:0/4173602916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f379c057cb0 con 0x7f37ac0c08d0 2023-06-09T11:34:50.656 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:50.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:49.439449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 msgr2=0x7f379006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 0x7f379006d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f37a800a010 tx=0x7f37a8008030 comp rx=0 tx=0).stop 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 msgr2=0x7f37ac157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f379c00ebe0 tx=0x7f379c00eef0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 shutdown_connections 2023-06-09T11:34:50.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37ac1593d0 0x7f37ac157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f379006ace0 0x7f379006d190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f37ac071910 0x7f37ac0c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 --2- 172.21.15.7:0/4173602916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37ac0c08d0 0x7f37ac157830 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 >> 172.21.15.7:0/4173602916 conn(0x7f37ac06a4d0 msgr2=0x7f37ac06dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 shutdown_connections 2023-06-09T11:34:50.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.653+0000 7f37b3bff700 1 -- 172.21.15.7:0/4173602916 wait complete. 2023-06-09T11:34:50.661 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 107 2023-06-09T11:34:50.677 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:34:50.677 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:34:50.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:50.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.870+0000 7f0cfedff700 1 Processor -- start 2023-06-09T11:34:50.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.870+0000 7f0cfedff700 1 -- start start 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfedff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0cf80cb9a0 0x7f0cf80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfedff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf8159b20 con 0x7f0cf815cd40 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfedff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf80cac80 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfedff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf81597d0 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfd39c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0cf815cd40 0x7f0cf80cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58376/0 (socket says 172.21.15.7:58376) 2023-06-09T11:34:50.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cfd39c700 1 -- 172.21.15.7:0/1085817359 learned_addr learned my addr 172.21.15.7:0/1085817359 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:50.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/1085817359 >> v2:172.21.15.117:3300/0 conn(0x7f0cf80cb9a0 0x7f0cf80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf77fe700 1 -- 172.21.15.7:0/1085817359 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853286940 0 0) 0x7f0cf8159b20 con 0x7f0cf815cd40 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf77fe700 1 -- 172.21.15.7:0/1085817359 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce80035b0 con 0x7f0cf815cd40 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1085817359 >> v1:172.21.15.117:6789/0 conn(0x7f0cf80cb5d0 legacy=0x7f0cf8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1085817359 >> v1:172.21.15.7:6789/0 conn(0x7f0cf815cd40 legacy=0x7f0cf80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.872+0000 7f0cf7fff700 1 -- 172.21.15.7:0/1085817359 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8071ab0 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/1085817359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf80cb9a0 0x7f0cf80b52b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009840 tx=0x7f0ce4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5eec43da5985a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cf77fe700 1 -- 172.21.15.7:0/1085817359 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4011830 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cf77fe700 1 -- 172.21.15.7:0/1085817359 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4011990 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cf77fe700 1 -- 172.21.15.7:0/1085817359 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4011d20 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cfedff700 1 -- 172.21.15.7:0/1085817359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf80cb9a0 msgr2=0x7f0cf80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.873+0000 7f0cfedff700 1 --2- 172.21.15.7:0/1085817359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf80cb9a0 0x7f0cf80b52b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4009840 tx=0x7f0ce4009b50 comp rx=0 tx=0).stop 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.874+0000 7f0cfedff700 1 -- 172.21.15.7:0/1085817359 shutdown_connections 2023-06-09T11:34:50.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.874+0000 7f0cfedff700 1 --2- 172.21.15.7:0/1085817359 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf80cb9a0 0x7f0cf80b52b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.874+0000 7f0cfedff700 1 -- 172.21.15.7:0/1085817359 >> 172.21.15.7:0/1085817359 conn(0x7f0cf8070af0 msgr2=0x7f0cf8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.874+0000 7f0cfedff700 1 -- 172.21.15.7:0/1085817359 shutdown_connections 2023-06-09T11:34:50.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.874+0000 7f0cfedff700 1 -- 172.21.15.7:0/1085817359 wait complete. 2023-06-09T11:34:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.875+0000 7f0cfedff700 1 Processor -- start 2023-06-09T11:34:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.876+0000 7f0cfedff700 1 -- start start 2023-06-09T11:34:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.876+0000 7f0cfedff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfedff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80cb9a0 0x7f0cf80c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfedff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf815cd40 0x7f0cf80c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfedff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf81f23d0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfedff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf815da80 con 0x7f0cf815cd40 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfedff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf80c3660 con 0x7f0cf80cb9a0 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51890/0 (socket says 172.21.15.7:51890) 2023-06-09T11:34:50.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 -- 172.21.15.7:0/269163658 learned_addr learned my addr 172.21.15.7:0/269163658 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80cb9a0 0x7f0cf80c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfd39c700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf815cd40 0x7f0cf80c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 -- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80cb9a0 msgr2=0x7f0cf80c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80cb9a0 0x7f0cf80c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 -- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf815cd40 msgr2=0x7f0cf80c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf815cd40 0x7f0cf80c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:50.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 -- 172.21.15.7:0/269163658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce40094a0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.877+0000 7f0cfcb9b700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c4e0 tx=0x7f0cec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.878+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0052d0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.878+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cec005430 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.878+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cec0038c0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.878+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf815d4f0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.878+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf815d210 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.879+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf815d7d0 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.880+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0cec003a20 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.882+0000 7f0cf57fa700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 0x7f0cdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:50.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.882+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f0cec088d60 con 0x7f0cf80cb5d0 2023-06-09T11:34:50.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.882+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 0x7f0cdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:50.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.883+0000 7f0cf7fff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 0x7f0cdc06d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80cbd70 tx=0x7f0ce4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:50.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:50.886+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0cec0538d0 con 0x7f0cf80cb5d0 2023-06-09T11:34:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.140+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cf81f26b0 con 0x7f0cf80cb5d0 2023-06-09T11:34:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.140+0000 7f0cf57fa700 1 -- 172.21.15.7:0/269163658 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f0cec056ef0 con 0x7f0cf80cb5d0 2023-06-09T11:34:51.147 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:51.147 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:49.439449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 msgr2=0x7f0cdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 0x7f0cdc06d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80cbd70 tx=0x7f0ce4008030 comp rx=0 tx=0).stop 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 msgr2=0x7f0cf81f2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0cec00c4e0 tx=0x7f0cec00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 shutdown_connections 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0cf815cd40 0x7f0cf80c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0cf80cb5d0 0x7f0cf81f2860 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0cdc06ace0 0x7f0cdc06d190 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 --2- 172.21.15.7:0/269163658 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0cf80cb9a0 0x7f0cf80c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:51.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 >> 172.21.15.7:0/269163658 conn(0x7f0cf8070af0 msgr2=0x7f0cf8156f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:51.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 shutdown_connections 2023-06-09T11:34:51.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:51.143+0000 7f0cfedff700 1 -- 172.21.15.7:0/269163658 wait complete. 2023-06-09T11:34:51.151 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 107 2023-06-09T11:34:51.167 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 107, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 107, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5741": { "addr": "172.21.15.117:6833/557223025", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 557223025, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 557223025, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5741, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:34:49.439449+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5741, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:34:53.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.374+0000 7f67831ff700 1 Processor -- start 2023-06-09T11:34:53.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.374+0000 7f67831ff700 1 -- start start 2023-06-09T11:34:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.375+0000 7f67831ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f677c0c1bc0 0x7f677c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:53.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.375+0000 7f67831ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0c0bf0 con 0x7f677c0c1f90 2023-06-09T11:34:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.375+0000 7f67831ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677c0702f0 con 0x7f677c0c0140 2023-06-09T11:34:53.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.375+0000 7f67831ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c070110 con 0x7f677c0c1bc0 2023-06-09T11:34:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f678179c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f677c0c1f90 0x7f677c160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45572/0 (socket says 172.21.15.7:45572) 2023-06-09T11:34:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f678179c700 1 -- 172.21.15.7:0/2310454177 learned_addr learned my addr 172.21.15.7:0/2310454177 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f6780f9b700 1 --2- 172.21.15.7:0/2310454177 >> v2:172.21.15.117:3300/0 conn(0x7f677c0c1bc0 0x7f677c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f677b7fe700 1 -- 172.21.15.7:0/2310454177 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845344238 0 0) 0x7f677c0c0bf0 con 0x7f677c0c1f90 2023-06-09T11:34:53.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f677b7fe700 1 -- 172.21.15.7:0/2310454177 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f676c0035b0 con 0x7f677c0c1f90 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f6780f9b700 1 -- 172.21.15.7:0/2310454177 >> v1:172.21.15.117:6789/0 conn(0x7f677c0c0140 legacy=0x7f677c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f6780f9b700 1 -- 172.21.15.7:0/2310454177 >> v1:172.21.15.7:6789/0 conn(0x7f677c0c1f90 legacy=0x7f677c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.376+0000 7f6780f9b700 1 -- 172.21.15.7:0/2310454177 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c0c0e90 con 0x7f677c0c1bc0 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f6780f9b700 1 --2- 172.21.15.7:0/2310454177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c1bc0 0x7f677c1599b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6770009840 tx=0x7f6770009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4bdfc90172c0bec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f677b7fe700 1 -- 172.21.15.7:0/2310454177 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770011830 con 0x7f677c0c1bc0 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f677b7fe700 1 -- 172.21.15.7:0/2310454177 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6770011990 con 0x7f677c0c1bc0 2023-06-09T11:34:53.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f677b7fe700 1 -- 172.21.15.7:0/2310454177 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770011d20 con 0x7f677c0c1bc0 2023-06-09T11:34:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f67831ff700 1 -- 172.21.15.7:0/2310454177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c1bc0 msgr2=0x7f677c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f67831ff700 1 --2- 172.21.15.7:0/2310454177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c1bc0 0x7f677c1599b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6770009840 tx=0x7f6770009b50 comp rx=0 tx=0).stop 2023-06-09T11:34:53.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f67831ff700 1 -- 172.21.15.7:0/2310454177 shutdown_connections 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f67831ff700 1 --2- 172.21.15.7:0/2310454177 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c1bc0 0x7f677c1599b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.377+0000 7f67831ff700 1 -- 172.21.15.7:0/2310454177 >> 172.21.15.7:0/2310454177 conn(0x7f677c06a4d0 msgr2=0x7f677c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.378+0000 7f67831ff700 1 -- 172.21.15.7:0/2310454177 shutdown_connections 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.378+0000 7f67831ff700 1 -- 172.21.15.7:0/2310454177 wait complete. 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.379+0000 7f67831ff700 1 Processor -- start 2023-06-09T11:34:53.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.379+0000 7f67831ff700 1 -- start start 2023-06-09T11:34:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f677c0c1f90 0x7f677c0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c0c4d50 con 0x7f677c0c1bc0 2023-06-09T11:34:53.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c151bd0 con 0x7f677c0c0140 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.380+0000 7f67831ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c151eb0 con 0x7f677c0c1f90 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f677bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:54146/0 (socket says 172.21.15.7:54146) 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 -- 172.21.15.7:0/1264772798 learned_addr learned my addr 172.21.15.7:0/1264772798 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:53.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f677bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51894/0 (socket says 172.21.15.7:51894) 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f678179c700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f677c0c1f90 0x7f677c0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 -- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f677c0c1f90 msgr2=0x7f677c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f677c0c1f90 0x7f677c0b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 -- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 msgr2=0x7f677c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f6780f9b700 1 -- 172.21.15.7:0/1264772798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700094a0 con 0x7f677c0c0140 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.381+0000 7f677bfff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:34:53.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f6780f9b700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fd0 tx=0x7f6770004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:53.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67700036a0 con 0x7f677c0c0140 2023-06-09T11:34:53.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f677001f070 con 0x7f677c0c0140 2023-06-09T11:34:53.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770003a60 con 0x7f677c0c0140 2023-06-09T11:34:53.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c0c1450 con 0x7f677c0c0140 2023-06-09T11:34:53.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.382+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c0c1170 con 0x7f677c0c0140 2023-06-09T11:34:53.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.383+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c0705d0 con 0x7f677c0c0140 2023-06-09T11:34:53.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.384+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6770003bc0 con 0x7f677c0c0140 2023-06-09T11:34:53.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.385+0000 7f67797fa700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 0x7f676006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:53.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.385+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f6770021070 con 0x7f677c0c0140 2023-06-09T11:34:53.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.385+0000 7f677bfff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 0x7f676006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:53.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.386+0000 7f677bfff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 0x7f676006d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f677c1ec6e0 tx=0x7f6768009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:53.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.389+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6770056040 con 0x7f677c0c0140 2023-06-09T11:34:53.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.641+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f677c074cf0 con 0x7f677c0c0140 2023-06-09T11:34:53.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.642+0000 7f67797fa700 1 -- 172.21.15.7:0/1264772798 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f6770015090 con 0x7f677c0c0140 2023-06-09T11:34:53.649 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:53.649 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:49.439449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 msgr2=0x7f676006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 0x7f676006d190 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f677c1ec6e0 tx=0x7f6768009040 comp rx=0 tx=0).stop 2023-06-09T11:34:53.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 msgr2=0x7f677c1effc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6770009fd0 tx=0x7f6770004750 comp rx=0 tx=0).stop 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 shutdown_connections 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f676006ace0 0x7f676006d190 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f677c0c1f90 0x7f677c0b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f677c0c0140 0x7f677c1effc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 --2- 172.21.15.7:0/1264772798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f677c0c1bc0 0x7f677c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:53.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 >> 172.21.15.7:0/1264772798 conn(0x7f677c06a4d0 msgr2=0x7f677c14ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 shutdown_connections 2023-06-09T11:34:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:53.645+0000 7f67831ff700 1 -- 172.21.15.7:0/1264772798 wait complete. 2023-06-09T11:34:53.653 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 107 2023-06-09T11:34:53.669 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:34:55.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:55.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.876+0000 7f8808bff700 1 Processor -- start 2023-06-09T11:34:55.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.876+0000 7f8808bff700 1 -- start start 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.877+0000 7f8808bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f88040cb9a0 0x7f88040b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.877+0000 7f8808bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f880415cd40 0x7f88040cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.877+0000 7f8808bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8804159b20 con 0x7f88040cb5d0 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.877+0000 7f8808bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040cac80 con 0x7f880415cd40 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.877+0000 7f8808bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88041597d0 con 0x7f88040cb9a0 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8802d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f880415cd40 0x7f88040cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f880259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f88040cb5d0 0x7f8804159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45574/0 (socket says 172.21.15.7:45574) 2023-06-09T11:34:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f880259c700 1 -- 172.21.15.7:0/190917692 learned_addr learned my addr 172.21.15.7:0/190917692 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:55.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f87fbfff700 1 --2- 172.21.15.7:0/190917692 >> v2:172.21.15.117:3300/0 conn(0x7f88040cb9a0 0x7f88040b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8801d9b700 1 -- 172.21.15.7:0/190917692 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442443823 0 0) 0x7f8804159b20 con 0x7f88040cb5d0 2023-06-09T11:34:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8801d9b700 1 -- 172.21.15.7:0/190917692 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87f00035b0 con 0x7f88040cb5d0 2023-06-09T11:34:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8802d9d700 1 -- 172.21.15.7:0/190917692 >> v2:172.21.15.117:3300/0 conn(0x7f88040cb9a0 msgr2=0x7f88040b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:34:55.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8802d9d700 1 --2- 172.21.15.7:0/190917692 >> v2:172.21.15.117:3300/0 conn(0x7f88040cb9a0 0x7f88040b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8802d9d700 1 -- 172.21.15.7:0/190917692 >> v1:172.21.15.7:6789/0 conn(0x7f88040cb5d0 legacy=0x7f8804159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.878+0000 7f8802d9d700 1 -- 172.21.15.7:0/190917692 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804071ab0 con 0x7f880415cd40 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8802d9d700 1 --2- 172.21.15.7:0/190917692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f880415cd40 0x7f88040cbea0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f88040c9c60 tx=0x7f87fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74da9c26251a5517 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8801d9b700 1 -- 172.21.15.7:0/190917692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00e070 con 0x7f880415cd40 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8801d9b700 1 -- 172.21.15.7:0/190917692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00bb20 con 0x7f880415cd40 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8801d9b700 1 -- 172.21.15.7:0/190917692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc012430 con 0x7f880415cd40 2023-06-09T11:34:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 -- 172.21.15.7:0/190917692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f880415cd40 msgr2=0x7f88040cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 --2- 172.21.15.7:0/190917692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f880415cd40 0x7f88040cbea0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f88040c9c60 tx=0x7f87fc00b330 comp rx=0 tx=0).stop 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 -- 172.21.15.7:0/190917692 shutdown_connections 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 --2- 172.21.15.7:0/190917692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f880415cd40 0x7f88040cbea0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 --2- 172.21.15.7:0/190917692 >> v2:172.21.15.117:3300/0 conn(0x7f88040cb9a0 0x7f88040b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.879+0000 7f8808bff700 1 -- 172.21.15.7:0/190917692 >> 172.21.15.7:0/190917692 conn(0x7f8804070af0 msgr2=0x7f8804070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.880+0000 7f8808bff700 1 -- 172.21.15.7:0/190917692 shutdown_connections 2023-06-09T11:34:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.880+0000 7f8808bff700 1 -- 172.21.15.7:0/190917692 wait complete. 2023-06-09T11:34:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.881+0000 7f8808bff700 1 Processor -- start 2023-06-09T11:34:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.882+0000 7f8808bff700 1 -- start start 2023-06-09T11:34:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.882+0000 7f8808bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.882+0000 7f8808bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88040cb9a0 0x7f88041f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f8808bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880415cd40 0x7f88041f3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f8808bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040c3ad0 con 0x7f88040cb5d0 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f8808bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88041f6d30 con 0x7f880415cd40 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f8808bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88041f7260 con 0x7f88040cb9a0 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51902/0 (socket says 172.21.15.7:51902) 2023-06-09T11:34:55.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 -- 172.21.15.7:0/3527800647 learned_addr learned my addr 172.21.15.7:0/3527800647 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f87fbfff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88040cb9a0 0x7f88041f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f8802d9d700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880415cd40 0x7f88041f3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 -- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88040cb9a0 msgr2=0x7f88041f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88040cb9a0 0x7f88041f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 -- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880415cd40 msgr2=0x7f88041f3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880415cd40 0x7f88041f3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 -- 172.21.15.7:0/3527800647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87fc009a70 con 0x7f88040cb5d0 2023-06-09T11:34:55.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.883+0000 7f880259c700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c730 tx=0x7f87f400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.884+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4005d70 con 0x7f88040cb5d0 2023-06-09T11:34:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.884+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f40070d0 con 0x7f88040cb5d0 2023-06-09T11:34:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.884+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4007440 con 0x7f88040cb5d0 2023-06-09T11:34:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.884+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880415d4f0 con 0x7f88040cb5d0 2023-06-09T11:34:55.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.884+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880415d210 con 0x7f88040cb5d0 2023-06-09T11:34:55.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.885+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880415d7d0 con 0x7f88040cb5d0 2023-06-09T11:34:55.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.886+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f87f40075a0 con 0x7f88040cb5d0 2023-06-09T11:34:55.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.887+0000 7f87faffd700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 0x7f87e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:55.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.887+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f87f4088c30 con 0x7f88040cb5d0 2023-06-09T11:34:55.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.887+0000 7f87fbfff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 0x7f87e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:55.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.888+0000 7f87fbfff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 0x7f87e406d1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f88040c6550 tx=0x7f87ec009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:55.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:55.892+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87f40538e0 con 0x7f88040cb5d0 2023-06-09T11:34:56.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.145+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8804074cf0 con 0x7f88040cb5d0 2023-06-09T11:34:56.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.146+0000 7f87faffd700 1 -- 172.21.15.7:0/3527800647 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f87f4056f00 con 0x7f88040cb5d0 2023-06-09T11:34:56.152 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:56.152 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":107,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:49.439449+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:56.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 msgr2=0x7f87e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:56.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 0x7f87e406d1c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f88040c6550 tx=0x7f87ec009210 comp rx=0 tx=0).stop 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 msgr2=0x7f88040c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f87f400c730 tx=0x7f87f400ca40 comp rx=0 tx=0).stop 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 shutdown_connections 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f880415cd40 0x7f88041f3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88040cb5d0 0x7f88040c4eb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f87e406ad10 0x7f87e406d1c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:56.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 --2- 172.21.15.7:0/3527800647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f88040cb9a0 0x7f88041f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:56.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.148+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 >> 172.21.15.7:0/3527800647 conn(0x7f8804070af0 msgr2=0x7f88040cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:56.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.149+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 shutdown_connections 2023-06-09T11:34:56.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:56.149+0000 7f8808bff700 1 -- 172.21.15.7:0/3527800647 wait complete. 2023-06-09T11:34:56.156 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 107 2023-06-09T11:34:56.172 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:34:58.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:34:58.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.381+0000 7f47207ff700 1 Processor -- start 2023-06-09T11:34:58.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.382+0000 7f47207ff700 1 -- start start 2023-06-09T11:34:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f47207ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4718071230 0x7f4718158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f47207ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47181580a0 con 0x7f47180c5b40 2023-06-09T11:34:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f47207ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180c6180 con 0x7f4718151510 2023-06-09T11:34:58.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f47207ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47181518e0 con 0x7f4718071230 2023-06-09T11:34:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f471dd9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f47180c5b40 0x7f4718070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45588/0 (socket says 172.21.15.7:45588) 2023-06-09T11:34:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f471dd9a700 1 -- 172.21.15.7:0/4049866994 learned_addr learned my addr 172.21.15.7:0/4049866994 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:34:58.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.383+0000 7f471ed9c700 1 --2- 172.21.15.7:0/4049866994 >> v2:172.21.15.7:3300/0 conn(0x7f4718071230 0x7f4718158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584360686 0 0) 0x7f47181580a0 con 0x7f47180c5b40 2023-06-09T11:34:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47040035b0 con 0x7f47180c5b40 2023-06-09T11:34:58.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371749895 0 0) 0x7f47180c6180 con 0x7f4718151510 2023-06-09T11:34:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47181580a0 con 0x7f4718151510 2023-06-09T11:34:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471ed9c700 1 -- 172.21.15.7:0/4049866994 >> v1:172.21.15.117:6789/0 conn(0x7f4718151510 legacy=0x7f471815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471ed9c700 1 -- 172.21.15.7:0/4049866994 >> v1:172.21.15.7:6789/0 conn(0x7f47180c5b40 legacy=0x7f4718070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471ed9c700 1 -- 172.21.15.7:0/4049866994 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718157490 con 0x7f4718071230 2023-06-09T11:34:58.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471ed9c700 1 --2- 172.21.15.7:0/4049866994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718158870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9579a34e73f8438 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471000e070 con 0x7f4718071230 2023-06-09T11:34:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471000bb20 con 0x7f4718071230 2023-06-09T11:34:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.384+0000 7f471d599700 1 -- 172.21.15.7:0/4049866994 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710012430 con 0x7f4718071230 2023-06-09T11:34:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 -- 172.21.15.7:0/4049866994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 msgr2=0x7f4718158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 --2- 172.21.15.7:0/4049866994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718158870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471000b330 comp rx=0 tx=0).stop 2023-06-09T11:34:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 -- 172.21.15.7:0/4049866994 shutdown_connections 2023-06-09T11:34:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 --2- 172.21.15.7:0/4049866994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718158870 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 -- 172.21.15.7:0/4049866994 >> 172.21.15.7:0/4049866994 conn(0x7f471806a4d0 msgr2=0x7f471806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 -- 172.21.15.7:0/4049866994 shutdown_connections 2023-06-09T11:34:58.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.385+0000 7f47207ff700 1 -- 172.21.15.7:0/4049866994 wait complete. 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.387+0000 7f47207ff700 1 Processor -- start 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.387+0000 7f47207ff700 1 -- start start 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47180c5b40 0x7f4718154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718151510 0x7f4718153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47181ed3a0 con 0x7f4718071230 2023-06-09T11:34:58.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4718159fd0 con 0x7f4718151510 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f47207ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47181edbc0 con 0x7f47180c5b40 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51924/0 (socket says 172.21.15.7:51924) 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 -- 172.21.15.7:0/1152645003 learned_addr learned my addr 172.21.15.7:0/1152645003 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471ed9c700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718151510 0x7f4718153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471dd9a700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47180c5b40 0x7f4718154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:58.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 -- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47180c5b40 msgr2=0x7f4718154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47180c5b40 0x7f4718154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 -- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718151510 msgr2=0x7f4718153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718151510 0x7f4718153640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.388+0000 7f471e59b700 1 -- 172.21.15.7:0/1152645003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710009a70 con 0x7f4718071230 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f471e59b700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f471400c550 tx=0x7f471400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140052d0 con 0x7f4718071230 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714005430 con 0x7f4718071230 2023-06-09T11:34:58.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140038c0 con 0x7f4718071230 2023-06-09T11:34:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4718157a50 con 0x7f4718071230 2023-06-09T11:34:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.389+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4718157770 con 0x7f4718071230 2023-06-09T11:34:58.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.390+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4718157d30 con 0x7f4718071230 2023-06-09T11:34:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.391+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4714003a20 con 0x7f4718071230 2023-06-09T11:34:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.392+0000 7f470effd700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 0x7f46fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:34:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.392+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f4714088f50 con 0x7f4718071230 2023-06-09T11:34:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.392+0000 7f471dd9a700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 0x7f46fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:34:58.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.393+0000 7f471dd9a700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 0x7f46fc06d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f471814d1d0 tx=0x7f4708009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:34:58.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.397+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4714053a30 con 0x7f4718071230 2023-06-09T11:34:58.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.650+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47181edea0 con 0x7f4718071230 2023-06-09T11:34:58.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.651+0000 7f470effd700 1 -- 172.21.15.7:0/1152645003 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f4714017020 con 0x7f4718071230 2023-06-09T11:34:58.657 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:34:58.657 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:57.606339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:resolve","state_seq":20,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:34:58.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 msgr2=0x7f46fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 0x7f46fc06d190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f471814d1d0 tx=0x7f4708009040 comp rx=0 tx=0).stop 2023-06-09T11:34:58.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 msgr2=0x7f4718151c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:34:58.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f471400c550 tx=0x7f471400c860 comp rx=0 tx=0).stop 2023-06-09T11:34:58.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 shutdown_connections 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4718151510 0x7f4718153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4718071230 0x7f4718151c70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f46fc06ace0 0x7f46fc06d190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 --2- 172.21.15.7:0/1152645003 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47180c5b40 0x7f4718154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 >> 172.21.15.7:0/1152645003 conn(0x7f471806a4d0 msgr2=0x7f47180c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 shutdown_connections 2023-06-09T11:34:58.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:34:58.654+0000 7f47207ff700 1 -- 172.21.15.7:0/1152645003 wait complete. 2023-06-09T11:34:58.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 108 2023-06-09T11:34:58.678 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:00.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.883+0000 7f81b53ff700 1 Processor -- start 2023-06-09T11:35:00.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.883+0000 7f81b53ff700 1 -- start start 2023-06-09T11:35:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81b53ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b00c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81b53ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b00c5a80 con 0x7f81b015b4a0 2023-06-09T11:35:00.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81b53ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81b015a500 con 0x7f81b0153880 2023-06-09T11:35:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81b53ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00c5f10 con 0x7f81b00c9f40 2023-06-09T11:35:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81ae7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f81b015b4a0 0x7f81b00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45594/0 (socket says 172.21.15.7:45594) 2023-06-09T11:35:00.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.884+0000 7f81ae7fc700 1 -- 172.21.15.7:0/920771802 learned_addr learned my addr 172.21.15.7:0/920771802 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:35:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81af7fe700 1 --2- 172.21.15.7:0/920771802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b00c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:00.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374645204 0 0) 0x7f81b00c5a80 con 0x7f81b015b4a0 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81980035b0 con 0x7f81b015b4a0 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700220795 0 0) 0x7f81a0001110 con 0x7f81b0153880 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b00c5a80 con 0x7f81b0153880 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81af7fe700 1 -- 172.21.15.7:0/920771802 >> v1:172.21.15.117:6789/0 conn(0x7f81b0153880 legacy=0x7f81b015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81af7fe700 1 -- 172.21.15.7:0/920771802 >> v1:172.21.15.7:6789/0 conn(0x7f81b015b4a0 legacy=0x7f81b00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81af7fe700 1 -- 172.21.15.7:0/920771802 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b01596e0 con 0x7f81b00c9f40 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.885+0000 7f81af7fe700 1 --2- 172.21.15.7:0/920771802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b00c5540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009dc0 tx=0x7f81a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e790b169faa0464e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:00.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800e070 con 0x7f81b00c9f40 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800bb20 con 0x7f81b00c9f40 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81adffb700 1 -- 172.21.15.7:0/920771802 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8012430 con 0x7f81b00c9f40 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 -- 172.21.15.7:0/920771802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 msgr2=0x7f81b00c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 --2- 172.21.15.7:0/920771802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b00c5540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009dc0 tx=0x7f81a800b330 comp rx=0 tx=0).stop 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 -- 172.21.15.7:0/920771802 shutdown_connections 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 --2- 172.21.15.7:0/920771802 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b00c5540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:00.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 -- 172.21.15.7:0/920771802 >> 172.21.15.7:0/920771802 conn(0x7f81b00737f0 msgr2=0x7f81b00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 -- 172.21.15.7:0/920771802 shutdown_connections 2023-06-09T11:35:00.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.886+0000 7f81b53ff700 1 -- 172.21.15.7:0/920771802 wait complete. 2023-06-09T11:35:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.888+0000 7f81b53ff700 1 Processor -- start 2023-06-09T11:35:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.888+0000 7f81b53ff700 1 -- start start 2023-06-09T11:35:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b0152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 0x7f81b01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:00.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b00cf210 con 0x7f81b015b4a0 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0152360 con 0x7f81b0153880 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81b53ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b0157c50 con 0x7f81b00c9f40 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81af7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51934/0 (socket says 172.21.15.7:51934) 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.889+0000 7f81af7fe700 1 -- 172.21.15.7:0/1732273701 learned_addr learned my addr 172.21.15.7:0/1732273701 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:00.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81ae7fc700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 0x7f81b01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81aeffd700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b0152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 -- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 msgr2=0x7f81b0152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b0152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 -- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 msgr2=0x7f81b01589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 0x7f81b01589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 -- 172.21.15.7:0/1732273701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8009a70 con 0x7f81b015b4a0 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81ae7fc700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 0x7f81b01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:35:00.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81af7fe700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f81a800b370 tx=0x7f81a8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:00.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a801b810 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a8020070 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b0159c60 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.890+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a801bbf0 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.891+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b0159980 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.891+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b0159f40 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.892+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f81a801bd50 con 0x7f81b015b4a0 2023-06-09T11:35:00.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.893+0000 7f81977fe700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 0x7f819c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.893+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f81a8022070 con 0x7f81b015b4a0 2023-06-09T11:35:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.893+0000 7f81aeffd700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 0x7f819c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:00.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.894+0000 7f81aeffd700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 0x7f819c06d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f81a0007fd0 tx=0x7f81a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:00.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:00.898+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f81a80568a0 con 0x7f81b015b4a0 2023-06-09T11:35:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.153+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81b00c5d00 con 0x7f81b015b4a0 2023-06-09T11:35:01.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.154+0000 7f81977fe700 1 -- 172.21.15.7:0/1732273701 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f81a802c090 con 0x7f81b015b4a0 2023-06-09T11:35:01.161 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:01.161 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:34:57.606339+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:resolve","state_seq":20,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:01.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 msgr2=0x7f819c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 0x7f819c06d190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f81a0007fd0 tx=0x7f81a0009040 comp rx=0 tx=0).stop 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 msgr2=0x7f81b00ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f81a800b370 tx=0x7f81a8004ad0 comp rx=0 tx=0).stop 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 shutdown_connections 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b015b4a0 0x7f81b00ce660 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:01.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f819c06ace0 0x7f819c06d190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f81b00c9f40 0x7f81b0152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 --2- 172.21.15.7:0/1732273701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f81b0153880 0x7f81b01589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.156+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 >> 172.21.15.7:0/1732273701 conn(0x7f81b00737f0 msgr2=0x7f81b01e3940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.157+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 shutdown_connections 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:01.157+0000 7f81b53ff700 1 -- 172.21.15.7:0/1732273701 wait complete. 2023-06-09T11:35:01.164 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 108 2023-06-09T11:35:01.181 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:03.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:03.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.386+0000 7f8cfe9ff700 1 Processor -- start 2023-06-09T11:35:03.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.386+0000 7f8cfe9ff700 1 -- start start 2023-06-09T11:35:03.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.387+0000 7f8cfe9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cf8151510 0x7f8cf80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.387+0000 7f8cfe9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 0x7f8cf8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.387+0000 7f8cfe9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf81580a0 con 0x7f8cf8071060 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.387+0000 7f8cfe9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf80714e0 con 0x7f8cf8151510 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.387+0000 7f8cfe9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81518e0 con 0x7f8cf815b380 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cfcf9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8cf8071060 0x7f8cf8158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46266/0 (socket says 172.21.15.7:46266) 2023-06-09T11:35:03.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cfcf9c700 1 -- 172.21.15.7:0/1468825097 learned_addr learned my addr 172.21.15.7:0/1468825097 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1468825097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 0x7f8cf8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/1468825097 >> v2:172.21.15.117:3300/0 conn(0x7f8cf8151510 0x7f8cf80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1468825097 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747092817 0 0) 0x7f8cf81580a0 con 0x7f8cf8071060 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1468825097 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce00035b0 con 0x7f8cf8071060 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1468825097 >> v2:172.21.15.117:3300/0 conn(0x7f8cf8151510 msgr2=0x7f8cf80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:03.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1468825097 >> v2:172.21.15.117:3300/0 conn(0x7f8cf8151510 0x7f8cf80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1468825097 >> v1:172.21.15.7:6789/0 conn(0x7f8cf8071060 legacy=0x7f8cf8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.388+0000 7f8cf77fe700 1 -- 172.21.15.7:0/1468825097 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf80c6180 con 0x7f8cf815b380 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/1468825097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 0x7f8cf8070b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009840 tx=0x7f8cec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29741c1e661ba66 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1468825097 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec011830 con 0x7f8cf815b380 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1468825097 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec011990 con 0x7f8cf815b380 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cf6ffd700 1 -- 172.21.15.7:0/1468825097 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec011d20 con 0x7f8cf815b380 2023-06-09T11:35:03.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/1468825097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 msgr2=0x7f8cf8070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/1468825097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 0x7f8cf8070b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009840 tx=0x7f8cec009b50 comp rx=0 tx=0).stop 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/1468825097 shutdown_connections 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/1468825097 >> v2:172.21.15.117:3300/0 conn(0x7f8cf8151510 0x7f8cf80c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/1468825097 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf815b380 0x7f8cf8070b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.389+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/1468825097 >> 172.21.15.7:0/1468825097 conn(0x7f8cf806a4d0 msgr2=0x7f8cf806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.390+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/1468825097 shutdown_connections 2023-06-09T11:35:03.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.390+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/1468825097 wait complete. 2023-06-09T11:35:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.391+0000 7f8cfe9ff700 1 Processor -- start 2023-06-09T11:35:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.391+0000 7f8cfe9ff700 1 -- start start 2023-06-09T11:35:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf8071060 0x7f8cf8154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 0x7f8cf8146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf80c3030 con 0x7f8cf815b380 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf8154bc0 con 0x7f8cf8151510 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.392+0000 7f8cfe9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf81ece10 con 0x7f8cf8071060 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43430/0 (socket says 172.21.15.7:43430) 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 -- 172.21.15.7:0/237904066 learned_addr learned my addr 172.21.15.7:0/237904066 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:03.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf8071060 0x7f8cf8154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 0x7f8cf8146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 -- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf8071060 msgr2=0x7f8cf8154380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf8071060 0x7f8cf8154380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 -- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 msgr2=0x7f8cf8146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 0x7f8cf8146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 -- 172.21.15.7:0/237904066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec0094a0 con 0x7f8cf815b380 2023-06-09T11:35:03.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cf77fe700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 0x7f8cf8146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cfcf9c700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000cb40 tx=0x7f8cf000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.393+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf000ab50 con 0x7f8cf815b380 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.394+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf000acb0 con 0x7f8cf815b380 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.394+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf8157770 con 0x7f8cf815b380 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.394+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0012620 con 0x7f8cf815b380 2023-06-09T11:35:03.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.394+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf8157490 con 0x7f8cf815b380 2023-06-09T11:35:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.395+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf8157a50 con 0x7f8cf815b380 2023-06-09T11:35:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.396+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cf00113a0 con 0x7f8cf815b380 2023-06-09T11:35:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.397+0000 7f8cf4ff9700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 0x7f8ce406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:03.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.397+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f8cf0089750 con 0x7f8cf815b380 2023-06-09T11:35:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.397+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 0x7f8ce406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:03.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.397+0000 7f8cf7fff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 0x7f8ce406d1c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009b70 tx=0x7f8ce8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:03.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.401+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8cf0054350 con 0x7f8cf815b380 2023-06-09T11:35:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.656+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cf8074cf0 con 0x7f8cf815b380 2023-06-09T11:35:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.656+0000 7f8cf4ff9700 1 -- 172.21.15.7:0/237904066 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f8cf0057970 con 0x7f8cf815b380 2023-06-09T11:35:03.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:03.663 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:03.627879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 msgr2=0x7f8ce406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 0x7f8ce406d1c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ce8009b70 tx=0x7f8ce8008040 comp rx=0 tx=0).stop 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 msgr2=0x7f8cf8147110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f8cf000cb40 tx=0x7f8cf000cf00 comp rx=0 tx=0).stop 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 shutdown_connections 2023-06-09T11:35:03.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cf815b380 0x7f8cf8147110 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ce406ad10 0x7f8ce406d1c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cf8071060 0x7f8cf8154380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 --2- 172.21.15.7:0/237904066 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cf8151510 0x7f8cf8146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 >> 172.21.15.7:0/237904066 conn(0x7f8cf806a4d0 msgr2=0x7f8cf8159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 shutdown_connections 2023-06-09T11:35:03.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:03.659+0000 7f8cfe9ff700 1 -- 172.21.15.7:0/237904066 wait complete. 2023-06-09T11:35:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 109 2023-06-09T11:35:03.684 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 109, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 109, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5692": { "addr": "172.21.15.117:6837/3311310614", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3311310614, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3311310614, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5692, "incarnation": 67, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 13 }, "gid_5741": { "addr": "172.21.15.117:6833/557223025", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 557223025, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 557223025, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5741, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 22 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 62, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:35:03.627879+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5741, "mds_1": 5692, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:35:05.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.890+0000 7f436c3ff700 1 Processor -- start 2023-06-09T11:35:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.890+0000 7f436c3ff700 1 -- start start 2023-06-09T11:35:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436c3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43640cb5d0 0x7f43640b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436c3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4364159b20 con 0x7f43640cb9a0 2023-06-09T11:35:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436c3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640cac80 con 0x7f436415cd40 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436c3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43641597d0 con 0x7f43640cb5d0 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43640cb5d0 0x7f43640b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f43640cb5d0 0x7f43640b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43436/0 (socket says 172.21.15.7:43436) 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 -- 172.21.15.7:0/1165957739 learned_addr learned my addr 172.21.15.7:0/1165957739 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436999a700 1 --1- 172.21.15.7:0/1165957739 >> v1:172.21.15.7:6789/0 conn(0x7f43640cb9a0 0x7f4364159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46274/0 (socket says 172.21.15.7:46274) 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 -- 172.21.15.7:0/1165957739 >> v1:172.21.15.117:6789/0 conn(0x7f436415cd40 legacy=0x7f43640cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 -- 172.21.15.7:0/1165957739 >> v1:172.21.15.7:6789/0 conn(0x7f43640cb9a0 legacy=0x7f4364159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f436a19b700 1 -- 172.21.15.7:0/1165957739 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364071ab0 con 0x7f43640cb5d0 2023-06-09T11:35:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.892+0000 7f4369199700 1 -- 172.21.15.7:0/1165957739 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380341613 0 0) 0x7f4364159b20 con 0x7f43640cb9a0 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f436a19b700 1 --2- 172.21.15.7:0/1165957739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43640cb5d0 0x7f43640b52b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f43600098f0 tx=0x7f4360009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549f326becd3349e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f4369199700 1 -- 172.21.15.7:0/1165957739 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000b8a0 con 0x7f43640cb5d0 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f4369199700 1 -- 172.21.15.7:0/1165957739 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436000ba00 con 0x7f43640cb5d0 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f4369199700 1 -- 172.21.15.7:0/1165957739 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000bd90 con 0x7f43640cb5d0 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f436c3ff700 1 -- 172.21.15.7:0/1165957739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43640cb5d0 msgr2=0x7f43640b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.893+0000 7f436c3ff700 1 --2- 172.21.15.7:0/1165957739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43640cb5d0 0x7f43640b52b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f43600098f0 tx=0x7f4360009c00 comp rx=0 tx=0).stop 2023-06-09T11:35:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.894+0000 7f436c3ff700 1 -- 172.21.15.7:0/1165957739 shutdown_connections 2023-06-09T11:35:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.894+0000 7f436c3ff700 1 --2- 172.21.15.7:0/1165957739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f43640cb5d0 0x7f43640b52b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.894+0000 7f436c3ff700 1 -- 172.21.15.7:0/1165957739 >> 172.21.15.7:0/1165957739 conn(0x7f4364070af0 msgr2=0x7f4364070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.894+0000 7f436c3ff700 1 -- 172.21.15.7:0/1165957739 shutdown_connections 2023-06-09T11:35:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.894+0000 7f436c3ff700 1 -- 172.21.15.7:0/1165957739 wait complete. 2023-06-09T11:35:05.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.895+0000 7f436c3ff700 1 Processor -- start 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.895+0000 7f436c3ff700 1 -- start start 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43640cb5d0 0x7f43641f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43640cb9a0 0x7f43641f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4364153e30 con 0x7f436415cd40 2023-06-09T11:35:05.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640c3750 con 0x7f43640cb9a0 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.896+0000 7f436c3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4364153a90 con 0x7f43640cb5d0 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43446/0 (socket says 172.21.15.7:43446) 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 -- 172.21.15.7:0/2176693301 learned_addr learned my addr 172.21.15.7:0/2176693301 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a19b700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43640cb5d0 0x7f43641f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436999a700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43640cb9a0 0x7f43641f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 -- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43640cb5d0 msgr2=0x7f43641f06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43640cb5d0 0x7f43641f06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 -- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43640cb9a0 msgr2=0x7f43641f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43640cb9a0 0x7f43641f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 -- 172.21.15.7:0/2176693301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360009550 con 0x7f436415cd40 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.897+0000 7f436a99c700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f435c00cbf0 tx=0x7f435c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.898+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c003c00 con 0x7f436415cd40 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.898+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c003d60 con 0x7f436415cd40 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.898+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00aa40 con 0x7f436415cd40 2023-06-09T11:35:05.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.898+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436415d4f0 con 0x7f436415cd40 2023-06-09T11:35:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.898+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436415d210 con 0x7f436415cd40 2023-06-09T11:35:05.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.899+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436415d7d0 con 0x7f436415cd40 2023-06-09T11:35:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.900+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f435c00aba0 con 0x7f436415cd40 2023-06-09T11:35:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.901+0000 7f435affd700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 0x7f434806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.901+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f435c089800 con 0x7f436415cd40 2023-06-09T11:35:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.901+0000 7f436a19b700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 0x7f434806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:05.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.902+0000 7f436a19b700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 0x7f434806d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f4360008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:05.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:05.906+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f435c054420 con 0x7f436415cd40 2023-06-09T11:35:06.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.155+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43640c2f90 con 0x7f436415cd40 2023-06-09T11:35:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.156+0000 7f435affd700 1 -- 172.21.15.7:0/2176693301 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f435c057a40 con 0x7f436415cd40 2023-06-09T11:35:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:06.162 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":109,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:03.627879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:reconnect","state_seq":22,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:06.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 msgr2=0x7f434806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:06.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 0x7f434806d190 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4360000c00 tx=0x7f4360008030 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 msgr2=0x7f43641526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f435c00cbf0 tx=0x7f435c00cf00 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 shutdown_connections 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f436415cd40 0x7f43641526b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f434806ace0 0x7f434806d190 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f43640cb5d0 0x7f43641f06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 --2- 172.21.15.7:0/2176693301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f43640cb9a0 0x7f43641f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:06.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 >> 172.21.15.7:0/2176693301 conn(0x7f4364070af0 msgr2=0x7f43640cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 shutdown_connections 2023-06-09T11:35:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:06.158+0000 7f436c3ff700 1 -- 172.21.15.7:0/2176693301 wait complete. 2023-06-09T11:35:06.166 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 109 2023-06-09T11:35:06.182 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:08.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.387+0000 7ffb887ff700 1 Processor -- start 2023-06-09T11:35:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.388+0000 7ffb887ff700 1 -- start start 2023-06-09T11:35:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb887ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb800cb5d0 0x7ffb800b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb887ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb887ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb80159b20 con 0x7ffb800cb9a0 2023-06-09T11:35:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb887ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb800cac80 con 0x7ffb800cb5d0 2023-06-09T11:35:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb887ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb801597d0 con 0x7ffb8015cd40 2023-06-09T11:35:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb8659b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ffb800cb5d0 0x7ffb800b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb85d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffb800cb9a0 0x7ffb80159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46284/0 (socket says 172.21.15.7:46284) 2023-06-09T11:35:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb85d9a700 1 -- 172.21.15.7:0/1741772660 learned_addr learned my addr 172.21.15.7:0/1741772660 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:35:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb86d9c700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb8659b700 1 -- 172.21.15.7:0/1741772660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 msgr2=0x7ffb800cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb8659b700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.389+0000 7ffb8659b700 1 -- 172.21.15.7:0/1741772660 >> v1:172.21.15.7:6789/0 conn(0x7ffb800cb9a0 legacy=0x7ffb80159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb8659b700 1 -- 172.21.15.7:0/1741772660 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb80071ab0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb85599700 1 -- 172.21.15.7:0/1741772660 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208647394 0 0) 0x7ffb80159b20 con 0x7ffb800cb9a0 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb86d9c700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb8659b700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb800b52b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009840 tx=0x7ffb7c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d755286cd25b25d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb85599700 1 -- 172.21.15.7:0/1741772660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011830 con 0x7ffb800cb5d0 2023-06-09T11:35:08.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb85599700 1 -- 172.21.15.7:0/1741772660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c011990 con 0x7ffb800cb5d0 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.390+0000 7ffb85599700 1 -- 172.21.15.7:0/1741772660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c011d20 con 0x7ffb800cb5d0 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 -- 172.21.15.7:0/1741772660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 msgr2=0x7ffb800b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb800b52b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009840 tx=0x7ffb7c009b50 comp rx=0 tx=0).stop 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 -- 172.21.15.7:0/1741772660 shutdown_connections 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 --2- 172.21.15.7:0/1741772660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb800b52b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 -- 172.21.15.7:0/1741772660 >> 172.21.15.7:0/1741772660 conn(0x7ffb80070af0 msgr2=0x7ffb80070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 -- 172.21.15.7:0/1741772660 shutdown_connections 2023-06-09T11:35:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.391+0000 7ffb887ff700 1 -- 172.21.15.7:0/1741772660 wait complete. 2023-06-09T11:35:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.392+0000 7ffb887ff700 1 Processor -- start 2023-06-09T11:35:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.393+0000 7ffb887ff700 1 -- start start 2023-06-09T11:35:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.393+0000 7ffb887ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb887ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb800cb9a0 0x7ffb800cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb887ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb887ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb801f1f80 con 0x7ffb800cb5d0 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb887ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb801f1ca0 con 0x7ffb800cb9a0 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb887ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800ce6e0 con 0x7ffb8015cd40 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43460/0 (socket says 172.21.15.7:43460) 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 -- 172.21.15.7:0/2944899208 learned_addr learned my addr 172.21.15.7:0/2944899208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb86d9c700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 -- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 msgr2=0x7ffb800cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 -- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb800cb9a0 msgr2=0x7ffb800cf1c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb800cb9a0 0x7ffb800cf1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 -- 172.21.15.7:0/2944899208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c0094a0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.394+0000 7ffb8659b700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.395+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0036a0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.395+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c01f070 con 0x7ffb800cb5d0 2023-06-09T11:35:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.395+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8015d4f0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.395+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c003a20 con 0x7ffb800cb5d0 2023-06-09T11:35:08.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.395+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8015d210 con 0x7ffb800cb5d0 2023-06-09T11:35:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.396+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8015d7d0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.397+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb7c003b80 con 0x7ffb800cb5d0 2023-06-09T11:35:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.398+0000 7ffb76ffd700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 0x7ffb6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.398+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7ffb7c021070 con 0x7ffb800cb5d0 2023-06-09T11:35:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.398+0000 7ffb85d9a700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 0x7ffb6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:08.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.399+0000 7ffb85d9a700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 0x7ffb6406d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007fd0 tx=0x7ffb70007940 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:08.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.403+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffb7c055ea0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.658+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb80074cf0 con 0x7ffb800cb5d0 2023-06-09T11:35:08.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.659+0000 7ffb76ffd700 1 -- 172.21.15.7:0/2944899208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5462 (secure 0 0 0) 0x7ffb7c010380 con 0x7ffb800cb5d0 2023-06-09T11:35:08.665 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:08.665 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:08.644378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 msgr2=0x7ffb6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:08.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 0x7ffb6406d190 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ffb70007fd0 tx=0x7ffb70007940 comp rx=0 tx=0).stop 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 msgr2=0x7ffb80150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0090b0 tx=0x7ffb7c004750 comp rx=0 tx=0).stop 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 shutdown_connections 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffb6406ace0 0x7ffb6406d190 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb8015cd40 0x7ffb800cf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffb800cb5d0 0x7ffb80150430 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 --2- 172.21.15.7:0/2944899208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb800cb9a0 0x7ffb800cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 >> 172.21.15.7:0/2944899208 conn(0x7ffb80070af0 msgr2=0x7ffb8015c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 shutdown_connections 2023-06-09T11:35:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:08.661+0000 7ffb887ff700 1 -- 172.21.15.7:0/2944899208 wait complete. 2023-06-09T11:35:08.669 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 110 2023-06-09T11:35:08.685 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:10.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 Processor -- start 2023-06-09T11:35:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 -- start start 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cc0156f20 0x7f4cc00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0071d50 0x7f4cc0072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cc00d01a0 con 0x7f4cc0156b50 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cc0072690 con 0x7f4cc0156f20 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.891+0000 7f4cc49fe700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00cb2b0 con 0x7f4cc0071d50 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cc0156f20 0x7f4cc00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4cc0156f20 0x7f4cc00cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43466/0 (socket says 172.21.15.7:43466) 2023-06-09T11:35:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 -- 172.21.15.7:0/4022449808 learned_addr learned my addr 172.21.15.7:0/4022449808 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbf59e700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0071d50 0x7f4cc0072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 -- 172.21.15.7:0/4022449808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0071d50 msgr2=0x7f4cc0072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0071d50 0x7f4cc0072150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 -- 172.21.15.7:0/4022449808 >> v1:172.21.15.117:6789/0 conn(0x7f4cc0156b50 legacy=0x7f4cc00caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbe59c700 1 -- 172.21.15.7:0/4022449808 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc00b54a0 con 0x7f4cc0156f20 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.892+0000 7f4cbdd9b700 1 -- 172.21.15.7:0/4022449808 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507073546 0 0) 0x7f4cc00d01a0 con 0x7f4cc0156b50 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cbe59c700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0156f20 0x7f4cc00cb510 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40098f0 tx=0x7f4cb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2001e8983acfa9d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cbdd9b700 1 -- 172.21.15.7:0/4022449808 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4011830 con 0x7f4cc0156f20 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cbdd9b700 1 -- 172.21.15.7:0/4022449808 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb4011990 con 0x7f4cc0156f20 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cbdd9b700 1 -- 172.21.15.7:0/4022449808 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4011d20 con 0x7f4cc0156f20 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 -- 172.21.15.7:0/4022449808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0156f20 msgr2=0x7f4cc00cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0156f20 0x7f4cc00cb510 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40098f0 tx=0x7f4cb4009c00 comp rx=0 tx=0).stop 2023-06-09T11:35:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 -- 172.21.15.7:0/4022449808 shutdown_connections 2023-06-09T11:35:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0071d50 0x7f4cc0072150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/4022449808 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0156f20 0x7f4cc00cb510 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 -- 172.21.15.7:0/4022449808 >> 172.21.15.7:0/4022449808 conn(0x7f4cc00737f0 msgr2=0x7f4cc0073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 -- 172.21.15.7:0/4022449808 shutdown_connections 2023-06-09T11:35:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.893+0000 7f4cc49fe700 1 -- 172.21.15.7:0/4022449808 wait complete. 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.895+0000 7f4cc49fe700 1 Processor -- start 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.895+0000 7f4cc49fe700 1 -- start start 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0156b50 0x7f4cc00cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 0x7f4cc01f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00c6c70 con 0x7f4cc0071d50 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00c5590 con 0x7f4cc0156b50 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.896+0000 7f4cc49fe700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc00c5870 con 0x7f4cc0156f20 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43480/0 (socket says 172.21.15.7:43480) 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 -- 172.21.15.7:0/2453171434 learned_addr learned my addr 172.21.15.7:0/2453171434 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbe59c700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0156b50 0x7f4cc00cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbf59e700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 0x7f4cc01f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 -- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 msgr2=0x7f4cc01f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 0x7f4cc01f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 -- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0156b50 msgr2=0x7f4cc00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0156b50 0x7f4cc00cdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 -- 172.21.15.7:0/2453171434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb4009550 con 0x7f4cc0071d50 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbf59e700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 0x7f4cc01f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4cbed9d700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c550 tx=0x7f4cb000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb00054e0 con 0x7f4cc0071d50 2023-06-09T11:35:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cb00036d0 con 0x7f4cc0071d50 2023-06-09T11:35:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.897+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0003a80 con 0x7f4cc0071d50 2023-06-09T11:35:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.898+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc00cf590 con 0x7f4cc0071d50 2023-06-09T11:35:10.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.898+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc00cba50 con 0x7f4cc0071d50 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.899+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc00cf870 con 0x7f4cc0071d50 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.900+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4cb0003be0 con 0x7f4cc0071d50 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.901+0000 7f4ca77fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 0x7f4cac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.901+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f4cb0088db0 con 0x7f4cc0071d50 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.901+0000 7f4cbe59c700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 0x7f4cac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:10.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.902+0000 7f4cbe59c700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 0x7f4cac06d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4014040 tx=0x7f4cb4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:10.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:10.906+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4cb0053a50 con 0x7f4cc0071d50 2023-06-09T11:35:11.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.162+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cc004f760 con 0x7f4cc0071d50 2023-06-09T11:35:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.162+0000 7f4ca77fe700 1 -- 172.21.15.7:0/2453171434 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f4cb0017020 con 0x7f4cc0071d50 2023-06-09T11:35:11.167 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:11.168 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:08.644378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.164+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 msgr2=0x7f4cac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.164+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 0x7f4cac06d190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4014040 tx=0x7f4cb4008030 comp rx=0 tx=0).stop 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.164+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 msgr2=0x7f4cc00c4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.164+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000c550 tx=0x7f4cb000c860 comp rx=0 tx=0).stop 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 shutdown_connections 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4cac06ace0 0x7f4cac06d190 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:11.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4cc0156f20 0x7f4cc01f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cc0071d50 0x7f4cc00c4e10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 --2- 172.21.15.7:0/2453171434 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4cc0156b50 0x7f4cc00cdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 >> 172.21.15.7:0/2453171434 conn(0x7f4cc00737f0 msgr2=0x7f4cc00c9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 shutdown_connections 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:11.165+0000 7f4cc49fe700 1 -- 172.21.15.7:0/2453171434 wait complete. 2023-06-09T11:35:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 110 2023-06-09T11:35:11.187 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:13.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.393+0000 7f85447ff700 1 Processor -- start 2023-06-09T11:35:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.394+0000 7f85447ff700 1 -- start start 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f853c153880 0x7f853c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c0c9f40 0x7f853c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f853c15b2d0 0x7f853c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c0c5f10 con 0x7f853c153880 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853c15a500 con 0x7f853c15b2d0 2023-06-09T11:35:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f85447ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c15da00 con 0x7f853c0c9f40 2023-06-09T11:35:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f854259b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f853c153880 0x7f853c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f854259b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f853c153880 0x7f853c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56468/0 (socket says 172.21.15.7:56468) 2023-06-09T11:35:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f854259b700 1 -- 172.21.15.7:0/4203294184 learned_addr learned my addr 172.21.15.7:0/4203294184 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f8541d9a700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c0c9f40 0x7f853c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.395+0000 7f8542d9c700 1 --2- 172.21.15.7:0/4203294184 >> v2:172.21.15.117:3300/0 conn(0x7f853c15b2d0 0x7f853c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 -- 172.21.15.7:0/4203294184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c0c9f40 msgr2=0x7f853c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c0c9f40 0x7f853c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 -- 172.21.15.7:0/4203294184 >> v2:172.21.15.117:3300/0 conn(0x7f853c15b2d0 msgr2=0x7f853c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 --2- 172.21.15.7:0/4203294184 >> v2:172.21.15.117:3300/0 conn(0x7f853c15b2d0 0x7f853c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 -- 172.21.15.7:0/4203294184 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853c15b750 con 0x7f853c153880 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f854259b700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c15d4c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f853c071db0 tx=0x7f8538009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c83ccb72fdc790a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f8541599700 1 -- 172.21.15.7:0/4203294184 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538011830 con 0x7f853c153880 2023-06-09T11:35:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f8541599700 1 -- 172.21.15.7:0/4203294184 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8538011990 con 0x7f853c153880 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.396+0000 7f8541599700 1 -- 172.21.15.7:0/4203294184 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538011d20 con 0x7f853c153880 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 -- 172.21.15.7:0/4203294184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 msgr2=0x7f853c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c15d4c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f853c071db0 tx=0x7f8538009bd0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 -- 172.21.15.7:0/4203294184 shutdown_connections 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 --2- 172.21.15.7:0/4203294184 >> v2:172.21.15.117:3300/0 conn(0x7f853c15b2d0 0x7f853c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c15d4c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 --2- 172.21.15.7:0/4203294184 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c0c9f40 0x7f853c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 -- 172.21.15.7:0/4203294184 >> 172.21.15.7:0/4203294184 conn(0x7f853c0737f0 msgr2=0x7f853c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 -- 172.21.15.7:0/4203294184 shutdown_connections 2023-06-09T11:35:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.397+0000 7f85447ff700 1 -- 172.21.15.7:0/4203294184 wait complete. 2023-06-09T11:35:13.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.399+0000 7f85447ff700 1 Processor -- start 2023-06-09T11:35:13.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.399+0000 7f85447ff700 1 -- start start 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 0x7f853c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c15b2d0 0x7f853c1ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c0b4c30 con 0x7f853c153880 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1eef70 con 0x7f853c0c9f40 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f85447ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853c1f25f0 con 0x7f853c15b2d0 2023-06-09T11:35:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56480/0 (socket says 172.21.15.7:56480) 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 -- 172.21.15.7:0/2393510996 learned_addr learned my addr 172.21.15.7:0/2393510996 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f854259b700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 0x7f853c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8542d9c700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c15b2d0 0x7f853c1ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 -- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c15b2d0 msgr2=0x7f853c1ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c15b2d0 0x7f853c1ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 -- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 msgr2=0x7f853c14b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 0x7f853c14b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f8541d9a700 1 -- 172.21.15.7:0/2393510996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8538009550 con 0x7f853c153880 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.400+0000 7f854259b700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 0x7f853c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f8541d9a700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e930 tx=0x7f852c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00ee80 con 0x7f853c153880 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00c850 con 0x7f853c153880 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00cbe0 con 0x7f853c153880 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c159980 con 0x7f853c153880 2023-06-09T11:35:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.401+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853c1596e0 con 0x7f853c153880 2023-06-09T11:35:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.402+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f853c159c60 con 0x7f853c153880 2023-06-09T11:35:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.403+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f852c00cd40 con 0x7f853c153880 2023-06-09T11:35:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.404+0000 7f8532ffd700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.404+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f852c089d80 con 0x7f853c153880 2023-06-09T11:35:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.404+0000 7f854259b700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.405+0000 7f854259b700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 0x7f852006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8538009c10 tx=0x7f8538008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:13.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.408+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f852c0548f0 con 0x7f853c153880 2023-06-09T11:35:13.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.660+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f853c074cf0 con 0x7f853c153880 2023-06-09T11:35:13.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.660+0000 7f8532ffd700 1 -- 172.21.15.7:0/2393510996 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f852c057f10 con 0x7f853c153880 2023-06-09T11:35:13.665 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:13.665 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":110,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:08.644378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:13.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 msgr2=0x7f852006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:13.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 0x7f852006d190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8538009c10 tx=0x7f8538008030 comp rx=0 tx=0).stop 2023-06-09T11:35:13.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 msgr2=0x7f853c1ef380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:13.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f852c00e930 tx=0x7f852c00ec40 comp rx=0 tx=0).stop 2023-06-09T11:35:13.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 shutdown_connections 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f852006ace0 0x7f852006d190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f853c15b2d0 0x7f853c1ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f853c0c9f40 0x7f853c14b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 --2- 172.21.15.7:0/2393510996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f853c153880 0x7f853c1ef380 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 >> 172.21.15.7:0/2393510996 conn(0x7f853c0737f0 msgr2=0x7f853c15b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 shutdown_connections 2023-06-09T11:35:13.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:13.663+0000 7f85447ff700 1 -- 172.21.15.7:0/2393510996 wait complete. 2023-06-09T11:35:13.670 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 110 2023-06-09T11:35:13.686 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:35:15.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.893+0000 7f2908bff700 1 Processor -- start 2023-06-09T11:35:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.893+0000 7f2908bff700 1 -- start start 2023-06-09T11:35:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.894+0000 7f2908bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040cb5d0 0x7f29040b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.894+0000 7f2908bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29040cb9a0 0x7f2904159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2908bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2904159b20 con 0x7f290415cd40 2023-06-09T11:35:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2908bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040cac80 con 0x7f29040cb9a0 2023-06-09T11:35:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2908bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29041597d0 con 0x7f29040cb5d0 2023-06-09T11:35:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29040cb9a0 0x7f2904159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29040cb9a0 0x7f2904159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56490/0 (socket says 172.21.15.7:56490) 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 -- 172.21.15.7:0/3721125281 learned_addr learned my addr 172.21.15.7:0/3721125281 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f290259c700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040cb5d0 0x7f29040b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 -- 172.21.15.7:0/3721125281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040cb5d0 msgr2=0x7f29040b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040cb5d0 0x7f29040b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 -- 172.21.15.7:0/3721125281 >> v1:172.21.15.117:6789/0 conn(0x7f290415cd40 legacy=0x7f29040cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.895+0000 7f2901d9b700 1 -- 172.21.15.7:0/3721125281 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904071ab0 con 0x7f29040cb9a0 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2901d9b700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f2904159fb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f28f80098f0 tx=0x7f28f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67f8ba6e557499a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f290159a700 1 -- 172.21.15.7:0/3721125281 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800b8a0 con 0x7f29040cb9a0 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f290159a700 1 -- 172.21.15.7:0/3721125281 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f800ba00 con 0x7f29040cb9a0 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f290159a700 1 -- 172.21.15.7:0/3721125281 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800bd90 con 0x7f29040cb9a0 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 -- 172.21.15.7:0/3721125281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 msgr2=0x7f2904159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f2904159fb0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f28f80098f0 tx=0x7f28f8009c00 comp rx=0 tx=0).stop 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 -- 172.21.15.7:0/3721125281 shutdown_connections 2023-06-09T11:35:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29040cb5d0 0x7f29040b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 --2- 172.21.15.7:0/3721125281 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f2904159fb0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.896+0000 7f2908bff700 1 -- 172.21.15.7:0/3721125281 >> 172.21.15.7:0/3721125281 conn(0x7f2904070af0 msgr2=0x7f2904070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.897+0000 7f2908bff700 1 -- 172.21.15.7:0/3721125281 shutdown_connections 2023-06-09T11:35:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.897+0000 7f2908bff700 1 -- 172.21.15.7:0/3721125281 wait complete. 2023-06-09T11:35:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.898+0000 7f2908bff700 1 Processor -- start 2023-06-09T11:35:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.898+0000 7f2908bff700 1 -- start start 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29040cb5d0 0x7f290415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f290415cd40 0x7f29040cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040ad700 con 0x7f29040cb9a0 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041efbb0 con 0x7f29040cb5d0 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.899+0000 7f2908bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040ce2a0 con 0x7f290415cd40 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56506/0 (socket says 172.21.15.7:56506) 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 -- 172.21.15.7:0/3888716413 learned_addr learned my addr 172.21.15.7:0/3888716413 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f290259c700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29040cb5d0 0x7f290415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2902d9d700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f290415cd40 0x7f29040cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 -- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f290415cd40 msgr2=0x7f29040cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f290415cd40 0x7f29040cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 -- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29040cb5d0 msgr2=0x7f290415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:15.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29040cb5d0 0x7f290415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 -- 172.21.15.7:0/3888716413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8009550 con 0x7f29040cb9a0 2023-06-09T11:35:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.900+0000 7f2901d9b700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009c40 tx=0x7f28f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:15.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.901+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8004300 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.901+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f801f070 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.901+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f80036a0 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.901+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290415d4f0 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.901+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290415d210 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.902+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290415d7d0 con 0x7f29040cb9a0 2023-06-09T11:35:15.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.903+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f28f8021030 con 0x7f29040cb9a0 2023-06-09T11:35:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.904+0000 7f28f2ffd700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 0x7f28e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.904+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f28f808ac90 con 0x7f29040cb9a0 2023-06-09T11:35:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.904+0000 7f290259c700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 0x7f28e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:15.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.905+0000 7f290259c700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 0x7f28e406d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006b20 tx=0x7f28f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:15.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:15.909+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f28f8055810 con 0x7f29040cb9a0 2023-06-09T11:35:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.158+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2904157530 con 0x7f29040cb9a0 2023-06-09T11:35:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.159+0000 7f28f2ffd700 1 -- 172.21.15.7:0/3888716413 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f28f8058e30 con 0x7f29040cb9a0 2023-06-09T11:35:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:16.165 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:14.658723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 msgr2=0x7f28e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 0x7f28e406d190 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f28f4006b20 tx=0x7f28f4008040 comp rx=0 tx=0).stop 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 msgr2=0x7f29040cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009c40 tx=0x7f28f8004750 comp rx=0 tx=0).stop 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 shutdown_connections 2023-06-09T11:35:16.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.161+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f28e406ace0 0x7f28e406d190 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f290415cd40 0x7f29040cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29040cb5d0 0x7f290415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 --2- 172.21.15.7:0/3888716413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29040cb9a0 0x7f29040cebe0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 >> 172.21.15.7:0/3888716413 conn(0x7f2904070af0 msgr2=0x7f290415c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 shutdown_connections 2023-06-09T11:35:16.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:16.162+0000 7f2908bff700 1 -- 172.21.15.7:0/3888716413 wait complete. 2023-06-09T11:35:16.169 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 111 2023-06-09T11:35:16.185 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=5630 2023-06-09T11:35:16.185 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.d 2023-06-09T11:35:17.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:17.359 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:18.185 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:35:18.186 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:35:18.186 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:35:18.191 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:35:18.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:18.338 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:35:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.398+0000 7f55873ff700 1 Processor -- start 2023-06-09T11:35:18.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.398+0000 7f55873ff700 1 -- start start 2023-06-09T11:35:18.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f55873ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5580151510 0x7f55800c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f55873ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5580071230 0x7f5580158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f55873ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55801580a0 con 0x7f558015b380 2023-06-09T11:35:18.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f55873ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55801518e0 con 0x7f5580071230 2023-06-09T11:35:18.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f55873ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55800c6180 con 0x7f5580151510 2023-06-09T11:35:18.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f558599c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5580071230 0x7f5580158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f558599c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5580071230 0x7f5580158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56518/0 (socket says 172.21.15.7:56518) 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.399+0000 7f558599c700 1 -- 172.21.15.7:0/2512994243 learned_addr learned my addr 172.21.15.7:0/2512994243 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558519b700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5580151510 0x7f55800c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558599c700 1 -- 172.21.15.7:0/2512994243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5580151510 msgr2=0x7f55800c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558599c700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5580151510 0x7f55800c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558599c700 1 -- 172.21.15.7:0/2512994243 >> v1:172.21.15.117:6789/0 conn(0x7f558015b380 legacy=0x7f5580070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558599c700 1 -- 172.21.15.7:0/2512994243 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5580157490 con 0x7f5580071230 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f5577fff700 1 -- 172.21.15.7:0/2512994243 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280022493 0 0) 0x7f55801580a0 con 0x7f558015b380 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f558599c700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f5580158870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5578009e50 tx=0x7f557800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdadf1f6d71d2cbf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f5577fff700 1 -- 172.21.15.7:0/2512994243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800e070 con 0x7f5580071230 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f5577fff700 1 -- 172.21.15.7:0/2512994243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800b910 con 0x7f5580071230 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.400+0000 7f5577fff700 1 -- 172.21.15.7:0/2512994243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557800bcc0 con 0x7f5580071230 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 -- 172.21.15.7:0/2512994243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 msgr2=0x7f5580158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f5580158870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5578009e50 tx=0x7f557800b330 comp rx=0 tx=0).stop 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 -- 172.21.15.7:0/2512994243 shutdown_connections 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f5580158870 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 --2- 172.21.15.7:0/2512994243 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5580151510 0x7f55800c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 -- 172.21.15.7:0/2512994243 >> 172.21.15.7:0/2512994243 conn(0x7f558006a4d0 msgr2=0x7f558006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 -- 172.21.15.7:0/2512994243 shutdown_connections 2023-06-09T11:35:18.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.401+0000 7f55873ff700 1 -- 172.21.15.7:0/2512994243 wait complete. 2023-06-09T11:35:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.403+0000 7f55873ff700 1 Processor -- start 2023-06-09T11:35:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.403+0000 7f55873ff700 1 -- start start 2023-06-09T11:35:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5580151510 0x7f55801e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558015b380 0x7f55801e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55801f2e90 con 0x7f5580071230 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55800c0f60 con 0x7f5580151510 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f55873ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5580156a40 con 0x7f558015b380 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f558519b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f558519b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56524/0 (socket says 172.21.15.7:56524) 2023-06-09T11:35:18.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f558519b700 1 -- 172.21.15.7:0/836364692 learned_addr learned my addr 172.21.15.7:0/836364692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f558499a700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5580151510 0x7f55801e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.404+0000 7f558599c700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558015b380 0x7f55801e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 -- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558015b380 msgr2=0x7f55801e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558015b380 0x7f55801e30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 -- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5580151510 msgr2=0x7f55801e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5580151510 0x7f55801e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 -- 172.21.15.7:0/836364692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578009b30 con 0x7f5580071230 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f558519b700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9e0 tx=0x7f557c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c009710 con 0x7f5580071230 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c00ee80 con 0x7f5580071230 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5580157a50 con 0x7f5580071230 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c00ca50 con 0x7f5580071230 2023-06-09T11:35:18.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.405+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5580157770 con 0x7f5580071230 2023-06-09T11:35:18.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.406+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5580157d30 con 0x7f5580071230 2023-06-09T11:35:18.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.407+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f557c00cbb0 con 0x7f5580071230 2023-06-09T11:35:18.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.408+0000 7f5575ffb700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:18.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.408+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f557c089d20 con 0x7f5580071230 2023-06-09T11:35:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.409+0000 7f558499a700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:18.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.409+0000 7f558499a700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 0x7f556406d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55800c5fd0 tx=0x7f5570009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:18.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.413+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f557c058330 con 0x7f5580071230 2023-06-09T11:35:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.663+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5580074cf0 con 0x7f5580071230 2023-06-09T11:35:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.663+0000 7f5575ffb700 1 -- 172.21.15.7:0/836364692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f557c021090 con 0x7f5580071230 2023-06-09T11:35:18.669 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:18.669 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":111,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:14.658723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 msgr2=0x7f556406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 0x7f556406d190 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55800c5fd0 tx=0x7f5570009040 comp rx=0 tx=0).stop 2023-06-09T11:35:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 msgr2=0x7f55801f21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f557c00e9e0 tx=0x7f557c00ecf0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 shutdown_connections 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f556406ace0 0x7f556406d190 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558015b380 0x7f55801e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5580071230 0x7f55801f21f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 --2- 172.21.15.7:0/836364692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5580151510 0x7f55801e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 >> 172.21.15.7:0/836364692 conn(0x7f558006a4d0 msgr2=0x7f558014cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 shutdown_connections 2023-06-09T11:35:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:18.666+0000 7f55873ff700 1 -- 172.21.15.7:0/836364692 wait complete. 2023-06-09T11:35:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 111 2023-06-09T11:35:18.690 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:35:20.448 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:35:20.443+0000 7f88c79f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:35:20.448 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:35:20.443+0000 7f88c79f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:35:20.449 INFO:tasks.ceph.mds.b.smithi117.stderr:2023-06-09T11:35:20.443+0000 7f88c79f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:35:20.691 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:35:20.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.921+0000 7fcb53fff700 1 Processor -- start 2023-06-09T11:35:20.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.922+0000 7fcb53fff700 1 -- start start 2023-06-09T11:35:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb4c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb53fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c0c5a80 con 0x7fcb4c153880 2023-06-09T11:35:20.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb53fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4c15a500 con 0x7fcb4c15b4a0 2023-06-09T11:35:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb53fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0c5f10 con 0x7fcb4c0c9f40 2023-06-09T11:35:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb51d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb4c153880 0x7fcb4c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60680/0 (socket says 172.21.15.7:60680) 2023-06-09T11:35:20.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.924+0000 7fcb51d9b700 1 -- 172.21.15.7:0/2422189896 learned_addr learned my addr 172.21.15.7:0/2422189896 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:35:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb5259c700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb4c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469466449 0 0) 0x7fcb4c0c5a80 con 0x7fcb4c153880 2023-06-09T11:35:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb380035b0 con 0x7fcb4c153880 2023-06-09T11:35:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635465523 0 0) 0x7fcb3c001110 con 0x7fcb4c15b4a0 2023-06-09T11:35:20.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb4c0c5a80 con 0x7fcb4c15b4a0 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 839374862 0 0) 0x7fcb380035b0 con 0x7fcb4c153880 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 msgr2=0x7fcb4c0c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb4c0c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 >> v1:172.21.15.117:6789/0 conn(0x7fcb4c15b4a0 legacy=0x7fcb4c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c1596e0 con 0x7fcb4c153880 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fcb48001670 con 0x7fcb4c153880 2023-06-09T11:35:20.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 >> v1:172.21.15.7:6789/0 conn(0x7fcb4c153880 legacy=0x7fcb4c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb38003bc0 0x7fcb38003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb38004850 0x7fcb38004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0c5f10 con 0x7fcb38003bc0 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3800baf0 con 0x7fcb38004850 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.925+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3800bcb0 con 0x7fcb4c0c9f40 2023-06-09T11:35:20.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb51d9b700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb38003bc0 0x7fcb38003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb38004850 0x7fcb38004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5259c700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 -- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 msgr2=0x7fcb38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb4c0c9f40 0x7fcb38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 -- 172.21.15.7:0/2422189896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb38003bc0 msgr2=0x7fcb38003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb38003bc0 0x7fcb38003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:20.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 -- 172.21.15.7:0/2422189896 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb380035b0 con 0x7fcb38004850 2023-06-09T11:35:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.926+0000 7fcb5159a700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb38004850 0x7fcb38004cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001970 tx=0x7fcb38005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ae9ab53da60e75 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c001600 con 0x7fcb38004850 2023-06-09T11:35:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0052d0 con 0x7fcb38004850 2023-06-09T11:35:20.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb50d99700 1 -- 172.21.15.7:0/2422189896 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c003680 con 0x7fcb38004850 2023-06-09T11:35:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb53fff700 1 -- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb38004850 msgr2=0x7fcb38004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:20.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb53fff700 1 --2- 172.21.15.7:0/2422189896 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb38004850 0x7fcb38004cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c001970 tx=0x7fcb38005880 comp rx=0 tx=0).stop 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.927+0000 7fcb5259c700 1 -- 172.21.15.7:0/2422189896 reap_dead start 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.928+0000 7fcb53fff700 1 -- 172.21.15.7:0/2422189896 shutdown_connections 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.928+0000 7fcb53fff700 1 -- 172.21.15.7:0/2422189896 >> 172.21.15.7:0/2422189896 conn(0x7fcb4c0737f0 msgr2=0x7fcb4c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.928+0000 7fcb53fff700 1 -- 172.21.15.7:0/2422189896 shutdown_connections 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.928+0000 7fcb53fff700 1 -- 172.21.15.7:0/2422189896 wait complete. 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.931+0000 7fcb53fff700 1 Processor -- start 2023-06-09T11:35:20.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.931+0000 7fcb53fff700 1 -- start start 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 0x7fcb4c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 0x7fcb4c1ef270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0c71e0 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c1efcf0 con 0x7fcb4c0c91a0 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb53fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4c0b3460 con 0x7fcb38004850 2023-06-09T11:35:20.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb5259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb5259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56536/0 (socket says 172.21.15.7:56536) 2023-06-09T11:35:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb5259c700 1 -- 172.21.15.7:0/3639874982 learned_addr learned my addr 172.21.15.7:0/3639874982 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:35:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb51d9b700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 0x7fcb4c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.932+0000 7fcb5159a700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 0x7fcb4c1ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 -- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 msgr2=0x7fcb4c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 0x7fcb4c0ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:20.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 -- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 msgr2=0x7fcb4c1ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 0x7fcb4c1ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 -- 172.21.15.7:0/3639874982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c159c60 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb51d9b700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 0x7fcb4c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5159a700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 0x7fcb4c1ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb5259c700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c06b800 tx=0x7fcb44011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400e040 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4400f040 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.933+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400fdd0 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.934+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4c159980 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.934+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4c159f40 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.935+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4c074cf0 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.936+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb44022460 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.937+0000 7fcb427fc700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 0x7fcb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:35:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.937+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fcb44089d50 con 0x7fcb4c0b2eb0 2023-06-09T11:35:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.937+0000 7fcb51d9b700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 0x7fcb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:35:20.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.938+0000 7fcb51d9b700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 0x7fcb3006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001eb0 tx=0x7fcb48001e20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:35:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:20.942+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcb440548c0 con 0x7fcb4c0b2eb0 2023-06-09T11:35:21.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.230+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb4c14dd20 con 0x7fcb4c0b2eb0 2023-06-09T11:35:21.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.231+0000 7fcb427fc700 1 -- 172.21.15.7:0/3639874982 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+6223 (secure 0 0 0) 0x7fcb44057ee0 con 0x7fcb4c0b2eb0 2023-06-09T11:35:21.236 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:35:21.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":112,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:35:14.658723+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:35:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 msgr2=0x7fcb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 0x7fcb3006d190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcb48001eb0 tx=0x7fcb48001e20 comp rx=0 tx=0).stop 2023-06-09T11:35:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 msgr2=0x7fcb4c1ef7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:35:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c06b800 tx=0x7fcb44011f60 comp rx=0 tx=0).stop 2023-06-09T11:35:21.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 shutdown_connections 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb4c0b2eb0 0x7fcb4c1ef7b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3006ace0 0x7fcb3006d190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb38004850 0x7fcb4c0ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 --2- 172.21.15.7:0/3639874982 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb4c0c91a0 0x7fcb4c1ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 >> 172.21.15.7:0/3639874982 conn(0x7fcb4c0737f0 msgr2=0x7fcb4c150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 shutdown_connections 2023-06-09T11:35:21.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:35:21.234+0000 7fcb53fff700 1 -- 172.21.15.7:0/3639874982 wait complete. 2023-06-09T11:35:21.241 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 112 2023-06-09T11:35:21.257 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:35:21.257 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 42 secs before thrashing 2023-06-09T11:35:47.516 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:35:47.521 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:03.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.475+0000 7f602e9ff700 1 Processor -- start 2023-06-09T11:36:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.475+0000 7f602e9ff700 1 -- start start 2023-06-09T11:36:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.476+0000 7f602e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c1bc0 0x7f60281599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.476+0000 7f602e9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60280c1f90 0x7f6028159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.476+0000 7f602e9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60280c0bf0 con 0x7f60280c0140 2023-06-09T11:36:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.476+0000 7f602e9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6028070110 con 0x7f60280c1f90 2023-06-09T11:36:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.476+0000 7f602e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6028071300 con 0x7f60280c1bc0 2023-06-09T11:36:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f6027fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c1bc0 0x7f60281599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60280c1f90 0x7f6028159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f60277fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f60280c0140 0x7f60281501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:38422/0 (socket says 172.21.15.7:38422) 2023-06-09T11:36:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f60277fe700 1 -- 172.21.15.7:0/1640668164 learned_addr learned my addr 172.21.15.7:0/1640668164 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 --2- 172.21.15.7:0/1640668164 >> v2:172.21.15.117:3300/0 conn(0x7f60280c1f90 0x7f6028159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:57686/0 (socket says 172.21.15.7:57686) 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 -- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c1bc0 msgr2=0x7f60281599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 --2- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c1bc0 0x7f60281599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 -- 172.21.15.7:0/1640668164 >> v1:172.21.15.117:6789/0 conn(0x7f60280c0140 legacy=0x7f60281501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f602cf9c700 1 -- 172.21.15.7:0/1640668164 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280702f0 con 0x7f60280c1f90 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.477+0000 7f6026ffd700 1 -- 172.21.15.7:0/1640668164 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302973569 0 0) 0x7f60280c0bf0 con 0x7f60280c0140 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602cf9c700 1 --2- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1f90 0x7f6028159ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b83e769138f632 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f6026ffd700 1 -- 172.21.15.7:0/1640668164 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602000e070 con 0x7f60280c1f90 2023-06-09T11:36:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f6026ffd700 1 -- 172.21.15.7:0/1640668164 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602000bb20 con 0x7f60280c1f90 2023-06-09T11:36:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f6026ffd700 1 -- 172.21.15.7:0/1640668164 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020012430 con 0x7f60280c1f90 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 -- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1f90 msgr2=0x7f6028159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 --2- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1f90 0x7f6028159ef0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6020009d90 tx=0x7f602000b330 comp rx=0 tx=0).stop 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 -- 172.21.15.7:0/1640668164 shutdown_connections 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 --2- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1f90 0x7f6028159ef0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 --2- 172.21.15.7:0/1640668164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c1bc0 0x7f60281599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.478+0000 7f602e9ff700 1 -- 172.21.15.7:0/1640668164 >> 172.21.15.7:0/1640668164 conn(0x7f602806a4d0 msgr2=0x7f602806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.479+0000 7f602e9ff700 1 -- 172.21.15.7:0/1640668164 shutdown_connections 2023-06-09T11:36:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.479+0000 7f602e9ff700 1 -- 172.21.15.7:0/1640668164 wait complete. 2023-06-09T11:36:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.480+0000 7f602e9ff700 1 Processor -- start 2023-06-09T11:36:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.480+0000 7f602e9ff700 1 -- start start 2023-06-09T11:36:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c0140 0x7f60280c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1bc0 0x7f60281ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60280c2a60 con 0x7f60280c1f90 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281ef720 con 0x7f60280c1bc0 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.481+0000 7f602e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60280c2c40 con 0x7f60280c0140 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36924/0 (socket says 172.21.15.7:36924) 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 -- 172.21.15.7:0/205898818 learned_addr learned my addr 172.21.15.7:0/205898818 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f6027fff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c0140 0x7f60280c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f60277fe700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1bc0 0x7f60281ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 -- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c0140 msgr2=0x7f60280c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c0140 0x7f60280c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 -- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1bc0 msgr2=0x7f60281ef1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1bc0 0x7f60281ef1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.482+0000 7f602cf9c700 1 -- 172.21.15.7:0/205898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009a70 con 0x7f60280c1f90 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f602cf9c700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f602806faf0 tx=0x7f6020004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020012430 con 0x7f60280c1f90 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020020070 con 0x7f60280c1f90 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60280c1170 con 0x7f60280c1f90 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020004280 con 0x7f60280c1f90 2023-06-09T11:36:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.483+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60280c0e90 con 0x7f60280c1f90 2023-06-09T11:36:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.484+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60280c1450 con 0x7f60280c1f90 2023-06-09T11:36:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.485+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60200043e0 con 0x7f60280c1f90 2023-06-09T11:36:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.486+0000 7f6024ff9700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.486+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f6020022070 con 0x7f60280c1f90 2023-06-09T11:36:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.486+0000 7f6027fff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.487+0000 7f6027fff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 0x7f601406d1c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6018009be0 tx=0x7f6018009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:03.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.491+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f60200565c0 con 0x7f60280c1f90 2023-06-09T11:36:03.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.745+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6028074cf0 con 0x7f60280c1f90 2023-06-09T11:36:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.745+0000 7f6024ff9700 1 -- 172.21.15.7:0/205898818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6222 (secure 0 0 0) 0x7f6020059be0 con 0x7f60280c1f90 2023-06-09T11:36:03.752 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:03.752 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:03.616447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 msgr2=0x7f601406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 0x7f601406d1c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6018009be0 tx=0x7f6018009210 comp rx=0 tx=0).stop 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 msgr2=0x7f60281efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f602806faf0 tx=0x7f6020004ad0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 shutdown_connections 2023-06-09T11:36:03.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60280c1f90 0x7f60281efc20 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60280c0140 0x7f60280c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 --2- 172.21.15.7:0/205898818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60280c1bc0 0x7f60281ef1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 >> 172.21.15.7:0/205898818 conn(0x7f602806a4d0 msgr2=0x7f60280c7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 shutdown_connections 2023-06-09T11:36:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.748+0000 7f602e9ff700 1 -- 172.21.15.7:0/205898818 wait complete. 2023-06-09T11:36:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 114 2023-06-09T11:36:03.772 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2023-06-09T11:36:03.772 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:36:03.772 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:36:03.793 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:36:03.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.991+0000 7fcfe45ff700 1 Processor -- start 2023-06-09T11:36:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.991+0000 7fcfe45ff700 1 -- start start 2023-06-09T11:36:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.992+0000 7fcfe45ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfdc15a0c0 0x7fcfdc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.992+0000 7fcfe45ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0c0bf0 con 0x7fcfdc0c0140 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.992+0000 7fcfe45ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0702f0 con 0x7fcfdc0c1bc0 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe45ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc070110 con 0x7fcfdc15a0c0 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe1b9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcfdc15a0c0 0x7fcfdc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe2b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcfdc0c0140 0x7fcfdc160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34456/0 (socket says 172.21.15.7:34456) 2023-06-09T11:36:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe2b9c700 1 -- 172.21.15.7:0/3924755921 learned_addr learned my addr 172.21.15.7:0/3924755921 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe1b9a700 1 -- 172.21.15.7:0/3924755921 >> v1:172.21.15.117:6789/0 conn(0x7fcfdc0c1bc0 legacy=0x7fcfdc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe1b9a700 1 -- 172.21.15.7:0/3924755921 >> v1:172.21.15.7:6789/0 conn(0x7fcfdc0c0140 legacy=0x7fcfdc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe1399700 1 -- 172.21.15.7:0/3924755921 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2492053303 0 0) 0x7fcfdc0702f0 con 0x7fcfdc0c1bc0 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.993+0000 7fcfe1b9a700 1 -- 172.21.15.7:0/3924755921 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0c0e90 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe1b9a700 1 --2- 172.21.15.7:0/3924755921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc1501b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009840 tx=0x7fcfcc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ccf01bce1c1108 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe1399700 1 -- 172.21.15.7:0/3924755921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc004230 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe1399700 1 -- 172.21.15.7:0/3924755921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc004390 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe1399700 1 -- 172.21.15.7:0/3924755921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0036a0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe45ff700 1 -- 172.21.15.7:0/3924755921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 msgr2=0x7fcfdc1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/3924755921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc1501b0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009840 tx=0x7fcfcc009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe45ff700 1 -- 172.21.15.7:0/3924755921 shutdown_connections 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/3924755921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc1501b0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.994+0000 7fcfe45ff700 1 -- 172.21.15.7:0/3924755921 >> 172.21.15.7:0/3924755921 conn(0x7fcfdc06a4d0 msgr2=0x7fcfdc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.995+0000 7fcfe45ff700 1 -- 172.21.15.7:0/3924755921 shutdown_connections 2023-06-09T11:36:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.995+0000 7fcfe45ff700 1 -- 172.21.15.7:0/3924755921 wait complete. 2023-06-09T11:36:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.996+0000 7fcfe45ff700 1 Processor -- start 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.996+0000 7fcfe45ff700 1 -- start start 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 0x7fcfdc0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 0x7fcfdc0c9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1ef000 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1ef420 con 0x7fcfdc0c0140 2023-06-09T11:36:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe45ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0c9d70 con 0x7fcfdc0c1bc0 2023-06-09T11:36:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.997+0000 7fcfe2b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36952/0 (socket says 172.21.15.7:36952) 2023-06-09T11:36:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 -- 172.21.15.7:0/1911238002 learned_addr learned my addr 172.21.15.7:0/1911238002 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe1b9a700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 0x7fcfdc0c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe239b700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 0x7fcfdc0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 -- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 msgr2=0x7fcfdc0c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 0x7fcfdc0c9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 -- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 msgr2=0x7fcfdc0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 0x7fcfdc0ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 -- 172.21.15.7:0/1911238002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0094a0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe1b9a700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 0x7fcfdc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe239b700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 0x7fcfdc0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe2b9c700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4001fd0 tx=0x7fcfd400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400ad60 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40070d0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc0c1450 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.998+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4007480 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:03.999+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc0c1170 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.000+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc0705d0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.001+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcfd40075e0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.001+0000 7fcfd2ffd700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 0x7fcfc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.002+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fcfd40895c0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.002+0000 7fcfe239b700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 0x7fcfc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.002+0000 7fcfe239b700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 0x7fcfc006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007fd0 tx=0x7fcfd8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.006+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcfd40541c0 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.264+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfdc1ef140 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.265+0000 7fcfd2ffd700 1 -- 172.21.15.7:0/1911238002 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) v1 ==== 95+0+6222 (secure 0 0 0) 0x7fcfdc1ef140 con 0x7fcfdc15a0c0 2023-06-09T11:36:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:04.272 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":114,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:03.616447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 msgr2=0x7fcfc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 0x7fcfc006d190 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8007fd0 tx=0x7fcfd8009040 comp rx=0 tx=0).stop 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 msgr2=0x7fcfdc0c9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4001fd0 tx=0x7fcfd400a340 comp rx=0 tx=0).stop 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 shutdown_connections 2023-06-09T11:36:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfdc15a0c0 0x7fcfdc0c9750 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcfdc0c0140 0x7fcfdc0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfc006ace0 0x7fcfc006d190 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 --2- 172.21.15.7:0/1911238002 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcfdc0c1bc0 0x7fcfdc0c9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.267+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 >> 172.21.15.7:0/1911238002 conn(0x7fcfdc06a4d0 msgr2=0x7fcfdc14ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.268+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 shutdown_connections 2023-06-09T11:36:04.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:04.268+0000 7fcfe45ff700 1 -- 172.21.15.7:0/1911238002 wait complete. 2023-06-09T11:36:04.276 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 114 2023-06-09T11:36:04.291 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:06.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:06.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.497+0000 7f29b73ff700 1 Processor -- start 2023-06-09T11:36:06.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.498+0000 7f29b73ff700 1 -- start start 2023-06-09T11:36:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b73ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29b015b060 0x7f29b00c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b73ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29b00c88e0 0x7f29b0163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b73ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00c9bd0 con 0x7f29b00c2ae0 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b73ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b0159360 con 0x7f29b00c88e0 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b73ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00c8e20 con 0x7f29b015b060 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b599c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29b00c88e0 0x7f29b0163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b599c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29b00c88e0 0x7f29b0163ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36962/0 (socket says 172.21.15.7:36962) 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b599c700 1 -- 172.21.15.7:0/2740943780 learned_addr learned my addr 172.21.15.7:0/2740943780 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.499+0000 7f29b519b700 1 --1- 172.21.15.7:0/2740943780 >> v1:172.21.15.7:6789/0 conn(0x7f29b00c2ae0 0x7f29b015a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34458/0 (socket says 172.21.15.7:34458) 2023-06-09T11:36:06.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b499a700 1 --2- 172.21.15.7:0/2740943780 >> v2:172.21.15.117:3300/0 conn(0x7f29b015b060 0x7f29b00c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b599c700 1 -- 172.21.15.7:0/2740943780 >> v2:172.21.15.117:3300/0 conn(0x7f29b015b060 msgr2=0x7f29b00c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b599c700 1 --2- 172.21.15.7:0/2740943780 >> v2:172.21.15.117:3300/0 conn(0x7f29b015b060 0x7f29b00c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b599c700 1 -- 172.21.15.7:0/2740943780 >> v1:172.21.15.7:6789/0 conn(0x7f29b00c2ae0 legacy=0x7f29b015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b599c700 1 -- 172.21.15.7:0/2740943780 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b0159570 con 0x7f29b00c88e0 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29b599c700 1 --2- 172.21.15.7:0/2740943780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b0163ce0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f29b00c4600 tx=0x7f29a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc5192decdfa6b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:06.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29a7fff700 1 -- 172.21.15.7:0/2740943780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a800e070 con 0x7f29b00c88e0 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29a7fff700 1 -- 172.21.15.7:0/2740943780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a800b9c0 con 0x7f29b00c88e0 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.500+0000 7f29a7fff700 1 -- 172.21.15.7:0/2740943780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a800bd70 con 0x7f29b00c88e0 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 -- 172.21.15.7:0/2740943780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 msgr2=0x7f29b0163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2740943780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b0163ce0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f29b00c4600 tx=0x7f29a800b330 comp rx=0 tx=0).stop 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 -- 172.21.15.7:0/2740943780 shutdown_connections 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2740943780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b0163ce0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2740943780 >> v2:172.21.15.117:3300/0 conn(0x7f29b015b060 0x7f29b00c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 -- 172.21.15.7:0/2740943780 >> 172.21.15.7:0/2740943780 conn(0x7f29b00cd330 msgr2=0x7f29b00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 -- 172.21.15.7:0/2740943780 shutdown_connections 2023-06-09T11:36:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.501+0000 7f29b73ff700 1 -- 172.21.15.7:0/2740943780 wait complete. 2023-06-09T11:36:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.503+0000 7f29b73ff700 1 Processor -- start 2023-06-09T11:36:06.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.504+0000 7f29b73ff700 1 -- start start 2023-06-09T11:36:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 0x7f29b0152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 0x7f29b01f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00c4d70 con 0x7f29b00c88e0 2023-06-09T11:36:06.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00c5190 con 0x7f29b015b060 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b73ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01f2dd0 con 0x7f29b00c2ae0 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b499a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b499a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36968/0 (socket says 172.21.15.7:36968) 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b499a700 1 -- 172.21.15.7:0/2092436803 learned_addr learned my addr 172.21.15.7:0/2092436803 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b599c700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 0x7f29b01f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.505+0000 7f29b519b700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 0x7f29b0152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 -- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 msgr2=0x7f29b0152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 0x7f29b0152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 -- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 msgr2=0x7f29b01f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 0x7f29b01f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 -- 172.21.15.7:0/2092436803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8009b30 con 0x7f29b00c88e0 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b599c700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 0x7f29b01f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b519b700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 0x7f29b0152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b499a700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f29a000d830 tx=0x7f29a000dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a00125f0 con 0x7f29b00c88e0 2023-06-09T11:36:06.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a0012750 con 0x7f29b00c88e0 2023-06-09T11:36:06.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a0012b00 con 0x7f29b00c88e0 2023-06-09T11:36:06.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b0159af0 con 0x7f29b00c88e0 2023-06-09T11:36:06.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.506+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b0159810 con 0x7f29b00c88e0 2023-06-09T11:36:06.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.507+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b0159dd0 con 0x7f29b00c88e0 2023-06-09T11:36:06.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.508+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29a0012c60 con 0x7f29b00c88e0 2023-06-09T11:36:06.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.509+0000 7f29a5ffb700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 0x7f299406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:06.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.509+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f29a0089c90 con 0x7f29b00c88e0 2023-06-09T11:36:06.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.509+0000 7f29b519b700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 0x7f299406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:06.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.510+0000 7f29b519b700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 0x7f299406d1c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000f60 tx=0x7f29ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:06.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.514+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f29a0054890 con 0x7f29b00c88e0 2023-06-09T11:36:06.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.769+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29b00c4eb0 con 0x7f29b00c88e0 2023-06-09T11:36:06.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.770+0000 7f29a5ffb700 1 -- 172.21.15.7:0/2092436803 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f29b00c4eb0 con 0x7f29b00c88e0 2023-06-09T11:36:06.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:06.776 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 msgr2=0x7f299406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 0x7f299406d1c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f29ac000f60 tx=0x7f29ac009040 comp rx=0 tx=0).stop 2023-06-09T11:36:06.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 msgr2=0x7f29b01f2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f29a000d830 tx=0x7f29a000dbf0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 shutdown_connections 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29b015b060 0x7f29b01f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f299406ad10 0x7f299406d1c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29b00c2ae0 0x7f29b0152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 --2- 172.21.15.7:0/2092436803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29b00c88e0 0x7f29b01f2350 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 >> 172.21.15.7:0/2092436803 conn(0x7f29b00cd330 msgr2=0x7f29b00c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:06.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 shutdown_connections 2023-06-09T11:36:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:06.772+0000 7f29b73ff700 1 -- 172.21.15.7:0/2092436803 wait complete. 2023-06-09T11:36:06.780 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:06.796 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:08.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.001+0000 7f1f011ff700 1 Processor -- start 2023-06-09T11:36:09.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.001+0000 7f1f011ff700 1 -- start start 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1f011ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1efc153880 0x7f1efc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1f011ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1efc0c9f40 0x7f1efc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1f011ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc15a500 con 0x7f1efc15b2d0 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1f011ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc0c5f10 con 0x7f1efc0c9f40 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1f011ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1efc15da00 con 0x7f1efc153880 2023-06-09T11:36:09.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1efc0c9f40 0x7f1efc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1efc0c9f40 0x7f1efc15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36984/0 (socket says 172.21.15.7:36984) 2023-06-09T11:36:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 -- 172.21.15.7:0/2714181557 learned_addr learned my addr 172.21.15.7:0/2714181557 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:09.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efb59e700 1 --1- 172.21.15.7:0/2714181557 >> v1:172.21.15.7:6789/0 conn(0x7f1efc15b2d0 0x7f1efc0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34468/0 (socket says 172.21.15.7:34468) 2023-06-09T11:36:09.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efad9d700 1 --2- 172.21.15.7:0/2714181557 >> v2:172.21.15.117:3300/0 conn(0x7f1efc153880 0x7f1efc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 -- 172.21.15.7:0/2714181557 >> v2:172.21.15.117:3300/0 conn(0x7f1efc153880 msgr2=0x7f1efc15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:09.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 --2- 172.21.15.7:0/2714181557 >> v2:172.21.15.117:3300/0 conn(0x7f1efc153880 0x7f1efc15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1efa59c700 1 -- 172.21.15.7:0/2714181557 >> v1:172.21.15.7:6789/0 conn(0x7f1efc15b2d0 legacy=0x7f1efc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1efa59c700 1 -- 172.21.15.7:0/2714181557 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc1596e0 con 0x7f1efc0c9f40 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.003+0000 7f1ef9d9b700 1 -- 172.21.15.7:0/2714181557 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4153549996 0 0) 0x7f1ef4001490 con 0x7f1efc15b2d0 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1efa59c700 1 --2- 172.21.15.7:0/2714181557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc15ad90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1efc08d340 tx=0x7f1ef0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608512b1ff0a28b4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1ef9d9b700 1 -- 172.21.15.7:0/2714181557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000b870 con 0x7f1efc0c9f40 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1ef9d9b700 1 -- 172.21.15.7:0/2714181557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef000b9d0 con 0x7f1efc0c9f40 2023-06-09T11:36:09.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1ef9d9b700 1 -- 172.21.15.7:0/2714181557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef000bd60 con 0x7f1efc0c9f40 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1f011ff700 1 -- 172.21.15.7:0/2714181557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 msgr2=0x7f1efc15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.004+0000 7f1f011ff700 1 --2- 172.21.15.7:0/2714181557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc15ad90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1efc08d340 tx=0x7f1ef0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 -- 172.21.15.7:0/2714181557 shutdown_connections 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 --2- 172.21.15.7:0/2714181557 >> v2:172.21.15.117:3300/0 conn(0x7f1efc153880 0x7f1efc15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 --2- 172.21.15.7:0/2714181557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc15ad90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 -- 172.21.15.7:0/2714181557 >> 172.21.15.7:0/2714181557 conn(0x7f1efc0737f0 msgr2=0x7f1efc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 -- 172.21.15.7:0/2714181557 shutdown_connections 2023-06-09T11:36:09.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.005+0000 7f1f011ff700 1 -- 172.21.15.7:0/2714181557 wait complete. 2023-06-09T11:36:09.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.007+0000 7f1f011ff700 1 Processor -- start 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.007+0000 7f1f011ff700 1 -- start start 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1efc153880 0x7f1efc1f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1efc15b2d0 0x7f1efc14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0cdc00 con 0x7f1efc0c9f40 2023-06-09T11:36:09.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0ce190 con 0x7f1efc153880 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.008+0000 7f1f011ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0b3950 con 0x7f1efc15b2d0 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36988/0 (socket says 172.21.15.7:36988) 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 -- 172.21.15.7:0/728362049 learned_addr learned my addr 172.21.15.7:0/728362049 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efa59c700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1efc153880 0x7f1efc1f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efb59e700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1efc15b2d0 0x7f1efc14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 -- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1efc15b2d0 msgr2=0x7f1efc14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1efc15b2d0 0x7f1efc14e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 -- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1efc153880 msgr2=0x7f1efc1f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1efc153880 0x7f1efc1f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 -- 172.21.15.7:0/728362049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef0009550 con 0x7f1efc0c9f40 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.009+0000 7f1efad9d700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00e810 tx=0x7f1eec00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.010+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec00c070 con 0x7f1efc0c9f40 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.010+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec013610 con 0x7f1efc0c9f40 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.010+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc159c60 con 0x7f1efc0c9f40 2023-06-09T11:36:09.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.010+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec0139c0 con 0x7f1efc0c9f40 2023-06-09T11:36:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.010+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc159980 con 0x7f1efc0c9f40 2023-06-09T11:36:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.011+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc159f40 con 0x7f1efc0c9f40 2023-06-09T11:36:09.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.012+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1eec013b20 con 0x7f1efc0c9f40 2023-06-09T11:36:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.013+0000 7f1ee37fe700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 0x7f1ee806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.013+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f1eec089790 con 0x7f1efc0c9f40 2023-06-09T11:36:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.013+0000 7f1efa59c700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 0x7f1ee806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:09.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.013+0000 7f1efa59c700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 0x7f1ee806d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c10 tx=0x7f1ef0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.017+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1eec054310 con 0x7f1efc0c9f40 2023-06-09T11:36:09.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.269+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1efc074cf0 con 0x7f1efc0c9f40 2023-06-09T11:36:09.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.270+0000 7f1ee37fe700 1 -- 172.21.15.7:0/728362049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f1eec057930 con 0x7f1efc0c9f40 2023-06-09T11:36:09.276 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:09.277 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 msgr2=0x7f1ee806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 0x7f1ee806d190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0009c10 tx=0x7f1ef0008030 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 msgr2=0x7f1efc0cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00e810 tx=0x7f1eec00eb20 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 shutdown_connections 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ee806ace0 0x7f1ee806d190 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1efc15b2d0 0x7f1efc14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1efc0c9f40 0x7f1efc0cd6c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 --2- 172.21.15.7:0/728362049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1efc153880 0x7f1efc1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 >> 172.21.15.7:0/728362049 conn(0x7f1efc0737f0 msgr2=0x7f1efc06ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 shutdown_connections 2023-06-09T11:36:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:09.272+0000 7f1f011ff700 1 -- 172.21.15.7:0/728362049 wait complete. 2023-06-09T11:36:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:09.296 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:11.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.503+0000 7fd7bc3ff700 1 Processor -- start 2023-06-09T11:36:11.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.503+0000 7fd7bc3ff700 1 -- start start 2023-06-09T11:36:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.504+0000 7fd7bc3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.504+0000 7fd7bc3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7b415cd40 0x7fd7b40cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.504+0000 7fd7bc3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b4159b20 con 0x7fd7b40cb5d0 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.504+0000 7fd7bc3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b40cac80 con 0x7fd7b415cd40 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.504+0000 7fd7bc3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41597d0 con 0x7fd7b40cb9a0 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba19b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd7b40cb5d0 0x7fd7b4159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34470/0 (socket says 172.21.15.7:34470) 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba19b700 1 -- 172.21.15.7:0/1577884620 learned_addr learned my addr 172.21.15.7:0/1577884620 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:11.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/1577884620 >> v2:172.21.15.7:3300/0 conn(0x7fd7b415cd40 0x7fd7b40cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7b999a700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 -- 172.21.15.7:0/1577884620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 msgr2=0x7fd7b40b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 -- 172.21.15.7:0/1577884620 >> v1:172.21.15.7:6789/0 conn(0x7fd7b40cb5d0 legacy=0x7fd7b4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 -- 172.21.15.7:0/1577884620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b4071ab0 con 0x7fd7b415cd40 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7b9199700 1 -- 172.21.15.7:0/1577884620 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4194790968 0 0) 0x7fd7b4159b20 con 0x7fd7b40cb5d0 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.505+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40cbea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009d90 tx=0x7fd7ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3184207cf0453f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:11.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7b9199700 1 -- 172.21.15.7:0/1577884620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00e070 con 0x7fd7b415cd40 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7b9199700 1 -- 172.21.15.7:0/1577884620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac00bbd0 con 0x7fd7b415cd40 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7b9199700 1 -- 172.21.15.7:0/1577884620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac012430 con 0x7fd7b415cd40 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/1577884620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 msgr2=0x7fd7b40cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40cbea0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009d90 tx=0x7fd7ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/1577884620 shutdown_connections 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40cbea0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/1577884620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/1577884620 >> 172.21.15.7:0/1577884620 conn(0x7fd7b4070af0 msgr2=0x7fd7b4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:11.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/1577884620 shutdown_connections 2023-06-09T11:36:11.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.506+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/1577884620 wait complete. 2023-06-09T11:36:11.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.509+0000 7fd7bc3ff700 1 Processor -- start 2023-06-09T11:36:11.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.509+0000 7fd7bc3ff700 1 -- start start 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 0x7fd7b41eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40ce650 con 0x7fd7b415cd40 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40cf490 con 0x7fd7b40cb5d0 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7bc3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40ce880 con 0x7fd7b40cb9a0 2023-06-09T11:36:11.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7ba99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7ba99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37014/0 (socket says 172.21.15.7:37014) 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.510+0000 7fd7ba99c700 1 -- 172.21.15.7:0/746481936 learned_addr learned my addr 172.21.15.7:0/746481936 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7b999a700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba19b700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 0x7fd7b41eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 -- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 msgr2=0x7fd7b40cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 -- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 msgr2=0x7fd7b41eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 0x7fd7b41eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 -- 172.21.15.7:0/746481936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac009a70 con 0x7fd7b415cd40 2023-06-09T11:36:11.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba19b700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 0x7fd7b41eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:11.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7ba99c700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac013000 tx=0x7fd7ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac012430 con 0x7fd7b415cd40 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac020070 con 0x7fd7b415cd40 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.511+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac004280 con 0x7fd7b415cd40 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.512+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b415d4f0 con 0x7fd7b415cd40 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.512+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b415d210 con 0x7fd7b415cd40 2023-06-09T11:36:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.513+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b415d7d0 con 0x7fd7b415cd40 2023-06-09T11:36:11.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.513+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7ac0043e0 con 0x7fd7b415cd40 2023-06-09T11:36:11.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.514+0000 7fd7aaffd700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 0x7fd79806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:11.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.515+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fd7ac022070 con 0x7fd7b415cd40 2023-06-09T11:36:11.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.515+0000 7fd7ba19b700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 0x7fd79806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:11.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.515+0000 7fd7ba19b700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 0x7fd79806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f60 tx=0x7fd7b0007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:11.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.519+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd7ac056360 con 0x7fd7b415cd40 2023-06-09T11:36:11.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.773+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b40cf600 con 0x7fd7b415cd40 2023-06-09T11:36:11.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.774+0000 7fd7aaffd700 1 -- 172.21.15.7:0/746481936 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7fd7ac059980 con 0x7fd7b415cd40 2023-06-09T11:36:11.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:11.780 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 msgr2=0x7fd79806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 0x7fd79806d190 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f60 tx=0x7fd7b0007ca0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 msgr2=0x7fd7b40c3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac013000 tx=0x7fd7ac004ad0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 shutdown_connections 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7b415cd40 0x7fd7b40c3400 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7b40cb5d0 0x7fd7b41eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd79806ace0 0x7fd79806d190 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 --2- 172.21.15.7:0/746481936 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7b40cb9a0 0x7fd7b40cf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:11.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 >> 172.21.15.7:0/746481936 conn(0x7fd7b4070af0 msgr2=0x7fd7b4159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 shutdown_connections 2023-06-09T11:36:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:11.776+0000 7fd7bc3ff700 1 -- 172.21.15.7:0/746481936 wait complete. 2023-06-09T11:36:11.784 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:11.800 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:13.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.006+0000 7f2bc03ff700 1 Processor -- start 2023-06-09T11:36:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.006+0000 7f2bc03ff700 1 -- start start 2023-06-09T11:36:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bc03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 0x7f2bb8158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bc03ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb81580a0 con 0x7f2bb80c5b40 2023-06-09T11:36:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bc03ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb80c6180 con 0x7f2bb8151510 2023-06-09T11:36:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bc03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb81518e0 con 0x7f2bb8071230 2023-06-09T11:36:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bbd99a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2bb80c5b40 0x7f2bb8070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52770/0 (socket says 172.21.15.7:52770) 2023-06-09T11:36:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.007+0000 7f2bbd99a700 1 -- 172.21.15.7:0/1090058230 learned_addr learned my addr 172.21.15.7:0/1090058230 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbe99c700 1 --2- 172.21.15.7:0/1090058230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 0x7f2bb8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212139669 0 0) 0x7f2bb81580a0 con 0x7f2bb80c5b40 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba40035b0 con 0x7f2bb80c5b40 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171712916 0 0) 0x7f2bb80c6180 con 0x7f2bb8151510 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb81580a0 con 0x7f2bb8151510 2023-06-09T11:36:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbe99c700 1 -- 172.21.15.7:0/1090058230 >> v1:172.21.15.117:6789/0 conn(0x7f2bb8151510 legacy=0x7f2bb815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbe99c700 1 -- 172.21.15.7:0/1090058230 >> v1:172.21.15.7:6789/0 conn(0x7f2bb80c5b40 legacy=0x7f2bb8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664023975 0 0) 0x7f2ba40035b0 con 0x7f2bb80c5b40 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.008+0000 7f2bbe99c700 1 -- 172.21.15.7:0/1090058230 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8157490 con 0x7f2bb8071230 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bbe99c700 1 --2- 172.21.15.7:0/1090058230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 0x7f2bb8158870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009dc0 tx=0x7f2bb000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb7bb9a861cfd61 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000e070 con 0x7f2bb8071230 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0003a20 con 0x7f2bb8071230 2023-06-09T11:36:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bbd199700 1 -- 172.21.15.7:0/1090058230 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8001670 con 0x7f2bb8071230 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bc03ff700 1 -- 172.21.15.7:0/1090058230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 msgr2=0x7f2bb8158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/1090058230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 0x7f2bb8158870 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009dc0 tx=0x7f2bb000b330 comp rx=0 tx=0).stop 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bc03ff700 1 -- 172.21.15.7:0/1090058230 shutdown_connections 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/1090058230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb8071230 0x7f2bb8158870 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.009+0000 7f2bc03ff700 1 -- 172.21.15.7:0/1090058230 >> 172.21.15.7:0/1090058230 conn(0x7f2bb806a4d0 msgr2=0x7f2bb806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.010+0000 7f2bc03ff700 1 -- 172.21.15.7:0/1090058230 shutdown_connections 2023-06-09T11:36:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.010+0000 7f2bc03ff700 1 -- 172.21.15.7:0/1090058230 wait complete. 2023-06-09T11:36:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.011+0000 7f2bc03ff700 1 Processor -- start 2023-06-09T11:36:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.011+0000 7f2bc03ff700 1 -- start start 2023-06-09T11:36:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb80c5b40 0x7f2bb814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb8151510 0x7f2bb814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb814f620 con 0x7f2bb8071230 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c4d90 con 0x7f2bb8151510 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.012+0000 7f2bc03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb81e1990 con 0x7f2bb80c5b40 2023-06-09T11:36:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35674/0 (socket says 172.21.15.7:35674) 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 -- 172.21.15.7:0/3916326562 learned_addr learned my addr 172.21.15.7:0/3916326562 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbd99a700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb80c5b40 0x7f2bb814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe99c700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb8151510 0x7f2bb814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 -- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb80c5b40 msgr2=0x7f2bb814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb80c5b40 0x7f2bb814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 -- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb8151510 msgr2=0x7f2bb814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb8151510 0x7f2bb814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 -- 172.21.15.7:0/3916326562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0009a70 con 0x7f2bb8071230 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.013+0000 7f2bbe19b700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c500 tx=0x7f2bb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.014+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40052d0 con 0x7f2bb8071230 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.014+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4005450 con 0x7f2bb8071230 2023-06-09T11:36:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.014+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40038c0 con 0x7f2bb8071230 2023-06-09T11:36:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.014+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8157a50 con 0x7f2bb8071230 2023-06-09T11:36:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.014+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8157770 con 0x7f2bb8071230 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.015+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8157d30 con 0x7f2bb8071230 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.016+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bb4003a20 con 0x7f2bb8071230 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.017+0000 7f2baeffd700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.017+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7f2bb4088f30 con 0x7f2bb8071230 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.017+0000 7f2bbd99a700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:14.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.018+0000 7f2bbd99a700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 0x7f2b9c06d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80b0f60 tx=0x7f2ba80015f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.021+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bb4053a10 con 0x7f2bb8071230 2023-06-09T11:36:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.275+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bb80c5070 con 0x7f2bb8071230 2023-06-09T11:36:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.276+0000 7f2baeffd700 1 -- 172.21.15.7:0/3916326562 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f2bb4017020 con 0x7f2bb8071230 2023-06-09T11:36:14.282 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:14.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 msgr2=0x7f2b9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 0x7f2b9c06d190 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80b0f60 tx=0x7f2ba80015f0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 msgr2=0x7f2bb814d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400c500 tx=0x7f2bb400c8c0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 shutdown_connections 2023-06-09T11:36:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bb8151510 0x7f2bb814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bb8071230 0x7f2bb814d1d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 --2- 172.21.15.7:0/3916326562 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bb80c5b40 0x7f2bb814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.278+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 >> 172.21.15.7:0/3916326562 conn(0x7f2bb806a4d0 msgr2=0x7f2bb81e1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.279+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 shutdown_connections 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:14.279+0000 7f2bc03ff700 1 -- 172.21.15.7:0/3916326562 wait complete. 2023-06-09T11:36:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:14.303 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:16.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.508+0000 7fa253dff700 1 Processor -- start 2023-06-09T11:36:16.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.508+0000 7fa253dff700 1 -- start start 2023-06-09T11:36:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa253dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24c0c5b40 0x7fa24c070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa253dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa24c071060 0x7fa24c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa253dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24c1580a0 con 0x7fa24c151510 2023-06-09T11:36:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa253dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24c1518e0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa253dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24c0c5f70 con 0x7fa24c071060 2023-06-09T11:36:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24c0c5b40 0x7fa24c070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa24c0c5b40 0x7fa24c070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35680/0 (socket says 172.21.15.7:35680) 2023-06-09T11:36:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 -- 172.21.15.7:0/1054863152 learned_addr learned my addr 172.21.15.7:0/1054863152 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25239c700 1 --2- 172.21.15.7:0/1054863152 >> v2:172.21.15.117:3300/0 conn(0x7fa24c071060 0x7fa24c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 -- 172.21.15.7:0/1054863152 >> v2:172.21.15.117:3300/0 conn(0x7fa24c071060 msgr2=0x7fa24c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:16.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 --2- 172.21.15.7:0/1054863152 >> v2:172.21.15.117:3300/0 conn(0x7fa24c071060 0x7fa24c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 -- 172.21.15.7:0/1054863152 >> v1:172.21.15.7:6789/0 conn(0x7fa24c151510 legacy=0x7fa24c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.510+0000 7fa25139a700 1 -- 172.21.15.7:0/1054863152 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0c6180 con 0x7fa24c0c5b40 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa25139a700 1 --2- 172.21.15.7:0/1054863152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c070b20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa24c06ff60 tx=0x7fa23c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a74d6462f6dc9bc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa250b99700 1 -- 172.21.15.7:0/1054863152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00b870 con 0x7fa24c0c5b40 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa250b99700 1 -- 172.21.15.7:0/1054863152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c00b9d0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa250b99700 1 -- 172.21.15.7:0/1054863152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c00bd60 con 0x7fa24c0c5b40 2023-06-09T11:36:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa253dff700 1 -- 172.21.15.7:0/1054863152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 msgr2=0x7fa24c070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa253dff700 1 --2- 172.21.15.7:0/1054863152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c070b20 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa24c06ff60 tx=0x7fa23c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.511+0000 7fa253dff700 1 -- 172.21.15.7:0/1054863152 shutdown_connections 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.512+0000 7fa253dff700 1 --2- 172.21.15.7:0/1054863152 >> v2:172.21.15.117:3300/0 conn(0x7fa24c071060 0x7fa24c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.512+0000 7fa253dff700 1 --2- 172.21.15.7:0/1054863152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c070b20 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.512+0000 7fa253dff700 1 -- 172.21.15.7:0/1054863152 >> 172.21.15.7:0/1054863152 conn(0x7fa24c06a4d0 msgr2=0x7fa24c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.512+0000 7fa253dff700 1 -- 172.21.15.7:0/1054863152 shutdown_connections 2023-06-09T11:36:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.512+0000 7fa253dff700 1 -- 172.21.15.7:0/1054863152 wait complete. 2023-06-09T11:36:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.514+0000 7fa253dff700 1 Processor -- start 2023-06-09T11:36:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.514+0000 7fa253dff700 1 -- start start 2023-06-09T11:36:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa24c071060 0x7fa24c150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24c151510 0x7fa24c155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c0cbff0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c0cb4b0 con 0x7fa24c151510 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.515+0000 7fa253dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c0cba30 con 0x7fa24c071060 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35688/0 (socket says 172.21.15.7:35688) 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 -- 172.21.15.7:0/654974730 learned_addr learned my addr 172.21.15.7:0/654974730 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa251b9b700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa24c071060 0x7fa24c150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25239c700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24c151510 0x7fa24c155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 -- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa24c071060 msgr2=0x7fa24c150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa24c071060 0x7fa24c150620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 -- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24c151510 msgr2=0x7fa24c155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24c151510 0x7fa24c155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 -- 172.21.15.7:0/654974730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c009550 con 0x7fa24c0c5b40 2023-06-09T11:36:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.516+0000 7fa25139a700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.517+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c004480 con 0x7fa24c0c5b40 2023-06-09T11:36:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.517+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23c0045e0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.517+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c157770 con 0x7fa24c0c5b40 2023-06-09T11:36:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.517+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0038e0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.517+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c157490 con 0x7fa24c0c5b40 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.518+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24c157a50 con 0x7fa24c0c5b40 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.519+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa23c021070 con 0x7fa24c0c5b40 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.520+0000 7fa2427fc700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 0x7fa23006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.520+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fa23c08aff0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.520+0000 7fa251b9b700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 0x7fa23006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.521+0000 7fa251b9b700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 0x7fa23006d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa248007fd0 tx=0x7fa248009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:16.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.524+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa23c0bd2d0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.779+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa24c074cf0 con 0x7fa24c0c5b40 2023-06-09T11:36:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.779+0000 7fa2427fc700 1 -- 172.21.15.7:0/654974730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7fa23c055b90 con 0x7fa24c0c5b40 2023-06-09T11:36:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:16.786 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 msgr2=0x7fa23006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 0x7fa23006d190 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa248007fd0 tx=0x7fa248009040 comp rx=0 tx=0).stop 2023-06-09T11:36:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 msgr2=0x7fa24c1ecf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa23c000c00 tx=0x7fa23c004750 comp rx=0 tx=0).stop 2023-06-09T11:36:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 shutdown_connections 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa24c151510 0x7fa24c155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa23006ace0 0x7fa23006d190 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa24c071060 0x7fa24c150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 --2- 172.21.15.7:0/654974730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa24c0c5b40 0x7fa24c1ecf70 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 >> 172.21.15.7:0/654974730 conn(0x7fa24c06a4d0 msgr2=0x7fa24c156930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 shutdown_connections 2023-06-09T11:36:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:16.782+0000 7fa253dff700 1 -- 172.21.15.7:0/654974730 wait complete. 2023-06-09T11:36:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:16.805 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:17.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:17.628 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:18.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.012+0000 7fe8c17ff700 1 Processor -- start 2023-06-09T11:36:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.012+0000 7fe8c17ff700 1 -- start start 2023-06-09T11:36:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8c17ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8c17ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 0x7fe8bc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8c17ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc159b20 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8c17ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc0cac80 con 0x7fe8bc15cd40 2023-06-09T11:36:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8c17ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc1597d0 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.013+0000 7fe8baffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe8bc0cb5d0 0x7fe8bc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52782/0 (socket says 172.21.15.7:52782) 2023-06-09T11:36:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8baffd700 1 -- 172.21.15.7:0/2689530450 learned_addr learned my addr 172.21.15.7:0/2689530450 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/2689530450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8bb7fe700 1 --2- 172.21.15.7:0/2689530450 >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 0x7fe8bc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8b9ffb700 1 -- 172.21.15.7:0/2689530450 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477972336 0 0) 0x7fe8bc159b20 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8b9ffb700 1 -- 172.21.15.7:0/2689530450 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a40035b0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8ba7fc700 1 -- 172.21.15.7:0/2689530450 >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 msgr2=0x7fe8bc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/2689530450 >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 0x7fe8bc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8ba7fc700 1 -- 172.21.15.7:0/2689530450 >> v1:172.21.15.7:6789/0 conn(0x7fe8bc0cb5d0 legacy=0x7fe8bc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8ba7fc700 1 -- 172.21.15.7:0/2689530450 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc071ab0 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.014+0000 7fe8bb7fe700 1 --2- 172.21.15.7:0/2689530450 >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 0x7fe8bc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/2689530450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0b52b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009840 tx=0x7fe8b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65e497bc6dbb02c0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8b9ffb700 1 -- 172.21.15.7:0/2689530450 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0011830 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8b9ffb700 1 -- 172.21.15.7:0/2689530450 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b0011990 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8b9ffb700 1 -- 172.21.15.7:0/2689530450 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0011d00 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 -- 172.21.15.7:0/2689530450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 msgr2=0x7fe8bc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/2689530450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0b52b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009840 tx=0x7fe8b0009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 -- 172.21.15.7:0/2689530450 shutdown_connections 2023-06-09T11:36:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/2689530450 >> v2:172.21.15.117:3300/0 conn(0x7fe8bc15cd40 0x7fe8bc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/2689530450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0b52b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.015+0000 7fe8c17ff700 1 -- 172.21.15.7:0/2689530450 >> 172.21.15.7:0/2689530450 conn(0x7fe8bc070af0 msgr2=0x7fe8bc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.016+0000 7fe8c17ff700 1 -- 172.21.15.7:0/2689530450 shutdown_connections 2023-06-09T11:36:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.016+0000 7fe8c17ff700 1 -- 172.21.15.7:0/2689530450 wait complete. 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.017+0000 7fe8c17ff700 1 Processor -- start 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.017+0000 7fe8c17ff700 1 -- start start 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 0x7fe8bc0c38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0c5860 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0c6510 con 0x7fe8bc0cb9a0 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8c17ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc152920 con 0x7fe8bc15cd40 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8baffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35696/0 (socket says 172.21.15.7:35696) 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.018+0000 7fe8baffd700 1 -- 172.21.15.7:0/341019425 learned_addr learned my addr 172.21.15.7:0/341019425 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8bb7fe700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 0x7fe8bc0c38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 -- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 msgr2=0x7fe8bc0c38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 0x7fe8bc0c38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 -- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc0cb9a0 msgr2=0x7fe8bc0c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 -- 172.21.15.7:0/341019425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4009a70 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8bb7fe700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 0x7fe8bc0c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8baffd700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c4d0 tx=0x7fe8ac00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac005d70 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac0070d0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac007440 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.019+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00094a0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.020+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc15d4f0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.021+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc15d210 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.022+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8ac0075a0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.023+0000 7fe8a37fe700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 0x7fe8a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.023+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 7238+0+0 (secure 0 0 0) 0x7fe8ac088d60 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.023+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 0x7fe8a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.024+0000 7fe8ba7fc700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 0x7fe8a806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cbd70 tx=0x7fe8b0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:19.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.028+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe8ac053980 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.278+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8bc15d7d0 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.279+0000 7fe8a37fe700 1 -- 172.21.15.7:0/341019425 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+6224 (secure 0 0 0) 0x7fe8ac017090 con 0x7fe8bc0cb5d0 2023-06-09T11:36:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:19.284 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":5852,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:05.622639+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5692,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5692":{"gid":5692,"name":"f","rank":1,"incarnation":67,"state":"up:active","state_seq":13,"addr":"172.21.15.117:6837/3311310614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3311310614},{"type":"v1","addr":"172.21.15.117:6837","nonce":3311310614}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.281+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 msgr2=0x7fe8a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.281+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 0x7fe8a806d190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cbd70 tx=0x7fe8b0008030 comp rx=0 tx=0).stop 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.281+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 msgr2=0x7fe8bc0c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.281+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac00c4d0 tx=0x7fe8ac00c7e0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 shutdown_connections 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe8a806ace0 0x7fe8a806d190 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8bc15cd40 0x7fe8bc0c38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8bc0cb5d0 0x7fe8bc0c5320 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 --2- 172.21.15.7:0/341019425 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8bc0cb9a0 0x7fe8bc0c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 >> 172.21.15.7:0/341019425 conn(0x7fe8bc070af0 msgr2=0x7fe8bc0b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 shutdown_connections 2023-06-09T11:36:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:19.282+0000 7fe8c17ff700 1 -- 172.21.15.7:0/341019425 wait complete. 2023-06-09T11:36:19.288 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2023-06-09T11:36:19.304 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:36:21.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.512+0000 7f8a19bff700 1 Processor -- start 2023-06-09T11:36:21.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.512+0000 7f8a19bff700 1 -- start start 2023-06-09T11:36:21.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.513+0000 7f8a19bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1415b4a0 0x7f8a1415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a19bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1415a500 con 0x7f8a140c9f40 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a19bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1415da00 con 0x7f8a14153880 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a19bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140c5f10 con 0x7f8a1415b4a0 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a12ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1415b4a0 0x7f8a1415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a13fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a140c9f40 0x7f8a140c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52788/0 (socket says 172.21.15.7:52788) 2023-06-09T11:36:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a13fff700 1 -- 172.21.15.7:0/3413428262 learned_addr learned my addr 172.21.15.7:0/3413428262 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a12ffd700 1 -- 172.21.15.7:0/3413428262 >> v1:172.21.15.117:6789/0 conn(0x7f8a14153880 legacy=0x7f8a1415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a12ffd700 1 -- 172.21.15.7:0/3413428262 >> v1:172.21.15.7:6789/0 conn(0x7f8a140c9f40 legacy=0x7f8a140c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a12ffd700 1 -- 172.21.15.7:0/3413428262 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a141596e0 con 0x7f8a1415b4a0 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.514+0000 7f8a127fc700 1 -- 172.21.15.7:0/3413428262 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900475791 0 0) 0x7f8a0c001490 con 0x7f8a140c9f40 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a12ffd700 1 --2- 172.21.15.7:0/3413428262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1415b4a0 0x7f8a1415d4c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009840 tx=0x7f8a08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469f1fcb574560b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a127fc700 1 -- 172.21.15.7:0/3413428262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011830 con 0x7f8a1415b4a0 2023-06-09T11:36:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a127fc700 1 -- 172.21.15.7:0/3413428262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a08011990 con 0x7f8a1415b4a0 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a127fc700 1 -- 172.21.15.7:0/3413428262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08011d20 con 0x7f8a1415b4a0 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a19bff700 1 -- 172.21.15.7:0/3413428262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1415b4a0 msgr2=0x7f8a1415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a19bff700 1 --2- 172.21.15.7:0/3413428262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1415b4a0 0x7f8a1415d4c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009840 tx=0x7f8a08009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a19bff700 1 -- 172.21.15.7:0/3413428262 shutdown_connections 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a19bff700 1 --2- 172.21.15.7:0/3413428262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1415b4a0 0x7f8a1415d4c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.515+0000 7f8a19bff700 1 -- 172.21.15.7:0/3413428262 >> 172.21.15.7:0/3413428262 conn(0x7f8a140737f0 msgr2=0x7f8a140720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.516+0000 7f8a19bff700 1 -- 172.21.15.7:0/3413428262 shutdown_connections 2023-06-09T11:36:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.516+0000 7f8a19bff700 1 -- 172.21.15.7:0/3413428262 wait complete. 2023-06-09T11:36:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.517+0000 7f8a19bff700 1 Processor -- start 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.517+0000 7f8a19bff700 1 -- start start 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a140c9f40 0x7f8a141efe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 0x7f8a140cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140cb870 con 0x7f8a14153880 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140ceab0 con 0x7f8a140c9f40 2023-06-09T11:36:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.518+0000 7f8a19bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141e5a90 con 0x7f8a1415b4a0 2023-06-09T11:36:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35718/0 (socket says 172.21.15.7:35718) 2023-06-09T11:36:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 -- 172.21.15.7:0/2500766967 learned_addr learned my addr 172.21.15.7:0/2500766967 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a137fe700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a140c9f40 0x7f8a141efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a13fff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 0x7f8a140cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 -- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 msgr2=0x7f8a140cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 0x7f8a140cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 -- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a140c9f40 msgr2=0x7f8a141efe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a140c9f40 0x7f8a141efe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 -- 172.21.15.7:0/2500766967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080094a0 con 0x7f8a14153880 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a13fff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 0x7f8a140cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.519+0000 7f8a12ffd700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8a140ca590 tx=0x7f8a08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.520+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0801a810 con 0x7f8a14153880 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.520+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0801a970 con 0x7f8a14153880 2023-06-09T11:36:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.520+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0801ad00 con 0x7f8a14153880 2023-06-09T11:36:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.520+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a14159c60 con 0x7f8a14153880 2023-06-09T11:36:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.520+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a14159980 con 0x7f8a14153880 2023-06-09T11:36:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.521+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a14159f40 con 0x7f8a14153880 2023-06-09T11:36:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.522+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a08022070 con 0x7f8a14153880 2023-06-09T11:36:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.523+0000 7f89fbfff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 0x7f8a0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.523+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f8a0808a880 con 0x7f8a14153880 2023-06-09T11:36:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.523+0000 7f8a137fe700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 0x7f8a0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.524+0000 7f8a137fe700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 0x7f8a0006d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8a04007fd0 tx=0x7f8a04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:21.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.527+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8a08055430 con 0x7f8a14153880 2023-06-09T11:36:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.782+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a14074cf0 con 0x7f8a14153880 2023-06-09T11:36:21.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.782+0000 7f89fbfff700 1 -- 172.21.15.7:0/2500766967 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5460 (secure 0 0 0) 0x7f8a0801ae60 con 0x7f8a14153880 2023-06-09T11:36:21.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:21.788 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:19.460966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 msgr2=0x7f8a0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 0x7f8a0006d190 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8a04007fd0 tx=0x7f8a04009040 comp rx=0 tx=0).stop 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 msgr2=0x7f8a140cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8a140ca590 tx=0x7f8a08004750 comp rx=0 tx=0).stop 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 shutdown_connections 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ace0 0x7f8a0006d190 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1415b4a0 0x7f8a140cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a140c9f40 0x7f8a141efe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 --2- 172.21.15.7:0/2500766967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a14153880 0x7f8a140cacf0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 >> 172.21.15.7:0/2500766967 conn(0x7f8a140737f0 msgr2=0x7f8a140720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 shutdown_connections 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:21.785+0000 7f8a19bff700 1 -- 172.21.15.7:0/2500766967 wait complete. 2023-06-09T11:36:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2023-06-09T11:36:21.807 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:36:21.807 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:36:21.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.002+0000 7fe38efff700 1 Processor -- start 2023-06-09T11:36:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.002+0000 7fe38efff700 1 -- start start 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38efff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe388153880 0x7fe38815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38efff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3880c9f40 0x7fe38815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38efff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38815a500 con 0x7fe38815b2d0 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38efff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3880c5f10 con 0x7fe388153880 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38efff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38815da00 con 0x7fe3880c9f40 2023-06-09T11:36:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38cd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe388153880 0x7fe38815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38cd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe388153880 0x7fe38815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35724/0 (socket says 172.21.15.7:35724) 2023-06-09T11:36:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe38cd9b700 1 -- 172.21.15.7:0/1409948566 learned_addr learned my addr 172.21.15.7:0/1409948566 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.004+0000 7fe387fff700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3880c9f40 0x7fe38815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe38cd9b700 1 -- 172.21.15.7:0/1409948566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3880c9f40 msgr2=0x7fe38815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe38cd9b700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3880c9f40 0x7fe38815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe38cd9b700 1 -- 172.21.15.7:0/1409948566 >> v1:172.21.15.117:6789/0 conn(0x7fe38815b2d0 legacy=0x7fe3880c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe38cd9b700 1 -- 172.21.15.7:0/1409948566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38815b750 con 0x7fe388153880 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe38cd9b700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe388153880 0x7fe38815d4c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0098f0 tx=0x7fe37c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40db183adcef41a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe3877fe700 1 -- 172.21.15.7:0/1409948566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00b8a0 con 0x7fe388153880 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe3877fe700 1 -- 172.21.15.7:0/1409948566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37c00ba00 con 0x7fe388153880 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.005+0000 7fe3877fe700 1 -- 172.21.15.7:0/1409948566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c00bd90 con 0x7fe388153880 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 -- 172.21.15.7:0/1409948566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe388153880 msgr2=0x7fe38815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe388153880 0x7fe38815d4c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0098f0 tx=0x7fe37c009c00 comp rx=0 tx=0).stop 2023-06-09T11:36:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 -- 172.21.15.7:0/1409948566 shutdown_connections 2023-06-09T11:36:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe388153880 0x7fe38815d4c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 --2- 172.21.15.7:0/1409948566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3880c9f40 0x7fe38815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 -- 172.21.15.7:0/1409948566 >> 172.21.15.7:0/1409948566 conn(0x7fe3880737f0 msgr2=0x7fe3880720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 -- 172.21.15.7:0/1409948566 shutdown_connections 2023-06-09T11:36:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.006+0000 7fe38efff700 1 -- 172.21.15.7:0/1409948566 wait complete. 2023-06-09T11:36:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.008+0000 7fe38efff700 1 Processor -- start 2023-06-09T11:36:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.009+0000 7fe38efff700 1 -- start start 2023-06-09T11:36:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3880c9f40 0x7fe3880c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 0x7fe3880cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe388158360 con 0x7fe38815b2d0 2023-06-09T11:36:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3880c69d0 con 0x7fe3880c9f40 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38efff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881f2730 con 0x7fe388153880 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38d59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38d59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35734/0 (socket says 172.21.15.7:35734) 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38d59c700 1 -- 172.21.15.7:0/2103770396 learned_addr learned my addr 172.21.15.7:0/2103770396 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe38cd9b700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3880c9f40 0x7fe3880c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.010+0000 7fe387fff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 0x7fe3880cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 -- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 msgr2=0x7fe3880cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 0x7fe3880cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 -- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3880c9f40 msgr2=0x7fe3880c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3880c9f40 0x7fe3880c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 -- 172.21.15.7:0/2103770396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009550 con 0x7fe38815b2d0 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe387fff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 0x7fe3880cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38d59c700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe38000cb50 tx=0x7fe38000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380003c00 con 0x7fe38815b2d0 2023-06-09T11:36:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe380003d60 con 0x7fe38815b2d0 2023-06-09T11:36:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe388159980 con 0x7fe38815b2d0 2023-06-09T11:36:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38000aac0 con 0x7fe38815b2d0 2023-06-09T11:36:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.011+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881596e0 con 0x7fe38815b2d0 2023-06-09T11:36:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.012+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe388159c60 con 0x7fe38815b2d0 2023-06-09T11:36:22.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.013+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe38000ac20 con 0x7fe38815b2d0 2023-06-09T11:36:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.014+0000 7fe3857fa700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 0x7fe36c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.014+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fe380089850 con 0x7fe38815b2d0 2023-06-09T11:36:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.015+0000 7fe38cd9b700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 0x7fe36c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:22.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.015+0000 7fe38cd9b700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 0x7fe36c06d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:22.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.019+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe380054370 con 0x7fe38815b2d0 2023-06-09T11:36:22.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.271+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe38815a500 con 0x7fe38815b2d0 2023-06-09T11:36:22.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.272+0000 7fe3857fa700 1 -- 172.21.15.7:0/2103770396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5460 (secure 0 0 0) 0x7fe38815a500 con 0x7fe38815b2d0 2023-06-09T11:36:22.277 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:22.277 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:19.460966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 msgr2=0x7fe36c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 0x7fe36c06d190 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009c40 tx=0x7fe37c008030 comp rx=0 tx=0).stop 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 msgr2=0x7fe3881f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe38000cb50 tx=0x7fe38000ce60 comp rx=0 tx=0).stop 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 shutdown_connections 2023-06-09T11:36:22.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe38815b2d0 0x7fe3881f2b80 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3880c9f40 0x7fe3880c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36c06ace0 0x7fe36c06d190 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 --2- 172.21.15.7:0/2103770396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe388153880 0x7fe3880cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 >> 172.21.15.7:0/2103770396 conn(0x7fe3880737f0 msgr2=0x7fe38815b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 shutdown_connections 2023-06-09T11:36:22.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:22.274+0000 7fe38efff700 1 -- 172.21.15.7:0/2103770396 wait complete. 2023-06-09T11:36:22.281 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2023-06-09T11:36:22.297 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 116, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 116, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5741": { "addr": "172.21.15.117:6833/557223025", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 557223025, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 557223025, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5741, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5852": { "addr": "172.21.15.117:6835/1343666954", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1343666954, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1343666954, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5852, "incarnation": 116, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 63, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:36:19.460966+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5741, "mds_1": 5852, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:36:24.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:24.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.514+0000 7fb734fff700 1 Processor -- start 2023-06-09T11:36:24.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.515+0000 7fb734fff700 1 -- start start 2023-06-09T11:36:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.516+0000 7fb734fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb73015b060 0x7fb7300c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.516+0000 7fb734fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.516+0000 7fb734fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300c9bd0 con 0x7fb7300c2ae0 2023-06-09T11:36:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.516+0000 7fb734fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb730159360 con 0x7fb73015b060 2023-06-09T11:36:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.516+0000 7fb734fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c8e20 con 0x7fb7300c88e0 2023-06-09T11:36:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb7300c2ae0 0x7fb73015a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44138/0 (socket says 172.21.15.7:44138) 2023-06-09T11:36:24.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72e59c700 1 -- 172.21.15.7:0/1118225136 learned_addr learned my addr 172.21.15.7:0/1118225136 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72dd9b700 1 --2- 172.21.15.7:0/1118225136 >> v2:172.21.15.117:3300/0 conn(0x7fb73015b060 0x7fb7300c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72ed9d700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033717678 0 0) 0x7fb7300c9bd0 con 0x7fb7300c2ae0 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7180035b0 con 0x7fb7300c2ae0 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72dd9b700 1 -- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 msgr2=0x7fb730163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72dd9b700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72dd9b700 1 -- 172.21.15.7:0/1118225136 >> v1:172.21.15.7:6789/0 conn(0x7fb7300c2ae0 legacy=0x7fb73015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475426989 0 0) 0x7fb7180035b0 con 0x7fb7300c2ae0 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72dd9b700 1 -- 172.21.15.7:0/1118225136 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730159570 con 0x7fb73015b060 2023-06-09T11:36:24.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.517+0000 7fb72ed9d700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.518+0000 7fb72dd9b700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73015b060 0x7fb7300c2eb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb724009840 tx=0x7fb724009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecc840a2eb20961 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.518+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011830 con 0x7fb73015b060 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.518+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb724011990 con 0x7fb73015b060 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.518+0000 7fb72d59a700 1 -- 172.21.15.7:0/1118225136 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724011d00 con 0x7fb73015b060 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 -- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73015b060 msgr2=0x7fb7300c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73015b060 0x7fb7300c2eb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb724009840 tx=0x7fb724009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 -- 172.21.15.7:0/1118225136 shutdown_connections 2023-06-09T11:36:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 --2- 172.21.15.7:0/1118225136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb73015b060 0x7fb7300c2eb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 -- 172.21.15.7:0/1118225136 >> 172.21.15.7:0/1118225136 conn(0x7fb7300cd330 msgr2=0x7fb7300cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:24.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 -- 172.21.15.7:0/1118225136 shutdown_connections 2023-06-09T11:36:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.519+0000 7fb734fff700 1 -- 172.21.15.7:0/1118225136 wait complete. 2023-06-09T11:36:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.521+0000 7fb734fff700 1 Processor -- start 2023-06-09T11:36:24.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.521+0000 7fb734fff700 1 -- start start 2023-06-09T11:36:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 0x7fb730147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301e3720 con 0x7fb73015b060 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7301e2280 con 0x7fb7300c2ae0 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.522+0000 7fb734fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cc0d0 con 0x7fb7300c88e0 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58774/0 (socket says 172.21.15.7:58774) 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 -- 172.21.15.7:0/3967972181 learned_addr learned my addr 172.21.15.7:0/3967972181 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72dd9b700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72e59c700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 0x7fb730147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 -- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 msgr2=0x7fb730148320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730148320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 -- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 msgr2=0x7fb730147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 0x7fb730147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 -- 172.21.15.7:0/3967972181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728009a70 con 0x7fb73015b060 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72e59c700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 0x7fb730147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:24.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.523+0000 7fb72ed9d700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb72800efc0 tx=0x7fb72800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.524+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800cb40 con 0x7fb73015b060 2023-06-09T11:36:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.524+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72800f040 con 0x7fb73015b060 2023-06-09T11:36:24.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.524+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7240094a0 con 0x7fb73015b060 2023-06-09T11:36:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.524+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728003a20 con 0x7fb73015b060 2023-06-09T11:36:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.524+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730159af0 con 0x7fb73015b060 2023-06-09T11:36:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.525+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb730159810 con 0x7fb73015b060 2023-06-09T11:36:24.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.526+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb728014480 con 0x7fb73015b060 2023-06-09T11:36:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.527+0000 7fb71effd700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 0x7fb71006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.528+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fb72808a350 con 0x7fb73015b060 2023-06-09T11:36:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.528+0000 7fb72e59c700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 0x7fb71006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:24.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.528+0000 7fb72e59c700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 0x7fb71006d1c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb720001820 tx=0x7fb720009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:24.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.532+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb728054f00 con 0x7fb73015b060 2023-06-09T11:36:24.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.786+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb730159dd0 con 0x7fb73015b060 2023-06-09T11:36:24.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.786+0000 7fb71effd700 1 -- 172.21.15.7:0/3967972181 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 116 v116) v1 ==== 95+0+5460 (secure 0 0 0) 0x7fb728058520 con 0x7fb73015b060 2023-06-09T11:36:24.791 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:24.791 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":116,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":116,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:19.460966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 msgr2=0x7fb71006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 0x7fb71006d1c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb720001820 tx=0x7fb720009040 comp rx=0 tx=0).stop 2023-06-09T11:36:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 msgr2=0x7fb7300cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb72800efc0 tx=0x7fb72800c3a0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 shutdown_connections 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb73015b060 0x7fb7300cc560 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7300c2ae0 0x7fb730147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb71006ad10 0x7fb71006d1c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 --2- 172.21.15.7:0/3967972181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7300c88e0 0x7fb730148320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 >> 172.21.15.7:0/3967972181 conn(0x7fb7300cd330 msgr2=0x7fb730157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 shutdown_connections 2023-06-09T11:36:24.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:24.789+0000 7fb734fff700 1 -- 172.21.15.7:0/3967972181 wait complete. 2023-06-09T11:36:24.796 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 116 2023-06-09T11:36:24.812 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:26.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.019+0000 7fc6655ff700 1 Processor -- start 2023-06-09T11:36:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.019+0000 7fc6655ff700 1 -- start start 2023-06-09T11:36:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.020+0000 7fc6655ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 0x7fc66015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.020+0000 7fc6655ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc66015b4a0 0x7fc6600c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.020+0000 7fc6655ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66015a500 con 0x7fc6600c9f40 2023-06-09T11:36:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.020+0000 7fc6655ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600c5f10 con 0x7fc66015b4a0 2023-06-09T11:36:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.020+0000 7fc6655ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66015da00 con 0x7fc660153880 2023-06-09T11:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc66015b4a0 0x7fc6600c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc66015b4a0 0x7fc6600c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58776/0 (socket says 172.21.15.7:58776) 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 -- 172.21.15.7:0/1505024714 learned_addr learned my addr 172.21.15.7:0/1505024714 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65effd700 1 --2- 172.21.15.7:0/1505024714 >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 0x7fc66015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 -- 172.21.15.7:0/1505024714 >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 msgr2=0x7fc66015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 --2- 172.21.15.7:0/1505024714 >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 0x7fc66015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 -- 172.21.15.7:0/1505024714 >> v1:172.21.15.117:6789/0 conn(0x7fc6600c9f40 legacy=0x7fc66015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 -- 172.21.15.7:0/1505024714 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6601596e0 con 0x7fc66015b4a0 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65effd700 1 --2- 172.21.15.7:0/1505024714 >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 0x7fc66015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.021+0000 7fc65f7fe700 1 --2- 172.21.15.7:0/1505024714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc66015b4a0 0x7fc6600c4e30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc66008d340 tx=0x7fc65800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ff49217e3ab42e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc65dffb700 1 -- 172.21.15.7:0/1505024714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800e070 con 0x7fc66015b4a0 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc65dffb700 1 -- 172.21.15.7:0/1505024714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc658003a20 con 0x7fc66015b4a0 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc65dffb700 1 -- 172.21.15.7:0/1505024714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658003dd0 con 0x7fc66015b4a0 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 -- 172.21.15.7:0/1505024714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc66015b4a0 msgr2=0x7fc6600c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1505024714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc66015b4a0 0x7fc6600c4e30 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fc66008d340 tx=0x7fc65800b330 comp rx=0 tx=0).stop 2023-06-09T11:36:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 -- 172.21.15.7:0/1505024714 shutdown_connections 2023-06-09T11:36:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1505024714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc66015b4a0 0x7fc6600c4e30 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1505024714 >> v2:172.21.15.117:3300/0 conn(0x7fc660153880 0x7fc66015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 -- 172.21.15.7:0/1505024714 >> 172.21.15.7:0/1505024714 conn(0x7fc6600737f0 msgr2=0x7fc6600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 -- 172.21.15.7:0/1505024714 shutdown_connections 2023-06-09T11:36:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.022+0000 7fc6655ff700 1 -- 172.21.15.7:0/1505024714 wait complete. 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.024+0000 7fc6655ff700 1 Processor -- start 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.024+0000 7fc6655ff700 1 -- start start 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 0x7fc6600b3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc66015b4a0 0x7fc6600b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66014da60 con 0x7fc6600c9f40 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600b30c0 con 0x7fc660153880 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc6655ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6601f3500 con 0x7fc66015b4a0 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc65effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc65effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58784/0 (socket says 172.21.15.7:58784) 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.025+0000 7fc65effd700 1 -- 172.21.15.7:0/1335303597 learned_addr learned my addr 172.21.15.7:0/1335303597 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65e7fc700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 0x7fc6600b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:27.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 -- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc66015b4a0 msgr2=0x7fc6600b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc66015b4a0 0x7fc6600b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 -- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 msgr2=0x7fc6600b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 0x7fc6600b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 -- 172.21.15.7:0/1335303597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658009b30 con 0x7fc6600c9f40 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65e7fc700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 0x7fc6600b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc65effd700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c750 tx=0x7fc65000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650011800 con 0x7fc6600c9f40 2023-06-09T11:36:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc650011960 con 0x7fc6600c9f40 2023-06-09T11:36:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650011cd0 con 0x7fc6600c9f40 2023-06-09T11:36:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc660159c60 con 0x7fc6600c9f40 2023-06-09T11:36:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.026+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc660159980 con 0x7fc6600c9f40 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.027+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc660159f40 con 0x7fc6600c9f40 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.028+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc650021350 con 0x7fc6600c9f40 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.029+0000 7fc6477fe700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 0x7fc64c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.029+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fc6500899b0 con 0x7fc6600c9f40 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.030+0000 7fc65e7fc700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 0x7fc64c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:27.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.030+0000 7fc65e7fc700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 0x7fc64c06d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cc5a0 tx=0x7fc654009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:27.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.033+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc650057f60 con 0x7fc6600c9f40 2023-06-09T11:36:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.291+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6600b2f00 con 0x7fc6600c9f40 2023-06-09T11:36:27.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.291+0000 7fc6477fe700 1 -- 172.21.15.7:0/1335303597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5462 (secure 0 0 0) 0x7fc650057af0 con 0x7fc6600c9f40 2023-06-09T11:36:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:27.297 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:26.468346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:resolve","state_seq":18,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 msgr2=0x7fc64c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 0x7fc64c06d190 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cc5a0 tx=0x7fc654009040 comp rx=0 tx=0).stop 2023-06-09T11:36:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 msgr2=0x7fc6600cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc65000c750 tx=0x7fc65000cb10 comp rx=0 tx=0).stop 2023-06-09T11:36:27.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 shutdown_connections 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc64c06ace0 0x7fc64c06d190 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc66015b4a0 0x7fc6600b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6600c9f40 0x7fc6600cc5f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 --2- 172.21.15.7:0/1335303597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc660153880 0x7fc6600b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 >> 172.21.15.7:0/1335303597 conn(0x7fc6600737f0 msgr2=0x7fc66015b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 shutdown_connections 2023-06-09T11:36:27.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:27.294+0000 7fc6655ff700 1 -- 172.21.15.7:0/1335303597 wait complete. 2023-06-09T11:36:27.301 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 117 2023-06-09T11:36:27.317 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:29.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:29.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.523+0000 7f714e1ff700 1 Processor -- start 2023-06-09T11:36:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.523+0000 7f714e1ff700 1 -- start start 2023-06-09T11:36:29.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.525+0000 7f714e1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 0x7f71480b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.525+0000 7f714e1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 0x7f7148159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.525+0000 7f714e1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7148159b20 con 0x7f714815cd40 2023-06-09T11:36:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.525+0000 7f714e1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480cac80 con 0x7f71480cb5d0 2023-06-09T11:36:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.525+0000 7f714e1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71481597d0 con 0x7f71480cb9a0 2023-06-09T11:36:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7147fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f714815cd40 0x7f71480cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44154/0 (socket says 172.21.15.7:44154) 2023-06-09T11:36:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7147fff700 1 -- 172.21.15.7:0/2859850787 learned_addr learned my addr 172.21.15.7:0/2859850787 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f71477fe700 1 --2- 172.21.15.7:0/2859850787 >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 0x7f71480b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2859850787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 0x7f7148159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f71467fc700 1 -- 172.21.15.7:0/2859850787 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987450039 0 0) 0x7f7148159b20 con 0x7f714815cd40 2023-06-09T11:36:29.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f71467fc700 1 -- 172.21.15.7:0/2859850787 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71300035b0 con 0x7f714815cd40 2023-06-09T11:36:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 -- 172.21.15.7:0/2859850787 >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 msgr2=0x7f71480b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2859850787 >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 0x7f71480b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 -- 172.21.15.7:0/2859850787 >> v1:172.21.15.7:6789/0 conn(0x7f714815cd40 legacy=0x7f71480cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 -- 172.21.15.7:0/2859850787 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148071ab0 con 0x7f71480cb9a0 2023-06-09T11:36:29.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f71477fe700 1 --2- 172.21.15.7:0/2859850787 >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 0x7f71480b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:36:29.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.526+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2859850787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 0x7f7148159fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f713c009840 tx=0x7f713c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3c1dd1b6a68820 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:29.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f71467fc700 1 -- 172.21.15.7:0/2859850787 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011830 con 0x7f71480cb9a0 2023-06-09T11:36:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f71467fc700 1 -- 172.21.15.7:0/2859850787 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c011990 con 0x7f71480cb9a0 2023-06-09T11:36:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f71467fc700 1 -- 172.21.15.7:0/2859850787 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c011d20 con 0x7f71480cb9a0 2023-06-09T11:36:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 -- 172.21.15.7:0/2859850787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 msgr2=0x7f7148159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2859850787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 0x7f7148159fb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f713c009840 tx=0x7f713c009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 -- 172.21.15.7:0/2859850787 shutdown_connections 2023-06-09T11:36:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2859850787 >> v2:172.21.15.117:3300/0 conn(0x7f71480cb5d0 0x7f71480b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2859850787 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb9a0 0x7f7148159fb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 -- 172.21.15.7:0/2859850787 >> 172.21.15.7:0/2859850787 conn(0x7f7148070af0 msgr2=0x7f7148070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 -- 172.21.15.7:0/2859850787 shutdown_connections 2023-06-09T11:36:29.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.527+0000 7f714e1ff700 1 -- 172.21.15.7:0/2859850787 wait complete. 2023-06-09T11:36:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.530+0000 7f714e1ff700 1 Processor -- start 2023-06-09T11:36:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.530+0000 7f714e1ff700 1 -- start start 2023-06-09T11:36:29.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb5d0 0x7f71480c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714815cd40 0x7f71480b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480c5440 con 0x7f71480cb9a0 2023-06-09T11:36:29.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71480d04f0 con 0x7f714815cd40 2023-06-09T11:36:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f714e1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480c8a70 con 0x7f71480cb5d0 2023-06-09T11:36:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.531+0000 7f7146ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58792/0 (socket says 172.21.15.7:58792) 2023-06-09T11:36:29.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 -- 172.21.15.7:0/2016208174 learned_addr learned my addr 172.21.15.7:0/2016208174 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7147fff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714815cd40 0x7f71480b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f71477fe700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb5d0 0x7f71480c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 -- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb5d0 msgr2=0x7f71480c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb5d0 0x7f71480c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 -- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714815cd40 msgr2=0x7f71480b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714815cd40 0x7f71480b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 -- 172.21.15.7:0/2016208174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380094a0 con 0x7f71480cb9a0 2023-06-09T11:36:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f7146ffd700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c004590 con 0x7f71480cb9a0 2023-06-09T11:36:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.532+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c01f070 con 0x7f71480cb9a0 2023-06-09T11:36:29.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.533+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c0038e0 con 0x7f71480cb9a0 2023-06-09T11:36:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.533+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c0094a0 con 0x7f71480cb9a0 2023-06-09T11:36:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.533+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714815d4f0 con 0x7f71480cb9a0 2023-06-09T11:36:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.534+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714815d210 con 0x7f71480cb9a0 2023-06-09T11:36:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.535+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f713c003a40 con 0x7f71480cb9a0 2023-06-09T11:36:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.536+0000 7f712ffff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 0x7f713406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.536+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f713c021070 con 0x7f71480cb9a0 2023-06-09T11:36:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.536+0000 7f71477fe700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 0x7f713406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.537+0000 7f71477fe700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 0x7f713406d1c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7138009d70 tx=0x7f7138009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.541+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f713c055930 con 0x7f71480cb9a0 2023-06-09T11:36:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.794+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f714815d7d0 con 0x7f71480cb9a0 2023-06-09T11:36:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.795+0000 7f712ffff700 1 -- 172.21.15.7:0/2016208174 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f713c058f50 con 0x7f71480cb9a0 2023-06-09T11:36:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":117,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:26.468346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:resolve","state_seq":18,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 msgr2=0x7f713406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 0x7f713406d1c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7138009d70 tx=0x7f7138009210 comp rx=0 tx=0).stop 2023-06-09T11:36:29.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 msgr2=0x7f71480cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f713c000c00 tx=0x7f713c004750 comp rx=0 tx=0).stop 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 shutdown_connections 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f714815cd40 0x7f71480b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f713406ad10 0x7f713406d1c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71480cb5d0 0x7f71480c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 --2- 172.21.15.7:0/2016208174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71480cb9a0 0x7f71480cd560 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 >> 172.21.15.7:0/2016208174 conn(0x7f7148070af0 msgr2=0x7f71480c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 shutdown_connections 2023-06-09T11:36:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:29.797+0000 7f714e1ff700 1 -- 172.21.15.7:0/2016208174 wait complete. 2023-06-09T11:36:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 117 2023-06-09T11:36:29.820 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:31.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:32.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.025+0000 7f14dc9ff700 1 Processor -- start 2023-06-09T11:36:32.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.025+0000 7f14dc9ff700 1 -- start start 2023-06-09T11:36:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14dc9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14d8153880 0x7f14d815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14dc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14dc9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14d815a500 con 0x7f14d80c9f40 2023-06-09T11:36:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14dc9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d80c5f10 con 0x7f14d8153880 2023-06-09T11:36:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14dc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d815da00 con 0x7f14d815b4a0 2023-06-09T11:36:32.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14d8153880 0x7f14d815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14d8153880 0x7f14d815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58798/0 (socket says 172.21.15.7:58798) 2023-06-09T11:36:32.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 -- 172.21.15.7:0/3629276577 learned_addr learned my addr 172.21.15.7:0/3629276577 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:32.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d5d9b700 1 --1- 172.21.15.7:0/3629276577 >> v1:172.21.15.7:6789/0 conn(0x7f14d80c9f40 0x7f14d815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44164/0 (socket says 172.21.15.7:44164) 2023-06-09T11:36:32.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d6d9d700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 -- 172.21.15.7:0/3629276577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 msgr2=0x7f14d80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 -- 172.21.15.7:0/3629276577 >> v1:172.21.15.7:6789/0 conn(0x7f14d80c9f40 legacy=0x7f14d815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.027+0000 7f14d659c700 1 -- 172.21.15.7:0/3629276577 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d81596e0 con 0x7f14d8153880 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14d659c700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d8153880 0x7f14d815d4c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f14c80098f0 tx=0x7f14c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df036d9b384eda server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14d559a700 1 -- 172.21.15.7:0/3629276577 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800b8a0 con 0x7f14d8153880 2023-06-09T11:36:32.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14d559a700 1 -- 172.21.15.7:0/3629276577 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c800ba00 con 0x7f14d8153880 2023-06-09T11:36:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14d559a700 1 -- 172.21.15.7:0/3629276577 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c800bd90 con 0x7f14d8153880 2023-06-09T11:36:32.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14dc9ff700 1 -- 172.21.15.7:0/3629276577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d8153880 msgr2=0x7f14d815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.028+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d8153880 0x7f14d815d4c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f14c80098f0 tx=0x7f14c8009c00 comp rx=0 tx=0).stop 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 -- 172.21.15.7:0/3629276577 shutdown_connections 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/3629276577 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d8153880 0x7f14d815d4c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 -- 172.21.15.7:0/3629276577 >> 172.21.15.7:0/3629276577 conn(0x7f14d80737f0 msgr2=0x7f14d80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 -- 172.21.15.7:0/3629276577 shutdown_connections 2023-06-09T11:36:32.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.029+0000 7f14dc9ff700 1 -- 172.21.15.7:0/3629276577 wait complete. 2023-06-09T11:36:32.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.030+0000 7f14dc9ff700 1 Processor -- start 2023-06-09T11:36:32.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 -- start start 2023-06-09T11:36:32.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14d8153880 0x7f14d80ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81efc90 con 0x7f14d80c9f40 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d814b5c0 con 0x7f14d8153880 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.031+0000 7f14dc9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d81f00b0 con 0x7f14d815b4a0 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58812/0 (socket says 172.21.15.7:58812) 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 -- 172.21.15.7:0/4267723841 learned_addr learned my addr 172.21.15.7:0/4267723841 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:32.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d6d9d700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d5d9b700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14d8153880 0x7f14d80ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 -- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 msgr2=0x7f14d80cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 -- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14d8153880 msgr2=0x7f14d80ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14d8153880 0x7f14d80ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 -- 172.21.15.7:0/4267723841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c8009550 con 0x7f14d80c9f40 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14d659c700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f14c8000c00 tx=0x7f14c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004480 con 0x7f14d80c9f40 2023-06-09T11:36:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.032+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c801f070 con 0x7f14d80c9f40 2023-06-09T11:36:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.033+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d8159c60 con 0x7f14d80c9f40 2023-06-09T11:36:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.033+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c80038c0 con 0x7f14d80c9f40 2023-06-09T11:36:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.033+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d8159980 con 0x7f14d80c9f40 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.034+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14d8159f40 con 0x7f14d80c9f40 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.035+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14c8021030 con 0x7f14d80c9f40 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.036+0000 7f14c6ffd700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 0x7f14b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.036+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f14c808aee0 con 0x7f14d80c9f40 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.036+0000 7f14d5d9b700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 0x7f14b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.037+0000 7f14d5d9b700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 0x7f14b806d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14d806b450 tx=0x7f14cc007d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.041+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f14c8055bb0 con 0x7f14d80c9f40 2023-06-09T11:36:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.296+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14d81eff00 con 0x7f14d80c9f40 2023-06-09T11:36:32.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.297+0000 7f14c6ffd700 1 -- 172.21.15.7:0/4267723841 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f14c80591d0 con 0x7f14d80c9f40 2023-06-09T11:36:32.302 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:32.302 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:31.623739+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 msgr2=0x7f14b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 0x7f14b806d190 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f14d806b450 tx=0x7f14cc007d10 comp rx=0 tx=0).stop 2023-06-09T11:36:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 msgr2=0x7f14d814b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:32.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f14c8000c00 tx=0x7f14c8004750 comp rx=0 tx=0).stop 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 shutdown_connections 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f14b806ace0 0x7f14b806d190 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14d815b4a0 0x7f14d80cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14d80c9f40 0x7f14d814b7d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 --2- 172.21.15.7:0/4267723841 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14d8153880 0x7f14d80ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 >> 172.21.15.7:0/4267723841 conn(0x7f14d80737f0 msgr2=0x7f14d814dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 shutdown_connections 2023-06-09T11:36:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:32.299+0000 7f14dc9ff700 1 -- 172.21.15.7:0/4267723841 wait complete. 2023-06-09T11:36:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 118 2023-06-09T11:36:32.322 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:34.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.529+0000 7fcda23ff700 1 Processor -- start 2023-06-09T11:36:34.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.529+0000 7fcda23ff700 1 -- start start 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcd9c073d80 0x7fcd9c06b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c074150 0x7fcd9c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c073a10 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c06a4d0 con 0x7fcd9c073d80 2023-06-09T11:36:34.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda23ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0cc280 con 0x7fcd9c074150 2023-06-09T11:36:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41596/0 (socket says 172.21.15.7:41596) 2023-06-09T11:36:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 -- 172.21.15.7:0/3365844852 learned_addr learned my addr 172.21.15.7:0/3365844852 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcda099c700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c074150 0x7fcd9c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9bfff700 1 --2- 172.21.15.7:0/3365844852 >> v2:172.21.15.117:3300/0 conn(0x7fcd9c073d80 0x7fcd9c06b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 -- 172.21.15.7:0/3365844852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c074150 msgr2=0x7fcd9c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c074150 0x7fcd9c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 -- 172.21.15.7:0/3365844852 >> v2:172.21.15.117:3300/0 conn(0x7fcd9c073d80 msgr2=0x7fcd9c06b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/3365844852 >> v2:172.21.15.117:3300/0 conn(0x7fcd9c073d80 0x7fcd9c06b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.531+0000 7fcd9b7fe700 1 -- 172.21.15.7:0/3365844852 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0c8700 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.532+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c070e90 tx=0x7fcd90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f8139cb5bf6c02 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.532+0000 7fcd9affd700 1 -- 172.21.15.7:0/3365844852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011830 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.532+0000 7fcd9affd700 1 -- 172.21.15.7:0/3365844852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd90011990 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.532+0000 7fcd9affd700 1 -- 172.21.15.7:0/3365844852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd90011d20 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 -- 172.21.15.7:0/3365844852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c0bf5d0 msgr2=0x7fcd9c1534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c070e90 tx=0x7fcd90009bd0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 -- 172.21.15.7:0/3365844852 shutdown_connections 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 --2- 172.21.15.7:0/3365844852 >> v2:172.21.15.117:3300/0 conn(0x7fcd9c073d80 0x7fcd9c06b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c074150 0x7fcd9c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 --2- 172.21.15.7:0/3365844852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c0bf5d0 0x7fcd9c1534b0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 -- 172.21.15.7:0/3365844852 >> 172.21.15.7:0/3365844852 conn(0x7fcd9c06d150 msgr2=0x7fcd9c06d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 -- 172.21.15.7:0/3365844852 shutdown_connections 2023-06-09T11:36:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.533+0000 7fcda23ff700 1 -- 172.21.15.7:0/3365844852 wait complete. 2023-06-09T11:36:34.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.534+0000 7fcda23ff700 1 Processor -- start 2023-06-09T11:36:34.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 -- start start 2023-06-09T11:36:34.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd9c074150 0x7fcd9c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 0x7fcd9c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1ef8d0 con 0x7fcd9c073d80 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0caa50 con 0x7fcd9c074150 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.535+0000 7fcda23ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c1ef330 con 0x7fcd9c0bf5d0 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41604/0 (socket says 172.21.15.7:41604) 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 -- 172.21.15.7:0/1164570470 learned_addr learned my addr 172.21.15.7:0/1164570470 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd9c074150 0x7fcd9c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcda099c700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 0x7fcd9c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 -- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 msgr2=0x7fcd9c15ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 0x7fcd9c15ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 -- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd9c074150 msgr2=0x7fcd9c15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd9c074150 0x7fcd9c15a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 -- 172.21.15.7:0/1164570470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd90009550 con 0x7fcd9c073d80 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcda099c700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 0x7fcd9c15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.536+0000 7fcd9bfff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00e930 tx=0x7fcd8c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.537+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00ee80 con 0x7fcd9c073d80 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.537+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00c850 con 0x7fcd9c073d80 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.537+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c00cbe0 con 0x7fcd9c073d80 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.537+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c0cc7b0 con 0x7fcd9c073d80 2023-06-09T11:36:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.537+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c0cc4d0 con 0x7fcd9c073d80 2023-06-09T11:36:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.538+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c0cca90 con 0x7fcd9c073d80 2023-06-09T11:36:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.539+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd8c00cd40 con 0x7fcd9c073d80 2023-06-09T11:36:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.539+0000 7fcd98ff9700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 0x7fcd8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.540+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fcd8c089a70 con 0x7fcd9c073d80 2023-06-09T11:36:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.540+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 0x7fcd8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.540+0000 7fcd9b7fe700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 0x7fcd8806d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009840 tx=0x7fcd90008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:34.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.544+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcd8c054500 con 0x7fcd9c073d80 2023-06-09T11:36:34.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.798+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd9c0c6980 con 0x7fcd9c073d80 2023-06-09T11:36:34.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.799+0000 7fcd98ff9700 1 -- 172.21.15.7:0/1164570470 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fcd8c017090 con 0x7fcd9c073d80 2023-06-09T11:36:34.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:34.804 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:31.623739+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 msgr2=0x7fcd8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 0x7fcd8806d190 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009840 tx=0x7fcd90008030 comp rx=0 tx=0).stop 2023-06-09T11:36:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 msgr2=0x7fcd9c154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00e930 tx=0x7fcd8c00ec40 comp rx=0 tx=0).stop 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 shutdown_connections 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcd8806ace0 0x7fcd8806d190 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcd9c0bf5d0 0x7fcd9c15ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd9c073d80 0x7fcd9c154f60 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 --2- 172.21.15.7:0/1164570470 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcd9c074150 0x7fcd9c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 >> 172.21.15.7:0/1164570470 conn(0x7fcd9c06d150 msgr2=0x7fcd9c0c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.801+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 shutdown_connections 2023-06-09T11:36:34.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:34.802+0000 7fcda23ff700 1 -- 172.21.15.7:0/1164570470 wait complete. 2023-06-09T11:36:34.808 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 118 2023-06-09T11:36:34.825 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 118, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 118, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5741": { "addr": "172.21.15.117:6833/557223025", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 557223025, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 557223025, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5741, "incarnation": 107, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:active", "state_seq": 26 }, "gid_5852": { "addr": "172.21.15.117:6835/1343666954", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1343666954, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1343666954, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5852, "incarnation": 116, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:reconnect", "state_seq": 20 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 } }, "last_failure": 0, "last_failure_osd_epoch": 63, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:36:31.623739+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5741, "mds_1": 5852, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:36:36.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.031+0000 7f08d31ff700 1 Processor -- start 2023-06-09T11:36:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.031+0000 7f08d31ff700 1 -- start start 2023-06-09T11:36:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d31ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08cc15b4a0 0x7f08cc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d31ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f08cc0c9f40 0x7f08cc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d31ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08cc15a500 con 0x7f08cc153880 2023-06-09T11:36:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d31ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc0c5f10 con 0x7f08cc15b4a0 2023-06-09T11:36:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d31ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08cc15da00 con 0x7f08cc0c9f40 2023-06-09T11:36:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08cc15b4a0 0x7f08cc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f08cc15b4a0 0x7f08cc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41608/0 (socket says 172.21.15.7:41608) 2023-06-09T11:36:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 -- 172.21.15.7:0/2549981362 learned_addr learned my addr 172.21.15.7:0/2549981362 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08d179c700 1 --2- 172.21.15.7:0/2549981362 >> v2:172.21.15.117:3300/0 conn(0x7f08cc0c9f40 0x7f08cc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 -- 172.21.15.7:0/2549981362 >> v2:172.21.15.117:3300/0 conn(0x7f08cc0c9f40 msgr2=0x7f08cc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 --2- 172.21.15.7:0/2549981362 >> v2:172.21.15.117:3300/0 conn(0x7f08cc0c9f40 0x7f08cc0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 -- 172.21.15.7:0/2549981362 >> v1:172.21.15.117:6789/0 conn(0x7f08cc153880 legacy=0x7f08cc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.033+0000 7f08cbfff700 1 -- 172.21.15.7:0/2549981362 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc1596e0 con 0x7f08cc15b4a0 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08cbfff700 1 --2- 172.21.15.7:0/2549981362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc15b4a0 0x7f08cc15d4c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0749f0 tx=0x7f08b8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8b2b658b701e78 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08cb7fe700 1 -- 172.21.15.7:0/2549981362 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800b870 con 0x7f08cc15b4a0 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08cb7fe700 1 -- 172.21.15.7:0/2549981362 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b800b9d0 con 0x7f08cc15b4a0 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08cb7fe700 1 -- 172.21.15.7:0/2549981362 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800bd60 con 0x7f08cc15b4a0 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08d31ff700 1 -- 172.21.15.7:0/2549981362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc15b4a0 msgr2=0x7f08cc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.034+0000 7f08d31ff700 1 --2- 172.21.15.7:0/2549981362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc15b4a0 0x7f08cc15d4c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0749f0 tx=0x7f08b8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 -- 172.21.15.7:0/2549981362 shutdown_connections 2023-06-09T11:36:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 --2- 172.21.15.7:0/2549981362 >> v2:172.21.15.117:3300/0 conn(0x7f08cc0c9f40 0x7f08cc0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 --2- 172.21.15.7:0/2549981362 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc15b4a0 0x7f08cc15d4c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 -- 172.21.15.7:0/2549981362 >> 172.21.15.7:0/2549981362 conn(0x7f08cc0737f0 msgr2=0x7f08cc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 -- 172.21.15.7:0/2549981362 shutdown_connections 2023-06-09T11:36:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.035+0000 7f08d31ff700 1 -- 172.21.15.7:0/2549981362 wait complete. 2023-06-09T11:36:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.037+0000 7f08d31ff700 1 Processor -- start 2023-06-09T11:36:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.037+0000 7f08d31ff700 1 -- start start 2023-06-09T11:36:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.038+0000 7f08d31ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08cc0c9f40 0x7f08cc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.038+0000 7f08d31ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d31ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08cc15b4a0 0x7f08cc14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d31ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc1594b0 con 0x7f08cc153880 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d31ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc0cea30 con 0x7f08cc15b4a0 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d31ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08cc14b630 con 0x7f08cc0c9f40 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41614/0 (socket says 172.21.15.7:41614) 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 -- 172.21.15.7:0/3196866999 learned_addr learned my addr 172.21.15.7:0/3196866999 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d0f9b700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08cc0c9f40 0x7f08cc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08d179c700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08cc15b4a0 0x7f08cc14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 -- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08cc0c9f40 msgr2=0x7f08cc0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08cc0c9f40 0x7f08cc0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 -- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08cc15b4a0 msgr2=0x7f08cc14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08cc15b4a0 0x7f08cc14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 -- 172.21.15.7:0/3196866999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009550 con 0x7f08cc153880 2023-06-09T11:36:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.039+0000 7f08cbfff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8014040 tx=0x7f08b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.040+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004520 con 0x7f08cc153880 2023-06-09T11:36:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.040+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b801e070 con 0x7f08cc153880 2023-06-09T11:36:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.040+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80038a0 con 0x7f08cc153880 2023-06-09T11:36:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.040+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08cc159c60 con 0x7f08cc153880 2023-06-09T11:36:37.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.040+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc159980 con 0x7f08cc153880 2023-06-09T11:36:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.041+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08cc159f40 con 0x7f08cc153880 2023-06-09T11:36:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.042+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f08b8003a00 con 0x7f08cc153880 2023-06-09T11:36:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.044+0000 7f08c97fa700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 0x7f08b006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.044+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f08b8020070 con 0x7f08cc153880 2023-06-09T11:36:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.044+0000 7f08d0f9b700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 0x7f08b006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.045+0000 7f08d0f9b700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 0x7f08b006d1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000f60 tx=0x7f08c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:37.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.048+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f08b80553b0 con 0x7f08cc153880 2023-06-09T11:36:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.302+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08cc0cf030 con 0x7f08cc153880 2023-06-09T11:36:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.303+0000 7f08c97fa700 1 -- 172.21.15.7:0/3196866999 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f08cc0cf030 con 0x7f08cc153880 2023-06-09T11:36:37.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:37.308 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:31.623739+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 msgr2=0x7f08b006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 0x7f08b006d1c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f08c0000f60 tx=0x7f08c0009040 comp rx=0 tx=0).stop 2023-06-09T11:36:37.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 msgr2=0x7f08cc14a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8014040 tx=0x7f08b8004750 comp rx=0 tx=0).stop 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 shutdown_connections 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f08cc15b4a0 0x7f08cc14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f08b006ad10 0x7f08b006d1c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f08cc0c9f40 0x7f08cc0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 --2- 172.21.15.7:0/3196866999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08cc153880 0x7f08cc14a7d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 >> 172.21.15.7:0/3196866999 conn(0x7f08cc0737f0 msgr2=0x7f08cc14c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:37.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 shutdown_connections 2023-06-09T11:36:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:37.305+0000 7f08d31ff700 1 -- 172.21.15.7:0/3196866999 wait complete. 2023-06-09T11:36:37.312 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 118 2023-06-09T11:36:37.328 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:39.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.537+0000 7fcc025ff700 1 Processor -- start 2023-06-09T11:36:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.537+0000 7fcc025ff700 1 -- start start 2023-06-09T11:36:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.538+0000 7fcc025ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc0c6280 0x7fcbfc0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.538+0000 7fcc025ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbfc06a4d0 0x7fcbfc0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.538+0000 7fcc025ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc15cac0 con 0x7fcbfc06e460 2023-06-09T11:36:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.538+0000 7fcc025ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc0c8870 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.538+0000 7fcc025ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0c37d0 con 0x7fcbfc0c6280 2023-06-09T11:36:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcbfc06a4d0 0x7fcbfc0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcc00b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcbfc06e460 0x7fcbfc06cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38486/0 (socket says 172.21.15.7:38486) 2023-06-09T11:36:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcc00b9c700 1 -- 172.21.15.7:0/4037994036 learned_addr learned my addr 172.21.15.7:0/4037994036 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfbfff700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc0c6280 0x7fcbfc0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 -- 172.21.15.7:0/4037994036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc0c6280 msgr2=0x7fcbfc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc0c6280 0x7fcbfc0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 -- 172.21.15.7:0/4037994036 >> v1:172.21.15.7:6789/0 conn(0x7fcbfc06e460 legacy=0x7fcbfc06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 -- 172.21.15.7:0/4037994036 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc15cd30 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfaffd700 1 -- 172.21.15.7:0/4037994036 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262925797 0 0) 0x7fcbfc15cac0 con 0x7fcbfc06e460 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.539+0000 7fcbfb7fe700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc0737f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009840 tx=0x7fcbf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c61bcf99828eb5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcbfaffd700 1 -- 172.21.15.7:0/4037994036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0011830 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcbfaffd700 1 -- 172.21.15.7:0/4037994036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf0011990 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcbfaffd700 1 -- 172.21.15.7:0/4037994036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0011d20 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 -- 172.21.15.7:0/4037994036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc06a4d0 msgr2=0x7fcbfc0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc0737f0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009840 tx=0x7fcbf0009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 -- 172.21.15.7:0/4037994036 shutdown_connections 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc0c6280 0x7fcbfc0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 --2- 172.21.15.7:0/4037994036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc0737f0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 -- 172.21.15.7:0/4037994036 >> 172.21.15.7:0/4037994036 conn(0x7fcbfc06f980 msgr2=0x7fcbfc0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 -- 172.21.15.7:0/4037994036 shutdown_connections 2023-06-09T11:36:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.540+0000 7fcc025ff700 1 -- 172.21.15.7:0/4037994036 wait complete. 2023-06-09T11:36:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.542+0000 7fcc025ff700 1 Processor -- start 2023-06-09T11:36:39.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.542+0000 7fcc025ff700 1 -- start start 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc1548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 0x7fcbfc1f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc1e5a20 con 0x7fcbfc0c6280 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc1f2f30 con 0x7fcbfc06a4d0 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc025ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc1e2f60 con 0x7fcbfc06e460 2023-06-09T11:36:39.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc00b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc00b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41630/0 (socket says 172.21.15.7:41630) 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.543+0000 7fcc00b9c700 1 -- 172.21.15.7:0/2704375056 learned_addr learned my addr 172.21.15.7:0/2704375056 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbfbfff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc1548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbfb7fe700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 0x7fcbfc1f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 -- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 msgr2=0x7fcbfc1f29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 0x7fcbfc1f29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 -- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 msgr2=0x7fcbfc1548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc1548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 -- 172.21.15.7:0/2704375056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00094a0 con 0x7fcbfc0c6280 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbfbfff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbfb7fe700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 0x7fcbfc1f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc00b9c700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4002230 tx=0x7fcbf400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400ac20 con 0x7fcbfc0c6280 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400ada0 con 0x7fcbfc0c6280 2023-06-09T11:36:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc15d2f0 con 0x7fcbfc0c6280 2023-06-09T11:36:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4007310 con 0x7fcbfc0c6280 2023-06-09T11:36:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.544+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc15d010 con 0x7fcbfc0c6280 2023-06-09T11:36:39.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.545+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc15d5d0 con 0x7fcbfc0c6280 2023-06-09T11:36:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.546+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcbf4007470 con 0x7fcbfc0c6280 2023-06-09T11:36:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.547+0000 7fcbf8ff9700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 0x7fcbe806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.547+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fcbf4089960 con 0x7fcbfc0c6280 2023-06-09T11:36:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.547+0000 7fcbfbfff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 0x7fcbe806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:39.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.548+0000 7fcbfbfff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 0x7fcbe806d1c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009820 tx=0x7fcbec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:39.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.552+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcbf4054480 con 0x7fcbfc0c6280 2023-06-09T11:36:39.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.807+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbfc074d50 con 0x7fcbfc0c6280 2023-06-09T11:36:39.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.808+0000 7fcbf8ff9700 1 -- 172.21.15.7:0/2704375056 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fcbf4018090 con 0x7fcbfc0c6280 2023-06-09T11:36:39.813 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:39.813 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:37.618764+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:39.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 msgr2=0x7fcbe806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 0x7fcbe806d1c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009820 tx=0x7fcbec008040 comp rx=0 tx=0).stop 2023-06-09T11:36:39.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 msgr2=0x7fcbfc06aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:39.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4002230 tx=0x7fcbf400a060 comp rx=0 tx=0).stop 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 shutdown_connections 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcbfc0c6280 0x7fcbfc06aca0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcbfc06a4d0 0x7fcbfc1548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbe806ad10 0x7fcbe806d1c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 --2- 172.21.15.7:0/2704375056 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcbfc06e460 0x7fcbfc1f29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 >> 172.21.15.7:0/2704375056 conn(0x7fcbfc06f980 msgr2=0x7fcbfc15b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:39.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 shutdown_connections 2023-06-09T11:36:39.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:39.810+0000 7fcc025ff700 1 -- 172.21.15.7:0/2704375056 wait complete. 2023-06-09T11:36:39.817 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 119 2023-06-09T11:36:39.833 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:41.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.040+0000 7f9dc77ff700 1 Processor -- start 2023-06-09T11:36:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.040+0000 7f9dc77ff700 1 -- start start 2023-06-09T11:36:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc77ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9dc01539b0 0x7f9dc0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dc00c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc77ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc015a420 con 0x7f9dc01535e0 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc77ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc00ca670 con 0x7f9dc01539b0 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5a00 con 0x7f9dc00c4ce0 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc559b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9dc01535e0 0x7f9dc00c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38498/0 (socket says 172.21.15.7:38498) 2023-06-09T11:36:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc559b700 1 -- 172.21.15.7:0/4129748640 learned_addr learned my addr 172.21.15.7:0/4129748640 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc4d9a700 1 --2- 172.21.15.7:0/4129748640 >> v2:172.21.15.117:3300/0 conn(0x7f9dc01539b0 0x7f9dc0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dc00c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804633840 0 0) 0x7f9dbc001110 con 0x7f9dc01535e0 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.042+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dac0035b0 con 0x7f9dc01535e0 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2883862654 0 0) 0x7f9dac0035b0 con 0x7f9dc01535e0 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 msgr2=0x7f9dc00c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dc00c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 >> v2:172.21.15.117:3300/0 conn(0x7f9dc01539b0 msgr2=0x7f9dc0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 --2- 172.21.15.7:0/4129748640 >> v2:172.21.15.117:3300/0 conn(0x7f9dc01539b0 0x7f9dc0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc006b460 con 0x7f9dc01535e0 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f9dbc001800 con 0x7f9dc01535e0 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 >> v1:172.21.15.7:6789/0 conn(0x7f9dc01535e0 legacy=0x7f9dc00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dac003a00 0x7f9dac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dac004d90 0x7f9dac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c5a00 con 0x7f9dac004d90 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac00ba10 con 0x7f9dac003a00 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac00bb50 con 0x7f9dc00c4ce0 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9dc559b700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dac003a00 0x7f9dac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dac004d90 0x7f9dac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.043+0000 7f9dc4d9a700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 -- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 msgr2=0x7f9dac004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dc00c4ce0 0x7f9dac004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 -- 172.21.15.7:0/4129748640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dac004d90 msgr2=0x7f9dac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dac004d90 0x7f9dac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 -- 172.21.15.7:0/4129748640 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac0035b0 con 0x7f9dac003a00 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dac004d90 0x7f9dac005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9dc559b700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dac003a00 0x7f9dac003e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c190 tx=0x7f9dac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e265af4bfa1c4a55 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc007d00 con 0x7f9dac003a00 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc00cc70 con 0x7f9dac003a00 2023-06-09T11:36:42.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.044+0000 7f9db7fff700 1 -- 172.21.15.7:0/4129748640 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc005960 con 0x7f9dac003a00 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 -- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dac003a00 msgr2=0x7f9dac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/4129748640 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dac003a00 0x7f9dac003e10 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00c190 tx=0x7f9dac005880 comp rx=0 tx=0).stop 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc5d9c700 1 -- 172.21.15.7:0/4129748640 reap_dead start 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 -- 172.21.15.7:0/4129748640 shutdown_connections 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 -- 172.21.15.7:0/4129748640 >> 172.21.15.7:0/4129748640 conn(0x7f9dc00737f0 msgr2=0x7f9dc0072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 -- 172.21.15.7:0/4129748640 shutdown_connections 2023-06-09T11:36:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.045+0000 7f9dc77ff700 1 -- 172.21.15.7:0/4129748640 wait complete. 2023-06-09T11:36:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.047+0000 7f9dc77ff700 1 Processor -- start 2023-06-09T11:36:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.047+0000 7f9dc77ff700 1 -- start start 2023-06-09T11:36:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 0x7f9dc0155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 0x7f9dc01581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 0x7f9dc0158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c95e0 con 0x7f9dc014fc40 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00c9930 con 0x7f9dc014f7d0 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b2120 con 0x7f9dac003a00 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 0x7f9dc0158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 0x7f9dc0155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc4d9a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 0x7f9dc01581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 0x7f9dc0155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:45336/0 (socket says 172.21.15.7:45336) 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc559b700 1 -- 172.21.15.7:0/1432066032 learned_addr learned my addr 172.21.15.7:0/1432066032 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 -- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 msgr2=0x7f9dc0155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 0x7f9dc0155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 -- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 msgr2=0x7f9dc01581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 0x7f9dc01581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc5d9c700 1 -- 172.21.15.7:0/1432066032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc006a850 con 0x7f9dc014fc40 2023-06-09T11:36:42.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.048+0000 7f9dc4d9a700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 0x7f9dc01581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9dc5d9c700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 0x7f9dc0158710 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0072a90 tx=0x7f9db800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db801c070 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800f040 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc006a570 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800e040 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.049+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc006ab30 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.050+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0158c60 con 0x7f9dc014fc40 2023-06-09T11:36:42.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.051+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9db8015430 con 0x7f9dc014fc40 2023-06-09T11:36:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.052+0000 7f9db5ffb700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 0x7f9da406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.052+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f9db8089690 con 0x7f9dc014fc40 2023-06-09T11:36:42.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.052+0000 7f9dc559b700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 0x7f9da406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.053+0000 7f9dc559b700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 0x7f9da406d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0012b0 tx=0x7f9dbc00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:42.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.056+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9dc0158c60 con 0x7f9dc014fc40 2023-06-09T11:36:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.310+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc0158c60 con 0x7f9dc014fc40 2023-06-09T11:36:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.311+0000 7f9db5ffb700 1 -- 172.21.15.7:0/1432066032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f9dc0158c60 con 0x7f9dc014fc40 2023-06-09T11:36:42.316 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:42.317 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":119,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:37.618764+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:rejoin","state_seq":22,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.313+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 msgr2=0x7f9da406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.313+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 0x7f9da406d190 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0012b0 tx=0x7f9dbc00a000 comp rx=0 tx=0).stop 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.313+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 msgr2=0x7f9dc0158710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.313+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 0x7f9dc0158710 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0072a90 tx=0x7f9db800f490 comp rx=0 tx=0).stop 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.313+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 shutdown_connections 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9dc014fc40 0x7f9dc0158710 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9da406ace0 0x7f9da406d190 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9dac003a00 0x7f9dc0155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 --2- 172.21.15.7:0/1432066032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9dc014f7d0 0x7f9dc01581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 >> 172.21.15.7:0/1432066032 conn(0x7f9dc00737f0 msgr2=0x7f9dc015c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 shutdown_connections 2023-06-09T11:36:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:42.314+0000 7f9dc77ff700 1 -- 172.21.15.7:0/1432066032 wait complete. 2023-06-09T11:36:42.320 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 119 2023-06-09T11:36:42.337 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:36:44.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:44.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.543+0000 7fcb501ff700 1 Processor -- start 2023-06-09T11:36:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.543+0000 7fcb501ff700 1 -- start start 2023-06-09T11:36:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c1bc0 0x7fcb481599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb480c0140 0x7fcb48159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 0x7fcb481501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb48070110 con 0x7fcb480c0140 2023-06-09T11:36:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb48071300 con 0x7fcb480c1f90 2023-06-09T11:36:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb501ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480cbd20 con 0x7fcb480c1bc0 2023-06-09T11:36:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.544+0000 7fcb4d79a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb480c0140 0x7fcb48159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcb480c0140 0x7fcb48159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59184/0 (socket says 172.21.15.7:59184) 2023-06-09T11:36:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 -- 172.21.15.7:0/2054444443 learned_addr learned my addr 172.21.15.7:0/2054444443 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:44.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4e79c700 1 --2- 172.21.15.7:0/2054444443 >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 0x7fcb481501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4df9b700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c1bc0 0x7fcb481599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 -- 172.21.15.7:0/2054444443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c1bc0 msgr2=0x7fcb481599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c1bc0 0x7fcb481599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 -- 172.21.15.7:0/2054444443 >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 msgr2=0x7fcb481501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 --2- 172.21.15.7:0/2054444443 >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 0x7fcb481501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 -- 172.21.15.7:0/2054444443 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480c0bf0 con 0x7fcb480c0140 2023-06-09T11:36:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4e79c700 1 --2- 172.21.15.7:0/2054444443 >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 0x7fcb481501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4d79a700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c0140 0x7fcb48159ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cc250 tx=0x7fcb38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe9b1978f70effc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4cf99700 1 -- 172.21.15.7:0/2054444443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800b860 con 0x7fcb480c0140 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4cf99700 1 -- 172.21.15.7:0/2054444443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3800b9c0 con 0x7fcb480c0140 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.545+0000 7fcb4cf99700 1 -- 172.21.15.7:0/2054444443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800bd50 con 0x7fcb480c0140 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 -- 172.21.15.7:0/2054444443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c0140 msgr2=0x7fcb48159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c0140 0x7fcb48159ef0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcb480cc250 tx=0x7fcb38009c00 comp rx=0 tx=0).stop 2023-06-09T11:36:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 -- 172.21.15.7:0/2054444443 shutdown_connections 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 --2- 172.21.15.7:0/2054444443 >> v2:172.21.15.117:3300/0 conn(0x7fcb480c1f90 0x7fcb481501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c1bc0 0x7fcb481599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 --2- 172.21.15.7:0/2054444443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c0140 0x7fcb48159ef0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 -- 172.21.15.7:0/2054444443 >> 172.21.15.7:0/2054444443 conn(0x7fcb4806a4d0 msgr2=0x7fcb4806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 -- 172.21.15.7:0/2054444443 shutdown_connections 2023-06-09T11:36:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.546+0000 7fcb501ff700 1 -- 172.21.15.7:0/2054444443 wait complete. 2023-06-09T11:36:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.548+0000 7fcb501ff700 1 Processor -- start 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.548+0000 7fcb501ff700 1 -- start start 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c0140 0x7fcb48149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb480c1bc0 0x7fcb4814a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb481596a0 con 0x7fcb480c1f90 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb48158830 con 0x7fcb480c1bc0 2023-06-09T11:36:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb501ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480adf10 con 0x7fcb480c0140 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb4e79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb4e79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59186/0 (socket says 172.21.15.7:59186) 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb4e79c700 1 -- 172.21.15.7:0/4237311728 learned_addr learned my addr 172.21.15.7:0/4237311728 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb4df9b700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c0140 0x7fcb48149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.549+0000 7fcb4d79a700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb480c1bc0 0x7fcb4814a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 -- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c0140 msgr2=0x7fcb48149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c0140 0x7fcb48149e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 -- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb480c1bc0 msgr2=0x7fcb4814a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb480c1bc0 0x7fcb4814a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 -- 172.21.15.7:0/4237311728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb38009550 con 0x7fcb480c1f90 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb4e79c700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcb40007df0 tx=0x7fcb4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4000dd20 con 0x7fcb480c1f90 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4000f040 con 0x7fcb480c1f90 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb40003c60 con 0x7fcb480c1f90 2023-06-09T11:36:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb480c0e90 con 0x7fcb480c1f90 2023-06-09T11:36:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.550+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb480702f0 con 0x7fcb480c1f90 2023-06-09T11:36:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.552+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb480c1170 con 0x7fcb480c1f90 2023-06-09T11:36:44.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.552+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb40004030 con 0x7fcb480c1f90 2023-06-09T11:36:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.553+0000 7fcb3e7fc700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 0x7fcb2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.553+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fcb40089900 con 0x7fcb480c1f90 2023-06-09T11:36:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.553+0000 7fcb4df9b700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 0x7fcb2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.554+0000 7fcb4df9b700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 0x7fcb2c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009950 tx=0x7fcb44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.558+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcb40054580 con 0x7fcb480c1f90 2023-06-09T11:36:44.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.810+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb481597e0 con 0x7fcb480c1f90 2023-06-09T11:36:44.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.810+0000 7fcb3e7fc700 1 -- 172.21.15.7:0/4237311728 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fcb481597e0 con 0x7fcb480c1f90 2023-06-09T11:36:44.816 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:44.816 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:43.648346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 msgr2=0x7fcb2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 0x7fcb2c06d190 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcb44009950 tx=0x7fcb44008040 comp rx=0 tx=0).stop 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 msgr2=0x7fcb48148a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fcb40007df0 tx=0x7fcb4000d3b0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 shutdown_connections 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb480c1f90 0x7fcb48148a80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb2c06ace0 0x7fcb2c06d190 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb480c0140 0x7fcb48149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 --2- 172.21.15.7:0/4237311728 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb480c1bc0 0x7fcb4814a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 >> 172.21.15.7:0/4237311728 conn(0x7fcb4806a4d0 msgr2=0x7fcb4814d050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 shutdown_connections 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:44.813+0000 7fcb501ff700 1 -- 172.21.15.7:0/4237311728 wait complete. 2023-06-09T11:36:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 120 2023-06-09T11:36:44.836 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=1, replacing gid=5692 2023-06-09T11:36:44.836 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 4 secs before reviving mds.f 2023-06-09T11:36:47.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:47.743 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:36:48.838 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:36:48.838 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:36:48.838 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:36:48.843 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:36:48.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:48.985 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:36:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.044+0000 7fa0967ff700 1 Processor -- start 2023-06-09T11:36:49.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.044+0000 7fa0967ff700 1 -- start start 2023-06-09T11:36:49.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.045+0000 7fa0967ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0901599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.045+0000 7fa0967ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0900c0140 0x7fa090159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa0967ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0900c0bf0 con 0x7fa0900c1f90 2023-06-09T11:36:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa0967ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa090070110 con 0x7fa0900c0140 2023-06-09T11:36:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa0967ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa090071300 con 0x7fa0900c1bc0 2023-06-09T11:36:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa094d9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0900c1f90 0x7fa0901501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59946/0 (socket says 172.21.15.7:59946) 2023-06-09T11:36:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa0900c0140 0x7fa090159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa094d9c700 1 -- 172.21.15.7:0/2925148590 learned_addr learned my addr 172.21.15.7:0/2925148590 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08ffff700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0901599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08effd700 1 -- 172.21.15.7:0/2925148590 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354171049 0 0) 0x7fa0900c0bf0 con 0x7fa0900c1f90 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08effd700 1 -- 172.21.15.7:0/2925148590 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0780057c0 con 0x7fa0900c1f90 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08f7fe700 1 -- 172.21.15.7:0/2925148590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 msgr2=0x7fa0901599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0901599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08f7fe700 1 -- 172.21.15.7:0/2925148590 >> v1:172.21.15.7:6789/0 conn(0x7fa0900c1f90 legacy=0x7fa0901501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.046+0000 7fa08f7fe700 1 -- 172.21.15.7:0/2925148590 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0900702f0 con 0x7fa0900c0140 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa090159ef0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa084009840 tx=0x7fa084009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d519a5af39361e4f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa08effd700 1 -- 172.21.15.7:0/2925148590 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084011830 con 0x7fa0900c0140 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa08effd700 1 -- 172.21.15.7:0/2925148590 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084011990 con 0x7fa0900c0140 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa08effd700 1 -- 172.21.15.7:0/2925148590 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084011d20 con 0x7fa0900c0140 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa0967ff700 1 -- 172.21.15.7:0/2925148590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 msgr2=0x7fa090159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.047+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa090159ef0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa084009840 tx=0x7fa084009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:49.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 -- 172.21.15.7:0/2925148590 shutdown_connections 2023-06-09T11:36:49.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0901599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2925148590 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa090159ef0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 -- 172.21.15.7:0/2925148590 >> 172.21.15.7:0/2925148590 conn(0x7fa09006a4d0 msgr2=0x7fa09006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:49.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 -- 172.21.15.7:0/2925148590 shutdown_connections 2023-06-09T11:36:49.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.048+0000 7fa0967ff700 1 -- 172.21.15.7:0/2925148590 wait complete. 2023-06-09T11:36:49.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.049+0000 7fa0967ff700 1 Processor -- start 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.049+0000 7fa0967ff700 1 -- start start 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0900ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 0x7fa0900caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900ca0a0 con 0x7fa0900c0140 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900c9140 con 0x7fa0900c1f90 2023-06-09T11:36:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.050+0000 7fa0967ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900c9420 con 0x7fa0900c1bc0 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59202/0 (socket says 172.21.15.7:59202) 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 -- 172.21.15.7:0/2840892134 learned_addr learned my addr 172.21.15.7:0/2840892134 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0900ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa094d9c700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 0x7fa0900caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 -- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 msgr2=0x7fa0900ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0900ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 -- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 msgr2=0x7fa0900caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 0x7fa0900caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 -- 172.21.15.7:0/2840892134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840094a0 con 0x7fa0900c0140 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0900ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa094d9c700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 0x7fa0900caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08ffff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa08000e9d0 tx=0x7fa08000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.051+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa080009730 con 0x7fa0900c0140 2023-06-09T11:36:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.052+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08000c850 con 0x7fa0900c0140 2023-06-09T11:36:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.052+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08000cbe0 con 0x7fa0900c0140 2023-06-09T11:36:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.052+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0900c1170 con 0x7fa0900c0140 2023-06-09T11:36:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.052+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0900c0e90 con 0x7fa0900c0140 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.053+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0900c1450 con 0x7fa0900c0140 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.054+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa08000cd40 con 0x7fa0900c0140 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.055+0000 7fa08cff9700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 0x7fa07c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.055+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fa080089e10 con 0x7fa0900c0140 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.055+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 0x7fa07c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.055+0000 7fa08f7fe700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 0x7fa07c06d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa09014e090 tx=0x7fa084008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.059+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa0800584f0 con 0x7fa0900c0140 2023-06-09T11:36:49.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.314+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa090074cf0 con 0x7fa0900c0140 2023-06-09T11:36:49.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.314+0000 7fa08cff9700 1 -- 172.21.15.7:0/2840892134 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fa080058080 con 0x7fa0900c0140 2023-06-09T11:36:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:49.321 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":120,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:43.648346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 msgr2=0x7fa07c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 0x7fa07c06d190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa09014e090 tx=0x7fa084008030 comp rx=0 tx=0).stop 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 msgr2=0x7fa0900c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa08000e9d0 tx=0x7fa08000ed90 comp rx=0 tx=0).stop 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 shutdown_connections 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa0900c1f90 0x7fa0900caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0900c0140 0x7fa0900c4e40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa07c06ace0 0x7fa07c06d190 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 --2- 172.21.15.7:0/2840892134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa0900c1bc0 0x7fa0900ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 >> 172.21.15.7:0/2840892134 conn(0x7fa09006a4d0 msgr2=0x7fa09014f440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 shutdown_connections 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:49.317+0000 7fa0967ff700 1 -- 172.21.15.7:0/2840892134 wait complete. 2023-06-09T11:36:49.324 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 120 2023-06-09T11:36:49.341 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-06-09T11:36:51.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:36:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.547+0000 7f35a63ff700 1 Processor -- start 2023-06-09T11:36:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.547+0000 7f35a63ff700 1 -- start start 2023-06-09T11:36:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.548+0000 7f35a63ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35a0153880 0x7f35a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.548+0000 7f35a63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f35a63ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a015a500 con 0x7f35a015b2d0 2023-06-09T11:36:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f35a63ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a00c5f10 con 0x7f35a0153880 2023-06-09T11:36:51.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f35a63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a015da00 con 0x7f35a00c9f40 2023-06-09T11:36:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f35a0153880 0x7f35a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f35a499c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f35a015b2d0 0x7f35a00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59956/0 (socket says 172.21.15.7:59956) 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f35a499c700 1 -- 172.21.15.7:0/2711279171 learned_addr learned my addr 172.21.15.7:0/2711279171 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359f7fe700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359ffff700 1 -- 172.21.15.7:0/2711279171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 msgr2=0x7f35a015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359ffff700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359ffff700 1 -- 172.21.15.7:0/2711279171 >> v1:172.21.15.7:6789/0 conn(0x7f35a015b2d0 legacy=0x7f35a00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359ffff700 1 -- 172.21.15.7:0/2711279171 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a015b750 con 0x7f35a0153880 2023-06-09T11:36:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.549+0000 7f359effd700 1 -- 172.21.15.7:0/2711279171 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661794752 0 0) 0x7f3598001490 con 0x7f35a015b2d0 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f359ffff700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a0153880 0x7f35a015d4c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3590009840 tx=0x7f3590009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8623d7479d88264e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f359effd700 1 -- 172.21.15.7:0/2711279171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590011830 con 0x7f35a0153880 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f359effd700 1 -- 172.21.15.7:0/2711279171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3590011990 con 0x7f35a0153880 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f359effd700 1 -- 172.21.15.7:0/2711279171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590011d20 con 0x7f35a0153880 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f35a63ff700 1 -- 172.21.15.7:0/2711279171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a0153880 msgr2=0x7f35a015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f35a63ff700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a0153880 0x7f35a015d4c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3590009840 tx=0x7f3590009b50 comp rx=0 tx=0).stop 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.550+0000 7f35a63ff700 1 -- 172.21.15.7:0/2711279171 shutdown_connections 2023-06-09T11:36:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.551+0000 7f35a63ff700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a0153880 0x7f35a015d4c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.551+0000 7f35a63ff700 1 --2- 172.21.15.7:0/2711279171 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.551+0000 7f35a63ff700 1 -- 172.21.15.7:0/2711279171 >> 172.21.15.7:0/2711279171 conn(0x7f35a00737f0 msgr2=0x7f35a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.551+0000 7f35a63ff700 1 -- 172.21.15.7:0/2711279171 shutdown_connections 2023-06-09T11:36:51.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.551+0000 7f35a63ff700 1 -- 172.21.15.7:0/2711279171 wait complete. 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.552+0000 7f35a63ff700 1 Processor -- start 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.552+0000 7f35a63ff700 1 -- start start 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a0158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35a0153880 0x7f35a0149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0158e50 con 0x7f35a015b2d0 2023-06-09T11:36:51.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0158050 con 0x7f35a0153880 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.553+0000 7f35a63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a014b570 con 0x7f35a00c9f40 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59228/0 (socket says 172.21.15.7:59228) 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 -- 172.21.15.7:0/1067028208 learned_addr learned my addr 172.21.15.7:0/1067028208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f359f7fe700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35a0153880 0x7f35a0149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f359ffff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a0158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 -- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 msgr2=0x7f35a0158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a0158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 -- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35a0153880 msgr2=0x7f35a0149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35a0153880 0x7f35a0149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 -- 172.21.15.7:0/1067028208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900094a0 con 0x7f35a015b2d0 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.554+0000 7f35a499c700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f359800cbd0 tx=0x7f359800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.555+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800aa60 con 0x7f35a015b2d0 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.555+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800abe0 con 0x7f35a015b2d0 2023-06-09T11:36:51.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.555+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980070d0 con 0x7f35a015b2d0 2023-06-09T11:36:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.555+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a0159980 con 0x7f35a015b2d0 2023-06-09T11:36:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.555+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a01596e0 con 0x7f35a015b2d0 2023-06-09T11:36:51.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.556+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a0159c60 con 0x7f35a015b2d0 2023-06-09T11:36:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.557+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3598007230 con 0x7f35a015b2d0 2023-06-09T11:36:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.558+0000 7f359cff9700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 0x7f358c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:36:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.558+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f3598089170 con 0x7f35a015b2d0 2023-06-09T11:36:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.558+0000 7f359ffff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 0x7f358c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:36:51.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.559+0000 7f359ffff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 0x7f358c06d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f35900090b0 tx=0x7f3590008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:36:51.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.562+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3598053d20 con 0x7f35a015b2d0 2023-06-09T11:36:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.813+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35a015a500 con 0x7f35a015b2d0 2023-06-09T11:36:51.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.814+0000 7f359cff9700 1 -- 172.21.15.7:0/1067028208 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+6222 (secure 0 0 0) 0x7f35a015a500 con 0x7f35a015b2d0 2023-06-09T11:36:51.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:36:51.820 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":121,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:36:43.648346+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:36:51.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 msgr2=0x7f358c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 0x7f358c06d190 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f35900090b0 tx=0x7f3590008030 comp rx=0 tx=0).stop 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 msgr2=0x7f35a0148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f359800cbd0 tx=0x7f359800cee0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 shutdown_connections 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35a015b2d0 0x7f35a0148bd0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f358c06ace0 0x7f358c06d190 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f35a00c9f40 0x7f35a0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 --2- 172.21.15.7:0/1067028208 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f35a0153880 0x7f35a0149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:36:51.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 >> 172.21.15.7:0/1067028208 conn(0x7f35a00737f0 msgr2=0x7f35a014d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:36:51.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 shutdown_connections 2023-06-09T11:36:51.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:36:51.816+0000 7f35a63ff700 1 -- 172.21.15.7:0/1067028208 wait complete. 2023-06-09T11:36:51.824 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 121 2023-06-09T11:36:51.840 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:36:51.840 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 69 secs before thrashing 2023-06-09T11:37:17.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:17.864 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:47.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:37:47.976 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:00.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.062+0000 7f1621dff700 1 Processor -- start 2023-06-09T11:38:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.062+0000 7f1621dff700 1 -- start start 2023-06-09T11:38:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.063+0000 7f1621dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c0c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.063+0000 7f1621dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f161c15b060 0x7f161c163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.063+0000 7f1621dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0c9bd0 con 0x7f161c0c88e0 2023-06-09T11:38:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.063+0000 7f1621dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c159360 con 0x7f161c15b060 2023-06-09T11:38:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.063+0000 7f1621dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0c8e20 con 0x7f161c0c2ae0 2023-06-09T11:38:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f161c15b060 0x7f161c163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f161c15b060 0x7f161c163ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45574/0 (socket says 172.21.15.7:45574) 2023-06-09T11:38:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 -- 172.21.15.7:0/4210356837 learned_addr learned my addr 172.21.15.7:0/4210356837 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161b7fe700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c0c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 -- 172.21.15.7:0/4210356837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 msgr2=0x7f161c0c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c0c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 -- 172.21.15.7:0/4210356837 >> v1:172.21.15.7:6789/0 conn(0x7f161c0c88e0 legacy=0x7f161c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 -- 172.21.15.7:0/4210356837 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c159570 con 0x7f161c15b060 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161b7fe700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c0c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161a7fc700 1 -- 172.21.15.7:0/4210356837 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808483871 0 0) 0x7f161c0c9bd0 con 0x7f161c0c88e0 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.064+0000 7f161bfff700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c163ce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04049fb98747b4d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f161a7fc700 1 -- 172.21.15.7:0/4210356837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400e070 con 0x7f161c15b060 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f161a7fc700 1 -- 172.21.15.7:0/4210356837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400b9c0 con 0x7f161c15b060 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f161a7fc700 1 -- 172.21.15.7:0/4210356837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400bd70 con 0x7f161c15b060 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 -- 172.21.15.7:0/4210356837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 msgr2=0x7f161c163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c163ce0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161400b330 comp rx=0 tx=0).stop 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 -- 172.21.15.7:0/4210356837 shutdown_connections 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c163ce0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 --2- 172.21.15.7:0/4210356837 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c0c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 -- 172.21.15.7:0/4210356837 >> 172.21.15.7:0/4210356837 conn(0x7f161c0cd330 msgr2=0x7f161c0cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:01.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 -- 172.21.15.7:0/4210356837 shutdown_connections 2023-06-09T11:38:01.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.065+0000 7f1621dff700 1 -- 172.21.15.7:0/4210356837 wait complete. 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.068+0000 7f1621dff700 1 Processor -- start 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.068+0000 7f1621dff700 1 -- start start 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f161c0c88e0 0x7f161c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c145270 con 0x7f161c15b060 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0c6470 con 0x7f161c0c88e0 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f1621dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c14f490 con 0x7f161c0c2ae0 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f161bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f161b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:42390/0 (socket says 172.21.15.7:42390) 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f161b7fe700 1 -- 172.21.15.7:0/3902049690 learned_addr learned my addr 172.21.15.7:0/3902049690 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.069+0000 7f161affd700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f161c0c88e0 0x7f161c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 -- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f161c0c88e0 msgr2=0x7f161c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f161c0c88e0 0x7f161c0cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 -- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 msgr2=0x7f161c143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 -- 172.21.15.7:0/3902049690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614009b30 con 0x7f161c0c2ae0 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161bfff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f161b7fe700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c750 tx=0x7f160c00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c011800 con 0x7f161c0c2ae0 2023-06-09T11:38:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c011960 con 0x7f161c0c2ae0 2023-06-09T11:38:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c011cd0 con 0x7f161c0c2ae0 2023-06-09T11:38:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.070+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c159af0 con 0x7f161c0c2ae0 2023-06-09T11:38:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.071+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c159810 con 0x7f161c0c2ae0 2023-06-09T11:38:01.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.072+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c159dd0 con 0x7f161c0c2ae0 2023-06-09T11:38:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.073+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f160c021350 con 0x7f161c0c2ae0 2023-06-09T11:38:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.074+0000 7f1603fff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 0x7f160806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.074+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f160c089830 con 0x7f161c0c2ae0 2023-06-09T11:38:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.074+0000 7f161affd700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 0x7f160806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.075+0000 7f161affd700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 0x7f160806d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f161c14a0a0 tx=0x7f1610009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.080+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f160c057df0 con 0x7f161c0c2ae0 2023-06-09T11:38:01.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.363+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f161c074cf0 con 0x7f161c0c2ae0 2023-06-09T11:38:01.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.363+0000 7f1603fff700 1 -- 172.21.15.7:0/3902049690 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f160c057980 con 0x7f161c0c2ae0 2023-06-09T11:38:01.369 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:01.369 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 msgr2=0x7f160806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 0x7f160806d190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f161c14a0a0 tx=0x7f1610009040 comp rx=0 tx=0).stop 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 msgr2=0x7f161c14dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f160c00c750 tx=0x7f160c00ca60 comp rx=0 tx=0).stop 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 shutdown_connections 2023-06-09T11:38:01.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f161c15b060 0x7f161c143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160806ace0 0x7f160806d190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f161c0c2ae0 0x7f161c14dc40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 --2- 172.21.15.7:0/3902049690 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f161c0c88e0 0x7f161c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 >> 172.21.15.7:0/3902049690 conn(0x7f161c0cd330 msgr2=0x7f161c1575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 shutdown_connections 2023-06-09T11:38:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.366+0000 7f1621dff700 1 -- 172.21.15.7:0/3902049690 wait complete. 2023-06-09T11:38:01.374 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:01.391 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=0) 2023-06-09T11:38:01.391 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:38:01.391 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:38:01.409 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:38:01.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.625+0000 7fc4775ff700 1 Processor -- start 2023-06-09T11:38:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.625+0000 7fc4775ff700 1 -- start start 2023-06-09T11:38:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.626+0000 7fc4775ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4700c5b40 0x7fc470070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.626+0000 7fc4775ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 0x7fc470158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.626+0000 7fc4775ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4701580a0 con 0x7fc470151510 2023-06-09T11:38:01.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.626+0000 7fc4775ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4701518e0 con 0x7fc470071060 2023-06-09T11:38:01.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.626+0000 7fc4775ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4700c5f70 con 0x7fc4700c5b40 2023-06-09T11:38:01.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc47539b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc470151510 0x7fc47015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42164/0 (socket says 172.21.15.7:42164) 2023-06-09T11:38:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc47539b700 1 -- 172.21.15.7:0/1256393993 learned_addr learned my addr 172.21.15.7:0/1256393993 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc475b9c700 1 --2- 172.21.15.7:0/1256393993 >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 0x7fc470158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 --2- 172.21.15.7:0/1256393993 >> v2:172.21.15.117:3300/0 conn(0x7fc4700c5b40 0x7fc470070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc467fff700 1 -- 172.21.15.7:0/1256393993 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443420708 0 0) 0x7fc4701580a0 con 0x7fc470151510 2023-06-09T11:38:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc467fff700 1 -- 172.21.15.7:0/1256393993 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc45c0035b0 con 0x7fc470151510 2023-06-09T11:38:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 -- 172.21.15.7:0/1256393993 >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 msgr2=0x7fc470158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:01.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 --2- 172.21.15.7:0/1256393993 >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 0x7fc470158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 -- 172.21.15.7:0/1256393993 >> v1:172.21.15.7:6789/0 conn(0x7fc470151510 legacy=0x7fc47015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 -- 172.21.15.7:0/1256393993 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4700c6180 con 0x7fc4700c5b40 2023-06-09T11:38:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc475b9c700 1 --2- 172.21.15.7:0/1256393993 >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 0x7fc470158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.627+0000 7fc474b9a700 1 --2- 172.21.15.7:0/1256393993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700c5b40 0x7fc470070b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc4700703a0 tx=0x7fc460009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601dc45084119b9b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc467fff700 1 -- 172.21.15.7:0/1256393993 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460011830 con 0x7fc4700c5b40 2023-06-09T11:38:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc467fff700 1 -- 172.21.15.7:0/1256393993 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc460011990 con 0x7fc4700c5b40 2023-06-09T11:38:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc467fff700 1 -- 172.21.15.7:0/1256393993 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460011d00 con 0x7fc4700c5b40 2023-06-09T11:38:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 -- 172.21.15.7:0/1256393993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700c5b40 msgr2=0x7fc470070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 --2- 172.21.15.7:0/1256393993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700c5b40 0x7fc470070b20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc4700703a0 tx=0x7fc460009b20 comp rx=0 tx=0).stop 2023-06-09T11:38:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 -- 172.21.15.7:0/1256393993 shutdown_connections 2023-06-09T11:38:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 --2- 172.21.15.7:0/1256393993 >> v2:172.21.15.7:3300/0 conn(0x7fc470071060 0x7fc470158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 --2- 172.21.15.7:0/1256393993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4700c5b40 0x7fc470070b20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 -- 172.21.15.7:0/1256393993 >> 172.21.15.7:0/1256393993 conn(0x7fc47006a4d0 msgr2=0x7fc47006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:01.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.628+0000 7fc4775ff700 1 -- 172.21.15.7:0/1256393993 shutdown_connections 2023-06-09T11:38:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.629+0000 7fc4775ff700 1 -- 172.21.15.7:0/1256393993 wait complete. 2023-06-09T11:38:01.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.630+0000 7fc4775ff700 1 Processor -- start 2023-06-09T11:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.630+0000 7fc4775ff700 1 -- start start 2023-06-09T11:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 0x7fc470153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 0x7fc4700cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700c20a0 con 0x7fc4700c5b40 2023-06-09T11:38:01.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470154b00 con 0x7fc470151510 2023-06-09T11:38:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc4775ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4700b07d0 con 0x7fc470071060 2023-06-09T11:38:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.631+0000 7fc474b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45602/0 (socket says 172.21.15.7:45602) 2023-06-09T11:38:01.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 -- 172.21.15.7:0/2280198431 learned_addr learned my addr 172.21.15.7:0/2280198431 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc47539b700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 0x7fc470153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 -- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 msgr2=0x7fc470153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc475b9c700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 0x7fc4700cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 0x7fc470153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 -- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 msgr2=0x7fc4700cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 0x7fc4700cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 -- 172.21.15.7:0/2280198431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468009a70 con 0x7fc4700c5b40 2023-06-09T11:38:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc47539b700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 0x7fc470153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:38:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc475b9c700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 0x7fc4700cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:38:01.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc474b9a700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc460009b60 tx=0x7fc460004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4600036a0 con 0x7fc4700c5b40 2023-06-09T11:38:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46001f070 con 0x7fc4700c5b40 2023-06-09T11:38:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4600094a0 con 0x7fc4700c5b40 2023-06-09T11:38:01.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.632+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460003a40 con 0x7fc4700c5b40 2023-06-09T11:38:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.633+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc470157770 con 0x7fc4700c5b40 2023-06-09T11:38:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.633+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc470157a50 con 0x7fc4700c5b40 2023-06-09T11:38:01.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.634+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc460003ba0 con 0x7fc4700c5b40 2023-06-09T11:38:01.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.635+0000 7fc465ffb700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 0x7fc45406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:01.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.635+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fc460021070 con 0x7fc4700c5b40 2023-06-09T11:38:01.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.636+0000 7fc47539b700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 0x7fc45406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:01.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.636+0000 7fc47539b700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 0x7fc45406d1c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0011e0 tx=0x7fc46c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:01.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.640+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4600bd2d0 con 0x7fc4700c5b40 2023-06-09T11:38:01.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.892+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc470157490 con 0x7fc4700c5b40 2023-06-09T11:38:01.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.893+0000 7fc465ffb700 1 -- 172.21.15.7:0/2280198431 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fc460015090 con 0x7fc4700c5b40 2023-06-09T11:38:01.899 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:01.899 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 msgr2=0x7fc45406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 0x7fc45406d1c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0011e0 tx=0x7fc46c009040 comp rx=0 tx=0).stop 2023-06-09T11:38:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 msgr2=0x7fc4700caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:01.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc460009b60 tx=0x7fc460004750 comp rx=0 tx=0).stop 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 shutdown_connections 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc470151510 0x7fc4700cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc45406ad10 0x7fc45406d1c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc470071060 0x7fc470153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 --2- 172.21.15.7:0/2280198431 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4700c5b40 0x7fc4700caf50 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 >> 172.21.15.7:0/2280198431 conn(0x7fc47006a4d0 msgr2=0x7fc470155990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:01.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 shutdown_connections 2023-06-09T11:38:01.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:01.895+0000 7fc4775ff700 1 -- 172.21.15.7:0/2280198431 wait complete. 2023-06-09T11:38:01.903 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:01.919 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:03.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.124+0000 7f54319ff700 1 Processor -- start 2023-06-09T11:38:04.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.125+0000 7f54319ff700 1 -- start start 2023-06-09T11:38:04.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f54319ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f542c0cc5b0 0x7f542c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:04.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f54319ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c159b20 con 0x7f542c0cb9a0 2023-06-09T11:38:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f54319ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542c0cac80 con 0x7f542c0cb5d0 2023-06-09T11:38:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f54319ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542c1597d0 con 0x7f542c0cc5b0 2023-06-09T11:38:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f542a7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f542c0cb9a0 0x7f542c0cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60624/0 (socket says 172.21.15.7:60624) 2023-06-09T11:38:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f542a7fc700 1 -- 172.21.15.7:0/1772522860 learned_addr learned my addr 172.21.15.7:0/1772522860 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:04.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f542b7fe700 1 --2- 172.21.15.7:0/1772522860 >> v2:172.21.15.7:3300/0 conn(0x7f542c0cc5b0 0x7f542c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:04.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.126+0000 7f5429ffb700 1 -- 172.21.15.7:0/1772522860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846756070 0 0) 0x7f542c159b20 con 0x7f542c0cb9a0 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f5429ffb700 1 -- 172.21.15.7:0/1772522860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54140035b0 con 0x7f542c0cb9a0 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f542b7fe700 1 -- 172.21.15.7:0/1772522860 >> v1:172.21.15.117:6789/0 conn(0x7f542c0cb5d0 legacy=0x7f542c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f542b7fe700 1 -- 172.21.15.7:0/1772522860 >> v1:172.21.15.7:6789/0 conn(0x7f542c0cb9a0 legacy=0x7f542c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f542b7fe700 1 -- 172.21.15.7:0/1772522860 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542c071ab0 con 0x7f542c0cc5b0 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f542b7fe700 1 --2- 172.21.15.7:0/1772522860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cc5b0 0x7f542c0b52b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c11f6b30e6fa9a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f5429ffb700 1 -- 172.21.15.7:0/1772522860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542400e070 con 0x7f542c0cc5b0 2023-06-09T11:38:04.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f5429ffb700 1 -- 172.21.15.7:0/1772522860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542400bb20 con 0x7f542c0cc5b0 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.127+0000 7f5429ffb700 1 -- 172.21.15.7:0/1772522860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5424012430 con 0x7f542c0cc5b0 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 -- 172.21.15.7:0/1772522860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cc5b0 msgr2=0x7f542c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 --2- 172.21.15.7:0/1772522860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cc5b0 0x7f542c0b52b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542400b330 comp rx=0 tx=0).stop 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 -- 172.21.15.7:0/1772522860 shutdown_connections 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 --2- 172.21.15.7:0/1772522860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cc5b0 0x7f542c0b52b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 -- 172.21.15.7:0/1772522860 >> 172.21.15.7:0/1772522860 conn(0x7f542c070af0 msgr2=0x7f542c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 -- 172.21.15.7:0/1772522860 shutdown_connections 2023-06-09T11:38:04.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.128+0000 7f54319ff700 1 -- 172.21.15.7:0/1772522860 wait complete. 2023-06-09T11:38:04.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.130+0000 7f54319ff700 1 Processor -- start 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.130+0000 7f54319ff700 1 -- start start 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 0x7f542c1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f542c0cc5b0 0x7f542c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1590a0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542c1efdc0 con 0x7f542c0cc5b0 2023-06-09T11:38:04.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f54319ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f542c0c5500 con 0x7f542c0cb9a0 2023-06-09T11:38:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42482/0 (socket says 172.21.15.7:42482) 2023-06-09T11:38:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 -- 172.21.15.7:0/418179161 learned_addr learned my addr 172.21.15.7:0/418179161 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542b7fe700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f542c0cc5b0 0x7f542c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:04.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542a7fc700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 0x7f542c1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 -- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 msgr2=0x7f542c1549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 0x7f542c1549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 -- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f542c0cc5b0 msgr2=0x7f542c154f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f542c0cc5b0 0x7f542c154f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542affd700 1 -- 172.21.15.7:0/418179161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5424009a70 con 0x7f542c0cb5d0 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.131+0000 7f542a7fc700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 0x7f542c1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f542affd700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c4e0 tx=0x7f541c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0052d0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541c005430 con 0x7f542c0cb5d0 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542c15d4f0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541c0038c0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.132+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f542c15d210 con 0x7f542c0cb5d0 2023-06-09T11:38:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.133+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542c15d7d0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.134+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f541c003a20 con 0x7f542c0cb5d0 2023-06-09T11:38:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.135+0000 7f5413fff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 0x7f541806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.135+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f541c088ed0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.135+0000 7f542a7fc700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 0x7f541806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:04.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.136+0000 7f542a7fc700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 0x7f541806d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f542c0b4580 tx=0x7f5420009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:04.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.139+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f541c053960 con 0x7f542c0cb5d0 2023-06-09T11:38:04.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.390+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f542c074cf0 con 0x7f542c0cb5d0 2023-06-09T11:38:04.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.391+0000 7f5413fff700 1 -- 172.21.15.7:0/418179161 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f541c017020 con 0x7f542c0cb5d0 2023-06-09T11:38:04.396 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:04.397 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 msgr2=0x7f541806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 0x7f541806d190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f542c0b4580 tx=0x7f5420009040 comp rx=0 tx=0).stop 2023-06-09T11:38:04.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 msgr2=0x7f542c155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c4e0 tx=0x7f541c00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 shutdown_connections 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f542c0cc5b0 0x7f542c154f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f542c0cb5d0 0x7f542c155c70 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f541806ace0 0x7f541806d190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 --2- 172.21.15.7:0/418179161 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f542c0cb9a0 0x7f542c1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.393+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 >> 172.21.15.7:0/418179161 conn(0x7f542c070af0 msgr2=0x7f542c0c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:04.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.394+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 shutdown_connections 2023-06-09T11:38:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:04.394+0000 7f54319ff700 1 -- 172.21.15.7:0/418179161 wait complete. 2023-06-09T11:38:04.401 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:04.417 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:06.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:06.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.623+0000 7fa6809ff700 1 Processor -- start 2023-06-09T11:38:06.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.623+0000 7fa6809ff700 1 -- start start 2023-06-09T11:38:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.624+0000 7fa6809ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa6809ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c159b20 con 0x7fa67c15cd40 2023-06-09T11:38:06.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa6809ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c0cac80 con 0x7fa67c0cb5d0 2023-06-09T11:38:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa6809ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c1597d0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42492/0 (socket says 172.21.15.7:42492) 2023-06-09T11:38:06.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 -- 172.21.15.7:0/413175032 learned_addr learned my addr 172.21.15.7:0/413175032 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 -- 172.21.15.7:0/413175032 >> v1:172.21.15.117:6789/0 conn(0x7fa67c0cb5d0 legacy=0x7fa67c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 -- 172.21.15.7:0/413175032 >> v1:172.21.15.7:6789/0 conn(0x7fa67c15cd40 legacy=0x7fa67c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa679d9b700 1 -- 172.21.15.7:0/413175032 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c071ab0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.625+0000 7fa67959a700 1 -- 172.21.15.7:0/413175032 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288383368 0 0) 0x7fa67c159b20 con 0x7fa67c15cd40 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa679d9b700 1 --2- 172.21.15.7:0/413175032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa6700098f0 tx=0x7fa670009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11631445ba03c013 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa67959a700 1 -- 172.21.15.7:0/413175032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000b8a0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa67959a700 1 -- 172.21.15.7:0/413175032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000ba00 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa67959a700 1 -- 172.21.15.7:0/413175032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67000bd90 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa6809ff700 1 -- 172.21.15.7:0/413175032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 msgr2=0x7fa67c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa6809ff700 1 --2- 172.21.15.7:0/413175032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fa6700098f0 tx=0x7fa670009c00 comp rx=0 tx=0).stop 2023-06-09T11:38:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa6809ff700 1 -- 172.21.15.7:0/413175032 shutdown_connections 2023-06-09T11:38:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa6809ff700 1 --2- 172.21.15.7:0/413175032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0b52b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.626+0000 7fa6809ff700 1 -- 172.21.15.7:0/413175032 >> 172.21.15.7:0/413175032 conn(0x7fa67c070af0 msgr2=0x7fa67c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.627+0000 7fa6809ff700 1 -- 172.21.15.7:0/413175032 shutdown_connections 2023-06-09T11:38:06.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.627+0000 7fa6809ff700 1 -- 172.21.15.7:0/413175032 wait complete. 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.628+0000 7fa6809ff700 1 Processor -- start 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.628+0000 7fa6809ff700 1 -- start start 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c0cb5d0 0x7fa67c155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 0x7fa67c0c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:06.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0c4610 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0c65b0 con 0x7fa67c0cb5d0 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa6809ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c1ef080 con 0x7fa67c15cd40 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa679d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa679d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42500/0 (socket says 172.21.15.7:42500) 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.629+0000 7fa679d9b700 1 -- 172.21.15.7:0/809290527 learned_addr learned my addr 172.21.15.7:0/809290527 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa67a59c700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c0cb5d0 0x7fa67c155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa67ad9d700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 0x7fa67c0c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:06.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 -- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 msgr2=0x7fa67c0c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 0x7fa67c0c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 -- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c0cb5d0 msgr2=0x7fa67c155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c0cb5d0 0x7fa67c155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 -- 172.21.15.7:0/809290527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa670009550 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa67ad9d700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 0x7fa67c0c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa679d9b700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa670004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:06.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011a70 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67001f070 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011df0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.630+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c15d4f0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.631+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c15d210 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.631+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c15d7d0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.632+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa67001a430 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.633+0000 7fa66affd700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 0x7fa65c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.633+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fa670021070 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.634+0000 7fa67a59c700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 0x7fa65c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.634+0000 7fa67a59c700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 0x7fa65c06d190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000f60 tx=0x7fa66c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.638+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa6700560c0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.895+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa67c0b3f50 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.895+0000 7fa66affd700 1 -- 172.21.15.7:0/809290527 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fa6700596e0 con 0x7fa67c0cb9a0 2023-06-09T11:38:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:06.901 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:06.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 msgr2=0x7fa65c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 0x7fa65c06d190 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa66c000f60 tx=0x7fa66c009040 comp rx=0 tx=0).stop 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 msgr2=0x7fa67c0c3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fa670000c00 tx=0x7fa670004750 comp rx=0 tx=0).stop 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 shutdown_connections 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa65c06ace0 0x7fa65c06d190 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa67c15cd40 0x7fa67c0c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa67c0cb5d0 0x7fa67c155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 --2- 172.21.15.7:0/809290527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa67c0cb9a0 0x7fa67c0c3340 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 >> 172.21.15.7:0/809290527 conn(0x7fa67c070af0 msgr2=0x7fa67c156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 shutdown_connections 2023-06-09T11:38:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:06.898+0000 7fa6809ff700 1 -- 172.21.15.7:0/809290527 wait complete. 2023-06-09T11:38:06.905 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:06.921 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:08.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:09.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.127+0000 7f49715ff700 1 Processor -- start 2023-06-09T11:38:09.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.127+0000 7f49715ff700 1 -- start start 2023-06-09T11:38:09.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f496c1593d0 0x7f496c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 0x7f496c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496c0714f0 con 0x7f496c1593d0 2023-06-09T11:38:09.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496c0cc140 con 0x7f496c071910 2023-06-09T11:38:09.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.128+0000 7f49715ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496c0c1830 con 0x7f496c0c08d0 2023-06-09T11:38:09.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f496c1593d0 0x7f496c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f496c1593d0 0x7f496c1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42506/0 (socket says 172.21.15.7:42506) 2023-06-09T11:38:09.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 -- 172.21.15.7:0/521643273 learned_addr learned my addr 172.21.15.7:0/521643273 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:09.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496b7fe700 1 --2- 172.21.15.7:0/521643273 >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 0x7f496c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496a7fc700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 -- 172.21.15.7:0/521643273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 msgr2=0x7f496c159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 -- 172.21.15.7:0/521643273 >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 msgr2=0x7f496c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 --2- 172.21.15.7:0/521643273 >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 0x7f496c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 -- 172.21.15.7:0/521643273 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c0c0560 con 0x7f496c1593d0 2023-06-09T11:38:09.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496b7fe700 1 --2- 172.21.15.7:0/521643273 >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 0x7f496c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496a7fc700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.129+0000 7f496affd700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1597a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f496c0c9050 tx=0x7f495c00ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db457276eba96cce server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f4969ffb700 1 -- 172.21.15.7:0/521643273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c00cb70 con 0x7f496c1593d0 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f4969ffb700 1 -- 172.21.15.7:0/521643273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c00ccd0 con 0x7f496c1593d0 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f4969ffb700 1 -- 172.21.15.7:0/521643273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c018620 con 0x7f496c1593d0 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 -- 172.21.15.7:0/521643273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 msgr2=0x7f496c1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1597a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f496c0c9050 tx=0x7f495c00ad70 comp rx=0 tx=0).stop 2023-06-09T11:38:09.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 -- 172.21.15.7:0/521643273 shutdown_connections 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 --2- 172.21.15.7:0/521643273 >> v2:172.21.15.117:3300/0 conn(0x7f496c071910 0x7f496c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1597a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 --2- 172.21.15.7:0/521643273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 -- 172.21.15.7:0/521643273 >> 172.21.15.7:0/521643273 conn(0x7f496c06a4d0 msgr2=0x7f496c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 -- 172.21.15.7:0/521643273 shutdown_connections 2023-06-09T11:38:09.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.130+0000 7f49715ff700 1 -- 172.21.15.7:0/521643273 wait complete. 2023-06-09T11:38:09.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.132+0000 7f49715ff700 1 Processor -- start 2023-06-09T11:38:09.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.132+0000 7f49715ff700 1 -- start start 2023-06-09T11:38:09.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 0x7f496c1f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c1def90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c14d6d0 con 0x7f496c1593d0 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496c14cb80 con 0x7f496c071910 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f49715ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496c1eebc0 con 0x7f496c0c08d0 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f496b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f496b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42516/0 (socket says 172.21.15.7:42516) 2023-06-09T11:38:09.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.133+0000 7f496b7fe700 1 -- 172.21.15.7:0/1214592657 learned_addr learned my addr 172.21.15.7:0/1214592657 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496affd700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 0x7f496c1f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496a7fc700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c1def90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 -- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 msgr2=0x7f496c1def90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c1def90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 -- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 msgr2=0x7f496c1f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 0x7f496c1f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 -- 172.21.15.7:0/1214592657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c00a6f0 con 0x7f496c1593d0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496a7fc700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c1def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496affd700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 0x7f496c1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f496b7fe700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4964007e10 tx=0x7f496400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400de90 con 0x7f496c1593d0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496400f040 con 0x7f496c1593d0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964003c60 con 0x7f496c1593d0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.134+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496c0c1a80 con 0x7f496c1593d0 2023-06-09T11:38:09.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.135+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496c072290 con 0x7f496c1593d0 2023-06-09T11:38:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.136+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496c0c1d60 con 0x7f496c1593d0 2023-06-09T11:38:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.136+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4964004030 con 0x7f496c1593d0 2023-06-09T11:38:09.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.137+0000 7f49537fe700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 0x7f495806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.138+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7f4964089af0 con 0x7f496c1593d0 2023-06-09T11:38:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.138+0000 7f496affd700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 0x7f495806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:09.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.138+0000 7f496affd700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 0x7f495806d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f495c00afd0 tx=0x7f495c01a040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:09.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.142+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f49640546a0 con 0x7f496c1593d0 2023-06-09T11:38:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.394+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f496c1f3080 con 0x7f496c1593d0 2023-06-09T11:38:09.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.395+0000 7f49537fe700 1 -- 172.21.15.7:0/1214592657 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f496c1f3080 con 0x7f496c1593d0 2023-06-09T11:38:09.401 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:09.401 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 msgr2=0x7f495806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:09.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 0x7f495806d190 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f495c00afd0 tx=0x7f495c01a040 comp rx=0 tx=0).stop 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 msgr2=0x7f496c1583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4964007e10 tx=0x7f496400d3b0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 shutdown_connections 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f496c1593d0 0x7f496c1583f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f496c071910 0x7f496c1f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f495806ace0 0x7f495806d190 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 --2- 172.21.15.7:0/1214592657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f496c0c08d0 0x7f496c1def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 >> 172.21.15.7:0/1214592657 conn(0x7f496c06a4d0 msgr2=0x7f496c06dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 shutdown_connections 2023-06-09T11:38:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:09.398+0000 7f49715ff700 1 -- 172.21.15.7:0/1214592657 wait complete. 2023-06-09T11:38:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:09.421 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:11.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.626+0000 7fd24cdff700 1 Processor -- start 2023-06-09T11:38:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.626+0000 7fd24cdff700 1 -- start start 2023-06-09T11:38:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24cdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 0x7fd24815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 0x7fd2481592e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd248159820 con 0x7fd248070950 2023-06-09T11:38:11.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24cdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd248071ec0 con 0x7fd2481534c0 2023-06-09T11:38:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd248071730 con 0x7fd248066a40 2023-06-09T11:38:11.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd23ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 0x7fd2481592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd246d9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fd248070950 0x7fd24806e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:36114/0 (socket says 172.21.15.7:36114) 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd24659c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 0x7fd24815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.628+0000 7fd246d9d700 1 -- 172.21.15.7:0/1123013202 learned_addr learned my addr 172.21.15.7:0/1123013202 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd245d9b700 1 -- 172.21.15.7:0/1123013202 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371273942 0 0) 0x7fd248159820 con 0x7fd248070950 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd23ffff700 1 -- 172.21.15.7:0/1123013202 >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 msgr2=0x7fd24815d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd23ffff700 1 --2- 172.21.15.7:0/1123013202 >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 0x7fd24815d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd23ffff700 1 -- 172.21.15.7:0/1123013202 >> v1:172.21.15.117:6789/0 conn(0x7fd248070950 legacy=0x7fd24806e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd23ffff700 1 -- 172.21.15.7:0/1123013202 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24815d050 con 0x7fd248066a40 2023-06-09T11:38:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd24659c700 1 --2- 172.21.15.7:0/1123013202 >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 0x7fd24815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:11.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.629+0000 7fd23ffff700 1 --2- 172.21.15.7:0/1123013202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 0x7fd2481592e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd230009840 tx=0x7fd230009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c559f45e1529ab server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd245d9b700 1 -- 172.21.15.7:0/1123013202 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011830 con 0x7fd248066a40 2023-06-09T11:38:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd245d9b700 1 -- 172.21.15.7:0/1123013202 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230011990 con 0x7fd248066a40 2023-06-09T11:38:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd245d9b700 1 -- 172.21.15.7:0/1123013202 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011d20 con 0x7fd248066a40 2023-06-09T11:38:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 -- 172.21.15.7:0/1123013202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 msgr2=0x7fd2481592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 --2- 172.21.15.7:0/1123013202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 0x7fd2481592e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd230009840 tx=0x7fd230009b50 comp rx=0 tx=0).stop 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 -- 172.21.15.7:0/1123013202 shutdown_connections 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 --2- 172.21.15.7:0/1123013202 >> v2:172.21.15.117:3300/0 conn(0x7fd2481534c0 0x7fd24815d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 --2- 172.21.15.7:0/1123013202 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd248066a40 0x7fd2481592e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 -- 172.21.15.7:0/1123013202 >> 172.21.15.7:0/1123013202 conn(0x7fd2480cba20 msgr2=0x7fd2480cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 -- 172.21.15.7:0/1123013202 shutdown_connections 2023-06-09T11:38:11.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.630+0000 7fd24cdff700 1 -- 172.21.15.7:0/1123013202 wait complete. 2023-06-09T11:38:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.633+0000 7fd24cdff700 1 Processor -- start 2023-06-09T11:38:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.633+0000 7fd24cdff700 1 -- start start 2023-06-09T11:38:11.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 0x7fd2481f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 0x7fd24813cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2481534c0 0x7fd24813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480c36c0 con 0x7fd248070950 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24813dd90 con 0x7fd248066a40 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd24cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481f25d0 con 0x7fd2481534c0 2023-06-09T11:38:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.634+0000 7fd23ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 0x7fd24813cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 0x7fd2481f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd23ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 0x7fd24813cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42520/0 (socket says 172.21.15.7:42520) 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd23ffff700 1 -- 172.21.15.7:0/3228992372 learned_addr learned my addr 172.21.15.7:0/3228992372 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd246d9d700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2481534c0 0x7fd24813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 -- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2481534c0 msgr2=0x7fd24813e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2481534c0 0x7fd24813e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 -- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 msgr2=0x7fd24813cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 0x7fd24813cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 -- 172.21.15.7:0/3228992372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380094a0 con 0x7fd248066a40 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.635+0000 7fd24659c700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 0x7fd2481f1c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e900 tx=0x7fd23800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.636+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238013680 con 0x7fd248066a40 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.636+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2380137e0 con 0x7fd248066a40 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.636+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2300094a0 con 0x7fd248066a40 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.636+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238013b70 con 0x7fd248066a40 2023-06-09T11:38:11.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.636+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd248071450 con 0x7fd248066a40 2023-06-09T11:38:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.637+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd248071170 con 0x7fd248066a40 2023-06-09T11:38:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.638+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd238013cd0 con 0x7fd248066a40 2023-06-09T11:38:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.639+0000 7fd23effd700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 0x7fd22806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:11.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.639+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fd238089950 con 0x7fd248066a40 2023-06-09T11:38:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.639+0000 7fd23ffff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 0x7fd22806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.640+0000 7fd23ffff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 0x7fd22806d1c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd2481422d0 tx=0x7fd230008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:11.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.644+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd238054470 con 0x7fd248066a40 2023-06-09T11:38:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.899+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd24815ca00 con 0x7fd248066a40 2023-06-09T11:38:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.900+0000 7fd23effd700 1 -- 172.21.15.7:0/3228992372 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fd238021090 con 0x7fd248066a40 2023-06-09T11:38:11.905 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:11.906 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 msgr2=0x7fd22806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 0x7fd22806d1c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fd2481422d0 tx=0x7fd230008030 comp rx=0 tx=0).stop 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 msgr2=0x7fd2481f1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 0x7fd2481f1c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e900 tx=0x7fd23800ec10 comp rx=0 tx=0).stop 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 shutdown_connections 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd22806ad10 0x7fd22806d1c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2481534c0 0x7fd24813e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd248066a40 0x7fd2481f1c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 --2- 172.21.15.7:0/3228992372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd248070950 0x7fd24813cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 >> 172.21.15.7:0/3228992372 conn(0x7fd2480cba20 msgr2=0x7fd24815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 shutdown_connections 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:11.902+0000 7fd24cdff700 1 -- 172.21.15.7:0/3228992372 wait complete. 2023-06-09T11:38:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:11.926 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:13.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.133+0000 7fc4f41ff700 1 Processor -- start 2023-06-09T11:38:14.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.134+0000 7fc4f41ff700 1 -- start start 2023-06-09T11:38:14.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f41ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4ec153880 0x7fc4ec15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f41ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ec15a500 con 0x7fc4ec0c9f40 2023-06-09T11:38:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f41ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ec0c5f10 con 0x7fc4ec153880 2023-06-09T11:38:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec15da00 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f279c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f279c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:49214/0 (socket says 172.21.15.7:49214) 2023-06-09T11:38:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f179a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fc4ec0c9f40 0x7fc4ec15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:35492/0 (socket says 172.21.15.7:35492) 2023-06-09T11:38:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f279c700 1 -- 172.21.15.7:0/809440162 learned_addr learned my addr 172.21.15.7:0/809440162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.135+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/809440162 >> v2:172.21.15.117:3300/0 conn(0x7fc4ec153880 0x7fc4ec15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f1f9b700 1 -- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 msgr2=0x7fc4ec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f1f9b700 1 -- 172.21.15.7:0/809440162 >> v1:172.21.15.117:6789/0 conn(0x7fc4ec0c9f40 legacy=0x7fc4ec15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f1f9b700 1 -- 172.21.15.7:0/809440162 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec1596e0 con 0x7fc4ec153880 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f0f99700 1 -- 172.21.15.7:0/809440162 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392283558 0 0) 0x7fc4dc001110 con 0x7fc4ec0c9f40 2023-06-09T11:38:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f279c700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.136+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec153880 0x7fc4ec15d4c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a47884c136ea35a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f0f99700 1 -- 172.21.15.7:0/809440162 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8011830 con 0x7fc4ec153880 2023-06-09T11:38:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f0f99700 1 -- 172.21.15.7:0/809440162 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8011990 con 0x7fc4ec153880 2023-06-09T11:38:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f0f99700 1 -- 172.21.15.7:0/809440162 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8011d20 con 0x7fc4ec153880 2023-06-09T11:38:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 -- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec153880 msgr2=0x7fc4ec15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec153880 0x7fc4ec15d4c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009840 tx=0x7fc4e8009b50 comp rx=0 tx=0).stop 2023-06-09T11:38:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 -- 172.21.15.7:0/809440162 shutdown_connections 2023-06-09T11:38:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec15b4a0 0x7fc4ec0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/809440162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec153880 0x7fc4ec15d4c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 -- 172.21.15.7:0/809440162 >> 172.21.15.7:0/809440162 conn(0x7fc4ec0737f0 msgr2=0x7fc4ec0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 -- 172.21.15.7:0/809440162 shutdown_connections 2023-06-09T11:38:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.137+0000 7fc4f41ff700 1 -- 172.21.15.7:0/809440162 wait complete. 2023-06-09T11:38:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.139+0000 7fc4f41ff700 1 Processor -- start 2023-06-09T11:38:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.139+0000 7fc4f41ff700 1 -- start start 2023-06-09T11:38:14.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 0x7fc4ec157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec153880 0x7fc4ec1f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec0b2eb0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1f2190 con 0x7fc4ec0c9f40 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ec1f2750 con 0x7fc4ec153880 2023-06-09T11:38:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f279c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f279c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53810/0 (socket says 172.21.15.7:53810) 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.140+0000 7fc4f279c700 1 -- 172.21.15.7:0/1824409064 learned_addr learned my addr 172.21.15.7:0/1824409064 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 0x7fc4ec157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f179a700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec153880 0x7fc4ec1f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 -- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec153880 msgr2=0x7fc4ec1f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec153880 0x7fc4ec1f2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 -- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 msgr2=0x7fc4ec157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 0x7fc4ec157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 -- 172.21.15.7:0/1824409064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e4009b30 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 0x7fc4ec157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f279c700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009e50 tx=0x7fc4e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e400cbf0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e400f040 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4014480 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.141+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80094a0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.142+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ec159c60 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.143+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ec159980 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.144+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc4e40145e0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.144+0000 7fc4e27fc700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 0x7fc4d006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.145+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 7324+0+0 (secure 0 0 0) 0x7fc4e408a1f0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.145+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 0x7fc4d006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:14.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.145+0000 7fc4f1f9b700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 0x7fc4d006d1c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80090b0 tx=0x7fc4e8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:14.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.149+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc4e4054c80 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.402+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4ec159f40 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.403+0000 7fc4e27fc700 1 -- 172.21.15.7:0/1824409064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fc4e40582a0 con 0x7fc4ec15b4a0 2023-06-09T11:38:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:14.409 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":124,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6583,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":121}],"filesystems":[{"mdsmap":{"epoch":124,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:37:44.643728+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":63,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5741,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5741":{"gid":5741,"name":"b","rank":0,"incarnation":107,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/557223025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":557223025},{"type":"v1","addr":"172.21.15.117:6833","nonce":557223025}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.405+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 msgr2=0x7fc4d006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.405+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 0x7fc4d006d1c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc4e80090b0 tx=0x7fc4e8008030 comp rx=0 tx=0).stop 2023-06-09T11:38:14.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 msgr2=0x7fc4ec06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009e50 tx=0x7fc4e400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 shutdown_connections 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4ec15b4a0 0x7fc4ec06a740 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4ec0c9f40 0x7fc4ec157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc4d006ad10 0x7fc4d006d1c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 --2- 172.21.15.7:0/1824409064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc4ec153880 0x7fc4ec1f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:14.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 >> 172.21.15.7:0/1824409064 conn(0x7fc4ec0737f0 msgr2=0x7fc4ec0cc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 shutdown_connections 2023-06-09T11:38:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:14.406+0000 7fc4f41ff700 1 -- 172.21.15.7:0/1824409064 wait complete. 2023-06-09T11:38:14.413 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 124 2023-06-09T11:38:14.429 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:38:16.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.634+0000 7f44d41ff700 1 Processor -- start 2023-06-09T11:38:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.634+0000 7f44d41ff700 1 -- start start 2023-06-09T11:38:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.635+0000 7f44d41ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44cc151510 0x7f44cc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.635+0000 7f44d41ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc1580a0 con 0x7f44cc15b380 2023-06-09T11:38:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.635+0000 7f44d41ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44cc0c6180 con 0x7f44cc071230 2023-06-09T11:38:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.635+0000 7f44d41ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44cc1518e0 con 0x7f44cc151510 2023-06-09T11:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44cc151510 0x7f44cc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f44cc151510 0x7f44cc0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53824/0 (socket says 172.21.15.7:53824) 2023-06-09T11:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 -- 172.21.15.7:0/3682001334 learned_addr learned my addr 172.21.15.7:0/3682001334 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 -- 172.21.15.7:0/3682001334 >> v1:172.21.15.117:6789/0 conn(0x7f44cc071230 legacy=0x7f44cc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 -- 172.21.15.7:0/3682001334 >> v1:172.21.15.7:6789/0 conn(0x7f44cc15b380 legacy=0x7f44cc070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 -- 172.21.15.7:0/3682001334 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc157490 con 0x7f44cc151510 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d0f99700 1 -- 172.21.15.7:0/3682001334 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242455905 0 0) 0x7f44cc1580a0 con 0x7f44cc15b380 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.636+0000 7f44d1f9b700 1 --2- 172.21.15.7:0/3682001334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc151510 0x7f44cc0c5b40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f44c80098f0 tx=0x7f44c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986887ea8bc6b129 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d0f99700 1 -- 172.21.15.7:0/3682001334 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800b8a0 con 0x7f44cc151510 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d0f99700 1 -- 172.21.15.7:0/3682001334 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800ba00 con 0x7f44cc151510 2023-06-09T11:38:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d0f99700 1 -- 172.21.15.7:0/3682001334 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800bd90 con 0x7f44cc151510 2023-06-09T11:38:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 -- 172.21.15.7:0/3682001334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc151510 msgr2=0x7f44cc0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 --2- 172.21.15.7:0/3682001334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc151510 0x7f44cc0c5b40 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f44c80098f0 tx=0x7f44c8009c00 comp rx=0 tx=0).stop 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 -- 172.21.15.7:0/3682001334 shutdown_connections 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 --2- 172.21.15.7:0/3682001334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc151510 0x7f44cc0c5b40 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 -- 172.21.15.7:0/3682001334 >> 172.21.15.7:0/3682001334 conn(0x7f44cc06a4d0 msgr2=0x7f44cc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 -- 172.21.15.7:0/3682001334 shutdown_connections 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.637+0000 7f44d41ff700 1 -- 172.21.15.7:0/3682001334 wait complete. 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.639+0000 7f44d41ff700 1 Processor -- start 2023-06-09T11:38:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.639+0000 7f44d41ff700 1 -- start start 2023-06-09T11:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44cc151510 0x7f44cc156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44cc15b380 0x7f44cc1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc14b4a0 con 0x7f44cc071230 2023-06-09T11:38:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0c73f0 con 0x7f44cc15b380 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44cc0c76d0 con 0x7f44cc151510 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53832/0 (socket says 172.21.15.7:53832) 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.640+0000 7f44d1f9b700 1 -- 172.21.15.7:0/4220948518 learned_addr learned my addr 172.21.15.7:0/4220948518 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d279c700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44cc15b380 0x7f44cc1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d179a700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44cc151510 0x7f44cc156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 -- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44cc151510 msgr2=0x7f44cc156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44cc151510 0x7f44cc156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 -- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44cc15b380 msgr2=0x7f44cc1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44cc15b380 0x7f44cc1569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 -- 172.21.15.7:0/4220948518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8009550 con 0x7f44cc071230 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44d1f9b700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0c60b0 tx=0x7f44c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c801a900 con 0x7f44cc071230 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c801f070 con 0x7f44cc071230 2023-06-09T11:38:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.641+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c801ac60 con 0x7f44cc071230 2023-06-09T11:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.642+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc157a50 con 0x7f44cc071230 2023-06-09T11:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.642+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44cc157770 con 0x7f44cc071230 2023-06-09T11:38:16.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.643+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44cc157d30 con 0x7f44cc071230 2023-06-09T11:38:16.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.644+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44c801b430 con 0x7f44cc071230 2023-06-09T11:38:16.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.644+0000 7f44c27fc700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 0x7f44b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:16.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.644+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f44c8021070 con 0x7f44cc071230 2023-06-09T11:38:16.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.645+0000 7f44d179a700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 0x7f44b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:16.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.645+0000 7f44d179a700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 0x7f44b006d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f44cc14e900 tx=0x7f44bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:16.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.649+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44c8055f00 con 0x7f44cc071230 2023-06-09T11:38:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.903+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44cc0c0350 con 0x7f44cc071230 2023-06-09T11:38:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.904+0000 7f44c27fc700 1 -- 172.21.15.7:0/4220948518 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f44c8059520 con 0x7f44cc071230 2023-06-09T11:38:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:16.909 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:14.487326+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 msgr2=0x7f44b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 0x7f44b006d190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f44cc14e900 tx=0x7f44bc009040 comp rx=0 tx=0).stop 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 msgr2=0x7f44cc14af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0c60b0 tx=0x7f44c8004750 comp rx=0 tx=0).stop 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 shutdown_connections 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44cc15b380 0x7f44cc1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44cc071230 0x7f44cc14af60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44b006ace0 0x7f44b006d190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 --2- 172.21.15.7:0/4220948518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44cc151510 0x7f44cc156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 >> 172.21.15.7:0/4220948518 conn(0x7f44cc06a4d0 msgr2=0x7f44cc14d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 shutdown_connections 2023-06-09T11:38:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:16.906+0000 7f44d41ff700 1 -- 172.21.15.7:0/4220948518 wait complete. 2023-06-09T11:38:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 125 2023-06-09T11:38:16.929 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:38:16.930 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:38:16.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.124+0000 7f8bdb7ff700 1 Processor -- start 2023-06-09T11:38:17.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.124+0000 7f8bdb7ff700 1 -- start start 2023-06-09T11:38:17.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.125+0000 7f8bdb7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8bd4153880 0x7f8bd415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:17.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bdb7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd415a500 con 0x7f8bd40c9f40 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bdb7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd415da00 con 0x7f8bd415b4a0 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bdb7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd40c5f10 con 0x7f8bd4153880 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bd8d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8bd40c9f40 0x7f8bd415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53542/0 (socket says 172.21.15.7:53542) 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bd8d9a700 1 -- 172.21.15.7:0/1902204897 learned_addr learned my addr 172.21.15.7:0/1902204897 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bd959b700 1 --2- 172.21.15.7:0/1902204897 >> v2:172.21.15.117:3300/0 conn(0x7f8bd4153880 0x7f8bd415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363234433 0 0) 0x7f8bc4001110 con 0x7f8bd40c9f40 2023-06-09T11:38:17.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.126+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc00035b0 con 0x7f8bd40c9f40 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825917378 0 0) 0x7f8bcc001490 con 0x7f8bd415b4a0 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc4001110 con 0x7f8bd415b4a0 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002089296 0 0) 0x7f8bc00035b0 con 0x7f8bd40c9f40 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bd959b700 1 -- 172.21.15.7:0/1902204897 >> v1:172.21.15.117:6789/0 conn(0x7f8bd415b4a0 legacy=0x7f8bd40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bd959b700 1 -- 172.21.15.7:0/1902204897 >> v1:172.21.15.7:6789/0 conn(0x7f8bd40c9f40 legacy=0x7f8bd415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bd959b700 1 -- 172.21.15.7:0/1902204897 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd41596e0 con 0x7f8bd4153880 2023-06-09T11:38:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bd959b700 1 --2- 172.21.15.7:0/1902204897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd4153880 0x7f8bd415d4c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009840 tx=0x7f8bd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37e43d13733b5ec4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4001800 con 0x7f8bd4153880 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0004030 con 0x7f8bd4153880 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.127+0000 7f8bcbfff700 1 -- 172.21.15.7:0/1902204897 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00043a0 con 0x7f8bd4153880 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/1902204897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd4153880 msgr2=0x7f8bd415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/1902204897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd4153880 0x7f8bd415d4c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009840 tx=0x7f8bd0009b50 comp rx=0 tx=0).stop 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/1902204897 shutdown_connections 2023-06-09T11:38:17.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/1902204897 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd4153880 0x7f8bd415d4c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/1902204897 >> 172.21.15.7:0/1902204897 conn(0x7f8bd40737f0 msgr2=0x7f8bd40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/1902204897 shutdown_connections 2023-06-09T11:38:17.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.128+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/1902204897 wait complete. 2023-06-09T11:38:17.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.130+0000 7f8bdb7ff700 1 Processor -- start 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.130+0000 7f8bdb7ff700 1 -- start start 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bd4153880 0x7f8bd4156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 0x7f8bd41ef980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41ef560 con 0x7f8bd40c9f40 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40cf3e0 con 0x7f8bd415b4a0 2023-06-09T11:38:17.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bdb7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd41eefb0 con 0x7f8bd4153880 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bd959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bd959b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53836/0 (socket says 172.21.15.7:53836) 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bd959b700 1 -- 172.21.15.7:0/3801017017 learned_addr learned my addr 172.21.15.7:0/3801017017 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bd8d9a700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bd4153880 0x7f8bd4156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.131+0000 7f8bd9d9c700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 0x7f8bd41ef980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 -- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bd4153880 msgr2=0x7f8bd4156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bd4153880 0x7f8bd4156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 -- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 msgr2=0x7f8bd41ef980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 0x7f8bd41ef980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 -- 172.21.15.7:0/3801017017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00094a0 con 0x7f8bd40c9f40 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd9d9c700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 0x7f8bd41ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bd959b700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009b90 tx=0x7f8bd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0004590 con 0x7f8bd40c9f40 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd001f070 con 0x7f8bd40c9f40 2023-06-09T11:38:17.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0011870 con 0x7f8bd40c9f40 2023-06-09T11:38:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd4159c60 con 0x7f8bd40c9f40 2023-06-09T11:38:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.132+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd4159980 con 0x7f8bd40c9f40 2023-06-09T11:38:17.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.133+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4159f40 con 0x7f8bd40c9f40 2023-06-09T11:38:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.134+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bd00119d0 con 0x7f8bd40c9f40 2023-06-09T11:38:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.136+0000 7f8bc9ffb700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.136+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f8bd0021070 con 0x7f8bd40c9f40 2023-06-09T11:38:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.136+0000 7f8bd8d9a700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:17.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.136+0000 7f8bd8d9a700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 0x7f8bb806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000fd0 tx=0x7f8bc4000f60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:17.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.140+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8bd0055b80 con 0x7f8bd40c9f40 2023-06-09T11:38:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.393+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bd404e8c0 con 0x7f8bd40c9f40 2023-06-09T11:38:17.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.394+0000 7f8bc9ffb700 1 -- 172.21.15.7:0/3801017017 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f8bd002b090 con 0x7f8bd40c9f40 2023-06-09T11:38:17.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:17.399 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:14.487326+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 msgr2=0x7f8bb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 0x7f8bb806d190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4000fd0 tx=0x7f8bc4000f60 comp rx=0 tx=0).stop 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 msgr2=0x7f8bd41f2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009b90 tx=0x7f8bd0004750 comp rx=0 tx=0).stop 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 shutdown_connections 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8bd415b4a0 0x7f8bd41ef980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8bd40c9f40 0x7f8bd41f2520 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8bb806ace0 0x7f8bb806d190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 --2- 172.21.15.7:0/3801017017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8bd4153880 0x7f8bd4156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 >> 172.21.15.7:0/3801017017 conn(0x7f8bd40737f0 msgr2=0x7f8bd414ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 shutdown_connections 2023-06-09T11:38:17.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:17.396+0000 7f8bdb7ff700 1 -- 172.21.15.7:0/3801017017 wait complete. 2023-06-09T11:38:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 125 2023-06-09T11:38:17.420 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 125, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 125, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5852": { "addr": "172.21.15.117:6835/1343666954", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1343666954, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1343666954, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5852, "incarnation": 116, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:38:14.487326+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 5852, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:38:18.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:18.079 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:19.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.638+0000 7fba07dff700 1 Processor -- start 2023-06-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.638+0000 7fba07dff700 1 -- start start 2023-06-09T11:38:19.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba001593d0 0x7fba001597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 0x7fba00159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba00071910 0x7fba00071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba000714f0 con 0x7fba00071910 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba000cc140 con 0x7fba001593d0 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba07dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba000c1830 con 0x7fba000c08d0 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba0639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba00071910 0x7fba00071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba0539a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 0x7fba00159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba0539a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 0x7fba00159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:49240/0 (socket says 172.21.15.7:49240) 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba05b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fba001593d0 0x7fba001597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba0539a700 1 -- 172.21.15.7:0/1414840275 learned_addr learned my addr 172.21.15.7:0/1414840275 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.639+0000 7fba0639c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fba00071910 0x7fba00071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53852/0 (socket says 172.21.15.7:53852) 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 -- 172.21.15.7:0/1414840275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 msgr2=0x7fba00159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:19.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 --2- 172.21.15.7:0/1414840275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 0x7fba00159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 -- 172.21.15.7:0/1414840275 >> v2:172.21.15.117:3300/0 conn(0x7fba001593d0 msgr2=0x7fba001597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 --2- 172.21.15.7:0/1414840275 >> v2:172.21.15.117:3300/0 conn(0x7fba001593d0 0x7fba001597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 -- 172.21.15.7:0/1414840275 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000c0560 con 0x7fba00071910 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba0639c700 1 --2- 172.21.15.7:0/1414840275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba00071910 0x7fba00071d50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fba0006ff10 tx=0x7fb9f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb2e10d13118b5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba04b99700 1 -- 172.21.15.7:0/1414840275 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f800e070 con 0x7fba00071910 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba04b99700 1 -- 172.21.15.7:0/1414840275 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f800bb20 con 0x7fba00071910 2023-06-09T11:38:19.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.640+0000 7fba04b99700 1 -- 172.21.15.7:0/1414840275 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8012430 con 0x7fba00071910 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 -- 172.21.15.7:0/1414840275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba00071910 msgr2=0x7fba00071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 --2- 172.21.15.7:0/1414840275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba00071910 0x7fba00071d50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fba0006ff10 tx=0x7fb9f800b330 comp rx=0 tx=0).stop 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 -- 172.21.15.7:0/1414840275 shutdown_connections 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 --2- 172.21.15.7:0/1414840275 >> v2:172.21.15.117:3300/0 conn(0x7fba001593d0 0x7fba001597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 --2- 172.21.15.7:0/1414840275 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba00071910 0x7fba00071d50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 --2- 172.21.15.7:0/1414840275 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba000c08d0 0x7fba00159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 -- 172.21.15.7:0/1414840275 >> 172.21.15.7:0/1414840275 conn(0x7fba0006a4d0 msgr2=0x7fba0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:19.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 -- 172.21.15.7:0/1414840275 shutdown_connections 2023-06-09T11:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.641+0000 7fba07dff700 1 -- 172.21.15.7:0/1414840275 wait complete. 2023-06-09T11:38:19.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.643+0000 7fba07dff700 1 Processor -- start 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.644+0000 7fba07dff700 1 -- start start 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00071910 0x7fba000af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba001593d0 0x7fba000ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000af890 con 0x7fba000c08d0 2023-06-09T11:38:19.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0014d0f0 con 0x7fba00071910 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba07dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0014dec0 con 0x7fba001593d0 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba0539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba0539a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53866/0 (socket says 172.21.15.7:53866) 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba0539a700 1 -- 172.21.15.7:0/2196908480 learned_addr learned my addr 172.21.15.7:0/2196908480 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba05b9b700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00071910 0x7fba000af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.645+0000 7fba0639c700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba001593d0 0x7fba000ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 -- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba001593d0 msgr2=0x7fba000ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:19.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba001593d0 0x7fba000ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 -- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00071910 msgr2=0x7fba000af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00071910 0x7fba000af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 -- 172.21.15.7:0/2196908480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8009a70 con 0x7fba000c08d0 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba0539a700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e960 tx=0x7fb9f000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0013680 con 0x7fba000c08d0 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f00137e0 con 0x7fba000c08d0 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba000c1a80 con 0x7fba000c08d0 2023-06-09T11:38:19.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0013b50 con 0x7fba000c08d0 2023-06-09T11:38:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.646+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba00072290 con 0x7fba000c08d0 2023-06-09T11:38:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.647+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba000c1d60 con 0x7fba000c08d0 2023-06-09T11:38:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.648+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9f0013cb0 con 0x7fba000c08d0 2023-06-09T11:38:19.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.649+0000 7fb9f67fc700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 0x7fb9e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.649+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fb9f0089d40 con 0x7fba000c08d0 2023-06-09T11:38:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.650+0000 7fba05b9b700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 0x7fb9e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:19.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.650+0000 7fba05b9b700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 0x7fb9e406d1c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00ac40 tx=0x7fb9fc00a320 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.654+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb9f0054800 con 0x7fba000c08d0 2023-06-09T11:38:19.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.909+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba00074cf0 con 0x7fba000c08d0 2023-06-09T11:38:19.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.909+0000 7fb9f67fc700 1 -- 172.21.15.7:0/2196908480 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fb9f0057e20 con 0x7fba000c08d0 2023-06-09T11:38:19.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:19.915 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:14.487326+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 msgr2=0x7fb9e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 0x7fb9e406d1c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00ac40 tx=0x7fb9fc00a320 comp rx=0 tx=0).stop 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 msgr2=0x7fba0006df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e960 tx=0x7fb9f000ec70 comp rx=0 tx=0).stop 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 shutdown_connections 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9e406ad10 0x7fb9e406d1c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fba001593d0 0x7fba000ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fba00071910 0x7fba000af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 --2- 172.21.15.7:0/2196908480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fba000c08d0 0x7fba0006df40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 >> 172.21.15.7:0/2196908480 conn(0x7fba0006a4d0 msgr2=0x7fba000c6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 shutdown_connections 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:19.912+0000 7fba07dff700 1 -- 172.21.15.7:0/2196908480 wait complete. 2023-06-09T11:38:19.919 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 125 2023-06-09T11:38:19.938 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:21.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.145+0000 7f10631ff700 1 Processor -- start 2023-06-09T11:38:22.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.145+0000 7f10631ff700 1 -- start start 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f10631ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c153880 0x7f105c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f10631ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f105c15b4a0 0x7f105c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f10631ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105c15a500 con 0x7f105c0c9f40 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f10631ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105c0c5f10 con 0x7f105c15b4a0 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f10631ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c15da00 con 0x7f105c153880 2023-06-09T11:38:22.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f106179c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f105c15b4a0 0x7f105c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f105bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f105c0c9f40 0x7f105c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53546/0 (socket says 172.21.15.7:53546) 2023-06-09T11:38:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f106179c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f105c15b4a0 0x7f105c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53878/0 (socket says 172.21.15.7:53878) 2023-06-09T11:38:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f105bfff700 1 -- 172.21.15.7:0/2482304673 learned_addr learned my addr 172.21.15.7:0/2482304673 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:22.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.146+0000 7f1060f9b700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c153880 0x7f105c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f106179c700 1 -- 172.21.15.7:0/2482304673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c153880 msgr2=0x7f105c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:22.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f106179c700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c153880 0x7f105c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f106179c700 1 -- 172.21.15.7:0/2482304673 >> v1:172.21.15.7:6789/0 conn(0x7f105c0c9f40 legacy=0x7f105c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f106179c700 1 -- 172.21.15.7:0/2482304673 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105c1596e0 con 0x7f105c15b4a0 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f105b7fe700 1 -- 172.21.15.7:0/2482304673 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276344057 0 0) 0x7f1048001110 con 0x7f105c0c9f40 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f106179c700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c15b4a0 0x7f105c0c4e30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d90 tx=0x7f105400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d48c7c8d5464cda server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f105b7fe700 1 -- 172.21.15.7:0/2482304673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105400e070 con 0x7f105c15b4a0 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f105b7fe700 1 -- 172.21.15.7:0/2482304673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105400bb20 con 0x7f105c15b4a0 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.147+0000 7f105b7fe700 1 -- 172.21.15.7:0/2482304673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054012430 con 0x7f105c15b4a0 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 -- 172.21.15.7:0/2482304673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c15b4a0 msgr2=0x7f105c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c15b4a0 0x7f105c0c4e30 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1054009d90 tx=0x7f105400b330 comp rx=0 tx=0).stop 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 -- 172.21.15.7:0/2482304673 shutdown_connections 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c15b4a0 0x7f105c0c4e30 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 --2- 172.21.15.7:0/2482304673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c153880 0x7f105c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 -- 172.21.15.7:0/2482304673 >> 172.21.15.7:0/2482304673 conn(0x7f105c0737f0 msgr2=0x7f105c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 -- 172.21.15.7:0/2482304673 shutdown_connections 2023-06-09T11:38:22.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.148+0000 7f10631ff700 1 -- 172.21.15.7:0/2482304673 wait complete. 2023-06-09T11:38:22.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.150+0000 7f10631ff700 1 Processor -- start 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.151+0000 7f10631ff700 1 -- start start 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c153880 0x7f105c158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c15b4a0 0x7f105c0b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0b4650 con 0x7f105c0c9f40 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0c7e00 con 0x7f105c153880 2023-06-09T11:38:22.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f10631ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105c0c8130 con 0x7f105c15b4a0 2023-06-09T11:38:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f1060f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f1060f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53894/0 (socket says 172.21.15.7:53894) 2023-06-09T11:38:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f1060f9b700 1 -- 172.21.15.7:0/2016840076 learned_addr learned my addr 172.21.15.7:0/2016840076 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f105bfff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c153880 0x7f105c158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.152+0000 7f106179c700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c15b4a0 0x7f105c0b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 -- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c15b4a0 msgr2=0x7f105c0b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c15b4a0 0x7f105c0b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 -- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c153880 msgr2=0x7f105c158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c153880 0x7f105c158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 -- 172.21.15.7:0/2016840076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1054009a70 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f1060f9b700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f105000e9e0 tx=0x7f105000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050009710 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105000ee80 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c159c60 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105000ca50 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.153+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c159980 con 0x7f105c0c9f40 2023-06-09T11:38:22.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.154+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c159f40 con 0x7f105c0c9f40 2023-06-09T11:38:22.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.155+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f105000cbb0 con 0x7f105c0c9f40 2023-06-09T11:38:22.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.156+0000 7f10597fa700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:22.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.157+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f1050089b00 con 0x7f105c0c9f40 2023-06-09T11:38:22.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.157+0000 7f105bfff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:22.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.157+0000 7f105bfff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 0x7f104006d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f105c1595f0 tx=0x7f1048007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:22.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.160+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f105008be60 con 0x7f105c0c9f40 2023-06-09T11:38:22.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.414+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f105c074cf0 con 0x7f105c0c9f40 2023-06-09T11:38:22.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.415+0000 7f10597fa700 1 -- 172.21.15.7:0/2016840076 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 125 v125) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f1050017090 con 0x7f105c0c9f40 2023-06-09T11:38:22.420 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:22.421 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":125,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":125,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:14.487326+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:22.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 msgr2=0x7f104006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 0x7f104006d190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f105c1595f0 tx=0x7f1048007c20 comp rx=0 tx=0).stop 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 msgr2=0x7f105c0c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f105000e9e0 tx=0x7f105000ecf0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 shutdown_connections 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f104006ace0 0x7f104006d190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f105c15b4a0 0x7f105c0b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f105c0c9f40 0x7f105c0c6560 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 --2- 172.21.15.7:0/2016840076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f105c153880 0x7f105c158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 >> 172.21.15.7:0/2016840076 conn(0x7f105c0737f0 msgr2=0x7f105c15c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 shutdown_connections 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:22.417+0000 7f10631ff700 1 -- 172.21.15.7:0/2016840076 wait complete. 2023-06-09T11:38:22.424 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 125 2023-06-09T11:38:22.440 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:24.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.647+0000 7f3cd3bff700 1 Processor -- start 2023-06-09T11:38:24.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.647+0000 7f3cd3bff700 1 -- start start 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.648+0000 7f3cd3bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ccc15a0c0 0x7f3ccc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.648+0000 7f3cd3bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0c0bf0 con 0x7f3ccc0c1bc0 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.648+0000 7f3cd3bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0702f0 con 0x7f3ccc0c0140 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.648+0000 7f3cd3bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc070110 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd199b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ccc0c1bc0 0x7f3ccc1599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42270/0 (socket says 172.21.15.7:42270) 2023-06-09T11:38:24.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd199b700 1 -- 172.21.15.7:0/2195766767 learned_addr learned my addr 172.21.15.7:0/2195766767 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:24.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd119a700 1 --2- 172.21.15.7:0/2195766767 >> v2:172.21.15.7:3300/0 conn(0x7f3ccc15a0c0 0x7f3ccc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd119a700 1 -- 172.21.15.7:0/2195766767 >> v1:172.21.15.117:6789/0 conn(0x7f3ccc0c0140 legacy=0x7f3ccc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd119a700 1 -- 172.21.15.7:0/2195766767 >> v1:172.21.15.7:6789/0 conn(0x7f3ccc0c1bc0 legacy=0x7f3ccc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd119a700 1 -- 172.21.15.7:0/2195766767 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0c0e90 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.649+0000 7f3cd119a700 1 --2- 172.21.15.7:0/2195766767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc1501b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009840 tx=0x7f3cbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bf53ef07433fac server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd0999700 1 -- 172.21.15.7:0/2195766767 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011830 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd0999700 1 -- 172.21.15.7:0/2195766767 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc011990 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd0999700 1 -- 172.21.15.7:0/2195766767 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc011d20 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 -- 172.21.15.7:0/2195766767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 msgr2=0x7f3ccc1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/2195766767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc1501b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009840 tx=0x7f3cbc009b50 comp rx=0 tx=0).stop 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 -- 172.21.15.7:0/2195766767 shutdown_connections 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/2195766767 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc1501b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 -- 172.21.15.7:0/2195766767 >> 172.21.15.7:0/2195766767 conn(0x7f3ccc06a4d0 msgr2=0x7f3ccc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 -- 172.21.15.7:0/2195766767 shutdown_connections 2023-06-09T11:38:24.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.650+0000 7f3cd3bff700 1 -- 172.21.15.7:0/2195766767 wait complete. 2023-06-09T11:38:24.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.652+0000 7f3cd3bff700 1 Processor -- start 2023-06-09T11:38:24.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.652+0000 7f3cd3bff700 1 -- start start 2023-06-09T11:38:24.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ccc0c0140 0x7f3ccc1ef770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ccc0c1bc0 0x7f3ccc0c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1532a0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1ef2e0 con 0x7f3ccc0c1bc0 2023-06-09T11:38:24.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd3bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc14e1a0 con 0x7f3ccc0c0140 2023-06-09T11:38:24.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52654/0 (socket says 172.21.15.7:52654) 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd219c700 1 -- 172.21.15.7:0/284590788 learned_addr learned my addr 172.21.15.7:0/284590788 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd199b700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ccc0c0140 0x7f3ccc1ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.653+0000 7f3cd119a700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ccc0c1bc0 0x7f3ccc0c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 -- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ccc0c0140 msgr2=0x7f3ccc1ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ccc0c0140 0x7f3ccc1ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 -- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ccc0c1bc0 msgr2=0x7f3ccc0c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ccc0c1bc0 0x7f3ccc0c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 -- 172.21.15.7:0/284590788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc0094a0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd219c700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400cc50 tx=0x7f3cc400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4003c00 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4003d60 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400aab0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.654+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc0c1450 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.655+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0c1170 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.656+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc0705d0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.656+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3cc400ac10 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.657+0000 7f3cc27fc700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 0x7f3cb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.658+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f3cc4089a90 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.658+0000 7f3cd199b700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 0x7f3cb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:24.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.658+0000 7f3cd199b700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 0x7f3cb006d1c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005910 tx=0x7f3cc8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:24.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.662+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3cc40544c0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.919+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ccc1ef4a0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.920+0000 7f3cc27fc700 1 -- 172.21.15.7:0/284590788 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f3cc4057ae0 con 0x7f3ccc15a0c0 2023-06-09T11:38:24.925 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:24.925 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:23.703285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 msgr2=0x7f3cb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 0x7f3cb006d1c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8005910 tx=0x7f3cc8009040 comp rx=0 tx=0).stop 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 msgr2=0x7f3ccc0c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400cc50 tx=0x7f3cc400cf60 comp rx=0 tx=0).stop 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 shutdown_connections 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ccc15a0c0 0x7f3ccc0c4930 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3cb006ad10 0x7f3cb006d1c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ccc0c0140 0x7f3ccc1ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 --2- 172.21.15.7:0/284590788 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ccc0c1bc0 0x7f3ccc0c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:24.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 >> 172.21.15.7:0/284590788 conn(0x7f3ccc06a4d0 msgr2=0x7f3ccc14d150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:24.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 shutdown_connections 2023-06-09T11:38:24.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:24.922+0000 7f3cd3bff700 1 -- 172.21.15.7:0/284590788 wait complete. 2023-06-09T11:38:24.929 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 126 2023-06-09T11:38:24.945 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:26.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:27.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.153+0000 7fcfa1fff700 1 Processor -- start 2023-06-09T11:38:27.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.153+0000 7fcfa1fff700 1 -- start start 2023-06-09T11:38:27.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.154+0000 7fcfa1fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf9c153880 0x7fcf9c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.154+0000 7fcfa1fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fcf9c15b4a0 0x7fcf9c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.154+0000 7fcfa1fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9c15a500 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.154+0000 7fcfa1fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c0c5f10 con 0x7fcf9c153880 2023-06-09T11:38:27.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.154+0000 7fcfa1fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9c15da00 con 0x7fcf9c15b4a0 2023-06-09T11:38:27.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcf9c153880 0x7fcf9c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcf9c0c9f40 0x7fcf9c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42272/0 (socket says 172.21.15.7:42272) 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9affd700 1 -- 172.21.15.7:0/4064284468 learned_addr learned my addr 172.21.15.7:0/4064284468 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9bfff700 1 --2- 172.21.15.7:0/4064284468 >> v2:172.21.15.117:3300/0 conn(0x7fcf9c15b4a0 0x7fcf9c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/4064284468 >> v2:172.21.15.117:3300/0 conn(0x7fcf9c15b4a0 msgr2=0x7fcf9c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 --2- 172.21.15.7:0/4064284468 >> v2:172.21.15.117:3300/0 conn(0x7fcf9c15b4a0 0x7fcf9c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/4064284468 >> v1:172.21.15.7:6789/0 conn(0x7fcf9c0c9f40 legacy=0x7fcf9c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9a7fc700 1 -- 172.21.15.7:0/4064284468 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149987071 0 0) 0x7fcf90001110 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/4064284468 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9c1596e0 con 0x7fcf9c153880 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.155+0000 7fcf9b7fe700 1 --2- 172.21.15.7:0/4064284468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c153880 0x7fcf9c15d4c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0c4640 tx=0x7fcf8c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2515ae28d0fcd3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcf9a7fc700 1 -- 172.21.15.7:0/4064284468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011830 con 0x7fcf9c153880 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcf9a7fc700 1 -- 172.21.15.7:0/4064284468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c011990 con 0x7fcf9c153880 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcf9a7fc700 1 -- 172.21.15.7:0/4064284468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c011d20 con 0x7fcf9c153880 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 -- 172.21.15.7:0/4064284468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c153880 msgr2=0x7fcf9c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/4064284468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c153880 0x7fcf9c15d4c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c0c4640 tx=0x7fcf8c009b20 comp rx=0 tx=0).stop 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 -- 172.21.15.7:0/4064284468 shutdown_connections 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/4064284468 >> v2:172.21.15.117:3300/0 conn(0x7fcf9c15b4a0 0x7fcf9c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/4064284468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c153880 0x7fcf9c15d4c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 -- 172.21.15.7:0/4064284468 >> 172.21.15.7:0/4064284468 conn(0x7fcf9c0737f0 msgr2=0x7fcf9c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 -- 172.21.15.7:0/4064284468 shutdown_connections 2023-06-09T11:38:27.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.156+0000 7fcfa1fff700 1 -- 172.21.15.7:0/4064284468 wait complete. 2023-06-09T11:38:27.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.158+0000 7fcfa1fff700 1 Processor -- start 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.158+0000 7fcfa1fff700 1 -- start start 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf9c153880 0x7fcf9c0cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf9c15b4a0 0x7fcf9c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c1565d0 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c14ce40 con 0x7fcf9c15b4a0 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcfa1fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c156010 con 0x7fcf9c153880 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcf9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcf9b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52666/0 (socket says 172.21.15.7:52666) 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.159+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/1864327939 learned_addr learned my addr 172.21.15.7:0/1864327939 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf9c153880 msgr2=0x7fcf9c0cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf9c153880 0x7fcf9c0cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf9c15b4a0 msgr2=0x7fcf9c148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf9c15b4a0 0x7fcf9c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 -- 172.21.15.7:0/1864327939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0094a0 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf9b7fe700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0090b0 tx=0x7fcf8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0038a0 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c01f070 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c159c60 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c003c00 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.160+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c159980 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.161+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c159f40 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.162+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf8c01a430 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.164+0000 7fcf83fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 0x7fcf8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.164+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fcf8c021070 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.164+0000 7fcf9affd700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 0x7fcf8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:27.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.164+0000 7fcf9affd700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 0x7fcf8806d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c14c7b0 tx=0x7fcf90001330 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:27.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.168+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcf8c055f80 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.423+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf9c074cf0 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.423+0000 7fcf83fff700 1 -- 172.21.15.7:0/1864327939 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 126 v126) v1 ==== 95+0+5467 (secure 0 0 0) 0x7fcf8c05de60 con 0x7fcf9c0c9f40 2023-06-09T11:38:27.429 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:27.429 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":126,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":126,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:23.703285+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 msgr2=0x7fcf8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 0x7fcf8806d190 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c14c7b0 tx=0x7fcf90001330 comp rx=0 tx=0).stop 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 msgr2=0x7fcf9c14c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0090b0 tx=0x7fcf8c004750 comp rx=0 tx=0).stop 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 shutdown_connections 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcf9c15b4a0 0x7fcf9c148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcf9c0c9f40 0x7fcf9c14c900 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcf8806ace0 0x7fcf8806d190 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 --2- 172.21.15.7:0/1864327939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcf9c153880 0x7fcf9c0cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 >> 172.21.15.7:0/1864327939 conn(0x7fcf9c0737f0 msgr2=0x7fcf9c1528d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 shutdown_connections 2023-06-09T11:38:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:27.426+0000 7fcfa1fff700 1 -- 172.21.15.7:0/1864327939 wait complete. 2023-06-09T11:38:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 126 2023-06-09T11:38:27.449 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:29.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.658+0000 7effd6dff700 1 Processor -- start 2023-06-09T11:38:29.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.658+0000 7effd6dff700 1 -- start start 2023-06-09T11:38:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.659+0000 7effd6dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effd00c5b40 0x7effd0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.659+0000 7effd6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.659+0000 7effd6dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd01580a0 con 0x7effd0151510 2023-06-09T11:38:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.659+0000 7effd6dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd00714e0 con 0x7effd00c5b40 2023-06-09T11:38:29.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.659+0000 7effd6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd01518e0 con 0x7effd0071060 2023-06-09T11:38:29.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effcffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7effd00c5b40 0x7effd0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54864/0 (socket says 172.21.15.7:54864) 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 -- 172.21.15.7:0/2863982261 learned_addr learned my addr 172.21.15.7:0/2863982261 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd4b9b700 1 --1- 172.21.15.7:0/2863982261 >> v1:172.21.15.117:6789/0 conn(0x7effd0151510 0x7effd015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:43738/0 (socket says 172.21.15.7:43738) 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 -- 172.21.15.7:0/2863982261 >> v2:172.21.15.117:3300/0 conn(0x7effd00c5b40 msgr2=0x7effd0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 --2- 172.21.15.7:0/2863982261 >> v2:172.21.15.117:3300/0 conn(0x7effd00c5b40 0x7effd0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 -- 172.21.15.7:0/2863982261 >> v1:172.21.15.117:6789/0 conn(0x7effd0151510 legacy=0x7effd015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.660+0000 7effd539c700 1 -- 172.21.15.7:0/2863982261 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd00c6180 con 0x7effd0071060 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd539c700 1 --2- 172.21.15.7:0/2863982261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7effc8009e50 tx=0x7effc800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520537cce11162fd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effcf7fe700 1 -- 172.21.15.7:0/2863982261 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc800e070 con 0x7effd0071060 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effcf7fe700 1 -- 172.21.15.7:0/2863982261 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc800b910 con 0x7effd0071060 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effcf7fe700 1 -- 172.21.15.7:0/2863982261 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc800bcc0 con 0x7effd0071060 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 -- 172.21.15.7:0/2863982261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 msgr2=0x7effd0158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 --2- 172.21.15.7:0/2863982261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7effc8009e50 tx=0x7effc800b330 comp rx=0 tx=0).stop 2023-06-09T11:38:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 -- 172.21.15.7:0/2863982261 shutdown_connections 2023-06-09T11:38:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 --2- 172.21.15.7:0/2863982261 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd0158870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 --2- 172.21.15.7:0/2863982261 >> v2:172.21.15.117:3300/0 conn(0x7effd00c5b40 0x7effd0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.661+0000 7effd6dff700 1 -- 172.21.15.7:0/2863982261 >> 172.21.15.7:0/2863982261 conn(0x7effd006a4d0 msgr2=0x7effd006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.662+0000 7effd6dff700 1 -- 172.21.15.7:0/2863982261 shutdown_connections 2023-06-09T11:38:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.662+0000 7effd6dff700 1 -- 172.21.15.7:0/2863982261 wait complete. 2023-06-09T11:38:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.663+0000 7effd6dff700 1 Processor -- start 2023-06-09T11:38:29.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.664+0000 7effd6dff700 1 -- start start 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.664+0000 7effd6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd6dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c5b40 0x7effd00c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd6dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd6dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd00c8480 con 0x7effd0151510 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd6dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd01f0530 con 0x7effd00c5b40 2023-06-09T11:38:29.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd00c6b80 con 0x7effd0071060 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52676/0 (socket says 172.21.15.7:52676) 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54870/0 (socket says 172.21.15.7:54870) 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd539c700 1 -- 172.21.15.7:0/3853718441 learned_addr learned my addr 172.21.15.7:0/3853718441 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effcffff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c5b40 0x7effd00c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 -- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c5b40 msgr2=0x7effd00c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c5b40 0x7effd00c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 -- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 msgr2=0x7effd00c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd4b9b700 1 -- 172.21.15.7:0/3853718441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8009b30 con 0x7effd0071060 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.665+0000 7effd539c700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effd4b9b700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7effc400c3f0 tx=0x7effc400c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4004a70 con 0x7effd0071060 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4004bd0 con 0x7effd0071060 2023-06-09T11:38:29.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc40070d0 con 0x7effd0071060 2023-06-09T11:38:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd0157770 con 0x7effd0071060 2023-06-09T11:38:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.666+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd0157490 con 0x7effd0071060 2023-06-09T11:38:29.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.667+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd0157a50 con 0x7effd0071060 2023-06-09T11:38:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.668+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7effc4007230 con 0x7effd0071060 2023-06-09T11:38:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.669+0000 7effcd7fa700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.669+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7effc40891c0 con 0x7effd0071060 2023-06-09T11:38:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.670+0000 7effcffff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:29.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.670+0000 7effcffff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 0x7effb406d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7effd00c2230 tx=0x7effbc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:29.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.675+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7effc4053bf0 con 0x7effd0071060 2023-06-09T11:38:29.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.930+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effd0074cf0 con 0x7effd0071060 2023-06-09T11:38:29.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.932+0000 7effcd7fa700 1 -- 172.21.15.7:0/3853718441 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+5469 (secure 0 0 0) 0x7effc4057210 con 0x7effd0071060 2023-06-09T11:38:29.936 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:29.937 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:29.716939+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:29.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.933+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 msgr2=0x7effb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.933+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 0x7effb406d190 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7effd00c2230 tx=0x7effbc009210 comp rx=0 tx=0).stop 2023-06-09T11:38:29.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.933+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 msgr2=0x7effd01efff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.933+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7effc400c3f0 tx=0x7effc400c700 comp rx=0 tx=0).stop 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 shutdown_connections 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7effd0151510 0x7effd00c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7effb406ace0 0x7effb406d190 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7effd0071060 0x7effd01efff0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 --2- 172.21.15.7:0/3853718441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7effd00c5b40 0x7effd00c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:29.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 >> 172.21.15.7:0/3853718441 conn(0x7effd006a4d0 msgr2=0x7effd01e5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 shutdown_connections 2023-06-09T11:38:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:29.934+0000 7effd6dff700 1 -- 172.21.15.7:0/3853718441 wait complete. 2023-06-09T11:38:29.941 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 127 2023-06-09T11:38:29.957 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 127, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 127, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5852": { "addr": "172.21.15.117:6835/1343666954", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1343666954, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1343666954, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5852, "incarnation": 116, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:reconnect", "state_seq": 27 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:38:29.716939+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 5852, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:38:31.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.164+0000 7f62787ff700 1 Processor -- start 2023-06-09T11:38:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.165+0000 7f62787ff700 1 -- start start 2023-06-09T11:38:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f62787ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62700cc5b0 0x7f62700b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f62787ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6270159b20 con 0x7f62700cb9a0 2023-06-09T11:38:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f62787ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62700cac80 con 0x7f62700cb5d0 2023-06-09T11:38:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f62787ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62701597d0 con 0x7f62700cc5b0 2023-06-09T11:38:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f6275d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62700cb9a0 0x7f62700cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42276/0 (socket says 172.21.15.7:42276) 2023-06-09T11:38:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f6275d9a700 1 -- 172.21.15.7:0/1587033324 learned_addr learned my addr 172.21.15.7:0/1587033324 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:32.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.166+0000 7f6276d9c700 1 --2- 172.21.15.7:0/1587033324 >> v2:172.21.15.117:3300/0 conn(0x7f62700cc5b0 0x7f62700b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644495603 0 0) 0x7f6270159b20 con 0x7f62700cb9a0 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f625c0035b0 con 0x7f62700cb9a0 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329228920 0 0) 0x7f62700cac80 con 0x7f62700cb5d0 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6270159b20 con 0x7f62700cb5d0 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6276d9c700 1 -- 172.21.15.7:0/1587033324 >> v1:172.21.15.117:6789/0 conn(0x7f62700cb5d0 legacy=0x7f6270159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6276d9c700 1 -- 172.21.15.7:0/1587033324 >> v1:172.21.15.7:6789/0 conn(0x7f62700cb9a0 legacy=0x7f62700cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6276d9c700 1 -- 172.21.15.7:0/1587033324 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270071ab0 con 0x7f62700cc5b0 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6276d9c700 1 --2- 172.21.15.7:0/1587033324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cc5b0 0x7f62700b52b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6268009d90 tx=0x7f626800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=120c6e8e942b34db server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.167+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626800e070 con 0x7f62700cc5b0 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626800bb20 con 0x7f62700cc5b0 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f6275599700 1 -- 172.21.15.7:0/1587033324 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268012430 con 0x7f62700cc5b0 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 -- 172.21.15.7:0/1587033324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cc5b0 msgr2=0x7f62700b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 --2- 172.21.15.7:0/1587033324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cc5b0 0x7f62700b52b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6268009d90 tx=0x7f626800b330 comp rx=0 tx=0).stop 2023-06-09T11:38:32.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 -- 172.21.15.7:0/1587033324 shutdown_connections 2023-06-09T11:38:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 --2- 172.21.15.7:0/1587033324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cc5b0 0x7f62700b52b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 -- 172.21.15.7:0/1587033324 >> 172.21.15.7:0/1587033324 conn(0x7f6270070af0 msgr2=0x7f6270070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 -- 172.21.15.7:0/1587033324 shutdown_connections 2023-06-09T11:38:32.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.168+0000 7f62787ff700 1 -- 172.21.15.7:0/1587033324 wait complete. 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.170+0000 7f62787ff700 1 Processor -- start 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.170+0000 7f62787ff700 1 -- start start 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62700cb5d0 0x7f62701f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cb9a0 0x7f62700cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62701f2b80 con 0x7f62700cc5b0 2023-06-09T11:38:32.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62700ce2d0 con 0x7f62700cb9a0 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f62787ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62701f28a0 con 0x7f62700cb5d0 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f6276d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f6276d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52678/0 (socket says 172.21.15.7:52678) 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f6276d9c700 1 -- 172.21.15.7:0/3396551685 learned_addr learned my addr 172.21.15.7:0/3396551685 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f627659b700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62700cb5d0 0x7f62701f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.171+0000 7f6275d9a700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cb9a0 0x7f62700cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:32.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 -- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62700cb5d0 msgr2=0x7f62701f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62700cb5d0 0x7f62701f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 -- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cb9a0 msgr2=0x7f62700cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cb9a0 0x7f62700cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 -- 172.21.15.7:0/3396551685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268009a70 con 0x7f62700cc5b0 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6276d9c700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6270067180 tx=0x7f6268004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268012430 con 0x7f62700cc5b0 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268020070 con 0x7f62700cc5b0 2023-06-09T11:38:32.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626801ba50 con 0x7f62700cc5b0 2023-06-09T11:38:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.172+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627015d4f0 con 0x7f62700cc5b0 2023-06-09T11:38:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.173+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627015d210 con 0x7f62700cc5b0 2023-06-09T11:38:32.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.174+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627015d7d0 con 0x7f62700cc5b0 2023-06-09T11:38:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.174+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f626801bbb0 con 0x7f62700cc5b0 2023-06-09T11:38:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.175+0000 7f6266ffd700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 0x7f625406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.176+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f6268022070 con 0x7f62700cc5b0 2023-06-09T11:38:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.176+0000 7f627659b700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 0x7f625406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:32.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.176+0000 7f627659b700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 0x7f625406d1c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f626c0011e0 tx=0x7f626c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:32.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.180+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6268056470 con 0x7f62700cc5b0 2023-06-09T11:38:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.432+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62700cd900 con 0x7f62700cc5b0 2023-06-09T11:38:32.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.433+0000 7f6266ffd700 1 -- 172.21.15.7:0/3396551685 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+5469 (secure 0 0 0) 0x7f62700cd900 con 0x7f62700cc5b0 2023-06-09T11:38:32.438 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:32.438 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:29.716939+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 msgr2=0x7f625406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 0x7f625406d1c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f626c0011e0 tx=0x7f626c009040 comp rx=0 tx=0).stop 2023-06-09T11:38:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 msgr2=0x7f62700cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f6270067180 tx=0x7f6268004ad0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 shutdown_connections 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62700cc5b0 0x7f62700cf760 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f625406ad10 0x7f625406d1c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62700cb5d0 0x7f62701f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 --2- 172.21.15.7:0/3396551685 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62700cb9a0 0x7f62700cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.435+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 >> 172.21.15.7:0/3396551685 conn(0x7f6270070af0 msgr2=0x7f62700b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.436+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 shutdown_connections 2023-06-09T11:38:32.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:32.436+0000 7f62787ff700 1 -- 172.21.15.7:0/3396551685 wait complete. 2023-06-09T11:38:32.442 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 127 2023-06-09T11:38:32.457 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:34.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.664+0000 7f2b821ff700 1 Processor -- start 2023-06-09T11:38:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.665+0000 7f2b821ff700 1 -- start start 2023-06-09T11:38:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b821ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b821ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2b7c15cd40 0x7f2b7c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b821ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c159b20 con 0x7f2b7c0cb9a0 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b821ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c0cac80 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b821ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c1597d0 con 0x7f2b7c15cd40 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b7b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b7b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43334/0 (socket says 172.21.15.7:43334) 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b7affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2b7c0cb9a0 0x7f2b7c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34118/0 (socket says 172.21.15.7:34118) 2023-06-09T11:38:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.666+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/3608323143 learned_addr learned my addr 172.21.15.7:0/3608323143 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7bfff700 1 --2- 172.21.15.7:0/3608323143 >> v2:172.21.15.117:3300/0 conn(0x7f2b7c15cd40 0x7f2b7c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/3608323143 >> v2:172.21.15.117:3300/0 conn(0x7f2b7c15cd40 msgr2=0x7f2b7c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/3608323143 >> v2:172.21.15.117:3300/0 conn(0x7f2b7c15cd40 0x7f2b7c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/3608323143 >> v1:172.21.15.7:6789/0 conn(0x7f2b7c0cb9a0 legacy=0x7f2b7c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/3608323143 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c071ab0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7a7fc700 1 -- 172.21.15.7:0/3608323143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2218326160 0 0) 0x7f2b7c159b20 con 0x7f2b7c0cb9a0 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/3608323143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0c9b80 tx=0x7f2b6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551ba3033e694769 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7a7fc700 1 -- 172.21.15.7:0/3608323143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00b870 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7a7fc700 1 -- 172.21.15.7:0/3608323143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c00b9d0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.667+0000 7f2b7a7fc700 1 -- 172.21.15.7:0/3608323143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c00bd60 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 -- 172.21.15.7:0/3608323143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 msgr2=0x7f2b7c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 --2- 172.21.15.7:0/3608323143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0c9b80 tx=0x7f2b6c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 -- 172.21.15.7:0/3608323143 shutdown_connections 2023-06-09T11:38:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 --2- 172.21.15.7:0/3608323143 >> v2:172.21.15.117:3300/0 conn(0x7f2b7c15cd40 0x7f2b7c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 --2- 172.21.15.7:0/3608323143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c0b52b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 -- 172.21.15.7:0/3608323143 >> 172.21.15.7:0/3608323143 conn(0x7f2b7c070af0 msgr2=0x7f2b7c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 -- 172.21.15.7:0/3608323143 shutdown_connections 2023-06-09T11:38:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.668+0000 7f2b821ff700 1 -- 172.21.15.7:0/3608323143 wait complete. 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.669+0000 7f2b821ff700 1 Processor -- start 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.670+0000 7f2b821ff700 1 -- start start 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b7c0cb9a0 0x7f2b7c0c3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b7c15cd40 0x7f2b7c0c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0b1c20 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c0c45f0 con 0x7f2b7c0cb9a0 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b821ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c1ef800 con 0x7f2b7c15cd40 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43338/0 (socket says 172.21.15.7:43338) 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/2967592938 learned_addr learned my addr 172.21.15.7:0/2967592938 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7affd700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b7c0cb9a0 0x7f2b7c0c3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7bfff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b7c15cd40 0x7f2b7c0c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b7c15cd40 msgr2=0x7f2b7c0c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b7c15cd40 0x7f2b7c0c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b7c0cb9a0 msgr2=0x7f2b7c0c3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b7c0cb9a0 0x7f2b7c0c3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.671+0000 7f2b7b7fe700 1 -- 172.21.15.7:0/2967592938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c009550 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b7b7fe700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c067180 tx=0x7f2b6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004250 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b6c01f070 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c0045b0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c15d4f0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.672+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c15d210 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.673+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c15d7d0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.674+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b6c021030 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.675+0000 7f2b63fff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 0x7f2b6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.675+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f2b6c08ab50 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.675+0000 7f2b7affd700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 0x7f2b6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.676+0000 7f2b7affd700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 0x7f2b6806d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0b39c0 tx=0x7f2b70007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.680+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2b6c0557d0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.930+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b7c0c48d0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.931+0000 7f2b63fff700 1 -- 172.21.15.7:0/2967592938 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 127 v127) v1 ==== 95+0+5469 (secure 0 0 0) 0x7f2b6c058df0 con 0x7f2b7c0cb5d0 2023-06-09T11:38:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:34.935 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":127,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":127,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:29.716939+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 msgr2=0x7f2b6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 0x7f2b6806d190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c0b39c0 tx=0x7f2b70007de0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 msgr2=0x7f2b7c1ef2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c067180 tx=0x7f2b6c004750 comp rx=0 tx=0).stop 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 shutdown_connections 2023-06-09T11:38:34.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2b6806ace0 0x7f2b6806d190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2b7c15cd40 0x7f2b7c0c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2b7c0cb5d0 0x7f2b7c1ef2c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 --2- 172.21.15.7:0/2967592938 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2b7c0cb9a0 0x7f2b7c0c3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 >> 172.21.15.7:0/2967592938 conn(0x7f2b7c070af0 msgr2=0x7f2b7c158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 shutdown_connections 2023-06-09T11:38:34.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:34.933+0000 7f2b821ff700 1 -- 172.21.15.7:0/2967592938 wait complete. 2023-06-09T11:38:34.940 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 127 2023-06-09T11:38:34.955 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:36.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.164+0000 7fcb5edff700 1 Processor -- start 2023-06-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.164+0000 7fcb5edff700 1 -- start start 2023-06-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.165+0000 7fcb5edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb58158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5edff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb581580a0 con 0x7fcb58151510 2023-06-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5edff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb580c6180 con 0x7fcb580c5b40 2023-06-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581518e0 con 0x7fcb58071230 2023-06-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5cb9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fcb58151510 0x7fcb5815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34132/0 (socket says 172.21.15.7:34132) 2023-06-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/2953641233 learned_addr learned my addr 172.21.15.7:0/2953641233 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb5d39c700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb58158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363544218 0 0) 0x7fcb581580a0 con 0x7fcb58151510 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb480035b0 con 0x7fcb58151510 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006810844 0 0) 0x7fcb580c6180 con 0x7fcb580c5b40 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb581580a0 con 0x7fcb580c5b40 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.166+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1451116866 0 0) 0x7fcb480035b0 con 0x7fcb58151510 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 msgr2=0x7fcb58158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb58158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 >> v1:172.21.15.117:6789/0 conn(0x7fcb580c5b40 legacy=0x7fcb58070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58157490 con 0x7fcb58151510 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fcb4c001670 con 0x7fcb58151510 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 >> v1:172.21.15.7:6789/0 conn(0x7fcb58151510 legacy=0x7fcb5815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb48003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48004d90 0x7fcb48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581518e0 con 0x7fcb48003bc0 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800bac0 con 0x7fcb48004d90 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4800bc30 con 0x7fcb58071230 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb48003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5d39c700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48004d90 0x7fcb48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb57fff700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 msgr2=0x7fcb48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb58071230 0x7fcb48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48004d90 msgr2=0x7fcb48005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb48004d90 0x7fcb48005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.167+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/2953641233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480035b0 con 0x7fcb48003bc0 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb48003fb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c210 tx=0x7fcb48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63a0bae00c4c61e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c00cd50 con 0x7fcb48003bc0 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c001af0 con 0x7fcb48003bc0 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb577fe700 1 -- 172.21.15.7:0/2953641233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c005ba0 con 0x7fcb48003bc0 2023-06-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb5edff700 1 -- 172.21.15.7:0/2953641233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 msgr2=0x7fcb48003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb5edff700 1 --2- 172.21.15.7:0/2953641233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb48003fb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c00c210 tx=0x7fcb48005880 comp rx=0 tx=0).stop 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb5d39c700 1 -- 172.21.15.7:0/2953641233 reap_dead start 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.168+0000 7fcb5edff700 1 -- 172.21.15.7:0/2953641233 shutdown_connections 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.169+0000 7fcb5edff700 1 -- 172.21.15.7:0/2953641233 >> 172.21.15.7:0/2953641233 conn(0x7fcb5806a4d0 msgr2=0x7fcb5806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.169+0000 7fcb5edff700 1 -- 172.21.15.7:0/2953641233 shutdown_connections 2023-06-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.169+0000 7fcb5edff700 1 -- 172.21.15.7:0/2953641233 wait complete. 2023-06-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.170+0000 7fcb5edff700 1 Processor -- start 2023-06-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.170+0000 7fcb5edff700 1 -- start start 2023-06-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 0x7fcb5814ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb581f2960 0x7fcb5814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb580c6900 con 0x7fcb48003bc0 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581ef8a0 con 0x7fcb581ef3f0 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.171+0000 7fcb5edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb581efb80 con 0x7fcb581f2960 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43366/0 (socket says 172.21.15.7:43366) 2023-06-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/1045994023 learned_addr learned my addr 172.21.15.7:0/1045994023 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb57fff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 0x7fcb5814ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5d39c700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb581f2960 0x7fcb5814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb581f2960 msgr2=0x7fcb5814b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb581f2960 0x7fcb5814b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 msgr2=0x7fcb5814ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 0x7fcb5814ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 -- 172.21.15.7:0/1045994023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58157a50 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb57fff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 0x7fcb5814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5cb9b700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c000c00 tx=0x7fcb4c001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0072d0 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c007430 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c0077c0 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.172+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb58157770 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.173+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb58157d30 con 0x7fcb48003bc0 2023-06-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.174+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb58074cf0 con 0x7fcb48003bc0 2023-06-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.174+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb4c01e430 con 0x7fcb48003bc0 2023-06-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.176+0000 7fcb557fa700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 0x7fcb3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.176+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fcb4c08a9e0 con 0x7fcb48003bc0 2023-06-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.176+0000 7fcb57fff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 0x7fcb3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.177+0000 7fcb57fff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 0x7fcb3c06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb58156b50 tx=0x7fcb44009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.180+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcb4c055650 con 0x7fcb48003bc0 2023-06-09T11:38:37.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.434+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb580b10b0 con 0x7fcb48003bc0 2023-06-09T11:38:37.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.435+0000 7fcb557fa700 1 -- 172.21.15.7:0/1045994023 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fcb4c058c70 con 0x7fcb48003bc0 2023-06-09T11:38:37.439 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:37.440 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":128,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:35.494553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.437+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 msgr2=0x7fcb3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.437+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 0x7fcb3c06d190 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb58156b50 tx=0x7fcb44009040 comp rx=0 tx=0).stop 2023-06-09T11:38:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.437+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 msgr2=0x7fcb580ca800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.437+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c000c00 tx=0x7fcb4c001980 comp rx=0 tx=0).stop 2023-06-09T11:38:37.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 shutdown_connections 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcb3c06ace0 0x7fcb3c06d190 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcb581f2960 0x7fcb5814b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb48003bc0 0x7fcb580ca800 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 --2- 172.21.15.7:0/1045994023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcb581ef3f0 0x7fcb5814ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 >> 172.21.15.7:0/1045994023 conn(0x7fcb5806a4d0 msgr2=0x7fcb580700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 shutdown_connections 2023-06-09T11:38:37.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:37.438+0000 7fcb5edff700 1 -- 172.21.15.7:0/1045994023 wait complete. 2023-06-09T11:38:37.444 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 128 2023-06-09T11:38:37.460 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:39.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.669+0000 7f22bb3ff700 1 Processor -- start 2023-06-09T11:38:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.670+0000 7f22bb3ff700 1 -- start start 2023-06-09T11:38:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22bb3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 0x7f22b415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22bb3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b415dcf0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22bb3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b415d450 con 0x7f22b40b5360 2023-06-09T11:38:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22bb3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b415d730 con 0x7f22b406b480 2023-06-09T11:38:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22b919b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f22b40ca1b0 0x7f22b41536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34144/0 (socket says 172.21.15.7:34144) 2023-06-09T11:38:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22b919b700 1 -- 172.21.15.7:0/933287133 learned_addr learned my addr 172.21.15.7:0/933287133 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:38:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22b999c700 1 --2- 172.21.15.7:0/933287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 0x7f22b415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074130913 0 0) 0x7f22b415dcf0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.671+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a00035b0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269991667 0 0) 0x7f22b415d450 con 0x7f22b40b5360 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22b415dcf0 con 0x7f22b40b5360 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632583143 0 0) 0x7f22a00035b0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22b999c700 1 -- 172.21.15.7:0/933287133 >> v1:172.21.15.117:6789/0 conn(0x7f22b40b5360 legacy=0x7f22b406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22b999c700 1 -- 172.21.15.7:0/933287133 >> v1:172.21.15.7:6789/0 conn(0x7f22b40ca1b0 legacy=0x7f22b41536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22b999c700 1 -- 172.21.15.7:0/933287133 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b415da10 con 0x7f22b406b480 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22b999c700 1 --2- 172.21.15.7:0/933287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 0x7f22b415cdc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009dc0 tx=0x7f22ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff5174607c1de41 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00e070 con 0x7f22b406b480 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac003a20 con 0x7f22b406b480 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.672+0000 7f22abfff700 1 -- 172.21.15.7:0/933287133 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0001670 con 0x7f22b406b480 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 -- 172.21.15.7:0/933287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 msgr2=0x7f22b415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/933287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 0x7f22b415cdc0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009dc0 tx=0x7f22ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 -- 172.21.15.7:0/933287133 shutdown_connections 2023-06-09T11:38:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/933287133 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b406b480 0x7f22b415cdc0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 -- 172.21.15.7:0/933287133 >> 172.21.15.7:0/933287133 conn(0x7f22b4072090 msgr2=0x7f22b4072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 -- 172.21.15.7:0/933287133 shutdown_connections 2023-06-09T11:38:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.673+0000 7f22bb3ff700 1 -- 172.21.15.7:0/933287133 wait complete. 2023-06-09T11:38:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.675+0000 7f22bb3ff700 1 Processor -- start 2023-06-09T11:38:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.676+0000 7f22bb3ff700 1 -- start start 2023-06-09T11:38:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 0x7f22b415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b40b5360 0x7f22b415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b41f02a0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b41efd40 con 0x7f22b406b480 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22bb3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b415be70 con 0x7f22b40b5360 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22b999c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22b999c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43376/0 (socket says 172.21.15.7:43376) 2023-06-09T11:38:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.677+0000 7f22b999c700 1 -- 172.21.15.7:0/1115156657 learned_addr learned my addr 172.21.15.7:0/1115156657 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b919b700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 0x7f22b415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b899a700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b40b5360 0x7f22b415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 -- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b40b5360 msgr2=0x7f22b415ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b40b5360 0x7f22b415ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 -- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 msgr2=0x7f22b415c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 0x7f22b415c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 -- 172.21.15.7:0/1115156657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ac009a70 con 0x7f22b40ca1b0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b919b700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 0x7f22b415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22b999c700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009d90 tx=0x7f22ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac0046f0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac004850 con 0x7f22b40ca1b0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b406a850 con 0x7f22b40ca1b0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac01d670 con 0x7f22b40ca1b0 2023-06-09T11:38:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.678+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b406a570 con 0x7f22b40ca1b0 2023-06-09T11:38:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.679+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b4153e00 con 0x7f22b40ca1b0 2023-06-09T11:38:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.680+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22ac01d7d0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.681+0000 7f22a9ffb700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 0x7f229806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.681+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f22ac08b0d0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.682+0000 7f22b919b700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 0x7f229806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:39.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.682+0000 7f22b919b700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 0x7f229806d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f22b0001380 tx=0x7f22b0007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.686+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f22ac055b00 con 0x7f22b40ca1b0 2023-06-09T11:38:39.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.936+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22b4074cf0 con 0x7f22b40ca1b0 2023-06-09T11:38:39.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.936+0000 7f22a9ffb700 1 -- 172.21.15.7:0/1115156657 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f22ac016090 con 0x7f22b40ca1b0 2023-06-09T11:38:39.941 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:39.941 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":128,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:35.494553+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 msgr2=0x7f229806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 0x7f229806d190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f22b0001380 tx=0x7f22b0007aa0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 msgr2=0x7f22b41e66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f22ac009d90 tx=0x7f22ac004ad0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 shutdown_connections 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22b40ca1b0 0x7f22b41e66e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22b406b480 0x7f22b415c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f229806ace0 0x7f229806d190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 --2- 172.21.15.7:0/1115156657 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f22b40b5360 0x7f22b415ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:39.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 >> 172.21.15.7:0/1115156657 conn(0x7f22b4072090 msgr2=0x7f22b40cb360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 shutdown_connections 2023-06-09T11:38:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:39.939+0000 7f22bb3ff700 1 -- 172.21.15.7:0/1115156657 wait complete. 2023-06-09T11:38:39.945 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 128 2023-06-09T11:38:39.961 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:41.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:42.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.169+0000 7fe3e85ff700 1 Processor -- start 2023-06-09T11:38:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.169+0000 7fe3e85ff700 1 -- start start 2023-06-09T11:38:42.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.170+0000 7fe3e85ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 0x7fe3e00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.170+0000 7fe3e85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.170+0000 7fe3e85ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e01580a0 con 0x7fe3e015b380 2023-06-09T11:38:42.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.170+0000 7fe3e85ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e01518e0 con 0x7fe3e0151510 2023-06-09T11:38:42.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.170+0000 7fe3e85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00c6180 con 0x7fe3e0071230 2023-06-09T11:38:42.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46456/0 (socket says 172.21.15.7:46456) 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 -- 172.21.15.7:0/1214634229 learned_addr learned my addr 172.21.15.7:0/1214634229 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e639b700 1 --2- 172.21.15.7:0/1214634229 >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 0x7fe3e00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 -- 172.21.15.7:0/1214634229 >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 msgr2=0x7fe3e00c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 --2- 172.21.15.7:0/1214634229 >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 0x7fe3e00c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 -- 172.21.15.7:0/1214634229 >> v1:172.21.15.117:6789/0 conn(0x7fe3e015b380 legacy=0x7fe3e0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e6b9c700 1 -- 172.21.15.7:0/1214634229 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e0157490 con 0x7fe3e0071230 2023-06-09T11:38:42.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.171+0000 7fe3e639b700 1 --2- 172.21.15.7:0/1214634229 >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 0x7fe3e00c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e6b9c700 1 --2- 172.21.15.7:0/1214634229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009e50 tx=0x7fe3d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7142a621d8492840 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e5399700 1 -- 172.21.15.7:0/1214634229 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800e070 con 0x7fe3e0071230 2023-06-09T11:38:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e5399700 1 -- 172.21.15.7:0/1214634229 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800b910 con 0x7fe3e0071230 2023-06-09T11:38:42.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e5399700 1 -- 172.21.15.7:0/1214634229 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800bcc0 con 0x7fe3e0071230 2023-06-09T11:38:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e85ff700 1 -- 172.21.15.7:0/1214634229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 msgr2=0x7fe3e0158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.172+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/1214634229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009e50 tx=0x7fe3d800b330 comp rx=0 tx=0).stop 2023-06-09T11:38:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 -- 172.21.15.7:0/1214634229 shutdown_connections 2023-06-09T11:38:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/1214634229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e0071230 0x7fe3e0158870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/1214634229 >> v2:172.21.15.117:3300/0 conn(0x7fe3e0151510 0x7fe3e00c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 -- 172.21.15.7:0/1214634229 >> 172.21.15.7:0/1214634229 conn(0x7fe3e006a4d0 msgr2=0x7fe3e006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 -- 172.21.15.7:0/1214634229 shutdown_connections 2023-06-09T11:38:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.173+0000 7fe3e85ff700 1 -- 172.21.15.7:0/1214634229 wait complete. 2023-06-09T11:38:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.174+0000 7fe3e85ff700 1 Processor -- start 2023-06-09T11:38:42.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.175+0000 7fe3e85ff700 1 -- start start 2023-06-09T11:38:42.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0151510 0x7fe3e00cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e015b380 0x7fe3e00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00c8fa0 con 0x7fe3e0071230 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01eda70 con 0x7fe3e0151510 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e01edd10 con 0x7fe3e015b380 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43380/0 (socket says 172.21.15.7:43380) 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 -- 172.21.15.7:0/803558890 learned_addr learned my addr 172.21.15.7:0/803558890 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e5b9a700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0151510 0x7fe3e00cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e6b9c700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e015b380 0x7fe3e00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 -- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e015b380 msgr2=0x7fe3e00cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e015b380 0x7fe3e00cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 -- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0151510 msgr2=0x7fe3e00cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0151510 0x7fe3e00cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.176+0000 7fe3e639b700 1 -- 172.21.15.7:0/803558890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0094a0 con 0x7fe3e0071230 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3e639b700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00ebe0 tx=0x7fe3dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00c850 con 0x7fe3e0071230 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc00c9b0 con 0x7fe3e0071230 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc00cd40 con 0x7fe3e0071230 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d8009b30 con 0x7fe3e0071230 2023-06-09T11:38:42.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.177+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e0157a50 con 0x7fe3e0071230 2023-06-09T11:38:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.178+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e0157770 con 0x7fe3e0071230 2023-06-09T11:38:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.179+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3dc013470 con 0x7fe3e0071230 2023-06-09T11:38:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.180+0000 7fe3d6ffd700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:42.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.180+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fe3dc089a70 con 0x7fe3e0071230 2023-06-09T11:38:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.180+0000 7fe3e5b9a700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:42.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.181+0000 7fe3e5b9a700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 0x7fe3c406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3e01eab80 tx=0x7fe3d0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:42.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.185+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe3dc0545c0 con 0x7fe3e0071230 2023-06-09T11:38:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.438+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3e0157d30 con 0x7fe3e0071230 2023-06-09T11:38:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.439+0000 7fe3d6ffd700 1 -- 172.21.15.7:0/803558890 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+5472 (secure 0 0 0) 0x7fe3dc00cea0 con 0x7fe3e0071230 2023-06-09T11:38:42.443 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:42.444 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:40.652295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:clientreplay","state_seq":31,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 msgr2=0x7fe3c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 0x7fe3c406d190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe3e01eab80 tx=0x7fe3d0009040 comp rx=0 tx=0).stop 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 msgr2=0x7fe3e0154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00ebe0 tx=0x7fe3dc00eef0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 shutdown_connections 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe3c406ace0 0x7fe3c406d190 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3e015b380 0x7fe3e00cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3e0071230 0x7fe3e0154700 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 --2- 172.21.15.7:0/803558890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe3e0151510 0x7fe3e00cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 >> 172.21.15.7:0/803558890 conn(0x7fe3e006a4d0 msgr2=0x7fe3e00c6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 shutdown_connections 2023-06-09T11:38:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:42.442+0000 7fe3e85ff700 1 -- 172.21.15.7:0/803558890 wait complete. 2023-06-09T11:38:42.448 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 129 2023-06-09T11:38:42.465 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 129, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 129, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5852": { "addr": "172.21.15.117:6835/1343666954", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 1343666954, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 1343666954, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 5852, "incarnation": 116, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 24 }, "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:clientreplay", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 64, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:38:40.652295+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 5852, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:38:44.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.674+0000 7fa1723ff700 1 Processor -- start 2023-06-09T11:38:44.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.674+0000 7fa1723ff700 1 -- start start 2023-06-09T11:38:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa1723ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa1723ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa1723ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa16c159b20 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa1723ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16c0cac80 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa1723ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1597d0 con 0x7fa16c15cd40 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa16bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa16bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47578/0 (socket says 172.21.15.7:47578) 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa16bfff700 1 -- 172.21.15.7:0/74730881 learned_addr learned my addr 172.21.15.7:0/74730881 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa16b7fe700 1 --1- 172.21.15.7:0/74730881 >> v1:172.21.15.7:6789/0 conn(0x7fa16c0cb9a0 0x7fa16c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58704/0 (socket says 172.21.15.7:58704) 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.675+0000 7fa17099c700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16bfff700 1 -- 172.21.15.7:0/74730881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 msgr2=0x7fa16c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16bfff700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16bfff700 1 -- 172.21.15.7:0/74730881 >> v1:172.21.15.7:6789/0 conn(0x7fa16c0cb9a0 legacy=0x7fa16c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16bfff700 1 -- 172.21.15.7:0/74730881 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c071ab0 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa17099c700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16bfff700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0098f0 tx=0x7fa15c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f435e89a30fbaf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16affd700 1 -- 172.21.15.7:0/74730881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00b8a0 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16affd700 1 -- 172.21.15.7:0/74730881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c00ba00 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.676+0000 7fa16affd700 1 -- 172.21.15.7:0/74730881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c00bd90 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 -- 172.21.15.7:0/74730881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb5d0 msgr2=0x7fa16c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0098f0 tx=0x7fa15c009c00 comp rx=0 tx=0).stop 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 -- 172.21.15.7:0/74730881 shutdown_connections 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 --2- 172.21.15.7:0/74730881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c0b52b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 -- 172.21.15.7:0/74730881 >> 172.21.15.7:0/74730881 conn(0x7fa16c070af0 msgr2=0x7fa16c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 -- 172.21.15.7:0/74730881 shutdown_connections 2023-06-09T11:38:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.677+0000 7fa1723ff700 1 -- 172.21.15.7:0/74730881 wait complete. 2023-06-09T11:38:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.678+0000 7fa1723ff700 1 Processor -- start 2023-06-09T11:38:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.679+0000 7fa1723ff700 1 -- start start 2023-06-09T11:38:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.679+0000 7fa1723ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c1f25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.679+0000 7fa1723ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa1723ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa1723ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0d05f0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa1723ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c1f2f10 con 0x7fa16c0cb5d0 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa1723ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16c0c3770 con 0x7fa16c15cd40 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47586/0 (socket says 172.21.15.7:47586) 2023-06-09T11:38:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 -- 172.21.15.7:0/3565506541 learned_addr learned my addr 172.21.15.7:0/3565506541 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16bfff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c1f25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa17099c700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 -- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 msgr2=0x7fa16c0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 -- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa16c0cb5d0 msgr2=0x7fa16c1f25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c1f25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 -- 172.21.15.7:0/3565506541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009550 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa17099c700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.680+0000 7fa16b7fe700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c4f0 tx=0x7fa16000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.681+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160005cb0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.681+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160005e10 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.681+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600072d0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.681+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16c15d4f0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.681+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16c15d210 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.682+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16c15d7d0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.683+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa160007430 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.684+0000 7fa168ff9700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 0x7fa15806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.684+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fa160088d90 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.684+0000 7fa16bfff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 0x7fa15806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.685+0000 7fa16bfff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 0x7fa15806d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009c40 tx=0x7fa15c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.689+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa160053900 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.940+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa16c0c35d0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.941+0000 7fa168ff9700 1 -- 172.21.15.7:0/3565506541 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 129 v129) v1 ==== 95+0+5472 (secure 0 0 0) 0x7fa16c0c35d0 con 0x7fa16c0cb9a0 2023-06-09T11:38:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:44.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":129,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":129,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:40.652295+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:clientreplay","state_seq":31,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 msgr2=0x7fa15806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 0x7fa15806d190 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009c40 tx=0x7fa15c008030 comp rx=0 tx=0).stop 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 msgr2=0x7fa16c0c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa16000c4f0 tx=0x7fa16000c800 comp rx=0 tx=0).stop 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 shutdown_connections 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa15806ace0 0x7fa15806d190 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa16c15cd40 0x7fa16c0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa16c0cb5d0 0x7fa16c1f25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 --2- 172.21.15.7:0/3565506541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa16c0cb9a0 0x7fa16c0c4860 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 >> 172.21.15.7:0/3565506541 conn(0x7fa16c070af0 msgr2=0x7fa16c0c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 shutdown_connections 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:44.943+0000 7fa1723ff700 1 -- 172.21.15.7:0/3565506541 wait complete. 2023-06-09T11:38:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 129 2023-06-09T11:38:44.966 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:38:46.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:47.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.174+0000 7fcc14fff700 1 Processor -- start 2023-06-09T11:38:47.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.175+0000 7fcc14fff700 1 -- start start 2023-06-09T11:38:47.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc14fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10151510 0x7fcc100c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc14fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1015b380 0x7fcc10070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc14fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc101580a0 con 0x7fcc10071060 2023-06-09T11:38:47.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc14fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc100714e0 con 0x7fcc1015b380 2023-06-09T11:38:47.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc14fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101518e0 con 0x7fcc10151510 2023-06-09T11:38:47.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1015b380 0x7fcc10070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc07fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcc1015b380 0x7fcc10070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47600/0 (socket says 172.21.15.7:47600) 2023-06-09T11:38:47.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc07fff700 1 -- 172.21.15.7:0/1102103011 learned_addr learned my addr 172.21.15.7:0/1102103011 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.176+0000 7fcc0e59c700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10151510 0x7fcc100c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc07fff700 1 -- 172.21.15.7:0/1102103011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10151510 msgr2=0x7fcc100c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc07fff700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10151510 0x7fcc100c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc07fff700 1 -- 172.21.15.7:0/1102103011 >> v1:172.21.15.117:6789/0 conn(0x7fcc10071060 legacy=0x7fcc10158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc07fff700 1 -- 172.21.15.7:0/1102103011 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100c6180 con 0x7fcc1015b380 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc0dd9b700 1 -- 172.21.15.7:0/1102103011 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2070511329 0 0) 0x7fcc101580a0 con 0x7fcc10071060 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc07fff700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015b380 0x7fcc10070b20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80098f0 tx=0x7fcbf8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec263bafd3a3851 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc0dd9b700 1 -- 172.21.15.7:0/1102103011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011830 con 0x7fcc1015b380 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc0dd9b700 1 -- 172.21.15.7:0/1102103011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8011990 con 0x7fcc1015b380 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.177+0000 7fcc0dd9b700 1 -- 172.21.15.7:0/1102103011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8011d20 con 0x7fcc1015b380 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 -- 172.21.15.7:0/1102103011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015b380 msgr2=0x7fcc10070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015b380 0x7fcc10070b20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80098f0 tx=0x7fcbf8009c00 comp rx=0 tx=0).stop 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 -- 172.21.15.7:0/1102103011 shutdown_connections 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10151510 0x7fcc100c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 --2- 172.21.15.7:0/1102103011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015b380 0x7fcc10070b20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 -- 172.21.15.7:0/1102103011 >> 172.21.15.7:0/1102103011 conn(0x7fcc1006a4d0 msgr2=0x7fcc1006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 -- 172.21.15.7:0/1102103011 shutdown_connections 2023-06-09T11:38:47.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.178+0000 7fcc14fff700 1 -- 172.21.15.7:0/1102103011 wait complete. 2023-06-09T11:38:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.179+0000 7fcc14fff700 1 Processor -- start 2023-06-09T11:38:47.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.180+0000 7fcc14fff700 1 -- start start 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.180+0000 7fcc14fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.180+0000 7fcc14fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc14fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1015b380 0x7fcc10145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc14fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10149780 con 0x7fcc10151510 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc14fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101488f0 con 0x7fcc1015b380 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc14fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc10148b70 con 0x7fcc10071060 2023-06-09T11:38:47.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40220/0 (socket says 172.21.15.7:40220) 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 -- 172.21.15.7:0/258429476 learned_addr learned my addr 172.21.15.7:0/258429476 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47604/0 (socket says 172.21.15.7:47604) 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0ed9d700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1015b380 0x7fcc10145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 -- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1015b380 msgr2=0x7fcc10145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1015b380 0x7fcc10145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 -- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 msgr2=0x7fcc101e0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc0e59c700 1 -- 172.21.15.7:0/258429476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8009550 con 0x7fcc10071060 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.181+0000 7fcc07fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc0e59c700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9b0 tx=0x7fcc0000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00009710 con 0x7fcc10071060 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000ee50 con 0x7fcc10071060 2023-06-09T11:38:47.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc10157770 con 0x7fcc10071060 2023-06-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000ca50 con 0x7fcc10071060 2023-06-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.182+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc10157490 con 0x7fcc10071060 2023-06-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.183+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10157a50 con 0x7fcc10071060 2023-06-09T11:38:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.184+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc0000cbb0 con 0x7fcc10071060 2023-06-09T11:38:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.185+0000 7fcc06ffd700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.185+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fcc000899d0 con 0x7fcc10071060 2023-06-09T11:38:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.185+0000 7fcc07fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.186+0000 7fcc07fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8014040 tx=0x7fcbf8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.190+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcc00057fb0 con 0x7fcc10071060 2023-06-09T11:38:47.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.444+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc10074cf0 con 0x7fcc10071060 2023-06-09T11:38:47.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.445+0000 7fcc06ffd700 1 -- 172.21.15.7:0/258429476 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fcc0000ce60 con 0x7fcc10071060 2023-06-09T11:38:47.449 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:47.449 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 msgr2=0x7fcbf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 0x7fcbf006d190 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8014040 tx=0x7fcbf8008030 comp rx=0 tx=0).stop 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 msgr2=0x7fcc101e06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000e9b0 tx=0x7fcc0000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 shutdown_connections 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcc1015b380 0x7fcc10145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcbf006ace0 0x7fcbf006d190 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcc10071060 0x7fcc101e06d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 --2- 172.21.15.7:0/258429476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc10151510 0x7fcc101e0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 >> 172.21.15.7:0/258429476 conn(0x7fcc1006a4d0 msgr2=0x7fcc1014cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 shutdown_connections 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.447+0000 7fcc14fff700 1 -- 172.21.15.7:0/258429476 wait complete. 2023-06-09T11:38:47.453 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 130 2023-06-09T11:38:47.470 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=0, replacing gid=5741 2023-06-09T11:38:47.470 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.b 2023-06-09T11:38:47.470 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:38:47.470 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:38:47.470 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:38:47.475 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:38:47.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:47.627 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:38:47.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.671+0000 7f3ded3ff700 1 Processor -- start 2023-06-09T11:38:47.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.671+0000 7f3ded3ff700 1 -- start start 2023-06-09T11:38:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3ded3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de8153880 0x7f3de815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3ded3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3de80c9f40 0x7f3de815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3ded3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3de815a500 con 0x7f3de815b2d0 2023-06-09T11:38:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3ded3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3de80c5f10 con 0x7f3de80c9f40 2023-06-09T11:38:47.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3ded3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de815da00 con 0x7f3de8153880 2023-06-09T11:38:47.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3de80c9f40 0x7f3de815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3de80c9f40 0x7f3de815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47614/0 (socket says 172.21.15.7:47614) 2023-06-09T11:38:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3de815b2d0 0x7f3de80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58716/0 (socket says 172.21.15.7:58716) 2023-06-09T11:38:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 -- 172.21.15.7:0/2335010954 learned_addr learned my addr 172.21.15.7:0/2335010954 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de6ffd700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de8153880 0x7f3de815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 -- 172.21.15.7:0/2335010954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de8153880 msgr2=0x7f3de815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de8153880 0x7f3de815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 -- 172.21.15.7:0/2335010954 >> v1:172.21.15.7:6789/0 conn(0x7f3de815b2d0 legacy=0x7f3de80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de67fc700 1 -- 172.21.15.7:0/2335010954 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de815b750 con 0x7f3de80c9f40 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.673+0000 7f3de5ffb700 1 -- 172.21.15.7:0/2335010954 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244287827 0 0) 0x7f3de0001490 con 0x7f3de815b2d0 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.674+0000 7f3de67fc700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de80c9f40 0x7f3de815ad90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0098f0 tx=0x7f3ddc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd36d4364098f7f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.674+0000 7f3de5ffb700 1 -- 172.21.15.7:0/2335010954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc011830 con 0x7f3de80c9f40 2023-06-09T11:38:47.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.674+0000 7f3de5ffb700 1 -- 172.21.15.7:0/2335010954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc011990 con 0x7f3de80c9f40 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.674+0000 7f3de5ffb700 1 -- 172.21.15.7:0/2335010954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc011d20 con 0x7f3de80c9f40 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 -- 172.21.15.7:0/2335010954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de80c9f40 msgr2=0x7f3de815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de80c9f40 0x7f3de815ad90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0098f0 tx=0x7f3ddc009c00 comp rx=0 tx=0).stop 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 -- 172.21.15.7:0/2335010954 shutdown_connections 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de8153880 0x7f3de815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/2335010954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de80c9f40 0x7f3de815ad90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 -- 172.21.15.7:0/2335010954 >> 172.21.15.7:0/2335010954 conn(0x7f3de80737f0 msgr2=0x7f3de80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:47.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 -- 172.21.15.7:0/2335010954 shutdown_connections 2023-06-09T11:38:47.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.675+0000 7f3ded3ff700 1 -- 172.21.15.7:0/2335010954 wait complete. 2023-06-09T11:38:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.676+0000 7f3ded3ff700 1 Processor -- start 2023-06-09T11:38:47.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 -- start start 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de8153880 0x7f3de80b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de815b2d0 0x7f3de806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de81518c0 con 0x7f3de815b2d0 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de8152f40 con 0x7f3de8153880 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.677+0000 7f3ded3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3de80cbbb0 con 0x7f3de80c9f40 2023-06-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40238/0 (socket says 172.21.15.7:40238) 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de815b2d0 0x7f3de806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 -- 172.21.15.7:0/1915602830 learned_addr learned my addr 172.21.15.7:0/1915602830 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de67fc700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de8153880 0x7f3de80b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 -- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de8153880 msgr2=0x7f3de80b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de8153880 0x7f3de80b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 -- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de815b2d0 msgr2=0x7f3de806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de815b2d0 0x7f3de806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 -- 172.21.15.7:0/1915602830 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc009550 con 0x7f3de80c9f40 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.678+0000 7f3de6ffd700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800e9e0 tx=0x7f3dd800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.679+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd8009710 con 0x7f3de80c9f40 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.679+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd800ee80 con 0x7f3de80c9f40 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.679+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd800ca50 con 0x7f3de80c9f40 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.679+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8159980 con 0x7f3de80c9f40 2023-06-09T11:38:47.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.679+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de81596e0 con 0x7f3de80c9f40 2023-06-09T11:38:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.680+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de8159c60 con 0x7f3de80c9f40 2023-06-09T11:38:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.681+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3dd800cbb0 con 0x7f3de80c9f40 2023-06-09T11:38:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.681+0000 7f3dcf7fe700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 0x7f3dd406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.682+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f3dd8089910 con 0x7f3de80c9f40 2023-06-09T11:38:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.682+0000 7f3de67fc700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 0x7f3dd406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:47.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.682+0000 7f3de67fc700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 0x7f3dd406d1c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009130 tx=0x7f3ddc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.686+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3dd8057ef0 con 0x7f3de80c9f40 2023-06-09T11:38:47.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.937+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3de80b36b0 con 0x7f3de80c9f40 2023-06-09T11:38:47.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.938+0000 7f3dcf7fe700 1 -- 172.21.15.7:0/1915602830 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f3dd8057a80 con 0x7f3de80c9f40 2023-06-09T11:38:47.943 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:47.943 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":130,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.940+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 msgr2=0x7f3dd406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.940+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 0x7f3dd406d1c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009130 tx=0x7f3ddc008030 comp rx=0 tx=0).stop 2023-06-09T11:38:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.940+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 msgr2=0x7f3de8151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.940+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3dd800e9e0 tx=0x7f3dd800ecf0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 shutdown_connections 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3de815b2d0 0x7f3de806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd406ad10 0x7f3dd406d1c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3de80c9f40 0x7f3de8151ce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 --2- 172.21.15.7:0/1915602830 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3de8153880 0x7f3de80b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 >> 172.21.15.7:0/1915602830 conn(0x7f3de80737f0 msgr2=0x7f3de814dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 shutdown_connections 2023-06-09T11:38:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:47.941+0000 7f3ded3ff700 1 -- 172.21.15.7:0/1915602830 wait complete. 2023-06-09T11:38:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 130 2023-06-09T11:38:47.962 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:38:48.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:48.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:38:49.963 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:38:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.170+0000 7fb532bff700 1 Processor -- start 2023-06-09T11:38:50.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.170+0000 7fb532bff700 1 -- start start 2023-06-09T11:38:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.171+0000 7fb532bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1bc0 0x7fb52c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.171+0000 7fb532bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb52c0c0140 0x7fb52c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb532bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb52c0c1f90 0x7fb52c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb532bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c070110 con 0x7fb52c0c1f90 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb532bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c071300 con 0x7fb52c0c0140 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb532bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0cbd20 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb52c0c1f90 0x7fb52c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb52c0c1f90 0x7fb52c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47632/0 (socket says 172.21.15.7:47632) 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 -- 172.21.15.7:0/1910887304 learned_addr learned my addr 172.21.15.7:0/1910887304 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb52bfff700 1 --2- 172.21.15.7:0/1910887304 >> v2:172.21.15.117:3300/0 conn(0x7fb52c0c0140 0x7fb52c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53099b700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1bc0 0x7fb52c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 -- 172.21.15.7:0/1910887304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1bc0 msgr2=0x7fb52c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1bc0 0x7fb52c1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 -- 172.21.15.7:0/1910887304 >> v2:172.21.15.117:3300/0 conn(0x7fb52c0c0140 msgr2=0x7fb52c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:38:50.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 --2- 172.21.15.7:0/1910887304 >> v2:172.21.15.117:3300/0 conn(0x7fb52c0c0140 0x7fb52c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.172+0000 7fb53119c700 1 -- 172.21.15.7:0/1910887304 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0c0bf0 con 0x7fb52c0c1f90 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb53119c700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1f90 0x7fb52c1501b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06f740 tx=0x7fb52400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1d4eae33be77ac server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb52b7fe700 1 -- 172.21.15.7:0/1910887304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524010070 con 0x7fb52c0c1f90 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb52b7fe700 1 -- 172.21.15.7:0/1910887304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400dcd0 con 0x7fb52c0c1f90 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb52b7fe700 1 -- 172.21.15.7:0/1910887304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52400b310 con 0x7fb52c0c1f90 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb532bff700 1 -- 172.21.15.7:0/1910887304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1f90 msgr2=0x7fb52c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:50.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.173+0000 7fb532bff700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1f90 0x7fb52c1501b0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb52c06f740 tx=0x7fb52400d460 comp rx=0 tx=0).stop 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 -- 172.21.15.7:0/1910887304 shutdown_connections 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1f90 0x7fb52c1501b0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 --2- 172.21.15.7:0/1910887304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1bc0 0x7fb52c1599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 --2- 172.21.15.7:0/1910887304 >> v2:172.21.15.117:3300/0 conn(0x7fb52c0c0140 0x7fb52c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 -- 172.21.15.7:0/1910887304 >> 172.21.15.7:0/1910887304 conn(0x7fb52c06a4d0 msgr2=0x7fb52c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 -- 172.21.15.7:0/1910887304 shutdown_connections 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.174+0000 7fb532bff700 1 -- 172.21.15.7:0/1910887304 wait complete. 2023-06-09T11:38:50.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.175+0000 7fb532bff700 1 Processor -- start 2023-06-09T11:38:50.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 -- start start 2023-06-09T11:38:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 0x7fb52c0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1f90 0x7fb52c14ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0af5b0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0ace80 con 0x7fb52c0c0140 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.176+0000 7fb532bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c1eecc0 con 0x7fb52c0c1f90 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47642/0 (socket says 172.21.15.7:47642) 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 -- 172.21.15.7:0/1916701111 learned_addr learned my addr 172.21.15.7:0/1916701111 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb53099b700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 0x7fb52c0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb53119c700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1f90 0x7fb52c14ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 -- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1f90 msgr2=0x7fb52c14ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1f90 0x7fb52c14ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 -- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 msgr2=0x7fb52c0c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 0x7fb52c0c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 -- 172.21.15.7:0/1916701111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52400bd40 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb53099b700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 0x7fb52c0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.177+0000 7fb52bfff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e960 tx=0x7fb51800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:50.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.178+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518009690 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.178+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51800c850 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.178+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51800cbc0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.178+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c0c0e90 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.178+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0702f0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.179+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c0c1170 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.180+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb51800cd20 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.181+0000 7fb5297fa700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 0x7fb51006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:38:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.181+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fb518089af0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.181+0000 7fb53099b700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 0x7fb51006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:38:50.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.182+0000 7fb53099b700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 0x7fb51006d1c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb520009c30 tx=0x7fb520008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:38:50.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.185+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb5180545b0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.440+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb52c074cf0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.441+0000 7fb5297fa700 1 -- 172.21.15.7:0/1916701111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fb518057bd0 con 0x7fb52c0c1bc0 2023-06-09T11:38:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:38:50.445 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 msgr2=0x7fb51006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 0x7fb51006d1c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb520009c30 tx=0x7fb520008040 comp rx=0 tx=0).stop 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 msgr2=0x7fb52c1ef150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e960 tx=0x7fb51800ed20 comp rx=0 tx=0).stop 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 shutdown_connections 2023-06-09T11:38:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb51006ad10 0x7fb51006d1c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb52c0c1f90 0x7fb52c14ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb52c0c0140 0x7fb52c0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 --2- 172.21.15.7:0/1916701111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb52c0c1bc0 0x7fb52c1ef150 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 >> 172.21.15.7:0/1916701111 conn(0x7fb52c06a4d0 msgr2=0x7fb52c14bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 shutdown_connections 2023-06-09T11:38:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:38:50.443+0000 7fb532bff700 1 -- 172.21.15.7:0/1916701111 wait complete. 2023-06-09T11:38:50.450 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:38:50.465 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:38:50.465 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 80 secs before thrashing 2023-06-09T11:38:54.677 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:38:54.672+0000 7fc7709f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:38:54.677 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:38:54.672+0000 7fc7709f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:38:54.677 INFO:tasks.ceph.mds.f.smithi117.stderr:2023-06-09T11:38:54.672+0000 7fc7709f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:39:18.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:18.302 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:48.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:39:48.410 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:10.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.679+0000 7f16873ff700 1 Processor -- start 2023-06-09T11:40:10.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.679+0000 7f16873ff700 1 -- start start 2023-06-09T11:40:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f16873ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16800cb9a0 0x7f16800b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f16873ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168015cd40 0x7f16800cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f16873ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1680159b20 con 0x7f16800cb5d0 2023-06-09T11:40:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f16873ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800cac80 con 0x7f16800cb9a0 2023-06-09T11:40:10.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f16873ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16801597d0 con 0x7f168015cd40 2023-06-09T11:40:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168519b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16800cb5d0 0x7f1680159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39980/0 (socket says 172.21.15.7:39980) 2023-06-09T11:40:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168519b700 1 -- 172.21.15.7:0/1565368801 learned_addr learned my addr 172.21.15.7:0/1565368801 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168499a700 1 --2- 172.21.15.7:0/1565368801 >> v2:172.21.15.7:3300/0 conn(0x7f16800cb9a0 0x7f16800b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168599c700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168015cd40 0x7f16800cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f1677fff700 1 -- 172.21.15.7:0/1565368801 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625877751 0 0) 0x7f1680159b20 con 0x7f16800cb5d0 2023-06-09T11:40:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168499a700 1 -- 172.21.15.7:0/1565368801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168015cd40 msgr2=0x7f16800cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:10.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.681+0000 7f168499a700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168015cd40 0x7f16800cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f168499a700 1 -- 172.21.15.7:0/1565368801 >> v1:172.21.15.7:6789/0 conn(0x7f16800cb5d0 legacy=0x7f1680159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f168499a700 1 -- 172.21.15.7:0/1565368801 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1680071ab0 con 0x7f16800cb9a0 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f168499a700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb9a0 0x7f16800b52b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1670009840 tx=0x7f1670009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2b89dbc0243dda server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f1677fff700 1 -- 172.21.15.7:0/1565368801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670011830 con 0x7f16800cb9a0 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f1677fff700 1 -- 172.21.15.7:0/1565368801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670011990 con 0x7f16800cb9a0 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f1677fff700 1 -- 172.21.15.7:0/1565368801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670011d20 con 0x7f16800cb9a0 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f16873ff700 1 -- 172.21.15.7:0/1565368801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb9a0 msgr2=0x7f16800b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.682+0000 7f16873ff700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb9a0 0x7f16800b52b0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f1670009840 tx=0x7f1670009b50 comp rx=0 tx=0).stop 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 -- 172.21.15.7:0/1565368801 shutdown_connections 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168015cd40 0x7f16800cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 --2- 172.21.15.7:0/1565368801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb9a0 0x7f16800b52b0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 -- 172.21.15.7:0/1565368801 >> 172.21.15.7:0/1565368801 conn(0x7f1680070af0 msgr2=0x7f1680070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 -- 172.21.15.7:0/1565368801 shutdown_connections 2023-06-09T11:40:10.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.683+0000 7f16873ff700 1 -- 172.21.15.7:0/1565368801 wait complete. 2023-06-09T11:40:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.684+0000 7f16873ff700 1 Processor -- start 2023-06-09T11:40:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.685+0000 7f16873ff700 1 -- start start 2023-06-09T11:40:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.685+0000 7f16873ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.685+0000 7f16873ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16800cb9a0 0x7f16800c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f16873ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f168015cd40 0x7f16801f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f16873ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680156730 con 0x7f16800cb5d0 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f16873ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16800cfaf0 con 0x7f168015cd40 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f16873ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16800cf620 con 0x7f16800cb9a0 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57214/0 (socket says 172.21.15.7:57214) 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 -- 172.21.15.7:0/3602628510 learned_addr learned my addr 172.21.15.7:0/3602628510 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:10.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168499a700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16800cb9a0 0x7f16800c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168599c700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f168015cd40 0x7f16801f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 -- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16800cb9a0 msgr2=0x7f16800c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16800cb9a0 0x7f16800c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 -- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f168015cd40 msgr2=0x7f16801f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f168015cd40 0x7f16801f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 -- 172.21.15.7:0/3602628510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16700094a0 con 0x7f16800cb5d0 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.686+0000 7f168519b700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f167c0011e0 tx=0x7f167c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:10.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.687+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c004c80 con 0x7f16800cb5d0 2023-06-09T11:40:10.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.687+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c004e00 con 0x7f16800cb5d0 2023-06-09T11:40:10.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.687+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168015d4f0 con 0x7f16800cb5d0 2023-06-09T11:40:10.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.687+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c007310 con 0x7f16800cb5d0 2023-06-09T11:40:10.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.687+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f168015d210 con 0x7f16800cb5d0 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.688+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f168015d7d0 con 0x7f16800cb5d0 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.689+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f167c007470 con 0x7f16800cb5d0 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.690+0000 7f1675ffb700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 0x7f166406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.690+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f167c0896e0 con 0x7f16800cb5d0 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.690+0000 7f168499a700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 0x7f166406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:10.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.691+0000 7f168499a700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 0x7f166406d1c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f16800cbd70 tx=0x7f1670008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:10.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.695+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f167c0541a0 con 0x7f16800cb5d0 2023-06-09T11:40:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.952+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16801f01c0 con 0x7f16800cb5d0 2023-06-09T11:40:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.953+0000 7f1675ffb700 1 -- 172.21.15.7:0/3602628510 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f167c0577c0 con 0x7f16800cb5d0 2023-06-09T11:40:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:10.958 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.955+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 msgr2=0x7f166406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.955+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 0x7f166406d1c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f16800cbd70 tx=0x7f1670008030 comp rx=0 tx=0).stop 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.955+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 msgr2=0x7f16800cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.955+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f167c0011e0 tx=0x7f167c00ca30 comp rx=0 tx=0).stop 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 shutdown_connections 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f168015cd40 0x7f16801f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16800cb5d0 0x7f16800cc010 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166406ad10 0x7f166406d1c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 --2- 172.21.15.7:0/3602628510 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16800cb9a0 0x7f16800c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 >> 172.21.15.7:0/3602628510 conn(0x7f1680070af0 msgr2=0x7f1680158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 shutdown_connections 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:10.956+0000 7f16873ff700 1 -- 172.21.15.7:0/3602628510 wait complete. 2023-06-09T11:40:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:10.978 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-06-09T11:40:10.979 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:40:10.979 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:40:10.997 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:40:10.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:11.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.192+0000 7f74893ff700 1 Processor -- start 2023-06-09T11:40:11.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.193+0000 7f74893ff700 1 -- start start 2023-06-09T11:40:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74893ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f74840cb9a0 0x7f74840b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74893ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f748415cd40 0x7f74840cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74893ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7484159b20 con 0x7f74840cb5d0 2023-06-09T11:40:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74893ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840cac80 con 0x7f748415cd40 2023-06-09T11:40:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74893ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74841597d0 con 0x7f74840cb9a0 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74837fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f748415cd40 0x7f74840cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f7482ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f74840cb5d0 0x7f7484159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39986/0 (socket says 172.21.15.7:39986) 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f7482ffd700 1 -- 172.21.15.7:0/675835794 learned_addr learned my addr 172.21.15.7:0/675835794 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74827fc700 1 --2- 172.21.15.7:0/675835794 >> v2:172.21.15.117:3300/0 conn(0x7f74840cb9a0 0x7f74840b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74837fe700 1 -- 172.21.15.7:0/675835794 >> v2:172.21.15.117:3300/0 conn(0x7f74840cb9a0 msgr2=0x7f74840b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74837fe700 1 --2- 172.21.15.7:0/675835794 >> v2:172.21.15.117:3300/0 conn(0x7f74840cb9a0 0x7f74840b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74837fe700 1 -- 172.21.15.7:0/675835794 >> v1:172.21.15.7:6789/0 conn(0x7f74840cb5d0 legacy=0x7f7484159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.194+0000 7f74837fe700 1 -- 172.21.15.7:0/675835794 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484071ab0 con 0x7f748415cd40 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f74837fe700 1 --2- 172.21.15.7:0/675835794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f748415cd40 0x7f74840cbea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f74840c9c60 tx=0x7f747c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c1f80e58043755 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f7481ffb700 1 -- 172.21.15.7:0/675835794 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00e070 con 0x7f748415cd40 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f7481ffb700 1 -- 172.21.15.7:0/675835794 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00bbd0 con 0x7f748415cd40 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f7481ffb700 1 -- 172.21.15.7:0/675835794 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c012430 con 0x7f748415cd40 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f74893ff700 1 -- 172.21.15.7:0/675835794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f748415cd40 msgr2=0x7f74840cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.195+0000 7f74893ff700 1 --2- 172.21.15.7:0/675835794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f748415cd40 0x7f74840cbea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f74840c9c60 tx=0x7f747c00b330 comp rx=0 tx=0).stop 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 -- 172.21.15.7:0/675835794 shutdown_connections 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 --2- 172.21.15.7:0/675835794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f748415cd40 0x7f74840cbea0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 --2- 172.21.15.7:0/675835794 >> v2:172.21.15.117:3300/0 conn(0x7f74840cb9a0 0x7f74840b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 -- 172.21.15.7:0/675835794 >> 172.21.15.7:0/675835794 conn(0x7f7484070af0 msgr2=0x7f7484070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 -- 172.21.15.7:0/675835794 shutdown_connections 2023-06-09T11:40:11.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.196+0000 7f74893ff700 1 -- 172.21.15.7:0/675835794 wait complete. 2023-06-09T11:40:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.197+0000 7f74893ff700 1 Processor -- start 2023-06-09T11:40:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.198+0000 7f74893ff700 1 -- start start 2023-06-09T11:40:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.198+0000 7f74893ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840cb5d0 0x7f74840cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.198+0000 7f74893ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74893ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f748415cd40 0x7f74840c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74893ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840cf920 con 0x7f74840cb9a0 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74893ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841f3090 con 0x7f748415cd40 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74893ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74840c5d70 con 0x7f74840cb5d0 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57232/0 (socket says 172.21.15.7:57232) 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 -- 172.21.15.7:0/784036543 learned_addr learned my addr 172.21.15.7:0/784036543 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:11.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74837fe700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f748415cd40 0x7f74840c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f7482ffd700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840cb5d0 0x7f74840cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 -- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840cb5d0 msgr2=0x7f74840cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840cb5d0 0x7f74840cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 -- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f748415cd40 msgr2=0x7f74840c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f748415cd40 0x7f74840c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 -- 172.21.15.7:0/784036543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009a70 con 0x7f74840cb9a0 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.199+0000 7f74827fc700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f747800e810 tx=0x7f747800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:11.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.200+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478013610 con 0x7f74840cb9a0 2023-06-09T11:40:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.200+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7478013770 con 0x7f74840cb9a0 2023-06-09T11:40:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.200+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7478013b00 con 0x7f74840cb9a0 2023-06-09T11:40:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.200+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748415d4f0 con 0x7f74840cb9a0 2023-06-09T11:40:11.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.200+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748415d210 con 0x7f74840cb9a0 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.201+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748415d7d0 con 0x7f74840cb9a0 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.202+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7478013c60 con 0x7f74840cb9a0 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.203+0000 7f746b7fe700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 0x7f747006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.203+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f7478089670 con 0x7f74840cb9a0 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.203+0000 7f7482ffd700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 0x7f747006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:11.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.204+0000 7f7482ffd700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 0x7f747006d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7474005910 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:11.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.207+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7478057c50 con 0x7f74840cb9a0 2023-06-09T11:40:11.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.461+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74840b4570 con 0x7f74840cb9a0 2023-06-09T11:40:11.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.462+0000 7f746b7fe700 1 -- 172.21.15.7:0/784036543 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f74780577e0 con 0x7f74840cb9a0 2023-06-09T11:40:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:11.467 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:11.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.464+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 msgr2=0x7f747006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.464+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 0x7f747006d190 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7474005910 tx=0x7f7474009040 comp rx=0 tx=0).stop 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.464+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 msgr2=0x7f7484158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.464+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f747800e810 tx=0x7f747800ebd0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.464+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 shutdown_connections 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f748415cd40 0x7f74840c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f747006ace0 0x7f747006d190 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74840cb5d0 0x7f74840cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 --2- 172.21.15.7:0/784036543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74840cb9a0 0x7f7484158bc0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 >> 172.21.15.7:0/784036543 conn(0x7f7484070af0 msgr2=0x7f7484157380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 shutdown_connections 2023-06-09T11:40:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:11.465+0000 7f74893ff700 1 -- 172.21.15.7:0/784036543 wait complete. 2023-06-09T11:40:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:11.487 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:13.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.695+0000 7fefea1ff700 1 Processor -- start 2023-06-09T11:40:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.695+0000 7fefea1ff700 1 -- start start 2023-06-09T11:40:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe4150390 0x7fefe406b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fefe406aee0 0x7fefe40c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4159670 0x7fefe4159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe40c3db0 con 0x7fefe4150390 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe406dce0 con 0x7fefe406aee0 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.696+0000 7fefea1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe406d0d0 con 0x7fefe4159670 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe4150390 0x7fefe406b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe4150390 0x7fefe406b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59448/0 (socket says 172.21.15.7:59448) 2023-06-09T11:40:13.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 -- 172.21.15.7:0/2330442507 learned_addr learned my addr 172.21.15.7:0/2330442507 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe3fff700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4159670 0x7fefe4159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe2ffd700 1 --2- 172.21.15.7:0/2330442507 >> v2:172.21.15.117:3300/0 conn(0x7fefe406aee0 0x7fefe40c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 -- 172.21.15.7:0/2330442507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4159670 msgr2=0x7fefe4159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4159670 0x7fefe4159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 -- 172.21.15.7:0/2330442507 >> v2:172.21.15.117:3300/0 conn(0x7fefe406aee0 msgr2=0x7fefe40c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2330442507 >> v2:172.21.15.117:3300/0 conn(0x7fefe406aee0 0x7fefe40c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 -- 172.21.15.7:0/2330442507 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe406d3b0 con 0x7fefe4150390 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.697+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4150390 0x7fefe406b670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fefe40c35f0 tx=0x7fefd4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf8f7b3f2bb8302 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefe27fc700 1 -- 172.21.15.7:0/2330442507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011830 con 0x7fefe4150390 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefe27fc700 1 -- 172.21.15.7:0/2330442507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd4011990 con 0x7fefe4150390 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefe27fc700 1 -- 172.21.15.7:0/2330442507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd4011d20 con 0x7fefe4150390 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 -- 172.21.15.7:0/2330442507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4150390 msgr2=0x7fefe406b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4150390 0x7fefe406b670 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fefe40c35f0 tx=0x7fefd4009bd0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 -- 172.21.15.7:0/2330442507 shutdown_connections 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4159670 0x7fefe4159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2330442507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4150390 0x7fefe406b670 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2330442507 >> v2:172.21.15.117:3300/0 conn(0x7fefe406aee0 0x7fefe40c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 -- 172.21.15.7:0/2330442507 >> 172.21.15.7:0/2330442507 conn(0x7fefe406f980 msgr2=0x7fefe40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 -- 172.21.15.7:0/2330442507 shutdown_connections 2023-06-09T11:40:13.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.698+0000 7fefea1ff700 1 -- 172.21.15.7:0/2330442507 wait complete. 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.700+0000 7fefea1ff700 1 Processor -- start 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.700+0000 7fefea1ff700 1 -- start start 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe406aee0 0x7fefe40caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 0x7fefe41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe41ef3d0 con 0x7fefe4159670 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe41eeb00 con 0x7fefe406aee0 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefea1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe41f21f0 con 0x7fefe4150390 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefe3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefe3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59462/0 (socket says 172.21.15.7:59462) 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.701+0000 7fefe3fff700 1 -- 172.21.15.7:0/2248581671 learned_addr learned my addr 172.21.15.7:0/2248581671 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe406aee0 0x7fefe40caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe2ffd700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 0x7fefe41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 -- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 msgr2=0x7fefe41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 0x7fefe41586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 -- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe406aee0 msgr2=0x7fefe40caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe406aee0 0x7fefe40caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 -- 172.21.15.7:0/2248581671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009550 con 0x7fefe4159670 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe2ffd700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 0x7fefe41586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefe3fff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009d30 tx=0x7fefdc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc009ba0 con 0x7fefe4159670 2023-06-09T11:40:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefdc00f040 con 0x7fefe4159670 2023-06-09T11:40:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe406d970 con 0x7fefe4159670 2023-06-09T11:40:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefdc014480 con 0x7fefe4159670 2023-06-09T11:40:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.702+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe406d690 con 0x7fefe4159670 2023-06-09T11:40:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.703+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe406e170 con 0x7fefe4159670 2023-06-09T11:40:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.704+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefdc0145e0 con 0x7fefe4159670 2023-06-09T11:40:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.705+0000 7fefcbfff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 0x7fefd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.705+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fefdc08a6f0 con 0x7fefe4159670 2023-06-09T11:40:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.706+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 0x7fefd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.706+0000 7fefe37fe700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 0x7fefd006d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009c10 tx=0x7fefd4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.710+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fefdc0551e0 con 0x7fefe4159670 2023-06-09T11:40:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.964+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefe4074cf0 con 0x7fefe4159670 2023-06-09T11:40:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.965+0000 7fefcbfff700 1 -- 172.21.15.7:0/2248581671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fefdc018090 con 0x7fefe4159670 2023-06-09T11:40:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:13.970 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 msgr2=0x7fefd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 0x7fefd006d190 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009c10 tx=0x7fefd4008030 comp rx=0 tx=0).stop 2023-06-09T11:40:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 msgr2=0x7fefe414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009d30 tx=0x7fefdc00c3a0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 shutdown_connections 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe4159670 0x7fefe414ec00 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe406aee0 0x7fefe40caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefd006ace0 0x7fefd006d190 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 --2- 172.21.15.7:0/2248581671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe4150390 0x7fefe41586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 >> 172.21.15.7:0/2248581671 conn(0x7fefe406f980 msgr2=0x7fefe40c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 shutdown_connections 2023-06-09T11:40:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:13.967+0000 7fefea1ff700 1 -- 172.21.15.7:0/2248581671 wait complete. 2023-06-09T11:40:13.974 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:13.990 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:15.990 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.195+0000 7f55939ff700 1 Processor -- start 2023-06-09T11:40:16.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.195+0000 7f55939ff700 1 -- start start 2023-06-09T11:40:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f55939ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f558c0c6280 0x7f558c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f55939ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f558c06a4d0 0x7f558c0737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f55939ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558c15cac0 con 0x7f558c06e460 2023-06-09T11:40:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f55939ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0c8870 con 0x7f558c06a4d0 2023-06-09T11:40:16.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f55939ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0c37d0 con 0x7f558c0c6280 2023-06-09T11:40:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f5590f9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f558c06a4d0 0x7f558c0737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f5591f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f558c06e460 0x7f558c06cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60172/0 (socket says 172.21.15.7:60172) 2023-06-09T11:40:16.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f5591f9c700 1 -- 172.21.15.7:0/54199720 learned_addr learned my addr 172.21.15.7:0/54199720 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:16.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.197+0000 7f559179b700 1 --2- 172.21.15.7:0/54199720 >> v2:172.21.15.117:3300/0 conn(0x7f558c0c6280 0x7f558c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5590f9a700 1 -- 172.21.15.7:0/54199720 >> v2:172.21.15.117:3300/0 conn(0x7f558c0c6280 msgr2=0x7f558c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:16.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5590f9a700 1 --2- 172.21.15.7:0/54199720 >> v2:172.21.15.117:3300/0 conn(0x7f558c0c6280 0x7f558c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5590f9a700 1 -- 172.21.15.7:0/54199720 >> v1:172.21.15.7:6789/0 conn(0x7f558c06e460 legacy=0x7f558c06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5590f9a700 1 -- 172.21.15.7:0/54199720 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c15cd30 con 0x7f558c06a4d0 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5583fff700 1 -- 172.21.15.7:0/54199720 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165402784 0 0) 0x7f558c15cac0 con 0x7f558c06e460 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5590f9a700 1 --2- 172.21.15.7:0/54199720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c0737f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f558c0743a0 tx=0x7f557c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561f0234c3192142 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5583fff700 1 -- 172.21.15.7:0/54199720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c011830 con 0x7f558c06a4d0 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5583fff700 1 -- 172.21.15.7:0/54199720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c011990 con 0x7f558c06a4d0 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.198+0000 7f5583fff700 1 -- 172.21.15.7:0/54199720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c011d20 con 0x7f558c06a4d0 2023-06-09T11:40:16.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 -- 172.21.15.7:0/54199720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 msgr2=0x7f558c0737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 --2- 172.21.15.7:0/54199720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c0737f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f558c0743a0 tx=0x7f557c009b20 comp rx=0 tx=0).stop 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 -- 172.21.15.7:0/54199720 shutdown_connections 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 --2- 172.21.15.7:0/54199720 >> v2:172.21.15.117:3300/0 conn(0x7f558c0c6280 0x7f558c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 --2- 172.21.15.7:0/54199720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c0737f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 -- 172.21.15.7:0/54199720 >> 172.21.15.7:0/54199720 conn(0x7f558c06f980 msgr2=0x7f558c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 -- 172.21.15.7:0/54199720 shutdown_connections 2023-06-09T11:40:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.199+0000 7f55939ff700 1 -- 172.21.15.7:0/54199720 wait complete. 2023-06-09T11:40:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.200+0000 7f55939ff700 1 Processor -- start 2023-06-09T11:40:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.201+0000 7f55939ff700 1 -- start start 2023-06-09T11:40:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558c06e460 0x7f558c13d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 0x7f558c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c141be0 con 0x7f558c06a4d0 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c1efdd0 con 0x7f558c0c6280 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f55939ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c1eef70 con 0x7f558c06e460 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59472/0 (socket says 172.21.15.7:59472) 2023-06-09T11:40:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 -- 172.21.15.7:0/820918460 learned_addr learned my addr 172.21.15.7:0/820918460 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f5590f9a700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558c06e460 0x7f558c13d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f5591f9c700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 0x7f558c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 -- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558c06e460 msgr2=0x7f558c13d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558c06e460 0x7f558c13d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 -- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 msgr2=0x7f558c1592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 0x7f558c1592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f559179b700 1 -- 172.21.15.7:0/820918460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0094a0 con 0x7f558c06a4d0 2023-06-09T11:40:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.202+0000 7f5591f9c700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 0x7f558c1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f559179b700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f558800e870 tx=0x7f558800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588009590 con 0x7f558c06a4d0 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558800ee70 con 0x7f558c06a4d0 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c15d2f0 con 0x7f558c06a4d0 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558800ca50 con 0x7f558c06a4d0 2023-06-09T11:40:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.203+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c15d010 con 0x7f558c06a4d0 2023-06-09T11:40:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.204+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558c15d5d0 con 0x7f558c06a4d0 2023-06-09T11:40:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.205+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f558800cbb0 con 0x7f558c06a4d0 2023-06-09T11:40:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.206+0000 7f5581ffb700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 0x7f557006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.206+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f55880899b0 con 0x7f558c06a4d0 2023-06-09T11:40:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.206+0000 7f5590f9a700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 0x7f557006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:16.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.207+0000 7f5590f9a700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 0x7f557006d1c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f557c0090b0 tx=0x7f557c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:16.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.210+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5588057f90 con 0x7f558c06a4d0 2023-06-09T11:40:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.463+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f558c15a2d0 con 0x7f558c06a4d0 2023-06-09T11:40:16.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.464+0000 7f5581ffb700 1 -- 172.21.15.7:0/820918460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f5588021070 con 0x7f558c06a4d0 2023-06-09T11:40:16.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:16.469 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 msgr2=0x7f557006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 0x7f557006d1c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f557c0090b0 tx=0x7f557c008030 comp rx=0 tx=0).stop 2023-06-09T11:40:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 msgr2=0x7f558c1416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f558800e870 tx=0x7f558800ec30 comp rx=0 tx=0).stop 2023-06-09T11:40:16.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 shutdown_connections 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f558c0c6280 0x7f558c1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f558c06a4d0 0x7f558c1416a0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f557006ad10 0x7f557006d1c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 --2- 172.21.15.7:0/820918460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f558c06e460 0x7f558c13d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 >> 172.21.15.7:0/820918460 conn(0x7f558c06f980 msgr2=0x7f558c0c4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 shutdown_connections 2023-06-09T11:40:16.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:16.466+0000 7f55939ff700 1 -- 172.21.15.7:0/820918460 wait complete. 2023-06-09T11:40:16.473 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:16.489 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:18.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:18.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:18.539 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.698+0000 7f1ceedff700 1 Processor -- start 2023-06-09T11:40:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.698+0000 7f1ceedff700 1 -- start start 2023-06-09T11:40:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1ceedff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1ce806d640 0x7f1ce80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1ceedff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce815cac0 con 0x7f1ce80c6280 2023-06-09T11:40:18.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1ceedff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ce80c37d0 con 0x7f1ce806a4d0 2023-06-09T11:40:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1ceedff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ce80c8870 con 0x7f1ce806d640 2023-06-09T11:40:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1cecb9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ce80c6280 0x7f1ce80737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60186/0 (socket says 172.21.15.7:60186) 2023-06-09T11:40:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1cecb9b700 1 -- 172.21.15.7:0/728163991 learned_addr learned my addr 172.21.15.7:0/728163991 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:18.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.699+0000 7f1ced39c700 1 --2- 172.21.15.7:0/728163991 >> v2:172.21.15.117:3300/0 conn(0x7f1ce806d640 0x7f1ce80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505262407 0 0) 0x7f1ce815cac0 con 0x7f1ce80c6280 2023-06-09T11:40:18.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cd80035b0 con 0x7f1ce80c6280 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ced39c700 1 -- 172.21.15.7:0/728163991 >> v1:172.21.15.117:6789/0 conn(0x7f1ce806a4d0 legacy=0x7f1ce806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333424116 0 0) 0x7f1ce80c37d0 con 0x7f1ce806a4d0 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ced39c700 1 -- 172.21.15.7:0/728163991 >> v1:172.21.15.7:6789/0 conn(0x7f1ce80c6280 legacy=0x7f1ce80737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ced39c700 1 -- 172.21.15.7:0/728163991 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce815cd30 con 0x7f1ce806d640 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ced39c700 1 --2- 172.21.15.7:0/728163991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce806d640 0x7f1ce80c76b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009840 tx=0x7f1ce0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f663d175cd0dbcfb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0011830 con 0x7f1ce806d640 2023-06-09T11:40:18.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce0011990 con 0x7f1ce806d640 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.700+0000 7f1ce77fe700 1 -- 172.21.15.7:0/728163991 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0011d20 con 0x7f1ce806d640 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 -- 172.21.15.7:0/728163991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce806d640 msgr2=0x7f1ce80c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 --2- 172.21.15.7:0/728163991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce806d640 0x7f1ce80c76b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009840 tx=0x7f1ce0009b50 comp rx=0 tx=0).stop 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 -- 172.21.15.7:0/728163991 shutdown_connections 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 --2- 172.21.15.7:0/728163991 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce806d640 0x7f1ce80c76b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 -- 172.21.15.7:0/728163991 >> 172.21.15.7:0/728163991 conn(0x7f1ce806f980 msgr2=0x7f1ce80cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 -- 172.21.15.7:0/728163991 shutdown_connections 2023-06-09T11:40:18.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.701+0000 7f1ceedff700 1 -- 172.21.15.7:0/728163991 wait complete. 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.703+0000 7f1ceedff700 1 Processor -- start 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.703+0000 7f1ceedff700 1 -- start start 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ce806a4d0 0x7f1ce81411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 0x7f1ce813d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:18.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce81e4b70 con 0x7f1ce806d640 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce8154770 con 0x7f1ce80c6280 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ceedff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce81e3530 con 0x7f1ce806a4d0 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ce7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59502/0 (socket says 172.21.15.7:59502) 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.704+0000 7f1ce7fff700 1 -- 172.21.15.7:0/3589489354 learned_addr learned my addr 172.21.15.7:0/3589489354 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ced39c700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 0x7f1ce813d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1cecb9b700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ce806a4d0 0x7f1ce81411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:18.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 -- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ce806a4d0 msgr2=0x7f1ce81411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ce806a4d0 0x7f1ce81411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 -- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 msgr2=0x7f1ce813d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 0x7f1ce813d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 -- 172.21.15.7:0/3589489354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce00094a0 con 0x7f1ce806d640 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ced39c700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 0x7f1ce813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:40:18.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce7fff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c750 tx=0x7f1cd400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd40054e0 con 0x7f1ce806d640 2023-06-09T11:40:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd4003680 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce815d2f0 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.705+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4003a30 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.706+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ce815d010 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.707+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ce815d5d0 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.707+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cd4003b90 con 0x7f1ce806d640 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.708+0000 7f1ce57fa700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 0x7f1ccc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.709+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f1cd4088e30 con 0x7f1ce806d640 2023-06-09T11:40:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.709+0000 7f1cecb9b700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 0x7f1ccc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:18.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.709+0000 7f1cecb9b700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 0x7f1ccc06d1c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005c90 tx=0x7f1cdc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.713+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1cd4053a50 con 0x7f1ce806d640 2023-06-09T11:40:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.965+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ce80c6970 con 0x7f1ce806d640 2023-06-09T11:40:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.966+0000 7f1ce57fa700 1 -- 172.21.15.7:0/3589489354 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f1cd4017090 con 0x7f1ce806d640 2023-06-09T11:40:18.971 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:18.971 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.968+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 msgr2=0x7f1ccc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.968+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 0x7f1ccc06d1c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005c90 tx=0x7f1cdc009040 comp rx=0 tx=0).stop 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.968+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 msgr2=0x7f1ce81e39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.968+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c750 tx=0x7f1cd400ca60 comp rx=0 tx=0).stop 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 shutdown_connections 2023-06-09T11:40:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ce80c6280 0x7f1ce813d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1ccc06ad10 0x7f1ccc06d1c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ce806a4d0 0x7f1ce81411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 --2- 172.21.15.7:0/3589489354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ce806d640 0x7f1ce81e39c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 >> 172.21.15.7:0/3589489354 conn(0x7f1ce806f980 msgr2=0x7f1ce806cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 shutdown_connections 2023-06-09T11:40:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:18.969+0000 7f1ceedff700 1 -- 172.21.15.7:0/3589489354 wait complete. 2023-06-09T11:40:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:18.992 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:20.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:21.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.199+0000 7fb4ca5ff700 1 Processor -- start 2023-06-09T11:40:21.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.200+0000 7fb4ca5ff700 1 -- start start 2023-06-09T11:40:21.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.201+0000 7fb4ca5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4c414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.201+0000 7fb4ca5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40cc140 con 0x7fb4c40c08d0 2023-06-09T11:40:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4ca5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40c1830 con 0x7fb4c4071910 2023-06-09T11:40:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4ca5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40714f0 con 0x7fb4c414f9a0 2023-06-09T11:40:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb4c40c08d0 0x7fb4c40cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60200/0 (socket says 172.21.15.7:60200) 2023-06-09T11:40:21.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c3fff700 1 -- 172.21.15.7:0/1524160462 learned_addr learned my addr 172.21.15.7:0/1524160462 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c8b9c700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4c414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758584891 0 0) 0x7fb4c40cc140 con 0x7fb4c40c08d0 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ac0057c0 con 0x7fb4c40c08d0 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2514386179 0 0) 0x7fb4ac0057c0 con 0x7fb4c40c08d0 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 msgr2=0x7fb4c414fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4c414fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 >> v1:172.21.15.117:6789/0 conn(0x7fb4c4071910 legacy=0x7fb4c4071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40c0560 con 0x7fb4c40c08d0 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb4b4001670 con 0x7fb4c40c08d0 2023-06-09T11:40:21.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.202+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 >> v1:172.21.15.7:6789/0 conn(0x7fb4c40c08d0 legacy=0x7fb4c40cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ac005c40 0x7fb4ac006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4ac006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac006fa0 0x7fb4ac007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40714f0 con 0x7fb4ac005c40 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00dd00 con 0x7fb4ac006fa0 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4ac00dec0 con 0x7fb4c414f9a0 2023-06-09T11:40:21.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ac005c40 0x7fb4ac006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c8b9c700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac006fa0 0x7fb4ac007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c37fe700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4ac006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 -- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 msgr2=0x7fb4ac006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c414f9a0 0x7fb4ac006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 -- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac006fa0 msgr2=0x7fb4ac007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac006fa0 0x7fb4ac007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 -- 172.21.15.7:0/1524160462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4ac0057c0 con 0x7fb4ac005c40 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.203+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ac005c40 0x7fb4ac006010 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c1e0 tx=0x7fb4ac007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a0685b9dc7b83 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4007b70 con 0x7fb4ac005c40 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400cd80 con 0x7fb4ac005c40 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4c2ffd700 1 -- 172.21.15.7:0/1524160462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005ba0 con 0x7fb4ac005c40 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1524160462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ac005c40 msgr2=0x7fb4ac006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1524160462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4ac005c40 0x7fb4ac006010 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400c1e0 tx=0x7fb4ac007a90 comp rx=0 tx=0).stop 2023-06-09T11:40:21.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4c8b9c700 1 -- 172.21.15.7:0/1524160462 reap_dead start 2023-06-09T11:40:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1524160462 shutdown_connections 2023-06-09T11:40:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1524160462 >> 172.21.15.7:0/1524160462 conn(0x7fb4c406a4d0 msgr2=0x7fb4c406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1524160462 shutdown_connections 2023-06-09T11:40:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.204+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1524160462 wait complete. 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.206+0000 7fb4ca5ff700 1 Processor -- start 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.206+0000 7fb4ca5ff700 1 -- start start 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 0x7fb4c40ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 0x7fb4c40c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40c2c80 con 0x7fb4c414dfc0 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c41ef9d0 con 0x7fb4ac005c40 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4ca5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c41ef410 con 0x7fb4c40c51d0 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.207+0000 7fb4c8b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59530/0 (socket says 172.21.15.7:59530) 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 -- 172.21.15.7:0/1502432684 learned_addr learned my addr 172.21.15.7:0/1502432684 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 -- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 msgr2=0x7fb4c40c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c37fe700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 0x7fb4c40c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 0x7fb4c40ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 0x7fb4c40c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 -- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 msgr2=0x7fb4c40ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 0x7fb4c40ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 -- 172.21.15.7:0/1502432684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40c1d60 con 0x7fb4c414dfc0 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c37fe700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 0x7fb4c40c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 0x7fb4c40ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c8b9c700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4c406fd50 tx=0x7fb4bc011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00e040 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4bc00f040 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00fd50 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.208+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c40c1a80 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.209+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c40c2040 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.210+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c40c2ef0 con 0x7fb4c414dfc0 2023-06-09T11:40:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.210+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4bc015430 con 0x7fb4c414dfc0 2023-06-09T11:40:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.211+0000 7fb4c0ff9700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.211+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fb4bc089f10 con 0x7fb4c414dfc0 2023-06-09T11:40:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.211+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.212+0000 7fb4c3fff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 0x7fb4b006d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4015040 tx=0x7fb4b4007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:21.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.216+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb4bc0549d0 con 0x7fb4c414dfc0 2023-06-09T11:40:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.470+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c4074cf0 con 0x7fb4c414dfc0 2023-06-09T11:40:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.471+0000 7fb4c0ff9700 1 -- 172.21.15.7:0/1502432684 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fb4bc057ff0 con 0x7fb4c414dfc0 2023-06-09T11:40:21.476 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:21.476 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.473+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 msgr2=0x7fb4b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.473+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 0x7fb4b006d190 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4015040 tx=0x7fb4b4007000 comp rx=0 tx=0).stop 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.473+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 msgr2=0x7fb4c41eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.473+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb4c406fd50 tx=0x7fb4bc011f80 comp rx=0 tx=0).stop 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 shutdown_connections 2023-06-09T11:40:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4c414dfc0 0x7fb4c41eeaa0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4ac005c40 0x7fb4c40ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb4b006ace0 0x7fb4b006d190 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 --2- 172.21.15.7:0/1502432684 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4c40c51d0 0x7fb4c40c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 >> 172.21.15.7:0/1502432684 conn(0x7fb4c406a4d0 msgr2=0x7fb4c41585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 shutdown_connections 2023-06-09T11:40:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:21.474+0000 7fb4ca5ff700 1 -- 172.21.15.7:0/1502432684 wait complete. 2023-06-09T11:40:21.481 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:21.497 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:23.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.703+0000 7fdc4e7ff700 1 Processor -- start 2023-06-09T11:40:23.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.703+0000 7fdc4e7ff700 1 -- start start 2023-06-09T11:40:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4e7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc480b5360 0x7fdc4815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4e7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca580 0x7fdc480c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4e7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4815dcf0 con 0x7fdc480ca1b0 2023-06-09T11:40:23.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4e7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc480c5e00 con 0x7fdc480b5360 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4e7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4815d450 con 0x7fdc480ca580 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc477fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc480b5360 0x7fdc4815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc477fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc480b5360 0x7fdc4815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42554/0 (socket says 172.21.15.7:42554) 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc477fe700 1 -- 172.21.15.7:0/1771090379 learned_addr learned my addr 172.21.15.7:0/1771090379 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc47fff700 1 --1- 172.21.15.7:0/1771090379 >> v1:172.21.15.7:6789/0 conn(0x7fdc480ca1b0 0x7fdc481536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39570/0 (socket says 172.21.15.7:39570) 2023-06-09T11:40:23.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.705+0000 7fdc4cd9c700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca580 0x7fdc480c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc46ffd700 1 -- 172.21.15.7:0/1771090379 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668618879 0 0) 0x7fdc4815dcf0 con 0x7fdc480ca1b0 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc477fe700 1 -- 172.21.15.7:0/1771090379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca580 msgr2=0x7fdc480c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc477fe700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca580 0x7fdc480c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc477fe700 1 -- 172.21.15.7:0/1771090379 >> v1:172.21.15.7:6789/0 conn(0x7fdc480ca1b0 legacy=0x7fdc481536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc477fe700 1 -- 172.21.15.7:0/1771090379 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4815d730 con 0x7fdc480b5360 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc477fe700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480b5360 0x7fdc4815cdc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098f0 tx=0x7fdc3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5b423583469dae server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc46ffd700 1 -- 172.21.15.7:0/1771090379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011830 con 0x7fdc480b5360 2023-06-09T11:40:23.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc46ffd700 1 -- 172.21.15.7:0/1771090379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c011990 con 0x7fdc480b5360 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.706+0000 7fdc46ffd700 1 -- 172.21.15.7:0/1771090379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011d20 con 0x7fdc480b5360 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/1771090379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480b5360 msgr2=0x7fdc4815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480b5360 0x7fdc4815cdc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098f0 tx=0x7fdc3c009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/1771090379 shutdown_connections 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca580 0x7fdc480c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/1771090379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480b5360 0x7fdc4815cdc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/1771090379 >> 172.21.15.7:0/1771090379 conn(0x7fdc48072090 msgr2=0x7fdc48072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/1771090379 shutdown_connections 2023-06-09T11:40:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.707+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/1771090379 wait complete. 2023-06-09T11:40:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.709+0000 7fdc4e7ff700 1 Processor -- start 2023-06-09T11:40:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.710+0000 7fdc4e7ff700 1 -- start start 2023-06-09T11:40:23.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 0x7fdc481f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 0x7fdc480b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480cd920 con 0x7fdc480ca580 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4815c280 con 0x7fdc480b5360 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4e7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481f3070 con 0x7fdc480ca1b0 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42566/0 (socket says 172.21.15.7:42566) 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4cd9c700 1 -- 172.21.15.7:0/4186751018 learned_addr learned my addr 172.21.15.7:0/4186751018 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 0x7fdc481f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc477fe700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 0x7fdc480b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.711+0000 7fdc4cd9c700 1 -- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 msgr2=0x7fdc480b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4cd9c700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 0x7fdc480b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4cd9c700 1 -- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 msgr2=0x7fdc481f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4cd9c700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 0x7fdc481f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4cd9c700 1 -- 172.21.15.7:0/4186751018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009550 con 0x7fdc480ca580 2023-06-09T11:40:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 0x7fdc481f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc477fe700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 0x7fdc480b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:40:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4cd9c700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009d30 tx=0x7fdc4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40009ba0 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4806a570 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40014480 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.712+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4815da10 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.713+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4806a850 con 0x7fdc480ca580 2023-06-09T11:40:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.714+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc400145e0 con 0x7fdc480ca580 2023-06-09T11:40:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.715+0000 7fdc44ff9700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 0x7fdc3406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.715+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fdc4008a2f0 con 0x7fdc480ca580 2023-06-09T11:40:23.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.715+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 0x7fdc3406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:23.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.716+0000 7fdc47fff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 0x7fdc3406d1c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000f60 tx=0x7fdc38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:23.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.720+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdc40054e40 con 0x7fdc480ca580 2023-06-09T11:40:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.975+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc481f31e0 con 0x7fdc480ca580 2023-06-09T11:40:23.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.975+0000 7fdc44ff9700 1 -- 172.21.15.7:0/4186751018 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fdc40058460 con 0x7fdc480ca580 2023-06-09T11:40:23.981 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:23.981 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 msgr2=0x7fdc3406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 0x7fdc3406d1c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdc38000f60 tx=0x7fdc38009040 comp rx=0 tx=0).stop 2023-06-09T11:40:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 msgr2=0x7fdc4814fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009d30 tx=0x7fdc4000c3a0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 shutdown_connections 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc480ca580 0x7fdc4814fcc0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc480b5360 0x7fdc481f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3406ad10 0x7fdc3406d1c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 --2- 172.21.15.7:0/4186751018 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc480ca1b0 0x7fdc480b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 >> 172.21.15.7:0/4186751018 conn(0x7fdc48072090 msgr2=0x7fdc481534c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 shutdown_connections 2023-06-09T11:40:23.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:23.978+0000 7fdc4e7ff700 1 -- 172.21.15.7:0/4186751018 wait complete. 2023-06-09T11:40:23.985 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:24.000 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:26.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:26.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.207+0000 7fe218bff700 1 Processor -- start 2023-06-09T11:40:26.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.207+0000 7fe218bff700 1 -- start start 2023-06-09T11:40:26.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.208+0000 7fe218bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe214151510 0x7fe2140c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.208+0000 7fe218bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe214071230 0x7fe214158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.208+0000 7fe218bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2141580a0 con 0x7fe21415b380 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.208+0000 7fe218bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2141518e0 con 0x7fe214071230 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.208+0000 7fe218bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2140c6180 con 0x7fe214151510 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe214071230 0x7fe214158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe214071230 0x7fe214158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42580/0 (socket says 172.21.15.7:42580) 2023-06-09T11:40:26.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 -- 172.21.15.7:0/3845583189 learned_addr learned my addr 172.21.15.7:0/3845583189 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe21259c700 1 --2- 172.21.15.7:0/3845583189 >> v2:172.21.15.117:3300/0 conn(0x7fe214151510 0x7fe2140c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 -- 172.21.15.7:0/3845583189 >> v2:172.21.15.117:3300/0 conn(0x7fe214151510 msgr2=0x7fe2140c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 --2- 172.21.15.7:0/3845583189 >> v2:172.21.15.117:3300/0 conn(0x7fe214151510 0x7fe2140c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 -- 172.21.15.7:0/3845583189 >> v1:172.21.15.117:6789/0 conn(0x7fe21415b380 legacy=0x7fe214070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 -- 172.21.15.7:0/3845583189 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe214157490 con 0x7fe214071230 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.209+0000 7fe212d9d700 1 --2- 172.21.15.7:0/3845583189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214071230 0x7fe214158870 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe21406ff10 tx=0x7fe20c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9235a3a9b98d10 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe21159a700 1 -- 172.21.15.7:0/3845583189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00e070 con 0x7fe214071230 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe21159a700 1 -- 172.21.15.7:0/3845583189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c003a20 con 0x7fe214071230 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe21159a700 1 -- 172.21.15.7:0/3845583189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c003dd0 con 0x7fe214071230 2023-06-09T11:40:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 -- 172.21.15.7:0/3845583189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214071230 msgr2=0x7fe214158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 --2- 172.21.15.7:0/3845583189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214071230 0x7fe214158870 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe21406ff10 tx=0x7fe20c00b330 comp rx=0 tx=0).stop 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 -- 172.21.15.7:0/3845583189 shutdown_connections 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 --2- 172.21.15.7:0/3845583189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214071230 0x7fe214158870 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 --2- 172.21.15.7:0/3845583189 >> v2:172.21.15.117:3300/0 conn(0x7fe214151510 0x7fe2140c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 -- 172.21.15.7:0/3845583189 >> 172.21.15.7:0/3845583189 conn(0x7fe21406a4d0 msgr2=0x7fe21406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 -- 172.21.15.7:0/3845583189 shutdown_connections 2023-06-09T11:40:26.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.210+0000 7fe218bff700 1 -- 172.21.15.7:0/3845583189 wait complete. 2023-06-09T11:40:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.212+0000 7fe218bff700 1 Processor -- start 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.212+0000 7fe218bff700 1 -- start start 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe214071230 0x7fe2140cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21415b380 0x7fe2140ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140c0fe0 con 0x7fe214151510 2023-06-09T11:40:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140cc070 con 0x7fe21415b380 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe218bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140c11d0 con 0x7fe214071230 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.213+0000 7fe211d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42594/0 (socket says 172.21.15.7:42594) 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 -- 172.21.15.7:0/564597306 learned_addr learned my addr 172.21.15.7:0/564597306 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe21259c700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe214071230 0x7fe2140cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe212d9d700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21415b380 0x7fe2140ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 -- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe214071230 msgr2=0x7fe2140cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe214071230 0x7fe2140cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 -- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21415b380 msgr2=0x7fe2140ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21415b380 0x7fe2140ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 -- 172.21.15.7:0/564597306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c009b30 con 0x7fe214151510 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe211d9b700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b4a0 tx=0x7fe20800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:26.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208011830 con 0x7fe214151510 2023-06-09T11:40:26.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208011990 con 0x7fe214151510 2023-06-09T11:40:26.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.214+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe214157a50 con 0x7fe214151510 2023-06-09T11:40:26.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.215+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208011d40 con 0x7fe214151510 2023-06-09T11:40:26.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.215+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe214157770 con 0x7fe214151510 2023-06-09T11:40:26.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.216+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214157d30 con 0x7fe214151510 2023-06-09T11:40:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.217+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe20801a400 con 0x7fe214151510 2023-06-09T11:40:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.217+0000 7fe202ffd700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 0x7fe1f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:26.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.218+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7fe208089700 con 0x7fe214151510 2023-06-09T11:40:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.218+0000 7fe21259c700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 0x7fe1f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:26.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.218+0000 7fe21259c700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 0x7fe1f406d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe204009930 tx=0x7fe204008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:26.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.222+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe208054310 con 0x7fe214151510 2023-06-09T11:40:26.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.479+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2140c5e70 con 0x7fe214151510 2023-06-09T11:40:26.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.479+0000 7fe202ffd700 1 -- 172.21.15.7:0/564597306 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7fe208057930 con 0x7fe214151510 2023-06-09T11:40:26.484 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:26.485 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.481+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 msgr2=0x7fe1f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.481+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 0x7fe1f406d190 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe204009930 tx=0x7fe204008040 comp rx=0 tx=0).stop 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.481+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 msgr2=0x7fe2140ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fe20800b4a0 tx=0x7fe20800b860 comp rx=0 tx=0).stop 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 shutdown_connections 2023-06-09T11:40:26.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe21415b380 0x7fe2140ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe1f406ace0 0x7fe1f406d190 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe214071230 0x7fe2140cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 --2- 172.21.15.7:0/564597306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe214151510 0x7fe2140ca1e0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 >> 172.21.15.7:0/564597306 conn(0x7fe21406a4d0 msgr2=0x7fe21414b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 shutdown_connections 2023-06-09T11:40:26.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:26.482+0000 7fe218bff700 1 -- 172.21.15.7:0/564597306 wait complete. 2023-06-09T11:40:26.489 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:26.505 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:28.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:28.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.711+0000 7f34b3bff700 1 Processor -- start 2023-06-09T11:40:28.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.711+0000 7f34b3bff700 1 -- start start 2023-06-09T11:40:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.712+0000 7f34b3bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34ac0c5b40 0x7f34ac070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.712+0000 7f34b3bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac1580a0 con 0x7f34ac151510 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.712+0000 7f34b3bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ac0c6180 con 0x7f34ac071060 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.712+0000 7f34b3bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34ac1518e0 con 0x7f34ac0c5b40 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.712+0000 7f34b119a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34ac0c5b40 0x7f34ac070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34ac0c5b40 0x7f34ac070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42600/0 (socket says 172.21.15.7:42600) 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b199b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f34ac151510 0x7f34ac15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39576/0 (socket says 172.21.15.7:39576) 2023-06-09T11:40:28.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 -- 172.21.15.7:0/2081575666 learned_addr learned my addr 172.21.15.7:0/2081575666 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:28.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 -- 172.21.15.7:0/2081575666 >> v1:172.21.15.117:6789/0 conn(0x7f34ac071060 legacy=0x7f34ac158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 -- 172.21.15.7:0/2081575666 >> v1:172.21.15.7:6789/0 conn(0x7f34ac151510 legacy=0x7f34ac15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 -- 172.21.15.7:0/2081575666 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac157490 con 0x7f34ac0c5b40 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b119a700 1 --2- 172.21.15.7:0/2081575666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac0c5b40 0x7f34ac070b20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098f0 tx=0x7f349c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cd47f3f72f4d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.713+0000 7f34b0999700 1 -- 172.21.15.7:0/2081575666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c004230 con 0x7f34ac0c5b40 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b0999700 1 -- 172.21.15.7:0/2081575666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c004390 con 0x7f34ac0c5b40 2023-06-09T11:40:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b0999700 1 -- 172.21.15.7:0/2081575666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349c0036a0 con 0x7f34ac0c5b40 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 -- 172.21.15.7:0/2081575666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac0c5b40 msgr2=0x7f34ac070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 --2- 172.21.15.7:0/2081575666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac0c5b40 0x7f34ac070b20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f349c0098f0 tx=0x7f349c009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 -- 172.21.15.7:0/2081575666 shutdown_connections 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 --2- 172.21.15.7:0/2081575666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac0c5b40 0x7f34ac070b20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 -- 172.21.15.7:0/2081575666 >> 172.21.15.7:0/2081575666 conn(0x7f34ac06a4d0 msgr2=0x7f34ac06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 -- 172.21.15.7:0/2081575666 shutdown_connections 2023-06-09T11:40:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.714+0000 7f34b3bff700 1 -- 172.21.15.7:0/2081575666 wait complete. 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.716+0000 7f34b3bff700 1 Processor -- start 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.716+0000 7f34b3bff700 1 -- start start 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34ac071060 0x7f34ac155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34ac0c5b40 0x7f34ac0c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:28.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0c54f0 con 0x7f34ac151510 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0aedc0 con 0x7f34ac0c5b40 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b3bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ac0af6a0 con 0x7f34ac071060 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.717+0000 7f34b219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b199b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34ac071060 0x7f34ac155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b119a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34ac0c5b40 0x7f34ac0c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:28.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42606/0 (socket says 172.21.15.7:42606) 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 -- 172.21.15.7:0/579949927 learned_addr learned my addr 172.21.15.7:0/579949927 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 -- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34ac071060 msgr2=0x7f34ac155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34ac071060 0x7f34ac155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 -- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34ac0c5b40 msgr2=0x7f34ac0c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34ac0c5b40 0x7f34ac0c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 -- 172.21.15.7:0/579949927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c009550 con 0x7f34ac151510 2023-06-09T11:40:28.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.718+0000 7f34b219c700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f34a400cc40 tx=0x7f34a400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.719+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400aa20 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.719+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400ab80 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.719+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4012420 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.719+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ac157a50 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.719+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ac157770 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.720+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ac157d30 con 0x7f34ac151510 2023-06-09T11:40:28.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.721+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34a4012580 con 0x7f34ac151510 2023-06-09T11:40:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.722+0000 7f34a27fc700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.722+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 7410+0+0 (secure 0 0 0) 0x7f34a408a090 con 0x7f34ac151510 2023-06-09T11:40:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.722+0000 7f34b199b700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:28.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.722+0000 7f34b199b700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 0x7f349006d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f34ac1ecd70 tx=0x7f34a8007f10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:28.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.726+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f34a4054b50 con 0x7f34ac151510 2023-06-09T11:40:28.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.978+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34ac074cf0 con 0x7f34ac151510 2023-06-09T11:40:28.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.979+0000 7f34a27fc700 1 -- 172.21.15.7:0/579949927 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 131 v131) v1 ==== 95+0+6227 (secure 0 0 0) 0x7f34a4016090 con 0x7f34ac151510 2023-06-09T11:40:28.984 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:28.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":131,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6718,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":131}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:38:46.503469+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":5852,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5852":{"gid":5852,"name":"d","rank":1,"incarnation":116,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6835/1343666954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":1343666954},{"type":"v1","addr":"172.21.15.117:6835","nonce":1343666954}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:28.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 msgr2=0x7f349006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 0x7f349006d190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f34ac1ecd70 tx=0x7f34a8007f10 comp rx=0 tx=0).stop 2023-06-09T11:40:28.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 msgr2=0x7f34ac1f28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f34a400cc40 tx=0x7f34a400cf50 comp rx=0 tx=0).stop 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 shutdown_connections 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34ac151510 0x7f34ac1f28a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f349006ace0 0x7f349006d190 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34ac071060 0x7f34ac155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 --2- 172.21.15.7:0/579949927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34ac0c5b40 0x7f34ac0c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:28.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 >> 172.21.15.7:0/579949927 conn(0x7f34ac06a4d0 msgr2=0x7f34ac14d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:28.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 shutdown_connections 2023-06-09T11:40:28.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:28.981+0000 7f34b3bff700 1 -- 172.21.15.7:0/579949927 wait complete. 2023-06-09T11:40:28.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 131 2023-06-09T11:40:29.004 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:40:31.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:31.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.211+0000 7fdd98bff700 1 Processor -- start 2023-06-09T11:40:31.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.212+0000 7fdd98bff700 1 -- start start 2023-06-09T11:40:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd98bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdd940cb5d0 0x7fdd940b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd98bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd94159b20 con 0x7fdd9415cd40 2023-06-09T11:40:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd98bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940cac80 con 0x7fdd940cb9a0 2023-06-09T11:40:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd98bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd941597d0 con 0x7fdd940cb5d0 2023-06-09T11:40:31.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd92d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdd9415cd40 0x7fdd940cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39590/0 (socket says 172.21.15.7:39590) 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.213+0000 7fdd92d9d700 1 -- 172.21.15.7:0/2176043754 learned_addr learned my addr 172.21.15.7:0/2176043754 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2176043754 >> v2:172.21.15.117:3300/0 conn(0x7fdd940cb5d0 0x7fdd940b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595275372 0 0) 0x7fdd94159b20 con 0x7fdd9415cd40 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd800035b0 con 0x7fdd9415cd40 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440688788 0 0) 0x7fdd940cac80 con 0x7fdd940cb9a0 2023-06-09T11:40:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd94159b20 con 0x7fdd940cb9a0 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1701920502 0 0) 0x7fdd800035b0 con 0x7fdd9415cd40 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 >> v2:172.21.15.117:3300/0 conn(0x7fdd940cb5d0 msgr2=0x7fdd940b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 --2- 172.21.15.7:0/2176043754 >> v2:172.21.15.117:3300/0 conn(0x7fdd940cb5d0 0x7fdd940b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 >> v1:172.21.15.117:6789/0 conn(0x7fdd940cb9a0 legacy=0x7fdd94159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd94071ab0 con 0x7fdd9415cd40 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fdd8c0019f0 con 0x7fdd9415cd40 2023-06-09T11:40:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 >> v1:172.21.15.7:6789/0 conn(0x7fdd9415cd40 legacy=0x7fdd940cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 0x7fdd80003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80004850 0x7fdd80004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd80005200 0x7fdd8000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd941597d0 con 0x7fdd80003be0 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8000c030 con 0x7fdd80004850 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.214+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8000c1f0 con 0x7fdd80005200 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 0x7fdd80003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd8bfff700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80004850 0x7fdd80004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd92d9d700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd80005200 0x7fdd8000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 -- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd80005200 msgr2=0x7fdd8000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd80005200 0x7fdd8000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 -- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80004850 msgr2=0x7fdd80004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80004850 0x7fdd80004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 -- 172.21.15.7:0/2176043754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800035b0 con 0x7fdd80003be0 2023-06-09T11:40:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd92d9d700 1 -- 172.21.15.7:0/2176043754 reap_dead start 2023-06-09T11:40:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.215+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 0x7fdd80003fd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009b60 tx=0x7fdd80005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb43971a57f670e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e6c0 con 0x7fdd80003be0 2023-06-09T11:40:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400e820 con 0x7fdd80003be0 2023-06-09T11:40:31.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd91d9b700 1 -- 172.21.15.7:0/2176043754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400eb90 con 0x7fdd80003be0 2023-06-09T11:40:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 -- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 msgr2=0x7fdd80003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 0x7fdd80003fd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fdd84009b60 tx=0x7fdd80005880 comp rx=0 tx=0).stop 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 -- 172.21.15.7:0/2176043754 shutdown_connections 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2176043754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd80003be0 0x7fdd80003fd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 -- 172.21.15.7:0/2176043754 >> 172.21.15.7:0/2176043754 conn(0x7fdd94070af0 msgr2=0x7fdd94070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 -- 172.21.15.7:0/2176043754 shutdown_connections 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.216+0000 7fdd98bff700 1 -- 172.21.15.7:0/2176043754 wait complete. 2023-06-09T11:40:31.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.218+0000 7fdd98bff700 1 Processor -- start 2023-06-09T11:40:31.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.218+0000 7fdd98bff700 1 -- start start 2023-06-09T11:40:31.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80003be0 0x7fdd940c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 0x7fdd94155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd941e50a0 con 0x7fdd940b3930 2023-06-09T11:40:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd941e3190 con 0x7fdd80003be0 2023-06-09T11:40:31.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.219+0000 7fdd98bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940c5ee0 con 0x7fdd941567b0 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42624/0 (socket says 172.21.15.7:42624) 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 -- 172.21.15.7:0/2782293572 learned_addr learned my addr 172.21.15.7:0/2782293572 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80003be0 0x7fdd940c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd92d9d700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 0x7fdd94155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 -- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 msgr2=0x7fdd94155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 0x7fdd94155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 -- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80003be0 msgr2=0x7fdd940c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80003be0 0x7fdd940c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 -- 172.21.15.7:0/2782293572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9415d4f0 con 0x7fdd940b3930 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd92d9d700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 0x7fdd94155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8bfff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00f7e0 tx=0x7fdd7c00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0052e0 con 0x7fdd940b3930 2023-06-09T11:40:31.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7c005460 con 0x7fdd940b3930 2023-06-09T11:40:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c0038c0 con 0x7fdd940b3930 2023-06-09T11:40:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.220+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9415d210 con 0x7fdd940b3930 2023-06-09T11:40:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.221+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9415d7d0 con 0x7fdd940b3930 2023-06-09T11:40:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.222+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd940b4880 con 0x7fdd940b3930 2023-06-09T11:40:31.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.223+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdd7c003a20 con 0x7fdd940b3930 2023-06-09T11:40:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.223+0000 7fdd8affd700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 0x7fdd7406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.224+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fdd7c089980 con 0x7fdd940b3930 2023-06-09T11:40:31.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.224+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 0x7fdd7406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.224+0000 7fdd9259c700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 0x7fdd7406d1c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdd840090b0 tx=0x7fdd8401f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.228+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdd7c054360 con 0x7fdd940b3930 2023-06-09T11:40:31.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.479+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd94158800 con 0x7fdd940b3930 2023-06-09T11:40:31.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.479+0000 7fdd8affd700 1 -- 172.21.15.7:0/2782293572 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+5462 (secure 0 0 0) 0x7fdd7c057980 con 0x7fdd940b3930 2023-06-09T11:40:31.485 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:31.485 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:29.518378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:31.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 msgr2=0x7fdd7406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 0x7fdd7406d1c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdd840090b0 tx=0x7fdd8401f040 comp rx=0 tx=0).stop 2023-06-09T11:40:31.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 msgr2=0x7fdd941eeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00f7e0 tx=0x7fdd7c00faf0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 shutdown_connections 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdd7406ad10 0x7fdd7406d1c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdd941567b0 0x7fdd94155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdd80003be0 0x7fdd940c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 --2- 172.21.15.7:0/2782293572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd940b3930 0x7fdd941eeee0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 >> 172.21.15.7:0/2782293572 conn(0x7fdd94070af0 msgr2=0x7fdd940cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:31.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 shutdown_connections 2023-06-09T11:40:31.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.482+0000 7fdd98bff700 1 -- 172.21.15.7:0/2782293572 wait complete. 2023-06-09T11:40:31.489 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 132 2023-06-09T11:40:31.505 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:40:31.505 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:40:31.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:31.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.701+0000 7fc8a4bff700 1 Processor -- start 2023-06-09T11:40:31.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.701+0000 7fc8a4bff700 1 -- start start 2023-06-09T11:40:31.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc8a4bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc8a01535e0 0x7fc8a0159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc8a4bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a015a420 con 0x7fc8a01539b0 2023-06-09T11:40:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc8a4bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a006b460 con 0x7fc8a00c4ce0 2023-06-09T11:40:31.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc8a4bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00ca670 con 0x7fc8a01535e0 2023-06-09T11:40:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc89dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc8a01539b0 0x7fc8a00c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39606/0 (socket says 172.21.15.7:39606) 2023-06-09T11:40:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc89dd9b700 1 -- 172.21.15.7:0/2237598143 learned_addr learned my addr 172.21.15.7:0/2237598143 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc89e59c700 1 --2- 172.21.15.7:0/2237598143 >> v2:172.21.15.117:3300/0 conn(0x7fc8a01535e0 0x7fc8a0159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.703+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328503561 0 0) 0x7fc894001110 con 0x7fc8a01539b0 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8880035b0 con 0x7fc8a01539b0 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3332155447 0 0) 0x7fc8880035b0 con 0x7fc8a01539b0 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 >> v2:172.21.15.117:3300/0 conn(0x7fc8a01535e0 msgr2=0x7fc8a0159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 --2- 172.21.15.7:0/2237598143 >> v2:172.21.15.117:3300/0 conn(0x7fc8a01535e0 0x7fc8a0159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 >> v1:172.21.15.117:6789/0 conn(0x7fc8a00c4ce0 legacy=0x7fc8a015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a006a570 con 0x7fc8a01539b0 2023-06-09T11:40:31.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fc894001800 con 0x7fc8a01539b0 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 >> v1:172.21.15.7:6789/0 conn(0x7fc8a01539b0 legacy=0x7fc8a00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc888003a50 0x7fc888003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 0x7fc888004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888005200 0x7fc88800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88800c0e0 con 0x7fc888004850 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88800c220 con 0x7fc888003a50 2023-06-09T11:40:31.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88800c3e0 con 0x7fc888005200 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89dd9b700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 0x7fc888004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.704+0000 7fc89e59c700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc888003a50 0x7fc888003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89ed9d700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888005200 0x7fc88800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 -- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888005200 msgr2=0x7fc88800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888005200 0x7fc88800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 -- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc888003a50 msgr2=0x7fc888003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:31.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc888003a50 0x7fc888003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 -- 172.21.15.7:0/2237598143 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8880035b0 con 0x7fc888004850 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89ed9d700 1 -- 172.21.15.7:0/2237598143 reap_dead start 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89dd9b700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 0x7fc888004cc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc894001eb0 tx=0x7fc888005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3476552a37ca16e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400cca0 con 0x7fc888004850 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400ce20 con 0x7fc888004850 2023-06-09T11:40:31.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.705+0000 7fc89d59a700 1 -- 172.21.15.7:0/2237598143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894005ba0 con 0x7fc888004850 2023-06-09T11:40:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 msgr2=0x7fc888004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 0x7fc888004cc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fc894001eb0 tx=0x7fc888005880 comp rx=0 tx=0).stop 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2237598143 shutdown_connections 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2237598143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc888004850 0x7fc888004cc0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2237598143 >> 172.21.15.7:0/2237598143 conn(0x7fc8a00737f0 msgr2=0x7fc8a0072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2237598143 shutdown_connections 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.706+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2237598143 wait complete. 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.708+0000 7fc8a4bff700 1 Processor -- start 2023-06-09T11:40:31.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.708+0000 7fc8a4bff700 1 -- start start 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888004850 0x7fc8a014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a014ef60 0x7fc8a0158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01ef690 con 0x7fc8a0158860 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a01efb80 con 0x7fc8a014ef60 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc8a4bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00c6c20 con 0x7fc888004850 2023-06-09T11:40:31.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42646/0 (socket says 172.21.15.7:42646) 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 -- 172.21.15.7:0/2589969482 learned_addr learned my addr 172.21.15.7:0/2589969482 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89e59c700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888004850 0x7fc8a014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89dd9b700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a014ef60 0x7fc8a0158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 -- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888004850 msgr2=0x7fc8a014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888004850 0x7fc8a014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 -- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a014ef60 msgr2=0x7fc8a0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a014ef60 0x7fc8a0158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 -- 172.21.15.7:0/2589969482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a006ab30 con 0x7fc8a0158860 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.709+0000 7fc89ed9d700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00ccb60 tx=0x7fc89800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.710+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89800eed0 con 0x7fc8a0158860 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.710+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898003c10 con 0x7fc8a0158860 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.710+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980050d0 con 0x7fc8a0158860 2023-06-09T11:40:31.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.710+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a006a850 con 0x7fc8a0158860 2023-06-09T11:40:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.710+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a006ae10 con 0x7fc8a0158860 2023-06-09T11:40:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.711+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a0074cf0 con 0x7fc8a0158860 2023-06-09T11:40:31.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.712+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc898005230 con 0x7fc8a0158860 2023-06-09T11:40:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.713+0000 7fc88effd700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 0x7fc88006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.713+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fc8980899a0 con 0x7fc8a0158860 2023-06-09T11:40:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.713+0000 7fc89e59c700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 0x7fc88006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.713+0000 7fc89e59c700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 0x7fc88006d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc890009ce0 tx=0x7fc890009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:31.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.717+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc898054410 con 0x7fc8a0158860 2023-06-09T11:40:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.970+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8a00c48b0 con 0x7fc8a0158860 2023-06-09T11:40:31.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.971+0000 7fc88effd700 1 -- 172.21.15.7:0/2589969482 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+5462 (secure 0 0 0) 0x7fc898057a30 con 0x7fc8a0158860 2023-06-09T11:40:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:31.976 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:29.518378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 msgr2=0x7fc88006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 0x7fc88006d190 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc890009ce0 tx=0x7fc890009210 comp rx=0 tx=0).stop 2023-06-09T11:40:31.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 msgr2=0x7fc8a00c76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00ccb60 tx=0x7fc89800ed50 comp rx=0 tx=0).stop 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 shutdown_connections 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8a0158860 0x7fc8a00c76c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88006ace0 0x7fc88006d190 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc888004850 0x7fc8a014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 --2- 172.21.15.7:0/2589969482 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8a014ef60 0x7fc8a0158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 >> 172.21.15.7:0/2589969482 conn(0x7fc8a00737f0 msgr2=0x7fc8a0152220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 shutdown_connections 2023-06-09T11:40:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:31.973+0000 7fc8a4bff700 1 -- 172.21.15.7:0/2589969482 wait complete. 2023-06-09T11:40:31.980 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 132 2023-06-09T11:40:31.996 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 132, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 132, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:40:29.518378+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:40:33.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:34.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.204+0000 7ff9571ff700 1 Processor -- start 2023-06-09T11:40:34.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.204+0000 7ff9571ff700 1 -- start start 2023-06-09T11:40:34.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff9571ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9500c5b40 0x7ff950070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff9571ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff950071060 0x7ff950158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff9571ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9501580a0 con 0x7ff950151510 2023-06-09T11:40:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff9571ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500714e0 con 0x7ff9500c5b40 2023-06-09T11:40:34.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff9571ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9501518e0 con 0x7ff950071060 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9500c5b40 0x7ff950070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff9500c5b40 0x7ff950070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43000/0 (socket says 172.21.15.7:43000) 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 -- 172.21.15.7:0/1323732806 learned_addr learned my addr 172.21.15.7:0/1323732806 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff95579c700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff950071060 0x7ff950158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 -- 172.21.15.7:0/1323732806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff950071060 msgr2=0x7ff950158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:34.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff950071060 0x7ff950158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 -- 172.21.15.7:0/1323732806 >> v1:172.21.15.117:6789/0 conn(0x7ff950151510 legacy=0x7ff95015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.206+0000 7ff94ffff700 1 -- 172.21.15.7:0/1323732806 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500c6180 con 0x7ff9500c5b40 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff94ffff700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500c5b40 0x7ff950070b20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098f0 tx=0x7ff93c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3af6d3b57386b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff94f7fe700 1 -- 172.21.15.7:0/1323732806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c011830 con 0x7ff9500c5b40 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff94f7fe700 1 -- 172.21.15.7:0/1323732806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c011990 con 0x7ff9500c5b40 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff94f7fe700 1 -- 172.21.15.7:0/1323732806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c011d20 con 0x7ff9500c5b40 2023-06-09T11:40:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff9571ff700 1 -- 172.21.15.7:0/1323732806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500c5b40 msgr2=0x7ff950070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.207+0000 7ff9571ff700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500c5b40 0x7ff950070b20 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098f0 tx=0x7ff93c009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 -- 172.21.15.7:0/1323732806 shutdown_connections 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff950071060 0x7ff950158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 --2- 172.21.15.7:0/1323732806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9500c5b40 0x7ff950070b20 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 -- 172.21.15.7:0/1323732806 >> 172.21.15.7:0/1323732806 conn(0x7ff95006a4d0 msgr2=0x7ff95006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 -- 172.21.15.7:0/1323732806 shutdown_connections 2023-06-09T11:40:34.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.208+0000 7ff9571ff700 1 -- 172.21.15.7:0/1323732806 wait complete. 2023-06-09T11:40:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.209+0000 7ff9571ff700 1 Processor -- start 2023-06-09T11:40:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.210+0000 7ff9571ff700 1 -- start start 2023-06-09T11:40:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.210+0000 7ff9571ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff9571ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9500c5b40 0x7ff9500c0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff9571ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff950151510 0x7ff9501f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff9571ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95014f200 con 0x7ff950071060 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff9571ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950159df0 con 0x7ff950151510 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff9571ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95014e610 con 0x7ff9500c5b40 2023-06-09T11:40:34.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43012/0 (socket says 172.21.15.7:43012) 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 -- 172.21.15.7:0/3863947805 learned_addr learned my addr 172.21.15.7:0/3863947805 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 -- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9500c5b40 msgr2=0x7ff9500c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9500c5b40 0x7ff9500c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 -- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff950151510 msgr2=0x7ff9501f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff95579c700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff950151510 0x7ff9501f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:34.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff950151510 0x7ff9501f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 -- 172.21.15.7:0/3863947805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff93c009550 con 0x7ff950071060 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.211+0000 7ff954f9b700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c450 tx=0x7ff94400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.212+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440052d0 con 0x7ff950071060 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.212+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944005430 con 0x7ff950071060 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.212+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9440038c0 con 0x7ff950071060 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.212+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff950157770 con 0x7ff950071060 2023-06-09T11:40:34.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.212+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff950157490 con 0x7ff950071060 2023-06-09T11:40:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.213+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff950157a50 con 0x7ff950071060 2023-06-09T11:40:34.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.214+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff944003a20 con 0x7ff950071060 2023-06-09T11:40:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.216+0000 7ff94d7fa700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 0x7ff93406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.216+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7ff944088ef0 con 0x7ff950071060 2023-06-09T11:40:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.216+0000 7ff94ffff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 0x7ff93406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:34.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.217+0000 7ff94ffff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 0x7ff93406d1c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff9500c1ca0 tx=0x7ff93c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:34.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.220+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff9440538d0 con 0x7ff950071060 2023-06-09T11:40:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.476+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff950074cf0 con 0x7ff950071060 2023-06-09T11:40:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.476+0000 7ff94d7fa700 1 -- 172.21.15.7:0/3863947805 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+5462 (secure 0 0 0) 0x7ff944056ef0 con 0x7ff950071060 2023-06-09T11:40:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:34.482 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":132,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":132,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:29.518378+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 msgr2=0x7ff93406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 0x7ff93406d1c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff9500c1ca0 tx=0x7ff93c010000 comp rx=0 tx=0).stop 2023-06-09T11:40:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 msgr2=0x7ff9500c0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff94400c450 tx=0x7ff94400c810 comp rx=0 tx=0).stop 2023-06-09T11:40:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 shutdown_connections 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff950151510 0x7ff9501f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff950071060 0x7ff9500c0220 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff93406ad10 0x7ff93406d1c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 --2- 172.21.15.7:0/3863947805 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff9500c5b40 0x7ff9500c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 >> 172.21.15.7:0/3863947805 conn(0x7ff95006a4d0 msgr2=0x7ff95015b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 shutdown_connections 2023-06-09T11:40:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:34.479+0000 7ff9571ff700 1 -- 172.21.15.7:0/3863947805 wait complete. 2023-06-09T11:40:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 132 2023-06-09T11:40:34.502 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:36.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.708+0000 7efc33bff700 1 Processor -- start 2023-06-09T11:40:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.709+0000 7efc33bff700 1 -- start start 2023-06-09T11:40:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc33bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efc2c0cb5d0 0x7efc2c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc33bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc33bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c159b20 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc33bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c0cac80 con 0x7efc2c0cb5d0 2023-06-09T11:40:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc33bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c1597d0 con 0x7efc2c15cd40 2023-06-09T11:40:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc3119a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efc2c0cb9a0 0x7efc2c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36802/0 (socket says 172.21.15.7:36802) 2023-06-09T11:40:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc3119a700 1 -- 172.21.15.7:0/2508677714 learned_addr learned my addr 172.21.15.7:0/2508677714 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc3219c700 1 --2- 172.21.15.7:0/2508677714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.710+0000 7efc3199b700 1 --2- 172.21.15.7:0/2508677714 >> v2:172.21.15.117:3300/0 conn(0x7efc2c0cb5d0 0x7efc2c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc3219c700 1 -- 172.21.15.7:0/2508677714 >> v2:172.21.15.117:3300/0 conn(0x7efc2c0cb5d0 msgr2=0x7efc2c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc3219c700 1 --2- 172.21.15.7:0/2508677714 >> v2:172.21.15.117:3300/0 conn(0x7efc2c0cb5d0 0x7efc2c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc3219c700 1 -- 172.21.15.7:0/2508677714 >> v1:172.21.15.7:6789/0 conn(0x7efc2c0cb9a0 legacy=0x7efc2c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc3219c700 1 -- 172.21.15.7:0/2508677714 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c071ab0 con 0x7efc2c15cd40 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc30999700 1 -- 172.21.15.7:0/2508677714 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246602654 0 0) 0x7efc2c159b20 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc3219c700 1 --2- 172.21.15.7:0/2508677714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0cbea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d90 tx=0x7efc2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4284a7af9c28920f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc30999700 1 -- 172.21.15.7:0/2508677714 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2400e070 con 0x7efc2c15cd40 2023-06-09T11:40:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc30999700 1 -- 172.21.15.7:0/2508677714 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2400bb20 con 0x7efc2c15cd40 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.711+0000 7efc30999700 1 -- 172.21.15.7:0/2508677714 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc24012430 con 0x7efc2c15cd40 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 -- 172.21.15.7:0/2508677714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 msgr2=0x7efc2c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 --2- 172.21.15.7:0/2508677714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0cbea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efc24009d90 tx=0x7efc2400b330 comp rx=0 tx=0).stop 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 -- 172.21.15.7:0/2508677714 shutdown_connections 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 --2- 172.21.15.7:0/2508677714 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0cbea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 --2- 172.21.15.7:0/2508677714 >> v2:172.21.15.117:3300/0 conn(0x7efc2c0cb5d0 0x7efc2c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 -- 172.21.15.7:0/2508677714 >> 172.21.15.7:0/2508677714 conn(0x7efc2c070af0 msgr2=0x7efc2c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 -- 172.21.15.7:0/2508677714 shutdown_connections 2023-06-09T11:40:36.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.712+0000 7efc33bff700 1 -- 172.21.15.7:0/2508677714 wait complete. 2023-06-09T11:40:36.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.714+0000 7efc33bff700 1 Processor -- start 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.714+0000 7efc33bff700 1 -- start start 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 0x7efc2c14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0ccca0 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c151740 con 0x7efc2c0cb5d0 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc33bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c150530 con 0x7efc2c15cd40 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3119a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43020/0 (socket says 172.21.15.7:43020) 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3119a700 1 -- 172.21.15.7:0/78146041 learned_addr learned my addr 172.21.15.7:0/78146041 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3199b700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 0x7efc2c14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3219c700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3119a700 1 -- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 msgr2=0x7efc2c0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.715+0000 7efc3119a700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc3119a700 1 -- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 msgr2=0x7efc2c14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc3119a700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 0x7efc2c14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc3119a700 1 -- 172.21.15.7:0/78146041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc24009a70 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc3199b700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 0x7efc2c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:36.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc3119a700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c570 tx=0x7efc1c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004c80 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c004e00 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c15d4f0 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c007310 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.716+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c15d210 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.717+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c15d7d0 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.718+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc1c007470 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.719+0000 7efc227fc700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 0x7efc1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:36.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.719+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7efc1c089880 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.720+0000 7efc3199b700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 0x7efc1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.720+0000 7efc3199b700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 0x7efc1006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc28009c30 tx=0x7efc28008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:36.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.724+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc1c0542f0 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.980+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc2c0b3f50 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.981+0000 7efc227fc700 1 -- 172.21.15.7:0/78146041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5464 (secure 0 0 0) 0x7efc1c057910 con 0x7efc2c0cb9a0 2023-06-09T11:40:36.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:36.986 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:36.730774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:resolve","state_seq":29,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:36.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 msgr2=0x7efc1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 0x7efc1006d190 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efc28009c30 tx=0x7efc28008040 comp rx=0 tx=0).stop 2023-06-09T11:40:36.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 msgr2=0x7efc2c1ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:36.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c570 tx=0x7efc1c00c930 comp rx=0 tx=0).stop 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 shutdown_connections 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1006ace0 0x7efc1006d190 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc2c15cd40 0x7efc2c0c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc2c0cb5d0 0x7efc2c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 --2- 172.21.15.7:0/78146041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc2c0cb9a0 0x7efc2c1ee700 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 >> 172.21.15.7:0/78146041 conn(0x7efc2c070af0 msgr2=0x7efc2c156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:36.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 shutdown_connections 2023-06-09T11:40:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:36.983+0000 7efc33bff700 1 -- 172.21.15.7:0/78146041 wait complete. 2023-06-09T11:40:36.990 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 133 2023-06-09T11:40:37.006 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:39.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.212+0000 7fcce91ff700 1 Processor -- start 2023-06-09T11:40:39.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.212+0000 7fcce91ff700 1 -- start start 2023-06-09T11:40:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.213+0000 7fcce91ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcce4151510 0x7fcce40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.213+0000 7fcce91ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce41580a0 con 0x7fcce415b380 2023-06-09T11:40:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.213+0000 7fcce91ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40c6180 con 0x7fcce4071230 2023-06-09T11:40:39.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.213+0000 7fcce91ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce41518e0 con 0x7fcce4151510 2023-06-09T11:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcce4151510 0x7fcce40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:39.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fcce4151510 0x7fcce40c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43032/0 (socket says 172.21.15.7:43032) 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 -- 172.21.15.7:0/888147003 learned_addr learned my addr 172.21.15.7:0/888147003 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 -- 172.21.15.7:0/888147003 >> v1:172.21.15.117:6789/0 conn(0x7fcce4071230 legacy=0x7fcce4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 -- 172.21.15.7:0/888147003 >> v1:172.21.15.7:6789/0 conn(0x7fcce415b380 legacy=0x7fcce4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 -- 172.21.15.7:0/888147003 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4157490 con 0x7fcce4151510 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.214+0000 7fcce2d9d700 1 --2- 172.21.15.7:0/888147003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40c5b40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098f0 tx=0x7fccd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd2e713e139ccd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce1d9b700 1 -- 172.21.15.7:0/888147003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400b8a0 con 0x7fcce4151510 2023-06-09T11:40:39.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce1d9b700 1 -- 172.21.15.7:0/888147003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd400ba00 con 0x7fcce4151510 2023-06-09T11:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce1d9b700 1 -- 172.21.15.7:0/888147003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400bd90 con 0x7fcce4151510 2023-06-09T11:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 -- 172.21.15.7:0/888147003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 msgr2=0x7fcce40c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 --2- 172.21.15.7:0/888147003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40c5b40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098f0 tx=0x7fccd4009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 -- 172.21.15.7:0/888147003 shutdown_connections 2023-06-09T11:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 --2- 172.21.15.7:0/888147003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40c5b40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 -- 172.21.15.7:0/888147003 >> 172.21.15.7:0/888147003 conn(0x7fcce406a4d0 msgr2=0x7fcce406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 -- 172.21.15.7:0/888147003 shutdown_connections 2023-06-09T11:40:39.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.215+0000 7fcce91ff700 1 -- 172.21.15.7:0/888147003 wait complete. 2023-06-09T11:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.217+0000 7fcce91ff700 1 Processor -- start 2023-06-09T11:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.217+0000 7fcce91ff700 1 -- start start 2023-06-09T11:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce4071230 0x7fcce40c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:39.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcce415b380 0x7fcce40cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41f04b0 con 0x7fcce4151510 2023-06-09T11:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce41eff50 con 0x7fcce4071230 2023-06-09T11:40:39.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce4150470 con 0x7fcce415b380 2023-06-09T11:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.218+0000 7fcce259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43036/0 (socket says 172.21.15.7:43036) 2023-06-09T11:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 -- 172.21.15.7:0/52158756 learned_addr learned my addr 172.21.15.7:0/52158756 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:39.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce2d9d700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce4071230 0x7fcce40c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce359e700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcce415b380 0x7fcce40cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 -- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcce415b380 msgr2=0x7fcce40cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcce415b380 0x7fcce40cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 -- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce4071230 msgr2=0x7fcce40c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce4071230 0x7fcce40c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 -- 172.21.15.7:0/52158756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4009550 con 0x7fcce4151510 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce259c700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c450 tx=0x7fccd800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:39.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80052d0 con 0x7fcce4151510 2023-06-09T11:40:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8005430 con 0x7fcce4151510 2023-06-09T11:40:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce4157a50 con 0x7fcce4151510 2023-06-09T11:40:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.219+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd80038c0 con 0x7fcce4151510 2023-06-09T11:40:39.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.220+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce4157770 con 0x7fcce4151510 2023-06-09T11:40:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.221+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce4157d30 con 0x7fcce4151510 2023-06-09T11:40:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.222+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fccd8003a20 con 0x7fcce4151510 2023-06-09T11:40:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.222+0000 7fcccb7fe700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.223+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fccd8088d80 con 0x7fcce4151510 2023-06-09T11:40:39.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.223+0000 7fcce2d9d700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.223+0000 7fcce2d9d700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 0x7fccd006d1c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009c40 tx=0x7fccd4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:39.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.227+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fccd80537f0 con 0x7fcce4151510 2023-06-09T11:40:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.478+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcce4074cf0 con 0x7fcce4151510 2023-06-09T11:40:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.478+0000 7fcccb7fe700 1 -- 172.21.15.7:0/52158756 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fccd800ce60 con 0x7fcce4151510 2023-06-09T11:40:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:39.484 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:36.730774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:resolve","state_seq":29,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 msgr2=0x7fccd006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 0x7fccd006d1c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009c40 tx=0x7fccd4008030 comp rx=0 tx=0).stop 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 msgr2=0x7fcce40ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fccd800c450 tx=0x7fccd800c760 comp rx=0 tx=0).stop 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 shutdown_connections 2023-06-09T11:40:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fccd006ad10 0x7fccd006d1c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fcce415b380 0x7fcce40cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fcce4071230 0x7fcce40c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 --2- 172.21.15.7:0/52158756 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcce4151510 0x7fcce40ae280 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 >> 172.21.15.7:0/52158756 conn(0x7fcce406a4d0 msgr2=0x7fcce414d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 shutdown_connections 2023-06-09T11:40:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:39.481+0000 7fcce91ff700 1 -- 172.21.15.7:0/52158756 wait complete. 2023-06-09T11:40:39.488 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 133 2023-06-09T11:40:39.504 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:41.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:41.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.711+0000 7fd1d61ff700 1 Processor -- start 2023-06-09T11:40:41.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.711+0000 7fd1d61ff700 1 -- start start 2023-06-09T11:40:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.712+0000 7fd1d61ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 0x7fd1d015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.712+0000 7fd1d61ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd1d00c9f40 0x7fd1d015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.712+0000 7fd1d61ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d015a500 con 0x7fd1d015b2d0 2023-06-09T11:40:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.712+0000 7fd1d61ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d00c5f10 con 0x7fd1d00c9f40 2023-06-09T11:40:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.712+0000 7fd1d61ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d015da00 con 0x7fd1d0153880 2023-06-09T11:40:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd1d015b2d0 0x7fd1d00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36816/0 (socket says 172.21.15.7:36816) 2023-06-09T11:40:41.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cffff700 1 -- 172.21.15.7:0/1924210134 learned_addr learned my addr 172.21.15.7:0/1924210134 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:41.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1924210134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 0x7fd1d015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/1924210134 >> v2:172.21.15.117:3300/0 conn(0x7fd1d00c9f40 0x7fd1d015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/1924210134 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1819272444 0 0) 0x7fd1c8001490 con 0x7fd1d015b2d0 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/1924210134 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b80035b0 con 0x7fd1d015b2d0 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cf7fe700 1 -- 172.21.15.7:0/1924210134 >> v2:172.21.15.117:3300/0 conn(0x7fd1d00c9f40 msgr2=0x7fd1d015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1924210134 >> v2:172.21.15.117:3300/0 conn(0x7fd1d00c9f40 0x7fd1d015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cf7fe700 1 -- 172.21.15.7:0/1924210134 >> v1:172.21.15.7:6789/0 conn(0x7fd1d015b2d0 legacy=0x7fd1d00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.713+0000 7fd1cf7fe700 1 -- 172.21.15.7:0/1924210134 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d015b750 con 0x7fd1d0153880 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.714+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1924210134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 0x7fd1d015d4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009840 tx=0x7fd1c0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ad9a21320d93155 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.714+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/1924210134 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011830 con 0x7fd1d0153880 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.714+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/1924210134 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c0011990 con 0x7fd1d0153880 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.714+0000 7fd1ce7fc700 1 -- 172.21.15.7:0/1924210134 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0011d00 con 0x7fd1d0153880 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1924210134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 msgr2=0x7fd1d015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1924210134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 0x7fd1d015d4c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009840 tx=0x7fd1c0009b50 comp rx=0 tx=0).stop 2023-06-09T11:40:41.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1924210134 shutdown_connections 2023-06-09T11:40:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1924210134 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d0153880 0x7fd1d015d4c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1924210134 >> v2:172.21.15.117:3300/0 conn(0x7fd1d00c9f40 0x7fd1d015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1924210134 >> 172.21.15.7:0/1924210134 conn(0x7fd1d00737f0 msgr2=0x7fd1d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1924210134 shutdown_connections 2023-06-09T11:40:41.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.715+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1924210134 wait complete. 2023-06-09T11:40:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.717+0000 7fd1d61ff700 1 Processor -- start 2023-06-09T11:40:41.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.717+0000 7fd1d61ff700 1 -- start start 2023-06-09T11:40:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.718+0000 7fd1d61ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d00c9f40 0x7fd1d006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.718+0000 7fd1d61ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1d61ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d015b2d0 0x7fd1d01f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1d61ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0152710 con 0x7fd1d0153880 2023-06-09T11:40:41.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1d61ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0152ac0 con 0x7fd1d00c9f40 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1d61ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01581b0 con 0x7fd1d015b2d0 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43048/0 (socket says 172.21.15.7:43048) 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 -- 172.21.15.7:0/1633821485 learned_addr learned my addr 172.21.15.7:0/1633821485 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d00c9f40 0x7fd1d006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1cffff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d015b2d0 0x7fd1d01f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 -- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d015b2d0 msgr2=0x7fd1d01f2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d015b2d0 0x7fd1d01f2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 -- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d00c9f40 msgr2=0x7fd1d006a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d00c9f40 0x7fd1d006a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 -- 172.21.15.7:0/1633821485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c00094a0 con 0x7fd1d0153880 2023-06-09T11:40:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.719+0000 7fd1ceffd700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e9b0 tx=0x7fd1c400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.720+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4009710 con 0x7fd1d0153880 2023-06-09T11:40:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.720+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c400ee50 con 0x7fd1d0153880 2023-06-09T11:40:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.720+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d0159980 con 0x7fd1d0153880 2023-06-09T11:40:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.720+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400ca50 con 0x7fd1d0153880 2023-06-09T11:40:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.720+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d01596e0 con 0x7fd1d0153880 2023-06-09T11:40:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.721+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0159c60 con 0x7fd1d0153880 2023-06-09T11:40:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.722+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1c400cbb0 con 0x7fd1d0153880 2023-06-09T11:40:41.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.723+0000 7fd1b7fff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 0x7fd1bc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.723+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fd1c40899c0 con 0x7fd1d0153880 2023-06-09T11:40:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.723+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 0x7fd1bc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:41.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.724+0000 7fd1cf7fe700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 0x7fd1bc06d1c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0000c00 tx=0x7fd1c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:41.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.727+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd1c4057f50 con 0x7fd1d0153880 2023-06-09T11:40:41.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.979+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1d0074cf0 con 0x7fd1d0153880 2023-06-09T11:40:41.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.980+0000 7fd1b7fff700 1 -- 172.21.15.7:0/1633821485 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 133 v133) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fd1c400ce60 con 0x7fd1d0153880 2023-06-09T11:40:41.985 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:41.985 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":133,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":133,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:36.730774+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:resolve","state_seq":29,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:41.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 msgr2=0x7fd1bc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 0x7fd1bc06d1c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0000c00 tx=0x7fd1c0008030 comp rx=0 tx=0).stop 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 msgr2=0x7fd1d01f2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd1c400e9b0 tx=0x7fd1c400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 shutdown_connections 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1bc06ad10 0x7fd1bc06d1c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd1d015b2d0 0x7fd1d01f2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd1d00c9f40 0x7fd1d006a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 --2- 172.21.15.7:0/1633821485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1d0153880 0x7fd1d01f2780 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:41.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.982+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 >> 172.21.15.7:0/1633821485 conn(0x7fd1d00737f0 msgr2=0x7fd1d01505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:41.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.983+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 shutdown_connections 2023-06-09T11:40:41.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:41.983+0000 7fd1d61ff700 1 -- 172.21.15.7:0/1633821485 wait complete. 2023-06-09T11:40:41.989 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 133 2023-06-09T11:40:42.005 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:44.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:44.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.213+0000 7fd2f87ff700 1 Processor -- start 2023-06-09T11:40:44.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.213+0000 7fd2f87ff700 1 -- start start 2023-06-09T11:40:44.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f87ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2f0071c80 0x7fd2f0159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f00af980 0x7fd2f015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f87ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f015ce00 con 0x7fd2f0153960 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f87ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f00adab0 con 0x7fd2f0071c80 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015dc60 con 0x7fd2f00af980 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f5d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2f0071c80 0x7fd2f0159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f5d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2f0071c80 0x7fd2f0159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36164/0 (socket says 172.21.15.7:36164) 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/1363210876 learned_addr learned my addr 172.21.15.7:0/1363210876 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.215+0000 7fd2f6d9c700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f00af980 0x7fd2f015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/1363210876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f00af980 msgr2=0x7fd2f015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5d9a700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f00af980 0x7fd2f015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/1363210876 >> v1:172.21.15.117:6789/0 conn(0x7fd2f0153960 legacy=0x7fd2f006fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/1363210876 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f015d0e0 con 0x7fd2f0071c80 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5599700 1 -- 172.21.15.7:0/1363210876 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072852649 0 0) 0x7fd2f015ce00 con 0x7fd2f0153960 2023-06-09T11:40:44.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5d9a700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0071c80 0x7fd2f0159bb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00098f0 tx=0x7fd2e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b250701e34a286c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:44.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5599700 1 -- 172.21.15.7:0/1363210876 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0011830 con 0x7fd2f0071c80 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5599700 1 -- 172.21.15.7:0/1363210876 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e0011990 con 0x7fd2f0071c80 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.216+0000 7fd2f5599700 1 -- 172.21.15.7:0/1363210876 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0011d20 con 0x7fd2f0071c80 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 -- 172.21.15.7:0/1363210876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0071c80 msgr2=0x7fd2f0159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0071c80 0x7fd2f0159bb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00098f0 tx=0x7fd2e0009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 -- 172.21.15.7:0/1363210876 shutdown_connections 2023-06-09T11:40:44.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f00af980 0x7fd2f015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/1363210876 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f0071c80 0x7fd2f0159bb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 -- 172.21.15.7:0/1363210876 >> 172.21.15.7:0/1363210876 conn(0x7fd2f00c9f00 msgr2=0x7fd2f00ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 -- 172.21.15.7:0/1363210876 shutdown_connections 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.217+0000 7fd2f87ff700 1 -- 172.21.15.7:0/1363210876 wait complete. 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.219+0000 7fd2f87ff700 1 Processor -- start 2023-06-09T11:40:44.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.219+0000 7fd2f87ff700 1 -- start start 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2f0071c80 0x7fd2f01f3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f0153960 0x7fd2f01eef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00b3030 con 0x7fd2f00af980 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f00b0c70 con 0x7fd2f0071c80 2023-06-09T11:40:44.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f01ef4a0 con 0x7fd2f0153960 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36170/0 (socket says 172.21.15.7:36170) 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/3482870683 learned_addr learned my addr 172.21.15.7:0/3482870683 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f659b700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2f0071c80 0x7fd2f01f3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f6d9c700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f0153960 0x7fd2f01eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f0153960 msgr2=0x7fd2f01eef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f0153960 0x7fd2f01eef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2f0071c80 msgr2=0x7fd2f01f3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2f0071c80 0x7fd2f01f3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 -- 172.21.15.7:0/3482870683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e0009550 con 0x7fd2f00af980 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.220+0000 7fd2f5d9a700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009f50 tx=0x7fd2e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.221+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0004300 con 0x7fd2f00af980 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.221+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e001f070 con 0x7fd2f00af980 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.221+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e00036a0 con 0x7fd2f00af980 2023-06-09T11:40:44.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.221+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f015d6a0 con 0x7fd2f00af980 2023-06-09T11:40:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.221+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f015d3c0 con 0x7fd2f00af980 2023-06-09T11:40:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.222+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f015d980 con 0x7fd2f00af980 2023-06-09T11:40:44.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.223+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2e0021030 con 0x7fd2f00af980 2023-06-09T11:40:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.224+0000 7fd2e6ffd700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 0x7fd2d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.224+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fd2e008ade0 con 0x7fd2f00af980 2023-06-09T11:40:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.224+0000 7fd2f659b700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 0x7fd2d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:44.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.225+0000 7fd2f659b700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 0x7fd2d406d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec000f60 tx=0x7fd2ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:44.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.228+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd2e0055970 con 0x7fd2f00af980 2023-06-09T11:40:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.482+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2f0074cf0 con 0x7fd2f00af980 2023-06-09T11:40:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.483+0000 7fd2e6ffd700 1 -- 172.21.15.7:0/3482870683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fd2e0006d80 con 0x7fd2f00af980 2023-06-09T11:40:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:44.488 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":134,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:42.744675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 msgr2=0x7fd2d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 0x7fd2d406d190 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec000f60 tx=0x7fd2ec009040 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 msgr2=0x7fd2f00b0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fd2e0009f50 tx=0x7fd2e0004750 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 shutdown_connections 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd2d406ace0 0x7fd2d406d190 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2f0153960 0x7fd2f01eef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2f0071c80 0x7fd2f01f3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 --2- 172.21.15.7:0/3482870683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2f00af980 0x7fd2f00b0730 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 >> 172.21.15.7:0/3482870683 conn(0x7fd2f00c9f00 msgr2=0x7fd2f0150730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:44.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 shutdown_connections 2023-06-09T11:40:44.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:44.485+0000 7fd2f87ff700 1 -- 172.21.15.7:0/3482870683 wait complete. 2023-06-09T11:40:44.492 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 134 2023-06-09T11:40:44.509 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 134, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 134, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:reconnect", "state_seq": 31 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:40:42.744675+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:40:46.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.717+0000 7fb450dff700 1 Processor -- start 2023-06-09T11:40:46.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.717+0000 7fb450dff700 1 -- start start 2023-06-09T11:40:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.718+0000 7fb450dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb44c151510 0x7fb44c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.718+0000 7fb450dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c1580a0 con 0x7fb44c15b380 2023-06-09T11:40:46.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.718+0000 7fb450dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb44c0c6180 con 0x7fb44c071230 2023-06-09T11:40:46.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.718+0000 7fb450dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb44c1518e0 con 0x7fb44c151510 2023-06-09T11:40:46.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb449d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb44c15b380 0x7fb44c070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50648/0 (socket says 172.21.15.7:50648) 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb449d9b700 1 -- 172.21.15.7:0/2038609023 learned_addr learned my addr 172.21.15.7:0/2038609023 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113096696 0 0) 0x7fb44c1580a0 con 0x7fb44c15b380 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44a59c700 1 --2- 172.21.15.7:0/2038609023 >> v2:172.21.15.117:3300/0 conn(0x7fb44c151510 0x7fb44c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4340035b0 con 0x7fb44c15b380 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4073341724 0 0) 0x7fb4340035b0 con 0x7fb44c15b380 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 >> v2:172.21.15.117:3300/0 conn(0x7fb44c151510 msgr2=0x7fb44c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:46.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 --2- 172.21.15.7:0/2038609023 >> v2:172.21.15.117:3300/0 conn(0x7fb44c151510 0x7fb44c0c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 >> v1:172.21.15.117:6789/0 conn(0x7fb44c071230 legacy=0x7fb44c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c157490 con 0x7fb44c15b380 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.719+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb440001670 con 0x7fb44c15b380 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 >> v1:172.21.15.7:6789/0 conn(0x7fb44c15b380 legacy=0x7fb44c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb434003a50 0x7fb434003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb434004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb434005200 0x7fb43400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1518e0 con 0x7fb434004850 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43400c0e0 con 0x7fb434005200 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43400c2a0 con 0x7fb434003a50 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb434004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44a59c700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb434003a50 0x7fb434003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44ad9d700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb434005200 0x7fb43400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 -- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb434003a50 msgr2=0x7fb434003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:46.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb434003a50 0x7fb434003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 -- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb434005200 msgr2=0x7fb43400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb434005200 0x7fb43400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb449d9b700 1 -- 172.21.15.7:0/2038609023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340035b0 con 0x7fb434004850 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.720+0000 7fb44ad9d700 1 -- 172.21.15.7:0/2038609023 reap_dead start 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb449d9b700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb434004cc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb440001eb0 tx=0x7fb434005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0640be08f02021b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440007b70 con 0x7fb434004850 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44000ccf0 con 0x7fb434004850 2023-06-09T11:40:46.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb44959a700 1 -- 172.21.15.7:0/2038609023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440005ba0 con 0x7fb434004850 2023-06-09T11:40:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 -- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 msgr2=0x7fb434004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb434004cc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb440001eb0 tx=0x7fb434005880 comp rx=0 tx=0).stop 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 -- 172.21.15.7:0/2038609023 shutdown_connections 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 --2- 172.21.15.7:0/2038609023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb434004cc0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 -- 172.21.15.7:0/2038609023 >> 172.21.15.7:0/2038609023 conn(0x7fb44c06a4d0 msgr2=0x7fb44c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.721+0000 7fb450dff700 1 -- 172.21.15.7:0/2038609023 shutdown_connections 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.722+0000 7fb450dff700 1 -- 172.21.15.7:0/2038609023 wait complete. 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.723+0000 7fb450dff700 1 Processor -- start 2023-06-09T11:40:46.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.723+0000 7fb450dff700 1 -- start start 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 0x7fb44c14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb44c0c0de0 0x7fb44c1e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1585b0 con 0x7fb434004850 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1f3100 con 0x7fb44c0c0de0 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.724+0000 7fb450dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c1563b0 con 0x7fb44c14c0c0 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36190/0 (socket says 172.21.15.7:36190) 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 -- 172.21.15.7:0/830648692 learned_addr learned my addr 172.21.15.7:0/830648692 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb449d9b700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 0x7fb44c14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44ad9d700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb44c0c0de0 0x7fb44c1e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 -- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 msgr2=0x7fb44c14e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 0x7fb44c14e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 -- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb44c0c0de0 msgr2=0x7fb44c1e4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb44c0c0de0 0x7fb44c1e4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 -- 172.21.15.7:0/830648692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c157a50 con 0x7fb434004850 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb449d9b700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 0x7fb44c14e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.725+0000 7fb44a59c700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0119d0 tx=0x7fb43c011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.726+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c011ec0 con 0x7fb434004850 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.726+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c004240 con 0x7fb434004850 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.726+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c00f730 con 0x7fb434004850 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.726+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c157770 con 0x7fb434004850 2023-06-09T11:40:46.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.726+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c157d30 con 0x7fb434004850 2023-06-09T11:40:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.727+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c074cf0 con 0x7fb434004850 2023-06-09T11:40:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.728+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb43c00f890 con 0x7fb434004850 2023-06-09T11:40:46.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.729+0000 7fb43affd700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 0x7fb42c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:46.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.729+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fb43c089e40 con 0x7fb434004850 2023-06-09T11:40:46.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.729+0000 7fb449d9b700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 0x7fb42c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:46.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.730+0000 7fb449d9b700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 0x7fb42c06d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb440015040 tx=0x7fb440007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:46.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.734+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb43c054940 con 0x7fb434004850 2023-06-09T11:40:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.987+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb44c0b10b0 con 0x7fb434004850 2023-06-09T11:40:46.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.987+0000 7fb43affd700 1 -- 172.21.15.7:0/830648692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 134 v134) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fb43c019090 con 0x7fb434004850 2023-06-09T11:40:46.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:46.992 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":134,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":134,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:42.744675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:reconnect","state_seq":31,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 msgr2=0x7fb42c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 0x7fb42c06d190 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb440015040 tx=0x7fb440007000 comp rx=0 tx=0).stop 2023-06-09T11:40:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 msgr2=0x7fb44c155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0119d0 tx=0x7fb43c011ce0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 shutdown_connections 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb44c0c0de0 0x7fb44c1e4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb434004850 0x7fb44c155e70 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb42c06ace0 0x7fb42c06d190 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 --2- 172.21.15.7:0/830648692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb44c14c0c0 0x7fb44c14e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 >> 172.21.15.7:0/830648692 conn(0x7fb44c06a4d0 msgr2=0x7fb44c06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 shutdown_connections 2023-06-09T11:40:46.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:46.990+0000 7fb450dff700 1 -- 172.21.15.7:0/830648692 wait complete. 2023-06-09T11:40:46.996 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 134 2023-06-09T11:40:47.012 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:48.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:48.655 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:40:49.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.214+0000 7fb708fff700 1 Processor -- start 2023-06-09T11:40:49.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.214+0000 7fb708fff700 1 -- start start 2023-06-09T11:40:49.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.215+0000 7fb708fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb7040c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb708fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb70415b380 0x7fb704070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb708fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb704071060 0x7fb704158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb708fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7040714e0 con 0x7fb704071060 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb708fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7041518e0 con 0x7fb70415b380 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb708fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041580a0 con 0x7fb704151510 2023-06-09T11:40:49.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb704071060 0x7fb704158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb704071060 0x7fb704158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36204/0 (socket says 172.21.15.7:36204) 2023-06-09T11:40:49.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 -- 172.21.15.7:0/1150495995 learned_addr learned my addr 172.21.15.7:0/1150495995 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb70259c700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb7040c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb6fbfff700 1 --2- 172.21.15.7:0/1150495995 >> v2:172.21.15.117:3300/0 conn(0x7fb70415b380 0x7fb704070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 -- 172.21.15.7:0/1150495995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 msgr2=0x7fb7040c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb7040c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 -- 172.21.15.7:0/1150495995 >> v2:172.21.15.117:3300/0 conn(0x7fb70415b380 msgr2=0x7fb704070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 --2- 172.21.15.7:0/1150495995 >> v2:172.21.15.117:3300/0 conn(0x7fb70415b380 0x7fb704070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.216+0000 7fb702d9d700 1 -- 172.21.15.7:0/1150495995 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040c6180 con 0x7fb704071060 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb702d9d700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb704158870 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb7040cc290 tx=0x7fb6fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf05cbf7cf285d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb701d9b700 1 -- 172.21.15.7:0/1150495995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00e070 con 0x7fb704071060 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb701d9b700 1 -- 172.21.15.7:0/1150495995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc00b9c0 con 0x7fb704071060 2023-06-09T11:40:49.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb701d9b700 1 -- 172.21.15.7:0/1150495995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc00bd70 con 0x7fb704071060 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb708fff700 1 -- 172.21.15.7:0/1150495995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 msgr2=0x7fb704158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.217+0000 7fb708fff700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb704158870 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fb7040cc290 tx=0x7fb6fc00b330 comp rx=0 tx=0).stop 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 -- 172.21.15.7:0/1150495995 shutdown_connections 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb704158870 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 --2- 172.21.15.7:0/1150495995 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb7040c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 --2- 172.21.15.7:0/1150495995 >> v2:172.21.15.117:3300/0 conn(0x7fb70415b380 0x7fb704070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 -- 172.21.15.7:0/1150495995 >> 172.21.15.7:0/1150495995 conn(0x7fb70406a4d0 msgr2=0x7fb70406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 -- 172.21.15.7:0/1150495995 shutdown_connections 2023-06-09T11:40:49.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.218+0000 7fb708fff700 1 -- 172.21.15.7:0/1150495995 wait complete. 2023-06-09T11:40:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.219+0000 7fb708fff700 1 Processor -- start 2023-06-09T11:40:49.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.220+0000 7fb708fff700 1 -- start start 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb70414ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 0x7fb70414b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041f2d20 con 0x7fb704071060 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704159110 con 0x7fb70415b380 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb708fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7041f2fe0 con 0x7fb704151510 2023-06-09T11:40:49.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36220/0 (socket says 172.21.15.7:36220) 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 -- 172.21.15.7:0/528449920 learned_addr learned my addr 172.21.15.7:0/528449920 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb6fbfff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb70414ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb702d9d700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 0x7fb70414b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 -- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 msgr2=0x7fb70414ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb70414ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 -- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 msgr2=0x7fb70414b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:49.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 0x7fb70414b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb70259c700 1 -- 172.21.15.7:0/528449920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6fc009b30 con 0x7fb704071060 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.221+0000 7fb702d9d700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 0x7fb70414b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb70259c700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400e810 tx=0x7fb6f400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4013610 con 0x7fb704071060 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4013790 con 0x7fb704071060 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb704157770 con 0x7fb704071060 2023-06-09T11:40:49.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4013b40 con 0x7fb704071060 2023-06-09T11:40:49.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.222+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb704157490 con 0x7fb704071060 2023-06-09T11:40:49.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.223+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb704157a50 con 0x7fb704071060 2023-06-09T11:40:49.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.224+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6f4013ca0 con 0x7fb704071060 2023-06-09T11:40:49.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.225+0000 7fb6faffd700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:49.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.225+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fb6f4089690 con 0x7fb704071060 2023-06-09T11:40:49.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.225+0000 7fb6fbfff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:49.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.226+0000 7fb6fbfff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb70415a950 tx=0x7fb6ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:49.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.229+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb6f4057b90 con 0x7fb704071060 2023-06-09T11:40:49.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.478+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb704074cf0 con 0x7fb704071060 2023-06-09T11:40:49.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.478+0000 7fb6faffd700 1 -- 172.21.15.7:0/528449920 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5463 (secure 0 0 0) 0x7fb6f4009440 con 0x7fb704071060 2023-06-09T11:40:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:49.483 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:48.758343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:49.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.480+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 msgr2=0x7fb6e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.480+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb70415a950 tx=0x7fb6ec008040 comp rx=0 tx=0).stop 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.480+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 msgr2=0x7fb7041f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.480+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400e810 tx=0x7fb6f400eb20 comp rx=0 tx=0).stop 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 shutdown_connections 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb70415b380 0x7fb70414b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb704071060 0x7fb7041f23d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6e406ace0 0x7fb6e406d190 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 --2- 172.21.15.7:0/528449920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb704151510 0x7fb70414ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 >> 172.21.15.7:0/528449920 conn(0x7fb70406a4d0 msgr2=0x7fb70414d020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 shutdown_connections 2023-06-09T11:40:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:49.481+0000 7fb708fff700 1 -- 172.21.15.7:0/528449920 wait complete. 2023-06-09T11:40:49.486 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 135 2023-06-09T11:40:49.502 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:51.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:51.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.711+0000 7f96a33ff700 1 Processor -- start 2023-06-09T11:40:51.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.711+0000 7f96a33ff700 1 -- start start 2023-06-09T11:40:51.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a33ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f969c06d640 0x7f969c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:51.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a33ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c15cac0 con 0x7f969c0c6280 2023-06-09T11:40:51.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a33ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f969c0c37d0 con 0x7f969c06a4d0 2023-06-09T11:40:51.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a33ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f969c0c8870 con 0x7f969c06d640 2023-06-09T11:40:51.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f969c06d640 0x7f969c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:51.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a119b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f969c0c6280 0x7f969c0737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50660/0 (socket says 172.21.15.7:50660) 2023-06-09T11:40:51.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.712+0000 7f96a119b700 1 -- 172.21.15.7:0/461491555 learned_addr learned my addr 172.21.15.7:0/461491555 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:51.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f96a199c700 1 -- 172.21.15.7:0/461491555 >> v1:172.21.15.117:6789/0 conn(0x7f969c06a4d0 legacy=0x7f969c06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:51.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f96a199c700 1 -- 172.21.15.7:0/461491555 >> v1:172.21.15.7:6789/0 conn(0x7f969c0c6280 legacy=0x7f969c0737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f96a199c700 1 -- 172.21.15.7:0/461491555 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c15cd30 con 0x7f969c06d640 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f96a199c700 1 --2- 172.21.15.7:0/461491555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c06d640 0x7f969c0c76b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b6307e0bb29df server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f9693fff700 1 -- 172.21.15.7:0/461491555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969400e070 con 0x7f969c06d640 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f9693fff700 1 -- 172.21.15.7:0/461491555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969400bb90 con 0x7f969c06d640 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.713+0000 7f9693fff700 1 -- 172.21.15.7:0/461491555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694003a20 con 0x7f969c06d640 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 -- 172.21.15.7:0/461491555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c06d640 msgr2=0x7f969c0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 --2- 172.21.15.7:0/461491555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c06d640 0x7f969c0c76b0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f9694009d90 tx=0x7f969400b330 comp rx=0 tx=0).stop 2023-06-09T11:40:51.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 -- 172.21.15.7:0/461491555 shutdown_connections 2023-06-09T11:40:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 --2- 172.21.15.7:0/461491555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c06d640 0x7f969c0c76b0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 -- 172.21.15.7:0/461491555 >> 172.21.15.7:0/461491555 conn(0x7f969c06f980 msgr2=0x7f969c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 -- 172.21.15.7:0/461491555 shutdown_connections 2023-06-09T11:40:51.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.714+0000 7f96a33ff700 1 -- 172.21.15.7:0/461491555 wait complete. 2023-06-09T11:40:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.716+0000 7f96a33ff700 1 Processor -- start 2023-06-09T11:40:51.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.717+0000 7f96a33ff700 1 -- start start 2023-06-09T11:40:51.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 0x7f969c141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:51.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f969c06d640 0x7f969c13ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c0ca620 con 0x7f969c0c6280 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f3170 con 0x7f969c06d640 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c1f23b0 con 0x7f969c06a4d0 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a199c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a119b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 0x7f969c141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:51.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a199c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36240/0 (socket says 172.21.15.7:36240) 2023-06-09T11:40:51.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a199c700 1 -- 172.21.15.7:0/4264809205 learned_addr learned my addr 172.21.15.7:0/4264809205 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:51.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.718+0000 7f96a099a700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f969c06d640 0x7f969c13ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 -- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 msgr2=0x7f969c141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 0x7f969c141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 -- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f969c06d640 msgr2=0x7f969c13ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f969c06d640 0x7f969c13ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 -- 172.21.15.7:0/4264809205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9694009a70 con 0x7f969c0c6280 2023-06-09T11:40:51.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a119b700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 0x7f969c141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a199c700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f969c068c30 tx=0x7f9694004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694012630 con 0x7f969c0c6280 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96940127b0 con 0x7f969c0c6280 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9694012b60 con 0x7f969c0c6280 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.719+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969c15d2f0 con 0x7f969c0c6280 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.720+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f969c15d010 con 0x7f969c0c6280 2023-06-09T11:40:51.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.721+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f969c15d5d0 con 0x7f969c0c6280 2023-06-09T11:40:51.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.722+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9694012cc0 con 0x7f969c0c6280 2023-06-09T11:40:51.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.722+0000 7f9691ffb700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 0x7f968006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:51.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.723+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f969408bf00 con 0x7f969c0c6280 2023-06-09T11:40:51.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.723+0000 7f96a119b700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 0x7f968006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:51.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.723+0000 7f96a119b700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 0x7f968006d1c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f60 tx=0x7f9698007f10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:51.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.727+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9694056a00 con 0x7f969c0c6280 2023-06-09T11:40:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.982+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f969c074cf0 con 0x7f969c0c6280 2023-06-09T11:40:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.982+0000 7f9691ffb700 1 -- 172.21.15.7:0/4264809205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f969405a020 con 0x7f969c0c6280 2023-06-09T11:40:51.987 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:51.987 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:48.758343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:51.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 msgr2=0x7f968006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 0x7f968006d1c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f60 tx=0x7f9698007f10 comp rx=0 tx=0).stop 2023-06-09T11:40:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 msgr2=0x7f969c1eeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f969c068c30 tx=0x7f9694004ad0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 shutdown_connections 2023-06-09T11:40:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f969c0c6280 0x7f969c1eeda0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f968006ad10 0x7f968006d1c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f969c06a4d0 0x7f969c141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 --2- 172.21.15.7:0/4264809205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f969c06d640 0x7f969c13ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 >> 172.21.15.7:0/4264809205 conn(0x7f969c06f980 msgr2=0x7f969c0c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 shutdown_connections 2023-06-09T11:40:51.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:51.985+0000 7f96a33ff700 1 -- 172.21.15.7:0/4264809205 wait complete. 2023-06-09T11:40:51.993 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 135 2023-06-09T11:40:52.007 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:54.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:54.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.215+0000 7f32d33ff700 1 Processor -- start 2023-06-09T11:40:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.216+0000 7f32d33ff700 1 -- start start 2023-06-09T11:40:54.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d33ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f32cc151510 0x7f32cc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d33ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc1580a0 con 0x7f32cc15b380 2023-06-09T11:40:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d33ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc0c6180 con 0x7f32cc071230 2023-06-09T11:40:54.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d33ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc1518e0 con 0x7f32cc151510 2023-06-09T11:40:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d099a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f32cc15b380 0x7f32cc070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49280/0 (socket says 172.21.15.7:49280) 2023-06-09T11:40:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d099a700 1 -- 172.21.15.7:0/2094565335 learned_addr learned my addr 172.21.15.7:0/2094565335 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32d119b700 1 --2- 172.21.15.7:0/2094565335 >> v2:172.21.15.117:3300/0 conn(0x7f32cc151510 0x7f32cc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:54.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.217+0000 7f32c3fff700 1 -- 172.21.15.7:0/2094565335 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919587342 0 0) 0x7f32cc1580a0 con 0x7f32cc15b380 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32c3fff700 1 -- 172.21.15.7:0/2094565335 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b80035b0 con 0x7f32cc15b380 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32d119b700 1 -- 172.21.15.7:0/2094565335 >> v1:172.21.15.117:6789/0 conn(0x7f32cc071230 legacy=0x7f32cc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32d119b700 1 -- 172.21.15.7:0/2094565335 >> v1:172.21.15.7:6789/0 conn(0x7f32cc15b380 legacy=0x7f32cc070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32d119b700 1 -- 172.21.15.7:0/2094565335 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc157490 con 0x7f32cc151510 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32d119b700 1 --2- 172.21.15.7:0/2094565335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc151510 0x7f32cc0c5b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009840 tx=0x7f32c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4215be443aa81cb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32c3fff700 1 -- 172.21.15.7:0/2094565335 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8011830 con 0x7f32cc151510 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32c3fff700 1 -- 172.21.15.7:0/2094565335 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8011990 con 0x7f32cc151510 2023-06-09T11:40:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.218+0000 7f32c3fff700 1 -- 172.21.15.7:0/2094565335 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8011d20 con 0x7f32cc151510 2023-06-09T11:40:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 -- 172.21.15.7:0/2094565335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc151510 msgr2=0x7f32cc0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 --2- 172.21.15.7:0/2094565335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc151510 0x7f32cc0c5b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009840 tx=0x7f32c8009b50 comp rx=0 tx=0).stop 2023-06-09T11:40:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 -- 172.21.15.7:0/2094565335 shutdown_connections 2023-06-09T11:40:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 --2- 172.21.15.7:0/2094565335 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc151510 0x7f32cc0c5b40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 -- 172.21.15.7:0/2094565335 >> 172.21.15.7:0/2094565335 conn(0x7f32cc06a4d0 msgr2=0x7f32cc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 -- 172.21.15.7:0/2094565335 shutdown_connections 2023-06-09T11:40:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.219+0000 7f32d33ff700 1 -- 172.21.15.7:0/2094565335 wait complete. 2023-06-09T11:40:54.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.221+0000 7f32d33ff700 1 Processor -- start 2023-06-09T11:40:54.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.221+0000 7f32d33ff700 1 -- start start 2023-06-09T11:40:54.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:54.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32cc151510 0x7f32cc0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc15b380 0x7f32cc1ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc158b50 con 0x7f32cc071230 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc1f13e0 con 0x7f32cc15b380 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc158870 con 0x7f32cc151510 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55314/0 (socket says 172.21.15.7:55314) 2023-06-09T11:40:54.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.222+0000 7f32d119b700 1 -- 172.21.15.7:0/3220289529 learned_addr learned my addr 172.21.15.7:0/3220289529 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:54.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d199c700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc15b380 0x7f32cc1ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d099a700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32cc151510 0x7f32cc0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 -- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32cc151510 msgr2=0x7f32cc0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32cc151510 0x7f32cc0c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 -- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc15b380 msgr2=0x7f32cc1ecf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc15b380 0x7f32cc1ecf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 -- 172.21.15.7:0/3220289529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c80094a0 con 0x7f32cc071230 2023-06-09T11:40:54.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d119b700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009fd0 tx=0x7f32c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c80038a0 con 0x7f32cc071230 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c801f070 con 0x7f32cc071230 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8003c40 con 0x7f32cc071230 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.223+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc157a50 con 0x7f32cc071230 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.224+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc157770 con 0x7f32cc071230 2023-06-09T11:40:54.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.225+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc157d30 con 0x7f32cc071230 2023-06-09T11:40:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.225+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f32c8026480 con 0x7f32cc071230 2023-06-09T11:40:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.226+0000 7f32c1ffb700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 0x7f32b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.226+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f32c8021070 con 0x7f32cc071230 2023-06-09T11:40:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.226+0000 7f32d099a700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 0x7f32b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.227+0000 7f32d099a700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 0x7f32b006d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f32cc14c660 tx=0x7f32bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.230+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f32c8055f00 con 0x7f32cc071230 2023-06-09T11:40:54.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.482+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32cc1f1230 con 0x7f32cc071230 2023-06-09T11:40:54.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.482+0000 7f32c1ffb700 1 -- 172.21.15.7:0/3220289529 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 135 v135) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f32c8059520 con 0x7f32cc071230 2023-06-09T11:40:54.486 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:54.486 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":135,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":135,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:48.758343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:rejoin","state_seq":33,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 msgr2=0x7f32b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 0x7f32b006d190 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f32cc14c660 tx=0x7f32bc009040 comp rx=0 tx=0).stop 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 msgr2=0x7f32cc0ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009fd0 tx=0x7f32c8004750 comp rx=0 tx=0).stop 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 shutdown_connections 2023-06-09T11:40:54.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32cc15b380 0x7f32cc1ecf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32cc071230 0x7f32cc0ca9b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f32b006ace0 0x7f32b006d190 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 --2- 172.21.15.7:0/3220289529 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32cc151510 0x7f32cc0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 >> 172.21.15.7:0/3220289529 conn(0x7f32cc06a4d0 msgr2=0x7f32cc14b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 shutdown_connections 2023-06-09T11:40:54.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:54.485+0000 7f32d33ff700 1 -- 172.21.15.7:0/3220289529 wait complete. 2023-06-09T11:40:54.491 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 135 2023-06-09T11:40:54.507 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:40:56.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.714+0000 7f4b091ff700 1 Processor -- start 2023-06-09T11:40:56.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.714+0000 7f4b091ff700 1 -- start start 2023-06-09T11:40:56.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.715+0000 7f4b091ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 0x7f4b040b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.715+0000 7f4b091ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b040cb9a0 0x7f4b04159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b091ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b04159b20 con 0x7f4b0415cd40 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b091ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b040cac80 con 0x7f4b040cb9a0 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b091ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b041597d0 con 0x7f4b040cb5d0 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b040cb9a0 0x7f4b04159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4b040cb9a0 0x7f4b04159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55326/0 (socket says 172.21.15.7:55326) 2023-06-09T11:40:56.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 -- 172.21.15.7:0/3492099542 learned_addr learned my addr 172.21.15.7:0/3492099542 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:56.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b0359e700 1 --1- 172.21.15.7:0/3492099542 >> v1:172.21.15.7:6789/0 conn(0x7f4b0415cd40 0x7f4b040cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49282/0 (socket says 172.21.15.7:49282) 2023-06-09T11:40:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 0x7f4b040b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 -- 172.21.15.7:0/3492099542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 msgr2=0x7f4b040b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 0x7f4b040b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 -- 172.21.15.7:0/3492099542 >> v1:172.21.15.7:6789/0 conn(0x7f4b0415cd40 legacy=0x7f4b040cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4afbfff700 1 -- 172.21.15.7:0/3492099542 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04071ab0 con 0x7f4b040cb9a0 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b0259c700 1 -- 172.21.15.7:0/3492099542 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506324916 0 0) 0x7f4b04159b20 con 0x7f4b0415cd40 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.716+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 0x7f4b040b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4afbfff700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b04159fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0098f0 tx=0x7f4aec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ef176c5e1bbb7c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b0259c700 1 -- 172.21.15.7:0/3492099542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00b8a0 con 0x7f4b040cb9a0 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b0259c700 1 -- 172.21.15.7:0/3492099542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec00ba00 con 0x7f4b040cb9a0 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b0259c700 1 -- 172.21.15.7:0/3492099542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00bd90 con 0x7f4b040cb9a0 2023-06-09T11:40:56.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 -- 172.21.15.7:0/3492099542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 msgr2=0x7f4b04159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b04159fb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0098f0 tx=0x7f4aec009c00 comp rx=0 tx=0).stop 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 -- 172.21.15.7:0/3492099542 shutdown_connections 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b040cb5d0 0x7f4b040b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 --2- 172.21.15.7:0/3492099542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b04159fb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.717+0000 7f4b091ff700 1 -- 172.21.15.7:0/3492099542 >> 172.21.15.7:0/3492099542 conn(0x7f4b04070af0 msgr2=0x7f4b04070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.718+0000 7f4b091ff700 1 -- 172.21.15.7:0/3492099542 shutdown_connections 2023-06-09T11:40:56.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.718+0000 7f4b091ff700 1 -- 172.21.15.7:0/3492099542 wait complete. 2023-06-09T11:40:56.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.720+0000 7f4b091ff700 1 Processor -- start 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.720+0000 7f4b091ff700 1 -- start start 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 0x7f4b04153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 0x7f4b041f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040cf770 con 0x7f4b040cb9a0 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b040cee80 con 0x7f4b040cb5d0 2023-06-09T11:40:56.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.721+0000 7f4b091ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04158700 con 0x7f4b0415cd40 2023-06-09T11:40:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55328/0 (socket says 172.21.15.7:55328) 2023-06-09T11:40:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 -- 172.21.15.7:0/2798647943 learned_addr learned my addr 172.21.15.7:0/2798647943 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4b0359e700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 0x7f4b041f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 0x7f4b04153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 -- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 msgr2=0x7f4b041f1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 0x7f4b041f1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 -- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 msgr2=0x7f4b04153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 0x7f4b04153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 -- 172.21.15.7:0/2798647943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec009550 con 0x7f4b040cb9a0 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 0x7f4b04153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4b0359e700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 0x7f4b041f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.722+0000 7f4afbfff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000c00 tx=0x7f4aec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.723+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec004550 con 0x7f4b040cb9a0 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.723+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec01f070 con 0x7f4b040cb9a0 2023-06-09T11:40:56.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.723+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0038e0 con 0x7f4b040cb9a0 2023-06-09T11:40:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.723+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0415d4f0 con 0x7f4b040cb9a0 2023-06-09T11:40:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.723+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0415d210 con 0x7f4b040cb9a0 2023-06-09T11:40:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.724+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b0415d7d0 con 0x7f4b040cb9a0 2023-06-09T11:40:56.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.725+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4aec003a40 con 0x7f4b040cb9a0 2023-06-09T11:40:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.726+0000 7f4afb7fe700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 0x7f4ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.726+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f4aec021070 con 0x7f4b040cb9a0 2023-06-09T11:40:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.726+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 0x7f4ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:56.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.727+0000 7f4b02d9d700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 0x7f4ae406d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4af40099f0 tx=0x7f4af4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.730+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4aec055b00 con 0x7f4b040cb9a0 2023-06-09T11:40:56.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.983+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b041f0b00 con 0x7f4b040cb9a0 2023-06-09T11:40:56.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.984+0000 7f4afb7fe700 1 -- 172.21.15.7:0/2798647943 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 136 v136) v1 ==== 95+0+5469 (secure 0 0 0) 0x7f4aec059120 con 0x7f4b040cb9a0 2023-06-09T11:40:56.988 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:56.988 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":136,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":136,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:54.779580+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:clientreplay","state_seq":35,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:56.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.986+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 msgr2=0x7f4ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 0x7f4ae406d190 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4af40099f0 tx=0x7f4af4008040 comp rx=0 tx=0).stop 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 msgr2=0x7f4b041f0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000c00 tx=0x7f4aec004750 comp rx=0 tx=0).stop 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 shutdown_connections 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4ae406ace0 0x7f4ae406d190 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b0415cd40 0x7f4b041f1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b040cb5d0 0x7f4b04153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 --2- 172.21.15.7:0/2798647943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b040cb9a0 0x7f4b041f0e10 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 >> 172.21.15.7:0/2798647943 conn(0x7f4b04070af0 msgr2=0x7f4b040b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 shutdown_connections 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:56.987+0000 7f4b091ff700 1 -- 172.21.15.7:0/2798647943 wait complete. 2023-06-09T11:40:56.992 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 136 2023-06-09T11:40:57.009 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 136, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 136, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_5994": { "addr": "172.21.15.7:6839/2829758995", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 2829758995, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 2829758995, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 5994, "incarnation": 73, "join_fscid": -1, "name": "e", "rank": 2, "state": "up:active", "state_seq": 41 }, "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:clientreplay", "state_seq": 35 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:40:54.779580+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 5994 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:40:59.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:40:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.218+0000 7f306e7ff700 1 Processor -- start 2023-06-09T11:40:59.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.218+0000 7f306e7ff700 1 -- start start 2023-06-09T11:40:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306e7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f30680cb5d0 0x7f30680b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306e7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f306815cd40 0x7f30680cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306e7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3068159b20 con 0x7f30680cb9a0 2023-06-09T11:40:59.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306e7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30680cac80 con 0x7f306815cd40 2023-06-09T11:40:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306e7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30681597d0 con 0x7f30680cb5d0 2023-06-09T11:40:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f30677fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f30680cb9a0 0x7f3068159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49294/0 (socket says 172.21.15.7:49294) 2023-06-09T11:40:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f30677fe700 1 -- 172.21.15.7:0/1094281732 learned_addr learned my addr 172.21.15.7:0/1094281732 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:40:59.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f306cd9c700 1 --2- 172.21.15.7:0/1094281732 >> v2:172.21.15.7:3300/0 conn(0x7f306815cd40 0x7f30680cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.220+0000 7f3067fff700 1 --2- 172.21.15.7:0/1094281732 >> v2:172.21.15.117:3300/0 conn(0x7f30680cb5d0 0x7f30680b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f3066ffd700 1 -- 172.21.15.7:0/1094281732 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353064604 0 0) 0x7f3068159b20 con 0x7f30680cb9a0 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f3066ffd700 1 -- 172.21.15.7:0/1094281732 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30500035b0 con 0x7f30680cb9a0 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f306cd9c700 1 -- 172.21.15.7:0/1094281732 >> v2:172.21.15.117:3300/0 conn(0x7f30680cb5d0 msgr2=0x7f30680b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f306cd9c700 1 --2- 172.21.15.7:0/1094281732 >> v2:172.21.15.117:3300/0 conn(0x7f30680cb5d0 0x7f30680b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f306cd9c700 1 -- 172.21.15.7:0/1094281732 >> v1:172.21.15.7:6789/0 conn(0x7f30680cb9a0 legacy=0x7f3068159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f306cd9c700 1 -- 172.21.15.7:0/1094281732 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068071ab0 con 0x7f306815cd40 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f306cd9c700 1 --2- 172.21.15.7:0/1094281732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30680cbea0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f306808d420 tx=0x7f306000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a37241d0b722e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:59.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f3066ffd700 1 -- 172.21.15.7:0/1094281732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000e070 con 0x7f306815cd40 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.221+0000 7f3066ffd700 1 -- 172.21.15.7:0/1094281732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000bb20 con 0x7f306815cd40 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f3066ffd700 1 -- 172.21.15.7:0/1094281732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060012430 con 0x7f306815cd40 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 -- 172.21.15.7:0/1094281732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 msgr2=0x7f30680cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 --2- 172.21.15.7:0/1094281732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30680cbea0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f306808d420 tx=0x7f306000b330 comp rx=0 tx=0).stop 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 -- 172.21.15.7:0/1094281732 shutdown_connections 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 --2- 172.21.15.7:0/1094281732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30680cbea0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 --2- 172.21.15.7:0/1094281732 >> v2:172.21.15.117:3300/0 conn(0x7f30680cb5d0 0x7f30680b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 -- 172.21.15.7:0/1094281732 >> 172.21.15.7:0/1094281732 conn(0x7f3068070af0 msgr2=0x7f3068070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 -- 172.21.15.7:0/1094281732 shutdown_connections 2023-06-09T11:40:59.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.222+0000 7f306e7ff700 1 -- 172.21.15.7:0/1094281732 wait complete. 2023-06-09T11:40:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.224+0000 7f306e7ff700 1 Processor -- start 2023-06-09T11:40:59.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.224+0000 7f306e7ff700 1 -- start start 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 0x7f30681efa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30680cb9a0 0x7f30681f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30680b1330 con 0x7f306815cd40 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30681f2ad0 con 0x7f30680cb5d0 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306e7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680cfd30 con 0x7f30680cb9a0 2023-06-09T11:40:59.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55342/0 (socket says 172.21.15.7:55342) 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f306cd9c700 1 -- 172.21.15.7:0/2365505845 learned_addr learned my addr 172.21.15.7:0/2365505845 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f3067fff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 0x7f30681efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.225+0000 7f30677fe700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30680cb9a0 0x7f30681f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 -- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30680cb9a0 msgr2=0x7f30681f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30680cb9a0 0x7f30681f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 -- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 msgr2=0x7f30681efa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 0x7f30681efa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 -- 172.21.15.7:0/2365505845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060009a70 con 0x7f306815cd40 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f3067fff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 0x7f30681efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306cd9c700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3060013000 tx=0x7f3060004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060012a70 con 0x7f306815cd40 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3060020070 con 0x7f306815cd40 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060012e30 con 0x7f306815cd40 2023-06-09T11:40:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306815d4f0 con 0x7f306815cd40 2023-06-09T11:40:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.226+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306815d210 con 0x7f306815cd40 2023-06-09T11:40:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.227+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306815d7d0 con 0x7f306815cd40 2023-06-09T11:40:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.228+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3060004040 con 0x7f306815cd40 2023-06-09T11:40:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.229+0000 7f3064ff9700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 0x7f305406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:40:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.229+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f3060022070 con 0x7f306815cd40 2023-06-09T11:40:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.229+0000 7f3067fff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 0x7f305406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:40:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.230+0000 7f3067fff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 0x7f305406d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3058009c30 tx=0x7f3058008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:40:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.234+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3060059ca0 con 0x7f306815cd40 2023-06-09T11:40:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.486+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30680cfea0 con 0x7f306815cd40 2023-06-09T11:40:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.487+0000 7f3064ff9700 1 -- 172.21.15.7:0/2365505845 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 137 v137) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f3060059830 con 0x7f306815cd40 2023-06-09T11:40:59.491 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:40:59.491 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":137,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":137,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:40:58.790633+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:clientreplay","state_seq":35,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:40:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.489+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 msgr2=0x7f305406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.489+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 0x7f305406d190 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3058009c30 tx=0x7f3058008040 comp rx=0 tx=0).stop 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.489+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 msgr2=0x7f30681f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.489+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3060013000 tx=0x7f3060004ad0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 shutdown_connections 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306815cd40 0x7f30681f22b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f30680cb5d0 0x7f30681efa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f305406ace0 0x7f305406d190 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 --2- 172.21.15.7:0/2365505845 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30680cb9a0 0x7f30681f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:40:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 >> 172.21.15.7:0/2365505845 conn(0x7f3068070af0 msgr2=0x7f3068158ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:40:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 shutdown_connections 2023-06-09T11:40:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:40:59.490+0000 7f306e7ff700 1 -- 172.21.15.7:0/2365505845 wait complete. 2023-06-09T11:40:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 137 2023-06-09T11:40:59.512 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:41:01.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:41:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.720+0000 7fb988fff700 1 Processor -- start 2023-06-09T11:41:01.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.720+0000 7fb988fff700 1 -- start start 2023-06-09T11:41:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb988fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb984153880 0x7fb98415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb988fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9840c9f40 0x7fb98415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb988fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb98415a500 con 0x7fb98415b2d0 2023-06-09T11:41:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb988fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9840c5f10 con 0x7fb984153880 2023-06-09T11:41:01.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb988fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98415da00 con 0x7fb9840c9f40 2023-06-09T11:41:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb984153880 0x7fb98415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb984153880 0x7fb98415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55346/0 (socket says 172.21.15.7:55346) 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 -- 172.21.15.7:0/885317922 learned_addr learned my addr 172.21.15.7:0/885317922 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb97bfff700 1 --2- 172.21.15.7:0/885317922 >> v2:172.21.15.117:3300/0 conn(0x7fb9840c9f40 0x7fb98415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 -- 172.21.15.7:0/885317922 >> v2:172.21.15.117:3300/0 conn(0x7fb9840c9f40 msgr2=0x7fb98415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 --2- 172.21.15.7:0/885317922 >> v2:172.21.15.117:3300/0 conn(0x7fb9840c9f40 0x7fb98415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 -- 172.21.15.7:0/885317922 >> v1:172.21.15.117:6789/0 conn(0x7fb98415b2d0 legacy=0x7fb9840c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.722+0000 7fb98259c700 1 -- 172.21.15.7:0/885317922 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9841596e0 con 0x7fb984153880 2023-06-09T11:41:01.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb98259c700 1 --2- 172.21.15.7:0/885317922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb984153880 0x7fb98415d4c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb9840cce30 tx=0x7fb974009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c76ba96bbcd4589e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb981d9b700 1 -- 172.21.15.7:0/885317922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974011830 con 0x7fb984153880 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb981d9b700 1 -- 172.21.15.7:0/885317922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb974011990 con 0x7fb984153880 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb981d9b700 1 -- 172.21.15.7:0/885317922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974011d20 con 0x7fb984153880 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb988fff700 1 -- 172.21.15.7:0/885317922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb984153880 msgr2=0x7fb98415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.723+0000 7fb988fff700 1 --2- 172.21.15.7:0/885317922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb984153880 0x7fb98415d4c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb9840cce30 tx=0x7fb974009bd0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 -- 172.21.15.7:0/885317922 shutdown_connections 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 --2- 172.21.15.7:0/885317922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb984153880 0x7fb98415d4c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 --2- 172.21.15.7:0/885317922 >> v2:172.21.15.117:3300/0 conn(0x7fb9840c9f40 0x7fb98415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 -- 172.21.15.7:0/885317922 >> 172.21.15.7:0/885317922 conn(0x7fb9840737f0 msgr2=0x7fb9840720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 -- 172.21.15.7:0/885317922 shutdown_connections 2023-06-09T11:41:01.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.724+0000 7fb988fff700 1 -- 172.21.15.7:0/885317922 wait complete. 2023-06-09T11:41:01.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.725+0000 7fb988fff700 1 Processor -- start 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.726+0000 7fb988fff700 1 -- start start 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.726+0000 7fb988fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.726+0000 7fb988fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb984153880 0x7fb98414b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb988fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb98415b2d0 0x7fb9841ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb988fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9840b3560 con 0x7fb9840c9f40 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb988fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841f3470 con 0x7fb98415b2d0 2023-06-09T11:41:01.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb988fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9841f2860 con 0x7fb984153880 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55348/0 (socket says 172.21.15.7:55348) 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 -- 172.21.15.7:0/4170932884 learned_addr learned my addr 172.21.15.7:0/4170932884 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb97bfff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb984153880 0x7fb98414b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb982d9d700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb98415b2d0 0x7fb9841ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 -- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb984153880 msgr2=0x7fb98414b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb984153880 0x7fb98414b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 -- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb98415b2d0 msgr2=0x7fb9841ef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb98415b2d0 0x7fb9841ef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 -- 172.21.15.7:0/4170932884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb974009550 con 0x7fb9840c9f40 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.727+0000 7fb98259c700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb974000c00 tx=0x7fb974004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.728+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974004570 con 0x7fb9840c9f40 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.728+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97401f070 con 0x7fb9840c9f40 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.728+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740038e0 con 0x7fb9840c9f40 2023-06-09T11:41:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.728+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb984159c60 con 0x7fb9840c9f40 2023-06-09T11:41:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.728+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb984159980 con 0x7fb9840c9f40 2023-06-09T11:41:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.729+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984159f40 con 0x7fb9840c9f40 2023-06-09T11:41:01.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.730+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb974021030 con 0x7fb9840c9f40 2023-06-09T11:41:01.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.731+0000 7fb97affd700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 0x7fb96406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:01.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.731+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fb97408ae70 con 0x7fb9840c9f40 2023-06-09T11:41:01.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.731+0000 7fb97bfff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 0x7fb96406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:01.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.732+0000 7fb97bfff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 0x7fb96406d1c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb98415c3b0 tx=0x7fb96c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:01.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.735+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb974055a90 con 0x7fb9840c9f40 2023-06-09T11:41:01.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.989+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb984074cf0 con 0x7fb9840c9f40 2023-06-09T11:41:01.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.990+0000 7fb97affd700 1 -- 172.21.15.7:0/4170932884 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fb974015090 con 0x7fb9840c9f40 2023-06-09T11:41:01.994 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:41:01.994 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:00.803084+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:41:01.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.992+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 msgr2=0x7fb96406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.992+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 0x7fb96406d1c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb98415c3b0 tx=0x7fb96c008040 comp rx=0 tx=0).stop 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 msgr2=0x7fb9840b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb974000c00 tx=0x7fb974004750 comp rx=0 tx=0).stop 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 shutdown_connections 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb98415b2d0 0x7fb9841ef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9840c9f40 0x7fb9840b2eb0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb96406ad10 0x7fb96406d1c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 --2- 172.21.15.7:0/4170932884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb984153880 0x7fb98414b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 >> 172.21.15.7:0/4170932884 conn(0x7fb9840737f0 msgr2=0x7fb98415b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 shutdown_connections 2023-06-09T11:41:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:01.993+0000 7fb988fff700 1 -- 172.21.15.7:0/4170932884 wait complete. 2023-06-09T11:41:01.998 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 138 2023-06-09T11:41:02.014 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=1, replacing gid=5852 2023-06-09T11:41:02.014 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.d 2023-06-09T11:41:03.015 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:41:03.015 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:41:03.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:41:03.020 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:41:03.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:41:03.169 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:41:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.228+0000 7f6dd21ff700 1 Processor -- start 2023-06-09T11:41:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.228+0000 7f6dd21ff700 1 -- start start 2023-06-09T11:41:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dd21ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6dcc0af980 0x7f6dcc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dd21ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc15ce00 con 0x7f6dcc153960 2023-06-09T11:41:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dd21ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc0adab0 con 0x7f6dcc0af980 2023-06-09T11:41:03.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc15dc60 con 0x7f6dcc071c80 2023-06-09T11:41:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dcb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6dcc153960 0x7f6dcc06fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41570/0 (socket says 172.21.15.7:41570) 2023-06-09T11:41:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.229+0000 7f6dcb7fe700 1 -- 172.21.15.7:0/3995474437 learned_addr learned my addr 172.21.15.7:0/3995474437 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:41:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcaffd700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/3995474437 >> v2:172.21.15.117:3300/0 conn(0x7f6dcc0af980 0x7f6dcc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dca7fc700 1 -- 172.21.15.7:0/3995474437 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232718603 0 0) 0x7f6dcc15ce00 con 0x7f6dcc153960 2023-06-09T11:41:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dca7fc700 1 -- 172.21.15.7:0/3995474437 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db40035b0 con 0x7f6dcc153960 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 -- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 msgr2=0x7f6dcc159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 -- 172.21.15.7:0/3995474437 >> v1:172.21.15.7:6789/0 conn(0x7f6dcc153960 legacy=0x7f6dcc06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 -- 172.21.15.7:0/3995474437 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc15d0e0 con 0x7f6dcc0af980 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.230+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc15a0f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24c36c42a0e3fbbb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dca7fc700 1 -- 172.21.15.7:0/3995474437 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400e070 con 0x7f6dcc0af980 2023-06-09T11:41:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dca7fc700 1 -- 172.21.15.7:0/3995474437 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc400bb20 con 0x7f6dcc0af980 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dca7fc700 1 -- 172.21.15.7:0/3995474437 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400be90 con 0x7f6dcc0af980 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 -- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 msgr2=0x7f6dcc15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc15a0f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009d90 tx=0x7f6dc400b330 comp rx=0 tx=0).stop 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 -- 172.21.15.7:0/3995474437 shutdown_connections 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc15a0f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/3995474437 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 -- 172.21.15.7:0/3995474437 >> 172.21.15.7:0/3995474437 conn(0x7f6dcc0c9f00 msgr2=0x7f6dcc0ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 -- 172.21.15.7:0/3995474437 shutdown_connections 2023-06-09T11:41:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.231+0000 7f6dd21ff700 1 -- 172.21.15.7:0/3995474437 wait complete. 2023-06-09T11:41:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.233+0000 7f6dd21ff700 1 Processor -- start 2023-06-09T11:41:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.233+0000 7f6dd21ff700 1 -- start start 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc151cf0 con 0x7f6dcc153960 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0b05c0 con 0x7f6dcc0af980 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dd21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc0b0840 con 0x7f6dcc071c80 2023-06-09T11:41:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35902/0 (socket says 172.21.15.7:35902) 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dcbfff700 1 -- 172.21.15.7:0/2152136676 learned_addr learned my addr 172.21.15.7:0/2152136676 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dcb7fe700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.234+0000 7f6dcaffd700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 -- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 msgr2=0x7f6dcc0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc0b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 -- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 msgr2=0x7f6dcc0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 -- 172.21.15.7:0/2152136676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc4009a70 con 0x7f6dcc153960 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dcbfff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4013000 tx=0x7f6dc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4004800 con 0x7f6dcc153960 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc4020070 con 0x7f6dcc153960 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc15d6a0 con 0x7f6dcc153960 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40276c0 con 0x7f6dcc153960 2023-06-09T11:41:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.235+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc15d3c0 con 0x7f6dcc153960 2023-06-09T11:41:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.237+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc15d980 con 0x7f6dcc153960 2023-06-09T11:41:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.237+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6dc4012430 con 0x7f6dcc153960 2023-06-09T11:41:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.238+0000 7f6db3fff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 0x7f6db806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.238+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f6dc4022070 con 0x7f6dcc153960 2023-06-09T11:41:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.239+0000 7f6dcb7fe700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 0x7f6db806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:03.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.239+0000 7f6dcb7fe700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 0x7f6db806d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc007fd0 tx=0x7f6dbc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:03.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.243+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6dc40560e0 con 0x7f6dcc153960 2023-06-09T11:41:03.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.497+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dcc151ea0 con 0x7f6dcc153960 2023-06-09T11:41:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.498+0000 7f6db3fff700 1 -- 172.21.15.7:0/2152136676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 138 v138) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f6dcc151ea0 con 0x7f6dcc153960 2023-06-09T11:41:03.502 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:41:03.502 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":138,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:00.803084+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:41:03.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 msgr2=0x7f6db806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 0x7f6db806d190 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc007fd0 tx=0x7f6dbc009040 comp rx=0 tx=0).stop 2023-06-09T11:41:03.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 msgr2=0x7f6dcc148cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:03.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4013000 tx=0x7f6dc4004ad0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 shutdown_connections 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6dcc153960 0x7f6dcc148cb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6db806ace0 0x7f6db806d190 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6dcc071c80 0x7f6dcc0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 --2- 172.21.15.7:0/2152136676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6dcc0af980 0x7f6dcc0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 >> 172.21.15.7:0/2152136676 conn(0x7f6dcc0c9f00 msgr2=0x7f6dcc159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 shutdown_connections 2023-06-09T11:41:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:03.500+0000 7f6dd21ff700 1 -- 172.21.15.7:0/2152136676 wait complete. 2023-06-09T11:41:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 138 2023-06-09T11:41:03.522 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:41:05.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:41:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.730+0000 7fe11e7ff700 1 Processor -- start 2023-06-09T11:41:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.730+0000 7fe11e7ff700 1 -- start start 2023-06-09T11:41:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.731+0000 7fe11e7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1180cb5d0 0x7fe1180b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11e7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 0x7fe1180cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11e7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe118159b20 con 0x7fe1180cb9a0 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11e7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180cac80 con 0x7fe1180cb5d0 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11e7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1181597d0 con 0x7fe11815cd40 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe117fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe1180cb5d0 0x7fe1180b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe1177fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe1180cb9a0 0x7fe118159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41586/0 (socket says 172.21.15.7:41586) 2023-06-09T11:41:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe1177fe700 1 -- 172.21.15.7:0/3058970100 learned_addr learned my addr 172.21.15.7:0/3058970100 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11cd9c700 1 --2- 172.21.15.7:0/3058970100 >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 0x7fe1180cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe117fff700 1 -- 172.21.15.7:0/3058970100 >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 msgr2=0x7fe1180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe117fff700 1 --2- 172.21.15.7:0/3058970100 >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 0x7fe1180cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe117fff700 1 -- 172.21.15.7:0/3058970100 >> v1:172.21.15.7:6789/0 conn(0x7fe1180cb9a0 legacy=0x7fe118159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe117fff700 1 -- 172.21.15.7:0/3058970100 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118071ab0 con 0x7fe1180cb5d0 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe116ffd700 1 -- 172.21.15.7:0/3058970100 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133338407 0 0) 0x7fe118159b20 con 0x7fe1180cb9a0 2023-06-09T11:41:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.732+0000 7fe11cd9c700 1 --2- 172.21.15.7:0/3058970100 >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 0x7fe1180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe117fff700 1 --2- 172.21.15.7:0/3058970100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180cb5d0 0x7fe1180b52b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1180c9b80 tx=0x7fe108009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32824f0348431ec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe116ffd700 1 -- 172.21.15.7:0/3058970100 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108011830 con 0x7fe1180cb5d0 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe116ffd700 1 -- 172.21.15.7:0/3058970100 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108011990 con 0x7fe1180cb5d0 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe116ffd700 1 -- 172.21.15.7:0/3058970100 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108011d20 con 0x7fe1180cb5d0 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3058970100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180cb5d0 msgr2=0x7fe1180b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3058970100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180cb5d0 0x7fe1180b52b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe1180c9b80 tx=0x7fe108009b20 comp rx=0 tx=0).stop 2023-06-09T11:41:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3058970100 shutdown_connections 2023-06-09T11:41:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3058970100 >> v2:172.21.15.117:3300/0 conn(0x7fe11815cd40 0x7fe1180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.733+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3058970100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180cb5d0 0x7fe1180b52b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.734+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3058970100 >> 172.21.15.7:0/3058970100 conn(0x7fe118070af0 msgr2=0x7fe118070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.734+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3058970100 shutdown_connections 2023-06-09T11:41:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.734+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3058970100 wait complete. 2023-06-09T11:41:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.735+0000 7fe11e7ff700 1 Processor -- start 2023-06-09T11:41:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.736+0000 7fe11e7ff700 1 -- start start 2023-06-09T11:41:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180cb5d0 0x7fe1181f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1180cb9a0 0x7fe1180c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180d0310 con 0x7fe11815cd40 2023-06-09T11:41:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe118154500 con 0x7fe1180cb9a0 2023-06-09T11:41:05.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.737+0000 7fe11e7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180cc4c0 con 0x7fe1180cb5d0 2023-06-09T11:41:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35928/0 (socket says 172.21.15.7:35928) 2023-06-09T11:41:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 -- 172.21.15.7:0/3809757033 learned_addr learned my addr 172.21.15.7:0/3809757033 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:41:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe117fff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180cb5d0 0x7fe1181f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe1177fe700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1180cb9a0 0x7fe1180c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 -- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180cb5d0 msgr2=0x7fe1181f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180cb5d0 0x7fe1181f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 -- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1180cb9a0 msgr2=0x7fe1180c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1180cb9a0 0x7fe1180c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 -- 172.21.15.7:0/3809757033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1080094a0 con 0x7fe11815cd40 2023-06-09T11:41:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe11cd9c700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe11000cd80 tx=0x7fe11000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11000f480 con 0x7fe11815cd40 2023-06-09T11:41:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.738+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11000f600 con 0x7fe11815cd40 2023-06-09T11:41:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.739+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11000f9b0 con 0x7fe11815cd40 2023-06-09T11:41:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.739+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11815d4f0 con 0x7fe11815cd40 2023-06-09T11:41:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.739+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11815d210 con 0x7fe11815cd40 2023-06-09T11:41:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.740+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11815d7d0 con 0x7fe11815cd40 2023-06-09T11:41:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.741+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe11000b040 con 0x7fe11815cd40 2023-06-09T11:41:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.742+0000 7fe114ff9700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:41:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.742+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fe11008a0a0 con 0x7fe11815cd40 2023-06-09T11:41:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.742+0000 7fe117fff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:41:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.742+0000 7fe117fff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe1080090b0 tx=0x7fe108008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:41:05.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:05.746+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe110054ba0 con 0x7fe11815cd40 2023-06-09T11:41:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.006+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe118074cf0 con 0x7fe11815cd40 2023-06-09T11:41:06.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.007+0000 7fe114ff9700 1 -- 172.21.15.7:0/3809757033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+6219 (secure 0 0 0) 0x7fe1100581c0 con 0x7fe11815cd40 2023-06-09T11:41:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:41:06.011 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":139,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":138,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:00.803084+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:41:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.009+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 msgr2=0x7fe10406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.009+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe1080090b0 tx=0x7fe108008030 comp rx=0 tx=0).stop 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.009+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 msgr2=0x7fe1180c2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.009+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fe11000cd80 tx=0x7fe11000a560 comp rx=0 tx=0).stop 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.009+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 shutdown_connections 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815cd40 0x7fe1180c2490 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:06.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180cb5d0 0x7fe1181f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 --2- 172.21.15.7:0/3809757033 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe1180cb9a0 0x7fe1180c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:41:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 >> 172.21.15.7:0/3809757033 conn(0x7fe118070af0 msgr2=0x7fe118158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:41:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 shutdown_connections 2023-06-09T11:41:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:41:06.010+0000 7fe11e7ff700 1 -- 172.21.15.7:0/3809757033 wait complete. 2023-06-09T11:41:06.015 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 139 2023-06-09T11:41:06.031 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:41:06.032 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 55 secs before thrashing 2023-06-09T11:41:18.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:18.775 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:21.369 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 1% (136/12590) Updating files: 1% (137/12590) Updating files: 1% (216/12590) Updating files: 1% (249/12590) Updating files: 2% (252/12590) Updating files: 2% (327/12590) Updating files: 3% (378/12590) Updating files: 3% (392/12590) Updating files: 3% (446/12590) Updating files: 3% (456/12590) Updating files: 3% (487/12590) Updating files: 3% (500/12590) Updating files: 4% (504/12590) Updating files: 4% (586/12590) Updating files: 4% (609/12590) Updating files: 5% (630/12590) Updating files: 5% (683/12590) Updating files: 6% (756/12590) Updating files: 6% (757/12590) Updating files: 6% (834/12590) Updating files: 6% (850/12590) Updating files: 7% (882/12590) Updating files: 7% (956/12590) Updating files: 8% (1008/12590) Updating files: 8% (1052/12590) Updating files: 8% (1080/12590) Updating files: 8% (1091/12590) Updating files: 8% (1115/12590) Updating files: 9% (1134/12590) Updating files: 9% (1183/12590) Updating files: 9% (1226/12590) Updating files: 10% (1259/12590) Updating files: 10% (1295/12590) Updating files: 10% (1338/12590) Updating files: 11% (1385/12590) Updating files: 11% (1388/12590) Updating files: 11% (1404/12590) Updating files: 12% (1511/12590) Updating files: 12% (1524/12590) Updating files: 12% (1552/12590) Updating files: 12% (1579/12590) Updating files: 12% (1611/12590) Updating files: 13% (1637/12590) Updating files: 13% (1652/12590) Updating files: 13% (1666/12590) Updating files: 13% (1682/12590) Updating files: 13% (1716/12590) Updating files: 13% (1750/12590) Updating files: 14% (1763/12590) Updating files: 14% (1767/12590) Updating files: 14% (1833/12590) Updating files: 14% (1862/12590) Updating files: 15% (1889/12590) Updating files: 15% (1891/12590) Updating files: 15% (1892/12590) Updating files: 15% (1940/12590) Updating files: 15% (1999/12590) Updating files: 16% (2015/12590) Updating files: 16% (2026/12590) Updating files: 16% (2085/12590) Updating files: 16% (2091/12590) Updating files: 17% (2141/12590) Updating files: 17% (2211/12590) Updating files: 17% (2240/12590) Updating files: 18% (2267/12590) Updating files: 18% (2295/12590) Updating files: 18% (2376/12590) Updating files: 19% (2393/12590) Updating files: 19% (2415/12590) Updating files: 19% (2418/12590) Updating files: 19% (2473/12590) Updating files: 19% (2484/12590) Updating files: 20% (2518/12590) Updating files: 20% (2541/12590) Updating files: 20% (2569/12590) Updating files: 20% (2613/12590) Updating files: 21% (2644/12590) Updating files: 21% (2710/12590) Updating files: 21% (2747/12590) Updating files: 22% (2770/12590) Updating files: 22% (2870/12590) Updating files: 23% (2896/12590) Updating files: 24% (3022/12590) Updating files: 24% (3033/12590) Updating files: 24% (3113/12590) Updating files: 25% (3148/12590) Updating files: 25% (3197/12590) Updating files: 25% (3227/12590) Updating files: 26% (3274/12590) Updating files: 26% (3284/12590) Updating files: 27% (3400/12590) Updating files: 27% (3431/12590) Updating files: 27% (3484/12590) Updating files: 28% (3526/12590) Updating files: 28% (3559/12590) Updating files: 29% (3652/12590) Updating files: 29% (3708/12590) Updating files: 30% (3777/12590) Updating files: 30% (3844/12590) Updating files: 30% (3852/12590) Updating files: 30% (3899/12590) Updating files: 31% (3903/12590) Updating files: 31% (3997/12590) Updating files: 32% (4029/12590) Updating files: 32% (4055/12590) Updating files: 32% (4113/12590) Updating files: 32% (4142/12590) Updating files: 33% (4155/12590) Updating files: 33% (4226/12590) Updating files: 33% (4262/12590) Updating files: 34% (4281/12590) Updating files: 34% (4338/12590) Updating files: 34% (4343/12590) Updating files: 34% (4352/12590) Updating files: 34% (4367/12590) Updating files: 34% (4376/12590) Updating files: 34% (4385/12590) Updating files: 34% (4395/12590) Updating files: 34% (4397/12590) Updating files: 34% (4403/12590) Updating files: 35% (4407/12590) Updating files: 35% (4418/12590) Updating files: 35% (4422/12590) Updating files: 35% (4423/12590) Updating files: 35% (4455/12590) Updating files: 35% (4470/12590) Updating files: 35% (4473/12590) Updating files: 35% (4486/12590) Updating files: 35% (4510/12590) Updating files: 36% (4533/12590) Updating files: 36% (4540/12590) Updating files: 36% (4552/12590) Updating files: 36% (4563/12590) Updating files: 36% (4598/12590) Updating files: 36% (4618/12590) Updating files: 36% (4651/12590) Updating files: 37% (4659/12590) Updating files: 37% (4688/12590) Updating files: 38% (4785/12590) Updating files: 38% (4820/12590) Updating files: 38% (4842/12590) Updating files: 38% (4858/12590) Updating files: 38% (4863/12590) Updating files: 38% (4869/12590) Updating files: 38% (4903/12590) Updating files: 39% (4911/12590) Updating files: 39% (4916/12590) Updating files: 39% (4923/12590) Updating files: 39% (4959/12590) Updating files: 39% (4971/12590) Updating files: 40% (5036/12590) Updating files: 40% (5055/12590) Updating files: 40% (5096/12590) Updating files: 41% (5162/12590) Updating files: 41% (5176/12590) Updating files: 41% (5190/12590) Updating files: 41% (5231/12590) Updating files: 41% (5263/12590) Updating files: 42% (5288/12590) Updating files: 42% (5303/12590) Updating files: 42% (5364/12590) Updating files: 42% (5385/12590) Updating files: 42% (5401/12590) Updating files: 42% (5409/12590) Updating files: 43% (5414/12590) Updating files: 43% (5427/12590) Updating files: 43% (5443/12590) Updating files: 43% (5455/12590) Updating files: 43% (5474/12590) Updating files: 43% (5526/12590) Updating files: 44% (5540/12590) Updating files: 44% (5558/12590) Updating files: 44% (5565/12590) Updating files: 44% (5618/12590) Updating files: 44% (5639/12590) Updating files: 44% (5644/12590) Updating files: 44% (5664/12590) Updating files: 45% (5666/12590) Updating files: 45% (5697/12590) Updating files: 45% (5757/12590) Updating files: 46% (5792/12590) Updating files: 46% (5796/12590) Updating files: 46% (5914/12590) Updating files: 47% (5918/12590) Updating files: 47% (6017/12590) Updating files: 48% (6044/12590) Updating files: 48% (6076/12590) Updating files: 48% (6147/12590) Updating files: 48% (6160/12590) Updating files: 49% (6170/12590) Updating files: 49% (6174/12590) Updating files: 49% (6176/12590) Updating files: 49% (6239/12590) Updating files: 49% (6251/12590) Updating files: 50% (6295/12590) Updating files: 50% (6316/12590) Updating files: 50% (6371/12590) Updating files: 50% (6393/12590) Updating files: 51% (6421/12590) Updating files: 51% (6472/12590) Updating files: 52% (6547/12590) Updating files: 52% (6618/12590) Updating files: 53% (6673/12590) Updating files: 53% (6677/12590) Updating files: 53% (6708/12590) Updating files: 54% (6799/12590) Updating files: 54% (6811/12590) Updating files: 54% (6826/12590) Updating files: 55% (6925/12590) Updating files: 55% (6943/12590) Updating files: 55% (6986/12590) Updating files: 56% (7051/12590) Updating files: 56% (7115/12590) Updating files: 56% (7150/12590) Updating files: 57% (7177/12590) Updating files: 57% (7195/12590) Updating files: 57% (7196/12590) Updating files: 57% (7200/12590) Updating files: 57% (7207/12590) Updating files: 57% (7288/12590) Updating files: 58% (7303/12590) Updating files: 59% (7429/12590) Updating files: 59% (7475/12590) Updating files: 60% (7554/12590) Updating files: 60% (7631/12590) Updating files: 61% (7680/12590) Updating files: 61% (7733/12590) Updating files: 61% (7788/12590) Updating files: 62% (7806/12590) Updating files: 62% (7844/12590) Updating files: 62% (7920/12590) Updating files: 63% (7932/12590) Updating files: 63% (7985/12590) Updating files: 63% (8016/12590) Updating files: 63% (8018/12590) Updating files: 63% (8038/12590) Updating files: 63% (8054/12590) Updating files: 64% (8058/12590) Updating files: 64% (8071/12590) Updating files: 64% (8084/12590) Updating files: 64% (8174/12590) Updating files: 65% (8184/12590) Updating files: 65% (8209/12590) Updating files: 65% (8249/12590) Updating files: 66% (8310/12590) Updating files: 66% (8348/12590) Updating files: 66% (8373/12590) Updating files: 67% (8436/12590) Updating files: 67% (8469/12590) Updating files: 68% (8562/12590) Updating files: 68% (8584/12590) Updating files: 68% (8670/12590) Updating files: 68% (8681/12590) Updating files: 69% (8688/12590) Updating files: 69% (8741/12590) Updating files: 69% (8776/12590) Updating files: 70% (8813/12590) Updating files: 70% (8929/12590) Updating files: 71% (8939/12590) Updating files: 71% (9062/12590) Updating files: 72% (9065/12590) Updating files: 72% (9115/12590) Updating files: 73% (9191/12590) Updating files: 73% (9256/12590) Updating files: 74% (9317/12590) Updating files: 74% (9368/12590) Updating files: 75% (9443/12590) Updating files: 75% (9457/12590) Updating files: 76% (9569/12590) Updating files: 76% (9570/12590) Updating files: 76% (9590/12590) Updating files: 77% (9695/12590) Updating files: 77% (9728/12590) Updating files: 78% (9821/12590) Updating files: 78% (9874/12590) Updating files: 78% (9943/12590) Updating files: 79% (9947/12590) Updating files: 79% (10000/12590) Updating files: 79% (10031/12590) Updating files: 80% (10072/12590) Updating files: 80% (10104/12590) Updating files: 81% (10198/12590) Updating files: 81% (10228/12590) Updating files: 81% (10271/12590) Updating files: 82% (10324/12590) Updating files: 82% (10345/12590) Updating files: 82% (10392/12590) Updating files: 82% (10414/12590) Updating files: 83% (10450/12590) Updating files: 83% (10471/12590) Updating files: 84% (10576/12590) Updating files: 84% (10609/12590) Updating files: 84% (10690/12590) Updating files: 84% (10701/12590) Updating files: 85% (10702/12590) Updating files: 85% (10772/12590) Updating files: 85% (10806/12590) Updating files: 86% (10828/12590) Updating files: 87% (10954/12590) Updating files: 87% (10992/12590) Updating files: 88% (11080/12590) Updating files: 88% (11191/12590) Updating files: 89% (11206/12590) Updating files: 89% (11216/12590) Updating files: 89% (11244/12590) Updating files: 89% (11261/12590) Updating files: 90% (11331/12590) Updating files: 90% (11343/12590) Updating files: 90% (11382/12590) Updating files: 90% (11404/12590) Updating files: 90% (11413/12590) Updating files: 91% (11457/12590) Updating files: 91% (11514/12590) Updating files: 91% (11575/12590) Updating files: 91% (11576/12590) Updating files: 92% (11583/12590) Updating files: 92% (11663/12590) Updating files: 93% (11709/12590) Updating files: 93% (11760/12590) Updating files: 94% (11835/12590) Updating files: 94% (11899/12590) Updating files: 95% (11961/12590) Updating files: 95% (12024/12590) Updating files: 96% (12087/12590) Updating files: 96% (12107/12590) Updating files: 96% (12109/12590) Updating files: 96% (12145/12590) Updating files: 96% (12146/12590) Updating files: 96% (12153/12590) Updating files: 96% (12164/12590) Updating files: 97% (12213/12590) Updating files: 97% (12246/12590) Updating files: 97% (12278/12590) Updating files: 97% (12321/12590) Updating files: 98% (12339/12590) Updating files: 98% (12417/12590) Updating files: 98% (12443/12590) Updating files: 98% (12460/12590) Updating files: 99% (12465/12590) Updating files: 99% (12468/12590) Updating files: 99% (12480/12590) Updating files: 99% (12486/12590) Updating files: 99% (12549/12590) Updating files: 99% (12564/12590) Updating files: 99% (12570/12590) Updating files: 100% (12590/12590) Updating files: 100% (12590/12590), done. 2023-06-09T11:41:21.902 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 204c175eea3 msg: blobhash on entity_name_t causes strange badness... should look into that, but avoid for now 2023-06-09T11:41:21.912 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.1 2023-06-09T11:41:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:41:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 2 -eq 48 ']' 2023-06-09T11:41:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.2 2023-06-09T11:41:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.2 2023-06-09T11:41:21.970 INFO:tasks.workunit.client.0.smithi007.stdout:v0.2 2023-06-09T11:41:21.970 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.2 2023-06-09T11:41:35.403 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 12% (28/222) Updating files: 13% (29/222) Updating files: 14% (32/222) Updating files: 15% (34/222) Updating files: 16% (36/222) Updating files: 17% (38/222) Updating files: 18% (40/222) Updating files: 19% (43/222) Updating files: 20% (45/222) Updating files: 21% (47/222) Updating files: 22% (49/222) Updating files: 22% (50/222) Updating files: 23% (52/222) Updating files: 24% (54/222) Updating files: 25% (56/222) Updating files: 26% (58/222) Updating files: 26% (59/222) Updating files: 27% (60/222) Updating files: 28% (63/222) Updating files: 29% (65/222) Updating files: 30% (67/222) Updating files: 31% (69/222) Updating files: 32% (72/222) Updating files: 33% (74/222) Updating files: 33% (75/222) Updating files: 34% (76/222) Updating files: 35% (78/222) Updating files: 36% (80/222) Updating files: 37% (83/222) Updating files: 38% (85/222) Updating files: 39% (87/222) Updating files: 40% (89/222) Updating files: 41% (92/222) Updating files: 42% (94/222) Updating files: 43% (96/222) Updating files: 44% (98/222) Updating files: 45% (100/222) Updating files: 46% (103/222) Updating files: 47% (105/222) Updating files: 48% (107/222) Updating files: 49% (109/222) Updating files: 50% (111/222) Updating files: 51% (114/222) Updating files: 52% (116/222) Updating files: 53% (118/222) Updating files: 54% (120/222) Updating files: 55% (123/222) Updating files: 56% (125/222) Updating files: 57% (127/222) Updating files: 58% (129/222) Updating files: 59% (131/222) Updating files: 60% (134/222) Updating files: 61% (136/222) Updating files: 62% (138/222) Updating files: 63% (140/222) Updating files: 64% (143/222) Updating files: 65% (145/222) Updating files: 66% (147/222) Updating files: 67% (149/222) Updating files: 68% (151/222) Updating files: 69% (154/222) Updating files: 70% (156/222) Updating files: 71% (158/222) Updating files: 72% (160/222) Updating files: 73% (163/222) Updating files: 74% (165/222) Updating files: 74% (166/222) Updating files: 75% (167/222) Updating files: 76% (169/222) Updating files: 77% (171/222) Updating files: 78% (174/222) Updating files: 78% (175/222) Updating files: 79% (176/222) Updating files: 80% (178/222) Updating files: 80% (179/222) Updating files: 81% (180/222) Updating files: 82% (183/222) Updating files: 83% (185/222) Updating files: 84% (187/222) Updating files: 85% (189/222) Updating files: 86% (191/222) Updating files: 87% (194/222) Updating files: 88% (196/222) Updating files: 89% (198/222) Updating files: 90% (200/222) Updating files: 91% (203/222) Updating files: 92% (205/222) Updating files: 93% (207/222) Updating files: 94% (209/222) Updating files: 95% (211/222) Updating files: 96% (214/222) Updating files: 97% (216/222) Updating files: 98% (218/222) Updating files: 99% (220/222) Updating files: 100% (222/222) Updating files: 100% (222/222), done. 2023-06-09T11:41:35.558 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 2b5f76f48e9 readme 2023-06-09T11:41:35.565 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.2 2023-06-09T11:41:35.599 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:41:35.600 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 3 -eq 48 ']' 2023-06-09T11:41:35.600 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.3 2023-06-09T11:41:35.600 INFO:tasks.workunit.client.0.smithi007.stdout:v0.3 2023-06-09T11:41:35.601 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.3 2023-06-09T11:41:35.601 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.3 2023-06-09T11:41:45.720 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 15% (47/295) Updating files: 16% (48/295) Updating files: 17% (51/295) Updating files: 18% (54/295) Updating files: 19% (57/295) Updating files: 20% (59/295) Updating files: 21% (62/295) Updating files: 22% (65/295) Updating files: 23% (68/295) Updating files: 24% (71/295) Updating files: 25% (74/295) Updating files: 26% (77/295) Updating files: 27% (80/295) Updating files: 27% (81/295) Updating files: 28% (83/295) Updating files: 29% (86/295) Updating files: 30% (89/295) Updating files: 31% (92/295) Updating files: 32% (95/295) Updating files: 33% (98/295) Updating files: 34% (101/295) Updating files: 35% (104/295) Updating files: 36% (107/295) Updating files: 37% (110/295) Updating files: 37% (111/295) Updating files: 38% (113/295) Updating files: 39% (116/295) Updating files: 40% (118/295) Updating files: 41% (121/295) Updating files: 42% (124/295) Updating files: 42% (125/295) Updating files: 43% (127/295) Updating files: 44% (130/295) Updating files: 45% (133/295) Updating files: 46% (136/295) Updating files: 47% (139/295) Updating files: 48% (142/295) Updating files: 49% (145/295) Updating files: 50% (148/295) Updating files: 51% (151/295) Updating files: 52% (154/295) Updating files: 53% (157/295) Updating files: 54% (160/295) Updating files: 55% (163/295) Updating files: 56% (166/295) Updating files: 57% (169/295) Updating files: 58% (172/295) Updating files: 59% (175/295) Updating files: 60% (177/295) Updating files: 61% (180/295) Updating files: 62% (183/295) Updating files: 63% (186/295) Updating files: 64% (189/295) Updating files: 65% (192/295) Updating files: 66% (195/295) Updating files: 67% (198/295) Updating files: 68% (201/295) Updating files: 69% (204/295) Updating files: 70% (207/295) Updating files: 71% (210/295) Updating files: 72% (213/295) Updating files: 73% (216/295) Updating files: 74% (219/295) Updating files: 75% (222/295) Updating files: 76% (225/295) Updating files: 77% (228/295) Updating files: 78% (231/295) Updating files: 78% (233/295) Updating files: 79% (234/295) Updating files: 80% (236/295) Updating files: 81% (239/295) Updating files: 82% (242/295) Updating files: 82% (243/295) Updating files: 83% (245/295) Updating files: 84% (248/295) Updating files: 85% (251/295) Updating files: 85% (252/295) Updating files: 86% (254/295) Updating files: 87% (257/295) Updating files: 88% (260/295) Updating files: 89% (263/295) Updating files: 90% (266/295) Updating files: 91% (269/295) Updating files: 92% (272/295) Updating files: 93% (275/295) Updating files: 94% (278/295) Updating files: 95% (281/295) Updating files: 96% (284/295) Updating files: 97% (287/295) Updating files: 98% (290/295) Updating files: 98% (292/295) Updating files: 99% (293/295) Updating files: 100% (295/295) Updating files: 100% (295/295), done. 2023-06-09T11:41:45.840 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 2e986fe9ddf include ceph.spec in configure.ac 2023-06-09T11:41:45.850 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.3 2023-06-09T11:41:45.889 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:41:45.890 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 4 -eq 48 ']' 2023-06-09T11:41:45.890 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.4 2023-06-09T11:41:45.890 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.4 2023-06-09T11:41:45.890 INFO:tasks.workunit.client.0.smithi007.stdout:v0.4 2023-06-09T11:41:45.890 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.4 2023-06-09T11:41:48.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:48.922 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:41:54.807 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 16% (40/247) Updating files: 17% (42/247) Updating files: 18% (45/247) Updating files: 19% (47/247) Updating files: 20% (50/247) Updating files: 20% (51/247) Updating files: 21% (52/247) Updating files: 22% (55/247) Updating files: 23% (57/247) Updating files: 24% (60/247) Updating files: 25% (62/247) Updating files: 26% (65/247) Updating files: 27% (67/247) Updating files: 28% (70/247) Updating files: 29% (72/247) Updating files: 30% (75/247) Updating files: 31% (77/247) Updating files: 32% (80/247) Updating files: 33% (82/247) Updating files: 34% (84/247) Updating files: 35% (87/247) Updating files: 36% (89/247) Updating files: 37% (92/247) Updating files: 38% (94/247) Updating files: 38% (95/247) Updating files: 39% (97/247) Updating files: 40% (99/247) Updating files: 41% (102/247) Updating files: 42% (104/247) Updating files: 43% (107/247) Updating files: 44% (109/247) Updating files: 45% (112/247) Updating files: 46% (114/247) Updating files: 47% (117/247) Updating files: 48% (119/247) Updating files: 49% (122/247) Updating files: 50% (124/247) Updating files: 51% (126/247) Updating files: 52% (129/247) Updating files: 53% (131/247) Updating files: 54% (134/247) Updating files: 55% (136/247) Updating files: 56% (139/247) Updating files: 57% (141/247) Updating files: 58% (144/247) Updating files: 59% (146/247) Updating files: 60% (149/247) Updating files: 61% (151/247) Updating files: 62% (154/247) Updating files: 63% (156/247) Updating files: 64% (159/247) Updating files: 65% (161/247) Updating files: 66% (164/247) Updating files: 67% (166/247) Updating files: 68% (168/247) Updating files: 69% (171/247) Updating files: 70% (173/247) Updating files: 71% (176/247) Updating files: 72% (178/247) Updating files: 73% (181/247) Updating files: 74% (183/247) Updating files: 75% (186/247) Updating files: 76% (188/247) Updating files: 77% (191/247) Updating files: 78% (193/247) Updating files: 79% (196/247) Updating files: 80% (198/247) Updating files: 81% (201/247) Updating files: 82% (203/247) Updating files: 83% (206/247) Updating files: 84% (208/247) Updating files: 85% (210/247) Updating files: 86% (213/247) Updating files: 87% (215/247) Updating files: 88% (218/247) Updating files: 89% (220/247) Updating files: 90% (223/247) Updating files: 91% (225/247) Updating files: 92% (228/247) Updating files: 93% (230/247) Updating files: 94% (233/247) Updating files: 95% (235/247) Updating files: 96% (238/247) Updating files: 97% (240/247) Updating files: 98% (243/247) Updating files: 98% (244/247) Updating files: 99% (245/247) Updating files: 100% (247/247) Updating files: 100% (247/247), done. 2023-06-09T11:41:54.947 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 13b1bf7c492 kclient: initializing kaddr (merge fix) 2023-06-09T11:41:54.955 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.4 2023-06-09T11:41:54.995 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:41:54.995 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 5 -eq 48 ']' 2023-06-09T11:41:54.996 INFO:tasks.workunit.client.0.smithi007.stdout:v0.5 2023-06-09T11:41:54.996 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.5 2023-06-09T11:41:54.996 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.5 2023-06-09T11:41:54.997 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.5 2023-06-09T11:42:01.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.228+0000 7fc57c7ff700 1 Processor -- start 2023-06-09T11:42:01.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.228+0000 7fc57c7ff700 1 -- start start 2023-06-09T11:42:01.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.229+0000 7fc57c7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5740c8e90 0x7fc57415d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57c7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57415dc90 con 0x7fc5740c9260 2023-06-09T11:42:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57c7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5740cfaf0 con 0x7fc5740b5370 2023-06-09T11:42:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57c7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5741597d0 con 0x7fc5740c8e90 2023-06-09T11:42:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5740c8e90 0x7fc57415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc5740c8e90 0x7fc57415d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42718/0 (socket says 172.21.15.7:42718) 2023-06-09T11:42:01.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 -- 172.21.15.7:0/3560937521 learned_addr learned my addr 172.21.15.7:0/3560937521 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 -- 172.21.15.7:0/3560937521 >> v1:172.21.15.117:6789/0 conn(0x7fc5740b5370 legacy=0x7fc5740c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 -- 172.21.15.7:0/3560937521 >> v1:172.21.15.7:6789/0 conn(0x7fc5740c9260 legacy=0x7fc5740cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc57a59b700 1 -- 172.21.15.7:0/3560937521 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5740cfdd0 con 0x7fc5740c8e90 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.230+0000 7fc579599700 1 -- 172.21.15.7:0/3560937521 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817326711 0 0) 0x7fc564001110 con 0x7fc5740c9260 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57a59b700 1 --2- 172.21.15.7:0/3560937521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c8e90 0x7fc57415d750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc5700098f0 tx=0x7fc570009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fb53830e62e36f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc579599700 1 -- 172.21.15.7:0/3560937521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000b8a0 con 0x7fc5740c8e90 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc579599700 1 -- 172.21.15.7:0/3560937521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57000ba00 con 0x7fc5740c8e90 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc579599700 1 -- 172.21.15.7:0/3560937521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57000bd90 con 0x7fc5740c8e90 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57c7ff700 1 -- 172.21.15.7:0/3560937521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c8e90 msgr2=0x7fc57415d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/3560937521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c8e90 0x7fc57415d750 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc5700098f0 tx=0x7fc570009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57c7ff700 1 -- 172.21.15.7:0/3560937521 shutdown_connections 2023-06-09T11:42:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/3560937521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c8e90 0x7fc57415d750 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.231+0000 7fc57c7ff700 1 -- 172.21.15.7:0/3560937521 >> 172.21.15.7:0/3560937521 conn(0x7fc574070af0 msgr2=0x7fc574070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.232+0000 7fc57c7ff700 1 -- 172.21.15.7:0/3560937521 shutdown_connections 2023-06-09T11:42:01.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.232+0000 7fc57c7ff700 1 -- 172.21.15.7:0/3560937521 wait complete. 2023-06-09T11:42:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.234+0000 7fc57c7ff700 1 Processor -- start 2023-06-09T11:42:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.234+0000 7fc57c7ff700 1 -- start start 2023-06-09T11:42:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5740b5370 0x7fc574156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 0x7fc5741e3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741e3770 con 0x7fc5740c9260 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5741e4680 con 0x7fc5740b5370 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.235+0000 7fc57c7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc574156640 con 0x7fc5740c8e90 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42724/0 (socket says 172.21.15.7:42724) 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 -- 172.21.15.7:0/4147086544 learned_addr learned my addr 172.21.15.7:0/4147086544 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:01.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57a59b700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5740b5370 0x7fc574156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc579d9a700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 0x7fc5741e3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 -- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 msgr2=0x7fc5741e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 0x7fc5741e3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 -- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5740b5370 msgr2=0x7fc574156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5740b5370 0x7fc574156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 -- 172.21.15.7:0/4147086544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc570009550 con 0x7fc5740c9260 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc579d9a700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 0x7fc5741e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:42:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.236+0000 7fc57ad9c700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00cbc0 tx=0x7fc56c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.237+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c003c00 con 0x7fc5740c9260 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.237+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56c003d60 con 0x7fc5740c9260 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.237+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc574159ca0 con 0x7fc5740c9260 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.237+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00ab40 con 0x7fc5740c9260 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.237+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5741599c0 con 0x7fc5740c9260 2023-06-09T11:42:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.238+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc574159f80 con 0x7fc5740c9260 2023-06-09T11:42:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.239+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc56c00aca0 con 0x7fc5740c9260 2023-06-09T11:42:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.240+0000 7fc56affd700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 0x7fc55806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.240+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fc56c089b20 con 0x7fc5740c9260 2023-06-09T11:42:01.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.240+0000 7fc57a59b700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 0x7fc55806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.240+0000 7fc57a59b700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 0x7fc55806d1c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc570000c00 tx=0x7fc570008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.244+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc56c054590 con 0x7fc5740c9260 2023-06-09T11:42:01.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.496+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc574074cf0 con 0x7fc5740c9260 2023-06-09T11:42:01.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.496+0000 7fc56affd700 1 -- 172.21.15.7:0/4147086544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fc56c057bb0 con 0x7fc5740c9260 2023-06-09T11:42:01.500 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:01.501 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:01.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.498+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 msgr2=0x7fc55806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.498+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 0x7fc55806d1c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc570000c00 tx=0x7fc570008030 comp rx=0 tx=0).stop 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.498+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 msgr2=0x7fc5741e4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.498+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00cbc0 tx=0x7fc56c00ced0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 shutdown_connections 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc5740c9260 0x7fc5741e4140 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc5740b5370 0x7fc574156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc55806ad10 0x7fc55806d1c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 --2- 172.21.15.7:0/4147086544 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc5740c8e90 0x7fc5741e3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 >> 172.21.15.7:0/4147086544 conn(0x7fc574070af0 msgr2=0x7fc5740c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:01.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 shutdown_connections 2023-06-09T11:42:01.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.499+0000 7fc57c7ff700 1 -- 172.21.15.7:0/4147086544 wait complete. 2023-06-09T11:42:01.504 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:01.520 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=2) 2023-06-09T11:42:01.521 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-06-09T11:42:01.521 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:42:01.542 INFO:tasks.ceph.mds.e:Stopped 2023-06-09T11:42:01.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:01.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.779+0000 7efea91ff700 1 Processor -- start 2023-06-09T11:42:01.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.779+0000 7efea91ff700 1 -- start start 2023-06-09T11:42:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efea415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea91ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7efea40c9f40 0x7efea40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea91ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea415a500 con 0x7efea4153880 2023-06-09T11:42:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea91ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea40c5f10 con 0x7efea40c9f40 2023-06-09T11:42:01.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea415da00 con 0x7efea415b4a0 2023-06-09T11:42:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea2d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efea4153880 0x7efea415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48144/0 (socket says 172.21.15.7:48144) 2023-06-09T11:42:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.780+0000 7efea2d9d700 1 -- 172.21.15.7:0/2297932948 learned_addr learned my addr 172.21.15.7:0/2297932948 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea359e700 1 --2- 172.21.15.7:0/2297932948 >> v2:172.21.15.117:3300/0 conn(0x7efea40c9f40 0x7efea40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea259c700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efea415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270180739 0 0) 0x7efe94001110 con 0x7efea4153880 2023-06-09T11:42:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c0035b0 con 0x7efea4153880 2023-06-09T11:42:01.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1934413564 0 0) 0x7efe8c0035b0 con 0x7efea4153880 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 msgr2=0x7efea415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efea415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 >> v2:172.21.15.117:3300/0 conn(0x7efea40c9f40 msgr2=0x7efea40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 --2- 172.21.15.7:0/2297932948 >> v2:172.21.15.117:3300/0 conn(0x7efea40c9f40 0x7efea40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea41596e0 con 0x7efea4153880 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7efe94001800 con 0x7efea4153880 2023-06-09T11:42:01.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 >> v1:172.21.15.7:6789/0 conn(0x7efea4153880 legacy=0x7efea415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c003a00 0x7efe8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c004850 0x7efe8c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efe8c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40c5f10 con 0x7efe8c003a00 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea415da00 con 0x7efe8c004850 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.781+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00ba40 con 0x7efea415b4a0 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea359e700 1 --2- 172.21.15.7:0/2297932948 >> v2:172.21.15.117:3300/0 conn(0x7efea40c9f40 0x7efea40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:01.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c003a00 0x7efe8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea259c700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c004850 0x7efe8c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea359e700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efe8c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 -- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 msgr2=0x7efe8c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea415b4a0 0x7efe8c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 -- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c004850 msgr2=0x7efe8c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c004850 0x7efe8c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 -- 172.21.15.7:0/2297932948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0035b0 con 0x7efe8c003a00 2023-06-09T11:42:01.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.782+0000 7efea2d9d700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c003a00 0x7efe8c003e10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efe9400c190 tx=0x7efe8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfd8b1b097c49d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94007d00 con 0x7efe8c003a00 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9400cc90 con 0x7efe8c003a00 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea1d9b700 1 -- 172.21.15.7:0/2297932948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94005960 con 0x7efe8c003a00 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 -- 172.21.15.7:0/2297932948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c003a00 msgr2=0x7efe8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 --2- 172.21.15.7:0/2297932948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe8c003a00 0x7efe8c003e10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7efe9400c190 tx=0x7efe8c005880 comp rx=0 tx=0).stop 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea359e700 1 -- 172.21.15.7:0/2297932948 reap_dead start 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 -- 172.21.15.7:0/2297932948 shutdown_connections 2023-06-09T11:42:01.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 -- 172.21.15.7:0/2297932948 >> 172.21.15.7:0/2297932948 conn(0x7efea40737f0 msgr2=0x7efea40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:01.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 -- 172.21.15.7:0/2297932948 shutdown_connections 2023-06-09T11:42:01.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.783+0000 7efea91ff700 1 -- 172.21.15.7:0/2297932948 wait complete. 2023-06-09T11:42:01.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.785+0000 7efea91ff700 1 Processor -- start 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.785+0000 7efea91ff700 1 -- start start 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c003a00 0x7efea414cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 0x7efea40c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea4158fb0 con 0x7efea4159270 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea41f2760 con 0x7efe8c003a00 2023-06-09T11:42:01.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea40b4990 con 0x7efea40b4430 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42742/0 (socket says 172.21.15.7:42742) 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea259c700 1 -- 172.21.15.7:0/1859850123 learned_addr learned my addr 172.21.15.7:0/1859850123 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea2d9d700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c003a00 0x7efea414cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.786+0000 7efea359e700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 0x7efea40c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 -- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 msgr2=0x7efea40c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 0x7efea40c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 -- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c003a00 msgr2=0x7efea414cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c003a00 0x7efea414cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 -- 172.21.15.7:0/1859850123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4159c60 con 0x7efea4159270 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea359e700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 0x7efea40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea259c700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efe980119a0 tx=0x7efe98011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800f940 con 0x7efea4159270 2023-06-09T11:42:01.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9800faa0 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea4159980 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800fe30 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.787+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea4159f40 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.788+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea4074cf0 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.789+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe98014440 con 0x7efea4159270 2023-06-09T11:42:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.790+0000 7efe8b7fe700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 0x7efe9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.790+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7efe9808a140 con 0x7efea4159270 2023-06-09T11:42:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.790+0000 7efea2d9d700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 0x7efe9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.791+0000 7efea2d9d700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 0x7efe9006d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe94015040 tx=0x7efe9400a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:01.795+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efe98054bb0 con 0x7efea4159270 2023-06-09T11:42:02.044 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2023-06-09T11:42:02.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.048+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efea40c4a00 con 0x7efea4159270 2023-06-09T11:42:02.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.049+0000 7efe8b7fe700 1 -- 172.21.15.7:0/1859850123 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7efe980581d0 con 0x7efea4159270 2023-06-09T11:42:02.052 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:02.053 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 msgr2=0x7efe9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 0x7efe9006d190 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe94015040 tx=0x7efe9400a000 comp rx=0 tx=0).stop 2023-06-09T11:42:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 msgr2=0x7efea40cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7efe980119a0 tx=0x7efe98011cb0 comp rx=0 tx=0).stop 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 shutdown_connections 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efe9006ace0 0x7efe9006d190 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efea40b4430 0x7efea40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe8c003a00 0x7efea414cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 --2- 172.21.15.7:0/1859850123 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efea4159270 0x7efea40cb5e0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 >> 172.21.15.7:0/1859850123 conn(0x7efea40737f0 msgr2=0x7efea40cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 shutdown_connections 2023-06-09T11:42:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:02.051+0000 7efea91ff700 1 -- 172.21.15.7:0/1859850123 wait complete. 2023-06-09T11:42:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:02.073 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:04.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:04.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.282+0000 7fdcc9dff700 1 Processor -- start 2023-06-09T11:42:04.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.282+0000 7fdcc9dff700 1 -- start start 2023-06-09T11:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.283+0000 7fdcc9dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.283+0000 7fdcc9dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 0x7fdcc415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.283+0000 7fdcc9dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc415a500 con 0x7fdcc415b2d0 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.283+0000 7fdcc9dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc40c5f10 con 0x7fdcc40c9f40 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.283+0000 7fdcc9dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc415da00 con 0x7fdcc4153880 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdcc415b2d0 0x7fdcc40c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40786/0 (socket says 172.21.15.7:40786) 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc3fff700 1 -- 172.21.15.7:0/3389090900 learned_addr learned my addr 172.21.15.7:0/3389090900 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3389090900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc2ffd700 1 --2- 172.21.15.7:0/3389090900 >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 0x7fdcc415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc27fc700 1 -- 172.21.15.7:0/3389090900 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533212140 0 0) 0x7fdcbc001490 con 0x7fdcc415b2d0 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc27fc700 1 -- 172.21.15.7:0/3389090900 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcac0035b0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc37fe700 1 -- 172.21.15.7:0/3389090900 >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 msgr2=0x7fdcc415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3389090900 >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 0x7fdcc415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc37fe700 1 -- 172.21.15.7:0/3389090900 >> v1:172.21.15.7:6789/0 conn(0x7fdcc415b2d0 legacy=0x7fdcc40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc37fe700 1 -- 172.21.15.7:0/3389090900 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc415b750 con 0x7fdcc4153880 2023-06-09T11:42:04.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.284+0000 7fdcc2ffd700 1 --2- 172.21.15.7:0/3389090900 >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 0x7fdcc415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.285+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3389090900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc415d4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009840 tx=0x7fdcb4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80cbc38d7ff9bbd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.285+0000 7fdcc27fc700 1 -- 172.21.15.7:0/3389090900 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011830 con 0x7fdcc4153880 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.285+0000 7fdcc27fc700 1 -- 172.21.15.7:0/3389090900 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4011990 con 0x7fdcc4153880 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.285+0000 7fdcc27fc700 1 -- 172.21.15.7:0/3389090900 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4011d00 con 0x7fdcc4153880 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3389090900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 msgr2=0x7fdcc415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3389090900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc415d4c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009840 tx=0x7fdcb4009b50 comp rx=0 tx=0).stop 2023-06-09T11:42:04.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3389090900 shutdown_connections 2023-06-09T11:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3389090900 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc415d4c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3389090900 >> v2:172.21.15.117:3300/0 conn(0x7fdcc40c9f40 0x7fdcc415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3389090900 >> 172.21.15.7:0/3389090900 conn(0x7fdcc40737f0 msgr2=0x7fdcc40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3389090900 shutdown_connections 2023-06-09T11:42:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.286+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3389090900 wait complete. 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.288+0000 7fdcc9dff700 1 Processor -- start 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.288+0000 7fdcc9dff700 1 -- start start 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcc40c9f40 0x7fdcc406a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc41f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4152710 con 0x7fdcc415b2d0 2023-06-09T11:42:04.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4152ac0 con 0x7fdcc40c9f40 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc9dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41581b0 con 0x7fdcc4153880 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45450/0 (socket says 172.21.15.7:45450) 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 -- 172.21.15.7:0/3733433411 learned_addr learned my addr 172.21.15.7:0/3733433411 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcc40c9f40 0x7fdcc406a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc2ffd700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc41f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 -- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 msgr2=0x7fdcc41f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc41f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 -- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcc40c9f40 msgr2=0x7fdcc406a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcc40c9f40 0x7fdcc406a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.289+0000 7fdcc3fff700 1 -- 172.21.15.7:0/3733433411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80094a0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcc3fff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00cae0 tx=0x7fdcbc00cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00a9a0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc00ab00 con 0x7fdcc415b2d0 2023-06-09T11:42:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc00ae90 con 0x7fdcc415b2d0 2023-06-09T11:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb40094a0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.290+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc4159980 con 0x7fdcc415b2d0 2023-06-09T11:42:04.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.291+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc41596e0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.292+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcbc01b3b0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.294+0000 7fdcabfff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 0x7fdcb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.294+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fdcbc0894d0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.294+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 0x7fdcb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:04.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.294+0000 7fdcc37fe700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 0x7fdcb006d1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:04.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.298+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdcbc0579e0 con 0x7fdcc415b2d0 2023-06-09T11:42:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.554+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcc4159c60 con 0x7fdcc415b2d0 2023-06-09T11:42:04.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.554+0000 7fdcabfff700 1 -- 172.21.15.7:0/3733433411 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fdcbc001b70 con 0x7fdcc415b2d0 2023-06-09T11:42:04.558 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:04.558 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 msgr2=0x7fdcb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 0x7fdcb006d1c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4000c00 tx=0x7fdcb4008030 comp rx=0 tx=0).stop 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 msgr2=0x7fdcc41f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00cae0 tx=0x7fdcbc00cdf0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 shutdown_connections 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcc415b2d0 0x7fdcc41f2cc0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcc40c9f40 0x7fdcc406a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdcb006ad10 0x7fdcb006d1c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 --2- 172.21.15.7:0/3733433411 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcc4153880 0x7fdcc41f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 >> 172.21.15.7:0/3733433411 conn(0x7fdcc40737f0 msgr2=0x7fdcc41505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 shutdown_connections 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:04.557+0000 7fdcc9dff700 1 -- 172.21.15.7:0/3733433411 wait complete. 2023-06-09T11:42:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:04.578 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:06.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.788+0000 7fb7cc7ff700 1 Processor -- start 2023-06-09T11:42:06.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.788+0000 7fb7cc7ff700 1 -- start start 2023-06-09T11:42:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.789+0000 7fb7cc7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7c41535e0 0x7fb7c4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.789+0000 7fb7cc7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb7c40c4ce0 0x7fb7c40c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.789+0000 7fb7cc7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c415a420 con 0x7fb7c41539b0 2023-06-09T11:42:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7cc7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c40ca670 con 0x7fb7c41535e0 2023-06-09T11:42:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7cc7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c40c5a00 con 0x7fb7c40c4ce0 2023-06-09T11:42:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7c41535e0 0x7fb7c4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb7c41535e0 0x7fb7c4159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45462/0 (socket says 172.21.15.7:45462) 2023-06-09T11:42:06.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 -- 172.21.15.7:0/1533105264 learned_addr learned my addr 172.21.15.7:0/1533105264 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:06.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7cad9c700 1 --2- 172.21.15.7:0/1533105264 >> v2:172.21.15.117:3300/0 conn(0x7fb7c40c4ce0 0x7fb7c40c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 -- 172.21.15.7:0/1533105264 >> v2:172.21.15.117:3300/0 conn(0x7fb7c40c4ce0 msgr2=0x7fb7c40c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 --2- 172.21.15.7:0/1533105264 >> v2:172.21.15.117:3300/0 conn(0x7fb7c40c4ce0 0x7fb7c40c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 -- 172.21.15.7:0/1533105264 >> v1:172.21.15.7:6789/0 conn(0x7fb7c41539b0 legacy=0x7fb7c40c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7ca59b700 1 -- 172.21.15.7:0/1533105264 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c406b460 con 0x7fb7c41535e0 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.790+0000 7fb7c9599700 1 -- 172.21.15.7:0/1533105264 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401645530 0 0) 0x7fb7b4001110 con 0x7fb7c41539b0 2023-06-09T11:42:06.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7ca59b700 1 --2- 172.21.15.7:0/1533105264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41535e0 0x7fb7c4159ee0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40749f0 tx=0x7fb7c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8062d525b23d44 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7c9599700 1 -- 172.21.15.7:0/1533105264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0011830 con 0x7fb7c41535e0 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7c9599700 1 -- 172.21.15.7:0/1533105264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c0011990 con 0x7fb7c41535e0 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7c9599700 1 -- 172.21.15.7:0/1533105264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0011d20 con 0x7fb7c41535e0 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/1533105264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41535e0 msgr2=0x7fb7c4159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/1533105264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41535e0 0x7fb7c4159ee0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fb7c40749f0 tx=0x7fb7c0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/1533105264 shutdown_connections 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/1533105264 >> v2:172.21.15.117:3300/0 conn(0x7fb7c40c4ce0 0x7fb7c40c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.791+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/1533105264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41535e0 0x7fb7c4159ee0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.792+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/1533105264 >> 172.21.15.7:0/1533105264 conn(0x7fb7c40737f0 msgr2=0x7fb7c4072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.792+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/1533105264 shutdown_connections 2023-06-09T11:42:06.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.792+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/1533105264 wait complete. 2023-06-09T11:42:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.793+0000 7fb7cc7ff700 1 Processor -- start 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.793+0000 7fb7cc7ff700 1 -- start start 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.794+0000 7fb7cc7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40c4ce0 0x7fb7c4151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.794+0000 7fb7cc7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c41535e0 0x7fb7c40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.794+0000 7fb7cc7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cc7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c414db00 con 0x7fb7c41539b0 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cc7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c414e090 con 0x7fb7c40c4ce0 2023-06-09T11:42:06.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cc7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c414e370 con 0x7fb7c41535e0 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45464/0 (socket says 172.21.15.7:45464) 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 -- 172.21.15.7:0/3119028518 learned_addr learned my addr 172.21.15.7:0/3119028518 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7ca59b700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40c4ce0 0x7fb7c4151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7c9d9a700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c41535e0 0x7fb7c40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 -- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c41535e0 msgr2=0x7fb7c40b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c41535e0 0x7fb7c40b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 -- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40c4ce0 msgr2=0x7fb7c4151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40c4ce0 0x7fb7c4151c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 -- 172.21.15.7:0/3119028518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c0009550 con 0x7fb7c41539b0 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.795+0000 7fb7cad9c700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00efc0 tx=0x7fb7bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.796+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc009ea0 con 0x7fb7c41539b0 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.796+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc00f040 con 0x7fb7c41539b0 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.796+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc014480 con 0x7fb7c41539b0 2023-06-09T11:42:06.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.796+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c406a850 con 0x7fb7c41539b0 2023-06-09T11:42:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.796+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c406a570 con 0x7fb7c41539b0 2023-06-09T11:42:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.797+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c406ab30 con 0x7fb7c41539b0 2023-06-09T11:42:06.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.798+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb7bc0145e0 con 0x7fb7c41539b0 2023-06-09T11:42:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.799+0000 7fb7baffd700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 0x7fb7a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.799+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fb7bc08a570 con 0x7fb7c41539b0 2023-06-09T11:42:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.799+0000 7fb7ca59b700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 0x7fb7a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:06.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.800+0000 7fb7ca59b700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 0x7fb7a806d1c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009c10 tx=0x7fb7c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:06.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:06.803+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb7bc055070 con 0x7fb7c41539b0 2023-06-09T11:42:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.059+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7c41e4880 con 0x7fb7c41539b0 2023-06-09T11:42:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.059+0000 7fb7baffd700 1 -- 172.21.15.7:0/3119028518 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fb7c41e4880 con 0x7fb7c41539b0 2023-06-09T11:42:07.063 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:07.063 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 msgr2=0x7fb7a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 0x7fb7a806d1c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009c10 tx=0x7fb7c0008030 comp rx=0 tx=0).stop 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 msgr2=0x7fb7c41e4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00efc0 tx=0x7fb7bc00c3a0 comp rx=0 tx=0).stop 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 shutdown_connections 2023-06-09T11:42:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7c41539b0 0x7fb7c41e4b60 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb7c40c4ce0 0x7fb7c4151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb7a806ad10 0x7fb7a806d1c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 --2- 172.21.15.7:0/3119028518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb7c41535e0 0x7fb7c40b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 >> 172.21.15.7:0/3119028518 conn(0x7fb7c40737f0 msgr2=0x7fb7c4158790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 shutdown_connections 2023-06-09T11:42:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:07.062+0000 7fb7cc7ff700 1 -- 172.21.15.7:0/3119028518 wait complete. 2023-06-09T11:42:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:07.084 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:08.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2023-06-09T11:42:09.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:09.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.291+0000 7f6a933ff700 1 Processor -- start 2023-06-09T11:42:09.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.291+0000 7f6a933ff700 1 -- start start 2023-06-09T11:42:09.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a933ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6a8c0cb9a0 0x7f6a8c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a933ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c159b20 con 0x7f6a8c15cd40 2023-06-09T11:42:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a933ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a8c0cac80 con 0x7f6a8c0cb5d0 2023-06-09T11:42:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a933ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a8c1597d0 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a9199c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6a8c15cd40 0x7f6a8c0cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40802/0 (socket says 172.21.15.7:40802) 2023-06-09T11:42:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a9199c700 1 -- 172.21.15.7:0/2003108646 learned_addr learned my addr 172.21.15.7:0/2003108646 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.292+0000 7f6a9099a700 1 --2- 172.21.15.7:0/2003108646 >> v2:172.21.15.117:3300/0 conn(0x7f6a8c0cb9a0 0x7f6a8c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a83fff700 1 -- 172.21.15.7:0/2003108646 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810283319 0 0) 0x7f6a8c159b20 con 0x7f6a8c15cd40 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a83fff700 1 -- 172.21.15.7:0/2003108646 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a780035b0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a9099a700 1 -- 172.21.15.7:0/2003108646 >> v1:172.21.15.117:6789/0 conn(0x7f6a8c0cb5d0 legacy=0x7f6a8c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a9099a700 1 -- 172.21.15.7:0/2003108646 >> v1:172.21.15.7:6789/0 conn(0x7f6a8c15cd40 legacy=0x7f6a8c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a9099a700 1 -- 172.21.15.7:0/2003108646 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8c071ab0 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a9099a700 1 --2- 172.21.15.7:0/2003108646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0b52b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009840 tx=0x7f6a7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8068ab3aae1500d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a83fff700 1 -- 172.21.15.7:0/2003108646 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c011830 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a83fff700 1 -- 172.21.15.7:0/2003108646 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7c011990 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.293+0000 7f6a83fff700 1 -- 172.21.15.7:0/2003108646 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c011d20 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 -- 172.21.15.7:0/2003108646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 msgr2=0x7f6a8c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 --2- 172.21.15.7:0/2003108646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0b52b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c009840 tx=0x7f6a7c009b50 comp rx=0 tx=0).stop 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 -- 172.21.15.7:0/2003108646 shutdown_connections 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 --2- 172.21.15.7:0/2003108646 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0b52b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 -- 172.21.15.7:0/2003108646 >> 172.21.15.7:0/2003108646 conn(0x7f6a8c070af0 msgr2=0x7f6a8c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 -- 172.21.15.7:0/2003108646 shutdown_connections 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.294+0000 7f6a933ff700 1 -- 172.21.15.7:0/2003108646 wait complete. 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.296+0000 7f6a933ff700 1 Processor -- start 2023-06-09T11:42:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.296+0000 7f6a933ff700 1 -- start start 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a8c0cb5d0 0x7f6a8c0aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0ab2c0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c1f2d20 con 0x7f6a8c0cb9a0 2023-06-09T11:42:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a933ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0c4af0 con 0x7f6a8c0cb5d0 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45472/0 (socket says 172.21.15.7:45472) 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 -- 172.21.15.7:0/1778898838 learned_addr learned my addr 172.21.15.7:0/1778898838 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9119b700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a8c0cb5d0 0x7f6a8c0aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9099a700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 -- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a8c0cb5d0 msgr2=0x7f6a8c0aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a8c0cb5d0 0x7f6a8c0aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 -- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 msgr2=0x7f6a8c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.297+0000 7f6a9199c700 1 -- 172.21.15.7:0/1778898838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0094a0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a9199c700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400caa0 tx=0x7f6a8400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400a9a0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8400ab00 con 0x7f6a8c15cd40 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c15d4f0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8400ae70 con 0x7f6a8c15cd40 2023-06-09T11:42:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.298+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c15d210 con 0x7f6a8c15cd40 2023-06-09T11:42:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.299+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c15d7d0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.300+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a8401b3b0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.301+0000 7f6a81ffb700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.301+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f6a84089590 con 0x7f6a8c15cd40 2023-06-09T11:42:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.301+0000 7f6a9119b700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.302+0000 7f6a9119b700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 0x7f6a7006d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007fd0 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:09.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.305+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6a84054170 con 0x7f6a8c15cd40 2023-06-09T11:42:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.554+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a8c074cf0 con 0x7f6a8c15cd40 2023-06-09T11:42:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.555+0000 7f6a81ffb700 1 -- 172.21.15.7:0/1778898838 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f6a8405b860 con 0x7f6a8c15cd40 2023-06-09T11:42:09.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:09.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 msgr2=0x7f6a7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 0x7f6a7006d190 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a88007fd0 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2023-06-09T11:42:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 msgr2=0x7f6a8c0c5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400caa0 tx=0x7f6a8400cdb0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 shutdown_connections 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8c15cd40 0x7f6a8c0c5ab0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6a7006ace0 0x7f6a7006d190 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6a8c0cb5d0 0x7f6a8c0aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 --2- 172.21.15.7:0/1778898838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6a8c0cb9a0 0x7f6a8c0c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.557+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 >> 172.21.15.7:0/1778898838 conn(0x7f6a8c070af0 msgr2=0x7f6a8c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.558+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 shutdown_connections 2023-06-09T11:42:09.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:09.558+0000 7f6a933ff700 1 -- 172.21.15.7:0/1778898838 wait complete. 2023-06-09T11:42:09.563 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:09.579 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:11.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:11.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.789+0000 7f2c035ff700 1 Processor -- start 2023-06-09T11:42:11.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.789+0000 7f2c035ff700 1 -- start start 2023-06-09T11:42:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bfc153880 0x7f2bfc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2bfc0c9f40 0x7f2bfc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 0x7f2bfc0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc0c5f10 con 0x7f2bfc153880 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc15a500 con 0x7f2bfc0c9f40 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c035ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc15da00 con 0x7f2bfc15b2d0 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c0139b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bfc153880 0x7f2bfc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c0139b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2bfc153880 0x7f2bfc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45482/0 (socket says 172.21.15.7:45482) 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.790+0000 7f2c0139b700 1 -- 172.21.15.7:0/191551880 learned_addr learned my addr 172.21.15.7:0/191551880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c01b9c700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 0x7f2bfc0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c00b9a700 1 --2- 172.21.15.7:0/191551880 >> v2:172.21.15.117:3300/0 conn(0x7f2bfc0c9f40 0x7f2bfc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 -- 172.21.15.7:0/191551880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 msgr2=0x7f2bfc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 0x7f2bfc0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 -- 172.21.15.7:0/191551880 >> v2:172.21.15.117:3300/0 conn(0x7f2bfc0c9f40 msgr2=0x7f2bfc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 --2- 172.21.15.7:0/191551880 >> v2:172.21.15.117:3300/0 conn(0x7f2bfc0c9f40 0x7f2bfc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 -- 172.21.15.7:0/191551880 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc1596e0 con 0x7f2bfc153880 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c01b9c700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 0x7f2bfc0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2c0139b700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc15d4c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0cce30 tx=0x7f2bf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc91e4592a089cfd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:11.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2bf3fff700 1 -- 172.21.15.7:0/191551880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8011830 con 0x7f2bfc153880 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2bf3fff700 1 -- 172.21.15.7:0/191551880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf8011990 con 0x7f2bfc153880 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.791+0000 7f2bf3fff700 1 -- 172.21.15.7:0/191551880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8011d20 con 0x7f2bfc153880 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 -- 172.21.15.7:0/191551880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 msgr2=0x7f2bfc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc15d4c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0cce30 tx=0x7f2bf8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 -- 172.21.15.7:0/191551880 shutdown_connections 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc15b2d0 0x7f2bfc0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 --2- 172.21.15.7:0/191551880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc15d4c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 --2- 172.21.15.7:0/191551880 >> v2:172.21.15.117:3300/0 conn(0x7f2bfc0c9f40 0x7f2bfc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 -- 172.21.15.7:0/191551880 >> 172.21.15.7:0/191551880 conn(0x7f2bfc0737f0 msgr2=0x7f2bfc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 -- 172.21.15.7:0/191551880 shutdown_connections 2023-06-09T11:42:11.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.792+0000 7f2c035ff700 1 -- 172.21.15.7:0/191551880 wait complete. 2023-06-09T11:42:11.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.794+0000 7f2c035ff700 1 Processor -- start 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.794+0000 7f2c035ff700 1 -- start start 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc0c9f40 0x7f2bfc1f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bfc15b2d0 0x7f2bfc0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc154380 con 0x7f2bfc153880 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc152830 con 0x7f2bfc15b2d0 2023-06-09T11:42:11.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c035ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0b5280 con 0x7f2bfc0c9f40 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45496/0 (socket says 172.21.15.7:45496) 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 -- 172.21.15.7:0/389972631 learned_addr learned my addr 172.21.15.7:0/389972631 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c0139b700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc0c9f40 0x7f2bfc1f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c01b9c700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bfc15b2d0 0x7f2bfc0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 -- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc0c9f40 msgr2=0x7f2bfc1f26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:11.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc0c9f40 0x7f2bfc1f26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 -- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bfc15b2d0 msgr2=0x7f2bfc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bfc15b2d0 0x7f2bfc0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.795+0000 7f2c00b9a700 1 -- 172.21.15.7:0/389972631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf8009550 con 0x7f2bfc153880 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2c00b9a700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e930 tx=0x7f2bec00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec013680 con 0x7f2bfc153880 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec0137e0 con 0x7f2bfc153880 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc159980 con 0x7f2bfc153880 2023-06-09T11:42:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec013b70 con 0x7f2bfc153880 2023-06-09T11:42:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.796+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc15b730 con 0x7f2bfc153880 2023-06-09T11:42:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.797+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc159c60 con 0x7f2bfc153880 2023-06-09T11:42:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.798+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bec013cd0 con 0x7f2bfc153880 2023-06-09T11:42:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.799+0000 7f2bf1ffb700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 0x7f2be006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.799+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f2bec089a40 con 0x7f2bfc153880 2023-06-09T11:42:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.800+0000 7f2c0139b700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 0x7f2be006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:11.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.800+0000 7f2c0139b700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 0x7f2be006d1c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009c10 tx=0x7f2bf8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:11.804+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bec08be60 con 0x7f2bfc153880 2023-06-09T11:42:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.055+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bfc152970 con 0x7f2bfc153880 2023-06-09T11:42:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.056+0000 7f2bf1ffb700 1 -- 172.21.15.7:0/389972631 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f2bfc152970 con 0x7f2bfc153880 2023-06-09T11:42:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:12.060 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 msgr2=0x7f2be006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 0x7f2be006d1c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009c10 tx=0x7f2bf8008030 comp rx=0 tx=0).stop 2023-06-09T11:42:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 msgr2=0x7f2bfc152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2bec00e930 tx=0x7f2bec00ec40 comp rx=0 tx=0).stop 2023-06-09T11:42:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 shutdown_connections 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2bfc15b2d0 0x7f2bfc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be006ad10 0x7f2be006d1c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2bfc0c9f40 0x7f2bfc1f26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 --2- 172.21.15.7:0/389972631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bfc153880 0x7f2bfc152c50 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 >> 172.21.15.7:0/389972631 conn(0x7f2bfc0737f0 msgr2=0x7f2bfc0cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 shutdown_connections 2023-06-09T11:42:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:12.058+0000 7f2c035ff700 1 -- 172.21.15.7:0/389972631 wait complete. 2023-06-09T11:42:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:12.079 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:14.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.288+0000 7fa93bdff700 1 Processor -- start 2023-06-09T11:42:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.289+0000 7fa93bdff700 1 -- start start 2023-06-09T11:42:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93bdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa93415b4a0 0x7fa93415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93bdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9340c9f40 0x7fa9340c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93bdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93415a500 con 0x7fa934153880 2023-06-09T11:42:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93bdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9340c5f10 con 0x7fa9340c9f40 2023-06-09T11:42:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93bdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93415da00 con 0x7fa93415b4a0 2023-06-09T11:42:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93a39c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9340c9f40 0x7fa9340c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93a39c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9340c9f40 0x7fa9340c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53010/0 (socket says 172.21.15.7:53010) 2023-06-09T11:42:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93a39c700 1 -- 172.21.15.7:0/1659127260 learned_addr learned my addr 172.21.15.7:0/1659127260 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.290+0000 7fa93939a700 1 --2- 172.21.15.7:0/1659127260 >> v2:172.21.15.117:3300/0 conn(0x7fa93415b4a0 0x7fa93415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa93a39c700 1 -- 172.21.15.7:0/1659127260 >> v2:172.21.15.117:3300/0 conn(0x7fa93415b4a0 msgr2=0x7fa93415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa93a39c700 1 --2- 172.21.15.7:0/1659127260 >> v2:172.21.15.117:3300/0 conn(0x7fa93415b4a0 0x7fa93415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa93a39c700 1 -- 172.21.15.7:0/1659127260 >> v1:172.21.15.117:6789/0 conn(0x7fa934153880 legacy=0x7fa93415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa93a39c700 1 -- 172.21.15.7:0/1659127260 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9341596e0 con 0x7fa9340c9f40 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa93a39c700 1 --2- 172.21.15.7:0/1659127260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9340c9f40 0x7fa9340c4e30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa9340cce30 tx=0x7fa92c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724304d952de7e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa938b99700 1 -- 172.21.15.7:0/1659127260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00e070 con 0x7fa9340c9f40 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa938b99700 1 -- 172.21.15.7:0/1659127260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c00b910 con 0x7fa9340c9f40 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.291+0000 7fa938b99700 1 -- 172.21.15.7:0/1659127260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c00bcc0 con 0x7fa9340c9f40 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 -- 172.21.15.7:0/1659127260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9340c9f40 msgr2=0x7fa9340c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1659127260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9340c9f40 0x7fa9340c4e30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa9340cce30 tx=0x7fa92c00b330 comp rx=0 tx=0).stop 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 -- 172.21.15.7:0/1659127260 shutdown_connections 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1659127260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9340c9f40 0x7fa9340c4e30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1659127260 >> v2:172.21.15.117:3300/0 conn(0x7fa93415b4a0 0x7fa93415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 -- 172.21.15.7:0/1659127260 >> 172.21.15.7:0/1659127260 conn(0x7fa9340737f0 msgr2=0x7fa9340720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 -- 172.21.15.7:0/1659127260 shutdown_connections 2023-06-09T11:42:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.292+0000 7fa93bdff700 1 -- 172.21.15.7:0/1659127260 wait complete. 2023-06-09T11:42:14.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.294+0000 7fa93bdff700 1 Processor -- start 2023-06-09T11:42:14.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.294+0000 7fa93bdff700 1 -- start start 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9340c9f40 0x7fa9341519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa93415b4a0 0x7fa93406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340ced00 con 0x7fa934153880 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934151540 con 0x7fa93415b4a0 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9340cd970 con 0x7fa9340c9f40 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93939a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93939a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53018/0 (socket says 172.21.15.7:53018) 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93939a700 1 -- 172.21.15.7:0/1118372614 learned_addr learned my addr 172.21.15.7:0/1118372614 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa939b9b700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9340c9f40 0x7fa9341519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.295+0000 7fa93a39c700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa93415b4a0 0x7fa93406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 -- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9340c9f40 msgr2=0x7fa9341519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9340c9f40 0x7fa9341519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 -- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa93415b4a0 msgr2=0x7fa93406a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa93415b4a0 0x7fa93406a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 -- 172.21.15.7:0/1118372614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c009b30 con 0x7fa934153880 2023-06-09T11:42:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93939a700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e870 tx=0x7fa92400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa924009590 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa924013680 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9240139f0 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa934159c60 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.296+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa934159980 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.297+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa934159f40 con 0x7fa934153880 2023-06-09T11:42:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.299+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa924013b50 con 0x7fa934153880 2023-06-09T11:42:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.300+0000 7fa92a7fc700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 0x7fa91806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.300+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7fa924089950 con 0x7fa934153880 2023-06-09T11:42:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.300+0000 7fa939b9b700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 0x7fa91806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.301+0000 7fa939b9b700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 0x7fa91806d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa930007fd0 tx=0x7fa930009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.304+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa924057e50 con 0x7fa934153880 2023-06-09T11:42:14.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.557+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa93414e350 con 0x7fa934153880 2023-06-09T11:42:14.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.557+0000 7fa92a7fc700 1 -- 172.21.15.7:0/1118372614 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fa9240579e0 con 0x7fa934153880 2023-06-09T11:42:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:14.561 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 msgr2=0x7fa91806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 0x7fa91806d190 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa930007fd0 tx=0x7fa930009040 comp rx=0 tx=0).stop 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 msgr2=0x7fa9340caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa92400e870 tx=0x7fa92400eb80 comp rx=0 tx=0).stop 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 shutdown_connections 2023-06-09T11:42:14.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa93415b4a0 0x7fa93406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa91806ace0 0x7fa91806d190 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9340c9f40 0x7fa9341519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 --2- 172.21.15.7:0/1118372614 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa934153880 0x7fa9340caa80 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 >> 172.21.15.7:0/1118372614 conn(0x7fa9340737f0 msgr2=0x7fa93414d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 shutdown_connections 2023-06-09T11:42:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:14.560+0000 7fa93bdff700 1 -- 172.21.15.7:0/1118372614 wait complete. 2023-06-09T11:42:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:14.582 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:16.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.790+0000 7f2c757ff700 1 Processor -- start 2023-06-09T11:42:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.790+0000 7f2c757ff700 1 -- start start 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c757ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c700c1bc0 0x7f2c701599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c757ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c70159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c757ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c700c0bf0 con 0x7f2c700c1f90 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c757ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c70070110 con 0x7f2c700c1bc0 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c757ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70071300 con 0x7f2c700c0140 2023-06-09T11:42:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c6effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c700c1bc0 0x7f2c701599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c6effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c700c1bc0 0x7f2c701599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53024/0 (socket says 172.21.15.7:53024) 2023-06-09T11:42:16.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.791+0000 7f2c6effd700 1 -- 172.21.15.7:0/3989728838 learned_addr learned my addr 172.21.15.7:0/3989728838 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6e7fc700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c70159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6effd700 1 -- 172.21.15.7:0/3989728838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 msgr2=0x7f2c70159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6effd700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c70159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6effd700 1 -- 172.21.15.7:0/3989728838 >> v1:172.21.15.117:6789/0 conn(0x7f2c700c1f90 legacy=0x7f2c701501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6effd700 1 -- 172.21.15.7:0/3989728838 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700702f0 con 0x7f2c700c1bc0 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6effd700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701599b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2c600098f0 tx=0x7f2c60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4b6cc9a4793225 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6dffb700 1 -- 172.21.15.7:0/3989728838 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000b8a0 con 0x7f2c700c1bc0 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6dffb700 1 -- 172.21.15.7:0/3989728838 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6000ba00 con 0x7f2c700c1bc0 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.792+0000 7f2c6dffb700 1 -- 172.21.15.7:0/3989728838 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6000bd90 con 0x7f2c700c1bc0 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 -- 172.21.15.7:0/3989728838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 msgr2=0x7f2c701599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701599b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2c600098f0 tx=0x7f2c60009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:16.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 -- 172.21.15.7:0/3989728838 shutdown_connections 2023-06-09T11:42:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701599b0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 --2- 172.21.15.7:0/3989728838 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c70159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 -- 172.21.15.7:0/3989728838 >> 172.21.15.7:0/3989728838 conn(0x7f2c7006a4d0 msgr2=0x7f2c7006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 -- 172.21.15.7:0/3989728838 shutdown_connections 2023-06-09T11:42:16.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.793+0000 7f2c757ff700 1 -- 172.21.15.7:0/3989728838 wait complete. 2023-06-09T11:42:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.795+0000 7f2c757ff700 1 Processor -- start 2023-06-09T11:42:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.795+0000 7f2c757ff700 1 -- start start 2023-06-09T11:42:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c700ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c700c1f90 0x7f2c700c2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700ad880 con 0x7f2c700c1bc0 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c701ef000 con 0x7f2c700c1f90 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c757ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70151cf0 con 0x7f2c700c0140 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c6e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53026/0 (socket says 172.21.15.7:53026) 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c6e7fc700 1 -- 172.21.15.7:0/2103223071 learned_addr learned my addr 172.21.15.7:0/2103223071 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c6effd700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c700ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.796+0000 7f2c6f7fe700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c700c1f90 0x7f2c700c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 -- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 msgr2=0x7f2c700ca260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c700ca260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 -- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c700c1f90 msgr2=0x7f2c700c2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c700c1f90 0x7f2c700c2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 -- 172.21.15.7:0/2103223071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60009550 con 0x7f2c700c1bc0 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c6e7fc700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e810 tx=0x7f2c6400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:16.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400c070 con 0x7f2c700c1bc0 2023-06-09T11:42:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c64013610 con 0x7f2c700c1bc0 2023-06-09T11:42:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c640139c0 con 0x7f2c700c1bc0 2023-06-09T11:42:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c700c1170 con 0x7f2c700c1bc0 2023-06-09T11:42:16.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.797+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c700c0e90 con 0x7f2c700c1bc0 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.798+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c700c1450 con 0x7f2c700c1bc0 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.799+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c64013b20 con 0x7f2c700c1bc0 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.800+0000 7f2c577fe700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 0x7f2c5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.800+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f2c640898b0 con 0x7f2c700c1bc0 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.800+0000 7f2c6effd700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 0x7f2c5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.801+0000 7f2c6effd700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 0x7f2c5c06d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009c40 tx=0x7f2c60008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:16.805+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2c640542a0 con 0x7f2c700c1bc0 2023-06-09T11:42:17.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.062+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c70070b40 con 0x7f2c700c1bc0 2023-06-09T11:42:17.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.063+0000 7f2c577fe700 1 -- 172.21.15.7:0/2103223071 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 140 v140) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f2c640578c0 con 0x7f2c700c1bc0 2023-06-09T11:42:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:17.067 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":140,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6859,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":139}],"filesystems":[{"mdsmap":{"epoch":140,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:41:16.534526+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":5994},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5994":{"gid":5994,"name":"e","rank":2,"incarnation":73,"state":"up:active","state_seq":41,"addr":"172.21.15.7:6839/2829758995","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":2829758995},{"type":"v1","addr":"172.21.15.7:6839","nonce":2829758995}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 msgr2=0x7f2c5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 0x7f2c5c06d190 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009c40 tx=0x7f2c60008030 comp rx=0 tx=0).stop 2023-06-09T11:42:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 msgr2=0x7f2c701ef490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400e810 tx=0x7f2c6400eb20 comp rx=0 tx=0).stop 2023-06-09T11:42:17.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 shutdown_connections 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c700c1f90 0x7f2c700c2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c5c06ace0 0x7f2c5c06d190 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c700c0140 0x7f2c700ca260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 --2- 172.21.15.7:0/2103223071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c700c1bc0 0x7f2c701ef490 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 >> 172.21.15.7:0/2103223071 conn(0x7f2c7006a4d0 msgr2=0x7f2c701538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 shutdown_connections 2023-06-09T11:42:17.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:17.065+0000 7f2c757ff700 1 -- 172.21.15.7:0/2103223071 wait complete. 2023-06-09T11:42:17.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 140 2023-06-09T11:42:17.087 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:42:19.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:19.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:19.139 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.299+0000 7f0b319ff700 1 Processor -- start 2023-06-09T11:42:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.299+0000 7f0b319ff700 1 -- start start 2023-06-09T11:42:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b319ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b2c072690 0x7f0b2c0c7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b319ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0c7080 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b319ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2c0c7eb0 con 0x7f0b2c0722c0 2023-06-09T11:42:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b319ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b2c157730 con 0x7f0b2c072690 2023-06-09T11:42:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b2b7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b2c0c7a50 0x7f0b2c150de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34930/0 (socket says 172.21.15.7:34930) 2023-06-09T11:42:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b2b7fe700 1 -- 172.21.15.7:0/2638361911 learned_addr learned my addr 172.21.15.7:0/2638361911 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b2a7fc700 1 --2- 172.21.15.7:0/2638361911 >> v2:172.21.15.117:3300/0 conn(0x7f0b2c072690 0x7f0b2c0c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.301+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003326966 0 0) 0x7f0b2c0c7080 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b140035b0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2246019759 0 0) 0x7f0b2c0c7eb0 con 0x7f0b2c0722c0 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b2c0c7080 con 0x7f0b2c0722c0 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b2a7fc700 1 -- 172.21.15.7:0/2638361911 >> v1:172.21.15.117:6789/0 conn(0x7f0b2c0722c0 legacy=0x7f0b2c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b2a7fc700 1 -- 172.21.15.7:0/2638361911 >> v1:172.21.15.7:6789/0 conn(0x7f0b2c0c7a50 legacy=0x7f0b2c150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b2a7fc700 1 -- 172.21.15.7:0/2638361911 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b2c1579f0 con 0x7f0b2c072690 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b2a7fc700 1 --2- 172.21.15.7:0/2638361911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c0c7510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009840 tx=0x7f0b20009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41ae7578c28be84d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20011830 con 0x7f0b2c072690 2023-06-09T11:42:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b20011990 con 0x7f0b2c072690 2023-06-09T11:42:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.302+0000 7f0b29ffb700 1 -- 172.21.15.7:0/2638361911 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20011d20 con 0x7f0b2c072690 2023-06-09T11:42:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 -- 172.21.15.7:0/2638361911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 msgr2=0x7f0b2c0c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 --2- 172.21.15.7:0/2638361911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c0c7510 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b20009840 tx=0x7f0b20009b50 comp rx=0 tx=0).stop 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 -- 172.21.15.7:0/2638361911 shutdown_connections 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 --2- 172.21.15.7:0/2638361911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c0c7510 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 -- 172.21.15.7:0/2638361911 >> 172.21.15.7:0/2638361911 conn(0x7f0b2c06dc00 msgr2=0x7f0b2c06e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 -- 172.21.15.7:0/2638361911 shutdown_connections 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.303+0000 7f0b319ff700 1 -- 172.21.15.7:0/2638361911 wait complete. 2023-06-09T11:42:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.305+0000 7f0b319ff700 1 Processor -- start 2023-06-09T11:42:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.305+0000 7f0b319ff700 1 -- start start 2023-06-09T11:42:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0722c0 0x7f0b2c147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c14ef50 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c1efc60 con 0x7f0b2c072690 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b319ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2c148490 con 0x7f0b2c0722c0 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53030/0 (socket says 172.21.15.7:53030) 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 -- 172.21.15.7:0/1061240784 learned_addr learned my addr 172.21.15.7:0/1061240784 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2a7fc700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2affd700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0722c0 0x7f0b2c147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 -- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0722c0 msgr2=0x7f0b2c147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0722c0 0x7f0b2c147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 -- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 msgr2=0x7f0b2c14f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c14f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.306+0000 7f0b2b7fe700 1 -- 172.21.15.7:0/1061240784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200094a0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b2b7fe700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400cbd0 tx=0x7f0b2400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2400aa00 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2400ab60 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c157fb0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24012400 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.307+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2c157cd0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.308+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2c158290 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.309+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b2401b3b0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.310+0000 7f0b13fff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 0x7f0b1806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.310+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 7496+0+0 (secure 0 0 0) 0x7f0b24089830 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.310+0000 7f0b2affd700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 0x7f0b1806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.311+0000 7f0b2affd700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 0x7f0b1806d1c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0011e0 tx=0x7f0b1c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.314+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0b24054210 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.568+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b2c074cf0 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.568+0000 7f0b13fff700 1 -- 172.21.15.7:0/1061240784 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f0b24057830 con 0x7f0b2c0c7a50 2023-06-09T11:42:19.573 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:19.573 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:19.541301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.571+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 msgr2=0x7f0b1806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.571+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 0x7f0b1806d1c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0011e0 tx=0x7f0b1c009040 comp rx=0 tx=0).stop 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.571+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 msgr2=0x7f0b2c1f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.571+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0b2400cbd0 tx=0x7f0b2400cee0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 shutdown_connections 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b2c0c7a50 0x7f0b2c1f00f0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b1806ad10 0x7f0b1806d1c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b2c0722c0 0x7f0b2c147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 --2- 172.21.15.7:0/1061240784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b2c072690 0x7f0b2c14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 >> 172.21.15.7:0/1061240784 conn(0x7f0b2c06dc00 msgr2=0x7f0b2c070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 shutdown_connections 2023-06-09T11:42:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.572+0000 7f0b319ff700 1 -- 172.21.15.7:0/1061240784 wait complete. 2023-06-09T11:42:19.578 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 141 2023-06-09T11:42:19.599 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-06-09T11:42:19.599 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:42:19.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:19.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.796+0000 7fd5863ff700 1 Processor -- start 2023-06-09T11:42:19.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.796+0000 7fd5863ff700 1 -- start start 2023-06-09T11:42:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd5863ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5800c0140 0x7fd580161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd5863ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5800c0bf0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd5863ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5800702f0 con 0x7fd58015a0c0 2023-06-09T11:42:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd5863ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd580070110 con 0x7fd5800c0140 2023-06-09T11:42:19.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd57ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd5800c1bc0 0x7fd5801599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34932/0 (socket says 172.21.15.7:34932) 2023-06-09T11:42:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd57ffff700 1 -- 172.21.15.7:0/2175515136 learned_addr learned my addr 172.21.15.7:0/2175515136 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:19.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.797+0000 7fd58499c700 1 --2- 172.21.15.7:0/2175515136 >> v2:172.21.15.117:3300/0 conn(0x7fd5800c0140 0x7fd580161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd57effd700 1 -- 172.21.15.7:0/2175515136 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473097920 0 0) 0x7fd5800c0bf0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd57effd700 1 -- 172.21.15.7:0/2175515136 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5680057c0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd58499c700 1 -- 172.21.15.7:0/2175515136 >> v1:172.21.15.117:6789/0 conn(0x7fd58015a0c0 legacy=0x7fd5801501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd58499c700 1 -- 172.21.15.7:0/2175515136 >> v1:172.21.15.7:6789/0 conn(0x7fd5800c1bc0 legacy=0x7fd5801599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd58499c700 1 -- 172.21.15.7:0/2175515136 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800c0e90 con 0x7fd5800c0140 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.798+0000 7fd58499c700 1 --2- 172.21.15.7:0/2175515136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd580161cf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd578009d90 tx=0x7fd57800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb74cfb8761348e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd57effd700 1 -- 172.21.15.7:0/2175515136 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57800e070 con 0x7fd5800c0140 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd57effd700 1 -- 172.21.15.7:0/2175515136 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57800bb20 con 0x7fd5800c0140 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd57effd700 1 -- 172.21.15.7:0/2175515136 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578012430 con 0x7fd5800c0140 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 -- 172.21.15.7:0/2175515136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 msgr2=0x7fd580161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 --2- 172.21.15.7:0/2175515136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd580161cf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd578009d90 tx=0x7fd57800b330 comp rx=0 tx=0).stop 2023-06-09T11:42:19.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 -- 172.21.15.7:0/2175515136 shutdown_connections 2023-06-09T11:42:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 --2- 172.21.15.7:0/2175515136 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd580161cf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 -- 172.21.15.7:0/2175515136 >> 172.21.15.7:0/2175515136 conn(0x7fd58006a4d0 msgr2=0x7fd58006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 -- 172.21.15.7:0/2175515136 shutdown_connections 2023-06-09T11:42:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.799+0000 7fd5863ff700 1 -- 172.21.15.7:0/2175515136 wait complete. 2023-06-09T11:42:19.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.802+0000 7fd5863ff700 1 Processor -- start 2023-06-09T11:42:19.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.802+0000 7fd5863ff700 1 -- start start 2023-06-09T11:42:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd58014b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd58015a0c0 0x7fd580158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800c9700 con 0x7fd5800c1bc0 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58014b7c0 con 0x7fd5800c0140 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd5863ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58014db60 con 0x7fd58015a0c0 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd57f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd57f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53046/0 (socket says 172.21.15.7:53046) 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.803+0000 7fd57f7fe700 1 -- 172.21.15.7:0/4221508297 learned_addr learned my addr 172.21.15.7:0/4221508297 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57ffff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd58014b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd58499c700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd58015a0c0 0x7fd580158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 -- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd58015a0c0 msgr2=0x7fd580158930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd58015a0c0 0x7fd580158930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 -- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 msgr2=0x7fd58014b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd58014b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 -- 172.21.15.7:0/4221508297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd578009a70 con 0x7fd5800c1bc0 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57f7fe700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c420 tx=0x7fd57400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740052d0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574005430 con 0x7fd5800c1bc0 2023-06-09T11:42:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.804+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5800c1450 con 0x7fd5800c1bc0 2023-06-09T11:42:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.805+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740038c0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.805+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5800c1170 con 0x7fd5800c1bc0 2023-06-09T11:42:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.806+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5800705d0 con 0x7fd5800c1bc0 2023-06-09T11:42:19.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.807+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd574003a20 con 0x7fd5800c1bc0 2023-06-09T11:42:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.808+0000 7fd57cff9700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.808+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fd574088d90 con 0x7fd5800c1bc0 2023-06-09T11:42:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.808+0000 7fd57ffff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:19.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.808+0000 7fd57ffff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd570007fd0 tx=0x7fd570007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:19.812+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd574053840 con 0x7fd5800c1bc0 2023-06-09T11:42:20.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.073+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd580074cf0 con 0x7fd5800c1bc0 2023-06-09T11:42:20.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.073+0000 7fd57cff9700 1 -- 172.21.15.7:0/4221508297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fd574056e60 con 0x7fd5800c1bc0 2023-06-09T11:42:20.078 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:20.078 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:19.541301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 msgr2=0x7fd56c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:20.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 0x7fd56c06d190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd570007fd0 tx=0x7fd570007dc0 comp rx=0 tx=0).stop 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 msgr2=0x7fd580154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c420 tx=0x7fd57400c730 comp rx=0 tx=0).stop 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 shutdown_connections 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd56c06ace0 0x7fd56c06d190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.075+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd58015a0c0 0x7fd580158930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.076+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5800c0140 0x7fd58014b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:20.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.076+0000 7fd5863ff700 1 --2- 172.21.15.7:0/4221508297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5800c1bc0 0x7fd580154be0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.076+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 >> 172.21.15.7:0/4221508297 conn(0x7fd58006a4d0 msgr2=0x7fd5800c8460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.076+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 shutdown_connections 2023-06-09T11:42:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:20.076+0000 7fd5863ff700 1 -- 172.21.15.7:0/4221508297 wait complete. 2023-06-09T11:42:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 141 2023-06-09T11:42:20.099 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 141, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 141, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:42:19.541301+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:42:22.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:22.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.306+0000 7f541cbff700 1 Processor -- start 2023-06-09T11:42:22.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.306+0000 7f541cbff700 1 -- start start 2023-06-09T11:42:22.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.307+0000 7f541cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 0x7f5418159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.307+0000 7f541cbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f54180c4ce0 0x7f54180c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541cbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541815a420 con 0x7f54181535e0 2023-06-09T11:42:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541cbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54180ca670 con 0x7f54180c4ce0 2023-06-09T11:42:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180c5a00 con 0x7f54181539b0 2023-06-09T11:42:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54181535e0 0x7f54180c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34936/0 (socket says 172.21.15.7:34936) 2023-06-09T11:42:22.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541659c700 1 -- 172.21.15.7:0/886529199 learned_addr learned my addr 172.21.15.7:0/886529199 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f5416d9d700 1 --2- 172.21.15.7:0/886529199 >> v2:172.21.15.117:3300/0 conn(0x7f54180c4ce0 0x7f54180c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f5415d9b700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 0x7f5418159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541559a700 1 -- 172.21.15.7:0/886529199 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 910152795 0 0) 0x7f5408001110 con 0x7f54181535e0 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f541559a700 1 -- 172.21.15.7:0/886529199 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54000035b0 con 0x7f54181535e0 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.308+0000 7f5416d9d700 1 -- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 msgr2=0x7f5418159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f5416d9d700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 0x7f5418159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f5416d9d700 1 -- 172.21.15.7:0/886529199 >> v1:172.21.15.7:6789/0 conn(0x7f54181535e0 legacy=0x7f54180c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f5416d9d700 1 -- 172.21.15.7:0/886529199 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f541806b460 con 0x7f54180c4ce0 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f5415d9b700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 0x7f5418159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f5416d9d700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54180c4ce0 0x7f54180c5180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb059a4a6d968ca8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f541559a700 1 -- 172.21.15.7:0/886529199 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f541000e070 con 0x7f54180c4ce0 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f541559a700 1 -- 172.21.15.7:0/886529199 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f541000bb20 con 0x7f54180c4ce0 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.309+0000 7f541559a700 1 -- 172.21.15.7:0/886529199 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410012430 con 0x7f54180c4ce0 2023-06-09T11:42:22.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 -- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54180c4ce0 msgr2=0x7f54180c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54180c4ce0 0x7f54180c5180 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d90 tx=0x7f541000b330 comp rx=0 tx=0).stop 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 -- 172.21.15.7:0/886529199 shutdown_connections 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54180c4ce0 0x7f54180c5180 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 --2- 172.21.15.7:0/886529199 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54181539b0 0x7f5418159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 -- 172.21.15.7:0/886529199 >> 172.21.15.7:0/886529199 conn(0x7f54180737f0 msgr2=0x7f5418072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 -- 172.21.15.7:0/886529199 shutdown_connections 2023-06-09T11:42:22.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.310+0000 7f541cbff700 1 -- 172.21.15.7:0/886529199 wait complete. 2023-06-09T11:42:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.312+0000 7f541cbff700 1 Processor -- start 2023-06-09T11:42:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.312+0000 7f541cbff700 1 -- start start 2023-06-09T11:42:22.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180c4ce0 0x7f54181f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54181539b0 0x7f54180c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180d0290 con 0x7f54181535e0 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54181f2890 con 0x7f54181539b0 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5418159200 con 0x7f54180c4ce0 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53062/0 (socket says 172.21.15.7:53062) 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 -- 172.21.15.7:0/478524943 learned_addr learned my addr 172.21.15.7:0/478524943 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f541659c700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180c4ce0 0x7f54181f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5416d9d700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54181539b0 0x7f54180c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 -- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180c4ce0 msgr2=0x7f54181f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180c4ce0 0x7f54181f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 -- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54181539b0 msgr2=0x7f54180c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54181539b0 0x7f54180c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.313+0000 7f5415d9b700 1 -- 172.21.15.7:0/478524943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0094a0 con 0x7f54181535e0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f5415d9b700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f540c00e9b0 tx=0x7f540c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c00ee50 con 0x7f54181535e0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c00c850 con 0x7f54181535e0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c00cc00 con 0x7f54181535e0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5410009a70 con 0x7f54181535e0 2023-06-09T11:42:22.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.314+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541806a850 con 0x7f54181535e0 2023-06-09T11:42:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.315+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541806a570 con 0x7f54181535e0 2023-06-09T11:42:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.316+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f540c013470 con 0x7f54181535e0 2023-06-09T11:42:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.317+0000 7f5406ffd700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 0x7f53f806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:22.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.317+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f540c089780 con 0x7f54181535e0 2023-06-09T11:42:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.317+0000 7f541659c700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 0x7f53f806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:22.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.318+0000 7f541659c700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 0x7f53f806d1c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5408007fd0 tx=0x7f5408007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:22.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.321+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f540c0541a0 con 0x7f54181535e0 2023-06-09T11:42:22.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.574+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f541806ab30 con 0x7f54181535e0 2023-06-09T11:42:22.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.574+0000 7f5406ffd700 1 -- 172.21.15.7:0/478524943 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f540c0577c0 con 0x7f54181535e0 2023-06-09T11:42:22.579 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:22.580 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:19.541301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:22.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.576+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 msgr2=0x7f53f806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.576+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 0x7f53f806d1c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5408007fd0 tx=0x7f5408007c40 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.576+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 msgr2=0x7f54180c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.576+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f540c00e9b0 tx=0x7f540c00ecc0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 shutdown_connections 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f54181539b0 0x7f54180c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f53f806ad10 0x7f53f806d1c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f54180c4ce0 0x7f54181f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 --2- 172.21.15.7:0/478524943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54181535e0 0x7f54180c7560 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:22.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 >> 172.21.15.7:0/478524943 conn(0x7f54180737f0 msgr2=0x7f54181519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:22.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 shutdown_connections 2023-06-09T11:42:22.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:22.577+0000 7f541cbff700 1 -- 172.21.15.7:0/478524943 wait complete. 2023-06-09T11:42:22.583 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 141 2023-06-09T11:42:22.600 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:24.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.799+0000 7fd4a89ff700 1 Processor -- start 2023-06-09T11:42:24.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.800+0000 7fd4a89ff700 1 -- start start 2023-06-09T11:42:24.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a89ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a89ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a4159b20 con 0x7fd4a415cd40 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a89ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a40cac80 con 0x7fd4a40cb9a0 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a89ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a41597d0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56088/0 (socket says 172.21.15.7:56088) 2023-06-09T11:42:24.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 -- 172.21.15.7:0/3025279114 learned_addr learned my addr 172.21.15.7:0/3025279114 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 -- 172.21.15.7:0/3025279114 >> v1:172.21.15.117:6789/0 conn(0x7fd4a40cb9a0 legacy=0x7fd4a4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:24.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 -- 172.21.15.7:0/3025279114 >> v1:172.21.15.7:6789/0 conn(0x7fd4a415cd40 legacy=0x7fd4a40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.801+0000 7fd4a259c700 1 -- 172.21.15.7:0/3025279114 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4071ab0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a259c700 1 --2- 172.21.15.7:0/3025279114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd4940098f0 tx=0x7fd494009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573b3b2f8669ca4e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a159a700 1 -- 172.21.15.7:0/3025279114 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400b8a0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a159a700 1 -- 172.21.15.7:0/3025279114 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49400ba00 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a159a700 1 -- 172.21.15.7:0/3025279114 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49400bd90 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a89ff700 1 -- 172.21.15.7:0/3025279114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a40cb5d0 msgr2=0x7fd4a40b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:24.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.802+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/3025279114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fd4940098f0 tx=0x7fd494009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.803+0000 7fd4a89ff700 1 -- 172.21.15.7:0/3025279114 shutdown_connections 2023-06-09T11:42:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.803+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/3025279114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a40cb5d0 0x7fd4a40b52b0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.803+0000 7fd4a89ff700 1 -- 172.21.15.7:0/3025279114 >> 172.21.15.7:0/3025279114 conn(0x7fd4a4070af0 msgr2=0x7fd4a4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.803+0000 7fd4a89ff700 1 -- 172.21.15.7:0/3025279114 shutdown_connections 2023-06-09T11:42:24.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.803+0000 7fd4a89ff700 1 -- 172.21.15.7:0/3025279114 wait complete. 2023-06-09T11:42:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.804+0000 7fd4a89ff700 1 Processor -- start 2023-06-09T11:42:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.804+0000 7fd4a89ff700 1 -- start start 2023-06-09T11:42:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a40cb9a0 0x7fd4a40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 0x7fd4a40ce890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a41f3100 con 0x7fd4a415cd40 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40cf440 con 0x7fd4a40cb9a0 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.805+0000 7fd4a89ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40c8e60 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 0x7fd4a40ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:55518/0 (socket says 172.21.15.7:55518) 2023-06-09T11:42:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 -- 172.21.15.7:0/804527700 learned_addr learned my addr 172.21.15.7:0/804527700 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:24.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a1d9b700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a40cb9a0 0x7fd4a40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 -- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a40cb9a0 msgr2=0x7fd4a40ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a40cb9a0 0x7fd4a40ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 -- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 msgr2=0x7fd4a40ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 0x7fd4a40ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:24.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 -- 172.21.15.7:0/804527700 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd494009550 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a2d9d700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 0x7fd4a40ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.806+0000 7fd4a259c700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd494009c40 tx=0x7fd494004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.807+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494004380 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.807+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49401f070 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.807+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4940036a0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.807+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a415d4f0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.807+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a415d210 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.808+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a415d7d0 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.809+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd494021030 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.810+0000 7fd492ffd700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 0x7fd48406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:24.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.810+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fd49408b220 con 0x7fd4a40cb5d0 2023-06-09T11:42:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.810+0000 7fd4a1d9b700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 0x7fd48406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:24.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.811+0000 7fd4a1d9b700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 0x7fd48406d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd4a415c250 tx=0x7fd498009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:24.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:24.818+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd494055d70 con 0x7fd4a40cb5d0 2023-06-09T11:42:25.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.074+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a4074cf0 con 0x7fd4a40cb5d0 2023-06-09T11:42:25.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.075+0000 7fd492ffd700 1 -- 172.21.15.7:0/804527700 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 141 v141) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fd494015090 con 0x7fd4a40cb5d0 2023-06-09T11:42:25.080 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:25.080 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":141,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":141,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:19.541301+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:25.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 msgr2=0x7fd48406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 0x7fd48406d190 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd4a415c250 tx=0x7fd498009040 comp rx=0 tx=0).stop 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 msgr2=0x7fd4a41f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd494009c40 tx=0x7fd494004750 comp rx=0 tx=0).stop 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 shutdown_connections 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4a415cd40 0x7fd4a40ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd48406ace0 0x7fd48406d190 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4a40cb5d0 0x7fd4a41f27b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 --2- 172.21.15.7:0/804527700 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4a40cb9a0 0x7fd4a40ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 >> 172.21.15.7:0/804527700 conn(0x7fd4a4070af0 msgr2=0x7fd4a40c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 shutdown_connections 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:25.078+0000 7fd4a89ff700 1 -- 172.21.15.7:0/804527700 wait complete. 2023-06-09T11:42:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 141 2023-06-09T11:42:25.100 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:27.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:27.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.307+0000 7f48643ff700 1 Processor -- start 2023-06-09T11:42:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.307+0000 7f48643ff700 1 -- start start 2023-06-09T11:42:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.308+0000 7f48643ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.308+0000 7f48643ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.308+0000 7f48643ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c159b20 con 0x7f485c0cb5d0 2023-06-09T11:42:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.308+0000 7f48643ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0cac80 con 0x7f485c15cd40 2023-06-09T11:42:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.308+0000 7f48643ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1597d0 con 0x7f485c0cb9a0 2023-06-09T11:42:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56114/0 (socket says 172.21.15.7:56114) 2023-06-09T11:42:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 -- 172.21.15.7:0/313455331 learned_addr learned my addr 172.21.15.7:0/313455331 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:27.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486199a700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 -- 172.21.15.7:0/313455331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 msgr2=0x7f485c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 -- 172.21.15.7:0/313455331 >> v1:172.21.15.117:6789/0 conn(0x7f485c0cb5d0 legacy=0x7f485c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f486299c700 1 -- 172.21.15.7:0/313455331 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c071ab0 con 0x7f485c15cd40 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.309+0000 7f4861199700 1 -- 172.21.15.7:0/313455331 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270900074 0 0) 0x7f485c159b20 con 0x7f485c0cb5d0 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f486299c700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0cbea0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4854009e50 tx=0x7f485400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dafb4c99e4e10e8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:27.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f4861199700 1 -- 172.21.15.7:0/313455331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400e070 con 0x7f485c15cd40 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f4861199700 1 -- 172.21.15.7:0/313455331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485400b910 con 0x7f485c15cd40 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f4861199700 1 -- 172.21.15.7:0/313455331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400bcc0 con 0x7f485c15cd40 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 -- 172.21.15.7:0/313455331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 msgr2=0x7f485c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0cbea0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4854009e50 tx=0x7f485400b330 comp rx=0 tx=0).stop 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 -- 172.21.15.7:0/313455331 shutdown_connections 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 --2- 172.21.15.7:0/313455331 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.310+0000 7f48643ff700 1 -- 172.21.15.7:0/313455331 >> 172.21.15.7:0/313455331 conn(0x7f485c070af0 msgr2=0x7f485c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:27.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.311+0000 7f48643ff700 1 -- 172.21.15.7:0/313455331 shutdown_connections 2023-06-09T11:42:27.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.311+0000 7f48643ff700 1 -- 172.21.15.7:0/313455331 wait complete. 2023-06-09T11:42:27.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.313+0000 7f48643ff700 1 Processor -- start 2023-06-09T11:42:27.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.313+0000 7f48643ff700 1 -- start start 2023-06-09T11:42:27.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb5d0 0x7f485c0cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1ef130 con 0x7f485c15cd40 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1ef320 con 0x7f485c0cb5d0 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.314+0000 7f48643ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1ef5e0 con 0x7f485c0cb9a0 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56118/0 (socket says 172.21.15.7:56118) 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 -- 172.21.15.7:0/464328592 learned_addr learned my addr 172.21.15.7:0/464328592 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486219b700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb5d0 0x7f485c0cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486199a700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 -- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 msgr2=0x7f485c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 -- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb5d0 msgr2=0x7f485c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb5d0 0x7f485c0cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 -- 172.21.15.7:0/464328592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009b30 con 0x7f485c15cd40 2023-06-09T11:42:27.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.315+0000 7f486299c700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f485c067180 tx=0x7f4854004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.316+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854004880 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.316+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48540092e0 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.316+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c15d4f0 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.316+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485401c660 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.316+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c15d210 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.317+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c15d7d0 con 0x7f485c15cd40 2023-06-09T11:42:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.318+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f485401c7c0 con 0x7f485c15cd40 2023-06-09T11:42:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.319+0000 7f4852ffd700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 0x7f484006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.319+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f485408b2a0 con 0x7f485c15cd40 2023-06-09T11:42:27.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.319+0000 7f486219b700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 0x7f484006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.320+0000 7f486219b700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 0x7f484006d1c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4858007fd0 tx=0x7f4858009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.323+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4854055cc0 con 0x7f485c15cd40 2023-06-09T11:42:27.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.573+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f485c074cf0 con 0x7f485c15cd40 2023-06-09T11:42:27.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.573+0000 7f4852ffd700 1 -- 172.21.15.7:0/464328592 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f48540592e0 con 0x7f485c15cd40 2023-06-09T11:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":142,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:26.548976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:resolve","state_seq":22,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:27.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 msgr2=0x7f484006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 0x7f484006d1c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4858007fd0 tx=0x7f4858009040 comp rx=0 tx=0).stop 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 msgr2=0x7f485c0ce970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f485c067180 tx=0x7f4854004ad0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 shutdown_connections 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0ce970 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb5d0 0x7f485c0cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ad10 0x7f484006d1c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 --2- 172.21.15.7:0/464328592 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb9a0 0x7f485c0cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 >> 172.21.15.7:0/464328592 conn(0x7f485c070af0 msgr2=0x7f485c158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 shutdown_connections 2023-06-09T11:42:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:27.576+0000 7f48643ff700 1 -- 172.21.15.7:0/464328592 wait complete. 2023-06-09T11:42:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 142 2023-06-09T11:42:27.598 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:29.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.804+0000 7fdea37ff700 1 Processor -- start 2023-06-09T11:42:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.804+0000 7fdea37ff700 1 -- start start 2023-06-09T11:42:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea37ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde9c0c9f40 0x7fde9c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea37ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c0c5a80 con 0x7fde9c15b4a0 2023-06-09T11:42:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea37ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c15a500 con 0x7fde9c153880 2023-06-09T11:42:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea37ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0c5f10 con 0x7fde9c0c9f40 2023-06-09T11:42:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea0d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde9c15b4a0 0x7fde9c0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58592/0 (socket says 172.21.15.7:58592) 2023-06-09T11:42:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.805+0000 7fdea0d9a700 1 -- 172.21.15.7:0/860781692 learned_addr learned my addr 172.21.15.7:0/860781692 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:29.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> v2:172.21.15.117:3300/0 conn(0x7fde9c0c9f40 0x7fde9c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.812 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625978311 0 0) 0x7fde9c0c5a80 con 0x7fde9c15b4a0 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde880035b0 con 0x7fde9c15b4a0 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744449302 0 0) 0x7fde98001110 con 0x7fde9c153880 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9c0c5a80 con 0x7fde9c153880 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933919041 0 0) 0x7fde880035b0 con 0x7fde9c15b4a0 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 >> v2:172.21.15.117:3300/0 conn(0x7fde9c0c9f40 msgr2=0x7fde9c0c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 --2- 172.21.15.7:0/860781692 >> v2:172.21.15.117:3300/0 conn(0x7fde9c0c9f40 0x7fde9c0c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 >> v1:172.21.15.117:6789/0 conn(0x7fde9c153880 legacy=0x7fde9c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c1596e0 con 0x7fde9c15b4a0 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fde8c001670 con 0x7fde9c15b4a0 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> v2:172.21.15.117:3300/0 conn(0x7fde9c0c9f40 0x7fde9c0c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 >> v1:172.21.15.7:6789/0 conn(0x7fde9c15b4a0 legacy=0x7fde9c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde88003be0 0x7fde88003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88004850 0x7fde88004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 0x7fde8800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c5f10 con 0x7fde88005250 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8800c160 con 0x7fde88003be0 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.806+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8800c320 con 0x7fde88004850 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 0x7fde8800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea159b700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde88003be0 0x7fde88003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea0d9a700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88004850 0x7fde88004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 -- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88004850 msgr2=0x7fde88004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:29.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88004850 0x7fde88004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 -- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde88003be0 msgr2=0x7fde88003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde88003be0 0x7fde88003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 -- 172.21.15.7:0/860781692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880035b0 con 0x7fde88005250 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 -- 172.21.15.7:0/860781692 reap_dead start 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 0x7fde8800ba10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e320 tx=0x7fde88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afb006c3dc708c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9400edc0 con 0x7fde88005250 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9400f040 con 0x7fde88005250 2023-06-09T11:42:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.807+0000 7fde93fff700 1 -- 172.21.15.7:0/860781692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940146c0 con 0x7fde88005250 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 -- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 msgr2=0x7fde8800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 0x7fde8800ba10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fde9400e320 tx=0x7fde88005880 comp rx=0 tx=0).stop 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 -- 172.21.15.7:0/860781692 shutdown_connections 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 --2- 172.21.15.7:0/860781692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde88005250 0x7fde8800ba10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 -- 172.21.15.7:0/860781692 >> 172.21.15.7:0/860781692 conn(0x7fde9c0737f0 msgr2=0x7fde9c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 -- 172.21.15.7:0/860781692 shutdown_connections 2023-06-09T11:42:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.808+0000 7fdea37ff700 1 -- 172.21.15.7:0/860781692 wait complete. 2023-06-09T11:42:29.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.810+0000 7fdea37ff700 1 Processor -- start 2023-06-09T11:42:29.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.810+0000 7fdea37ff700 1 -- start start 2023-06-09T11:42:29.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88005250 0x7fde9c14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c14dd90 0x7fde9c0b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c156270 con 0x7fde9c0b35c0 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1f24e0 con 0x7fde9c14dd90 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea37ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1f27c0 con 0x7fde88005250 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea1d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea1d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56124/0 (socket says 172.21.15.7:56124) 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea1d9c700 1 -- 172.21.15.7:0/3031360670 learned_addr learned my addr 172.21.15.7:0/3031360670 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.811+0000 7fdea0d9a700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c14dd90 0x7fde9c0b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea159b700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88005250 0x7fde9c14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 -- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88005250 msgr2=0x7fde9c14d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88005250 0x7fde9c14d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 -- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c14dd90 msgr2=0x7fde9c0b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c14dd90 0x7fde9c0b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:29.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 -- 172.21.15.7:0/3031360670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c159c60 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea1d9c700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0b57d0 tx=0x7fde94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94004450 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde940045b0 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94014480 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c159980 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.812+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c159f40 con 0x7fde9c0b35c0 2023-06-09T11:42:29.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.813+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c153f10 con 0x7fde9c0b35c0 2023-06-09T11:42:29.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.814+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde94023070 con 0x7fde9c0b35c0 2023-06-09T11:42:29.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.815+0000 7fde91ffb700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:29.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.815+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fde9408b440 con 0x7fde9c0b35c0 2023-06-09T11:42:29.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.815+0000 7fdea159b700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.816+0000 7fdea159b700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 0x7fde8006d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:29.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:29.820+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde94056010 con 0x7fde9c0b35c0 2023-06-09T11:42:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.071+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde9c074cf0 con 0x7fde9c0b35c0 2023-06-09T11:42:30.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.072+0000 7fde91ffb700 1 -- 172.21.15.7:0/3031360670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 142 v142) v1 ==== 95+0+5467 (secure 0 0 0) 0x7fde94059630 con 0x7fde9c0b35c0 2023-06-09T11:42:30.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:30.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":142,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":142,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:26.548976+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:resolve","state_seq":22,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 msgr2=0x7fde8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 0x7fde8006d190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fde98000c00 tx=0x7fde98009040 comp rx=0 tx=0).stop 2023-06-09T11:42:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 msgr2=0x7fde9c0cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0b57d0 tx=0x7fde94009750 comp rx=0 tx=0).stop 2023-06-09T11:42:30.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 shutdown_connections 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde9c0b35c0 0x7fde9c0cbc00 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8006ace0 0x7fde8006d190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde88005250 0x7fde9c14d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 --2- 172.21.15.7:0/3031360670 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9c14dd90 0x7fde9c0b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 >> 172.21.15.7:0/3031360670 conn(0x7fde9c0737f0 msgr2=0x7fde9c0c9670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 shutdown_connections 2023-06-09T11:42:30.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:30.074+0000 7fdea37ff700 1 -- 172.21.15.7:0/3031360670 wait complete. 2023-06-09T11:42:30.081 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 142 2023-06-09T11:42:30.097 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:32.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.305+0000 7efc3c9ff700 1 Processor -- start 2023-06-09T11:42:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.306+0000 7efc3c9ff700 1 -- start start 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c9090 0x7efc380c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3c9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc380cad00 0x7efc380cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3c9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3815cec0 con 0x7efc380cfaf0 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3c9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3815d160 con 0x7efc380cad00 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3815d440 con 0x7efc380c9090 2023-06-09T11:42:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc35d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc380cad00 0x7efc380cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc35d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efc380cad00 0x7efc380cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56128/0 (socket says 172.21.15.7:56128) 2023-06-09T11:42:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc35d9b700 1 -- 172.21.15.7:0/1323785779 learned_addr learned my addr 172.21.15.7:0/1323785779 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.307+0000 7efc3659c700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c9090 0x7efc380c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc35d9b700 1 -- 172.21.15.7:0/1323785779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c9090 msgr2=0x7efc380c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc35d9b700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c9090 0x7efc380c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc35d9b700 1 -- 172.21.15.7:0/1323785779 >> v1:172.21.15.117:6789/0 conn(0x7efc380cfaf0 legacy=0x7efc381598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc35d9b700 1 -- 172.21.15.7:0/1323785779 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3815d720 con 0x7efc380cad00 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc3559a700 1 -- 172.21.15.7:0/1323785779 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746000470 0 0) 0x7efc3815cec0 con 0x7efc380cfaf0 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc35d9b700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380cf5b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0098f0 tx=0x7efc2c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8651481a6b66bc7b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc3559a700 1 -- 172.21.15.7:0/1323785779 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c011830 con 0x7efc380cad00 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc3559a700 1 -- 172.21.15.7:0/1323785779 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c011990 con 0x7efc380cad00 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.308+0000 7efc3559a700 1 -- 172.21.15.7:0/1323785779 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c011d20 con 0x7efc380cad00 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 -- 172.21.15.7:0/1323785779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 msgr2=0x7efc380cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380cf5b0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0098f0 tx=0x7efc2c009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 -- 172.21.15.7:0/1323785779 shutdown_connections 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380c9090 0x7efc380c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/1323785779 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380cf5b0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 -- 172.21.15.7:0/1323785779 >> 172.21.15.7:0/1323785779 conn(0x7efc38070af0 msgr2=0x7efc38070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 -- 172.21.15.7:0/1323785779 shutdown_connections 2023-06-09T11:42:32.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.309+0000 7efc3c9ff700 1 -- 172.21.15.7:0/1323785779 wait complete. 2023-06-09T11:42:32.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.311+0000 7efc3c9ff700 1 Processor -- start 2023-06-09T11:42:32.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.311+0000 7efc3c9ff700 1 -- start start 2023-06-09T11:42:32.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c9090 0x7efc381e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380cfaf0 0x7efc38151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381e3be0 con 0x7efc380cad00 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38151850 con 0x7efc380c9090 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380c6280 con 0x7efc380cfaf0 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56140/0 (socket says 172.21.15.7:56140) 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 -- 172.21.15.7:0/2701642128 learned_addr learned my addr 172.21.15.7:0/2701642128 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:32.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc3659c700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c9090 0x7efc381e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc36d9d700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380cfaf0 0x7efc38151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 -- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380cfaf0 msgr2=0x7efc38151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380cfaf0 0x7efc38151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 -- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c9090 msgr2=0x7efc381e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.312+0000 7efc35d9b700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c9090 0x7efc381e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc35d9b700 1 -- 172.21.15.7:0/2701642128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c009550 con 0x7efc380cad00 2023-06-09T11:42:32.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc35d9b700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009c40 tx=0x7efc2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:32.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c004510 con 0x7efc380cad00 2023-06-09T11:42:32.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c01e070 con 0x7efc380cad00 2023-06-09T11:42:32.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0038c0 con 0x7efc380cad00 2023-06-09T11:42:32.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc380cb130 con 0x7efc380cad00 2023-06-09T11:42:32.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.313+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc380ca620 con 0x7efc380cad00 2023-06-09T11:42:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.314+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc380cb2c0 con 0x7efc380cad00 2023-06-09T11:42:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.315+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efc2c003a20 con 0x7efc380cad00 2023-06-09T11:42:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.316+0000 7efc26ffd700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 0x7efc1806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.316+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7efc2c08a950 con 0x7efc380cad00 2023-06-09T11:42:32.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.316+0000 7efc3659c700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 0x7efc1806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:32.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.317+0000 7efc3659c700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 0x7efc1806d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efc28009b40 tx=0x7efc28009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:32.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.321+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efc2c055380 con 0x7efc380cad00 2023-06-09T11:42:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.572+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc38074cf0 con 0x7efc380cad00 2023-06-09T11:42:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.574+0000 7efc26ffd700 1 -- 172.21.15.7:0/2701642128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5469 (secure 0 0 0) 0x7efc2c0589a0 con 0x7efc380cad00 2023-06-09T11:42:32.579 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:32.579 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:32.571228+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 msgr2=0x7efc1806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 0x7efc1806d190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7efc28009b40 tx=0x7efc28009210 comp rx=0 tx=0).stop 2023-06-09T11:42:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 msgr2=0x7efc380c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009c40 tx=0x7efc2c004750 comp rx=0 tx=0).stop 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 shutdown_connections 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efc1806ace0 0x7efc1806d190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efc380cfaf0 0x7efc38151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efc380c9090 0x7efc381e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 --2- 172.21.15.7:0/2701642128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc380cad00 0x7efc380c57c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.577+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 >> 172.21.15.7:0/2701642128 conn(0x7efc38070af0 msgr2=0x7efc380c80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.578+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 shutdown_connections 2023-06-09T11:42:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:32.578+0000 7efc3c9ff700 1 -- 172.21.15.7:0/2701642128 wait complete. 2023-06-09T11:42:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 143 2023-06-09T11:42:32.605 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 143, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 143, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:reconnect", "state_seq": 24 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:42:32.571228+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:42:34.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:34.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.813+0000 7f899f7ff700 1 Processor -- start 2023-06-09T11:42:34.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.813+0000 7f899f7ff700 1 -- start start 2023-06-09T11:42:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899f7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f89980b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899f7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8998159b20 con 0x7f89980cb9a0 2023-06-09T11:42:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899f7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89980cac80 con 0x7f899815cd40 2023-06-09T11:42:34.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899f7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89981597d0 con 0x7f89980cb5d0 2023-06-09T11:42:34.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899cd9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f89980cb9a0 0x7f8998159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34376/0 (socket says 172.21.15.7:34376) 2023-06-09T11:42:34.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.814+0000 7f899cd9a700 1 -- 172.21.15.7:0/3130597891 learned_addr learned my addr 172.21.15.7:0/3130597891 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f899d59b700 1 --2- 172.21.15.7:0/3130597891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f89980b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705815434 0 0) 0x7f8998159b20 con 0x7f89980cb9a0 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89840035b0 con 0x7f89980cb9a0 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330473329 0 0) 0x7f89980cac80 con 0x7f899815cd40 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8998159b20 con 0x7f899815cd40 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f899d59b700 1 -- 172.21.15.7:0/3130597891 >> v1:172.21.15.117:6789/0 conn(0x7f899815cd40 legacy=0x7f89980cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f899d59b700 1 -- 172.21.15.7:0/3130597891 >> v1:172.21.15.7:6789/0 conn(0x7f89980cb9a0 legacy=0x7f8998159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f899d59b700 1 -- 172.21.15.7:0/3130597891 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998071ab0 con 0x7f89980cb5d0 2023-06-09T11:42:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.815+0000 7f899d59b700 1 --2- 172.21.15.7:0/3130597891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f89980b52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8994009870 tx=0x7f8994009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b46dc021510d7473 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994011830 con 0x7f89980cb5d0 2023-06-09T11:42:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994011990 con 0x7f89980cb5d0 2023-06-09T11:42:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f898ffff700 1 -- 172.21.15.7:0/3130597891 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994011d20 con 0x7f89980cb5d0 2023-06-09T11:42:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 -- 172.21.15.7:0/3130597891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 msgr2=0x7f89980b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 --2- 172.21.15.7:0/3130597891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f89980b52b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8994009870 tx=0x7f8994009b80 comp rx=0 tx=0).stop 2023-06-09T11:42:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 -- 172.21.15.7:0/3130597891 shutdown_connections 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 --2- 172.21.15.7:0/3130597891 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f89980b52b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 -- 172.21.15.7:0/3130597891 >> 172.21.15.7:0/3130597891 conn(0x7f8998070af0 msgr2=0x7f8998070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 -- 172.21.15.7:0/3130597891 shutdown_connections 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.816+0000 7f899f7ff700 1 -- 172.21.15.7:0/3130597891 wait complete. 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.818+0000 7f899f7ff700 1 Processor -- start 2023-06-09T11:42:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.818+0000 7f899f7ff700 1 -- start start 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f8998156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89980cb9a0 0x7f8998154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89981ef290 con 0x7f899815cd40 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89981eeff0 con 0x7f89980cb9a0 2023-06-09T11:42:34.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899f7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980c6a70 con 0x7f89980cb5d0 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.819+0000 7f899dd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45008/0 (socket says 172.21.15.7:45008) 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 -- 172.21.15.7:0/53435088 learned_addr learned my addr 172.21.15.7:0/53435088 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899d59b700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f8998156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899cd9a700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89980cb9a0 0x7f8998154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 -- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 msgr2=0x7f8998156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:34.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f8998156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 -- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89980cb9a0 msgr2=0x7f8998154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89980cb9a0 0x7f8998154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 -- 172.21.15.7:0/53435088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940094a0 con 0x7f899815cd40 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899dd9c700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f899000caf0 tx=0x7f899000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990003c00 con 0x7f899815cd40 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990003d60 con 0x7f899815cd40 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000aa20 con 0x7f899815cd40 2023-06-09T11:42:34.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.820+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899815d4f0 con 0x7f899815cd40 2023-06-09T11:42:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.821+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899815d210 con 0x7f899815cd40 2023-06-09T11:42:34.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.822+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899815d7d0 con 0x7f899815cd40 2023-06-09T11:42:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.823+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f899000ab80 con 0x7f899815cd40 2023-06-09T11:42:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.824+0000 7f898dffb700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 0x7f897c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.824+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f8990089a20 con 0x7f899815cd40 2023-06-09T11:42:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.824+0000 7f899d59b700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 0x7f897c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.825+0000 7f899d59b700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 0x7f897c06d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f89940097c0 tx=0x7f899401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:34.828+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8990054440 con 0x7f899815cd40 2023-06-09T11:42:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.080+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89980ccf90 con 0x7f899815cd40 2023-06-09T11:42:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.080+0000 7f898dffb700 1 -- 172.21.15.7:0/53435088 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5469 (secure 0 0 0) 0x7f8990057a60 con 0x7f899815cd40 2023-06-09T11:42:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:35.085 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:32.571228+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 msgr2=0x7f897c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 0x7f897c06d190 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f89940097c0 tx=0x7f899401c040 comp rx=0 tx=0).stop 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 msgr2=0x7f89980c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f899000caf0 tx=0x7f899000ce00 comp rx=0 tx=0).stop 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 shutdown_connections 2023-06-09T11:42:35.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f899815cd40 0x7f89980c77f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f897c06ace0 0x7f897c06d190 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89980cb5d0 0x7f8998156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 --2- 172.21.15.7:0/53435088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89980cb9a0 0x7f8998154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 >> 172.21.15.7:0/53435088 conn(0x7f8998070af0 msgr2=0x7f8998158e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 shutdown_connections 2023-06-09T11:42:35.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:35.083+0000 7f899f7ff700 1 -- 172.21.15.7:0/53435088 wait complete. 2023-06-09T11:42:35.090 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 143 2023-06-09T11:42:35.106 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:37.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.314+0000 7fa7795ff700 1 Processor -- start 2023-06-09T11:42:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.314+0000 7fa7795ff700 1 -- start start 2023-06-09T11:42:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.315+0000 7fa7795ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa774151510 0x7fa7740c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.315+0000 7fa7795ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77415b380 0x7fa774070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.315+0000 7fa7795ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7741580a0 con 0x7fa774071060 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.315+0000 7fa7795ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740714e0 con 0x7fa774151510 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.315+0000 7fa7795ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7741518e0 con 0x7fa77415b380 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa774151510 0x7fa7740c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa774151510 0x7fa7740c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45016/0 (socket says 172.21.15.7:45016) 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 -- 172.21.15.7:0/3554273397 learned_addr learned my addr 172.21.15.7:0/3554273397 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa7727fc700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77415b380 0x7fa774070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 -- 172.21.15.7:0/3554273397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77415b380 msgr2=0x7fa774070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77415b380 0x7fa774070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 -- 172.21.15.7:0/3554273397 >> v1:172.21.15.117:6789/0 conn(0x7fa774071060 legacy=0x7fa774158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 -- 172.21.15.7:0/3554273397 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740c6180 con 0x7fa774151510 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.316+0000 7fa772ffd700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774151510 0x7fa7740c5b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa7640098f0 tx=0x7fa764009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29e93f6a791a358 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa771ffb700 1 -- 172.21.15.7:0/3554273397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400b8a0 con 0x7fa774151510 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa771ffb700 1 -- 172.21.15.7:0/3554273397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76400ba00 con 0x7fa774151510 2023-06-09T11:42:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa771ffb700 1 -- 172.21.15.7:0/3554273397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400bd90 con 0x7fa774151510 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 -- 172.21.15.7:0/3554273397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774151510 msgr2=0x7fa7740c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774151510 0x7fa7740c5b40 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa7640098f0 tx=0x7fa764009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 -- 172.21.15.7:0/3554273397 shutdown_connections 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774151510 0x7fa7740c5b40 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 --2- 172.21.15.7:0/3554273397 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa77415b380 0x7fa774070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 -- 172.21.15.7:0/3554273397 >> 172.21.15.7:0/3554273397 conn(0x7fa77406a4d0 msgr2=0x7fa77406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 -- 172.21.15.7:0/3554273397 shutdown_connections 2023-06-09T11:42:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.317+0000 7fa7795ff700 1 -- 172.21.15.7:0/3554273397 wait complete. 2023-06-09T11:42:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.319+0000 7fa7795ff700 1 Processor -- start 2023-06-09T11:42:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.319+0000 7fa7795ff700 1 -- start start 2023-06-09T11:42:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa774151510 0x7fa7740c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa77415b380 0x7fa7741f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774145c30 con 0x7fa774071060 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c8fa0 con 0x7fa77415b380 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa7795ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c05e0 con 0x7fa774151510 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa772ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa772ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45028/0 (socket says 172.21.15.7:45028) 2023-06-09T11:42:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.320+0000 7fa772ffd700 1 -- 172.21.15.7:0/520026865 learned_addr learned my addr 172.21.15.7:0/520026865 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 -- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa774151510 msgr2=0x7fa7740c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa774151510 0x7fa7740c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 -- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa77415b380 msgr2=0x7fa7741f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa77415b380 0x7fa7741f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 -- 172.21.15.7:0/520026865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa764009550 con 0x7fa774071060 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa772ffd700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa764000c00 tx=0x7fa764004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004480 con 0x7fa774071060 2023-06-09T11:42:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76401f070 con 0x7fa774071060 2023-06-09T11:42:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774157770 con 0x7fa774071060 2023-06-09T11:42:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640038c0 con 0x7fa774071060 2023-06-09T11:42:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.321+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa774157490 con 0x7fa774071060 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.323+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774157a50 con 0x7fa774071060 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.323+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa764021030 con 0x7fa774071060 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.324+0000 7fa75b7fe700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 0x7fa76006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.324+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fa76408b050 con 0x7fa774071060 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.324+0000 7fa7727fc700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 0x7fa76006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.325+0000 7fa7727fc700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 0x7fa76006d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b0f20 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:37.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.328+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa7640bd2d0 con 0x7fa774071060 2023-06-09T11:42:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.581+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa774074cf0 con 0x7fa774071060 2023-06-09T11:42:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.582+0000 7fa75b7fe700 1 -- 172.21.15.7:0/520026865 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+5469 (secure 0 0 0) 0x7fa764055c20 con 0x7fa774071060 2023-06-09T11:42:37.586 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:37.586 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":143,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:32.571228+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:reconnect","state_seq":24,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:37.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 msgr2=0x7fa76006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 0x7fa76006d190 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b0f20 tx=0x7fa768008040 comp rx=0 tx=0).stop 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 msgr2=0x7fa7740c0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa764000c00 tx=0x7fa764004750 comp rx=0 tx=0).stop 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 shutdown_connections 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa77415b380 0x7fa7741f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa774071060 0x7fa7740c0a70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa76006ace0 0x7fa76006d190 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 --2- 172.21.15.7:0/520026865 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa774151510 0x7fa7740c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 >> 172.21.15.7:0/520026865 conn(0x7fa77406a4d0 msgr2=0x7fa77415a940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 shutdown_connections 2023-06-09T11:42:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:37.584+0000 7fa7795ff700 1 -- 172.21.15.7:0/520026865 wait complete. 2023-06-09T11:42:37.590 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 143 2023-06-09T11:42:37.606 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:39.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:39.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.813+0000 7f1f981ff700 1 Processor -- start 2023-06-09T11:42:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.814+0000 7f1f981ff700 1 -- start start 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f981ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f900c5b40 0x7f1f90070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f981ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f90071060 0x7f1f90158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f981ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f901580a0 con 0x7f1f90151510 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f981ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900714e0 con 0x7f1f900c5b40 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f981ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901518e0 con 0x7f1f90071060 2023-06-09T11:42:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f9579a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f900c5b40 0x7f1f90070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f9579a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f900c5b40 0x7f1f90070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45030/0 (socket says 172.21.15.7:45030) 2023-06-09T11:42:39.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.815+0000 7f1f9579a700 1 -- 172.21.15.7:0/2677703908 learned_addr learned my addr 172.21.15.7:0/2677703908 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9679c700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f90071060 0x7f1f90158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9579a700 1 -- 172.21.15.7:0/2677703908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f90071060 msgr2=0x7f1f90158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9579a700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f90071060 0x7f1f90158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:39.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9579a700 1 -- 172.21.15.7:0/2677703908 >> v1:172.21.15.7:6789/0 conn(0x7f1f90151510 legacy=0x7f1f9015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9579a700 1 -- 172.21.15.7:0/2677703908 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900c6180 con 0x7f1f900c5b40 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f94f99700 1 -- 172.21.15.7:0/2677703908 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085790910 0 0) 0x7f1f901580a0 con 0x7f1f90151510 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f9579a700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f900c5b40 0x7f1f90070b20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098f0 tx=0x7f1f80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca90bd071e94994b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f94f99700 1 -- 172.21.15.7:0/2677703908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011830 con 0x7f1f900c5b40 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f94f99700 1 -- 172.21.15.7:0/2677703908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80011990 con 0x7f1f900c5b40 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.816+0000 7f1f94f99700 1 -- 172.21.15.7:0/2677703908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80011d20 con 0x7f1f900c5b40 2023-06-09T11:42:39.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 -- 172.21.15.7:0/2677703908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f900c5b40 msgr2=0x7f1f90070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f900c5b40 0x7f1f90070b20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1f800098f0 tx=0x7f1f80009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 -- 172.21.15.7:0/2677703908 shutdown_connections 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f90071060 0x7f1f90158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 --2- 172.21.15.7:0/2677703908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f900c5b40 0x7f1f90070b20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 -- 172.21.15.7:0/2677703908 >> 172.21.15.7:0/2677703908 conn(0x7f1f9006a4d0 msgr2=0x7f1f9006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 -- 172.21.15.7:0/2677703908 shutdown_connections 2023-06-09T11:42:39.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.817+0000 7f1f981ff700 1 -- 172.21.15.7:0/2677703908 wait complete. 2023-06-09T11:42:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.819+0000 7f1f981ff700 1 Processor -- start 2023-06-09T11:42:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.819+0000 7f1f981ff700 1 -- start start 2023-06-09T11:42:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f900c5b40 0x7f1f900caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 0x7f1f901ed730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901edc70 con 0x7f1f90071060 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90159910 con 0x7f1f90151510 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f981ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90159bf0 con 0x7f1f900c5b40 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f95f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f95f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45038/0 (socket says 172.21.15.7:45038) 2023-06-09T11:42:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.820+0000 7f1f95f9b700 1 -- 172.21.15.7:0/1744808391 learned_addr learned my addr 172.21.15.7:0/1744808391 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f9579a700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f900c5b40 0x7f1f900caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f9679c700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 0x7f1f901ed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 -- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f900c5b40 msgr2=0x7f1f900caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f900c5b40 0x7f1f900caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 -- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 msgr2=0x7f1f901ed730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 0x7f1f901ed730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 -- 172.21.15.7:0/1744808391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80009550 con 0x7f1f90071060 2023-06-09T11:42:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f9679c700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 0x7f1f901ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f95f9b700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c550 tx=0x7f1f8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c0054e0 con 0x7f1f90071060 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c003680 con 0x7f1f90071060 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f90157770 con 0x7f1f90071060 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c003a30 con 0x7f1f90071060 2023-06-09T11:42:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.821+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f90157490 con 0x7f1f90071060 2023-06-09T11:42:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.822+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90157a50 con 0x7f1f90071060 2023-06-09T11:42:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.823+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f8c003b90 con 0x7f1f90071060 2023-06-09T11:42:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.824+0000 7f1f867fc700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 0x7f1f7406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.824+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f1f8c088ee0 con 0x7f1f90071060 2023-06-09T11:42:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.824+0000 7f1f9579a700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 0x7f1f7406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.825+0000 7f1f9579a700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 0x7f1f7406d1c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009c40 tx=0x7f1f80008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:39.829+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f8c053990 con 0x7f1f90071060 2023-06-09T11:42:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.081+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f90074cf0 con 0x7f1f90071060 2023-06-09T11:42:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.082+0000 7f1f867fc700 1 -- 172.21.15.7:0/1744808391 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f1f8c017020 con 0x7f1f90071060 2023-06-09T11:42:40.086 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:40.086 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:37.712559+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 msgr2=0x7f1f7406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 0x7f1f7406d1c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009c40 tx=0x7f1f80008030 comp rx=0 tx=0).stop 2023-06-09T11:42:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 msgr2=0x7f1f901eac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c550 tx=0x7f1f8c00c910 comp rx=0 tx=0).stop 2023-06-09T11:42:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 shutdown_connections 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f90151510 0x7f1f901ed730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f90071060 0x7f1f901eac90 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f7406ad10 0x7f1f7406d1c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 --2- 172.21.15.7:0/1744808391 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f900c5b40 0x7f1f900caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 >> 172.21.15.7:0/1744808391 conn(0x7f1f9006a4d0 msgr2=0x7f1f9014c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 shutdown_connections 2023-06-09T11:42:40.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:40.084+0000 7f1f981ff700 1 -- 172.21.15.7:0/1744808391 wait complete. 2023-06-09T11:42:40.091 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 144 2023-06-09T11:42:40.107 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:42.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.316+0000 7fb395dff700 1 Processor -- start 2023-06-09T11:42:42.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.316+0000 7fb395dff700 1 -- start start 2023-06-09T11:42:42.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb395dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb390153b30 0x7fb3900ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb395dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390071fa0 0x7fb3900723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb395dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39015a390 con 0x7fb390153760 2023-06-09T11:42:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb395dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900c5f40 con 0x7fb390153b30 2023-06-09T11:42:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb395dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900ca720 con 0x7fb390071fa0 2023-06-09T11:42:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb38f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb390153760 0x7fb3900b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34392/0 (socket says 172.21.15.7:34392) 2023-06-09T11:42:42.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb38f7fe700 1 -- 172.21.15.7:0/2988855454 learned_addr learned my addr 172.21.15.7:0/2988855454 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> v2:172.21.15.117:3300/0 conn(0x7fb390153b30 0x7fb3900ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb38ffff700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390071fa0 0x7fb3900723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.318+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034663617 0 0) 0x7fb39015a390 con 0x7fb390153760 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3780035b0 con 0x7fb390153760 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1295474412 0 0) 0x7fb3780035b0 con 0x7fb390153760 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390071fa0 msgr2=0x7fb3900723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390071fa0 0x7fb3900723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 >> v2:172.21.15.117:3300/0 conn(0x7fb390153b30 msgr2=0x7fb3900ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 --2- 172.21.15.7:0/2988855454 >> v2:172.21.15.117:3300/0 conn(0x7fb390153b30 0x7fb3900ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900c5a20 con 0x7fb390153760 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fb380001670 con 0x7fb390153760 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 >> v1:172.21.15.7:6789/0 conn(0x7fb390153760 legacy=0x7fb3900b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb378003a00 0x7fb378003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3780043e0 0x7fb3780048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb378004f20 0x7fb37800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900c5f40 con 0x7fb3780043e0 2023-06-09T11:42:42.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900ca720 con 0x7fb378004f20 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800bf50 con 0x7fb378003a00 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> v2:172.21.15.117:3300/0 conn(0x7fb390153b30 0x7fb3900ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38ffff700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390071fa0 0x7fb3900723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3780043e0 0x7fb3780048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38f7fe700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb378003a00 0x7fb378003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.319+0000 7fb38ffff700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb378004f20 0x7fb37800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 -- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb378003a00 msgr2=0x7fb378003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb378003a00 0x7fb378003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 -- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb378004f20 msgr2=0x7fb37800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb378004f20 0x7fb37800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 -- 172.21.15.7:0/2988855454 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780035b0 con 0x7fb3780043e0 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38effd700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3780043e0 0x7fb3780048b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb384009d00 tx=0x7fb378005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459f6d12b4e74cdc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384009e50 con 0x7fb3780043e0 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38400e9a0 con 0x7fb3780043e0 2023-06-09T11:42:42.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.320+0000 7fb38e7fc700 1 -- 172.21.15.7:0/2988855454 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38400ed30 con 0x7fb3780043e0 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 -- 172.21.15.7:0/2988855454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3780043e0 msgr2=0x7fb3780048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 --2- 172.21.15.7:0/2988855454 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3780043e0 0x7fb3780048b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb384009d00 tx=0x7fb378005880 comp rx=0 tx=0).stop 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb38ffff700 1 -- 172.21.15.7:0/2988855454 reap_dead start 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 -- 172.21.15.7:0/2988855454 shutdown_connections 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 -- 172.21.15.7:0/2988855454 >> 172.21.15.7:0/2988855454 conn(0x7fb3900737f0 msgr2=0x7fb390073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 -- 172.21.15.7:0/2988855454 shutdown_connections 2023-06-09T11:42:42.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.321+0000 7fb395dff700 1 -- 172.21.15.7:0/2988855454 wait complete. 2023-06-09T11:42:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.323+0000 7fb395dff700 1 Processor -- start 2023-06-09T11:42:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.323+0000 7fb395dff700 1 -- start start 2023-06-09T11:42:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3780043e0 0x7fb3901f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390156950 0x7fb39014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390155130 con 0x7fb390158cf0 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900b4b50 con 0x7fb3780043e0 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb395dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900b4d50 con 0x7fb390156950 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3780043e0 0x7fb3901f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45050/0 (socket says 172.21.15.7:45050) 2023-06-09T11:42:42.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 -- 172.21.15.7:0/1724089364 learned_addr learned my addr 172.21.15.7:0/1724089364 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38ffff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390156950 0x7fb39014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 -- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390156950 msgr2=0x7fb39014cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390156950 0x7fb39014cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 -- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3780043e0 msgr2=0x7fb3901f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3780043e0 0x7fb3901f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.324+0000 7fb38effd700 1 -- 172.21.15.7:0/1724089364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390159a60 con 0x7fb390158cf0 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb38effd700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb3780056f0 tx=0x7fb3840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:42.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384021070 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb384009e50 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb384006e60 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390159780 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.325+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb390159d40 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.326+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390074cf0 con 0x7fb390158cf0 2023-06-09T11:42:42.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.327+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb384003ad0 con 0x7fb390158cf0 2023-06-09T11:42:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.328+0000 7fb377fff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.328+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fb38408ab20 con 0x7fb390158cf0 2023-06-09T11:42:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.328+0000 7fb38f7fe700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:42.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.329+0000 7fb38f7fe700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 0x7fb37c06d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb380001eb0 tx=0x7fb380007980 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:42.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.332+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb3840556f0 con 0x7fb390158cf0 2023-06-09T11:42:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.584+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb39015ca00 con 0x7fb390158cf0 2023-06-09T11:42:42.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.584+0000 7fb377fff700 1 -- 172.21.15.7:0/1724089364 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 144 v144) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fb384058d10 con 0x7fb390158cf0 2023-06-09T11:42:42.589 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:42.589 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":144,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":144,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:37.712559+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:rejoin","state_seq":26,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 msgr2=0x7fb37c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 0x7fb37c06d190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb380001eb0 tx=0x7fb380007980 comp rx=0 tx=0).stop 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 msgr2=0x7fb390151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fb3780056f0 tx=0x7fb3840047c0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 shutdown_connections 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb37c06ace0 0x7fb37c06d190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb390156950 0x7fb39014cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3780043e0 0x7fb3901f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 --2- 172.21.15.7:0/1724089364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb390158cf0 0x7fb390151a80 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 >> 172.21.15.7:0/1724089364 conn(0x7fb3900737f0 msgr2=0x7fb3900662d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 shutdown_connections 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:42.587+0000 7fb395dff700 1 -- 172.21.15.7:0/1724089364 wait complete. 2023-06-09T11:42:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 144 2023-06-09T11:42:42.610 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:44.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.818+0000 7f0b73fff700 1 Processor -- start 2023-06-09T11:42:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.818+0000 7f0b73fff700 1 -- start start 2023-06-09T11:42:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 0x7f0b6c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b73fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b6c06e460 0x7f0b6c06cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b73fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6c15cac0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b73fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c8870 con 0x7f0b6c06e460 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b73fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c0c37d0 con 0x7f0b6c0c6280 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b7159a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b6c06a4d0 0x7f0b6c0737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39250/0 (socket says 172.21.15.7:39250) 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b7159a700 1 -- 172.21.15.7:0/2322090384 learned_addr learned my addr 172.21.15.7:0/2322090384 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/2322090384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 0x7f0b6c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.820+0000 7f0b7259c700 1 --2- 172.21.15.7:0/2322090384 >> v2:172.21.15.117:3300/0 conn(0x7f0b6c06e460 0x7f0b6c06cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b70d99700 1 -- 172.21.15.7:0/2322090384 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609861210 0 0) 0x7f0b6c15cac0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b70d99700 1 -- 172.21.15.7:0/2322090384 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b580035b0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b71d9b700 1 -- 172.21.15.7:0/2322090384 >> v2:172.21.15.117:3300/0 conn(0x7f0b6c06e460 msgr2=0x7f0b6c06cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/2322090384 >> v2:172.21.15.117:3300/0 conn(0x7f0b6c06e460 0x7f0b6c06cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b71d9b700 1 -- 172.21.15.7:0/2322090384 >> v1:172.21.15.7:6789/0 conn(0x7f0b6c06a4d0 legacy=0x7f0b6c0737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b71d9b700 1 -- 172.21.15.7:0/2322090384 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c15cd30 con 0x7f0b6c0c6280 2023-06-09T11:42:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/2322090384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 0x7f0b6c0c76b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009840 tx=0x7f0b68009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83649aae763e5443 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b70d99700 1 -- 172.21.15.7:0/2322090384 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68011830 con 0x7f0b6c0c6280 2023-06-09T11:42:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b70d99700 1 -- 172.21.15.7:0/2322090384 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68011990 con 0x7f0b6c0c6280 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.821+0000 7f0b70d99700 1 -- 172.21.15.7:0/2322090384 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68011d20 con 0x7f0b6c0c6280 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 -- 172.21.15.7:0/2322090384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 msgr2=0x7f0b6c0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 --2- 172.21.15.7:0/2322090384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 0x7f0b6c0c76b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009840 tx=0x7f0b68009b50 comp rx=0 tx=0).stop 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 -- 172.21.15.7:0/2322090384 shutdown_connections 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 --2- 172.21.15.7:0/2322090384 >> v2:172.21.15.117:3300/0 conn(0x7f0b6c06e460 0x7f0b6c06cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 --2- 172.21.15.7:0/2322090384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c0c6280 0x7f0b6c0c76b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 -- 172.21.15.7:0/2322090384 >> 172.21.15.7:0/2322090384 conn(0x7f0b6c06f980 msgr2=0x7f0b6c0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 -- 172.21.15.7:0/2322090384 shutdown_connections 2023-06-09T11:42:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.822+0000 7f0b73fff700 1 -- 172.21.15.7:0/2322090384 wait complete. 2023-06-09T11:42:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.824+0000 7f0b73fff700 1 Processor -- start 2023-06-09T11:42:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.824+0000 7f0b73fff700 1 -- start start 2023-06-09T11:42:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c06e460 0x7f0b6c1e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6c0c6280 0x7f0b6c1e6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c15b8a0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c155200 con 0x7f0b6c0c6280 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b73fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c15a1f0 con 0x7f0b6c06e460 2023-06-09T11:42:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b71d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51562/0 (socket says 172.21.15.7:51562) 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b71d9b700 1 -- 172.21.15.7:0/3224430522 learned_addr learned my addr 172.21.15.7:0/3224430522 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b7259c700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6c0c6280 0x7f0b6c1e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b7159a700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c06e460 0x7f0b6c1e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b71d9b700 1 -- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c06e460 msgr2=0x7f0b6c1e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.825+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c06e460 0x7f0b6c1e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b71d9b700 1 -- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6c0c6280 msgr2=0x7f0b6c1e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6c0c6280 0x7f0b6c1e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b71d9b700 1 -- 172.21.15.7:0/3224430522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b680094a0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b71d9b700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0b68014040 tx=0x7f0b68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b680038a0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6801f070 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c15d2f0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68003c00 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.826+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c15d010 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.827+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c15d5d0 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.828+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b6801a430 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.829+0000 7f0b627fc700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 0x7f0b5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.829+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f0b68021070 con 0x7f0b6c06a4d0 2023-06-09T11:42:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.829+0000 7f0b7159a700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 0x7f0b5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.830+0000 7f0b7159a700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 0x7f0b5006d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c72f0 tx=0x7f0b5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:44.834+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0b68055e20 con 0x7f0b6c06a4d0 2023-06-09T11:42:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.087+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b6c074cf0 con 0x7f0b6c06a4d0 2023-06-09T11:42:45.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.087+0000 7f0b627fc700 1 -- 172.21.15.7:0/3224430522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5472 (secure 0 0 0) 0x7f0b68059440 con 0x7f0b6c06a4d0 2023-06-09T11:42:45.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:45.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:43.740975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:clientreplay","state_seq":28,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:45.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 msgr2=0x7f0b5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 0x7f0b5006d190 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0c72f0 tx=0x7f0b5c009040 comp rx=0 tx=0).stop 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 msgr2=0x7f0b6c154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f0b68014040 tx=0x7f0b68004750 comp rx=0 tx=0).stop 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 shutdown_connections 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b6c0c6280 0x7f0b6c1e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:45.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b6c06a4d0 0x7f0b6c154cc0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b5006ace0 0x7f0b5006d190 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 --2- 172.21.15.7:0/3224430522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b6c06e460 0x7f0b6c1e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 >> 172.21.15.7:0/3224430522 conn(0x7f0b6c06f980 msgr2=0x7f0b6c06db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 shutdown_connections 2023-06-09T11:42:45.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:45.090+0000 7f0b73fff700 1 -- 172.21.15.7:0/3224430522 wait complete. 2023-06-09T11:42:45.097 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 145 2023-06-09T11:42:45.113 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 145, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 145, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6583": { "addr": "172.21.15.117:6837/1520141131", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 1520141131, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 1520141131, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6583, "incarnation": 125, "join_fscid": -1, "name": "f", "rank": 0, "state": "up:active", "state_seq": 33 }, "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:clientreplay", "state_seq": 28 } }, "last_failure": 0, "last_failure_osd_epoch": 66, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:42:43.740975+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6583, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:42:47.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:47.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.321+0000 7f076f1ff700 1 Processor -- start 2023-06-09T11:42:47.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.321+0000 7f076f1ff700 1 -- start start 2023-06-09T11:42:47.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f076f1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0768153880 0x7f076815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f076f1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076815a500 con 0x7f07680c9f40 2023-06-09T11:42:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f076f1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076815da00 con 0x7f076815b4a0 2023-06-09T11:42:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f076f1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07680c5f10 con 0x7f0768153880 2023-06-09T11:42:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f0767fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f07680c9f40 0x7f076815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39260/0 (socket says 172.21.15.7:39260) 2023-06-09T11:42:47.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.322+0000 7f0767fff700 1 -- 172.21.15.7:0/2987890095 learned_addr learned my addr 172.21.15.7:0/2987890095 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:47.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f076cf9b700 1 --2- 172.21.15.7:0/2987890095 >> v2:172.21.15.117:3300/0 conn(0x7f0768153880 0x7f076815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f07677fe700 1 -- 172.21.15.7:0/2987890095 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549288323 0 0) 0x7f0754001110 con 0x7f07680c9f40 2023-06-09T11:42:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f07677fe700 1 -- 172.21.15.7:0/2987890095 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07580035b0 con 0x7f07680c9f40 2023-06-09T11:42:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f076cf9b700 1 -- 172.21.15.7:0/2987890095 >> v1:172.21.15.117:6789/0 conn(0x7f076815b4a0 legacy=0x7f07680c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f076cf9b700 1 -- 172.21.15.7:0/2987890095 >> v1:172.21.15.7:6789/0 conn(0x7f07680c9f40 legacy=0x7f076815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f076cf9b700 1 -- 172.21.15.7:0/2987890095 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07681596e0 con 0x7f0768153880 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.323+0000 7f076cf9b700 1 --2- 172.21.15.7:0/2987890095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0768153880 0x7f076815d4c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f075c009840 tx=0x7f075c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22063a13058735f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f07677fe700 1 -- 172.21.15.7:0/2987890095 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c011830 con 0x7f0768153880 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f07677fe700 1 -- 172.21.15.7:0/2987890095 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c011990 con 0x7f0768153880 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f07677fe700 1 -- 172.21.15.7:0/2987890095 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c011d20 con 0x7f0768153880 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 -- 172.21.15.7:0/2987890095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0768153880 msgr2=0x7f076815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 --2- 172.21.15.7:0/2987890095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0768153880 0x7f076815d4c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f075c009840 tx=0x7f075c009b50 comp rx=0 tx=0).stop 2023-06-09T11:42:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 -- 172.21.15.7:0/2987890095 shutdown_connections 2023-06-09T11:42:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 --2- 172.21.15.7:0/2987890095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0768153880 0x7f076815d4c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 -- 172.21.15.7:0/2987890095 >> 172.21.15.7:0/2987890095 conn(0x7f07680737f0 msgr2=0x7f07680720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 -- 172.21.15.7:0/2987890095 shutdown_connections 2023-06-09T11:42:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.324+0000 7f076f1ff700 1 -- 172.21.15.7:0/2987890095 wait complete. 2023-06-09T11:42:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.326+0000 7f076f1ff700 1 Processor -- start 2023-06-09T11:42:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.326+0000 7f076f1ff700 1 -- start start 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07680c9f40 0x7f07680ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 0x7f07681f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768147260 con 0x7f0768153880 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768146f00 con 0x7f076815b4a0 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f076f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07680cb930 con 0x7f07680c9f40 2023-06-09T11:42:47.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.327+0000 7f0767fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51576/0 (socket says 172.21.15.7:51576) 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 -- 172.21.15.7:0/453391776 learned_addr learned my addr 172.21.15.7:0/453391776 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f076cf9b700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07680c9f40 0x7f07680ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f076d79c700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 0x7f07681f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 -- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07680c9f40 msgr2=0x7f07680ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07680c9f40 0x7f07680ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 -- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 msgr2=0x7f07681f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 0x7f07681f3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 -- 172.21.15.7:0/453391776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0094a0 con 0x7f0768153880 2023-06-09T11:42:47.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f076d79c700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 0x7f07681f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f0767fff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0754007fd0 tx=0x7f075400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754005d70 con 0x7f0768153880 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.328+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07540070d0 con 0x7f0768153880 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.329+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0768159c60 con 0x7f0768153880 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.329+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754007460 con 0x7f0768153880 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.329+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0768159980 con 0x7f0768153880 2023-06-09T11:42:47.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.330+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768159f40 con 0x7f0768153880 2023-06-09T11:42:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.331+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f07540075c0 con 0x7f0768153880 2023-06-09T11:42:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.332+0000 7f07657fa700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 0x7f074c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:47.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.332+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f0754088fe0 con 0x7f0768153880 2023-06-09T11:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.332+0000 7f076cf9b700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 0x7f074c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:47.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.333+0000 7f076cf9b700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 0x7f074c06d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f075c0090b0 tx=0x7f075c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:47.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.336+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f07540539b0 con 0x7f0768153880 2023-06-09T11:42:47.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.585+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f076815a500 con 0x7f0768153880 2023-06-09T11:42:47.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.585+0000 7f07657fa700 1 -- 172.21.15.7:0/453391776 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+5472 (secure 0 0 0) 0x7f076815a500 con 0x7f0768153880 2023-06-09T11:42:47.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:47.590 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":145,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:43.740975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:clientreplay","state_seq":28,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 msgr2=0x7f074c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 0x7f074c06d190 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f075c0090b0 tx=0x7f075c008030 comp rx=0 tx=0).stop 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 msgr2=0x7f07681f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0754007fd0 tx=0x7f075400ca90 comp rx=0 tx=0).stop 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 shutdown_connections 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f076815b4a0 0x7f07681f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f074c06ace0 0x7f074c06d190 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07680c9f40 0x7f07680ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 --2- 172.21.15.7:0/453391776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0768153880 0x7f07681f2d40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 >> 172.21.15.7:0/453391776 conn(0x7f07680737f0 msgr2=0x7f076814de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 shutdown_connections 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:47.588+0000 7f076f1ff700 1 -- 172.21.15.7:0/453391776 wait complete. 2023-06-09T11:42:47.594 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 145 2023-06-09T11:42:47.610 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:42:49.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:49.244 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:42:49.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:49.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.820+0000 7f725a1ff700 1 Processor -- start 2023-06-09T11:42:49.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.820+0000 7f725a1ff700 1 -- start start 2023-06-09T11:42:49.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f725a1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72540c0140 0x7f7254161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f725a1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540c0bf0 con 0x7f725415a0c0 2023-06-09T11:42:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f725a1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72540702f0 con 0x7f72540c1bc0 2023-06-09T11:42:49.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f725a1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7254070110 con 0x7f72540c0140 2023-06-09T11:42:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f7253fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f72540c0140 0x7f7254161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f7252ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f725415a0c0 0x7f72541501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39276/0 (socket says 172.21.15.7:39276) 2023-06-09T11:42:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.821+0000 7f7252ffd700 1 -- 172.21.15.7:0/3146117643 learned_addr learned my addr 172.21.15.7:0/3146117643 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:42:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f7253fff700 1 -- 172.21.15.7:0/3146117643 >> v1:172.21.15.117:6789/0 conn(0x7f72540c1bc0 legacy=0x7f72541599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:49.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f7253fff700 1 -- 172.21.15.7:0/3146117643 >> v1:172.21.15.7:6789/0 conn(0x7f725415a0c0 legacy=0x7f72541501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f7253fff700 1 -- 172.21.15.7:0/3146117643 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540c0e90 con 0x7f72540c0140 2023-06-09T11:42:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f7253fff700 1 --2- 172.21.15.7:0/3146117643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540c0140 0x7f7254161cf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bfa0 tx=0x7f724c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b55addddee444e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f72527fc700 1 -- 172.21.15.7:0/3146117643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c010070 con 0x7f72540c0140 2023-06-09T11:42:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f72527fc700 1 -- 172.21.15.7:0/3146117643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00b2e0 con 0x7f72540c0140 2023-06-09T11:42:49.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.822+0000 7f72527fc700 1 -- 172.21.15.7:0/3146117643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00dcf0 con 0x7f72540c0140 2023-06-09T11:42:49.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 -- 172.21.15.7:0/3146117643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540c0140 msgr2=0x7f7254161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:49.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 --2- 172.21.15.7:0/3146117643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540c0140 0x7f7254161cf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bfa0 tx=0x7f724c00d330 comp rx=0 tx=0).stop 2023-06-09T11:42:49.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 -- 172.21.15.7:0/3146117643 shutdown_connections 2023-06-09T11:42:49.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 --2- 172.21.15.7:0/3146117643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72540c0140 0x7f7254161cf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:49.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 -- 172.21.15.7:0/3146117643 >> 172.21.15.7:0/3146117643 conn(0x7f725406a4d0 msgr2=0x7f725406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 -- 172.21.15.7:0/3146117643 shutdown_connections 2023-06-09T11:42:49.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.823+0000 7f725a1ff700 1 -- 172.21.15.7:0/3146117643 wait complete. 2023-06-09T11:42:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.825+0000 7f725a1ff700 1 Processor -- start 2023-06-09T11:42:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.825+0000 7f725a1ff700 1 -- start start 2023-06-09T11:42:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72540c0140 0x7f72540c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72540c1bc0 0x7f72541eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:49.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72541eafc0 con 0x7f725415a0c0 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540c3940 con 0x7f72540c1bc0 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f725a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540c4da0 con 0x7f72540c0140 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f7253fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f7253fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51600/0 (socket says 172.21.15.7:51600) 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f7253fff700 1 -- 172.21.15.7:0/2604486151 learned_addr learned my addr 172.21.15.7:0/2604486151 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f7252ffd700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72540c1bc0 0x7f72541eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:49.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.826+0000 7f72537fe700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72540c0140 0x7f72540c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 -- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72540c0140 msgr2=0x7f72540c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72540c0140 0x7f72540c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 -- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72540c1bc0 msgr2=0x7f72541eb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72540c1bc0 0x7f72541eb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:49.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 -- 172.21.15.7:0/2604486151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c00bc80 con 0x7f725415a0c0 2023-06-09T11:42:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f7253fff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bfd0 tx=0x7f724c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c00dea0 con 0x7f725415a0c0 2023-06-09T11:42:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c00b450 con 0x7f725415a0c0 2023-06-09T11:42:49.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c009540 con 0x7f725415a0c0 2023-06-09T11:42:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72540c1450 con 0x7f725415a0c0 2023-06-09T11:42:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.827+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72540c1170 con 0x7f725415a0c0 2023-06-09T11:42:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.828+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72540705d0 con 0x7f725415a0c0 2023-06-09T11:42:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.829+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f724c0096a0 con 0x7f725415a0c0 2023-06-09T11:42:49.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.830+0000 7f723bfff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.830+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f724c08cfe0 con 0x7f725415a0c0 2023-06-09T11:42:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.831+0000 7f72537fe700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:49.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.831+0000 7f72537fe700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 0x7f724006d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f7244009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:49.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:49.835+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f724c057ab0 con 0x7f725415a0c0 2023-06-09T11:42:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.089+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72541eb100 con 0x7f725415a0c0 2023-06-09T11:42:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.090+0000 7f723bfff700 1 -- 172.21.15.7:0/2604486151 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f72541eb100 con 0x7f725415a0c0 2023-06-09T11:42:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:50.095 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:49.757447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 msgr2=0x7f724006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 0x7f724006d190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7244007fd0 tx=0x7f7244009040 comp rx=0 tx=0).stop 2023-06-09T11:42:50.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 msgr2=0x7f725414b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f724c00bfd0 tx=0x7f724c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 shutdown_connections 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f725415a0c0 0x7f725414b610 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f724006ace0 0x7f724006d190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f72540c0140 0x7f72540c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 --2- 172.21.15.7:0/2604486151 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f72540c1bc0 0x7f72541eb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:50.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 >> 172.21.15.7:0/2604486151 conn(0x7f725406a4d0 msgr2=0x7f725414ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 shutdown_connections 2023-06-09T11:42:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:50.092+0000 7f725a1ff700 1 -- 172.21.15.7:0/2604486151 wait complete. 2023-06-09T11:42:50.099 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 146 2023-06-09T11:42:50.115 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=2, replacing gid=5994 2023-06-09T11:42:50.115 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.e 2023-06-09T11:42:52.116 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-06-09T11:42:52.116 INFO:tasks.ceph.mds.e:Restarting daemon 2023-06-09T11:42:52.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-06-09T11:42:52.121 INFO:tasks.ceph.mds.e:Started 2023-06-09T11:42:52.122 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:52.280 INFO:tasks.ceph.mds.e.smithi007.stdout:starting mds.e at 2023-06-09T11:42:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.330+0000 7fbe1dfff700 1 Processor -- start 2023-06-09T11:42:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.330+0000 7fbe1dfff700 1 -- start start 2023-06-09T11:42:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe1dfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 0x7fbe1815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe1dfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe180c9f40 0x7fbe1815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe1dfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe1815a500 con 0x7fbe1815b2d0 2023-06-09T11:42:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe1dfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe180c5f10 con 0x7fbe180c9f40 2023-06-09T11:42:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe1dfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1815da00 con 0x7fbe18153880 2023-06-09T11:42:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe180c9f40 0x7fbe1815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe180c9f40 0x7fbe1815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51640/0 (socket says 172.21.15.7:51640) 2023-06-09T11:42:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 -- 172.21.15.7:0/3129168351 learned_addr learned my addr 172.21.15.7:0/3129168351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe177fe700 1 --2- 172.21.15.7:0/3129168351 >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 0x7fbe1815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 -- 172.21.15.7:0/3129168351 >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 msgr2=0x7fbe1815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 --2- 172.21.15.7:0/3129168351 >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 0x7fbe1815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 -- 172.21.15.7:0/3129168351 >> v1:172.21.15.117:6789/0 conn(0x7fbe1815b2d0 legacy=0x7fbe180c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe16ffd700 1 -- 172.21.15.7:0/3129168351 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe181596e0 con 0x7fbe180c9f40 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.332+0000 7fbe177fe700 1 --2- 172.21.15.7:0/3129168351 >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 0x7fbe1815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe16ffd700 1 --2- 172.21.15.7:0/3129168351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180c9f40 0x7fbe1815ad90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbe1808d340 tx=0x7fbe0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0d8bf6fe8c37a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe167fc700 1 -- 172.21.15.7:0/3129168351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c004230 con 0x7fbe180c9f40 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe167fc700 1 -- 172.21.15.7:0/3129168351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c004390 con 0x7fbe180c9f40 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe167fc700 1 -- 172.21.15.7:0/3129168351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0036a0 con 0x7fbe180c9f40 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 -- 172.21.15.7:0/3129168351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180c9f40 msgr2=0x7fbe1815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/3129168351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180c9f40 0x7fbe1815ad90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fbe1808d340 tx=0x7fbe0c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 -- 172.21.15.7:0/3129168351 shutdown_connections 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/3129168351 >> v2:172.21.15.117:3300/0 conn(0x7fbe18153880 0x7fbe1815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/3129168351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe180c9f40 0x7fbe1815ad90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 -- 172.21.15.7:0/3129168351 >> 172.21.15.7:0/3129168351 conn(0x7fbe180737f0 msgr2=0x7fbe180720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.333+0000 7fbe1dfff700 1 -- 172.21.15.7:0/3129168351 shutdown_connections 2023-06-09T11:42:52.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.334+0000 7fbe1dfff700 1 -- 172.21.15.7:0/3129168351 wait complete. 2023-06-09T11:42:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.335+0000 7fbe1dfff700 1 Processor -- start 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.335+0000 7fbe1dfff700 1 -- start start 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180c9f40 0x7fbe1815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18153880 0x7fbe18158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180cc320 con 0x7fbe1815b2d0 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1814b510 con 0x7fbe18153880 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.336+0000 7fbe1dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1814bdb0 con 0x7fbe180c9f40 2023-06-09T11:42:52.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:51646/0 (socket says 172.21.15.7:51646) 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 -- 172.21.15.7:0/80390061 learned_addr learned my addr 172.21.15.7:0/80390061 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe177fe700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180c9f40 0x7fbe1815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe16ffd700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18153880 0x7fbe18158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 -- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180c9f40 msgr2=0x7fbe1815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180c9f40 0x7fbe1815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 -- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18153880 msgr2=0x7fbe18158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18153880 0x7fbe18158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 -- 172.21.15.7:0/80390061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c009550 con 0x7fbe1815b2d0 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.337+0000 7fbe17fff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000b970 tx=0x7fbe1000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.338+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000d860 con 0x7fbe1815b2d0 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.338+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000d9c0 con 0x7fbe1815b2d0 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.338+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1000dd70 con 0x7fbe1815b2d0 2023-06-09T11:42:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.338+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18159c60 con 0x7fbe1815b2d0 2023-06-09T11:42:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.338+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe18159980 con 0x7fbe1815b2d0 2023-06-09T11:42:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.339+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe18159f40 con 0x7fbe1815b2d0 2023-06-09T11:42:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.340+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe1000e430 con 0x7fbe1815b2d0 2023-06-09T11:42:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.341+0000 7fbdfffff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.341+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fbe10089b70 con 0x7fbe1815b2d0 2023-06-09T11:42:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.342+0000 7fbe177fe700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:52.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.342+0000 7fbe177fe700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe08006fd0 tx=0x7fbe08008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.346+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbe100546d0 con 0x7fbe1815b2d0 2023-06-09T11:42:52.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.599+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1808c990 con 0x7fbe1815b2d0 2023-06-09T11:42:52.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.600+0000 7fbdfffff700 1 -- 172.21.15.7:0/80390061 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fbe10057cf0 con 0x7fbe1815b2d0 2023-06-09T11:42:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:52.605 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":146,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:49.757447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 msgr2=0x7fbe0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fbe08006fd0 tx=0x7fbe08008040 comp rx=0 tx=0).stop 2023-06-09T11:42:52.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 msgr2=0x7fbe180cf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000b970 tx=0x7fbe1000bc80 comp rx=0 tx=0).stop 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 shutdown_connections 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe1815b2d0 0x7fbe180cf5c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe180c9f40 0x7fbe1815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 --2- 172.21.15.7:0/80390061 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe18153880 0x7fbe18158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:52.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.602+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 >> 172.21.15.7:0/80390061 conn(0x7fbe180737f0 msgr2=0x7fbe1814d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.603+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 shutdown_connections 2023-06-09T11:42:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:52.603+0000 7fbe1dfff700 1 -- 172.21.15.7:0/80390061 wait complete. 2023-06-09T11:42:52.609 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 146 2023-06-09T11:42:52.625 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-06-09T11:42:54.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:42:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.837+0000 7fbc993ff700 1 Processor -- start 2023-06-09T11:42:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.838+0000 7fbc993ff700 1 -- start start 2023-06-09T11:42:54.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc993ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94153880 0x7fbc9415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc993ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 0x7fbc940c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc993ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9415a500 con 0x7fbc940c9f40 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc993ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc940c5f10 con 0x7fbc94153880 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc993ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9415da00 con 0x7fbc9415b4a0 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc92ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94153880 0x7fbc9415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc92ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc94153880 0x7fbc9415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53814/0 (socket says 172.21.15.7:53814) 2023-06-09T11:42:54.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc927fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fbc940c9f40 0x7fbc9415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:60298/0 (socket says 172.21.15.7:60298) 2023-06-09T11:42:54.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc92ffd700 1 -- 172.21.15.7:0/2881837382 learned_addr learned my addr 172.21.15.7:0/2881837382 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.839+0000 7fbc937fe700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 0x7fbc940c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc92ffd700 1 -- 172.21.15.7:0/2881837382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 msgr2=0x7fbc940c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:42:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc92ffd700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 0x7fbc940c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc92ffd700 1 -- 172.21.15.7:0/2881837382 >> v1:172.21.15.7:6789/0 conn(0x7fbc940c9f40 legacy=0x7fbc9415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:54.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc92ffd700 1 -- 172.21.15.7:0/2881837382 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc941596e0 con 0x7fbc94153880 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc937fe700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 0x7fbc940c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc91ffb700 1 -- 172.21.15.7:0/2881837382 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072011568 0 0) 0x7fbc88001110 con 0x7fbc940c9f40 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc92ffd700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94153880 0x7fbc9415d4c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbc840098f0 tx=0x7fbc84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dcde7eadb4bbc5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc91ffb700 1 -- 172.21.15.7:0/2881837382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400b8a0 con 0x7fbc94153880 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc91ffb700 1 -- 172.21.15.7:0/2881837382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8400ba00 con 0x7fbc94153880 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.840+0000 7fbc91ffb700 1 -- 172.21.15.7:0/2881837382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8400bd90 con 0x7fbc94153880 2023-06-09T11:42:54.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 -- 172.21.15.7:0/2881837382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94153880 msgr2=0x7fbc9415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94153880 0x7fbc9415d4c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbc840098f0 tx=0x7fbc84009c00 comp rx=0 tx=0).stop 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 -- 172.21.15.7:0/2881837382 shutdown_connections 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc9415b4a0 0x7fbc940c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 --2- 172.21.15.7:0/2881837382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc94153880 0x7fbc9415d4c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 -- 172.21.15.7:0/2881837382 >> 172.21.15.7:0/2881837382 conn(0x7fbc940737f0 msgr2=0x7fbc940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 -- 172.21.15.7:0/2881837382 shutdown_connections 2023-06-09T11:42:54.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.841+0000 7fbc993ff700 1 -- 172.21.15.7:0/2881837382 wait complete. 2023-06-09T11:42:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.842+0000 7fbc993ff700 1 Processor -- start 2023-06-09T11:42:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.843+0000 7fbc993ff700 1 -- start start 2023-06-09T11:42:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.843+0000 7fbc993ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.843+0000 7fbc993ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc94153880 0x7fbc940b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.843+0000 7fbc993ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc9415b4a0 0x7fbc940b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc993ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940b3dc0 con 0x7fbc940c9f40 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc993ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940b3f90 con 0x7fbc9415b4a0 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc993ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc940b4230 con 0x7fbc94153880 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53818/0 (socket says 172.21.15.7:53818) 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 -- 172.21.15.7:0/614290608 learned_addr learned my addr 172.21.15.7:0/614290608 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:42:54.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc927fc700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc94153880 0x7fbc940b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc937fe700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc9415b4a0 0x7fbc940b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 -- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc94153880 msgr2=0x7fbc940b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc94153880 0x7fbc940b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 -- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc9415b4a0 msgr2=0x7fbc940b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc9415b4a0 0x7fbc940b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 -- 172.21.15.7:0/614290608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84009550 con 0x7fbc940c9f40 2023-06-09T11:42:54.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.844+0000 7fbc92ffd700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009c40 tx=0x7fbc84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.845+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc840036a0 con 0x7fbc940c9f40 2023-06-09T11:42:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.845+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc84003800 con 0x7fbc940c9f40 2023-06-09T11:42:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.845+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc84003b70 con 0x7fbc940c9f40 2023-06-09T11:42:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.845+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc94159c60 con 0x7fbc940c9f40 2023-06-09T11:42:54.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.845+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc94159980 con 0x7fbc940c9f40 2023-06-09T11:42:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.846+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc94159f40 con 0x7fbc940c9f40 2023-06-09T11:42:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.847+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc84003cd0 con 0x7fbc940c9f40 2023-06-09T11:42:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.848+0000 7fbc7b7fe700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 0x7fbc8006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:42:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.848+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fbc84021030 con 0x7fbc940c9f40 2023-06-09T11:42:54.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.848+0000 7fbc927fc700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 0x7fbc8006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:42:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.849+0000 7fbc927fc700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 0x7fbc8006d160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbc941eccf0 tx=0x7fbc88007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:42:54.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:54.852+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbc84055d90 con 0x7fbc940c9f40 2023-06-09T11:42:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.104+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc940cf2b0 con 0x7fbc940c9f40 2023-06-09T11:42:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.105+0000 7fbc7b7fe700 1 -- 172.21.15.7:0/614290608 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 147 v147) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fbc840593b0 con 0x7fbc940c9f40 2023-06-09T11:42:55.109 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:42:55.110 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":147,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:42:49.757447+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:42:55.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 msgr2=0x7fbc8006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 0x7fbc8006d160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbc941eccf0 tx=0x7fbc88007a40 comp rx=0 tx=0).stop 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 msgr2=0x7fbc94158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbc84009c40 tx=0x7fbc84004750 comp rx=0 tx=0).stop 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 shutdown_connections 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc9415b4a0 0x7fbc940b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:55.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc940c9f40 0x7fbc94158690 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbc8006acb0 0x7fbc8006d160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 --2- 172.21.15.7:0/614290608 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc94153880 0x7fbc940b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 >> 172.21.15.7:0/614290608 conn(0x7fbc940737f0 msgr2=0x7fbc940720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 shutdown_connections 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:42:55.107+0000 7fbc993ff700 1 -- 172.21.15.7:0/614290608 wait complete. 2023-06-09T11:42:55.113 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 147 2023-06-09T11:42:55.129 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-06-09T11:42:55.129 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 70 secs before thrashing 2023-06-09T11:43:05.169 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 15% (39/245) Updating files: 16% (40/245) Updating files: 17% (42/245) Updating files: 18% (45/245) Updating files: 19% (47/245) Updating files: 20% (49/245) Updating files: 21% (52/245) Updating files: 22% (54/245) Updating files: 22% (56/245) Updating files: 23% (57/245) Updating files: 24% (59/245) Updating files: 24% (60/245) Updating files: 25% (62/245) Updating files: 26% (64/245) Updating files: 26% (66/245) Updating files: 27% (67/245) Updating files: 28% (69/245) Updating files: 28% (70/245) Updating files: 29% (72/245) Updating files: 30% (74/245) Updating files: 31% (76/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 37% (93/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 40% (98/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 54% (134/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 80% (197/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 86% (212/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2023-06-09T11:43:05.688 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 1dd420957f9 journal: detect size of raw block devices properly 2023-06-09T11:43:05.695 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.5 2023-06-09T11:43:06.329 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:43:06.329 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 6 -eq 48 ']' 2023-06-09T11:43:06.329 INFO:tasks.workunit.client.0.smithi007.stdout:v0.6 2023-06-09T11:43:06.330 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.6 2023-06-09T11:43:06.330 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.6 2023-06-09T11:43:06.330 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.6 2023-06-09T11:43:20.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:20.007 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:30.985 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 17% (40/227) Updating files: 18% (41/227) Updating files: 19% (44/227) Updating files: 20% (46/227) Updating files: 21% (48/227) Updating files: 22% (50/227) Updating files: 22% (51/227) Updating files: 23% (53/227) Updating files: 23% (54/227) Updating files: 24% (55/227) Updating files: 25% (57/227) Updating files: 26% (60/227) Updating files: 26% (61/227) Updating files: 27% (62/227) Updating files: 28% (64/227) Updating files: 28% (65/227) Updating files: 29% (66/227) Updating files: 30% (69/227) Updating files: 31% (71/227) Updating files: 32% (73/227) Updating files: 33% (75/227) Updating files: 34% (78/227) Updating files: 35% (80/227) Updating files: 36% (82/227) Updating files: 37% (84/227) Updating files: 37% (86/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 54% (124/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 74% (170/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 81% (185/227) Updating files: 82% (187/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 84% (192/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2023-06-09T11:43:31.117 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at a4c752df713 v0.6 2023-06-09T11:43:31.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.6 2023-06-09T11:43:31.551 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:43:31.552 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 7 -eq 48 ']' 2023-06-09T11:43:31.552 INFO:tasks.workunit.client.0.smithi007.stdout:v0.7 2023-06-09T11:43:31.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.7 2023-06-09T11:43:31.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.7 2023-06-09T11:43:31.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.7 2023-06-09T11:43:50.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:50.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:43:50.831 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 12% (25/201) Updating files: 13% (27/201) Updating files: 14% (29/201) Updating files: 15% (31/201) Updating files: 16% (33/201) Updating files: 17% (35/201) Updating files: 18% (37/201) Updating files: 19% (39/201) Updating files: 20% (41/201) Updating files: 21% (43/201) Updating files: 22% (45/201) Updating files: 23% (47/201) Updating files: 24% (49/201) Updating files: 25% (51/201) Updating files: 26% (53/201) Updating files: 27% (55/201) Updating files: 28% (57/201) Updating files: 29% (59/201) Updating files: 30% (61/201) Updating files: 31% (63/201) Updating files: 31% (64/201) Updating files: 32% (65/201) Updating files: 33% (67/201) Updating files: 33% (68/201) Updating files: 34% (69/201) Updating files: 35% (71/201) Updating files: 36% (73/201) Updating files: 37% (75/201) Updating files: 38% (77/201) Updating files: 39% (79/201) Updating files: 40% (81/201) Updating files: 41% (83/201) Updating files: 42% (85/201) Updating files: 43% (87/201) Updating files: 44% (89/201) Updating files: 45% (91/201) Updating files: 46% (93/201) Updating files: 47% (95/201) Updating files: 48% (97/201) Updating files: 49% (99/201) Updating files: 50% (101/201) Updating files: 51% (103/201) Updating files: 52% (105/201) Updating files: 53% (107/201) Updating files: 54% (109/201) Updating files: 55% (111/201) Updating files: 56% (113/201) Updating files: 57% (115/201) Updating files: 58% (117/201) Updating files: 59% (119/201) Updating files: 60% (121/201) Updating files: 61% (123/201) Updating files: 62% (125/201) Updating files: 63% (127/201) Updating files: 64% (129/201) Updating files: 64% (130/201) Updating files: 65% (131/201) Updating files: 66% (133/201) Updating files: 67% (135/201) Updating files: 68% (137/201) Updating files: 69% (139/201) Updating files: 70% (141/201) Updating files: 71% (143/201) Updating files: 72% (145/201) Updating files: 73% (147/201) Updating files: 74% (149/201) Updating files: 75% (151/201) Updating files: 76% (153/201) Updating files: 77% (155/201) Updating files: 78% (157/201) Updating files: 79% (159/201) Updating files: 80% (161/201) Updating files: 81% (163/201) Updating files: 82% (165/201) Updating files: 83% (167/201) Updating files: 84% (169/201) Updating files: 85% (171/201) Updating files: 86% (173/201) Updating files: 87% (175/201) Updating files: 88% (177/201) Updating files: 89% (179/201) Updating files: 90% (181/201) Updating files: 91% (183/201) Updating files: 92% (185/201) Updating files: 93% (187/201) Updating files: 94% (189/201) Updating files: 95% (191/201) Updating files: 96% (193/201) Updating files: 96% (194/201) Updating files: 97% (195/201) Updating files: 98% (197/201) Updating files: 99% (199/201) Updating files: 100% (201/201) Updating files: 100% (201/201), done. 2023-06-09T11:43:51.560 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at c46e1b320c6 mkcephfs: btrfsctl -a between mkfs and mount 2023-06-09T11:43:51.565 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.7 2023-06-09T11:43:51.593 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:43:51.594 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 8 -eq 48 ']' 2023-06-09T11:43:51.594 INFO:tasks.workunit.client.0.smithi007.stdout:v0.8 2023-06-09T11:43:51.594 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.8 2023-06-09T11:43:51.594 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.8 2023-06-09T11:43:51.595 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.8 2023-06-09T11:44:03.474 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 4% (9/225) Updating files: 5% (12/225) Updating files: 6% (14/225) Updating files: 7% (16/225) Updating files: 8% (18/225) Updating files: 9% (21/225) Updating files: 10% (23/225) Updating files: 11% (25/225) Updating files: 12% (27/225) Updating files: 13% (30/225) Updating files: 14% (32/225) Updating files: 15% (34/225) Updating files: 16% (36/225) Updating files: 17% (39/225) Updating files: 18% (41/225) Updating files: 19% (43/225) Updating files: 20% (45/225) Updating files: 21% (48/225) Updating files: 22% (50/225) Updating files: 23% (52/225) Updating files: 24% (54/225) Updating files: 25% (57/225) Updating files: 26% (59/225) Updating files: 26% (60/225) Updating files: 27% (61/225) Updating files: 28% (63/225) Updating files: 29% (66/225) Updating files: 30% (68/225) Updating files: 31% (70/225) Updating files: 32% (72/225) Updating files: 33% (75/225) Updating files: 34% (77/225) Updating files: 35% (79/225) Updating files: 36% (81/225) Updating files: 37% (84/225) Updating files: 38% (86/225) Updating files: 39% (88/225) Updating files: 40% (90/225) Updating files: 41% (93/225) Updating files: 42% (95/225) Updating files: 43% (97/225) Updating files: 44% (99/225) Updating files: 45% (102/225) Updating files: 46% (104/225) Updating files: 47% (106/225) Updating files: 48% (108/225) Updating files: 49% (111/225) Updating files: 50% (113/225) Updating files: 51% (115/225) Updating files: 52% (117/225) Updating files: 52% (119/225) Updating files: 53% (120/225) Updating files: 54% (122/225) Updating files: 55% (124/225) Updating files: 56% (126/225) Updating files: 57% (129/225) Updating files: 58% (131/225) Updating files: 59% (133/225) Updating files: 60% (135/225) Updating files: 61% (138/225) Updating files: 62% (140/225) Updating files: 63% (142/225) Updating files: 64% (144/225) Updating files: 65% (147/225) Updating files: 66% (149/225) Updating files: 67% (151/225) Updating files: 68% (153/225) Updating files: 69% (156/225) Updating files: 70% (158/225) Updating files: 71% (160/225) Updating files: 72% (162/225) Updating files: 73% (165/225) Updating files: 74% (167/225) Updating files: 75% (169/225) Updating files: 76% (171/225) Updating files: 77% (174/225) Updating files: 78% (176/225) Updating files: 79% (178/225) Updating files: 80% (180/225) Updating files: 81% (183/225) Updating files: 82% (185/225) Updating files: 83% (187/225) Updating files: 83% (188/225) Updating files: 84% (189/225) Updating files: 85% (192/225) Updating files: 86% (194/225) Updating files: 87% (196/225) Updating files: 87% (197/225) Updating files: 88% (198/225) Updating files: 89% (201/225) Updating files: 90% (203/225) Updating files: 91% (205/225) Updating files: 92% (207/225) Updating files: 93% (210/225) Updating files: 94% (212/225) Updating files: 95% (214/225) Updating files: 96% (216/225) Updating files: 97% (219/225) Updating files: 98% (221/225) Updating files: 99% (223/225) Updating files: 100% (225/225) Updating files: 100% (225/225), done. 2023-06-09T11:44:03.651 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 284161c00ab kclient: kill dput/d_drop debug cruft 2023-06-09T11:44:03.659 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.8 2023-06-09T11:44:03.692 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:44:03.692 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 9 -eq 48 ']' 2023-06-09T11:44:03.692 INFO:tasks.workunit.client.0.smithi007.stdout:v0.9 2023-06-09T11:44:03.693 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.9 2023-06-09T11:44:03.693 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.9 2023-06-09T11:44:03.693 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.9 2023-06-09T11:44:05.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:05.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.353+0000 7fc8a1bff700 1 Processor -- start 2023-06-09T11:44:05.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.353+0000 7fc8a1bff700 1 -- start start 2023-06-09T11:44:05.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc8a1bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc89c151510 0x7fc89c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc8a1bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c1580a0 con 0x7fc89c15b380 2023-06-09T11:44:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc8a1bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0c6180 con 0x7fc89c071230 2023-06-09T11:44:05.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc8a1bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c1518e0 con 0x7fc89c151510 2023-06-09T11:44:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc89b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc89c151510 0x7fc89c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc89affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc89c15b380 0x7fc89c070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58214/0 (socket says 172.21.15.7:58214) 2023-06-09T11:44:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.355+0000 7fc89affd700 1 -- 172.21.15.7:0/676575273 learned_addr learned my addr 172.21.15.7:0/676575273 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.356+0000 7fc89a7fc700 1 -- 172.21.15.7:0/676575273 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969210640 0 0) 0x7fc89c1580a0 con 0x7fc89c15b380 2023-06-09T11:44:05.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.356+0000 7fc89b7fe700 1 -- 172.21.15.7:0/676575273 >> v1:172.21.15.117:6789/0 conn(0x7fc89c071230 legacy=0x7fc89c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.356+0000 7fc89b7fe700 1 -- 172.21.15.7:0/676575273 >> v1:172.21.15.7:6789/0 conn(0x7fc89c15b380 legacy=0x7fc89c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.356+0000 7fc89b7fe700 1 -- 172.21.15.7:0/676575273 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c157490 con 0x7fc89c151510 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.356+0000 7fc89b7fe700 1 --2- 172.21.15.7:0/676575273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c151510 0x7fc89c0c5b40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009840 tx=0x7fc88c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb9819ecef43621 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc89a7fc700 1 -- 172.21.15.7:0/676575273 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c011830 con 0x7fc89c151510 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc89a7fc700 1 -- 172.21.15.7:0/676575273 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c011990 con 0x7fc89c151510 2023-06-09T11:44:05.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc89a7fc700 1 -- 172.21.15.7:0/676575273 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c011d20 con 0x7fc89c151510 2023-06-09T11:44:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 -- 172.21.15.7:0/676575273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c151510 msgr2=0x7fc89c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/676575273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c151510 0x7fc89c0c5b40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009840 tx=0x7fc88c009b50 comp rx=0 tx=0).stop 2023-06-09T11:44:05.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 -- 172.21.15.7:0/676575273 shutdown_connections 2023-06-09T11:44:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/676575273 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c151510 0x7fc89c0c5b40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 -- 172.21.15.7:0/676575273 >> 172.21.15.7:0/676575273 conn(0x7fc89c06a4d0 msgr2=0x7fc89c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 -- 172.21.15.7:0/676575273 shutdown_connections 2023-06-09T11:44:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.357+0000 7fc8a1bff700 1 -- 172.21.15.7:0/676575273 wait complete. 2023-06-09T11:44:05.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.360+0000 7fc8a1bff700 1 Processor -- start 2023-06-09T11:44:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.360+0000 7fc8a1bff700 1 -- start start 2023-06-09T11:44:05.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc89c151510 0x7fc89c14a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc89c15b380 0x7fc89c14a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0caeb0 con 0x7fc89c071230 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c5260 con 0x7fc89c151510 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc8a1bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c3530 con 0x7fc89c15b380 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc89b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35888/0 (socket says 172.21.15.7:35888) 2023-06-09T11:44:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.361+0000 7fc89b7fe700 1 -- 172.21.15.7:0/1371926119 learned_addr learned my addr 172.21.15.7:0/1371926119 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:05.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 -- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc89c15b380 msgr2=0x7fc89c14a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc89c15b380 0x7fc89c14a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 -- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc89c151510 msgr2=0x7fc89c14a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc89c151510 0x7fc89c14a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 -- 172.21.15.7:0/1371926119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0094a0 con 0x7fc89c071230 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc89b7fe700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0090b0 tx=0x7fc88c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01a900 con 0x7fc89c071230 2023-06-09T11:44:05.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c01aa60 con 0x7fc89c071230 2023-06-09T11:44:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c157a50 con 0x7fc89c071230 2023-06-09T11:44:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.362+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c01add0 con 0x7fc89c071230 2023-06-09T11:44:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.363+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c157770 con 0x7fc89c071230 2023-06-09T11:44:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.364+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c157d30 con 0x7fc89c071230 2023-06-09T11:44:05.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.364+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc88c022070 con 0x7fc89c071230 2023-06-09T11:44:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.365+0000 7fc883fff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.366+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fc88c08ace0 con 0x7fc89c071230 2023-06-09T11:44:05.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.366+0000 7fc89affd700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.366+0000 7fc89affd700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 0x7fc88806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc890000c00 tx=0x7fc890007a50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.371+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc88c055960 con 0x7fc89c071230 2023-06-09T11:44:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.646+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc89c1efe30 con 0x7fc89c071230 2023-06-09T11:44:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.647+0000 7fc883fff700 1 -- 172.21.15.7:0/1371926119 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fc88c058f80 con 0x7fc89c071230 2023-06-09T11:44:05.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:05.651 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 msgr2=0x7fc88806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 0x7fc88806d190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc890000c00 tx=0x7fc890007a50 comp rx=0 tx=0).stop 2023-06-09T11:44:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 msgr2=0x7fc89c0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0090b0 tx=0x7fc88c004750 comp rx=0 tx=0).stop 2023-06-09T11:44:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 shutdown_connections 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc88806ace0 0x7fc88806d190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc89c15b380 0x7fc89c14a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc89c071230 0x7fc89c0c7d40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 --2- 172.21.15.7:0/1371926119 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc89c151510 0x7fc89c14a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.649+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 >> 172.21.15.7:0/1371926119 conn(0x7fc89c06a4d0 msgr2=0x7fc89c14cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.650+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 shutdown_connections 2023-06-09T11:44:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.650+0000 7fc8a1bff700 1 -- 172.21.15.7:0/1371926119 wait complete. 2023-06-09T11:44:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:05.671 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=0) 2023-06-09T11:44:05.671 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:44:05.671 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:44:05.689 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:44:05.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.887+0000 7ff8e07ff700 1 Processor -- start 2023-06-09T11:44:05.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.887+0000 7ff8e07ff700 1 -- start start 2023-06-09T11:44:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.888+0000 7ff8e07ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d815a0c0 0x7ff8d81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.888+0000 7ff8e07ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8d80c0140 0x7ff8d81506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.888+0000 7ff8e07ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d80c0bf0 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.888+0000 7ff8e07ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d8070110 con 0x7ff8d80c0140 2023-06-09T11:44:05.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.888+0000 7ff8e07ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8071300 con 0x7ff8d815a0c0 2023-06-09T11:44:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff8d80c0140 0x7ff8d81506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8de59b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff8d80c1bc0 0x7ff8d81599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58216/0 (socket says 172.21.15.7:58216) 2023-06-09T11:44:05.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8de59b700 1 -- 172.21.15.7:0/2203010027 learned_addr learned my addr 172.21.15.7:0/2203010027 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ddd9a700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d815a0c0 0x7ff8d81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 -- 172.21.15.7:0/2203010027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d815a0c0 msgr2=0x7ff8d81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d815a0c0 0x7ff8d81501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 -- 172.21.15.7:0/2203010027 >> v1:172.21.15.7:6789/0 conn(0x7ff8d80c1bc0 legacy=0x7ff8d81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 -- 172.21.15.7:0/2203010027 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d80702f0 con 0x7ff8d80c0140 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8dd599700 1 -- 172.21.15.7:0/2203010027 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487206435 0 0) 0x7ff8d80c0bf0 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.889+0000 7ff8ded9c700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c0140 0x7ff8d81506f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b7fce6e39c1656 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8dd599700 1 -- 172.21.15.7:0/2203010027 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000e070 con 0x7ff8d80c0140 2023-06-09T11:44:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8dd599700 1 -- 172.21.15.7:0/2203010027 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d000bb20 con 0x7ff8d80c0140 2023-06-09T11:44:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8dd599700 1 -- 172.21.15.7:0/2203010027 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0012430 con 0x7ff8d80c0140 2023-06-09T11:44:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 -- 172.21.15.7:0/2203010027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c0140 msgr2=0x7ff8d81506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c0140 0x7ff8d81506f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009d90 tx=0x7ff8d000b330 comp rx=0 tx=0).stop 2023-06-09T11:44:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 -- 172.21.15.7:0/2203010027 shutdown_connections 2023-06-09T11:44:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c0140 0x7ff8d81506f0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/2203010027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d815a0c0 0x7ff8d81501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 -- 172.21.15.7:0/2203010027 >> 172.21.15.7:0/2203010027 conn(0x7ff8d806a4d0 msgr2=0x7ff8d806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:05.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 -- 172.21.15.7:0/2203010027 shutdown_connections 2023-06-09T11:44:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.890+0000 7ff8e07ff700 1 -- 172.21.15.7:0/2203010027 wait complete. 2023-06-09T11:44:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.892+0000 7ff8e07ff700 1 Processor -- start 2023-06-09T11:44:05.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.892+0000 7ff8e07ff700 1 -- start start 2023-06-09T11:44:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d80c0140 0x7ff8d80afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d815a0c0 0x7ff8d814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d80af580 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d81f0bc0 con 0x7ff8d815a0c0 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8e07ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d8072290 con 0x7ff8d80c0140 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.893+0000 7ff8ddd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35902/0 (socket says 172.21.15.7:35902) 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 -- 172.21.15.7:0/1241793358 learned_addr learned my addr 172.21.15.7:0/1241793358 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8de59b700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d80c0140 0x7ff8d80afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ded9c700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d815a0c0 0x7ff8d814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 -- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d80c0140 msgr2=0x7ff8d80afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d80c0140 0x7ff8d80afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 -- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d815a0c0 msgr2=0x7ff8d814b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d815a0c0 0x7ff8d814b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 -- 172.21.15.7:0/1241793358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0009a70 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ddd9a700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e9d0 tx=0x7ff8c800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8009730 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c800ee70 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.894+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d80c1170 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.895+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c800ca70 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.895+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d80c0e90 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.896+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d80c1450 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.897+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8c800cbd0 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.898+0000 7ff8ceffd700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 0x7ff8bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.898+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7ff8c8089e40 con 0x7ff8d80c1bc0 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.898+0000 7ff8de59b700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 0x7ff8bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:05.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.899+0000 7ff8de59b700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 0x7ff8bc06d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:05.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:05.902+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff8c8058380 con 0x7ff8d80c1bc0 2023-06-09T11:44:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.156+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8d8074cf0 con 0x7ff8d80c1bc0 2023-06-09T11:44:06.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.156+0000 7ff8ceffd700 1 -- 172.21.15.7:0/1241793358 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7ff8c8057f10 con 0x7ff8d80c1bc0 2023-06-09T11:44:06.160 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:06.160 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 msgr2=0x7ff8bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 0x7ff8bc06d190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4007fd0 tx=0x7ff8d4007de0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 msgr2=0x7ff8d81eff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e9d0 tx=0x7ff8c800ece0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 shutdown_connections 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff8d815a0c0 0x7ff8d814b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff8bc06ace0 0x7ff8bc06d190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.158+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff8d80c0140 0x7ff8d80afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.159+0000 7ff8e07ff700 1 --2- 172.21.15.7:0/1241793358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8d80c1bc0 0x7ff8d81eff40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.159+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 >> 172.21.15.7:0/1241793358 conn(0x7ff8d806a4d0 msgr2=0x7ff8d814e700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.159+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 shutdown_connections 2023-06-09T11:44:06.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:06.159+0000 7ff8e07ff700 1 -- 172.21.15.7:0/1241793358 wait complete. 2023-06-09T11:44:06.164 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:06.179 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:08.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:08.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.387+0000 7eff2bfff700 1 Processor -- start 2023-06-09T11:44:08.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.387+0000 7eff2bfff700 1 -- start start 2023-06-09T11:44:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 0x7eff2415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff240c9f40 0x7eff2415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7eff2415b2d0 0x7eff240c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff240c5f10 con 0x7eff240c9f40 2023-06-09T11:44:08.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff2415a500 con 0x7eff2415b2d0 2023-06-09T11:44:08.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2415da00 con 0x7eff24153880 2023-06-09T11:44:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff240c9f40 0x7eff2415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7eff240c9f40 0x7eff2415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35910/0 (socket says 172.21.15.7:35910) 2023-06-09T11:44:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 -- 172.21.15.7:0/1869966109 learned_addr learned my addr 172.21.15.7:0/1869966109 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:08.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2a59c700 1 --2- 172.21.15.7:0/1869966109 >> v2:172.21.15.117:3300/0 conn(0x7eff2415b2d0 0x7eff240c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff29d9b700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 0x7eff2415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 -- 172.21.15.7:0/1869966109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 msgr2=0x7eff2415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 0x7eff2415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 -- 172.21.15.7:0/1869966109 >> v2:172.21.15.117:3300/0 conn(0x7eff2415b2d0 msgr2=0x7eff240c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 --2- 172.21.15.7:0/1869966109 >> v2:172.21.15.117:3300/0 conn(0x7eff2415b2d0 0x7eff240c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff2959a700 1 -- 172.21.15.7:0/1869966109 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2415b750 con 0x7eff240c9f40 2023-06-09T11:44:08.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.389+0000 7eff29d9b700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 0x7eff2415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff2959a700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff240c9f40 0x7eff2415ad90 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7eff24071db0 tx=0x7eff14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9320b20042da0e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff28d99700 1 -- 172.21.15.7:0/1869966109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400b870 con 0x7eff240c9f40 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff28d99700 1 -- 172.21.15.7:0/1869966109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400b9d0 con 0x7eff240c9f40 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff28d99700 1 -- 172.21.15.7:0/1869966109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400bd60 con 0x7eff240c9f40 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff2bfff700 1 -- 172.21.15.7:0/1869966109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff240c9f40 msgr2=0x7eff2415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.390+0000 7eff2bfff700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff240c9f40 0x7eff2415ad90 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7eff24071db0 tx=0x7eff14009bd0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 -- 172.21.15.7:0/1869966109 shutdown_connections 2023-06-09T11:44:08.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 --2- 172.21.15.7:0/1869966109 >> v2:172.21.15.117:3300/0 conn(0x7eff2415b2d0 0x7eff240c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff24153880 0x7eff2415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 --2- 172.21.15.7:0/1869966109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff240c9f40 0x7eff2415ad90 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 -- 172.21.15.7:0/1869966109 >> 172.21.15.7:0/1869966109 conn(0x7eff240737f0 msgr2=0x7eff240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 -- 172.21.15.7:0/1869966109 shutdown_connections 2023-06-09T11:44:08.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.391+0000 7eff2bfff700 1 -- 172.21.15.7:0/1869966109 wait complete. 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.392+0000 7eff2bfff700 1 Processor -- start 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.392+0000 7eff2bfff700 1 -- start start 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 0x7eff24150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff2415b2d0 0x7eff2414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff240b4420 con 0x7eff24153880 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff240b4790 con 0x7eff240c9f40 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.393+0000 7eff2bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff240b4a70 con 0x7eff2415b2d0 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35924/0 (socket says 172.21.15.7:35924) 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 -- 172.21.15.7:0/3194536518 learned_addr learned my addr 172.21.15.7:0/3194536518 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2a59c700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff2415b2d0 0x7eff2414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff29d9b700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 0x7eff24150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 -- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff2415b2d0 msgr2=0x7eff2414af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff2415b2d0 0x7eff2414af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 -- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 msgr2=0x7eff24150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 0x7eff24150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 -- 172.21.15.7:0/3194536518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14009550 con 0x7eff24153880 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff29d9b700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 0x7eff24150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.394+0000 7eff2959a700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7eff2406b580 tx=0x7eff14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:08.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.395+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14004480 con 0x7eff24153880 2023-06-09T11:44:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.395+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff140045e0 con 0x7eff24153880 2023-06-09T11:44:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.395+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff24159980 con 0x7eff24153880 2023-06-09T11:44:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.395+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff140038c0 con 0x7eff24153880 2023-06-09T11:44:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.395+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff241596e0 con 0x7eff24153880 2023-06-09T11:44:08.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.396+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff24159c60 con 0x7eff24153880 2023-06-09T11:44:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.397+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff14021070 con 0x7eff24153880 2023-06-09T11:44:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.398+0000 7eff1a7fc700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 0x7eff0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.398+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7eff1408b120 con 0x7eff24153880 2023-06-09T11:44:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.398+0000 7eff29d9b700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 0x7eff0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:08.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.399+0000 7eff29d9b700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 0x7eff0806d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7eff20006ba0 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:08.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.402+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7eff14055bd0 con 0x7eff24153880 2023-06-09T11:44:08.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.654+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff24074cf0 con 0x7eff24153880 2023-06-09T11:44:08.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.655+0000 7eff1a7fc700 1 -- 172.21.15.7:0/3194536518 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7eff140591f0 con 0x7eff24153880 2023-06-09T11:44:08.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 msgr2=0x7eff0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 0x7eff0806d190 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7eff20006ba0 tx=0x7eff20008040 comp rx=0 tx=0).stop 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 msgr2=0x7eff24151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7eff2406b580 tx=0x7eff14004750 comp rx=0 tx=0).stop 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 shutdown_connections 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7eff0806ace0 0x7eff0806d190 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7eff2415b2d0 0x7eff2414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7eff240c9f40 0x7eff24150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 --2- 172.21.15.7:0/3194536518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7eff24153880 0x7eff24151440 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 >> 172.21.15.7:0/3194536518 conn(0x7eff240737f0 msgr2=0x7eff24150790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.657+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 shutdown_connections 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:08.658+0000 7eff2bfff700 1 -- 172.21.15.7:0/3194536518 wait complete. 2023-06-09T11:44:08.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:08.679 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:10.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.887+0000 7fc28ffff700 1 Processor -- start 2023-06-09T11:44:10.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.888+0000 7fc28ffff700 1 -- start start 2023-06-09T11:44:10.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 0x7fc288158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2881580a0 con 0x7fc2880c5b40 2023-06-09T11:44:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28ffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2880c6180 con 0x7fc288151510 2023-06-09T11:44:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2881518e0 con 0x7fc288071230 2023-06-09T11:44:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28d59a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc2880c5b40 0x7fc288070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58230/0 (socket says 172.21.15.7:58230) 2023-06-09T11:44:10.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28d59a700 1 -- 172.21.15.7:0/885474139 learned_addr learned my addr 172.21.15.7:0/885474139 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:10.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.889+0000 7fc28e59c700 1 --2- 172.21.15.7:0/885474139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 0x7fc288158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567381797 0 0) 0x7fc2881580a0 con 0x7fc2880c5b40 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2740035b0 con 0x7fc2880c5b40 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134557128 0 0) 0x7fc2880c6180 con 0x7fc288151510 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2881580a0 con 0x7fc288151510 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28e59c700 1 -- 172.21.15.7:0/885474139 >> v1:172.21.15.117:6789/0 conn(0x7fc288151510 legacy=0x7fc28815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28e59c700 1 -- 172.21.15.7:0/885474139 >> v1:172.21.15.7:6789/0 conn(0x7fc2880c5b40 legacy=0x7fc288070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:10.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28e59c700 1 -- 172.21.15.7:0/885474139 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288157490 con 0x7fc288071230 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.890+0000 7fc28e59c700 1 --2- 172.21.15.7:0/885474139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 0x7fc288158870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc280009dc0 tx=0x7fc28000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90772d28bec5774 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28000e070 con 0x7fc288071230 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28000bb20 con 0x7fc288071230 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28cd99700 1 -- 172.21.15.7:0/885474139 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc280012430 con 0x7fc288071230 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 -- 172.21.15.7:0/885474139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 msgr2=0x7fc288158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 --2- 172.21.15.7:0/885474139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 0x7fc288158870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc280009dc0 tx=0x7fc28000b330 comp rx=0 tx=0).stop 2023-06-09T11:44:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 -- 172.21.15.7:0/885474139 shutdown_connections 2023-06-09T11:44:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 --2- 172.21.15.7:0/885474139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288071230 0x7fc288158870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 -- 172.21.15.7:0/885474139 >> 172.21.15.7:0/885474139 conn(0x7fc28806a4d0 msgr2=0x7fc28806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 -- 172.21.15.7:0/885474139 shutdown_connections 2023-06-09T11:44:10.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.891+0000 7fc28ffff700 1 -- 172.21.15.7:0/885474139 wait complete. 2023-06-09T11:44:10.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.893+0000 7fc28ffff700 1 Processor -- start 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.893+0000 7fc28ffff700 1 -- start start 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc288071230 0x7fc2881f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288151510 0x7fc28814aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880c7450 con 0x7fc2880c5b40 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880cb7a0 con 0x7fc288071230 2023-06-09T11:44:10.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2880cb1c0 con 0x7fc288151510 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35926/0 (socket says 172.21.15.7:35926) 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.894+0000 7fc28d59a700 1 -- 172.21.15.7:0/3799789049 learned_addr learned my addr 172.21.15.7:0/3799789049 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28e59c700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288151510 0x7fc28814aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28dd9b700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc288071230 0x7fc2881f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 -- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288151510 msgr2=0x7fc28814aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:10.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288151510 0x7fc28814aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 -- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc288071230 msgr2=0x7fc2881f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc288071230 0x7fc2881f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 -- 172.21.15.7:0/3799789049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280009a70 con 0x7fc2880c5b40 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28d59a700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c730 tx=0x7fc27800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278005d70 con 0x7fc2880c5b40 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780070d0 con 0x7fc2880c5b40 2023-06-09T11:44:10.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc288157a50 con 0x7fc2880c5b40 2023-06-09T11:44:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278007480 con 0x7fc2880c5b40 2023-06-09T11:44:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.895+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc288157770 con 0x7fc2880c5b40 2023-06-09T11:44:10.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.896+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc288157d30 con 0x7fc2880c5b40 2023-06-09T11:44:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.897+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2780075e0 con 0x7fc2880c5b40 2023-06-09T11:44:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.898+0000 7fc27e7fc700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.898+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fc278089130 con 0x7fc2880c5b40 2023-06-09T11:44:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.899+0000 7fc28dd9b700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:10.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.899+0000 7fc28dd9b700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 0x7fc26c06d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc284007fd0 tx=0x7fc284009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:10.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:10.903+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc278053b50 con 0x7fc2880c5b40 2023-06-09T11:44:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.153+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc288074cf0 con 0x7fc2880c5b40 2023-06-09T11:44:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.154+0000 7fc27e7fc700 1 -- 172.21.15.7:0/3799789049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fc278057170 con 0x7fc2880c5b40 2023-06-09T11:44:11.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:11.158 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.156+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 msgr2=0x7fc26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.156+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 0x7fc26c06d190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc284007fd0 tx=0x7fc284009040 comp rx=0 tx=0).stop 2023-06-09T11:44:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.156+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 msgr2=0x7fc28814eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.156+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c730 tx=0x7fc27800caf0 comp rx=0 tx=0).stop 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 shutdown_connections 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26c06ace0 0x7fc26c06d190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc288151510 0x7fc28814aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc288071230 0x7fc2881f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 --2- 172.21.15.7:0/3799789049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2880c5b40 0x7fc28814eb80 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 >> 172.21.15.7:0/3799789049 conn(0x7fc28806a4d0 msgr2=0x7fc28814d100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 shutdown_connections 2023-06-09T11:44:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:11.157+0000 7fc28ffff700 1 -- 172.21.15.7:0/3799789049 wait complete. 2023-06-09T11:44:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:11.178 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:13.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.385+0000 7f738ebff700 1 Processor -- start 2023-06-09T11:44:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.385+0000 7f738ebff700 1 -- start start 2023-06-09T11:44:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738ebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f738815cd40 0x7f73880cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738ebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7388159b20 con 0x7f73880cb9a0 2023-06-09T11:44:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738ebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880cac80 con 0x7f738815cd40 2023-06-09T11:44:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73881597d0 con 0x7f73880cb5d0 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738d19c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f738815cd40 0x7f73880cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738c99b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738d19c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f738815cd40 0x7f73880cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738d19c700 1 -- 172.21.15.7:0/1216349983 learned_addr learned my addr 172.21.15.7:0/1216349983 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f738c99b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:35402/0 (socket says 172.21.15.7:35402) 2023-06-09T11:44:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.387+0000 7f7387fff700 1 --1- 172.21.15.7:0/1216349983 >> v1:172.21.15.117:6789/0 conn(0x7f73880cb9a0 0x7f7388159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:58304/0 (socket says 172.21.15.7:58304) 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738d19c700 1 -- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 msgr2=0x7f73880b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738d19c700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738d19c700 1 -- 172.21.15.7:0/1216349983 >> v1:172.21.15.117:6789/0 conn(0x7f73880cb9a0 legacy=0x7f7388159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738d19c700 1 -- 172.21.15.7:0/1216349983 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388071ab0 con 0x7f738815cd40 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738c99b700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f73877fe700 1 -- 172.21.15.7:0/1216349983 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684503602 0 0) 0x7f7388159b20 con 0x7f73880cb9a0 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f738d19c700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73880cbea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7380009e50 tx=0x7f738000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f982b8e6846b3833 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f73877fe700 1 -- 172.21.15.7:0/1216349983 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000e070 con 0x7f738815cd40 2023-06-09T11:44:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f73877fe700 1 -- 172.21.15.7:0/1216349983 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738000b910 con 0x7f738815cd40 2023-06-09T11:44:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.388+0000 7f73877fe700 1 -- 172.21.15.7:0/1216349983 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000bcc0 con 0x7f738815cd40 2023-06-09T11:44:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 -- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 msgr2=0x7f73880cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73880cbea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7380009e50 tx=0x7f738000b330 comp rx=0 tx=0).stop 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 -- 172.21.15.7:0/1216349983 shutdown_connections 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73880cbea0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 --2- 172.21.15.7:0/1216349983 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb5d0 0x7f73880b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 -- 172.21.15.7:0/1216349983 >> 172.21.15.7:0/1216349983 conn(0x7f7388070af0 msgr2=0x7f7388070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 -- 172.21.15.7:0/1216349983 shutdown_connections 2023-06-09T11:44:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.389+0000 7f738ebff700 1 -- 172.21.15.7:0/1216349983 wait complete. 2023-06-09T11:44:13.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.391+0000 7f738ebff700 1 Processor -- start 2023-06-09T11:44:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.392+0000 7f738ebff700 1 -- start start 2023-06-09T11:44:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb9a0 0x7f7388150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73881ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881527b0 con 0x7f73880cb5d0 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73881521d0 con 0x7f738815cd40 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880cda10 con 0x7f73880cb9a0 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42902/0 (socket says 172.21.15.7:42902) 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738c99b700 1 -- 172.21.15.7:0/4260111494 learned_addr learned my addr 172.21.15.7:0/4260111494 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f738d19c700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73881ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.393+0000 7f7387fff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb9a0 0x7f7388150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 -- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb9a0 msgr2=0x7f7388150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb9a0 0x7f7388150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 -- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 msgr2=0x7f73881ee700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73881ee700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 -- 172.21.15.7:0/4260111494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c0094a0 con 0x7f73880cb5d0 2023-06-09T11:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738c99b700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f737c00ebe0 tx=0x7f737c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c009800 con 0x7f73880cb5d0 2023-06-09T11:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00c910 con 0x7f73880cb5d0 2023-06-09T11:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7380009b30 con 0x7f73880cb5d0 2023-06-09T11:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00cc80 con 0x7f73880cb5d0 2023-06-09T11:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.394+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738815d4f0 con 0x7f73880cb5d0 2023-06-09T11:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.395+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738815d210 con 0x7f73880cb5d0 2023-06-09T11:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.396+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f737c013470 con 0x7f73880cb5d0 2023-06-09T11:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.397+0000 7f73857fa700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 0x7f736c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.398+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f737c089bf0 con 0x7f73880cb5d0 2023-06-09T11:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.398+0000 7f7387fff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 0x7f736c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.398+0000 7f7387fff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 0x7f736c06d1c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f73881ec330 tx=0x7f7374007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.402+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f737c054610 con 0x7f73880cb5d0 2023-06-09T11:44:13.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.654+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f738815d7d0 con 0x7f73880cb5d0 2023-06-09T11:44:13.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.655+0000 7f73857fa700 1 -- 172.21.15.7:0/4260111494 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f737c057c30 con 0x7f73880cb5d0 2023-06-09T11:44:13.658 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:13.658 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:13.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 msgr2=0x7f736c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 0x7f736c06d1c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f73881ec330 tx=0x7f7374007be0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 msgr2=0x7f73880b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f737c00ebe0 tx=0x7f737c00eef0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 shutdown_connections 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f738815cd40 0x7f73881ee700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f73880cb5d0 0x7f73880b3550 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f736c06ad10 0x7f736c06d1c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 --2- 172.21.15.7:0/4260111494 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f73880cb9a0 0x7f7388150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 >> 172.21.15.7:0/4260111494 conn(0x7f7388070af0 msgr2=0x7f7388070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 shutdown_connections 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:13.657+0000 7f738ebff700 1 -- 172.21.15.7:0/4260111494 wait complete. 2023-06-09T11:44:13.662 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:13.678 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:15.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.886+0000 7f3ee5bff700 1 Processor -- start 2023-06-09T11:44:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.887+0000 7f3ee5bff700 1 -- start start 2023-06-09T11:44:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3ee5bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3ee015b060 0x7f3ee00c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3ee5bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee00c9bd0 con 0x7f3ee00c88e0 2023-06-09T11:44:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3ee5bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee0159570 con 0x7f3ee00c2ae0 2023-06-09T11:44:15.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3ee5bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee0159360 con 0x7f3ee015b060 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3edffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ee00c88e0 0x7f3ee0163ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51374/0 (socket says 172.21.15.7:51374) 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3edffff700 1 -- 172.21.15.7:0/827369348 learned_addr learned my addr 172.21.15.7:0/827369348 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.888+0000 7f3edeffd700 1 --2- 172.21.15.7:0/827369348 >> v2:172.21.15.117:3300/0 conn(0x7f3ee015b060 0x7f3ee00c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3ede7fc700 1 -- 172.21.15.7:0/827369348 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862199685 0 0) 0x7f3ee00c9bd0 con 0x7f3ee00c88e0 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3ede7fc700 1 -- 172.21.15.7:0/827369348 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec80035b0 con 0x7f3ee00c88e0 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3edeffd700 1 -- 172.21.15.7:0/827369348 >> v1:172.21.15.117:6789/0 conn(0x7f3ee00c2ae0 legacy=0x7f3ee015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3edeffd700 1 -- 172.21.15.7:0/827369348 >> v1:172.21.15.7:6789/0 conn(0x7f3ee00c88e0 legacy=0x7f3ee0163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3edeffd700 1 -- 172.21.15.7:0/827369348 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0159810 con 0x7f3ee015b060 2023-06-09T11:44:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3edeffd700 1 --2- 172.21.15.7:0/827369348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee015b060 0x7f3ee00c2eb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009840 tx=0x7f3ed4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfd7c6b17f25f8f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3ede7fc700 1 -- 172.21.15.7:0/827369348 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4011830 con 0x7f3ee015b060 2023-06-09T11:44:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3ede7fc700 1 -- 172.21.15.7:0/827369348 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed4011990 con 0x7f3ee015b060 2023-06-09T11:44:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.889+0000 7f3ede7fc700 1 -- 172.21.15.7:0/827369348 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4011d00 con 0x7f3ee015b060 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 -- 172.21.15.7:0/827369348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee015b060 msgr2=0x7f3ee00c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/827369348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee015b060 0x7f3ee00c2eb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009840 tx=0x7f3ed4009b50 comp rx=0 tx=0).stop 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 -- 172.21.15.7:0/827369348 shutdown_connections 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/827369348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee015b060 0x7f3ee00c2eb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 -- 172.21.15.7:0/827369348 >> 172.21.15.7:0/827369348 conn(0x7f3ee00cd330 msgr2=0x7f3ee00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 -- 172.21.15.7:0/827369348 shutdown_connections 2023-06-09T11:44:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.890+0000 7f3ee5bff700 1 -- 172.21.15.7:0/827369348 wait complete. 2023-06-09T11:44:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.892+0000 7f3ee5bff700 1 Processor -- start 2023-06-09T11:44:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.893+0000 7f3ee5bff700 1 -- start start 2023-06-09T11:44:15.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee00c2ae0 0x7f3ee014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee00c88e0 0x7f3ee0146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee015bca0 con 0x7f3ee015b060 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee00ccd90 con 0x7f3ee00c2ae0 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3ee5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee015bf10 con 0x7f3ee00c88e0 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42912/0 (socket says 172.21.15.7:42912) 2023-06-09T11:44:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edffff700 1 -- 172.21.15.7:0/2129891388 learned_addr learned my addr 172.21.15.7:0/2129891388 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edeffd700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee00c88e0 0x7f3ee0146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edf7fe700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee00c2ae0 0x7f3ee014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.894+0000 7f3edffff700 1 -- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee00c88e0 msgr2=0x7f3ee0146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3edffff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee00c88e0 0x7f3ee0146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3edffff700 1 -- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee00c2ae0 msgr2=0x7f3ee014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3edffff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee00c2ae0 0x7f3ee014e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3edffff700 1 -- 172.21.15.7:0/2129891388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed40094a0 con 0x7f3ee015b060 2023-06-09T11:44:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3edffff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800cb40 tx=0x7f3ed800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed800a9e0 con 0x7f3ee015b060 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed800ab40 con 0x7f3ee015b060 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8012400 con 0x7f3ee015b060 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee0159dd0 con 0x7f3ee015b060 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.895+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee0159af0 con 0x7f3ee015b060 2023-06-09T11:44:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.896+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee015a0b0 con 0x7f3ee015b060 2023-06-09T11:44:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.897+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ed801b3b0 con 0x7f3ee015b060 2023-06-09T11:44:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.898+0000 7f3ec7fff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 0x7f3ecc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.898+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7f3ed80896f0 con 0x7f3ee015b060 2023-06-09T11:44:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.898+0000 7f3edf7fe700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 0x7f3ecc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.899+0000 7f3edf7fe700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 0x7f3ecc06d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00018a0 tx=0x7f3ed0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:15.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:15.903+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3ed8054130 con 0x7f3ee015b060 2023-06-09T11:44:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.153+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ee0074cf0 con 0x7f3ee015b060 2023-06-09T11:44:16.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.154+0000 7f3ec7fff700 1 -- 172.21.15.7:0/2129891388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f3ed8057750 con 0x7f3ee015b060 2023-06-09T11:44:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:16.157 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 msgr2=0x7f3ecc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 0x7f3ecc06d190 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00018a0 tx=0x7f3ed0009040 comp rx=0 tx=0).stop 2023-06-09T11:44:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 msgr2=0x7f3ee01e49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800cb40 tx=0x7f3ed800ce50 comp rx=0 tx=0).stop 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 shutdown_connections 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ee015b060 0x7f3ee01e49a0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ee00c2ae0 0x7f3ee014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3ecc06ace0 0x7f3ecc06d190 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 --2- 172.21.15.7:0/2129891388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ee00c88e0 0x7f3ee0146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 >> 172.21.15.7:0/2129891388 conn(0x7f3ee00cd330 msgr2=0x7f3ee015b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 shutdown_connections 2023-06-09T11:44:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:16.156+0000 7f3ee5bff700 1 -- 172.21.15.7:0/2129891388 wait complete. 2023-06-09T11:44:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:16.177 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:18.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.387+0000 7fad9a1ff700 1 Processor -- start 2023-06-09T11:44:18.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.387+0000 7fad9a1ff700 1 -- start start 2023-06-09T11:44:18.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad9a1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad940cb9a0 0x7fad940b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad9a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 0x7fad940cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad9a1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad94159b20 con 0x7fad940cb5d0 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad9a1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad940cac80 con 0x7fad940cb9a0 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad9a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad941597d0 con 0x7fad9415cd40 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad92ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad940cb9a0 0x7fad940b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad92ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fad940cb9a0 0x7fad940b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42922/0 (socket says 172.21.15.7:42922) 2023-06-09T11:44:18.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad92ffd700 1 -- 172.21.15.7:0/2026608731 learned_addr learned my addr 172.21.15.7:0/2026608731 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:18.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.388+0000 7fad937fe700 1 --1- 172.21.15.7:0/2026608731 >> v1:172.21.15.7:6789/0 conn(0x7fad940cb5d0 0x7fad94159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51384/0 (socket says 172.21.15.7:51384) 2023-06-09T11:44:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad93fff700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 0x7fad940cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad92ffd700 1 -- 172.21.15.7:0/2026608731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 msgr2=0x7fad940cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad92ffd700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 0x7fad940cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad92ffd700 1 -- 172.21.15.7:0/2026608731 >> v1:172.21.15.7:6789/0 conn(0x7fad940cb5d0 legacy=0x7fad94159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad92ffd700 1 -- 172.21.15.7:0/2026608731 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94071ab0 con 0x7fad940cb9a0 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad93fff700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 0x7fad940cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad927fc700 1 -- 172.21.15.7:0/2026608731 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180042597 0 0) 0x7fad94159b20 con 0x7fad940cb5d0 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad92ffd700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb9a0 0x7fad940b52b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fad880098f0 tx=0x7fad88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32104538cc5b5c0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad927fc700 1 -- 172.21.15.7:0/2026608731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800b8a0 con 0x7fad940cb9a0 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad927fc700 1 -- 172.21.15.7:0/2026608731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800ba00 con 0x7fad940cb9a0 2023-06-09T11:44:18.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.389+0000 7fad927fc700 1 -- 172.21.15.7:0/2026608731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800bd90 con 0x7fad940cb9a0 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 -- 172.21.15.7:0/2026608731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb9a0 msgr2=0x7fad940b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb9a0 0x7fad940b52b0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fad880098f0 tx=0x7fad88009c00 comp rx=0 tx=0).stop 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 -- 172.21.15.7:0/2026608731 shutdown_connections 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad9415cd40 0x7fad940cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/2026608731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb9a0 0x7fad940b52b0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 -- 172.21.15.7:0/2026608731 >> 172.21.15.7:0/2026608731 conn(0x7fad94070af0 msgr2=0x7fad94070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 -- 172.21.15.7:0/2026608731 shutdown_connections 2023-06-09T11:44:18.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.390+0000 7fad9a1ff700 1 -- 172.21.15.7:0/2026608731 wait complete. 2023-06-09T11:44:18.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.392+0000 7fad9a1ff700 1 Processor -- start 2023-06-09T11:44:18.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.392+0000 7fad9a1ff700 1 -- start start 2023-06-09T11:44:18.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad940cb9a0 0x7fad941f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9415cd40 0x7fad941f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940d0450 con 0x7fad940cb5d0 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940cff80 con 0x7fad9415cd40 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad9a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad941f31f0 con 0x7fad940cb9a0 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad937fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42924/0 (socket says 172.21.15.7:42924) 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad937fe700 1 -- 172.21.15.7:0/3948930117 learned_addr learned my addr 172.21.15.7:0/3948930117 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:18.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad92ffd700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad940cb9a0 0x7fad941f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.393+0000 7fad93fff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9415cd40 0x7fad941f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 -- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad940cb9a0 msgr2=0x7fad941f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad940cb9a0 0x7fad941f22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 -- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9415cd40 msgr2=0x7fad941f2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9415cd40 0x7fad941f2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 -- 172.21.15.7:0/3948930117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad88009550 con 0x7fad940cb5d0 2023-06-09T11:44:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad937fe700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fad8400c750 tx=0x7fad8400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:18.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84004c80 con 0x7fad940cb5d0 2023-06-09T11:44:18.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84004e00 con 0x7fad940cb5d0 2023-06-09T11:44:18.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84007310 con 0x7fad940cb5d0 2023-06-09T11:44:18.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9415d4f0 con 0x7fad940cb5d0 2023-06-09T11:44:18.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.394+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9415d210 con 0x7fad940cb5d0 2023-06-09T11:44:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.395+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9415d7d0 con 0x7fad940cb5d0 2023-06-09T11:44:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.396+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad84007470 con 0x7fad940cb5d0 2023-06-09T11:44:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.397+0000 7fad7bfff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 0x7fad8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.397+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 7582+0+0 (secure 0 0 0) 0x7fad84089930 con 0x7fad940cb5d0 2023-06-09T11:44:18.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.397+0000 7fad92ffd700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 0x7fad8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:18.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.398+0000 7fad92ffd700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 0x7fad8006d190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fad940cbd70 tx=0x7fad88008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:18.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.402+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fad84054350 con 0x7fad940cb5d0 2023-06-09T11:44:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.656+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad9408d240 con 0x7fad940cb5d0 2023-06-09T11:44:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.657+0000 7fad7bfff700 1 -- 172.21.15.7:0/3948930117 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 151 v151) v1 ==== 95+0+6225 (secure 0 0 0) 0x7fad84057970 con 0x7fad940cb5d0 2023-06-09T11:44:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:18.661 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":151,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7437,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":147}],"filesystems":[{"mdsmap":{"epoch":151,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:43:41.734861+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":66,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6583,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6583":{"gid":6583,"name":"f","rank":0,"incarnation":125,"state":"up:active","state_seq":33,"addr":"172.21.15.117:6837/1520141131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":1520141131},{"type":"v1","addr":"172.21.15.117:6837","nonce":1520141131}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.659+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 msgr2=0x7fad8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.659+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 0x7fad8006d190 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fad940cbd70 tx=0x7fad88008030 comp rx=0 tx=0).stop 2023-06-09T11:44:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.659+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 msgr2=0x7fad941e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.659+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fad8400c750 tx=0x7fad8400cb10 comp rx=0 tx=0).stop 2023-06-09T11:44:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 shutdown_connections 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fad9415cd40 0x7fad941f2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad940cb5d0 0x7fad941e5f20 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fad8006ace0 0x7fad8006d190 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 --2- 172.21.15.7:0/3948930117 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fad940cb9a0 0x7fad941f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 >> 172.21.15.7:0/3948930117 conn(0x7fad94070af0 msgr2=0x7fad940cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 shutdown_connections 2023-06-09T11:44:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:18.660+0000 7fad9a1ff700 1 -- 172.21.15.7:0/3948930117 wait complete. 2023-06-09T11:44:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 151 2023-06-09T11:44:18.681 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:44:20.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:20.280 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:20.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.894+0000 7f2d989ff700 1 Processor -- start 2023-06-09T11:44:20.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.894+0000 7f2d989ff700 1 -- start start 2023-06-09T11:44:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d989ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 0x7f2d9415d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d989ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2d940c9260 0x7f2d940cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d989ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9415dc90 con 0x7f2d940b5370 2023-06-09T11:44:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d989ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d941597d0 con 0x7f2d940c9260 2023-06-09T11:44:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d989ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940cfaf0 con 0x7f2d940c8e90 2023-06-09T11:44:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d92d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d940b5370 0x7f2d940c9fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:51396/0 (socket says 172.21.15.7:51396) 2023-06-09T11:44:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d92d9d700 1 -- 172.21.15.7:0/2206991954 learned_addr learned my addr 172.21.15.7:0/2206991954 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:20.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d91d9b700 1 --2- 172.21.15.7:0/2206991954 >> v2:172.21.15.117:3300/0 conn(0x7f2d940c9260 0x7f2d940cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.895+0000 7f2d9259c700 1 --2- 172.21.15.7:0/2206991954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 0x7f2d9415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9159a700 1 -- 172.21.15.7:0/2206991954 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071712307 0 0) 0x7f2d8c001490 con 0x7f2d940b5370 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9159a700 1 -- 172.21.15.7:0/2206991954 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d7c0035b0 con 0x7f2d940b5370 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9259c700 1 -- 172.21.15.7:0/2206991954 >> v2:172.21.15.117:3300/0 conn(0x7f2d940c9260 msgr2=0x7f2d940cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9259c700 1 --2- 172.21.15.7:0/2206991954 >> v2:172.21.15.117:3300/0 conn(0x7f2d940c9260 0x7f2d940cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9259c700 1 -- 172.21.15.7:0/2206991954 >> v1:172.21.15.7:6789/0 conn(0x7f2d940b5370 legacy=0x7f2d940c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9259c700 1 -- 172.21.15.7:0/2206991954 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d940cfdd0 con 0x7f2d940c8e90 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9259c700 1 --2- 172.21.15.7:0/2206991954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 0x7f2d9415d750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009870 tx=0x7f2d84009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fed9cdfb20bd54 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.896+0000 7f2d9159a700 1 -- 172.21.15.7:0/2206991954 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011830 con 0x7f2d940c8e90 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d9159a700 1 -- 172.21.15.7:0/2206991954 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84011990 con 0x7f2d940c8e90 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d9159a700 1 -- 172.21.15.7:0/2206991954 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011d20 con 0x7f2d940c8e90 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 -- 172.21.15.7:0/2206991954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 msgr2=0x7f2d9415d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 --2- 172.21.15.7:0/2206991954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 0x7f2d9415d750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009870 tx=0x7f2d84009b80 comp rx=0 tx=0).stop 2023-06-09T11:44:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 -- 172.21.15.7:0/2206991954 shutdown_connections 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 --2- 172.21.15.7:0/2206991954 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c8e90 0x7f2d9415d750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 --2- 172.21.15.7:0/2206991954 >> v2:172.21.15.117:3300/0 conn(0x7f2d940c9260 0x7f2d940cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 -- 172.21.15.7:0/2206991954 >> 172.21.15.7:0/2206991954 conn(0x7f2d94070af0 msgr2=0x7f2d94070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 -- 172.21.15.7:0/2206991954 shutdown_connections 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.897+0000 7f2d989ff700 1 -- 172.21.15.7:0/2206991954 wait complete. 2023-06-09T11:44:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.899+0000 7f2d989ff700 1 Processor -- start 2023-06-09T11:44:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.900+0000 7f2d989ff700 1 -- start start 2023-06-09T11:44:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d940c8e90 0x7f2d940c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 0x7f2d941f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c4b40 con 0x7f2d940b5370 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c3cb0 con 0x7f2d940c8e90 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d989ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d940c3f50 con 0x7f2d940c9260 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d9259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42938/0 (socket says 172.21.15.7:42938) 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d9259c700 1 -- 172.21.15.7:0/1856980505 learned_addr learned my addr 172.21.15.7:0/1856980505 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d92d9d700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 0x7f2d941f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.901+0000 7f2d91d9b700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d940c8e90 0x7f2d940c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 -- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 msgr2=0x7f2d941f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 0x7f2d941f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 -- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d940c8e90 msgr2=0x7f2d940c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d940c8e90 0x7f2d940c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 -- 172.21.15.7:0/1856980505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d840094a0 con 0x7f2d940b5370 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d92d9d700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 0x7f2d941f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:20.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d9259c700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840036a0 con 0x7f2d940b5370 2023-06-09T11:44:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8401f070 con 0x7f2d940b5370 2023-06-09T11:44:20.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d94159ca0 con 0x7f2d940b5370 2023-06-09T11:44:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84003a40 con 0x7f2d940b5370 2023-06-09T11:44:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.902+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d941599c0 con 0x7f2d940b5370 2023-06-09T11:44:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.903+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d94159f80 con 0x7f2d940b5370 2023-06-09T11:44:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.904+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2d84003ba0 con 0x7f2d940b5370 2023-06-09T11:44:20.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.905+0000 7f2d82ffd700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 0x7f2d7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.905+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f2d84021070 con 0x7f2d940b5370 2023-06-09T11:44:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.905+0000 7f2d91d9b700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 0x7f2d7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.906+0000 7f2d91d9b700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 0x7f2d7406d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d940cfa20 tx=0x7f2d88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:20.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:20.910+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2d84055e40 con 0x7f2d940b5370 2023-06-09T11:44:21.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.161+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d94074cf0 con 0x7f2d940b5370 2023-06-09T11:44:21.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.161+0000 7f2d82ffd700 1 -- 172.21.15.7:0/1856980505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5460 (secure 0 0 0) 0x7f2d84015020 con 0x7f2d940b5370 2023-06-09T11:44:21.165 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:21.165 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:19.574095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:21.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 msgr2=0x7f2d7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 0x7f2d7406d190 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d940cfa20 tx=0x7f2d88008040 comp rx=0 tx=0).stop 2023-06-09T11:44:21.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 msgr2=0x7f2d9415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2d84000c00 tx=0x7f2d84004750 comp rx=0 tx=0).stop 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 shutdown_connections 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2d7406ace0 0x7f2d7406d190 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2d940c9260 0x7f2d941f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2d940b5370 0x7f2d9415be00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 --2- 172.21.15.7:0/1856980505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2d940c8e90 0x7f2d940c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 >> 172.21.15.7:0/1856980505 conn(0x7f2d94070af0 msgr2=0x7f2d94157c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:21.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 shutdown_connections 2023-06-09T11:44:21.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.164+0000 7f2d989ff700 1 -- 172.21.15.7:0/1856980505 wait complete. 2023-06-09T11:44:21.169 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 152 2023-06-09T11:44:21.185 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:44:21.186 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:44:21.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.382+0000 7f9196fff700 1 Processor -- start 2023-06-09T11:44:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.382+0000 7f9196fff700 1 -- start start 2023-06-09T11:44:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f9196fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f91901535e0 0x7f9190159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f9196fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91901539b0 0x7f91900c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f9196fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919015a420 con 0x7f91900c4ce0 2023-06-09T11:44:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f9196fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91900c5a00 con 0x7f91901539b0 2023-06-09T11:44:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f9196fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919006b460 con 0x7f91901535e0 2023-06-09T11:44:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f918ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91901539b0 0x7f91900c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f918ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f91901539b0 0x7f91900c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42948/0 (socket says 172.21.15.7:42948) 2023-06-09T11:44:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.383+0000 7f918ffff700 1 -- 172.21.15.7:0/360858620 learned_addr learned my addr 172.21.15.7:0/360858620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f9194d9b700 1 --2- 172.21.15.7:0/360858620 >> v2:172.21.15.117:3300/0 conn(0x7f91901535e0 0x7f9190159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918ffff700 1 -- 172.21.15.7:0/360858620 >> v2:172.21.15.117:3300/0 conn(0x7f91901535e0 msgr2=0x7f9190159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918ffff700 1 --2- 172.21.15.7:0/360858620 >> v2:172.21.15.117:3300/0 conn(0x7f91901535e0 0x7f9190159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918ffff700 1 -- 172.21.15.7:0/360858620 >> v1:172.21.15.117:6789/0 conn(0x7f91900c4ce0 legacy=0x7f919015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918ffff700 1 -- 172.21.15.7:0/360858620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919006a570 con 0x7f91901539b0 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918ffff700 1 --2- 172.21.15.7:0/360858620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901539b0 0x7f91900c9f60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f9190073d90 tx=0x7f917c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad99fe0b7b861116 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918f7fe700 1 -- 172.21.15.7:0/360858620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011830 con 0x7f91901539b0 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918f7fe700 1 -- 172.21.15.7:0/360858620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c011990 con 0x7f91901539b0 2023-06-09T11:44:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.384+0000 7f918f7fe700 1 -- 172.21.15.7:0/360858620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c011d20 con 0x7f91901539b0 2023-06-09T11:44:21.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 -- 172.21.15.7:0/360858620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901539b0 msgr2=0x7f91900c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 --2- 172.21.15.7:0/360858620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901539b0 0x7f91900c9f60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f9190073d90 tx=0x7f917c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 -- 172.21.15.7:0/360858620 shutdown_connections 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 --2- 172.21.15.7:0/360858620 >> v2:172.21.15.117:3300/0 conn(0x7f91901535e0 0x7f9190159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 --2- 172.21.15.7:0/360858620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901539b0 0x7f91900c9f60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 -- 172.21.15.7:0/360858620 >> 172.21.15.7:0/360858620 conn(0x7f91900737f0 msgr2=0x7f9190072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 -- 172.21.15.7:0/360858620 shutdown_connections 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.385+0000 7f9196fff700 1 -- 172.21.15.7:0/360858620 wait complete. 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.387+0000 7f9196fff700 1 Processor -- start 2023-06-09T11:44:21.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.387+0000 7f9196fff700 1 -- start start 2023-06-09T11:44:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 0x7f91901584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 0x7f919014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901ef440 con 0x7f91901535e0 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91901e4d30 con 0x7f91900c4ce0 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9196fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91900b2e70 con 0x7f91901539b0 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42962/0 (socket says 172.21.15.7:42962) 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 -- 172.21.15.7:0/2691828988 learned_addr learned my addr 172.21.15.7:0/2691828988 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f9194d9b700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 0x7f91901584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f919559c700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 0x7f919014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 -- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 msgr2=0x7f919014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 0x7f919014ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 -- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 msgr2=0x7f91901584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.388+0000 7f918ffff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 0x7f91901584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f918ffff700 1 -- 172.21.15.7:0/2691828988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009550 con 0x7f91901535e0 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f9194d9b700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 0x7f91901584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f919559c700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 0x7f919014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:21.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f918ffff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f917c014040 tx=0x7f917c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:21.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c0036a0 con 0x7f91901535e0 2023-06-09T11:44:21.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c01e070 con 0x7f91901535e0 2023-06-09T11:44:21.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919006ab30 con 0x7f91901535e0 2023-06-09T11:44:21.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c003a60 con 0x7f91901535e0 2023-06-09T11:44:21.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.389+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919006a850 con 0x7f91901535e0 2023-06-09T11:44:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.390+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919006ae10 con 0x7f91901535e0 2023-06-09T11:44:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.391+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f917c003bc0 con 0x7f91901535e0 2023-06-09T11:44:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.393+0000 7f918d7fa700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 0x7f917406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.393+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f917c08aa60 con 0x7f91901535e0 2023-06-09T11:44:21.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.393+0000 7f9194d9b700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 0x7f917406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:21.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.394+0000 7f9194d9b700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 0x7f917406d1c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9184009b60 tx=0x7f9184008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:21.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.397+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f917c0554b0 con 0x7f91901535e0 2023-06-09T11:44:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.649+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91901ef580 con 0x7f91901535e0 2023-06-09T11:44:21.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.650+0000 7f918d7fa700 1 -- 172.21.15.7:0/2691828988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5460 (secure 0 0 0) 0x7f917c020090 con 0x7f91901535e0 2023-06-09T11:44:21.653 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:21.653 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:19.574095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.652+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 msgr2=0x7f917406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.652+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 0x7f917406d1c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9184009b60 tx=0x7f9184008040 comp rx=0 tx=0).stop 2023-06-09T11:44:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.652+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 msgr2=0x7f91901ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:21.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.652+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f917c014040 tx=0x7f917c004750 comp rx=0 tx=0).stop 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 shutdown_connections 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f917406ad10 0x7f917406d1c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91901539b0 0x7f919014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91900c4ce0 0x7f91901584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 --2- 172.21.15.7:0/2691828988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901535e0 0x7f91901ef860 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 >> 172.21.15.7:0/2691828988 conn(0x7f91900737f0 msgr2=0x7f919015aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:21.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 shutdown_connections 2023-06-09T11:44:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:21.653+0000 7f9196fff700 1 -- 172.21.15.7:0/2691828988 wait complete. 2023-06-09T11:44:21.658 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 152 2023-06-09T11:44:21.674 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 152, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 152, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 30 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:44:19.574095+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:44:23.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.882+0000 7ffaebbff700 1 Processor -- start 2023-06-09T11:44:23.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.882+0000 7ffaebbff700 1 -- start start 2023-06-09T11:44:23.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffaebbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffae415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffaebbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae415a500 con 0x7ffae4153880 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffaebbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae415da00 con 0x7ffae40c9f40 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffaebbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40c5f10 con 0x7ffae415b4a0 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffae999b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ffae4153880 0x7ffae415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53616/0 (socket says 172.21.15.7:53616) 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffae999b700 1 -- 172.21.15.7:0/1374881185 learned_addr learned my addr 172.21.15.7:0/1374881185 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:23.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffae919a700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffae415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299231516 0 0) 0x7ffae0001110 con 0x7ffae4153880 2023-06-09T11:44:23.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.884+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffad00035b0 con 0x7ffae4153880 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807433404 0 0) 0x7ffad00035b0 con 0x7ffae4153880 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 msgr2=0x7ffae415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffae415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 >> v1:172.21.15.117:6789/0 conn(0x7ffae40c9f40 legacy=0x7ffae40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae41596e0 con 0x7ffae4153880 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7ffae0001800 con 0x7ffae4153880 2023-06-09T11:44:23.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 >> v1:172.21.15.7:6789/0 conn(0x7ffae4153880 legacy=0x7ffae415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad0003a30 0x7ffad0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffad0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0004d90 0x7ffad0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40c5f10 con 0x7ffad0003a30 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad000bac0 con 0x7ffad0004d90 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad000bc50 con 0x7ffae415b4a0 2023-06-09T11:44:23.892 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae999b700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad0003a30 0x7ffad0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffae919a700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffad0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.885+0000 7ffaea19c700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0004d90 0x7ffad0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 -- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 msgr2=0x7ffad0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae415b4a0 0x7ffad0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 -- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0004d90 msgr2=0x7ffad0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0004d90 0x7ffad0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 -- 172.21.15.7:0/1374881185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00035b0 con 0x7ffad0003a30 2023-06-09T11:44:23.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffaea19c700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0004d90 0x7ffad0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae999b700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad0003a30 0x7ffad0003e00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c240 tx=0x7ffad0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61802be745859207 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000cde0 con 0x7ffad0003a30 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae0005960 con 0x7ffad0003a30 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.886+0000 7ffae8999700 1 -- 172.21.15.7:0/1374881185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0005d10 con 0x7ffad0003a30 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 -- 172.21.15.7:0/1374881185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad0003a30 msgr2=0x7ffad0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 --2- 172.21.15.7:0/1374881185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffad0003a30 0x7ffad0003e00 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ffae000c240 tx=0x7ffad0005880 comp rx=0 tx=0).stop 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaea19c700 1 -- 172.21.15.7:0/1374881185 reap_dead start 2023-06-09T11:44:23.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 -- 172.21.15.7:0/1374881185 shutdown_connections 2023-06-09T11:44:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 -- 172.21.15.7:0/1374881185 >> 172.21.15.7:0/1374881185 conn(0x7ffae40737f0 msgr2=0x7ffae40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 -- 172.21.15.7:0/1374881185 shutdown_connections 2023-06-09T11:44:23.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.887+0000 7ffaebbff700 1 -- 172.21.15.7:0/1374881185 wait complete. 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.888+0000 7ffaebbff700 1 Processor -- start 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.889+0000 7ffaebbff700 1 -- start start 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 0x7ffae40cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 0x7ffae40c4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41502d0 con 0x7ffae41efde0 2023-06-09T11:44:23.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41ef0f0 con 0x7ffad0003a30 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaebbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40cf310 con 0x7ffae40ce670 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53910/0 (socket says 172.21.15.7:53910) 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 -- 172.21.15.7:0/3231958859 learned_addr learned my addr 172.21.15.7:0/3231958859 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae999b700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 0x7ffae40cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaea19c700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 0x7ffae40c4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 -- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 msgr2=0x7ffae40c4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:23.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 0x7ffae40c4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 -- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 msgr2=0x7ffae40cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 0x7ffae40cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae919a700 1 -- 172.21.15.7:0/3231958859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae4159c60 con 0x7ffae41efde0 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffaea19c700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 0x7ffae40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.890+0000 7ffae999b700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 0x7ffae40cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffae919a700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ffad4011aa0 tx=0x7ffad4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:23.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400f920 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad400fa80 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad400fe30 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae4159980 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.891+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae4159f40 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.892+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae41efc30 con 0x7ffae41efde0 2023-06-09T11:44:23.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.893+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffad4014440 con 0x7ffae41efde0 2023-06-09T11:44:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.894+0000 7ffada7fc700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 0x7ffac806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.894+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7ffad408a270 con 0x7ffae41efde0 2023-06-09T11:44:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.894+0000 7ffae999b700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 0x7ffac806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:23.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.895+0000 7ffae999b700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 0x7ffac806d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001ac0 tx=0x7ffae0007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:23.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:23.898+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ffad4054da0 con 0x7ffae41efde0 2023-06-09T11:44:24.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.152+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffae4074cf0 con 0x7ffae41efde0 2023-06-09T11:44:24.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.152+0000 7ffada7fc700 1 -- 172.21.15.7:0/3231958859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5460 (secure 0 0 0) 0x7ffad40583c0 con 0x7ffae41efde0 2023-06-09T11:44:24.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:24.156 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:19.574095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 msgr2=0x7ffac806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:24.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 0x7ffac806d190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ffae0001ac0 tx=0x7ffae0007000 comp rx=0 tx=0).stop 2023-06-09T11:44:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 msgr2=0x7ffae414cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7ffad4011aa0 tx=0x7ffad4011e60 comp rx=0 tx=0).stop 2023-06-09T11:44:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 shutdown_connections 2023-06-09T11:44:24.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ffac806ace0 0x7ffac806d190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffae40ce670 0x7ffae40c4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffad0003a30 0x7ffae40cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:24.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 --2- 172.21.15.7:0/3231958859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffae41efde0 0x7ffae414cb10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 >> 172.21.15.7:0/3231958859 conn(0x7ffae40737f0 msgr2=0x7ffae415b870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 shutdown_connections 2023-06-09T11:44:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:24.155+0000 7ffaebbff700 1 -- 172.21.15.7:0/3231958859 wait complete. 2023-06-09T11:44:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 152 2023-06-09T11:44:24.176 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:26.177 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:26.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.386+0000 7fa7d95ff700 1 Processor -- start 2023-06-09T11:44:26.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.387+0000 7fa7d95ff700 1 -- start start 2023-06-09T11:44:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d95ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7d4151510 0x7fa7d40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d95ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d4158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d95ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d41580a0 con 0x7fa7d415b380 2023-06-09T11:44:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d95ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d41518e0 con 0x7fa7d4151510 2023-06-09T11:44:26.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d95ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c6180 con 0x7fa7d4071230 2023-06-09T11:44:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d2ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7d4151510 0x7fa7d40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d4158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.388+0000 7fa7d2ffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa7d4151510 0x7fa7d40c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:49208/0 (socket says 172.21.15.7:49208) 2023-06-09T11:44:26.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/1747352904 learned_addr learned my addr 172.21.15.7:0/1747352904 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 msgr2=0x7fa7d4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d4158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/1747352904 >> v1:172.21.15.117:6789/0 conn(0x7fa7d415b380 legacy=0x7fa7d4070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/1747352904 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d4157490 con 0x7fa7d4151510 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d37fe700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.389+0000 7fa7d2ffd700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d40c5b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40098f0 tx=0x7fa7c4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=808616a4c45b086d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d1ffb700 1 -- 172.21.15.7:0/1747352904 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4011830 con 0x7fa7d4151510 2023-06-09T11:44:26.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d1ffb700 1 -- 172.21.15.7:0/1747352904 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c4011990 con 0x7fa7d4151510 2023-06-09T11:44:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d1ffb700 1 -- 172.21.15.7:0/1747352904 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4011d20 con 0x7fa7d4151510 2023-06-09T11:44:26.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 -- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 msgr2=0x7fa7d40c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d40c5b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40098f0 tx=0x7fa7c4009c00 comp rx=0 tx=0).stop 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 -- 172.21.15.7:0/1747352904 shutdown_connections 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/1747352904 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d40c5b40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 -- 172.21.15.7:0/1747352904 >> 172.21.15.7:0/1747352904 conn(0x7fa7d406a4d0 msgr2=0x7fa7d406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 -- 172.21.15.7:0/1747352904 shutdown_connections 2023-06-09T11:44:26.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.390+0000 7fa7d95ff700 1 -- 172.21.15.7:0/1747352904 wait complete. 2023-06-09T11:44:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.392+0000 7fa7d95ff700 1 Processor -- start 2023-06-09T11:44:26.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.392+0000 7fa7d95ff700 1 -- start start 2023-06-09T11:44:26.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d41596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d415b380 0x7fa7d41e1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d414e8f0 con 0x7fa7d415b380 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d41e0fb0 con 0x7fa7d4151510 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.393+0000 7fa7d95ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d40c0c10 con 0x7fa7d4071230 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d415b380 0x7fa7d41e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:33228/0 (socket says 172.21.15.7:33228) 2023-06-09T11:44:26.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/2560826528 learned_addr learned my addr 172.21.15.7:0/2560826528 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d27fc700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d41596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 msgr2=0x7fa7d41596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d41596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d415b380 msgr2=0x7fa7d41e1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d415b380 0x7fa7d41e1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 -- 172.21.15.7:0/2560826528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc009a70 con 0x7fa7d4071230 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.394+0000 7fa7d2ffd700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40cc2c0 tx=0x7fa7c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:26.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.395+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4004400 con 0x7fa7d4071230 2023-06-09T11:44:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.395+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c401f070 con 0x7fa7d4071230 2023-06-09T11:44:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.395+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c4009550 con 0x7fa7d4071230 2023-06-09T11:44:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.395+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40038e0 con 0x7fa7d4071230 2023-06-09T11:44:26.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.395+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d4157a50 con 0x7fa7d4071230 2023-06-09T11:44:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.396+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d4157770 con 0x7fa7d4071230 2023-06-09T11:44:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.397+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7c4021030 con 0x7fa7d4071230 2023-06-09T11:44:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.398+0000 7fa7bb7fe700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 0x7fa7c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.398+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fa7c408aec0 con 0x7fa7d4071230 2023-06-09T11:44:26.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.398+0000 7fa7d27fc700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 0x7fa7c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:26.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.399+0000 7fa7d27fc700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 0x7fa7c006d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40c8a50 tx=0x7fa7c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:26.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.404+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa7c4055ad0 con 0x7fa7d4071230 2023-06-09T11:44:26.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.656+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d4157d30 con 0x7fa7d4071230 2023-06-09T11:44:26.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.657+0000 7fa7bb7fe700 1 -- 172.21.15.7:0/2560826528 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5460 (secure 0 0 0) 0x7fa7c4015090 con 0x7fa7d4071230 2023-06-09T11:44:26.660 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:26.661 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:19.574095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:26.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 msgr2=0x7fa7c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 0x7fa7c006d190 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40c8a50 tx=0x7fa7c8009040 comp rx=0 tx=0).stop 2023-06-09T11:44:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 msgr2=0x7fa7d41e0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40cc2c0 tx=0x7fa7c4004750 comp rx=0 tx=0).stop 2023-06-09T11:44:26.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 shutdown_connections 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7d415b380 0x7fa7d41e1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7c006ace0 0x7fa7c006d190 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7d4071230 0x7fa7d41e0a70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 --2- 172.21.15.7:0/2560826528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7d4151510 0x7fa7d41596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.659+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 >> 172.21.15.7:0/2560826528 conn(0x7fa7d406a4d0 msgr2=0x7fa7d4156330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:26.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.660+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 shutdown_connections 2023-06-09T11:44:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:26.660+0000 7fa7d95ff700 1 -- 172.21.15.7:0/2560826528 wait complete. 2023-06-09T11:44:26.665 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 152 2023-06-09T11:44:26.681 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:28.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.891+0000 7f6caa9ff700 1 Processor -- start 2023-06-09T11:44:28.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.892+0000 7f6caa9ff700 1 -- start start 2023-06-09T11:44:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6caa9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6ca40c6280 0x7f6ca40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6caa9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ca406e460 0x7f6ca406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6caa9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca415cac0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6caa9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca40c8870 con 0x7f6ca406e460 2023-06-09T11:44:28.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6caa9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca40c37d0 con 0x7f6ca40c6280 2023-06-09T11:44:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6ca8f9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ca406e460 0x7f6ca406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6ca37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6ca406a4d0 0x7f6ca40737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53628/0 (socket says 172.21.15.7:53628) 2023-06-09T11:44:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.894+0000 7f6ca37fe700 1 -- 172.21.15.7:0/3444478668 learned_addr learned my addr 172.21.15.7:0/3444478668 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3444478668 >> v2:172.21.15.117:3300/0 conn(0x7f6ca40c6280 0x7f6ca40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca8f9c700 1 -- 172.21.15.7:0/3444478668 >> v2:172.21.15.117:3300/0 conn(0x7f6ca40c6280 msgr2=0x7f6ca40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca8f9c700 1 --2- 172.21.15.7:0/3444478668 >> v2:172.21.15.117:3300/0 conn(0x7f6ca40c6280 0x7f6ca40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca8f9c700 1 -- 172.21.15.7:0/3444478668 >> v1:172.21.15.7:6789/0 conn(0x7f6ca406a4d0 legacy=0x7f6ca40737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca8f9c700 1 -- 172.21.15.7:0/3444478668 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca415cd30 con 0x7f6ca406e460 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/3444478668 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848091665 0 0) 0x7f6ca415cac0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.895+0000 7f6ca8f9c700 1 --2- 172.21.15.7:0/3444478668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406e460 0x7f6ca406cf30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40743a0 tx=0x7f6c9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab311d819e4e88b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/3444478668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c00e070 con 0x7f6ca406e460 2023-06-09T11:44:28.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/3444478668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c00bb20 con 0x7f6ca406e460 2023-06-09T11:44:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6ca2ffd700 1 -- 172.21.15.7:0/3444478668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c012430 con 0x7f6ca406e460 2023-06-09T11:44:28.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3444478668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406e460 msgr2=0x7f6ca406cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3444478668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406e460 0x7f6ca406cf30 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40743a0 tx=0x7f6c9c00b330 comp rx=0 tx=0).stop 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3444478668 shutdown_connections 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3444478668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406e460 0x7f6ca406cf30 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3444478668 >> v2:172.21.15.117:3300/0 conn(0x7f6ca40c6280 0x7f6ca40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3444478668 >> 172.21.15.7:0/3444478668 conn(0x7f6ca406f980 msgr2=0x7f6ca40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3444478668 shutdown_connections 2023-06-09T11:44:28.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.896+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3444478668 wait complete. 2023-06-09T11:44:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.898+0000 7f6caa9ff700 1 Processor -- start 2023-06-09T11:44:28.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.898+0000 7f6caa9ff700 1 -- start start 2023-06-09T11:44:28.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 0x7f6ca41ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ca40c6280 0x7f6ca41411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41ef2c0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41eece0 con 0x7f6ca406e460 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6caa9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca4156050 con 0x7f6ca40c6280 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6ca3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.899+0000 7f6ca3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53930/0 (socket says 172.21.15.7:53930) 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3614679513 learned_addr learned my addr 172.21.15.7:0/3614679513 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:28.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 0x7f6ca41ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca8f9c700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ca40c6280 0x7f6ca41411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ca40c6280 msgr2=0x7f6ca41411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ca40c6280 0x7f6ca41411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 msgr2=0x7f6ca41ef750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 0x7f6ca41ef750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 -- 172.21.15.7:0/3614679513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c009a70 con 0x7f6ca406a4d0 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 0x7f6ca41ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:28.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.900+0000 7f6ca3fff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e870 tx=0x7f6c9400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:28.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.901+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94009590 con 0x7f6ca406a4d0 2023-06-09T11:44:28.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.901+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c94013680 con 0x7f6ca406a4d0 2023-06-09T11:44:28.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.901+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca415d2f0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.901+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c940139f0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.901+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca415d010 con 0x7f6ca406a4d0 2023-06-09T11:44:28.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.902+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca415d5d0 con 0x7f6ca406a4d0 2023-06-09T11:44:28.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.904+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c94013b50 con 0x7f6ca406a4d0 2023-06-09T11:44:28.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.905+0000 7f6ca0ff9700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.906+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f6c94089e90 con 0x7f6ca406a4d0 2023-06-09T11:44:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.906+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.906+0000 7f6ca37fe700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 0x7f6c9006d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40c5210 tx=0x7f6c98007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:28.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:28.910+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c940547c0 con 0x7f6ca406a4d0 2023-06-09T11:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.163+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ca4074cf0 con 0x7f6ca406a4d0 2023-06-09T11:44:29.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.164+0000 7f6ca0ff9700 1 -- 172.21.15.7:0/3614679513 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 152 v152) v1 ==== 95+0+5460 (secure 0 0 0) 0x7f6c94017070 con 0x7f6ca406a4d0 2023-06-09T11:44:29.167 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:29.168 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":152,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":152,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:19.574095+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:29.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 msgr2=0x7f6c9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 0x7f6c9006d190 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40c5210 tx=0x7f6c98007ca0 comp rx=0 tx=0).stop 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 msgr2=0x7f6ca41570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400e870 tx=0x7f6c9400eb80 comp rx=0 tx=0).stop 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 shutdown_connections 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c9006ace0 0x7f6c9006d190 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ca40c6280 0x7f6ca41411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:29.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ca406a4d0 0x7f6ca41570c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 --2- 172.21.15.7:0/3614679513 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ca406e460 0x7f6ca41ef750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 >> 172.21.15.7:0/3614679513 conn(0x7f6ca406f980 msgr2=0x7f6ca40cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 shutdown_connections 2023-06-09T11:44:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:29.166+0000 7f6caa9ff700 1 -- 172.21.15.7:0/3614679513 wait complete. 2023-06-09T11:44:29.171 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 152 2023-06-09T11:44:29.188 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:31.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.395+0000 7f7902bff700 1 Processor -- start 2023-06-09T11:44:31.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.395+0000 7f7902bff700 1 -- start start 2023-06-09T11:44:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.396+0000 7f7902bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78fc1593d0 0x7f78fc1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.396+0000 7f7902bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 0x7f78fc159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.396+0000 7f7902bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc0cc140 con 0x7f78fc0c08d0 2023-06-09T11:44:31.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.396+0000 7f7902bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc0714f0 con 0x7f78fc1593d0 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.396+0000 7f7902bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c1830 con 0x7f78fc072020 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78fc1593d0 0x7f78fc1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f78fbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f78fc0c08d0 0x7f78fc071910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53642/0 (socket says 172.21.15.7:53642) 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f78fbfff700 1 -- 172.21.15.7:0/956237650 learned_addr learned my addr 172.21.15.7:0/956237650 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f78fc1593d0 0x7f78fc1597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53938/0 (socket says 172.21.15.7:53938) 2023-06-09T11:44:31.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790119c700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 0x7f78fc159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 -- 172.21.15.7:0/956237650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 msgr2=0x7f78fc159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 0x7f78fc159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 -- 172.21.15.7:0/956237650 >> v1:172.21.15.7:6789/0 conn(0x7f78fc0c08d0 legacy=0x7f78fc071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790099b700 1 -- 172.21.15.7:0/956237650 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc0c0560 con 0x7f78fc1593d0 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f78fb7fe700 1 -- 172.21.15.7:0/956237650 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666110525 0 0) 0x7f78fc0cc140 con 0x7f78fc0c08d0 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.397+0000 7f790119c700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 0x7f78fc159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f790099b700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc1597a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f78f000bae0 tx=0x7f78f000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51fd67c85f972a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:31.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f78fb7fe700 1 -- 172.21.15.7:0/956237650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f000dd30 con 0x7f78fc1593d0 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f78fb7fe700 1 -- 172.21.15.7:0/956237650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f000de90 con 0x7f78fc1593d0 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f78fb7fe700 1 -- 172.21.15.7:0/956237650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f0010690 con 0x7f78fc1593d0 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 -- 172.21.15.7:0/956237650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 msgr2=0x7f78fc1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc1597a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f78f000bae0 tx=0x7f78f000bdf0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 -- 172.21.15.7:0/956237650 shutdown_connections 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc072020 0x7f78fc159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 --2- 172.21.15.7:0/956237650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc1597a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.398+0000 7f7902bff700 1 -- 172.21.15.7:0/956237650 >> 172.21.15.7:0/956237650 conn(0x7f78fc06a4d0 msgr2=0x7f78fc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.399+0000 7f7902bff700 1 -- 172.21.15.7:0/956237650 shutdown_connections 2023-06-09T11:44:31.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.399+0000 7f7902bff700 1 -- 172.21.15.7:0/956237650 wait complete. 2023-06-09T11:44:31.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.400+0000 7f7902bff700 1 Processor -- start 2023-06-09T11:44:31.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.400+0000 7f7902bff700 1 -- start start 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 0x7f78fc14db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc0c08d0 0x7f78fc155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc156910 con 0x7f78fc1593d0 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc156e60 con 0x7f78fc072020 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.401+0000 7f7902bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1f20a0 con 0x7f78fc0c08d0 2023-06-09T11:44:31.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53948/0 (socket says 172.21.15.7:53948) 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 -- 172.21.15.7:0/2291833619 learned_addr learned my addr 172.21.15.7:0/2291833619 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790099b700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 0x7f78fc14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f78fbfff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc0c08d0 0x7f78fc155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 -- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc0c08d0 msgr2=0x7f78fc155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc0c08d0 0x7f78fc155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 -- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 msgr2=0x7f78fc14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 0x7f78fc14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 -- 172.21.15.7:0/2291833619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f000b740 con 0x7f78fc1593d0 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790099b700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 0x7f78fc14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f790119c700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007f50 tx=0x7f78f400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.402+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f400dec0 con 0x7f78fc1593d0 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.403+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f400f040 con 0x7f78fc1593d0 2023-06-09T11:44:31.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.403+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0c1d60 con 0x7f78fc1593d0 2023-06-09T11:44:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.403+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4003c40 con 0x7f78fc1593d0 2023-06-09T11:44:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.403+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc0c1a80 con 0x7f78fc1593d0 2023-06-09T11:44:31.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.404+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc0c2040 con 0x7f78fc1593d0 2023-06-09T11:44:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.405+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f78f4004030 con 0x7f78fc1593d0 2023-06-09T11:44:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.406+0000 7f78f97fa700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 0x7f78e006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.406+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f78f4089cd0 con 0x7f78fc1593d0 2023-06-09T11:44:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.406+0000 7f790099b700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 0x7f78e006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:31.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.407+0000 7f790099b700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 0x7f78e006d1c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78f000be30 tx=0x7f78f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:31.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.410+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f78f4054690 con 0x7f78fc1593d0 2023-06-09T11:44:31.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.659+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78fc0c0140 con 0x7f78fc1593d0 2023-06-09T11:44:31.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.659+0000 7f78f97fa700 1 -- 172.21.15.7:0/2291833619 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f78f4057cb0 con 0x7f78fc1593d0 2023-06-09T11:44:31.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:31.664 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":153,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:29.760234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:resolve","state_seq":26,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:31.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 msgr2=0x7f78e006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 0x7f78e006d1c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f78f000be30 tx=0x7f78f0008040 comp rx=0 tx=0).stop 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 msgr2=0x7f78fc0c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f78f4007f50 tx=0x7f78f400d460 comp rx=0 tx=0).stop 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 shutdown_connections 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f78fc1593d0 0x7f78fc0c71b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f78fc072020 0x7f78fc14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f78e006ad10 0x7f78e006d1c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 --2- 172.21.15.7:0/2291833619 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f78fc0c08d0 0x7f78fc155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 >> 172.21.15.7:0/2291833619 conn(0x7f78fc06a4d0 msgr2=0x7f78fc06a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 shutdown_connections 2023-06-09T11:44:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:31.662+0000 7f7902bff700 1 -- 172.21.15.7:0/2291833619 wait complete. 2023-06-09T11:44:31.668 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 153 2023-06-09T11:44:31.684 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:33.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:33.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.893+0000 7f39313ff700 1 Processor -- start 2023-06-09T11:44:33.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.893+0000 7f39313ff700 1 -- start start 2023-06-09T11:44:33.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f39313ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15a0c0 0x7f392c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f39313ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f392c0c0140 0x7f392c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f39313ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f392c0c0bf0 con 0x7f392c0c1bc0 2023-06-09T11:44:33.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f39313ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c070110 con 0x7f392c15a0c0 2023-06-09T11:44:33.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f39313ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392c071300 con 0x7f392c0c0140 2023-06-09T11:44:33.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15a0c0 0x7f392c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f392c15a0c0 0x7f392c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36364/0 (socket says 172.21.15.7:36364) 2023-06-09T11:44:33.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 -- 172.21.15.7:0/2372844925 learned_addr learned my addr 172.21.15.7:0/2372844925 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:33.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2372844925 >> v2:172.21.15.117:3300/0 conn(0x7f392c0c0140 0x7f392c1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 -- 172.21.15.7:0/2372844925 >> v2:172.21.15.117:3300/0 conn(0x7f392c0c0140 msgr2=0x7f392c1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 --2- 172.21.15.7:0/2372844925 >> v2:172.21.15.117:3300/0 conn(0x7f392c0c0140 0x7f392c1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.895+0000 7f392a7fc700 1 -- 172.21.15.7:0/2372844925 >> v1:172.21.15.117:6789/0 conn(0x7f392c0c1bc0 legacy=0x7f392c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.896+0000 7f392a7fc700 1 -- 172.21.15.7:0/2372844925 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0702f0 con 0x7f392c15a0c0 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.896+0000 7f392a7fc700 1 --2- 172.21.15.7:0/2372844925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15a0c0 0x7f392c1501b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f392c071620 tx=0x7f392000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc2f8f5ffc49889 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.896+0000 7f3929ffb700 1 -- 172.21.15.7:0/2372844925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392000dd30 con 0x7f392c15a0c0 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.896+0000 7f3929ffb700 1 -- 172.21.15.7:0/2372844925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392000de90 con 0x7f392c15a0c0 2023-06-09T11:44:33.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.896+0000 7f3929ffb700 1 -- 172.21.15.7:0/2372844925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920010690 con 0x7f392c15a0c0 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 -- 172.21.15.7:0/2372844925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15a0c0 msgr2=0x7f392c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 --2- 172.21.15.7:0/2372844925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15a0c0 0x7f392c1501b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f392c071620 tx=0x7f392000be70 comp rx=0 tx=0).stop 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 -- 172.21.15.7:0/2372844925 shutdown_connections 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 --2- 172.21.15.7:0/2372844925 >> v2:172.21.15.117:3300/0 conn(0x7f392c0c0140 0x7f392c1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 --2- 172.21.15.7:0/2372844925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c15a0c0 0x7f392c1501b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 -- 172.21.15.7:0/2372844925 >> 172.21.15.7:0/2372844925 conn(0x7f392c06a4d0 msgr2=0x7f392c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:33.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 -- 172.21.15.7:0/2372844925 shutdown_connections 2023-06-09T11:44:33.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.897+0000 7f39313ff700 1 -- 172.21.15.7:0/2372844925 wait complete. 2023-06-09T11:44:33.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.898+0000 7f39313ff700 1 Processor -- start 2023-06-09T11:44:33.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.899+0000 7f39313ff700 1 -- start start 2023-06-09T11:44:33.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.899+0000 7f39313ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c0140 0x7f392c0ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.899+0000 7f39313ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f39313ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c15a0c0 0x7f392c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f39313ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c1eba40 con 0x7f392c0c1bc0 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f39313ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f392c1eab00 con 0x7f392c0c0140 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f39313ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f392c1eadc0 con 0x7f392c15a0c0 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36366/0 (socket says 172.21.15.7:36366) 2023-06-09T11:44:33.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 -- 172.21.15.7:0/2160820795 learned_addr learned my addr 172.21.15.7:0/2160820795 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392affd700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c0140 0x7f392c0ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392b7fe700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c15a0c0 0x7f392c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 -- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c15a0c0 msgr2=0x7f392c157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c15a0c0 0x7f392c157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 -- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c0140 msgr2=0x7f392c0ca390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c0140 0x7f392c0ca390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:33.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 -- 172.21.15.7:0/2160820795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392000b7f0 con 0x7f392c0c1bc0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.900+0000 7f392a7fc700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.901+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392001f070 con 0x7f392c0c1bc0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.901+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3920003ba0 con 0x7f392c0c1bc0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.901+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920010470 con 0x7f392c0c1bc0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.901+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f392c0c1170 con 0x7f392c0c1bc0 2023-06-09T11:44:33.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.901+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f392c0c0e90 con 0x7f392c0c1bc0 2023-06-09T11:44:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.902+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f392c0c1450 con 0x7f392c0c1bc0 2023-06-09T11:44:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.903+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f392002c430 con 0x7f392c0c1bc0 2023-06-09T11:44:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.904+0000 7f39137fe700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 0x7f391806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.904+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f392008cdb0 con 0x7f392c0c1bc0 2023-06-09T11:44:33.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.904+0000 7f392affd700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 0x7f391806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:33.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.904+0000 7f392affd700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 0x7f391806d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:33.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:33.908+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3920091070 con 0x7f392c0c1bc0 2023-06-09T11:44:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.157+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f392c1575a0 con 0x7f392c0c1bc0 2023-06-09T11:44:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.158+0000 7f39137fe700 1 -- 172.21.15.7:0/2160820795 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 153 v153) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f3920057800 con 0x7f392c0c1bc0 2023-06-09T11:44:34.162 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:34.163 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":153,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":153,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:29.760234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:resolve","state_seq":26,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 msgr2=0x7f391806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 0x7f391806d190 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f391c000f60 tx=0x7f391c009040 comp rx=0 tx=0).stop 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 msgr2=0x7f392c1577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f3920000c00 tx=0x7f3920004750 comp rx=0 tx=0).stop 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 shutdown_connections 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f391806ace0 0x7f391806d190 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f392c15a0c0 0x7f392c157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f392c0c0140 0x7f392c0ca390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 --2- 172.21.15.7:0/2160820795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f392c0c1bc0 0x7f392c1577a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 >> 172.21.15.7:0/2160820795 conn(0x7f392c06a4d0 msgr2=0x7f392c0c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 shutdown_connections 2023-06-09T11:44:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:34.160+0000 7f39313ff700 1 -- 172.21.15.7:0/2160820795 wait complete. 2023-06-09T11:44:34.166 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 153 2023-06-09T11:44:34.183 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 153, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 153, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 30 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:resolve", "state_seq": 26 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:44:29.760234+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:44:36.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:36.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.391+0000 7fdc061ff700 1 Processor -- start 2023-06-09T11:44:36.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.391+0000 7fdc061ff700 1 -- start start 2023-06-09T11:44:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdc061ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 0x7fdc00070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdc061ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc00071060 0x7fdc00158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdc061ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc001580a0 con 0x7fdc00151510 2023-06-09T11:44:36.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdc061ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc001518e0 con 0x7fdc00071060 2023-06-09T11:44:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdc061ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc000c5f70 con 0x7fdc000c5b40 2023-06-09T11:44:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc00071060 0x7fdc00158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc00071060 0x7fdc00158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36370/0 (socket says 172.21.15.7:36370) 2023-06-09T11:44:36.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 -- 172.21.15.7:0/797150942 learned_addr learned my addr 172.21.15.7:0/797150942 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:36.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbf7fff700 1 --2- 172.21.15.7:0/797150942 >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 0x7fdc00070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 -- 172.21.15.7:0/797150942 >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 msgr2=0x7fdc00070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 --2- 172.21.15.7:0/797150942 >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 0x7fdc00070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 -- 172.21.15.7:0/797150942 >> v1:172.21.15.117:6789/0 conn(0x7fdc00151510 legacy=0x7fdc0015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbfffff700 1 -- 172.21.15.7:0/797150942 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000c6180 con 0x7fdc00071060 2023-06-09T11:44:36.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.393+0000 7fdbf7fff700 1 --2- 172.21.15.7:0/797150942 >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 0x7fdc00070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdbfffff700 1 --2- 172.21.15.7:0/797150942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc00158870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc000703a0 tx=0x7fdbf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a9b7567de0f9c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdbfeffd700 1 -- 172.21.15.7:0/797150942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800e070 con 0x7fdc00071060 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdbfeffd700 1 -- 172.21.15.7:0/797150942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800b9c0 con 0x7fdc00071060 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdbfeffd700 1 -- 172.21.15.7:0/797150942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf800bd70 con 0x7fdc00071060 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdc061ff700 1 -- 172.21.15.7:0/797150942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 msgr2=0x7fdc00158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.394+0000 7fdc061ff700 1 --2- 172.21.15.7:0/797150942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc00158870 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fdc000703a0 tx=0x7fdbf800b330 comp rx=0 tx=0).stop 2023-06-09T11:44:36.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 -- 172.21.15.7:0/797150942 shutdown_connections 2023-06-09T11:44:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 --2- 172.21.15.7:0/797150942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc00158870 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 --2- 172.21.15.7:0/797150942 >> v2:172.21.15.117:3300/0 conn(0x7fdc000c5b40 0x7fdc00070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 -- 172.21.15.7:0/797150942 >> 172.21.15.7:0/797150942 conn(0x7fdc0006a4d0 msgr2=0x7fdc0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 -- 172.21.15.7:0/797150942 shutdown_connections 2023-06-09T11:44:36.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.395+0000 7fdc061ff700 1 -- 172.21.15.7:0/797150942 wait complete. 2023-06-09T11:44:36.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.396+0000 7fdc061ff700 1 Processor -- start 2023-06-09T11:44:36.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.397+0000 7fdc061ff700 1 -- start start 2023-06-09T11:44:36.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.397+0000 7fdc061ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdc061ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc000c5b40 0x7fdc000c1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdc061ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc00151510 0x7fdc001f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdc061ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c9e70 con 0x7fdc00071060 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdc061ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000cbab0 con 0x7fdc00151510 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdc061ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000c36a0 con 0x7fdc000c5b40 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36378/0 (socket says 172.21.15.7:36378) 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 -- 172.21.15.7:0/3366258683 learned_addr learned my addr 172.21.15.7:0/3366258683 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:36.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbf7fff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc000c5b40 0x7fdc000c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbfffff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc00151510 0x7fdc001f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 -- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc000c5b40 msgr2=0x7fdc000c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc000c5b40 0x7fdc000c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 -- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc00151510 msgr2=0x7fdc001f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc00151510 0x7fdc001f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.398+0000 7fdbff7fe700 1 -- 172.21.15.7:0/3366258683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf8009b30 con 0x7fdc00071060 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdbff7fe700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c510 tx=0x7fdbf000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:36.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00052d0 con 0x7fdc00071060 2023-06-09T11:44:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf0005430 con 0x7fdc00071060 2023-06-09T11:44:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc00157770 con 0x7fdc00071060 2023-06-09T11:44:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf00038c0 con 0x7fdc00071060 2023-06-09T11:44:36.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.399+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc00157490 con 0x7fdc00071060 2023-06-09T11:44:36.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.400+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc00157a50 con 0x7fdc00071060 2023-06-09T11:44:36.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.401+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbf0003a20 con 0x7fdc00071060 2023-06-09T11:44:36.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.402+0000 7fdbfcff9700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 0x7fdbe006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:36.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.402+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fdbf0088f90 con 0x7fdc00071060 2023-06-09T11:44:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.402+0000 7fdbf7fff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 0x7fdbe006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:36.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.403+0000 7fdbf7fff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 0x7fdbe006d1c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc0014d030 tx=0x7fdbe8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:36.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.406+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdbf00538c0 con 0x7fdc00071060 2023-06-09T11:44:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.659+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc000c3420 con 0x7fdc00071060 2023-06-09T11:44:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.659+0000 7fdbfcff9700 1 -- 172.21.15.7:0/3366258683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fdbf0017020 con 0x7fdc00071060 2023-06-09T11:44:36.664 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:36.664 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:35.778956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 msgr2=0x7fdbe006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 0x7fdbe006d1c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc0014d030 tx=0x7fdbe8008040 comp rx=0 tx=0).stop 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 msgr2=0x7fdc000c2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000c510 tx=0x7fdbf000c820 comp rx=0 tx=0).stop 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 shutdown_connections 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc00151510 0x7fdc001f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc00071060 0x7fdc000c2a50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdbe006ad10 0x7fdbe006d1c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 --2- 172.21.15.7:0/3366258683 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc000c5b40 0x7fdc000c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 >> 172.21.15.7:0/3366258683 conn(0x7fdc0006a4d0 msgr2=0x7fdc0014d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 shutdown_connections 2023-06-09T11:44:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:36.662+0000 7fdc061ff700 1 -- 172.21.15.7:0/3366258683 wait complete. 2023-06-09T11:44:36.668 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 154 2023-06-09T11:44:36.684 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:38.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:38.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.893+0000 7fdb3a1ff700 1 Processor -- start 2023-06-09T11:44:38.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.893+0000 7fdb3a1ff700 1 -- start start 2023-06-09T11:44:38.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb3a1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb34151510 0x7fdb340c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb3a1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb3415b380 0x7fdb34070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb3a1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb341580a0 con 0x7fdb34071060 2023-06-09T11:44:38.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb3a1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb341518e0 con 0x7fdb3415b380 2023-06-09T11:44:38.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb3a1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb34071480 con 0x7fdb34151510 2023-06-09T11:44:38.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb3415b380 0x7fdb34070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb3415b380 0x7fdb34070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36386/0 (socket says 172.21.15.7:36386) 2023-06-09T11:44:38.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 -- 172.21.15.7:0/1852994035 learned_addr learned my addr 172.21.15.7:0/1852994035 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb337fe700 1 --2- 172.21.15.7:0/1852994035 >> v2:172.21.15.117:3300/0 conn(0x7fdb34151510 0x7fdb340c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 -- 172.21.15.7:0/1852994035 >> v2:172.21.15.117:3300/0 conn(0x7fdb34151510 msgr2=0x7fdb340c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/1852994035 >> v2:172.21.15.117:3300/0 conn(0x7fdb34151510 0x7fdb340c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 -- 172.21.15.7:0/1852994035 >> v1:172.21.15.117:6789/0 conn(0x7fdb34071060 legacy=0x7fdb34158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.895+0000 7fdb32ffd700 1 -- 172.21.15.7:0/1852994035 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb340c6180 con 0x7fdb3415b380 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/1852994035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3415b380 0x7fdb34070b20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406ff10 tx=0x7fdb28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b0986fd70ed155 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb327fc700 1 -- 172.21.15.7:0/1852994035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28004230 con 0x7fdb3415b380 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb327fc700 1 -- 172.21.15.7:0/1852994035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28004390 con 0x7fdb3415b380 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb327fc700 1 -- 172.21.15.7:0/1852994035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb280036a0 con 0x7fdb3415b380 2023-06-09T11:44:38.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/1852994035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3415b380 msgr2=0x7fdb34070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/1852994035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3415b380 0x7fdb34070b20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fdb3406ff10 tx=0x7fdb28009bd0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/1852994035 shutdown_connections 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/1852994035 >> v2:172.21.15.117:3300/0 conn(0x7fdb34151510 0x7fdb340c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/1852994035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb3415b380 0x7fdb34070b20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.896+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/1852994035 >> 172.21.15.7:0/1852994035 conn(0x7fdb3406a4d0 msgr2=0x7fdb3406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:38.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.897+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/1852994035 shutdown_connections 2023-06-09T11:44:38.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.897+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/1852994035 wait complete. 2023-06-09T11:44:38.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.898+0000 7fdb3a1ff700 1 Processor -- start 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.898+0000 7fdb3a1ff700 1 -- start start 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 0x7fdb34158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 0x7fdb341448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3414a840 con 0x7fdb34071060 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34159ef0 con 0x7fdb3415b380 2023-06-09T11:44:38.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.899+0000 7fdb3a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341ef6d0 con 0x7fdb34151510 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36400/0 (socket says 172.21.15.7:36400) 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 -- 172.21.15.7:0/512835727 learned_addr learned my addr 172.21.15.7:0/512835727 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 0x7fdb34158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb33fff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 0x7fdb341448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 -- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 msgr2=0x7fdb34158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:38.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 0x7fdb34158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 -- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 msgr2=0x7fdb341448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 0x7fdb341448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 -- 172.21.15.7:0/512835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb28009550 con 0x7fdb34071060 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 0x7fdb34158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb33fff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 0x7fdb341448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.900+0000 7fdb337fe700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400e810 tx=0x7fdb2400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:38.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.901+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24013610 con 0x7fdb34071060 2023-06-09T11:44:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.901+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb24013770 con 0x7fdb34071060 2023-06-09T11:44:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.901+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb24013b20 con 0x7fdb34071060 2023-06-09T11:44:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.901+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb34157770 con 0x7fdb34071060 2023-06-09T11:44:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.901+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb34157490 con 0x7fdb34071060 2023-06-09T11:44:38.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.902+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34157a50 con 0x7fdb34071060 2023-06-09T11:44:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.903+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb24013c80 con 0x7fdb34071060 2023-06-09T11:44:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.904+0000 7fdb1bfff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.904+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fdb24089930 con 0x7fdb34071060 2023-06-09T11:44:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.904+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:38.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.904+0000 7fdb32ffd700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 0x7fdb2006d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb3414ce60 tx=0x7fdb28008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:38.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:38.908+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdb24054270 con 0x7fdb34071060 2023-06-09T11:44:39.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.163+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb34074cf0 con 0x7fdb34071060 2023-06-09T11:44:39.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.163+0000 7fdb1bfff700 1 -- 172.21.15.7:0/512835727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 154 v154) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fdb24057890 con 0x7fdb34071060 2023-06-09T11:44:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:39.168 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":154,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":154,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:35.778956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:reconnect","state_seq":28,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.165+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 msgr2=0x7fdb2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.165+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 0x7fdb2006d190 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdb3414ce60 tx=0x7fdb28008030 comp rx=0 tx=0).stop 2023-06-09T11:44:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 msgr2=0x7fdb340c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400e810 tx=0x7fdb2400ebd0 comp rx=0 tx=0).stop 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 shutdown_connections 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb3415b380 0x7fdb341448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb34071060 0x7fdb340c8fa0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb2006ace0 0x7fdb2006d190 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 --2- 172.21.15.7:0/512835727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb34151510 0x7fdb34158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 >> 172.21.15.7:0/512835727 conn(0x7fdb3406a4d0 msgr2=0x7fdb34145e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 shutdown_connections 2023-06-09T11:44:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:39.166+0000 7fdb3a1ff700 1 -- 172.21.15.7:0/512835727 wait complete. 2023-06-09T11:44:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 154 2023-06-09T11:44:39.188 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:41.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.395+0000 7fd4d7dff700 1 Processor -- start 2023-06-09T11:44:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.395+0000 7fd4d7dff700 1 -- start start 2023-06-09T11:44:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.396+0000 7fd4d7dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 0x7fd4d00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.396+0000 7fd4d7dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.396+0000 7fd4d7dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d0159b20 con 0x7fd4d015cd40 2023-06-09T11:44:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.396+0000 7fd4d7dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00cac80 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.396+0000 7fd4d7dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01597d0 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d5b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 0x7fd4d00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:42854/0 (socket says 172.21.15.7:42854) 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 -- 172.21.15.7:0/2979671825 learned_addr learned my addr 172.21.15.7:0/2979671825 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 -- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 msgr2=0x7fd4d00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 0x7fd4d00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 -- 172.21.15.7:0/2979671825 >> v1:172.21.15.117:6789/0 conn(0x7fd4d015cd40 legacy=0x7fd4d00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 -- 172.21.15.7:0/2979671825 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d0071ab0 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d5b9b700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 0x7fd4d00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.397+0000 7fd4d539a700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098f0 tx=0x7fd4c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc973ec008d7c5d9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d4b99700 1 -- 172.21.15.7:0/2979671825 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011830 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d4b99700 1 -- 172.21.15.7:0/2979671825 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c0011990 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d4b99700 1 -- 172.21.15.7:0/2979671825 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0011d20 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 -- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 msgr2=0x7fd4d0159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098f0 tx=0x7fd4c0009c00 comp rx=0 tx=0).stop 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 -- 172.21.15.7:0/2979671825 shutdown_connections 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d00cb5d0 0x7fd4d00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/2979671825 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d0159fb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 -- 172.21.15.7:0/2979671825 >> 172.21.15.7:0/2979671825 conn(0x7fd4d0070af0 msgr2=0x7fd4d0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 -- 172.21.15.7:0/2979671825 shutdown_connections 2023-06-09T11:44:41.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.398+0000 7fd4d7dff700 1 -- 172.21.15.7:0/2979671825 wait complete. 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.400+0000 7fd4d7dff700 1 Processor -- start 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.400+0000 7fd4d7dff700 1 -- start start 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d01e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d015cd40 0x7fd4d01f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01e5230 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01e5590 con 0x7fd4d00cb9a0 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d7dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00c8b10 con 0x7fd4d015cd40 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36414/0 (socket says 172.21.15.7:36414) 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.401+0000 7fd4d5b9b700 1 -- 172.21.15.7:0/1688165152 learned_addr learned my addr 172.21.15.7:0/1688165152 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d639c700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d015cd40 0x7fd4d01f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d539a700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d01e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 -- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d015cd40 msgr2=0x7fd4d01f0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d015cd40 0x7fd4d01f0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 -- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 msgr2=0x7fd4d01e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d01e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 -- 172.21.15.7:0/1688165152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0094a0 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d5b9b700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00ec10 tx=0x7fd4cc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00c850 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00c9b0 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00cd20 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.402+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c0009550 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.403+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d015d4f0 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.403+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d015d210 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.404+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4cc013470 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.405+0000 7fd4c67fc700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 0x7fd4b406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.406+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd4cc089b40 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.406+0000 7fd4d539a700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 0x7fd4b406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:41.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.406+0000 7fd4d539a700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 0x7fd4b406d1c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00cbd70 tx=0x7fd4c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:41.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.410+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd4cc054590 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.658+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4d015d7d0 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.658+0000 7fd4c67fc700 1 -- 172.21.15.7:0/1688165152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fd4cc017020 con 0x7fd4d00cb5d0 2023-06-09T11:44:41.663 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:41.663 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":155,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:40.796597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.660+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 msgr2=0x7fd4b406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 0x7fd4b406d1c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd4d00cbd70 tx=0x7fd4c0008030 comp rx=0 tx=0).stop 2023-06-09T11:44:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 msgr2=0x7fd4d01e4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00ec10 tx=0x7fd4cc00ef20 comp rx=0 tx=0).stop 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 shutdown_connections 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd4b406ad10 0x7fd4b406d1c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4d015cd40 0x7fd4d01f0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4d00cb5d0 0x7fd4d01e4cf0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 --2- 172.21.15.7:0/1688165152 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4d00cb9a0 0x7fd4d01e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 >> 172.21.15.7:0/1688165152 conn(0x7fd4d0070af0 msgr2=0x7fd4d00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 shutdown_connections 2023-06-09T11:44:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:41.661+0000 7fd4d7dff700 1 -- 172.21.15.7:0/1688165152 wait complete. 2023-06-09T11:44:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 155 2023-06-09T11:44:41.683 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:43.684 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.891+0000 7f1b897ff700 1 Processor -- start 2023-06-09T11:44:43.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.892+0000 7f1b897ff700 1 -- start start 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1b84151510 0x7f1b840c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b8415b380 0x7f1b84070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b84071060 0x7f1b84158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b841518e0 con 0x7f1b8415b380 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b840714c0 con 0x7f1b84151510 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b897ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841580a0 con 0x7f1b84071060 2023-06-09T11:44:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b827fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b8415b380 0x7f1b84070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b827fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1b8415b380 0x7f1b84070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46428/0 (socket says 172.21.15.7:46428) 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.893+0000 7f1b827fc700 1 -- 172.21.15.7:0/3536109886 learned_addr learned my addr 172.21.15.7:0/3536109886 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 -- 172.21.15.7:0/3536109886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b84071060 msgr2=0x7f1b84158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 --2- 172.21.15.7:0/3536109886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b84071060 0x7f1b84158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 -- 172.21.15.7:0/3536109886 >> v2:172.21.15.117:3300/0 conn(0x7f1b84151510 msgr2=0x7f1b840c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 --2- 172.21.15.7:0/3536109886 >> v2:172.21.15.117:3300/0 conn(0x7f1b84151510 0x7f1b840c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 -- 172.21.15.7:0/3536109886 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840c6180 con 0x7f1b8415b380 2023-06-09T11:44:43.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b827fc700 1 --2- 172.21.15.7:0/3536109886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b8415b380 0x7f1b84070b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1b8406ff10 tx=0x7f1b78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda9988d9c1992c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b81ffb700 1 -- 172.21.15.7:0/3536109886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800b870 con 0x7f1b8415b380 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b81ffb700 1 -- 172.21.15.7:0/3536109886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b7800b9d0 con 0x7f1b8415b380 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.894+0000 7f1b81ffb700 1 -- 172.21.15.7:0/3536109886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800bd60 con 0x7f1b8415b380 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 -- 172.21.15.7:0/3536109886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b8415b380 msgr2=0x7f1b84070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 --2- 172.21.15.7:0/3536109886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b8415b380 0x7f1b84070b20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f1b8406ff10 tx=0x7f1b78009bd0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 -- 172.21.15.7:0/3536109886 shutdown_connections 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 --2- 172.21.15.7:0/3536109886 >> v2:172.21.15.117:3300/0 conn(0x7f1b84151510 0x7f1b840c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 --2- 172.21.15.7:0/3536109886 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b84071060 0x7f1b84158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 --2- 172.21.15.7:0/3536109886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b8415b380 0x7f1b84070b20 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 -- 172.21.15.7:0/3536109886 >> 172.21.15.7:0/3536109886 conn(0x7f1b8406a4d0 msgr2=0x7f1b8406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 -- 172.21.15.7:0/3536109886 shutdown_connections 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.895+0000 7f1b897ff700 1 -- 172.21.15.7:0/3536109886 wait complete. 2023-06-09T11:44:43.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.897+0000 7f1b897ff700 1 Processor -- start 2023-06-09T11:44:43.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.897+0000 7f1b897ff700 1 -- start start 2023-06-09T11:44:43.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b84151510 0x7f1b840c0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8415b380 0x7f1b840c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84147760 con 0x7f1b84071060 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841f3410 con 0x7f1b84151510 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b897ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840c69b0 con 0x7f1b8415b380 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46442/0 (socket says 172.21.15.7:46442) 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 -- 172.21.15.7:0/1343905673 learned_addr learned my addr 172.21.15.7:0/1343905673 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b827fc700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b84151510 0x7f1b840c0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:43.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b837fe700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8415b380 0x7f1b840c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 -- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8415b380 msgr2=0x7f1b840c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8415b380 0x7f1b840c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 -- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b84151510 msgr2=0x7f1b840c0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b84151510 0x7f1b840c0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.898+0000 7f1b82ffd700 1 -- 172.21.15.7:0/1343905673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78009550 con 0x7f1b84071060 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b82ffd700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400b530 tx=0x7f1b7400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74011830 con 0x7f1b84071060 2023-06-09T11:44:43.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b74011990 con 0x7f1b84071060 2023-06-09T11:44:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b74011d20 con 0x7f1b84071060 2023-06-09T11:44:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84157770 con 0x7f1b84071060 2023-06-09T11:44:43.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.899+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84157490 con 0x7f1b84071060 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.900+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84157a50 con 0x7f1b84071060 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.901+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b7401a400 con 0x7f1b84071060 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.902+0000 7f1b6b7fe700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 0x7f1b7006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.902+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f1b74089bc0 con 0x7f1b84071060 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.902+0000 7f1b827fc700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 0x7f1b7006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:43.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.903+0000 7f1b827fc700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 0x7f1b7006d1c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1b78014040 tx=0x7f1b78008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:43.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:43.906+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1b74054610 con 0x7f1b84071060 2023-06-09T11:44:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.162+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b84074cf0 con 0x7f1b84071060 2023-06-09T11:44:44.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.163+0000 7f1b6b7fe700 1 -- 172.21.15.7:0/1343905673 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f1b74057c30 con 0x7f1b84071060 2023-06-09T11:44:44.167 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:44.167 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":155,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:40.796597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 msgr2=0x7f1b7006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 0x7f1b7006d1c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1b78014040 tx=0x7f1b78008030 comp rx=0 tx=0).stop 2023-06-09T11:44:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 msgr2=0x7f1b840b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400b530 tx=0x7f1b7400b840 comp rx=0 tx=0).stop 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 shutdown_connections 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1b7006ad10 0x7f1b7006d1c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1b8415b380 0x7f1b840c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1b84071060 0x7f1b840b0110 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 --2- 172.21.15.7:0/1343905673 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1b84151510 0x7f1b840c0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:44.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 >> 172.21.15.7:0/1343905673 conn(0x7f1b8406a4d0 msgr2=0x7f1b8415a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 shutdown_connections 2023-06-09T11:44:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:44.165+0000 7f1b897ff700 1 -- 172.21.15.7:0/1343905673 wait complete. 2023-06-09T11:44:44.171 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 155 2023-06-09T11:44:44.187 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:44:46.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.395+0000 7f05cabff700 1 Processor -- start 2023-06-09T11:44:46.399 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.395+0000 7f05cabff700 1 -- start start 2023-06-09T11:44:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05cabff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05cabff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.400 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05cabff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c415a500 con 0x7f05c4153880 2023-06-09T11:44:46.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05cabff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c40c5f10 con 0x7f05c40c9f40 2023-06-09T11:44:46.401 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05cabff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c415da00 con 0x7f05c415b4a0 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05c3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.396+0000 7f05c919c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c919c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:58440/0 (socket says 172.21.15.7:58440) 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c899b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f05c4153880 0x7f05c415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:37782/0 (socket says 172.21.15.7:37782) 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40048/0 (socket says 172.21.15.7:40048) 2023-06-09T11:44:46.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c919c700 1 -- 172.21.15.7:0/935361163 learned_addr learned my addr 172.21.15.7:0/935361163 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 -- 172.21.15.7:0/935361163 >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 msgr2=0x7f05c40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 --2- 172.21.15.7:0/935361163 >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 -- 172.21.15.7:0/935361163 >> v1:172.21.15.117:6789/0 conn(0x7f05c4153880 legacy=0x7f05c415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 -- 172.21.15.7:0/935361163 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c41596e0 con 0x7f05c415b4a0 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c919c700 1 --2- 172.21.15.7:0/935361163 >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.397+0000 7f05c3fff700 1 --2- 172.21.15.7:0/935361163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009840 tx=0x7f05b0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395f1801bec6b4ba server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:46.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05c37fe700 1 -- 172.21.15.7:0/935361163 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011830 con 0x7f05c415b4a0 2023-06-09T11:44:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05c37fe700 1 -- 172.21.15.7:0/935361163 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0011990 con 0x7f05c415b4a0 2023-06-09T11:44:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05c37fe700 1 -- 172.21.15.7:0/935361163 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011d20 con 0x7f05c415b4a0 2023-06-09T11:44:46.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 -- 172.21.15.7:0/935361163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 msgr2=0x7f05c415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 --2- 172.21.15.7:0/935361163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009840 tx=0x7f05b0009b50 comp rx=0 tx=0).stop 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 -- 172.21.15.7:0/935361163 shutdown_connections 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 --2- 172.21.15.7:0/935361163 >> v2:172.21.15.117:3300/0 conn(0x7f05c40c9f40 0x7f05c40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 --2- 172.21.15.7:0/935361163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c415b4a0 0x7f05c415d4c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 -- 172.21.15.7:0/935361163 >> 172.21.15.7:0/935361163 conn(0x7f05c40737f0 msgr2=0x7f05c40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 -- 172.21.15.7:0/935361163 shutdown_connections 2023-06-09T11:44:46.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.398+0000 7f05cabff700 1 -- 172.21.15.7:0/935361163 wait complete. 2023-06-09T11:44:46.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.400+0000 7f05cabff700 1 Processor -- start 2023-06-09T11:44:46.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.400+0000 7f05cabff700 1 -- start start 2023-06-09T11:44:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40c9f40 0x7f05c414b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415b4a0 0x7f05c40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4158420 con 0x7f05c4153880 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c41492d0 con 0x7f05c415b4a0 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05cabff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4158fd0 con 0x7f05c40c9f40 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05c3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46456/0 (socket says 172.21.15.7:46456) 2023-06-09T11:44:46.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.401+0000 7f05c3fff700 1 -- 172.21.15.7:0/2441448597 learned_addr learned my addr 172.21.15.7:0/2441448597 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c899b700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40c9f40 0x7f05c414b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c919c700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415b4a0 0x7f05c40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 -- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40c9f40 msgr2=0x7f05c414b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40c9f40 0x7f05c414b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 -- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415b4a0 msgr2=0x7f05c40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415b4a0 0x7f05c40caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 -- 172.21.15.7:0/2441448597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc009b30 con 0x7f05c4153880 2023-06-09T11:44:46.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c3fff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f05b00090b0 tx=0x7f05b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b00038a0 con 0x7f05c4153880 2023-06-09T11:44:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b001e070 con 0x7f05c4153880 2023-06-09T11:44:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0003c40 con 0x7f05c4153880 2023-06-09T11:44:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.402+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b00094a0 con 0x7f05c4153880 2023-06-09T11:44:46.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.403+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c4159c60 con 0x7f05c4153880 2023-06-09T11:44:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.404+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c4159980 con 0x7f05c4153880 2023-06-09T11:44:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.404+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05b002d450 con 0x7f05c4153880 2023-06-09T11:44:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.406+0000 7f05c17fa700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 0x7f05a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.406+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f05b008abb0 con 0x7f05c4153880 2023-06-09T11:44:46.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.406+0000 7f05c899b700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 0x7f05a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:46.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.406+0000 7f05c899b700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 0x7f05a806d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05b8000f60 tx=0x7f05b8007d10 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.410+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f05b0055600 con 0x7f05c4153880 2023-06-09T11:44:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.661+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05c4159f40 con 0x7f05c4153880 2023-06-09T11:44:46.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.661+0000 7f05c17fa700 1 -- 172.21.15.7:0/2441448597 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 155 v155) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f05b0020020 con 0x7f05c4153880 2023-06-09T11:44:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:46.666 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":155,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":155,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:40.796597+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:rejoin","state_seq":30,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 msgr2=0x7f05a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 0x7f05a806d190 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f05b8000f60 tx=0x7f05b8007d10 comp rx=0 tx=0).stop 2023-06-09T11:44:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 msgr2=0x7f05c40cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:46.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f05b00090b0 tx=0x7f05b0004750 comp rx=0 tx=0).stop 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 shutdown_connections 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05c415b4a0 0x7f05c40caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05a806ace0 0x7f05a806d190 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05c40c9f40 0x7f05c414b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 --2- 172.21.15.7:0/2441448597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05c4153880 0x7f05c40cd990 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 >> 172.21.15.7:0/2441448597 conn(0x7f05c40737f0 msgr2=0x7f05c41e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 shutdown_connections 2023-06-09T11:44:46.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:46.664+0000 7f05cabff700 1 -- 172.21.15.7:0/2441448597 wait complete. 2023-06-09T11:44:46.670 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 155 2023-06-09T11:44:46.687 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 155, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 155, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6718": { "addr": "172.21.15.117:6833/1900293448", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 1900293448, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 1900293448, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6718, "incarnation": 132, "join_fscid": -1, "name": "b", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 30 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:rejoin", "state_seq": 30 } }, "last_failure": 0, "last_failure_osd_epoch": 67, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:44:40.796597+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 6718, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:44:48.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:48.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.896+0000 7f5a19fff700 1 Processor -- start 2023-06-09T11:44:48.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.896+0000 7f5a19fff700 1 -- start start 2023-06-09T11:44:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.897+0000 7f5a19fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a1415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a19fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1415a500 con 0x7f5a140c9f40 2023-06-09T11:44:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a19fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1415da00 con 0x7f5a14153880 2023-06-09T11:44:48.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a19fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140c5f10 con 0x7f5a1415b4a0 2023-06-09T11:44:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a13fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5a140c9f40 0x7f5a140c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56806/0 (socket says 172.21.15.7:56806) 2023-06-09T11:44:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a13fff700 1 -- 172.21.15.7:0/1580274591 learned_addr learned my addr 172.21.15.7:0/1580274591 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/1580274591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a1415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:48.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458424684 0 0) 0x7f5a0c001490 con 0x7f5a140c9f40 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.898+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59fc0035b0 con 0x7f5a140c9f40 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370514463 0 0) 0x7f5a04001110 con 0x7f5a14153880 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a12ffd700 1 -- 172.21.15.7:0/1580274591 >> v1:172.21.15.117:6789/0 conn(0x7f5a14153880 legacy=0x7f5a1415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a12ffd700 1 -- 172.21.15.7:0/1580274591 >> v1:172.21.15.7:6789/0 conn(0x7f5a140c9f40 legacy=0x7f5a140c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a12ffd700 1 -- 172.21.15.7:0/1580274591 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a141596e0 con 0x7f5a1415b4a0 2023-06-09T11:44:48.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/1580274591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a1415d4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ece86a5569a6010 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c001b80 con 0x7f5a1415b4a0 2023-06-09T11:44:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08004030 con 0x7f5a1415b4a0 2023-06-09T11:44:48.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.899+0000 7f5a127fc700 1 -- 172.21.15.7:0/1580274591 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a080043a0 con 0x7f5a1415b4a0 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 -- 172.21.15.7:0/1580274591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 msgr2=0x7f5a1415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 --2- 172.21.15.7:0/1580274591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a1415d4c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009870 tx=0x7f5a08009b80 comp rx=0 tx=0).stop 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 -- 172.21.15.7:0/1580274591 shutdown_connections 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 --2- 172.21.15.7:0/1580274591 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a1415d4c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 -- 172.21.15.7:0/1580274591 >> 172.21.15.7:0/1580274591 conn(0x7f5a140737f0 msgr2=0x7f5a140720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 -- 172.21.15.7:0/1580274591 shutdown_connections 2023-06-09T11:44:48.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.900+0000 7f5a19fff700 1 -- 172.21.15.7:0/1580274591 wait complete. 2023-06-09T11:44:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.902+0000 7f5a19fff700 1 Processor -- start 2023-06-09T11:44:48.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.903+0000 7f5a19fff700 1 -- start start 2023-06-09T11:44:48.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 0x7f5a140cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a141ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a141eef50 con 0x7f5a140c9f40 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b2100 con 0x7f5a14153880 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a19fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1414f170 con 0x7f5a1415b4a0 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46472/0 (socket says 172.21.15.7:46472) 2023-06-09T11:44:48.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 -- 172.21.15.7:0/232482848 learned_addr learned my addr 172.21.15.7:0/232482848 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a13fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a141ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 0x7f5a140cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 -- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 msgr2=0x7f5a141ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a141ef710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 -- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 msgr2=0x7f5a140cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 0x7f5a140cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:48.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a137fe700 1 -- 172.21.15.7:0/232482848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080094a0 con 0x7f5a140c9f40 2023-06-09T11:44:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a13fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a141ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.904+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 0x7f5a140cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:44:48.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f5a137fe700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c750 tx=0x7f5a0400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a040054e0 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04003680 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a14159c60 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04003a30 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.905+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a14159980 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.906+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14159f40 con 0x7f5a140c9f40 2023-06-09T11:44:48.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.907+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a04003b90 con 0x7f5a140c9f40 2023-06-09T11:44:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.908+0000 7f59fbfff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.908+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f5a04088e90 con 0x7f5a140c9f40 2023-06-09T11:44:48.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.908+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:48.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.909+0000 7f5a12ffd700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a1414d5f0 tx=0x7f5a08008000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:48.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:48.912+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5a04057390 con 0x7f5a140c9f40 2023-06-09T11:44:49.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.165+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a1415da00 con 0x7f5a140c9f40 2023-06-09T11:44:49.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.166+0000 7f59fbfff700 1 -- 172.21.15.7:0/232482848 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f5a1415da00 con 0x7f5a140c9f40 2023-06-09T11:44:49.170 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:49.170 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:49.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 msgr2=0x7f5a0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 0x7f5a0006d190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5a1414d5f0 tx=0x7f5a08008000 comp rx=0 tx=0).stop 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 msgr2=0x7f5a14150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c750 tx=0x7f5a0400cb10 comp rx=0 tx=0).stop 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 shutdown_connections 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5a0006ace0 0x7f5a0006d190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5a1415b4a0 0x7f5a141ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a140c9f40 0x7f5a14150620 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 --2- 172.21.15.7:0/232482848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a14153880 0x7f5a140cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 >> 172.21.15.7:0/232482848 conn(0x7f5a140737f0 msgr2=0x7f5a1414b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 shutdown_connections 2023-06-09T11:44:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.168+0000 7f5a19fff700 1 -- 172.21.15.7:0/232482848 wait complete. 2023-06-09T11:44:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 156 2023-06-09T11:44:49.191 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e has gained rank=0, replacing gid=6583 2023-06-09T11:44:49.191 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 0 secs before reviving mds.f 2023-06-09T11:44:49.191 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:44:49.191 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:44:49.191 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:44:49.196 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:44:49.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:49.360 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:44:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.411+0000 7fe55b5ff700 1 Processor -- start 2023-06-09T11:44:49.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.411+0000 7fe55b5ff700 1 -- start start 2023-06-09T11:44:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe55b5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 0x7fe55415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:49.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe55b5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55415a500 con 0x7fe5540c9f40 2023-06-09T11:44:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe55b5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55415da00 con 0x7fe55415b4a0 2023-06-09T11:44:49.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe55b5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540c5f10 con 0x7fe554153880 2023-06-09T11:44:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe558b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe5540c9f40 0x7fe55415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56808/0 (socket says 172.21.15.7:56808) 2023-06-09T11:44:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.412+0000 7fe558b9a700 1 -- 172.21.15.7:0/1713014660 learned_addr learned my addr 172.21.15.7:0/1713014660 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:44:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe55939b700 1 --2- 172.21.15.7:0/1713014660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 0x7fe55415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 487667642 0 0) 0x7fe544001110 con 0x7fe5540c9f40 2023-06-09T11:44:49.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5400035b0 con 0x7fe5540c9f40 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172487893 0 0) 0x7fe54c001490 con 0x7fe55415b4a0 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe544001110 con 0x7fe55415b4a0 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe55939b700 1 -- 172.21.15.7:0/1713014660 >> v1:172.21.15.117:6789/0 conn(0x7fe55415b4a0 legacy=0x7fe5540c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe55939b700 1 -- 172.21.15.7:0/1713014660 >> v1:172.21.15.7:6789/0 conn(0x7fe5540c9f40 legacy=0x7fe55415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe55939b700 1 -- 172.21.15.7:0/1713014660 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5541596e0 con 0x7fe554153880 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.413+0000 7fe55939b700 1 --2- 172.21.15.7:0/1713014660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 0x7fe55415d4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe550009870 tx=0x7fe550009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd1a04f49a11ef0c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550011830 con 0x7fe554153880 2023-06-09T11:44:49.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550011990 con 0x7fe554153880 2023-06-09T11:44:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe54bfff700 1 -- 172.21.15.7:0/1713014660 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550011d20 con 0x7fe554153880 2023-06-09T11:44:49.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1713014660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 msgr2=0x7fe55415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1713014660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 0x7fe55415d4c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe550009870 tx=0x7fe550009b80 comp rx=0 tx=0).stop 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1713014660 shutdown_connections 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1713014660 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe554153880 0x7fe55415d4c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1713014660 >> 172.21.15.7:0/1713014660 conn(0x7fe5540737f0 msgr2=0x7fe5540720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1713014660 shutdown_connections 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.414+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1713014660 wait complete. 2023-06-09T11:44:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.416+0000 7fe55b5ff700 1 Processor -- start 2023-06-09T11:44:49.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.416+0000 7fe55b5ff700 1 -- start start 2023-06-09T11:44:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5540c9f40 0x7fe55414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 0x7fe554146b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55414ae40 con 0x7fe554153880 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541531d0 con 0x7fe5540c9f40 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe55b5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554149aa0 con 0x7fe55415b4a0 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe558b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe558b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46482/0 (socket says 172.21.15.7:46482) 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.417+0000 7fe558b9a700 1 -- 172.21.15.7:0/1203628870 learned_addr learned my addr 172.21.15.7:0/1203628870 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe55939b700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5540c9f40 0x7fe55414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe559b9c700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 0x7fe554146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 -- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 msgr2=0x7fe554146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 0x7fe554146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 -- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5540c9f40 msgr2=0x7fe55414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5540c9f40 0x7fe55414c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 -- 172.21.15.7:0/1203628870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5500094a0 con 0x7fe554153880 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe559b9c700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 0x7fe554146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe558b9a700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c760 tx=0x7fe54400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544005d70 con 0x7fe554153880 2023-06-09T11:44:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544007070 con 0x7fe554153880 2023-06-09T11:44:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544007420 con 0x7fe554153880 2023-06-09T11:44:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.418+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe554159c60 con 0x7fe554153880 2023-06-09T11:44:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.419+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe554159980 con 0x7fe554153880 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.419+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe554159f40 con 0x7fe554153880 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.420+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe544007580 con 0x7fe554153880 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.421+0000 7fe549ffb700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 0x7fe53806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.421+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fe544088eb0 con 0x7fe554153880 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.422+0000 7fe55939b700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 0x7fe53806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:49.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.422+0000 7fe55939b700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 0x7fe53806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097c0 tx=0x7fe55001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.426+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe544053870 con 0x7fe554153880 2023-06-09T11:44:49.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.681+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe55414b640 con 0x7fe554153880 2023-06-09T11:44:49.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.682+0000 7fe549ffb700 1 -- 172.21.15.7:0/1203628870 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 156 v156) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fe55414b640 con 0x7fe554153880 2023-06-09T11:44:49.686 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:49.686 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":156,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 msgr2=0x7fe53806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 0x7fe53806d190 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe5500097c0 tx=0x7fe55001c040 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 msgr2=0x7fe554146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fe54400c760 tx=0x7fe54400ca70 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 shutdown_connections 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe53806ace0 0x7fe53806d190 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55415b4a0 0x7fe554146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5540c9f40 0x7fe55414c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 --2- 172.21.15.7:0/1203628870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe554153880 0x7fe554146640 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:49.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 >> 172.21.15.7:0/1203628870 conn(0x7fe5540737f0 msgr2=0x7fe55414ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 shutdown_connections 2023-06-09T11:44:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:49.684+0000 7fe55b5ff700 1 -- 172.21.15.7:0/1203628870 wait complete. 2023-06-09T11:44:49.690 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 156 2023-06-09T11:44:49.706 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-06-09T11:44:50.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:50.484 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:44:51.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:44:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.960+0000 7fd27f1ff700 1 Processor -- start 2023-06-09T11:44:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.960+0000 7fd27f1ff700 1 -- start start 2023-06-09T11:44:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27f1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2780c5b40 0x7fd278070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27f1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 0x7fd278158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27f1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2781580a0 con 0x7fd278151510 2023-06-09T11:44:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27f1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2781518e0 con 0x7fd2780c5b40 2023-06-09T11:44:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27f1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2780c5f70 con 0x7fd278071060 2023-06-09T11:44:51.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd277fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2780c5b40 0x7fd278070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd277fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2780c5b40 0x7fd278070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46496/0 (socket says 172.21.15.7:46496) 2023-06-09T11:44:51.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27cf9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd278151510 0x7fd27815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56818/0 (socket says 172.21.15.7:56818) 2023-06-09T11:44:51.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd277fff700 1 -- 172.21.15.7:0/4288700943 learned_addr learned my addr 172.21.15.7:0/4288700943 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.963+0000 7fd27d79c700 1 --2- 172.21.15.7:0/4288700943 >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 0x7fd278158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd277fff700 1 -- 172.21.15.7:0/4288700943 >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 msgr2=0x7fd278158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd277fff700 1 --2- 172.21.15.7:0/4288700943 >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 0x7fd278158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd277fff700 1 -- 172.21.15.7:0/4288700943 >> v1:172.21.15.7:6789/0 conn(0x7fd278151510 legacy=0x7fd27815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd277fff700 1 -- 172.21.15.7:0/4288700943 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2780c6180 con 0x7fd2780c5b40 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd27d79c700 1 --2- 172.21.15.7:0/4288700943 >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 0x7fd278158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:51.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd277fff700 1 --2- 172.21.15.7:0/4288700943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd278070b20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd27806ff60 tx=0x7fd264009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b0e80dcde55498 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd2777fe700 1 -- 172.21.15.7:0/4288700943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264011830 con 0x7fd2780c5b40 2023-06-09T11:44:51.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd2777fe700 1 -- 172.21.15.7:0/4288700943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264011990 con 0x7fd2780c5b40 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.964+0000 7fd2777fe700 1 -- 172.21.15.7:0/4288700943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264011d20 con 0x7fd2780c5b40 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4288700943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 msgr2=0x7fd278070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4288700943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd278070b20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd27806ff60 tx=0x7fd264009bd0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4288700943 shutdown_connections 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4288700943 >> v2:172.21.15.117:3300/0 conn(0x7fd278071060 0x7fd278158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4288700943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd278070b20 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4288700943 >> 172.21.15.7:0/4288700943 conn(0x7fd27806a4d0 msgr2=0x7fd27806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:51.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4288700943 shutdown_connections 2023-06-09T11:44:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.965+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4288700943 wait complete. 2023-06-09T11:44:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.967+0000 7fd27f1ff700 1 Processor -- start 2023-06-09T11:44:51.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.968+0000 7fd27f1ff700 1 -- start start 2023-06-09T11:44:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 0x7fd278155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278151510 0x7fd2780cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780c1360 con 0x7fd2780c5b40 2023-06-09T11:44:51.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780c0b50 con 0x7fd278151510 2023-06-09T11:44:51.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2781f6220 con 0x7fd278071060 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd277fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27cf9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 0x7fd278155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd277fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46500/0 (socket says 172.21.15.7:46500) 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd277fff700 1 -- 172.21.15.7:0/4060404765 learned_addr learned my addr 172.21.15.7:0/4060404765 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.970+0000 7fd27d79c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278151510 0x7fd2780cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 -- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 msgr2=0x7fd278155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 0x7fd278155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 -- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278151510 msgr2=0x7fd2780cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278151510 0x7fd2780cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 -- 172.21.15.7:0/4060404765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd270009a70 con 0x7fd2780c5b40 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd27cf9b700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 0x7fd278155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd277fff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640043c0 con 0x7fd2780c5b40 2023-06-09T11:44:51.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd26401f070 con 0x7fd2780c5b40 2023-06-09T11:44:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.971+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd264009550 con 0x7fd2780c5b40 2023-06-09T11:44:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.972+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2640036a0 con 0x7fd2780c5b40 2023-06-09T11:44:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.972+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd278157770 con 0x7fd2780c5b40 2023-06-09T11:44:51.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.973+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd278157490 con 0x7fd2780c5b40 2023-06-09T11:44:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.974+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd264021030 con 0x7fd2780c5b40 2023-06-09T11:44:51.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.976+0000 7fd2757fa700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 0x7fd25c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:44:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.976+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd26408ad90 con 0x7fd2780c5b40 2023-06-09T11:44:51.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.976+0000 7fd27cf9b700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 0x7fd25c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:44:51.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.978+0000 7fd27cf9b700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 0x7fd25c06d1c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000f60 tx=0x7fd26c007ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:44:51.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:51.982+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd2640557e0 con 0x7fd2780c5b40 2023-06-09T11:44:52.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.245+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd278157a50 con 0x7fd2780c5b40 2023-06-09T11:44:52.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.246+0000 7fd2757fa700 1 -- 172.21.15.7:0/4060404765 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+6222 (secure 0 0 0) 0x7fd264015020 con 0x7fd2780c5b40 2023-06-09T11:44:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:44:52.250 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 msgr2=0x7fd25c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 0x7fd25c06d1c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd26c000f60 tx=0x7fd26c007ed0 comp rx=0 tx=0).stop 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 msgr2=0x7fd2780cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264004750 comp rx=0 tx=0).stop 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 shutdown_connections 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd278151510 0x7fd2780cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:52.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd25c06ad10 0x7fd25c06d1c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd278071060 0x7fd278155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 --2- 172.21.15.7:0/4060404765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2780c5b40 0x7fd2780cb4a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 >> 172.21.15.7:0/4060404765 conn(0x7fd27806a4d0 msgr2=0x7fd27815af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.248+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 shutdown_connections 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:44:52.249+0000 7fd27f1ff700 1 -- 172.21.15.7:0/4060404765 wait complete. 2023-06-09T11:44:52.254 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 157 2023-06-09T11:44:52.271 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:44:52.271 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 16 secs before thrashing 2023-06-09T11:44:52.783 INFO:tasks.ceph.mds.e.smithi007.stderr:2023-06-09T11:44:52.779+0000 7fbaf65f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-09T11:44:52.783 INFO:tasks.ceph.mds.e.smithi007.stderr:2023-06-09T11:44:52.779+0000 7fbaf65f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:44:52.784 INFO:tasks.ceph.mds.e.smithi007.stderr:2023-06-09T11:44:52.779+0000 7fbaf65f1700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-06-09T11:45:03.264 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 2% (4/193) Updating files: 3% (6/193) Updating files: 4% (8/193) Updating files: 5% (10/193) Updating files: 6% (12/193) Updating files: 7% (14/193) Updating files: 8% (16/193) Updating files: 9% (18/193) Updating files: 10% (20/193) Updating files: 11% (22/193) Updating files: 11% (23/193) Updating files: 12% (24/193) Updating files: 13% (26/193) Updating files: 14% (28/193) Updating files: 15% (29/193) Updating files: 16% (31/193) Updating files: 17% (33/193) Updating files: 18% (35/193) Updating files: 19% (37/193) Updating files: 20% (39/193) Updating files: 21% (41/193) Updating files: 22% (43/193) Updating files: 23% (45/193) Updating files: 23% (46/193) Updating files: 24% (47/193) Updating files: 25% (49/193) Updating files: 26% (51/193) Updating files: 27% (53/193) Updating files: 28% (55/193) Updating files: 29% (56/193) Updating files: 30% (58/193) Updating files: 31% (60/193) Updating files: 32% (62/193) Updating files: 33% (64/193) Updating files: 34% (66/193) Updating files: 35% (68/193) Updating files: 36% (70/193) Updating files: 37% (72/193) Updating files: 38% (74/193) Updating files: 39% (76/193) Updating files: 40% (78/193) Updating files: 41% (80/193) Updating files: 42% (82/193) Updating files: 43% (83/193) Updating files: 44% (85/193) Updating files: 45% (87/193) Updating files: 46% (89/193) Updating files: 47% (91/193) Updating files: 48% (93/193) Updating files: 49% (95/193) Updating files: 50% (97/193) Updating files: 51% (99/193) Updating files: 52% (101/193) Updating files: 53% (103/193) Updating files: 54% (105/193) Updating files: 55% (107/193) Updating files: 56% (109/193) Updating files: 57% (111/193) Updating files: 58% (112/193) Updating files: 59% (114/193) Updating files: 60% (116/193) Updating files: 61% (118/193) Updating files: 62% (120/193) Updating files: 63% (122/193) Updating files: 64% (124/193) Updating files: 65% (126/193) Updating files: 66% (128/193) Updating files: 67% (130/193) Updating files: 68% (132/193) Updating files: 69% (134/193) Updating files: 69% (135/193) Updating files: 70% (136/193) Updating files: 71% (138/193) Updating files: 72% (139/193) Updating files: 73% (141/193) Updating files: 74% (143/193) Updating files: 75% (145/193) Updating files: 76% (147/193) Updating files: 77% (149/193) Updating files: 78% (151/193) Updating files: 79% (153/193) Updating files: 80% (155/193) Updating files: 81% (157/193) Updating files: 82% (159/193) Updating files: 83% (161/193) Updating files: 84% (163/193) Updating files: 85% (165/193) Updating files: 86% (166/193) Updating files: 87% (168/193) Updating files: 88% (170/193) Updating files: 89% (172/193) Updating files: 90% (174/193) Updating files: 91% (176/193) Updating files: 92% (178/193) Updating files: 93% (180/193) Updating files: 94% (182/193) Updating files: 94% (183/193) Updating files: 95% (184/193) Updating files: 96% (186/193) Updating files: 97% (188/193) Updating files: 98% (190/193) Updating files: 99% (192/193) Updating files: 100% (193/193) Updating files: 100% (193/193), done. 2023-06-09T11:45:04.393 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 41481853924 kclient: strip out kernel version compatibility cruft 2023-06-09T11:45:04.400 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.9 2023-06-09T11:45:04.431 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:45:04.432 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 10 -eq 48 ']' 2023-06-09T11:45:04.432 INFO:tasks.workunit.client.0.smithi007.stdout:v0.10 2023-06-09T11:45:04.432 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.10 2023-06-09T11:45:04.433 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.10 2023-06-09T11:45:04.433 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.10 2023-06-09T11:45:08.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.496+0000 7febe6bff700 1 Processor -- start 2023-06-09T11:45:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.496+0000 7febe6bff700 1 -- start start 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.497+0000 7febe6bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7febe0159eb0 0x7febe0071910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.497+0000 7febe6bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00cc140 con 0x7febe00c08d0 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.497+0000 7febe6bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00c1830 con 0x7febe01593d0 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.497+0000 7febe6bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe00714f0 con 0x7febe0159eb0 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febe519c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febe00c08d0 0x7febe014f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38906/0 (socket says 172.21.15.7:38906) 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febe519c700 1 -- 172.21.15.7:0/316107023 learned_addr learned my addr 172.21.15.7:0/316107023 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdffff700 1 --2- 172.21.15.7:0/316107023 >> v2:172.21.15.117:3300/0 conn(0x7febe0159eb0 0x7febe0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673794585 0 0) 0x7febe00cc140 con 0x7febe00c08d0 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febd00035b0 con 0x7febe00c08d0 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdffff700 1 -- 172.21.15.7:0/316107023 >> v1:172.21.15.117:6789/0 conn(0x7febe01593d0 legacy=0x7febe01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052707326 0 0) 0x7febe00c1830 con 0x7febe01593d0 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdffff700 1 -- 172.21.15.7:0/316107023 >> v1:172.21.15.7:6789/0 conn(0x7febe00c08d0 legacy=0x7febe014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.498+0000 7febdffff700 1 -- 172.21.15.7:0/316107023 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00c0560 con 0x7febe0159eb0 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febdffff700 1 --2- 172.21.15.7:0/316107023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe0159eb0 0x7febe0071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7febcc00a9e0 tx=0x7febcc00acf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5ce3c5996991a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00cb70 con 0x7febe0159eb0 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00ccd0 con 0x7febe0159eb0 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febdf7fe700 1 -- 172.21.15.7:0/316107023 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00ae80 con 0x7febe0159eb0 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febe6bff700 1 -- 172.21.15.7:0/316107023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe0159eb0 msgr2=0x7febe0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.499+0000 7febe6bff700 1 --2- 172.21.15.7:0/316107023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe0159eb0 0x7febe0071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7febcc00a9e0 tx=0x7febcc00acf0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.500+0000 7febe6bff700 1 -- 172.21.15.7:0/316107023 shutdown_connections 2023-06-09T11:45:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.500+0000 7febe6bff700 1 --2- 172.21.15.7:0/316107023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe0159eb0 0x7febe0071910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.500+0000 7febe6bff700 1 -- 172.21.15.7:0/316107023 >> 172.21.15.7:0/316107023 conn(0x7febe006a4d0 msgr2=0x7febe006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.500+0000 7febe6bff700 1 -- 172.21.15.7:0/316107023 shutdown_connections 2023-06-09T11:45:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.500+0000 7febe6bff700 1 -- 172.21.15.7:0/316107023 wait complete. 2023-06-09T11:45:08.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.502+0000 7febe6bff700 1 Processor -- start 2023-06-09T11:45:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.502+0000 7febe6bff700 1 -- start start 2023-06-09T11:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.503+0000 7febe6bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.503+0000 7febe6bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 0x7febe01efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe6bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febe0159eb0 0x7febe0071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe6bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01f2ca0 con 0x7febe01593d0 2023-06-09T11:45:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe6bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe01ee840 con 0x7febe00c08d0 2023-06-09T11:45:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe6bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00cad10 con 0x7febe0159eb0 2023-06-09T11:45:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe499b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:33210/0 (socket says 172.21.15.7:33210) 2023-06-09T11:45:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 0x7febe01efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe499b700 1 -- 172.21.15.7:0/3640439653 learned_addr learned my addr 172.21.15.7:0/3640439653 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe519c700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febe0159eb0 0x7febe0071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 -- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febe0159eb0 msgr2=0x7febe0071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febe0159eb0 0x7febe0071f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 -- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 msgr2=0x7febe01f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febdffff700 1 -- 172.21.15.7:0/3640439653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc00a640 con 0x7febe01593d0 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.504+0000 7febe499b700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febdffff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 0x7febe01efcb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7febcc017000 tx=0x7febcc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00ae80 con 0x7febe01593d0 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00c970 con 0x7febe01593d0 2023-06-09T11:45:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc023620 con 0x7febe01593d0 2023-06-09T11:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe00c1d60 con 0x7febe01593d0 2023-06-09T11:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.505+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00c1a80 con 0x7febe01593d0 2023-06-09T11:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.507+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe00c2040 con 0x7febe01593d0 2023-06-09T11:45:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.507+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febcc021070 con 0x7febe01593d0 2023-06-09T11:45:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.509+0000 7febdd7fa700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.509+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7febcc08a810 con 0x7febe01593d0 2023-06-09T11:45:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.509+0000 7febe499b700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.510+0000 7febe499b700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 0x7febc406d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7febd4000f60 tx=0x7febd4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:08.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.514+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7febcc055200 con 0x7febe01593d0 2023-06-09T11:45:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.777+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febe01ef430 con 0x7febe01593d0 2023-06-09T11:45:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.777+0000 7febdd7fa700 1 -- 172.21.15.7:0/3640439653 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+6222 (secure 0 0 0) 0x7febcc058820 con 0x7febe01593d0 2023-06-09T11:45:08.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:08.782 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 msgr2=0x7febc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 0x7febc406d190 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7febd4000f60 tx=0x7febd4009040 comp rx=0 tx=0).stop 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 msgr2=0x7febe01efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 0x7febe01efcb0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7febcc017000 tx=0x7febcc0047c0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 shutdown_connections 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7febc406ace0 0x7febc406d190 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7febe0159eb0 0x7febe0071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7febe00c08d0 0x7febe01f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 --2- 172.21.15.7:0/3640439653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7febe01593d0 0x7febe01efcb0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 >> 172.21.15.7:0/3640439653 conn(0x7febe006a4d0 msgr2=0x7febe00c7750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 shutdown_connections 2023-06-09T11:45:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:08.780+0000 7febe6bff700 1 -- 172.21.15.7:0/3640439653 wait complete. 2023-06-09T11:45:08.786 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 157 2023-06-09T11:45:08.802 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=1) 2023-06-09T11:45:08.802 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:45:08.802 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:45:08.820 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:45:08.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.017+0000 7fa44f5ff700 1 Processor -- start 2023-06-09T11:45:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.017+0000 7fa44f5ff700 1 -- start start 2023-06-09T11:45:09.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.018+0000 7fa44f5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa448153880 0x7fa44815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.018+0000 7fa44f5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44815a500 con 0x7fa4480c9f40 2023-06-09T11:45:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.018+0000 7fa44f5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44815da00 con 0x7fa44815b4a0 2023-06-09T11:45:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.018+0000 7fa44f5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4480c5f10 con 0x7fa448153880 2023-06-09T11:45:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa448153880 0x7fa44815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa448153880 0x7fa44815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34360/0 (socket says 172.21.15.7:34360) 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 -- 172.21.15.7:0/2872787435 learned_addr learned my addr 172.21.15.7:0/2872787435 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44cb9a700 1 --1- 172.21.15.7:0/2872787435 >> v1:172.21.15.7:6789/0 conn(0x7fa4480c9f40 0x7fa44815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38908/0 (socket says 172.21.15.7:38908) 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 -- 172.21.15.7:0/2872787435 >> v1:172.21.15.117:6789/0 conn(0x7fa44815b4a0 legacy=0x7fa4480c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 -- 172.21.15.7:0/2872787435 >> v1:172.21.15.7:6789/0 conn(0x7fa4480c9f40 legacy=0x7fa44815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 -- 172.21.15.7:0/2872787435 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4481596e0 con 0x7fa448153880 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa43ffff700 1 -- 172.21.15.7:0/2872787435 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906221395 0 0) 0x7fa438001110 con 0x7fa4480c9f40 2023-06-09T11:45:09.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.019+0000 7fa44d39b700 1 --2- 172.21.15.7:0/2872787435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa44815d4c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa4440098f0 tx=0x7fa444009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed39ad7d18225f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa43ffff700 1 -- 172.21.15.7:0/2872787435 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400b8a0 con 0x7fa448153880 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa43ffff700 1 -- 172.21.15.7:0/2872787435 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44400ba00 con 0x7fa448153880 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa43ffff700 1 -- 172.21.15.7:0/2872787435 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400bd90 con 0x7fa448153880 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 -- 172.21.15.7:0/2872787435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 msgr2=0x7fa44815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/2872787435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa44815d4c0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa4440098f0 tx=0x7fa444009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 -- 172.21.15.7:0/2872787435 shutdown_connections 2023-06-09T11:45:09.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/2872787435 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa44815d4c0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 -- 172.21.15.7:0/2872787435 >> 172.21.15.7:0/2872787435 conn(0x7fa4480737f0 msgr2=0x7fa4480720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:09.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.020+0000 7fa44f5ff700 1 -- 172.21.15.7:0/2872787435 shutdown_connections 2023-06-09T11:45:09.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.021+0000 7fa44f5ff700 1 -- 172.21.15.7:0/2872787435 wait complete. 2023-06-09T11:45:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.022+0000 7fa44f5ff700 1 Processor -- start 2023-06-09T11:45:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.022+0000 7fa44f5ff700 1 -- start start 2023-06-09T11:45:09.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 0x7fa4480ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 0x7fa4481efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4481590a0 con 0x7fa448153880 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4480b3830 con 0x7fa44815b4a0 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4481f2810 con 0x7fa4480c9f40 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:09.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.023+0000 7fa44cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34366/0 (socket says 172.21.15.7:34366) 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44d39b700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 0x7fa4480ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44db9c700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 0x7fa4481efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 -- 172.21.15.7:0/282997163 learned_addr learned my addr 172.21.15.7:0/282997163 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 -- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 msgr2=0x7fa4480ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 0x7fa4480ce790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 -- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 msgr2=0x7fa4481efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 0x7fa4481efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 -- 172.21.15.7:0/282997163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009550 con 0x7fa448153880 2023-06-09T11:45:09.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44d39b700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 0x7fa4480ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:09.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44db9c700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 0x7fa4481efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:09.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa44cb9a700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c550 tx=0x7fa43800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.024+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438004c80 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.025+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa438004de0 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.025+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa448159c60 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.025+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438007310 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.025+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa448159980 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.026+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa448159f40 con 0x7fa448153880 2023-06-09T11:45:09.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.027+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa438007470 con 0x7fa448153880 2023-06-09T11:45:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.028+0000 7fa43dffb700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 0x7fa42c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.028+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fa438089650 con 0x7fa448153880 2023-06-09T11:45:09.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.028+0000 7fa44d39b700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 0x7fa42c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:09.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.029+0000 7fa44d39b700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 0x7fa42c06d1c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa444000c00 tx=0x7fa444008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:09.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.033+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa438053f80 con 0x7fa448153880 2023-06-09T11:45:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.284+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa44815a500 con 0x7fa448153880 2023-06-09T11:45:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.284+0000 7fa43dffb700 1 -- 172.21.15.7:0/282997163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+6222 (secure 0 0 0) 0x7fa44815a500 con 0x7fa448153880 2023-06-09T11:45:09.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:09.289 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:09.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 msgr2=0x7fa42c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 0x7fa42c06d1c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa444000c00 tx=0x7fa444008030 comp rx=0 tx=0).stop 2023-06-09T11:45:09.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 msgr2=0x7fa4481ef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:09.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c550 tx=0x7fa43800c860 comp rx=0 tx=0).stop 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 shutdown_connections 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa44815b4a0 0x7fa4481efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa42c06ad10 0x7fa42c06d1c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4480c9f40 0x7fa4480ce790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 --2- 172.21.15.7:0/282997163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa448153880 0x7fa4481ef540 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 >> 172.21.15.7:0/282997163 conn(0x7fa4480737f0 msgr2=0x7fa4480cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 shutdown_connections 2023-06-09T11:45:09.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:09.287+0000 7fa44f5ff700 1 -- 172.21.15.7:0/282997163 wait complete. 2023-06-09T11:45:09.293 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 157 2023-06-09T11:45:09.309 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:11.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:11.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.516+0000 7fee245ff700 1 Processor -- start 2023-06-09T11:45:11.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.516+0000 7fee245ff700 1 -- start start 2023-06-09T11:45:11.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.517+0000 7fee245ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.517+0000 7fee245ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1c0c9f40 0x7fee1c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.517+0000 7fee245ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1c15a500 con 0x7fee1c15b2d0 2023-06-09T11:45:11.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.517+0000 7fee245ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1c0c5f10 con 0x7fee1c0c9f40 2023-06-09T11:45:11.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.517+0000 7fee245ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c15da00 con 0x7fee1c153880 2023-06-09T11:45:11.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fee1c0c9f40 0x7fee1c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee22b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee1c15b2d0 0x7fee1c0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38912/0 (socket says 172.21.15.7:38912) 2023-06-09T11:45:11.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee22b9c700 1 -- 172.21.15.7:0/3694869735 learned_addr learned my addr 172.21.15.7:0/3694869735 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:11.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee2239b700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 -- 172.21.15.7:0/3694869735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 msgr2=0x7fee1c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:11.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 -- 172.21.15.7:0/3694869735 >> v1:172.21.15.7:6789/0 conn(0x7fee1c15b2d0 legacy=0x7fee1c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 -- 172.21.15.7:0/3694869735 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c15b750 con 0x7fee1c0c9f40 2023-06-09T11:45:11.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21399700 1 -- 172.21.15.7:0/3694869735 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144203473 0 0) 0x7fee14001490 con 0x7fee1c15b2d0 2023-06-09T11:45:11.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.518+0000 7fee21b9a700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c15ad90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009840 tx=0x7fee0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a74b960009f025 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:11.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee21399700 1 -- 172.21.15.7:0/3694869735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c011830 con 0x7fee1c0c9f40 2023-06-09T11:45:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee21399700 1 -- 172.21.15.7:0/3694869735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0c011990 con 0x7fee1c0c9f40 2023-06-09T11:45:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee21399700 1 -- 172.21.15.7:0/3694869735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0c011d20 con 0x7fee1c0c9f40 2023-06-09T11:45:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 -- 172.21.15.7:0/3694869735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 msgr2=0x7fee1c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c15ad90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fee0c009840 tx=0x7fee0c009b50 comp rx=0 tx=0).stop 2023-06-09T11:45:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 -- 172.21.15.7:0/3694869735 shutdown_connections 2023-06-09T11:45:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 --2- 172.21.15.7:0/3694869735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c15ad90 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 -- 172.21.15.7:0/3694869735 >> 172.21.15.7:0/3694869735 conn(0x7fee1c0737f0 msgr2=0x7fee1c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 -- 172.21.15.7:0/3694869735 shutdown_connections 2023-06-09T11:45:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.519+0000 7fee245ff700 1 -- 172.21.15.7:0/3694869735 wait complete. 2023-06-09T11:45:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.522+0000 7fee245ff700 1 Processor -- start 2023-06-09T11:45:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.522+0000 7fee245ff700 1 -- start start 2023-06-09T11:45:11.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 0x7fee1c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c1518c0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c152f40 con 0x7fee1c15b2d0 2023-06-09T11:45:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee245ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c0cbbb0 con 0x7fee1c153880 2023-06-09T11:45:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee2239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.523+0000 7fee2239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34392/0 (socket says 172.21.15.7:34392) 2023-06-09T11:45:11.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 -- 172.21.15.7:0/376129389 learned_addr learned my addr 172.21.15.7:0/376129389 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee21b9a700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee22b9c700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 0x7fee1c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 -- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 msgr2=0x7fee1c0b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c0b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 -- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 msgr2=0x7fee1c06a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 0x7fee1c06a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 -- 172.21.15.7:0/376129389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0094a0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee22b9c700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 0x7fee1c06a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:45:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee2239b700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e840 tx=0x7fee1800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.524+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18013610 con 0x7fee1c0c9f40 2023-06-09T11:45:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.525+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee18013770 con 0x7fee1c0c9f40 2023-06-09T11:45:11.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.525+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18013b20 con 0x7fee1c0c9f40 2023-06-09T11:45:11.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.525+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c159980 con 0x7fee1c0c9f40 2023-06-09T11:45:11.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.525+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1c1596e0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.526+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1c159c60 con 0x7fee1c0c9f40 2023-06-09T11:45:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.527+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee18013c80 con 0x7fee1c0c9f40 2023-06-09T11:45:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.528+0000 7fee12ffd700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 0x7fee0006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.528+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fee18089640 con 0x7fee1c0c9f40 2023-06-09T11:45:11.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.528+0000 7fee21b9a700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 0x7fee0006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.529+0000 7fee21b9a700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 0x7fee0006d1c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0090b0 tx=0x7fee0c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:11.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.533+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fee180540b0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.783+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee1c0b36b0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.783+0000 7fee12ffd700 1 -- 172.21.15.7:0/376129389 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 157 v157) v1 ==== 95+0+6222 (secure 0 0 0) 0x7fee180576d0 con 0x7fee1c0c9f40 2023-06-09T11:45:11.787 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:11.787 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":157,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":156,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:44:46.810405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 msgr2=0x7fee0006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 0x7fee0006d1c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0090b0 tx=0x7fee0c008030 comp rx=0 tx=0).stop 2023-06-09T11:45:11.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 msgr2=0x7fee1c151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fee1800e840 tx=0x7fee1800eb50 comp rx=0 tx=0).stop 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 shutdown_connections 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee1c15b2d0 0x7fee1c06a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee1c0c9f40 0x7fee1c151ce0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee0006ad10 0x7fee0006d1c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 --2- 172.21.15.7:0/376129389 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee1c153880 0x7fee1c0b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:11.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 >> 172.21.15.7:0/376129389 conn(0x7fee1c0737f0 msgr2=0x7fee1c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 shutdown_connections 2023-06-09T11:45:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:11.786+0000 7fee245ff700 1 -- 172.21.15.7:0/376129389 wait complete. 2023-06-09T11:45:11.791 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 157 2023-06-09T11:45:11.807 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:13.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.015+0000 7fd3a93ff700 1 Processor -- start 2023-06-09T11:45:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.015+0000 7fd3a93ff700 1 -- start start 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.016+0000 7fd3a93ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3a415a0c0 0x7fd3a41501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.016+0000 7fd3a93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd3a41506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.016+0000 7fd3a93ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a40c0bf0 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.016+0000 7fd3a93ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a4070110 con 0x7fd3a415a0c0 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.016+0000 7fd3a93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a4071300 con 0x7fd3a40c0140 2023-06-09T11:45:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd3a40c1bc0 0x7fd3a41599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42414/0 (socket says 172.21.15.7:42414) 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a2ffd700 1 -- 172.21.15.7:0/641005695 learned_addr learned my addr 172.21.15.7:0/641005695 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd3a41506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/641005695 >> v2:172.21.15.117:3300/0 conn(0x7fd3a415a0c0 0x7fd3a41501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532873219 0 0) 0x7fd3a40c0bf0 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd38c0035b0 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656412049 0 0) 0x7fd38c0035b0 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 msgr2=0x7fd3a41506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd3a41506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 >> v2:172.21.15.117:3300/0 conn(0x7fd3a415a0c0 msgr2=0x7fd3a41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 --2- 172.21.15.7:0/641005695 >> v2:172.21.15.117:3300/0 conn(0x7fd3a415a0c0 0x7fd3a41501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40702f0 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fd394001670 con 0x7fd3a40c1bc0 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.017+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 >> v1:172.21.15.7:6789/0 conn(0x7fd3a40c1bc0 legacy=0x7fd3a41599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c003a30 0x7fd38c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd38c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c004d90 0x7fd38c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a4070110 con 0x7fd38c003a30 2023-06-09T11:45:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c00dc20 con 0x7fd38c004d90 2023-06-09T11:45:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c00dd60 con 0x7fd3a40c0140 2023-06-09T11:45:14.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a2ffd700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c003a30 0x7fd38c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd38c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c004d90 0x7fd38c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 -- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 msgr2=0x7fd38c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a40c0140 0x7fd38c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 -- 172.21.15.7:0/641005695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c003a30 msgr2=0x7fd38c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c003a30 0x7fd38c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a37fe700 1 -- 172.21.15.7:0/641005695 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0035b0 con 0x7fd38c004d90 2023-06-09T11:45:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.018+0000 7fd3a2ffd700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c003a30 0x7fd38c003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c004d90 0x7fd38c005200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009fc0 tx=0x7fd38c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=700fbeaf7322a86c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c00ec30 con 0x7fd38c004d90 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39c00f040 con 0x7fd38c004d90 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a1ffb700 1 -- 172.21.15.7:0/641005695 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39c014480 con 0x7fd38c004d90 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a93ff700 1 -- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c004d90 msgr2=0x7fd38c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/641005695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c004d90 0x7fd38c005200 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009fc0 tx=0x7fd38c005880 comp rx=0 tx=0).stop 2023-06-09T11:45:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a37fe700 1 -- 172.21.15.7:0/641005695 reap_dead start 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a93ff700 1 -- 172.21.15.7:0/641005695 shutdown_connections 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.019+0000 7fd3a93ff700 1 -- 172.21.15.7:0/641005695 >> 172.21.15.7:0/641005695 conn(0x7fd3a406a4d0 msgr2=0x7fd3a406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.020+0000 7fd3a93ff700 1 -- 172.21.15.7:0/641005695 shutdown_connections 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.020+0000 7fd3a93ff700 1 -- 172.21.15.7:0/641005695 wait complete. 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.021+0000 7fd3a93ff700 1 Processor -- start 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.021+0000 7fd3a93ff700 1 -- start start 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 0x7fd3a40ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a40c4620 0x7fd3a40c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40c4fd0 con 0x7fd38c004d90 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40c32a0 con 0x7fd3a40c4620 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.022+0000 7fd3a93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a40c3580 con 0x7fd3a414f8d0 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38778/0 (socket says 172.21.15.7:38778) 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 -- 172.21.15.7:0/3493486348 learned_addr learned my addr 172.21.15.7:0/3493486348 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 0x7fd3a40ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a37fe700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a40c4620 0x7fd3a40c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 -- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 msgr2=0x7fd3a40ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 0x7fd3a40ca8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 -- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a40c4620 msgr2=0x7fd3a40c4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a40c4620 0x7fd3a40c4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 -- 172.21.15.7:0/3493486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a40c1170 con 0x7fd38c004d90 2023-06-09T11:45:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 0x7fd3a40ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:45:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd3a2ffd700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd394001200 tx=0x7fd3940078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400f910 con 0x7fd38c004d90 2023-06-09T11:45:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.023+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39400fa70 con 0x7fd38c004d90 2023-06-09T11:45:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.024+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39400fe00 con 0x7fd38c004d90 2023-06-09T11:45:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.024+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a40c0e90 con 0x7fd38c004d90 2023-06-09T11:45:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.024+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a40c1450 con 0x7fd38c004d90 2023-06-09T11:45:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.025+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a4074cf0 con 0x7fd38c004d90 2023-06-09T11:45:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.026+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd39401d460 con 0x7fd38c004d90 2023-06-09T11:45:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.026+0000 7fd38b7fe700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 0x7fd39006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.027+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd394089d20 con 0x7fd38c004d90 2023-06-09T11:45:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.027+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 0x7fd39006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.027+0000 7fd3a27fc700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 0x7fd39006d1c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd3a414c080 tx=0x7fd398008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.031+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd3940546e0 con 0x7fd38c004d90 2023-06-09T11:45:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.282+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3a40c77d0 con 0x7fd38c004d90 2023-06-09T11:45:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.283+0000 7fd38b7fe700 1 -- 172.21.15.7:0/3493486348 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+6223 (secure 0 0 0) 0x7fd394057d00 con 0x7fd38c004d90 2023-06-09T11:45:14.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:14.286 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:12.761207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 msgr2=0x7fd39006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 0x7fd39006d1c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd3a414c080 tx=0x7fd398008040 comp rx=0 tx=0).stop 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 msgr2=0x7fd3a41ef3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fd394001200 tx=0x7fd3940078e0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 shutdown_connections 2023-06-09T11:45:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3a40c4620 0x7fd3a40c4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c004d90 0x7fd3a41ef3b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd39006ad10 0x7fd39006d1c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 --2- 172.21.15.7:0/3493486348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3a414f8d0 0x7fd3a40ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.285+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 >> 172.21.15.7:0/3493486348 conn(0x7fd3a406a4d0 msgr2=0x7fd3a41551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.286+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 shutdown_connections 2023-06-09T11:45:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:14.286+0000 7fd3a93ff700 1 -- 172.21.15.7:0/3493486348 wait complete. 2023-06-09T11:45:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 158 2023-06-09T11:45:14.306 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:16.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.514+0000 7f094a1ff700 1 Processor -- start 2023-06-09T11:45:16.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.514+0000 7f094a1ff700 1 -- start start 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.515+0000 7f094a1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09441539b0 0x7f0944159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.515+0000 7f094a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09440c4ce0 0x7f09440c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f094a1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094415a420 con 0x7f09441535e0 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f094a1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09440ca670 con 0x7f09441539b0 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f094a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09440c5a00 con 0x7f09440c4ce0 2023-06-09T11:45:16.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09441539b0 0x7f0944159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f09441539b0 0x7f0944159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38784/0 (socket says 172.21.15.7:38784) 2023-06-09T11:45:16.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 -- 172.21.15.7:0/2829331566 learned_addr learned my addr 172.21.15.7:0/2829331566 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0943fff700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09440c4ce0 0x7f09440c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 -- 172.21.15.7:0/2829331566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09440c4ce0 msgr2=0x7f09440c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09440c4ce0 0x7f09440c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 -- 172.21.15.7:0/2829331566 >> v1:172.21.15.117:6789/0 conn(0x7f09441535e0 legacy=0x7f09440c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 -- 172.21.15.7:0/2829331566 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094406b460 con 0x7f09441539b0 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.516+0000 7f0942ffd700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09441539b0 0x7f0944159ee0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f09380098f0 tx=0x7f0938009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41372962142065d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f09427fc700 1 -- 172.21.15.7:0/2829331566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093800b8a0 con 0x7f09441539b0 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f09427fc700 1 -- 172.21.15.7:0/2829331566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093800ba00 con 0x7f09441539b0 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f09427fc700 1 -- 172.21.15.7:0/2829331566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093800bd90 con 0x7f09441539b0 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 -- 172.21.15.7:0/2829331566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09441539b0 msgr2=0x7f0944159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09441539b0 0x7f0944159ee0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f09380098f0 tx=0x7f0938009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:16.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 -- 172.21.15.7:0/2829331566 shutdown_connections 2023-06-09T11:45:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09440c4ce0 0x7f09440c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 --2- 172.21.15.7:0/2829331566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09441539b0 0x7f0944159ee0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.517+0000 7f094a1ff700 1 -- 172.21.15.7:0/2829331566 >> 172.21.15.7:0/2829331566 conn(0x7f09440737f0 msgr2=0x7f0944072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.518+0000 7f094a1ff700 1 -- 172.21.15.7:0/2829331566 shutdown_connections 2023-06-09T11:45:16.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.518+0000 7f094a1ff700 1 -- 172.21.15.7:0/2829331566 wait complete. 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.519+0000 7f094a1ff700 1 Processor -- start 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.519+0000 7f094a1ff700 1 -- start start 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09441535e0 0x7f0944158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09441539b0 0x7f0944158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944156a50 con 0x7f09440c4ce0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094414ec90 con 0x7f09441539b0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.520+0000 7f094a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944157a20 con 0x7f09441535e0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38788/0 (socket says 172.21.15.7:38788) 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 -- 172.21.15.7:0/552315511 learned_addr learned my addr 172.21.15.7:0/552315511 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f0942ffd700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09441535e0 0x7f0944158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f0943fff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09441539b0 0x7f0944158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 -- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09441535e0 msgr2=0x7f0944158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09441535e0 0x7f0944158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 -- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09441539b0 msgr2=0x7f0944158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09441539b0 0x7f0944158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 -- 172.21.15.7:0/552315511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0938009550 con 0x7f09440c4ce0 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.521+0000 7f09437fe700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f093400c4e0 tx=0x7f093400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.522+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09340052d0 con 0x7f09440c4ce0 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.522+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0934005430 con 0x7f09440c4ce0 2023-06-09T11:45:16.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.522+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09340038c0 con 0x7f09440c4ce0 2023-06-09T11:45:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.522+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094406a850 con 0x7f09440c4ce0 2023-06-09T11:45:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.522+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094406a570 con 0x7f09440c4ce0 2023-06-09T11:45:16.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.523+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094406ab30 con 0x7f09440c4ce0 2023-06-09T11:45:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.524+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0934003a20 con 0x7f09440c4ce0 2023-06-09T11:45:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.524+0000 7f092bfff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 0x7f093006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.525+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f0934088d20 con 0x7f09440c4ce0 2023-06-09T11:45:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.525+0000 7f0942ffd700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 0x7f093006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:16.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.525+0000 7f0942ffd700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 0x7f093006d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0944153da0 tx=0x7f0938008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:16.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.529+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f093408f080 con 0x7f09440c4ce0 2023-06-09T11:45:16.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.782+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f094415a420 con 0x7f09440c4ce0 2023-06-09T11:45:16.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.783+0000 7f092bfff700 1 -- 172.21.15.7:0/552315511 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+6223 (secure 0 0 0) 0x7f094415a420 con 0x7f09440c4ce0 2023-06-09T11:45:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:16.787 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:12.761207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.785+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 msgr2=0x7f093006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.785+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 0x7f093006d190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0944153da0 tx=0x7f0938008030 comp rx=0 tx=0).stop 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.785+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 msgr2=0x7f0944154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.785+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f093400c4e0 tx=0x7f093400c7f0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 shutdown_connections 2023-06-09T11:45:16.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f09441539b0 0x7f0944158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09440c4ce0 0x7f0944154040 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f093006ace0 0x7f093006d190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 --2- 172.21.15.7:0/552315511 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f09441535e0 0x7f0944158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 >> 172.21.15.7:0/552315511 conn(0x7f09440737f0 msgr2=0x7f09440cb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 shutdown_connections 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:16.786+0000 7f094a1ff700 1 -- 172.21.15.7:0/552315511 wait complete. 2023-06-09T11:45:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 158 2023-06-09T11:45:16.807 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:18.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.014+0000 7fd451fff700 1 Processor -- start 2023-06-09T11:45:19.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.014+0000 7fd451fff700 1 -- start start 2023-06-09T11:45:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd451fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd44c1593d0 0x7fd44c1597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd451fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0cc140 con 0x7fd44c072020 2023-06-09T11:45:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd451fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0c1830 con 0x7fd44c0c08d0 2023-06-09T11:45:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd451fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c0714f0 con 0x7fd44c1593d0 2023-06-09T11:45:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd44bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd44c072020 0x7fd44c14f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42418/0 (socket says 172.21.15.7:42418) 2023-06-09T11:45:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd44bfff700 1 -- 172.21.15.7:0/2296730090 learned_addr learned my addr 172.21.15.7:0/2296730090 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.016+0000 7fd44b7fe700 1 --2- 172.21.15.7:0/2296730090 >> v2:172.21.15.117:3300/0 conn(0x7fd44c1593d0 0x7fd44c1597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44a7fc700 1 -- 172.21.15.7:0/2296730090 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103060678 0 0) 0x7fd44c0cc140 con 0x7fd44c072020 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44a7fc700 1 -- 172.21.15.7:0/2296730090 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4340034d0 con 0x7fd44c072020 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44b7fe700 1 -- 172.21.15.7:0/2296730090 >> v1:172.21.15.117:6789/0 conn(0x7fd44c0c08d0 legacy=0x7fd44c071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44b7fe700 1 -- 172.21.15.7:0/2296730090 >> v1:172.21.15.7:6789/0 conn(0x7fd44c072020 legacy=0x7fd44c14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44b7fe700 1 -- 172.21.15.7:0/2296730090 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0c0560 con 0x7fd44c1593d0 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.017+0000 7fd44b7fe700 1 --2- 172.21.15.7:0/2296730090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1597a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009840 tx=0x7fd43c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89b2cb924c0529d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd44a7fc700 1 -- 172.21.15.7:0/2296730090 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c011830 con 0x7fd44c1593d0 2023-06-09T11:45:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd44a7fc700 1 -- 172.21.15.7:0/2296730090 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c011990 con 0x7fd44c1593d0 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd44a7fc700 1 -- 172.21.15.7:0/2296730090 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c011d20 con 0x7fd44c1593d0 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 -- 172.21.15.7:0/2296730090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 msgr2=0x7fd44c1597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 --2- 172.21.15.7:0/2296730090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1597a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009840 tx=0x7fd43c009b50 comp rx=0 tx=0).stop 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 -- 172.21.15.7:0/2296730090 shutdown_connections 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 --2- 172.21.15.7:0/2296730090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1597a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 -- 172.21.15.7:0/2296730090 >> 172.21.15.7:0/2296730090 conn(0x7fd44c06a4d0 msgr2=0x7fd44c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 -- 172.21.15.7:0/2296730090 shutdown_connections 2023-06-09T11:45:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.018+0000 7fd451fff700 1 -- 172.21.15.7:0/2296730090 wait complete. 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.020+0000 7fd451fff700 1 Processor -- start 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.020+0000 7fd451fff700 1 -- start start 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd44c072020 0x7fd44c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1eadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c14ae90 con 0x7fd44c0c08d0 2023-06-09T11:45:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c14a9c0 con 0x7fd44c1593d0 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd451fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c0c26c0 con 0x7fd44c072020 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd44affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd44affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38802/0 (socket says 172.21.15.7:38802) 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd44affd700 1 -- 172.21.15.7:0/248776821 learned_addr learned my addr 172.21.15.7:0/248776821 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd44b7fe700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd44c072020 0x7fd44c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.021+0000 7fd44bfff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1eadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 -- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd44c072020 msgr2=0x7fd44c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd44c072020 0x7fd44c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 -- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 msgr2=0x7fd44c1eadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1eadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 -- 172.21.15.7:0/248776821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0094a0 con 0x7fd44c0c08d0 2023-06-09T11:45:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd44affd700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c4e0 tx=0x7fd44000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400052d0 con 0x7fd44c0c08d0 2023-06-09T11:45:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd440005430 con 0x7fd44c0c08d0 2023-06-09T11:45:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4400038c0 con 0x7fd44c0c08d0 2023-06-09T11:45:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c0c1d60 con 0x7fd44c0c08d0 2023-06-09T11:45:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.022+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0c1a80 con 0x7fd44c0c08d0 2023-06-09T11:45:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.023+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c0c2040 con 0x7fd44c0c08d0 2023-06-09T11:45:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.024+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd440003a20 con 0x7fd44c0c08d0 2023-06-09T11:45:19.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.025+0000 7fd433fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 0x7fd43806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.025+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd440088f50 con 0x7fd44c0c08d0 2023-06-09T11:45:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.025+0000 7fd44b7fe700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 0x7fd43806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:19.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.026+0000 7fd44b7fe700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 0x7fd43806d1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0090b0 tx=0x7fd43c01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.030+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd440053880 con 0x7fd44c0c08d0 2023-06-09T11:45:19.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.282+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd44c074cf0 con 0x7fd44c0c08d0 2023-06-09T11:45:19.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.283+0000 7fd433fff700 1 -- 172.21.15.7:0/248776821 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+6223 (secure 0 0 0) 0x7fd440017020 con 0x7fd44c0c08d0 2023-06-09T11:45:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:19.287 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:12.761207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:19.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.285+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 msgr2=0x7fd43806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.285+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 0x7fd43806d1c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd43c0090b0 tx=0x7fd43c01c040 comp rx=0 tx=0).stop 2023-06-09T11:45:19.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.285+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 msgr2=0x7fd44c1582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:19.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.285+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c4e0 tx=0x7fd44000c7f0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 shutdown_connections 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd44c1593d0 0x7fd44c1eadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd43806ad10 0x7fd43806d1c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd44c072020 0x7fd44c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 --2- 172.21.15.7:0/248776821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd44c0c08d0 0x7fd44c1582b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 >> 172.21.15.7:0/248776821 conn(0x7fd44c06a4d0 msgr2=0x7fd44c0c6990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:19.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 shutdown_connections 2023-06-09T11:45:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:19.286+0000 7fd451fff700 1 -- 172.21.15.7:0/248776821 wait complete. 2023-06-09T11:45:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 158 2023-06-09T11:45:19.307 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:20.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:20.594 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:21.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.516+0000 7fd17ffff700 1 Processor -- start 2023-06-09T11:45:21.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.516+0000 7fd17ffff700 1 -- start start 2023-06-09T11:45:21.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.517+0000 7fd17ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd178151510 0x7fd1780c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.517+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd178070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.517+0000 7fd17ffff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1781580a0 con 0x7fd178071060 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.517+0000 7fd17ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1780714e0 con 0x7fd178151510 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.517+0000 7fd17ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781518e0 con 0x7fd17815b380 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd178071060 0x7fd178158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42424/0 (socket says 172.21.15.7:42424) 2023-06-09T11:45:21.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17e59c700 1 -- 172.21.15.7:0/3693524546 learned_addr learned my addr 172.21.15.7:0/3693524546 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3693524546 >> v2:172.21.15.117:3300/0 conn(0x7fd178151510 0x7fd1780c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17d59a700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd178070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17cd99700 1 -- 172.21.15.7:0/3693524546 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285701421 0 0) 0x7fd1781580a0 con 0x7fd178071060 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17cd99700 1 -- 172.21.15.7:0/3693524546 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1640035b0 con 0x7fd178071060 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 msgr2=0x7fd178070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd178070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3693524546 >> v1:172.21.15.7:6789/0 conn(0x7fd178071060 legacy=0x7fd178158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.518+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3693524546 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780c6180 con 0x7fd178151510 2023-06-09T11:45:21.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.519+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c5b40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd174009840 tx=0x7fd174009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a99b2e637321fb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.519+0000 7fd17cd99700 1 -- 172.21.15.7:0/3693524546 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174011830 con 0x7fd178151510 2023-06-09T11:45:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.519+0000 7fd17cd99700 1 -- 172.21.15.7:0/3693524546 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174011990 con 0x7fd178151510 2023-06-09T11:45:21.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.519+0000 7fd17cd99700 1 -- 172.21.15.7:0/3693524546 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174011d20 con 0x7fd178151510 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 -- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 msgr2=0x7fd1780c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c5b40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd174009840 tx=0x7fd174009b50 comp rx=0 tx=0).stop 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 -- 172.21.15.7:0/3693524546 shutdown_connections 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c5b40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3693524546 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd178070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 -- 172.21.15.7:0/3693524546 >> 172.21.15.7:0/3693524546 conn(0x7fd17806a4d0 msgr2=0x7fd17806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:21.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 -- 172.21.15.7:0/3693524546 shutdown_connections 2023-06-09T11:45:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.520+0000 7fd17ffff700 1 -- 172.21.15.7:0/3693524546 wait complete. 2023-06-09T11:45:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.522+0000 7fd17ffff700 1 Processor -- start 2023-06-09T11:45:21.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.522+0000 7fd17ffff700 1 -- start start 2023-06-09T11:45:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd1781e3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781e2d20 con 0x7fd178071060 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd178159460 con 0x7fd178151510 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd178158500 con 0x7fd17815b380 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17d59a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38808/0 (socket says 172.21.15.7:38808) 2023-06-09T11:45:21.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3421311977 learned_addr learned my addr 172.21.15.7:0/3421311977 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17e59c700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd1781e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 msgr2=0x7fd1781e3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd1781e3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 msgr2=0x7fd1780c1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.523+0000 7fd17dd9b700 1 -- 172.21.15.7:0/3421311977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740094a0 con 0x7fd178071060 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd17dd9b700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1740090b0 tx=0x7fd174004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:21.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174004590 con 0x7fd178071060 2023-06-09T11:45:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17401f070 con 0x7fd178071060 2023-06-09T11:45:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178157770 con 0x7fd178071060 2023-06-09T11:45:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740038e0 con 0x7fd178071060 2023-06-09T11:45:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.524+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd178157490 con 0x7fd178071060 2023-06-09T11:45:21.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.525+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178157a50 con 0x7fd178071060 2023-06-09T11:45:21.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.526+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd174003a40 con 0x7fd178071060 2023-06-09T11:45:21.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.528+0000 7fd16e7fc700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 0x7fd15c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:21.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.528+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7fd174021070 con 0x7fd178071060 2023-06-09T11:45:21.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.528+0000 7fd17d59a700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 0x7fd15c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:21.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.529+0000 7fd17d59a700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 0x7fd15c06d1c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1780cc180 tx=0x7fd168009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:21.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.532+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd1740555e0 con 0x7fd178071060 2023-06-09T11:45:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.787+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd178074cf0 con 0x7fd178071060 2023-06-09T11:45:21.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.788+0000 7fd16e7fc700 1 -- 172.21.15.7:0/3421311977 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+6223 (secure 0 0 0) 0x7fd174058c00 con 0x7fd178071060 2023-06-09T11:45:21.791 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:21.792 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:12.761207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.790+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 msgr2=0x7fd15c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.790+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 0x7fd15c06d1c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd1780cc180 tx=0x7fd168009210 comp rx=0 tx=0).stop 2023-06-09T11:45:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.790+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 msgr2=0x7fd178159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.790+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1740090b0 tx=0x7fd174004750 comp rx=0 tx=0).stop 2023-06-09T11:45:21.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 shutdown_connections 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ad10 0x7fd15c06d1c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd17815b380 0x7fd1781e3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd178071060 0x7fd178159ec0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 --2- 172.21.15.7:0/3421311977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd178151510 0x7fd1780c1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 >> 172.21.15.7:0/3421311977 conn(0x7fd17806a4d0 msgr2=0x7fd17814d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 shutdown_connections 2023-06-09T11:45:21.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:21.791+0000 7fd17ffff700 1 -- 172.21.15.7:0/3421311977 wait complete. 2023-06-09T11:45:21.796 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 158 2023-06-09T11:45:21.812 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:23.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.020+0000 7f62965ff700 1 Processor -- start 2023-06-09T11:45:24.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.021+0000 7f62965ff700 1 -- start start 2023-06-09T11:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f62965ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f62900c9fe0 0x7f62900cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f62965ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f62965ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629015cec0 con 0x7f62900c9090 2023-06-09T11:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f62965ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62900ca440 con 0x7f62900c9fe0 2023-06-09T11:45:24.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f62965ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629015d160 con 0x7f62900cfaf0 2023-06-09T11:45:24.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f6294b9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f6294b9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39204/0 (socket says 172.21.15.7:39204) 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f6294b9c700 1 -- 172.21.15.7:0/3423319981 learned_addr learned my addr 172.21.15.7:0/3423319981 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.022+0000 7f628f7fe700 1 --2- 172.21.15.7:0/3423319981 >> v2:172.21.15.117:3300/0 conn(0x7f62900c9fe0 0x7f62900cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f6294b9c700 1 -- 172.21.15.7:0/3423319981 >> v2:172.21.15.117:3300/0 conn(0x7f62900c9fe0 msgr2=0x7f62900cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f6294b9c700 1 --2- 172.21.15.7:0/3423319981 >> v2:172.21.15.117:3300/0 conn(0x7f62900c9fe0 0x7f62900cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f6294b9c700 1 -- 172.21.15.7:0/3423319981 >> v1:172.21.15.117:6789/0 conn(0x7f62900c9090 legacy=0x7f62900cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f628effd700 1 -- 172.21.15.7:0/3423319981 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842189111 0 0) 0x7f629015cec0 con 0x7f62900c9090 2023-06-09T11:45:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f6294b9c700 1 -- 172.21.15.7:0/3423319981 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629015d440 con 0x7f62900cfaf0 2023-06-09T11:45:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f6294b9c700 1 --2- 172.21.15.7:0/3423319981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6288009e50 tx=0x7f628800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89813c73e14d2e1f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f628effd700 1 -- 172.21.15.7:0/3423319981 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800e070 con 0x7f62900cfaf0 2023-06-09T11:45:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f628effd700 1 -- 172.21.15.7:0/3423319981 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628800b910 con 0x7f62900cfaf0 2023-06-09T11:45:24.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.023+0000 7f628effd700 1 -- 172.21.15.7:0/3423319981 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628800bcc0 con 0x7f62900cfaf0 2023-06-09T11:45:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 -- 172.21.15.7:0/3423319981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 msgr2=0x7f62901598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 --2- 172.21.15.7:0/3423319981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6288009e50 tx=0x7f628800b330 comp rx=0 tx=0).stop 2023-06-09T11:45:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 -- 172.21.15.7:0/3423319981 shutdown_connections 2023-06-09T11:45:24.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 --2- 172.21.15.7:0/3423319981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901598b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 --2- 172.21.15.7:0/3423319981 >> v2:172.21.15.117:3300/0 conn(0x7f62900c9fe0 0x7f62900cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 -- 172.21.15.7:0/3423319981 >> 172.21.15.7:0/3423319981 conn(0x7f6290070af0 msgr2=0x7f6290070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 -- 172.21.15.7:0/3423319981 shutdown_connections 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.024+0000 7f62965ff700 1 -- 172.21.15.7:0/3423319981 wait complete. 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.026+0000 7f62965ff700 1 Processor -- start 2023-06-09T11:45:24.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.026+0000 7f62965ff700 1 -- start start 2023-06-09T11:45:24.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62900c9090 0x7f6290153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290158830 con 0x7f62900c9fe0 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6290159330 con 0x7f62900c9090 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f62965ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62900c7b60 con 0x7f62900cfaf0 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f628f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f628f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57830/0 (socket says 172.21.15.7:57830) 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f628f7fe700 1 -- 172.21.15.7:0/2427762207 learned_addr learned my addr 172.21.15.7:0/2427762207 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f6294b9c700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.027+0000 7f628ffff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62900c9090 0x7f6290153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 -- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 msgr2=0x7f62901e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 -- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62900c9090 msgr2=0x7f6290153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62900c9090 0x7f6290153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 -- 172.21.15.7:0/2427762207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288009b30 con 0x7f62900c9fe0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628f7fe700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f628400e8a0 tx=0x7f628400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6284009590 con 0x7f62900c9fe0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6284013680 con 0x7f62900c9fe0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62900ca620 con 0x7f62900c9fe0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62840139f0 con 0x7f62900c9fe0 2023-06-09T11:45:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.028+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629015d720 con 0x7f62900c9fe0 2023-06-09T11:45:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.029+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62900b53f0 con 0x7f62900c9fe0 2023-06-09T11:45:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.030+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6284013b50 con 0x7f62900c9fe0 2023-06-09T11:45:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.031+0000 7f628cff9700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 0x7f627c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.031+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 7668+0+0 (secure 0 0 0) 0x7f6284089950 con 0x7f62900c9fe0 2023-06-09T11:45:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.031+0000 7f628ffff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 0x7f627c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.032+0000 7f628ffff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 0x7f627c06d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6280000f60 tx=0x7f6280009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.035+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62840543a0 con 0x7f62900c9fe0 2023-06-09T11:45:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.294+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6290074cf0 con 0x7f62900c9fe0 2023-06-09T11:45:24.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.294+0000 7f628cff9700 1 -- 172.21.15.7:0/2427762207 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 158 v158) v1 ==== 95+0+6223 (secure 0 0 0) 0x7f62840579c0 con 0x7f62900c9fe0 2023-06-09T11:45:24.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:24.298 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":158,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7141,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":157}],"filesystems":[{"mdsmap":{"epoch":158,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:12.761207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":6718,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6718":{"gid":6718,"name":"b","rank":1,"incarnation":132,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6833/1900293448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":1900293448},{"type":"v1","addr":"172.21.15.117:6833","nonce":1900293448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 msgr2=0x7f627c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 0x7f627c06d190 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6280000f60 tx=0x7f6280009040 comp rx=0 tx=0).stop 2023-06-09T11:45:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 msgr2=0x7f6290154420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f628400e8a0 tx=0x7f628400ebb0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 shutdown_connections 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f627c06ace0 0x7f627c06d190 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62900cfaf0 0x7f62901e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62900c9090 0x7f6290153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 --2- 172.21.15.7:0/2427762207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62900c9fe0 0x7f6290154420 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 >> 172.21.15.7:0/2427762207 conn(0x7f6290070af0 msgr2=0x7f6290156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 shutdown_connections 2023-06-09T11:45:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:24.297+0000 7f62965ff700 1 -- 172.21.15.7:0/2427762207 wait complete. 2023-06-09T11:45:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 158 2023-06-09T11:45:24.318 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:45:26.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.526+0000 7f45b87ff700 1 Processor -- start 2023-06-09T11:45:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.526+0000 7f45b87ff700 1 -- start start 2023-06-09T11:45:26.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.527+0000 7f45b87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.527+0000 7f45b87ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 0x7f45b00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.527+0000 7f45b87ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b0159b20 con 0x7f45b00cb9a0 2023-06-09T11:45:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.527+0000 7f45b87ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b00cac80 con 0x7f45b015cd40 2023-06-09T11:45:26.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.527+0000 7f45b87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b01597d0 con 0x7f45b00cb5d0 2023-06-09T11:45:26.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39232/0 (socket says 172.21.15.7:39232) 2023-06-09T11:45:26.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 -- 172.21.15.7:0/1255584468 learned_addr learned my addr 172.21.15.7:0/1255584468 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:26.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b5d9a700 1 --1- 172.21.15.7:0/1255584468 >> v1:172.21.15.117:6789/0 conn(0x7f45b00cb9a0 0x7f45b0159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:43798/0 (socket says 172.21.15.7:43798) 2023-06-09T11:45:26.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b6d9c700 1 --2- 172.21.15.7:0/1255584468 >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 0x7f45b00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 -- 172.21.15.7:0/1255584468 >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 msgr2=0x7f45b00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 --2- 172.21.15.7:0/1255584468 >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 0x7f45b00cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 -- 172.21.15.7:0/1255584468 >> v1:172.21.15.117:6789/0 conn(0x7f45b00cb9a0 legacy=0x7f45b0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b659b700 1 -- 172.21.15.7:0/1255584468 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0071ab0 con 0x7f45b00cb5d0 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.528+0000 7f45b6d9c700 1 --2- 172.21.15.7:0/1255584468 >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 0x7f45b00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:26.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.529+0000 7f45b659b700 1 --2- 172.21.15.7:0/1255584468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0098f0 tx=0x7f45ac009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411dc0bd0c3bb706 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:26.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.529+0000 7f45b5599700 1 -- 172.21.15.7:0/1255584468 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac011830 con 0x7f45b00cb5d0 2023-06-09T11:45:26.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.529+0000 7f45b5599700 1 -- 172.21.15.7:0/1255584468 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ac011990 con 0x7f45b00cb5d0 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.529+0000 7f45b5599700 1 -- 172.21.15.7:0/1255584468 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ac011d20 con 0x7f45b00cb5d0 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 -- 172.21.15.7:0/1255584468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 msgr2=0x7f45b00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 --2- 172.21.15.7:0/1255584468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0098f0 tx=0x7f45ac009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 -- 172.21.15.7:0/1255584468 shutdown_connections 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 --2- 172.21.15.7:0/1255584468 >> v2:172.21.15.117:3300/0 conn(0x7f45b015cd40 0x7f45b00cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 --2- 172.21.15.7:0/1255584468 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00b52b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 -- 172.21.15.7:0/1255584468 >> 172.21.15.7:0/1255584468 conn(0x7f45b0070af0 msgr2=0x7f45b0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:26.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 -- 172.21.15.7:0/1255584468 shutdown_connections 2023-06-09T11:45:26.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.530+0000 7f45b87ff700 1 -- 172.21.15.7:0/1255584468 wait complete. 2023-06-09T11:45:26.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.532+0000 7f45b87ff700 1 Processor -- start 2023-06-09T11:45:26.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.532+0000 7f45b87ff700 1 -- start start 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00cb9a0 0x7f45b00c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00ceea0 con 0x7f45b015cd40 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b00d0240 con 0x7f45b00cb9a0 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0159500 con 0x7f45b00cb5d0 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57838/0 (socket says 172.21.15.7:57838) 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b6d9c700 1 -- 172.21.15.7:0/4145000286 learned_addr learned my addr 172.21.15.7:0/4145000286 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b659b700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.533+0000 7f45b5d9a700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00cb9a0 0x7f45b00c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 -- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 msgr2=0x7f45b00cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 -- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00cb9a0 msgr2=0x7f45b00c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00cb9a0 0x7f45b00c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 -- 172.21.15.7:0/4145000286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009a70 con 0x7f45b015cd40 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b6d9c700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f45a800ef90 tx=0x7f45a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a800cc90 con 0x7f45b015cd40 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a800f040 con 0x7f45b015cd40 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a80146c0 con 0x7f45b015cd40 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac009550 con 0x7f45b015cd40 2023-06-09T11:45:26.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.534+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b015d4f0 con 0x7f45b015cd40 2023-06-09T11:45:26.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.535+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b015d210 con 0x7f45b015cd40 2023-06-09T11:45:26.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.536+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45a8014820 con 0x7f45b015cd40 2023-06-09T11:45:26.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.538+0000 7f45a6ffd700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 0x7f459406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:26.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.538+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f45a808a720 con 0x7f45b015cd40 2023-06-09T11:45:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.538+0000 7f45b659b700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 0x7f459406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:26.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.539+0000 7f45b659b700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 0x7f459406d1c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009c40 tx=0x7f45ac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:26.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.542+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f45a8055000 con 0x7f45b015cd40 2023-06-09T11:45:26.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.795+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45b015d7d0 con 0x7f45b015cd40 2023-06-09T11:45:26.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.795+0000 7f45a6ffd700 1 -- 172.21.15.7:0/4145000286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f45a8018070 con 0x7f45b015cd40 2023-06-09T11:45:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:26.799 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":160,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:25.601471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 msgr2=0x7f459406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 0x7f459406d1c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009c40 tx=0x7f45ac008030 comp rx=0 tx=0).stop 2023-06-09T11:45:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 msgr2=0x7f45b0152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:26.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f45a800ef90 tx=0x7f45a800c3a0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 shutdown_connections 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45b015cd40 0x7f45b0152b00 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f459406ad10 0x7f459406d1c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f45b00cb5d0 0x7f45b00cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 --2- 172.21.15.7:0/4145000286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45b00cb9a0 0x7f45b00c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 >> 172.21.15.7:0/4145000286 conn(0x7f45b0070af0 msgr2=0x7f45b015c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:26.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 shutdown_connections 2023-06-09T11:45:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:26.798+0000 7f45b87ff700 1 -- 172.21.15.7:0/4145000286 wait complete. 2023-06-09T11:45:26.803 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 160 2023-06-09T11:45:26.820 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:45:26.820 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:45:26.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.016+0000 7f1a247ff700 1 Processor -- start 2023-06-09T11:45:27.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.016+0000 7f1a247ff700 1 -- start start 2023-06-09T11:45:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.017+0000 7f1a247ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1a1c153880 0x7f1a1c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.017+0000 7f1a247ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.017+0000 7f1a247ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c15a500 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.017+0000 7f1a247ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c0c5f10 con 0x7f1a1c153880 2023-06-09T11:45:27.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.017+0000 7f1a247ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c15da00 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39260/0 (socket says 172.21.15.7:39260) 2023-06-09T11:45:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 -- 172.21.15.7:0/1858229523 learned_addr learned my addr 172.21.15.7:0/1858229523 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a2259b700 1 --2- 172.21.15.7:0/1858229523 >> v2:172.21.15.117:3300/0 conn(0x7f1a1c153880 0x7f1a1c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 -- 172.21.15.7:0/1858229523 >> v2:172.21.15.117:3300/0 conn(0x7f1a1c153880 msgr2=0x7f1a1c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 --2- 172.21.15.7:0/1858229523 >> v2:172.21.15.117:3300/0 conn(0x7f1a1c153880 0x7f1a1c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 -- 172.21.15.7:0/1858229523 >> v1:172.21.15.117:6789/0 conn(0x7f1a1c0c9f40 legacy=0x7f1a1c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.018+0000 7f1a22d9c700 1 -- 172.21.15.7:0/1858229523 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c1596e0 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a22d9c700 1 --2- 172.21.15.7:0/1858229523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009e50 tx=0x7f1a1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6ce6fce5ae67a6e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a21599700 1 -- 172.21.15.7:0/1858229523 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400e070 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a21599700 1 -- 172.21.15.7:0/1858229523 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1400b910 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a21599700 1 -- 172.21.15.7:0/1858229523 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400bcc0 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 -- 172.21.15.7:0/1858229523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 msgr2=0x7f1a1c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 --2- 172.21.15.7:0/1858229523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009e50 tx=0x7f1a1400b330 comp rx=0 tx=0).stop 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 -- 172.21.15.7:0/1858229523 shutdown_connections 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 --2- 172.21.15.7:0/1858229523 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c0c4e30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 --2- 172.21.15.7:0/1858229523 >> v2:172.21.15.117:3300/0 conn(0x7f1a1c153880 0x7f1a1c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.019+0000 7f1a247ff700 1 -- 172.21.15.7:0/1858229523 >> 172.21.15.7:0/1858229523 conn(0x7f1a1c0737f0 msgr2=0x7f1a1c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.020+0000 7f1a247ff700 1 -- 172.21.15.7:0/1858229523 shutdown_connections 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.020+0000 7f1a247ff700 1 -- 172.21.15.7:0/1858229523 wait complete. 2023-06-09T11:45:27.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.021+0000 7f1a247ff700 1 Processor -- start 2023-06-09T11:45:27.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.021+0000 7f1a247ff700 1 -- start start 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1c153880 0x7f1a1c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0cc480 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c151140 con 0x7f1a1c153880 2023-06-09T11:45:27.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a247ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c1525f0 con 0x7f1a1c15b4a0 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.022+0000 7f1a2259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57848/0 (socket says 172.21.15.7:57848) 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 -- 172.21.15.7:0/3994885230 learned_addr learned my addr 172.21.15.7:0/3994885230 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a21d9a700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1c153880 0x7f1a1c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a22d9c700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 -- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 msgr2=0x7f1a1c148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 -- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1c153880 msgr2=0x7f1a1c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1c153880 0x7f1a1c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 -- 172.21.15.7:0/3994885230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a14009b30 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.023+0000 7f1a2259b700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e9e0 tx=0x7f1a1800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.024+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a18009710 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.024+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a1800c850 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.024+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a1800cbc0 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.024+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c159c60 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.024+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c159980 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.025+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c159f40 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.026+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a1800cd20 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.027+0000 7f1a12ffd700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 0x7f1a0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.027+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f1a1808a0d0 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.027+0000 7f1a21d9a700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 0x7f1a0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.027+0000 7f1a21d9a700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 0x7f1a0006d190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c14bc50 tx=0x7f1a0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.031+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1a18058440 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.282+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a1c0cd3a0 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.283+0000 7f1a12ffd700 1 -- 172.21.15.7:0/3994885230 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f1a1c0cd3a0 con 0x7f1a1c0c9f40 2023-06-09T11:45:27.286 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:27.286 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":160,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:25.601471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 msgr2=0x7f1a0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 0x7f1a0006d190 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c14bc50 tx=0x7f1a0c009040 comp rx=0 tx=0).stop 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 msgr2=0x7f1a1c151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800e9e0 tx=0x7f1a1800eda0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 shutdown_connections 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a0006ace0 0x7f1a0006d190 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a1c15b4a0 0x7f1a1c148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a1c0c9f40 0x7f1a1c151a90 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 --2- 172.21.15.7:0/3994885230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a1c153880 0x7f1a1c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 >> 172.21.15.7:0/3994885230 conn(0x7f1a1c0737f0 msgr2=0x7f1a1c0cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.285+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 shutdown_connections 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:27.286+0000 7f1a247ff700 1 -- 172.21.15.7:0/3994885230 wait complete. 2023-06-09T11:45:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 160 2023-06-09T11:45:27.307 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 160, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 160, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 30 }, "gid_7141": { "addr": "172.21.15.117:6837/3727664816", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3727664816, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3727664816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7141, "incarnation": 159, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 68, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:45:25.601471+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7141, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:45:29.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:29.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.515+0000 7f0b437ff700 1 Processor -- start 2023-06-09T11:45:29.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.515+0000 7f0b437ff700 1 -- start start 2023-06-09T11:45:29.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b437ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 0x7f0b3c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b437ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 0x7f0b3c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b437ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c1580a0 con 0x7f0b3c15b380 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b437ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c1518e0 con 0x7f0b3c071230 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b437ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c6180 con 0x7f0b3c151510 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.516+0000 7f0b40d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0b3c15b380 0x7f0b3c070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39346/0 (socket says 172.21.15.7:39346) 2023-06-09T11:45:29.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b40d9a700 1 -- 172.21.15.7:0/3292324038 learned_addr learned my addr 172.21.15.7:0/3292324038 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 --2- 172.21.15.7:0/3292324038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 0x7f0b3c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b41d9c700 1 --2- 172.21.15.7:0/3292324038 >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 0x7f0b3c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b33fff700 1 -- 172.21.15.7:0/3292324038 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095048376 0 0) 0x7f0b3c1580a0 con 0x7f0b3c15b380 2023-06-09T11:45:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b33fff700 1 -- 172.21.15.7:0/3292324038 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b280035b0 con 0x7f0b3c15b380 2023-06-09T11:45:29.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 -- 172.21.15.7:0/3292324038 >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 msgr2=0x7f0b3c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 --2- 172.21.15.7:0/3292324038 >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 0x7f0b3c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 -- 172.21.15.7:0/3292324038 >> v1:172.21.15.7:6789/0 conn(0x7f0b3c15b380 legacy=0x7f0b3c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 -- 172.21.15.7:0/3292324038 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c157490 con 0x7f0b3c151510 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b41d9c700 1 --2- 172.21.15.7:0/3292324038 >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 0x7f0b3c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.517+0000 7f0b4159b700 1 --2- 172.21.15.7:0/3292324038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 0x7f0b3c0c5b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009840 tx=0x7f0b38009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8109e82e5eaef733 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b33fff700 1 -- 172.21.15.7:0/3292324038 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38011830 con 0x7f0b3c151510 2023-06-09T11:45:29.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b33fff700 1 -- 172.21.15.7:0/3292324038 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38011990 con 0x7f0b3c151510 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b33fff700 1 -- 172.21.15.7:0/3292324038 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38011d00 con 0x7f0b3c151510 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 -- 172.21.15.7:0/3292324038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 msgr2=0x7f0b3c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 --2- 172.21.15.7:0/3292324038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 0x7f0b3c0c5b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009840 tx=0x7f0b38009b50 comp rx=0 tx=0).stop 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 -- 172.21.15.7:0/3292324038 shutdown_connections 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 --2- 172.21.15.7:0/3292324038 >> v2:172.21.15.117:3300/0 conn(0x7f0b3c071230 0x7f0b3c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 --2- 172.21.15.7:0/3292324038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c151510 0x7f0b3c0c5b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 -- 172.21.15.7:0/3292324038 >> 172.21.15.7:0/3292324038 conn(0x7f0b3c06a4d0 msgr2=0x7f0b3c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:29.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.518+0000 7f0b437ff700 1 -- 172.21.15.7:0/3292324038 shutdown_connections 2023-06-09T11:45:29.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.519+0000 7f0b437ff700 1 -- 172.21.15.7:0/3292324038 wait complete. 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.521+0000 7f0b437ff700 1 Processor -- start 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.521+0000 7f0b437ff700 1 -- start start 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b3c071230 0x7f0b3c0c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c15b380 0x7f0b3c148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c1efa90 con 0x7f0b3c151510 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c0c85b0 con 0x7f0b3c071230 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.522+0000 7f0b437ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c1eee10 con 0x7f0b3c15b380 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57852/0 (socket says 172.21.15.7:57852) 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 -- 172.21.15.7:0/4181987323 learned_addr learned my addr 172.21.15.7:0/4181987323 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b41d9c700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c15b380 0x7f0b3c148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b4159b700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b3c071230 0x7f0b3c0c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 -- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c15b380 msgr2=0x7f0b3c148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c15b380 0x7f0b3c148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 -- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b3c071230 msgr2=0x7f0b3c0c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b3c071230 0x7f0b3c0c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 -- 172.21.15.7:0/4181987323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b34009a70 con 0x7f0b3c151510 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.523+0000 7f0b40d9a700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c450 tx=0x7f0b2c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.524+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c005d70 con 0x7f0b3c151510 2023-06-09T11:45:29.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.524+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c00ce10 con 0x7f0b3c151510 2023-06-09T11:45:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.524+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c0072d0 con 0x7f0b3c151510 2023-06-09T11:45:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.524+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b380094a0 con 0x7f0b3c151510 2023-06-09T11:45:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.524+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c157a50 con 0x7f0b3c151510 2023-06-09T11:45:29.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.525+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c157770 con 0x7f0b3c151510 2023-06-09T11:45:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.526+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b2c007430 con 0x7f0b3c151510 2023-06-09T11:45:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.527+0000 7f0b31ffb700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 0x7f0b2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.527+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f0b2c088e60 con 0x7f0b3c151510 2023-06-09T11:45:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.527+0000 7f0b4159b700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 0x7f0b2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:29.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.528+0000 7f0b4159b700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 0x7f0b2006d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b38008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:29.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.531+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0b2c0537f0 con 0x7f0b3c151510 2023-06-09T11:45:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.802+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b3c157d30 con 0x7f0b3c151510 2023-06-09T11:45:29.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.802+0000 7f0b31ffb700 1 -- 172.21.15.7:0/4181987323 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 160 v160) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f0b2c017030 con 0x7f0b3c151510 2023-06-09T11:45:29.806 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:29.806 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":160,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":160,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:25.601471+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 msgr2=0x7f0b2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 0x7f0b2006d190 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0b38000c00 tx=0x7f0b38008030 comp rx=0 tx=0).stop 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 msgr2=0x7f0b3c14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00c450 tx=0x7f0b2c00c760 comp rx=0 tx=0).stop 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 shutdown_connections 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b2006ace0 0x7f0b2006d190 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b3c15b380 0x7f0b3c148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b3c071230 0x7f0b3c0c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 --2- 172.21.15.7:0/4181987323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b3c151510 0x7f0b3c14e250 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 >> 172.21.15.7:0/4181987323 conn(0x7f0b3c06a4d0 msgr2=0x7f0b3c14b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 shutdown_connections 2023-06-09T11:45:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:29.805+0000 7f0b437ff700 1 -- 172.21.15.7:0/4181987323 wait complete. 2023-06-09T11:45:29.810 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 160 2023-06-09T11:45:29.826 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:31.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:32.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.035+0000 7fa977fff700 1 Processor -- start 2023-06-09T11:45:32.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.035+0000 7fa977fff700 1 -- start start 2023-06-09T11:45:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.036+0000 7fa977fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9700c6280 0x7fa9700c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.036+0000 7fa977fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa97006e460 0x7fa97006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.036+0000 7fa977fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97015cac0 con 0x7fa97006a4d0 2023-06-09T11:45:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.036+0000 7fa977fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9700c8870 con 0x7fa9700c6280 2023-06-09T11:45:32.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.036+0000 7fa977fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9700c37d0 con 0x7fa97006e460 2023-06-09T11:45:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9700c6280 0x7fa9700c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa9700c6280 0x7fa9700c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57868/0 (socket says 172.21.15.7:57868) 2023-06-09T11:45:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 -- 172.21.15.7:0/1259672235 learned_addr learned my addr 172.21.15.7:0/1259672235 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:32.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa97659c700 1 --2- 172.21.15.7:0/1259672235 >> v2:172.21.15.117:3300/0 conn(0x7fa97006e460 0x7fa97006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 -- 172.21.15.7:0/1259672235 >> v2:172.21.15.117:3300/0 conn(0x7fa97006e460 msgr2=0x7fa97006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:32.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 --2- 172.21.15.7:0/1259672235 >> v2:172.21.15.117:3300/0 conn(0x7fa97006e460 0x7fa97006cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 -- 172.21.15.7:0/1259672235 >> v1:172.21.15.117:6789/0 conn(0x7fa97006a4d0 legacy=0x7fa9700737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 -- 172.21.15.7:0/1259672235 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa97015cd30 con 0x7fa9700c6280 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa974d99700 1 -- 172.21.15.7:0/1259672235 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026970505 0 0) 0x7fa97015cac0 con 0x7fa97006a4d0 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.037+0000 7fa975d9b700 1 --2- 172.21.15.7:0/1259672235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9700c6280 0x7fa9700c76b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa97013f730 tx=0x7fa96c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a531b4282f914233 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa974d99700 1 -- 172.21.15.7:0/1259672235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c011830 con 0x7fa9700c6280 2023-06-09T11:45:32.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa974d99700 1 -- 172.21.15.7:0/1259672235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c011990 con 0x7fa9700c6280 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa974d99700 1 -- 172.21.15.7:0/1259672235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c011d20 con 0x7fa9700c6280 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 -- 172.21.15.7:0/1259672235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9700c6280 msgr2=0x7fa9700c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 --2- 172.21.15.7:0/1259672235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9700c6280 0x7fa9700c76b0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fa97013f730 tx=0x7fa96c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 -- 172.21.15.7:0/1259672235 shutdown_connections 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 --2- 172.21.15.7:0/1259672235 >> v2:172.21.15.117:3300/0 conn(0x7fa97006e460 0x7fa97006cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 --2- 172.21.15.7:0/1259672235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9700c6280 0x7fa9700c76b0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 -- 172.21.15.7:0/1259672235 >> 172.21.15.7:0/1259672235 conn(0x7fa97006f980 msgr2=0x7fa9700cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 -- 172.21.15.7:0/1259672235 shutdown_connections 2023-06-09T11:45:32.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.038+0000 7fa977fff700 1 -- 172.21.15.7:0/1259672235 wait complete. 2023-06-09T11:45:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.040+0000 7fa977fff700 1 Processor -- start 2023-06-09T11:45:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.040+0000 7fa977fff700 1 -- start start 2023-06-09T11:45:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 0x7fa97015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 0x7fa9701f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97015c270 con 0x7fa97006e460 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97015ab30 con 0x7fa9700c6280 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa977fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9701f0550 con 0x7fa97006a4d0 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa97559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa97559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57870/0 (socket says 172.21.15.7:57870) 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.041+0000 7fa97559a700 1 -- 172.21.15.7:0/2635008609 learned_addr learned my addr 172.21.15.7:0/2635008609 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:32.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa975d9b700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 0x7fa97015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97659c700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 0x7fa9701f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 -- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 msgr2=0x7fa97015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 0x7fa97015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 -- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 msgr2=0x7fa9701f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 0x7fa9701f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 -- 172.21.15.7:0/2635008609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c009550 con 0x7fa97006e460 2023-06-09T11:45:32.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa975d9b700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 0x7fa97015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97659c700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 0x7fa9701f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa97559a700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa9600011e0 tx=0x7fa96000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9600054e0 con 0x7fa97006e460 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa960003680 con 0x7fa97006e460 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960003a30 con 0x7fa97006e460 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97015d2f0 con 0x7fa97006e460 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.042+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97015d010 con 0x7fa97006e460 2023-06-09T11:45:32.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.043+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97015d5d0 con 0x7fa97006e460 2023-06-09T11:45:32.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.044+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa960003b90 con 0x7fa97006e460 2023-06-09T11:45:32.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.045+0000 7fa9667fc700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 0x7fa95406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:32.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.045+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fa960089150 con 0x7fa97006e460 2023-06-09T11:45:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.046+0000 7fa975d9b700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 0x7fa95406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:32.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.046+0000 7fa975d9b700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 0x7fa95406d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:32.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.050+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa960053a30 con 0x7fa97006e460 2023-06-09T11:45:32.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.299+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa970156670 con 0x7fa97006e460 2023-06-09T11:45:32.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.300+0000 7fa9667fc700 1 -- 172.21.15.7:0/2635008609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5461 (secure 0 0 0) 0x7fa960017030 con 0x7fa97006e460 2023-06-09T11:45:32.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:32.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:31.810125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:32.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 msgr2=0x7fa95406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 0x7fa95406d190 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa96c000c00 tx=0x7fa96c008030 comp rx=0 tx=0).stop 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 msgr2=0x7fa9701f0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa9600011e0 tx=0x7fa96000cae0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 shutdown_connections 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9700c6280 0x7fa9701f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa95406ace0 0x7fa95406d190 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa97006a4d0 0x7fa97015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 --2- 172.21.15.7:0/2635008609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa97006e460 0x7fa9701f0010 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 >> 172.21.15.7:0/2635008609 conn(0x7fa97006f980 msgr2=0x7fa9700c43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.302+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 shutdown_connections 2023-06-09T11:45:32.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:32.303+0000 7fa977fff700 1 -- 172.21.15.7:0/2635008609 wait complete. 2023-06-09T11:45:32.308 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 161 2023-06-09T11:45:32.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:34.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.532+0000 7f8fc1dff700 1 Processor -- start 2023-06-09T11:45:34.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.532+0000 7f8fc1dff700 1 -- start start 2023-06-09T11:45:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.533+0000 7f8fc1dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.533+0000 7f8fc1dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 0x7f8fbc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.533+0000 7f8fc1dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc0c0bf0 con 0x7f8fbc0c1bc0 2023-06-09T11:45:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.533+0000 7f8fc1dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc070110 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.533+0000 7f8fc1dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc071300 con 0x7f8fbc0c0140 2023-06-09T11:45:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43704/0 (socket says 172.21.15.7:43704) 2023-06-09T11:45:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 -- 172.21.15.7:0/3506787240 learned_addr learned my addr 172.21.15.7:0/3506787240 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:34.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbbfff700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 0x7f8fbc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 -- 172.21.15.7:0/3506787240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 msgr2=0x7f8fbc1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 0x7f8fbc1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 -- 172.21.15.7:0/3506787240 >> v1:172.21.15.117:6789/0 conn(0x7f8fbc0c1bc0 legacy=0x7f8fbc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbaffd700 1 -- 172.21.15.7:0/3506787240 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc0702f0 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.534+0000 7f8fbbfff700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 0x7f8fbc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fbaffd700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bbc0 tx=0x7f8fb000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61867f0e041cac3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fba7fc700 1 -- 172.21.15.7:0/3506787240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000dd50 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fba7fc700 1 -- 172.21.15.7:0/3506787240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0010470 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fba7fc700 1 -- 172.21.15.7:0/3506787240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0010800 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 -- 172.21.15.7:0/3506787240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 msgr2=0x7f8fbc1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bbc0 tx=0x7f8fb000bed0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 -- 172.21.15.7:0/3506787240 shutdown_connections 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c0140 0x7f8fbc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/3506787240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1501b0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.535+0000 7f8fc1dff700 1 -- 172.21.15.7:0/3506787240 >> 172.21.15.7:0/3506787240 conn(0x7f8fbc06a4d0 msgr2=0x7f8fbc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.536+0000 7f8fc1dff700 1 -- 172.21.15.7:0/3506787240 shutdown_connections 2023-06-09T11:45:34.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.536+0000 7f8fc1dff700 1 -- 172.21.15.7:0/3506787240 wait complete. 2023-06-09T11:45:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.537+0000 7f8fc1dff700 1 Processor -- start 2023-06-09T11:45:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.537+0000 7f8fc1dff700 1 -- start start 2023-06-09T11:45:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc0c0140 0x7f8fbc1eeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c1bc0 0x7f8fbc14ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c2ab0 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc156680 con 0x7f8fbc0c0140 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.538+0000 7f8fc1dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc155790 con 0x7f8fbc0c1bc0 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43706/0 (socket says 172.21.15.7:43706) 2023-06-09T11:45:34.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 -- 172.21.15.7:0/2277178907 learned_addr learned my addr 172.21.15.7:0/2277178907 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbaffd700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c1bc0 0x7f8fbc14ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbb7fe700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc0c0140 0x7f8fbc1eeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 -- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c1bc0 msgr2=0x7f8fbc14ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c1bc0 0x7f8fbc14ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 -- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc0c0140 msgr2=0x7f8fbc1eeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc0c0140 0x7f8fbc1eeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 -- 172.21.15.7:0/2277178907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb000b7f0 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fbbfff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ef0 tx=0x7f8fb400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.539+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400de60 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.540+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400f040 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.540+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003c60 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.540+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0c1170 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.540+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc0c0e90 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.541+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc0c1450 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.542+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8fb4004030 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.542+0000 7f8fa3fff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 0x7f8fa806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:34.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.543+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f8fb4089e70 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.543+0000 7f8fbb7fe700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 0x7f8fa806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:34.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.543+0000 7f8fbb7fe700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 0x7f8fa806d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000f60 tx=0x7f8fac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:34.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.547+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8fb4054750 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.798+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fbc1567f0 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.799+0000 7f8fa3fff700 1 -- 172.21.15.7:0/2277178907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 161 v161) v1 ==== 95+0+5461 (secure 0 0 0) 0x7f8fb401a070 con 0x7f8fbc15a0c0 2023-06-09T11:45:34.802 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:34.802 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":161,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":161,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:31.810125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:34.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 msgr2=0x7f8fa806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 0x7f8fa806d190 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8fac000f60 tx=0x7f8fac009040 comp rx=0 tx=0).stop 2023-06-09T11:45:34.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 msgr2=0x7f8fbc1f0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:34.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4007ef0 tx=0x7f8fb400d3b0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 shutdown_connections 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8fbc15a0c0 0x7f8fbc1f0010 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8fbc0c0140 0x7f8fbc1eeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8fa806ace0 0x7f8fa806d190 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 --2- 172.21.15.7:0/2277178907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8fbc0c1bc0 0x7f8fbc14ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 >> 172.21.15.7:0/2277178907 conn(0x7f8fbc06a4d0 msgr2=0x7f8fbc14ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 shutdown_connections 2023-06-09T11:45:34.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:34.802+0000 7f8fc1dff700 1 -- 172.21.15.7:0/2277178907 wait complete. 2023-06-09T11:45:34.807 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 161 2023-06-09T11:45:34.823 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:36.823 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.030+0000 7fc7745ff700 1 Processor -- start 2023-06-09T11:45:37.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.030+0000 7fc7745ff700 1 -- start start 2023-06-09T11:45:37.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc7745ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc7745ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc76c0cb9a0 0x7fc76c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc7745ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc76c159b20 con 0x7fc76c15cd40 2023-06-09T11:45:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc7745ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc76c0cac80 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc7745ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1597d0 con 0x7fc76c0cb5d0 2023-06-09T11:45:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.031+0000 7fc771b9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc76c0cb9a0 0x7fc76c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc772b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc76c15cd40 0x7fc76c0cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54006/0 (socket says 172.21.15.7:54006) 2023-06-09T11:45:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc772b9c700 1 -- 172.21.15.7:0/2924946341 learned_addr learned my addr 172.21.15.7:0/2924946341 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc77239b700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771b9a700 1 -- 172.21.15.7:0/2924946341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 msgr2=0x7fc76c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771b9a700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771b9a700 1 -- 172.21.15.7:0/2924946341 >> v1:172.21.15.7:6789/0 conn(0x7fc76c15cd40 legacy=0x7fc76c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771b9a700 1 -- 172.21.15.7:0/2924946341 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc76c071ab0 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771b9a700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c159fb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009840 tx=0x7fc75c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0234c49fa40ad41 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.032+0000 7fc771399700 1 -- 172.21.15.7:0/2924946341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c011830 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc771399700 1 -- 172.21.15.7:0/2924946341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c011990 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc771399700 1 -- 172.21.15.7:0/2924946341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c011d20 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 -- 172.21.15.7:0/2924946341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c0cb9a0 msgr2=0x7fc76c159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c159fb0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009840 tx=0x7fc75c009b50 comp rx=0 tx=0).stop 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 -- 172.21.15.7:0/2924946341 shutdown_connections 2023-06-09T11:45:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 --2- 172.21.15.7:0/2924946341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c159fb0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 -- 172.21.15.7:0/2924946341 >> 172.21.15.7:0/2924946341 conn(0x7fc76c070af0 msgr2=0x7fc76c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 -- 172.21.15.7:0/2924946341 shutdown_connections 2023-06-09T11:45:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.033+0000 7fc7745ff700 1 -- 172.21.15.7:0/2924946341 wait complete. 2023-06-09T11:45:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.035+0000 7fc7745ff700 1 Processor -- start 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.035+0000 7fc7745ff700 1 -- start start 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c1e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c1e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1e5230 con 0x7fc76c15cd40 2023-06-09T11:45:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c1e5590 con 0x7fc76c0cb9a0 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc7745ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76c0c8b10 con 0x7fc76c0cb5d0 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc772b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc772b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43726/0 (socket says 172.21.15.7:43726) 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc772b9c700 1 -- 172.21.15.7:0/1800526108 learned_addr learned my addr 172.21.15.7:0/1800526108 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc77239b700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c1e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.036+0000 7fc771b9a700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c1e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 -- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 msgr2=0x7fc76c1e4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c1e4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 -- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc76c0cb9a0 msgr2=0x7fc76c1e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c1e5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 -- 172.21.15.7:0/1800526108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75c0094a0 con 0x7fc76c15cd40 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc772b9c700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ca20 tx=0x7fc76400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400a9a0 con 0x7fc76c15cd40 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76400ab00 con 0x7fc76c15cd40 2023-06-09T11:45:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76400ae70 con 0x7fc76c15cd40 2023-06-09T11:45:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc76c15d4f0 con 0x7fc76c15cd40 2023-06-09T11:45:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.037+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc76c15d210 con 0x7fc76c15cd40 2023-06-09T11:45:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.038+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc76c15d7d0 con 0x7fc76c15cd40 2023-06-09T11:45:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.039+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc76401b3b0 con 0x7fc76c15cd40 2023-06-09T11:45:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.040+0000 7fc762ffd700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 0x7fc75006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.040+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fc7640894b0 con 0x7fc76c15cd40 2023-06-09T11:45:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.041+0000 7fc77239b700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 0x7fc75006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:37.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.041+0000 7fc77239b700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 0x7fc75006d1c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc768006b20 tx=0x7fc768008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:37.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.045+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc764053f60 con 0x7fc76c15cd40 2023-06-09T11:45:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.298+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc76c074cf0 con 0x7fc76c15cd40 2023-06-09T11:45:37.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.299+0000 7fc762ffd700 1 -- 172.21.15.7:0/1800526108 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 162 v162) v1 ==== 95+0+5460 (secure 0 0 0) 0x7fc764057580 con 0x7fc76c15cd40 2023-06-09T11:45:37.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:37.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":162,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":162,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:35.594112+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:37.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 msgr2=0x7fc75006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 0x7fc75006d1c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc768006b20 tx=0x7fc768008040 comp rx=0 tx=0).stop 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 msgr2=0x7fc76c1f0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ca20 tx=0x7fc76400cd30 comp rx=0 tx=0).stop 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 shutdown_connections 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc76c15cd40 0x7fc76c1f0490 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc75006ad10 0x7fc75006d1c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc76c0cb5d0 0x7fc76c1e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 --2- 172.21.15.7:0/1800526108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc76c0cb9a0 0x7fc76c1e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 >> 172.21.15.7:0/1800526108 conn(0x7fc76c070af0 msgr2=0x7fc76c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 shutdown_connections 2023-06-09T11:45:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:37.301+0000 7fc7745ff700 1 -- 172.21.15.7:0/1800526108 wait complete. 2023-06-09T11:45:37.307 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 162 2023-06-09T11:45:37.323 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:39.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.531+0000 7f1fa35ff700 1 Processor -- start 2023-06-09T11:45:39.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.531+0000 7f1fa35ff700 1 -- start start 2023-06-09T11:45:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.532+0000 7f1fa35ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.532+0000 7f1fa35ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0c5a80 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.532+0000 7f1fa35ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c15a500 con 0x7f1f9c153880 2023-06-09T11:45:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.532+0000 7f1fa35ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c5f10 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa0b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f9c15b4a0 0x7f1f9c0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54018/0 (socket says 172.21.15.7:54018) 2023-06-09T11:45:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa0b9a700 1 -- 172.21.15.7:0/3528995781 learned_addr learned my addr 172.21.15.7:0/3528995781 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:39.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa1b9c700 1 --2- 172.21.15.7:0/3528995781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912797897 0 0) 0x7f1f9c0c5a80 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f880035b0 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912290036 0 0) 0x7f1f98001110 con 0x7f1f9c153880 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9c0c5a80 con 0x7f1f9c153880 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/3528995781 >> v1:172.21.15.117:6789/0 conn(0x7f1f9c153880 legacy=0x7f1f9c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/3528995781 >> v1:172.21.15.7:6789/0 conn(0x7f1f9c15b4a0 legacy=0x7f1f9c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.533+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/3528995781 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c1596e0 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa1b9c700 1 --2- 172.21.15.7:0/3528995781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0c5540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009dc0 tx=0x7f1f9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12db52d92ecb52b8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400e070 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9400bb20 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1f93fff700 1 -- 172.21.15.7:0/3528995781 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94012430 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa35ff700 1 -- 172.21.15.7:0/3528995781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 msgr2=0x7f1f9c0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/3528995781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0c5540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009dc0 tx=0x7f1f9400b330 comp rx=0 tx=0).stop 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa35ff700 1 -- 172.21.15.7:0/3528995781 shutdown_connections 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/3528995781 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0c5540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.534+0000 7f1fa35ff700 1 -- 172.21.15.7:0/3528995781 >> 172.21.15.7:0/3528995781 conn(0x7f1f9c0737f0 msgr2=0x7f1f9c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.535+0000 7f1fa35ff700 1 -- 172.21.15.7:0/3528995781 shutdown_connections 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.535+0000 7f1fa35ff700 1 -- 172.21.15.7:0/3528995781 wait complete. 2023-06-09T11:45:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.537+0000 7f1fa35ff700 1 Processor -- start 2023-06-09T11:45:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.537+0000 7f1fa35ff700 1 -- start start 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c153880 0x7f1f9c0ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1ef520 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1e4030 con 0x7f1f9c153880 2023-06-09T11:45:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.538+0000 7f1fa35ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1e3d50 con 0x7f1f9c0c9f40 2023-06-09T11:45:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43736/0 (socket says 172.21.15.7:43736) 2023-06-09T11:45:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/2043334111 learned_addr learned my addr 172.21.15.7:0/2043334111 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa0b9a700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c153880 0x7f1f9c0ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa139b700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 msgr2=0x7f1f9c0b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c153880 msgr2=0x7f1f9c0ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c153880 0x7f1f9c0ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 -- 172.21.15.7:0/2043334111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94009a70 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.539+0000 7f1fa1b9c700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b370 tx=0x7f1f94004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.540+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9401b810 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.540+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94020070 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.540+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c159c60 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.540+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9401bbb0 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.540+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c159980 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.541+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c159f40 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.542+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f9401bd10 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.543+0000 7f1f91ffb700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 0x7f1f8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.543+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f1f94022070 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.543+0000 7f1fa139b700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 0x7f1f8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:39.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.544+0000 7f1fa139b700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 0x7f1f8006d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1f980013b0 tx=0x7f1f98009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.547+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f94056480 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.803+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f9c1ef6b0 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.803+0000 7f1f91ffb700 1 -- 172.21.15.7:0/2043334111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f1f9402c070 con 0x7f1f9c15b4a0 2023-06-09T11:45:39.808 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:39.808 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:37.601177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 msgr2=0x7f1f8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 0x7f1f8006d190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1f980013b0 tx=0x7f1f98009040 comp rx=0 tx=0).stop 2023-06-09T11:45:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 msgr2=0x7f1f9c151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b370 tx=0x7f1f94004ad0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 shutdown_connections 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f9c15b4a0 0x7f1f9c151b10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f8006ace0 0x7f1f8006d190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f9c0c9f40 0x7f1f9c0b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 --2- 172.21.15.7:0/2043334111 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f9c153880 0x7f1f9c0ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 >> 172.21.15.7:0/2043334111 conn(0x7f1f9c0737f0 msgr2=0x7f1f9c14dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 shutdown_connections 2023-06-09T11:45:39.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:39.806+0000 7f1fa35ff700 1 -- 172.21.15.7:0/2043334111 wait complete. 2023-06-09T11:45:39.812 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 163 2023-06-09T11:45:39.828 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 163, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 163, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6859": { "addr": "172.21.15.117:6835/477143596", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 477143596, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 477143596, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6859, "incarnation": 141, "join_fscid": -1, "name": "d", "rank": 2, "state": "up:active", "state_seq": 30 }, "gid_7141": { "addr": "172.21.15.117:6837/3727664816", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3727664816, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3727664816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7141, "incarnation": 159, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 14 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 68, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:45:37.601177+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7141, "mds_2": 6859 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:45:41.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.036+0000 7f4555fff700 1 Processor -- start 2023-06-09T11:45:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.036+0000 7f4555fff700 1 -- start start 2023-06-09T11:45:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f4555fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 0x7f455015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f4555fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 0x7f45500c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f4555fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455015a500 con 0x7f45500c9f40 2023-06-09T11:45:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f4555fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45500c5f10 con 0x7f4550153880 2023-06-09T11:45:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f4555fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455015da00 con 0x7f455015b4a0 2023-06-09T11:45:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f454effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f45500c9f40 0x7f455015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54024/0 (socket says 172.21.15.7:54024) 2023-06-09T11:45:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f454effd700 1 -- 172.21.15.7:0/1930871083 learned_addr learned my addr 172.21.15.7:0/1930871083 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:45:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f454f7fe700 1 --2- 172.21.15.7:0/1930871083 >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 0x7f455015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.037+0000 7f454ffff700 1 --2- 172.21.15.7:0/1930871083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 0x7f45500c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454e7fc700 1 -- 172.21.15.7:0/1930871083 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778365297 0 0) 0x7f4544001110 con 0x7f45500c9f40 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454e7fc700 1 -- 172.21.15.7:0/1930871083 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45380035b0 con 0x7f45500c9f40 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454ffff700 1 -- 172.21.15.7:0/1930871083 >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 msgr2=0x7f455015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454ffff700 1 --2- 172.21.15.7:0/1930871083 >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 0x7f455015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454ffff700 1 -- 172.21.15.7:0/1930871083 >> v1:172.21.15.7:6789/0 conn(0x7f45500c9f40 legacy=0x7f455015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454ffff700 1 -- 172.21.15.7:0/1930871083 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45501596e0 con 0x7f455015b4a0 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454f7fe700 1 --2- 172.21.15.7:0/1930871083 >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 0x7f455015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.038+0000 7f454ffff700 1 --2- 172.21.15.7:0/1930871083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 0x7f45500c4e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4548009d90 tx=0x7f454800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89ce11fe48c88651 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f454e7fc700 1 -- 172.21.15.7:0/1930871083 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800e070 con 0x7f455015b4a0 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f454e7fc700 1 -- 172.21.15.7:0/1930871083 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454800bb20 con 0x7f455015b4a0 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f454e7fc700 1 -- 172.21.15.7:0/1930871083 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454800be90 con 0x7f455015b4a0 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 -- 172.21.15.7:0/1930871083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 msgr2=0x7f45500c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 --2- 172.21.15.7:0/1930871083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 0x7f45500c4e30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4548009d90 tx=0x7f454800b330 comp rx=0 tx=0).stop 2023-06-09T11:45:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 -- 172.21.15.7:0/1930871083 shutdown_connections 2023-06-09T11:45:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 --2- 172.21.15.7:0/1930871083 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f455015b4a0 0x7f45500c4e30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 --2- 172.21.15.7:0/1930871083 >> v2:172.21.15.117:3300/0 conn(0x7f4550153880 0x7f455015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 -- 172.21.15.7:0/1930871083 >> 172.21.15.7:0/1930871083 conn(0x7f45500737f0 msgr2=0x7f45500720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 -- 172.21.15.7:0/1930871083 shutdown_connections 2023-06-09T11:45:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.039+0000 7f4555fff700 1 -- 172.21.15.7:0/1930871083 wait complete. 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.041+0000 7f4555fff700 1 Processor -- start 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.041+0000 7f4555fff700 1 -- start start 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4550153880 0x7f45501eec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455015b4a0 0x7f4550158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500cf120 con 0x7f45500c9f40 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b3880 con 0x7f455015b4a0 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f4555fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45500b2fe0 con 0x7f4550153880 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f454f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f454f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43744/0 (socket says 172.21.15.7:43744) 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.042+0000 7f454f7fe700 1 -- 172.21.15.7:0/3217038939 learned_addr learned my addr 172.21.15.7:0/3217038939 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454effd700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4550153880 0x7f45501eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 -- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4550153880 msgr2=0x7f45501eec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4550153880 0x7f45501eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 -- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455015b4a0 msgr2=0x7f4550158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455015b4a0 0x7f4550158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 -- 172.21.15.7:0/3217038939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400094a0 con 0x7f45500c9f40 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f454f7fe700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f454000ebe0 tx=0x7f4540009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000c910 con 0x7f45500c9f40 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000ca70 con 0x7f45500c9f40 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4548009a70 con 0x7f45500c9f40 2023-06-09T11:45:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000cde0 con 0x7f45500c9f40 2023-06-09T11:45:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.043+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4550159c60 con 0x7f45500c9f40 2023-06-09T11:45:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.044+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4550159980 con 0x7f45500c9f40 2023-06-09T11:45:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.045+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4540013470 con 0x7f45500c9f40 2023-06-09T11:45:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.047+0000 7f4537fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 0x7f453c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.047+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f454008a0d0 con 0x7f45500c9f40 2023-06-09T11:45:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.047+0000 7f454effd700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 0x7f453c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.048+0000 7f454effd700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 0x7f453c06d1c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4544007fd0 tx=0x7f4544009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:42.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.052+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4540054a40 con 0x7f45500c9f40 2023-06-09T11:45:42.311 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.307+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4550159f40 con 0x7f45500c9f40 2023-06-09T11:45:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.308+0000 7f4537fff700 1 -- 172.21.15.7:0/3217038939 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 163 v163) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f4540058060 con 0x7f45500c9f40 2023-06-09T11:45:42.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:42.312 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":163,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":163,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:37.601177+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:42.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 msgr2=0x7f453c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 0x7f453c06d1c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4544007fd0 tx=0x7f4544009040 comp rx=0 tx=0).stop 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 msgr2=0x7f45501571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f454000ebe0 tx=0x7f4540009ec0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 shutdown_connections 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f455015b4a0 0x7f4550158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45500c9f40 0x7f45501571a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f453c06ad10 0x7f453c06d1c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 --2- 172.21.15.7:0/3217038939 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4550153880 0x7f45501eec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 >> 172.21.15.7:0/3217038939 conn(0x7f45500737f0 msgr2=0x7f455014d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 shutdown_connections 2023-06-09T11:45:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:42.310+0000 7f4555fff700 1 -- 172.21.15.7:0/3217038939 wait complete. 2023-06-09T11:45:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 163 2023-06-09T11:45:42.332 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:44.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.541+0000 7f1cca5ff700 1 Processor -- start 2023-06-09T11:45:44.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.541+0000 7f1cca5ff700 1 -- start start 2023-06-09T11:45:44.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc41593d0 0x7f1cc41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1cc40c08d0 0x7f1cc4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cc4071910 0x7f1cc4071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc40714f0 con 0x7f1cc4071910 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc40cc140 con 0x7f1cc40c08d0 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cca5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c1830 con 0x7f1cc41593d0 2023-06-09T11:45:44.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.542+0000 7f1cc8b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cc4071910 0x7f1cc4071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1cc4071910 0x7f1cc4071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36242/0 (socket says 172.21.15.7:36242) 2023-06-09T11:45:44.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2659774044 learned_addr learned my addr 172.21.15.7:0/2659774044 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:44.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc37fe700 1 --2- 172.21.15.7:0/2659774044 >> v2:172.21.15.117:3300/0 conn(0x7f1cc40c08d0 0x7f1cc4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc3fff700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc41593d0 0x7f1cc41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2659774044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc41593d0 msgr2=0x7f1cc41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc41593d0 0x7f1cc41597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2659774044 >> v2:172.21.15.117:3300/0 conn(0x7f1cc40c08d0 msgr2=0x7f1cc4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2659774044 >> v2:172.21.15.117:3300/0 conn(0x7f1cc40c08d0 0x7f1cc4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2659774044 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc40c0560 con 0x7f1cc4071910 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4071910 0x7f1cc4071d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1cc406f740 tx=0x7f1cbc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac73bebd9412d95c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc2ffd700 1 -- 172.21.15.7:0/2659774044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00e070 con 0x7f1cc4071910 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc2ffd700 1 -- 172.21.15.7:0/2659774044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00b910 con 0x7f1cc4071910 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.543+0000 7f1cc2ffd700 1 -- 172.21.15.7:0/2659774044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00bcc0 con 0x7f1cc4071910 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2659774044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4071910 msgr2=0x7f1cc4071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:44.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4071910 0x7f1cc4071d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f1cc406f740 tx=0x7f1cbc00b330 comp rx=0 tx=0).stop 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2659774044 shutdown_connections 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc4071910 0x7f1cc4071d50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2659774044 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc41593d0 0x7f1cc41597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2659774044 >> v2:172.21.15.117:3300/0 conn(0x7f1cc40c08d0 0x7f1cc4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2659774044 >> 172.21.15.7:0/2659774044 conn(0x7f1cc406a4d0 msgr2=0x7f1cc406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2659774044 shutdown_connections 2023-06-09T11:45:44.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.544+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2659774044 wait complete. 2023-06-09T11:45:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.546+0000 7f1cca5ff700 1 Processor -- start 2023-06-09T11:45:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.546+0000 7f1cca5ff700 1 -- start start 2023-06-09T11:45:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc4071910 0x7f1cc4146030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc40c08d0 0x7f1cc40ca290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc40c32f0 con 0x7f1cc41593d0 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4158320 con 0x7f1cc4071910 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cca5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc41585e0 con 0x7f1cc40c08d0 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cc8b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cc8b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36258/0 (socket says 172.21.15.7:36258) 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2933183501 learned_addr learned my addr 172.21.15.7:0/2933183501 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cc37fe700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc40c08d0 0x7f1cc40ca290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.547+0000 7f1cc3fff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc4071910 0x7f1cc4146030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc40c08d0 msgr2=0x7f1cc40ca290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc40c08d0 0x7f1cc40ca290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc4071910 msgr2=0x7f1cc4146030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc4071910 0x7f1cc4146030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 -- 172.21.15.7:0/2933183501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc009b30 con 0x7f1cc41593d0 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc8b9c700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc406fbf0 tx=0x7f1cbc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc0099b0 con 0x7f1cc41593d0 2023-06-09T11:45:44.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00ce80 con 0x7f1cc41593d0 2023-06-09T11:45:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc40c1a80 con 0x7f1cc41593d0 2023-06-09T11:45:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc004490 con 0x7f1cc41593d0 2023-06-09T11:45:44.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.548+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc4072290 con 0x7f1cc41593d0 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.549+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc40c1d60 con 0x7f1cc41593d0 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.550+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cbc0045f0 con 0x7f1cc41593d0 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.551+0000 7f1cc0ff9700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 0x7f1cb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.551+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f1cbc08ade0 con 0x7f1cc41593d0 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.551+0000 7f1cc3fff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 0x7f1cb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.552+0000 7f1cc3fff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 0x7f1cb006d1c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009b40 tx=0x7f1cb4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:44.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.555+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1cbc055900 con 0x7f1cc41593d0 2023-06-09T11:45:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.809+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1cc4074cf0 con 0x7f1cc41593d0 2023-06-09T11:45:44.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.810+0000 7f1cc0ff9700 1 -- 172.21.15.7:0/2933183501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f1cbc02a090 con 0x7f1cc41593d0 2023-06-09T11:45:44.815 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:44.815 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:42.620654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:44.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 msgr2=0x7f1cb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:44.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 0x7f1cb006d1c0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009b40 tx=0x7f1cb4009210 comp rx=0 tx=0).stop 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 msgr2=0x7f1cc40ca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1cc406fbf0 tx=0x7f1cbc004ad0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 shutdown_connections 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cc41593d0 0x7f1cc40ca7d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1cc4071910 0x7f1cc4146030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb006ad10 0x7f1cb006d1c0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 --2- 172.21.15.7:0/2933183501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1cc40c08d0 0x7f1cc40ca290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 >> 172.21.15.7:0/2933183501 conn(0x7f1cc406a4d0 msgr2=0x7f1cc40c5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 shutdown_connections 2023-06-09T11:45:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:44.812+0000 7f1cca5ff700 1 -- 172.21.15.7:0/2933183501 wait complete. 2023-06-09T11:45:44.818 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 164 2023-06-09T11:45:44.834 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:46.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.041+0000 7fe8bcfff700 1 Processor -- start 2023-06-09T11:45:47.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.041+0000 7fe8bcfff700 1 -- start start 2023-06-09T11:45:47.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b815d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c9260 0x7fe8b80cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8b80b5370 0x7fe8b80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b81597d0 con 0x7fe8b80b5370 2023-06-09T11:45:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b815dc90 con 0x7fe8b80c9260 2023-06-09T11:45:47.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.042+0000 7fe8bcfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80cfaf0 con 0x7fe8b80c8e90 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8b80b5370 0x7fe8b80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8b80b5370 0x7fe8b80c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36268/0 (socket says 172.21.15.7:36268) 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 -- 172.21.15.7:0/3070526482 learned_addr learned my addr 172.21.15.7:0/3070526482 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b659c700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8affff700 1 --2- 172.21.15.7:0/3070526482 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c9260 0x7fe8b80cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 -- 172.21.15.7:0/3070526482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 msgr2=0x7fe8b815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 -- 172.21.15.7:0/3070526482 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c9260 msgr2=0x7fe8b80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 --2- 172.21.15.7:0/3070526482 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c9260 0x7fe8b80cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.043+0000 7fe8b6d9d700 1 -- 172.21.15.7:0/3070526482 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b80cfdd0 con 0x7fe8b80b5370 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8b6d9d700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c9fe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009e50 tx=0x7fe8b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c90bb78b1ca49e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8b5d9b700 1 -- 172.21.15.7:0/3070526482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000e070 con 0x7fe8b80b5370 2023-06-09T11:45:47.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8b5d9b700 1 -- 172.21.15.7:0/3070526482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b000b8d0 con 0x7fe8b80b5370 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8b5d9b700 1 -- 172.21.15.7:0/3070526482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000bc80 con 0x7fe8b80b5370 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 -- 172.21.15.7:0/3070526482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 msgr2=0x7fe8b80c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c9fe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009e50 tx=0x7fe8b000b330 comp rx=0 tx=0).stop 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 -- 172.21.15.7:0/3070526482 shutdown_connections 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c9fe0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/3070526482 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/3070526482 >> v2:172.21.15.117:3300/0 conn(0x7fe8b80c9260 0x7fe8b80cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.044+0000 7fe8bcfff700 1 -- 172.21.15.7:0/3070526482 >> 172.21.15.7:0/3070526482 conn(0x7fe8b8070af0 msgr2=0x7fe8b8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.045+0000 7fe8bcfff700 1 -- 172.21.15.7:0/3070526482 shutdown_connections 2023-06-09T11:45:47.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.045+0000 7fe8bcfff700 1 -- 172.21.15.7:0/3070526482 wait complete. 2023-06-09T11:45:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.046+0000 7fe8bcfff700 1 Processor -- start 2023-06-09T11:45:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.046+0000 7fe8bcfff700 1 -- start start 2023-06-09T11:45:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b8157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c9260 0x7fe8b81f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80ab160 con 0x7fe8b80b5370 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81f0440 con 0x7fe8b80c9260 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.047+0000 7fe8bcfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b80aba10 con 0x7fe8b80c8e90 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36274/0 (socket says 172.21.15.7:36274) 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 -- 172.21.15.7:0/1396523091 learned_addr learned my addr 172.21.15.7:0/1396523091 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b6d9d700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c9260 0x7fe8b81f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8affff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 -- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 msgr2=0x7fe8b8157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b8157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 -- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c9260 msgr2=0x7fe8b81f08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c9260 0x7fe8b81f08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 -- 172.21.15.7:0/1396523091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b0009b30 con 0x7fe8b80b5370 2023-06-09T11:45:47.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8b659c700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800e960 tx=0x7fe8a800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.048+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8009690 con 0x7fe8b80b5370 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.049+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a800c850 con 0x7fe8b80b5370 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.049+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b8159ca0 con 0x7fe8b80b5370 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.049+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800cbc0 con 0x7fe8b80b5370 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.049+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b81599c0 con 0x7fe8b80b5370 2023-06-09T11:45:47.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.050+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b8159f80 con 0x7fe8b80b5370 2023-06-09T11:45:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.051+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8a800cd20 con 0x7fe8b80b5370 2023-06-09T11:45:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.052+0000 7fe8aeffd700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 0x7fe89806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.052+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fe8a8089ce0 con 0x7fe8b80b5370 2023-06-09T11:45:47.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.052+0000 7fe8affff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 0x7fe89806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:47.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.053+0000 7fe8affff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 0x7fe89806d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c3130 tx=0x7fe8a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:47.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.056+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe8a8058050 con 0x7fe8b80b5370 2023-06-09T11:45:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.310+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8b815cc50 con 0x7fe8b80b5370 2023-06-09T11:45:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.310+0000 7fe8aeffd700 1 -- 172.21.15.7:0/1396523091 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 164 v164) v1 ==== 95+0+5459 (secure 0 0 0) 0x7fe8a8017020 con 0x7fe8b80b5370 2023-06-09T11:45:47.315 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:47.315 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":164,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":164,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:42.620654+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.312+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 msgr2=0x7fe89806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.312+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 0x7fe89806d190 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fe8b80c3130 tx=0x7fe8a0008040 comp rx=0 tx=0).stop 2023-06-09T11:45:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.312+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 msgr2=0x7fe8b80c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.312+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800e960 tx=0x7fe8a800ed20 comp rx=0 tx=0).stop 2023-06-09T11:45:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 shutdown_connections 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe8b80c9260 0x7fe8b81f08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8b80b5370 0x7fe8b80c81e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe89806ace0 0x7fe89806d190 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 --2- 172.21.15.7:0/1396523091 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8b80c8e90 0x7fe8b8157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 >> 172.21.15.7:0/1396523091 conn(0x7fe8b8070af0 msgr2=0x7fe8b8155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 shutdown_connections 2023-06-09T11:45:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:47.313+0000 7fe8bcfff700 1 -- 172.21.15.7:0/1396523091 wait complete. 2023-06-09T11:45:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 164 2023-06-09T11:45:47.335 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:45:47.787 INFO:tasks.ceph.mds.e.smithi007.stderr:2023-06-09T11:45:47.782+0000 7fbaf65f1700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-06-09T11:45:49.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:49.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.541+0000 7f6ed8bff700 1 Processor -- start 2023-06-09T11:45:49.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.541+0000 7f6ed8bff700 1 -- start start 2023-06-09T11:45:49.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed8bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ed4153880 0x7f6ed415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:49.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed8bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed415a500 con 0x7f6ed415b4a0 2023-06-09T11:45:49.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed8bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed415da00 con 0x7f6ed40c9f40 2023-06-09T11:45:49.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed8bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed40c5f10 con 0x7f6ed4153880 2023-06-09T11:45:49.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ed4153880 0x7f6ed415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:49.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6ed4153880 0x7f6ed415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36282/0 (socket says 172.21.15.7:36282) 2023-06-09T11:45:49.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 -- 172.21.15.7:0/2401668041 learned_addr learned my addr 172.21.15.7:0/2401668041 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:49.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 -- 172.21.15.7:0/2401668041 >> v1:172.21.15.117:6789/0 conn(0x7f6ed40c9f40 legacy=0x7f6ed415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 -- 172.21.15.7:0/2401668041 >> v1:172.21.15.7:6789/0 conn(0x7f6ed415b4a0 legacy=0x7f6ed40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.543+0000 7f6ed259c700 1 -- 172.21.15.7:0/2401668041 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed41596e0 con 0x7f6ed4153880 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed259c700 1 --2- 172.21.15.7:0/2401668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed4153880 0x7f6ed415d4c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098f0 tx=0x7f6ec4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86869361c053d437 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed1d9b700 1 -- 172.21.15.7:0/2401668041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400b8a0 con 0x7f6ed4153880 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed1d9b700 1 -- 172.21.15.7:0/2401668041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec400ba00 con 0x7f6ed4153880 2023-06-09T11:45:49.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed1d9b700 1 -- 172.21.15.7:0/2401668041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400bd90 con 0x7f6ed4153880 2023-06-09T11:45:49.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed8bff700 1 -- 172.21.15.7:0/2401668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed4153880 msgr2=0x7f6ed415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.544+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/2401668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed4153880 0x7f6ed415d4c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098f0 tx=0x7f6ec4009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.545+0000 7f6ed8bff700 1 -- 172.21.15.7:0/2401668041 shutdown_connections 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.545+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/2401668041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed4153880 0x7f6ed415d4c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.545+0000 7f6ed8bff700 1 -- 172.21.15.7:0/2401668041 >> 172.21.15.7:0/2401668041 conn(0x7f6ed40737f0 msgr2=0x7f6ed40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.545+0000 7f6ed8bff700 1 -- 172.21.15.7:0/2401668041 shutdown_connections 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.545+0000 7f6ed8bff700 1 -- 172.21.15.7:0/2401668041 wait complete. 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.546+0000 7f6ed8bff700 1 Processor -- start 2023-06-09T11:45:49.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.546+0000 7f6ed8bff700 1 -- start start 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 0x7f6ed40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 0x7f6ed4157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed41efe20 con 0x7f6ed415b4a0 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed41eefb0 con 0x7f6ed4153880 2023-06-09T11:45:49.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.547+0000 7f6ed8bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed415c9b0 con 0x7f6ed40c9f40 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36298/0 (socket says 172.21.15.7:36298) 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 -- 172.21.15.7:0/1269677030 learned_addr learned my addr 172.21.15.7:0/1269677030 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed259c700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 0x7f6ed40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ecbfff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 0x7f6ed4157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 -- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 msgr2=0x7f6ed40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 0x7f6ed40b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 -- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 msgr2=0x7f6ed4157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 0x7f6ed4157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 -- 172.21.15.7:0/1269677030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4009550 con 0x7f6ed415b4a0 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed259c700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 0x7f6ed40b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ecbfff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 0x7f6ed4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed2d9d700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cbb0 tx=0x7f6ecc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc003c00 con 0x7f6ed415b4a0 2023-06-09T11:45:49.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc003d60 con 0x7f6ed415b4a0 2023-06-09T11:45:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.548+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed4159c60 con 0x7f6ed415b4a0 2023-06-09T11:45:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.549+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00ab70 con 0x7f6ed415b4a0 2023-06-09T11:45:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.549+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed4159980 con 0x7f6ed415b4a0 2023-06-09T11:45:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.550+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed4159f40 con 0x7f6ed415b4a0 2023-06-09T11:45:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.551+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ecc00acd0 con 0x7f6ed415b4a0 2023-06-09T11:45:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.551+0000 7f6ecaffd700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 0x7f6eb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.552+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f6ecc089a70 con 0x7f6ed415b4a0 2023-06-09T11:45:49.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.552+0000 7f6ed259c700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 0x7f6eb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:49.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.552+0000 7f6ed259c700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 0x7f6eb406d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098f0 tx=0x7f6ec4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:49.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.556+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6ecc0543e0 con 0x7f6ed415b4a0 2023-06-09T11:45:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.812+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ed415a500 con 0x7f6ed415b4a0 2023-06-09T11:45:49.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.812+0000 7f6ecaffd700 1 -- 172.21.15.7:0/1269677030 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f6ed415a500 con 0x7f6ed415b4a0 2023-06-09T11:45:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:49.818 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":165,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:47.762655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:49.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 msgr2=0x7f6eb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 0x7f6eb406d190 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40098f0 tx=0x7f6ec4008030 comp rx=0 tx=0).stop 2023-06-09T11:45:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 msgr2=0x7f6ed4158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00cbb0 tx=0x7f6ecc00cf70 comp rx=0 tx=0).stop 2023-06-09T11:45:49.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 shutdown_connections 2023-06-09T11:45:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ed415b4a0 0x7f6ed4158340 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6eb406ace0 0x7f6eb406d190 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6ed40c9f40 0x7f6ed40b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 --2- 172.21.15.7:0/1269677030 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6ed4153880 0x7f6ed4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 >> 172.21.15.7:0/1269677030 conn(0x7f6ed40737f0 msgr2=0x7f6ed414b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 shutdown_connections 2023-06-09T11:45:49.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:49.815+0000 7f6ed8bff700 1 -- 172.21.15.7:0/1269677030 wait complete. 2023-06-09T11:45:49.824 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 165 2023-06-09T11:45:49.837 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=6718 2023-06-09T11:45:49.838 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 2 secs before reviving mds.b 2023-06-09T11:45:50.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:50.715 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:45:51.839 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:45:51.840 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:45:51.840 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:45:51.845 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:45:51.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:51.996 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:45:52.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.053+0000 7efd92fff700 1 Processor -- start 2023-06-09T11:45:52.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.053+0000 7efd92fff700 1 -- start start 2023-06-09T11:45:52.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd92fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd8c0c2d90 0x7efd8c0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:52.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd92fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c0c88d0 con 0x7efd8c15a0f0 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd92fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8c153e40 con 0x7efd8c153a70 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd92fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8c0c2bf0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd8bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd8c0c2d90 0x7efd8c0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd8bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7efd8c0c2d90 0x7efd8c0c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36308/0 (socket says 172.21.15.7:36308) 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd9159c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efd8c15a0f0 0x7efd8c0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59800/0 (socket says 172.21.15.7:59800) 2023-06-09T11:45:52.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.054+0000 7efd8bfff700 1 -- 172.21.15.7:0/1162190924 learned_addr learned my addr 172.21.15.7:0/1162190924 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8b7fe700 1 -- 172.21.15.7:0/1162190924 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168473570 0 0) 0x7efd80001110 con 0x7efd8c153a70 2023-06-09T11:45:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8bfff700 1 -- 172.21.15.7:0/1162190924 >> v1:172.21.15.117:6789/0 conn(0x7efd8c153a70 legacy=0x7efd8c0ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8bfff700 1 -- 172.21.15.7:0/1162190924 >> v1:172.21.15.7:6789/0 conn(0x7efd8c15a0f0 legacy=0x7efd8c0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8bfff700 1 -- 172.21.15.7:0/1162190924 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c0ca350 con 0x7efd8c0c2d90 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8bfff700 1 --2- 172.21.15.7:0/1162190924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0c26b0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efd780098f0 tx=0x7efd78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a09487bc6c8d164 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8b7fe700 1 -- 172.21.15.7:0/1162190924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78004230 con 0x7efd8c0c2d90 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8b7fe700 1 -- 172.21.15.7:0/1162190924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd78004390 con 0x7efd8c0c2d90 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.055+0000 7efd8b7fe700 1 -- 172.21.15.7:0/1162190924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780036a0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 -- 172.21.15.7:0/1162190924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 msgr2=0x7efd8c0c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 --2- 172.21.15.7:0/1162190924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0c26b0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efd780098f0 tx=0x7efd78009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:52.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 -- 172.21.15.7:0/1162190924 shutdown_connections 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 --2- 172.21.15.7:0/1162190924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0c26b0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 -- 172.21.15.7:0/1162190924 >> 172.21.15.7:0/1162190924 conn(0x7efd8c06e410 msgr2=0x7efd8c06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 -- 172.21.15.7:0/1162190924 shutdown_connections 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.056+0000 7efd92fff700 1 -- 172.21.15.7:0/1162190924 wait complete. 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.058+0000 7efd92fff700 1 Processor -- start 2023-06-09T11:45:52.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.058+0000 7efd92fff700 1 -- start start 2023-06-09T11:45:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c153a70 0x7efd8c1eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 0x7efd8c1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:52.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c1e35f0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c1f24a0 con 0x7efd8c153a70 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd92fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c0b2290 con 0x7efd8c15a0f0 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36324/0 (socket says 172.21.15.7:36324) 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 -- 172.21.15.7:0/3430184108 learned_addr learned my addr 172.21.15.7:0/3430184108 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd8bfff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c153a70 0x7efd8c1eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:52.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd9159c700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 0x7efd8c1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 -- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 msgr2=0x7efd8c1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 0x7efd8c1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 -- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c153a70 msgr2=0x7efd8c1eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c153a70 0x7efd8c1eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd90d9b700 1 -- 172.21.15.7:0/3430184108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78009550 con 0x7efd8c0c2d90 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.059+0000 7efd9159c700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 0x7efd8c1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:45:52.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd90d9b700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c730 tx=0x7efd8000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800054e0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd80003680 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80003a30 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c0c85f0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.060+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c0c8350 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.061+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c0c9d00 con 0x7efd8c0c2d90 2023-06-09T11:45:52.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.062+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efd80003b90 con 0x7efd8c0c2d90 2023-06-09T11:45:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.063+0000 7efd897fa700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 0x7efd7006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.063+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7efd800889a0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.063+0000 7efd8bfff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 0x7efd7006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.064+0000 7efd8bfff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 0x7efd7006d1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efd78009130 tx=0x7efd78008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:52.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.068+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7efd800533a0 con 0x7efd8c0c2d90 2023-06-09T11:45:52.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.320+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd8c1f3300 con 0x7efd8c0c2d90 2023-06-09T11:45:52.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.321+0000 7efd897fa700 1 -- 172.21.15.7:0/3430184108 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 165 v165) v1 ==== 95+0+5459 (secure 0 0 0) 0x7efd80017090 con 0x7efd8c0c2d90 2023-06-09T11:45:52.325 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:52.326 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":165,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:47.762655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:52.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 msgr2=0x7efd7006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 0x7efd7006d1c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efd78009130 tx=0x7efd78008030 comp rx=0 tx=0).stop 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 msgr2=0x7efd8c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efd8000c730 tx=0x7efd8000ca40 comp rx=0 tx=0).stop 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 shutdown_connections 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7efd7006ad10 0x7efd7006d1c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efd8c15a0f0 0x7efd8c1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd8c0c2d90 0x7efd8c0cc5b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 --2- 172.21.15.7:0/3430184108 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efd8c153a70 0x7efd8c1eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 >> 172.21.15.7:0/3430184108 conn(0x7efd8c06e410 msgr2=0x7efd8c159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.323+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 shutdown_connections 2023-06-09T11:45:52.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:52.324+0000 7efd92fff700 1 -- 172.21.15.7:0/3430184108 wait complete. 2023-06-09T11:45:52.330 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 165 2023-06-09T11:45:52.345 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:45:54.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:45:54.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.565+0000 7f073d3ff700 1 Processor -- start 2023-06-09T11:45:54.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.565+0000 7f073d3ff700 1 -- start start 2023-06-09T11:45:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.566+0000 7f073d3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0738153880 0x7f073815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f073d3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073815a500 con 0x7f07380c9f40 2023-06-09T11:45:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f073d3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073815da00 con 0x7f073815b4a0 2023-06-09T11:45:54.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f073d3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380c5f10 con 0x7f0738153880 2023-06-09T11:45:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0738153880 0x7f073815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0738153880 0x7f073815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37222/0 (socket says 172.21.15.7:37222) 2023-06-09T11:45:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 -- 172.21.15.7:0/1862740621 learned_addr learned my addr 172.21.15.7:0/1862740621 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:54.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f072ffff700 1 --1- 172.21.15.7:0/1862740621 >> v1:172.21.15.7:6789/0 conn(0x7f07380c9f40 0x7f073815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50218/0 (socket says 172.21.15.7:50218) 2023-06-09T11:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 -- 172.21.15.7:0/1862740621 >> v1:172.21.15.117:6789/0 conn(0x7f073815b4a0 legacy=0x7f07380c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 -- 172.21.15.7:0/1862740621 >> v1:172.21.15.7:6789/0 conn(0x7f07380c9f40 legacy=0x7f073815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.567+0000 7f0736ffd700 1 -- 172.21.15.7:0/1862740621 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07381596e0 con 0x7f0738153880 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f0736ffd700 1 --2- 172.21.15.7:0/1862740621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f073815d4c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f07280098f0 tx=0x7f0728009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335eb998525b6793 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f07367fc700 1 -- 172.21.15.7:0/1862740621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728004230 con 0x7f0738153880 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f07367fc700 1 -- 172.21.15.7:0/1862740621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728004390 con 0x7f0738153880 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f07367fc700 1 -- 172.21.15.7:0/1862740621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07280036a0 con 0x7f0738153880 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f073d3ff700 1 -- 172.21.15.7:0/1862740621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 msgr2=0x7f073815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1862740621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f073815d4c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f07280098f0 tx=0x7f0728009c00 comp rx=0 tx=0).stop 2023-06-09T11:45:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f073d3ff700 1 -- 172.21.15.7:0/1862740621 shutdown_connections 2023-06-09T11:45:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1862740621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f073815d4c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.568+0000 7f073d3ff700 1 -- 172.21.15.7:0/1862740621 >> 172.21.15.7:0/1862740621 conn(0x7f07380737f0 msgr2=0x7f07380720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.569+0000 7f073d3ff700 1 -- 172.21.15.7:0/1862740621 shutdown_connections 2023-06-09T11:45:54.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.569+0000 7f073d3ff700 1 -- 172.21.15.7:0/1862740621 wait complete. 2023-06-09T11:45:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.570+0000 7f073d3ff700 1 Processor -- start 2023-06-09T11:45:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.570+0000 7f073d3ff700 1 -- start start 2023-06-09T11:45:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07380c9f40 0x7f07380b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:54.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 0x7f0738148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b54f0 con 0x7f0738153880 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b31f0 con 0x7f07380c9f40 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.571+0000 7f073d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b2f50 con 0x7f073815b4a0 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37232/0 (socket says 172.21.15.7:37232) 2023-06-09T11:45:54.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 -- 172.21.15.7:0/1781692697 learned_addr learned my addr 172.21.15.7:0/1781692697 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f0736ffd700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07380c9f40 0x7f07380b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f07377fe700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 0x7f0738148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 -- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 msgr2=0x7f0738148ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 0x7f0738148ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 -- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07380c9f40 msgr2=0x7f07380b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07380c9f40 0x7f07380b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 -- 172.21.15.7:0/1781692697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009550 con 0x7f0738153880 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f07377fe700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 0x7f0738148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072ffff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f072000c4c0 tx=0x7f072000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.572+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0720005cb0 con 0x7f0738153880 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.573+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0720005e10 con 0x7f0738153880 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.573+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200072d0 con 0x7f0738153880 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.573+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0738159c60 con 0x7f0738153880 2023-06-09T11:45:54.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.573+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738159980 con 0x7f0738153880 2023-06-09T11:45:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.574+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0738159f40 con 0x7f0738153880 2023-06-09T11:45:54.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.575+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0720007430 con 0x7f0738153880 2023-06-09T11:45:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.576+0000 7f072f7fe700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 0x7f071806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:45:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.576+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f0720088db0 con 0x7f0738153880 2023-06-09T11:45:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.576+0000 7f0736ffd700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 0x7f071806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:45:54.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.576+0000 7f0736ffd700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 0x7f071806d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0728009130 tx=0x7f0728008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:45:54.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.580+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f072008d070 con 0x7f0738153880 2023-06-09T11:45:54.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.832+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f073815a500 con 0x7f0738153880 2023-06-09T11:45:54.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.833+0000 7f072f7fe700 1 -- 172.21.15.7:0/1781692697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 166 v166) v1 ==== 95+0+6220 (secure 0 0 0) 0x7f073815a500 con 0x7f0738153880 2023-06-09T11:45:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:45:54.838 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":166,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":165,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:45:47.762655+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 msgr2=0x7f071806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 0x7f071806d1c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0728009130 tx=0x7f0728008030 comp rx=0 tx=0).stop 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 msgr2=0x7f07381e5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f072000c4c0 tx=0x7f072000c7d0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 shutdown_connections 2023-06-09T11:45:54.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f071806ad10 0x7f071806d1c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f073815b4a0 0x7f0738148ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07380c9f40 0x7f07380b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 --2- 172.21.15.7:0/1781692697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0738153880 0x7f07381e5d60 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.835+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 >> 172.21.15.7:0/1781692697 conn(0x7f07380737f0 msgr2=0x7f073815c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.836+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 shutdown_connections 2023-06-09T11:45:54.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:45:54.836+0000 7f073d3ff700 1 -- 172.21.15.7:0/1781692697 wait complete. 2023-06-09T11:45:54.842 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 166 2023-06-09T11:45:54.857 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:45:54.858 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 37 secs before thrashing 2023-06-09T11:45:58.069 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 8% (14/164) Updating files: 9% (15/164) Updating files: 10% (17/164) Updating files: 11% (19/164) Updating files: 12% (20/164) Updating files: 12% (21/164) Updating files: 13% (22/164) Updating files: 14% (23/164) Updating files: 15% (25/164) Updating files: 16% (27/164) Updating files: 17% (28/164) Updating files: 18% (30/164) Updating files: 19% (32/164) Updating files: 20% (33/164) Updating files: 21% (35/164) Updating files: 22% (37/164) Updating files: 23% (38/164) Updating files: 24% (40/164) Updating files: 25% (41/164) Updating files: 26% (43/164) Updating files: 27% (45/164) Updating files: 28% (46/164) Updating files: 29% (48/164) Updating files: 30% (50/164) Updating files: 31% (51/164) Updating files: 32% (53/164) Updating files: 33% (55/164) Updating files: 34% (56/164) Updating files: 35% (58/164) Updating files: 36% (60/164) Updating files: 37% (61/164) Updating files: 38% (63/164) Updating files: 39% (64/164) Updating files: 40% (66/164) Updating files: 41% (68/164) Updating files: 42% (69/164) Updating files: 43% (71/164) Updating files: 44% (73/164) Updating files: 45% (74/164) Updating files: 46% (76/164) Updating files: 47% (78/164) Updating files: 48% (79/164) Updating files: 49% (81/164) Updating files: 50% (82/164) Updating files: 51% (84/164) Updating files: 52% (86/164) Updating files: 53% (87/164) Updating files: 54% (89/164) Updating files: 55% (91/164) Updating files: 56% (92/164) Updating files: 57% (94/164) Updating files: 58% (96/164) Updating files: 59% (97/164) Updating files: 60% (99/164) Updating files: 61% (101/164) Updating files: 62% (102/164) Updating files: 63% (104/164) Updating files: 64% (105/164) Updating files: 65% (107/164) Updating files: 66% (109/164) Updating files: 67% (110/164) Updating files: 68% (112/164) Updating files: 69% (114/164) Updating files: 70% (115/164) Updating files: 71% (117/164) Updating files: 72% (119/164) Updating files: 73% (120/164) Updating files: 74% (122/164) Updating files: 75% (123/164) Updating files: 76% (125/164) Updating files: 77% (127/164) Updating files: 78% (128/164) Updating files: 79% (130/164) Updating files: 80% (132/164) Updating files: 81% (133/164) Updating files: 82% (135/164) Updating files: 83% (137/164) Updating files: 84% (138/164) Updating files: 85% (140/164) Updating files: 85% (141/164) Updating files: 86% (142/164) Updating files: 87% (143/164) Updating files: 88% (145/164) Updating files: 89% (146/164) Updating files: 90% (148/164) Updating files: 91% (150/164) Updating files: 92% (151/164) Updating files: 93% (153/164) Updating files: 94% (155/164) Updating files: 95% (156/164) Updating files: 96% (158/164) Updating files: 97% (160/164) Updating files: 98% (161/164) Updating files: 99% (163/164) Updating files: 100% (164/164) Updating files: 100% (164/164), done. 2023-06-09T11:46:00.576 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 007313fe844 v0.10 2023-06-09T11:46:00.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.10 2023-06-09T11:46:00.614 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:46:00.614 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 11 -eq 48 ']' 2023-06-09T11:46:00.614 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.11 2023-06-09T11:46:00.615 INFO:tasks.workunit.client.0.smithi007.stdout:v0.11 2023-06-09T11:46:00.615 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.11 2023-06-09T11:46:00.615 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.11 2023-06-09T11:46:08.391 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 14% (10/69) Updating files: 15% (11/69) Updating files: 17% (12/69) Updating files: 18% (13/69) Updating files: 20% (14/69) Updating files: 21% (15/69) Updating files: 23% (16/69) Updating files: 24% (17/69) Updating files: 26% (18/69) Updating files: 27% (19/69) Updating files: 28% (20/69) Updating files: 30% (21/69) Updating files: 31% (22/69) Updating files: 33% (23/69) Updating files: 34% (24/69) Updating files: 36% (25/69) Updating files: 37% (26/69) Updating files: 39% (27/69) Updating files: 40% (28/69) Updating files: 42% (29/69) Updating files: 43% (30/69) Updating files: 44% (31/69) Updating files: 46% (32/69) Updating files: 47% (33/69) Updating files: 49% (34/69) Updating files: 50% (35/69) Updating files: 52% (36/69) Updating files: 53% (37/69) Updating files: 55% (38/69) Updating files: 56% (39/69) Updating files: 57% (40/69) Updating files: 59% (41/69) Updating files: 60% (42/69) Updating files: 62% (43/69) Updating files: 63% (44/69) Updating files: 65% (45/69) Updating files: 66% (46/69) Updating files: 68% (47/69) Updating files: 69% (48/69) Updating files: 71% (49/69) Updating files: 72% (50/69) Updating files: 73% (51/69) Updating files: 75% (52/69) Updating files: 76% (53/69) Updating files: 78% (54/69) Updating files: 79% (55/69) Updating files: 81% (56/69) Updating files: 82% (57/69) Updating files: 84% (58/69) Updating files: 85% (59/69) Updating files: 86% (60/69) Updating files: 88% (61/69) Updating files: 89% (62/69) Updating files: 91% (63/69) Updating files: 92% (64/69) Updating files: 94% (65/69) Updating files: 95% (66/69) Updating files: 97% (67/69) Updating files: 98% (68/69) Updating files: 100% (69/69) Updating files: 100% (69/69), done. 2023-06-09T11:46:08.573 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 9fa6f5e7d09 v0.11 2023-06-09T11:46:08.578 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.11 2023-06-09T11:46:08.609 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:46:08.609 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 12 -eq 48 ']' 2023-06-09T11:46:08.609 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.12 2023-06-09T11:46:08.610 INFO:tasks.workunit.client.0.smithi007.stdout:v0.12 2023-06-09T11:46:08.610 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.12 2023-06-09T11:46:08.610 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.12 2023-06-09T11:46:15.562 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 12% (9/75) Updating files: 13% (10/75) Updating files: 14% (11/75) Updating files: 16% (12/75) Updating files: 17% (13/75) Updating files: 18% (14/75) Updating files: 20% (15/75) Updating files: 21% (16/75) Updating files: 22% (17/75) Updating files: 24% (18/75) Updating files: 25% (19/75) Updating files: 26% (20/75) Updating files: 28% (21/75) Updating files: 29% (22/75) Updating files: 30% (23/75) Updating files: 32% (24/75) Updating files: 33% (25/75) Updating files: 34% (26/75) Updating files: 36% (27/75) Updating files: 37% (28/75) Updating files: 38% (29/75) Updating files: 40% (30/75) Updating files: 41% (31/75) Updating files: 42% (32/75) Updating files: 44% (33/75) Updating files: 45% (34/75) Updating files: 46% (35/75) Updating files: 48% (36/75) Updating files: 49% (37/75) Updating files: 50% (38/75) Updating files: 52% (39/75) Updating files: 53% (40/75) Updating files: 54% (41/75) Updating files: 56% (42/75) Updating files: 57% (43/75) Updating files: 58% (44/75) Updating files: 60% (45/75) Updating files: 61% (46/75) Updating files: 62% (47/75) Updating files: 64% (48/75) Updating files: 65% (49/75) Updating files: 66% (50/75) Updating files: 68% (51/75) Updating files: 69% (52/75) Updating files: 70% (53/75) Updating files: 72% (54/75) Updating files: 73% (55/75) Updating files: 74% (56/75) Updating files: 76% (57/75) Updating files: 77% (58/75) Updating files: 78% (59/75) Updating files: 80% (60/75) Updating files: 81% (61/75) Updating files: 82% (62/75) Updating files: 84% (63/75) Updating files: 85% (64/75) Updating files: 86% (65/75) Updating files: 88% (66/75) Updating files: 89% (67/75) Updating files: 90% (68/75) Updating files: 92% (69/75) Updating files: 93% (70/75) Updating files: 94% (71/75) Updating files: 96% (72/75) Updating files: 97% (73/75) Updating files: 98% (74/75) Updating files: 100% (75/75) Updating files: 100% (75/75), done. 2023-06-09T11:46:16.418 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at f524150822a v0.12 2023-06-09T11:46:16.424 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.12 2023-06-09T11:46:16.454 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:46:16.454 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 13 -eq 48 ']' 2023-06-09T11:46:16.455 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.13 2023-06-09T11:46:16.455 INFO:tasks.workunit.client.0.smithi007.stdout:v0.13 2023-06-09T11:46:16.455 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.13 2023-06-09T11:46:16.455 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.13 2023-06-09T11:46:20.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:20.952 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:25.226 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 10% (14/133) Updating files: 11% (15/133) Updating files: 12% (16/133) Updating files: 13% (18/133) Updating files: 14% (19/133) Updating files: 15% (20/133) Updating files: 16% (22/133) Updating files: 17% (23/133) Updating files: 18% (24/133) Updating files: 19% (26/133) Updating files: 20% (27/133) Updating files: 21% (28/133) Updating files: 21% (29/133) Updating files: 22% (30/133) Updating files: 23% (31/133) Updating files: 24% (32/133) Updating files: 25% (34/133) Updating files: 26% (35/133) Updating files: 27% (36/133) Updating files: 28% (38/133) Updating files: 29% (39/133) Updating files: 30% (40/133) Updating files: 31% (42/133) Updating files: 32% (43/133) Updating files: 33% (44/133) Updating files: 34% (46/133) Updating files: 35% (47/133) Updating files: 36% (48/133) Updating files: 37% (50/133) Updating files: 38% (51/133) Updating files: 39% (52/133) Updating files: 40% (54/133) Updating files: 41% (55/133) Updating files: 42% (56/133) Updating files: 43% (58/133) Updating files: 44% (59/133) Updating files: 45% (60/133) Updating files: 46% (62/133) Updating files: 47% (63/133) Updating files: 48% (64/133) Updating files: 49% (66/133) Updating files: 50% (67/133) Updating files: 51% (68/133) Updating files: 52% (70/133) Updating files: 53% (71/133) Updating files: 54% (72/133) Updating files: 55% (74/133) Updating files: 56% (75/133) Updating files: 57% (76/133) Updating files: 58% (78/133) Updating files: 59% (79/133) Updating files: 60% (80/133) Updating files: 61% (82/133) Updating files: 62% (83/133) Updating files: 63% (84/133) Updating files: 64% (86/133) Updating files: 65% (87/133) Updating files: 66% (88/133) Updating files: 67% (90/133) Updating files: 68% (91/133) Updating files: 69% (92/133) Updating files: 70% (94/133) Updating files: 71% (95/133) Updating files: 72% (96/133) Updating files: 73% (98/133) Updating files: 74% (99/133) Updating files: 75% (100/133) Updating files: 76% (102/133) Updating files: 77% (103/133) Updating files: 78% (104/133) Updating files: 79% (106/133) Updating files: 80% (107/133) Updating files: 81% (108/133) Updating files: 82% (110/133) Updating files: 83% (111/133) Updating files: 84% (112/133) Updating files: 85% (114/133) Updating files: 86% (115/133) Updating files: 87% (116/133) Updating files: 88% (118/133) Updating files: 89% (119/133) Updating files: 90% (120/133) Updating files: 91% (122/133) Updating files: 92% (123/133) Updating files: 93% (124/133) Updating files: 94% (126/133) Updating files: 95% (127/133) Updating files: 96% (128/133) Updating files: 97% (130/133) Updating files: 98% (131/133) Updating files: 99% (132/133) Updating files: 100% (133/133) Updating files: 100% (133/133), done. 2023-06-09T11:46:25.403 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 2b32297267d v0.13 2023-06-09T11:46:25.408 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.13 2023-06-09T11:46:25.453 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:46:25.453 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 14 -eq 48 ']' 2023-06-09T11:46:25.454 INFO:tasks.workunit.client.0.smithi007.stdout:v0.14 2023-06-09T11:46:25.454 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.14 2023-06-09T11:46:25.454 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.14 2023-06-09T11:46:25.455 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.14 2023-06-09T11:46:31.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.061+0000 7f2eef1ff700 1 Processor -- start 2023-06-09T11:46:32.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.061+0000 7f2eef1ff700 1 -- start start 2023-06-09T11:46:32.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eef1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ee8151510 0x7f2ee80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eef1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ee815b380 0x7f2ee8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eef1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee81580a0 con 0x7f2ee8071060 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eef1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee81518e0 con 0x7f2ee8151510 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eef1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee8071480 con 0x7f2ee815b380 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.062+0000 7f2eecf9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ee8151510 0x7f2ee80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ee8151510 0x7f2ee80c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34000/0 (socket says 172.21.15.7:34000) 2023-06-09T11:46:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 -- 172.21.15.7:0/1438185159 learned_addr learned my addr 172.21.15.7:0/1438185159 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:32.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2ee7fff700 1 --2- 172.21.15.7:0/1438185159 >> v2:172.21.15.117:3300/0 conn(0x7f2ee815b380 0x7f2ee8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 -- 172.21.15.7:0/1438185159 >> v2:172.21.15.117:3300/0 conn(0x7f2ee815b380 msgr2=0x7f2ee8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 --2- 172.21.15.7:0/1438185159 >> v2:172.21.15.117:3300/0 conn(0x7f2ee815b380 0x7f2ee8070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 -- 172.21.15.7:0/1438185159 >> v1:172.21.15.117:6789/0 conn(0x7f2ee8071060 legacy=0x7f2ee8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 -- 172.21.15.7:0/1438185159 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80c6180 con 0x7f2ee8151510 2023-06-09T11:46:32.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2eecf9b700 1 --2- 172.21.15.7:0/1438185159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8151510 0x7f2ee80c5b40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2ee806b170 tx=0x7f2edc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745a39f74dc027a3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2ee77fe700 1 -- 172.21.15.7:0/1438185159 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011830 con 0x7f2ee8151510 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2ee77fe700 1 -- 172.21.15.7:0/1438185159 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc011990 con 0x7f2ee8151510 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.063+0000 7f2ee77fe700 1 -- 172.21.15.7:0/1438185159 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc011d20 con 0x7f2ee8151510 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 -- 172.21.15.7:0/1438185159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8151510 msgr2=0x7f2ee80c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/1438185159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8151510 0x7f2ee80c5b40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2ee806b170 tx=0x7f2edc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 -- 172.21.15.7:0/1438185159 shutdown_connections 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/1438185159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8151510 0x7f2ee80c5b40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/1438185159 >> v2:172.21.15.117:3300/0 conn(0x7f2ee815b380 0x7f2ee8070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 -- 172.21.15.7:0/1438185159 >> 172.21.15.7:0/1438185159 conn(0x7f2ee806a4d0 msgr2=0x7f2ee806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 -- 172.21.15.7:0/1438185159 shutdown_connections 2023-06-09T11:46:32.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.064+0000 7f2eef1ff700 1 -- 172.21.15.7:0/1438185159 wait complete. 2023-06-09T11:46:32.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.066+0000 7f2eef1ff700 1 Processor -- start 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.066+0000 7f2eef1ff700 1 -- start start 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 0x7f2ee80c1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 0x7f2ee80c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80c6670 con 0x7f2ee8071060 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80c6950 con 0x7f2ee8151510 2023-06-09T11:46:32.072 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eef1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80c6c30 con 0x7f2ee815b380 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34016/0 (socket says 172.21.15.7:34016) 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 -- 172.21.15.7:0/328721424 learned_addr learned my addr 172.21.15.7:0/328721424 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 -- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 msgr2=0x7f2ee80c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2ee7fff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 0x7f2ee80c1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eed79c700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 0x7f2ee80c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 0x7f2ee80c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 -- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 msgr2=0x7f2ee80c1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 0x7f2ee80c1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.067+0000 7f2eecf9b700 1 -- 172.21.15.7:0/328721424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc009550 con 0x7f2ee8071060 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2ee7fff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 0x7f2ee80c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2eed79c700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 0x7f2ee80c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2eecf9b700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004570 con 0x7f2ee8071060 2023-06-09T11:46:32.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc01f070 con 0x7f2ee8071060 2023-06-09T11:46:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee8157770 con 0x7f2ee8071060 2023-06-09T11:46:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0038e0 con 0x7f2ee8071060 2023-06-09T11:46:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.068+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee8157490 con 0x7f2ee8071060 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.069+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8157a50 con 0x7f2ee8071060 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.070+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2edc021030 con 0x7f2ee8071060 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.071+0000 7f2ee57fa700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 0x7f2ecc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.071+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f2edc08b050 con 0x7f2ee8071060 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.072+0000 7f2ee7fff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 0x7f2ecc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.072+0000 7f2ee7fff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 0x7f2ecc06d1c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80c56c0 tx=0x7f2ed4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.076+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2edc0559c0 con 0x7f2ee8071060 2023-06-09T11:46:32.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.326+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ee8074cf0 con 0x7f2ee8071060 2023-06-09T11:46:32.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.327+0000 7f2ee57fa700 1 -- 172.21.15.7:0/328721424 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f2edc015020 con 0x7f2ee8071060 2023-06-09T11:46:32.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:32.331 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 msgr2=0x7f2ecc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 0x7f2ecc06d1c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80c56c0 tx=0x7f2ed4008040 comp rx=0 tx=0).stop 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 msgr2=0x7f2ee81f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2edc000c00 tx=0x7f2edc004750 comp rx=0 tx=0).stop 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 shutdown_connections 2023-06-09T11:46:32.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ecc06ad10 0x7f2ecc06d1c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ee815b380 0x7f2ee80c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ee8071060 0x7f2ee81f2f20 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 --2- 172.21.15.7:0/328721424 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ee8151510 0x7f2ee80c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.329+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 >> 172.21.15.7:0/328721424 conn(0x7f2ee806a4d0 msgr2=0x7f2ee814cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.330+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 shutdown_connections 2023-06-09T11:46:32.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.330+0000 7f2eef1ff700 1 -- 172.21.15.7:0/328721424 wait complete. 2023-06-09T11:46:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:32.350 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=2) 2023-06-09T11:46:32.350 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:46:32.351 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:46:32.372 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:46:32.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:32.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.568+0000 7fc00e5ff700 1 Processor -- start 2023-06-09T11:46:32.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.568+0000 7fc00e5ff700 1 -- start start 2023-06-09T11:46:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc0080c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc00815b380 0x7fc008070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc008071060 0x7fc008158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0080714e0 con 0x7fc00815b380 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0081518e0 con 0x7fc008071060 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0081580a0 con 0x7fc008151510 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc0077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc00815b380 0x7fc008070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc0077fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc00815b380 0x7fc008070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34032/0 (socket says 172.21.15.7:34032) 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc0077fe700 1 -- 172.21.15.7:0/2989552394 learned_addr learned my addr 172.21.15.7:0/2989552394 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc00cb9c700 1 --2- 172.21.15.7:0/2989552394 >> v2:172.21.15.117:3300/0 conn(0x7fc008071060 0x7fc008158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc007fff700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc0080c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc0077fe700 1 -- 172.21.15.7:0/2989552394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 msgr2=0x7fc0080c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.570+0000 7fc0077fe700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc0080c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc0077fe700 1 -- 172.21.15.7:0/2989552394 >> v2:172.21.15.117:3300/0 conn(0x7fc008071060 msgr2=0x7fc008158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc0077fe700 1 --2- 172.21.15.7:0/2989552394 >> v2:172.21.15.117:3300/0 conn(0x7fc008071060 0x7fc008158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc0077fe700 1 -- 172.21.15.7:0/2989552394 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0080c6180 con 0x7fc00815b380 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc007fff700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc0080c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:46:32.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc0077fe700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00815b380 0x7fc008070b20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc0080adac0 tx=0x7fbffc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb5836122b3f45f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc006ffd700 1 -- 172.21.15.7:0/2989552394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011830 con 0x7fc00815b380 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc006ffd700 1 -- 172.21.15.7:0/2989552394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc011990 con 0x7fc00815b380 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc006ffd700 1 -- 172.21.15.7:0/2989552394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011d20 con 0x7fc00815b380 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc00e5ff700 1 -- 172.21.15.7:0/2989552394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00815b380 msgr2=0x7fc008070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.571+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00815b380 0x7fc008070b20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc0080adac0 tx=0x7fbffc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 -- 172.21.15.7:0/2989552394 shutdown_connections 2023-06-09T11:46:32.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/2989552394 >> v2:172.21.15.117:3300/0 conn(0x7fc008071060 0x7fc008158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc0080c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/2989552394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc00815b380 0x7fc008070b20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 -- 172.21.15.7:0/2989552394 >> 172.21.15.7:0/2989552394 conn(0x7fc00806a4d0 msgr2=0x7fc00806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 -- 172.21.15.7:0/2989552394 shutdown_connections 2023-06-09T11:46:32.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.572+0000 7fc00e5ff700 1 -- 172.21.15.7:0/2989552394 wait complete. 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.573+0000 7fc00e5ff700 1 Processor -- start 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.573+0000 7fc00e5ff700 1 -- start start 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc008158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 0x7fc0080c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00815a320 con 0x7fc008071060 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0080b0f40 con 0x7fc00815b380 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.574+0000 7fc00e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc008146be0 con 0x7fc008151510 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34042/0 (socket says 172.21.15.7:34042) 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 -- 172.21.15.7:0/1284336246 learned_addr learned my addr 172.21.15.7:0/1284336246 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc0077fe700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc008158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc00cb9c700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 0x7fc0080c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 -- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 msgr2=0x7fc008158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc008158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 -- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 msgr2=0x7fc0080c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 0x7fc0080c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 -- 172.21.15.7:0/1284336246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009550 con 0x7fc008071060 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc0077fe700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc008158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc00cb9c700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 0x7fc0080c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:32.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc007fff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e840 tx=0x7fbff800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.575+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8004230 con 0x7fc008071060 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.576+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8004390 con 0x7fc008071060 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.576+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80036a0 con 0x7fc008071060 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.576+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc008157770 con 0x7fc008071060 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.576+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc008157490 con 0x7fc008071060 2023-06-09T11:46:32.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.577+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc008157a50 con 0x7fc008071060 2023-06-09T11:46:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.578+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbff8003800 con 0x7fc008071060 2023-06-09T11:46:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.578+0000 7fc004ff9700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 0x7fbff406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.579+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fbff80895c0 con 0x7fc008071060 2023-06-09T11:46:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.579+0000 7fc0077fe700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 0x7fbff406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:32.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.579+0000 7fc0077fe700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 0x7fbff406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc008150b70 tx=0x7fbffc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:32.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.583+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbff8057a50 con 0x7fc008071060 2023-06-09T11:46:32.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.839+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc008159230 con 0x7fc008071060 2023-06-09T11:46:32.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.840+0000 7fc004ff9700 1 -- 172.21.15.7:0/1284336246 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7fbff80094c0 con 0x7fc008071060 2023-06-09T11:46:32.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:32.843 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:32.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 msgr2=0x7fbff406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 0x7fbff406d190 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc008150b70 tx=0x7fbffc008030 comp rx=0 tx=0).stop 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 msgr2=0x7fc008158410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbff800e840 tx=0x7fbff800ec00 comp rx=0 tx=0).stop 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 shutdown_connections 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc00815b380 0x7fc0080c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc008071060 0x7fc008158410 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbff406ace0 0x7fbff406d190 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 --2- 172.21.15.7:0/1284336246 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc008151510 0x7fc008158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 >> 172.21.15.7:0/1284336246 conn(0x7fc00806a4d0 msgr2=0x7fc00814cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.842+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 shutdown_connections 2023-06-09T11:46:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:32.843+0000 7fc00e5ff700 1 -- 172.21.15.7:0/1284336246 wait complete. 2023-06-09T11:46:32.847 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:32.863 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:46:34.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:35.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.071+0000 7fd8859ff700 1 Processor -- start 2023-06-09T11:46:35.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.071+0000 7fd8859ff700 1 -- start start 2023-06-09T11:46:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd8859ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd880153880 0x7fd88015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd8859ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015b4a0 0x7fd8800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd8859ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88015a500 con 0x7fd8800c9f40 2023-06-09T11:46:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd8859ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800c5f10 con 0x7fd880153880 2023-06-09T11:46:35.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd8859ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88015da00 con 0x7fd88015b4a0 2023-06-09T11:46:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd8800c9f40 0x7fd88015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58718/0 (socket says 172.21.15.7:58718) 2023-06-09T11:46:35.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87e7fc700 1 -- 172.21.15.7:0/2564812260 learned_addr learned my addr 172.21.15.7:0/2564812260 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87effd700 1 --2- 172.21.15.7:0/2564812260 >> v2:172.21.15.117:3300/0 conn(0x7fd880153880 0x7fd88015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87f7fe700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015b4a0 0x7fd8800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87dffb700 1 -- 172.21.15.7:0/2564812260 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663601460 0 0) 0x7fd874001110 con 0x7fd8800c9f40 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.073+0000 7fd87dffb700 1 -- 172.21.15.7:0/2564812260 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8680035b0 con 0x7fd8800c9f40 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87effd700 1 -- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015b4a0 msgr2=0x7fd8800c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87effd700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015b4a0 0x7fd8800c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87effd700 1 -- 172.21.15.7:0/2564812260 >> v1:172.21.15.7:6789/0 conn(0x7fd8800c9f40 legacy=0x7fd88015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87effd700 1 -- 172.21.15.7:0/2564812260 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8801596e0 con 0x7fd880153880 2023-06-09T11:46:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87effd700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd880153880 0x7fd88015d4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd870009840 tx=0x7fd870009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94717442f7836aa server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87dffb700 1 -- 172.21.15.7:0/2564812260 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870011830 con 0x7fd880153880 2023-06-09T11:46:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87dffb700 1 -- 172.21.15.7:0/2564812260 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd870011990 con 0x7fd880153880 2023-06-09T11:46:35.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.074+0000 7fd87dffb700 1 -- 172.21.15.7:0/2564812260 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870011d20 con 0x7fd880153880 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 -- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd880153880 msgr2=0x7fd88015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd880153880 0x7fd88015d4c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd870009840 tx=0x7fd870009b50 comp rx=0 tx=0).stop 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 -- 172.21.15.7:0/2564812260 shutdown_connections 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd88015b4a0 0x7fd8800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 --2- 172.21.15.7:0/2564812260 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd880153880 0x7fd88015d4c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 -- 172.21.15.7:0/2564812260 >> 172.21.15.7:0/2564812260 conn(0x7fd8800737f0 msgr2=0x7fd8800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 -- 172.21.15.7:0/2564812260 shutdown_connections 2023-06-09T11:46:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.075+0000 7fd8859ff700 1 -- 172.21.15.7:0/2564812260 wait complete. 2023-06-09T11:46:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.077+0000 7fd8859ff700 1 Processor -- start 2023-06-09T11:46:35.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.078+0000 7fd8859ff700 1 -- start start 2023-06-09T11:46:35.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 0x7fd8801e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd88015b4a0 0x7fd8800cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88014a130 con 0x7fd880153880 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801f3470 con 0x7fd88015b4a0 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd8859ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88014a3d0 con 0x7fd8800c9f40 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87e7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52886/0 (socket says 172.21.15.7:52886) 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87e7fc700 1 -- 172.21.15.7:0/3335985713 learned_addr learned my addr 172.21.15.7:0/3335985713 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87effd700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 0x7fd8801e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87f7fe700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd88015b4a0 0x7fd8800cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87e7fc700 1 -- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 msgr2=0x7fd8801e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.079+0000 7fd87e7fc700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 0x7fd8801e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd87e7fc700 1 -- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd88015b4a0 msgr2=0x7fd8800cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd87e7fc700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd88015b4a0 0x7fd8800cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd87e7fc700 1 -- 172.21.15.7:0/3335985713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700094a0 con 0x7fd880153880 2023-06-09T11:46:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd87effd700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 0x7fd8801e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:46:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd87e7fc700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4d0 tx=0x7fd87400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874005d70 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8740070d0 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874007480 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880159c60 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.080+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880159980 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.081+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880159f40 con 0x7fd880153880 2023-06-09T11:46:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.082+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8740075e0 con 0x7fd880153880 2023-06-09T11:46:35.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.083+0000 7fd867fff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 0x7fd86c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:35.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.083+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7fd874088ff0 con 0x7fd880153880 2023-06-09T11:46:35.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.084+0000 7fd87effd700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 0x7fd86c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:35.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.084+0000 7fd87effd700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 0x7fd86c06d1c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:35.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.088+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd8740bb2d0 con 0x7fd880153880 2023-06-09T11:46:35.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.341+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd88015a500 con 0x7fd880153880 2023-06-09T11:46:35.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.341+0000 7fd867fff700 1 -- 172.21.15.7:0/3335985713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7fd88015a500 con 0x7fd880153880 2023-06-09T11:46:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:35.345 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 msgr2=0x7fd86c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 0x7fd86c06d1c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd870008030 comp rx=0 tx=0).stop 2023-06-09T11:46:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 msgr2=0x7fd8800cbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:35.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd87400c4d0 tx=0x7fd87400c890 comp rx=0 tx=0).stop 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 shutdown_connections 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd88015b4a0 0x7fd8800cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd86c06ad10 0x7fd86c06d1c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd8800c9f40 0x7fd8801e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 --2- 172.21.15.7:0/3335985713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd880153880 0x7fd8800cbe00 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 >> 172.21.15.7:0/3335985713 conn(0x7fd8800737f0 msgr2=0x7fd88015b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 shutdown_connections 2023-06-09T11:46:35.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:35.344+0000 7fd8859ff700 1 -- 172.21.15.7:0/3335985713 wait complete. 2023-06-09T11:46:35.349 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:35.364 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:46:37.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.572+0000 7f7d0b1ff700 1 Processor -- start 2023-06-09T11:46:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.572+0000 7f7d0b1ff700 1 -- start start 2023-06-09T11:46:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.573+0000 7f7d0b1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7d041593d0 0x7f7d041597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.573+0000 7f7d0b1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d04072020 0x7f7d04159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.573+0000 7f7d0b1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040cc140 con 0x7f7d040c08d0 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.573+0000 7f7d0b1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040714f0 con 0x7f7d04072020 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.573+0000 7f7d0b1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040c1830 con 0x7f7d041593d0 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d04072020 0x7f7d04159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7d04072020 0x7f7d04159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52894/0 (socket says 172.21.15.7:52894) 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 -- 172.21.15.7:0/409772882 learned_addr learned my addr 172.21.15.7:0/409772882 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d08f9b700 1 --2- 172.21.15.7:0/409772882 >> v2:172.21.15.117:3300/0 conn(0x7f7d041593d0 0x7f7d041597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 -- 172.21.15.7:0/409772882 >> v2:172.21.15.117:3300/0 conn(0x7f7d041593d0 msgr2=0x7f7d041597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:37.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 --2- 172.21.15.7:0/409772882 >> v2:172.21.15.117:3300/0 conn(0x7f7d041593d0 0x7f7d041597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 -- 172.21.15.7:0/409772882 >> v1:172.21.15.7:6789/0 conn(0x7f7d040c08d0 legacy=0x7f7d04071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 -- 172.21.15.7:0/409772882 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040c0560 con 0x7f7d04072020 2023-06-09T11:46:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d037fe700 1 -- 172.21.15.7:0/409772882 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891000555 0 0) 0x7f7d040cc140 con 0x7f7d040c08d0 2023-06-09T11:46:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.574+0000 7f7d0979c700 1 --2- 172.21.15.7:0/409772882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04159ce0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406ff10 tx=0x7f7cfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ccf1b038ec9b7d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d037fe700 1 -- 172.21.15.7:0/409772882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00e070 con 0x7f7d04072020 2023-06-09T11:46:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d037fe700 1 -- 172.21.15.7:0/409772882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc00b9c0 con 0x7f7d04072020 2023-06-09T11:46:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d037fe700 1 -- 172.21.15.7:0/409772882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00bd70 con 0x7f7d04072020 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/409772882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 msgr2=0x7f7d04159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/409772882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04159ce0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406ff10 tx=0x7f7cfc00b330 comp rx=0 tx=0).stop 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/409772882 shutdown_connections 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/409772882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04159ce0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/409772882 >> v2:172.21.15.117:3300/0 conn(0x7f7d041593d0 0x7f7d041597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/409772882 >> 172.21.15.7:0/409772882 conn(0x7f7d0406a4d0 msgr2=0x7f7d0406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/409772882 shutdown_connections 2023-06-09T11:46:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.575+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/409772882 wait complete. 2023-06-09T11:46:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.577+0000 7f7d0b1ff700 1 Processor -- start 2023-06-09T11:46:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.577+0000 7f7d0b1ff700 1 -- start start 2023-06-09T11:46:37.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c08d0 0x7f7d041409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d041593d0 0x7f7d04140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04141ac0 con 0x7f7d04072020 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041434e0 con 0x7f7d041593d0 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d0b1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04142c20 con 0x7f7d040c08d0 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.578+0000 7f7d08f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52900/0 (socket says 172.21.15.7:52900) 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 -- 172.21.15.7:0/1827125220 learned_addr learned my addr 172.21.15.7:0/1827125220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d0979c700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d041593d0 0x7f7d04140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d03fff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c08d0 0x7f7d041409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 -- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c08d0 msgr2=0x7f7d041409b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c08d0 0x7f7d041409b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 -- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d041593d0 msgr2=0x7f7d04140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d041593d0 0x7f7d04140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 -- 172.21.15.7:0/1827125220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009b30 con 0x7f7d04072020 2023-06-09T11:46:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d08f9b700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e810 tx=0x7f7cf800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.579+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8013610 con 0x7f7d04072020 2023-06-09T11:46:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.580+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8013770 con 0x7f7d04072020 2023-06-09T11:46:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.580+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d040c1d60 con 0x7f7d04072020 2023-06-09T11:46:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.580+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8013b20 con 0x7f7d04072020 2023-06-09T11:46:37.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.580+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d040c1a80 con 0x7f7d04072020 2023-06-09T11:46:37.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.581+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d040c2040 con 0x7f7d04072020 2023-06-09T11:46:37.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.581+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cf8013c80 con 0x7f7d04072020 2023-06-09T11:46:37.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.583+0000 7f7d017fa700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 0x7f7ce806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:37.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.583+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f7cf80897b0 con 0x7f7d04072020 2023-06-09T11:46:37.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.583+0000 7f7d03fff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 0x7f7ce806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:37.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.583+0000 7f7d03fff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 0x7f7ce806d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7d041429d0 tx=0x7f7cf0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:37.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.586+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7cf8054130 con 0x7f7d04072020 2023-06-09T11:46:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.842+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d04074cf0 con 0x7f7d04072020 2023-06-09T11:46:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.843+0000 7f7d017fa700 1 -- 172.21.15.7:0/1827125220 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f7cf8057750 con 0x7f7d04072020 2023-06-09T11:46:37.846 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:37.846 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 msgr2=0x7f7ce806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 0x7f7ce806d190 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f7d041429d0 tx=0x7f7cf0009040 comp rx=0 tx=0).stop 2023-06-09T11:46:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 msgr2=0x7f7d04138ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e810 tx=0x7f7cf800eb20 comp rx=0 tx=0).stop 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 shutdown_connections 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7d041593d0 0x7f7d04140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d04072020 0x7f7d04138ef0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7ce806ace0 0x7f7ce806d190 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 --2- 172.21.15.7:0/1827125220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7d040c08d0 0x7f7d041409b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 >> 172.21.15.7:0/1827125220 conn(0x7f7d0406a4d0 msgr2=0x7f7d040c6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 shutdown_connections 2023-06-09T11:46:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:37.845+0000 7f7d0b1ff700 1 -- 172.21.15.7:0/1827125220 wait complete. 2023-06-09T11:46:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:37.866 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:46:39.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.074+0000 7f0ed13ff700 1 Processor -- start 2023-06-09T11:46:40.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.075+0000 7f0ed13ff700 1 -- start start 2023-06-09T11:46:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ed13ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ecc0cb9a0 0x7f0ecc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ed13ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc159b20 con 0x7f0ecc0cb5d0 2023-06-09T11:46:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ed13ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0cac80 con 0x7f0ecc15cd40 2023-06-09T11:46:40.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ed13ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc1597d0 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ec3fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ecc0cb9a0 0x7f0ecc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ecaffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0ecc0cb5d0 0x7f0ecc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58734/0 (socket says 172.21.15.7:58734) 2023-06-09T11:46:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ecaffd700 1 -- 172.21.15.7:0/585896658 learned_addr learned my addr 172.21.15.7:0/585896658 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:40.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ec3fff700 1 -- 172.21.15.7:0/585896658 >> v1:172.21.15.117:6789/0 conn(0x7f0ecc15cd40 legacy=0x7f0ecc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ec3fff700 1 -- 172.21.15.7:0/585896658 >> v1:172.21.15.7:6789/0 conn(0x7f0ecc0cb5d0 legacy=0x7f0ecc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.076+0000 7f0ec3fff700 1 -- 172.21.15.7:0/585896658 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc071ab0 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0ec3fff700 1 --2- 172.21.15.7:0/585896658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc0cb9a0 0x7f0ecc0b52b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009840 tx=0x7f0eb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2c2350c4d87e3e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0eca7fc700 1 -- 172.21.15.7:0/585896658 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4011830 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0eca7fc700 1 -- 172.21.15.7:0/585896658 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb4011990 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0eca7fc700 1 -- 172.21.15.7:0/585896658 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4011d20 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0ed13ff700 1 -- 172.21.15.7:0/585896658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc0cb9a0 msgr2=0x7f0ecc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.077+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/585896658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc0cb9a0 0x7f0ecc0b52b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009840 tx=0x7f0eb4009b50 comp rx=0 tx=0).stop 2023-06-09T11:46:40.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.078+0000 7f0ed13ff700 1 -- 172.21.15.7:0/585896658 shutdown_connections 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.078+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/585896658 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc0cb9a0 0x7f0ecc0b52b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.078+0000 7f0ed13ff700 1 -- 172.21.15.7:0/585896658 >> 172.21.15.7:0/585896658 conn(0x7f0ecc070af0 msgr2=0x7f0ecc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.078+0000 7f0ed13ff700 1 -- 172.21.15.7:0/585896658 shutdown_connections 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.078+0000 7f0ed13ff700 1 -- 172.21.15.7:0/585896658 wait complete. 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.079+0000 7f0ed13ff700 1 Processor -- start 2023-06-09T11:46:40.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.079+0000 7f0ed13ff700 1 -- start start 2023-06-09T11:46:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 0x7f0ecc1540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ecc0cb9a0 0x7f0ecc150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1eeda0 con 0x7f0ecc15cd40 2023-06-09T11:46:40.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0d0480 con 0x7f0ecc0cb5d0 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.080+0000 7f0ed13ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0ceca0 con 0x7f0ecc0cb9a0 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52916/0 (socket says 172.21.15.7:52916) 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 -- 172.21.15.7:0/2712393364 learned_addr learned my addr 172.21.15.7:0/2712393364 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecaffd700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 0x7f0ecc1540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ec3fff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ecc0cb9a0 0x7f0ecc150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:40.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 -- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ecc0cb9a0 msgr2=0x7f0ecc150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ecc0cb9a0 0x7f0ecc150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 -- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 msgr2=0x7f0ecc1540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 0x7f0ecc1540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 -- 172.21.15.7:0/2712393364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40094a0 con 0x7f0ecc15cd40 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecaffd700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 0x7f0ecc1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.081+0000 7f0ecb7fe700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400cbc0 tx=0x7f0ec400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:40.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.082+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4003c00 con 0x7f0ecc15cd40 2023-06-09T11:46:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.082+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4003d60 con 0x7f0ecc15cd40 2023-06-09T11:46:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.082+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400ab90 con 0x7f0ecc15cd40 2023-06-09T11:46:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.082+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc15d4f0 con 0x7f0ecc15cd40 2023-06-09T11:46:40.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.082+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc15d210 con 0x7f0ecc15cd40 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.083+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc15d7d0 con 0x7f0ecc15cd40 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.084+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ec400acf0 con 0x7f0ecc15cd40 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.084+0000 7f0ec37fe700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 0x7f0eac06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.085+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f0ec4089b60 con 0x7f0ecc15cd40 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.085+0000 7f0ecaffd700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 0x7f0eac06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:40.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.085+0000 7f0ecaffd700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 0x7f0eac06d1c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc005910 tx=0x7f0ebc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:40.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.089+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0ec4054560 con 0x7f0ecc15cd40 2023-06-09T11:46:40.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.340+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ecc0cee10 con 0x7f0ecc15cd40 2023-06-09T11:46:40.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.341+0000 7f0ec37fe700 1 -- 172.21.15.7:0/2712393364 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f0ec4057b80 con 0x7f0ecc15cd40 2023-06-09T11:46:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:40.345 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 msgr2=0x7f0eac06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 0x7f0eac06d1c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc005910 tx=0x7f0ebc009040 comp rx=0 tx=0).stop 2023-06-09T11:46:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 msgr2=0x7f0ecc0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400cbc0 tx=0x7f0ec400cf80 comp rx=0 tx=0).stop 2023-06-09T11:46:40.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 shutdown_connections 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ecc15cd40 0x7f0ecc0cf130 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ecc0cb5d0 0x7f0ecc1540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0eac06ad10 0x7f0eac06d1c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 --2- 172.21.15.7:0/2712393364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ecc0cb9a0 0x7f0ecc150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 >> 172.21.15.7:0/2712393364 conn(0x7f0ecc070af0 msgr2=0x7f0ecc158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 shutdown_connections 2023-06-09T11:46:40.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:40.344+0000 7f0ed13ff700 1 -- 172.21.15.7:0/2712393364 wait complete. 2023-06-09T11:46:40.349 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:40.364 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:46:42.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:42.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.572+0000 7f7e7cdff700 1 Processor -- start 2023-06-09T11:46:42.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.572+0000 7f7e7cdff700 1 -- start start 2023-06-09T11:46:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e780c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7cdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e780c5a80 con 0x7f7e78153880 2023-06-09T11:46:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7815a500 con 0x7f7e7815b4a0 2023-06-09T11:46:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780c5f10 con 0x7f7e780c9f40 2023-06-09T11:46:42.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7e78153880 0x7f7e7815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58738/0 (socket says 172.21.15.7:58738) 2023-06-09T11:46:42.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7659c700 1 -- 172.21.15.7:0/447011357 learned_addr learned my addr 172.21.15.7:0/447011357 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e780c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895911749 0 0) 0x7f7e780c5a80 con 0x7f7e78153880 2023-06-09T11:46:42.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.574+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e600035b0 con 0x7f7e78153880 2023-06-09T11:46:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3138283221 0 0) 0x7f7e600035b0 con 0x7f7e78153880 2023-06-09T11:46:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 msgr2=0x7f7e780c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e780c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 >> v1:172.21.15.117:6789/0 conn(0x7f7e7815b4a0 legacy=0x7f7e780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e781596e0 con 0x7f7e78153880 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f7e68001670 con 0x7f7e78153880 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 >> v1:172.21.15.7:6789/0 conn(0x7f7e78153880 legacy=0x7f7e7815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e60003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e60003f70 0x7f7e60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e60004d90 0x7f7e60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780c5f10 con 0x7f7e60004d90 2023-06-09T11:46:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6000bb50 con 0x7f7e60003f70 2023-06-09T11:46:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6000bd10 con 0x7f7e780c9f40 2023-06-09T11:46:42.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e60004d90 0x7f7e60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e75d9b700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e60003f70 0x7f7e60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.575+0000 7f7e7659c700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e60003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 -- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 msgr2=0x7f7e60003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:42.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e780c9f40 0x7f7e60003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 -- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e60003f70 msgr2=0x7f7e60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e60003f70 0x7f7e60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 -- 172.21.15.7:0/447011357 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e600035b0 con 0x7f7e60004d90 2023-06-09T11:46:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e60004d90 0x7f7e60005200 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009f90 tx=0x7f7e60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b99fac8c5d7b711 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:42.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7000ec30 con 0x7f7e60004d90 2023-06-09T11:46:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7000f040 con 0x7f7e60004d90 2023-06-09T11:46:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.576+0000 7f7e7559a700 1 -- 172.21.15.7:0/447011357 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e70014480 con 0x7f7e60004d90 2023-06-09T11:46:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 -- 172.21.15.7:0/447011357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e60004d90 msgr2=0x7f7e60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/447011357 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e60004d90 0x7f7e60005200 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7e70009f90 tx=0x7f7e60005880 comp rx=0 tx=0).stop 2023-06-09T11:46:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e76d9d700 1 -- 172.21.15.7:0/447011357 reap_dead start 2023-06-09T11:46:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 -- 172.21.15.7:0/447011357 shutdown_connections 2023-06-09T11:46:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 -- 172.21.15.7:0/447011357 >> 172.21.15.7:0/447011357 conn(0x7f7e780737f0 msgr2=0x7f7e780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 -- 172.21.15.7:0/447011357 shutdown_connections 2023-06-09T11:46:42.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.577+0000 7f7e7cdff700 1 -- 172.21.15.7:0/447011357 wait complete. 2023-06-09T11:46:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.579+0000 7f7e7cdff700 1 Processor -- start 2023-06-09T11:46:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.579+0000 7f7e7cdff700 1 -- start start 2023-06-09T11:46:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e60004d90 0x7f7e78157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 0x7f7e781eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e78152490 con 0x7f7e78159220 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e781ef520 con 0x7f7e780b3fd0 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e7cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e780b3e00 con 0x7f7e60004d90 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52934/0 (socket says 172.21.15.7:52934) 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 -- 172.21.15.7:0/2355240220 learned_addr learned my addr 172.21.15.7:0/2355240220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:42.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 -- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e60004d90 msgr2=0x7f7e78157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e60004d90 0x7f7e78157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 0x7f7e781eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 -- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 msgr2=0x7f7e781eefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 0x7f7e781eefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 -- 172.21.15.7:0/2355240220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78159c60 con 0x7f7e78159220 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e76d9d700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 0x7f7e781eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.580+0000 7f7e75d9b700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00f820 tx=0x7f7e6c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:42.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.581+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0052e0 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.581+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c005460 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.581+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e78159980 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.581+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0038c0 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.581+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e78159f40 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.582+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e78074cf0 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.583+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e6c003a20 con 0x7f7e78159220 2023-06-09T11:46:42.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.584+0000 7f7e66ffd700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.584+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7754+0+0 (secure 0 0 0) 0x7f7e6c089900 con 0x7f7e78159220 2023-06-09T11:46:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.584+0000 7f7e7659c700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.584+0000 7f7e7659c700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e68007d30 tx=0x7f7e68001b40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:42.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.588+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7e6c054300 con 0x7f7e78159220 2023-06-09T11:46:42.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.839+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e7815a500 con 0x7f7e78159220 2023-06-09T11:46:42.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.840+0000 7f7e66ffd700 1 -- 172.21.15.7:0/2355240220 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 168 v168) v1 ==== 95+0+6224 (secure 0 0 0) 0x7f7e7815a500 con 0x7f7e78159220 2023-06-09T11:46:42.844 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:42.844 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":168,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7279,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":166}],"filesystems":[{"mdsmap":{"epoch":168,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:20.603235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":68,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":6859},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6859":{"gid":6859,"name":"d","rank":2,"incarnation":141,"state":"up:active","state_seq":30,"addr":"172.21.15.117:6835/477143596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":477143596},{"type":"v1","addr":"172.21.15.117:6835","nonce":477143596}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 msgr2=0x7f7e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 0x7f7e5806d190 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7e68007d30 tx=0x7f7e68001b40 comp rx=0 tx=0).stop 2023-06-09T11:46:42.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 msgr2=0x7f7e781eeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00f820 tx=0x7f7e6c00fbe0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 shutdown_connections 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e780b3fd0 0x7f7e781eefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e5806ace0 0x7f7e5806d190 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e60004d90 0x7f7e78157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 --2- 172.21.15.7:0/2355240220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e78159220 0x7f7e781eeaa0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:42.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 >> 172.21.15.7:0/2355240220 conn(0x7f7e780737f0 msgr2=0x7f7e78152830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:42.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 shutdown_connections 2023-06-09T11:46:42.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:42.842+0000 7f7e7cdff700 1 -- 172.21.15.7:0/2355240220 wait complete. 2023-06-09T11:46:42.848 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 168 2023-06-09T11:46:42.864 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:46:44.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:45.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.069+0000 7fbdf2fff700 1 Processor -- start 2023-06-09T11:46:45.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.069+0000 7fbdf2fff700 1 -- start start 2023-06-09T11:46:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf2fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdec153960 0x7fbdec06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf2fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 0x7fbdec070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf2fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec15ce00 con 0x7fbdec0af980 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf2fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdec0adab0 con 0x7fbdec153960 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf2fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdec15dc60 con 0x7fbdec071c80 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf0d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdec153960 0x7fbdec06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf0d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbdec153960 0x7fbdec06fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52004/0 (socket says 172.21.15.7:52004) 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/2209816802 learned_addr learned my addr 172.21.15.7:0/2209816802 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:45.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.071+0000 7fbdebfff700 1 --2- 172.21.15.7:0/2209816802 >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 0x7fbdec070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/2209816802 >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 msgr2=0x7fbdec070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:45.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdf0d9b700 1 --2- 172.21.15.7:0/2209816802 >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 0x7fbdec070040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/2209816802 >> v1:172.21.15.117:6789/0 conn(0x7fbdec0af980 legacy=0x7fbdec159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/2209816802 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec15d0e0 con 0x7fbdec153960 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdebfff700 1 --2- 172.21.15.7:0/2209816802 >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 0x7fbdec070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdf0d9b700 1 --2- 172.21.15.7:0/2209816802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec153960 0x7fbdec06fb00 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0719a0 tx=0x7fbde0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b39b80e47f53c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdeb7fe700 1 -- 172.21.15.7:0/2209816802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde000b870 con 0x7fbdec153960 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdeb7fe700 1 -- 172.21.15.7:0/2209816802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde000b9d0 con 0x7fbdec153960 2023-06-09T11:46:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.072+0000 7fbdeb7fe700 1 -- 172.21.15.7:0/2209816802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde000bd60 con 0x7fbdec153960 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 -- 172.21.15.7:0/2209816802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec153960 msgr2=0x7fbdec06fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/2209816802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec153960 0x7fbdec06fb00 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0719a0 tx=0x7fbde0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 -- 172.21.15.7:0/2209816802 shutdown_connections 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/2209816802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec153960 0x7fbdec06fb00 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/2209816802 >> v2:172.21.15.117:3300/0 conn(0x7fbdec071c80 0x7fbdec070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 -- 172.21.15.7:0/2209816802 >> 172.21.15.7:0/2209816802 conn(0x7fbdec0c9f00 msgr2=0x7fbdec0ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 -- 172.21.15.7:0/2209816802 shutdown_connections 2023-06-09T11:46:45.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.073+0000 7fbdf2fff700 1 -- 172.21.15.7:0/2209816802 wait complete. 2023-06-09T11:46:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.074+0000 7fbdf2fff700 1 Processor -- start 2023-06-09T11:46:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.075+0000 7fbdf2fff700 1 -- start start 2023-06-09T11:46:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 0x7fbdec0c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdec153960 0x7fbdec0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c55e0 con 0x7fbdec071c80 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c4aa0 con 0x7fbdec0af980 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf2fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c4d40 con 0x7fbdec153960 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52012/0 (socket says 172.21.15.7:52012) 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/190553604 learned_addr learned my addr 172.21.15.7:0/190553604 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdebfff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 0x7fbdec0c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf159c700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdec153960 0x7fbdec0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdec153960 msgr2=0x7fbdec0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdec153960 0x7fbdec0c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 msgr2=0x7fbdec0c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 0x7fbdec0c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdf0d9b700 1 -- 172.21.15.7:0/190553604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0009550 con 0x7fbdec071c80 2023-06-09T11:46:45.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.076+0000 7fbdebfff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 0x7fbdec0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbdf0d9b700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbde0000c00 tx=0x7fbde0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0004520 con 0x7fbdec071c80 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde001e070 con 0x7fbdec071c80 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde00038a0 con 0x7fbdec071c80 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec15d6a0 con 0x7fbdec071c80 2023-06-09T11:46:45.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.077+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec15d3c0 con 0x7fbdec071c80 2023-06-09T11:46:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.078+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec15d980 con 0x7fbdec071c80 2023-06-09T11:46:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.079+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbde0003a00 con 0x7fbdec071c80 2023-06-09T11:46:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.081+0000 7fbde97fa700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 0x7fbdd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.081+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fbde0020070 con 0x7fbdec071c80 2023-06-09T11:46:45.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.081+0000 7fbdebfff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 0x7fbdd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.082+0000 7fbdebfff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 0x7fbdd006d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbdec1599b0 tx=0x7fbdd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.087+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbde0055b10 con 0x7fbdec071c80 2023-06-09T11:46:45.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.356+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdec0ae7d0 con 0x7fbdec071c80 2023-06-09T11:46:45.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.357+0000 7fbde97fa700 1 -- 172.21.15.7:0/190553604 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fbde002a090 con 0x7fbdec071c80 2023-06-09T11:46:45.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:45.362 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":169,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:44.608118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 msgr2=0x7fbdd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 0x7fbdd006d190 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbdec1599b0 tx=0x7fbdd8008040 comp rx=0 tx=0).stop 2023-06-09T11:46:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 msgr2=0x7fbdec0b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fbde0000c00 tx=0x7fbde0004750 comp rx=0 tx=0).stop 2023-06-09T11:46:45.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 shutdown_connections 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbdd006ace0 0x7fbdd006d190 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbdec153960 0x7fbdec0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbdec071c80 0x7fbdec0b0790 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 --2- 172.21.15.7:0/190553604 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbdec0af980 0x7fbdec0c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 >> 172.21.15.7:0/190553604 conn(0x7fbdec0c9f00 msgr2=0x7fbdec151ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 shutdown_connections 2023-06-09T11:46:45.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.360+0000 7fbdf2fff700 1 -- 172.21.15.7:0/190553604 wait complete. 2023-06-09T11:46:45.366 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 169 2023-06-09T11:46:45.382 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:46:45.382 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:46:45.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:45.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.579+0000 7fe55a3ff700 1 Processor -- start 2023-06-09T11:46:45.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.579+0000 7fe55a3ff700 1 -- start start 2023-06-09T11:46:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe55a3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5541535e0 0x7fe554159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe55a3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5541539b0 0x7fe5540c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe55a3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55415a420 con 0x7fe5540c4ce0 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe55a3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5540c5a00 con 0x7fe5541539b0 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe55a3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55406b460 con 0x7fe5541535e0 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5541539b0 0x7fe5540c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5541539b0 0x7fe5540c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52024/0 (socket says 172.21.15.7:52024) 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 -- 172.21.15.7:0/3838234318 learned_addr learned my addr 172.21.15.7:0/3838234318 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe553fff700 1 --2- 172.21.15.7:0/3838234318 >> v2:172.21.15.117:3300/0 conn(0x7fe5541535e0 0x7fe554159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 -- 172.21.15.7:0/3838234318 >> v2:172.21.15.117:3300/0 conn(0x7fe5541535e0 msgr2=0x7fe554159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3838234318 >> v2:172.21.15.117:3300/0 conn(0x7fe5541535e0 0x7fe554159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 -- 172.21.15.7:0/3838234318 >> v1:172.21.15.117:6789/0 conn(0x7fe5540c4ce0 legacy=0x7fe55415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.581+0000 7fe5537fe700 1 -- 172.21.15.7:0/3838234318 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55406a570 con 0x7fe5541539b0 2023-06-09T11:46:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3838234318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5541539b0 0x7fe5540c9f60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe554073d90 tx=0x7fe548009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1b292639ec1d5b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe552ffd700 1 -- 172.21.15.7:0/3838234318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800b870 con 0x7fe5541539b0 2023-06-09T11:46:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe552ffd700 1 -- 172.21.15.7:0/3838234318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54800b9d0 con 0x7fe5541539b0 2023-06-09T11:46:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe552ffd700 1 -- 172.21.15.7:0/3838234318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800bd60 con 0x7fe5541539b0 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3838234318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5541539b0 msgr2=0x7fe5540c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.582+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3838234318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5541539b0 0x7fe5540c9f60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe554073d90 tx=0x7fe548009bd0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3838234318 shutdown_connections 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3838234318 >> v2:172.21.15.117:3300/0 conn(0x7fe5541535e0 0x7fe554159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3838234318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5541539b0 0x7fe5540c9f60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3838234318 >> 172.21.15.7:0/3838234318 conn(0x7fe5540737f0 msgr2=0x7fe554072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3838234318 shutdown_connections 2023-06-09T11:46:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.583+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3838234318 wait complete. 2023-06-09T11:46:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.585+0000 7fe55a3ff700 1 Processor -- start 2023-06-09T11:46:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.585+0000 7fe55a3ff700 1 -- start start 2023-06-09T11:46:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 0x7fe55414feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 0x7fe5540ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5540ce8e0 con 0x7fe5540c4ce0 2023-06-09T11:46:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541ef380 con 0x7fe5541539b0 2023-06-09T11:46:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.586+0000 7fe55a3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541ef870 con 0x7fe5541535e0 2023-06-09T11:46:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52030/0 (socket says 172.21.15.7:52030) 2023-06-09T11:46:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 -- 172.21.15.7:0/3151606309 learned_addr learned my addr 172.21.15.7:0/3151606309 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 0x7fe55414feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe55899c700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 0x7fe5540ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 -- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 msgr2=0x7fe55414feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 0x7fe55414feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 -- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 msgr2=0x7fe5540ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 0x7fe5540ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 -- 172.21.15.7:0/3151606309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe548009550 con 0x7fe5540c4ce0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 0x7fe55414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe55899c700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 0x7fe5540ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.587+0000 7fe553fff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe54400e810 tx=0x7fe54400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.588+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544013610 con 0x7fe5540c4ce0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.588+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe544013770 con 0x7fe5540c4ce0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.588+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe544013b20 con 0x7fe5540c4ce0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.588+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55406ab30 con 0x7fe5540c4ce0 2023-06-09T11:46:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.588+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55406a850 con 0x7fe5540c4ce0 2023-06-09T11:46:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.589+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55406ae10 con 0x7fe5540c4ce0 2023-06-09T11:46:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.590+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe544013c80 con 0x7fe5540c4ce0 2023-06-09T11:46:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.590+0000 7fe550ff9700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 0x7fe54006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.591+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fe5440898a0 con 0x7fe5540c4ce0 2023-06-09T11:46:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.591+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 0x7fe54006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.591+0000 7fe5537fe700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 0x7fe54006d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe554153da0 tx=0x7fe548008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.595+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe5440541f0 con 0x7fe5540c4ce0 2023-06-09T11:46:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.850+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe554074cf0 con 0x7fe5540c4ce0 2023-06-09T11:46:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.851+0000 7fe550ff9700 1 -- 172.21.15.7:0/3151606309 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fe544021090 con 0x7fe5540c4ce0 2023-06-09T11:46:45.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:45.856 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":169,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:44.608118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 msgr2=0x7fe54006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 0x7fe54006d190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe554153da0 tx=0x7fe548008030 comp rx=0 tx=0).stop 2023-06-09T11:46:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 msgr2=0x7fe5540c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fe54400e810 tx=0x7fe54400ebd0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 shutdown_connections 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5541539b0 0x7fe5540ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5540c4ce0 0x7fe5540c9230 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54006ace0 0x7fe54006d190 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 --2- 172.21.15.7:0/3151606309 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5541535e0 0x7fe55414feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 >> 172.21.15.7:0/3151606309 conn(0x7fe5540737f0 msgr2=0x7fe554159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 shutdown_connections 2023-06-09T11:46:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:45.853+0000 7fe55a3ff700 1 -- 172.21.15.7:0/3151606309 wait complete. 2023-06-09T11:46:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 169 2023-06-09T11:46:45.876 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 169, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 169, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7141": { "addr": "172.21.15.117:6837/3727664816", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3727664816, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3727664816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7141, "incarnation": 159, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7279": { "addr": "172.21.15.117:6833/3302752268", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 3302752268, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 3302752268, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7279, "incarnation": 169, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:46:44.608118+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7141, "mds_2": 7279 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:46:47.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:48.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.084+0000 7f57457ff700 1 Processor -- start 2023-06-09T11:46:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.084+0000 7f57457ff700 1 -- start start 2023-06-09T11:46:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.085+0000 7f57457ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f57400cc5b0 0x7f57400b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:48.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.085+0000 7f57457ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5740159b20 con 0x7f57400cb9a0 2023-06-09T11:46:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.085+0000 7f57457ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400cac80 con 0x7f57400cb5d0 2023-06-09T11:46:48.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.085+0000 7f57457ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57401597d0 con 0x7f57400cc5b0 2023-06-09T11:46:48.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f57400cb9a0 0x7f57400cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45954/0 (socket says 172.21.15.7:45954) 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573e7fc700 1 -- 172.21.15.7:0/4108858823 learned_addr learned my addr 172.21.15.7:0/4108858823 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573f7fe700 1 --2- 172.21.15.7:0/4108858823 >> v2:172.21.15.117:3300/0 conn(0x7f57400cc5b0 0x7f57400b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573dffb700 1 -- 172.21.15.7:0/4108858823 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719867590 0 0) 0x7f5740159b20 con 0x7f57400cb9a0 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573dffb700 1 -- 172.21.15.7:0/4108858823 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57280035b0 con 0x7f57400cb9a0 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573f7fe700 1 -- 172.21.15.7:0/4108858823 >> v1:172.21.15.117:6789/0 conn(0x7f57400cb5d0 legacy=0x7f5740159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573f7fe700 1 -- 172.21.15.7:0/4108858823 >> v1:172.21.15.7:6789/0 conn(0x7f57400cb9a0 legacy=0x7f57400cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.086+0000 7f573f7fe700 1 -- 172.21.15.7:0/4108858823 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5740071ab0 con 0x7f57400cc5b0 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f573f7fe700 1 --2- 172.21.15.7:0/4108858823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cc5b0 0x7f57400b52b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5738009d90 tx=0x7f573800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fdfd034fe3f19bb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f573dffb700 1 -- 172.21.15.7:0/4108858823 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800e070 con 0x7f57400cc5b0 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f573dffb700 1 -- 172.21.15.7:0/4108858823 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573800bb20 con 0x7f57400cc5b0 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f573dffb700 1 -- 172.21.15.7:0/4108858823 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738012430 con 0x7f57400cc5b0 2023-06-09T11:46:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f57457ff700 1 -- 172.21.15.7:0/4108858823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cc5b0 msgr2=0x7f57400b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f57457ff700 1 --2- 172.21.15.7:0/4108858823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cc5b0 0x7f57400b52b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f5738009d90 tx=0x7f573800b330 comp rx=0 tx=0).stop 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f57457ff700 1 -- 172.21.15.7:0/4108858823 shutdown_connections 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f57457ff700 1 --2- 172.21.15.7:0/4108858823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cc5b0 0x7f57400b52b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.087+0000 7f57457ff700 1 -- 172.21.15.7:0/4108858823 >> 172.21.15.7:0/4108858823 conn(0x7f5740070af0 msgr2=0x7f5740070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.088+0000 7f57457ff700 1 -- 172.21.15.7:0/4108858823 shutdown_connections 2023-06-09T11:46:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.088+0000 7f57457ff700 1 -- 172.21.15.7:0/4108858823 wait complete. 2023-06-09T11:46:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.089+0000 7f57457ff700 1 Processor -- start 2023-06-09T11:46:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.089+0000 7f57457ff700 1 -- start start 2023-06-09T11:46:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 0x7f57401ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400cb9a0 0x7f57400b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57401ef800 con 0x7f57400cc5b0 2023-06-09T11:46:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400c56c0 con 0x7f57400cb5d0 2023-06-09T11:46:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.090+0000 7f57457ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57400c59a0 con 0x7f57400cb9a0 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52032/0 (socket says 172.21.15.7:52032) 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 -- 172.21.15.7:0/2637851867 learned_addr learned my addr 172.21.15.7:0/2637851867 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573effd700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 0x7f57401ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573e7fc700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400cb9a0 0x7f57400b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 -- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400cb9a0 msgr2=0x7f57400b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400cb9a0 0x7f57400b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 -- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 msgr2=0x7f57401ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 0x7f57401ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 -- 172.21.15.7:0/2637851867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738009a70 con 0x7f57400cc5b0 2023-06-09T11:46:48.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573effd700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 0x7f57401ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f573f7fe700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5740067180 tx=0x7f5738004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738012430 con 0x7f57400cc5b0 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738020070 con 0x7f57400cc5b0 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.091+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738004280 con 0x7f57400cc5b0 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.092+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574015d4f0 con 0x7f57400cc5b0 2023-06-09T11:46:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.092+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574015d210 con 0x7f57400cc5b0 2023-06-09T11:46:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.093+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574015d7d0 con 0x7f57400cc5b0 2023-06-09T11:46:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.093+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57380043e0 con 0x7f57400cc5b0 2023-06-09T11:46:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.094+0000 7f57277fe700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 0x7f572c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.094+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f5738022070 con 0x7f57400cc5b0 2023-06-09T11:46:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.095+0000 7f573effd700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 0x7f572c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.095+0000 7f573effd700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 0x7f572c06d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.098+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5738056360 con 0x7f57400cc5b0 2023-06-09T11:46:48.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.353+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5740074cf0 con 0x7f57400cc5b0 2023-06-09T11:46:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.354+0000 7f57277fe700 1 -- 172.21.15.7:0/2637851867 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f573802c070 con 0x7f57400cc5b0 2023-06-09T11:46:48.358 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:48.359 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":169,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:44.608118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:48.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 msgr2=0x7f572c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 0x7f572c06d190 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5730000f60 tx=0x7f5730009040 comp rx=0 tx=0).stop 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 msgr2=0x7f57400c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f5740067180 tx=0x7f5738004ad0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 shutdown_connections 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f57400cc5b0 0x7f57400c3f50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57400cb5d0 0x7f57401ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f572c06ace0 0x7f572c06d190 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 --2- 172.21.15.7:0/2637851867 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f57400cb9a0 0x7f57400b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 >> 172.21.15.7:0/2637851867 conn(0x7f5740070af0 msgr2=0x7f57400c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 shutdown_connections 2023-06-09T11:46:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:48.356+0000 7f57457ff700 1 -- 172.21.15.7:0/2637851867 wait complete. 2023-06-09T11:46:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 169 2023-06-09T11:46:48.378 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:46:50.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.586+0000 7f3503fff700 1 Processor -- start 2023-06-09T11:46:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.586+0000 7f3503fff700 1 -- start start 2023-06-09T11:46:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34fc151510 0x7f34fc0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f34fc15b380 0x7f34fc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc071060 0x7f34fc158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc1518e0 con 0x7f34fc151510 2023-06-09T11:46:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0714c0 con 0x7f34fc15b380 2023-06-09T11:46:50.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.587+0000 7f3503fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1580a0 con 0x7f34fc071060 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34fc151510 0x7f34fc0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34fc151510 0x7f34fc0c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52044/0 (socket says 172.21.15.7:52044) 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 -- 172.21.15.7:0/2053222400 learned_addr learned my addr 172.21.15.7:0/2053222400 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f350159a700 1 --2- 172.21.15.7:0/2053222400 >> v2:172.21.15.117:3300/0 conn(0x7f34fc15b380 0x7f34fc070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f350259c700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc071060 0x7f34fc158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 -- 172.21.15.7:0/2053222400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc071060 msgr2=0x7f34fc158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:50.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc071060 0x7f34fc158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 -- 172.21.15.7:0/2053222400 >> v2:172.21.15.117:3300/0 conn(0x7f34fc15b380 msgr2=0x7f34fc070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 --2- 172.21.15.7:0/2053222400 >> v2:172.21.15.117:3300/0 conn(0x7f34fc15b380 0x7f34fc070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.588+0000 7f3501d9b700 1 -- 172.21.15.7:0/2053222400 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc0c6180 con 0x7f34fc151510 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3501d9b700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc151510 0x7f34fc0c5b40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06b170 tx=0x7f34f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7737f33c17d7c76 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3500d99700 1 -- 172.21.15.7:0/2053222400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011830 con 0x7f34fc151510 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3500d99700 1 -- 172.21.15.7:0/2053222400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f8011990 con 0x7f34fc151510 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3500d99700 1 -- 172.21.15.7:0/2053222400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8011d20 con 0x7f34fc151510 2023-06-09T11:46:50.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 -- 172.21.15.7:0/2053222400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc151510 msgr2=0x7f34fc0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc151510 0x7f34fc0c5b40 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f34fc06b170 tx=0x7f34f8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 -- 172.21.15.7:0/2053222400 shutdown_connections 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc071060 0x7f34fc158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 --2- 172.21.15.7:0/2053222400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc151510 0x7f34fc0c5b40 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 --2- 172.21.15.7:0/2053222400 >> v2:172.21.15.117:3300/0 conn(0x7f34fc15b380 0x7f34fc070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 -- 172.21.15.7:0/2053222400 >> 172.21.15.7:0/2053222400 conn(0x7f34fc06a4d0 msgr2=0x7f34fc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:50.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.589+0000 7f3503fff700 1 -- 172.21.15.7:0/2053222400 shutdown_connections 2023-06-09T11:46:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.590+0000 7f3503fff700 1 -- 172.21.15.7:0/2053222400 wait complete. 2023-06-09T11:46:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.592+0000 7f3503fff700 1 Processor -- start 2023-06-09T11:46:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.592+0000 7f3503fff700 1 -- start start 2023-06-09T11:46:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 0x7f34fc0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34fc15b380 0x7f34fc0aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc15a800 con 0x7f34fc071060 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0af160 con 0x7f34fc15b380 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.593+0000 7f3503fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0b02e0 con 0x7f34fc151510 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f350159a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 0x7f34fc0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52052/0 (socket says 172.21.15.7:52052) 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 -- 172.21.15.7:0/3819086743 learned_addr learned my addr 172.21.15.7:0/3819086743 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:50.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f350259c700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34fc15b380 0x7f34fc0aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 -- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 msgr2=0x7f34fc0ad990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 0x7f34fc0ad990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 -- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34fc15b380 msgr2=0x7f34fc0aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34fc15b380 0x7f34fc0aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 -- 172.21.15.7:0/3819086743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8009550 con 0x7f34fc071060 2023-06-09T11:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f350159a700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 0x7f34fc0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.594+0000 7f3501d9b700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009c10 tx=0x7f34f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.595+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004240 con 0x7f34fc071060 2023-06-09T11:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.595+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f801f070 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.595+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc157770 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.595+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f80036a0 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.595+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc157490 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.596+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc157a50 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.596+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f34f8021030 con 0x7f34fc071060 2023-06-09T11:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.597+0000 7f34f27fc700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.598+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f34f808aef0 con 0x7f34fc071060 2023-06-09T11:46:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.598+0000 7f350159a700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:50.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.598+0000 7f350159a700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 0x7f34e006d190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f34fc150c00 tx=0x7f34ec009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:50.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.602+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f34f80559e0 con 0x7f34fc071060 2023-06-09T11:46:50.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.857+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34fc0c5a20 con 0x7f34fc071060 2023-06-09T11:46:50.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.858+0000 7f34f27fc700 1 -- 172.21.15.7:0/3819086743 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 169 v169) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f34f8015020 con 0x7f34fc071060 2023-06-09T11:46:50.862 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:50.862 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":169,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":169,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:44.608118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 msgr2=0x7f34e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:50.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 0x7f34e006d190 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f34fc150c00 tx=0x7f34ec009210 comp rx=0 tx=0).stop 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 msgr2=0x7f34fc146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009c10 tx=0x7f34f8004750 comp rx=0 tx=0).stop 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 shutdown_connections 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34fc15b380 0x7f34fc0aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34fc071060 0x7f34fc146f40 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f34e006ace0 0x7f34e006d190 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 --2- 172.21.15.7:0/3819086743 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f34fc151510 0x7f34fc0ad990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:50.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 >> 172.21.15.7:0/3819086743 conn(0x7f34fc06a4d0 msgr2=0x7f34fc0c7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:50.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 shutdown_connections 2023-06-09T11:46:50.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:50.860+0000 7f3503fff700 1 -- 172.21.15.7:0/3819086743 wait complete. 2023-06-09T11:46:50.866 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 169 2023-06-09T11:46:50.882 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:46:51.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:51.122 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:46:52.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:53.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.090+0000 7f1cd1bff700 1 Processor -- start 2023-06-09T11:46:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.090+0000 7f1cd1bff700 1 -- start start 2023-06-09T11:46:53.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1cd1bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ccc153880 0x7f1ccc15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1cd1bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc15a500 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1cd1bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc15da00 con 0x7f1ccc0c9f40 2023-06-09T11:46:53.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1cd1bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0c5f10 con 0x7f1ccc153880 2023-06-09T11:46:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ccc153880 0x7f1ccc15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1ccc15b4a0 0x7f1ccc0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33348/0 (socket says 172.21.15.7:33348) 2023-06-09T11:46:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccb7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1ccc153880 0x7f1ccc15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39028/0 (socket says 172.21.15.7:39028) 2023-06-09T11:46:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccbfff700 1 -- 172.21.15.7:0/1273011440 learned_addr learned my addr 172.21.15.7:0/1273011440 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:53.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccb7fe700 1 -- 172.21.15.7:0/1273011440 >> v1:172.21.15.117:6789/0 conn(0x7f1ccc0c9f40 legacy=0x7f1ccc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccb7fe700 1 -- 172.21.15.7:0/1273011440 >> v1:172.21.15.7:6789/0 conn(0x7f1ccc15b4a0 legacy=0x7f1ccc0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1ccb7fe700 1 -- 172.21.15.7:0/1273011440 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc1596e0 con 0x7f1ccc153880 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.092+0000 7f1cca7fc700 1 -- 172.21.15.7:0/1273011440 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991045301 0 0) 0x7f1cc4001490 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1ccb7fe700 1 --2- 172.21.15.7:0/1273011440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc153880 0x7f1ccc15d4c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009840 tx=0x7f1cbc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850d8f8c6662767d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1cca7fc700 1 -- 172.21.15.7:0/1273011440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc011830 con 0x7f1ccc153880 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1cca7fc700 1 -- 172.21.15.7:0/1273011440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc011990 con 0x7f1ccc153880 2023-06-09T11:46:53.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1cca7fc700 1 -- 172.21.15.7:0/1273011440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc011d20 con 0x7f1ccc153880 2023-06-09T11:46:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1cd1bff700 1 -- 172.21.15.7:0/1273011440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc153880 msgr2=0x7f1ccc15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.093+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/1273011440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc153880 0x7f1ccc15d4c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009840 tx=0x7f1cbc009b50 comp rx=0 tx=0).stop 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.094+0000 7f1cd1bff700 1 -- 172.21.15.7:0/1273011440 shutdown_connections 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.094+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/1273011440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc153880 0x7f1ccc15d4c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.094+0000 7f1cd1bff700 1 -- 172.21.15.7:0/1273011440 >> 172.21.15.7:0/1273011440 conn(0x7f1ccc0737f0 msgr2=0x7f1ccc0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.094+0000 7f1cd1bff700 1 -- 172.21.15.7:0/1273011440 shutdown_connections 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.094+0000 7f1cd1bff700 1 -- 172.21.15.7:0/1273011440 wait complete. 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.095+0000 7f1cd1bff700 1 Processor -- start 2023-06-09T11:46:53.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 -- start start 2023-06-09T11:46:53.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ccc0c9f40 0x7f1ccc1e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:53.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ccc153880 0x7f1ccc0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:53.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:53.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc1e4a10 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc1e4040 con 0x7f1ccc0c9f40 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.096+0000 7f1cd1bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cb2a0 con 0x7f1ccc153880 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39042/0 (socket says 172.21.15.7:39042) 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 -- 172.21.15.7:0/3700350532 learned_addr learned my addr 172.21.15.7:0/3700350532 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccb7fe700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ccc0c9f40 0x7f1ccc1e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccaffd700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ccc153880 0x7f1ccc0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:53.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 -- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ccc153880 msgr2=0x7f1ccc0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ccc153880 0x7f1ccc0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 -- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ccc0c9f40 msgr2=0x7f1ccc1e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ccc0c9f40 0x7f1ccc1e44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 -- 172.21.15.7:0/3700350532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0094a0 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.097+0000 7f1ccbfff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400cbd0 tx=0x7f1cc400cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.098+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400aa60 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.098+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc400abe0 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.098+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40070d0 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.098+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc159c60 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.098+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc159980 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.099+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc159f40 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.100+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cc4007230 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.101+0000 7f1cb3fff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 0x7f1cb806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.101+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f1cc4089360 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.101+0000 7f1ccb7fe700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 0x7f1cb806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:53.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.102+0000 7f1ccb7fe700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 0x7f1cb806d160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0090b0 tx=0x7f1cbc01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.105+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1cc4053d30 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.357+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ccc15a500 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.357+0000 7f1cb3fff700 1 -- 172.21.15.7:0/3700350532 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f1ccc15a500 con 0x7f1ccc15b4a0 2023-06-09T11:46:53.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:53.362 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:51.818490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:resolve","state_seq":16,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 msgr2=0x7f1cb806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 0x7f1cb806d160 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0090b0 tx=0x7f1cbc01c040 comp rx=0 tx=0).stop 2023-06-09T11:46:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 msgr2=0x7f1ccc06ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:53.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400cbd0 tx=0x7f1cc400cee0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 shutdown_connections 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ccc15b4a0 0x7f1ccc06ace0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1ccc0c9f40 0x7f1ccc1e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1cb806acb0 0x7f1cb806d160 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 --2- 172.21.15.7:0/3700350532 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1ccc153880 0x7f1ccc0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 >> 172.21.15.7:0/3700350532 conn(0x7f1ccc0737f0 msgr2=0x7f1ccc15c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 shutdown_connections 2023-06-09T11:46:53.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:53.360+0000 7f1cd1bff700 1 -- 172.21.15.7:0/3700350532 wait complete. 2023-06-09T11:46:53.366 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 170 2023-06-09T11:46:53.382 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:46:55.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.590+0000 7fa7fe3ff700 1 Processor -- start 2023-06-09T11:46:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.590+0000 7fa7fe3ff700 1 -- start start 2023-06-09T11:46:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fe3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7f8153880 0x7fa7f815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fe3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f80c9f40 0x7fa7f815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fe3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7f815a500 con 0x7fa7f815b2d0 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fe3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f80c5f10 con 0x7fa7f8153880 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fe3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f815da00 con 0x7fa7f80c9f40 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7f8153880 0x7fa7f815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7f7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa7f8153880 0x7fa7f815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39054/0 (socket says 172.21.15.7:39054) 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7f7fff700 1 -- 172.21.15.7:0/3573350795 learned_addr learned my addr 172.21.15.7:0/3573350795 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.591+0000 7fa7fc99c700 1 --1- 172.21.15.7:0/3573350795 >> v1:172.21.15.7:6789/0 conn(0x7fa7f815b2d0 0x7fa7f80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33360/0 (socket says 172.21.15.7:33360) 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f77fe700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f80c9f40 0x7fa7f815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f7fff700 1 -- 172.21.15.7:0/3573350795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f80c9f40 msgr2=0x7fa7f815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f7fff700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f80c9f40 0x7fa7f815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f7fff700 1 -- 172.21.15.7:0/3573350795 >> v1:172.21.15.7:6789/0 conn(0x7fa7f815b2d0 legacy=0x7fa7f80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f7fff700 1 -- 172.21.15.7:0/3573350795 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f815b750 con 0x7fa7f8153880 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f6ffd700 1 -- 172.21.15.7:0/3573350795 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3057820941 0 0) 0x7fa7f0001490 con 0x7fa7f815b2d0 2023-06-09T11:46:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f7fff700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f815d4c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80098f0 tx=0x7fa7e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f256e6a38528207a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f6ffd700 1 -- 172.21.15.7:0/3573350795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800b8a0 con 0x7fa7f8153880 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f6ffd700 1 -- 172.21.15.7:0/3573350795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e800ba00 con 0x7fa7f8153880 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.592+0000 7fa7f6ffd700 1 -- 172.21.15.7:0/3573350795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e800bd90 con 0x7fa7f8153880 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/3573350795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 msgr2=0x7fa7f815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f815d4c0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa7e80098f0 tx=0x7fa7e8009c00 comp rx=0 tx=0).stop 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/3573350795 shutdown_connections 2023-06-09T11:46:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f815d4c0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/3573350795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f80c9f40 0x7fa7f815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/3573350795 >> 172.21.15.7:0/3573350795 conn(0x7fa7f80737f0 msgr2=0x7fa7f80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/3573350795 shutdown_connections 2023-06-09T11:46:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.593+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/3573350795 wait complete. 2023-06-09T11:46:55.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.595+0000 7fa7fe3ff700 1 Processor -- start 2023-06-09T11:46:55.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.595+0000 7fa7fe3ff700 1 -- start start 2023-06-09T11:46:55.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f80c9f40 0x7fa7f814c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f815b2d0 0x7fa7f80b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8151d60 con 0x7fa7f8153880 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f8152320 con 0x7fa7f80c9f40 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7fe3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f814ce50 con 0x7fa7f815b2d0 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:55.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7f77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39068/0 (socket says 172.21.15.7:39068) 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7f77fe700 1 -- 172.21.15.7:0/1036409704 learned_addr learned my addr 172.21.15.7:0/1036409704 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7f77fe700 1 -- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f815b2d0 msgr2=0x7fa7f80b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.596+0000 7fa7f77fe700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f815b2d0 0x7fa7f80b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f77fe700 1 -- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f80c9f40 msgr2=0x7fa7f814c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f77fe700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f80c9f40 0x7fa7f814c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f77fe700 1 -- 172.21.15.7:0/1036409704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e8009550 con 0x7fa7f8153880 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f77fe700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e840 tx=0x7fa7ec00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:55.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec013610 con 0x7fa7f8153880 2023-06-09T11:46:55.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec013770 con 0x7fa7f8153880 2023-06-09T11:46:55.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f8159980 con 0x7fa7f8153880 2023-06-09T11:46:55.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec013b20 con 0x7fa7f8153880 2023-06-09T11:46:55.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.597+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f81596e0 con 0x7fa7f8153880 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.598+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f8159c60 con 0x7fa7f8153880 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.599+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa7ec013c80 con 0x7fa7f8153880 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.600+0000 7fa7f4ff9700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 0x7fa7e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.600+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fa7ec0898e0 con 0x7fa7f8153880 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.600+0000 7fa7f7fff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 0x7fa7e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.601+0000 7fa7f7fff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 0x7fa7e406d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009c40 tx=0x7fa7e8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:55.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.604+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa7ec054220 con 0x7fa7f8153880 2023-06-09T11:46:55.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.857+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7f81524d0 con 0x7fa7f8153880 2023-06-09T11:46:55.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.857+0000 7fa7f4ff9700 1 -- 172.21.15.7:0/1036409704 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 170 v170) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fa7f81524d0 con 0x7fa7f8153880 2023-06-09T11:46:55.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:55.862 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":170,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":170,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:51.818490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:resolve","state_seq":16,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:55.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.859+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 msgr2=0x7fa7e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.859+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 0x7fa7e406d190 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8009c40 tx=0x7fa7e8010000 comp rx=0 tx=0).stop 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.859+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 msgr2=0x7fa7f81571a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.859+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00e840 tx=0x7fa7ec00ec00 comp rx=0 tx=0).stop 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 shutdown_connections 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa7e406ace0 0x7fa7e406d190 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa7f815b2d0 0x7fa7f80b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa7f80c9f40 0x7fa7f814c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 --2- 172.21.15.7:0/1036409704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa7f8153880 0x7fa7f81571a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 >> 172.21.15.7:0/1036409704 conn(0x7fa7f80737f0 msgr2=0x7fa7f814e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 shutdown_connections 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:55.860+0000 7fa7fe3ff700 1 -- 172.21.15.7:0/1036409704 wait complete. 2023-06-09T11:46:55.865 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 170 2023-06-09T11:46:55.882 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:46:57.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:46:58.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.089+0000 7fa6361ff700 1 Processor -- start 2023-06-09T11:46:58.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.090+0000 7fa6361ff700 1 -- start start 2023-06-09T11:46:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa6361ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa630153a70 0x7fa6300c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa6361ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6300c2d90 0x7fa63015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa6361ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6300c88d0 con 0x7fa6300ca7e0 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa6361ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6300c2bf0 con 0x7fa6300c2d90 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa6361ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa630153e40 con 0x7fa630153a70 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa62effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6300c2d90 0x7fa63015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa62ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6300ca7e0 0x7fa6300cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33364/0 (socket says 172.21.15.7:33364) 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa62ffff700 1 -- 172.21.15.7:0/2062497782 learned_addr learned my addr 172.21.15.7:0/2062497782 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:46:58.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa62effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa6300c2d90 0x7fa63015a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39078/0 (socket says 172.21.15.7:39078) 2023-06-09T11:46:58.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.091+0000 7fa62f7fe700 1 --2- 172.21.15.7:0/2062497782 >> v2:172.21.15.117:3300/0 conn(0x7fa630153a70 0x7fa6300c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62effd700 1 -- 172.21.15.7:0/2062497782 >> v2:172.21.15.117:3300/0 conn(0x7fa630153a70 msgr2=0x7fa6300c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62effd700 1 --2- 172.21.15.7:0/2062497782 >> v2:172.21.15.117:3300/0 conn(0x7fa630153a70 0x7fa6300c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62effd700 1 -- 172.21.15.7:0/2062497782 >> v1:172.21.15.7:6789/0 conn(0x7fa6300ca7e0 legacy=0x7fa6300cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62effd700 1 -- 172.21.15.7:0/2062497782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63015a630 con 0x7fa6300c2d90 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62e7fc700 1 -- 172.21.15.7:0/2062497782 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288569047 0 0) 0x7fa6300c88d0 con 0x7fa6300ca7e0 2023-06-09T11:46:58.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62effd700 1 --2- 172.21.15.7:0/2062497782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa63015a0f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa630071c50 tx=0x7fa624009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beb2211646a3fb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62e7fc700 1 -- 172.21.15.7:0/2062497782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624011830 con 0x7fa6300c2d90 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62e7fc700 1 -- 172.21.15.7:0/2062497782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624011990 con 0x7fa6300c2d90 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.092+0000 7fa62e7fc700 1 -- 172.21.15.7:0/2062497782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624011d20 con 0x7fa6300c2d90 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 -- 172.21.15.7:0/2062497782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 msgr2=0x7fa63015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 --2- 172.21.15.7:0/2062497782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa63015a0f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa630071c50 tx=0x7fa624009b20 comp rx=0 tx=0).stop 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 -- 172.21.15.7:0/2062497782 shutdown_connections 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 --2- 172.21.15.7:0/2062497782 >> v2:172.21.15.117:3300/0 conn(0x7fa630153a70 0x7fa6300c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 --2- 172.21.15.7:0/2062497782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa63015a0f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 -- 172.21.15.7:0/2062497782 >> 172.21.15.7:0/2062497782 conn(0x7fa63006e410 msgr2=0x7fa63006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:58.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 -- 172.21.15.7:0/2062497782 shutdown_connections 2023-06-09T11:46:58.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.093+0000 7fa6361ff700 1 -- 172.21.15.7:0/2062497782 wait complete. 2023-06-09T11:46:58.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.095+0000 7fa6361ff700 1 Processor -- start 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.095+0000 7fa6361ff700 1 -- start start 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6300ca7e0 0x7fa630151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa630153a70 0x7fa6301f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300c5390 con 0x7fa6300c2d90 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6300cdb60 con 0x7fa630153a70 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa6361ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6301efcf0 con 0x7fa6300ca7e0 2023-06-09T11:46:58.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39084/0 (socket says 172.21.15.7:39084) 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 -- 172.21.15.7:0/1777763068 learned_addr learned my addr 172.21.15.7:0/1777763068 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62ffff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa630153a70 0x7fa6301f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62effd700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6300ca7e0 0x7fa630151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 -- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6300ca7e0 msgr2=0x7fa630151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6300ca7e0 0x7fa630151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 -- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa630153a70 msgr2=0x7fa6301f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa630153a70 0x7fa6301f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.096+0000 7fa62f7fe700 1 -- 172.21.15.7:0/1777763068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240094a0 con 0x7fa6300c2d90 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa62f7fe700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa62000e8a0 tx=0x7fa62000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa620009590 con 0x7fa6300c2d90 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62000eea0 con 0x7fa6300c2d90 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6300c8350 con 0x7fa6300c2d90 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62000c9f0 con 0x7fa6300c2d90 2023-06-09T11:46:58.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.097+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6300ca350 con 0x7fa6300c2d90 2023-06-09T11:46:58.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.098+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6300c85f0 con 0x7fa6300c2d90 2023-06-09T11:46:58.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.100+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa62000cb50 con 0x7fa6300c2d90 2023-06-09T11:46:58.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.101+0000 7fa617fff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 0x7fa61c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:46:58.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.101+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fa62008a2e0 con 0x7fa6300c2d90 2023-06-09T11:46:58.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.102+0000 7fa62effd700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 0x7fa61c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:46:58.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.102+0000 7fa62effd700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 0x7fa61c06d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa6240090b0 tx=0x7fa624008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:46:58.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.104+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa620058730 con 0x7fa6300c2d90 2023-06-09T11:46:58.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.361+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa630074d20 con 0x7fa6300c2d90 2023-06-09T11:46:58.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.361+0000 7fa617fff700 1 -- 172.21.15.7:0/1777763068 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5467 (secure 0 0 0) 0x7fa620017090 con 0x7fa6300c2d90 2023-06-09T11:46:58.366 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:46:58.366 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:57.620363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:46:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 msgr2=0x7fa61c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 0x7fa61c06d190 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa6240090b0 tx=0x7fa624008030 comp rx=0 tx=0).stop 2023-06-09T11:46:58.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 msgr2=0x7fa6300cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa62000e8a0 tx=0x7fa62000ec60 comp rx=0 tx=0).stop 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 shutdown_connections 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa630153a70 0x7fa6301f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6300c2d90 0x7fa6300cd620 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa61c06ace0 0x7fa61c06d190 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 --2- 172.21.15.7:0/1777763068 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6300ca7e0 0x7fa630151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 >> 172.21.15.7:0/1777763068 conn(0x7fa63006e410 msgr2=0x7fa6301555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:46:58.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 shutdown_connections 2023-06-09T11:46:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:46:58.364+0000 7fa6361ff700 1 -- 172.21.15.7:0/1777763068 wait complete. 2023-06-09T11:46:58.370 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 171 2023-06-09T11:46:58.386 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 171, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 171, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7141": { "addr": "172.21.15.117:6837/3727664816", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3727664816, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3727664816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7141, "incarnation": 159, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7279": { "addr": "172.21.15.117:6833/3302752268", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 3302752268, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 3302752268, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7279, "incarnation": 169, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:reconnect", "state_seq": 18 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:46:57.620363+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7141, "mds_2": 7279 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:47:00.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:00.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.594+0000 7f3f50bff700 1 Processor -- start 2023-06-09T11:47:00.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.594+0000 7f3f50bff700 1 -- start start 2023-06-09T11:47:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f50bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f50bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f4c0c9f40 0x7f3f4c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f50bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c15a500 con 0x7f3f4c153880 2023-06-09T11:47:00.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f50bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c0c5f10 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f50bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c15da00 con 0x7f3f4c0c9f40 2023-06-09T11:47:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39088/0 (socket says 172.21.15.7:39088) 2023-06-09T11:47:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 -- 172.21.15.7:0/1347706229 learned_addr learned my addr 172.21.15.7:0/1347706229 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f4ad9d700 1 --2- 172.21.15.7:0/1347706229 >> v2:172.21.15.117:3300/0 conn(0x7f3f4c0c9f40 0x7f3f4c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 -- 172.21.15.7:0/1347706229 >> v2:172.21.15.117:3300/0 conn(0x7f3f4c0c9f40 msgr2=0x7f3f4c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:00.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 --2- 172.21.15.7:0/1347706229 >> v2:172.21.15.117:3300/0 conn(0x7f3f4c0c9f40 0x7f3f4c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 -- 172.21.15.7:0/1347706229 >> v1:172.21.15.7:6789/0 conn(0x7f3f4c153880 legacy=0x7f3f4c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.596+0000 7f3f49d9b700 1 -- 172.21.15.7:0/1347706229 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c1596e0 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f49d9b700 1 --2- 172.21.15.7:0/1347706229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0749f0 tx=0x7f3f40009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa0630b170ea7a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f4959a700 1 -- 172.21.15.7:0/1347706229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011830 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f4959a700 1 -- 172.21.15.7:0/1347706229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40011990 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f4959a700 1 -- 172.21.15.7:0/1347706229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011d20 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 -- 172.21.15.7:0/1347706229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 msgr2=0x7f3f4c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 --2- 172.21.15.7:0/1347706229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0749f0 tx=0x7f3f40009bd0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 -- 172.21.15.7:0/1347706229 shutdown_connections 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 --2- 172.21.15.7:0/1347706229 >> v2:172.21.15.117:3300/0 conn(0x7f3f4c0c9f40 0x7f3f4c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 --2- 172.21.15.7:0/1347706229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c15d4c0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.597+0000 7f3f50bff700 1 -- 172.21.15.7:0/1347706229 >> 172.21.15.7:0/1347706229 conn(0x7f3f4c0737f0 msgr2=0x7f3f4c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.598+0000 7f3f50bff700 1 -- 172.21.15.7:0/1347706229 shutdown_connections 2023-06-09T11:47:00.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.598+0000 7f3f50bff700 1 -- 172.21.15.7:0/1347706229 wait complete. 2023-06-09T11:47:00.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.599+0000 7f3f50bff700 1 Processor -- start 2023-06-09T11:47:00.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.599+0000 7f3f50bff700 1 -- start start 2023-06-09T11:47:00.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 0x7f3f4c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f4c153880 0x7f3f4c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c1594b0 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0cea30 con 0x7f3f4c0c9f40 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f50bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c14b630 con 0x7f3f4c153880 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.600+0000 7f3f4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39102/0 (socket says 172.21.15.7:39102) 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 -- 172.21.15.7:0/4029941162 learned_addr learned my addr 172.21.15.7:0/4029941162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:00.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f49d9b700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f4c153880 0x7f3f4c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4a59c700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 0x7f3f4c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 -- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f4c153880 msgr2=0x7f3f4c14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f4c153880 0x7f3f4c14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 -- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 msgr2=0x7f3f4c0caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 0x7f3f4c0caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 -- 172.21.15.7:0/4029941162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40009550 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4a59c700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 0x7f3f4c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:00.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f4ad9d700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009dd0 tx=0x7f3f4400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44009c40 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.601+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400f040 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.602+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c159c60 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.602+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44014480 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.602+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c159980 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.603+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c159f40 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.604+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f440145e0 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.605+0000 7f3f3affd700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 0x7f3f2c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.605+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f3f4408a690 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.605+0000 7f3f4a59c700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 0x7f3f2c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:00.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.605+0000 7f3f4a59c700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 0x7f3f2c06d1c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000f60 tx=0x7f3f3c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.609+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3f44055060 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.860+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f4c0cf030 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.861+0000 7f3f3affd700 1 -- 172.21.15.7:0/4029941162 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 171 v171) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f3f4c0cf030 con 0x7f3f4c15b4a0 2023-06-09T11:47:00.865 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:00.865 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":171,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":171,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:46:57.620363+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:reconnect","state_seq":18,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 msgr2=0x7f3f2c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 0x7f3f2c06d1c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c000f60 tx=0x7f3f3c009040 comp rx=0 tx=0).stop 2023-06-09T11:47:00.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 msgr2=0x7f3f4c14b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009dd0 tx=0x7f3f4400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 shutdown_connections 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f4c15b4a0 0x7f3f4c14b0f0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f4c0c9f40 0x7f3f4c0caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f2c06ad10 0x7f3f2c06d1c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 --2- 172.21.15.7:0/4029941162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f4c153880 0x7f3f4c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:00.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 >> 172.21.15.7:0/4029941162 conn(0x7f3f4c0737f0 msgr2=0x7f3f4c14c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 shutdown_connections 2023-06-09T11:47:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:00.863+0000 7f3f50bff700 1 -- 172.21.15.7:0/4029941162 wait complete. 2023-06-09T11:47:00.869 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 171 2023-06-09T11:47:00.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:47:02.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.093+0000 7fd1753ff700 1 Processor -- start 2023-06-09T11:47:03.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.093+0000 7fd1753ff700 1 -- start start 2023-06-09T11:47:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd1753ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd1753ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd1753ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17015a500 con 0x7fd1700c9f40 2023-06-09T11:47:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd1753ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700c5f10 con 0x7fd17015b4a0 2023-06-09T11:47:03.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd1753ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17015da00 con 0x7fd170153880 2023-06-09T11:47:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:42988/0 (socket says 172.21.15.7:42988) 2023-06-09T11:47:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:37472/0 (socket says 172.21.15.7:37472) 2023-06-09T11:47:03.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16f7fe700 1 -- 172.21.15.7:0/812990361 learned_addr learned my addr 172.21.15.7:0/812990361 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.095+0000 7fd16e7fc700 1 --1- 172.21.15.7:0/812990361 >> v1:172.21.15.117:6789/0 conn(0x7fd1700c9f40 0x7fd17015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:49886/0 (socket says 172.21.15.7:49886) 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16effd700 1 -- 172.21.15.7:0/812990361 >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 msgr2=0x7fd1700c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16effd700 1 --2- 172.21.15.7:0/812990361 >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16effd700 1 -- 172.21.15.7:0/812990361 >> v1:172.21.15.117:6789/0 conn(0x7fd1700c9f40 legacy=0x7fd17015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16effd700 1 -- 172.21.15.7:0/812990361 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1701596e0 con 0x7fd170153880 2023-06-09T11:47:03.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16f7fe700 1 --2- 172.21.15.7:0/812990361 >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.096+0000 7fd16effd700 1 --2- 172.21.15.7:0/812990361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd160009840 tx=0x7fd160009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec3b81e4bcda52f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd16dffb700 1 -- 172.21.15.7:0/812990361 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160011830 con 0x7fd170153880 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd16dffb700 1 -- 172.21.15.7:0/812990361 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160011990 con 0x7fd170153880 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd16dffb700 1 -- 172.21.15.7:0/812990361 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160011d20 con 0x7fd170153880 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 -- 172.21.15.7:0/812990361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 msgr2=0x7fd17015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 --2- 172.21.15.7:0/812990361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd160009840 tx=0x7fd160009b50 comp rx=0 tx=0).stop 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 -- 172.21.15.7:0/812990361 shutdown_connections 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 --2- 172.21.15.7:0/812990361 >> v2:172.21.15.117:3300/0 conn(0x7fd17015b4a0 0x7fd1700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 --2- 172.21.15.7:0/812990361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd17015d4c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 -- 172.21.15.7:0/812990361 >> 172.21.15.7:0/812990361 conn(0x7fd1700737f0 msgr2=0x7fd1700720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 -- 172.21.15.7:0/812990361 shutdown_connections 2023-06-09T11:47:03.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.097+0000 7fd1753ff700 1 -- 172.21.15.7:0/812990361 wait complete. 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.099+0000 7fd1753ff700 1 Processor -- start 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.099+0000 7fd1753ff700 1 -- start start 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd1700cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 0x7fd1701f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1701f2420 con 0x7fd1700c9f40 2023-06-09T11:47:03.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700cca40 con 0x7fd17015b4a0 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd1753ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b3e00 con 0x7fd170153880 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd16effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd16effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41740/0 (socket says 172.21.15.7:41740) 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.100+0000 7fd16effd700 1 -- 172.21.15.7:0/2757580432 learned_addr learned my addr 172.21.15.7:0/2757580432 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 -- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 msgr2=0x7fd1700cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16e7fc700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd1700cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16f7fe700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 0x7fd1701f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd1700cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 -- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 msgr2=0x7fd1701f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 0x7fd1701f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 -- 172.21.15.7:0/2757580432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168009b30 con 0x7fd1700c9f40 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16e7fc700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd1700cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16f7fe700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 0x7fd1701f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:47:03.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd16effd700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600038a0 con 0x7fd1700c9f40 2023-06-09T11:47:03.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16001f070 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160003c20 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.101+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1600094a0 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.102+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd170159c60 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.103+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170159980 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.104+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd160026480 con 0x7fd1700c9f40 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.105+0000 7fd1577fe700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:03.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.105+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fd160021070 con 0x7fd1700c9f40 2023-06-09T11:47:03.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.105+0000 7fd16e7fc700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:03.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.105+0000 7fd16e7fc700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 0x7fd15c06d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd1700ce830 tx=0x7fd164009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:03.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.109+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd160055ec0 con 0x7fd1700c9f40 2023-06-09T11:47:03.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.359+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd170159f40 con 0x7fd1700c9f40 2023-06-09T11:47:03.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.360+0000 7fd1577fe700 1 -- 172.21.15.7:0/2757580432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fd16002b090 con 0x7fd1700c9f40 2023-06-09T11:47:03.364 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:03.364 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:02.782247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 msgr2=0x7fd15c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 0x7fd15c06d190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd1700ce830 tx=0x7fd164009040 comp rx=0 tx=0).stop 2023-06-09T11:47:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 msgr2=0x7fd1700b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fd160009fd0 tx=0x7fd160004750 comp rx=0 tx=0).stop 2023-06-09T11:47:03.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 shutdown_connections 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd17015b4a0 0x7fd1701f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd1700c9f40 0x7fd1700b38c0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd15c06ace0 0x7fd15c06d190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 --2- 172.21.15.7:0/2757580432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd170153880 0x7fd1700cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 >> 172.21.15.7:0/2757580432 conn(0x7fd1700737f0 msgr2=0x7fd17006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:03.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 shutdown_connections 2023-06-09T11:47:03.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:03.363+0000 7fd1753ff700 1 -- 172.21.15.7:0/2757580432 wait complete. 2023-06-09T11:47:03.369 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 172 2023-06-09T11:47:03.385 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:47:05.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.592+0000 7fa1371ff700 1 Processor -- start 2023-06-09T11:47:05.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.592+0000 7fa1371ff700 1 -- start start 2023-06-09T11:47:05.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.593+0000 7fa1371ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa130153880 0x7fa13015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.593+0000 7fa1371ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa13015b4a0 0x7fa1300c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.593+0000 7fa1371ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa13015a500 con 0x7fa1300c9f40 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.593+0000 7fa1371ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1300c5f10 con 0x7fa13015b4a0 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa1371ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13015da00 con 0x7fa130153880 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa13015b4a0 0x7fa1300c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa13015b4a0 0x7fa1300c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41748/0 (socket says 172.21.15.7:41748) 2023-06-09T11:47:05.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 -- 172.21.15.7:0/1479174371 learned_addr learned my addr 172.21.15.7:0/1479174371 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa134f9b700 1 --2- 172.21.15.7:0/1479174371 >> v2:172.21.15.117:3300/0 conn(0x7fa130153880 0x7fa13015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 -- 172.21.15.7:0/1479174371 >> v2:172.21.15.117:3300/0 conn(0x7fa130153880 msgr2=0x7fa13015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 --2- 172.21.15.7:0/1479174371 >> v2:172.21.15.117:3300/0 conn(0x7fa130153880 0x7fa13015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 -- 172.21.15.7:0/1479174371 >> v1:172.21.15.7:6789/0 conn(0x7fa1300c9f40 legacy=0x7fa13015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 -- 172.21.15.7:0/1479174371 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1301596e0 con 0x7fa13015b4a0 2023-06-09T11:47:05.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.594+0000 7fa13579c700 1 --2- 172.21.15.7:0/1479174371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13015b4a0 0x7fa1300c4e30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa13008d340 tx=0x7fa12800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b88d3d459f44d01 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa12f7fe700 1 -- 172.21.15.7:0/1479174371 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800e070 con 0x7fa13015b4a0 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa12f7fe700 1 -- 172.21.15.7:0/1479174371 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12800b9c0 con 0x7fa13015b4a0 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa12f7fe700 1 -- 172.21.15.7:0/1479174371 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800bd70 con 0x7fa13015b4a0 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 -- 172.21.15.7:0/1479174371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13015b4a0 msgr2=0x7fa1300c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1479174371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13015b4a0 0x7fa1300c4e30 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa13008d340 tx=0x7fa12800b330 comp rx=0 tx=0).stop 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 -- 172.21.15.7:0/1479174371 shutdown_connections 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1479174371 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa13015b4a0 0x7fa1300c4e30 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1479174371 >> v2:172.21.15.117:3300/0 conn(0x7fa130153880 0x7fa13015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 -- 172.21.15.7:0/1479174371 >> 172.21.15.7:0/1479174371 conn(0x7fa1300737f0 msgr2=0x7fa1300720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 -- 172.21.15.7:0/1479174371 shutdown_connections 2023-06-09T11:47:05.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.595+0000 7fa1371ff700 1 -- 172.21.15.7:0/1479174371 wait complete. 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.597+0000 7fa1371ff700 1 Processor -- start 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.597+0000 7fa1371ff700 1 -- start start 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 0x7fa1300ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 0x7fa1301e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300ceb40 con 0x7fa130153880 2023-06-09T11:47:05.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300cd8e0 con 0x7fa1300c9f40 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.598+0000 7fa1371ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1300cf320 con 0x7fa13015b4a0 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41760/0 (socket says 172.21.15.7:41760) 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 -- 172.21.15.7:0/1821089297 learned_addr learned my addr 172.21.15.7:0/1821089297 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa13579c700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 0x7fa1301e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa134f9b700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 0x7fa1300ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 -- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 msgr2=0x7fa1301e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 0x7fa1301e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 -- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 msgr2=0x7fa1300ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 0x7fa1300ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 -- 172.21.15.7:0/1821089297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009b30 con 0x7fa130153880 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa13579c700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 0x7fa1301e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa134f9b700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 0x7fa1300ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:05.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12ffff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c510 tx=0x7fa11c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.599+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0052d0 con 0x7fa130153880 2023-06-09T11:47:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.600+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c005430 con 0x7fa130153880 2023-06-09T11:47:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.600+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c0038c0 con 0x7fa130153880 2023-06-09T11:47:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.600+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa130159c60 con 0x7fa130153880 2023-06-09T11:47:05.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.600+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa130159980 con 0x7fa130153880 2023-06-09T11:47:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.601+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa130159f40 con 0x7fa130153880 2023-06-09T11:47:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.602+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa11c003a20 con 0x7fa130153880 2023-06-09T11:47:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.603+0000 7fa12d7fa700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 0x7fa11406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.603+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fa11c088fb0 con 0x7fa130153880 2023-06-09T11:47:05.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.604+0000 7fa134f9b700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 0x7fa11406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:05.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.604+0000 7fa134f9b700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 0x7fa11406d1c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa1240099f0 tx=0x7fa124008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:05.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.608+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa11c053860 con 0x7fa130153880 2023-06-09T11:47:05.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.862+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa13015a500 con 0x7fa130153880 2023-06-09T11:47:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.863+0000 7fa12d7fa700 1 -- 172.21.15.7:0/1821089297 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fa13015a500 con 0x7fa130153880 2023-06-09T11:47:05.867 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:05.867 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:02.782247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.865+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 msgr2=0x7fa11406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.865+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 0x7fa11406d1c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa1240099f0 tx=0x7fa124008040 comp rx=0 tx=0).stop 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.865+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 msgr2=0x7fa1300cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.865+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00c510 tx=0x7fa11c00c8d0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 shutdown_connections 2023-06-09T11:47:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa11406ad10 0x7fa11406d1c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa13015b4a0 0x7fa1301e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1300c9f40 0x7fa1300ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 --2- 172.21.15.7:0/1821089297 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa130153880 0x7fa1300cd3a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 >> 172.21.15.7:0/1821089297 conn(0x7fa1300737f0 msgr2=0x7fa13014e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 shutdown_connections 2023-06-09T11:47:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:05.866+0000 7fa1371ff700 1 -- 172.21.15.7:0/1821089297 wait complete. 2023-06-09T11:47:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 172 2023-06-09T11:47:05.888 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:47:07.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:08.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.095+0000 7f74299ff700 1 Processor -- start 2023-06-09T11:47:08.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.095+0000 7f74299ff700 1 -- start start 2023-06-09T11:47:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74299ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 0x7f742415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74299ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74240c9f40 0x7f742415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74299ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742415a500 con 0x7f742415b2d0 2023-06-09T11:47:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74299ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240c5f10 con 0x7f74240c9f40 2023-06-09T11:47:08.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74299ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742415da00 con 0x7f7424153880 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74227fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74240c9f40 0x7f742415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74227fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f74240c9f40 0x7f742415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41772/0 (socket says 172.21.15.7:41772) 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74227fc700 1 -- 172.21.15.7:0/1726555965 learned_addr learned my addr 172.21.15.7:0/1726555965 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f74237fe700 1 --1- 172.21.15.7:0/1726555965 >> v1:172.21.15.7:6789/0 conn(0x7f742415b2d0 0x7f74240c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40820/0 (socket says 172.21.15.7:40820) 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.097+0000 7f7422ffd700 1 --2- 172.21.15.7:0/1726555965 >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 0x7f742415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f74227fc700 1 -- 172.21.15.7:0/1726555965 >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 msgr2=0x7f742415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f74227fc700 1 --2- 172.21.15.7:0/1726555965 >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 0x7f742415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f74227fc700 1 -- 172.21.15.7:0/1726555965 >> v1:172.21.15.7:6789/0 conn(0x7f742415b2d0 legacy=0x7f74240c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f74227fc700 1 -- 172.21.15.7:0/1726555965 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74241596e0 con 0x7f74240c9f40 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f7422ffd700 1 --2- 172.21.15.7:0/1726555965 >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 0x7f742415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f7421ffb700 1 -- 172.21.15.7:0/1726555965 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483014288 0 0) 0x7f741c001490 con 0x7f742415b2d0 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f74227fc700 1 --2- 172.21.15.7:0/1726555965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74240c9f40 0x7f742415ad90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f742408d340 tx=0x7f7418009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f301b1281fdd8e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f7421ffb700 1 -- 172.21.15.7:0/1726555965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800b870 con 0x7f74240c9f40 2023-06-09T11:47:08.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f7421ffb700 1 -- 172.21.15.7:0/1726555965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741800b9d0 con 0x7f74240c9f40 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.098+0000 7f7421ffb700 1 -- 172.21.15.7:0/1726555965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741800bd60 con 0x7f74240c9f40 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 -- 172.21.15.7:0/1726555965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74240c9f40 msgr2=0x7f742415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 --2- 172.21.15.7:0/1726555965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74240c9f40 0x7f742415ad90 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f742408d340 tx=0x7f7418009bd0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 -- 172.21.15.7:0/1726555965 shutdown_connections 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 --2- 172.21.15.7:0/1726555965 >> v2:172.21.15.117:3300/0 conn(0x7f7424153880 0x7f742415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 --2- 172.21.15.7:0/1726555965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74240c9f40 0x7f742415ad90 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 -- 172.21.15.7:0/1726555965 >> 172.21.15.7:0/1726555965 conn(0x7f74240737f0 msgr2=0x7f74240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 -- 172.21.15.7:0/1726555965 shutdown_connections 2023-06-09T11:47:08.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.099+0000 7f74299ff700 1 -- 172.21.15.7:0/1726555965 wait complete. 2023-06-09T11:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.100+0000 7f74299ff700 1 Processor -- start 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.101+0000 7f74299ff700 1 -- start start 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.101+0000 7f74299ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74240c9f40 0x7f7424154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74299ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74299ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f742415b2d0 0x7f74241eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74299ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7424153e90 con 0x7f7424153880 2023-06-09T11:47:08.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74299ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74240b4030 con 0x7f742415b2d0 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74299ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74241540a0 con 0x7f74240c9f40 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41780/0 (socket says 172.21.15.7:41780) 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 -- 172.21.15.7:0/1905895356 learned_addr learned my addr 172.21.15.7:0/1905895356 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74237fe700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f742415b2d0 0x7f74241eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f7422ffd700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74240c9f40 0x7f7424154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 -- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74240c9f40 msgr2=0x7f7424154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74240c9f40 0x7f7424154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 -- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f742415b2d0 msgr2=0x7f74241eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f742415b2d0 0x7f74241eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.102+0000 7f74227fc700 1 -- 172.21.15.7:0/1905895356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7418009550 con 0x7f7424153880 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f74227fc700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7418009c10 tx=0x7f7418004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418004250 con 0x7f7424153880 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741801f070 con 0x7f7424153880 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7424159c60 con 0x7f7424153880 2023-06-09T11:47:08.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180045d0 con 0x7f7424153880 2023-06-09T11:47:08.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.103+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7424159980 con 0x7f7424153880 2023-06-09T11:47:08.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.104+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7424159f40 con 0x7f7424153880 2023-06-09T11:47:08.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.105+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7418021030 con 0x7f7424153880 2023-06-09T11:47:08.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.106+0000 7f740bfff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:08.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.106+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f741808af20 con 0x7f7424153880 2023-06-09T11:47:08.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.106+0000 7f7422ffd700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:08.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.107+0000 7f7422ffd700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 0x7f741006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7414009930 tx=0x7f7414008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:08.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.111+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f74180558f0 con 0x7f7424153880 2023-06-09T11:47:08.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.361+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7424074cf0 con 0x7f7424153880 2023-06-09T11:47:08.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.362+0000 7f740bfff700 1 -- 172.21.15.7:0/1905895356 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 172 v172) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f7418015090 con 0x7f7424153880 2023-06-09T11:47:08.366 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:08.366 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":172,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":172,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:02.782247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:rejoin","state_seq":20,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 msgr2=0x7f741006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 0x7f741006d190 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7414009930 tx=0x7f7414008040 comp rx=0 tx=0).stop 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 msgr2=0x7f74240b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7418009c10 tx=0x7f7418004750 comp rx=0 tx=0).stop 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 shutdown_connections 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f742415b2d0 0x7f74241eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f741006ace0 0x7f741006d190 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f74240c9f40 0x7f7424154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 --2- 172.21.15.7:0/1905895356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7424153880 0x7f74240b35c0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 >> 172.21.15.7:0/1905895356 conn(0x7f74240737f0 msgr2=0x7f74240cc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 shutdown_connections 2023-06-09T11:47:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:08.364+0000 7f74299ff700 1 -- 172.21.15.7:0/1905895356 wait complete. 2023-06-09T11:47:08.370 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 172 2023-06-09T11:47:08.387 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:47:10.387 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.594+0000 7f44f25ff700 1 Processor -- start 2023-06-09T11:47:10.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.594+0000 7f44f25ff700 1 -- start start 2023-06-09T11:47:10.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44f25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 0x7f44ec0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44f25ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec0c5a80 con 0x7f44ec15b4a0 2023-06-09T11:47:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44f25ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec15a500 con 0x7f44ec153880 2023-06-09T11:47:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44f25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec0c5f10 con 0x7f44ec0c9f40 2023-06-09T11:47:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f44ec15b4a0 0x7f44ec0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40826/0 (socket says 172.21.15.7:40826) 2023-06-09T11:47:10.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eb7fe700 1 -- 172.21.15.7:0/3021916759 learned_addr learned my addr 172.21.15.7:0/3021916759 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44f0b9c700 1 --2- 172.21.15.7:0/3021916759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 0x7f44ec0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501322200 0 0) 0x7f44ec0c5a80 con 0x7f44ec15b4a0 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d40035b0 con 0x7f44ec15b4a0 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890533576 0 0) 0x7f44dc001110 con 0x7f44ec153880 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.596+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44ec0c5a80 con 0x7f44ec153880 2023-06-09T11:47:10.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504556372 0 0) 0x7f44d40035b0 con 0x7f44ec15b4a0 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44f0b9c700 1 -- 172.21.15.7:0/3021916759 >> v1:172.21.15.117:6789/0 conn(0x7f44ec153880 legacy=0x7f44ec15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44f0b9c700 1 -- 172.21.15.7:0/3021916759 >> v1:172.21.15.7:6789/0 conn(0x7f44ec15b4a0 legacy=0x7f44ec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44f0b9c700 1 -- 172.21.15.7:0/3021916759 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec1596e0 con 0x7f44ec0c9f40 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44f0b9c700 1 --2- 172.21.15.7:0/3021916759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 0x7f44ec0c5540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009dc0 tx=0x7f44e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ae24b829250a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400e070 con 0x7f44ec0c9f40 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4003a20 con 0x7f44ec0c9f40 2023-06-09T11:47:10.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.597+0000 7f44eaffd700 1 -- 172.21.15.7:0/3021916759 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0001670 con 0x7f44ec0c9f40 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 -- 172.21.15.7:0/3021916759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 msgr2=0x7f44ec0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 --2- 172.21.15.7:0/3021916759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 0x7f44ec0c5540 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009dc0 tx=0x7f44e400b330 comp rx=0 tx=0).stop 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 -- 172.21.15.7:0/3021916759 shutdown_connections 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 --2- 172.21.15.7:0/3021916759 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec0c9f40 0x7f44ec0c5540 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 -- 172.21.15.7:0/3021916759 >> 172.21.15.7:0/3021916759 conn(0x7f44ec0737f0 msgr2=0x7f44ec0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 -- 172.21.15.7:0/3021916759 shutdown_connections 2023-06-09T11:47:10.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.598+0000 7f44f25ff700 1 -- 172.21.15.7:0/3021916759 wait complete. 2023-06-09T11:47:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.599+0000 7f44f25ff700 1 Processor -- start 2023-06-09T11:47:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.600+0000 7f44f25ff700 1 -- start start 2023-06-09T11:47:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.600+0000 7f44f25ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ec0c9f40 0x7f44ec158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f25ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f25ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec15b4a0 0x7f44ec148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f25ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec14d850 con 0x7f44ec153880 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f25ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec158aa0 con 0x7f44ec0c9f40 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f25ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec1f0080 con 0x7f44ec15b4a0 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41788/0 (socket says 172.21.15.7:41788) 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 -- 172.21.15.7:0/2809647968 learned_addr learned my addr 172.21.15.7:0/2809647968 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44f0b9c700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec15b4a0 0x7f44ec148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44ebfff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ec0c9f40 0x7f44ec158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 -- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec15b4a0 msgr2=0x7f44ec148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec15b4a0 0x7f44ec148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 -- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ec0c9f40 msgr2=0x7f44ec158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ec0c9f40 0x7f44ec158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.601+0000 7f44eb7fe700 1 -- 172.21.15.7:0/2809647968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009a70 con 0x7f44ec153880 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44eb7fe700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c500 tx=0x7f44e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0004c80 con 0x7f44ec153880 2023-06-09T11:47:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0004e00 con 0x7f44ec153880 2023-06-09T11:47:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec159c60 con 0x7f44ec153880 2023-06-09T11:47:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e0007310 con 0x7f44ec153880 2023-06-09T11:47:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.602+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec159980 con 0x7f44ec153880 2023-06-09T11:47:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.603+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44ec159f40 con 0x7f44ec153880 2023-06-09T11:47:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.604+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44e0007470 con 0x7f44ec153880 2023-06-09T11:47:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.605+0000 7f44e8ff9700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 0x7f44d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.605+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f44e0089980 con 0x7f44ec153880 2023-06-09T11:47:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.606+0000 7f44ebfff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 0x7f44d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.606+0000 7f44ebfff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 0x7f44d806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0012b0 tx=0x7f44dc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.610+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f44e0054350 con 0x7f44ec153880 2023-06-09T11:47:10.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.863+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44ec158be0 con 0x7f44ec153880 2023-06-09T11:47:10.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.864+0000 7f44e8ff9700 1 -- 172.21.15.7:0/2809647968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5470 (secure 0 0 0) 0x7f44ec158be0 con 0x7f44ec153880 2023-06-09T11:47:10.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:10.868 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:08.628384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:clientreplay","state_seq":22,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 msgr2=0x7f44d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 0x7f44d806d190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0012b0 tx=0x7f44dc009040 comp rx=0 tx=0).stop 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 msgr2=0x7f44ec147ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f44e000c500 tx=0x7f44e000c8c0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 shutdown_connections 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f44d806ace0 0x7f44d806d190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f44ec15b4a0 0x7f44ec148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f44ec0c9f40 0x7f44ec158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 --2- 172.21.15.7:0/2809647968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44ec153880 0x7f44ec147ee0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 >> 172.21.15.7:0/2809647968 conn(0x7f44ec0737f0 msgr2=0x7f44ec15c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 shutdown_connections 2023-06-09T11:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:10.866+0000 7f44f25ff700 1 -- 172.21.15.7:0/2809647968 wait complete. 2023-06-09T11:47:10.872 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 173 2023-06-09T11:47:10.889 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 173, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 173, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7141": { "addr": "172.21.15.117:6837/3727664816", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3727664816, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3727664816, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7141, "incarnation": 159, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7279": { "addr": "172.21.15.117:6833/3302752268", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 3302752268, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 3302752268, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7279, "incarnation": 169, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:clientreplay", "state_seq": 22 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 69, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:47:08.628384+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7141, "mds_2": 7279 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:47:12.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:13.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.095+0000 7f1af69ff700 1 Processor -- start 2023-06-09T11:47:13.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.096+0000 7f1af69ff700 1 -- start start 2023-06-09T11:47:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af69ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1af0153880 0x7f1af015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af69ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af69ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af015a500 con 0x7f1af00c9f40 2023-06-09T11:47:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af69ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af00c5f10 con 0x7f1af0153880 2023-06-09T11:47:13.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af69ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af015da00 con 0x7f1af015b4a0 2023-06-09T11:47:13.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1aeffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1af0153880 0x7f1af015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1af4f9c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1aeffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1af0153880 0x7f1af015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:51492/0 (socket says 172.21.15.7:51492) 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.097+0000 7f1aeffff700 1 -- 172.21.15.7:0/3102457829 learned_addr learned my addr 172.21.15.7:0/3102457829 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeffff700 1 -- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 msgr2=0x7f1af00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeffff700 1 --2- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeffff700 1 -- 172.21.15.7:0/3102457829 >> v1:172.21.15.117:6789/0 conn(0x7f1af00c9f40 legacy=0x7f1af015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeffff700 1 -- 172.21.15.7:0/3102457829 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af01596e0 con 0x7f1af0153880 2023-06-09T11:47:13.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeffff700 1 --2- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0153880 0x7f1af015d4c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54577f1f2784915 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.098+0000 7f1aeeffd700 1 -- 172.21.15.7:0/3102457829 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011830 con 0x7f1af0153880 2023-06-09T11:47:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1aeeffd700 1 -- 172.21.15.7:0/3102457829 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0011990 con 0x7f1af0153880 2023-06-09T11:47:13.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1aeeffd700 1 -- 172.21.15.7:0/3102457829 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0011d20 con 0x7f1af0153880 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 -- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0153880 msgr2=0x7f1af015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 --2- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0153880 0x7f1af015d4c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098f0 tx=0x7f1ae0009c00 comp rx=0 tx=0).stop 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 -- 172.21.15.7:0/3102457829 shutdown_connections 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 --2- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 --2- 172.21.15.7:0/3102457829 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af0153880 0x7f1af015d4c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 -- 172.21.15.7:0/3102457829 >> 172.21.15.7:0/3102457829 conn(0x7f1af00737f0 msgr2=0x7f1af00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 -- 172.21.15.7:0/3102457829 shutdown_connections 2023-06-09T11:47:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.099+0000 7f1af69ff700 1 -- 172.21.15.7:0/3102457829 wait complete. 2023-06-09T11:47:13.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.101+0000 7f1af69ff700 1 Processor -- start 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.101+0000 7f1af69ff700 1 -- start start 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.102+0000 7f1af69ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 0x7f1af00cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.102+0000 7f1af69ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.102+0000 7f1af69ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1af69ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00cec30 con 0x7f1af0153880 2023-06-09T11:47:13.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1af69ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af00cc940 con 0x7f1af00c9f40 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1af69ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af006b2c0 con 0x7f1af015b4a0 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 0x7f1af00cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33170/0 (socket says 172.21.15.7:33170) 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 -- 172.21.15.7:0/1931820006 learned_addr learned my addr 172.21.15.7:0/1931820006 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:13.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1af4f9c700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 -- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 msgr2=0x7f1af00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 -- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 msgr2=0x7f1af00cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 0x7f1af00cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aef7fe700 1 -- 172.21.15.7:0/1931820006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0009550 con 0x7f1af0153880 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.103+0000 7f1aeffff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 0x7f1af00cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1aef7fe700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c510 tx=0x7f1ae400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40052d0 con 0x7f1af0153880 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae4005430 con 0x7f1af0153880 2023-06-09T11:47:13.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40038c0 con 0x7f1af0153880 2023-06-09T11:47:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af0159c60 con 0x7f1af0153880 2023-06-09T11:47:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.104+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af0159980 con 0x7f1af0153880 2023-06-09T11:47:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.105+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af0159f40 con 0x7f1af0153880 2023-06-09T11:47:13.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.106+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1ae4003a20 con 0x7f1af0153880 2023-06-09T11:47:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.107+0000 7f1aecff9700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 0x7f1adc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.107+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f1ae4088e30 con 0x7f1af0153880 2023-06-09T11:47:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.107+0000 7f1aeffff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 0x7f1adc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:13.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.108+0000 7f1aeffff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 0x7f1adc06d1c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009c40 tx=0x7f1ae0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:13.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.111+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1ae40536e0 con 0x7f1af0153880 2023-06-09T11:47:13.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.363+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1af015a500 con 0x7f1af0153880 2023-06-09T11:47:13.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.363+0000 7f1aecff9700 1 -- 172.21.15.7:0/1931820006 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 173 v173) v1 ==== 95+0+5470 (secure 0 0 0) 0x7f1af015a500 con 0x7f1af0153880 2023-06-09T11:47:13.367 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:13.368 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":173,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":173,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:08.628384+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:clientreplay","state_seq":22,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.365+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 msgr2=0x7f1adc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.365+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 0x7f1adc06d1c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009c40 tx=0x7f1ae0008030 comp rx=0 tx=0).stop 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.365+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 msgr2=0x7f1af00caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.365+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400c510 tx=0x7f1ae400c820 comp rx=0 tx=0).stop 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 shutdown_connections 2023-06-09T11:47:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1adc06ad10 0x7f1adc06d1c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1af015b4a0 0x7f1af00cb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1af00c9f40 0x7f1af00cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 --2- 172.21.15.7:0/1931820006 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af0153880 0x7f1af00caed0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 >> 172.21.15.7:0/1931820006 conn(0x7f1af00737f0 msgr2=0x7f1af014dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 shutdown_connections 2023-06-09T11:47:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:13.366+0000 7f1af69ff700 1 -- 172.21.15.7:0/1931820006 wait complete. 2023-06-09T11:47:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 173 2023-06-09T11:47:13.388 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:47:15.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:15.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.594+0000 7f667e5ff700 1 Processor -- start 2023-06-09T11:47:15.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.595+0000 7f667e5ff700 1 -- start start 2023-06-09T11:47:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f667e5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66780c9f40 0x7f66780c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f667e5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66780c5a80 con 0x7f667815b4a0 2023-06-09T11:47:15.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f667e5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667815a500 con 0x7f6678153880 2023-06-09T11:47:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f667e5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66780c5f10 con 0x7f66780c9f40 2023-06-09T11:47:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f66777fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f667815b4a0 0x7f66780c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44022/0 (socket says 172.21.15.7:44022) 2023-06-09T11:47:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f66777fe700 1 -- 172.21.15.7:0/2440390551 learned_addr learned my addr 172.21.15.7:0/2440390551 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:15.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.596+0000 7f667cb9c700 1 --2- 172.21.15.7:0/2440390551 >> v2:172.21.15.7:3300/0 conn(0x7f66780c9f40 0x7f66780c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:15.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f6676ffd700 1 -- 172.21.15.7:0/2440390551 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838842972 0 0) 0x7f66780c5a80 con 0x7f667815b4a0 2023-06-09T11:47:15.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f6676ffd700 1 -- 172.21.15.7:0/2440390551 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66600035b0 con 0x7f667815b4a0 2023-06-09T11:47:15.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f667cb9c700 1 -- 172.21.15.7:0/2440390551 >> v1:172.21.15.117:6789/0 conn(0x7f6678153880 legacy=0x7f667815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f667cb9c700 1 -- 172.21.15.7:0/2440390551 >> v1:172.21.15.7:6789/0 conn(0x7f667815b4a0 legacy=0x7f66780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f667cb9c700 1 -- 172.21.15.7:0/2440390551 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66781596e0 con 0x7f66780c9f40 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f667cb9c700 1 --2- 172.21.15.7:0/2440390551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66780c9f40 0x7f66780c5540 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f667000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544233bc3b9dc704 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f6676ffd700 1 -- 172.21.15.7:0/2440390551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667000e070 con 0x7f66780c9f40 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f6676ffd700 1 -- 172.21.15.7:0/2440390551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667000bb20 con 0x7f66780c9f40 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.597+0000 7f6676ffd700 1 -- 172.21.15.7:0/2440390551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667000be90 con 0x7f66780c9f40 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 -- 172.21.15.7:0/2440390551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66780c9f40 msgr2=0x7f66780c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 --2- 172.21.15.7:0/2440390551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66780c9f40 0x7f66780c5540 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f6670009d90 tx=0x7f667000b330 comp rx=0 tx=0).stop 2023-06-09T11:47:15.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 -- 172.21.15.7:0/2440390551 shutdown_connections 2023-06-09T11:47:15.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 --2- 172.21.15.7:0/2440390551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66780c9f40 0x7f66780c5540 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 -- 172.21.15.7:0/2440390551 >> 172.21.15.7:0/2440390551 conn(0x7f66780737f0 msgr2=0x7f66780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:15.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 -- 172.21.15.7:0/2440390551 shutdown_connections 2023-06-09T11:47:15.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.598+0000 7f667e5ff700 1 -- 172.21.15.7:0/2440390551 wait complete. 2023-06-09T11:47:15.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.600+0000 7f667e5ff700 1 Processor -- start 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.600+0000 7f667e5ff700 1 -- start start 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66780c9f40 0x7f66780cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667815b4a0 0x7f66780cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780cef80 con 0x7f6678153880 2023-06-09T11:47:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66781516c0 con 0x7f667815b4a0 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6678151960 con 0x7f66780c9f40 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33184/0 (socket says 172.21.15.7:33184) 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 -- 172.21.15.7:0/506784319 learned_addr learned my addr 172.21.15.7:0/506784319 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f6677fff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66780c9f40 0x7f66780cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f667cb9c700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667815b4a0 0x7f66780cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 -- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66780c9f40 msgr2=0x7f66780cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66780c9f40 0x7f66780cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 -- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667815b4a0 msgr2=0x7f66780cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667815b4a0 0x7f66780cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.601+0000 7f66777fe700 1 -- 172.21.15.7:0/506784319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6670009a70 con 0x7f6678153880 2023-06-09T11:47:15.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f66777fe700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f666c001eb0 tx=0x7f666c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c005d70 con 0x7f6678153880 2023-06-09T11:47:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c0070d0 con 0x7f6678153880 2023-06-09T11:47:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6678159c60 con 0x7f6678153880 2023-06-09T11:47:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c007460 con 0x7f6678153880 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.602+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6678159980 con 0x7f6678153880 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.603+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678159f40 con 0x7f6678153880 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.604+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f666c0075c0 con 0x7f6678153880 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.605+0000 7f6674ff9700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 0x7f666406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.605+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f666c088f10 con 0x7f6678153880 2023-06-09T11:47:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.605+0000 7f6677fff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 0x7f666406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:15.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.606+0000 7f6677fff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 0x7f666406d1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:15.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.610+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f666c0538e0 con 0x7f6678153880 2023-06-09T11:47:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.862+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6678074cf0 con 0x7f6678153880 2023-06-09T11:47:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.863+0000 7f6674ff9700 1 -- 172.21.15.7:0/506784319 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f666c017090 con 0x7f6678153880 2023-06-09T11:47:15.867 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:15.867 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":174,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:13.786133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 msgr2=0x7f666406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 0x7f666406d1c0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6668007fd0 tx=0x7f6668009040 comp rx=0 tx=0).stop 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 msgr2=0x7f6678152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f666c001eb0 tx=0x7f666c00c7b0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 shutdown_connections 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f667815b4a0 0x7f66780cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f666406ad10 0x7f666406d1c0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f66780c9f40 0x7f66780cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 --2- 172.21.15.7:0/506784319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6678153880 0x7f6678152a00 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.865+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 >> 172.21.15.7:0/506784319 conn(0x7f66780737f0 msgr2=0x7f66780caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.866+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 shutdown_connections 2023-06-09T11:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:15.866+0000 7f667e5ff700 1 -- 172.21.15.7:0/506784319 wait complete. 2023-06-09T11:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 174 2023-06-09T11:47:15.887 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=6859 2023-06-09T11:47:15.887 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.d 2023-06-09T11:47:20.888 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:47:20.889 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:47:20.889 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:47:20.894 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:47:20.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:21.048 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:47:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.096+0000 7f49c51ff700 1 Processor -- start 2023-06-09T11:47:21.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.096+0000 7f49c51ff700 1 -- start start 2023-06-09T11:47:21.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.097+0000 7f49c51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.097+0000 7f49c51ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49c015b380 0x7f49c0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.097+0000 7f49c51ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c01580a0 con 0x7f49c0071060 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.097+0000 7f49c51ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00714e0 con 0x7f49c015b380 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.097+0000 7f49c51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01518e0 con 0x7f49c0151510 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49c015b380 0x7f49c0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f49c015b380 0x7f49c0070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33186/0 (socket says 172.21.15.7:33186) 2023-06-09T11:47:21.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49bf59e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f49c0071060 0x7f49c0158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44032/0 (socket says 172.21.15.7:44032) 2023-06-09T11:47:21.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 -- 172.21.15.7:0/2623879957 learned_addr learned my addr 172.21.15.7:0/2623879957 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49bed9d700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 -- 172.21.15.7:0/2623879957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 msgr2=0x7f49c00c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c00c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 -- 172.21.15.7:0/2623879957 >> v1:172.21.15.7:6789/0 conn(0x7f49c0071060 legacy=0x7f49c0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49bdd9b700 1 -- 172.21.15.7:0/2623879957 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874812106 0 0) 0x7f49c01580a0 con 0x7f49c0071060 2023-06-09T11:47:21.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 -- 172.21.15.7:0/2623879957 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00c6180 con 0x7f49c015b380 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.098+0000 7f49be59c700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c015b380 0x7f49c0070b20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098f0 tx=0x7f49b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a36b30d718cf0d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49bdd9b700 1 -- 172.21.15.7:0/2623879957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011830 con 0x7f49c015b380 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49bdd9b700 1 -- 172.21.15.7:0/2623879957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4011990 con 0x7f49c015b380 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49bdd9b700 1 -- 172.21.15.7:0/2623879957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4011d20 con 0x7f49c015b380 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 -- 172.21.15.7:0/2623879957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c015b380 msgr2=0x7f49c0070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c015b380 0x7f49c0070b20 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f49b40098f0 tx=0x7f49b4009c00 comp rx=0 tx=0).stop 2023-06-09T11:47:21.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 -- 172.21.15.7:0/2623879957 shutdown_connections 2023-06-09T11:47:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c00c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2623879957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c015b380 0x7f49c0070b20 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 -- 172.21.15.7:0/2623879957 >> 172.21.15.7:0/2623879957 conn(0x7f49c006a4d0 msgr2=0x7f49c006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 -- 172.21.15.7:0/2623879957 shutdown_connections 2023-06-09T11:47:21.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.099+0000 7f49c51ff700 1 -- 172.21.15.7:0/2623879957 wait complete. 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.101+0000 7f49c51ff700 1 Processor -- start 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.101+0000 7f49c51ff700 1 -- start start 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c015a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c015b380 0x7f49c00c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c015a700 con 0x7f49c0071060 2023-06-09T11:47:21.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c02c0 con 0x7f49c015b380 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49c51ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c0560 con 0x7f49c0151510 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49bed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33190/0 (socket says 172.21.15.7:33190) 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.102+0000 7f49bed9d700 1 -- 172.21.15.7:0/2240381573 learned_addr learned my addr 172.21.15.7:0/2240381573 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bf59e700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c015b380 0x7f49c00c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49be59c700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c015a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 -- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 msgr2=0x7f49c015a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c015a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 -- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c015b380 msgr2=0x7f49c00c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c015b380 0x7f49c00c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 -- 172.21.15.7:0/2240381573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b4009550 con 0x7f49c0071060 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49bed9d700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e9b0 tx=0x7f49b000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0009710 con 0x7f49c0071060 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b000c850 con 0x7f49c0071060 2023-06-09T11:47:21.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b000cbc0 con 0x7f49c0071060 2023-06-09T11:47:21.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.103+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c0157770 con 0x7f49c0071060 2023-06-09T11:47:21.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.104+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0157490 con 0x7f49c0071060 2023-06-09T11:47:21.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.105+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c0157a50 con 0x7f49c0071060 2023-06-09T11:47:21.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.105+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49b000cd20 con 0x7f49c0071060 2023-06-09T11:47:21.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.106+0000 7f49a77fe700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:21.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.106+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f49b0089c70 con 0x7f49c0071060 2023-06-09T11:47:21.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.106+0000 7f49be59c700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:21.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.107+0000 7f49be59c700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49b4014040 tx=0x7f49b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:21.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.111+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f49b00580d0 con 0x7f49c0071060 2023-06-09T11:47:21.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:21.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:21.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.363+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c0074cf0 con 0x7f49c0071060 2023-06-09T11:47:21.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.364+0000 7f49a77fe700 1 -- 172.21.15.7:0/2240381573 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 174 v174) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f49b0017090 con 0x7f49c0071060 2023-06-09T11:47:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:21.368 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":174,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:13.786133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 msgr2=0x7f49ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f49b4014040 tx=0x7f49b4008030 comp rx=0 tx=0).stop 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 msgr2=0x7f49c00c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f49b000e9b0 tx=0x7f49b000ed70 comp rx=0 tx=0).stop 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 shutdown_connections 2023-06-09T11:47:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f49c015b380 0x7f49c00c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49c0071060 0x7f49c00c8fa0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 --2- 172.21.15.7:0/2240381573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f49c0151510 0x7f49c015a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 >> 172.21.15.7:0/2240381573 conn(0x7f49c006a4d0 msgr2=0x7f49c0149310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 shutdown_connections 2023-06-09T11:47:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:21.367+0000 7f49c51ff700 1 -- 172.21.15.7:0/2240381573 wait complete. 2023-06-09T11:47:21.373 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 174 2023-06-09T11:47:21.389 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:47:23.390 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:23.566 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 27% (41/151) Updating files: 27% (42/151) Updating files: 28% (43/151) Updating files: 29% (44/151) Updating files: 30% (46/151) Updating files: 31% (47/151) Updating files: 32% (49/151) Updating files: 33% (50/151) Updating files: 34% (52/151) Updating files: 35% (53/151) Updating files: 36% (55/151) Updating files: 37% (56/151) Updating files: 38% (58/151) Updating files: 39% (59/151) Updating files: 40% (61/151) Updating files: 41% (62/151) Updating files: 42% (64/151) Updating files: 43% (65/151) Updating files: 44% (67/151) Updating files: 45% (68/151) Updating files: 46% (70/151) Updating files: 47% (71/151) Updating files: 47% (72/151) Updating files: 48% (73/151) Updating files: 49% (74/151) Updating files: 50% (76/151) Updating files: 51% (78/151) Updating files: 52% (79/151) Updating files: 53% (81/151) Updating files: 54% (82/151) Updating files: 55% (84/151) Updating files: 56% (85/151) Updating files: 57% (87/151) Updating files: 58% (88/151) Updating files: 59% (90/151) Updating files: 60% (91/151) Updating files: 61% (93/151) Updating files: 62% (94/151) Updating files: 63% (96/151) Updating files: 64% (97/151) Updating files: 65% (99/151) Updating files: 66% (100/151) Updating files: 67% (102/151) Updating files: 68% (103/151) Updating files: 69% (105/151) Updating files: 70% (106/151) Updating files: 70% (107/151) Updating files: 71% (108/151) Updating files: 72% (109/151) Updating files: 73% (111/151) Updating files: 74% (112/151) Updating files: 75% (114/151) Updating files: 76% (115/151) Updating files: 77% (117/151) Updating files: 78% (118/151) Updating files: 79% (120/151) Updating files: 80% (121/151) Updating files: 81% (123/151) Updating files: 82% (124/151) Updating files: 83% (126/151) Updating files: 84% (127/151) Updating files: 85% (129/151) Updating files: 86% (130/151) Updating files: 87% (132/151) Updating files: 88% (133/151) Updating files: 88% (134/151) Updating files: 89% (135/151) Updating files: 90% (136/151) Updating files: 91% (138/151) Updating files: 92% (139/151) Updating files: 93% (141/151) Updating files: 94% (142/151) Updating files: 95% (144/151) Updating files: 96% (145/151) Updating files: 97% (147/151) Updating files: 98% (148/151) Updating files: 99% (150/151) Updating files: 100% (151/151) Updating files: 100% (151/151), done. 2023-06-09T11:47:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.628+0000 7f90a899d700 1 Processor -- start 2023-06-09T11:47:23.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.628+0000 7f90a899d700 1 -- start start 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a899d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a40c8190 0x7f90a415ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a899d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f90a4150f80 0x7f90a41513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a899d700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a415a9f0 con 0x7f90a40c7dc0 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a899d700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a41518e0 con 0x7f90a4150f80 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a899d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a415a6f0 con 0x7f90a40c8190 2023-06-09T11:47:23.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a37fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90a40c7dc0 0x7f90a406c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41888/0 (socket says 172.21.15.7:41888) 2023-06-09T11:47:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a37fe700 1 -- 172.21.15.7:0/247957328 learned_addr learned my addr 172.21.15.7:0/247957328 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:23.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a3fff700 1 --2- 172.21.15.7:0/247957328 >> v2:172.21.15.117:3300/0 conn(0x7f90a4150f80 0x7f90a41513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a40c8190 0x7f90a415ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.630+0000 7f90a27fc700 1 -- 172.21.15.7:0/247957328 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279124065 0 0) 0x7f90a415a9f0 con 0x7f90a40c7dc0 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a27fc700 1 -- 172.21.15.7:0/247957328 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f908c0035b0 con 0x7f90a40c7dc0 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a3fff700 1 -- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a40c8190 msgr2=0x7f90a415ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a3fff700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a40c8190 0x7f90a415ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a3fff700 1 -- 172.21.15.7:0/247957328 >> v1:172.21.15.7:6789/0 conn(0x7f90a40c7dc0 legacy=0x7f90a406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a3fff700 1 -- 172.21.15.7:0/247957328 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40c39c0 con 0x7f90a4150f80 2023-06-09T11:47:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.631+0000 7f90a3fff700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a4150f80 0x7f90a41513a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d90 tx=0x7f909c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4675d2dca92e3a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.632+0000 7f90a27fc700 1 -- 172.21.15.7:0/247957328 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c00e070 con 0x7f90a4150f80 2023-06-09T11:47:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.632+0000 7f90a27fc700 1 -- 172.21.15.7:0/247957328 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c00bb20 con 0x7f90a4150f80 2023-06-09T11:47:23.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.632+0000 7f90a27fc700 1 -- 172.21.15.7:0/247957328 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c012430 con 0x7f90a4150f80 2023-06-09T11:47:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.632+0000 7f90a899d700 1 -- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a4150f80 msgr2=0x7f90a41513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.632+0000 7f90a899d700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a4150f80 0x7f90a41513a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f909c009d90 tx=0x7f909c00b330 comp rx=0 tx=0).stop 2023-06-09T11:47:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 -- 172.21.15.7:0/247957328 shutdown_connections 2023-06-09T11:47:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a4150f80 0x7f90a41513a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 --2- 172.21.15.7:0/247957328 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a40c8190 0x7f90a415ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 -- 172.21.15.7:0/247957328 >> 172.21.15.7:0/247957328 conn(0x7f90a406dfa0 msgr2=0x7f90a406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 -- 172.21.15.7:0/247957328 shutdown_connections 2023-06-09T11:47:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.633+0000 7f90a899d700 1 -- 172.21.15.7:0/247957328 wait complete. 2023-06-09T11:47:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.635+0000 7f90a899d700 1 Processor -- start 2023-06-09T11:47:23.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.635+0000 7f90a899d700 1 -- start start 2023-06-09T11:47:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a40c7dc0 0x7f90a40c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a4150f80 0x7f90a4155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4156530 con 0x7f90a40c8190 2023-06-09T11:47:23.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a40ccd60 con 0x7f90a40c7dc0 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a899d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a41ef200 con 0x7f90a4150f80 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a40c7dc0 0x7f90a40c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43962/0 (socket says 172.21.15.7:43962) 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 -- 172.21.15.7:0/339315978 learned_addr learned my addr 172.21.15.7:0/339315978 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:23.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a3fff700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a4150f80 0x7f90a4155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 -- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a4150f80 msgr2=0x7f90a4155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a4150f80 0x7f90a4155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 -- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a40c7dc0 msgr2=0x7f90a40c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a40c7dc0 0x7f90a40c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.637+0000 7f90a2ffd700 1 -- 172.21.15.7:0/339315978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009a70 con 0x7f90a40c8190 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f90a2ffd700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9b0 tx=0x7f909800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9098009710 con 0x7f90a40c8190 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800ee50 con 0x7f90a40c8190 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40c3f80 con 0x7f90a40c8190 2023-06-09T11:47:23.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800ca50 con 0x7f90a40c8190 2023-06-09T11:47:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.638+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a40c3ca0 con 0x7f90a40c8190 2023-06-09T11:47:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.639+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a406da70 con 0x7f90a40c8190 2023-06-09T11:47:23.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.641+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f909800cbb0 con 0x7f90a40c8190 2023-06-09T11:47:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.642+0000 7f908bfff700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.642+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f9098089fc0 con 0x7f90a40c8190 2023-06-09T11:47:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.642+0000 7f90a37fe700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:23.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.642+0000 7f90a37fe700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9094007fd0 tx=0x7f9094007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:23.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.647+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9098058300 con 0x7f90a40c8190 2023-06-09T11:47:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.914+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90a404f760 con 0x7f90a40c8190 2023-06-09T11:47:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.914+0000 7f908bfff700 1 -- 172.21.15.7:0/339315978 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 175 v175) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f9098021020 con 0x7f90a40c8190 2023-06-09T11:47:23.918 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:23.918 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":175,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":174,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:13.786133+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.916+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 msgr2=0x7f909006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.916+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f9094007fd0 tx=0x7f9094007dc0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.916+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 msgr2=0x7f90a41557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.916+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9b0 tx=0x7f909800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 shutdown_connections 2023-06-09T11:47:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90a4150f80 0x7f90a4155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90a40c7dc0 0x7f90a40c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 --2- 172.21.15.7:0/339315978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90a40c8190 0x7f90a41557f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 >> 172.21.15.7:0/339315978 conn(0x7f90a406dfa0 msgr2=0x7f90a4159980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 shutdown_connections 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:23.917+0000 7f90a899d700 1 -- 172.21.15.7:0/339315978 wait complete. 2023-06-09T11:47:23.921 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 175 2023-06-09T11:47:23.937 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:47:23.937 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 13 secs before thrashing 2023-06-09T11:47:24.816 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 1ec1e8d6fb6 v0.14 2023-06-09T11:47:24.822 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.14 2023-06-09T11:47:24.852 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:47:24.852 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 15 -eq 48 ']' 2023-06-09T11:47:24.852 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.15 2023-06-09T11:47:24.853 INFO:tasks.workunit.client.0.smithi007.stdout:v0.15 2023-06-09T11:47:24.853 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.15 2023-06-09T11:47:24.853 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.15 2023-06-09T11:47:36.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.145+0000 7f16307ff700 1 Processor -- start 2023-06-09T11:47:37.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.146+0000 7f16307ff700 1 -- start start 2023-06-09T11:47:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f16307ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1628151510 0x7f16280c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f16307ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f16307ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16281580a0 con 0x7f1628071060 2023-06-09T11:47:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f16307ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16280714e0 con 0x7f1628151510 2023-06-09T11:47:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f16307ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16281518e0 con 0x7f162815b380 2023-06-09T11:47:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f162e59b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1628151510 0x7f16280c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.147+0000 7f162dd9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162ed9c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f1628071060 0x7f1628158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:36692/0 (socket says 172.21.15.7:36692) 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1628151510 0x7f16280c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:58230/0 (socket says 172.21.15.7:58230) 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162ed9c700 1 -- 172.21.15.7:0/3364841558 learned_addr learned my addr 172.21.15.7:0/3364841558 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162dd9a700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:52586/0 (socket says 172.21.15.7:52586) 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 -- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 msgr2=0x7f1628070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 -- 172.21.15.7:0/3364841558 >> v1:172.21.15.117:6789/0 conn(0x7f1628071060 legacy=0x7f1628158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 -- 172.21.15.7:0/3364841558 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280c6180 con 0x7f1628151510 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162dd9a700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162d599700 1 -- 172.21.15.7:0/3364841558 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536866747 0 0) 0x7f16281580a0 con 0x7f1628071060 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.148+0000 7f162e59b700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1628151510 0x7f16280c5b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1624009840 tx=0x7f1624009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ffb8e4778c2e05 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f162d599700 1 -- 172.21.15.7:0/3364841558 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624011830 con 0x7f1628151510 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f162d599700 1 -- 172.21.15.7:0/3364841558 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624011990 con 0x7f1628151510 2023-06-09T11:47:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f162d599700 1 -- 172.21.15.7:0/3364841558 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624011d20 con 0x7f1628151510 2023-06-09T11:47:37.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 -- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1628151510 msgr2=0x7f16280c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1628151510 0x7f16280c5b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1624009840 tx=0x7f1624009b50 comp rx=0 tx=0).stop 2023-06-09T11:47:37.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 -- 172.21.15.7:0/3364841558 shutdown_connections 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1628151510 0x7f16280c5b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 --2- 172.21.15.7:0/3364841558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f162815b380 0x7f1628070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 -- 172.21.15.7:0/3364841558 >> 172.21.15.7:0/3364841558 conn(0x7f162806a4d0 msgr2=0x7f162806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 -- 172.21.15.7:0/3364841558 shutdown_connections 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.149+0000 7f16307ff700 1 -- 172.21.15.7:0/3364841558 wait complete. 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.151+0000 7f16307ff700 1 Processor -- start 2023-06-09T11:47:37.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.151+0000 7f16307ff700 1 -- start start 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1628071060 0x7f16281ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f162815b380 0x7f1628154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16281efd40 con 0x7f1628151510 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628159470 con 0x7f162815b380 2023-06-09T11:47:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f16307ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1628159750 con 0x7f1628071060 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f162dd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f162dd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45630/0 (socket says 172.21.15.7:45630) 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.152+0000 7f162dd9a700 1 -- 172.21.15.7:0/25285252 learned_addr learned my addr 172.21.15.7:0/25285252 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162ed9c700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f162815b380 0x7f1628154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162e59b700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1628071060 0x7f16281ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 -- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1628071060 msgr2=0x7f16281ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1628071060 0x7f16281ef800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 -- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f162815b380 msgr2=0x7f1628154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f162815b380 0x7f1628154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 -- 172.21.15.7:0/25285252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16180094a0 con 0x7f1628151510 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f162dd9a700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f161800ebe0 tx=0x7f161800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618009760 con 0x7f1628151510 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161800c850 con 0x7f1628151510 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.153+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16240094a0 con 0x7f1628151510 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.154+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800cbe0 con 0x7f1628151510 2023-06-09T11:47:37.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.154+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1628157770 con 0x7f1628151510 2023-06-09T11:47:37.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.155+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1628157490 con 0x7f1628151510 2023-06-09T11:47:37.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.156+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f161800cd40 con 0x7f1628151510 2023-06-09T11:47:37.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.157+0000 7f161effd700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 0x7f160c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.157+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f1618089c20 con 0x7f1628151510 2023-06-09T11:47:37.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.157+0000 7f162e59b700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 0x7f160c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.158+0000 7f162e59b700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 0x7f160c06d1c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f16240090b0 tx=0x7f1624008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.162+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f16180544d0 con 0x7f1628151510 2023-06-09T11:47:37.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.420+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1628157a50 con 0x7f1628151510 2023-06-09T11:47:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.420+0000 7f161effd700 1 -- 172.21.15.7:0/25285252 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f1618021020 con 0x7f1628151510 2023-06-09T11:47:37.423 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:37.424 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 msgr2=0x7f160c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 0x7f160c06d1c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f16240090b0 tx=0x7f1624008030 comp rx=0 tx=0).stop 2023-06-09T11:47:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 msgr2=0x7f16280c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f161800ebe0 tx=0x7f161800eef0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 shutdown_connections 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f162815b380 0x7f1628154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f160c06ad10 0x7f160c06d1c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1628071060 0x7f16281ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 --2- 172.21.15.7:0/25285252 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1628151510 0x7f16280c7a60 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 >> 172.21.15.7:0/25285252 conn(0x7f162806a4d0 msgr2=0x7f162814ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 shutdown_connections 2023-06-09T11:47:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.423+0000 7f16307ff700 1 -- 172.21.15.7:0/25285252 wait complete. 2023-06-09T11:47:37.428 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:37.443 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=1) 2023-06-09T11:47:37.444 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:47:37.444 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:47:37.464 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:47:37.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.661+0000 7f4e945ff700 1 Processor -- start 2023-06-09T11:47:37.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.661+0000 7f4e945ff700 1 -- start start 2023-06-09T11:47:37.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e945ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4e8c151510 0x7f4e8c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e945ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c1580a0 con 0x7f4e8c071230 2023-06-09T11:47:37.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e945ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c0c6180 con 0x7f4e8c15b380 2023-06-09T11:47:37.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e945ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c1518e0 con 0x7f4e8c151510 2023-06-09T11:47:37.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e92b9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4e8c071230 0x7f4e8c158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44498/0 (socket says 172.21.15.7:44498) 2023-06-09T11:47:37.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e92b9c700 1 -- 172.21.15.7:0/2355793955 learned_addr learned my addr 172.21.15.7:0/2355793955 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:37.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e9239b700 1 --2- 172.21.15.7:0/2355793955 >> v2:172.21.15.117:3300/0 conn(0x7f4e8c151510 0x7f4e8c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551281132 0 0) 0x7f4e8c1580a0 con 0x7f4e8c071230 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e780035b0 con 0x7f4e8c071230 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864422817 0 0) 0x7f4e8c0c6180 con 0x7f4e8c15b380 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.663+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e8c1580a0 con 0x7f4e8c15b380 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e9239b700 1 -- 172.21.15.7:0/2355793955 >> v1:172.21.15.117:6789/0 conn(0x7f4e8c15b380 legacy=0x7f4e8c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e9239b700 1 -- 172.21.15.7:0/2355793955 >> v1:172.21.15.7:6789/0 conn(0x7f4e8c071230 legacy=0x7f4e8c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e9239b700 1 -- 172.21.15.7:0/2355793955 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c157490 con 0x7f4e8c151510 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e9239b700 1 --2- 172.21.15.7:0/2355793955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c151510 0x7f4e8c0c5b40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009840 tx=0x7f4e88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9bfd9bf5de852b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88011830 con 0x7f4e8c151510 2023-06-09T11:47:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e88011990 con 0x7f4e8c151510 2023-06-09T11:47:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.664+0000 7f4e91399700 1 -- 172.21.15.7:0/2355793955 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88011d20 con 0x7f4e8c151510 2023-06-09T11:47:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 -- 172.21.15.7:0/2355793955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c151510 msgr2=0x7f4e8c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 --2- 172.21.15.7:0/2355793955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c151510 0x7f4e8c0c5b40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009840 tx=0x7f4e88009b50 comp rx=0 tx=0).stop 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 -- 172.21.15.7:0/2355793955 shutdown_connections 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 --2- 172.21.15.7:0/2355793955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c151510 0x7f4e8c0c5b40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 -- 172.21.15.7:0/2355793955 >> 172.21.15.7:0/2355793955 conn(0x7f4e8c06a4d0 msgr2=0x7f4e8c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 -- 172.21.15.7:0/2355793955 shutdown_connections 2023-06-09T11:47:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.665+0000 7f4e945ff700 1 -- 172.21.15.7:0/2355793955 wait complete. 2023-06-09T11:47:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.667+0000 7f4e945ff700 1 Processor -- start 2023-06-09T11:47:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.667+0000 7f4e945ff700 1 -- start start 2023-06-09T11:47:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c071230 0x7f4e8c0c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e8c15b380 0x7f4e8c0ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0ca4f0 con 0x7f4e8c151510 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c1f08e0 con 0x7f4e8c071230 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.668+0000 7f4e945ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c158b50 con 0x7f4e8c15b380 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45646/0 (socket says 172.21.15.7:45646) 2023-06-09T11:47:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 -- 172.21.15.7:0/3992719020 learned_addr learned my addr 172.21.15.7:0/3992719020 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e9239b700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c071230 0x7f4e8c0c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e92b9c700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e8c15b380 0x7f4e8c0ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 -- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e8c15b380 msgr2=0x7f4e8c0ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e8c15b380 0x7f4e8c0ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 -- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c071230 msgr2=0x7f4e8c0c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c071230 0x7f4e8c0c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 -- 172.21.15.7:0/3992719020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880094a0 con 0x7f4e8c151510 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e91b9a700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c500 tx=0x7f4e7c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.669+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0052d0 con 0x7f4e8c151510 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.670+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c005450 con 0x7f4e8c151510 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.670+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c157a50 con 0x7f4e8c151510 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.670+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c0038c0 con 0x7f4e8c151510 2023-06-09T11:47:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.670+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c157770 con 0x7f4e8c151510 2023-06-09T11:47:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.671+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c157d30 con 0x7f4e8c151510 2023-06-09T11:47:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.672+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4e7c003a20 con 0x7f4e8c151510 2023-06-09T11:47:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.673+0000 7f4e82ffd700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 0x7f4e7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.673+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f4e7c088ec0 con 0x7f4e8c151510 2023-06-09T11:47:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.673+0000 7f4e9239b700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 0x7f4e7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.673+0000 7f4e9239b700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 0x7f4e7006d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009fd0 tx=0x7f4e8801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:37.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.677+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4e7c053890 con 0x7f4e8c151510 2023-06-09T11:47:37.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.942+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e8c074cf0 con 0x7f4e8c151510 2023-06-09T11:47:37.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.943+0000 7f4e82ffd700 1 -- 172.21.15.7:0/3992719020 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f4e7c017090 con 0x7f4e8c151510 2023-06-09T11:47:37.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:37.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.945+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 msgr2=0x7f4e7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.945+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 0x7f4e7006d190 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009fd0 tx=0x7f4e8801c040 comp rx=0 tx=0).stop 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.945+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 msgr2=0x7f4e8c0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.945+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00c500 tx=0x7f4e7c00c810 comp rx=0 tx=0).stop 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 shutdown_connections 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4e7006ace0 0x7f4e7006d190 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4e8c15b380 0x7f4e8c0ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4e8c071230 0x7f4e8c0c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 --2- 172.21.15.7:0/3992719020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e8c151510 0x7f4e8c0cb950 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 >> 172.21.15.7:0/3992719020 conn(0x7f4e8c06a4d0 msgr2=0x7f4e8c155b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 shutdown_connections 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:37.946+0000 7f4e945ff700 1 -- 172.21.15.7:0/3992719020 wait complete. 2023-06-09T11:47:37.950 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:37.967 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:39.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:40.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.174+0000 7f380d9ff700 1 Processor -- start 2023-06-09T11:47:40.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.175+0000 7f380d9ff700 1 -- start start 2023-06-09T11:47:40.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f380d9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f38080cb5d0 0x7f38080b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f380d9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f38080cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f380d9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3808159b20 con 0x7f38080cb9a0 2023-06-09T11:47:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f380d9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38080cac80 con 0x7f38080cb5d0 2023-06-09T11:47:40.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f380d9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38081597d0 con 0x7f380815cd40 2023-06-09T11:47:40.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f38067fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f38080cb9a0 0x7f3808159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:44500/0 (socket says 172.21.15.7:44500) 2023-06-09T11:47:40.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f38067fc700 1 -- 172.21.15.7:0/2259017514 learned_addr learned my addr 172.21.15.7:0/2259017514 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:40.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f3806ffd700 1 --2- 172.21.15.7:0/2259017514 >> v2:172.21.15.117:3300/0 conn(0x7f38080cb5d0 0x7f38080b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f38077fe700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f38080cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586648605 0 0) 0x7f3808159b20 con 0x7f38080cb9a0 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.176+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f00035b0 con 0x7f38080cb9a0 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2295910772 0 0) 0x7f37f00035b0 con 0x7f38080cb9a0 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 msgr2=0x7f38080cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f38080cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 >> v2:172.21.15.117:3300/0 conn(0x7f38080cb5d0 msgr2=0x7f38080b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 --2- 172.21.15.7:0/2259017514 >> v2:172.21.15.117:3300/0 conn(0x7f38080cb5d0 0x7f38080b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808071ab0 con 0x7f38080cb9a0 2023-06-09T11:47:40.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f37fc001670 con 0x7f38080cb9a0 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 >> v1:172.21.15.7:6789/0 conn(0x7f38080cb9a0 legacy=0x7f3808159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f37f0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37f0003f80 0x7f37f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0004d90 0x7f37f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080cac80 con 0x7f37f0003f80 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081597d0 con 0x7f37f0004d90 2023-06-09T11:47:40.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f000ba40 con 0x7f380815cd40 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f38067fc700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37f0003f80 0x7f37f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f3806ffd700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f37f0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.177+0000 7f38077fe700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0004d90 0x7f37f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 -- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 msgr2=0x7f37f0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815cd40 0x7f37f0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 -- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0004d90 msgr2=0x7f37f0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:40.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0004d90 0x7f37f0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 -- 172.21.15.7:0/2259017514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00035b0 con 0x7f37f0003f80 2023-06-09T11:47:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f38067fc700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37f0003f80 0x7f37f0004850 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c2f0 tx=0x7f37f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b64183803907a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00ce30 con 0x7f37f0003f80 2023-06-09T11:47:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc005960 con 0x7f37f0003f80 2023-06-09T11:47:40.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.178+0000 7f3805ffb700 1 -- 172.21.15.7:0/2259017514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc005d10 con 0x7f37f0003f80 2023-06-09T11:47:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 -- 172.21.15.7:0/2259017514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37f0003f80 msgr2=0x7f37f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2259017514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37f0003f80 0x7f37f0004850 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f37fc00c2f0 tx=0x7f37f0005880 comp rx=0 tx=0).stop 2023-06-09T11:47:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f38077fe700 1 -- 172.21.15.7:0/2259017514 reap_dead start 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 -- 172.21.15.7:0/2259017514 shutdown_connections 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 -- 172.21.15.7:0/2259017514 >> 172.21.15.7:0/2259017514 conn(0x7f3808070af0 msgr2=0x7f3808070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 -- 172.21.15.7:0/2259017514 shutdown_connections 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.179+0000 7f380d9ff700 1 -- 172.21.15.7:0/2259017514 wait complete. 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.181+0000 7f380d9ff700 1 Processor -- start 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.181+0000 7f380d9ff700 1 -- start start 2023-06-09T11:47:40.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0003f80 0x7f3808153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 0x7f38080c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080ce390 con 0x7f38080cd8c0 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c8c00 con 0x7f37f0003f80 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f380d9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080c4420 con 0x7f380815c710 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45672/0 (socket says 172.21.15.7:45672) 2023-06-09T11:47:40.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 -- 172.21.15.7:0/2573833121 learned_addr learned my addr 172.21.15.7:0/2573833121 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f3806ffd700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0003f80 0x7f3808153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38077fe700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 0x7f38080c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 -- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 msgr2=0x7f38080c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 0x7f38080c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 -- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0003f80 msgr2=0x7f3808153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0003f80 0x7f3808153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.182+0000 7f38067fc700 1 -- 172.21.15.7:0/2573833121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380815d4f0 con 0x7f38080cd8c0 2023-06-09T11:47:40.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f38077fe700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 0x7f38080c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f38067fc700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc00cea0 con 0x7f38080cd8c0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc004a80 con 0x7f38080cd8c0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc004e10 con 0x7f38080cd8c0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380815d210 con 0x7f38080cd8c0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.183+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380815d7d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.184+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38080c45d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.185+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f37fc0070d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.186+0000 7f37effff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 0x7f37f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.186+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f37fc08b090 con 0x7f38080cd8c0 2023-06-09T11:47:40.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.186+0000 7f3806ffd700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 0x7f37f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.187+0000 7f3806ffd700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 0x7f37f406d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009c60 tx=0x7f37f8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.191+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f38080c45d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.450+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38080c45d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.450+0000 7f37effff700 1 -- 172.21.15.7:0/2573833121 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f38080c45d0 con 0x7f38080cd8c0 2023-06-09T11:47:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:40.454 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 msgr2=0x7f37f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 0x7f37f406d190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009c60 tx=0x7f37f8009210 comp rx=0 tx=0).stop 2023-06-09T11:47:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 msgr2=0x7f3808154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f37fc000c00 tx=0x7f37fc001a00 comp rx=0 tx=0).stop 2023-06-09T11:47:40.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 shutdown_connections 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f37f406ace0 0x7f37f406d190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f380815c710 0x7f38080c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f37f0003f80 0x7f3808153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 --2- 172.21.15.7:0/2573833121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38080cd8c0 0x7f3808154090 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 >> 172.21.15.7:0/2573833121 conn(0x7f3808070af0 msgr2=0x7f38080cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 shutdown_connections 2023-06-09T11:47:40.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:40.453+0000 7f380d9ff700 1 -- 172.21.15.7:0/2573833121 wait complete. 2023-06-09T11:47:40.458 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:40.474 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:42.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.682+0000 7ff1b7dff700 1 Processor -- start 2023-06-09T11:47:42.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.682+0000 7ff1b7dff700 1 -- start start 2023-06-09T11:47:42.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.683+0000 7ff1b7dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b00c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.683+0000 7ff1b7dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1b006e460 0x7ff1b006cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.683+0000 7ff1b7dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b015cac0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b7dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b00c8870 con 0x7ff1b006e460 2023-06-09T11:47:42.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b7dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00c37d0 con 0x7ff1b00c6280 2023-06-09T11:47:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1b006e460 0x7ff1b006cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b5b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b00c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff1b006e460 0x7ff1b006cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:58288/0 (socket says 172.21.15.7:58288) 2023-06-09T11:47:42.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 -- 172.21.15.7:0/4221567621 learned_addr learned my addr 172.21.15.7:0/4221567621 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 -- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 msgr2=0x7ff1b00c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 --2- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b00c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 -- 172.21.15.7:0/4221567621 >> v1:172.21.15.117:6789/0 conn(0x7ff1b006a4d0 legacy=0x7ff1b00737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.684+0000 7ff1b639c700 1 -- 172.21.15.7:0/4221567621 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b015cd30 con 0x7ff1b006e460 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.685+0000 7ff1b639c700 1 --2- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b006cf30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009e50 tx=0x7ff1a800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8645f2709b49fb4b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.685+0000 7ff1b4b99700 1 -- 172.21.15.7:0/4221567621 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800e070 con 0x7ff1b006e460 2023-06-09T11:47:42.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.685+0000 7ff1b4b99700 1 -- 172.21.15.7:0/4221567621 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a800b910 con 0x7ff1b006e460 2023-06-09T11:47:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.685+0000 7ff1b4b99700 1 -- 172.21.15.7:0/4221567621 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a800bcc0 con 0x7ff1b006e460 2023-06-09T11:47:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 -- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 msgr2=0x7ff1b006cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b006cf30 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009e50 tx=0x7ff1a800b330 comp rx=0 tx=0).stop 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 -- 172.21.15.7:0/4221567621 shutdown_connections 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b006cf30 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/4221567621 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b00c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 -- 172.21.15.7:0/4221567621 >> 172.21.15.7:0/4221567621 conn(0x7ff1b006f980 msgr2=0x7ff1b00cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 -- 172.21.15.7:0/4221567621 shutdown_connections 2023-06-09T11:47:42.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.686+0000 7ff1b7dff700 1 -- 172.21.15.7:0/4221567621 wait complete. 2023-06-09T11:47:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.688+0000 7ff1b7dff700 1 Processor -- start 2023-06-09T11:47:42.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.688+0000 7ff1b7dff700 1 -- start start 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b0156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b01e5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b01e52a0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b015bbc0 con 0x7ff1b006e460 2023-06-09T11:47:42.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b7dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b015b920 con 0x7ff1b00c6280 2023-06-09T11:47:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45674/0 (socket says 172.21.15.7:45674) 2023-06-09T11:47:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 -- 172.21.15.7:0/739034521 learned_addr learned my addr 172.21.15.7:0/739034521 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b539a700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b0156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b639c700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b01e5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 -- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 msgr2=0x7ff1b01e5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b01e5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 -- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 msgr2=0x7ff1b0156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b0156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.690+0000 7ff1b5b9b700 1 -- 172.21.15.7:0/739034521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a8009b30 con 0x7ff1b006a4d0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1b5b9b700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00e870 tx=0x7ff1ac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac009590 con 0x7ff1b006a4d0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ac013680 con 0x7ff1b006a4d0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0139f0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b015d2f0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.691+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b015d010 con 0x7ff1b006a4d0 2023-06-09T11:47:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.692+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b015d5d0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.693+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff1ac013b50 con 0x7ff1b006a4d0 2023-06-09T11:47:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.694+0000 7ff1a67fc700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 0x7ff19406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:42.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.694+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7ff1ac0899f0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.694+0000 7ff1b539a700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 0x7ff19406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:42.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.695+0000 7ff1b539a700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 0x7ff19406d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00c5cc0 tx=0x7ff1a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:42.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.698+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff1ac057e50 con 0x7ff1b006a4d0 2023-06-09T11:47:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.952+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1b0066ab0 con 0x7ff1b006a4d0 2023-06-09T11:47:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.952+0000 7ff1a67fc700 1 -- 172.21.15.7:0/739034521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7ff1ac021090 con 0x7ff1b006a4d0 2023-06-09T11:47:42.956 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:42.956 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 msgr2=0x7ff19406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 0x7ff19406d190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00c5cc0 tx=0x7ff1a0009040 comp rx=0 tx=0).stop 2023-06-09T11:47:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 msgr2=0x7ff1b01412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:42.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac00e870 tx=0x7ff1ac00eb80 comp rx=0 tx=0).stop 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 shutdown_connections 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff19406ace0 0x7ff19406d190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff1b00c6280 0x7ff1b01e5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1b006a4d0 0x7ff1b01412c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 --2- 172.21.15.7:0/739034521 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff1b006e460 0x7ff1b0156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:42.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 >> 172.21.15.7:0/739034521 conn(0x7ff1b006f980 msgr2=0x7ff1b006daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 shutdown_connections 2023-06-09T11:47:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:42.955+0000 7ff1b7dff700 1 -- 172.21.15.7:0/739034521 wait complete. 2023-06-09T11:47:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:42.976 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:44.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:45.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.184+0000 7ff74cbff700 1 Processor -- start 2023-06-09T11:47:45.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.185+0000 7ff74cbff700 1 -- start start 2023-06-09T11:47:45.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.186+0000 7ff74cbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff748153b30 0x7ff7480ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.186+0000 7ff74cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff7480723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74cbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74815a390 con 0x7ff748153760 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74cbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7480c5f40 con 0x7ff748153b30 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480ca720 con 0x7ff748071fa0 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff748153760 0x7ff7480b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48546/0 (socket says 172.21.15.7:48546) 2023-06-09T11:47:45.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74659c700 1 -- 172.21.15.7:0/2728721395 learned_addr learned my addr 172.21.15.7:0/2728721395 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:45.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162744282 0 0) 0x7ff74815a390 con 0x7ff748153760 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff745d9b700 1 --2- 172.21.15.7:0/2728721395 >> v2:172.21.15.117:3300/0 conn(0x7ff748153b30 0x7ff7480ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7300035b0 con 0x7ff748153760 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff746d9d700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff7480723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1839072493 0 0) 0x7ff7300035b0 con 0x7ff748153760 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 msgr2=0x7ff7480723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff7480723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 >> v2:172.21.15.117:3300/0 conn(0x7ff748153b30 msgr2=0x7ff7480ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 --2- 172.21.15.7:0/2728721395 >> v2:172.21.15.117:3300/0 conn(0x7ff748153b30 0x7ff7480ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.187+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480c5a20 con 0x7ff748153760 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7ff738001670 con 0x7ff748153760 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 >> v1:172.21.15.7:6789/0 conn(0x7ff748153760 legacy=0x7ff7480b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff730003a00 0x7ff730003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff730004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730004d90 0x7ff730005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480c5f40 con 0x7ff730003a00 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480ca720 con 0x7ff730004d90 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73000ba40 con 0x7ff748071fa0 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff730003a00 0x7ff730003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff745d9b700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff730004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff746d9d700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730004d90 0x7ff730005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 -- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 msgr2=0x7ff730004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:45.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff730004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 -- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730004d90 msgr2=0x7ff730005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730004d90 0x7ff730005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff74659c700 1 -- 172.21.15.7:0/2728721395 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300035b0 con 0x7ff730003a00 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff745d9b700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748071fa0 0x7ff730004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.188+0000 7ff746d9d700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730004d90 0x7ff730005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74659c700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff730003a00 0x7ff730003e10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c210 tx=0x7ff730005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e42bd9c0674f47 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:45.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738007b70 con 0x7ff730003a00 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73800cd50 con 0x7ff730003a00 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74559a700 1 -- 172.21.15.7:0/2728721395 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738005ba0 con 0x7ff730003a00 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 -- 172.21.15.7:0/2728721395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff730003a00 msgr2=0x7ff730003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 --2- 172.21.15.7:0/2728721395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff730003a00 0x7ff730003e10 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff73800c210 tx=0x7ff730005880 comp rx=0 tx=0).stop 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff746d9d700 1 -- 172.21.15.7:0/2728721395 reap_dead start 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 -- 172.21.15.7:0/2728721395 shutdown_connections 2023-06-09T11:47:45.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 -- 172.21.15.7:0/2728721395 >> 172.21.15.7:0/2728721395 conn(0x7ff7480737f0 msgr2=0x7ff748073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:45.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 -- 172.21.15.7:0/2728721395 shutdown_connections 2023-06-09T11:47:45.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.189+0000 7ff74cbff700 1 -- 172.21.15.7:0/2728721395 wait complete. 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.191+0000 7ff74cbff700 1 Processor -- start 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.191+0000 7ff74cbff700 1 -- start start 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730003a00 0x7ff7480c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748156240 0x7ff7481566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480c7920 con 0x7ff7480c8560 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff748156c20 con 0x7ff730003a00 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff74cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7480c9bf0 con 0x7ff748156240 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff745d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff745d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47456/0 (socket says 172.21.15.7:47456) 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.192+0000 7ff745d9b700 1 -- 172.21.15.7:0/1203059589 learned_addr learned my addr 172.21.15.7:0/1203059589 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff74659c700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730003a00 0x7ff7480c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff746d9d700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748156240 0x7ff7481566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 -- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748156240 msgr2=0x7ff7481566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748156240 0x7ff7481566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 -- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730003a00 msgr2=0x7ff7480c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730003a00 0x7ff7480c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 -- 172.21.15.7:0/1203059589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748159a60 con 0x7ff7480c8560 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff745d9b700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c010920 tx=0x7ff73c010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c010e70 con 0x7ff7480c8560 2023-06-09T11:47:45.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00e8d0 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff748159780 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.193+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00ec80 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.194+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff748159d40 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.195+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7480c93b0 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.195+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff73c016460 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.196+0000 7ff736ffd700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 0x7ff72806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.196+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7ff73c089f90 con 0x7ff7480c8560 2023-06-09T11:47:45.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.197+0000 7ff74659c700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 0x7ff72806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.197+0000 7ff74659c700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 0x7ff72806d1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff738015040 tx=0x7ff73800a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:45.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.201+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff73c054840 con 0x7ff7480c8560 2023-06-09T11:47:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.454+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff748074cf0 con 0x7ff7480c8560 2023-06-09T11:47:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.455+0000 7ff736ffd700 1 -- 172.21.15.7:0/1203059589 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7ff73c024090 con 0x7ff7480c8560 2023-06-09T11:47:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:45.458 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 msgr2=0x7ff72806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 0x7ff72806d1c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff738015040 tx=0x7ff73800a000 comp rx=0 tx=0).stop 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 msgr2=0x7ff748154b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff73c010920 tx=0x7ff73c010c30 comp rx=0 tx=0).stop 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 shutdown_connections 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff72806ad10 0x7ff72806d1c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff748156240 0x7ff7481566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff730003a00 0x7ff7480c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 --2- 172.21.15.7:0/1203059589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff7480c8560 0x7ff748154b30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 >> 172.21.15.7:0/1203059589 conn(0x7ff7480737f0 msgr2=0x7ff7480c49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 shutdown_connections 2023-06-09T11:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:45.457+0000 7ff74cbff700 1 -- 172.21.15.7:0/1203059589 wait complete. 2023-06-09T11:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:45.478 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:47.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.687+0000 7fd3bdfff700 1 Processor -- start 2023-06-09T11:47:47.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.687+0000 7fd3bdfff700 1 -- start start 2023-06-09T11:47:47.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80b5360 0x7fd3b80c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd3b80ca580 0x7fd3b81536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b80c5e00 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b815dcf0 con 0x7fd3b80ca580 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.688+0000 7fd3bdfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b815d450 con 0x7fd3b80b5360 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47462/0 (socket says 172.21.15.7:47462) 2023-06-09T11:47:47.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 -- 172.21.15.7:0/1771457709 learned_addr learned my addr 172.21.15.7:0/1771457709 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:47.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b6ffd700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80b5360 0x7fd3b80c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b7fff700 1 --2- 172.21.15.7:0/1771457709 >> v2:172.21.15.117:3300/0 conn(0x7fd3b80ca580 0x7fd3b81536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 -- 172.21.15.7:0/1771457709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80b5360 msgr2=0x7fd3b80c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:47.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80b5360 0x7fd3b80c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 -- 172.21.15.7:0/1771457709 >> v2:172.21.15.117:3300/0 conn(0x7fd3b80ca580 msgr2=0x7fd3b81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1771457709 >> v2:172.21.15.117:3300/0 conn(0x7fd3b80ca580 0x7fd3b81536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.689+0000 7fd3b77fe700 1 -- 172.21.15.7:0/1771457709 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b815d730 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd3b808d4a0 tx=0x7fd3a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3382d06054725588 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3b67fc700 1 -- 172.21.15.7:0/1771457709 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800b870 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3b67fc700 1 -- 172.21.15.7:0/1771457709 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a800b9d0 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3b67fc700 1 -- 172.21.15.7:0/1771457709 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800bd60 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1771457709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca1b0 msgr2=0x7fd3b815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:47.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.690+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fd3b808d4a0 tx=0x7fd3a8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1771457709 shutdown_connections 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1771457709 >> v2:172.21.15.117:3300/0 conn(0x7fd3b80ca580 0x7fd3b81536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca1b0 0x7fd3b815cdc0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1771457709 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80b5360 0x7fd3b80c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1771457709 >> 172.21.15.7:0/1771457709 conn(0x7fd3b8072090 msgr2=0x7fd3b8072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1771457709 shutdown_connections 2023-06-09T11:47:47.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.691+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1771457709 wait complete. 2023-06-09T11:47:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.692+0000 7fd3bdfff700 1 Processor -- start 2023-06-09T11:47:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.692+0000 7fd3bdfff700 1 -- start start 2023-06-09T11:47:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3b80b5360 0x7fd3b80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80ca1b0 0x7fd3b815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b80b4960 con 0x7fd3b80ca580 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b80b3a70 con 0x7fd3b80b5360 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.693+0000 7fd3bdfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b80b3d50 con 0x7fd3b80ca1b0 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47468/0 (socket says 172.21.15.7:47468) 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 -- 172.21.15.7:0/1352357609 learned_addr learned my addr 172.21.15.7:0/1352357609 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:47.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3b80b5360 0x7fd3b80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b6ffd700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80ca1b0 0x7fd3b815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 -- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80ca1b0 msgr2=0x7fd3b815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80ca1b0 0x7fd3b815a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 -- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3b80b5360 msgr2=0x7fd3b80b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3b80b5360 0x7fd3b80b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 -- 172.21.15.7:0/1352357609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a8009550 con 0x7fd3b80ca580 2023-06-09T11:47:47.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.694+0000 7fd3b7fff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007e60 tx=0x7fd3b000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.695+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b000ed30 con 0x7fd3b80ca580 2023-06-09T11:47:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.695+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0014420 con 0x7fd3b80ca580 2023-06-09T11:47:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.695+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b806a570 con 0x7fd3b80ca580 2023-06-09T11:47:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.695+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b00147d0 con 0x7fd3b80ca580 2023-06-09T11:47:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.695+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b815da10 con 0x7fd3b80ca580 2023-06-09T11:47:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.696+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b806a850 con 0x7fd3b80ca580 2023-06-09T11:47:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.697+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3b0014930 con 0x7fd3b80ca580 2023-06-09T11:47:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.698+0000 7fd39ffff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 0x7fd3a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.698+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7fd3b008a230 con 0x7fd3b80ca580 2023-06-09T11:47:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.698+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 0x7fd3a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:47.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.699+0000 7fd3b77fe700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 0x7fd3a406d1c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8000c00 tx=0x7fd3a8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:47.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.702+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd3b0054ae0 con 0x7fd3b80ca580 2023-06-09T11:47:47.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.958+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3b80b4b10 con 0x7fd3b80ca580 2023-06-09T11:47:47.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.959+0000 7fd39ffff700 1 -- 172.21.15.7:0/1352357609 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fd3b80b4b10 con 0x7fd3b80ca580 2023-06-09T11:47:47.962 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:47.962 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:47.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 msgr2=0x7fd3a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 0x7fd3a406d1c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8000c00 tx=0x7fd3a8008030 comp rx=0 tx=0).stop 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 msgr2=0x7fd3b80cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0007e60 tx=0x7fd3b000e3c0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 shutdown_connections 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3b80ca580 0x7fd3b80cd780 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd3b80b5360 0x7fd3b80b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd3a406ad10 0x7fd3a406d1c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 --2- 172.21.15.7:0/1352357609 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd3b80ca1b0 0x7fd3b815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.961+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 >> 172.21.15.7:0/1352357609 conn(0x7fd3b8072090 msgr2=0x7fd3b814f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.962+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 shutdown_connections 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:47.962+0000 7fd3bdfff700 1 -- 172.21.15.7:0/1352357609 wait complete. 2023-06-09T11:47:47.966 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:47.983 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:49.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.190+0000 7f060f9ff700 1 Processor -- start 2023-06-09T11:47:50.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.190+0000 7f060f9ff700 1 -- start start 2023-06-09T11:47:50.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060f9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06080cb5d0 0x7f06080b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060f9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f06080cb9a0 0x7f0608159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060f9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608159b20 con 0x7f060815cd40 2023-06-09T11:47:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060f9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06080cac80 con 0x7f06080cb5d0 2023-06-09T11:47:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060f9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06081597d0 con 0x7f06080cb9a0 2023-06-09T11:47:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060d79b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f06080cb5d0 0x7f06080b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.191+0000 7f060df9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f060815cd40 0x7f06080cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48550/0 (socket says 172.21.15.7:48550) 2023-06-09T11:47:50.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060df9c700 1 -- 172.21.15.7:0/1439687793 learned_addr learned my addr 172.21.15.7:0/1439687793 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:50.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060cf9a700 1 --2- 172.21.15.7:0/1439687793 >> v2:172.21.15.117:3300/0 conn(0x7f06080cb9a0 0x7f0608159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060d79b700 1 -- 172.21.15.7:0/1439687793 >> v2:172.21.15.117:3300/0 conn(0x7f06080cb9a0 msgr2=0x7f0608159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060d79b700 1 --2- 172.21.15.7:0/1439687793 >> v2:172.21.15.117:3300/0 conn(0x7f06080cb9a0 0x7f0608159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060d79b700 1 -- 172.21.15.7:0/1439687793 >> v1:172.21.15.7:6789/0 conn(0x7f060815cd40 legacy=0x7f06080cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060d79b700 1 -- 172.21.15.7:0/1439687793 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608071ab0 con 0x7f06080cb5d0 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f05fffff700 1 -- 172.21.15.7:0/1439687793 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4079930961 0 0) 0x7f0608159b20 con 0x7f060815cd40 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f060d79b700 1 --2- 172.21.15.7:0/1439687793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080b52b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f06080c9c60 tx=0x7f0604009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3358604090cd8c1c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:50.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.192+0000 7f05fffff700 1 -- 172.21.15.7:0/1439687793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604011830 con 0x7f06080cb5d0 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f05fffff700 1 -- 172.21.15.7:0/1439687793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0604011990 con 0x7f06080cb5d0 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f05fffff700 1 -- 172.21.15.7:0/1439687793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604011d20 con 0x7f06080cb5d0 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 -- 172.21.15.7:0/1439687793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 msgr2=0x7f06080b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 --2- 172.21.15.7:0/1439687793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080b52b0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f06080c9c60 tx=0x7f0604009b20 comp rx=0 tx=0).stop 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 -- 172.21.15.7:0/1439687793 shutdown_connections 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 --2- 172.21.15.7:0/1439687793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080b52b0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 --2- 172.21.15.7:0/1439687793 >> v2:172.21.15.117:3300/0 conn(0x7f06080cb9a0 0x7f0608159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 -- 172.21.15.7:0/1439687793 >> 172.21.15.7:0/1439687793 conn(0x7f0608070af0 msgr2=0x7f0608070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 -- 172.21.15.7:0/1439687793 shutdown_connections 2023-06-09T11:47:50.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.193+0000 7f060f9ff700 1 -- 172.21.15.7:0/1439687793 wait complete. 2023-06-09T11:47:50.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.195+0000 7f060f9ff700 1 Processor -- start 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.195+0000 7f060f9ff700 1 -- start start 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06080cb9a0 0x7f06080c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 0x7f06080c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06080c7350 con 0x7f06080cb5d0 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06080cd910 con 0x7f06080cb9a0 2023-06-09T11:47:50.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060f9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06081e5290 con 0x7f060815cd40 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060d79b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47484/0 (socket says 172.21.15.7:47484) 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060d79b700 1 -- 172.21.15.7:0/3056310201 learned_addr learned my addr 172.21.15.7:0/3056310201 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.196+0000 7f060cf9a700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06080cb9a0 0x7f06080c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060df9c700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 0x7f06080c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 -- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 msgr2=0x7f06080c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 0x7f06080c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 -- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06080cb9a0 msgr2=0x7f06080c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06080cb9a0 0x7f06080c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 -- 172.21.15.7:0/3056310201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06040094a0 con 0x7f06080cb5d0 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060df9c700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 0x7f06080c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060d79b700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f06040090b0 tx=0x7f0604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06040036a0 con 0x7f06080cb5d0 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060401f070 con 0x7f06080cb5d0 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060815d4f0 con 0x7f06080cb5d0 2023-06-09T11:47:50.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.197+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0604003a00 con 0x7f06080cb5d0 2023-06-09T11:47:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.198+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060815d210 con 0x7f06080cb5d0 2023-06-09T11:47:50.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.198+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060815d7d0 con 0x7f06080cb5d0 2023-06-09T11:47:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.199+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0604003b60 con 0x7f06080cb5d0 2023-06-09T11:47:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.201+0000 7f05fdffb700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 0x7f05ec06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.201+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f0604021070 con 0x7f06080cb5d0 2023-06-09T11:47:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.201+0000 7f060cf9a700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 0x7f05ec06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:50.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.202+0000 7f060cf9a700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 0x7f05ec06d1c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f06080ccce0 tx=0x7f05f8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:50.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.206+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f06040556d0 con 0x7f06080cb5d0 2023-06-09T11:47:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.459+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06080c7630 con 0x7f06080cb5d0 2023-06-09T11:47:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.460+0000 7f05fdffb700 1 -- 172.21.15.7:0/3056310201 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f060402b020 con 0x7f06080cb5d0 2023-06-09T11:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 msgr2=0x7f05ec06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 0x7f05ec06d1c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f06080ccce0 tx=0x7f05f8009210 comp rx=0 tx=0).stop 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 msgr2=0x7f06080cf440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f06040090b0 tx=0x7f0604004750 comp rx=0 tx=0).stop 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 shutdown_connections 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f05ec06ad10 0x7f05ec06d1c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f060815cd40 0x7f06080c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06080cb5d0 0x7f06080cf440 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 --2- 172.21.15.7:0/3056310201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f06080cb9a0 0x7f06080c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.462+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 >> 172.21.15.7:0/3056310201 conn(0x7f0608070af0 msgr2=0x7f06080cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.463+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 shutdown_connections 2023-06-09T11:47:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:50.463+0000 7f060f9ff700 1 -- 172.21.15.7:0/3056310201 wait complete. 2023-06-09T11:47:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:50.484 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:51.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:51.446 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:47:52.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.694+0000 7f76af9ff700 1 Processor -- start 2023-06-09T11:47:52.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.694+0000 7f76af9ff700 1 -- start start 2023-06-09T11:47:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76af9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f76a815a0c0 0x7f76a81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76af9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a80c0140 0x7f76a81506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76af9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80c0bf0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76af9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a8070110 con 0x7f76a815a0c0 2023-06-09T11:47:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76af9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8071300 con 0x7f76a80c0140 2023-06-09T11:47:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76ad79b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f76a80c1bc0 0x7f76a81599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48564/0 (socket says 172.21.15.7:48564) 2023-06-09T11:47:52.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.695+0000 7f76ad79b700 1 -- 172.21.15.7:0/3390871560 learned_addr learned my addr 172.21.15.7:0/3390871560 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:52.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76adf9c700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a80c0140 0x7f76a81506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76acf9a700 1 --2- 172.21.15.7:0/3390871560 >> v2:172.21.15.117:3300/0 conn(0x7f76a815a0c0 0x7f76a81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546221620 0 0) 0x7f76a80c0bf0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76940035b0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76acf9a700 1 -- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a80c0140 msgr2=0x7f76a81506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76acf9a700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a80c0140 0x7f76a81506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3194836975 0 0) 0x7f76940035b0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76acf9a700 1 -- 172.21.15.7:0/3390871560 >> v1:172.21.15.7:6789/0 conn(0x7f76a80c1bc0 legacy=0x7f76a81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.696+0000 7f76acf9a700 1 -- 172.21.15.7:0/3390871560 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80702f0 con 0x7f76a815a0c0 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76acf9a700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a815a0c0 0x7f76a81501b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7698009870 tx=0x7f7698009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec30d3394236fde2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4001670 con 0x7f76a815a0c0 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7698004030 con 0x7f76a815a0c0 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f769ffff700 1 -- 172.21.15.7:0/3390871560 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980043c0 con 0x7f76a815a0c0 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 -- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a815a0c0 msgr2=0x7f76a81501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a815a0c0 0x7f76a81501b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7698009870 tx=0x7f7698009b80 comp rx=0 tx=0).stop 2023-06-09T11:47:52.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 -- 172.21.15.7:0/3390871560 shutdown_connections 2023-06-09T11:47:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a80c0140 0x7f76a81506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 --2- 172.21.15.7:0/3390871560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a815a0c0 0x7f76a81501b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.697+0000 7f76af9ff700 1 -- 172.21.15.7:0/3390871560 >> 172.21.15.7:0/3390871560 conn(0x7f76a806a4d0 msgr2=0x7f76a806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.698+0000 7f76af9ff700 1 -- 172.21.15.7:0/3390871560 shutdown_connections 2023-06-09T11:47:52.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.698+0000 7f76af9ff700 1 -- 172.21.15.7:0/3390871560 wait complete. 2023-06-09T11:47:52.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.699+0000 7f76af9ff700 1 Processor -- start 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.699+0000 7f76af9ff700 1 -- start start 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 0x7f76a80ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 0x7f76a81f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8162d60 con 0x7f76a80c1bc0 2023-06-09T11:47:52.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a81617b0 con 0x7f76a80c0140 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76af9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80c9c70 con 0x7f76a815a0c0 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.700+0000 7f76acf9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47500/0 (socket says 172.21.15.7:47500) 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 -- 172.21.15.7:0/2800568656 learned_addr learned my addr 172.21.15.7:0/2800568656 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76ad79b700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 0x7f76a80ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76adf9c700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 0x7f76a81f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 -- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 msgr2=0x7f76a81f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 0x7f76a81f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 -- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 msgr2=0x7f76a80ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 0x7f76a80ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 -- 172.21.15.7:0/2800568656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76980094a0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76ad79b700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 0x7f76a80ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76adf9c700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 0x7f76a81f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76acf9a700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f76980090b0 tx=0x7f76980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:52.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698003e50 con 0x7f76a80c1bc0 2023-06-09T11:47:52.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7698011630 con 0x7f76a80c1bc0 2023-06-09T11:47:52.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76980119e0 con 0x7f76a80c1bc0 2023-06-09T11:47:52.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.701+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a80c1170 con 0x7f76a80c1bc0 2023-06-09T11:47:52.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.702+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a80c0e90 con 0x7f76a80c1bc0 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.703+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a80c1450 con 0x7f76a80c1bc0 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.703+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7698011b40 con 0x7f76a80c1bc0 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.704+0000 7f769dffb700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.705+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7840+0+0 (secure 0 0 0) 0x7f769808b260 con 0x7f76a80c1bc0 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.705+0000 7f76ad79b700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:52.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.705+0000 7f76ad79b700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 0x7f768c06d1c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001eb0 tx=0x7f76a4001d80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:52.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.709+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7698055d50 con 0x7f76a80c1bc0 2023-06-09T11:47:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.960+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76a8161920 con 0x7f76a80c1bc0 2023-06-09T11:47:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.961+0000 7f769dffb700 1 -- 172.21.15.7:0/2800568656 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 177 v177) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f769802a090 con 0x7f76a80c1bc0 2023-06-09T11:47:52.965 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:52.965 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":177,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7405,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":175}],"filesystems":[{"mdsmap":{"epoch":177,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:29.069536+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":69,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7141,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7141":{"gid":7141,"name":"f","rank":1,"incarnation":159,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6837/3727664816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3727664816},{"type":"v1","addr":"172.21.15.117:6837","nonce":3727664816}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 msgr2=0x7f768c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 0x7f768c06d1c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f76a4001eb0 tx=0x7f76a4001d80 comp rx=0 tx=0).stop 2023-06-09T11:47:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 msgr2=0x7f76a8161c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f76980090b0 tx=0x7f76980047c0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 shutdown_connections 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f768c06ad10 0x7f768c06d1c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f76a815a0c0 0x7f76a81f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f76a80c0140 0x7f76a80ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 --2- 172.21.15.7:0/2800568656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f76a80c1bc0 0x7f76a8161c40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 >> 172.21.15.7:0/2800568656 conn(0x7f76a806a4d0 msgr2=0x7f76a814ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 shutdown_connections 2023-06-09T11:47:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:52.964+0000 7f76af9ff700 1 -- 172.21.15.7:0/2800568656 wait complete. 2023-06-09T11:47:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 177 2023-06-09T11:47:52.986 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:47:54.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.193+0000 7f6c29dff700 1 Processor -- start 2023-06-09T11:47:55.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.193+0000 7f6c29dff700 1 -- start start 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c29dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6c240cb9a0 0x7f6c240b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c29dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c2415cd40 0x7f6c240cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c29dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c24159b20 con 0x7f6c240cb5d0 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c29dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240cac80 con 0x7f6c2415cd40 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c29dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c241597d0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c2415cd40 0x7f6c240cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c1bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6c240cb9a0 0x7f6c240b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f6c2415cd40 0x7f6c240cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37510/0 (socket says 172.21.15.7:37510) 2023-06-09T11:47:55.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 -- 172.21.15.7:0/1755438063 learned_addr learned my addr 172.21.15.7:0/1755438063 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:55.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 -- 172.21.15.7:0/1755438063 >> v2:172.21.15.117:3300/0 conn(0x7f6c240cb9a0 msgr2=0x7f6c240b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:55.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 --2- 172.21.15.7:0/1755438063 >> v2:172.21.15.117:3300/0 conn(0x7f6c240cb9a0 0x7f6c240b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 -- 172.21.15.7:0/1755438063 >> v1:172.21.15.117:6789/0 conn(0x7f6c240cb5d0 legacy=0x7f6c24159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.195+0000 7f6c23fff700 1 -- 172.21.15.7:0/1755438063 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24071ab0 con 0x7f6c2415cd40 2023-06-09T11:47:55.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c23fff700 1 --2- 172.21.15.7:0/1755438063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2415cd40 0x7f6c240cbea0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6c240c9c60 tx=0x7f6c1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de68b2e7dde5d72 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c22ffd700 1 -- 172.21.15.7:0/1755438063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00e070 con 0x7f6c2415cd40 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c22ffd700 1 -- 172.21.15.7:0/1755438063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c00b910 con 0x7f6c2415cd40 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c22ffd700 1 -- 172.21.15.7:0/1755438063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c00bcc0 con 0x7f6c2415cd40 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c29dff700 1 -- 172.21.15.7:0/1755438063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2415cd40 msgr2=0x7f6c240cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.196+0000 7f6c29dff700 1 --2- 172.21.15.7:0/1755438063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2415cd40 0x7f6c240cbea0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f6c240c9c60 tx=0x7f6c1c00b330 comp rx=0 tx=0).stop 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 -- 172.21.15.7:0/1755438063 shutdown_connections 2023-06-09T11:47:55.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 --2- 172.21.15.7:0/1755438063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c2415cd40 0x7f6c240cbea0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 --2- 172.21.15.7:0/1755438063 >> v2:172.21.15.117:3300/0 conn(0x7f6c240cb9a0 0x7f6c240b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 -- 172.21.15.7:0/1755438063 >> 172.21.15.7:0/1755438063 conn(0x7f6c24070af0 msgr2=0x7f6c24070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 -- 172.21.15.7:0/1755438063 shutdown_connections 2023-06-09T11:47:55.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.197+0000 7f6c29dff700 1 -- 172.21.15.7:0/1755438063 wait complete. 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.198+0000 7f6c29dff700 1 Processor -- start 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 -- start start 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240cb5d0 0x7f6c24152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c2415cd40 0x7f6c24150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c241f0290 con 0x7f6c240cb9a0 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cfdc0 con 0x7f6c240cb5d0 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.199+0000 7f6c29dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c240cd560 con 0x7f6c2415cd40 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37522/0 (socket says 172.21.15.7:37522) 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 -- 172.21.15.7:0/163372067 learned_addr learned my addr 172.21.15.7:0/163372067 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c237fe700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240cb5d0 0x7f6c24152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c23fff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c2415cd40 0x7f6c24150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 -- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c2415cd40 msgr2=0x7f6c24150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c2415cd40 0x7f6c24150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 -- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240cb5d0 msgr2=0x7f6c24152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240cb5d0 0x7f6c24152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 -- 172.21.15.7:0/163372067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009b30 con 0x7f6c240cb9a0 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.200+0000 7f6c1bfff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00e980 tx=0x7f6c0c00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.201+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c009710 con 0x7f6c240cb9a0 2023-06-09T11:47:55.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.201+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c00c690 con 0x7f6c240cb9a0 2023-06-09T11:47:55.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.201+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00ca40 con 0x7f6c240cb9a0 2023-06-09T11:47:55.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.201+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2415d4f0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.201+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2415d210 con 0x7f6c240cb9a0 2023-06-09T11:47:55.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.202+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2415d7d0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.203+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c0c00cba0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.204+0000 7f6c20ff9700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 0x7f6c0406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.204+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f6c0c089fe0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.204+0000 7f6c237fe700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 0x7f6c0406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.205+0000 7f6c237fe700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 0x7f6c0406d1c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.208+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c0c054810 con 0x7f6c240cb9a0 2023-06-09T11:47:55.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.472+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c2415ade0 con 0x7f6c240cb9a0 2023-06-09T11:47:55.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.473+0000 7f6c20ff9700 1 -- 172.21.15.7:0/163372067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f6c0c021020 con 0x7f6c240cb9a0 2023-06-09T11:47:55.477 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:55.477 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:54.624665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 msgr2=0x7f6c0406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 0x7f6c0406d1c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c14007fd0 tx=0x7f6c14009040 comp rx=0 tx=0).stop 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 msgr2=0x7f6c241538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00e980 tx=0x7f6c0c00ec90 comp rx=0 tx=0).stop 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 shutdown_connections 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c0406ad10 0x7f6c0406d1c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c2415cd40 0x7f6c24150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c240cb5d0 0x7f6c24152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 --2- 172.21.15.7:0/163372067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c240cb9a0 0x7f6c241538d0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 >> 172.21.15.7:0/163372067 conn(0x7f6c24070af0 msgr2=0x7f6c240cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 shutdown_connections 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.476+0000 7f6c29dff700 1 -- 172.21.15.7:0/163372067 wait complete. 2023-06-09T11:47:55.481 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 178 2023-06-09T11:47:55.498 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:47:55.498 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:47:55.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.694+0000 7fb3545ff700 1 Processor -- start 2023-06-09T11:47:55.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.694+0000 7fb3545ff700 1 -- start start 2023-06-09T11:47:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34c153880 0x7fb34c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c0c9f40 0x7fb34c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb34c15b2d0 0x7fb34c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0c5f10 con 0x7fb34c153880 2023-06-09T11:47:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c15a500 con 0x7fb34c15b2d0 2023-06-09T11:47:55.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb3545ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c15da00 con 0x7fb34c0c9f40 2023-06-09T11:47:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.695+0000 7fb35239b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34c153880 0x7fb34c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb34c153880 0x7fb34c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37532/0 (socket says 172.21.15.7:37532) 2023-06-09T11:47:55.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 -- 172.21.15.7:0/4082422969 learned_addr learned my addr 172.21.15.7:0/4082422969 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb351b9a700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c0c9f40 0x7fb34c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb352b9c700 1 --2- 172.21.15.7:0/4082422969 >> v2:172.21.15.117:3300/0 conn(0x7fb34c15b2d0 0x7fb34c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 -- 172.21.15.7:0/4082422969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c0c9f40 msgr2=0x7fb34c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c0c9f40 0x7fb34c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 -- 172.21.15.7:0/4082422969 >> v2:172.21.15.117:3300/0 conn(0x7fb34c15b2d0 msgr2=0x7fb34c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 --2- 172.21.15.7:0/4082422969 >> v2:172.21.15.117:3300/0 conn(0x7fb34c15b2d0 0x7fb34c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 -- 172.21.15.7:0/4082422969 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c15b750 con 0x7fb34c153880 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb35239b700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c153880 0x7fb34c15d4c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb34c071db0 tx=0x7fb348009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302907846ab5ae80 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb351399700 1 -- 172.21.15.7:0/4082422969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800b870 con 0x7fb34c153880 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb351399700 1 -- 172.21.15.7:0/4082422969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34800b9d0 con 0x7fb34c153880 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.696+0000 7fb351399700 1 -- 172.21.15.7:0/4082422969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800bd60 con 0x7fb34c153880 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 -- 172.21.15.7:0/4082422969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c153880 msgr2=0x7fb34c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c153880 0x7fb34c15d4c0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb34c071db0 tx=0x7fb348009bd0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 -- 172.21.15.7:0/4082422969 shutdown_connections 2023-06-09T11:47:55.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 --2- 172.21.15.7:0/4082422969 >> v2:172.21.15.117:3300/0 conn(0x7fb34c15b2d0 0x7fb34c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c153880 0x7fb34c15d4c0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 --2- 172.21.15.7:0/4082422969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c0c9f40 0x7fb34c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 -- 172.21.15.7:0/4082422969 >> 172.21.15.7:0/4082422969 conn(0x7fb34c0737f0 msgr2=0x7fb34c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 -- 172.21.15.7:0/4082422969 shutdown_connections 2023-06-09T11:47:55.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.697+0000 7fb3545ff700 1 -- 172.21.15.7:0/4082422969 wait complete. 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.699+0000 7fb3545ff700 1 Processor -- start 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.699+0000 7fb3545ff700 1 -- start start 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c153880 0x7fb34c1ef380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb34c15b2d0 0x7fb34c1ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b4c30 con 0x7fb34c0c9f40 2023-06-09T11:47:55.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1eef70 con 0x7fb34c15b2d0 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb3545ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c1f25f0 con 0x7fb34c153880 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb35239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb35239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37536/0 (socket says 172.21.15.7:37536) 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb35239b700 1 -- 172.21.15.7:0/3780398607 learned_addr learned my addr 172.21.15.7:0/3780398607 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb352b9c700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb34c15b2d0 0x7fb34c1ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.700+0000 7fb351b9a700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c153880 0x7fb34c1ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 -- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c153880 msgr2=0x7fb34c1ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c153880 0x7fb34c1ef380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 -- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb34c15b2d0 msgr2=0x7fb34c1ef8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb34c15b2d0 0x7fb34c1ef8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 -- 172.21.15.7:0/3780398607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348009550 con 0x7fb34c0c9f40 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb35239b700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb348000c00 tx=0x7fb348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348004480 con 0x7fb34c0c9f40 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3480045e0 con 0x7fb34c0c9f40 2023-06-09T11:47:55.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3480038e0 con 0x7fb34c0c9f40 2023-06-09T11:47:55.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c159980 con 0x7fb34c0c9f40 2023-06-09T11:47:55.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.701+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c1596e0 con 0x7fb34c0c9f40 2023-06-09T11:47:55.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.702+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c159c60 con 0x7fb34c0c9f40 2023-06-09T11:47:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.703+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb348021070 con 0x7fb34c0c9f40 2023-06-09T11:47:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.705+0000 7fb342ffd700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 0x7fb33006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.705+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fb34808b210 con 0x7fb34c0c9f40 2023-06-09T11:47:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.705+0000 7fb351b9a700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 0x7fb33006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:55.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.706+0000 7fb351b9a700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 0x7fb33006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb34c15a290 tx=0x7fb33c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:55.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.709+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb348055b60 con 0x7fb34c0c9f40 2023-06-09T11:47:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.958+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb34c074cf0 con 0x7fb34c0c9f40 2023-06-09T11:47:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.959+0000 7fb342ffd700 1 -- 172.21.15.7:0/3780398607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+5463 (secure 0 0 0) 0x7fb348015090 con 0x7fb34c0c9f40 2023-06-09T11:47:55.963 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:55.963 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:54.624665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 msgr2=0x7fb33006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 0x7fb33006d190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb34c15a290 tx=0x7fb33c008040 comp rx=0 tx=0).stop 2023-06-09T11:47:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 msgr2=0x7fb34c14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fb348000c00 tx=0x7fb348004750 comp rx=0 tx=0).stop 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 shutdown_connections 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb34c15b2d0 0x7fb34c1ef8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb34c0c9f40 0x7fb34c14b160 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb33006ace0 0x7fb33006d190 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 --2- 172.21.15.7:0/3780398607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb34c153880 0x7fb34c1ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 >> 172.21.15.7:0/3780398607 conn(0x7fb34c0737f0 msgr2=0x7fb34c15b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:55.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 shutdown_connections 2023-06-09T11:47:55.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:55.961+0000 7fb3545ff700 1 -- 172.21.15.7:0/3780398607 wait complete. 2023-06-09T11:47:55.967 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 178 2023-06-09T11:47:55.984 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 178, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 178, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7279": { "addr": "172.21.15.117:6833/3302752268", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 3302752268, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 3302752268, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7279, "incarnation": 169, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:replay", "state_seq": 1 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 70, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:47:54.624665+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 7279 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:47:57.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:47:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.191+0000 7f1d77bff700 1 Processor -- start 2023-06-09T11:47:58.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.191+0000 7f1d77bff700 1 -- start start 2023-06-09T11:47:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.192+0000 7f1d77bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1d700cc5b0 0x7f1d700b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:58.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.192+0000 7f1d77bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d70159b20 con 0x7f1d700cb9a0 2023-06-09T11:47:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.192+0000 7f1d77bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d700cac80 con 0x7f1d700cb5d0 2023-06-09T11:47:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.192+0000 7f1d77bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d701597d0 con 0x7f1d700cc5b0 2023-06-09T11:47:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7519a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1d700cb9a0 0x7f1d700cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43742/0 (socket says 172.21.15.7:43742) 2023-06-09T11:47:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7519a700 1 -- 172.21.15.7:0/783703317 learned_addr learned my addr 172.21.15.7:0/783703317 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:47:58.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7619c700 1 --2- 172.21.15.7:0/783703317 >> v2:172.21.15.7:3300/0 conn(0x7f1d700cc5b0 0x7f1d700b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418853960 0 0) 0x7f1d70159b20 con 0x7f1d700cb9a0 2023-06-09T11:47:58.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d5c0035b0 con 0x7f1d700cb9a0 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444582171 0 0) 0x7f1d700cac80 con 0x7f1d700cb5d0 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7619c700 1 -- 172.21.15.7:0/783703317 >> v1:172.21.15.117:6789/0 conn(0x7f1d700cb5d0 legacy=0x7f1d70159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7619c700 1 -- 172.21.15.7:0/783703317 >> v1:172.21.15.7:6789/0 conn(0x7f1d700cb9a0 legacy=0x7f1d700cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7619c700 1 -- 172.21.15.7:0/783703317 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d70071ab0 con 0x7f1d700cc5b0 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.193+0000 7f1d7619c700 1 --2- 172.21.15.7:0/783703317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cc5b0 0x7f1d700b52b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009d90 tx=0x7f1d6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b97dfc431689a42 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800e070 con 0x7f1d700cc5b0 2023-06-09T11:47:58.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6800bb20 con 0x7f1d700cc5b0 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d74999700 1 -- 172.21.15.7:0/783703317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800bed0 con 0x7f1d700cc5b0 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 -- 172.21.15.7:0/783703317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cc5b0 msgr2=0x7f1d700b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 --2- 172.21.15.7:0/783703317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cc5b0 0x7f1d700b52b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009d90 tx=0x7f1d6800b330 comp rx=0 tx=0).stop 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 -- 172.21.15.7:0/783703317 shutdown_connections 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 --2- 172.21.15.7:0/783703317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cc5b0 0x7f1d700b52b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 -- 172.21.15.7:0/783703317 >> 172.21.15.7:0/783703317 conn(0x7f1d70070af0 msgr2=0x7f1d70070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.194+0000 7f1d77bff700 1 -- 172.21.15.7:0/783703317 shutdown_connections 2023-06-09T11:47:58.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.195+0000 7f1d77bff700 1 -- 172.21.15.7:0/783703317 wait complete. 2023-06-09T11:47:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.196+0000 7f1d77bff700 1 Processor -- start 2023-06-09T11:47:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.196+0000 7f1d77bff700 1 -- start start 2023-06-09T11:47:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d700cb9a0 0x7f1d70158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:58.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 0x7f1d70152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d70150fe0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d701f21d0 con 0x7f1d700cc5b0 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d77bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d701518c0 con 0x7f1d700cb9a0 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d7599b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d7599b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37562/0 (socket says 172.21.15.7:37562) 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.197+0000 7f1d7599b700 1 -- 172.21.15.7:0/1650613383 learned_addr learned my addr 172.21.15.7:0/1650613383 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:47:58.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7519a700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d700cb9a0 0x7f1d70158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7619c700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 0x7f1d70152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 -- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d700cb9a0 msgr2=0x7f1d70158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d700cb9a0 0x7f1d70158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 -- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 msgr2=0x7f1d70152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 0x7f1d70152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 -- 172.21.15.7:0/1650613383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009a70 con 0x7f1d700cb5d0 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7619c700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 0x7f1d70152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:47:58.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d7599b700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0011e0 tx=0x7f1d6c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c0054e0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c003680 con 0x7f1d700cb5d0 2023-06-09T11:47:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.198+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7015d4f0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.199+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c003a30 con 0x7f1d700cb5d0 2023-06-09T11:47:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.199+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7015d210 con 0x7f1d700cb5d0 2023-06-09T11:47:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.200+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7015d7d0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.201+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d6c003b90 con 0x7f1d700cb5d0 2023-06-09T11:47:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.202+0000 7f1d667fc700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 0x7f1d5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:47:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.202+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f1d6c088fe0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.202+0000 7f1d7519a700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 0x7f1d5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:47:58.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.203+0000 7f1d7519a700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 0x7f1d5406d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1d701ebe60 tx=0x7f1d60009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:47:58.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.207+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1d6c053970 con 0x7f1d700cb5d0 2023-06-09T11:47:58.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.458+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d70074cf0 con 0x7f1d700cb5d0 2023-06-09T11:47:58.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.458+0000 7f1d667fc700 1 -- 172.21.15.7:0/1650613383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f1d6c017020 con 0x7f1d700cb5d0 2023-06-09T11:47:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:47:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:54.624665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:47:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 msgr2=0x7f1d5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 0x7f1d5406d190 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1d701ebe60 tx=0x7f1d60009040 comp rx=0 tx=0).stop 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 msgr2=0x7f1d70158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c0011e0 tx=0x7f1d6c00ca30 comp rx=0 tx=0).stop 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 shutdown_connections 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1d700cc5b0 0x7f1d70152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d700cb5d0 0x7f1d70158580 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1d5406ace0 0x7f1d5406d190 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 --2- 172.21.15.7:0/1650613383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1d700cb9a0 0x7f1d70158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:47:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 >> 172.21.15.7:0/1650613383 conn(0x7f1d70070af0 msgr2=0x7f1d700b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:47:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 shutdown_connections 2023-06-09T11:47:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:47:58.461+0000 7f1d77bff700 1 -- 172.21.15.7:0/1650613383 wait complete. 2023-06-09T11:47:58.513 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 178 2023-06-09T11:47:58.513 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:00.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:00.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.724+0000 7f28645ff700 1 Processor -- start 2023-06-09T11:48:00.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.724+0000 7f28645ff700 1 -- start start 2023-06-09T11:48:00.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.725+0000 7f28645ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.725+0000 7f28645ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 0x7f285c0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.725+0000 7f28645ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c15a420 con 0x7f285c0c4ce0 2023-06-09T11:48:00.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.725+0000 7f28645ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285c0c5a00 con 0x7f285c1539b0 2023-06-09T11:48:00.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.725+0000 7f28645ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c06b460 con 0x7f285c1535e0 2023-06-09T11:48:00.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46142/0 (socket says 172.21.15.7:46142) 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 -- 172.21.15.7:0/3650773162 learned_addr learned my addr 172.21.15.7:0/3650773162 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f2861b9a700 1 --2- 172.21.15.7:0/3650773162 >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 0x7f285c0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 -- 172.21.15.7:0/3650773162 >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 msgr2=0x7f285c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 --2- 172.21.15.7:0/3650773162 >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 0x7f285c0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 -- 172.21.15.7:0/3650773162 >> v1:172.21.15.117:6789/0 conn(0x7f285c0c4ce0 legacy=0x7f285c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 -- 172.21.15.7:0/3650773162 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c06a570 con 0x7f285c1535e0 2023-06-09T11:48:00.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f2861b9a700 1 --2- 172.21.15.7:0/3650773162 >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 0x7f285c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f2861399700 1 -- 172.21.15.7:0/3650773162 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824862221 0 0) 0x7f2854001490 con 0x7f285c0c4ce0 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.726+0000 7f286239b700 1 --2- 172.21.15.7:0/3650773162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28580098f0 tx=0x7f2858009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131bb26d324fafc2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f2861399700 1 -- 172.21.15.7:0/3650773162 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858011830 con 0x7f285c1535e0 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f2861399700 1 -- 172.21.15.7:0/3650773162 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2858011990 con 0x7f285c1535e0 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f2861399700 1 -- 172.21.15.7:0/3650773162 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2858011d20 con 0x7f285c1535e0 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 -- 172.21.15.7:0/3650773162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 msgr2=0x7f285c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 --2- 172.21.15.7:0/3650773162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28580098f0 tx=0x7f2858009c00 comp rx=0 tx=0).stop 2023-06-09T11:48:00.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 -- 172.21.15.7:0/3650773162 shutdown_connections 2023-06-09T11:48:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 --2- 172.21.15.7:0/3650773162 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c159ee0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 --2- 172.21.15.7:0/3650773162 >> v2:172.21.15.117:3300/0 conn(0x7f285c1539b0 0x7f285c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 -- 172.21.15.7:0/3650773162 >> 172.21.15.7:0/3650773162 conn(0x7f285c0737f0 msgr2=0x7f285c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 -- 172.21.15.7:0/3650773162 shutdown_connections 2023-06-09T11:48:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.727+0000 7f28645ff700 1 -- 172.21.15.7:0/3650773162 wait complete. 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.729+0000 7f28645ff700 1 Processor -- start 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.729+0000 7f28645ff700 1 -- start start 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c1f0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1539b0 0x7f285c1f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0c7800 con 0x7f285c1539b0 2023-06-09T11:48:00.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c1efce0 con 0x7f285c0c4ce0 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f28645ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0c6a60 con 0x7f285c1535e0 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f286239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f2862b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1539b0 0x7f285c1f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f2861b9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c1f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f286239b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:39424/0 (socket says 172.21.15.7:39424) 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.730+0000 7f286239b700 1 -- 172.21.15.7:0/2672124827 learned_addr learned my addr 172.21.15.7:0/2672124827 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 -- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 msgr2=0x7f285c158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:00.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 -- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1539b0 msgr2=0x7f285c1f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1539b0 0x7f285c1f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 -- 172.21.15.7:0/2672124827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0094a0 con 0x7f285c1535e0 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f286239b700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2861b9a700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c1f0170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e8d0 tx=0x7f284c00ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c013680 con 0x7f285c1535e0 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c0137e0 con 0x7f285c1535e0 2023-06-09T11:48:00.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858009550 con 0x7f285c1535e0 2023-06-09T11:48:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.731+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c013b90 con 0x7f285c1535e0 2023-06-09T11:48:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.732+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c06ab30 con 0x7f285c1535e0 2023-06-09T11:48:00.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.733+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c06a850 con 0x7f285c1535e0 2023-06-09T11:48:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.734+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f284c013cf0 con 0x7f285c1535e0 2023-06-09T11:48:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.735+0000 7f2852ffd700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 0x7f284006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.735+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f284c089be0 con 0x7f285c1535e0 2023-06-09T11:48:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.735+0000 7f286239b700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 0x7f284006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:00.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.735+0000 7f286239b700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 0x7f284006d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2858009c40 tx=0x7f2858008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:00.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.740+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f284c054410 con 0x7f285c1535e0 2023-06-09T11:48:00.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.993+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f285c06ae10 con 0x7f285c1535e0 2023-06-09T11:48:00.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.993+0000 7f2852ffd700 1 -- 172.21.15.7:0/2672124827 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 178 v178) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f284c021020 con 0x7f285c1535e0 2023-06-09T11:48:00.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:00.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":178,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":178,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:47:54.624665+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 msgr2=0x7f284006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 0x7f284006d190 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2858009c40 tx=0x7f2858008030 comp rx=0 tx=0).stop 2023-06-09T11:48:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 msgr2=0x7f285c1f0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:01.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c1f0170 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e8d0 tx=0x7f284c00ebe0 comp rx=0 tx=0).stop 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 shutdown_connections 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f285c1539b0 0x7f285c1f06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f285c0c4ce0 0x7f285c158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f284006ace0 0x7f284006d190 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 --2- 172.21.15.7:0/2672124827 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f285c1535e0 0x7f285c1f0170 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 >> 172.21.15.7:0/2672124827 conn(0x7f285c0737f0 msgr2=0x7f285c152870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 shutdown_connections 2023-06-09T11:48:01.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:00.996+0000 7f28645ff700 1 -- 172.21.15.7:0/2672124827 wait complete. 2023-06-09T11:48:01.002 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 178 2023-06-09T11:48:01.018 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:03.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:03.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.227+0000 7fa4077ff700 1 Processor -- start 2023-06-09T11:48:03.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.227+0000 7fa4077ff700 1 -- start start 2023-06-09T11:48:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa4077ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa4000c9090 0x7fa4000c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa4077ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 0x7fa4000cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa4077ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40015cec0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa4077ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40015d160 con 0x7fa4000c9090 2023-06-09T11:48:03.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa4077ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40015d440 con 0x7fa4000cad00 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa405d9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa4000cfaf0 0x7fa4001598b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50016/0 (socket says 172.21.15.7:50016) 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa405d9c700 1 -- 172.21.15.7:0/832554807 learned_addr learned my addr 172.21.15.7:0/832554807 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa404d9a700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 0x7fa4000cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.228+0000 7fa40559b700 1 --2- 172.21.15.7:0/832554807 >> v2:172.21.15.117:3300/0 conn(0x7fa4000c9090 0x7fa4000c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa3f7fff700 1 -- 172.21.15.7:0/832554807 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848410861 0 0) 0x7fa40015cec0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa3f7fff700 1 -- 172.21.15.7:0/832554807 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3ec0035b0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa40559b700 1 -- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 msgr2=0x7fa4000cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa40559b700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 0x7fa4000cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa40559b700 1 -- 172.21.15.7:0/832554807 >> v1:172.21.15.7:6789/0 conn(0x7fa4000cfaf0 legacy=0x7fa4001598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa40559b700 1 -- 172.21.15.7:0/832554807 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40015d720 con 0x7fa4000c9090 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa404d9a700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 0x7fa4000cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa40559b700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000c9090 0x7fa4000c9fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009840 tx=0x7fa3fc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68af3a0eed5e28f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa3f7fff700 1 -- 172.21.15.7:0/832554807 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc011830 con 0x7fa4000c9090 2023-06-09T11:48:03.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa3f7fff700 1 -- 172.21.15.7:0/832554807 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc011990 con 0x7fa4000c9090 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.229+0000 7fa3f7fff700 1 -- 172.21.15.7:0/832554807 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc011d20 con 0x7fa4000c9090 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 -- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000c9090 msgr2=0x7fa4000c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000c9090 0x7fa4000c9fe0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc009840 tx=0x7fa3fc009b50 comp rx=0 tx=0).stop 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 -- 172.21.15.7:0/832554807 shutdown_connections 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000c9090 0x7fa4000c9fe0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 --2- 172.21.15.7:0/832554807 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000cad00 0x7fa4000cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 -- 172.21.15.7:0/832554807 >> 172.21.15.7:0/832554807 conn(0x7fa400070af0 msgr2=0x7fa400070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 -- 172.21.15.7:0/832554807 shutdown_connections 2023-06-09T11:48:03.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.230+0000 7fa4077ff700 1 -- 172.21.15.7:0/832554807 wait complete. 2023-06-09T11:48:03.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.232+0000 7fa4077ff700 1 Processor -- start 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.232+0000 7fa4077ff700 1 -- start start 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000c9090 0x7fa4000c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000cad00 0x7fa4001f2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000c5700 con 0x7fa4000cfaf0 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000c5c80 con 0x7fa4000cad00 2023-06-09T11:48:03.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa4077ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000c3190 con 0x7fa4000c9090 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa405d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa405d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59048/0 (socket says 172.21.15.7:59048) 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa405d9c700 1 -- 172.21.15.7:0/929116074 learned_addr learned my addr 172.21.15.7:0/929116074 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa40559b700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000c9090 0x7fa4000c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.233+0000 7fa404d9a700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000cad00 0x7fa4001f2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 -- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000c9090 msgr2=0x7fa4000c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000c9090 0x7fa4000c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 -- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000cad00 msgr2=0x7fa4001f2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000cad00 0x7fa4001f2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 -- 172.21.15.7:0/929116074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00094a0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa405d9c700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800caa0 tx=0x7fa3f800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800a980 con 0x7fa4000cfaf0 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f800aae0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc0094a0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800ae70 con 0x7fa4000cfaf0 2023-06-09T11:48:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.234+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4000cb130 con 0x7fa4000cfaf0 2023-06-09T11:48:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.235+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4000ca620 con 0x7fa4000cfaf0 2023-06-09T11:48:03.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.236+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3f801b3b0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.237+0000 7fa3f5ffb700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 0x7fa3e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.237+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fa3f8089810 con 0x7fa4000cfaf0 2023-06-09T11:48:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.237+0000 7fa40559b700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 0x7fa3e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:03.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.238+0000 7fa40559b700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 0x7fa3e406d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000c00 tx=0x7fa3fc01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:03.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.242+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa3f80541a0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.498+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4000cb2c0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.499+0000 7fa3f5ffb700 1 -- 172.21.15.7:0/929116074 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fa3f80577c0 con 0x7fa4000cfaf0 2023-06-09T11:48:03.503 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:03.503 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:02.839208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 msgr2=0x7fa3e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 0x7fa3e406d190 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc000c00 tx=0x7fa3fc01c040 comp rx=0 tx=0).stop 2023-06-09T11:48:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 msgr2=0x7fa4000cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:03.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800caa0 tx=0x7fa3f800cdb0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 shutdown_connections 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa4000cfaf0 0x7fa4000cf1b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa3e406ace0 0x7fa3e406d190 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4000c9090 0x7fa4000c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 --2- 172.21.15.7:0/929116074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4000cad00 0x7fa4001f2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 >> 172.21.15.7:0/929116074 conn(0x7fa400070af0 msgr2=0x7fa4000c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 shutdown_connections 2023-06-09T11:48:03.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:03.501+0000 7fa4077ff700 1 -- 172.21.15.7:0/929116074 wait complete. 2023-06-09T11:48:03.507 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 179 2023-06-09T11:48:03.523 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:05.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.731+0000 7f9fbdbff700 1 Processor -- start 2023-06-09T11:48:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.731+0000 7f9fbdbff700 1 -- start start 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fbdbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb81539b0 0x7f9fb8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fbdbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fb80c4ce0 0x7f9fb80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fbdbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb815a420 con 0x7f9fb81535e0 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fbdbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb80ca670 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fbdbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c5a00 con 0x7f9fb81539b0 2023-06-09T11:48:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fb80c4ce0 0x7f9fb80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9fb80c4ce0 0x7f9fb80c5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59058/0 (socket says 172.21.15.7:59058) 2023-06-09T11:48:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 -- 172.21.15.7:0/2867318026 learned_addr learned my addr 172.21.15.7:0/2867318026 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb6ffd700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb81539b0 0x7f9fb8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 -- 172.21.15.7:0/2867318026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb81539b0 msgr2=0x7f9fb8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb81539b0 0x7f9fb8159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 -- 172.21.15.7:0/2867318026 >> v1:172.21.15.7:6789/0 conn(0x7f9fb81535e0 legacy=0x7f9fb80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb7fff700 1 -- 172.21.15.7:0/2867318026 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb806b460 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.733+0000 7f9fb67fc700 1 -- 172.21.15.7:0/2867318026 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2772059826 0 0) 0x7f9fa8001110 con 0x7f9fb81535e0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fb7fff700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb80c4ce0 0x7f9fb80c5180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aa8371908dbb31 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fb67fc700 1 -- 172.21.15.7:0/2867318026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000e070 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fb67fc700 1 -- 172.21.15.7:0/2867318026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb000b910 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fb67fc700 1 -- 172.21.15.7:0/2867318026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000bcc0 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fbdbff700 1 -- 172.21.15.7:0/2867318026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb80c4ce0 msgr2=0x7f9fb80c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.734+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb80c4ce0 0x7f9fb80c5180 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009e50 tx=0x7f9fb000b330 comp rx=0 tx=0).stop 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 -- 172.21.15.7:0/2867318026 shutdown_connections 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb80c4ce0 0x7f9fb80c5180 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/2867318026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb81539b0 0x7f9fb8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 -- 172.21.15.7:0/2867318026 >> 172.21.15.7:0/2867318026 conn(0x7f9fb80737f0 msgr2=0x7f9fb8072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 -- 172.21.15.7:0/2867318026 shutdown_connections 2023-06-09T11:48:05.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.735+0000 7f9fbdbff700 1 -- 172.21.15.7:0/2867318026 wait complete. 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.736+0000 7f9fbdbff700 1 Processor -- start 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.736+0000 7f9fbdbff700 1 -- start start 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb80c4ce0 0x7f9fb814f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb81535e0 0x7f9fb8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81557f0 con 0x7f9fb81539b0 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb81f1820 con 0x7f9fb81535e0 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.737+0000 7f9fbdbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb8072320 con 0x7f9fb80c4ce0 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59072/0 (socket says 172.21.15.7:59072) 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 -- 172.21.15.7:0/1250091912 learned_addr learned my addr 172.21.15.7:0/1250091912 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 -- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb80c4ce0 msgr2=0x7f9fb814f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb80c4ce0 0x7f9fb814f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:05.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 -- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb81535e0 msgr2=0x7f9fb8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb81535e0 0x7f9fb8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 -- 172.21.15.7:0/1250091912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb0009b30 con 0x7f9fb81539b0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fb7fff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b680 tx=0x7f9fb0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00092e0 con 0x7f9fb81539b0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb001c440 con 0x7f9fb81539b0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb806a850 con 0x7f9fb81539b0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.738+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb001c810 con 0x7f9fb81539b0 2023-06-09T11:48:05.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.739+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb806a570 con 0x7f9fb81539b0 2023-06-09T11:48:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.739+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb806ab30 con 0x7f9fb81539b0 2023-06-09T11:48:05.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.741+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fb001c970 con 0x7f9fb81539b0 2023-06-09T11:48:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.741+0000 7f9f9ffff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.741+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f9fb008b650 con 0x7f9fb81539b0 2023-06-09T11:48:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.742+0000 7f9fb77fe700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:05.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.742+0000 7f9fb77fe700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 0x7f9fa406d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8001690 tx=0x7f9fa8007850 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:05.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.746+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9fb0055fa0 con 0x7f9fb81539b0 2023-06-09T11:48:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.998+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb8074cf0 con 0x7f9fb81539b0 2023-06-09T11:48:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:05.999+0000 7f9f9ffff700 1 -- 172.21.15.7:0/1250091912 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f9fb00595c0 con 0x7f9fb81539b0 2023-06-09T11:48:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:06.003 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:02.839208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 msgr2=0x7f9fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 0x7f9fa406d190 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8001690 tx=0x7f9fa8007850 comp rx=0 tx=0).stop 2023-06-09T11:48:06.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 msgr2=0x7f9fb8155ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000b680 tx=0x7f9fb0004ad0 comp rx=0 tx=0).stop 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 shutdown_connections 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9fb81539b0 0x7f9fb8155ab0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9fb80c4ce0 0x7f9fb814f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 --2- 172.21.15.7:0/1250091912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9fb81535e0 0x7f9fb8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 >> 172.21.15.7:0/1250091912 conn(0x7f9fb80737f0 msgr2=0x7f9fb80cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 shutdown_connections 2023-06-09T11:48:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:06.001+0000 7f9fbdbff700 1 -- 172.21.15.7:0/1250091912 wait complete. 2023-06-09T11:48:06.007 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 179 2023-06-09T11:48:06.023 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:08.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:08.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.230+0000 7fe11e3ff700 1 Processor -- start 2023-06-09T11:48:08.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.230+0000 7fe11e3ff700 1 -- start start 2023-06-09T11:48:08.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11e3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe11815b4a0 0x7fe11815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180c9f40 0x7fe1180c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11e3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11815a500 con 0x7fe118153880 2023-06-09T11:48:08.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11e3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180c5f10 con 0x7fe11815b4a0 2023-06-09T11:48:08.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11815da00 con 0x7fe1180c9f40 2023-06-09T11:48:08.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe1177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe11815b4a0 0x7fe11815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe1177fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe11815b4a0 0x7fe11815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59082/0 (socket says 172.21.15.7:59082) 2023-06-09T11:48:08.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe1177fe700 1 -- 172.21.15.7:0/594276790 learned_addr learned my addr 172.21.15.7:0/594276790 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:08.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.232+0000 7fe11c99c700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180c9f40 0x7fe1180c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe1177fe700 1 -- 172.21.15.7:0/594276790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180c9f40 msgr2=0x7fe1180c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:08.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe1177fe700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180c9f40 0x7fe1180c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe1177fe700 1 -- 172.21.15.7:0/594276790 >> v1:172.21.15.117:6789/0 conn(0x7fe118153880 legacy=0x7fe11815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe1177fe700 1 -- 172.21.15.7:0/594276790 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1181596e0 con 0x7fe11815b4a0 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe1177fe700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815b4a0 0x7fe11815d4c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0098f0 tx=0x7fe10c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913bcea4160a07ec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe116ffd700 1 -- 172.21.15.7:0/594276790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c011830 con 0x7fe11815b4a0 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe116ffd700 1 -- 172.21.15.7:0/594276790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe10c011990 con 0x7fe11815b4a0 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.233+0000 7fe116ffd700 1 -- 172.21.15.7:0/594276790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10c011d20 con 0x7fe11815b4a0 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 -- 172.21.15.7:0/594276790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815b4a0 msgr2=0x7fe11815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815b4a0 0x7fe11815d4c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe10c0098f0 tx=0x7fe10c009c00 comp rx=0 tx=0).stop 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 -- 172.21.15.7:0/594276790 shutdown_connections 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe1180c9f40 0x7fe1180c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/594276790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe11815b4a0 0x7fe11815d4c0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 -- 172.21.15.7:0/594276790 >> 172.21.15.7:0/594276790 conn(0x7fe1180737f0 msgr2=0x7fe1180720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 -- 172.21.15.7:0/594276790 shutdown_connections 2023-06-09T11:48:08.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.234+0000 7fe11e3ff700 1 -- 172.21.15.7:0/594276790 wait complete. 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.235+0000 7fe11e3ff700 1 Processor -- start 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.236+0000 7fe11e3ff700 1 -- start start 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe118153880 0x7fe118149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11815b4a0 0x7fe11814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe118151670 con 0x7fe1180c9f40 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181529e0 con 0x7fe11815b4a0 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe11814b660 con 0x7fe118153880 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59096/0 (socket says 172.21.15.7:59096) 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe1177fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe118153880 0x7fe118149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 -- 172.21.15.7:0/3870851284 learned_addr learned my addr 172.21.15.7:0/3870851284 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe11c99c700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11815b4a0 0x7fe11814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 -- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe118153880 msgr2=0x7fe118149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe118153880 0x7fe118149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 -- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11815b4a0 msgr2=0x7fe11814eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11815b4a0 0x7fe11814eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.237+0000 7fe117fff700 1 -- 172.21.15.7:0/3870851284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c009550 con 0x7fe1180c9f40 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe117fff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c4e0 tx=0x7fe10800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080052d0 con 0x7fe1180c9f40 2023-06-09T11:48:08.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108005430 con 0x7fe1180c9f40 2023-06-09T11:48:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1080038c0 con 0x7fe1180c9f40 2023-06-09T11:48:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118159c60 con 0x7fe1180c9f40 2023-06-09T11:48:08.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.238+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe118159980 con 0x7fe1180c9f40 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.239+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe118159f40 con 0x7fe1180c9f40 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.240+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe108003a20 con 0x7fe1180c9f40 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.241+0000 7fe114ff9700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.241+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7fe108088f40 con 0x7fe1180c9f40 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.242+0000 7fe1177fe700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.242+0000 7fe1177fe700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c40 tx=0x7fe10c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:08.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.246+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe108053770 con 0x7fe1180c9f40 2023-06-09T11:48:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.500+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe11815a500 con 0x7fe1180c9f40 2023-06-09T11:48:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.500+0000 7fe114ff9700 1 -- 172.21.15.7:0/3870851284 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 179 v179) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fe11815a500 con 0x7fe1180c9f40 2023-06-09T11:48:08.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:08.504 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":179,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":179,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:02.839208+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:resolve","state_seq":12,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 msgr2=0x7fe10406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009c40 tx=0x7fe10c008030 comp rx=0 tx=0).stop 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 msgr2=0x7fe118151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c4e0 tx=0x7fe10800c7f0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 shutdown_connections 2023-06-09T11:48:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe11815b4a0 0x7fe11814eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1180c9f40 0x7fe118151130 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe10406ace0 0x7fe10406d190 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 --2- 172.21.15.7:0/3870851284 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe118153880 0x7fe118149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 >> 172.21.15.7:0/3870851284 conn(0x7fe1180737f0 msgr2=0x7fe11815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 shutdown_connections 2023-06-09T11:48:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:08.503+0000 7fe11e3ff700 1 -- 172.21.15.7:0/3870851284 wait complete. 2023-06-09T11:48:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 179 2023-06-09T11:48:08.525 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 179, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 179, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7279": { "addr": "172.21.15.117:6833/3302752268", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 3302752268, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 3302752268, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7279, "incarnation": 169, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 24 }, "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:resolve", "state_seq": 12 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 70, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:48:02.839208+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 7279 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:48:10.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.733+0000 7f249adff700 1 Processor -- start 2023-06-09T11:48:10.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.733+0000 7f249adff700 1 -- start start 2023-06-09T11:48:10.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.734+0000 7f249adff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249adff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2494159b20 con 0x7f24940cb9a0 2023-06-09T11:48:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249adff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24940cac80 con 0x7f24940cb5d0 2023-06-09T11:48:10.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24941597d0 con 0x7f249415cd40 2023-06-09T11:48:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249939c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f2498b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249939c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:35530/0 (socket says 172.21.15.7:35530) 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f2498b9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:44418/0 (socket says 172.21.15.7:44418) 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249939c700 1 -- 172.21.15.7:0/724418457 learned_addr learned my addr 172.21.15.7:0/724418457 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249939c700 1 -- 172.21.15.7:0/724418457 >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 msgr2=0x7f24940b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f24937fe700 1 -- 172.21.15.7:0/724418457 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696162511 0 0) 0x7f2494159b20 con 0x7f24940cb9a0 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.735+0000 7f249939c700 1 --2- 172.21.15.7:0/724418457 >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f249939c700 1 -- 172.21.15.7:0/724418457 >> v1:172.21.15.117:6789/0 conn(0x7f24940cb9a0 legacy=0x7f2494159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:10.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f249939c700 1 -- 172.21.15.7:0/724418457 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494071ab0 con 0x7f249415cd40 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f2498b9b700 1 --2- 172.21.15.7:0/724418457 >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f249939c700 1 --2- 172.21.15.7:0/724418457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f248c009e50 tx=0x7f248c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e55889809a1f38a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f24937fe700 1 -- 172.21.15.7:0/724418457 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c00e070 con 0x7f249415cd40 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f24937fe700 1 -- 172.21.15.7:0/724418457 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c00b910 con 0x7f249415cd40 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.736+0000 7f24937fe700 1 -- 172.21.15.7:0/724418457 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c00bcc0 con 0x7f249415cd40 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 -- 172.21.15.7:0/724418457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 msgr2=0x7f24940cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:10.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 --2- 172.21.15.7:0/724418457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f248c009e50 tx=0x7f248c00b330 comp rx=0 tx=0).stop 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 -- 172.21.15.7:0/724418457 shutdown_connections 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 --2- 172.21.15.7:0/724418457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f249415cd40 0x7f24940cbea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 --2- 172.21.15.7:0/724418457 >> v2:172.21.15.117:3300/0 conn(0x7f24940cb5d0 0x7f24940b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 -- 172.21.15.7:0/724418457 >> 172.21.15.7:0/724418457 conn(0x7f2494070af0 msgr2=0x7f2494070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 -- 172.21.15.7:0/724418457 shutdown_connections 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.737+0000 7f249adff700 1 -- 172.21.15.7:0/724418457 wait complete. 2023-06-09T11:48:10.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.738+0000 7f249adff700 1 Processor -- start 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.739+0000 7f249adff700 1 -- start start 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.739+0000 7f249adff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 0x7f2494158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.739+0000 7f249adff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 0x7f24940c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249adff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 0x7f24941528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249adff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24941f2aa0 con 0x7f24940cb9a0 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249adff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24940c6440 con 0x7f249415cd40 2023-06-09T11:48:10.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249adff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2494153260 con 0x7f24940cb5d0 2023-06-09T11:48:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2493fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 0x7f24940c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 0x7f2494158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2493fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 0x7f24940c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59102/0 (socket says 172.21.15.7:59102) 2023-06-09T11:48:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2493fff700 1 -- 172.21.15.7:0/1453148957 learned_addr learned my addr 172.21.15.7:0/1453148957 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:10.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249939c700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 0x7f24941528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 -- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 msgr2=0x7f24941528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 0x7f24941528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:10.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 -- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 msgr2=0x7f24940c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 0x7f24940c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f2498b9b700 1 -- 172.21.15.7:0/1453148957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880094a0 con 0x7f24940cb5d0 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.740+0000 7f249939c700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 0x7f24941528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f2498b9b700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 0x7f2494158cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9b0 tx=0x7f248800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800ee50 con 0x7f24940cb5d0 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800c850 con 0x7f24940cb5d0 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c009b30 con 0x7f24940cb5d0 2023-06-09T11:48:10.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800cc00 con 0x7f24940cb5d0 2023-06-09T11:48:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.741+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249415d4f0 con 0x7f24940cb5d0 2023-06-09T11:48:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.742+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249415d210 con 0x7f24940cb5d0 2023-06-09T11:48:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.743+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2488013470 con 0x7f24940cb5d0 2023-06-09T11:48:10.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.744+0000 7f24917fa700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 0x7f247806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.744+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f2488089ff0 con 0x7f24940cb5d0 2023-06-09T11:48:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.745+0000 7f2493fff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 0x7f247806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:10.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.745+0000 7f2493fff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 0x7f247806d190 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f24940c2e30 tx=0x7f2480009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:10.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:10.749+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f24880bc2d0 con 0x7f24940cb5d0 2023-06-09T11:48:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.001+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f249415d7d0 con 0x7f24940cb5d0 2023-06-09T11:48:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.003+0000 7f24917fa700 1 -- 172.21.15.7:0/1453148957 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f2488017090 con 0x7f24940cb5d0 2023-06-09T11:48:11.006 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:11.006 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":180,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:08.642155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.004+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 msgr2=0x7f247806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.004+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 0x7f247806d190 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f24940c2e30 tx=0x7f2480009040 comp rx=0 tx=0).stop 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.004+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 msgr2=0x7f2494158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.004+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 0x7f2494158cb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f248800e9b0 tx=0x7f248800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 shutdown_connections 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f249415cd40 0x7f24941528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f247806ace0 0x7f247806d190 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24940cb5d0 0x7f2494158cb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 --2- 172.21.15.7:0/1453148957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24940cb9a0 0x7f24940c3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 >> 172.21.15.7:0/1453148957 conn(0x7f2494070af0 msgr2=0x7f2494156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 shutdown_connections 2023-06-09T11:48:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:11.005+0000 7f249adff700 1 -- 172.21.15.7:0/1453148957 wait complete. 2023-06-09T11:48:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 180 2023-06-09T11:48:11.027 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:13.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:13.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.234+0000 7f60c71ff700 1 Processor -- start 2023-06-09T11:48:13.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.235+0000 7f60c71ff700 1 -- start start 2023-06-09T11:48:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c71ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60c0151510 0x7f60c00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c71ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c71ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c01580a0 con 0x7f60c0071060 2023-06-09T11:48:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c71ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c00714e0 con 0x7f60c0151510 2023-06-09T11:48:13.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c71ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01518e0 con 0x7f60c015b380 2023-06-09T11:48:13.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c4f9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60c0151510 0x7f60c00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60bffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60bffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:49586/0 (socket says 172.21.15.7:49586) 2023-06-09T11:48:13.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c4f9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f60c0151510 0x7f60c00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:41236/0 (socket says 172.21.15.7:41236) 2023-06-09T11:48:13.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60bffff700 1 -- 172.21.15.7:0/2221567143 learned_addr learned my addr 172.21.15.7:0/2221567143 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.236+0000 7f60c579c700 1 --1- 172.21.15.7:0/2221567143 >> v1:172.21.15.117:6789/0 conn(0x7f60c0071060 0x7f60c0158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:48886/0 (socket says 172.21.15.7:48886) 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60c4f9b700 1 -- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 msgr2=0x7f60c0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60c4f9b700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60c4f9b700 1 -- 172.21.15.7:0/2221567143 >> v1:172.21.15.117:6789/0 conn(0x7f60c0071060 legacy=0x7f60c0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60c4f9b700 1 -- 172.21.15.7:0/2221567143 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00c6180 con 0x7f60c0151510 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60bffff700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:48:13.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60bf7fe700 1 -- 172.21.15.7:0/2221567143 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063035044 0 0) 0x7f60c01580a0 con 0x7f60c0071060 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60c4f9b700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c0151510 0x7f60c00c5b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009840 tx=0x7f60b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66619bfa8b09f72c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.237+0000 7f60bf7fe700 1 -- 172.21.15.7:0/2221567143 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4011830 con 0x7f60c0151510 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60bf7fe700 1 -- 172.21.15.7:0/2221567143 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b4011990 con 0x7f60c0151510 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60bf7fe700 1 -- 172.21.15.7:0/2221567143 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4011d20 con 0x7f60c0151510 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 -- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c0151510 msgr2=0x7f60c00c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c0151510 0x7f60c00c5b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f60b4009840 tx=0x7f60b4009b50 comp rx=0 tx=0).stop 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 -- 172.21.15.7:0/2221567143 shutdown_connections 2023-06-09T11:48:13.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c0151510 0x7f60c00c5b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 --2- 172.21.15.7:0/2221567143 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c015b380 0x7f60c0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 -- 172.21.15.7:0/2221567143 >> 172.21.15.7:0/2221567143 conn(0x7f60c006a4d0 msgr2=0x7f60c006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 -- 172.21.15.7:0/2221567143 shutdown_connections 2023-06-09T11:48:13.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.238+0000 7f60c71ff700 1 -- 172.21.15.7:0/2221567143 wait complete. 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.240+0000 7f60c71ff700 1 Processor -- start 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.240+0000 7f60c71ff700 1 -- start start 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c0071060 0x7f60c00c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c015b380 0x7f60c0155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01551f0 con 0x7f60c0151510 2023-06-09T11:48:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c01557b0 con 0x7f60c015b380 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c71ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c014b430 con 0x7f60c0071060 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34528/0 (socket says 172.21.15.7:34528) 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60bffff700 1 -- 172.21.15.7:0/1095746942 learned_addr learned my addr 172.21.15.7:0/1095746942 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c4f9b700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c0071060 0x7f60c00c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.241+0000 7f60c579c700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c015b380 0x7f60c0155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 -- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c0071060 msgr2=0x7f60c00c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c0071060 0x7f60c00c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 -- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c015b380 msgr2=0x7f60c0155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c015b380 0x7f60c0155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 -- 172.21.15.7:0/1095746942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009550 con 0x7f60c0151510 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bffff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00e9e0 tx=0x7f60ac00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00ee80 con 0x7f60c0151510 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac00c850 con 0x7f60c0151510 2023-06-09T11:48:13.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00cbc0 con 0x7f60c0151510 2023-06-09T11:48:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b40094a0 con 0x7f60c0151510 2023-06-09T11:48:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.242+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c0157770 con 0x7f60c0151510 2023-06-09T11:48:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.244+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0157490 con 0x7f60c0151510 2023-06-09T11:48:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.244+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f60ac00cd20 con 0x7f60c0151510 2023-06-09T11:48:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.246+0000 7f60bd7fa700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.246+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f60ac089fe0 con 0x7f60c0151510 2023-06-09T11:48:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.246+0000 7f60c4f9b700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.246+0000 7f60c4f9b700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 0x7f60a406d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60b40090b0 tx=0x7f60b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:13.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.249+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f60ac054870 con 0x7f60c0151510 2023-06-09T11:48:13.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.499+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60c0157a50 con 0x7f60c0151510 2023-06-09T11:48:13.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.500+0000 7f60bd7fa700 1 -- 172.21.15.7:0/1095746942 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 180 v180) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f60ac021090 con 0x7f60c0151510 2023-06-09T11:48:13.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:13.504 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":180,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":180,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:08.642155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:reconnect","state_seq":14,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 msgr2=0x7f60a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 0x7f60a406d190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f60b40090b0 tx=0x7f60b4008030 comp rx=0 tx=0).stop 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 msgr2=0x7f60c014aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00e9e0 tx=0x7f60ac00ecf0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 shutdown_connections 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f60c015b380 0x7f60c0155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f60a406ace0 0x7f60a406d190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f60c0071060 0x7f60c00c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 --2- 172.21.15.7:0/1095746942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f60c0151510 0x7f60c014aef0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.502+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 >> 172.21.15.7:0/1095746942 conn(0x7f60c006a4d0 msgr2=0x7f60c014d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.503+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 shutdown_connections 2023-06-09T11:48:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:13.503+0000 7f60c71ff700 1 -- 172.21.15.7:0/1095746942 wait complete. 2023-06-09T11:48:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 180 2023-06-09T11:48:13.524 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:15.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:15.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.733+0000 7f3ad69ff700 1 Processor -- start 2023-06-09T11:48:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.733+0000 7f3ad69ff700 1 -- start start 2023-06-09T11:48:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.734+0000 7f3ad69ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ad0153880 0x7f3ad015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.734+0000 7f3ad69ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3ad00c9f40 0x7f3ad015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.734+0000 7f3ad69ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad015a500 con 0x7f3ad015b2d0 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.734+0000 7f3ad69ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad00c5f10 con 0x7f3ad0153880 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.734+0000 7f3ad69ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ad015da00 con 0x7f3ad00c9f40 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ad0153880 0x7f3ad015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ad0153880 0x7f3ad015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34540/0 (socket says 172.21.15.7:34540) 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3ad4f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ad015b2d0 0x7f3ad00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40682/0 (socket says 172.21.15.7:40682) 2023-06-09T11:48:15.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 -- 172.21.15.7:0/140419246 learned_addr learned my addr 172.21.15.7:0/140419246 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acf7fe700 1 --2- 172.21.15.7:0/140419246 >> v2:172.21.15.117:3300/0 conn(0x7f3ad00c9f40 0x7f3ad015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 -- 172.21.15.7:0/140419246 >> v2:172.21.15.117:3300/0 conn(0x7f3ad00c9f40 msgr2=0x7f3ad015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 --2- 172.21.15.7:0/140419246 >> v2:172.21.15.117:3300/0 conn(0x7f3ad00c9f40 0x7f3ad015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 -- 172.21.15.7:0/140419246 >> v1:172.21.15.7:6789/0 conn(0x7f3ad015b2d0 legacy=0x7f3ad00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 -- 172.21.15.7:0/140419246 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad01596e0 con 0x7f3ad0153880 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3aceffd700 1 -- 172.21.15.7:0/140419246 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266400087 0 0) 0x7f3ac8001490 con 0x7f3ad015b2d0 2023-06-09T11:48:15.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.735+0000 7f3acffff700 1 --2- 172.21.15.7:0/140419246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0153880 0x7f3ad015d4c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00cce30 tx=0x7f3ac0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a0283732f6c15 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3aceffd700 1 -- 172.21.15.7:0/140419246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000b870 con 0x7f3ad0153880 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3aceffd700 1 -- 172.21.15.7:0/140419246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac000b9d0 con 0x7f3ad0153880 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3aceffd700 1 -- 172.21.15.7:0/140419246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac000bd60 con 0x7f3ad0153880 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 -- 172.21.15.7:0/140419246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0153880 msgr2=0x7f3ad015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/140419246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0153880 0x7f3ad015d4c0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00cce30 tx=0x7f3ac0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 -- 172.21.15.7:0/140419246 shutdown_connections 2023-06-09T11:48:15.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/140419246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0153880 0x7f3ad015d4c0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/140419246 >> v2:172.21.15.117:3300/0 conn(0x7f3ad00c9f40 0x7f3ad015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 -- 172.21.15.7:0/140419246 >> 172.21.15.7:0/140419246 conn(0x7f3ad00737f0 msgr2=0x7f3ad00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 -- 172.21.15.7:0/140419246 shutdown_connections 2023-06-09T11:48:15.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.736+0000 7f3ad69ff700 1 -- 172.21.15.7:0/140419246 wait complete. 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.738+0000 7f3ad69ff700 1 Processor -- start 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.738+0000 7f3ad69ff700 1 -- start start 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad0153880 0x7f3ad01efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 0x7f3ad006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad006b360 con 0x7f3ad00c9f40 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01ef710 con 0x7f3ad015b2d0 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3ad69ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad01ef0e0 con 0x7f3ad0153880 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3acffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3acffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34556/0 (socket says 172.21.15.7:34556) 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.739+0000 7f3acffff700 1 -- 172.21.15.7:0/511428720 learned_addr learned my addr 172.21.15.7:0/511428720 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3ad4f9c700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 0x7f3ad006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acf7fe700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad0153880 0x7f3ad01efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 -- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad0153880 msgr2=0x7f3ad01efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad0153880 0x7f3ad01efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 -- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 msgr2=0x7f3ad006a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 0x7f3ad006a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 -- 172.21.15.7:0/511428720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac0009550 con 0x7f3ad00c9f40 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3ad4f9c700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 0x7f3ad006a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3acffff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009c10 tx=0x7f3ac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:15.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0004310 con 0x7f3ad00c9f40 2023-06-09T11:48:15.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac001f070 con 0x7f3ad00c9f40 2023-06-09T11:48:15.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00036a0 con 0x7f3ad00c9f40 2023-06-09T11:48:15.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.740+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad0159c60 con 0x7f3ad00c9f40 2023-06-09T11:48:15.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.741+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad0159980 con 0x7f3ad00c9f40 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.742+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0159f40 con 0x7f3ad00c9f40 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.743+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ac0021030 con 0x7f3ad00c9f40 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.743+0000 7f3accff9700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 0x7f3abc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.744+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f3ac008af00 con 0x7f3ad00c9f40 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.744+0000 7f3acf7fe700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 0x7f3abc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:15.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.744+0000 7f3acf7fe700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 0x7f3abc06d1c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3ad015cc10 tx=0x7f3ac4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:15.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.748+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3ac0055850 con 0x7f3ad00c9f40 2023-06-09T11:48:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.997+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ad0074cf0 con 0x7f3ad00c9f40 2023-06-09T11:48:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:15.998+0000 7f3accff9700 1 -- 172.21.15.7:0/511428720 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f3ac0015090 con 0x7f3ad00c9f40 2023-06-09T11:48:16.002 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:16.002 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":181,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:13.801744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 msgr2=0x7f3abc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 0x7f3abc06d1c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3ad015cc10 tx=0x7f3ac4008040 comp rx=0 tx=0).stop 2023-06-09T11:48:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 msgr2=0x7f3ad00cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009c10 tx=0x7f3ac0004750 comp rx=0 tx=0).stop 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 shutdown_connections 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad015b2d0 0x7f3ad006a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad00c9f40 0x7f3ad00cb860 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3abc06ad10 0x7f3abc06d1c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 --2- 172.21.15.7:0/511428720 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ad0153880 0x7f3ad01efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 >> 172.21.15.7:0/511428720 conn(0x7f3ad00737f0 msgr2=0x7f3ad015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 shutdown_connections 2023-06-09T11:48:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:16.000+0000 7f3ad69ff700 1 -- 172.21.15.7:0/511428720 wait complete. 2023-06-09T11:48:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 181 2023-06-09T11:48:16.022 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:18.023 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:18.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.229+0000 7f9df5dff700 1 Processor -- start 2023-06-09T11:48:18.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.229+0000 7f9df5dff700 1 -- start start 2023-06-09T11:48:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9df5dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9df015b4a0 0x7f9df015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9df5dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df015a500 con 0x7f9df00c9f40 2023-06-09T11:48:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9df5dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df015da00 con 0x7f9df0153880 2023-06-09T11:48:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9df5dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df00c5f10 con 0x7f9df015b4a0 2023-06-09T11:48:18.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9deeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9df015b4a0 0x7f9df015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:18.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9deeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9df015b4a0 0x7f9df015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34570/0 (socket says 172.21.15.7:34570) 2023-06-09T11:48:18.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.230+0000 7f9deeffd700 1 -- 172.21.15.7:0/2407162735 learned_addr learned my addr 172.21.15.7:0/2407162735 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9deeffd700 1 -- 172.21.15.7:0/2407162735 >> v1:172.21.15.117:6789/0 conn(0x7f9df0153880 legacy=0x7f9df015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9deeffd700 1 -- 172.21.15.7:0/2407162735 >> v1:172.21.15.7:6789/0 conn(0x7f9df00c9f40 legacy=0x7f9df00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9deeffd700 1 -- 172.21.15.7:0/2407162735 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df01596e0 con 0x7f9df015b4a0 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9deeffd700 1 --2- 172.21.15.7:0/2407162735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df015b4a0 0x7f9df015d4c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098f0 tx=0x7f9de4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa1a292faebf23e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9dee7fc700 1 -- 172.21.15.7:0/2407162735 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400b8a0 con 0x7f9df015b4a0 2023-06-09T11:48:18.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9dee7fc700 1 -- 172.21.15.7:0/2407162735 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de400ba00 con 0x7f9df015b4a0 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.231+0000 7f9dee7fc700 1 -- 172.21.15.7:0/2407162735 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de400bd90 con 0x7f9df015b4a0 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 -- 172.21.15.7:0/2407162735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df015b4a0 msgr2=0x7f9df015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 --2- 172.21.15.7:0/2407162735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df015b4a0 0x7f9df015d4c0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f9de40098f0 tx=0x7f9de4009c00 comp rx=0 tx=0).stop 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 -- 172.21.15.7:0/2407162735 shutdown_connections 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 --2- 172.21.15.7:0/2407162735 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df015b4a0 0x7f9df015d4c0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 -- 172.21.15.7:0/2407162735 >> 172.21.15.7:0/2407162735 conn(0x7f9df00737f0 msgr2=0x7f9df00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:18.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 -- 172.21.15.7:0/2407162735 shutdown_connections 2023-06-09T11:48:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.232+0000 7f9df5dff700 1 -- 172.21.15.7:0/2407162735 wait complete. 2023-06-09T11:48:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.233+0000 7f9df5dff700 1 Processor -- start 2023-06-09T11:48:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.234+0000 7f9df5dff700 1 -- start start 2023-06-09T11:48:18.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.234+0000 7f9df5dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 0x7f9df01e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.234+0000 7f9df5dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9df5dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9df015b4a0 0x7f9df0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9df5dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00cd3a0 con 0x7f9df0153880 2023-06-09T11:48:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9df5dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00cc060 con 0x7f9df00c9f40 2023-06-09T11:48:18.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9df5dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df00cf100 con 0x7f9df015b4a0 2023-06-09T11:48:18.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34580/0 (socket says 172.21.15.7:34580) 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 -- 172.21.15.7:0/3626533852 learned_addr learned my addr 172.21.15.7:0/3626533852 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9def7fe700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 0x7f9df01e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deffff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9df015b4a0 0x7f9df0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 -- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9df015b4a0 msgr2=0x7f9df0158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9df015b4a0 0x7f9df0158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 -- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 msgr2=0x7f9df01e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 0x7f9df01e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 -- 172.21.15.7:0/3626533852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de4009550 con 0x7f9df0153880 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9def7fe700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 0x7f9df01e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.235+0000 7f9deeffd700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9de4014040 tx=0x7f9de4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.236+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40038a0 con 0x7f9df0153880 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.236+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de401e070 con 0x7f9df0153880 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.236+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4003c60 con 0x7f9df0153880 2023-06-09T11:48:18.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.236+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df0159c60 con 0x7f9df0153880 2023-06-09T11:48:18.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.236+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df0159980 con 0x7f9df0153880 2023-06-09T11:48:18.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.237+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df0159f40 con 0x7f9df0153880 2023-06-09T11:48:18.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.238+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9de4011420 con 0x7f9df0153880 2023-06-09T11:48:18.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.239+0000 7f9dd7fff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:18.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.240+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f9de4020070 con 0x7f9df0153880 2023-06-09T11:48:18.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.240+0000 7f9def7fe700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:18.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.240+0000 7f9def7fe700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 0x7f9ddc06d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.244+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9de4055b10 con 0x7f9df0153880 2023-06-09T11:48:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.501+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9df00cdc00 con 0x7f9df0153880 2023-06-09T11:48:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.501+0000 7f9dd7fff700 1 -- 172.21.15.7:0/3626533852 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 181 v181) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f9de4015090 con 0x7f9df0153880 2023-06-09T11:48:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:18.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":181,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":181,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:13.801744+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 msgr2=0x7f9ddc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 0x7f9ddc06d190 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9de0000f60 tx=0x7f9de0009040 comp rx=0 tx=0).stop 2023-06-09T11:48:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 msgr2=0x7f9df00b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9de4014040 tx=0x7f9de4004750 comp rx=0 tx=0).stop 2023-06-09T11:48:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 shutdown_connections 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ddc06ace0 0x7f9ddc06d190 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9df015b4a0 0x7f9df0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9df00c9f40 0x7f9df01e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 --2- 172.21.15.7:0/3626533852 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9df0153880 0x7f9df00b42a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 >> 172.21.15.7:0/3626533852 conn(0x7f9df00737f0 msgr2=0x7f9df014f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 shutdown_connections 2023-06-09T11:48:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:18.504+0000 7f9df5dff700 1 -- 172.21.15.7:0/3626533852 wait complete. 2023-06-09T11:48:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 181 2023-06-09T11:48:18.526 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:48:20.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.734+0000 7f0fc0bff700 1 Processor -- start 2023-06-09T11:48:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.735+0000 7f0fc0bff700 1 -- start start 2023-06-09T11:48:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fc0bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fc0bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc0c0bf0 con 0x7f0fbc0c0140 2023-06-09T11:48:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fc0bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fbc0702f0 con 0x7f0fbc0c1f90 2023-06-09T11:48:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fc0bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fbc070110 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fba59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fba59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34582/0 (socket says 172.21.15.7:34582) 2023-06-09T11:48:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fb3fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0fbc0c0140 0x7f0fbc1501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40696/0 (socket says 172.21.15.7:40696) 2023-06-09T11:48:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.736+0000 7f0fba59c700 1 -- 172.21.15.7:0/1738718163 learned_addr learned my addr 172.21.15.7:0/1738718163 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fba59c700 1 -- 172.21.15.7:0/1738718163 >> v1:172.21.15.117:6789/0 conn(0x7f0fbc0c1f90 legacy=0x7f0fbc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fba59c700 1 -- 172.21.15.7:0/1738718163 >> v1:172.21.15.7:6789/0 conn(0x7f0fbc0c0140 legacy=0x7f0fbc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fba59c700 1 -- 172.21.15.7:0/1738718163 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fbc0c0e90 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fb9d9b700 1 -- 172.21.15.7:0/1738718163 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907185061 0 0) 0x7f0fbc0c0bf0 con 0x7f0fbc0c0140 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fba59c700 1 --2- 172.21.15.7:0/1738718163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0098f0 tx=0x7f0fac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cdd5ca99e8189a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fb9d9b700 1 -- 172.21.15.7:0/1738718163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac004230 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fb9d9b700 1 -- 172.21.15.7:0/1738718163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fac004390 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.737+0000 7f0fb9d9b700 1 -- 172.21.15.7:0/1738718163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fac0036a0 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 -- 172.21.15.7:0/1738718163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 msgr2=0x7f0fbc1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/1738718163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0098f0 tx=0x7f0fac009c00 comp rx=0 tx=0).stop 2023-06-09T11:48:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 -- 172.21.15.7:0/1738718163 shutdown_connections 2023-06-09T11:48:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/1738718163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc1599b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 -- 172.21.15.7:0/1738718163 >> 172.21.15.7:0/1738718163 conn(0x7f0fbc06a4d0 msgr2=0x7f0fbc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 -- 172.21.15.7:0/1738718163 shutdown_connections 2023-06-09T11:48:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.738+0000 7f0fc0bff700 1 -- 172.21.15.7:0/1738718163 wait complete. 2023-06-09T11:48:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.740+0000 7f0fc0bff700 1 Processor -- start 2023-06-09T11:48:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.740+0000 7f0fc0bff700 1 -- start start 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fbc0c0140 0x7f0fbc155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fbc0c1f90 0x7f0fbc145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1f0210 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1f0490 con 0x7f0fbc0c0140 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fc0bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fbc1f0770 con 0x7f0fbc0c1f90 2023-06-09T11:48:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34594/0 (socket says 172.21.15.7:34594) 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 -- 172.21.15.7:0/2633543740 learned_addr learned my addr 172.21.15.7:0/2633543740 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fba59c700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fbc0c0140 0x7f0fbc155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fbad9d700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fbc0c1f90 0x7f0fbc145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 -- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fbc0c1f90 msgr2=0x7f0fbc145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fbc0c1f90 0x7f0fbc145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 -- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fbc0c0140 msgr2=0x7f0fbc155900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fbc0c0140 0x7f0fbc155900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.741+0000 7f0fb3fff700 1 -- 172.21.15.7:0/2633543740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac009550 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fb3fff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400c520 tx=0x7f0fa400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4004c80 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa4004e00 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4007310 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc0c1450 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.742+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fbc0c1170 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.743+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fbc0705d0 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.744+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fa4007470 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.745+0000 7f0fb2ffd700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 0x7f0f9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.745+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f0fa40897c0 con 0x7f0fbc0c1bc0 2023-06-09T11:48:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.745+0000 7f0fba59c700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 0x7f0f9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.746+0000 7f0fba59c700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 0x7f0f9c06d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009130 tx=0x7f0fac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:20.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:20.750+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0fa4054120 con 0x7f0fbc0c1bc0 2023-06-09T11:48:21.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.005+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fbc074cf0 con 0x7f0fbc0c1bc0 2023-06-09T11:48:21.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.005+0000 7f0fb2ffd700 1 -- 172.21.15.7:0/2633543740 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f0fa4057740 con 0x7f0fbc0c1bc0 2023-06-09T11:48:21.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:21.010 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:19.824467+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:21.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 msgr2=0x7f0f9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:21.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 0x7f0f9c06d190 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009130 tx=0x7f0fac008030 comp rx=0 tx=0).stop 2023-06-09T11:48:21.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 msgr2=0x7f0fbc144c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:21.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400c520 tx=0x7f0fa400c830 comp rx=0 tx=0).stop 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 shutdown_connections 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f9c06ace0 0x7f0f9c06d190 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fbc0c1f90 0x7f0fbc145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fbc0c0140 0x7f0fbc155900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 --2- 172.21.15.7:0/2633543740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fbc0c1bc0 0x7f0fbc144c00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 >> 172.21.15.7:0/2633543740 conn(0x7f0fbc06a4d0 msgr2=0x7f0fbc0c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 shutdown_connections 2023-06-09T11:48:21.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:21.008+0000 7f0fc0bff700 1 -- 172.21.15.7:0/2633543740 wait complete. 2023-06-09T11:48:21.014 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 182 2023-06-09T11:48:21.030 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d has gained rank=1, replacing gid=7141 2023-06-09T11:48:21.030 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 1 secs before reviving mds.f 2023-06-09T11:48:21.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:21.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:22.031 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:48:22.031 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:48:22.032 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:48:22.036 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:48:22.037 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:22.185 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:48:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.244+0000 7f41975ff700 1 Processor -- start 2023-06-09T11:48:22.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.244+0000 7f41975ff700 1 -- start start 2023-06-09T11:48:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f41975ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 0x7f41900b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f41975ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4190159b20 con 0x7f41900cb5d0 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f41975ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41900cac80 con 0x7f41900cb9a0 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f41975ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41901597d0 con 0x7f41900cc5b0 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f419539b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f41900cb5d0 0x7f4190159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40700/0 (socket says 172.21.15.7:40700) 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f419539b700 1 -- 172.21.15.7:0/834162796 learned_addr learned my addr 172.21.15.7:0/834162796 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.245+0000 7f4195b9c700 1 --2- 172.21.15.7:0/834162796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 0x7f41900b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796811455 0 0) 0x7f4190159b20 con 0x7f41900cb5d0 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f417c0035b0 con 0x7f41900cb5d0 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944025421 0 0) 0x7f41900cac80 con 0x7f41900cb9a0 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4190159b20 con 0x7f41900cb9a0 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4195b9c700 1 -- 172.21.15.7:0/834162796 >> v1:172.21.15.117:6789/0 conn(0x7f41900cb9a0 legacy=0x7f41900cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4195b9c700 1 -- 172.21.15.7:0/834162796 >> v1:172.21.15.7:6789/0 conn(0x7f41900cb5d0 legacy=0x7f4190159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4195b9c700 1 -- 172.21.15.7:0/834162796 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190071ab0 con 0x7f41900cc5b0 2023-06-09T11:48:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4195b9c700 1 --2- 172.21.15.7:0/834162796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 0x7f41900b52b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4188009dc0 tx=0x7f418800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4031abde794f909d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800e070 con 0x7f41900cc5b0 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418800bb20 con 0x7f41900cc5b0 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.246+0000 7f4187fff700 1 -- 172.21.15.7:0/834162796 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188012430 con 0x7f41900cc5b0 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 -- 172.21.15.7:0/834162796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 msgr2=0x7f41900b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 --2- 172.21.15.7:0/834162796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 0x7f41900b52b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4188009dc0 tx=0x7f418800b330 comp rx=0 tx=0).stop 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 -- 172.21.15.7:0/834162796 shutdown_connections 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 --2- 172.21.15.7:0/834162796 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cc5b0 0x7f41900b52b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.253 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 -- 172.21.15.7:0/834162796 >> 172.21.15.7:0/834162796 conn(0x7f4190070af0 msgr2=0x7f4190070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:22.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 -- 172.21.15.7:0/834162796 shutdown_connections 2023-06-09T11:48:22.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.247+0000 7f41975ff700 1 -- 172.21.15.7:0/834162796 wait complete. 2023-06-09T11:48:22.254 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.249+0000 7f41975ff700 1 Processor -- start 2023-06-09T11:48:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.249+0000 7f41975ff700 1 -- start start 2023-06-09T11:48:22.255 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41900cb5d0 0x7f41901f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cb9a0 0x7f41901ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4190153ec0 con 0x7f41900cc5b0 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41901ef350 con 0x7f41900cb5d0 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f41975ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4190151770 con 0x7f41900cb9a0 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f4195b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f4195b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34610/0 (socket says 172.21.15.7:34610) 2023-06-09T11:48:22.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f4195b9c700 1 -- 172.21.15.7:0/3652993907 learned_addr learned my addr 172.21.15.7:0/3652993907 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f419539b700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41900cb5d0 0x7f41901f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.250+0000 7f4194b9a700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cb9a0 0x7f41901ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 -- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cb9a0 msgr2=0x7f41901ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cb9a0 0x7f41901ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 -- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41900cb5d0 msgr2=0x7f41901f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41900cb5d0 0x7f41901f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 -- 172.21.15.7:0/3652993907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4188009a70 con 0x7f41900cc5b0 2023-06-09T11:48:22.257 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4195b9c700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4188014040 tx=0x7f4188004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188012430 con 0x7f41900cc5b0 2023-06-09T11:48:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4188020070 con 0x7f41900cc5b0 2023-06-09T11:48:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418801ca50 con 0x7f41900cc5b0 2023-06-09T11:48:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419015d4f0 con 0x7f41900cc5b0 2023-06-09T11:48:22.258 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.251+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419015d210 con 0x7f41900cc5b0 2023-06-09T11:48:22.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.252+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419015d7d0 con 0x7f41900cc5b0 2023-06-09T11:48:22.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.253+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f418801cbb0 con 0x7f41900cc5b0 2023-06-09T11:48:22.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.255+0000 7f4185ffb700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 0x7f417406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:22.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.255+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f4188022070 con 0x7f41900cc5b0 2023-06-09T11:48:22.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.255+0000 7f419539b700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 0x7f417406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:22.260 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.255+0000 7f419539b700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 0x7f417406d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f418c007fd0 tx=0x7f418c007e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:22.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.259+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4188056410 con 0x7f41900cc5b0 2023-06-09T11:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.513+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41901ef4c0 con 0x7f41900cc5b0 2023-06-09T11:48:22.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.513+0000 7f4185ffb700 1 -- 172.21.15.7:0/3652993907 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 182 v182) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f418802c070 con 0x7f41900cc5b0 2023-06-09T11:48:22.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:22.518 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":182,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:19.824467+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 msgr2=0x7f417406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 0x7f417406d190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f418c007fd0 tx=0x7f418c007e00 comp rx=0 tx=0).stop 2023-06-09T11:48:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 msgr2=0x7f41900c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4188014040 tx=0x7f4188004ad0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 shutdown_connections 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41900cc5b0 0x7f41900c5c30 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f41900cb5d0 0x7f41901f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f417406ace0 0x7f417406d190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 --2- 172.21.15.7:0/3652993907 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f41900cb9a0 0x7f41901ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 >> 172.21.15.7:0/3652993907 conn(0x7f4190070af0 msgr2=0x7f419015c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 shutdown_connections 2023-06-09T11:48:22.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:22.516+0000 7f41975ff700 1 -- 172.21.15.7:0/3652993907 wait complete. 2023-06-09T11:48:22.522 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 182 2023-06-09T11:48:22.538 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-06-09T11:48:24.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:48:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.747+0000 7f20c49ff700 1 Processor -- start 2023-06-09T11:48:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.747+0000 7f20c49ff700 1 -- start start 2023-06-09T11:48:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20c49ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f20c015b4a0 0x7f20c015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20c49ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f20c00c9f40 0x7f20c00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20c49ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c015a500 con 0x7f20c0153880 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20c49ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c00c5f10 con 0x7f20c00c9f40 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20c49ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c015da00 con 0x7f20c015b4a0 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20be59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20c0153880 0x7f20c015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43592/0 (socket says 172.21.15.7:43592) 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.748+0000 7f20be59c700 1 -- 172.21.15.7:0/4161344461 learned_addr learned my addr 172.21.15.7:0/4161344461 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:48:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 --2- 172.21.15.7:0/4161344461 >> v2:172.21.15.7:3300/0 conn(0x7f20c00c9f40 0x7f20c00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20b7fff700 1 --2- 172.21.15.7:0/4161344461 >> v2:172.21.15.117:3300/0 conn(0x7f20c015b4a0 0x7f20c015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bdd9b700 1 -- 172.21.15.7:0/4161344461 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955849981 0 0) 0x7f20b0001110 con 0x7f20c0153880 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bdd9b700 1 -- 172.21.15.7:0/4161344461 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac0035b0 con 0x7f20c0153880 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 -- 172.21.15.7:0/4161344461 >> v2:172.21.15.117:3300/0 conn(0x7f20c015b4a0 msgr2=0x7f20c015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 --2- 172.21.15.7:0/4161344461 >> v2:172.21.15.117:3300/0 conn(0x7f20c015b4a0 0x7f20c015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:24.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 -- 172.21.15.7:0/4161344461 >> v1:172.21.15.7:6789/0 conn(0x7f20c0153880 legacy=0x7f20c015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 -- 172.21.15.7:0/4161344461 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c01596e0 con 0x7f20c00c9f40 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.749+0000 7f20bed9d700 1 --2- 172.21.15.7:0/4161344461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c00c9f40 0x7f20c00c4e30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f20c00cce30 tx=0x7f20b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3bb299c1a8e8ad server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20bdd9b700 1 -- 172.21.15.7:0/4161344461 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800e070 con 0x7f20c00c9f40 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20bdd9b700 1 -- 172.21.15.7:0/4161344461 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b800bb20 con 0x7f20c00c9f40 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20bdd9b700 1 -- 172.21.15.7:0/4161344461 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800be90 con 0x7f20c00c9f40 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 -- 172.21.15.7:0/4161344461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c00c9f40 msgr2=0x7f20c00c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 --2- 172.21.15.7:0/4161344461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c00c9f40 0x7f20c00c4e30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f20c00cce30 tx=0x7f20b800b330 comp rx=0 tx=0).stop 2023-06-09T11:48:24.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 -- 172.21.15.7:0/4161344461 shutdown_connections 2023-06-09T11:48:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 --2- 172.21.15.7:0/4161344461 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c00c9f40 0x7f20c00c4e30 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 --2- 172.21.15.7:0/4161344461 >> v2:172.21.15.117:3300/0 conn(0x7f20c015b4a0 0x7f20c015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 -- 172.21.15.7:0/4161344461 >> 172.21.15.7:0/4161344461 conn(0x7f20c00737f0 msgr2=0x7f20c00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 -- 172.21.15.7:0/4161344461 shutdown_connections 2023-06-09T11:48:24.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.750+0000 7f20c49ff700 1 -- 172.21.15.7:0/4161344461 wait complete. 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.752+0000 7f20c49ff700 1 Processor -- start 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.752+0000 7f20c49ff700 1 -- start start 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 0x7f20c01f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 0x7f20c01e5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0159370 con 0x7f20c0153880 2023-06-09T11:48:24.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01f28d0 con 0x7f20c00c9f40 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20c49ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c01f32a0 con 0x7f20c015b4a0 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20b7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40874/0 (socket says 172.21.15.7:40874) 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.753+0000 7f20b7fff700 1 -- 172.21.15.7:0/2472093019 learned_addr learned my addr 172.21.15.7:0/2472093019 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20bed9d700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 0x7f20c01e5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20be59c700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 0x7f20c01f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 -- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 msgr2=0x7f20c01e5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 0x7f20c01e5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 -- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 msgr2=0x7f20c01f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 0x7f20c01f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 -- 172.21.15.7:0/2472093019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b8009a70 con 0x7f20c0153880 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20bed9d700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 0x7f20c01e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20be59c700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 0x7f20c01f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b7fff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e870 tx=0x7f20a800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:24.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8009590 con 0x7f20c0153880 2023-06-09T11:48:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a800ee70 con 0x7f20c0153880 2023-06-09T11:48:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800ca50 con 0x7f20c0153880 2023-06-09T11:48:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c0159c60 con 0x7f20c0153880 2023-06-09T11:48:24.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.754+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c0159980 con 0x7f20c0153880 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.755+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c0159f40 con 0x7f20c0153880 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.756+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20a800cbb0 con 0x7f20c0153880 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.757+0000 7f20b6ffd700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.757+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f20a8089f20 con 0x7f20c0153880 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.758+0000 7f20be59c700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:48:24.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.758+0000 7f20be59c700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 0x7f20a006d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:48:24.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:24.762+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f20a8058300 con 0x7f20c0153880 2023-06-09T11:48:25.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.056+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20c00d04d0 con 0x7f20c0153880 2023-06-09T11:48:25.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.057+0000 7f20b6ffd700 1 -- 172.21.15.7:0/2472093019 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 183 v183) v1 ==== 95+0+6225 (secure 0 0 0) 0x7f20a8017090 con 0x7f20c0153880 2023-06-09T11:48:25.061 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:48:25.061 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":183,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":182,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:19.824467+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:48:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 msgr2=0x7f20a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 0x7f20a006d190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0009040 comp rx=0 tx=0).stop 2023-06-09T11:48:25.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 msgr2=0x7f20c00cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f20a800e870 tx=0x7f20a800ec30 comp rx=0 tx=0).stop 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 shutdown_connections 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20a006ace0 0x7f20a006d190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20c015b4a0 0x7f20c01e5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20c00c9f40 0x7f20c01f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 --2- 172.21.15.7:0/2472093019 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20c0153880 0x7f20c00cbda0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 >> 172.21.15.7:0/2472093019 conn(0x7f20c00737f0 msgr2=0x7f20c015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:48:25.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 shutdown_connections 2023-06-09T11:48:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:48:25.059+0000 7f20c49ff700 1 -- 172.21.15.7:0/2472093019 wait complete. 2023-06-09T11:48:25.065 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 183 2023-06-09T11:48:25.081 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:48:25.081 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 72 secs before thrashing 2023-06-09T11:48:26.173 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 12% (14/112) Updating files: 13% (15/112) Updating files: 14% (16/112) Updating files: 15% (17/112) Updating files: 16% (18/112) Updating files: 17% (20/112) Updating files: 18% (21/112) Updating files: 19% (22/112) Updating files: 20% (23/112) Updating files: 21% (24/112) Updating files: 22% (25/112) Updating files: 23% (26/112) Updating files: 24% (27/112) Updating files: 25% (28/112) Updating files: 26% (30/112) Updating files: 27% (31/112) Updating files: 28% (32/112) Updating files: 29% (33/112) Updating files: 30% (34/112) Updating files: 31% (35/112) Updating files: 32% (36/112) Updating files: 33% (37/112) Updating files: 34% (39/112) Updating files: 35% (40/112) Updating files: 36% (41/112) Updating files: 37% (42/112) Updating files: 38% (43/112) Updating files: 39% (44/112) Updating files: 40% (45/112) Updating files: 41% (46/112) Updating files: 41% (47/112) Updating files: 42% (48/112) Updating files: 43% (49/112) Updating files: 44% (50/112) Updating files: 45% (51/112) Updating files: 46% (52/112) Updating files: 47% (53/112) Updating files: 48% (54/112) Updating files: 49% (55/112) Updating files: 50% (56/112) Updating files: 51% (58/112) Updating files: 52% (59/112) Updating files: 53% (60/112) Updating files: 54% (61/112) Updating files: 55% (62/112) Updating files: 56% (63/112) Updating files: 57% (64/112) Updating files: 58% (65/112) Updating files: 59% (67/112) Updating files: 60% (68/112) Updating files: 61% (69/112) Updating files: 62% (70/112) Updating files: 63% (71/112) Updating files: 64% (72/112) Updating files: 65% (73/112) Updating files: 66% (74/112) Updating files: 67% (76/112) Updating files: 68% (77/112) Updating files: 69% (78/112) Updating files: 70% (79/112) Updating files: 71% (80/112) Updating files: 72% (81/112) Updating files: 73% (82/112) Updating files: 74% (83/112) Updating files: 75% (84/112) Updating files: 76% (86/112) Updating files: 77% (87/112) Updating files: 78% (88/112) Updating files: 79% (89/112) Updating files: 80% (90/112) Updating files: 81% (91/112) Updating files: 82% (92/112) Updating files: 83% (93/112) Updating files: 84% (95/112) Updating files: 85% (96/112) Updating files: 86% (97/112) Updating files: 87% (98/112) Updating files: 88% (99/112) Updating files: 89% (100/112) Updating files: 90% (101/112) Updating files: 91% (102/112) Updating files: 92% (104/112) Updating files: 93% (105/112) Updating files: 94% (106/112) Updating files: 95% (107/112) Updating files: 96% (108/112) Updating files: 97% (109/112) Updating files: 98% (110/112) Updating files: 99% (111/112) Updating files: 100% (112/112) Updating files: 100% (112/112), done. 2023-06-09T11:48:26.359 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at ab3c421eb95 v0.15 2023-06-09T11:48:26.364 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.15 2023-06-09T11:48:26.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:48:26.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 16 -eq 48 ']' 2023-06-09T11:48:26.391 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.16 2023-06-09T11:48:26.392 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.16 2023-06-09T11:48:26.392 INFO:tasks.workunit.client.0.smithi007.stdout:v0.16 2023-06-09T11:48:26.392 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.16 2023-06-09T11:48:40.391 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 18% (20/107) Updating files: 19% (21/107) Updating files: 20% (22/107) Updating files: 21% (23/107) Updating files: 22% (24/107) Updating files: 23% (25/107) Updating files: 24% (26/107) Updating files: 25% (27/107) Updating files: 26% (28/107) Updating files: 27% (29/107) Updating files: 28% (30/107) Updating files: 28% (31/107) Updating files: 29% (32/107) Updating files: 30% (33/107) Updating files: 31% (34/107) Updating files: 32% (35/107) Updating files: 33% (36/107) Updating files: 34% (37/107) Updating files: 35% (38/107) Updating files: 36% (39/107) Updating files: 37% (40/107) Updating files: 38% (41/107) Updating files: 39% (42/107) Updating files: 40% (43/107) Updating files: 41% (44/107) Updating files: 42% (45/107) Updating files: 43% (47/107) Updating files: 44% (48/107) Updating files: 45% (49/107) Updating files: 46% (50/107) Updating files: 47% (51/107) Updating files: 48% (52/107) Updating files: 49% (53/107) Updating files: 50% (54/107) Updating files: 51% (55/107) Updating files: 52% (56/107) Updating files: 53% (57/107) Updating files: 54% (58/107) Updating files: 55% (59/107) Updating files: 56% (60/107) Updating files: 57% (61/107) Updating files: 58% (63/107) Updating files: 59% (64/107) Updating files: 60% (65/107) Updating files: 61% (66/107) Updating files: 62% (67/107) Updating files: 63% (68/107) Updating files: 64% (69/107) Updating files: 65% (70/107) Updating files: 66% (71/107) Updating files: 67% (72/107) Updating files: 68% (73/107) Updating files: 69% (74/107) Updating files: 70% (75/107) Updating files: 71% (76/107) Updating files: 72% (78/107) Updating files: 73% (79/107) Updating files: 74% (80/107) Updating files: 75% (81/107) Updating files: 76% (82/107) Updating files: 77% (83/107) Updating files: 78% (84/107) Updating files: 79% (85/107) Updating files: 80% (86/107) Updating files: 81% (87/107) Updating files: 82% (88/107) Updating files: 83% (89/107) Updating files: 84% (90/107) Updating files: 85% (91/107) Updating files: 86% (93/107) Updating files: 87% (94/107) Updating files: 88% (95/107) Updating files: 89% (96/107) Updating files: 90% (97/107) Updating files: 91% (98/107) Updating files: 92% (99/107) Updating files: 93% (100/107) Updating files: 94% (101/107) Updating files: 95% (102/107) Updating files: 96% (103/107) Updating files: 97% (104/107) Updating files: 98% (105/107) Updating files: 99% (106/107) Updating files: 100% (107/107) Updating files: 100% (107/107), done. 2023-06-09T11:48:41.527 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 38dbaa5175c v0.16 2023-06-09T11:48:41.532 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.16 2023-06-09T11:48:41.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:48:41.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 17 -eq 48 ']' 2023-06-09T11:48:41.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.17 2023-06-09T11:48:41.583 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.17 2023-06-09T11:48:41.583 INFO:tasks.workunit.client.0.smithi007.stdout:v0.17 2023-06-09T11:48:41.584 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.17 2023-06-09T11:48:51.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:48:51.693 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:04.734 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 2% (6/217) Updating files: 3% (7/217) Updating files: 4% (9/217) Updating files: 5% (11/217) Updating files: 6% (14/217) Updating files: 7% (16/217) Updating files: 8% (18/217) Updating files: 9% (20/217) Updating files: 10% (22/217) Updating files: 11% (24/217) Updating files: 12% (27/217) Updating files: 13% (29/217) Updating files: 14% (31/217) Updating files: 15% (33/217) Updating files: 16% (35/217) Updating files: 17% (37/217) Updating files: 17% (38/217) Updating files: 18% (40/217) Updating files: 19% (42/217) Updating files: 20% (44/217) Updating files: 21% (46/217) Updating files: 22% (48/217) Updating files: 23% (50/217) Updating files: 23% (51/217) Updating files: 24% (53/217) Updating files: 25% (55/217) Updating files: 26% (57/217) Updating files: 27% (59/217) Updating files: 28% (61/217) Updating files: 29% (63/217) Updating files: 30% (66/217) Updating files: 31% (68/217) Updating files: 32% (70/217) Updating files: 33% (72/217) Updating files: 34% (74/217) Updating files: 35% (76/217) Updating files: 36% (79/217) Updating files: 37% (81/217) Updating files: 38% (83/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 46% (101/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 55% (121/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 76% (167/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2023-06-09T11:49:05.718 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at dac3a8c35fb v0.17 2023-06-09T11:49:05.723 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.17 2023-06-09T11:49:05.769 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:49:05.769 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 18 -eq 48 ']' 2023-06-09T11:49:05.769 INFO:tasks.workunit.client.0.smithi007.stdout:v0.18 2023-06-09T11:49:05.770 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.18 2023-06-09T11:49:05.770 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.18 2023-06-09T11:49:05.770 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.18 2023-06-09T11:49:21.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:21.805 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:35.619 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 14% (110/779) Updating files: 15% (117/779) Updating files: 16% (125/779) Updating files: 17% (133/779) Updating files: 18% (141/779) Updating files: 19% (149/779) Updating files: 19% (155/779) Updating files: 20% (156/779) Updating files: 21% (164/779) Updating files: 22% (172/779) Updating files: 23% (180/779) Updating files: 24% (187/779) Updating files: 25% (195/779) Updating files: 26% (203/779) Updating files: 27% (211/779) Updating files: 28% (219/779) Updating files: 29% (226/779) Updating files: 30% (234/779) Updating files: 31% (242/779) Updating files: 32% (250/779) Updating files: 33% (258/779) Updating files: 34% (265/779) Updating files: 35% (273/779) Updating files: 36% (281/779) Updating files: 37% (289/779) Updating files: 38% (297/779) Updating files: 39% (304/779) Updating files: 40% (312/779) Updating files: 41% (320/779) Updating files: 42% (328/779) Updating files: 43% (335/779) Updating files: 44% (343/779) Updating files: 45% (351/779) Updating files: 46% (359/779) Updating files: 47% (367/779) Updating files: 47% (368/779) Updating files: 48% (374/779) Updating files: 49% (382/779) Updating files: 50% (390/779) Updating files: 51% (398/779) Updating files: 51% (404/779) Updating files: 52% (406/779) Updating files: 53% (413/779) Updating files: 54% (421/779) Updating files: 55% (429/779) Updating files: 56% (437/779) Updating files: 57% (445/779) Updating files: 58% (452/779) Updating files: 59% (460/779) Updating files: 60% (468/779) Updating files: 61% (476/779) Updating files: 61% (481/779) Updating files: 62% (483/779) Updating files: 63% (491/779) Updating files: 64% (499/779) Updating files: 65% (507/779) Updating files: 66% (515/779) Updating files: 67% (522/779) Updating files: 68% (530/779) Updating files: 69% (538/779) Updating files: 69% (541/779) Updating files: 70% (546/779) Updating files: 70% (551/779) Updating files: 71% (554/779) Updating files: 72% (561/779) Updating files: 73% (569/779) Updating files: 74% (577/779) Updating files: 74% (584/779) Updating files: 75% (585/779) Updating files: 75% (588/779) Updating files: 76% (593/779) Updating files: 77% (600/779) Updating files: 77% (606/779) Updating files: 78% (608/779) Updating files: 79% (616/779) Updating files: 79% (621/779) Updating files: 80% (624/779) Updating files: 81% (631/779) Updating files: 81% (634/779) Updating files: 82% (639/779) Updating files: 83% (647/779) Updating files: 84% (655/779) Updating files: 85% (663/779) Updating files: 86% (670/779) Updating files: 87% (678/779) Updating files: 88% (686/779) Updating files: 88% (688/779) Updating files: 89% (694/779) Updating files: 90% (702/779) Updating files: 91% (709/779) Updating files: 91% (715/779) Updating files: 91% (716/779) Updating files: 92% (717/779) Updating files: 92% (718/779) Updating files: 92% (720/779) Updating files: 92% (721/779) Updating files: 92% (722/779) Updating files: 93% (725/779) Updating files: 94% (733/779) Updating files: 95% (741/779) Updating files: 95% (744/779) Updating files: 96% (748/779) Updating files: 97% (756/779) Updating files: 98% (764/779) Updating files: 99% (772/779) Updating files: 99% (774/779) Updating files: 100% (779/779) Updating files: 100% (779/779), done. 2023-06-09T11:49:35.808 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 2f10137986a v0.18 2023-06-09T11:49:35.815 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.18 2023-06-09T11:49:35.854 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:49:35.854 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 19 -eq 48 ']' 2023-06-09T11:49:35.855 INFO:tasks.workunit.client.0.smithi007.stdout:v0.19 2023-06-09T11:49:35.855 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.19 2023-06-09T11:49:35.855 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.19 2023-06-09T11:49:35.855 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.19 2023-06-09T11:49:37.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:37.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.314+0000 7f3c66fff700 1 Processor -- start 2023-06-09T11:49:37.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.314+0000 7f3c66fff700 1 -- start start 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 0x7f3c60159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 0x7f3c600cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c60159b20 con 0x7f3c600cb9a0 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c600cac80 con 0x7f3c600cb5d0 2023-06-09T11:49:37.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c66fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601597d0 con 0x7f3c6015cd40 2023-06-09T11:49:37.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c5ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 0x7f3c60159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c64d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.315+0000 7f3c6559c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 0x7f3c600cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c64d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:33852/0 (socket says 172.21.15.7:33852) 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c64d9b700 1 -- 172.21.15.7:0/182362748 learned_addr learned my addr 172.21.15.7:0/182362748 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 -- 172.21.15.7:0/182362748 >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 msgr2=0x7f3c600b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 -- 172.21.15.7:0/182362748 >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 msgr2=0x7f3c60159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 0x7f3c60159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 -- 172.21.15.7:0/182362748 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c60071ab0 con 0x7f3c6015cd40 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c64d9b700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c5ffff700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 0x7f3c60159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.316+0000 7f3c6559c700 1 --2- 172.21.15.7:0/182362748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 0x7f3c600cbea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c6008d420 tx=0x7f3c5800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f22f01480f49713 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c5f7fe700 1 -- 172.21.15.7:0/182362748 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5800e070 con 0x7f3c6015cd40 2023-06-09T11:49:37.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c5f7fe700 1 -- 172.21.15.7:0/182362748 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5800bb20 con 0x7f3c6015cd40 2023-06-09T11:49:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c5f7fe700 1 -- 172.21.15.7:0/182362748 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58012430 con 0x7f3c6015cd40 2023-06-09T11:49:37.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 -- 172.21.15.7:0/182362748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 msgr2=0x7f3c600cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 --2- 172.21.15.7:0/182362748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 0x7f3c600cbea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c6008d420 tx=0x7f3c5800b330 comp rx=0 tx=0).stop 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 -- 172.21.15.7:0/182362748 shutdown_connections 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.117:3300/0 conn(0x7f3c600cb5d0 0x7f3c600b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 --2- 172.21.15.7:0/182362748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c6015cd40 0x7f3c600cbea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 --2- 172.21.15.7:0/182362748 >> v2:172.21.15.7:3300/0 conn(0x7f3c600cb9a0 0x7f3c60159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 -- 172.21.15.7:0/182362748 >> 172.21.15.7:0/182362748 conn(0x7f3c60070af0 msgr2=0x7f3c60070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 -- 172.21.15.7:0/182362748 shutdown_connections 2023-06-09T11:49:37.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.317+0000 7f3c66fff700 1 -- 172.21.15.7:0/182362748 wait complete. 2023-06-09T11:49:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.319+0000 7f3c66fff700 1 Processor -- start 2023-06-09T11:49:37.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.320+0000 7f3c66fff700 1 -- start start 2023-06-09T11:49:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.320+0000 7f3c66fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 0x7f3c600c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c66fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 0x7f3c600c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c66fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c6015cd40 0x7f3c600cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c66fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c600cec30 con 0x7f3c600cb9a0 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c66fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6014c740 con 0x7f3c6015cd40 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c66fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c601ef6d0 con 0x7f3c600cb5d0 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c64d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 0x7f3c600c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 0x7f3c600c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c64d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 0x7f3c600c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47838/0 (socket says 172.21.15.7:47838) 2023-06-09T11:49:37.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c64d9b700 1 -- 172.21.15.7:0/2487781977 learned_addr learned my addr 172.21.15.7:0/2487781977 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c6559c700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c6015cd40 0x7f3c600cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 -- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 msgr2=0x7f3c600c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 0x7f3c600c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 -- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c6015cd40 msgr2=0x7f3c600cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c6015cd40 0x7f3c600cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.321+0000 7f3c5ffff700 1 -- 172.21.15.7:0/2487781977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c54009550 con 0x7f3c600cb9a0 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c5ffff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 0x7f3c600c80d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e9b0 tx=0x7f3c4c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00ee50 con 0x7f3c600cb9a0 2023-06-09T11:49:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c00c850 con 0x7f3c600cb9a0 2023-06-09T11:49:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00cc00 con 0x7f3c600cb9a0 2023-06-09T11:49:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c0094a0 con 0x7f3c600cb9a0 2023-06-09T11:49:37.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.322+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c58009a70 con 0x7f3c600cb9a0 2023-06-09T11:49:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.323+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6015d4f0 con 0x7f3c600cb9a0 2023-06-09T11:49:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.324+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c4c00cd60 con 0x7f3c600cb9a0 2023-06-09T11:49:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.326+0000 7f3c5d7fa700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.326+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f3c4c089c60 con 0x7f3c600cb9a0 2023-06-09T11:49:37.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.326+0000 7f3c64d9b700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.327+0000 7f3c64d9b700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 0x7f3c4406d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3c54006d30 tx=0x7f3c540092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.330+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3c4c0544a0 con 0x7f3c600cb9a0 2023-06-09T11:49:37.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.628+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c6015d7d0 con 0x7f3c600cb9a0 2023-06-09T11:49:37.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.629+0000 7f3c5d7fa700 1 -- 172.21.15.7:0/2487781977 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f3c4c021020 con 0x7f3c600cb9a0 2023-06-09T11:49:37.632 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:37.632 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:37.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 msgr2=0x7f3c4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 0x7f3c4406d190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3c54006d30 tx=0x7f3c540092c0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 msgr2=0x7f3c600c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 0x7f3c600c80d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00e9b0 tx=0x7f3c4c00ecc0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 shutdown_connections 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3c6015cd40 0x7f3c600cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3c4406ace0 0x7f3c4406d190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3c600cb5d0 0x7f3c600c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 --2- 172.21.15.7:0/2487781977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c600cb9a0 0x7f3c600c80d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.632+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 >> 172.21.15.7:0/2487781977 conn(0x7f3c60070af0 msgr2=0x7f3c60158410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.633+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 shutdown_connections 2023-06-09T11:49:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.633+0000 7f3c66fff700 1 -- 172.21.15.7:0/2487781977 wait complete. 2023-06-09T11:49:37.637 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:37.658 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.b (rank=2) 2023-06-09T11:49:37.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T11:49:37.659 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:49:37.678 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T11:49:37.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:37.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.891+0000 7f460cfff700 1 Processor -- start 2023-06-09T11:49:37.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.891+0000 7f460cfff700 1 -- start start 2023-06-09T11:49:37.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.892+0000 7f460cfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46081535e0 0x7f4608159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.892+0000 7f460cfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460815a420 con 0x7f46081539b0 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.892+0000 7f460cfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460806b460 con 0x7f46080c4ce0 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.892+0000 7f460cfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080ca670 con 0x7f46081535e0 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46081535e0 0x7f4608159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f46081535e0 0x7f4608159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53832/0 (socket says 172.21.15.7:53832) 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 -- 172.21.15.7:0/3079655274 learned_addr learned my addr 172.21.15.7:0/3079655274 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:37.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f4605d9b700 1 --1- 172.21.15.7:0/3079655274 >> v1:172.21.15.7:6789/0 conn(0x7f46081539b0 0x7f46080c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57814/0 (socket says 172.21.15.7:57814) 2023-06-09T11:49:37.897 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 -- 172.21.15.7:0/3079655274 >> v1:172.21.15.117:6789/0 conn(0x7f46080c4ce0 legacy=0x7f460815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 -- 172.21.15.7:0/3079655274 >> v1:172.21.15.7:6789/0 conn(0x7f46081539b0 legacy=0x7f46080c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 -- 172.21.15.7:0/3079655274 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460806a570 con 0x7f46081535e0 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460559a700 1 -- 172.21.15.7:0/3079655274 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748187138 0 0) 0x7f45fc001110 con 0x7f46081539b0 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.893+0000 7f460659c700 1 --2- 172.21.15.7:0/3079655274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46081535e0 0x7f4608159ee0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45f80098f0 tx=0x7f45f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760cd8b7329e4ce1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460559a700 1 -- 172.21.15.7:0/3079655274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800b8a0 con 0x7f46081535e0 2023-06-09T11:49:37.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460559a700 1 -- 172.21.15.7:0/3079655274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800ba00 con 0x7f46081535e0 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460559a700 1 -- 172.21.15.7:0/3079655274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800bd90 con 0x7f46081535e0 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 -- 172.21.15.7:0/3079655274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46081535e0 msgr2=0x7f4608159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 --2- 172.21.15.7:0/3079655274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46081535e0 0x7f4608159ee0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f45f80098f0 tx=0x7f45f8009c00 comp rx=0 tx=0).stop 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 -- 172.21.15.7:0/3079655274 shutdown_connections 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 --2- 172.21.15.7:0/3079655274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46081535e0 0x7f4608159ee0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 -- 172.21.15.7:0/3079655274 >> 172.21.15.7:0/3079655274 conn(0x7f46080737f0 msgr2=0x7f4608072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:37.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 -- 172.21.15.7:0/3079655274 shutdown_connections 2023-06-09T11:49:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.894+0000 7f460cfff700 1 -- 172.21.15.7:0/3079655274 wait complete. 2023-06-09T11:49:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.897+0000 7f460cfff700 1 Processor -- start 2023-06-09T11:49:37.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.897+0000 7f460cfff700 1 -- start start 2023-06-09T11:49:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 0x7f4608155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 0x7f46081f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460814e940 con 0x7f46080c4ce0 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080cfd00 con 0x7f46081539b0 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46081f3550 con 0x7f46081535e0 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.898+0000 7f460659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53842/0 (socket says 172.21.15.7:53842) 2023-06-09T11:49:37.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 -- 172.21.15.7:0/2204974809 learned_addr learned my addr 172.21.15.7:0/2204974809 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:37.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f4605d9b700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 0x7f4608155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 -- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 msgr2=0x7f4608155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f4606d9d700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 0x7f46081f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 0x7f4608155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 -- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 msgr2=0x7f46081f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 0x7f46081f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 -- 172.21.15.7:0/2204974809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8009550 con 0x7f46080c4ce0 2023-06-09T11:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f4605d9b700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 0x7f4608155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f4606d9d700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 0x7f46081f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460659c700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4608072cf0 tx=0x7f45f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8011a70 con 0x7f46080c4ce0 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f801f070 con 0x7f46080c4ce0 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8011df0 con 0x7f46080c4ce0 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.899+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460806ab30 con 0x7f46080c4ce0 2023-06-09T11:49:37.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.900+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460806a850 con 0x7f46080c4ce0 2023-06-09T11:49:37.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.901+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460806ae10 con 0x7f46080c4ce0 2023-06-09T11:49:37.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.902+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f45f801a430 con 0x7f46080c4ce0 2023-06-09T11:49:37.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.902+0000 7f45f6ffd700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 0x7f45e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:37.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.903+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f45f8021070 con 0x7f46080c4ce0 2023-06-09T11:49:37.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.903+0000 7f4605d9b700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 0x7f45e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:37.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.903+0000 7f4605d9b700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 0x7f45e806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:37.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:37.907+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f45f8055fa0 con 0x7f46080c4ce0 2023-06-09T11:49:38.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.168+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4608155550 con 0x7f46080c4ce0 2023-06-09T11:49:38.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.169+0000 7f45f6ffd700 1 -- 172.21.15.7:0/2204974809 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f45f80595c0 con 0x7f46080c4ce0 2023-06-09T11:49:38.171 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:38.172 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:38.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 msgr2=0x7f45e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:38.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 0x7f45e806d190 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45fc007fd0 tx=0x7f45fc009040 comp rx=0 tx=0).stop 2023-06-09T11:49:38.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 msgr2=0x7f460814e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f4608072cf0 tx=0x7f45f8004750 comp rx=0 tx=0).stop 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 shutdown_connections 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f46081539b0 0x7f46081f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46080c4ce0 0x7f460814e400 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f45e806ace0 0x7f45e806d190 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 --2- 172.21.15.7:0/2204974809 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46081535e0 0x7f4608155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:38.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 >> 172.21.15.7:0/2204974809 conn(0x7f46080737f0 msgr2=0x7f46080c5140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 shutdown_connections 2023-06-09T11:49:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:38.171+0000 7f460cfff700 1 -- 172.21.15.7:0/2204974809 wait complete. 2023-06-09T11:49:38.176 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:38.192 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:49:40.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.401+0000 7f3285dff700 1 Processor -- start 2023-06-09T11:49:40.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.402+0000 7f3285dff700 1 -- start start 2023-06-09T11:49:40.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f3285dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f3285dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 0x7f32800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f3285dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f328015a500 con 0x7f32800c9f40 2023-06-09T11:49:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f3285dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32800c5f10 con 0x7f328015b4a0 2023-06-09T11:49:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f3285dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328015da00 con 0x7f3280153880 2023-06-09T11:49:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47854/0 (socket says 172.21.15.7:47854) 2023-06-09T11:49:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f327f7fe700 1 -- 172.21.15.7:0/3461032197 learned_addr learned my addr 172.21.15.7:0/3461032197 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:40.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.403+0000 7f327ffff700 1 --2- 172.21.15.7:0/3461032197 >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 0x7f32800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327f7fe700 1 -- 172.21.15.7:0/3461032197 >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 msgr2=0x7f32800c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327f7fe700 1 --2- 172.21.15.7:0/3461032197 >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 0x7f32800c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327f7fe700 1 -- 172.21.15.7:0/3461032197 >> v1:172.21.15.117:6789/0 conn(0x7f32800c9f40 legacy=0x7f328015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327f7fe700 1 -- 172.21.15.7:0/3461032197 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32801596e0 con 0x7f3280153880 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327ffff700 1 --2- 172.21.15.7:0/3461032197 >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 0x7f32800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327f7fe700 1 --2- 172.21.15.7:0/3461032197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32700098f0 tx=0x7f3270009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3986fec7b566bc1f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:40.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.404+0000 7f327e7fc700 1 -- 172.21.15.7:0/3461032197 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011830 con 0x7f3280153880 2023-06-09T11:49:40.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f327e7fc700 1 -- 172.21.15.7:0/3461032197 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3270011990 con 0x7f3280153880 2023-06-09T11:49:40.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f327e7fc700 1 -- 172.21.15.7:0/3461032197 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270011d20 con 0x7f3280153880 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 -- 172.21.15.7:0/3461032197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 msgr2=0x7f328015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 --2- 172.21.15.7:0/3461032197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32700098f0 tx=0x7f3270009c00 comp rx=0 tx=0).stop 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 -- 172.21.15.7:0/3461032197 shutdown_connections 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 --2- 172.21.15.7:0/3461032197 >> v2:172.21.15.117:3300/0 conn(0x7f328015b4a0 0x7f32800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 --2- 172.21.15.7:0/3461032197 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f328015d4c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 -- 172.21.15.7:0/3461032197 >> 172.21.15.7:0/3461032197 conn(0x7f32800737f0 msgr2=0x7f32800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 -- 172.21.15.7:0/3461032197 shutdown_connections 2023-06-09T11:49:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.405+0000 7f3285dff700 1 -- 172.21.15.7:0/3461032197 wait complete. 2023-06-09T11:49:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.407+0000 7f3285dff700 1 Processor -- start 2023-06-09T11:49:40.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.407+0000 7f3285dff700 1 -- start start 2023-06-09T11:49:40.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f32800cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 0x7f32801f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32801f2420 con 0x7f32800c9f40 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32800cca40 con 0x7f328015b4a0 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f3285dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32800b3e00 con 0x7f3280153880 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.408+0000 7f327f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53846/0 (socket says 172.21.15.7:53846) 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 -- 172.21.15.7:0/112380963 learned_addr learned my addr 172.21.15.7:0/112380963 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327effd700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f32800cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327ffff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 0x7f32801f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 -- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 msgr2=0x7f32800cbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f32800cbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 -- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 msgr2=0x7f32801f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 0x7f32801f2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 -- 172.21.15.7:0/112380963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278009a70 con 0x7f32800c9f40 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327ffff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 0x7f32801f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f327f7fe700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.409+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3270004480 con 0x7f32800c9f40 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.410+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327001f070 con 0x7f32800c9f40 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.410+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32700038a0 con 0x7f32800c9f40 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.410+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3270009550 con 0x7f32800c9f40 2023-06-09T11:49:40.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.410+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3280159c60 con 0x7f32800c9f40 2023-06-09T11:49:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.411+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3280159980 con 0x7f32800c9f40 2023-06-09T11:49:40.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.412+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3270021030 con 0x7f32800c9f40 2023-06-09T11:49:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.413+0000 7f3267fff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.413+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f327008b100 con 0x7f32800c9f40 2023-06-09T11:49:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.413+0000 7f327effd700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:40.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.414+0000 7f327effd700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 0x7f326c06d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f32800ce830 tx=0x7f3274009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.418+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3270055b70 con 0x7f32800c9f40 2023-06-09T11:49:40.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.671+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3280159f40 con 0x7f32800c9f40 2023-06-09T11:49:40.674 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.672+0000 7f3267fff700 1 -- 172.21.15.7:0/112380963 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f3270059190 con 0x7f32800c9f40 2023-06-09T11:49:40.675 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:40.675 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 msgr2=0x7f326c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 0x7f326c06d190 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f32800ce830 tx=0x7f3274009040 comp rx=0 tx=0).stop 2023-06-09T11:49:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 msgr2=0x7f32800b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f3270000c00 tx=0x7f3270004750 comp rx=0 tx=0).stop 2023-06-09T11:49:40.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 shutdown_connections 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f328015b4a0 0x7f32801f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32800c9f40 0x7f32800b38c0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f326c06ace0 0x7f326c06d190 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 --2- 172.21.15.7:0/112380963 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3280153880 0x7f32800cbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 >> 172.21.15.7:0/112380963 conn(0x7f32800737f0 msgr2=0x7f328006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 shutdown_connections 2023-06-09T11:49:40.678 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:40.674+0000 7f3285dff700 1 -- 172.21.15.7:0/112380963 wait complete. 2023-06-09T11:49:40.679 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:40.694 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:49:42.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:42.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.905+0000 7f3df4fff700 1 Processor -- start 2023-06-09T11:49:42.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.905+0000 7f3df4fff700 1 -- start start 2023-06-09T11:49:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3df4fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3df00cb5d0 0x7f3df00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3df4fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df0159b20 con 0x7f3df00cb9a0 2023-06-09T11:49:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3df4fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df00cac80 con 0x7f3df015cd40 2023-06-09T11:49:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3df4fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df01597d0 con 0x7f3df00cb5d0 2023-06-09T11:49:42.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3dedd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3df00cb9a0 0x7f3df0159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57820/0 (socket says 172.21.15.7:57820) 2023-06-09T11:49:42.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3dedd9b700 1 -- 172.21.15.7:0/2718923060 learned_addr learned my addr 172.21.15.7:0/2718923060 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.906+0000 7f3dee59c700 1 --2- 172.21.15.7:0/2718923060 >> v2:172.21.15.117:3300/0 conn(0x7f3df00cb5d0 0x7f3df00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:42.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621412443 0 0) 0x7f3df0159b20 con 0x7f3df00cb9a0 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd80035b0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880095219 0 0) 0x7f3df00cac80 con 0x7f3df015cd40 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df0159b20 con 0x7f3df015cd40 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3dee59c700 1 -- 172.21.15.7:0/2718923060 >> v1:172.21.15.117:6789/0 conn(0x7f3df015cd40 legacy=0x7f3df00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3dee59c700 1 -- 172.21.15.7:0/2718923060 >> v1:172.21.15.7:6789/0 conn(0x7f3df00cb9a0 legacy=0x7f3df0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:42.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3dee59c700 1 -- 172.21.15.7:0/2718923060 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df0071ab0 con 0x7f3df00cb5d0 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.907+0000 7f3dee59c700 1 --2- 172.21.15.7:0/2718923060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00b52b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009840 tx=0x7f3de0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70e20cd3cde9ed51 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011830 con 0x7f3df00cb5d0 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de0011990 con 0x7f3df00cb5d0 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3ded59a700 1 -- 172.21.15.7:0/2718923060 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de0011d20 con 0x7f3df00cb5d0 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 -- 172.21.15.7:0/2718923060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 msgr2=0x7f3df00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 --2- 172.21.15.7:0/2718923060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00b52b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009840 tx=0x7f3de0009b50 comp rx=0 tx=0).stop 2023-06-09T11:49:42.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 -- 172.21.15.7:0/2718923060 shutdown_connections 2023-06-09T11:49:42.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 --2- 172.21.15.7:0/2718923060 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00b52b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:42.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 -- 172.21.15.7:0/2718923060 >> 172.21.15.7:0/2718923060 conn(0x7f3df0070af0 msgr2=0x7f3df0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:42.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 -- 172.21.15.7:0/2718923060 shutdown_connections 2023-06-09T11:49:42.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.908+0000 7f3df4fff700 1 -- 172.21.15.7:0/2718923060 wait complete. 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.910+0000 7f3df4fff700 1 Processor -- start 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.910+0000 7f3df4fff700 1 -- start start 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015cd40 0x7f3df0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00b2240 con 0x7f3df00cb9a0 2023-06-09T11:49:42.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01ef900 con 0x7f3df00cb5d0 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3df4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df01efbe0 con 0x7f3df015cd40 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3dedd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3dedd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53860/0 (socket says 172.21.15.7:53860) 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.911+0000 7f3dedd9b700 1 -- 172.21.15.7:0/3051430507 learned_addr learned my addr 172.21.15.7:0/3051430507 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3deed9d700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015cd40 0x7f3df0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dee59c700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 -- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015cd40 msgr2=0x7f3df0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015cd40 0x7f3df0150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 -- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 msgr2=0x7f3df00ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 -- 172.21.15.7:0/3051430507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00094a0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3dedd9b700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005910 tx=0x7f3de400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4005d70 con 0x7f3df00cb9a0 2023-06-09T11:49:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de40070d0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de4007460 con 0x7f3df00cb9a0 2023-06-09T11:49:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df015d4f0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.912+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df015d210 con 0x7f3df00cb9a0 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.913+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df015d7d0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.914+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3de40075c0 con 0x7f3df00cb9a0 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.915+0000 7f3ddeffd700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 0x7f3dd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.915+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f3de4089260 con 0x7f3df00cb9a0 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.915+0000 7f3dee59c700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 0x7f3dd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:42.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.916+0000 7f3dee59c700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 0x7f3dd006d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009fd0 tx=0x7f3de001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:42.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:42.920+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3de4053b20 con 0x7f3df00cb9a0 2023-06-09T11:49:43.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.173+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3df0074cf0 con 0x7f3df00cb9a0 2023-06-09T11:49:43.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.174+0000 7f3ddeffd700 1 -- 172.21.15.7:0/3051430507 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f3de4057140 con 0x7f3df00cb9a0 2023-06-09T11:49:43.177 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:43.177 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.176+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 msgr2=0x7f3dd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.176+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 0x7f3dd006d190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009fd0 tx=0x7f3de001c040 comp rx=0 tx=0).stop 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.176+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 msgr2=0x7f3df00ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.176+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f3de4005910 tx=0x7f3de400ca90 comp rx=0 tx=0).stop 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 shutdown_connections 2023-06-09T11:49:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3dd006ace0 0x7f3dd006d190 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3df015cd40 0x7f3df0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3df00cb5d0 0x7f3df00ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 --2- 172.21.15.7:0/3051430507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3df00cb9a0 0x7f3df00ce860 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 >> 172.21.15.7:0/3051430507 conn(0x7f3df0070af0 msgr2=0x7f3df00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 shutdown_connections 2023-06-09T11:49:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:43.177+0000 7f3df4fff700 1 -- 172.21.15.7:0/3051430507 wait complete. 2023-06-09T11:49:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:43.197 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:49:45.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.405+0000 7f559a9ff700 1 Processor -- start 2023-06-09T11:49:45.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.405+0000 7f559a9ff700 1 -- start start 2023-06-09T11:49:45.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.406+0000 7f559a9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5594153880 0x7f559415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.406+0000 7f559a9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 0x7f559415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f559a9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559415a500 con 0x7f559415b2d0 2023-06-09T11:49:45.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f559a9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940c5f10 con 0x7f5594153880 2023-06-09T11:49:45.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f559a9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559415da00 con 0x7f55940c9f40 2023-06-09T11:49:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f55937fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 0x7f559415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f5598f9c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f559415b2d0 0x7f55940c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:56516/0 (socket says 172.21.15.7:56516) 2023-06-09T11:49:45.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f5598f9c700 1 -- 172.21.15.7:0/2569056839 learned_addr learned my addr 172.21.15.7:0/2569056839 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.407+0000 7f5593fff700 1 --2- 172.21.15.7:0/2569056839 >> v2:172.21.15.117:3300/0 conn(0x7f5594153880 0x7f559415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f5592ffd700 1 -- 172.21.15.7:0/2569056839 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916528628 0 0) 0x7f558c001490 con 0x7f559415b2d0 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f5592ffd700 1 -- 172.21.15.7:0/2569056839 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f557c0035b0 con 0x7f559415b2d0 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f55937fe700 1 -- 172.21.15.7:0/2569056839 >> v2:172.21.15.117:3300/0 conn(0x7f5594153880 msgr2=0x7f559415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f55937fe700 1 --2- 172.21.15.7:0/2569056839 >> v2:172.21.15.117:3300/0 conn(0x7f5594153880 0x7f559415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f55937fe700 1 -- 172.21.15.7:0/2569056839 >> v1:172.21.15.117:6789/0 conn(0x7f559415b2d0 legacy=0x7f55940c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f55937fe700 1 -- 172.21.15.7:0/2569056839 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559415b750 con 0x7f55940c9f40 2023-06-09T11:49:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f55937fe700 1 --2- 172.21.15.7:0/2569056839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 0x7f559415ad90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5588009840 tx=0x7f5588009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34b9074beca1de server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f5592ffd700 1 -- 172.21.15.7:0/2569056839 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588011830 con 0x7f55940c9f40 2023-06-09T11:49:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f5592ffd700 1 -- 172.21.15.7:0/2569056839 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5588011990 con 0x7f55940c9f40 2023-06-09T11:49:45.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.408+0000 7f5592ffd700 1 -- 172.21.15.7:0/2569056839 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588011d20 con 0x7f55940c9f40 2023-06-09T11:49:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 -- 172.21.15.7:0/2569056839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 msgr2=0x7f559415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 --2- 172.21.15.7:0/2569056839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 0x7f559415ad90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5588009840 tx=0x7f5588009b50 comp rx=0 tx=0).stop 2023-06-09T11:49:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 -- 172.21.15.7:0/2569056839 shutdown_connections 2023-06-09T11:49:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 --2- 172.21.15.7:0/2569056839 >> v2:172.21.15.117:3300/0 conn(0x7f5594153880 0x7f559415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 --2- 172.21.15.7:0/2569056839 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f55940c9f40 0x7f559415ad90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 -- 172.21.15.7:0/2569056839 >> 172.21.15.7:0/2569056839 conn(0x7f55940737f0 msgr2=0x7f55940720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 -- 172.21.15.7:0/2569056839 shutdown_connections 2023-06-09T11:49:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.409+0000 7f559a9ff700 1 -- 172.21.15.7:0/2569056839 wait complete. 2023-06-09T11:49:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.411+0000 7f559a9ff700 1 Processor -- start 2023-06-09T11:49:45.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.411+0000 7f559a9ff700 1 -- start start 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5594153880 0x7f55940ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 0x7f55940ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594158330 con 0x7f55940c9f40 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55941efd90 con 0x7f559415b2d0 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f559a9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559414de50 con 0x7f5594153880 2023-06-09T11:49:45.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f5593fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f5593fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35280/0 (socket says 172.21.15.7:35280) 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f5593fff700 1 -- 172.21.15.7:0/650972388 learned_addr learned my addr 172.21.15.7:0/650972388 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f5593fff700 1 -- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5594153880 msgr2=0x7f55940ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.412+0000 7f5593fff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5594153880 0x7f55940ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5593fff700 1 -- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 msgr2=0x7f55940ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5598f9c700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 0x7f55940ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5593fff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 0x7f55940ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5593fff700 1 -- 172.21.15.7:0/650972388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880094a0 con 0x7f55940c9f40 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5598f9c700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 0x7f55940ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5593fff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9b0 tx=0x7f558400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558400c850 con 0x7f55940c9f40 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558400c9b0 con 0x7f55940c9f40 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558400cd20 con 0x7f55940c9f40 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5594159980 con 0x7f55940c9f40 2023-06-09T11:49:45.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.413+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55941596e0 con 0x7f55940c9f40 2023-06-09T11:49:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.414+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5594159c60 con 0x7f55940c9f40 2023-06-09T11:49:45.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.415+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5584013470 con 0x7f55940c9f40 2023-06-09T11:49:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.416+0000 7f5590ff9700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 0x7f558006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.416+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f5584089db0 con 0x7f55940c9f40 2023-06-09T11:49:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.417+0000 7f55937fe700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 0x7f558006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.417+0000 7f55937fe700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 0x7f558006d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f55940b22f0 tx=0x7f5588010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:45.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.421+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f558408e070 con 0x7f55940c9f40 2023-06-09T11:49:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.674+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5594074cf0 con 0x7f55940c9f40 2023-06-09T11:49:45.677 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.674+0000 7f5590ff9700 1 -- 172.21.15.7:0/650972388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f5584017090 con 0x7f55940c9f40 2023-06-09T11:49:45.678 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:45.678 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 msgr2=0x7f558006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 0x7f558006d190 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f55940b22f0 tx=0x7f5588010000 comp rx=0 tx=0).stop 2023-06-09T11:49:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 msgr2=0x7f55940cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f558400e9b0 tx=0x7f558400ed70 comp rx=0 tx=0).stop 2023-06-09T11:49:45.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 shutdown_connections 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f559415b2d0 0x7f55940ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55940c9f40 0x7f55940cbdc0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f558006ace0 0x7f558006d190 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 --2- 172.21.15.7:0/650972388 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5594153880 0x7f55940ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 >> 172.21.15.7:0/650972388 conn(0x7f55940737f0 msgr2=0x7f559415b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 shutdown_connections 2023-06-09T11:49:45.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:45.677+0000 7f559a9ff700 1 -- 172.21.15.7:0/650972388 wait complete. 2023-06-09T11:49:45.682 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:45.698 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:49:47.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:47.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.906+0000 7f50cbfff700 1 Processor -- start 2023-06-09T11:49:47.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.906+0000 7f50cbfff700 1 -- start start 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50cbfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c415a500 con 0x7f50c415b4a0 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50cbfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c415da00 con 0x7f50c40c9f40 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50cbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c5f10 con 0x7f50c4153880 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50ca59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f50c415b4a0 0x7f50c40c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50046/0 (socket says 172.21.15.7:50046) 2023-06-09T11:49:47.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50ca59c700 1 -- 172.21.15.7:0/798182605 learned_addr learned my addr 172.21.15.7:0/798182605 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.907+0000 7f50c9d9b700 1 --2- 172.21.15.7:0/798182605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903728493 0 0) 0x7f50bc001490 con 0x7f50c415b4a0 2023-06-09T11:49:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50b00035b0 con 0x7f50c415b4a0 2023-06-09T11:49:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229146497 0 0) 0x7f50b4001110 con 0x7f50c40c9f40 2023-06-09T11:49:47.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50bc001490 con 0x7f50c40c9f40 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c9d9b700 1 -- 172.21.15.7:0/798182605 >> v1:172.21.15.117:6789/0 conn(0x7f50c40c9f40 legacy=0x7f50c415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c9d9b700 1 -- 172.21.15.7:0/798182605 >> v1:172.21.15.7:6789/0 conn(0x7f50c415b4a0 legacy=0x7f50c40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c9d9b700 1 -- 172.21.15.7:0/798182605 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c41596e0 con 0x7f50c4153880 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.908+0000 7f50c9d9b700 1 --2- 172.21.15.7:0/798182605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c415d4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009870 tx=0x7f50c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ca6e0342d2b13d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011830 con 0x7f50c4153880 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0011990 con 0x7f50c4153880 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50c8d99700 1 -- 172.21.15.7:0/798182605 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0011d20 con 0x7f50c4153880 2023-06-09T11:49:47.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 -- 172.21.15.7:0/798182605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 msgr2=0x7f50c415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 --2- 172.21.15.7:0/798182605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c415d4c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009870 tx=0x7f50c0009b80 comp rx=0 tx=0).stop 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 -- 172.21.15.7:0/798182605 shutdown_connections 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 --2- 172.21.15.7:0/798182605 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c415d4c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 -- 172.21.15.7:0/798182605 >> 172.21.15.7:0/798182605 conn(0x7f50c40737f0 msgr2=0x7f50c40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 -- 172.21.15.7:0/798182605 shutdown_connections 2023-06-09T11:49:47.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.909+0000 7f50cbfff700 1 -- 172.21.15.7:0/798182605 wait complete. 2023-06-09T11:49:47.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.911+0000 7f50cbfff700 1 Processor -- start 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.911+0000 7f50cbfff700 1 -- start start 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c41e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415b4a0 0x7f50c41ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40cdd70 con 0x7f50c40c9f40 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40cc6a0 con 0x7f50c415b4a0 2023-06-09T11:49:47.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50cbfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40cc940 con 0x7f50c4153880 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50c9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50c9d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35290/0 (socket says 172.21.15.7:35290) 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.912+0000 7f50c9d9b700 1 -- 172.21.15.7:0/2332340540 learned_addr learned my addr 172.21.15.7:0/2332340540 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50ca59c700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415b4a0 0x7f50c41ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c959a700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c41e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 -- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 msgr2=0x7f50c41e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:47.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c41e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:47.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 -- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415b4a0 msgr2=0x7f50c41ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:47.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415b4a0 0x7f50c41ef5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:47.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 -- 172.21.15.7:0/2332340540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00094a0 con 0x7f50c40c9f40 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.913+0000 7f50c9d9b700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090b0 tx=0x7f50c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.914+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c001a6e0 con 0x7f50c40c9f40 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.914+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c001a840 con 0x7f50c40c9f40 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.914+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c001abb0 con 0x7f50c40c9f40 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.914+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c4159c60 con 0x7f50c40c9f40 2023-06-09T11:49:47.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.914+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4159980 con 0x7f50c40c9f40 2023-06-09T11:49:47.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.915+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c4159f40 con 0x7f50c40c9f40 2023-06-09T11:49:47.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.916+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50c0022070 con 0x7f50c40c9f40 2023-06-09T11:49:47.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.917+0000 7f50ba7fc700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 0x7f50a806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:47.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.917+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7926+0+0 (secure 0 0 0) 0x7f50c008a730 con 0x7f50c40c9f40 2023-06-09T11:49:47.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.917+0000 7f50c959a700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 0x7f50a806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:47.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.917+0000 7f50c959a700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 0x7f50a806d1c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f50c40cc3d0 tx=0x7f50b4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:47.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:47.921+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f50c0055250 con 0x7f50c40c9f40 2023-06-09T11:49:48.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.176+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50c41ef400 con 0x7f50c40c9f40 2023-06-09T11:49:48.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.177+0000 7f50ba7fc700 1 -- 172.21.15.7:0/2332340540 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 185 v185) v1 ==== 95+0+6229 (secure 0 0 0) 0x7f50c0029020 con 0x7f50c40c9f40 2023-06-09T11:49:48.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:48.180 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":185,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":6650,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":183}],"filesystems":[{"mdsmap":{"epoch":185,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:48:52.815616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7279":{"gid":7279,"name":"b","rank":2,"incarnation":169,"state":"up:active","state_seq":24,"addr":"172.21.15.117:6833/3302752268","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3302752268},{"type":"v1","addr":"172.21.15.117:6833","nonce":3302752268}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 msgr2=0x7f50a806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 0x7f50a806d1c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f50c40cc3d0 tx=0x7f50b4009040 comp rx=0 tx=0).stop 2023-06-09T11:49:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 msgr2=0x7f50c40b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f50c00090b0 tx=0x7f50c0004750 comp rx=0 tx=0).stop 2023-06-09T11:49:48.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 shutdown_connections 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50c415b4a0 0x7f50c41ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50c40c9f40 0x7f50c40b1ea0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a806ad10 0x7f50a806d1c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 --2- 172.21.15.7:0/2332340540 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50c4153880 0x7f50c41e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 >> 172.21.15.7:0/2332340540 conn(0x7f50c40737f0 msgr2=0x7f50c41503f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 shutdown_connections 2023-06-09T11:49:48.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:48.179+0000 7f50cbfff700 1 -- 172.21.15.7:0/2332340540 wait complete. 2023-06-09T11:49:48.184 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 185 2023-06-09T11:49:48.200 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is laggy/crashed, in failed state, or mds.b is removed from mdsmap 2023-06-09T11:49:50.201 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.426+0000 7f2c0bdff700 1 Processor -- start 2023-06-09T11:49:50.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.426+0000 7f2c0bdff700 1 -- start start 2023-06-09T11:49:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0bdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2c040c0140 0x7f2c04161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0bdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c040c0bf0 con 0x7f2c0415a0c0 2023-06-09T11:49:50.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0bdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c040702f0 con 0x7f2c040c1bc0 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0bdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04070110 con 0x7f2c040c0140 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0939a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2c0415a0c0 0x7f2c041501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50062/0 (socket says 172.21.15.7:50062) 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0939a700 1 -- 172.21.15.7:0/3441333176 learned_addr learned my addr 172.21.15.7:0/3441333176 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0a39c700 1 --2- 172.21.15.7:0/3441333176 >> v2:172.21.15.7:3300/0 conn(0x7f2c040c0140 0x7f2c04161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c08b99700 1 -- 172.21.15.7:0/3441333176 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392145876 0 0) 0x7f2c040c0bf0 con 0x7f2c0415a0c0 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c08b99700 1 -- 172.21.15.7:0/3441333176 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bf00057c0 con 0x7f2c0415a0c0 2023-06-09T11:49:50.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0a39c700 1 -- 172.21.15.7:0/3441333176 >> v1:172.21.15.117:6789/0 conn(0x7f2c040c1bc0 legacy=0x7f2c041599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0a39c700 1 -- 172.21.15.7:0/3441333176 >> v1:172.21.15.7:6789/0 conn(0x7f2c0415a0c0 legacy=0x7f2c041501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.428+0000 7f2c0a39c700 1 -- 172.21.15.7:0/3441333176 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c040c0e90 con 0x7f2c040c0140 2023-06-09T11:49:50.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c0a39c700 1 --2- 172.21.15.7:0/3441333176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c0140 0x7f2c04161cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5641ef5d2c8d689b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c08b99700 1 -- 172.21.15.7:0/3441333176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00e070 con 0x7f2c040c0140 2023-06-09T11:49:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c08b99700 1 -- 172.21.15.7:0/3441333176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc00bb20 con 0x7f2c040c0140 2023-06-09T11:49:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c08b99700 1 -- 172.21.15.7:0/3441333176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc012430 con 0x7f2c040c0140 2023-06-09T11:49:50.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c0bdff700 1 -- 172.21.15.7:0/3441333176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c0140 msgr2=0x7f2c04161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/3441333176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c0140 0x7f2c04161cf0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009d90 tx=0x7f2bfc00b330 comp rx=0 tx=0).stop 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c0bdff700 1 -- 172.21.15.7:0/3441333176 shutdown_connections 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.429+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/3441333176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c0140 0x7f2c04161cf0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.430+0000 7f2c0bdff700 1 -- 172.21.15.7:0/3441333176 >> 172.21.15.7:0/3441333176 conn(0x7f2c0406a4d0 msgr2=0x7f2c0406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.430+0000 7f2c0bdff700 1 -- 172.21.15.7:0/3441333176 shutdown_connections 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.430+0000 7f2c0bdff700 1 -- 172.21.15.7:0/3441333176 wait complete. 2023-06-09T11:49:50.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.431+0000 7f2c0bdff700 1 Processor -- start 2023-06-09T11:49:50.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.431+0000 7f2c0bdff700 1 -- start start 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c040c0140 0x7f2c040c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c0415a0c0 0x7f2c04158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041ea810 con 0x7f2c040c1bc0 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041ee740 con 0x7f2c040c0140 2023-06-09T11:49:50.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.432+0000 7f2c0bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c041eec80 con 0x7f2c0415a0c0 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35304/0 (socket says 172.21.15.7:35304) 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 -- 172.21.15.7:0/1928133563 learned_addr learned my addr 172.21.15.7:0/1928133563 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 -- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c0415a0c0 msgr2=0x7f2c04158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c0415a0c0 0x7f2c04158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 -- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c040c0140 msgr2=0x7f2c040c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c040c0140 0x7f2c040c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 -- 172.21.15.7:0/1928133563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc009a70 con 0x7f2c040c1bc0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2c0939a700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005910 tx=0x7f2bf400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4005d70 con 0x7f2c040c1bc0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf40070d0 con 0x7f2c040c1bc0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.433+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007440 con 0x7f2c040c1bc0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.434+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c040c1450 con 0x7f2c040c1bc0 2023-06-09T11:49:50.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.434+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c040c1170 con 0x7f2c040c1bc0 2023-06-09T11:49:50.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.434+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c040705d0 con 0x7f2c040c1bc0 2023-06-09T11:49:50.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.436+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bf40075a0 con 0x7f2c040c1bc0 2023-06-09T11:49:50.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.437+0000 7f2bfa7fc700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 0x7f2be806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.437+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f2bf4089060 con 0x7f2c040c1bc0 2023-06-09T11:49:50.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.437+0000 7f2c09b9b700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 0x7f2be806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.437+0000 7f2c09b9b700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 0x7f2be806d1c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000f60 tx=0x7f2c00007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.441+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2bf40bb2d0 con 0x7f2c040c1bc0 2023-06-09T11:49:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.692+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c041ea950 con 0x7f2c040c1bc0 2023-06-09T11:49:50.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.693+0000 7f2bfa7fc700 1 -- 172.21.15.7:0/1928133563 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f2bf4017090 con 0x7f2c040c1bc0 2023-06-09T11:49:50.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:50.696 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":186,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:49.651979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 msgr2=0x7f2be806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 0x7f2be806d1c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2c00000f60 tx=0x7f2c00007a40 comp rx=0 tx=0).stop 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 msgr2=0x7f2c04158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4005910 tx=0x7f2bf400c860 comp rx=0 tx=0).stop 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 shutdown_connections 2023-06-09T11:49:50.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2be806ad10 0x7f2be806d1c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c0415a0c0 0x7f2c04158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c040c0140 0x7f2c040c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 --2- 172.21.15.7:0/1928133563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c040c1bc0 0x7f2c04158930 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 >> 172.21.15.7:0/1928133563 conn(0x7f2c0406a4d0 msgr2=0x7f2c040722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 shutdown_connections 2023-06-09T11:49:50.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.695+0000 7f2c0bdff700 1 -- 172.21.15.7:0/1928133563 wait complete. 2023-06-09T11:49:50.700 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 186 2023-06-09T11:49:50.716 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b down, removed from mdsmap 2023-06-09T11:49:50.716 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:49:50.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.913+0000 7f3a849ff700 1 Processor -- start 2023-06-09T11:49:50.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.913+0000 7f3a849ff700 1 -- start start 2023-06-09T11:49:50.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.914+0000 7f3a849ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a8015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.914+0000 7f3a849ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3a8015b4a0 0x7f3a800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.914+0000 7f3a849ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8015a500 con 0x7f3a800c9f40 2023-06-09T11:49:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a849ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a800c5f10 con 0x7f3a8015b4a0 2023-06-09T11:49:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a849ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8015da00 con 0x7f3a80153880 2023-06-09T11:49:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7dd9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3a800c9f40 0x7f3a8015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50068/0 (socket says 172.21.15.7:50068) 2023-06-09T11:49:50.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7dd9b700 1 -- 172.21.15.7:0/2613494928 learned_addr learned my addr 172.21.15.7:0/2613494928 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7ed9d700 1 --2- 172.21.15.7:0/2613494928 >> v2:172.21.15.117:3300/0 conn(0x7f3a8015b4a0 0x7f3a800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a8015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005685645 0 0) 0x7f3a74001110 con 0x7f3a800c9f40 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.915+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a680035b0 con 0x7f3a800c9f40 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3012479170 0 0) 0x7f3a680035b0 con 0x7f3a800c9f40 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 msgr2=0x7f3a8015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a8015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 >> v2:172.21.15.117:3300/0 conn(0x7f3a8015b4a0 msgr2=0x7f3a800c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:50.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 --2- 172.21.15.7:0/2613494928 >> v2:172.21.15.117:3300/0 conn(0x7f3a8015b4a0 0x7f3a800c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a801596e0 con 0x7f3a800c9f40 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f3a74001800 con 0x7f3a800c9f40 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 >> v1:172.21.15.7:6789/0 conn(0x7f3a800c9f40 legacy=0x7f3a8015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a68004430 0x7f3a680048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a68005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800c5f10 con 0x7f3a68003a00 2023-06-09T11:49:50.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8015da00 con 0x7f3a68004430 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800ba10 con 0x7f3a80153880 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7ed9d700 1 --2- 172.21.15.7:0/2613494928 >> v2:172.21.15.117:3300/0 conn(0x7f3a8015b4a0 0x7f3a800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7dd9b700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a68004430 0x7f3a680048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7ed9d700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a68005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 -- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 msgr2=0x7f3a68005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a80153880 0x7f3a68005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 -- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a68004430 msgr2=0x7f3a680048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a68004430 0x7f3a680048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.916+0000 7f3a7e59c700 1 -- 172.21.15.7:0/2613494928 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a680035b0 con 0x7f3a68003a00 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a68003e10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009b90 tx=0x7f3a68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883feaad1792b44f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000e6c0 con 0x7f3a68003a00 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7000e820 con 0x7f3a68003a00 2023-06-09T11:49:50.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a7d59a700 1 -- 172.21.15.7:0/2613494928 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000eb90 con 0x7f3a68003a00 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a849ff700 1 -- 172.21.15.7:0/2613494928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 msgr2=0x7f3a68003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a849ff700 1 --2- 172.21.15.7:0/2613494928 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a68003e10 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009b90 tx=0x7f3a68005880 comp rx=0 tx=0).stop 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.917+0000 7f3a7ed9d700 1 -- 172.21.15.7:0/2613494928 reap_dead start 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.918+0000 7f3a849ff700 1 -- 172.21.15.7:0/2613494928 shutdown_connections 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.918+0000 7f3a849ff700 1 -- 172.21.15.7:0/2613494928 >> 172.21.15.7:0/2613494928 conn(0x7f3a800737f0 msgr2=0x7f3a800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.918+0000 7f3a849ff700 1 -- 172.21.15.7:0/2613494928 shutdown_connections 2023-06-09T11:49:50.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.918+0000 7f3a849ff700 1 -- 172.21.15.7:0/2613494928 wait complete. 2023-06-09T11:49:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.919+0000 7f3a849ff700 1 Processor -- start 2023-06-09T11:49:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.920+0000 7f3a849ff700 1 -- start start 2023-06-09T11:49:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.920+0000 7f3a849ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.920+0000 7f3a849ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a800ce130 0x7f3a800ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a849ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a800cb700 0x7f3a800cbb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a849ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800cb270 con 0x7f3a68003a00 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a849ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800cc0b0 con 0x7f3a800ce130 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a849ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800cd530 con 0x7f3a800cb700 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35322/0 (socket says 172.21.15.7:35322) 2023-06-09T11:49:50.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 -- 172.21.15.7:0/850016748 learned_addr learned my addr 172.21.15.7:0/850016748 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 -- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a800cb700 msgr2=0x7f3a800cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a800cb700 0x7f3a800cbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 -- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a800ce130 msgr2=0x7f3a800ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a800ce130 0x7f3a800ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 -- 172.21.15.7:0/850016748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a80159c60 con 0x7f3a68003a00 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.921+0000 7f3a7e59c700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005770 tx=0x7f3a700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.922+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000e6c0 con 0x7f3a68003a00 2023-06-09T11:49:50.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.922+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a70003ad0 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.922+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a70003e80 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.922+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a80159980 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.922+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a80159f40 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.923+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a801f04e0 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.924+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a70013470 con 0x7f3a68003a00 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.924+0000 7f3a6effd700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 0x7f3a6006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:50.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.925+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f3a7008b600 con 0x7f3a68003a00 2023-06-09T11:49:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.925+0000 7f3a7dd9b700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 0x7f3a6006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.925+0000 7f3a7dd9b700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 0x7f3a6006d1c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3a740012b0 tx=0x7f3a74001d00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:50.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:50.929+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3a801f04e0 con 0x7f3a68003a00 2023-06-09T11:49:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.183+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3a801f04e0 con 0x7f3a68003a00 2023-06-09T11:49:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.184+0000 7f3a6effd700 1 -- 172.21.15.7:0/850016748 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f3a801f04e0 con 0x7f3a68003a00 2023-06-09T11:49:51.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:51.187 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":186,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:49.651979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.186+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 msgr2=0x7f3a6006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 0x7f3a6006d1c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3a740012b0 tx=0x7f3a74001d00 comp rx=0 tx=0).stop 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 msgr2=0x7f3a801effa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f3a68005770 tx=0x7f3a700047c0 comp rx=0 tx=0).stop 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 shutdown_connections 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3a6006ad10 0x7f3a6006d1c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a800cb700 0x7f3a800cbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a68003a00 0x7f3a801effa0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 --2- 172.21.15.7:0/850016748 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a800ce130 0x7f3a800ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 >> 172.21.15.7:0/850016748 conn(0x7f3a800737f0 msgr2=0x7f3a800cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 shutdown_connections 2023-06-09T11:49:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:51.187+0000 7f3a849ff700 1 -- 172.21.15.7:0/850016748 wait complete. 2023-06-09T11:49:51.191 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 186 2023-06-09T11:49:51.208 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 186, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 186, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6650": { "addr": "172.21.15.117:6837/3744293936", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3744293936, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3744293936, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6650, "incarnation": 186, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:replay", "state_seq": 1 }, "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 71, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:49:49.651979+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 6650 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:49:51.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:51.919 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:49:53.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.416+0000 7fefe57ff700 1 Processor -- start 2023-06-09T11:49:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.417+0000 7fefe57ff700 1 -- start start 2023-06-09T11:49:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefe57ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00c9090 0x7fefe00c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefe57ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe01598b0 0x7fefe0159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefe57ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe015cec0 con 0x7fefe00cad00 2023-06-09T11:49:53.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefe57ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe015d160 con 0x7fefe01598b0 2023-06-09T11:49:53.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefe57ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe015d440 con 0x7fefe00c9090 2023-06-09T11:49:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefdf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe01598b0 0x7fefe0159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefdf7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fefe01598b0 0x7fefe0159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45248/0 (socket says 172.21.15.7:45248) 2023-06-09T11:49:53.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefdf7fe700 1 -- 172.21.15.7:0/1852725035 learned_addr learned my addr 172.21.15.7:0/1852725035 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.418+0000 7fefdeffd700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00c9090 0x7fefe00c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefdf7fe700 1 -- 172.21.15.7:0/1852725035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00c9090 msgr2=0x7fefe00c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefdf7fe700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00c9090 0x7fefe00c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefdf7fe700 1 -- 172.21.15.7:0/1852725035 >> v1:172.21.15.117:6789/0 conn(0x7fefe00cad00 legacy=0x7fefe00cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefdf7fe700 1 -- 172.21.15.7:0/1852725035 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe015d720 con 0x7fefe01598b0 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefdf7fe700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe01598b0 0x7fefe0159d10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e427b151c339b239 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefde7fc700 1 -- 172.21.15.7:0/1852725035 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800e070 con 0x7fefe01598b0 2023-06-09T11:49:53.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefde7fc700 1 -- 172.21.15.7:0/1852725035 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800b910 con 0x7fefe01598b0 2023-06-09T11:49:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.419+0000 7fefde7fc700 1 -- 172.21.15.7:0/1852725035 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800bcc0 con 0x7fefe01598b0 2023-06-09T11:49:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 -- 172.21.15.7:0/1852725035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe01598b0 msgr2=0x7fefe0159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe01598b0 0x7fefe0159d10 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).stop 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 -- 172.21.15.7:0/1852725035 shutdown_connections 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe01598b0 0x7fefe0159d10 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 --2- 172.21.15.7:0/1852725035 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00c9090 0x7fefe00c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 -- 172.21.15.7:0/1852725035 >> 172.21.15.7:0/1852725035 conn(0x7fefe0070af0 msgr2=0x7fefe0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 -- 172.21.15.7:0/1852725035 shutdown_connections 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.420+0000 7fefe57ff700 1 -- 172.21.15.7:0/1852725035 wait complete. 2023-06-09T11:49:53.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.422+0000 7fefe57ff700 1 Processor -- start 2023-06-09T11:49:53.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.422+0000 7fefe57ff700 1 -- start start 2023-06-09T11:49:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00cad00 0x7fefe0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 0x7fefe01ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01545f0 con 0x7fefe00c9090 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01f2ee0 con 0x7fefe01598b0 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefe57ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe01524c0 con 0x7fefe00cad00 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45262/0 (socket says 172.21.15.7:45262) 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 -- 172.21.15.7:0/3598966522 learned_addr learned my addr 172.21.15.7:0/3598966522 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:53.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefd7fff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00cad00 0x7fefe0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdf7fe700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 0x7fefe01ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 -- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00cad00 msgr2=0x7fefe0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00cad00 0x7fefe0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 -- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 msgr2=0x7fefe01ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 0x7fefe01ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdeffd700 1 -- 172.21.15.7:0/3598966522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009b30 con 0x7fefe00c9090 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.423+0000 7fefdf7fe700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 0x7fefe01ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:53.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefdeffd700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e870 tx=0x7fefd000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0009590 con 0x7fefe00c9090 2023-06-09T11:49:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000ee70 con 0x7fefe00c9090 2023-06-09T11:49:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ca50 con 0x7fefe00c9090 2023-06-09T11:49:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe015a250 con 0x7fefe00c9090 2023-06-09T11:49:53.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.424+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00ca620 con 0x7fefe00c9090 2023-06-09T11:49:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.425+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe00cb130 con 0x7fefe00c9090 2023-06-09T11:49:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.426+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fefd000cbb0 con 0x7fefe00c9090 2023-06-09T11:49:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.427+0000 7fefd77fe700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.427+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fefd0089c90 con 0x7fefe00c9090 2023-06-09T11:49:53.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.427+0000 7fefd7fff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:53.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.428+0000 7fefd7fff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 0x7fefc006d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fefe00b1e40 tx=0x7fefc8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.432+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fefd0057f90 con 0x7fefe00c9090 2023-06-09T11:49:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.687+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefe00caae0 con 0x7fefe00c9090 2023-06-09T11:49:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.687+0000 7fefd77fe700 1 -- 172.21.15.7:0/3598966522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fefd0017090 con 0x7fefe00c9090 2023-06-09T11:49:53.690 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:53.690 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":186,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:49.651979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 msgr2=0x7fefc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 0x7fefc006d190 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fefe00b1e40 tx=0x7fefc8009040 comp rx=0 tx=0).stop 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 msgr2=0x7fefe0151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e870 tx=0x7fefd000ec30 comp rx=0 tx=0).stop 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 shutdown_connections 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fefe01598b0 0x7fefe01ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fefe00c9090 0x7fefe0151ca0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fefc006ace0 0x7fefc006d190 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.689+0000 7fefe57ff700 1 --2- 172.21.15.7:0/3598966522 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fefe00cad00 0x7fefe0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.690+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 >> 172.21.15.7:0/3598966522 conn(0x7fefe0070af0 msgr2=0x7fefe0158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.690+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 shutdown_connections 2023-06-09T11:49:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:53.690+0000 7fefe57ff700 1 -- 172.21.15.7:0/3598966522 wait complete. 2023-06-09T11:49:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 186 2023-06-09T11:49:53.710 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:49:55.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:55.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.917+0000 7f16f6dff700 1 Processor -- start 2023-06-09T11:49:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.917+0000 7f16f6dff700 1 -- start start 2023-06-09T11:49:55.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.918+0000 7f16f6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 0x7f16f01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.918+0000 7f16f6dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f16f00c1f90 0x7f16f0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.918+0000 7f16f6dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f00c0bf0 con 0x7f16f00c0140 2023-06-09T11:49:55.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.918+0000 7f16f6dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f0070110 con 0x7f16f00c1f90 2023-06-09T11:49:55.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.918+0000 7f16f6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0071300 con 0x7f16f00c1bc0 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16effff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f16f00c0140 0x7f16f01501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58978/0 (socket says 172.21.15.7:58978) 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16effff700 1 -- 172.21.15.7:0/1442175315 learned_addr learned my addr 172.21.15.7:0/1442175315 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f4b9b700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 0x7f16f01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f539c700 1 --2- 172.21.15.7:0/1442175315 >> v2:172.21.15.117:3300/0 conn(0x7f16f00c1f90 0x7f16f0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16ef7fe700 1 -- 172.21.15.7:0/1442175315 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660041523 0 0) 0x7f16f00c0bf0 con 0x7f16f00c0140 2023-06-09T11:49:55.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16ef7fe700 1 -- 172.21.15.7:0/1442175315 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e00035b0 con 0x7f16f00c0140 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f539c700 1 -- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 msgr2=0x7f16f01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f539c700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 0x7f16f01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f539c700 1 -- 172.21.15.7:0/1442175315 >> v1:172.21.15.7:6789/0 conn(0x7f16f00c0140 legacy=0x7f16f01501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f539c700 1 -- 172.21.15.7:0/1442175315 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00702f0 con 0x7f16f00c1f90 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.919+0000 7f16f4b9b700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 0x7f16f01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f539c700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c1f90 0x7f16f0159ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009d90 tx=0x7f16e800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb6d0783185f9605 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:55.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16ef7fe700 1 -- 172.21.15.7:0/1442175315 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800e070 con 0x7f16f00c1f90 2023-06-09T11:49:55.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16ef7fe700 1 -- 172.21.15.7:0/1442175315 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e800bb20 con 0x7f16f00c1f90 2023-06-09T11:49:55.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16ef7fe700 1 -- 172.21.15.7:0/1442175315 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8012430 con 0x7f16f00c1f90 2023-06-09T11:49:55.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 -- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c1f90 msgr2=0x7f16f0159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:55.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c1f90 0x7f16f0159ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009d90 tx=0x7f16e800b330 comp rx=0 tx=0).stop 2023-06-09T11:49:55.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 -- 172.21.15.7:0/1442175315 shutdown_connections 2023-06-09T11:49:55.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c1f90 0x7f16f0159ef0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:55.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 --2- 172.21.15.7:0/1442175315 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1bc0 0x7f16f01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:55.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.920+0000 7f16f6dff700 1 -- 172.21.15.7:0/1442175315 >> 172.21.15.7:0/1442175315 conn(0x7f16f006a4d0 msgr2=0x7f16f006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:55.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.921+0000 7f16f6dff700 1 -- 172.21.15.7:0/1442175315 shutdown_connections 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.921+0000 7f16f6dff700 1 -- 172.21.15.7:0/1442175315 wait complete. 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.922+0000 7f16f6dff700 1 Processor -- start 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.922+0000 7f16f6dff700 1 -- start start 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 0x7f16f00c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1f90 0x7f16f00c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f01537e0 con 0x7f16f00c1bc0 2023-06-09T11:49:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0150870 con 0x7f16f00c0140 2023-06-09T11:49:55.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.923+0000 7f16f6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00ca6f0 con 0x7f16f00c1f90 2023-06-09T11:49:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16f4b9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 0x7f16f00c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45276/0 (socket says 172.21.15.7:45276) 2023-06-09T11:49:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 -- 172.21.15.7:0/40086238 learned_addr learned my addr 172.21.15.7:0/40086238 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:55.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16f539c700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1f90 0x7f16f00c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 -- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1f90 msgr2=0x7f16f00c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1f90 0x7f16f00c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 -- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 msgr2=0x7f16f00c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 0x7f16f00c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 -- 172.21.15.7:0/40086238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e40094a0 con 0x7f16f00c1bc0 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16f4b9b700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 0x7f16f00c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:49:55.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.924+0000 7f16effff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c700 tx=0x7f16dc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.925+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc005d70 con 0x7f16f00c1bc0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.925+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16dc0070d0 con 0x7f16f00c1bc0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.925+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16dc007460 con 0x7f16f00c1bc0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.925+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e8009a70 con 0x7f16f00c1bc0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.925+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f00c1170 con 0x7f16f00c1bc0 2023-06-09T11:49:55.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.926+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f00c0e90 con 0x7f16f00c1bc0 2023-06-09T11:49:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.927+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f16dc0075c0 con 0x7f16f00c1bc0 2023-06-09T11:49:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.928+0000 7f16ed7fa700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.928+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f16dc0890b0 con 0x7f16f00c1bc0 2023-06-09T11:49:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.928+0000 7f16f4b9b700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:55.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.929+0000 7f16f4b9b700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 0x7f16d406d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16e40114b0 tx=0x7f16e4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:55.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:55.932+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f16dc0539b0 con 0x7f16f00c1bc0 2023-06-09T11:49:56.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.183+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16f00c1450 con 0x7f16f00c1bc0 2023-06-09T11:49:56.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.183+0000 7f16ed7fa700 1 -- 172.21.15.7:0/40086238 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 186 v186) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f16dc017090 con 0x7f16f00c1bc0 2023-06-09T11:49:56.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:56.186 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":186,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":186,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:49.651979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:56.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.185+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 msgr2=0x7f16d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.185+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 0x7f16d406d190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f16e40114b0 tx=0x7f16e4009210 comp rx=0 tx=0).stop 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.185+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 msgr2=0x7f16f01ef0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.185+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00c700 tx=0x7f16dc00ca10 comp rx=0 tx=0).stop 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 shutdown_connections 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f16d406ace0 0x7f16d406d190 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16f00c1f90 0x7f16f00c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16f00c0140 0x7f16f00c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:56.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 --2- 172.21.15.7:0/40086238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16f00c1bc0 0x7f16f01ef0d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:56.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 >> 172.21.15.7:0/40086238 conn(0x7f16f006a4d0 msgr2=0x7f16f014f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:56.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 shutdown_connections 2023-06-09T11:49:56.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:56.186+0000 7f16f6dff700 1 -- 172.21.15.7:0/40086238 wait complete. 2023-06-09T11:49:56.190 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 186 2023-06-09T11:49:56.206 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:49:58.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:49:58.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.415+0000 7f90383ff700 1 Processor -- start 2023-06-09T11:49:58.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.415+0000 7f90383ff700 1 -- start start 2023-06-09T11:49:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.416+0000 7f90383ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f90300c1bc0 0x7f90301599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.416+0000 7f90383ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300c0bf0 con 0x7f90300c0140 2023-06-09T11:49:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.416+0000 7f90383ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300702f0 con 0x7f90300c1f90 2023-06-09T11:49:58.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.416+0000 7f90383ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9030070110 con 0x7f90300c1bc0 2023-06-09T11:49:58.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903599a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f90300c0140 0x7f90301501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58988/0 (socket says 172.21.15.7:58988) 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903599a700 1 -- 172.21.15.7:0/2680397784 learned_addr learned my addr 172.21.15.7:0/2680397784 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903619b700 1 --2- 172.21.15.7:0/2680397784 >> v2:172.21.15.117:3300/0 conn(0x7f90300c1bc0 0x7f90301599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 253104915 0 0) 0x7f90300c0bf0 con 0x7f90300c0140 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901c0035b0 con 0x7f90300c0140 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421921417 0 0) 0x7f90300702f0 con 0x7f90300c1f90 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90300c0bf0 con 0x7f90300c1f90 2023-06-09T11:49:58.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903619b700 1 -- 172.21.15.7:0/2680397784 >> v1:172.21.15.117:6789/0 conn(0x7f90300c1f90 legacy=0x7f9030160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903619b700 1 -- 172.21.15.7:0/2680397784 >> v1:172.21.15.7:6789/0 conn(0x7f90300c0140 legacy=0x7f90301501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.417+0000 7f903619b700 1 -- 172.21.15.7:0/2680397784 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300c0e90 con 0x7f90300c1bc0 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f903619b700 1 --2- 172.21.15.7:0/2680397784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1bc0 0x7f90301599b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f902c009840 tx=0x7f902c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9aa2edda12f9fc0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011830 con 0x7f90300c1bc0 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c011990 con 0x7f90300c1bc0 2023-06-09T11:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f9035199700 1 -- 172.21.15.7:0/2680397784 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011d20 con 0x7f90300c1bc0 2023-06-09T11:49:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f90383ff700 1 -- 172.21.15.7:0/2680397784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1bc0 msgr2=0x7f90301599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.418+0000 7f90383ff700 1 --2- 172.21.15.7:0/2680397784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1bc0 0x7f90301599b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f902c009840 tx=0x7f902c009b50 comp rx=0 tx=0).stop 2023-06-09T11:49:58.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.419+0000 7f90383ff700 1 -- 172.21.15.7:0/2680397784 shutdown_connections 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.419+0000 7f90383ff700 1 --2- 172.21.15.7:0/2680397784 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1bc0 0x7f90301599b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.419+0000 7f90383ff700 1 -- 172.21.15.7:0/2680397784 >> 172.21.15.7:0/2680397784 conn(0x7f903006a4d0 msgr2=0x7f903006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.419+0000 7f90383ff700 1 -- 172.21.15.7:0/2680397784 shutdown_connections 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.419+0000 7f90383ff700 1 -- 172.21.15.7:0/2680397784 wait complete. 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.420+0000 7f90383ff700 1 Processor -- start 2023-06-09T11:49:58.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.420+0000 7f90383ff700 1 -- start start 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 0x7f90300c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 0x7f903014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300afa00 con 0x7f90300c1bc0 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300b09d0 con 0x7f90300c1f90 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.421+0000 7f90383ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300adaf0 con 0x7f90300c0140 2023-06-09T11:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45280/0 (socket says 172.21.15.7:45280) 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 -- 172.21.15.7:0/1458060726 learned_addr learned my addr 172.21.15.7:0/1458060726 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903619b700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 0x7f90300c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903699c700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 0x7f903014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 -- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 msgr2=0x7f90300c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 0x7f90300c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 -- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 msgr2=0x7f903014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 0x7f903014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 -- 172.21.15.7:0/1458060726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0094a0 con 0x7f90300c1bc0 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903619b700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 0x7f90300c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903699c700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 0x7f903014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.422+0000 7f903599a700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f902000c700 tx=0x7f902000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.423+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020005d70 con 0x7f90300c1bc0 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.423+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90200070d0 con 0x7f90300c1bc0 2023-06-09T11:49:58.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.423+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020007480 con 0x7f90300c1bc0 2023-06-09T11:49:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.423+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90300c1450 con 0x7f90300c1bc0 2023-06-09T11:49:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.423+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90300c1170 con 0x7f90300c1bc0 2023-06-09T11:49:58.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.424+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90300705d0 con 0x7f90300c1bc0 2023-06-09T11:49:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.425+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90200075e0 con 0x7f90300c1bc0 2023-06-09T11:49:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.426+0000 7f9026ffd700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 0x7f901406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:49:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.426+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f90200892b0 con 0x7f90300c1bc0 2023-06-09T11:49:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.426+0000 7f903619b700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 0x7f901406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:49:58.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.427+0000 7f903619b700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 0x7f901406d1c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f902c000c00 tx=0x7f902c01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:49:58.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.430+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9020053b30 con 0x7f90300c1bc0 2023-06-09T11:49:58.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.687+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90300b0b40 con 0x7f90300c1bc0 2023-06-09T11:49:58.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.688+0000 7f9026ffd700 1 -- 172.21.15.7:0/1458060726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5466 (secure 0 0 0) 0x7f9020057150 con 0x7f90300c1bc0 2023-06-09T11:49:58.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:49:58.691 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:57.662725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:49:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 msgr2=0x7f901406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 0x7f901406d1c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f902c000c00 tx=0x7f902c01c040 comp rx=0 tx=0).stop 2023-06-09T11:49:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 msgr2=0x7f90300ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:49:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f902000c700 tx=0x7f902000cac0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 shutdown_connections 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90300c1f90 0x7f903014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f901406ad10 0x7f901406d1c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90300c0140 0x7f90300c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 --2- 172.21.15.7:0/1458060726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90300c1bc0 0x7f90300ae210 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 >> 172.21.15.7:0/1458060726 conn(0x7f903006a4d0 msgr2=0x7f90300c8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 shutdown_connections 2023-06-09T11:49:58.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:49:58.690+0000 7f90383ff700 1 -- 172.21.15.7:0/1458060726 wait complete. 2023-06-09T11:49:58.695 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 187 2023-06-09T11:49:58.711 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:00.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:00.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.926+0000 7fe5ea3ff700 1 Processor -- start 2023-06-09T11:50:00.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.926+0000 7fe5ea3ff700 1 -- start start 2023-06-09T11:50:00.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5e415b380 0x7fe5e4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 0x7fe5e4158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e40714e0 con 0x7fe5e415b380 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e41518e0 con 0x7fe5e4071060 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.927+0000 7fe5ea3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41580a0 con 0x7fe5e4151510 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5e415b380 0x7fe5e4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe5e415b380 0x7fe5e4070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45290/0 (socket says 172.21.15.7:45290) 2023-06-09T11:50:00.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 -- 172.21.15.7:0/1189460931 learned_addr learned my addr 172.21.15.7:0/1189460931 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:00.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e3fff700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e899c700 1 --2- 172.21.15.7:0/1189460931 >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 0x7fe5e4158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 -- 172.21.15.7:0/1189460931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 msgr2=0x7fe5e40c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e40c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 -- 172.21.15.7:0/1189460931 >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 msgr2=0x7fe5e4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/1189460931 >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 0x7fe5e4158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 -- 172.21.15.7:0/1189460931 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40c6180 con 0x7fe5e415b380 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e899c700 1 --2- 172.21.15.7:0/1189460931 >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 0x7fe5e4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.928+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e415b380 0x7fe5e4070b20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40adac0 tx=0x7fe5d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e3746bee565614 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5e2ffd700 1 -- 172.21.15.7:0/1189460931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8011830 con 0x7fe5e415b380 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5e2ffd700 1 -- 172.21.15.7:0/1189460931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8011990 con 0x7fe5e415b380 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5e2ffd700 1 -- 172.21.15.7:0/1189460931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8011d20 con 0x7fe5e415b380 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/1189460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e415b380 msgr2=0x7fe5e4070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e415b380 0x7fe5e4070b20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fe5e40adac0 tx=0x7fe5d8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/1189460931 shutdown_connections 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/1189460931 >> v2:172.21.15.117:3300/0 conn(0x7fe5e4071060 0x7fe5e4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e40c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/1189460931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e415b380 0x7fe5e4070b20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/1189460931 >> 172.21.15.7:0/1189460931 conn(0x7fe5e406a4d0 msgr2=0x7fe5e406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/1189460931 shutdown_connections 2023-06-09T11:50:00.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.929+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/1189460931 wait complete. 2023-06-09T11:50:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.931+0000 7fe5ea3ff700 1 Processor -- start 2023-06-09T11:50:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.931+0000 7fe5ea3ff700 1 -- start start 2023-06-09T11:50:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e4158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 0x7fe5e414bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40c99f0 con 0x7fe5e4071060 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e41588c0 con 0x7fe5e415b380 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5ea3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e40c6f10 con 0x7fe5e4151510 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.932+0000 7fe5e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45296/0 (socket says 172.21.15.7:45296) 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 -- 172.21.15.7:0/3591836824 learned_addr learned my addr 172.21.15.7:0/3591836824 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:00.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e4158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e899c700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 0x7fe5e414bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 -- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 msgr2=0x7fe5e4158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e4158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 -- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 msgr2=0x7fe5e414bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 0x7fe5e414bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 -- 172.21.15.7:0/3591836824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d8009550 con 0x7fe5e4071060 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e899c700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 0x7fe5e414bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:50:00.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e3fff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400e9b0 tx=0x7fe5d400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4009710 con 0x7fe5e4071060 2023-06-09T11:50:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400c7f0 con 0x7fe5e4071060 2023-06-09T11:50:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.933+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400cb60 con 0x7fe5e4071060 2023-06-09T11:50:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.934+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4157770 con 0x7fe5e4071060 2023-06-09T11:50:00.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.934+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4157490 con 0x7fe5e4071060 2023-06-09T11:50:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.935+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4157a50 con 0x7fe5e4071060 2023-06-09T11:50:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.936+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe5d400ccc0 con 0x7fe5e4071060 2023-06-09T11:50:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.937+0000 7fe5e0ff9700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 0x7fe5d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:00.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.937+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fe5d408a1c0 con 0x7fe5e4071060 2023-06-09T11:50:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.938+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 0x7fe5d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:00.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.938+0000 7fe5e37fe700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 0x7fe5d006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8014040 tx=0x7fe5d8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:00.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:00.942+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe5d4058430 con 0x7fe5e4071060 2023-06-09T11:50:01.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.195+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5e41f0d80 con 0x7fe5e4071060 2023-06-09T11:50:01.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.195+0000 7fe5e0ff9700 1 -- 172.21.15.7:0/3591836824 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 187 v187) v1 ==== 95+0+5466 (secure 0 0 0) 0x7fe5d4021090 con 0x7fe5e4071060 2023-06-09T11:50:01.199 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:01.200 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":187,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":187,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:49:57.662725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:resolve","state_seq":25,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 msgr2=0x7fe5d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 0x7fe5d006d190 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8014040 tx=0x7fe5d8008030 comp rx=0 tx=0).stop 2023-06-09T11:50:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 msgr2=0x7fe5e40cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400e9b0 tx=0x7fe5d400ed70 comp rx=0 tx=0).stop 2023-06-09T11:50:01.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 shutdown_connections 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe5e415b380 0x7fe5e414bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5e4071060 0x7fe5e40cbb20 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe5d006ace0 0x7fe5d006d190 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 --2- 172.21.15.7:0/3591836824 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5e4151510 0x7fe5e4158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 >> 172.21.15.7:0/3591836824 conn(0x7fe5e406a4d0 msgr2=0x7fe5e4156c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 shutdown_connections 2023-06-09T11:50:01.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:01.198+0000 7fe5ea3ff700 1 -- 172.21.15.7:0/3591836824 wait complete. 2023-06-09T11:50:01.204 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 187 2023-06-09T11:50:01.220 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:03.221 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:03.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.444+0000 7f21143ff700 1 Processor -- start 2023-06-09T11:50:03.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.444+0000 7f21143ff700 1 -- start start 2023-06-09T11:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.445+0000 7f21143ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f210c151510 0x7f210c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.445+0000 7f21143ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f210c071230 0x7f210c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.445+0000 7f21143ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c1580a0 con 0x7f210c15b380 2023-06-09T11:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.445+0000 7f21143ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c1518e0 con 0x7f210c151510 2023-06-09T11:50:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.445+0000 7f21143ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c0c6180 con 0x7f210c071230 2023-06-09T11:50:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211199a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f210c15b380 0x7f210c070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59336/0 (socket says 172.21.15.7:59336) 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 --2- 172.21.15.7:0/68139240 >> v2:172.21.15.7:3300/0 conn(0x7f210c151510 0x7f210c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211199a700 1 -- 172.21.15.7:0/68139240 learned_addr learned my addr 172.21.15.7:0/68139240 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211299c700 1 --2- 172.21.15.7:0/68139240 >> v2:172.21.15.117:3300/0 conn(0x7f210c071230 0x7f210c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 -- 172.21.15.7:0/68139240 >> v2:172.21.15.117:3300/0 conn(0x7f210c071230 msgr2=0x7f210c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 --2- 172.21.15.7:0/68139240 >> v2:172.21.15.117:3300/0 conn(0x7f210c071230 0x7f210c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 -- 172.21.15.7:0/68139240 >> v1:172.21.15.7:6789/0 conn(0x7f210c15b380 legacy=0x7f210c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f2111199700 1 -- 172.21.15.7:0/68139240 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120506180 0 0) 0x7f210c1580a0 con 0x7f210c15b380 2023-06-09T11:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 -- 172.21.15.7:0/68139240 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c157490 con 0x7f210c151510 2023-06-09T11:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.446+0000 7f211219b700 1 --2- 172.21.15.7:0/68139240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c0c5b40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f210c06fec0 tx=0x7f2108009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e5e186e2e039d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f2111199700 1 -- 172.21.15.7:0/68139240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108011830 con 0x7f210c151510 2023-06-09T11:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f2111199700 1 -- 172.21.15.7:0/68139240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2108011990 con 0x7f210c151510 2023-06-09T11:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f2111199700 1 -- 172.21.15.7:0/68139240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2108011d20 con 0x7f210c151510 2023-06-09T11:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 -- 172.21.15.7:0/68139240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 msgr2=0x7f210c0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 --2- 172.21.15.7:0/68139240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c0c5b40 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f210c06fec0 tx=0x7f2108009b20 comp rx=0 tx=0).stop 2023-06-09T11:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 -- 172.21.15.7:0/68139240 shutdown_connections 2023-06-09T11:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 --2- 172.21.15.7:0/68139240 >> v2:172.21.15.117:3300/0 conn(0x7f210c071230 0x7f210c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 --2- 172.21.15.7:0/68139240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c0c5b40 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 -- 172.21.15.7:0/68139240 >> 172.21.15.7:0/68139240 conn(0x7f210c06a4d0 msgr2=0x7f210c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.447+0000 7f21143ff700 1 -- 172.21.15.7:0/68139240 shutdown_connections 2023-06-09T11:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.448+0000 7f21143ff700 1 -- 172.21.15.7:0/68139240 wait complete. 2023-06-09T11:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.450+0000 7f21143ff700 1 Processor -- start 2023-06-09T11:50:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.451+0000 7f21143ff700 1 -- start start 2023-06-09T11:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f210c071230 0x7f210c15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f210c15b380 0x7f210c0c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c14e080 con 0x7f210c151510 2023-06-09T11:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0cb100 con 0x7f210c15b380 2023-06-09T11:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f21143ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c1f1220 con 0x7f210c071230 2023-06-09T11:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f211199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.452+0000 7f211199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54486/0 (socket says 172.21.15.7:54486) 2023-06-09T11:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 -- 172.21.15.7:0/3374675012 learned_addr learned my addr 172.21.15.7:0/3374675012 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:03.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211219b700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f210c071230 0x7f210c15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211299c700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f210c15b380 0x7f210c0c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 -- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f210c071230 msgr2=0x7f210c15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f210c071230 0x7f210c15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 -- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f210c15b380 msgr2=0x7f210c0c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f210c15b380 0x7f210c0c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 -- 172.21.15.7:0/3374675012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21080094a0 con 0x7f210c151510 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f211199a700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c550 tx=0x7f20fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc004c80 con 0x7f210c151510 2023-06-09T11:50:03.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc004e00 con 0x7f210c151510 2023-06-09T11:50:03.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c157a50 con 0x7f210c151510 2023-06-09T11:50:03.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.453+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc007310 con 0x7f210c151510 2023-06-09T11:50:03.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.454+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c157770 con 0x7f210c151510 2023-06-09T11:50:03.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.455+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c157d30 con 0x7f210c151510 2023-06-09T11:50:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.456+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20fc007470 con 0x7f210c151510 2023-06-09T11:50:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.457+0000 7f2102ffd700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 0x7f20f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.457+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f20fc089a00 con 0x7f210c151510 2023-06-09T11:50:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.457+0000 7f211219b700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 0x7f20f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:03.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.458+0000 7f211219b700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 0x7f20f006d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f21080090b0 tx=0x7f2108008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:03.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.461+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f20fc054270 con 0x7f210c151510 2023-06-09T11:50:03.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.712+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f210c074cf0 con 0x7f210c151510 2023-06-09T11:50:03.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.712+0000 7f2102ffd700 1 -- 172.21.15.7:0/3374675012 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) v1 ==== 95+0+5468 (secure 0 0 0) 0x7f20fc057890 con 0x7f210c151510 2023-06-09T11:50:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:03.717 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":188,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:02.755175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 msgr2=0x7f20f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 0x7f20f006d190 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f21080090b0 tx=0x7f2108008030 comp rx=0 tx=0).stop 2023-06-09T11:50:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 msgr2=0x7f210c14d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c550 tx=0x7f20fc00c910 comp rx=0 tx=0).stop 2023-06-09T11:50:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 shutdown_connections 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f210c15b380 0x7f210c0c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20f006ace0 0x7f20f006d190 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f210c071230 0x7f210c15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 --2- 172.21.15.7:0/3374675012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c151510 0x7f210c14d9f0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 >> 172.21.15.7:0/3374675012 conn(0x7f210c06a4d0 msgr2=0x7f210c14c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 shutdown_connections 2023-06-09T11:50:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:03.715+0000 7f21143ff700 1 -- 172.21.15.7:0/3374675012 wait complete. 2023-06-09T11:50:03.721 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 188 2023-06-09T11:50:03.738 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 188, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 188, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_6650": { "addr": "172.21.15.117:6837/3744293936", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3744293936, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3744293936, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6650, "incarnation": 186, "join_fscid": -1, "name": "f", "rank": 2, "state": "up:reconnect", "state_seq": 27 }, "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 } }, "last_failure": 0, "last_failure_osd_epoch": 71, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:50:02.755175+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 6650 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:50:05.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.938+0000 7f18e0bff700 1 Processor -- start 2023-06-09T11:50:05.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.938+0000 7f18e0bff700 1 -- start start 2023-06-09T11:50:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.939+0000 7f18e0bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f18dc0c1bc0 0x7f18dc1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.939+0000 7f18e0bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.939+0000 7f18e0bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0c0bf0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.939+0000 7f18e0bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc070110 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.939+0000 7f18e0bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc071300 con 0x7f18dc0c0140 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18dad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18dc0c1f90 0x7f18dc1501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59346/0 (socket says 172.21.15.7:59346) 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18dad9d700 1 -- 172.21.15.7:0/4202063573 learned_addr learned my addr 172.21.15.7:0/4202063573 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18da59c700 1 --2- 172.21.15.7:0/4202063573 >> v2:172.21.15.117:3300/0 conn(0x7f18dc0c1bc0 0x7f18dc1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18d9d9b700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18d959a700 1 -- 172.21.15.7:0/4202063573 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523746434 0 0) 0x7f18dc0c0bf0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18d959a700 1 -- 172.21.15.7:0/4202063573 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c40035b0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18da59c700 1 -- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 msgr2=0x7f18dc159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18da59c700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18da59c700 1 -- 172.21.15.7:0/4202063573 >> v1:172.21.15.7:6789/0 conn(0x7f18dc0c1f90 legacy=0x7f18dc1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18da59c700 1 -- 172.21.15.7:0/4202063573 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0702f0 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.940+0000 7f18d9d9b700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18da59c700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1599b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009840 tx=0x7f18cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8555f39f86aff09 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18d959a700 1 -- 172.21.15.7:0/4202063573 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc011830 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18d959a700 1 -- 172.21.15.7:0/4202063573 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc011990 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18d959a700 1 -- 172.21.15.7:0/4202063573 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc011d20 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18e0bff700 1 -- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 msgr2=0x7f18dc1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.941+0000 7f18e0bff700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1599b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009840 tx=0x7f18cc009b50 comp rx=0 tx=0).stop 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 -- 172.21.15.7:0/4202063573 shutdown_connections 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1599b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 --2- 172.21.15.7:0/4202063573 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 -- 172.21.15.7:0/4202063573 >> 172.21.15.7:0/4202063573 conn(0x7f18dc06a4d0 msgr2=0x7f18dc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 -- 172.21.15.7:0/4202063573 shutdown_connections 2023-06-09T11:50:05.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.942+0000 7f18e0bff700 1 -- 172.21.15.7:0/4202063573 wait complete. 2023-06-09T11:50:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.943+0000 7f18e0bff700 1 Processor -- start 2023-06-09T11:50:05.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 -- start start 2023-06-09T11:50:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1ef280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0ae330 con 0x7f18dc0c1f90 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0cafc0 con 0x7f18dc0c1bc0 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.944+0000 7f18e0bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1eeaa0 con 0x7f18dc0c0140 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54494/0 (socket says 172.21.15.7:54494) 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 -- 172.21.15.7:0/1490056364 learned_addr learned my addr 172.21.15.7:0/1490056364 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:05.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18d9d9b700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18da59c700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 -- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 msgr2=0x7f18dc0c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc0c2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 -- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 msgr2=0x7f18dc1ef280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1ef280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.945+0000 7f18dad9d700 1 -- 172.21.15.7:0/1490056364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00094a0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18dad9d700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f18d400caa0 tx=0x7f18d400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400a9e0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d400ab40 con 0x7f18dc0c1f90 2023-06-09T11:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4012400 con 0x7f18dc0c1f90 2023-06-09T11:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0094a0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.946+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc0c1170 con 0x7f18dc0c1f90 2023-06-09T11:50:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.947+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc0c0e90 con 0x7f18dc0c1f90 2023-06-09T11:50:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.948+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18d401b3b0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.949+0000 7f18caffd700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 0x7f18bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.949+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f18d40899a0 con 0x7f18dc0c1f90 2023-06-09T11:50:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.949+0000 7f18da59c700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 0x7f18bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.950+0000 7f18da59c700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 0x7f18bc06d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:05.953+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f18d4054240 con 0x7f18dc0c1f90 2023-06-09T11:50:06.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.205+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18dc0c1450 con 0x7f18dc0c1f90 2023-06-09T11:50:06.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.205+0000 7f18caffd700 1 -- 172.21.15.7:0/1490056364 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 188 v188) v1 ==== 95+0+5468 (secure 0 0 0) 0x7f18d4017070 con 0x7f18dc0c1f90 2023-06-09T11:50:06.209 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:06.209 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":188,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":188,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:02.755175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:reconnect","state_seq":27,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.207+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 msgr2=0x7f18bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:06.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.207+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 0x7f18bc06d190 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000c00 tx=0x7f18cc008030 comp rx=0 tx=0).stop 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.207+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 msgr2=0x7f18dc0c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.207+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f18d400caa0 tx=0x7f18d400cdb0 comp rx=0 tx=0).stop 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 shutdown_connections 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f18dc0c1f90 0x7f18dc0c4d40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f18bc06ace0 0x7f18bc06d190 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:06.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f18dc0c0140 0x7f18dc0c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 --2- 172.21.15.7:0/1490056364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f18dc0c1bc0 0x7f18dc1ef280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 >> 172.21.15.7:0/1490056364 conn(0x7f18dc06a4d0 msgr2=0x7f18dc0c6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 shutdown_connections 2023-06-09T11:50:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:06.208+0000 7f18e0bff700 1 -- 172.21.15.7:0/1490056364 wait complete. 2023-06-09T11:50:06.213 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 188 2023-06-09T11:50:06.229 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:08.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.438+0000 7f8de21ff700 1 Processor -- start 2023-06-09T11:50:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.438+0000 7f8de21ff700 1 -- start start 2023-06-09T11:50:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8de21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 0x7f8ddc0c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8de21ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8de21ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0c88d0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8de21ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc0c2bf0 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8de21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc153e40 con 0x7f8ddc153a70 2023-06-09T11:50:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8ddaffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8ddbfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8ddc0ca7e0 0x7f8ddc0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:59356/0 (socket says 172.21.15.7:59356) 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8ddaffd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:51628/0 (socket says 172.21.15.7:51628) 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8ddbfff700 1 -- 172.21.15.7:0/1076395554 learned_addr learned my addr 172.21.15.7:0/1076395554 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.440+0000 7f8ddb7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 0x7f8ddc0c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddaffd700 1 -- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 msgr2=0x7f8ddc0c26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8dda7fc700 1 -- 172.21.15.7:0/1076395554 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686138266 0 0) 0x7f8ddc0c88d0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddaffd700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 0x7f8ddc0c26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddaffd700 1 -- 172.21.15.7:0/1076395554 >> v1:172.21.15.7:6789/0 conn(0x7f8ddc0ca7e0 legacy=0x7f8ddc0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddaffd700 1 -- 172.21.15.7:0/1076395554 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc15a630 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddb7fe700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 0x7f8ddc0c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8ddaffd700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009840 tx=0x7f8dd0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da8d6cf38bee1d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.441+0000 7f8dda7fc700 1 -- 172.21.15.7:0/1076395554 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0011830 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8dda7fc700 1 -- 172.21.15.7:0/1076395554 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd0011990 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8dda7fc700 1 -- 172.21.15.7:0/1076395554 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0011d20 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 -- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc0c2d90 msgr2=0x7f8ddc15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009840 tx=0x7f8dd0009b50 comp rx=0 tx=0).stop 2023-06-09T11:50:08.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 -- 172.21.15.7:0/1076395554 shutdown_connections 2023-06-09T11:50:08.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc153a70 0x7f8ddc0c26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 --2- 172.21.15.7:0/1076395554 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc0c2d90 0x7f8ddc15a0f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 -- 172.21.15.7:0/1076395554 >> 172.21.15.7:0/1076395554 conn(0x7f8ddc06e410 msgr2=0x7f8ddc06e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 -- 172.21.15.7:0/1076395554 shutdown_connections 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.442+0000 7f8de21ff700 1 -- 172.21.15.7:0/1076395554 wait complete. 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.444+0000 7f8de21ff700 1 Processor -- start 2023-06-09T11:50:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.444+0000 7f8de21ff700 1 -- start start 2023-06-09T11:50:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc0c2d90 0x7f8ddc151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc153a70 0x7f8ddc06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cd2d0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06bd30 con 0x7f8ddc153a70 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8de21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0cd520 con 0x7f8ddc0c2d90 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54504/0 (socket says 172.21.15.7:54504) 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8ddaffd700 1 -- 172.21.15.7:0/2876874418 learned_addr learned my addr 172.21.15.7:0/2876874418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8ddbfff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc153a70 0x7f8ddc06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.445+0000 7f8ddb7fe700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc0c2d90 0x7f8ddc151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 -- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc0c2d90 msgr2=0x7f8ddc151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc0c2d90 0x7f8ddc151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 -- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc153a70 msgr2=0x7f8ddc06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc153a70 0x7f8ddc06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 -- 172.21.15.7:0/2876874418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0094a0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8ddaffd700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00090b0 tx=0x7f8dd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0004550 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd001f070 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd00038e0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd00094a0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.446+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc0c8350 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.447+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc0ca350 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.448+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8dd0003a40 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.449+0000 7f8dc3fff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 0x7f8dc806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.449+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f8dd0021070 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.450+0000 7f8ddb7fe700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 0x7f8dc806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:08.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.450+0000 7f8ddb7fe700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 0x7f8dc806d1c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009cd0 tx=0x7f8dcc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:08.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.454+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8dd0055c40 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.710+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ddc0c85f0 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.710+0000 7f8dc3fff700 1 -- 172.21.15.7:0/2876874418 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f8dd002b090 con 0x7f8ddc0ca7e0 2023-06-09T11:50:08.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:08.715 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":189,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:08.677305+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:08.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 msgr2=0x7f8dc806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 0x7f8dc806d1c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009cd0 tx=0x7f8dcc009210 comp rx=0 tx=0).stop 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 msgr2=0x7f8ddc06c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00090b0 tx=0x7f8dd0004750 comp rx=0 tx=0).stop 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 shutdown_connections 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8ddc153a70 0x7f8ddc06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8dc806ad10 0x7f8dc806d1c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8ddc0c2d90 0x7f8ddc151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 --2- 172.21.15.7:0/2876874418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8ddc0ca7e0 0x7f8ddc06c1c0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 >> 172.21.15.7:0/2876874418 conn(0x7f8ddc06e410 msgr2=0x7f8ddc1e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 shutdown_connections 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:08.713+0000 7f8de21ff700 1 -- 172.21.15.7:0/2876874418 wait complete. 2023-06-09T11:50:08.718 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 189 2023-06-09T11:50:08.734 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:10.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.944+0000 7fe37a5ff700 1 Processor -- start 2023-06-09T11:50:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.944+0000 7fe37a5ff700 1 -- start start 2023-06-09T11:50:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.945+0000 7fe37a5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.945+0000 7fe37a5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe37415b4a0 0x7fe3740c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.945+0000 7fe37a5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37415a500 con 0x7fe3740c9f40 2023-06-09T11:50:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.945+0000 7fe37a5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740c5f10 con 0x7fe37415b4a0 2023-06-09T11:50:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.945+0000 7fe37a5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37415da00 con 0x7fe374153880 2023-06-09T11:50:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe378b9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe37415b4a0 0x7fe3740c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40806/0 (socket says 172.21.15.7:40806) 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 -- 172.21.15.7:0/1764582704 learned_addr learned my addr 172.21.15.7:0/1764582704 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe3737fe700 1 --1- 172.21.15.7:0/1764582704 >> v1:172.21.15.117:6789/0 conn(0x7fe3740c9f40 0x7fe37415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:54358/0 (socket says 172.21.15.7:54358) 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 -- 172.21.15.7:0/1764582704 >> v2:172.21.15.117:3300/0 conn(0x7fe37415b4a0 msgr2=0x7fe3740c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe372ffd700 1 -- 172.21.15.7:0/1764582704 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390296804 0 0) 0x7fe368001110 con 0x7fe3740c9f40 2023-06-09T11:50:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 --2- 172.21.15.7:0/1764582704 >> v2:172.21.15.117:3300/0 conn(0x7fe37415b4a0 0x7fe3740c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 -- 172.21.15.7:0/1764582704 >> v1:172.21.15.117:6789/0 conn(0x7fe3740c9f40 legacy=0x7fe37415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 -- 172.21.15.7:0/1764582704 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3741596e0 con 0x7fe374153880 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.946+0000 7fe373fff700 1 --2- 172.21.15.7:0/1764582704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3640098f0 tx=0x7fe364009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2c0618e9d9e609 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe372ffd700 1 -- 172.21.15.7:0/1764582704 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364011830 con 0x7fe374153880 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe372ffd700 1 -- 172.21.15.7:0/1764582704 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe364011990 con 0x7fe374153880 2023-06-09T11:50:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe372ffd700 1 -- 172.21.15.7:0/1764582704 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364011d20 con 0x7fe374153880 2023-06-09T11:50:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1764582704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 msgr2=0x7fe37415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1764582704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe3640098f0 tx=0x7fe364009c00 comp rx=0 tx=0).stop 2023-06-09T11:50:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1764582704 shutdown_connections 2023-06-09T11:50:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1764582704 >> v2:172.21.15.117:3300/0 conn(0x7fe37415b4a0 0x7fe3740c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1764582704 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe374153880 0x7fe37415d4c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1764582704 >> 172.21.15.7:0/1764582704 conn(0x7fe3740737f0 msgr2=0x7fe3740720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.947+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1764582704 shutdown_connections 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.948+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1764582704 wait complete. 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.949+0000 7fe37a5ff700 1 Processor -- start 2023-06-09T11:50:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.949+0000 7fe37a5ff700 1 -- start start 2023-06-09T11:50:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37415b4a0 0x7fe374158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3741571a0 con 0x7fe374153880 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740b21e0 con 0x7fe37415b4a0 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe37a5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740b1f40 con 0x7fe3740c9f40 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.950+0000 7fe3737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe3737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54518/0 (socket says 172.21.15.7:54518) 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe3737fe700 1 -- 172.21.15.7:0/1731156313 learned_addr learned my addr 172.21.15.7:0/1731156313 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:40810/0 (socket says 172.21.15.7:40810) 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe378b9c700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37415b4a0 0x7fe374158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 -- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37415b4a0 msgr2=0x7fe374158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37415b4a0 0x7fe374158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 -- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 msgr2=0x7fe374157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 -- 172.21.15.7:0/1731156313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe364009550 con 0x7fe3740c9f40 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe3737fe700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe373fff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe364016000 tx=0x7fe364004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.951+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364004380 con 0x7fe3740c9f40 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.952+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36401f070 con 0x7fe3740c9f40 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.952+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3640036a0 con 0x7fe3740c9f40 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.952+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe374159c60 con 0x7fe3740c9f40 2023-06-09T11:50:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.952+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe374159980 con 0x7fe3740c9f40 2023-06-09T11:50:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.953+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374159f40 con 0x7fe3740c9f40 2023-06-09T11:50:10.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.954+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe364021030 con 0x7fe3740c9f40 2023-06-09T11:50:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.954+0000 7fe370ff9700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 0x7fe36006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.955+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fe36408b120 con 0x7fe3740c9f40 2023-06-09T11:50:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.955+0000 7fe3737fe700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 0x7fe36006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:10.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.955+0000 7fe3737fe700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 0x7fe36006d1c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ce270 tx=0x7fe368007ce0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:10.960+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe364055ac0 con 0x7fe3740c9f40 2023-06-09T11:50:11.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.213+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe374074cf0 con 0x7fe3740c9f40 2023-06-09T11:50:11.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.213+0000 7fe370ff9700 1 -- 172.21.15.7:0/1731156313 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fe3640590e0 con 0x7fe3740c9f40 2023-06-09T11:50:11.217 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:11.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":189,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:08.677305+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 msgr2=0x7fe36006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 0x7fe36006d1c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe3740ce270 tx=0x7fe368007ce0 comp rx=0 tx=0).stop 2023-06-09T11:50:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 msgr2=0x7fe37414b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:11.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe364016000 tx=0x7fe364004750 comp rx=0 tx=0).stop 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 shutdown_connections 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe37415b4a0 0x7fe374158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe36006ad10 0x7fe36006d1c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe3740c9f40 0x7fe37414b540 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 --2- 172.21.15.7:0/1731156313 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe374153880 0x7fe374157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 >> 172.21.15.7:0/1731156313 conn(0x7fe3740737f0 msgr2=0x7fe37415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 shutdown_connections 2023-06-09T11:50:11.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:11.216+0000 7fe37a5ff700 1 -- 172.21.15.7:0/1731156313 wait complete. 2023-06-09T11:50:11.222 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 189 2023-06-09T11:50:11.238 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:13.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.445+0000 7f7bc93ff700 1 Processor -- start 2023-06-09T11:50:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.445+0000 7f7bc93ff700 1 -- start start 2023-06-09T11:50:13.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc93ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7bc415b4a0 0x7f7bc415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:13.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc93ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc415a500 con 0x7f7bc4153880 2023-06-09T11:50:13.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc93ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc415da00 con 0x7f7bc40c9f40 2023-06-09T11:50:13.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc93ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc40c5f10 con 0x7f7bc415b4a0 2023-06-09T11:50:13.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bc4153880 0x7f7bc415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54608/0 (socket says 172.21.15.7:54608) 2023-06-09T11:50:13.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc2ffd700 1 -- 172.21.15.7:0/1855007979 learned_addr learned my addr 172.21.15.7:0/1855007979 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:13.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/1855007979 >> v2:172.21.15.7:3300/0 conn(0x7f7bc415b4a0 0x7f7bc415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:13.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc1ffb700 1 -- 172.21.15.7:0/1855007979 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033367812 0 0) 0x7f7bb4001110 con 0x7f7bc4153880 2023-06-09T11:50:13.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc27fc700 1 -- 172.21.15.7:0/1855007979 >> v1:172.21.15.117:6789/0 conn(0x7f7bc40c9f40 legacy=0x7f7bc40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc27fc700 1 -- 172.21.15.7:0/1855007979 >> v1:172.21.15.7:6789/0 conn(0x7f7bc4153880 legacy=0x7f7bc415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.447+0000 7f7bc27fc700 1 -- 172.21.15.7:0/1855007979 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc41596e0 con 0x7f7bc415b4a0 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/1855007979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc415b4a0 0x7f7bc415d4c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009840 tx=0x7f7bb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2953903db62e65 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc1ffb700 1 -- 172.21.15.7:0/1855007979 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8011830 con 0x7f7bc415b4a0 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc1ffb700 1 -- 172.21.15.7:0/1855007979 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb8011990 con 0x7f7bc415b4a0 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc1ffb700 1 -- 172.21.15.7:0/1855007979 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8011d20 con 0x7f7bc415b4a0 2023-06-09T11:50:13.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc93ff700 1 -- 172.21.15.7:0/1855007979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc415b4a0 msgr2=0x7f7bc415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.448+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/1855007979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc415b4a0 0x7f7bc415d4c0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009840 tx=0x7f7bb8009b50 comp rx=0 tx=0).stop 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.449+0000 7f7bc93ff700 1 -- 172.21.15.7:0/1855007979 shutdown_connections 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.449+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/1855007979 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc415b4a0 0x7f7bc415d4c0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.449+0000 7f7bc93ff700 1 -- 172.21.15.7:0/1855007979 >> 172.21.15.7:0/1855007979 conn(0x7f7bc40737f0 msgr2=0x7f7bc40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.449+0000 7f7bc93ff700 1 -- 172.21.15.7:0/1855007979 shutdown_connections 2023-06-09T11:50:13.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.449+0000 7f7bc93ff700 1 -- 172.21.15.7:0/1855007979 wait complete. 2023-06-09T11:50:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.451+0000 7f7bc93ff700 1 Processor -- start 2023-06-09T11:50:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.451+0000 7f7bc93ff700 1 -- start start 2023-06-09T11:50:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 0x7f7bc4149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc415b4a0 0x7f7bc40cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40ced20 con 0x7f7bc40c9f40 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc414be40 con 0x7f7bc4153880 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.452+0000 7f7bc93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc40cacb0 con 0x7f7bc415b4a0 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38364/0 (socket says 172.21.15.7:38364) 2023-06-09T11:50:13.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 -- 172.21.15.7:0/242709671 learned_addr learned my addr 172.21.15.7:0/242709671 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 0x7f7bc4149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:13.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc37fe700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc415b4a0 0x7f7bc40cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 -- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc415b4a0 msgr2=0x7f7bc40cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc415b4a0 0x7f7bc40cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 -- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 msgr2=0x7f7bc4149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 0x7f7bc4149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 -- 172.21.15.7:0/242709671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80094a0 con 0x7f7bc40c9f40 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 0x7f7bc4149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:50:13.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.453+0000 7f7bc2ffd700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c550 tx=0x7f7bb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.454+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4004c80 con 0x7f7bc40c9f40 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.454+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bb4004de0 con 0x7f7bc40c9f40 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.454+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4007310 con 0x7f7bc40c9f40 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.454+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bc4159c60 con 0x7f7bc40c9f40 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.454+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc4159980 con 0x7f7bc40c9f40 2023-06-09T11:50:13.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.455+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc4159f40 con 0x7f7bc40c9f40 2023-06-09T11:50:13.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.456+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7bb4007470 con 0x7f7bc40c9f40 2023-06-09T11:50:13.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.457+0000 7f7bab7fe700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 0x7f7bb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:13.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.457+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f7bb4089860 con 0x7f7bc40c9f40 2023-06-09T11:50:13.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.457+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 0x7f7bb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:13.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.457+0000 7f7bc27fc700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 0x7f7bb006d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80090b0 tx=0x7f7bb801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:13.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.461+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7bb40540d0 con 0x7f7bc40c9f40 2023-06-09T11:50:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.721+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bc415a500 con 0x7f7bc40c9f40 2023-06-09T11:50:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.722+0000 7f7bab7fe700 1 -- 172.21.15.7:0/242709671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 189 v189) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f7bc415a500 con 0x7f7bc40c9f40 2023-06-09T11:50:13.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:13.726 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":189,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":189,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:08.677305+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:rejoin","state_seq":29,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 msgr2=0x7f7bb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 0x7f7bb006d190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7bb80090b0 tx=0x7f7bb801c040 comp rx=0 tx=0).stop 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 msgr2=0x7f7bc40cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f7bb400c550 tx=0x7f7bb400c910 comp rx=0 tx=0).stop 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 shutdown_connections 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.724+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7bb006ace0 0x7f7bb006d190 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bc415b4a0 0x7f7bc40cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bc40c9f40 0x7f7bc40cefe0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 --2- 172.21.15.7:0/242709671 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bc4153880 0x7f7bc4149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 >> 172.21.15.7:0/242709671 conn(0x7f7bc40737f0 msgr2=0x7f7bc415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 shutdown_connections 2023-06-09T11:50:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:13.725+0000 7f7bc93ff700 1 -- 172.21.15.7:0/242709671 wait complete. 2023-06-09T11:50:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 189 2023-06-09T11:50:13.746 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:50:15.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:15.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.958+0000 7fa5753ff700 1 Processor -- start 2023-06-09T11:50:15.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.958+0000 7fa5753ff700 1 -- start start 2023-06-09T11:50:15.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.959+0000 7fa5753ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 0x7fa57015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.959+0000 7fa5753ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa5700c9f40 0x7fa57015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.959+0000 7fa5753ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa57015b2d0 0x7fa5700c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa5753ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5700c5f10 con 0x7fa57015b2d0 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa5753ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57015a500 con 0x7fa5700c9f40 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa5753ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015da00 con 0x7fa570153880 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa57015b2d0 0x7fa5700c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa57015b2d0 0x7fa5700c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38380/0 (socket says 172.21.15.7:38380) 2023-06-09T11:50:15.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 -- 172.21.15.7:0/2919072132 learned_addr learned my addr 172.21.15.7:0/2919072132 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56e7fc700 1 --2- 172.21.15.7:0/2919072132 >> v2:172.21.15.117:3300/0 conn(0x7fa5700c9f40 0x7fa57015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56effd700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 0x7fa57015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 -- 172.21.15.7:0/2919072132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 msgr2=0x7fa57015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 0x7fa57015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 -- 172.21.15.7:0/2919072132 >> v2:172.21.15.117:3300/0 conn(0x7fa5700c9f40 msgr2=0x7fa57015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/2919072132 >> v2:172.21.15.117:3300/0 conn(0x7fa5700c9f40 0x7fa57015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56f7fe700 1 -- 172.21.15.7:0/2919072132 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57015b750 con 0x7fa57015b2d0 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.960+0000 7fa56effd700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 0x7fa57015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa5700c4e30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa5700cd140 tx=0x7fa56800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bbda6bb712dadc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa56dffb700 1 -- 172.21.15.7:0/2919072132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800e070 con 0x7fa57015b2d0 2023-06-09T11:50:15.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa56dffb700 1 -- 172.21.15.7:0/2919072132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800b910 con 0x7fa57015b2d0 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa56dffb700 1 -- 172.21.15.7:0/2919072132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800bcc0 con 0x7fa57015b2d0 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa5753ff700 1 -- 172.21.15.7:0/2919072132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 msgr2=0x7fa5700c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.961+0000 7fa5753ff700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa5700c4e30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa5700cd140 tx=0x7fa56800b330 comp rx=0 tx=0).stop 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 -- 172.21.15.7:0/2919072132 shutdown_connections 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa5700c4e30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 --2- 172.21.15.7:0/2919072132 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa570153880 0x7fa57015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 --2- 172.21.15.7:0/2919072132 >> v2:172.21.15.117:3300/0 conn(0x7fa5700c9f40 0x7fa57015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 -- 172.21.15.7:0/2919072132 >> 172.21.15.7:0/2919072132 conn(0x7fa5700737f0 msgr2=0x7fa5700720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 -- 172.21.15.7:0/2919072132 shutdown_connections 2023-06-09T11:50:15.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.962+0000 7fa5753ff700 1 -- 172.21.15.7:0/2919072132 wait complete. 2023-06-09T11:50:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.963+0000 7fa5753ff700 1 Processor -- start 2023-06-09T11:50:15.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.964+0000 7fa5753ff700 1 -- start start 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 0x7fa5700cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa570153880 0x7fa5701ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700b2250 con 0x7fa57015b2d0 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701f3250 con 0x7fa570153880 2023-06-09T11:50:15.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa5753ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5701f36e0 con 0x7fa5700c9f40 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 0x7fa5700cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56e7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa570153880 0x7fa5701ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 0x7fa5700cbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:53460/0 (socket says 172.21.15.7:53460) 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38394/0 (socket says 172.21.15.7:38394) 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56effd700 1 -- 172.21.15.7:0/3170378933 learned_addr learned my addr 172.21.15.7:0/3170378933 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 -- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 msgr2=0x7fa5700cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 0x7fa5700cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 -- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa570153880 msgr2=0x7fa5701ef310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa570153880 0x7fa5701ef310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.965+0000 7fa56f7fe700 1 -- 172.21.15.7:0/3170378933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa568009b30 con 0x7fa57015b2d0 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa56f7fe700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa568015000 tx=0x7fa568004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680099b0 con 0x7fa57015b2d0 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680092e0 con 0x7fa57015b2d0 2023-06-09T11:50:15.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568004470 con 0x7fa57015b2d0 2023-06-09T11:50:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa570159980 con 0x7fa57015b2d0 2023-06-09T11:50:15.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.966+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701596e0 con 0x7fa57015b2d0 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.967+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa570159c60 con 0x7fa57015b2d0 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.968+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5680045d0 con 0x7fa57015b2d0 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.969+0000 7fa5577fe700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 0x7fa55c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.969+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fa56808aba0 con 0x7fa57015b2d0 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.969+0000 7fa56effd700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 0x7fa55c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.970+0000 7fa56effd700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 0x7fa55c06d1c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa560009a90 tx=0x7fa560008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:15.974+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa568058f30 con 0x7fa57015b2d0 2023-06-09T11:50:16.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.225+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa570074cf0 con 0x7fa57015b2d0 2023-06-09T11:50:16.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.226+0000 7fa5577fe700 1 -- 172.21.15.7:0/3170378933 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) v1 ==== 95+0+5465 (secure 0 0 0) 0x7fa56801f020 con 0x7fa57015b2d0 2023-06-09T11:50:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:16.230 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":190,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:13.833315+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:16.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 msgr2=0x7fa55c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:16.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 0x7fa55c06d1c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa560009a90 tx=0x7fa560008040 comp rx=0 tx=0).stop 2023-06-09T11:50:16.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 msgr2=0x7fa570152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:16.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa568015000 tx=0x7fa568004ad0 comp rx=0 tx=0).stop 2023-06-09T11:50:16.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 shutdown_connections 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa57015b2d0 0x7fa570152140 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa55c06ad10 0x7fa55c06d1c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa5700c9f40 0x7fa5700cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 --2- 172.21.15.7:0/3170378933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa570153880 0x7fa5701ef310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 >> 172.21.15.7:0/3170378933 conn(0x7fa5700737f0 msgr2=0x7fa5701507b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 shutdown_connections 2023-06-09T11:50:16.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:16.228+0000 7fa5753ff700 1 -- 172.21.15.7:0/3170378933 wait complete. 2023-06-09T11:50:16.234 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 190 2023-06-09T11:50:16.250 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=2, replacing gid=7279 2023-06-09T11:50:16.250 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 6 secs before reviving mds.b 2023-06-09T11:50:22.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:22.045 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:22.251 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.b 2023-06-09T11:50:22.251 INFO:tasks.ceph.mds.b:Restarting daemon 2023-06-09T11:50:22.251 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-06-09T11:50:22.255 INFO:tasks.ceph.mds.b:Started 2023-06-09T11:50:22.256 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:22.396 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:50:22.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.454+0000 7f2148dff700 1 Processor -- start 2023-06-09T11:50:22.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.454+0000 7f2148dff700 1 -- start start 2023-06-09T11:50:22.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2148dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 0x7f21440c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2148dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f214406e460 0x7f214406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2148dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f214415cac0 con 0x7f214406a4d0 2023-06-09T11:50:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2148dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21440c8870 con 0x7f214406e460 2023-06-09T11:50:22.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2148dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21440c37d0 con 0x7f21440c6280 2023-06-09T11:50:22.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f213bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f214406a4d0 0x7f21440737f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54614/0 (socket says 172.21.15.7:54614) 2023-06-09T11:50:22.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2142d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f214406e460 0x7f214406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f213bfff700 1 -- 172.21.15.7:0/1668475906 learned_addr learned my addr 172.21.15.7:0/1668475906 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:22.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f214259c700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 0x7f21440c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.456+0000 7f2141d9b700 1 -- 172.21.15.7:0/1668475906 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319862074 0 0) 0x7f214415cac0 con 0x7f214406a4d0 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2142d9d700 1 -- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 msgr2=0x7f21440c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2142d9d700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 0x7f21440c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2142d9d700 1 -- 172.21.15.7:0/1668475906 >> v1:172.21.15.7:6789/0 conn(0x7f214406a4d0 legacy=0x7f21440737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2142d9d700 1 -- 172.21.15.7:0/1668475906 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214415cd30 con 0x7f214406e460 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f214259c700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 0x7f21440c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2142d9d700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406e460 0x7f214406cf30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f213c009d90 tx=0x7f213c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=415151e07e68d5fe server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2141d9b700 1 -- 172.21.15.7:0/1668475906 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c00e070 con 0x7f214406e460 2023-06-09T11:50:22.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2141d9b700 1 -- 172.21.15.7:0/1668475906 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c00bb20 con 0x7f214406e460 2023-06-09T11:50:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.457+0000 7f2141d9b700 1 -- 172.21.15.7:0/1668475906 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c012430 con 0x7f214406e460 2023-06-09T11:50:22.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 -- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406e460 msgr2=0x7f214406cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406e460 0x7f214406cf30 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f213c009d90 tx=0x7f213c00b330 comp rx=0 tx=0).stop 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 -- 172.21.15.7:0/1668475906 shutdown_connections 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406e460 0x7f214406cf30 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 --2- 172.21.15.7:0/1668475906 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21440c6280 0x7f21440c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 -- 172.21.15.7:0/1668475906 >> 172.21.15.7:0/1668475906 conn(0x7f214406f980 msgr2=0x7f21440cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 -- 172.21.15.7:0/1668475906 shutdown_connections 2023-06-09T11:50:22.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.458+0000 7f2148dff700 1 -- 172.21.15.7:0/1668475906 wait complete. 2023-06-09T11:50:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.460+0000 7f2148dff700 1 Processor -- start 2023-06-09T11:50:22.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.460+0000 7f2148dff700 1 -- start start 2023-06-09T11:50:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406a4d0 0x7f2144153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f214406e460 0x7f21441f2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21441421d0 con 0x7f21440c6280 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214406e0b0 con 0x7f214406a4d0 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2148dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21441f2930 con 0x7f214406e460 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2142d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38396/0 (socket says 172.21.15.7:38396) 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f2142d9d700 1 -- 172.21.15.7:0/1561933734 learned_addr learned my addr 172.21.15.7:0/1561933734 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f214259c700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406a4d0 0x7f2144153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.461+0000 7f213bfff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f214406e460 0x7f21441f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 -- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f214406e460 msgr2=0x7f21441f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f214406e460 0x7f21441f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 -- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406a4d0 msgr2=0x7f2144153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406a4d0 0x7f2144153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 -- 172.21.15.7:0/1561933734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340094a0 con 0x7f21440c6280 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2142d9d700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2144068c30 tx=0x7f213c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c012430 con 0x7f21440c6280 2023-06-09T11:50:22.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c020070 con 0x7f21440c6280 2023-06-09T11:50:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c009a70 con 0x7f21440c6280 2023-06-09T11:50:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c004280 con 0x7f21440c6280 2023-06-09T11:50:22.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.462+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f214415d2f0 con 0x7f21440c6280 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.464+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214415d010 con 0x7f21440c6280 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.464+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f213c0043e0 con 0x7f21440c6280 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.465+0000 7f213affd700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 0x7f212406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.465+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f213c022070 con 0x7f21440c6280 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.466+0000 7f214259c700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 0x7f212406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:22.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.466+0000 7f214259c700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 0x7f212406d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2134009cd0 tx=0x7f2134009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.470+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f213c056580 con 0x7f21440c6280 2023-06-09T11:50:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.720+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f214415d5d0 con 0x7f21440c6280 2023-06-09T11:50:22.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.720+0000 7f213affd700 1 -- 172.21.15.7:0/1561933734 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 190 v190) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f213c059ba0 con 0x7f21440c6280 2023-06-09T11:50:22.724 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:22.724 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":190,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":190,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:13.833315+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 msgr2=0x7f212406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 0x7f212406d190 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2134009cd0 tx=0x7f2134009210 comp rx=0 tx=0).stop 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 msgr2=0x7f21441f1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f2144068c30 tx=0x7f213c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 shutdown_connections 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21440c6280 0x7f21441f1260 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f214406a4d0 0x7f2144153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f212406ace0 0x7f212406d190 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.722+0000 7f2148dff700 1 --2- 172.21.15.7:0/1561933734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f214406e460 0x7f21441f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.723+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 >> 172.21.15.7:0/1561933734 conn(0x7f214406f980 msgr2=0x7f214415c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.723+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 shutdown_connections 2023-06-09T11:50:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:22.723+0000 7f2148dff700 1 -- 172.21.15.7:0/1561933734 wait complete. 2023-06-09T11:50:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 190 2023-06-09T11:50:22.744 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.b is in active, standby or standby-replay 2023-06-09T11:50:24.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:50:24.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.970+0000 7fe561bff700 1 Processor -- start 2023-06-09T11:50:24.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.970+0000 7fe561bff700 1 -- start start 2023-06-09T11:50:24.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.971+0000 7fe561bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe55c151510 0x7fe55c0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.971+0000 7fe561bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe55c071230 0x7fe55c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.971+0000 7fe561bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c1580a0 con 0x7fe55c15b380 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.971+0000 7fe561bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c1518e0 con 0x7fe55c071230 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.971+0000 7fe561bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0c6180 con 0x7fe55c151510 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe55c071230 0x7fe55c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55affd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe55c15b380 0x7fe55c070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35472/0 (socket says 172.21.15.7:35472) 2023-06-09T11:50:24.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe55c071230 0x7fe55c158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56592/0 (socket says 172.21.15.7:56592) 2023-06-09T11:50:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55affd700 1 -- 172.21.15.7:0/2442534143 learned_addr learned my addr 172.21.15.7:0/2442534143 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:50:24.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55b7fe700 1 --2- 172.21.15.7:0/2442534143 >> v2:172.21.15.117:3300/0 conn(0x7fe55c151510 0x7fe55c0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 -- 172.21.15.7:0/2442534143 >> v2:172.21.15.117:3300/0 conn(0x7fe55c151510 msgr2=0x7fe55c0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:50:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 --2- 172.21.15.7:0/2442534143 >> v2:172.21.15.117:3300/0 conn(0x7fe55c151510 0x7fe55c0c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 -- 172.21.15.7:0/2442534143 >> v1:172.21.15.7:6789/0 conn(0x7fe55c15b380 legacy=0x7fe55c070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55bfff700 1 -- 172.21.15.7:0/2442534143 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c157490 con 0x7fe55c071230 2023-06-09T11:50:24.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.972+0000 7fe55a7fc700 1 -- 172.21.15.7:0/2442534143 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142634673 0 0) 0x7fe55c1580a0 con 0x7fe55c15b380 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe55bfff700 1 --2- 172.21.15.7:0/2442534143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c071230 0x7fe55c158870 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe55c06ff10 tx=0x7fe55400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f038746d53e978b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe55a7fc700 1 -- 172.21.15.7:0/2442534143 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55400e070 con 0x7fe55c071230 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe55a7fc700 1 -- 172.21.15.7:0/2442534143 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55400bb20 con 0x7fe55c071230 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe55a7fc700 1 -- 172.21.15.7:0/2442534143 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554012430 con 0x7fe55c071230 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 -- 172.21.15.7:0/2442534143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c071230 msgr2=0x7fe55c158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 --2- 172.21.15.7:0/2442534143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c071230 0x7fe55c158870 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe55c06ff10 tx=0x7fe55400b330 comp rx=0 tx=0).stop 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 -- 172.21.15.7:0/2442534143 shutdown_connections 2023-06-09T11:50:24.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 --2- 172.21.15.7:0/2442534143 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c071230 0x7fe55c158870 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 --2- 172.21.15.7:0/2442534143 >> v2:172.21.15.117:3300/0 conn(0x7fe55c151510 0x7fe55c0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 -- 172.21.15.7:0/2442534143 >> 172.21.15.7:0/2442534143 conn(0x7fe55c06a4d0 msgr2=0x7fe55c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 -- 172.21.15.7:0/2442534143 shutdown_connections 2023-06-09T11:50:24.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.973+0000 7fe561bff700 1 -- 172.21.15.7:0/2442534143 wait complete. 2023-06-09T11:50:24.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.975+0000 7fe561bff700 1 Processor -- start 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.975+0000 7fe561bff700 1 -- start start 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe55c071230 0x7fe55c158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55c15b380 0x7fe55c0c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c1e3540 con 0x7fe55c151510 2023-06-09T11:50:24.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0c0510 con 0x7fe55c071230 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe561bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0c07f0 con 0x7fe55c15b380 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe55affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.976+0000 7fe55affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56606/0 (socket says 172.21.15.7:56606) 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 -- 172.21.15.7:0/3779151453 learned_addr learned my addr 172.21.15.7:0/3779151453 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55b7fe700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe55c071230 0x7fe55c158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55bfff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55c15b380 0x7fe55c0c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 -- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55c15b380 msgr2=0x7fe55c0c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:24.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55c15b380 0x7fe55c0c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 -- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe55c071230 msgr2=0x7fe55c158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe55c071230 0x7fe55c158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 -- 172.21.15.7:0/3779151453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe554009a70 con 0x7fe55c151510 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe55affd700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c550 tx=0x7fe55000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550004c80 con 0x7fe55c151510 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550004e00 con 0x7fe55c151510 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550007310 con 0x7fe55c151510 2023-06-09T11:50:24.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.977+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c157a50 con 0x7fe55c151510 2023-06-09T11:50:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.978+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c157770 con 0x7fe55c151510 2023-06-09T11:50:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.979+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c157d30 con 0x7fe55c151510 2023-06-09T11:50:24.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.980+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe550007470 con 0x7fe55c151510 2023-06-09T11:50:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.981+0000 7fe543fff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:50:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.981+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fe5500897f0 con 0x7fe55c151510 2023-06-09T11:50:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.981+0000 7fe55b7fe700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:50:24.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.981+0000 7fe55b7fe700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 0x7fe54806d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009be0 tx=0x7fe54c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:50:24.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:24.985+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe5500540f0 con 0x7fe55c151510 2023-06-09T11:50:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.239+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe55c074cf0 con 0x7fe55c151510 2023-06-09T11:50:25.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.240+0000 7fe543fff700 1 -- 172.21.15.7:0/3779151453 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 192 v192) v1 ==== 95+0+6226 (secure 0 0 0) 0x7fe550057710 con 0x7fe55c151510 2023-06-09T11:50:25.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:50:25.244 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":192,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":192,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:24.407366+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:50:25.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 msgr2=0x7fe54806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:25.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 0x7fe54806d190 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009be0 tx=0x7fe54c009210 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 msgr2=0x7fe55c0c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe55000c550 tx=0x7fe55000c860 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 shutdown_connections 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe54806ace0 0x7fe54806d190 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe55c15b380 0x7fe55c0c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe55c071230 0x7fe55c158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 --2- 172.21.15.7:0/3779151453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe55c151510 0x7fe55c0c7e70 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:50:25.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.242+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 >> 172.21.15.7:0/3779151453 conn(0x7fe55c06a4d0 msgr2=0x7fe55c14cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:50:25.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.243+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 shutdown_connections 2023-06-09T11:50:25.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:50:25.243+0000 7fe561bff700 1 -- 172.21.15.7:0/3779151453 wait complete. 2023-06-09T11:50:25.248 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 192 2023-06-09T11:50:25.264 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b reported in up:standby state 2023-06-09T11:50:25.264 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 118 secs before thrashing 2023-06-09T11:50:26.363 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 11% (26/217) Updating files: 12% (27/217) Updating files: 13% (29/217) Updating files: 14% (31/217) Updating files: 15% (33/217) Updating files: 16% (35/217) Updating files: 17% (37/217) Updating files: 18% (40/217) Updating files: 19% (42/217) Updating files: 20% (44/217) Updating files: 21% (46/217) Updating files: 22% (48/217) Updating files: 22% (49/217) Updating files: 23% (50/217) Updating files: 24% (53/217) Updating files: 25% (55/217) Updating files: 26% (57/217) Updating files: 27% (59/217) Updating files: 28% (61/217) Updating files: 29% (63/217) Updating files: 30% (66/217) Updating files: 31% (68/217) Updating files: 32% (70/217) Updating files: 33% (72/217) Updating files: 34% (74/217) Updating files: 35% (76/217) Updating files: 36% (79/217) Updating files: 37% (81/217) Updating files: 38% (83/217) Updating files: 39% (85/217) Updating files: 40% (87/217) Updating files: 41% (89/217) Updating files: 42% (92/217) Updating files: 43% (94/217) Updating files: 44% (96/217) Updating files: 45% (98/217) Updating files: 46% (100/217) Updating files: 47% (102/217) Updating files: 48% (105/217) Updating files: 49% (107/217) Updating files: 50% (109/217) Updating files: 51% (111/217) Updating files: 52% (113/217) Updating files: 53% (116/217) Updating files: 54% (118/217) Updating files: 55% (120/217) Updating files: 56% (122/217) Updating files: 57% (124/217) Updating files: 58% (126/217) Updating files: 59% (129/217) Updating files: 60% (131/217) Updating files: 61% (133/217) Updating files: 62% (135/217) Updating files: 63% (137/217) Updating files: 64% (139/217) Updating files: 65% (142/217) Updating files: 66% (144/217) Updating files: 67% (146/217) Updating files: 68% (148/217) Updating files: 69% (150/217) Updating files: 70% (152/217) Updating files: 71% (155/217) Updating files: 72% (157/217) Updating files: 73% (159/217) Updating files: 74% (161/217) Updating files: 75% (163/217) Updating files: 76% (165/217) Updating files: 77% (168/217) Updating files: 78% (170/217) Updating files: 79% (172/217) Updating files: 80% (174/217) Updating files: 81% (176/217) Updating files: 82% (178/217) Updating files: 83% (181/217) Updating files: 84% (183/217) Updating files: 85% (185/217) Updating files: 86% (187/217) Updating files: 87% (189/217) Updating files: 88% (191/217) Updating files: 89% (194/217) Updating files: 90% (196/217) Updating files: 91% (198/217) Updating files: 92% (200/217) Updating files: 93% (202/217) Updating files: 94% (204/217) Updating files: 95% (207/217) Updating files: 96% (209/217) Updating files: 97% (211/217) Updating files: 98% (213/217) Updating files: 99% (215/217) Updating files: 100% (217/217) Updating files: 100% (217/217), done. 2023-06-09T11:50:26.511 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 98f5be53f85 v0.19 2023-06-09T11:50:26.516 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.19 2023-06-09T11:50:26.552 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:50:26.552 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 20 -eq 48 ']' 2023-06-09T11:50:26.552 INFO:tasks.workunit.client.0.smithi007.stdout:v0.20 2023-06-09T11:50:26.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.20 2023-06-09T11:50:26.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.20 2023-06-09T11:50:26.553 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.20 2023-06-09T11:50:36.975 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 23% (58/245) Updating files: 24% (59/245) Updating files: 25% (62/245) Updating files: 26% (64/245) Updating files: 27% (67/245) Updating files: 28% (69/245) Updating files: 29% (72/245) Updating files: 30% (74/245) Updating files: 31% (76/245) Updating files: 32% (79/245) Updating files: 33% (81/245) Updating files: 34% (84/245) Updating files: 35% (86/245) Updating files: 36% (89/245) Updating files: 37% (91/245) Updating files: 37% (93/245) Updating files: 38% (94/245) Updating files: 39% (96/245) Updating files: 40% (98/245) Updating files: 41% (101/245) Updating files: 42% (103/245) Updating files: 43% (106/245) Updating files: 44% (108/245) Updating files: 45% (111/245) Updating files: 46% (113/245) Updating files: 46% (115/245) Updating files: 47% (116/245) Updating files: 48% (118/245) Updating files: 49% (121/245) Updating files: 50% (123/245) Updating files: 51% (125/245) Updating files: 52% (128/245) Updating files: 53% (130/245) Updating files: 54% (133/245) Updating files: 55% (135/245) Updating files: 56% (138/245) Updating files: 57% (140/245) Updating files: 58% (143/245) Updating files: 59% (145/245) Updating files: 60% (147/245) Updating files: 61% (150/245) Updating files: 62% (152/245) Updating files: 63% (155/245) Updating files: 64% (157/245) Updating files: 65% (160/245) Updating files: 66% (162/245) Updating files: 67% (165/245) Updating files: 68% (167/245) Updating files: 69% (170/245) Updating files: 70% (172/245) Updating files: 70% (173/245) Updating files: 71% (174/245) Updating files: 72% (177/245) Updating files: 73% (179/245) Updating files: 74% (182/245) Updating files: 75% (184/245) Updating files: 76% (187/245) Updating files: 77% (189/245) Updating files: 78% (192/245) Updating files: 79% (194/245) Updating files: 80% (196/245) Updating files: 81% (199/245) Updating files: 82% (201/245) Updating files: 83% (204/245) Updating files: 84% (206/245) Updating files: 85% (209/245) Updating files: 86% (211/245) Updating files: 87% (214/245) Updating files: 88% (216/245) Updating files: 88% (218/245) Updating files: 89% (219/245) Updating files: 90% (221/245) Updating files: 90% (222/245) Updating files: 91% (223/245) Updating files: 92% (226/245) Updating files: 93% (228/245) Updating files: 94% (231/245) Updating files: 95% (233/245) Updating files: 96% (236/245) Updating files: 97% (238/245) Updating files: 98% (241/245) Updating files: 99% (243/245) Updating files: 100% (245/245) Updating files: 100% (245/245), done. 2023-06-09T11:50:37.171 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 266d2076b6c debian: fix up debian scripts 2023-06-09T11:50:37.175 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.20 2023-06-09T11:50:37.211 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:50:37.212 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 21 -eq 48 ']' 2023-06-09T11:50:37.212 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.21 2023-06-09T11:50:37.212 INFO:tasks.workunit.client.0.smithi007.stdout:v0.21 2023-06-09T11:50:37.212 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.21 2023-06-09T11:50:37.213 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.21 2023-06-09T11:50:52.152 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:52.156 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:50:56.566 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 7% (41/537) Updating files: 8% (43/537) Updating files: 9% (49/537) Updating files: 10% (54/537) Updating files: 11% (60/537) Updating files: 11% (61/537) Updating files: 12% (65/537) Updating files: 13% (70/537) Updating files: 14% (76/537) Updating files: 15% (81/537) Updating files: 16% (86/537) Updating files: 17% (92/537) Updating files: 18% (97/537) Updating files: 19% (103/537) Updating files: 20% (108/537) Updating files: 21% (113/537) Updating files: 22% (119/537) Updating files: 22% (121/537) Updating files: 23% (124/537) Updating files: 24% (129/537) Updating files: 25% (135/537) Updating files: 25% (138/537) Updating files: 26% (140/537) Updating files: 27% (145/537) Updating files: 28% (151/537) Updating files: 29% (156/537) Updating files: 30% (162/537) Updating files: 31% (167/537) Updating files: 32% (172/537) Updating files: 33% (178/537) Updating files: 34% (183/537) Updating files: 35% (188/537) Updating files: 36% (194/537) Updating files: 37% (199/537) Updating files: 38% (205/537) Updating files: 39% (210/537) Updating files: 39% (213/537) Updating files: 40% (215/537) Updating files: 41% (221/537) Updating files: 41% (224/537) Updating files: 42% (226/537) Updating files: 43% (231/537) Updating files: 43% (234/537) Updating files: 44% (237/537) Updating files: 45% (242/537) Updating files: 45% (246/537) Updating files: 46% (248/537) Updating files: 47% (253/537) Updating files: 47% (255/537) Updating files: 48% (258/537) Updating files: 49% (264/537) Updating files: 50% (269/537) Updating files: 51% (274/537) Updating files: 52% (280/537) Updating files: 53% (285/537) Updating files: 54% (290/537) Updating files: 55% (296/537) Updating files: 56% (301/537) Updating files: 57% (307/537) Updating files: 58% (312/537) Updating files: 59% (317/537) Updating files: 60% (323/537) Updating files: 61% (328/537) Updating files: 62% (333/537) Updating files: 63% (339/537) Updating files: 64% (344/537) Updating files: 65% (350/537) Updating files: 66% (355/537) Updating files: 67% (360/537) Updating files: 68% (366/537) Updating files: 69% (371/537) Updating files: 70% (376/537) Updating files: 71% (382/537) Updating files: 72% (387/537) Updating files: 73% (393/537) Updating files: 74% (398/537) Updating files: 75% (403/537) Updating files: 75% (407/537) Updating files: 76% (409/537) Updating files: 77% (414/537) Updating files: 78% (419/537) Updating files: 79% (425/537) Updating files: 80% (430/537) Updating files: 81% (435/537) Updating files: 82% (441/537) Updating files: 83% (446/537) Updating files: 83% (450/537) Updating files: 84% (452/537) Updating files: 84% (455/537) Updating files: 85% (457/537) Updating files: 86% (462/537) Updating files: 87% (468/537) Updating files: 88% (473/537) Updating files: 89% (478/537) Updating files: 90% (484/537) Updating files: 91% (489/537) Updating files: 91% (493/537) Updating files: 92% (495/537) Updating files: 93% (500/537) Updating files: 94% (505/537) Updating files: 95% (511/537) Updating files: 95% (513/537) Updating files: 96% (516/537) Updating files: 97% (521/537) Updating files: 98% (527/537) Updating files: 99% (532/537) Updating files: 100% (537/537) Updating files: 100% (537/537), done. 2023-06-09T11:50:56.677 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 090436f56ac v0.21 2023-06-09T11:50:56.685 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.21 2023-06-09T11:50:56.719 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:50:56.719 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 22 -eq 48 ']' 2023-06-09T11:50:56.719 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.22 2023-06-09T11:50:56.719 INFO:tasks.workunit.client.0.smithi007.stdout:v0.22 2023-06-09T11:50:56.720 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.22 2023-06-09T11:50:56.720 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.22 2023-06-09T11:51:04.382 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 22% (46/204) Updating files: 23% (47/204) Updating files: 24% (49/204) Updating files: 25% (51/204) Updating files: 26% (54/204) Updating files: 27% (56/204) Updating files: 28% (58/204) Updating files: 29% (60/204) Updating files: 30% (62/204) Updating files: 31% (64/204) Updating files: 32% (66/204) Updating files: 33% (68/204) Updating files: 34% (70/204) Updating files: 35% (72/204) Updating files: 36% (74/204) Updating files: 37% (76/204) Updating files: 38% (78/204) Updating files: 39% (80/204) Updating files: 40% (82/204) Updating files: 41% (84/204) Updating files: 42% (86/204) Updating files: 43% (88/204) Updating files: 44% (90/204) Updating files: 45% (92/204) Updating files: 46% (94/204) Updating files: 47% (96/204) Updating files: 48% (98/204) Updating files: 49% (100/204) Updating files: 50% (102/204) Updating files: 51% (105/204) Updating files: 52% (107/204) Updating files: 53% (109/204) Updating files: 54% (111/204) Updating files: 55% (113/204) Updating files: 56% (115/204) Updating files: 57% (117/204) Updating files: 58% (119/204) Updating files: 59% (121/204) Updating files: 60% (123/204) Updating files: 61% (125/204) Updating files: 62% (127/204) Updating files: 63% (129/204) Updating files: 64% (131/204) Updating files: 65% (133/204) Updating files: 66% (135/204) Updating files: 67% (137/204) Updating files: 68% (139/204) Updating files: 69% (141/204) Updating files: 70% (143/204) Updating files: 71% (145/204) Updating files: 72% (147/204) Updating files: 73% (149/204) Updating files: 74% (151/204) Updating files: 75% (153/204) Updating files: 75% (155/204) Updating files: 76% (156/204) Updating files: 77% (158/204) Updating files: 78% (160/204) Updating files: 79% (162/204) Updating files: 80% (164/204) Updating files: 81% (166/204) Updating files: 82% (168/204) Updating files: 83% (170/204) Updating files: 84% (172/204) Updating files: 85% (174/204) Updating files: 86% (176/204) Updating files: 87% (178/204) Updating files: 88% (180/204) Updating files: 89% (182/204) Updating files: 90% (184/204) Updating files: 91% (186/204) Updating files: 92% (188/204) Updating files: 93% (190/204) Updating files: 94% (192/204) Updating files: 95% (194/204) Updating files: 96% (196/204) Updating files: 97% (198/204) Updating files: 98% (200/204) Updating files: 99% (202/204) Updating files: 100% (204/204) Updating files: 100% (204/204), done. 2023-06-09T11:51:04.709 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 8a7c95f60ad v0.22 2023-06-09T11:51:04.716 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.22 2023-06-09T11:51:04.778 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:51:04.778 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 23 -eq 48 ']' 2023-06-09T11:51:04.779 INFO:tasks.workunit.client.0.smithi007.stdout:v0.23 2023-06-09T11:51:04.779 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.23 2023-06-09T11:51:04.779 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.23 2023-06-09T11:51:04.779 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.23 2023-06-09T11:51:15.585 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 14% (24/163) Updating files: 15% (25/163) Updating files: 16% (27/163) Updating files: 17% (28/163) Updating files: 18% (30/163) Updating files: 19% (31/163) Updating files: 20% (33/163) Updating files: 21% (35/163) Updating files: 22% (36/163) Updating files: 23% (38/163) Updating files: 24% (40/163) Updating files: 25% (41/163) Updating files: 26% (43/163) Updating files: 27% (45/163) Updating files: 28% (46/163) Updating files: 29% (48/163) Updating files: 30% (49/163) Updating files: 31% (51/163) Updating files: 32% (53/163) Updating files: 33% (54/163) Updating files: 34% (56/163) Updating files: 35% (58/163) Updating files: 36% (59/163) Updating files: 37% (61/163) Updating files: 38% (62/163) Updating files: 39% (64/163) Updating files: 40% (66/163) Updating files: 41% (67/163) Updating files: 42% (69/163) Updating files: 43% (71/163) Updating files: 44% (72/163) Updating files: 45% (74/163) Updating files: 46% (75/163) Updating files: 47% (77/163) Updating files: 48% (79/163) Updating files: 49% (80/163) Updating files: 50% (82/163) Updating files: 51% (84/163) Updating files: 52% (85/163) Updating files: 53% (87/163) Updating files: 54% (89/163) Updating files: 55% (90/163) Updating files: 56% (92/163) Updating files: 57% (93/163) Updating files: 58% (95/163) Updating files: 59% (97/163) Updating files: 60% (98/163) Updating files: 61% (100/163) Updating files: 62% (102/163) Updating files: 63% (103/163) Updating files: 64% (105/163) Updating files: 65% (106/163) Updating files: 66% (108/163) Updating files: 67% (110/163) Updating files: 68% (111/163) Updating files: 69% (113/163) Updating files: 70% (115/163) Updating files: 71% (116/163) Updating files: 72% (118/163) Updating files: 73% (119/163) Updating files: 74% (121/163) Updating files: 75% (123/163) Updating files: 76% (124/163) Updating files: 77% (126/163) Updating files: 78% (128/163) Updating files: 79% (129/163) Updating files: 80% (131/163) Updating files: 81% (133/163) Updating files: 82% (134/163) Updating files: 83% (136/163) Updating files: 84% (137/163) Updating files: 85% (139/163) Updating files: 86% (141/163) Updating files: 87% (142/163) Updating files: 88% (144/163) Updating files: 89% (146/163) Updating files: 90% (147/163) Updating files: 91% (149/163) Updating files: 92% (150/163) Updating files: 93% (152/163) Updating files: 94% (154/163) Updating files: 95% (155/163) Updating files: 96% (157/163) Updating files: 97% (159/163) Updating files: 98% (160/163) Updating files: 99% (162/163) Updating files: 100% (163/163) Updating files: 100% (163/163), done. 2023-06-09T11:51:15.748 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 5d1d8d0c460 v0.23 2023-06-09T11:51:15.754 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.23 2023-06-09T11:51:15.789 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:51:15.790 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 24 -eq 48 ']' 2023-06-09T11:51:15.790 INFO:tasks.workunit.client.0.smithi007.stdout:v0.24 2023-06-09T11:51:15.790 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.24 2023-06-09T11:51:15.790 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.24 2023-06-09T11:51:15.791 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.24 2023-06-09T11:51:22.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:22.263 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:29.486 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 3% (6/185) Updating files: 4% (8/185) Updating files: 5% (10/185) Updating files: 6% (12/185) Updating files: 7% (13/185) Updating files: 8% (15/185) Updating files: 9% (17/185) Updating files: 10% (19/185) Updating files: 11% (21/185) Updating files: 12% (23/185) Updating files: 13% (25/185) Updating files: 14% (26/185) Updating files: 15% (28/185) Updating files: 16% (30/185) Updating files: 17% (32/185) Updating files: 18% (34/185) Updating files: 19% (36/185) Updating files: 20% (37/185) Updating files: 21% (39/185) Updating files: 22% (41/185) Updating files: 23% (43/185) Updating files: 24% (45/185) Updating files: 25% (47/185) Updating files: 25% (48/185) Updating files: 26% (49/185) Updating files: 27% (50/185) Updating files: 28% (52/185) Updating files: 29% (54/185) Updating files: 30% (56/185) Updating files: 31% (58/185) Updating files: 31% (59/185) Updating files: 32% (60/185) Updating files: 33% (62/185) Updating files: 34% (63/185) Updating files: 35% (65/185) Updating files: 36% (67/185) Updating files: 37% (69/185) Updating files: 38% (71/185) Updating files: 38% (72/185) Updating files: 39% (73/185) Updating files: 40% (74/185) Updating files: 41% (76/185) Updating files: 42% (78/185) Updating files: 43% (80/185) Updating files: 44% (82/185) Updating files: 45% (84/185) Updating files: 46% (86/185) Updating files: 47% (87/185) Updating files: 48% (89/185) Updating files: 49% (91/185) Updating files: 50% (93/185) Updating files: 51% (95/185) Updating files: 52% (97/185) Updating files: 53% (99/185) Updating files: 54% (100/185) Updating files: 55% (102/185) Updating files: 56% (104/185) Updating files: 56% (105/185) Updating files: 57% (106/185) Updating files: 58% (108/185) Updating files: 59% (110/185) Updating files: 60% (111/185) Updating files: 61% (113/185) Updating files: 62% (115/185) Updating files: 63% (117/185) Updating files: 64% (119/185) Updating files: 65% (121/185) Updating files: 66% (123/185) Updating files: 67% (124/185) Updating files: 68% (126/185) Updating files: 69% (128/185) Updating files: 70% (130/185) Updating files: 71% (132/185) Updating files: 71% (133/185) Updating files: 72% (134/185) Updating files: 73% (136/185) Updating files: 74% (137/185) Updating files: 75% (139/185) Updating files: 76% (141/185) Updating files: 77% (143/185) Updating files: 78% (145/185) Updating files: 79% (147/185) Updating files: 80% (148/185) Updating files: 81% (150/185) Updating files: 82% (152/185) Updating files: 83% (154/185) Updating files: 84% (156/185) Updating files: 85% (158/185) Updating files: 86% (160/185) Updating files: 87% (161/185) Updating files: 88% (163/185) Updating files: 89% (165/185) Updating files: 90% (167/185) Updating files: 91% (169/185) Updating files: 92% (171/185) Updating files: 93% (173/185) Updating files: 94% (174/185) Updating files: 95% (176/185) Updating files: 96% (178/185) Updating files: 97% (180/185) Updating files: 98% (182/185) Updating files: 99% (184/185) Updating files: 100% (185/185) Updating files: 100% (185/185), done. 2023-06-09T11:51:30.836 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 180a4176035 v0.24 2023-06-09T11:51:30.842 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.24 2023-06-09T11:51:30.892 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:51:30.893 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 25 -eq 48 ']' 2023-06-09T11:51:30.893 INFO:tasks.workunit.client.0.smithi007.stdout:v0.25 2023-06-09T11:51:30.894 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.25 2023-06-09T11:51:30.894 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.25 2023-06-09T11:51:30.894 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.25 2023-06-09T11:51:52.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:52.381 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:51:55.293 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 6% (38/622) Updating files: 6% (43/622) Updating files: 7% (44/622) Updating files: 8% (50/622) Updating files: 9% (56/622) Updating files: 10% (63/622) Updating files: 11% (69/622) Updating files: 12% (75/622) Updating files: 13% (81/622) Updating files: 14% (88/622) Updating files: 15% (94/622) Updating files: 16% (100/622) Updating files: 16% (105/622) Updating files: 17% (106/622) Updating files: 18% (112/622) Updating files: 18% (115/622) Updating files: 19% (119/622) Updating files: 19% (120/622) Updating files: 20% (125/622) Updating files: 21% (131/622) Updating files: 21% (133/622) Updating files: 22% (137/622) Updating files: 23% (144/622) Updating files: 24% (150/622) Updating files: 25% (156/622) Updating files: 26% (162/622) Updating files: 27% (168/622) Updating files: 28% (175/622) Updating files: 29% (181/622) Updating files: 30% (187/622) Updating files: 30% (192/622) Updating files: 31% (193/622) Updating files: 31% (197/622) Updating files: 32% (200/622) Updating files: 33% (206/622) Updating files: 34% (212/622) Updating files: 35% (218/622) Updating files: 36% (224/622) Updating files: 37% (231/622) Updating files: 38% (237/622) Updating files: 39% (243/622) Updating files: 40% (249/622) Updating files: 41% (256/622) Updating files: 41% (260/622) Updating files: 42% (262/622) Updating files: 43% (268/622) Updating files: 44% (274/622) Updating files: 45% (280/622) Updating files: 46% (287/622) Updating files: 47% (293/622) Updating files: 48% (299/622) Updating files: 49% (305/622) Updating files: 50% (311/622) Updating files: 51% (318/622) Updating files: 52% (324/622) Updating files: 53% (330/622) Updating files: 54% (336/622) Updating files: 55% (343/622) Updating files: 56% (349/622) Updating files: 57% (355/622) Updating files: 58% (361/622) Updating files: 59% (367/622) Updating files: 59% (368/622) Updating files: 60% (374/622) Updating files: 61% (380/622) Updating files: 61% (385/622) Updating files: 62% (386/622) Updating files: 63% (392/622) Updating files: 64% (399/622) Updating files: 65% (405/622) Updating files: 65% (408/622) Updating files: 66% (411/622) Updating files: 67% (417/622) Updating files: 68% (423/622) Updating files: 69% (430/622) Updating files: 70% (436/622) Updating files: 71% (442/622) Updating files: 72% (448/622) Updating files: 73% (455/622) Updating files: 74% (461/622) Updating files: 75% (467/622) Updating files: 76% (473/622) Updating files: 77% (479/622) Updating files: 77% (480/622) Updating files: 78% (486/622) Updating files: 79% (492/622) Updating files: 80% (498/622) Updating files: 81% (504/622) Updating files: 82% (511/622) Updating files: 83% (517/622) Updating files: 83% (520/622) Updating files: 84% (523/622) Updating files: 84% (525/622) Updating files: 85% (529/622) Updating files: 85% (534/622) Updating files: 86% (535/622) Updating files: 87% (542/622) Updating files: 88% (548/622) Updating files: 89% (554/622) Updating files: 90% (560/622) Updating files: 91% (567/622) Updating files: 92% (573/622) Updating files: 93% (579/622) Updating files: 94% (585/622) Updating files: 95% (591/622) Updating files: 96% (598/622) Updating files: 96% (600/622) Updating files: 97% (604/622) Updating files: 98% (610/622) Updating files: 99% (616/622) Updating files: 100% (622/622) Updating files: 100% (622/622), done. 2023-06-09T11:51:55.425 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 3fa121bf0a8 v0.25 2023-06-09T11:51:55.431 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.25 2023-06-09T11:51:55.823 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:51:55.824 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 26 -eq 48 ']' 2023-06-09T11:51:55.824 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.26 2023-06-09T11:51:55.824 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.26 2023-06-09T11:51:55.824 INFO:tasks.workunit.client.0.smithi007.stdout:v0.26 2023-06-09T11:51:55.825 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.26 2023-06-09T11:52:18.885 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 13% (33/239) Updating files: 14% (34/239) Updating files: 15% (36/239) Updating files: 15% (38/239) Updating files: 16% (39/239) Updating files: 17% (41/239) Updating files: 18% (44/239) Updating files: 19% (46/239) Updating files: 19% (47/239) Updating files: 20% (48/239) Updating files: 21% (51/239) Updating files: 22% (53/239) Updating files: 23% (55/239) Updating files: 24% (58/239) Updating files: 25% (60/239) Updating files: 26% (63/239) Updating files: 27% (65/239) Updating files: 28% (67/239) Updating files: 29% (70/239) Updating files: 30% (72/239) Updating files: 30% (73/239) Updating files: 31% (75/239) Updating files: 32% (77/239) Updating files: 32% (78/239) Updating files: 33% (79/239) Updating files: 34% (82/239) Updating files: 35% (84/239) Updating files: 36% (87/239) Updating files: 37% (89/239) Updating files: 38% (91/239) Updating files: 39% (94/239) Updating files: 40% (96/239) Updating files: 41% (98/239) Updating files: 42% (101/239) Updating files: 43% (103/239) Updating files: 44% (106/239) Updating files: 45% (108/239) Updating files: 46% (110/239) Updating files: 47% (113/239) Updating files: 48% (115/239) Updating files: 49% (118/239) Updating files: 50% (120/239) Updating files: 50% (121/239) Updating files: 51% (122/239) Updating files: 52% (125/239) Updating files: 53% (127/239) Updating files: 54% (130/239) Updating files: 55% (132/239) Updating files: 56% (134/239) Updating files: 57% (137/239) Updating files: 58% (139/239) Updating files: 59% (142/239) Updating files: 60% (144/239) Updating files: 61% (146/239) Updating files: 62% (149/239) Updating files: 63% (151/239) Updating files: 64% (153/239) Updating files: 65% (156/239) Updating files: 66% (158/239) Updating files: 67% (161/239) Updating files: 68% (163/239) Updating files: 68% (164/239) Updating files: 69% (165/239) Updating files: 70% (168/239) Updating files: 71% (170/239) Updating files: 72% (173/239) Updating files: 73% (175/239) Updating files: 74% (177/239) Updating files: 75% (180/239) Updating files: 76% (182/239) Updating files: 77% (185/239) Updating files: 78% (187/239) Updating files: 79% (189/239) Updating files: 80% (192/239) Updating files: 81% (194/239) Updating files: 82% (196/239) Updating files: 83% (199/239) Updating files: 84% (201/239) Updating files: 85% (204/239) Updating files: 86% (206/239) Updating files: 87% (208/239) Updating files: 88% (211/239) Updating files: 89% (213/239) Updating files: 90% (216/239) Updating files: 91% (218/239) Updating files: 92% (220/239) Updating files: 93% (223/239) Updating files: 94% (225/239) Updating files: 95% (228/239) Updating files: 96% (230/239) Updating files: 97% (232/239) Updating files: 98% (235/239) Updating files: 99% (237/239) Updating files: 100% (239/239) Updating files: 100% (239/239), done. 2023-06-09T11:52:19.058 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 9981ff90968 v0.26 2023-06-09T11:52:19.065 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.26 2023-06-09T11:52:19.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:52:19.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 27 -eq 48 ']' 2023-06-09T11:52:19.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.27 2023-06-09T11:52:19.123 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.27 2023-06-09T11:52:19.124 INFO:tasks.workunit.client.0.smithi007.stdout:v0.27 2023-06-09T11:52:19.124 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.27 2023-06-09T11:52:22.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:22.559 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:23.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.490+0000 7feb551ff700 1 Processor -- start 2023-06-09T11:52:23.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.490+0000 7feb551ff700 1 -- start start 2023-06-09T11:52:23.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb551ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c1bc0 0x7feb501599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb551ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb500c1f90 0x7feb50159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb551ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb500c0bf0 con 0x7feb500c0140 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb551ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb50070110 con 0x7feb500c1f90 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb551ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb50071300 con 0x7feb500c1bc0 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb500c1f90 0x7feb50159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb500c1f90 0x7feb50159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59482/0 (socket says 172.21.15.7:59482) 2023-06-09T11:52:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 -- 172.21.15.7:0/3086331479 learned_addr learned my addr 172.21.15.7:0/3086331479 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:23.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4ed9d700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c1bc0 0x7feb501599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 -- 172.21.15.7:0/3086331479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c1bc0 msgr2=0x7feb501599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:23.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c1bc0 0x7feb501599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 -- 172.21.15.7:0/3086331479 >> v1:172.21.15.117:6789/0 conn(0x7feb500c0140 legacy=0x7feb501501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.492+0000 7feb4f59e700 1 -- 172.21.15.7:0/3086331479 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb500702f0 con 0x7feb500c1f90 2023-06-09T11:52:23.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb4f59e700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb50159ef0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7feb48009e50 tx=0x7feb4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c1a04da900998b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb4dd9b700 1 -- 172.21.15.7:0/3086331479 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800e070 con 0x7feb500c1f90 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb4dd9b700 1 -- 172.21.15.7:0/3086331479 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4800b910 con 0x7feb500c1f90 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb4dd9b700 1 -- 172.21.15.7:0/3086331479 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4800bcc0 con 0x7feb500c1f90 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb551ff700 1 -- 172.21.15.7:0/3086331479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 msgr2=0x7feb50159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.493+0000 7feb551ff700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb50159ef0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7feb48009e50 tx=0x7feb4800b330 comp rx=0 tx=0).stop 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 -- 172.21.15.7:0/3086331479 shutdown_connections 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb50159ef0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 --2- 172.21.15.7:0/3086331479 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c1bc0 0x7feb501599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 -- 172.21.15.7:0/3086331479 >> 172.21.15.7:0/3086331479 conn(0x7feb5006a4d0 msgr2=0x7feb5006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 -- 172.21.15.7:0/3086331479 shutdown_connections 2023-06-09T11:52:23.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.494+0000 7feb551ff700 1 -- 172.21.15.7:0/3086331479 wait complete. 2023-06-09T11:52:23.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.495+0000 7feb551ff700 1 Processor -- start 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.495+0000 7feb551ff700 1 -- start start 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c0140 0x7feb501eeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb500c1bc0 0x7feb501f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501df430 con 0x7feb500c1f90 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501f20a0 con 0x7feb500c1bc0 2023-06-09T11:52:23.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.496+0000 7feb551ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb500c2b10 con 0x7feb500c0140 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59484/0 (socket says 172.21.15.7:59484) 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 -- 172.21.15.7:0/550590321 learned_addr learned my addr 172.21.15.7:0/550590321 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4ed9d700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c0140 0x7feb501eeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4e59c700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb500c1bc0 0x7feb501f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 -- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c0140 msgr2=0x7feb501eeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c0140 0x7feb501eeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 -- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb500c1bc0 msgr2=0x7feb501f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb500c1bc0 0x7feb501f2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 -- 172.21.15.7:0/550590321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48009b30 con 0x7feb500c1f90 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.497+0000 7feb4f59e700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7feb5006faf0 tx=0x7feb48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.498+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb48004800 con 0x7feb500c1f90 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.498+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb48004960 con 0x7feb500c1f90 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.498+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb500c1170 con 0x7feb500c1f90 2023-06-09T11:52:23.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.498+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4801c680 con 0x7feb500c1f90 2023-06-09T11:52:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.498+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb500c0e90 con 0x7feb500c1f90 2023-06-09T11:52:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.499+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb500c1450 con 0x7feb500c1f90 2023-06-09T11:52:23.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.500+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb4801c7e0 con 0x7feb500c1f90 2023-06-09T11:52:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.501+0000 7feb377fe700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 0x7feb3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.501+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7feb4808b780 con 0x7feb500c1f90 2023-06-09T11:52:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.501+0000 7feb4ed9d700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 0x7feb3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:23.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.502+0000 7feb4ed9d700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 0x7feb3c06d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feb40009be0 tx=0x7feb40009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:23.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.505+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feb48056080 con 0x7feb500c1f90 2023-06-09T11:52:23.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.758+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb5006b180 con 0x7feb500c1f90 2023-06-09T11:52:23.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.758+0000 7feb377fe700 1 -- 172.21.15.7:0/550590321 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7feb480596a0 con 0x7feb500c1f90 2023-06-09T11:52:23.762 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:23.762 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:23.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.760+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 msgr2=0x7feb3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.760+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 0x7feb3c06d190 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7feb40009be0 tx=0x7feb40009210 comp rx=0 tx=0).stop 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.760+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 msgr2=0x7feb500c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.760+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7feb5006faf0 tx=0x7feb48004ad0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 shutdown_connections 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb500c1f90 0x7feb500c5cb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3c06ace0 0x7feb3c06d190 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb500c0140 0x7feb501eeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 --2- 172.21.15.7:0/550590321 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb500c1bc0 0x7feb501f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 >> 172.21.15.7:0/550590321 conn(0x7feb5006a4d0 msgr2=0x7feb5014d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 shutdown_connections 2023-06-09T11:52:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:23.761+0000 7feb551ff700 1 -- 172.21.15.7:0/550590321 wait complete. 2023-06-09T11:52:23.766 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:23.782 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.f (rank=2) 2023-06-09T11:52:23.782 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T11:52:23.783 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:52:23.802 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T11:52:23.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:24.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.000+0000 7f4aa4fff700 1 Processor -- start 2023-06-09T11:52:24.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.000+0000 7f4aa4fff700 1 -- start start 2023-06-09T11:52:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.001+0000 7f4aa4fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aa015b4a0 0x7f4aa015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.001+0000 7f4aa4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa00c9f40 0x7f4aa00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.001+0000 7f4aa4fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa015a500 con 0x7f4aa0153880 2023-06-09T11:52:24.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.001+0000 7f4aa4fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa00c5f10 con 0x7f4aa015b4a0 2023-06-09T11:52:24.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.001+0000 7f4aa4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa015da00 con 0x7f4aa00c9f40 2023-06-09T11:52:24.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aa015b4a0 0x7f4aa015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4aa015b4a0 0x7f4aa015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59488/0 (socket says 172.21.15.7:59488) 2023-06-09T11:52:24.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/2119039408 learned_addr learned my addr 172.21.15.7:0/2119039408 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:24.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9ed9d700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa00c9f40 0x7f4aa00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/2119039408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa00c9f40 msgr2=0x7f4aa00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:24.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa00c9f40 0x7f4aa00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/2119039408 >> v1:172.21.15.117:6789/0 conn(0x7f4aa0153880 legacy=0x7f4aa015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/2119039408 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa01596e0 con 0x7f4aa015b4a0 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9dd9b700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa015b4a0 0x7f4aa015d4c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4a940098f0 tx=0x7f4a94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a59093dcd77d303 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.002+0000 7f4a9d59a700 1 -- 172.21.15.7:0/2119039408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94011830 con 0x7f4aa015b4a0 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4a9d59a700 1 -- 172.21.15.7:0/2119039408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a94011990 con 0x7f4aa015b4a0 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4a9d59a700 1 -- 172.21.15.7:0/2119039408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94011d20 con 0x7f4aa015b4a0 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 -- 172.21.15.7:0/2119039408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa015b4a0 msgr2=0x7f4aa015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa015b4a0 0x7f4aa015d4c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f4a940098f0 tx=0x7f4a94009c00 comp rx=0 tx=0).stop 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 -- 172.21.15.7:0/2119039408 shutdown_connections 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa00c9f40 0x7f4aa00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/2119039408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa015b4a0 0x7f4aa015d4c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 -- 172.21.15.7:0/2119039408 >> 172.21.15.7:0/2119039408 conn(0x7f4aa00737f0 msgr2=0x7f4aa00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 -- 172.21.15.7:0/2119039408 shutdown_connections 2023-06-09T11:52:24.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.003+0000 7f4aa4fff700 1 -- 172.21.15.7:0/2119039408 wait complete. 2023-06-09T11:52:24.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.005+0000 7f4aa4fff700 1 Processor -- start 2023-06-09T11:52:24.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.005+0000 7f4aa4fff700 1 -- start start 2023-06-09T11:52:24.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 0x7f4aa0151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa015b4a0 0x7f4aa014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0151670 con 0x7f4aa0153880 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01529e0 con 0x7f4aa00c9f40 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4aa4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa014b660 con 0x7f4aa015b4a0 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4a9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4a9dd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59498/0 (socket says 172.21.15.7:59498) 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/626491078 learned_addr learned my addr 172.21.15.7:0/626491078 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4a9ed9d700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa015b4a0 0x7f4aa014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.006+0000 7f4a9e59c700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 0x7f4aa0151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa015b4a0 msgr2=0x7f4aa014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa015b4a0 0x7f4aa014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 msgr2=0x7f4aa0151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 0x7f4aa0151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 -- 172.21.15.7:0/626491078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94009550 con 0x7f4aa0153880 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9e59c700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 0x7f4aa0151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:52:24.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a9dd9b700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94004530 con 0x7f4aa0153880 2023-06-09T11:52:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a9401f070 con 0x7f4aa0153880 2023-06-09T11:52:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa0159c60 con 0x7f4aa0153880 2023-06-09T11:52:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a940038e0 con 0x7f4aa0153880 2023-06-09T11:52:24.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.007+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa0159980 con 0x7f4aa0153880 2023-06-09T11:52:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.008+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa0159f40 con 0x7f4aa0153880 2023-06-09T11:52:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.009+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4a94021030 con 0x7f4aa0153880 2023-06-09T11:52:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.010+0000 7f4a8effd700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 0x7f4a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.010+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f4a9408b110 con 0x7f4aa0153880 2023-06-09T11:52:24.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.010+0000 7f4a9e59c700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 0x7f4a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:24.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.011+0000 7f4a9e59c700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 0x7f4a8006d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4a90000f60 tx=0x7f4a90009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:24.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.014+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4a94055a10 con 0x7f4aa0153880 2023-06-09T11:52:24.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.265+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aa0074cf0 con 0x7f4aa0153880 2023-06-09T11:52:24.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.266+0000 7f4a8effd700 1 -- 172.21.15.7:0/626491078 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f4a94059030 con 0x7f4aa0153880 2023-06-09T11:52:24.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:24.270 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.268+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 msgr2=0x7f4a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.268+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 0x7f4a8006d190 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4a90000f60 tx=0x7f4a90009040 comp rx=0 tx=0).stop 2023-06-09T11:52:24.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.268+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 msgr2=0x7f4aa0149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.268+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f4a94000c00 tx=0x7f4a94004750 comp rx=0 tx=0).stop 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 shutdown_connections 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4a8006ace0 0x7f4a8006d190 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4aa015b4a0 0x7f4aa014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4aa00c9f40 0x7f4aa0151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 --2- 172.21.15.7:0/626491078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4aa0153880 0x7f4aa0149e70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 >> 172.21.15.7:0/626491078 conn(0x7f4aa00737f0 msgr2=0x7f4aa015b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 shutdown_connections 2023-06-09T11:52:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:24.269+0000 7f4aa4fff700 1 -- 172.21.15.7:0/626491078 wait complete. 2023-06-09T11:52:24.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:24.290 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:26.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:26.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.498+0000 7f4b5b7ff700 1 Processor -- start 2023-06-09T11:52:26.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.498+0000 7f4b5b7ff700 1 -- start start 2023-06-09T11:52:26.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b5b7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4b54153880 0x7f4b5415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b5b7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b540c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b5b7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5415a500 con 0x7f4b540c9f40 2023-06-09T11:52:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b5b7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540c5f10 con 0x7f4b54153880 2023-06-09T11:52:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b5b7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415da00 con 0x7f4b5415b4a0 2023-06-09T11:52:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b58d9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4b540c9f40 0x7f4b5415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56264/0 (socket says 172.21.15.7:56264) 2023-06-09T11:52:26.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.499+0000 7f4b58d9a700 1 -- 172.21.15.7:0/2804673984 learned_addr learned my addr 172.21.15.7:0/2804673984 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b5959b700 1 --2- 172.21.15.7:0/2804673984 >> v2:172.21.15.117:3300/0 conn(0x7f4b54153880 0x7f4b5415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b540c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792859426 0 0) 0x7f4b44001110 con 0x7f4b540c9f40 2023-06-09T11:52:26.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b400035b0 con 0x7f4b540c9f40 2023-06-09T11:52:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 561187736 0 0) 0x7f4b400035b0 con 0x7f4b540c9f40 2023-06-09T11:52:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 msgr2=0x7f4b540c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b540c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 >> v2:172.21.15.117:3300/0 conn(0x7f4b54153880 msgr2=0x7f4b5415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:26.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 --2- 172.21.15.7:0/2804673984 >> v2:172.21.15.117:3300/0 conn(0x7f4b54153880 0x7f4b5415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b541596e0 con 0x7f4b540c9f40 2023-06-09T11:52:26.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f4b44001800 con 0x7f4b540c9f40 2023-06-09T11:52:26.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 >> v1:172.21.15.7:6789/0 conn(0x7f4b540c9f40 legacy=0x7f4b5415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b40003a00 0x7f4b40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b40004d90 0x7f4b40005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540c5f10 con 0x7f4b40004d90 2023-06-09T11:52:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415da00 con 0x7f4b40003a00 2023-06-09T11:52:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.500+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4000ba40 con 0x7f4b5415b4a0 2023-06-09T11:52:26.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b40003a00 0x7f4b40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b58d9a700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b40004d90 0x7f4b40005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 -- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 msgr2=0x7f4b40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:26.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b5415b4a0 0x7f4b40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 -- 172.21.15.7:0/2804673984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b40004d90 msgr2=0x7f4b40005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b40004d90 0x7f4b40005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 -- 172.21.15.7:0/2804673984 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b400035b0 con 0x7f4b40003a00 2023-06-09T11:52:26.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b40004d90 0x7f4b40005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:52:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.501+0000 7f4b5959b700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b40003a00 0x7f4b40003e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009b60 tx=0x7f4b40005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f223d0a65235c4e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000e6c0 con 0x7f4b40003a00 2023-06-09T11:52:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5000e820 con 0x7f4b40003a00 2023-06-09T11:52:26.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b4bfff700 1 -- 172.21.15.7:0/2804673984 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000eb90 con 0x7f4b40003a00 2023-06-09T11:52:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b40003a00 msgr2=0x7f4b40003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/2804673984 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b40003a00 0x7f4b40003e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4b50009b60 tx=0x7f4b40005880 comp rx=0 tx=0).stop 2023-06-09T11:52:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b59d9c700 1 -- 172.21.15.7:0/2804673984 reap_dead start 2023-06-09T11:52:26.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/2804673984 shutdown_connections 2023-06-09T11:52:26.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/2804673984 >> 172.21.15.7:0/2804673984 conn(0x7f4b540737f0 msgr2=0x7f4b540720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:26.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/2804673984 shutdown_connections 2023-06-09T11:52:26.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.502+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/2804673984 wait complete. 2023-06-09T11:52:26.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.504+0000 7f4b5b7ff700 1 Processor -- start 2023-06-09T11:52:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.504+0000 7f4b5b7ff700 1 -- start start 2023-06-09T11:52:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b40003a00 0x7f4b5415c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b541f30f0 0x7f4b54157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b54152f90 con 0x7f4b541590a0 2023-06-09T11:52:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541f35a0 con 0x7f4b541f30f0 2023-06-09T11:52:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b5b7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b541ef2d0 con 0x7f4b40003a00 2023-06-09T11:52:26.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b59d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b59d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59530/0 (socket says 172.21.15.7:59530) 2023-06-09T11:52:26.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b59d9c700 1 -- 172.21.15.7:0/1763377903 learned_addr learned my addr 172.21.15.7:0/1763377903 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:26.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.505+0000 7f4b58d9a700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b541f30f0 0x7f4b54157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b5959b700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b40003a00 0x7f4b5415c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 -- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b40003a00 msgr2=0x7f4b5415c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b40003a00 0x7f4b5415c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 -- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b541f30f0 msgr2=0x7f4b54157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b541f30f0 0x7f4b54157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 -- 172.21.15.7:0/1763377903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54159c60 con 0x7f4b541590a0 2023-06-09T11:52:26.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b59d9c700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b57d0 tx=0x7f4b4c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:26.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00e040 con 0x7f4b541590a0 2023-06-09T11:52:26.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c00f040 con 0x7f4b541590a0 2023-06-09T11:52:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b54159980 con 0x7f4b541590a0 2023-06-09T11:52:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c00fbd0 con 0x7f4b541590a0 2023-06-09T11:52:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.506+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b54159f40 con 0x7f4b541590a0 2023-06-09T11:52:26.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.507+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b54074cf0 con 0x7f4b541590a0 2023-06-09T11:52:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.508+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b4c00fd30 con 0x7f4b541590a0 2023-06-09T11:52:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.509+0000 7f4b49ffb700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 0x7f4b3806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.510+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f4b4c08a3d0 con 0x7f4b541590a0 2023-06-09T11:52:26.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.510+0000 7f4b5959b700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 0x7f4b3806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:26.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.510+0000 7f4b5959b700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 0x7f4b3806d1c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4b500090b0 tx=0x7f4b50008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:26.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.514+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4b4c054bb0 con 0x7f4b541590a0 2023-06-09T11:52:26.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.772+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b540c4a00 con 0x7f4b541590a0 2023-06-09T11:52:26.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.772+0000 7f4b49ffb700 1 -- 172.21.15.7:0/1763377903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f4b4c0581d0 con 0x7f4b541590a0 2023-06-09T11:52:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:26.777 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:26.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 msgr2=0x7f4b3806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 0x7f4b3806d1c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4b500090b0 tx=0x7f4b50008030 comp rx=0 tx=0).stop 2023-06-09T11:52:26.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 msgr2=0x7f4b54157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:26.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f4b540b57d0 tx=0x7f4b4c00f490 comp rx=0 tx=0).stop 2023-06-09T11:52:26.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 shutdown_connections 2023-06-09T11:52:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b541590a0 0x7f4b54157aa0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b3806ad10 0x7f4b3806d1c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4b40003a00 0x7f4b5415c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 --2- 172.21.15.7:0/1763377903 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4b541f30f0 0x7f4b54157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 >> 172.21.15.7:0/1763377903 conn(0x7f4b540737f0 msgr2=0x7f4b541e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 shutdown_connections 2023-06-09T11:52:26.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:26.775+0000 7f4b5b7ff700 1 -- 172.21.15.7:0/1763377903 wait complete. 2023-06-09T11:52:26.783 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:26.797 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:28.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.005+0000 7fd8a3fff700 1 Processor -- start 2023-06-09T11:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.005+0000 7fd8a3fff700 1 -- start start 2023-06-09T11:52:29.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd89c0ca1b0 0x7fd89c15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a3fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c15dcf0 con 0x7fd89c0ca580 2023-06-09T11:52:29.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a3fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c15d450 con 0x7fd89c0b5360 2023-06-09T11:52:29.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a3fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c15d730 con 0x7fd89c0ca1b0 2023-06-09T11:52:29.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a259c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd89c0ca580 0x7fd89c06ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56270/0 (socket says 172.21.15.7:56270) 2023-06-09T11:52:29.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a259c700 1 -- 172.21.15.7:0/2379848237 learned_addr learned my addr 172.21.15.7:0/2379848237 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:29.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/2379848237 >> v2:172.21.15.117:3300/0 conn(0x7fd89c0ca1b0 0x7fd89c15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494431359 0 0) 0x7fd89c15dcf0 con 0x7fd89c0ca580 2023-06-09T11:52:29.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.007+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8880035b0 con 0x7fd89c0ca580 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034326198 0 0) 0x7fd8880035b0 con 0x7fd89c0ca580 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 >> v2:172.21.15.117:3300/0 conn(0x7fd89c0ca1b0 msgr2=0x7fd89c15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 --2- 172.21.15.7:0/2379848237 >> v2:172.21.15.117:3300/0 conn(0x7fd89c0ca1b0 0x7fd89c15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 >> v1:172.21.15.117:6789/0 conn(0x7fd89c0b5360 legacy=0x7fd89c1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c15da10 con 0x7fd89c0ca580 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fd8940019f0 con 0x7fd89c0ca580 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 >> v1:172.21.15.7:6789/0 conn(0x7fd89c0ca580 legacy=0x7fd89c06ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888003a00 0x7fd888003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 0x7fd888004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd888005200 0x7fd88800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c15d730 con 0x7fd888004850 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88800c060 con 0x7fd888003a00 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88800c220 con 0x7fd888005200 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a159a700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 0x7fd888004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888003a00 0x7fd888003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.008+0000 7fd8a259c700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd888005200 0x7fd88800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 -- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd888005200 msgr2=0x7fd88800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd888005200 0x7fd88800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 -- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888003a00 msgr2=0x7fd888003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888003a00 0x7fd888003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 -- 172.21.15.7:0/2379848237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880035b0 con 0x7fd888004850 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888003a00 0x7fd888003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:52:29.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a259c700 1 -- 172.21.15.7:0/2379848237 reap_dead start 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a159a700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 0x7fd888004cc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd88c001800 tx=0x7fd888005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4613b98e59a89b86 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0012f0 con 0x7fd888004850 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c005310 con 0x7fd888004850 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.009+0000 7fd8a0d99700 1 -- 172.21.15.7:0/2379848237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c006860 con 0x7fd888004850 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 -- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 msgr2=0x7fd888004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 0x7fd888004cc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd88c001800 tx=0x7fd888005880 comp rx=0 tx=0).stop 2023-06-09T11:52:29.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 -- 172.21.15.7:0/2379848237 shutdown_connections 2023-06-09T11:52:29.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/2379848237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd888004850 0x7fd888004cc0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 -- 172.21.15.7:0/2379848237 >> 172.21.15.7:0/2379848237 conn(0x7fd89c072090 msgr2=0x7fd89c072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:29.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 -- 172.21.15.7:0/2379848237 shutdown_connections 2023-06-09T11:52:29.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.010+0000 7fd8a3fff700 1 -- 172.21.15.7:0/2379848237 wait complete. 2023-06-09T11:52:29.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.012+0000 7fd8a3fff700 1 Processor -- start 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.012+0000 7fd8a3fff700 1 -- start start 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888004850 0x7fd89c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd89c156b10 0x7fd89c150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c1effb0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c151820 con 0x7fd888004850 2023-06-09T11:52:29.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a3fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c1ef0d0 con 0x7fd89c156b10 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59554/0 (socket says 172.21.15.7:59554) 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 -- 172.21.15.7:0/1298002563 learned_addr learned my addr 172.21.15.7:0/1298002563 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888004850 0x7fd89c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a159a700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd89c156b10 0x7fd89c150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 -- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd89c156b10 msgr2=0x7fd89c150a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd89c156b10 0x7fd89c150a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 -- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888004850 msgr2=0x7fd89c1512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888004850 0x7fd89c1512e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.013+0000 7fd8a259c700 1 -- 172.21.15.7:0/1298002563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c06a850 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8a259c700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd894000f80 tx=0x7fd89400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400ad60 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8940133f0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c06a570 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940137a0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.014+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c15cdc0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.015+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c1ef420 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.016+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd894013900 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.017+0000 7fd8927fc700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 0x7fd88006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:29.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.017+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fd89408a4d0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.017+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 0x7fd88006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:29.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.018+0000 7fd8a1d9b700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 0x7fd88006d1c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd89c064570 tx=0x7fd898009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:29.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.021+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd894054cb0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.274+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd89c1ef290 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.274+0000 7fd8927fc700 1 -- 172.21.15.7:0/1298002563 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fd8940582d0 con 0x7fd89c1ef5d0 2023-06-09T11:52:29.279 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:29.279 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 msgr2=0x7fd88006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 0x7fd88006d1c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd89c064570 tx=0x7fd898009210 comp rx=0 tx=0).stop 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 msgr2=0x7fd89c1efa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd894000f80 tx=0x7fd89400a5f0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 shutdown_connections 2023-06-09T11:52:29.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd89c1ef5d0 0x7fd89c1efa70 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd888004850 0x7fd89c1512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd88006ad10 0x7fd88006d1c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 --2- 172.21.15.7:0/1298002563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd89c156b10 0x7fd89c150a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 >> 172.21.15.7:0/1298002563 conn(0x7fd89c072090 msgr2=0x7fd89c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 shutdown_connections 2023-06-09T11:52:29.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:29.277+0000 7fd8a3fff700 1 -- 172.21.15.7:0/1298002563 wait complete. 2023-06-09T11:52:29.283 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:29.299 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:31.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:31.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.509+0000 7fb5dc1ff700 1 Processor -- start 2023-06-09T11:52:31.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.509+0000 7fb5dc1ff700 1 -- start start 2023-06-09T11:52:31.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.510+0000 7fb5dc1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5d4153880 0x7fb5d415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:31.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.510+0000 7fb5dc1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d415a500 con 0x7fb5d415b4a0 2023-06-09T11:52:31.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.510+0000 7fb5dc1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d415da00 con 0x7fb5d40c9f40 2023-06-09T11:52:31.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.510+0000 7fb5dc1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40c5f10 con 0x7fb5d4153880 2023-06-09T11:52:31.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d9f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb5d4153880 0x7fb5d415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:31.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5da79c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb5d415b4a0 0x7fb5d40c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56286/0 (socket says 172.21.15.7:56286) 2023-06-09T11:52:31.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5da79c700 1 -- 172.21.15.7:0/374287081 learned_addr learned my addr 172.21.15.7:0/374287081 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:31.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d8f99700 1 -- 172.21.15.7:0/374287081 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711422420 0 0) 0x7fb5cc001490 con 0x7fb5d415b4a0 2023-06-09T11:52:31.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/374287081 >> v1:172.21.15.117:6789/0 conn(0x7fb5d40c9f40 legacy=0x7fb5d415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/374287081 >> v1:172.21.15.7:6789/0 conn(0x7fb5d415b4a0 legacy=0x7fb5d40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/374287081 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d41596e0 con 0x7fb5d4153880 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.511+0000 7fb5d9f9b700 1 --2- 172.21.15.7:0/374287081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d4153880 0x7fb5d415d4c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009840 tx=0x7fb5d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebda61c0ed30b56 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5d8f99700 1 -- 172.21.15.7:0/374287081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011830 con 0x7fb5d4153880 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5d8f99700 1 -- 172.21.15.7:0/374287081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d0011990 con 0x7fb5d4153880 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5d8f99700 1 -- 172.21.15.7:0/374287081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011d20 con 0x7fb5d4153880 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/374287081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d4153880 msgr2=0x7fb5d415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/374287081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d4153880 0x7fb5d415d4c0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009840 tx=0x7fb5d0009b50 comp rx=0 tx=0).stop 2023-06-09T11:52:31.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/374287081 shutdown_connections 2023-06-09T11:52:31.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/374287081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d4153880 0x7fb5d415d4c0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/374287081 >> 172.21.15.7:0/374287081 conn(0x7fb5d40737f0 msgr2=0x7fb5d40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:31.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/374287081 shutdown_connections 2023-06-09T11:52:31.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.512+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/374287081 wait complete. 2023-06-09T11:52:31.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.514+0000 7fb5dc1ff700 1 Processor -- start 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.514+0000 7fb5dc1ff700 1 -- start start 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 0x7fb5d414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 0x7fb5d40b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d41541c0 con 0x7fb5d40c9f40 2023-06-09T11:52:31.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d41ef260 con 0x7fb5d415b4a0 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5dc1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d41ef540 con 0x7fb5d4153880 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5d9f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5d9f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59564/0 (socket says 172.21.15.7:59564) 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.515+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/2503166076 learned_addr learned my addr 172.21.15.7:0/2503166076 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d979a700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 0x7fb5d414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5da79c700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 0x7fb5d40b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 msgr2=0x7fb5d414c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 0x7fb5d414c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 msgr2=0x7fb5d40b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 0x7fb5d40b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 -- 172.21.15.7:0/2503166076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00094a0 con 0x7fb5d40c9f40 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d979a700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 0x7fb5d414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5da79c700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 0x7fb5d40b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5d9f9b700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00090b0 tx=0x7fb5d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:31.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d001a900 con 0x7fb5d40c9f40 2023-06-09T11:52:31.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d001aa60 con 0x7fb5d40c9f40 2023-06-09T11:52:31.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d001add0 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.516+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d4159c60 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.517+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4159980 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.518+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4159f40 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.518+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5d0022070 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.519+0000 7fb5ca7fc700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.519+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fb5d008aea0 con 0x7fb5d40c9f40 2023-06-09T11:52:31.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.520+0000 7fb5d979a700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:31.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.520+0000 7fb5d979a700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 0x7fb5b806d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40ce890 tx=0x7fb5c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:31.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.524+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb5d0055850 con 0x7fb5d40c9f40 2023-06-09T11:52:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.775+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d4074cf0 con 0x7fb5d40c9f40 2023-06-09T11:52:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.775+0000 7fb5ca7fc700 1 -- 172.21.15.7:0/2503166076 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fb5d0029090 con 0x7fb5d40c9f40 2023-06-09T11:52:31.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:31.780 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 msgr2=0x7fb5b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 0x7fb5b806d190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40ce890 tx=0x7fb5c4009040 comp rx=0 tx=0).stop 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 msgr2=0x7fb5d4155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00090b0 tx=0x7fb5d0004750 comp rx=0 tx=0).stop 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 shutdown_connections 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb5d415b4a0 0x7fb5d40b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb5d40c9f40 0x7fb5d4155d60 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb5b806ace0 0x7fb5b806d190 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 --2- 172.21.15.7:0/2503166076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb5d4153880 0x7fb5d414c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 >> 172.21.15.7:0/2503166076 conn(0x7fb5d40737f0 msgr2=0x7fb5d40cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 shutdown_connections 2023-06-09T11:52:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:31.778+0000 7fb5dc1ff700 1 -- 172.21.15.7:0/2503166076 wait complete. 2023-06-09T11:52:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:31.800 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:33.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.008+0000 7fbc1f5ff700 1 Processor -- start 2023-06-09T11:52:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.008+0000 7fbc1f5ff700 1 -- start start 2023-06-09T11:52:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbc180cb5d0 0x7fbc180b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc180cb9a0 0x7fbc18159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 0x7fbc180cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc18159b20 con 0x7fbc180cb9a0 2023-06-09T11:52:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc180cac80 con 0x7fbc180cb5d0 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181597d0 con 0x7fbc1815cd40 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc180cb9a0 0x7fbc18159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbc180cb9a0 0x7fbc18159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46268/0 (socket says 172.21.15.7:46268) 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.010+0000 7fbc1cb9a700 1 -- 172.21.15.7:0/1884318740 learned_addr learned my addr 172.21.15.7:0/1884318740 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1db9c700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 0x7fbc180cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1d39b700 1 --2- 172.21.15.7:0/1884318740 >> v2:172.21.15.117:3300/0 conn(0x7fbc180cb5d0 0x7fbc180b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 -- 172.21.15.7:0/1884318740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 msgr2=0x7fbc180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 0x7fbc180cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 -- 172.21.15.7:0/1884318740 >> v2:172.21.15.117:3300/0 conn(0x7fbc180cb5d0 msgr2=0x7fbc180b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/1884318740 >> v2:172.21.15.117:3300/0 conn(0x7fbc180cb5d0 0x7fbc180b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 -- 172.21.15.7:0/1884318740 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18071ab0 con 0x7fbc180cb9a0 2023-06-09T11:52:34.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1db9c700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 0x7fbc180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb9a0 0x7fbc18159fb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbc1808d420 tx=0x7fbc08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a91cae228f79e6a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1884318740 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08011830 con 0x7fbc180cb9a0 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1884318740 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc08011990 con 0x7fbc180cb9a0 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.011+0000 7fbc0ffff700 1 -- 172.21.15.7:0/1884318740 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc08011d20 con 0x7fbc180cb9a0 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/1884318740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb9a0 msgr2=0x7fbc18159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb9a0 0x7fbc18159fb0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbc1808d420 tx=0x7fbc08009bd0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/1884318740 shutdown_connections 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/1884318740 >> v2:172.21.15.117:3300/0 conn(0x7fbc180cb5d0 0x7fbc180b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc1815cd40 0x7fbc180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/1884318740 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb9a0 0x7fbc18159fb0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/1884318740 >> 172.21.15.7:0/1884318740 conn(0x7fbc18070af0 msgr2=0x7fbc18070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/1884318740 shutdown_connections 2023-06-09T11:52:34.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.012+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/1884318740 wait complete. 2023-06-09T11:52:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.014+0000 7fbc1f5ff700 1 Processor -- start 2023-06-09T11:52:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.014+0000 7fbc1f5ff700 1 -- start start 2023-06-09T11:52:34.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc180cb9a0 0x7fbc181f2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1815cd40 0x7fbc181e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180c3e40 con 0x7fbc180cb5d0 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc181f2e90 con 0x7fbc1815cd40 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc180cff20 con 0x7fbc180cb9a0 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1d39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1d39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46280/0 (socket says 172.21.15.7:46280) 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1d39b700 1 -- 172.21.15.7:0/835420702 learned_addr learned my addr 172.21.15.7:0/835420702 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc180cb9a0 0x7fbc181f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.015+0000 7fbc1db9c700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1815cd40 0x7fbc181e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 -- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc180cb9a0 msgr2=0x7fbc181f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc180cb9a0 0x7fbc181f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 -- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1815cd40 msgr2=0x7fbc181e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:34.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1815cd40 0x7fbc181e8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 -- 172.21.15.7:0/835420702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009550 con 0x7fbc180cb5d0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1d39b700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e930 tx=0x7fbc1400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14013680 con 0x7fbc180cb5d0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc140137e0 con 0x7fbc180cb5d0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14013b70 con 0x7fbc180cb5d0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1815d4f0 con 0x7fbc180cb5d0 2023-06-09T11:52:34.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.016+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1815d210 con 0x7fbc180cb5d0 2023-06-09T11:52:34.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.017+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1815d7d0 con 0x7fbc180cb5d0 2023-06-09T11:52:34.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.018+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbc14013cd0 con 0x7fbc180cb5d0 2023-06-09T11:52:34.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.019+0000 7fbc0dffb700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 0x7fbbfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.019+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fbc14089cc0 con 0x7fbc180cb5d0 2023-06-09T11:52:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.020+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 0x7fbbfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.020+0000 7fbc1cb9a700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 0x7fbbfc06d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbc180cbd70 tx=0x7fbc08008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:34.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.024+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbc14090080 con 0x7fbc180cb5d0 2023-06-09T11:52:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.276+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc1808d160 con 0x7fbc180cb5d0 2023-06-09T11:52:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.276+0000 7fbc0dffb700 1 -- 172.21.15.7:0/835420702 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fbc14021070 con 0x7fbc180cb5d0 2023-06-09T11:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:34.280 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 msgr2=0x7fbbfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 0x7fbbfc06d190 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbc180cbd70 tx=0x7fbc08008030 comp rx=0 tx=0).stop 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 msgr2=0x7fbc181f00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400e930 tx=0x7fbc1400ec40 comp rx=0 tx=0).stop 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 shutdown_connections 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbc1815cd40 0x7fbc181e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbc180cb5d0 0x7fbc181f00b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbbfc06ace0 0x7fbbfc06d190 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 --2- 172.21.15.7:0/835420702 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbc180cb9a0 0x7fbc181f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 >> 172.21.15.7:0/835420702 conn(0x7fbc18070af0 msgr2=0x7fbc180cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 shutdown_connections 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:34.279+0000 7fbc1f5ff700 1 -- 172.21.15.7:0/835420702 wait complete. 2023-06-09T11:52:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:34.302 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:36.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.509+0000 7fce7e7ff700 1 Processor -- start 2023-06-09T11:52:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.509+0000 7fce7e7ff700 1 -- start start 2023-06-09T11:52:36.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7e7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce780c5b40 0x7fce78070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7e7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce781580a0 con 0x7fce78071060 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7e7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce780c6180 con 0x7fce78151510 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7e7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce781518e0 con 0x7fce780c5b40 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce777fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fce780c5b40 0x7fce78070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7cd9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce78071060 0x7fce78158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52386/0 (socket says 172.21.15.7:52386) 2023-06-09T11:52:36.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce7cd9c700 1 -- 172.21.15.7:0/4230828934 learned_addr learned my addr 172.21.15.7:0/4230828934 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:36.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce777fe700 1 -- 172.21.15.7:0/4230828934 >> v1:172.21.15.117:6789/0 conn(0x7fce78151510 legacy=0x7fce7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce777fe700 1 -- 172.21.15.7:0/4230828934 >> v1:172.21.15.7:6789/0 conn(0x7fce78071060 legacy=0x7fce78158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.511+0000 7fce777fe700 1 -- 172.21.15.7:0/4230828934 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78157490 con 0x7fce780c5b40 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.512+0000 7fce777fe700 1 --2- 172.21.15.7:0/4230828934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce780c5b40 0x7fce78070b20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009840 tx=0x7fce6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2e3c13c21db585 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.512+0000 7fce76ffd700 1 -- 172.21.15.7:0/4230828934 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c011830 con 0x7fce780c5b40 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.512+0000 7fce76ffd700 1 -- 172.21.15.7:0/4230828934 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c011990 con 0x7fce780c5b40 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.512+0000 7fce76ffd700 1 -- 172.21.15.7:0/4230828934 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c011d20 con 0x7fce780c5b40 2023-06-09T11:52:36.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 -- 172.21.15.7:0/4230828934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce780c5b40 msgr2=0x7fce78070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/4230828934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce780c5b40 0x7fce78070b20 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009840 tx=0x7fce6c009b50 comp rx=0 tx=0).stop 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 -- 172.21.15.7:0/4230828934 shutdown_connections 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/4230828934 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce780c5b40 0x7fce78070b20 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 -- 172.21.15.7:0/4230828934 >> 172.21.15.7:0/4230828934 conn(0x7fce7806a4d0 msgr2=0x7fce7806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 -- 172.21.15.7:0/4230828934 shutdown_connections 2023-06-09T11:52:36.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.513+0000 7fce7e7ff700 1 -- 172.21.15.7:0/4230828934 wait complete. 2023-06-09T11:52:36.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.514+0000 7fce7e7ff700 1 Processor -- start 2023-06-09T11:52:36.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.515+0000 7fce7e7ff700 1 -- start start 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce780c5b40 0x7fce7814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce78151510 0x7fce78145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce781ed6d0 con 0x7fce78071060 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7814cca0 con 0x7fce780c5b40 2023-06-09T11:52:36.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7e7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce781496f0 con 0x7fce78151510 2023-06-09T11:52:36.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:36.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46286/0 (socket says 172.21.15.7:46286) 2023-06-09T11:52:36.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 -- 172.21.15.7:0/1728118064 learned_addr learned my addr 172.21.15.7:0/1728118064 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:36.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce777fe700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce780c5b40 0x7fce7814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:36.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce7cd9c700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce78151510 0x7fce78145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:36.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 -- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce78151510 msgr2=0x7fce78145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce78151510 0x7fce78145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 -- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce780c5b40 msgr2=0x7fce7814e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce780c5b40 0x7fce7814e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.516+0000 7fce77fff700 1 -- 172.21.15.7:0/1728118064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094a0 con 0x7fce78071060 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce77fff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c540 tx=0x7fce6800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680052d0 con 0x7fce78071060 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68005430 con 0x7fce78071060 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce680038c0 con 0x7fce78071060 2023-06-09T11:52:36.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce78157a50 con 0x7fce78071060 2023-06-09T11:52:36.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.517+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce78157770 con 0x7fce78071060 2023-06-09T11:52:36.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.518+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce78157d30 con 0x7fce78071060 2023-06-09T11:52:36.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.519+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce68003a20 con 0x7fce78071060 2023-06-09T11:52:36.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.520+0000 7fce74ff9700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:36.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.520+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7fce680890f0 con 0x7fce78071060 2023-06-09T11:52:36.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.520+0000 7fce777fe700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:36.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.521+0000 7fce777fe700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 0x7fce6406d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0090b0 tx=0x7fce6c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:36.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.525+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fce680538d0 con 0x7fce78071060 2023-06-09T11:52:36.778 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.775+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce781491a0 con 0x7fce78071060 2023-06-09T11:52:36.779 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.776+0000 7fce74ff9700 1 -- 172.21.15.7:0/1728118064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fce68017020 con 0x7fce78071060 2023-06-09T11:52:36.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:36.780 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:36.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 msgr2=0x7fce6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 0x7fce6406d190 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0090b0 tx=0x7fce6c008030 comp rx=0 tx=0).stop 2023-06-09T11:52:36.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 msgr2=0x7fce781edaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:36.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c540 tx=0x7fce6800c850 comp rx=0 tx=0).stop 2023-06-09T11:52:36.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 shutdown_connections 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fce6406ace0 0x7fce6406d190 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fce78151510 0x7fce78145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78071060 0x7fce781edaf0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 --2- 172.21.15.7:0/1728118064 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fce780c5b40 0x7fce7814e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 >> 172.21.15.7:0/1728118064 conn(0x7fce7806a4d0 msgr2=0x7fce7806a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 shutdown_connections 2023-06-09T11:52:36.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:36.778+0000 7fce7e7ff700 1 -- 172.21.15.7:0/1728118064 wait complete. 2023-06-09T11:52:36.784 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:36.800 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:38.801 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:39.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.009+0000 7f4862fff700 1 Processor -- start 2023-06-09T11:52:39.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.009+0000 7f4862fff700 1 -- start start 2023-06-09T11:52:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.010+0000 7f4862fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.010+0000 7f4862fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.010+0000 7f4862fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c159b20 con 0x7f485c0cb9a0 2023-06-09T11:52:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.010+0000 7f4862fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0cac80 con 0x7f485c15cd40 2023-06-09T11:52:39.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.010+0000 7f4862fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1597d0 con 0x7f485c0cb5d0 2023-06-09T11:52:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f486159c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:38618/0 (socket says 172.21.15.7:38618) 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 -- 172.21.15.7:0/3750521488 learned_addr learned my addr 172.21.15.7:0/3750521488 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f485bfff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f485c0cb9a0 0x7f485c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:54414/0 (socket says 172.21.15.7:54414) 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f485b7fe700 1 -- 172.21.15.7:0/3750521488 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369742936 0 0) 0x7f485c159b20 con 0x7f485c0cb9a0 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f485b7fe700 1 -- 172.21.15.7:0/3750521488 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f484c0035b0 con 0x7f485c0cb9a0 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 -- 172.21.15.7:0/3750521488 >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 msgr2=0x7f485c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:39.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 --2- 172.21.15.7:0/3750521488 >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 -- 172.21.15.7:0/3750521488 >> v1:172.21.15.117:6789/0 conn(0x7f485c0cb9a0 legacy=0x7f485c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f4860d9b700 1 -- 172.21.15.7:0/3750521488 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c071ab0 con 0x7f485c0cb5d0 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.011+0000 7f486159c700 1 --2- 172.21.15.7:0/3750521488 >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f4860d9b700 1 --2- 172.21.15.7:0/3750521488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48500098f0 tx=0x7f4850009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f013df491104ae7f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f485b7fe700 1 -- 172.21.15.7:0/3750521488 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850011830 con 0x7f485c0cb5d0 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f485b7fe700 1 -- 172.21.15.7:0/3750521488 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4850011990 con 0x7f485c0cb5d0 2023-06-09T11:52:39.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f485b7fe700 1 -- 172.21.15.7:0/3750521488 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4850011d20 con 0x7f485c0cb5d0 2023-06-09T11:52:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f4862fff700 1 -- 172.21.15.7:0/3750521488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 msgr2=0x7f485c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.012+0000 7f4862fff700 1 --2- 172.21.15.7:0/3750521488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f48500098f0 tx=0x7f4850009c00 comp rx=0 tx=0).stop 2023-06-09T11:52:39.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 -- 172.21.15.7:0/3750521488 shutdown_connections 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 --2- 172.21.15.7:0/3750521488 >> v2:172.21.15.117:3300/0 conn(0x7f485c15cd40 0x7f485c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 --2- 172.21.15.7:0/3750521488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c0b52b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 -- 172.21.15.7:0/3750521488 >> 172.21.15.7:0/3750521488 conn(0x7f485c070af0 msgr2=0x7f485c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 -- 172.21.15.7:0/3750521488 shutdown_connections 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.013+0000 7f4862fff700 1 -- 172.21.15.7:0/3750521488 wait complete. 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.014+0000 7f4862fff700 1 Processor -- start 2023-06-09T11:52:39.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.015+0000 7f4862fff700 1 -- start start 2023-06-09T11:52:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.015+0000 7f4862fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4862fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb9a0 0x7f485c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4862fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4862fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1532f0 con 0x7f485c15cd40 2023-06-09T11:52:39.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4862fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c1f2180 con 0x7f485c0cb9a0 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4862fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0ac6d0 con 0x7f485c0cb5d0 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46298/0 (socket says 172.21.15.7:46298) 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 -- 172.21.15.7:0/335648617 learned_addr learned my addr 172.21.15.7:0/335648617 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f4860d9b700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f485bfff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb9a0 0x7f485c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 -- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 msgr2=0x7f485c152db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c152db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 -- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb9a0 msgr2=0x7f485c0cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb9a0 0x7f485c0cdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.016+0000 7f486159c700 1 -- 172.21.15.7:0/335648617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009a70 con 0x7f485c15cd40 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f486159c700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f485400ef90 tx=0x7f485400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400cc90 con 0x7f485c15cd40 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485400f040 con 0x7f485c15cd40 2023-06-09T11:52:39.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48540146c0 con 0x7f485c15cd40 2023-06-09T11:52:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850009550 con 0x7f485c15cd40 2023-06-09T11:52:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.017+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c15d4f0 con 0x7f485c15cd40 2023-06-09T11:52:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.018+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c15d210 con 0x7f485c15cd40 2023-06-09T11:52:39.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.019+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4854014820 con 0x7f485c15cd40 2023-06-09T11:52:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.021+0000 7f48597fa700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.021+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 8012+0+0 (secure 0 0 0) 0x7f485408a810 con 0x7f485c15cd40 2023-06-09T11:52:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.021+0000 7f4860d9b700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:39.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.022+0000 7f4860d9b700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 0x7f484006d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4850009c40 tx=0x7f4850008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:39.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.025+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4854055110 con 0x7f485c15cd40 2023-06-09T11:52:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.276+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f485c15d7d0 con 0x7f485c15cd40 2023-06-09T11:52:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.277+0000 7f48597fa700 1 -- 172.21.15.7:0/335648617 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 193 v193) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f4854018020 con 0x7f485c15cd40 2023-06-09T11:52:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:39.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":193,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7666,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":191}],"filesystems":[{"mdsmap":{"epoch":193,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:50:40.841460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":6650},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6650":{"gid":6650,"name":"f","rank":2,"incarnation":186,"state":"up:active","state_seq":31,"addr":"172.21.15.117:6837/3744293936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3744293936},{"type":"v1","addr":"172.21.15.117:6837","nonce":3744293936}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 msgr2=0x7f484006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 0x7f484006d190 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4850009c40 tx=0x7f4850008030 comp rx=0 tx=0).stop 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 msgr2=0x7f485c0c7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f485400ef90 tx=0x7f485400c3a0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 shutdown_connections 2023-06-09T11:52:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f485c15cd40 0x7f485c0c7990 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f484006ace0 0x7f484006d190 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f485c0cb5d0 0x7f485c152db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 --2- 172.21.15.7:0/335648617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f485c0cb9a0 0x7f485c0cdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 >> 172.21.15.7:0/335648617 conn(0x7f485c070af0 msgr2=0x7f485c158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 shutdown_connections 2023-06-09T11:52:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:39.279+0000 7f4862fff700 1 -- 172.21.15.7:0/335648617 wait complete. 2023-06-09T11:52:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 193 2023-06-09T11:52:39.300 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is laggy/crashed, in failed state, or mds.f is removed from mdsmap 2023-06-09T11:52:41.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:41.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.509+0000 7f5e04dff700 1 Processor -- start 2023-06-09T11:52:41.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.509+0000 7f5e04dff700 1 -- start start 2023-06-09T11:52:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.510+0000 7f5e04dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5e001593d0 0x7f5e001597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.510+0000 7f5e04dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e000c08d0 0x7f5e00159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5e04dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e000cc140 con 0x7f5e00071910 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5e04dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000714f0 con 0x7f5e000c08d0 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5e04dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e000c1830 con 0x7f5e001593d0 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5df7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5e000c08d0 0x7f5e00159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5dfed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e00071910 0x7f5e00071d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52396/0 (socket says 172.21.15.7:52396) 2023-06-09T11:52:41.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5dfed9d700 1 -- 172.21.15.7:0/3253188247 learned_addr learned my addr 172.21.15.7:0/3253188247 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5dfe59c700 1 --2- 172.21.15.7:0/3253188247 >> v2:172.21.15.117:3300/0 conn(0x7f5e001593d0 0x7f5e001597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5df7fff700 1 -- 172.21.15.7:0/3253188247 >> v2:172.21.15.117:3300/0 conn(0x7f5e001593d0 msgr2=0x7f5e001597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5df7fff700 1 --2- 172.21.15.7:0/3253188247 >> v2:172.21.15.117:3300/0 conn(0x7f5e001593d0 0x7f5e001597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5df7fff700 1 -- 172.21.15.7:0/3253188247 >> v1:172.21.15.7:6789/0 conn(0x7f5e00071910 legacy=0x7f5e00071d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5df7fff700 1 -- 172.21.15.7:0/3253188247 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000c0560 con 0x7f5e000c08d0 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.511+0000 7f5dfdd9b700 1 -- 172.21.15.7:0/3253188247 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341427403 0 0) 0x7f5e000cc140 con 0x7f5e00071910 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5df7fff700 1 --2- 172.21.15.7:0/3253188247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e00159ce0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006ff10 tx=0x7f5de800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fd865beb1d1f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5dfdd9b700 1 -- 172.21.15.7:0/3253188247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800db60 con 0x7f5e000c08d0 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5dfdd9b700 1 -- 172.21.15.7:0/3253188247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800dcc0 con 0x7f5e000c08d0 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5dfdd9b700 1 -- 172.21.15.7:0/3253188247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8017630 con 0x7f5e000c08d0 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5e04dff700 1 -- 172.21.15.7:0/3253188247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 msgr2=0x7f5e00159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5e04dff700 1 --2- 172.21.15.7:0/3253188247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e00159ce0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f5e0006ff10 tx=0x7f5de800bdc0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5e04dff700 1 -- 172.21.15.7:0/3253188247 shutdown_connections 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5e04dff700 1 --2- 172.21.15.7:0/3253188247 >> v2:172.21.15.117:3300/0 conn(0x7f5e001593d0 0x7f5e001597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.512+0000 7f5e04dff700 1 --2- 172.21.15.7:0/3253188247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e00159ce0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.513+0000 7f5e04dff700 1 -- 172.21.15.7:0/3253188247 >> 172.21.15.7:0/3253188247 conn(0x7f5e0006a4d0 msgr2=0x7f5e0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.513+0000 7f5e04dff700 1 -- 172.21.15.7:0/3253188247 shutdown_connections 2023-06-09T11:52:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.513+0000 7f5e04dff700 1 -- 172.21.15.7:0/3253188247 wait complete. 2023-06-09T11:52:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.514+0000 7f5e04dff700 1 Processor -- start 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.514+0000 7f5e04dff700 1 -- start start 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 0x7f5e000c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 0x7f5e00147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000c4ae0 con 0x7f5e000c08d0 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000c4da0 con 0x7f5e00071910 2023-06-09T11:52:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.515+0000 7f5e04dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e000c2520 con 0x7f5e001593d0 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46330/0 (socket says 172.21.15.7:46330) 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 -- 172.21.15.7:0/2580095324 learned_addr learned my addr 172.21.15.7:0/2580095324 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5dfe59c700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 0x7f5e000c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 -- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 msgr2=0x7f5e00147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5dfed9d700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 0x7f5e00147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 0x7f5e00147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 -- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 msgr2=0x7f5e000c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 0x7f5e000c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 -- 172.21.15.7:0/2580095324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de800b740 con 0x7f5e000c08d0 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5dfe59c700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 0x7f5e000c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5dfed9d700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 0x7f5e00147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.516+0000 7f5df7fff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f5de8019040 tx=0x7f5de8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.517+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80179f0 con 0x7f5e000c08d0 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.517+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8017b50 con 0x7f5e000c08d0 2023-06-09T11:52:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.517+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de80036a0 con 0x7f5e000c08d0 2023-06-09T11:52:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.517+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e000c1d60 con 0x7f5e000c08d0 2023-06-09T11:52:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.517+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e000c1a80 con 0x7f5e000c08d0 2023-06-09T11:52:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.518+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e000c2040 con 0x7f5e000c08d0 2023-06-09T11:52:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.519+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5de8015070 con 0x7f5e000c08d0 2023-06-09T11:52:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.520+0000 7f5df6ffd700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 0x7f5de006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.520+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f5de808d920 con 0x7f5e000c08d0 2023-06-09T11:52:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.520+0000 7f5dfe59c700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 0x7f5de006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.521+0000 7f5dfe59c700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 0x7f5de006d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009820 tx=0x7f5df0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.524+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5de80581c0 con 0x7f5e000c08d0 2023-06-09T11:52:41.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.779+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e000c5810 con 0x7f5e000c08d0 2023-06-09T11:52:41.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.779+0000 7f5df6ffd700 1 -- 172.21.15.7:0/2580095324 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f5de8026090 con 0x7f5e000c08d0 2023-06-09T11:52:41.783 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:41.783 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:39.696310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.781+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 msgr2=0x7f5de006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.781+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 0x7f5de006d190 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5df0009820 tx=0x7f5df0008040 comp rx=0 tx=0).stop 2023-06-09T11:52:41.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.781+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 msgr2=0x7f5e000c3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.781+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f5de8019040 tx=0x7f5de8004750 comp rx=0 tx=0).stop 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 shutdown_connections 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5de006ace0 0x7f5de006d190 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5e001593d0 0x7f5e00147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5e00071910 0x7f5e000c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 --2- 172.21.15.7:0/2580095324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5e000c08d0 0x7f5e000c3470 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:41.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 >> 172.21.15.7:0/2580095324 conn(0x7f5e0006a4d0 msgr2=0x7f5e000c6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 shutdown_connections 2023-06-09T11:52:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:41.782+0000 7f5e04dff700 1 -- 172.21.15.7:0/2580095324 wait complete. 2023-06-09T11:52:41.787 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 194 2023-06-09T11:52:41.803 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f down, removed from mdsmap 2023-06-09T11:52:41.803 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:52:41.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.001+0000 7f231abff700 1 Processor -- start 2023-06-09T11:52:42.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.002+0000 7f231abff700 1 -- start start 2023-06-09T11:52:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231abff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f231415b4a0 0x7f231415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231abff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23140c9f40 0x7f23140c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231abff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231415a500 con 0x7f2314153880 2023-06-09T11:52:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231abff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23140c5f10 con 0x7f23140c9f40 2023-06-09T11:52:42.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231abff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231415da00 con 0x7f231415b4a0 2023-06-09T11:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231919c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23140c9f40 0x7f23140c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231919c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f23140c9f40 0x7f23140c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46342/0 (socket says 172.21.15.7:46342) 2023-06-09T11:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f231919c700 1 -- 172.21.15.7:0/2126591208 learned_addr learned my addr 172.21.15.7:0/2126591208 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.003+0000 7f2313fff700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f231415b4a0 0x7f231415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f231919c700 1 -- 172.21.15.7:0/2126591208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f231415b4a0 msgr2=0x7f231415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f231919c700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f231415b4a0 0x7f231415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f231919c700 1 -- 172.21.15.7:0/2126591208 >> v1:172.21.15.117:6789/0 conn(0x7f2314153880 legacy=0x7f231415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f231919c700 1 -- 172.21.15.7:0/2126591208 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23141596e0 con 0x7f23140c9f40 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f23137fe700 1 -- 172.21.15.7:0/2126591208 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242661718 0 0) 0x7f2308001110 con 0x7f2314153880 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f231919c700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f23140c4e30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f230c009e50 tx=0x7f230c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86939858eda1ea50 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f23137fe700 1 -- 172.21.15.7:0/2126591208 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c00e070 con 0x7f23140c9f40 2023-06-09T11:52:42.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f23137fe700 1 -- 172.21.15.7:0/2126591208 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230c00b910 con 0x7f23140c9f40 2023-06-09T11:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.004+0000 7f23137fe700 1 -- 172.21.15.7:0/2126591208 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230c00bcc0 con 0x7f23140c9f40 2023-06-09T11:52:42.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 -- 172.21.15.7:0/2126591208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 msgr2=0x7f23140c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f23140c4e30 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f230c009e50 tx=0x7f230c00b330 comp rx=0 tx=0).stop 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 -- 172.21.15.7:0/2126591208 shutdown_connections 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f23140c4e30 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 --2- 172.21.15.7:0/2126591208 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f231415b4a0 0x7f231415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 -- 172.21.15.7:0/2126591208 >> 172.21.15.7:0/2126591208 conn(0x7f23140737f0 msgr2=0x7f23140720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 -- 172.21.15.7:0/2126591208 shutdown_connections 2023-06-09T11:52:42.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.005+0000 7f231abff700 1 -- 172.21.15.7:0/2126591208 wait complete. 2023-06-09T11:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.007+0000 7f231abff700 1 Processor -- start 2023-06-09T11:52:42.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.007+0000 7f231abff700 1 -- start start 2023-06-09T11:52:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2314153880 0x7f23140cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f231415b4a0 0x7f23141e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2314151940 con 0x7f23140c9f40 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2314152df0 con 0x7f231415b4a0 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231abff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2314152b10 con 0x7f2314153880 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46344/0 (socket says 172.21.15.7:46344) 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 -- 172.21.15.7:0/2881733443 learned_addr learned my addr 172.21.15.7:0/2881733443 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f2313fff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2314153880 0x7f23140cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231919c700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f231415b4a0 0x7f23141e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 -- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2314153880 msgr2=0x7f23140cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2314153880 0x7f23140cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 -- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f231415b4a0 msgr2=0x7f23141e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.008+0000 7f231899b700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f231415b4a0 0x7f23141e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f231899b700 1 -- 172.21.15.7:0/2881733443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c009b30 con 0x7f23140c9f40 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f231899b700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f230800c750 tx=0x7f230800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:42.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23080054e0 con 0x7f23140c9f40 2023-06-09T11:52:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308003680 con 0x7f23140c9f40 2023-06-09T11:52:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2314159c60 con 0x7f23140c9f40 2023-06-09T11:52:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308003a30 con 0x7f23140c9f40 2023-06-09T11:52:42.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.009+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2314159980 con 0x7f23140c9f40 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.010+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2314159f40 con 0x7f23140c9f40 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.011+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2308003b90 con 0x7f23140c9f40 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.012+0000 7f23117fa700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 0x7f22f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.012+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f2308088fd0 con 0x7f23140c9f40 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.013+0000 7f2313fff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 0x7f22f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.013+0000 7f2313fff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 0x7f22f806d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2314158b50 tx=0x7f2300009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:42.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.016+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2308057280 con 0x7f23140c9f40 2023-06-09T11:52:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.267+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f231415a500 con 0x7f23140c9f40 2023-06-09T11:52:42.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.268+0000 7f23117fa700 1 -- 172.21.15.7:0/2881733443 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f231415a500 con 0x7f23140c9f40 2023-06-09T11:52:42.271 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:42.272 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:39.696310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 msgr2=0x7f22f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 0x7f22f806d190 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f2314158b50 tx=0x7f2300009210 comp rx=0 tx=0).stop 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 msgr2=0x7f2314158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f230800c750 tx=0x7f230800ca60 comp rx=0 tx=0).stop 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 shutdown_connections 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f231415b4a0 0x7f23141e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23140c9f40 0x7f2314158bf0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f22f806ace0 0x7f22f806d190 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 --2- 172.21.15.7:0/2881733443 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2314153880 0x7f23140cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 >> 172.21.15.7:0/2881733443 conn(0x7f23140737f0 msgr2=0x7f231415ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 shutdown_connections 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:42.270+0000 7f231abff700 1 -- 172.21.15.7:0/2881733443 wait complete. 2023-06-09T11:52:42.275 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 194 2023-06-09T11:52:42.294 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 194, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 194, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 72, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:52:39.696310+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:52:44.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.502+0000 7f5c7f5ff700 1 Processor -- start 2023-06-09T11:52:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.503+0000 7f5c7f5ff700 1 -- start start 2023-06-09T11:52:44.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781535e0 0x7f5c78159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7f5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5c780c4ce0 0x7f5c780c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7f5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7815a420 con 0x7f5c781539b0 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7f5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780ca670 con 0x7f5c780c4ce0 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c5a00 con 0x7f5c781535e0 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7cb9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5c781539b0 0x7f5c780c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57704/0 (socket says 172.21.15.7:57704) 2023-06-09T11:52:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7cb9a700 1 -- 172.21.15.7:0/72242301 learned_addr learned my addr 172.21.15.7:0/72242301 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7db9c700 1 --2- 172.21.15.7:0/72242301 >> v2:172.21.15.117:3300/0 conn(0x7f5c780c4ce0 0x7f5c780c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.504+0000 7f5c7d39b700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781535e0 0x7f5c78159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c6ffff700 1 -- 172.21.15.7:0/72242301 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729781902 0 0) 0x7f5c68001110 con 0x7f5c781539b0 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c6ffff700 1 -- 172.21.15.7:0/72242301 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c640035b0 con 0x7f5c781539b0 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c7db9c700 1 -- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781535e0 msgr2=0x7f5c78159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c7db9c700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781535e0 0x7f5c78159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c7db9c700 1 -- 172.21.15.7:0/72242301 >> v1:172.21.15.7:6789/0 conn(0x7f5c781539b0 legacy=0x7f5c780c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c7db9c700 1 -- 172.21.15.7:0/72242301 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7806b460 con 0x7f5c780c4ce0 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c7db9c700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c780c4ce0 0x7f5c780c5180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43e071d779580d0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c6ffff700 1 -- 172.21.15.7:0/72242301 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000e070 con 0x7f5c780c4ce0 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c6ffff700 1 -- 172.21.15.7:0/72242301 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000bb20 con 0x7f5c780c4ce0 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.505+0000 7f5c6ffff700 1 -- 172.21.15.7:0/72242301 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70012430 con 0x7f5c780c4ce0 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c780c4ce0 msgr2=0x7f5c780c5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c780c4ce0 0x7f5c780c5180 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009d90 tx=0x7f5c7000b330 comp rx=0 tx=0).stop 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/72242301 shutdown_connections 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c780c4ce0 0x7f5c780c5180 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/72242301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781535e0 0x7f5c78159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/72242301 >> 172.21.15.7:0/72242301 conn(0x7f5c780737f0 msgr2=0x7f5c78072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/72242301 shutdown_connections 2023-06-09T11:52:44.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.506+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/72242301 wait complete. 2023-06-09T11:52:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.508+0000 7f5c7f5ff700 1 Processor -- start 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.509+0000 7f5c7f5ff700 1 -- start start 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 0x7f5c780c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781539b0 0x7f5c780c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c8700 con 0x7f5c780c4ce0 2023-06-09T11:52:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780c89d0 con 0x7f5c781535e0 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78152c40 con 0x7f5c781539b0 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60478/0 (socket says 172.21.15.7:60478) 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 -- 172.21.15.7:0/2840621953 learned_addr learned my addr 172.21.15.7:0/2840621953 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7cb9a700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 0x7f5c780c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7db9c700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781539b0 0x7f5c780c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 -- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781539b0 msgr2=0x7f5c780c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781539b0 0x7f5c780c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 -- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 msgr2=0x7f5c780c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 0x7f5c780c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 -- 172.21.15.7:0/2840621953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009a70 con 0x7f5c780c4ce0 2023-06-09T11:52:44.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7cb9a700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 0x7f5c780c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:52:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.510+0000 7f5c7d39b700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e9b0 tx=0x7f5c7400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.511+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74009710 con 0x7f5c780c4ce0 2023-06-09T11:52:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.511+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400c850 con 0x7f5c780c4ce0 2023-06-09T11:52:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.511+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400cbc0 con 0x7f5c780c4ce0 2023-06-09T11:52:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.511+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7806a850 con 0x7f5c780c4ce0 2023-06-09T11:52:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.511+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7806a570 con 0x7f5c780c4ce0 2023-06-09T11:52:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.512+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7806ab30 con 0x7f5c780c4ce0 2023-06-09T11:52:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.513+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c7400cd20 con 0x7f5c780c4ce0 2023-06-09T11:52:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.514+0000 7f5c6dffb700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 0x7f5c5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.514+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f5c7408a150 con 0x7f5c780c4ce0 2023-06-09T11:52:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.514+0000 7f5c7cb9a700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 0x7f5c5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.515+0000 7f5c7cb9a700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 0x7f5c5c06d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c5520 tx=0x7f5c68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:44.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.518+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5c74058490 con 0x7f5c780c4ce0 2023-06-09T11:52:44.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.785+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c780c7b30 con 0x7f5c780c4ce0 2023-06-09T11:52:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.786+0000 7f5c6dffb700 1 -- 172.21.15.7:0/2840621953 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f5c74058020 con 0x7f5c780c4ce0 2023-06-09T11:52:44.789 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:44.790 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:39.696310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 msgr2=0x7f5c5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 0x7f5c5c06d190 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5c780c5520 tx=0x7f5c68009040 comp rx=0 tx=0).stop 2023-06-09T11:52:44.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 msgr2=0x7f5c78156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400e9b0 tx=0x7f5c7400ed70 comp rx=0 tx=0).stop 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 shutdown_connections 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c5c06ace0 0x7f5c5c06d190 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5c781539b0 0x7f5c780c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c780c4ce0 0x7f5c78156710 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 --2- 172.21.15.7:0/2840621953 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5c781535e0 0x7f5c780c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:44.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.788+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 >> 172.21.15.7:0/2840621953 conn(0x7f5c780737f0 msgr2=0x7f5c781509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.789+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 shutdown_connections 2023-06-09T11:52:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:44.789+0000 7f5c7f5ff700 1 -- 172.21.15.7:0/2840621953 wait complete. 2023-06-09T11:52:44.794 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 194 2023-06-09T11:52:44.810 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:52:46.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.019+0000 7f4c451ff700 1 Processor -- start 2023-06-09T11:52:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.019+0000 7f4c451ff700 1 -- start start 2023-06-09T11:52:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4c40151510 0x7f4c400c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c40070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40071060 0x7f4c40158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c400714e0 con 0x7f4c40071060 2023-06-09T11:52:47.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c401518e0 con 0x7f4c40151510 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401580a0 con 0x7f4c4015b380 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c3f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40071060 0x7f4c40158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c3f59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4c40071060 0x7f4c40158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60480/0 (socket says 172.21.15.7:60480) 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c3f59e700 1 -- 172.21.15.7:0/1153442238 learned_addr learned my addr 172.21.15.7:0/1153442238 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c3ed9d700 1 --2- 172.21.15.7:0/1153442238 >> v2:172.21.15.117:3300/0 conn(0x7f4c40151510 0x7f4c400c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.020+0000 7f4c3e59c700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c40070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 -- 172.21.15.7:0/1153442238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 msgr2=0x7f4c40070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:47.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c40070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 -- 172.21.15.7:0/1153442238 >> v2:172.21.15.117:3300/0 conn(0x7f4c40151510 msgr2=0x7f4c400c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 --2- 172.21.15.7:0/1153442238 >> v2:172.21.15.117:3300/0 conn(0x7f4c40151510 0x7f4c400c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 -- 172.21.15.7:0/1153442238 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c400c6180 con 0x7f4c40071060 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3f59e700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40071060 0x7f4c40158870 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4c4006ff10 tx=0x7f4c3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd23e7e1ac0cce98 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3dd9b700 1 -- 172.21.15.7:0/1153442238 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800e070 con 0x7f4c40071060 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3dd9b700 1 -- 172.21.15.7:0/1153442238 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800b910 con 0x7f4c40071060 2023-06-09T11:52:47.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.021+0000 7f4c3dd9b700 1 -- 172.21.15.7:0/1153442238 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800bcc0 con 0x7f4c40071060 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 -- 172.21.15.7:0/1153442238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40071060 msgr2=0x7f4c40158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40071060 0x7f4c40158870 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f4c4006ff10 tx=0x7f4c3800b330 comp rx=0 tx=0).stop 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 -- 172.21.15.7:0/1153442238 shutdown_connections 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 --2- 172.21.15.7:0/1153442238 >> v2:172.21.15.117:3300/0 conn(0x7f4c40151510 0x7f4c400c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40071060 0x7f4c40158870 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 --2- 172.21.15.7:0/1153442238 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c40070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 -- 172.21.15.7:0/1153442238 >> 172.21.15.7:0/1153442238 conn(0x7f4c4006a4d0 msgr2=0x7f4c4006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 -- 172.21.15.7:0/1153442238 shutdown_connections 2023-06-09T11:52:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.022+0000 7f4c451ff700 1 -- 172.21.15.7:0/1153442238 wait complete. 2023-06-09T11:52:47.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.025+0000 7f4c451ff700 1 Processor -- start 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.025+0000 7f4c451ff700 1 -- start start 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c40071060 0x7f4c401470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c401f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c400c1d70 con 0x7f4c40151510 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401f0010 con 0x7f4c40071060 2023-06-09T11:52:47.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c401f2c40 con 0x7f4c4015b380 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c3e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c3ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c40071060 0x7f4c401470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c3e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60484/0 (socket says 172.21.15.7:60484) 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c3e59c700 1 -- 172.21.15.7:0/165146430 learned_addr learned my addr 172.21.15.7:0/165146430 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.026+0000 7f4c3f59e700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c401f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 -- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 msgr2=0x7f4c401f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:47.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c401f2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 -- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c40071060 msgr2=0x7f4c401470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c40071060 0x7f4c401470d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 -- 172.21.15.7:0/165146430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c38009b30 con 0x7f4c40151510 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c3e59c700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400e870 tx=0x7f4c3400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c34009590 con 0x7f4c40151510 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c34013680 con 0x7f4c40151510 2023-06-09T11:52:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c340139f0 con 0x7f4c40151510 2023-06-09T11:52:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c40157770 con 0x7f4c40151510 2023-06-09T11:52:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.027+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c40157490 con 0x7f4c40151510 2023-06-09T11:52:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.028+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c40157a50 con 0x7f4c40151510 2023-06-09T11:52:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.029+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4c34013b50 con 0x7f4c40151510 2023-06-09T11:52:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.030+0000 7f4c277fe700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 0x7f4c2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.030+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f4c34089bb0 con 0x7f4c40151510 2023-06-09T11:52:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.031+0000 7f4c3ed9d700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 0x7f4c2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.031+0000 7f4c3ed9d700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 0x7f4c2c06d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009b60 tx=0x7f4c30009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.035+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4c34057dc0 con 0x7f4c40151510 2023-06-09T11:52:47.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.287+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c4015ae40 con 0x7f4c40151510 2023-06-09T11:52:47.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.288+0000 7f4c277fe700 1 -- 172.21.15.7:0/165146430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 194 v194) v1 ==== 95+0+5463 (secure 0 0 0) 0x7f4c34021020 con 0x7f4c40151510 2023-06-09T11:52:47.292 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:47.292 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":194,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":194,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:39.696310+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:47.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.290+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 msgr2=0x7f4c2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:47.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.290+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 0x7f4c2c06d190 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c30009b60 tx=0x7f4c30009210 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 msgr2=0x7f4c401f2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4c3400e870 tx=0x7f4c3400eb80 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 shutdown_connections 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4c2c06ace0 0x7f4c2c06d190 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4c4015b380 0x7f4c401f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c40071060 0x7f4c401470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 --2- 172.21.15.7:0/165146430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c40151510 0x7f4c401f2110 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:47.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 >> 172.21.15.7:0/165146430 conn(0x7f4c4006a4d0 msgr2=0x7f4c4014cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 shutdown_connections 2023-06-09T11:52:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:47.291+0000 7f4c451ff700 1 -- 172.21.15.7:0/165146430 wait complete. 2023-06-09T11:52:47.296 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 194 2023-06-09T11:52:47.312 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:52:49.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.520+0000 7f62d53ff700 1 Processor -- start 2023-06-09T11:52:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.521+0000 7f62d53ff700 1 -- start start 2023-06-09T11:52:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62d53ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62d015b4a0 0x7f62d015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62d53ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d015a500 con 0x7f62d0153880 2023-06-09T11:52:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62d53ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d015da00 con 0x7f62d00c9f40 2023-06-09T11:52:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62d53ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d00c5f10 con 0x7f62d015b4a0 2023-06-09T11:52:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62ce7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62d015b4a0 0x7f62d015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62ce7fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f62d015b4a0 0x7f62d015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60494/0 (socket says 172.21.15.7:60494) 2023-06-09T11:52:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.522+0000 7f62ce7fc700 1 -- 172.21.15.7:0/3377833172 learned_addr learned my addr 172.21.15.7:0/3377833172 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62ce7fc700 1 -- 172.21.15.7:0/3377833172 >> v1:172.21.15.117:6789/0 conn(0x7f62d00c9f40 legacy=0x7f62d00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62ce7fc700 1 -- 172.21.15.7:0/3377833172 >> v1:172.21.15.7:6789/0 conn(0x7f62d0153880 legacy=0x7f62d015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62ce7fc700 1 -- 172.21.15.7:0/3377833172 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d01596e0 con 0x7f62d015b4a0 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62ce7fc700 1 --2- 172.21.15.7:0/3377833172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d015d4c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f62c40098f0 tx=0x7f62c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314fb6160b389a3d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62cdffb700 1 -- 172.21.15.7:0/3377833172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400b8a0 con 0x7f62d015b4a0 2023-06-09T11:52:49.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62cdffb700 1 -- 172.21.15.7:0/3377833172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c400ba00 con 0x7f62d015b4a0 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.523+0000 7f62cdffb700 1 -- 172.21.15.7:0/3377833172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c400bd90 con 0x7f62d015b4a0 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 -- 172.21.15.7:0/3377833172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 msgr2=0x7f62d015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 --2- 172.21.15.7:0/3377833172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d015d4c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f62c40098f0 tx=0x7f62c4009c00 comp rx=0 tx=0).stop 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 -- 172.21.15.7:0/3377833172 shutdown_connections 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 --2- 172.21.15.7:0/3377833172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d015d4c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 -- 172.21.15.7:0/3377833172 >> 172.21.15.7:0/3377833172 conn(0x7f62d00737f0 msgr2=0x7f62d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 -- 172.21.15.7:0/3377833172 shutdown_connections 2023-06-09T11:52:49.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.524+0000 7f62d53ff700 1 -- 172.21.15.7:0/3377833172 wait complete. 2023-06-09T11:52:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.526+0000 7f62d53ff700 1 Processor -- start 2023-06-09T11:52:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.526+0000 7f62d53ff700 1 -- start start 2023-06-09T11:52:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d0151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d01e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01f2650 con 0x7f62d015b4a0 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d0152610 con 0x7f62d00c9f40 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62d53ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d01f2370 con 0x7f62d0153880 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60504/0 (socket says 172.21.15.7:60504) 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 -- 172.21.15.7:0/2588337374 learned_addr learned my addr 172.21.15.7:0/2588337374 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62ceffd700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d0151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62ce7fc700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d01e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 -- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 msgr2=0x7f62d01e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d01e6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 -- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 msgr2=0x7f62d0151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d0151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.527+0000 7f62cf7fe700 1 -- 172.21.15.7:0/2588337374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4009550 con 0x7f62d015b4a0 2023-06-09T11:52:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62cf7fe700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f62c800cc50 tx=0x7f62c800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003c00 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c8003d60 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d0159c60 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c800aa50 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.528+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d0159980 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.529+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0159f40 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.530+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f62c800abb0 con 0x7f62d015b4a0 2023-06-09T11:52:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.531+0000 7f62b77fe700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.531+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f62c8089a20 con 0x7f62d015b4a0 2023-06-09T11:52:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.531+0000 7f62ceffd700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.532+0000 7f62ceffd700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 0x7f62bc06d1c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f62c0007fd0 tx=0x7f62c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.535+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f62c80542c0 con 0x7f62d015b4a0 2023-06-09T11:52:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.792+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62d015da00 con 0x7f62d015b4a0 2023-06-09T11:52:49.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.793+0000 7f62b77fe700 1 -- 172.21.15.7:0/2588337374 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f62d015da00 con 0x7f62d015b4a0 2023-06-09T11:52:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:49.797 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":195,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:47.870185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:resolve","state_seq":38,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:49.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 msgr2=0x7f62bc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 0x7f62bc06d1c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f62c0007fd0 tx=0x7f62c0009040 comp rx=0 tx=0).stop 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 msgr2=0x7f62d01f2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f62c800cc50 tx=0x7f62c800cf60 comp rx=0 tx=0).stop 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 shutdown_connections 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62d015b4a0 0x7f62d01f2ae0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f62d00c9f40 0x7f62d0151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f62bc06ad10 0x7f62bc06d1c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 --2- 172.21.15.7:0/2588337374 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f62d0153880 0x7f62d01e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 >> 172.21.15.7:0/2588337374 conn(0x7f62d00737f0 msgr2=0x7f62d006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 shutdown_connections 2023-06-09T11:52:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:49.796+0000 7f62d53ff700 1 -- 172.21.15.7:0/2588337374 wait complete. 2023-06-09T11:52:49.802 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 195 2023-06-09T11:52:49.817 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:52:51.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.028+0000 7f4df3fff700 1 Processor -- start 2023-06-09T11:52:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.028+0000 7f4df3fff700 1 -- start start 2023-06-09T11:52:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.028+0000 7f4df3fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 0x7f4dec15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.028+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df3fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec15a500 con 0x7f4dec15b2d0 2023-06-09T11:52:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df3fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0c5f10 con 0x7f4dec153880 2023-06-09T11:52:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df3fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec15da00 con 0x7f4dec0c9f40 2023-06-09T11:52:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df159a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df1d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 0x7f4dec15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df159a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:54776/0 (socket says 172.21.15.7:54776) 2023-06-09T11:52:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df159a700 1 -- 172.21.15.7:0/2578188149 learned_addr learned my addr 172.21.15.7:0/2578188149 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df259c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f4dec15b2d0 0x7f4dec0c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:56240/0 (socket says 172.21.15.7:56240) 2023-06-09T11:52:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.029+0000 7f4df1d9b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 0x7f4dec15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:39332/0 (socket says 172.21.15.7:39332) 2023-06-09T11:52:52.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df159a700 1 -- 172.21.15.7:0/2578188149 >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 msgr2=0x7f4dec15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:52.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df159a700 1 --2- 172.21.15.7:0/2578188149 >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 0x7f4dec15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df159a700 1 -- 172.21.15.7:0/2578188149 >> v1:172.21.15.117:6789/0 conn(0x7f4dec15b2d0 legacy=0x7f4dec0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df159a700 1 -- 172.21.15.7:0/2578188149 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec15b750 con 0x7f4dec0c9f40 2023-06-09T11:52:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df159a700 1 --2- 172.21.15.7:0/2578188149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0098f0 tx=0x7f4ddc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f91aba7e2f2c75b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df0d99700 1 -- 172.21.15.7:0/2578188149 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc011830 con 0x7f4dec0c9f40 2023-06-09T11:52:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df0d99700 1 -- 172.21.15.7:0/2578188149 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ddc011990 con 0x7f4dec0c9f40 2023-06-09T11:52:52.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.030+0000 7f4df0d99700 1 -- 172.21.15.7:0/2578188149 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ddc011d20 con 0x7f4dec0c9f40 2023-06-09T11:52:52.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 -- 172.21.15.7:0/2578188149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 msgr2=0x7f4dec15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 --2- 172.21.15.7:0/2578188149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0098f0 tx=0x7f4ddc009c00 comp rx=0 tx=0).stop 2023-06-09T11:52:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 -- 172.21.15.7:0/2578188149 shutdown_connections 2023-06-09T11:52:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 --2- 172.21.15.7:0/2578188149 >> v2:172.21.15.117:3300/0 conn(0x7f4dec153880 0x7f4dec15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 --2- 172.21.15.7:0/2578188149 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec0c9f40 0x7f4dec15ad90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 -- 172.21.15.7:0/2578188149 >> 172.21.15.7:0/2578188149 conn(0x7f4dec0737f0 msgr2=0x7f4dec0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:52.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 -- 172.21.15.7:0/2578188149 shutdown_connections 2023-06-09T11:52:52.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.031+0000 7f4df3fff700 1 -- 172.21.15.7:0/2578188149 wait complete. 2023-06-09T11:52:52.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.033+0000 7f4df3fff700 1 Processor -- start 2023-06-09T11:52:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.033+0000 7f4df3fff700 1 -- start start 2023-06-09T11:52:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.033+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 0x7f4dec0b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec153880 0x7f4dec0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 0x7f4dec1f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df3fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1f3640 con 0x7f4dec15b2d0 2023-06-09T11:52:52.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df3fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0b50c0 con 0x7f4dec0c9f40 2023-06-09T11:52:52.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df3fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec0caf00 con 0x7f4dec153880 2023-06-09T11:52:52.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 0x7f4dec1f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df1d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 0x7f4dec0b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df1d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 0x7f4dec0b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:39336/0 (socket says 172.21.15.7:39336) 2023-06-09T11:52:52.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df159a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec153880 0x7f4dec0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df1d9b700 1 -- 172.21.15.7:0/1487997418 learned_addr learned my addr 172.21.15.7:0/1487997418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:52.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 -- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec153880 msgr2=0x7f4dec0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec153880 0x7f4dec0cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 -- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 msgr2=0x7f4dec0b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 0x7f4dec0b4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.034+0000 7f4df259c700 1 -- 172.21.15.7:0/1487997418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc009550 con 0x7f4dec15b2d0 2023-06-09T11:52:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4df259c700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 0x7f4dec1f2930 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4de400caa0 tx=0x7f4de400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400a940 con 0x7f4dec15b2d0 2023-06-09T11:52:52.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de400aaa0 con 0x7f4dec15b2d0 2023-06-09T11:52:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de400ae10 con 0x7f4dec15b2d0 2023-06-09T11:52:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec159980 con 0x7f4dec15b2d0 2023-06-09T11:52:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.035+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec1596e0 con 0x7f4dec15b2d0 2023-06-09T11:52:52.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.036+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec159c60 con 0x7f4dec15b2d0 2023-06-09T11:52:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.037+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4de4007070 con 0x7f4dec15b2d0 2023-06-09T11:52:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.038+0000 7f4de27fc700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 0x7f4dd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.039+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f4de4089680 con 0x7f4dec15b2d0 2023-06-09T11:52:52.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.039+0000 7f4df1d9b700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 0x7f4dd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:52.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.039+0000 7f4df1d9b700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 0x7f4dd006d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ca0 tx=0x7f4de8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:52.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.043+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4de4053f70 con 0x7f4dec15b2d0 2023-06-09T11:52:52.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.299+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dec0b38c0 con 0x7f4dec15b2d0 2023-06-09T11:52:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.300+0000 7f4de27fc700 1 -- 172.21.15.7:0/1487997418 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 195 v195) v1 ==== 95+0+5465 (secure 0 0 0) 0x7f4dec0b38c0 con 0x7f4dec15b2d0 2023-06-09T11:52:52.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:52.304 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":195,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":195,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:47.870185+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:resolve","state_seq":38,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:52.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.302+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 msgr2=0x7f4dd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.302+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 0x7f4dd006d190 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009ca0 tx=0x7f4de8009210 comp rx=0 tx=0).stop 2023-06-09T11:52:52.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.302+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 msgr2=0x7f4dec1f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:52.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.302+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 0x7f4dec1f2930 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4de400caa0 tx=0x7f4de400cdb0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 shutdown_connections 2023-06-09T11:52:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4dec15b2d0 0x7f4dec1f2930 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4dec0c9f40 0x7f4dec0b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4dd006ace0 0x7f4dd006d190 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 --2- 172.21.15.7:0/1487997418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4dec153880 0x7f4dec0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 >> 172.21.15.7:0/1487997418 conn(0x7f4dec0737f0 msgr2=0x7f4dec0720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 shutdown_connections 2023-06-09T11:52:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:52.303+0000 7f4df3fff700 1 -- 172.21.15.7:0/1487997418 wait complete. 2023-06-09T11:52:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 195 2023-06-09T11:52:52.324 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:52:52.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:52.738 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:52:54.324 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:54.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.533+0000 7f00aebff700 1 Processor -- start 2023-06-09T11:52:54.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.534+0000 7f00aebff700 1 -- start start 2023-06-09T11:52:54.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00aebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f00a815b4a0 0x7f00a815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00aebff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a815a500 con 0x7f00a80c9f40 2023-06-09T11:52:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00aebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a815da00 con 0x7f00a8153880 2023-06-09T11:52:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00aebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a80c5f10 con 0x7f00a815b4a0 2023-06-09T11:52:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00ad19c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f00a80c9f40 0x7f00a80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43838/0 (socket says 172.21.15.7:43838) 2023-06-09T11:52:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.535+0000 7f00ad19c700 1 -- 172.21.15.7:0/4175215622 learned_addr learned my addr 172.21.15.7:0/4175215622 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:54.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a7fff700 1 --2- 172.21.15.7:0/4175215622 >> v2:172.21.15.117:3300/0 conn(0x7f00a815b4a0 0x7f00a815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:54.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a77fe700 1 -- 172.21.15.7:0/4175215622 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794325802 0 0) 0x7f00a0001490 con 0x7f00a80c9f40 2023-06-09T11:52:54.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a77fe700 1 -- 172.21.15.7:0/4175215622 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00980035b0 con 0x7f00a80c9f40 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a7fff700 1 -- 172.21.15.7:0/4175215622 >> v1:172.21.15.117:6789/0 conn(0x7f00a8153880 legacy=0x7f00a815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a7fff700 1 -- 172.21.15.7:0/4175215622 >> v1:172.21.15.7:6789/0 conn(0x7f00a80c9f40 legacy=0x7f00a80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a7fff700 1 -- 172.21.15.7:0/4175215622 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a81596e0 con 0x7f00a815b4a0 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.536+0000 7f00a7fff700 1 --2- 172.21.15.7:0/4175215622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a815d4c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0094009840 tx=0x7f0094009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bd13adffbb7a49 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00a77fe700 1 -- 172.21.15.7:0/4175215622 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094011830 con 0x7f00a815b4a0 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00a77fe700 1 -- 172.21.15.7:0/4175215622 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0094011990 con 0x7f00a815b4a0 2023-06-09T11:52:54.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00a77fe700 1 -- 172.21.15.7:0/4175215622 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094011d20 con 0x7f00a815b4a0 2023-06-09T11:52:54.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 -- 172.21.15.7:0/4175215622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 msgr2=0x7f00a815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 --2- 172.21.15.7:0/4175215622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a815d4c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0094009840 tx=0x7f0094009b50 comp rx=0 tx=0).stop 2023-06-09T11:52:54.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 -- 172.21.15.7:0/4175215622 shutdown_connections 2023-06-09T11:52:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 --2- 172.21.15.7:0/4175215622 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a815d4c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 -- 172.21.15.7:0/4175215622 >> 172.21.15.7:0/4175215622 conn(0x7f00a80737f0 msgr2=0x7f00a80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 -- 172.21.15.7:0/4175215622 shutdown_connections 2023-06-09T11:52:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.537+0000 7f00aebff700 1 -- 172.21.15.7:0/4175215622 wait complete. 2023-06-09T11:52:54.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.539+0000 7f00aebff700 1 Processor -- start 2023-06-09T11:52:54.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.539+0000 7f00aebff700 1 -- start start 2023-06-09T11:52:54.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a80c9f40 0x7f00a80c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:54.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a8148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80c6da0 con 0x7f00a8153880 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a81f0120 con 0x7f00a815b4a0 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00aebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80c7c70 con 0x7f00a80c9f40 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48612/0 (socket says 172.21.15.7:48612) 2023-06-09T11:52:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.540+0000 7f00a7fff700 1 -- 172.21.15.7:0/3591950242 learned_addr learned my addr 172.21.15.7:0/3591950242 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00ad19c700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a8148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00ac99b700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a80c9f40 0x7f00a80c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 -- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a80c9f40 msgr2=0x7f00a80c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a80c9f40 0x7f00a80c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 -- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 msgr2=0x7f00a8148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a8148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 -- 172.21.15.7:0/3591950242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00940094a0 con 0x7f00a8153880 2023-06-09T11:52:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00ad19c700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a8148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:52:54.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a7fff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f0094004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940038a0 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009401e070 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a8159c60 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.541+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0094003c00 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.542+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a8159980 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.542+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a8159f40 con 0x7f00a8153880 2023-06-09T11:52:54.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.543+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f009402d450 con 0x7f00a8153880 2023-06-09T11:52:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.545+0000 7f00a57fa700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 0x7f008c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.545+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f009408ac80 con 0x7f00a8153880 2023-06-09T11:52:54.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.545+0000 7f00ac99b700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 0x7f008c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:54.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.546+0000 7f00ac99b700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 0x7f008c06d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f009c000f60 tx=0x7f009c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:54.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.549+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0094055520 con 0x7f00a8153880 2023-06-09T11:52:54.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.808+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00a8074cf0 con 0x7f00a8153880 2023-06-09T11:52:54.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.809+0000 7f00a57fa700 1 -- 172.21.15.7:0/3591950242 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f0094015020 con 0x7f00a8153880 2023-06-09T11:52:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:54.812 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":196,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:53.892082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:reconnect","state_seq":40,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 msgr2=0x7f008c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 0x7f008c06d190 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f009c000f60 tx=0x7f009c009040 comp rx=0 tx=0).stop 2023-06-09T11:52:54.814 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 msgr2=0x7f00a80b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0094000c00 tx=0x7f0094004750 comp rx=0 tx=0).stop 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 shutdown_connections 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f00a815b4a0 0x7f00a8148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f008c06ace0 0x7f008c06d190 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f00a80c9f40 0x7f00a80c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 --2- 172.21.15.7:0/3591950242 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00a8153880 0x7f00a80b3fb0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.811+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 >> 172.21.15.7:0/3591950242 conn(0x7f00a80737f0 msgr2=0x7f00a814ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.812+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 shutdown_connections 2023-06-09T11:52:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:54.812+0000 7f00aebff700 1 -- 172.21.15.7:0/3591950242 wait complete. 2023-06-09T11:52:54.816 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 196 2023-06-09T11:52:54.833 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 196, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 196, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7405": { "addr": "172.21.15.117:6835/3642322665", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3642322665, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3642322665, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7405, "incarnation": 178, "join_fscid": -1, "name": "d", "rank": 1, "state": "up:active", "state_seq": 18 }, "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:reconnect", "state_seq": 40 } }, "last_failure": 0, "last_failure_osd_epoch": 72, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:52:53.892082+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7405, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:52:56.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.043+0000 7f708dfff700 1 Processor -- start 2023-06-09T11:52:57.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.043+0000 7f708dfff700 1 -- start start 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f708dfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f708815a0c0 0x7f70881501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f708dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f708dfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880c0bf0 con 0x7f70880c1bc0 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f708dfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7088070110 con 0x7f708815a0c0 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f708dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7088071300 con 0x7f70880c0140 2023-06-09T11:52:57.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f70877fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f70880c1bc0 0x7f70881599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43840/0 (socket says 172.21.15.7:43840) 2023-06-09T11:52:57.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f70877fe700 1 -- 172.21.15.7:0/2241028730 learned_addr learned my addr 172.21.15.7:0/2241028730 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:57.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.044+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2241028730 >> v2:172.21.15.7:3300/0 conn(0x7f708815a0c0 0x7f70881501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7087fff700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f70867fc700 1 -- 172.21.15.7:0/2241028730 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035586581 0 0) 0x7f70880c0bf0 con 0x7f70880c1bc0 2023-06-09T11:52:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f70867fc700 1 -- 172.21.15.7:0/2241028730 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70700035b0 con 0x7f70880c1bc0 2023-06-09T11:52:57.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7086ffd700 1 -- 172.21.15.7:0/2241028730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 msgr2=0x7f70881506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7086ffd700 1 -- 172.21.15.7:0/2241028730 >> v1:172.21.15.7:6789/0 conn(0x7f70880c1bc0 legacy=0x7f70881599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7086ffd700 1 -- 172.21.15.7:0/2241028730 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880702f0 con 0x7f708815a0c0 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.045+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f708815a0c0 0x7f70881501b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39e1ecdd57c1d93 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f70867fc700 1 -- 172.21.15.7:0/2241028730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c011830 con 0x7f708815a0c0 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f70867fc700 1 -- 172.21.15.7:0/2241028730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c011990 con 0x7f708815a0c0 2023-06-09T11:52:57.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f70867fc700 1 -- 172.21.15.7:0/2241028730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c011d20 con 0x7f708815a0c0 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 -- 172.21.15.7:0/2241028730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f708815a0c0 msgr2=0x7f70881501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f708815a0c0 0x7f70881501b0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f707c009870 tx=0x7f707c009b80 comp rx=0 tx=0).stop 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 -- 172.21.15.7:0/2241028730 shutdown_connections 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 --2- 172.21.15.7:0/2241028730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f708815a0c0 0x7f70881501b0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 -- 172.21.15.7:0/2241028730 >> 172.21.15.7:0/2241028730 conn(0x7f708806a4d0 msgr2=0x7f708806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 -- 172.21.15.7:0/2241028730 shutdown_connections 2023-06-09T11:52:57.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.046+0000 7f708dfff700 1 -- 172.21.15.7:0/2241028730 wait complete. 2023-06-09T11:52:57.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.048+0000 7f708dfff700 1 Processor -- start 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.048+0000 7f708dfff700 1 -- start start 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f708815a0c0 0x7f70881592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708814b730 con 0x7f70880c1bc0 2023-06-09T11:52:57.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880c6770 con 0x7f708815a0c0 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f708dfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70881f2790 con 0x7f70880c0140 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f7086ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f7086ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48624/0 (socket says 172.21.15.7:48624) 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.049+0000 7f7086ffd700 1 -- 172.21.15.7:0/2247007045 learned_addr learned my addr 172.21.15.7:0/2247007045 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f70877fe700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7087fff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f708815a0c0 0x7f70881592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 -- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 msgr2=0x7f70881eebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881eebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 -- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f708815a0c0 msgr2=0x7f70881592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f708815a0c0 0x7f70881592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 -- 172.21.15.7:0/2247007045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c0094a0 con 0x7f70880c1bc0 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f7086ffd700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f707c0090b0 tx=0x7f707c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0036a0 con 0x7f70880c1bc0 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c01e070 con 0x7f70880c1bc0 2023-06-09T11:52:57.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c003a40 con 0x7f70880c1bc0 2023-06-09T11:52:57.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.050+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70880c1170 con 0x7f70880c1bc0 2023-06-09T11:52:57.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.051+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70880c0e90 con 0x7f70880c1bc0 2023-06-09T11:52:57.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.052+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70880c1450 con 0x7f70880c1bc0 2023-06-09T11:52:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.053+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f707c003ba0 con 0x7f70880c1bc0 2023-06-09T11:52:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.054+0000 7f706ffff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 0x7f707406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.054+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f707c08ad70 con 0x7f70880c1bc0 2023-06-09T11:52:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.054+0000 7f70877fe700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 0x7f707406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:57.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.055+0000 7f70877fe700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 0x7f707406d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f70780011e0 tx=0x7f7078009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:57.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.059+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f707c055580 con 0x7f70880c1bc0 2023-06-09T11:52:57.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.315+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7088074cf0 con 0x7f70880c1bc0 2023-06-09T11:52:57.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.316+0000 7f706ffff700 1 -- 172.21.15.7:0/2247007045 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 196 v196) v1 ==== 95+0+5467 (secure 0 0 0) 0x7f707c015020 con 0x7f70880c1bc0 2023-06-09T11:52:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:57.319 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":196,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":196,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:53.892082+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:reconnect","state_seq":40,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 msgr2=0x7f707406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 0x7f707406d190 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f70780011e0 tx=0x7f7078009040 comp rx=0 tx=0).stop 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 msgr2=0x7f70881f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f707c0090b0 tx=0x7f707c004750 comp rx=0 tx=0).stop 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 shutdown_connections 2023-06-09T11:52:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f708815a0c0 0x7f70881592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f707406ace0 0x7f707406d190 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70880c0140 0x7f70881eebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 --2- 172.21.15.7:0/2247007045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70880c1bc0 0x7f70881f2c20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 >> 172.21.15.7:0/2247007045 conn(0x7f708806a4d0 msgr2=0x7f708814de40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 shutdown_connections 2023-06-09T11:52:57.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:57.318+0000 7f708dfff700 1 -- 172.21.15.7:0/2247007045 wait complete. 2023-06-09T11:52:57.323 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 196 2023-06-09T11:52:57.339 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:52:59.340 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:52:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.549+0000 7fee807ff700 1 Processor -- start 2023-06-09T11:52:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.549+0000 7fee807ff700 1 -- start start 2023-06-09T11:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.550+0000 7fee807ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee7815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.550+0000 7fee807ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7815a500 con 0x7fee780c9f40 2023-06-09T11:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee807ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7815da00 con 0x7fee7815b4a0 2023-06-09T11:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee807ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c5f10 con 0x7fee78153880 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7dd9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fee780c9f40 0x7fee7815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43854/0 (socket says 172.21.15.7:43854) 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7dd9a700 1 -- 172.21.15.7:0/2188653358 learned_addr learned my addr 172.21.15.7:0/2188653358 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7e59b700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee7815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1056605148 0 0) 0x7fee68001110 con 0x7fee780c9f40 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee640035b0 con 0x7fee780c9f40 2023-06-09T11:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2952485330 0 0) 0x7fee640035b0 con 0x7fee780c9f40 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 msgr2=0x7fee7815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee7815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 >> v1:172.21.15.117:6789/0 conn(0x7fee7815b4a0 legacy=0x7fee780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.551+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee781596e0 con 0x7fee780c9f40 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fee68001800 con 0x7fee780c9f40 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 >> v1:172.21.15.7:6789/0 conn(0x7fee780c9f40 legacy=0x7fee7815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64003a30 0x7fee64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64004d90 0x7fee64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c5f10 con 0x7fee64004d90 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6400bac0 con 0x7fee64003a30 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6400bc50 con 0x7fee78153880 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64003a30 0x7fee64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7ed9c700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64004d90 0x7fee64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7dd9a700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 -- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 msgr2=0x7fee64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee78153880 0x7fee64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 -- 172.21.15.7:0/2188653358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64004d90 msgr2=0x7fee64005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64004d90 0x7fee64005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.552+0000 7fee7e59b700 1 -- 172.21.15.7:0/2188653358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640035b0 con 0x7fee64003a30 2023-06-09T11:52:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee7e59b700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64003a30 0x7fee64003e00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b60 tx=0x7fee64005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5627de41ce43640d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400e6c0 con 0x7fee64003a30 2023-06-09T11:52:59.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7400e820 con 0x7fee64003a30 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee7d599700 1 -- 172.21.15.7:0/2188653358 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400eb90 con 0x7fee64003a30 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee807ff700 1 -- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64003a30 msgr2=0x7fee64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee807ff700 1 --2- 172.21.15.7:0/2188653358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee64003a30 0x7fee64003e00 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fee74009b60 tx=0x7fee64005880 comp rx=0 tx=0).stop 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.553+0000 7fee7ed9c700 1 -- 172.21.15.7:0/2188653358 reap_dead start 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.554+0000 7fee807ff700 1 -- 172.21.15.7:0/2188653358 shutdown_connections 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.554+0000 7fee807ff700 1 -- 172.21.15.7:0/2188653358 >> 172.21.15.7:0/2188653358 conn(0x7fee780737f0 msgr2=0x7fee780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.554+0000 7fee807ff700 1 -- 172.21.15.7:0/2188653358 shutdown_connections 2023-06-09T11:52:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.554+0000 7fee807ff700 1 -- 172.21.15.7:0/2188653358 wait complete. 2023-06-09T11:52:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.556+0000 7fee807ff700 1 Processor -- start 2023-06-09T11:52:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.556+0000 7fee807ff700 1 -- start start 2023-06-09T11:52:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.557+0000 7fee807ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.557+0000 7fee807ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 0x7fee780cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.557+0000 7fee807ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 0x7fee7814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.557+0000 7fee807ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b4570 con 0x7fee64003a30 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.557+0000 7fee807ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b2eb0 con 0x7fee780ce620 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee807ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780b3190 con 0x7fee780cc720 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48644/0 (socket says 172.21.15.7:48644) 2023-06-09T11:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 -- 172.21.15.7:0/978380989 learned_addr learned my addr 172.21.15.7:0/978380989 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7dd9a700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 0x7fee780cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7ed9c700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 0x7fee7814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 -- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 msgr2=0x7fee7814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 0x7fee7814ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 -- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 msgr2=0x7fee780cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 0x7fee780cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 -- 172.21.15.7:0/978380989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78159c60 con 0x7fee64003a30 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7dd9a700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 0x7fee780cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7ed9c700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 0x7fee7814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.558+0000 7fee7e59b700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e4b0 tx=0x7fee740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.559+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7400e6c0 con 0x7fee64003a30 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.559+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee74003ad0 con 0x7fee64003a30 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.559+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee74003e80 con 0x7fee64003a30 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.559+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78159980 con 0x7fee64003a30 2023-06-09T11:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.559+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee78159f40 con 0x7fee64003a30 2023-06-09T11:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.560+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7815c5e0 con 0x7fee64003a30 2023-06-09T11:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.561+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee74013470 con 0x7fee64003a30 2023-06-09T11:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.562+0000 7fee6effd700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 0x7fee5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:52:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.562+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fee7408b480 con 0x7fee64003a30 2023-06-09T11:52:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.562+0000 7fee7dd9a700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 0x7fee5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:52:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.563+0000 7fee7dd9a700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 0x7fee5c06d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fee781eced0 tx=0x7fee68009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:52:59.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.566+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fee74055d20 con 0x7fee64003a30 2023-06-09T11:52:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.820+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee780cbcd0 con 0x7fee64003a30 2023-06-09T11:52:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.821+0000 7fee6effd700 1 -- 172.21.15.7:0/978380989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fee74059340 con 0x7fee64003a30 2023-06-09T11:52:59.823 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:52:59.824 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":197,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:58.905001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:rejoin","state_seq":42,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:52:59.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 msgr2=0x7fee5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 0x7fee5c06d190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fee781eced0 tx=0x7fee68009040 comp rx=0 tx=0).stop 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 msgr2=0x7fee7815bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fee7400e4b0 tx=0x7fee740047c0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 shutdown_connections 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fee5c06ace0 0x7fee5c06d190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fee780cc720 0x7fee7814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fee64003a30 0x7fee7815bba0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 --2- 172.21.15.7:0/978380989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fee780ce620 0x7fee780cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 >> 172.21.15.7:0/978380989 conn(0x7fee780737f0 msgr2=0x7fee781e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 shutdown_connections 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:52:59.823+0000 7fee807ff700 1 -- 172.21.15.7:0/978380989 wait complete. 2023-06-09T11:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 197 2023-06-09T11:52:59.843 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:53:01.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:53:02.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.053+0000 7fd240fff700 1 Processor -- start 2023-06-09T11:53:02.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.053+0000 7fd240fff700 1 -- start start 2023-06-09T11:53:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.054+0000 7fd240fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd23c15a0c0 0x7fd23c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.054+0000 7fd240fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c0c0bf0 con 0x7fd23c0c0140 2023-06-09T11:53:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.054+0000 7fd240fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c0702f0 con 0x7fd23c0c1bc0 2023-06-09T11:53:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.054+0000 7fd240fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c070110 con 0x7fd23c15a0c0 2023-06-09T11:53:02.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23ad9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd23c0c0140 0x7fd23c160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43868/0 (socket says 172.21.15.7:43868) 2023-06-09T11:53:02.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23ad9d700 1 -- 172.21.15.7:0/2981220091 learned_addr learned my addr 172.21.15.7:0/2981220091 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:53:02.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd239d9b700 1 --2- 172.21.15.7:0/2981220091 >> v2:172.21.15.117:3300/0 conn(0x7fd23c15a0c0 0x7fd23c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835416577 0 0) 0x7fd23c0c0bf0 con 0x7fd23c0c0140 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2240035b0 con 0x7fd23c0c0140 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 124017618 0 0) 0x7fd23c0702f0 con 0x7fd23c0c1bc0 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd23c0c0bf0 con 0x7fd23c0c1bc0 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd239d9b700 1 -- 172.21.15.7:0/2981220091 >> v1:172.21.15.117:6789/0 conn(0x7fd23c0c1bc0 legacy=0x7fd23c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd239d9b700 1 -- 172.21.15.7:0/2981220091 >> v1:172.21.15.7:6789/0 conn(0x7fd23c0c0140 legacy=0x7fd23c160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.055+0000 7fd239d9b700 1 -- 172.21.15.7:0/2981220091 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0c0e90 con 0x7fd23c15a0c0 2023-06-09T11:53:02.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.056+0000 7fd239d9b700 1 --2- 172.21.15.7:0/2981220091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1501b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd230009840 tx=0x7fd230009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5b1899edecaecd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:02.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.056+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011830 con 0x7fd23c15a0c0 2023-06-09T11:53:02.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.056+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230011990 con 0x7fd23c15a0c0 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.056+0000 7fd23959a700 1 -- 172.21.15.7:0/2981220091 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011d20 con 0x7fd23c15a0c0 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 -- 172.21.15.7:0/2981220091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c15a0c0 msgr2=0x7fd23c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 --2- 172.21.15.7:0/2981220091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1501b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd230009840 tx=0x7fd230009b50 comp rx=0 tx=0).stop 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 -- 172.21.15.7:0/2981220091 shutdown_connections 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 --2- 172.21.15.7:0/2981220091 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1501b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 -- 172.21.15.7:0/2981220091 >> 172.21.15.7:0/2981220091 conn(0x7fd23c06a4d0 msgr2=0x7fd23c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 -- 172.21.15.7:0/2981220091 shutdown_connections 2023-06-09T11:53:02.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.057+0000 7fd240fff700 1 -- 172.21.15.7:0/2981220091 wait complete. 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.058+0000 7fd240fff700 1 Processor -- start 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.059+0000 7fd240fff700 1 -- start start 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.059+0000 7fd240fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0c0140 0x7fd23c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd240fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd23c0c1bc0 0x7fd23c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd240fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd240fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c1576f0 con 0x7fd23c15a0c0 2023-06-09T11:53:02.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd240fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c14f820 con 0x7fd23c0c0140 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd240fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c14f260 con 0x7fd23c0c1bc0 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48652/0 (socket says 172.21.15.7:48652) 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 -- 172.21.15.7:0/225752128 learned_addr learned my addr 172.21.15.7:0/225752128 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23a59c700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0c0140 0x7fd23c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd239d9b700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd23c0c1bc0 0x7fd23c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 -- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd23c0c1bc0 msgr2=0x7fd23c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd23c0c1bc0 0x7fd23c0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 -- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0c0140 msgr2=0x7fd23c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0c0140 0x7fd23c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.060+0000 7fd23ad9d700 1 -- 172.21.15.7:0/225752128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2300094a0 con 0x7fd23c15a0c0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd23ad9d700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd23400cba0 tx=0x7fd23400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400aa40 con 0x7fd23c15a0c0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400aba0 con 0x7fd23c15a0c0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0c1450 con 0x7fd23c15a0c0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234012400 con 0x7fd23c15a0c0 2023-06-09T11:53:02.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.061+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0c1170 con 0x7fd23c15a0c0 2023-06-09T11:53:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.062+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c0705d0 con 0x7fd23c15a0c0 2023-06-09T11:53:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.063+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd23401b3b0 con 0x7fd23c15a0c0 2023-06-09T11:53:02.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.064+0000 7fd22affd700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 0x7fd21c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.064+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fd234089a00 con 0x7fd23c15a0c0 2023-06-09T11:53:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.064+0000 7fd23a59c700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 0x7fd21c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:02.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.065+0000 7fd23a59c700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 0x7fd21c06d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0011e0 tx=0x7fd22c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:02.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.068+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd234054210 con 0x7fd23c15a0c0 2023-06-09T11:53:02.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.320+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd23c0c7450 con 0x7fd23c15a0c0 2023-06-09T11:53:02.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.320+0000 7fd22affd700 1 -- 172.21.15.7:0/225752128 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5464 (secure 0 0 0) 0x7fd234017020 con 0x7fd23c15a0c0 2023-06-09T11:53:02.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:53:02.323 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":197,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:58.905001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:rejoin","state_seq":42,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:53:02.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 msgr2=0x7fd21c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 0x7fd21c06d190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0011e0 tx=0x7fd22c009040 comp rx=0 tx=0).stop 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 msgr2=0x7fd23c1f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd23400cba0 tx=0x7fd23400ceb0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 shutdown_connections 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23c15a0c0 0x7fd23c1f2320 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.326 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd23c0c0140 0x7fd23c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd21c06ace0 0x7fd21c06d190 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 --2- 172.21.15.7:0/225752128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd23c0c1bc0 0x7fd23c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 >> 172.21.15.7:0/225752128 conn(0x7fd23c06a4d0 msgr2=0x7fd23c14d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 shutdown_connections 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:02.323+0000 7fd240fff700 1 -- 172.21.15.7:0/225752128 wait complete. 2023-06-09T11:53:02.327 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 197 2023-06-09T11:53:02.344 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:53:04.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:53:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.554+0000 7f9188bff700 1 Processor -- start 2023-06-09T11:53:04.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.554+0000 7f9188bff700 1 -- start start 2023-06-09T11:53:04.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.555+0000 7f9188bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9184153880 0x7f918415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.555+0000 7f9188bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f918415b4a0 0x7f91840c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.555+0000 7f9188bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f918415a500 con 0x7f91840c9f40 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.555+0000 7f9188bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91840c5f10 con 0x7f918415b4a0 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.555+0000 7f9188bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918415da00 con 0x7f9184153880 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f918415b4a0 0x7f91840c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9181d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f91840c9f40 0x7f918415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48136/0 (socket says 172.21.15.7:48136) 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f918415b4a0 0x7f91840c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36916/0 (socket says 172.21.15.7:36916) 2023-06-09T11:53:04.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9181d9b700 1 -- 172.21.15.7:0/2492676711 learned_addr learned my addr 172.21.15.7:0/2492676711 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:53:04.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f918259c700 1 --2- 172.21.15.7:0/2492676711 >> v2:172.21.15.117:3300/0 conn(0x7f9184153880 0x7f918415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 -- 172.21.15.7:0/2492676711 >> v2:172.21.15.117:3300/0 conn(0x7f9184153880 msgr2=0x7f918415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 --2- 172.21.15.7:0/2492676711 >> v2:172.21.15.117:3300/0 conn(0x7f9184153880 0x7f918415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 -- 172.21.15.7:0/2492676711 >> v1:172.21.15.7:6789/0 conn(0x7f91840c9f40 legacy=0x7f918415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.556+0000 7f9182d9d700 1 -- 172.21.15.7:0/2492676711 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91841596e0 con 0x7f918415b4a0 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9182d9d700 1 --2- 172.21.15.7:0/2492676711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f918415b4a0 0x7f91840c4e30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f918408d340 tx=0x7f917c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e90718dbfdde4a5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f918159a700 1 -- 172.21.15.7:0/2492676711 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c00e070 con 0x7f918415b4a0 2023-06-09T11:53:04.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f918159a700 1 -- 172.21.15.7:0/2492676711 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c00bbd0 con 0x7f918415b4a0 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f918159a700 1 -- 172.21.15.7:0/2492676711 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c012430 con 0x7f918415b4a0 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 -- 172.21.15.7:0/2492676711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f918415b4a0 msgr2=0x7f91840c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 --2- 172.21.15.7:0/2492676711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f918415b4a0 0x7f91840c4e30 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f918408d340 tx=0x7f917c00b330 comp rx=0 tx=0).stop 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 -- 172.21.15.7:0/2492676711 shutdown_connections 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 --2- 172.21.15.7:0/2492676711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f918415b4a0 0x7f91840c4e30 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 --2- 172.21.15.7:0/2492676711 >> v2:172.21.15.117:3300/0 conn(0x7f9184153880 0x7f918415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.557+0000 7f9188bff700 1 -- 172.21.15.7:0/2492676711 >> 172.21.15.7:0/2492676711 conn(0x7f91840737f0 msgr2=0x7f91840720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:04.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.558+0000 7f9188bff700 1 -- 172.21.15.7:0/2492676711 shutdown_connections 2023-06-09T11:53:04.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.558+0000 7f9188bff700 1 -- 172.21.15.7:0/2492676711 wait complete. 2023-06-09T11:53:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.559+0000 7f9188bff700 1 Processor -- start 2023-06-09T11:53:04.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.559+0000 7f9188bff700 1 -- start start 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9184153880 0x7f91840cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f918415b4a0 0x7f918414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840c9c20 con 0x7f91840c9f40 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91840cd3a0 con 0x7f918415b4a0 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f9188bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91840cc0e0 con 0x7f9184153880 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.560+0000 7f918259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36922/0 (socket says 172.21.15.7:36922) 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 -- 172.21.15.7:0/3886040880 learned_addr learned my addr 172.21.15.7:0/3886040880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9181d9b700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9184153880 0x7f91840cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9182d9d700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f918415b4a0 0x7f918414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 -- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9184153880 msgr2=0x7f91840cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9184153880 0x7f91840cc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 -- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f918415b4a0 msgr2=0x7f918414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f918415b4a0 0x7f918414b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 -- 172.21.15.7:0/3886040880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c009a70 con 0x7f91840c9f40 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f918259c700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f917400e840 tx=0x7f917400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174013610 con 0x7f91840c9f40 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9174013770 con 0x7f91840c9f40 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9174013b20 con 0x7f91840c9f40 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.561+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9184159c60 con 0x7f91840c9f40 2023-06-09T11:53:04.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.562+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9184159980 con 0x7f91840c9f40 2023-06-09T11:53:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.563+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9184159f40 con 0x7f91840c9f40 2023-06-09T11:53:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.564+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9174013c80 con 0x7f91840c9f40 2023-06-09T11:53:04.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.564+0000 7f9172ffd700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 0x7f916406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.565+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f91740898e0 con 0x7f91840c9f40 2023-06-09T11:53:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.565+0000 7f9181d9b700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 0x7f916406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:04.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.565+0000 7f9181d9b700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 0x7f916406d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f91840ce0b0 tx=0x7f9178009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:04.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.569+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9174054220 con 0x7f91840c9f40 2023-06-09T11:53:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.822+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9184074cf0 con 0x7f91840c9f40 2023-06-09T11:53:04.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.823+0000 7f9172ffd700 1 -- 172.21.15.7:0/3886040880 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 197 v197) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f9174021020 con 0x7f91840c9f40 2023-06-09T11:53:04.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:53:04.826 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":197,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":197,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:52:58.905001+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:rejoin","state_seq":42,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 msgr2=0x7f916406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 0x7f916406d190 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f91840ce0b0 tx=0x7f9178009040 comp rx=0 tx=0).stop 2023-06-09T11:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 msgr2=0x7f91841f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f917400e840 tx=0x7f917400ec00 comp rx=0 tx=0).stop 2023-06-09T11:53:04.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 shutdown_connections 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f918415b4a0 0x7f918414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91840c9f40 0x7f91841f23b0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f916406ace0 0x7f916406d190 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 --2- 172.21.15.7:0/3886040880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9184153880 0x7f91840cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 >> 172.21.15.7:0/3886040880 conn(0x7f91840737f0 msgr2=0x7f918415b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.825+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 shutdown_connections 2023-06-09T11:53:04.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:04.826+0000 7f9188bff700 1 -- 172.21.15.7:0/3886040880 wait complete. 2023-06-09T11:53:04.830 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 197 2023-06-09T11:53:04.846 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:53:06.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:53:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.056+0000 7f87661ff700 1 Processor -- start 2023-06-09T11:53:07.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.056+0000 7f87661ff700 1 -- start start 2023-06-09T11:53:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f87661ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 0x7f876015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f87661ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600c88d0 con 0x7f8760153a70 2023-06-09T11:53:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f87661ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8760153e40 con 0x7f87600c2d90 2023-06-09T11:53:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f87661ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876015a630 con 0x7f87600c26b0 2023-06-09T11:53:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875f7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8760153a70 0x7f87600ca7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48152/0 (socket says 172.21.15.7:48152) 2023-06-09T11:53:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875f7fe700 1 -- 172.21.15.7:0/1099689940 learned_addr learned my addr 172.21.15.7:0/1099689940 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:53:07.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875ffff700 1 --2- 172.21.15.7:0/1099689940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 0x7f876015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644838850 0 0) 0x7f87600c88d0 con 0x7f8760153a70 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87480035b0 con 0x7f8760153a70 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.058+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052657757 0 0) 0x7f8754001110 con 0x7f87600c2d90 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87600c88d0 con 0x7f87600c2d90 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875ffff700 1 -- 172.21.15.7:0/1099689940 >> v1:172.21.15.117:6789/0 conn(0x7f87600c2d90 legacy=0x7f87600cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875ffff700 1 -- 172.21.15.7:0/1099689940 >> v1:172.21.15.7:6789/0 conn(0x7f8760153a70 legacy=0x7f87600ca7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875ffff700 1 -- 172.21.15.7:0/1099689940 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600ca350 con 0x7f87600c26b0 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875ffff700 1 --2- 172.21.15.7:0/1099689940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 0x7f876015a0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8758009dc0 tx=0x7f875800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30dcae810d1729b5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800e070 con 0x7f87600c26b0 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875800bb20 con 0x7f87600c26b0 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.059+0000 7f875e7fc700 1 -- 172.21.15.7:0/1099689940 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758012430 con 0x7f87600c26b0 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 -- 172.21.15.7:0/1099689940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 msgr2=0x7f876015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 --2- 172.21.15.7:0/1099689940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 0x7f876015a0f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8758009dc0 tx=0x7f875800b330 comp rx=0 tx=0).stop 2023-06-09T11:53:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 -- 172.21.15.7:0/1099689940 shutdown_connections 2023-06-09T11:53:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 --2- 172.21.15.7:0/1099689940 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c26b0 0x7f876015a0f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 -- 172.21.15.7:0/1099689940 >> 172.21.15.7:0/1099689940 conn(0x7f876006e410 msgr2=0x7f876006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 -- 172.21.15.7:0/1099689940 shutdown_connections 2023-06-09T11:53:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.060+0000 7f87661ff700 1 -- 172.21.15.7:0/1099689940 wait complete. 2023-06-09T11:53:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.062+0000 7f87661ff700 1 Processor -- start 2023-06-09T11:53:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.062+0000 7f87661ff700 1 -- start start 2023-06-09T11:53:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87600c26b0 0x7f87601df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 0x7f87600c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600c4790 con 0x7f8760153a70 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600c5030 con 0x7f87600c26b0 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f87661ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87601dfc30 con 0x7f87600c2d90 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f875ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f875ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36936/0 (socket says 172.21.15.7:36936) 2023-06-09T11:53:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f875ffff700 1 -- 172.21.15.7:0/34535312 learned_addr learned my addr 172.21.15.7:0/34535312 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f875f7fe700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87600c26b0 0x7f87601df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.063+0000 7f875effd700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 0x7f87600c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 -- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 msgr2=0x7f87600c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 0x7f87600c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 -- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87600c26b0 msgr2=0x7f87601df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87600c26b0 0x7f87601df6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 -- 172.21.15.7:0/34535312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758009a70 con 0x7f8760153a70 2023-06-09T11:53:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875effd700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 0x7f87600c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:53:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f875ffff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f87600caf60 tx=0x7f8758004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875801c810 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8758020070 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875801cbf0 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87600c85f0 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.064+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87600c8350 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.065+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87600c9d00 con 0x7f8760153a70 2023-06-09T11:53:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.066+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f875801cd50 con 0x7f8760153a70 2023-06-09T11:53:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.067+0000 7f8747fff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 0x7f874c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.067+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f8758022070 con 0x7f8760153a70 2023-06-09T11:53:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.068+0000 7f875f7fe700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 0x7f874c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:07.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.068+0000 7f875f7fe700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 0x7f874c06d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8750000f60 tx=0x7f8750007e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:07.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.072+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f87580564f0 con 0x7f8760153a70 2023-06-09T11:53:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.325+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8760074cf0 con 0x7f8760153a70 2023-06-09T11:53:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.325+0000 7f8747fff700 1 -- 172.21.15.7:0/34535312 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f875802c020 con 0x7f8760153a70 2023-06-09T11:53:07.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:53:07.328 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":198,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:53:04.922730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:53:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 msgr2=0x7f874c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 0x7f874c06d190 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8750000f60 tx=0x7f8750007e00 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 msgr2=0x7f87601f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f87600caf60 tx=0x7f8758004ad0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 shutdown_connections 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8760153a70 0x7f87601f2790 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f87600c26b0 0x7f87601df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f874c06ace0 0x7f874c06d190 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 --2- 172.21.15.7:0/34535312 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f87600c2d90 0x7f87600c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 >> 172.21.15.7:0/34535312 conn(0x7f876006e410 msgr2=0x7f8760156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 shutdown_connections 2023-06-09T11:53:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:07.328+0000 7f87661ff700 1 -- 172.21.15.7:0/34535312 wait complete. 2023-06-09T11:53:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 198 2023-06-09T11:53:07.348 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=2, replacing gid=6650 2023-06-09T11:53:07.349 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 7 secs before reviving mds.f 2023-06-09T11:53:14.349 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.f 2023-06-09T11:53:14.350 INFO:tasks.ceph.mds.f:Restarting daemon 2023-06-09T11:53:14.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2023-06-09T11:53:14.355 INFO:tasks.ceph.mds.f:Started 2023-06-09T11:53:14.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:53:14.501 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T11:53:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.562+0000 7f609bfff700 1 Processor -- start 2023-06-09T11:53:14.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.562+0000 7f609bfff700 1 -- start start 2023-06-09T11:53:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094151510 0x7f60940c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f609415b380 0x7f6094070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60941580a0 con 0x7f6094071060 2023-06-09T11:53:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60940714e0 con 0x7f609415b380 2023-06-09T11:53:14.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60941518e0 con 0x7f6094151510 2023-06-09T11:53:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f609415b380 0x7f6094070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f609415b380 0x7f6094070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47924/0 (socket says 172.21.15.7:47924) 2023-06-09T11:53:14.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6094071060 0x7f6094158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45908/0 (socket says 172.21.15.7:45908) 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 -- 172.21.15.7:0/4043068892 learned_addr learned my addr 172.21.15.7:0/4043068892 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f6099d9b700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094151510 0x7f60940c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 -- 172.21.15.7:0/4043068892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094151510 msgr2=0x7f60940c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094151510 0x7f60940c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 -- 172.21.15.7:0/4043068892 >> v1:172.21.15.7:6789/0 conn(0x7f6094071060 legacy=0x7f6094158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f6098d99700 1 -- 172.21.15.7:0/4043068892 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797424448 0 0) 0x7f60941580a0 con 0x7f6094071060 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.564+0000 7f609959a700 1 -- 172.21.15.7:0/4043068892 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940c6180 con 0x7f609415b380 2023-06-09T11:53:14.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f609959a700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609415b380 0x7f6094070b20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f60840098f0 tx=0x7f6084009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c1e2af8f20f0f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f6098d99700 1 -- 172.21.15.7:0/4043068892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084011830 con 0x7f609415b380 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f6098d99700 1 -- 172.21.15.7:0/4043068892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084011990 con 0x7f609415b380 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f6098d99700 1 -- 172.21.15.7:0/4043068892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084011d20 con 0x7f609415b380 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f609bfff700 1 -- 172.21.15.7:0/4043068892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609415b380 msgr2=0x7f6094070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.565+0000 7f609bfff700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609415b380 0x7f6094070b20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f60840098f0 tx=0x7f6084009c00 comp rx=0 tx=0).stop 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 -- 172.21.15.7:0/4043068892 shutdown_connections 2023-06-09T11:53:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094151510 0x7f60940c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 --2- 172.21.15.7:0/4043068892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f609415b380 0x7f6094070b20 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 -- 172.21.15.7:0/4043068892 >> 172.21.15.7:0/4043068892 conn(0x7f609406a4d0 msgr2=0x7f609406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 -- 172.21.15.7:0/4043068892 shutdown_connections 2023-06-09T11:53:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.566+0000 7f609bfff700 1 -- 172.21.15.7:0/4043068892 wait complete. 2023-06-09T11:53:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.567+0000 7f609bfff700 1 Processor -- start 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 -- start start 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094071060 0x7f60941ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f609415b380 0x7f60941f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941efe30 con 0x7f6094151510 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60940c7150 con 0x7f609415b380 2023-06-09T11:53:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.568+0000 7f609bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940c8350 con 0x7f6094071060 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47928/0 (socket says 172.21.15.7:47928) 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 -- 172.21.15.7:0/3284705156 learned_addr learned my addr 172.21.15.7:0/3284705156 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f6099d9b700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094071060 0x7f60941ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609a59c700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f609415b380 0x7f60941f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 -- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094071060 msgr2=0x7f60941ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094071060 0x7f60941ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 -- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f609415b380 msgr2=0x7f60941f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f609415b380 0x7f60941f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 -- 172.21.15.7:0/3284705156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6084009550 con 0x7f6094151510 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.569+0000 7f609959a700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6084009130 tx=0x7f6084004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.570+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084004510 con 0x7f6094151510 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.570+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608401e070 con 0x7f6094151510 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.570+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840038c0 con 0x7f6094151510 2023-06-09T11:53:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.570+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6094157770 con 0x7f6094151510 2023-06-09T11:53:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.570+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6094157490 con 0x7f6094151510 2023-06-09T11:53:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.571+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6094157a50 con 0x7f6094151510 2023-06-09T11:53:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.572+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6084003a20 con 0x7f6094151510 2023-06-09T11:53:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.573+0000 7f608a7fc700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 0x7f607806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.573+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f608408ab00 con 0x7f6094151510 2023-06-09T11:53:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.573+0000 7f6099d9b700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 0x7f607806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.574+0000 7f6099d9b700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 0x7f607806d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6090009be0 tx=0x7f6090009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:14.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.578+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f608408ce60 con 0x7f6094151510 2023-06-09T11:53:14.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.833+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6094074cf0 con 0x7f6094151510 2023-06-09T11:53:14.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.834+0000 7f608a7fc700 1 -- 172.21.15.7:0/3284705156 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 198 v198) v1 ==== 95+0+5464 (secure 0 0 0) 0x7f6084015020 con 0x7f6094151510 2023-06-09T11:53:14.836 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:53:14.837 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":198,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":198,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:53:04.922730+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 msgr2=0x7f607806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 0x7f607806d190 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6090009be0 tx=0x7f6090009210 comp rx=0 tx=0).stop 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 msgr2=0x7f60940affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f6084009130 tx=0x7f6084004750 comp rx=0 tx=0).stop 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 shutdown_connections 2023-06-09T11:53:14.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f609415b380 0x7f60941f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f607806ace0 0x7f607806d190 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6094071060 0x7f60941ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 --2- 172.21.15.7:0/3284705156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6094151510 0x7f60940affb0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 >> 172.21.15.7:0/3284705156 conn(0x7f609406a4d0 msgr2=0x7f609414cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 shutdown_connections 2023-06-09T11:53:14.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:14.836+0000 7f609bfff700 1 -- 172.21.15.7:0/3284705156 wait complete. 2023-06-09T11:53:14.841 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 198 2023-06-09T11:53:14.857 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.f is in active, standby or standby-replay 2023-06-09T11:53:16.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:53:17.073 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.071+0000 7f20d6dff700 1 Processor -- start 2023-06-09T11:53:17.074 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.071+0000 7f20d6dff700 1 -- start start 2023-06-09T11:53:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.072+0000 7f20d6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.072+0000 7f20d6dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d015a500 con 0x7f20d015b4a0 2023-06-09T11:53:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.072+0000 7f20d6dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d015da00 con 0x7f20d00c9f40 2023-06-09T11:53:17.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.072+0000 7f20d6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c5f10 con 0x7f20d0153880 2023-06-09T11:53:17.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d539c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f20d015b4a0 0x7f20d00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:45912/0 (socket says 172.21.15.7:45912) 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d539c700 1 -- 172.21.15.7:0/15294483 learned_addr learned my addr 172.21.15.7:0/15294483 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/15294483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613077557 0 0) 0x7f20bc001110 con 0x7f20d00c9f40 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c00035b0 con 0x7f20d00c9f40 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4271976865 0 0) 0x7f20c8001490 con 0x7f20d015b4a0 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20bc001110 con 0x7f20d015b4a0 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d4b9b700 1 -- 172.21.15.7:0/15294483 >> v1:172.21.15.117:6789/0 conn(0x7f20d00c9f40 legacy=0x7f20d015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d4b9b700 1 -- 172.21.15.7:0/15294483 >> v1:172.21.15.7:6789/0 conn(0x7f20d015b4a0 legacy=0x7f20d00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.073+0000 7f20d4b9b700 1 -- 172.21.15.7:0/15294483 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d01596e0 con 0x7f20d0153880 2023-06-09T11:53:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/15294483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d015d4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ad3266dcdb4c98f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4011830 con 0x7f20d0153880 2023-06-09T11:53:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c4011990 con 0x7f20d0153880 2023-06-09T11:53:17.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20cf7fe700 1 -- 172.21.15.7:0/15294483 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4011d20 con 0x7f20d0153880 2023-06-09T11:53:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20d6dff700 1 -- 172.21.15.7:0/15294483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 msgr2=0x7f20d015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.074+0000 7f20d6dff700 1 --2- 172.21.15.7:0/15294483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d015d4c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c4009b80 comp rx=0 tx=0).stop 2023-06-09T11:53:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.075+0000 7f20d6dff700 1 -- 172.21.15.7:0/15294483 shutdown_connections 2023-06-09T11:53:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.075+0000 7f20d6dff700 1 --2- 172.21.15.7:0/15294483 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d015d4c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.075+0000 7f20d6dff700 1 -- 172.21.15.7:0/15294483 >> 172.21.15.7:0/15294483 conn(0x7f20d00737f0 msgr2=0x7f20d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.075+0000 7f20d6dff700 1 -- 172.21.15.7:0/15294483 shutdown_connections 2023-06-09T11:53:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.075+0000 7f20d6dff700 1 -- 172.21.15.7:0/15294483 wait complete. 2023-06-09T11:53:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.076+0000 7f20d6dff700 1 Processor -- start 2023-06-09T11:53:17.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.076+0000 7f20d6dff700 1 -- start start 2023-06-09T11:53:17.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 0x7f20d01495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d00c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00c7f10 con 0x7f20d015b4a0 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0149af0 con 0x7f20d00c9f40 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.077+0000 7f20d6dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00ce8d0 con 0x7f20d0153880 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:17.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47940/0 (socket says 172.21.15.7:47940) 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 -- 172.21.15.7:0/3284755127 learned_addr learned my addr 172.21.15.7:0/3284755127 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20cffff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d00c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 0x7f20d01495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 -- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 msgr2=0x7f20d00c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d00c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 -- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 msgr2=0x7f20d01495b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 0x7f20d01495b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 -- 172.21.15.7:0/3284755127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40094a0 con 0x7f20d015b4a0 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 0x7f20d01495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.078+0000 7f20d539c700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ca70 tx=0x7f20c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.079+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c800aaa0 con 0x7f20d015b4a0 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.079+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c800ac00 con 0x7f20d015b4a0 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.079+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8012400 con 0x7f20d015b4a0 2023-06-09T11:53:17.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.079+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d0159c60 con 0x7f20d015b4a0 2023-06-09T11:53:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.079+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d0159980 con 0x7f20d015b4a0 2023-06-09T11:53:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.080+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d0159f40 con 0x7f20d015b4a0 2023-06-09T11:53:17.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.081+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20c80113a0 con 0x7f20d015b4a0 2023-06-09T11:53:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.082+0000 7f20cd7fa700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 0x7f20b406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:53:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.082+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f20c8089860 con 0x7f20d015b4a0 2023-06-09T11:53:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.082+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 0x7f20b406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:53:17.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.083+0000 7f20d4b9b700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 0x7f20b406d1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20c40097c0 tx=0x7f20c401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:53:17.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.086+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f20c8054070 con 0x7f20d015b4a0 2023-06-09T11:53:17.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.349+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20d015a500 con 0x7f20d015b4a0 2023-06-09T11:53:17.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.349+0000 7f20cd7fa700 1 -- 172.21.15.7:0/3284755127 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 200 v200) v1 ==== 95+0+6226 (secure 0 0 0) 0x7f20d015a500 con 0x7f20d015b4a0 2023-06-09T11:53:17.352 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:53:17.352 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":200,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":200,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:53:15.940307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:53:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 msgr2=0x7f20b406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 0x7f20b406d1c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f20c40097c0 tx=0x7f20c401c040 comp rx=0 tx=0).stop 2023-06-09T11:53:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 msgr2=0x7f20d015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:53:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ca70 tx=0x7f20c800ce30 comp rx=0 tx=0).stop 2023-06-09T11:53:17.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 shutdown_connections 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f20d015b4a0 0x7f20d015c060 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f20d00c9f40 0x7f20d01495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f20b406ad10 0x7f20b406d1c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 --2- 172.21.15.7:0/3284755127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f20d0153880 0x7f20d00c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 >> 172.21.15.7:0/3284755127 conn(0x7f20d00737f0 msgr2=0x7f20d014b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 shutdown_connections 2023-06-09T11:53:17.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:53:17.352+0000 7f20d6dff700 1 -- 172.21.15.7:0/3284755127 wait complete. 2023-06-09T11:53:17.357 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 200 2023-06-09T11:53:17.373 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f reported in up:standby state 2023-06-09T11:53:17.373 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 93 secs before thrashing 2023-06-09T11:53:22.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:22.849 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:26.983 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 27% (69/253) Updating files: 28% (71/253) Updating files: 29% (74/253) Updating files: 30% (76/253) Updating files: 30% (78/253) Updating files: 31% (79/253) Updating files: 32% (81/253) Updating files: 33% (84/253) Updating files: 34% (87/253) Updating files: 34% (88/253) Updating files: 35% (89/253) Updating files: 35% (91/253) Updating files: 36% (92/253) Updating files: 37% (94/253) Updating files: 38% (97/253) Updating files: 39% (99/253) Updating files: 40% (102/253) Updating files: 41% (104/253) Updating files: 42% (107/253) Updating files: 43% (109/253) Updating files: 44% (112/253) Updating files: 45% (114/253) Updating files: 46% (117/253) Updating files: 47% (119/253) Updating files: 48% (122/253) Updating files: 49% (124/253) Updating files: 49% (125/253) Updating files: 50% (127/253) Updating files: 50% (128/253) Updating files: 51% (130/253) Updating files: 52% (132/253) Updating files: 53% (135/253) Updating files: 54% (137/253) Updating files: 55% (140/253) Updating files: 56% (142/253) Updating files: 57% (145/253) Updating files: 58% (147/253) Updating files: 59% (150/253) Updating files: 60% (152/253) Updating files: 61% (155/253) Updating files: 62% (157/253) Updating files: 63% (160/253) Updating files: 64% (162/253) Updating files: 64% (163/253) Updating files: 65% (165/253) Updating files: 66% (167/253) Updating files: 67% (170/253) Updating files: 67% (172/253) Updating files: 68% (173/253) Updating files: 69% (175/253) Updating files: 70% (178/253) Updating files: 71% (180/253) Updating files: 72% (183/253) Updating files: 73% (185/253) Updating files: 74% (188/253) Updating files: 75% (190/253) Updating files: 75% (191/253) Updating files: 76% (193/253) Updating files: 77% (195/253) Updating files: 77% (197/253) Updating files: 78% (198/253) Updating files: 79% (200/253) Updating files: 80% (203/253) Updating files: 81% (205/253) Updating files: 82% (208/253) Updating files: 83% (210/253) Updating files: 84% (213/253) Updating files: 85% (216/253) Updating files: 86% (218/253) Updating files: 87% (221/253) Updating files: 88% (223/253) Updating files: 89% (226/253) Updating files: 90% (228/253) Updating files: 91% (231/253) Updating files: 92% (233/253) Updating files: 93% (236/253) Updating files: 94% (238/253) Updating files: 95% (241/253) Updating files: 96% (243/253) Updating files: 97% (246/253) Updating files: 98% (248/253) Updating files: 99% (251/253) Updating files: 100% (253/253) Updating files: 100% (253/253), done. 2023-06-09T11:53:28.064 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 793034c62c8 v0.27 2023-06-09T11:53:28.071 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.27 2023-06-09T11:53:28.113 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:53:28.113 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 28 -eq 48 ']' 2023-06-09T11:53:28.113 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.28 2023-06-09T11:53:28.114 INFO:tasks.workunit.client.0.smithi007.stdout:v0.28 2023-06-09T11:53:28.114 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.28 2023-06-09T11:53:28.114 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.28 2023-06-09T11:53:44.140 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 7% (18/227) Updating files: 8% (19/227) Updating files: 9% (21/227) Updating files: 10% (23/227) Updating files: 11% (25/227) Updating files: 12% (28/227) Updating files: 13% (30/227) Updating files: 14% (32/227) Updating files: 14% (33/227) Updating files: 15% (35/227) Updating files: 16% (37/227) Updating files: 17% (39/227) Updating files: 18% (41/227) Updating files: 18% (42/227) Updating files: 19% (44/227) Updating files: 20% (46/227) Updating files: 21% (48/227) Updating files: 22% (50/227) Updating files: 23% (53/227) Updating files: 24% (55/227) Updating files: 25% (57/227) Updating files: 26% (60/227) Updating files: 27% (62/227) Updating files: 28% (64/227) Updating files: 29% (66/227) Updating files: 30% (69/227) Updating files: 31% (71/227) Updating files: 32% (73/227) Updating files: 33% (75/227) Updating files: 34% (78/227) Updating files: 35% (80/227) Updating files: 36% (82/227) Updating files: 37% (84/227) Updating files: 38% (87/227) Updating files: 39% (89/227) Updating files: 40% (91/227) Updating files: 40% (92/227) Updating files: 41% (94/227) Updating files: 42% (96/227) Updating files: 43% (98/227) Updating files: 44% (100/227) Updating files: 45% (103/227) Updating files: 46% (105/227) Updating files: 47% (107/227) Updating files: 48% (109/227) Updating files: 48% (110/227) Updating files: 49% (112/227) Updating files: 50% (114/227) Updating files: 51% (116/227) Updating files: 52% (119/227) Updating files: 53% (121/227) Updating files: 54% (123/227) Updating files: 55% (125/227) Updating files: 56% (128/227) Updating files: 57% (130/227) Updating files: 58% (132/227) Updating files: 59% (134/227) Updating files: 60% (137/227) Updating files: 61% (139/227) Updating files: 62% (141/227) Updating files: 63% (144/227) Updating files: 64% (146/227) Updating files: 65% (148/227) Updating files: 66% (150/227) Updating files: 67% (153/227) Updating files: 68% (155/227) Updating files: 69% (157/227) Updating files: 70% (159/227) Updating files: 71% (162/227) Updating files: 72% (164/227) Updating files: 73% (166/227) Updating files: 74% (168/227) Updating files: 75% (171/227) Updating files: 76% (173/227) Updating files: 77% (175/227) Updating files: 77% (177/227) Updating files: 78% (178/227) Updating files: 79% (180/227) Updating files: 80% (182/227) Updating files: 81% (184/227) Updating files: 82% (187/227) Updating files: 82% (188/227) Updating files: 83% (189/227) Updating files: 84% (191/227) Updating files: 85% (193/227) Updating files: 86% (196/227) Updating files: 87% (198/227) Updating files: 88% (200/227) Updating files: 89% (203/227) Updating files: 90% (205/227) Updating files: 91% (207/227) Updating files: 92% (209/227) Updating files: 93% (212/227) Updating files: 94% (214/227) Updating files: 95% (216/227) Updating files: 96% (218/227) Updating files: 97% (221/227) Updating files: 98% (223/227) Updating files: 99% (225/227) Updating files: 100% (227/227) Updating files: 100% (227/227), done. 2023-06-09T11:53:45.223 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 071881d7e55 v0.28 2023-06-09T11:53:45.230 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.28 2023-06-09T11:53:45.273 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:53:45.273 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 29 -eq 48 ']' 2023-06-09T11:53:45.274 INFO:tasks.workunit.client.0.smithi007.stdout:v0.29 2023-06-09T11:53:45.274 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.29 2023-06-09T11:53:45.274 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.29 2023-06-09T11:53:45.274 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.29 2023-06-09T11:53:52.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:53:52.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:00.912 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 6% (10/159) Updating files: 7% (12/159) Updating files: 8% (13/159) Updating files: 9% (15/159) Updating files: 10% (16/159) Updating files: 11% (18/159) Updating files: 12% (20/159) Updating files: 13% (21/159) Updating files: 14% (23/159) Updating files: 15% (24/159) Updating files: 15% (25/159) Updating files: 16% (26/159) Updating files: 17% (28/159) Updating files: 18% (29/159) Updating files: 19% (31/159) Updating files: 20% (32/159) Updating files: 21% (34/159) Updating files: 22% (35/159) Updating files: 23% (37/159) Updating files: 24% (39/159) Updating files: 25% (40/159) Updating files: 26% (42/159) Updating files: 27% (43/159) Updating files: 28% (45/159) Updating files: 29% (47/159) Updating files: 30% (48/159) Updating files: 31% (50/159) Updating files: 32% (51/159) Updating files: 33% (53/159) Updating files: 34% (55/159) Updating files: 35% (56/159) Updating files: 36% (58/159) Updating files: 37% (59/159) Updating files: 38% (61/159) Updating files: 38% (62/159) Updating files: 39% (63/159) Updating files: 40% (64/159) Updating files: 41% (66/159) Updating files: 42% (67/159) Updating files: 43% (69/159) Updating files: 44% (70/159) Updating files: 45% (72/159) Updating files: 45% (73/159) Updating files: 46% (74/159) Updating files: 47% (75/159) Updating files: 48% (77/159) Updating files: 49% (78/159) Updating files: 50% (80/159) Updating files: 51% (82/159) Updating files: 52% (83/159) Updating files: 53% (85/159) Updating files: 54% (86/159) Updating files: 54% (87/159) Updating files: 55% (88/159) Updating files: 56% (90/159) Updating files: 57% (91/159) Updating files: 58% (93/159) Updating files: 59% (94/159) Updating files: 60% (96/159) Updating files: 61% (97/159) Updating files: 62% (99/159) Updating files: 63% (101/159) Updating files: 64% (102/159) Updating files: 65% (104/159) Updating files: 66% (105/159) Updating files: 66% (106/159) Updating files: 67% (107/159) Updating files: 67% (108/159) Updating files: 68% (109/159) Updating files: 69% (110/159) Updating files: 70% (112/159) Updating files: 71% (113/159) Updating files: 71% (114/159) Updating files: 72% (115/159) Updating files: 73% (117/159) Updating files: 74% (118/159) Updating files: 75% (120/159) Updating files: 76% (121/159) Updating files: 77% (123/159) Updating files: 78% (125/159) Updating files: 79% (126/159) Updating files: 80% (128/159) Updating files: 81% (129/159) Updating files: 82% (131/159) Updating files: 83% (132/159) Updating files: 84% (134/159) Updating files: 85% (136/159) Updating files: 86% (137/159) Updating files: 87% (139/159) Updating files: 88% (140/159) Updating files: 89% (142/159) Updating files: 90% (144/159) Updating files: 91% (145/159) Updating files: 92% (147/159) Updating files: 93% (148/159) Updating files: 94% (150/159) Updating files: 95% (152/159) Updating files: 96% (153/159) Updating files: 97% (155/159) Updating files: 98% (156/159) Updating files: 98% (157/159) Updating files: 99% (158/159) Updating files: 100% (159/159) Updating files: 100% (159/159), done. 2023-06-09T11:54:02.301 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 8e69c39f699 v0.29 2023-06-09T11:54:02.308 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.29 2023-06-09T11:54:02.351 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:54:02.351 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 30 -eq 48 ']' 2023-06-09T11:54:02.351 INFO:tasks.workunit.client.0.smithi007.stdout:v0.30 2023-06-09T11:54:02.352 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.30 2023-06-09T11:54:02.352 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.30 2023-06-09T11:54:02.352 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.30 2023-06-09T11:54:23.110 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:23.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:28.023 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 38% (109/285) Updating files: 38% (111/285) Updating files: 39% (112/285) Updating files: 40% (114/285) Updating files: 41% (117/285) Updating files: 42% (120/285) Updating files: 42% (121/285) Updating files: 43% (123/285) Updating files: 44% (126/285) Updating files: 45% (129/285) Updating files: 46% (132/285) Updating files: 47% (134/285) Updating files: 48% (137/285) Updating files: 49% (140/285) Updating files: 50% (143/285) Updating files: 51% (146/285) Updating files: 52% (149/285) Updating files: 53% (152/285) Updating files: 54% (154/285) Updating files: 55% (157/285) Updating files: 56% (160/285) Updating files: 57% (163/285) Updating files: 57% (164/285) Updating files: 58% (166/285) Updating files: 59% (169/285) Updating files: 59% (170/285) Updating files: 60% (171/285) Updating files: 60% (173/285) Updating files: 61% (174/285) Updating files: 62% (177/285) Updating files: 63% (180/285) Updating files: 64% (183/285) Updating files: 65% (186/285) Updating files: 66% (189/285) Updating files: 66% (190/285) Updating files: 67% (191/285) Updating files: 68% (194/285) Updating files: 69% (197/285) Updating files: 70% (200/285) Updating files: 71% (203/285) Updating files: 72% (206/285) Updating files: 73% (209/285) Updating files: 74% (211/285) Updating files: 75% (214/285) Updating files: 76% (217/285) Updating files: 76% (219/285) Updating files: 77% (220/285) Updating files: 78% (223/285) Updating files: 79% (226/285) Updating files: 79% (227/285) Updating files: 80% (228/285) Updating files: 81% (231/285) Updating files: 82% (234/285) Updating files: 83% (237/285) Updating files: 84% (240/285) Updating files: 85% (243/285) Updating files: 86% (246/285) Updating files: 87% (248/285) Updating files: 88% (251/285) Updating files: 89% (254/285) Updating files: 90% (257/285) Updating files: 91% (260/285) Updating files: 92% (263/285) Updating files: 93% (266/285) Updating files: 94% (268/285) Updating files: 95% (271/285) Updating files: 96% (274/285) Updating files: 97% (277/285) Updating files: 97% (279/285) Updating files: 98% (280/285) Updating files: 99% (283/285) Updating files: 100% (285/285) Updating files: 100% (285/285), done. 2023-06-09T11:54:28.253 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 64b1b2c70f0 v0.30 2023-06-09T11:54:28.259 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.30 2023-06-09T11:54:28.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:54:28.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 31 -eq 48 ']' 2023-06-09T11:54:28.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.31 2023-06-09T11:54:28.333 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.31 2023-06-09T11:54:28.333 INFO:tasks.workunit.client.0.smithi007.stdout:v0.31 2023-06-09T11:54:28.334 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.31 2023-06-09T11:54:47.570 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 5% (21/398) Updating files: 5% (22/398) Updating files: 6% (24/398) Updating files: 7% (28/398) Updating files: 8% (32/398) Updating files: 9% (36/398) Updating files: 10% (40/398) Updating files: 11% (44/398) Updating files: 12% (48/398) Updating files: 13% (52/398) Updating files: 13% (53/398) Updating files: 14% (56/398) Updating files: 15% (60/398) Updating files: 16% (64/398) Updating files: 17% (68/398) Updating files: 18% (72/398) Updating files: 19% (76/398) Updating files: 20% (80/398) Updating files: 21% (84/398) Updating files: 22% (88/398) Updating files: 23% (92/398) Updating files: 24% (96/398) Updating files: 25% (100/398) Updating files: 26% (104/398) Updating files: 27% (108/398) Updating files: 28% (112/398) Updating files: 29% (116/398) Updating files: 30% (120/398) Updating files: 31% (124/398) Updating files: 32% (128/398) Updating files: 33% (132/398) Updating files: 34% (136/398) Updating files: 34% (137/398) Updating files: 35% (140/398) Updating files: 35% (141/398) Updating files: 36% (144/398) Updating files: 37% (148/398) Updating files: 38% (152/398) Updating files: 38% (154/398) Updating files: 39% (156/398) Updating files: 40% (160/398) Updating files: 40% (161/398) Updating files: 41% (164/398) Updating files: 42% (168/398) Updating files: 43% (172/398) Updating files: 44% (176/398) Updating files: 45% (180/398) Updating files: 46% (184/398) Updating files: 47% (188/398) Updating files: 48% (192/398) Updating files: 49% (196/398) Updating files: 50% (199/398) Updating files: 51% (203/398) Updating files: 52% (207/398) Updating files: 53% (211/398) Updating files: 54% (215/398) Updating files: 55% (219/398) Updating files: 56% (223/398) Updating files: 57% (227/398) Updating files: 57% (228/398) Updating files: 58% (231/398) Updating files: 59% (235/398) Updating files: 60% (239/398) Updating files: 61% (243/398) Updating files: 62% (247/398) Updating files: 63% (251/398) Updating files: 64% (255/398) Updating files: 65% (259/398) Updating files: 66% (263/398) Updating files: 67% (267/398) Updating files: 68% (271/398) Updating files: 69% (275/398) Updating files: 70% (279/398) Updating files: 71% (283/398) Updating files: 72% (287/398) Updating files: 73% (291/398) Updating files: 74% (295/398) Updating files: 74% (298/398) Updating files: 75% (299/398) Updating files: 76% (303/398) Updating files: 76% (305/398) Updating files: 77% (307/398) Updating files: 78% (311/398) Updating files: 78% (312/398) Updating files: 79% (315/398) Updating files: 80% (319/398) Updating files: 81% (323/398) Updating files: 81% (326/398) Updating files: 82% (327/398) Updating files: 83% (331/398) Updating files: 84% (335/398) Updating files: 85% (339/398) Updating files: 86% (343/398) Updating files: 87% (347/398) Updating files: 88% (351/398) Updating files: 89% (355/398) Updating files: 90% (359/398) Updating files: 91% (363/398) Updating files: 92% (367/398) Updating files: 93% (371/398) Updating files: 94% (375/398) Updating files: 95% (379/398) Updating files: 96% (383/398) Updating files: 97% (387/398) Updating files: 98% (391/398) Updating files: 99% (395/398) Updating files: 100% (398/398) Updating files: 100% (398/398), done. 2023-06-09T11:54:47.713 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 9019c6ce640 v0.31 2023-06-09T11:54:47.718 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.31 2023-06-09T11:54:47.762 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:54:47.762 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 32 -eq 48 ']' 2023-06-09T11:54:47.762 INFO:tasks.workunit.client.0.smithi007.stdout:v0.32 2023-06-09T11:54:47.762 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.32 2023-06-09T11:54:47.763 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.32 2023-06-09T11:54:47.763 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.32 2023-06-09T11:54:50.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:54:50.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.572+0000 7f5f10fff700 1 Processor -- start 2023-06-09T11:54:50.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.572+0000 7f5f10fff700 1 -- start start 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f10fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f0c0bed60 0x7f5f0c159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f10fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 0x7f5f0c071510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f10fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c070d50 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f10fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c159ab0 con 0x7f5f0c0bed60 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f10fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c159d90 con 0x7f5f0c070440 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f03fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f0c0bed60 0x7f5f0c159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f03fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5f0c0bed60 0x7f5f0c159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50476/0 (socket says 172.21.15.7:50476) 2023-06-09T11:54:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f03fff700 1 -- 172.21.15.7:0/2871635776 learned_addr learned my addr 172.21.15.7:0/2871635776 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:50.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.573+0000 7f5f0ad9d700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 0x7f5f0c071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f03fff700 1 -- 172.21.15.7:0/2871635776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 msgr2=0x7f5f0c071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:54:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f03fff700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 0x7f5f0c071510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f03fff700 1 -- 172.21.15.7:0/2871635776 >> v1:172.21.15.117:6789/0 conn(0x7f5f0c0c61c0 legacy=0x7f5f0c0be650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f03fff700 1 -- 172.21.15.7:0/2871635776 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c15a070 con 0x7f5f0c0bed60 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f0ad9d700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 0x7f5f0c071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f03fff700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0bed60 0x7f5f0c159470 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40098f0 tx=0x7f5ef4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8884c0aa64f9485d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f09d9b700 1 -- 172.21.15.7:0/2871635776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400b8a0 con 0x7f5f0c0bed60 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f09d9b700 1 -- 172.21.15.7:0/2871635776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400ba00 con 0x7f5f0c0bed60 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.574+0000 7f5f09d9b700 1 -- 172.21.15.7:0/2871635776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400bd90 con 0x7f5f0c0bed60 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 -- 172.21.15.7:0/2871635776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0bed60 msgr2=0x7f5f0c159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0bed60 0x7f5f0c159470 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f5ef40098f0 tx=0x7f5ef4009c00 comp rx=0 tx=0).stop 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 -- 172.21.15.7:0/2871635776 shutdown_connections 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c070440 0x7f5f0c071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 --2- 172.21.15.7:0/2871635776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0bed60 0x7f5f0c159470 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 -- 172.21.15.7:0/2871635776 >> 172.21.15.7:0/2871635776 conn(0x7f5f0c06a4d0 msgr2=0x7f5f0c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 -- 172.21.15.7:0/2871635776 shutdown_connections 2023-06-09T11:54:50.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.575+0000 7f5f10fff700 1 -- 172.21.15.7:0/2871635776 wait complete. 2023-06-09T11:54:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.577+0000 7f5f10fff700 1 Processor -- start 2023-06-09T11:54:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.577+0000 7f5f10fff700 1 -- start start 2023-06-09T11:54:50.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f0c070440 0x7f5f0c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c0bed60 0x7f5f0c1efc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1ef7c0 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1ef220 con 0x7f5f0c070440 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f10fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c1ef4e0 con 0x7f5f0c0bed60 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f0ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f0ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50478/0 (socket says 172.21.15.7:50478) 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f0ad9d700 1 -- 172.21.15.7:0/1663718877 learned_addr learned my addr 172.21.15.7:0/1663718877 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f0a59c700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f0c070440 0x7f5f0c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.578+0000 7f5f03fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c0bed60 0x7f5f0c1efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 -- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c0bed60 msgr2=0x7f5f0c1efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c0bed60 0x7f5f0c1efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 -- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f0c070440 msgr2=0x7f5f0c1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f0c070440 0x7f5f0c1564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 -- 172.21.15.7:0/1663718877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef4009550 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f0ad9d700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5f04007ef0 tx=0x7f5f0400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0400dd80 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0400f040 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04003c60 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c15a350 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.579+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c070ff0 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.580+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c0c8040 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.581+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f04004030 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.582+0000 7f5f02ffd700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 0x7f5eec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.582+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f5f04089d60 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.582+0000 7f5f0a59c700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 0x7f5eec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:50.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.583+0000 7f5f0a59c700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 0x7f5eec06d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5efc007fd0 tx=0x7f5efc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:50.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.587+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5f04054580 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.843+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f0c074cf0 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.843+0000 7f5f02ffd700 1 -- 172.21.15.7:0/1663718877 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+6222 (secure 0 0 0) 0x7f5f04057ba0 con 0x7f5f0c0c61c0 2023-06-09T11:54:50.847 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:54:50.847 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:45.727223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:54:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 msgr2=0x7f5eec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 0x7f5eec06d190 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5efc007fd0 tx=0x7f5efc009040 comp rx=0 tx=0).stop 2023-06-09T11:54:50.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 msgr2=0x7f5f0c0c4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f5f04007ef0 tx=0x7f5f0400d3b0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 shutdown_connections 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0c0c61c0 0x7f5f0c0c4850 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f0c070440 0x7f5f0c1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5eec06ace0 0x7f5eec06d190 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.845+0000 7f5f10fff700 1 --2- 172.21.15.7:0/1663718877 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5f0c0bed60 0x7f5f0c1efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:50.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.846+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 >> 172.21.15.7:0/1663718877 conn(0x7f5f0c06a4d0 msgr2=0x7f5f0c0c6590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.846+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 shutdown_connections 2023-06-09T11:54:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:50.846+0000 7f5f10fff700 1 -- 172.21.15.7:0/1663718877 wait complete. 2023-06-09T11:54:50.851 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 202 2023-06-09T11:54:50.867 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.d (rank=1) 2023-06-09T11:54:50.867 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T11:54:50.868 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:54:50.891 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T11:54:50.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:54:51.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.089+0000 7fa6bc3ff700 1 Processor -- start 2023-06-09T11:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.089+0000 7fa6bc3ff700 1 -- start start 2023-06-09T11:54:51.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6bc3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa6b4153880 0x7fa6b415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6bc3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b415a500 con 0x7fa6b40c9f40 2023-06-09T11:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6bc3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b415da00 con 0x7fa6b415b4a0 2023-06-09T11:54:51.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6bc3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b40c5f10 con 0x7fa6b4153880 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6b999a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa6b40c9f40 0x7fa6b415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35588/0 (socket says 172.21.15.7:35588) 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.090+0000 7fa6b999a700 1 -- 172.21.15.7:0/1472250875 learned_addr learned my addr 172.21.15.7:0/1472250875 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6ba19b700 1 --2- 172.21.15.7:0/1472250875 >> v2:172.21.15.117:3300/0 conn(0x7fa6b4153880 0x7fa6b415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6b9199700 1 -- 172.21.15.7:0/1472250875 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089335711 0 0) 0x7fa6a4001110 con 0x7fa6b40c9f40 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6b9199700 1 -- 172.21.15.7:0/1472250875 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6a00035b0 con 0x7fa6b40c9f40 2023-06-09T11:54:51.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6ba19b700 1 -- 172.21.15.7:0/1472250875 >> v1:172.21.15.117:6789/0 conn(0x7fa6b415b4a0 legacy=0x7fa6b40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6ba19b700 1 -- 172.21.15.7:0/1472250875 >> v1:172.21.15.7:6789/0 conn(0x7fa6b40c9f40 legacy=0x7fa6b415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.091+0000 7fa6ba19b700 1 -- 172.21.15.7:0/1472250875 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b41596e0 con 0x7fa6b4153880 2023-06-09T11:54:51.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6ba19b700 1 --2- 172.21.15.7:0/1472250875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b4153880 0x7fa6b415d4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009840 tx=0x7fa6b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91e153ed79172c55 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6b9199700 1 -- 172.21.15.7:0/1472250875 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0011830 con 0x7fa6b4153880 2023-06-09T11:54:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6b9199700 1 -- 172.21.15.7:0/1472250875 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0011990 con 0x7fa6b4153880 2023-06-09T11:54:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6b9199700 1 -- 172.21.15.7:0/1472250875 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0011d20 con 0x7fa6b4153880 2023-06-09T11:54:51.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/1472250875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b4153880 msgr2=0x7fa6b415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/1472250875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b4153880 0x7fa6b415d4c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009840 tx=0x7fa6b0009b50 comp rx=0 tx=0).stop 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/1472250875 shutdown_connections 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/1472250875 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b4153880 0x7fa6b415d4c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.092+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/1472250875 >> 172.21.15.7:0/1472250875 conn(0x7fa6b40737f0 msgr2=0x7fa6b40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.093+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/1472250875 shutdown_connections 2023-06-09T11:54:51.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.093+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/1472250875 wait complete. 2023-06-09T11:54:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.094+0000 7fa6bc3ff700 1 Processor -- start 2023-06-09T11:54:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.094+0000 7fa6bc3ff700 1 -- start start 2023-06-09T11:54:51.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b40c9f40 0x7fa6b41effa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6b4153880 0x7fa6b414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b41f04e0 con 0x7fa6b415b4a0 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40c96f0 con 0x7fa6b40c9f40 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.095+0000 7fa6bc3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b40b3840 con 0x7fa6b4153880 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:51.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba19b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b40c9f40 0x7fa6b41effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50484/0 (socket says 172.21.15.7:50484) 2023-06-09T11:54:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 -- 172.21.15.7:0/430015076 learned_addr learned my addr 172.21.15.7:0/430015076 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:51.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6b999a700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6b4153880 0x7fa6b414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 -- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6b4153880 msgr2=0x7fa6b414bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6b4153880 0x7fa6b414bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 -- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b40c9f40 msgr2=0x7fa6b41effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b40c9f40 0x7fa6b41effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.096+0000 7fa6ba99c700 1 -- 172.21.15.7:0/430015076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00094a0 con 0x7fa6b415b4a0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6ba99c700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00cb50 tx=0x7fa6ac00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac003c00 con 0x7fa6b415b4a0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ac003d60 con 0x7fa6b415b4a0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b4159c60 con 0x7fa6b415b4a0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00aa20 con 0x7fa6b415b4a0 2023-06-09T11:54:51.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.097+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b4159980 con 0x7fa6b415b4a0 2023-06-09T11:54:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.098+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b4159f40 con 0x7fa6b415b4a0 2023-06-09T11:54:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.099+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa6ac00ab80 con 0x7fa6b415b4a0 2023-06-09T11:54:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.100+0000 7fa6aaffd700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 0x7fa69806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.100+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fa6ac089b20 con 0x7fa6b415b4a0 2023-06-09T11:54:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.101+0000 7fa6ba19b700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 0x7fa69806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:51.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.101+0000 7fa6ba19b700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 0x7fa69806d1c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00090b0 tx=0x7fa6b0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:51.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.105+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa6ac054330 con 0x7fa6b415b4a0 2023-06-09T11:54:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.353+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6b415da00 con 0x7fa6b415b4a0 2023-06-09T11:54:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.354+0000 7fa6aaffd700 1 -- 172.21.15.7:0/430015076 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 202 v202) v1 ==== 95+0+6222 (secure 0 0 0) 0x7fa6b415da00 con 0x7fa6b415b4a0 2023-06-09T11:54:51.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:54:51.358 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":202,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":202,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:45.727223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:54:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 msgr2=0x7fa69806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 0x7fa69806d1c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00090b0 tx=0x7fa6b0008030 comp rx=0 tx=0).stop 2023-06-09T11:54:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 msgr2=0x7fa6b415bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00cb50 tx=0x7fa6ac00ce60 comp rx=0 tx=0).stop 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 shutdown_connections 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6b415b4a0 0x7fa6b415bb60 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa6b40c9f40 0x7fa6b41effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa69806ad10 0x7fa69806d1c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.356+0000 7fa6bc3ff700 1 --2- 172.21.15.7:0/430015076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa6b4153880 0x7fa6b414bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.357+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 >> 172.21.15.7:0/430015076 conn(0x7fa6b40737f0 msgr2=0x7fa6b414ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.357+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 shutdown_connections 2023-06-09T11:54:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:51.357+0000 7fa6bc3ff700 1 -- 172.21.15.7:0/430015076 wait complete. 2023-06-09T11:54:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 202 2023-06-09T11:54:51.378 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:54:53.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:53.233 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:54:53.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:54:53.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.577+0000 7fb0d71ff700 1 Processor -- start 2023-06-09T11:54:53.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.577+0000 7fb0d71ff700 1 -- start start 2023-06-09T11:54:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d71ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0d015b4a0 0x7fb0d015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d71ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d015a500 con 0x7fb0d0153880 2023-06-09T11:54:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d71ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d015da00 con 0x7fb0d00c9f40 2023-06-09T11:54:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d71ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00c5f10 con 0x7fb0d015b4a0 2023-06-09T11:54:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d4f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0d0153880 0x7fb0d015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46384/0 (socket says 172.21.15.7:46384) 2023-06-09T11:54:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0d4f9b700 1 -- 172.21.15.7:0/829458887 learned_addr learned my addr 172.21.15.7:0/829458887 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:54:53.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cffff700 1 --2- 172.21.15.7:0/829458887 >> v2:172.21.15.7:3300/0 conn(0x7fb0d015b4a0 0x7fb0d015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cf7fe700 1 -- 172.21.15.7:0/829458887 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954640247 0 0) 0x7fb0c4001110 con 0x7fb0d0153880 2023-06-09T11:54:53.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cf7fe700 1 -- 172.21.15.7:0/829458887 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0c00035b0 con 0x7fb0d0153880 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cffff700 1 -- 172.21.15.7:0/829458887 >> v1:172.21.15.117:6789/0 conn(0x7fb0d00c9f40 legacy=0x7fb0d00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cffff700 1 -- 172.21.15.7:0/829458887 >> v1:172.21.15.7:6789/0 conn(0x7fb0d0153880 legacy=0x7fb0d015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.579+0000 7fb0cffff700 1 -- 172.21.15.7:0/829458887 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d01596e0 con 0x7fb0d015b4a0 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0cffff700 1 --2- 172.21.15.7:0/829458887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d015b4a0 0x7fb0d015d4c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009840 tx=0x7fb0bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63570491be7f74b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0cf7fe700 1 -- 172.21.15.7:0/829458887 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc011830 con 0x7fb0d015b4a0 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0cf7fe700 1 -- 172.21.15.7:0/829458887 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc011990 con 0x7fb0d015b4a0 2023-06-09T11:54:53.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0cf7fe700 1 -- 172.21.15.7:0/829458887 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc011d20 con 0x7fb0d015b4a0 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0d71ff700 1 -- 172.21.15.7:0/829458887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d015b4a0 msgr2=0x7fb0d015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.580+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/829458887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d015b4a0 0x7fb0d015d4c0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009840 tx=0x7fb0bc009b50 comp rx=0 tx=0).stop 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.581+0000 7fb0d71ff700 1 -- 172.21.15.7:0/829458887 shutdown_connections 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.581+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/829458887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d015b4a0 0x7fb0d015d4c0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.581+0000 7fb0d71ff700 1 -- 172.21.15.7:0/829458887 >> 172.21.15.7:0/829458887 conn(0x7fb0d00737f0 msgr2=0x7fb0d00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.581+0000 7fb0d71ff700 1 -- 172.21.15.7:0/829458887 shutdown_connections 2023-06-09T11:54:53.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.581+0000 7fb0d71ff700 1 -- 172.21.15.7:0/829458887 wait complete. 2023-06-09T11:54:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.583+0000 7fb0d71ff700 1 Processor -- start 2023-06-09T11:54:53.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.583+0000 7fb0d71ff700 1 -- start start 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.584+0000 7fb0d71ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 0x7fb0d0149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d71ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d71ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 0x7fb0d01f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d71ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01f6df0 con 0x7fb0d0153880 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d71ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0150ce0 con 0x7fb0d015b4a0 2023-06-09T11:54:53.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d71ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00cf2d0 con 0x7fb0d00c9f40 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39788/0 (socket says 172.21.15.7:39788) 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 -- 172.21.15.7:0/2202323351 learned_addr learned my addr 172.21.15.7:0/2202323351 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d4f9b700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 0x7fb0d0149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d579c700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 0x7fb0d01f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 -- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 msgr2=0x7fb0d0149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 0x7fb0d0149540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 -- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 msgr2=0x7fb0d01f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 0x7fb0d01f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0cffff700 1 -- 172.21.15.7:0/2202323351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc0094a0 con 0x7fb0d0153880 2023-06-09T11:54:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d4f9b700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 0x7fb0d0149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:54:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.585+0000 7fb0d579c700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 0x7fb0d01f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:54:53.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0cffff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000c00 tx=0x7fb0bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0038a0 con 0x7fb0d0153880 2023-06-09T11:54:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc01e070 con 0x7fb0d0153880 2023-06-09T11:54:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc003c00 con 0x7fb0d0153880 2023-06-09T11:54:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d0159c60 con 0x7fb0d0153880 2023-06-09T11:54:53.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.586+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d0159980 con 0x7fb0d0153880 2023-06-09T11:54:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.587+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0159f40 con 0x7fb0d0153880 2023-06-09T11:54:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.588+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb0bc02d450 con 0x7fb0d0153880 2023-06-09T11:54:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.589+0000 7fb0cd7fa700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 0x7fb0b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:53.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.589+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7fb0bc08ac50 con 0x7fb0d0153880 2023-06-09T11:54:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.589+0000 7fb0d4f9b700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 0x7fb0b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:53.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.590+0000 7fb0d4f9b700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 0x7fb0b406d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4001270 tx=0x7fb0c4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.593+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb0bc0554f0 con 0x7fb0d0153880 2023-06-09T11:54:53.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.849+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0d00ccf00 con 0x7fb0d0153880 2023-06-09T11:54:53.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.849+0000 7fb0cd7fa700 1 -- 172.21.15.7:0/2202323351 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+6221 (secure 0 0 0) 0x7fb0bc015070 con 0x7fb0d0153880 2023-06-09T11:54:53.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:54:53.853 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:51.911234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:54:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 msgr2=0x7fb0b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 0x7fb0b406d190 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4001270 tx=0x7fb0c4009040 comp rx=0 tx=0).stop 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 msgr2=0x7fb0d01f4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000c00 tx=0x7fb0bc004750 comp rx=0 tx=0).stop 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 shutdown_connections 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0d015b4a0 0x7fb0d01f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb0b406ace0 0x7fb0b406d190 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0d00c9f40 0x7fb0d0149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 --2- 172.21.15.7:0/2202323351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0d0153880 0x7fb0d01f4590 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 >> 172.21.15.7:0/2202323351 conn(0x7fb0d00737f0 msgr2=0x7fb0d014b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 shutdown_connections 2023-06-09T11:54:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:53.852+0000 7fb0d71ff700 1 -- 172.21.15.7:0/2202323351 wait complete. 2023-06-09T11:54:53.857 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 203 2023-06-09T11:54:53.873 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:54:55.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:54:56.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.082+0000 7f75f93ff700 1 Processor -- start 2023-06-09T11:54:56.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.082+0000 7f75f93ff700 1 -- start start 2023-06-09T11:54:56.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f4153880 0x7f75f415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75f40c9f40 0x7f75f415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f75f415b2d0 0x7f75f40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40c5f10 con 0x7f75f40c9f40 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f415a500 con 0x7f75f415b2d0 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.083+0000 7f75f93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f415da00 con 0x7f75f4153880 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75f40c9f40 0x7f75f415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f75f40c9f40 0x7f75f415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39804/0 (socket says 172.21.15.7:39804) 2023-06-09T11:54:56.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 -- 172.21.15.7:0/816488301 learned_addr learned my addr 172.21.15.7:0/816488301 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f37fe700 1 --2- 172.21.15.7:0/816488301 >> v2:172.21.15.117:3300/0 conn(0x7f75f415b2d0 0x7f75f40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f2ffd700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f4153880 0x7f75f415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 -- 172.21.15.7:0/816488301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f4153880 msgr2=0x7f75f415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f4153880 0x7f75f415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 -- 172.21.15.7:0/816488301 >> v2:172.21.15.117:3300/0 conn(0x7f75f415b2d0 msgr2=0x7f75f40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 --2- 172.21.15.7:0/816488301 >> v2:172.21.15.117:3300/0 conn(0x7f75f415b2d0 0x7f75f40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.084+0000 7f75f27fc700 1 -- 172.21.15.7:0/816488301 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f415b750 con 0x7f75f40c9f40 2023-06-09T11:54:56.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f27fc700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f40c9f40 0x7f75f415ad90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f75f4071db0 tx=0x7f75e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491df343f6256f41 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f1ffb700 1 -- 172.21.15.7:0/816488301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8011830 con 0x7f75f40c9f40 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f1ffb700 1 -- 172.21.15.7:0/816488301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8011990 con 0x7f75f40c9f40 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f1ffb700 1 -- 172.21.15.7:0/816488301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8011d20 con 0x7f75f40c9f40 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 -- 172.21.15.7:0/816488301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f40c9f40 msgr2=0x7f75f415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f40c9f40 0x7f75f415ad90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f75f4071db0 tx=0x7f75e8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 -- 172.21.15.7:0/816488301 shutdown_connections 2023-06-09T11:54:56.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 --2- 172.21.15.7:0/816488301 >> v2:172.21.15.117:3300/0 conn(0x7f75f415b2d0 0x7f75f40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f4153880 0x7f75f415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 --2- 172.21.15.7:0/816488301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f40c9f40 0x7f75f415ad90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 -- 172.21.15.7:0/816488301 >> 172.21.15.7:0/816488301 conn(0x7f75f40737f0 msgr2=0x7f75f40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.085+0000 7f75f93ff700 1 -- 172.21.15.7:0/816488301 shutdown_connections 2023-06-09T11:54:56.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.086+0000 7f75f93ff700 1 -- 172.21.15.7:0/816488301 wait complete. 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.087+0000 7f75f93ff700 1 Processor -- start 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.087+0000 7f75f93ff700 1 -- start start 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f40c9f40 0x7f75f40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f415b2d0 0x7f75f414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40b44c0 con 0x7f75f4153880 2023-06-09T11:54:56.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f41498d0 con 0x7f75f40c9f40 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f93ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40b2e00 con 0x7f75f415b2d0 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.088+0000 7f75f27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39806/0 (socket says 172.21.15.7:39806) 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 -- 172.21.15.7:0/1520892057 learned_addr learned my addr 172.21.15.7:0/1520892057 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f37fe700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f415b2d0 0x7f75f414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f2ffd700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f40c9f40 0x7f75f40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 -- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f415b2d0 msgr2=0x7f75f414b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:56.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f415b2d0 0x7f75f414b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 -- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f40c9f40 msgr2=0x7f75f40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f40c9f40 0x7f75f40b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 -- 172.21.15.7:0/1520892057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e8009550 con 0x7f75f4153880 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75f27fc700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8004240 con 0x7f75f4153880 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.089+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e801f070 con 0x7f75f4153880 2023-06-09T11:54:56.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.090+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f4159980 con 0x7f75f4153880 2023-06-09T11:54:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.090+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e80036a0 con 0x7f75f4153880 2023-06-09T11:54:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.090+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41596e0 con 0x7f75f4153880 2023-06-09T11:54:56.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.091+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4159c60 con 0x7f75f4153880 2023-06-09T11:54:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.091+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75e8021030 con 0x7f75f4153880 2023-06-09T11:54:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.093+0000 7f75db7fe700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 0x7f75e006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.093+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f75e808aef0 con 0x7f75f4153880 2023-06-09T11:54:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.093+0000 7f75f2ffd700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 0x7f75e006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:56.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.094+0000 7f75f2ffd700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 0x7f75e006d1c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009b60 tx=0x7f75e4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:56.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.098+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f75e80558b0 con 0x7f75f4153880 2023-06-09T11:54:56.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.346+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f40b48e0 con 0x7f75f4153880 2023-06-09T11:54:56.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.347+0000 7f75db7fe700 1 -- 172.21.15.7:0/1520892057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f75e8015090 con 0x7f75f4153880 2023-06-09T11:54:56.350 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:54:56.350 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:51.911234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:54:56.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 msgr2=0x7f75e006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 0x7f75e006d1c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009b60 tx=0x7f75e4009210 comp rx=0 tx=0).stop 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 msgr2=0x7f75f4149390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f75e8000c00 tx=0x7f75e8004750 comp rx=0 tx=0).stop 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 shutdown_connections 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f75e006ad10 0x7f75e006d1c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f75f415b2d0 0x7f75f414b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f75f40c9f40 0x7f75f40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 --2- 172.21.15.7:0/1520892057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f4153880 0x7f75f4149390 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:56.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 >> 172.21.15.7:0/1520892057 conn(0x7f75f40737f0 msgr2=0x7f75f414f190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:56.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 shutdown_connections 2023-06-09T11:54:56.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:56.349+0000 7f75f93ff700 1 -- 172.21.15.7:0/1520892057 wait complete. 2023-06-09T11:54:56.354 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 203 2023-06-09T11:54:56.370 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:54:58.371 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:54:58.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.579+0000 7f1a7a1ff700 1 Processor -- start 2023-06-09T11:54:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.579+0000 7f1a7a1ff700 1 -- start start 2023-06-09T11:54:58.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a7a1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a7415b4a0 0x7f1a7415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a7a1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7415a500 con 0x7f1a740c9f40 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a7a1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a7415da00 con 0x7f1a74153880 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a7a1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a740c5f10 con 0x7f1a7415b4a0 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a72ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1a7415b4a0 0x7f1a7415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a73fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1a740c9f40 0x7f1a740c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46396/0 (socket says 172.21.15.7:46396) 2023-06-09T11:54:58.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.580+0000 7f1a73fff700 1 -- 172.21.15.7:0/2922038878 learned_addr learned my addr 172.21.15.7:0/2922038878 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:54:58.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a72ffd700 1 -- 172.21.15.7:0/2922038878 >> v1:172.21.15.117:6789/0 conn(0x7f1a74153880 legacy=0x7f1a7415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a72ffd700 1 -- 172.21.15.7:0/2922038878 >> v1:172.21.15.7:6789/0 conn(0x7f1a740c9f40 legacy=0x7f1a740c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a727fc700 1 -- 172.21.15.7:0/2922038878 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694481812 0 0) 0x7f1a6c001490 con 0x7f1a740c9f40 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a72ffd700 1 -- 172.21.15.7:0/2922038878 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a741596e0 con 0x7f1a7415b4a0 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a72ffd700 1 --2- 172.21.15.7:0/2922038878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7415b4a0 0x7f1a7415d4c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009840 tx=0x7f1a68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c87ea882c69a58 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a727fc700 1 -- 172.21.15.7:0/2922038878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011830 con 0x7f1a7415b4a0 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a727fc700 1 -- 172.21.15.7:0/2922038878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a68011990 con 0x7f1a7415b4a0 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.581+0000 7f1a727fc700 1 -- 172.21.15.7:0/2922038878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a68011d20 con 0x7f1a7415b4a0 2023-06-09T11:54:58.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/2922038878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7415b4a0 msgr2=0x7f1a7415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/2922038878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7415b4a0 0x7f1a7415d4c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1a68009840 tx=0x7f1a68009b50 comp rx=0 tx=0).stop 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/2922038878 shutdown_connections 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/2922038878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a7415b4a0 0x7f1a7415d4c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/2922038878 >> 172.21.15.7:0/2922038878 conn(0x7f1a740737f0 msgr2=0x7f1a740720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/2922038878 shutdown_connections 2023-06-09T11:54:58.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.582+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/2922038878 wait complete. 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.584+0000 7f1a7a1ff700 1 Processor -- start 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.584+0000 7f1a7a1ff700 1 -- start start 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a74153880 0x7f1a740ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 0x7f1a74151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:58.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74157be0 con 0x7f1a740c9f40 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a74150c40 con 0x7f1a74153880 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a7a1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a740b4c30 con 0x7f1a7415b4a0 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39816/0 (socket says 172.21.15.7:39816) 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 -- 172.21.15.7:0/1740190882 learned_addr learned my addr 172.21.15.7:0/1740190882 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a72ffd700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a74153880 0x7f1a740ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a73fff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 0x7f1a74151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 -- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 msgr2=0x7f1a74151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 0x7f1a74151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 -- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a74153880 msgr2=0x7f1a740ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a74153880 0x7f1a740ce850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a737fe700 1 -- 172.21.15.7:0/1740190882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680094a0 con 0x7f1a740c9f40 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.585+0000 7f1a73fff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 0x7f1a74151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:54:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a737fe700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c4e0 tx=0x7f1a6400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a640052d0 con 0x7f1a740c9f40 2023-06-09T11:54:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64005430 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a74159c60 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a640038c0 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.586+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a74159980 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.587+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a74159f40 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.588+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a64003a20 con 0x7f1a740c9f40 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.589+0000 7f1a5bfff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 0x7f1a6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:54:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.589+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f1a64088e90 con 0x7f1a740c9f40 2023-06-09T11:54:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.589+0000 7f1a72ffd700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 0x7f1a6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:54:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.590+0000 7f1a72ffd700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 0x7f1a6006d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a680090b0 tx=0x7f1a6801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:54:58.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.593+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1a640536a0 con 0x7f1a740c9f40 2023-06-09T11:54:58.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.844+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a741f05e0 con 0x7f1a740c9f40 2023-06-09T11:54:58.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.845+0000 7f1a5bfff700 1 -- 172.21.15.7:0/1740190882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f1a741f05e0 con 0x7f1a740c9f40 2023-06-09T11:54:58.849 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:54:58.849 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:51.911234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 msgr2=0x7f1a6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 0x7f1a6006d190 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1a680090b0 tx=0x7f1a6801c040 comp rx=0 tx=0).stop 2023-06-09T11:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 msgr2=0x7f1a741f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400c4e0 tx=0x7f1a6400c7f0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 shutdown_connections 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1a6006ace0 0x7f1a6006d190 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1a7415b4a0 0x7f1a74151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a740c9f40 0x7f1a741f00a0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 --2- 172.21.15.7:0/1740190882 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1a74153880 0x7f1a740ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 >> 172.21.15.7:0/1740190882 conn(0x7f1a740737f0 msgr2=0x7f1a7414c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.847+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 shutdown_connections 2023-06-09T11:54:58.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:54:58.848+0000 7f1a7a1ff700 1 -- 172.21.15.7:0/1740190882 wait complete. 2023-06-09T11:54:58.853 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 203 2023-06-09T11:54:58.869 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:55:00.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.079+0000 7f2172dff700 1 Processor -- start 2023-06-09T11:55:01.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.079+0000 7f2172dff700 1 -- start start 2023-06-09T11:55:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f2172dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f216c153b30 0x7f216c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f2172dff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c15a390 con 0x7f216c071fa0 2023-06-09T11:55:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f2172dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0ca720 con 0x7f216c153760 2023-06-09T11:55:01.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f2172dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0c5f40 con 0x7f216c153b30 2023-06-09T11:55:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f217139c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f216c071fa0 0x7f216c15cc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46412/0 (socket says 172.21.15.7:46412) 2023-06-09T11:55:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.080+0000 7f217139c700 1 -- 172.21.15.7:0/2315028088 learned_addr learned my addr 172.21.15.7:0/2315028088 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216bfff700 1 --2- 172.21.15.7:0/2315028088 >> v2:172.21.15.117:3300/0 conn(0x7f216c153b30 0x7f216c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063962758 0 0) 0x7f216c15a390 con 0x7f216c071fa0 2023-06-09T11:55:01.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f215c0035b0 con 0x7f216c071fa0 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702985696 0 0) 0x7f215c0035b0 con 0x7f216c071fa0 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 >> v2:172.21.15.117:3300/0 conn(0x7f216c153b30 msgr2=0x7f216c0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 --2- 172.21.15.7:0/2315028088 >> v2:172.21.15.117:3300/0 conn(0x7f216c153b30 0x7f216c0ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 >> v1:172.21.15.117:6789/0 conn(0x7f216c153760 legacy=0x7f216c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0c5a20 con 0x7f216c071fa0 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f21640019f0 con 0x7f216c071fa0 2023-06-09T11:55:01.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 >> v1:172.21.15.7:6789/0 conn(0x7f216c071fa0 legacy=0x7f216c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f215c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c004850 0x7f215c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215c005200 0x7f215c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0c5f40 con 0x7f215c003a00 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c00c060 con 0x7f215c004850 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.081+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c00c220 con 0x7f215c005200 2023-06-09T11:55:01.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f215c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f217139c700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215c005200 0x7f215c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f216bfff700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c004850 0x7f215c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 -- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215c005200 msgr2=0x7f215c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f215c005200 0x7f215c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 -- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c004850 msgr2=0x7f215c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f215c004850 0x7f215c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 -- 172.21.15.7:0/2315028088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0035b0 con 0x7f215c003a00 2023-06-09T11:55:01.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f217139c700 1 -- 172.21.15.7:0/2315028088 reap_dead start 2023-06-09T11:55:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.082+0000 7f2170b9b700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f215c003e10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f21600017c0 tx=0x7f215c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fff21c9ee0ad6ec server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600052d0 con 0x7f215c003a00 2023-06-09T11:55:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160005430 con 0x7f215c003a00 2023-06-09T11:55:01.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f216b7fe700 1 -- 172.21.15.7:0/2315028088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600038c0 con 0x7f215c003a00 2023-06-09T11:55:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 -- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 msgr2=0x7f215c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f215c003e10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f21600017c0 tx=0x7f215c005880 comp rx=0 tx=0).stop 2023-06-09T11:55:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 -- 172.21.15.7:0/2315028088 shutdown_connections 2023-06-09T11:55:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 --2- 172.21.15.7:0/2315028088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f215c003e10 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 -- 172.21.15.7:0/2315028088 >> 172.21.15.7:0/2315028088 conn(0x7f216c0737f0 msgr2=0x7f216c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 -- 172.21.15.7:0/2315028088 shutdown_connections 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.083+0000 7f2172dff700 1 -- 172.21.15.7:0/2315028088 wait complete. 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.085+0000 7f2172dff700 1 Processor -- start 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.085+0000 7f2172dff700 1 -- start start 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f216c1574e0 0x7f216c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f216c158780 0x7f216c14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0b4900 con 0x7f215c003a00 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c157ad0 con 0x7f216c1574e0 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2172dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1f32e0 con 0x7f216c158780 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2170b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2170b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39834/0 (socket says 172.21.15.7:39834) 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f2170b9b700 1 -- 172.21.15.7:0/74205182 learned_addr learned my addr 172.21.15.7:0/74205182 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f217139c700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f216c158780 0x7f216c14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.086+0000 7f216bfff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f216c1574e0 0x7f216c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 -- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f216c158780 msgr2=0x7f216c14dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f216c158780 0x7f216c14dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 -- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f216c1574e0 msgr2=0x7f216c15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f216c1574e0 0x7f216c15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 -- 172.21.15.7:0/74205182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c159a60 con 0x7f215c003a00 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2170b9b700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c030 tx=0x7f2160001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160001290 con 0x7f215c003a00 2023-06-09T11:55:01.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216001d480 con 0x7f215c003a00 2023-06-09T11:55:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216001d830 con 0x7f215c003a00 2023-06-09T11:55:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c159780 con 0x7f215c003a00 2023-06-09T11:55:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.087+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c159d40 con 0x7f215c003a00 2023-06-09T11:55:01.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.088+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c074cf0 con 0x7f215c003a00 2023-06-09T11:55:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.089+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2160019040 con 0x7f215c003a00 2023-06-09T11:55:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.091+0000 7f21697fa700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 0x7f215006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.091+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f216008a650 con 0x7f215c003a00 2023-06-09T11:55:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.091+0000 7f216bfff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 0x7f215006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:01.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.092+0000 7f216bfff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 0x7f215006d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f216c1496d0 tx=0x7f2158009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:01.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.095+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f21600550d0 con 0x7f215c003a00 2023-06-09T11:55:01.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.347+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f216c0b2070 con 0x7f215c003a00 2023-06-09T11:55:01.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.347+0000 7f21697fa700 1 -- 172.21.15.7:0/74205182 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f21600586f0 con 0x7f215c003a00 2023-06-09T11:55:01.351 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:01.351 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:51.911234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 msgr2=0x7f215006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 0x7f215006d190 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f216c1496d0 tx=0x7f2158009210 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 msgr2=0x7f216c159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f215c00c030 tx=0x7f2160001120 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 shutdown_connections 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f215006ace0 0x7f215006d190 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f216c158780 0x7f216c14dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f215c003a00 0x7f216c159000 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 --2- 172.21.15.7:0/74205182 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f216c1574e0 0x7f216c15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:01.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 >> 172.21.15.7:0/74205182 conn(0x7f216c0737f0 msgr2=0x7f216c14f360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:01.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 shutdown_connections 2023-06-09T11:55:01.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:01.350+0000 7f2172dff700 1 -- 172.21.15.7:0/74205182 wait complete. 2023-06-09T11:55:01.355 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 203 2023-06-09T11:55:01.371 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:55:03.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.581+0000 7f27f1bff700 1 Processor -- start 2023-06-09T11:55:03.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.581+0000 7f27f1bff700 1 -- start start 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27f1bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27ec158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27f1bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec1580a0 con 0x7f27ec151510 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27f1bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec0c6180 con 0x7f27ec0c5b40 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27f1bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1518e0 con 0x7f27ec071230 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27eb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f27ec151510 0x7f27ec15b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47270/0 (socket says 172.21.15.7:47270) 2023-06-09T11:55:03.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27eb7fe700 1 -- 172.21.15.7:0/2903964031 learned_addr learned my addr 172.21.15.7:0/2903964031 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.583+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27ec158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3878509052 0 0) 0x7f27ec1580a0 con 0x7f27ec151510 2023-06-09T11:55:03.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27d40035b0 con 0x7f27ec151510 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2791878922 0 0) 0x7f27ec0c6180 con 0x7f27ec0c5b40 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ec1580a0 con 0x7f27ec0c5b40 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1506554757 0 0) 0x7f27d40035b0 con 0x7f27ec151510 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 msgr2=0x7f27ec158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27ec158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 >> v1:172.21.15.117:6789/0 conn(0x7f27ec0c5b40 legacy=0x7f27ec070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec157490 con 0x7f27ec151510 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f27dc001670 con 0x7f27ec151510 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27ec158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 >> v1:172.21.15.7:6789/0 conn(0x7f27ec151510 legacy=0x7f27ec15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d4003bc0 0x7f27d4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27d4004850 0x7f27d4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1518e0 con 0x7f27d4004850 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27d400bac0 con 0x7f27d4003bc0 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27d400bc50 con 0x7f27ec071230 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27eaffd700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27d4004850 0x7f27d4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27eb7fe700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d4003bc0 0x7f27d4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.584+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 -- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 msgr2=0x7f27d4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:03.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27ec071230 0x7f27d4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 -- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d4003bc0 msgr2=0x7f27d4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27d4003bc0 0x7f27d4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 -- 172.21.15.7:0/2903964031 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e4009a70 con 0x7f27d4004850 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27eaffd700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27d4004850 0x7f27d4004cc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f27e0001970 tx=0x7f27d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f7569fc0d55f02 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0001430 con 0x7f27d4004850 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00052d0 con 0x7f27d4004850 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.585+0000 7f27ea7fc700 1 -- 172.21.15.7:0/2903964031 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003680 con 0x7f27d4004850 2023-06-09T11:55:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 -- 172.21.15.7:0/2903964031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27d4004850 msgr2=0x7f27d4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2903964031 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27d4004850 0x7f27d4004cc0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f27e0001970 tx=0x7f27d4005880 comp rx=0 tx=0).stop 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27ebfff700 1 -- 172.21.15.7:0/2903964031 reap_dead start 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 -- 172.21.15.7:0/2903964031 shutdown_connections 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 -- 172.21.15.7:0/2903964031 >> 172.21.15.7:0/2903964031 conn(0x7f27ec06a4d0 msgr2=0x7f27ec06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 -- 172.21.15.7:0/2903964031 shutdown_connections 2023-06-09T11:55:03.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.586+0000 7f27f1bff700 1 -- 172.21.15.7:0/2903964031 wait complete. 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.588+0000 7f27f1bff700 1 Processor -- start 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.588+0000 7f27f1bff700 1 -- start start 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d4004850 0x7f27ec146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27ec14cb60 0x7f27ec14d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec14ea60 con 0x7f27ec14dd00 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec158760 con 0x7f27ec14cb60 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27f1bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec158d00 con 0x7f27d4004850 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54980/0 (socket says 172.21.15.7:54980) 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27ebfff700 1 -- 172.21.15.7:0/2189845973 learned_addr learned my addr 172.21.15.7:0/2189845973 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27eb7fe700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d4004850 0x7f27ec146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27eaffd700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27ec14cb60 0x7f27ec14d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27ebfff700 1 -- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d4004850 msgr2=0x7f27ec146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.589+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d4004850 0x7f27ec146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27ebfff700 1 -- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27ec14cb60 msgr2=0x7f27ec14d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27ec14cb60 0x7f27ec14d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27ebfff700 1 -- 172.21.15.7:0/2189845973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec157a50 con 0x7f27ec14dd00 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27ebfff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f27ec06d970 tx=0x7f27e400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e401c070 con 0x7f27ec14dd00 2023-06-09T11:55:03.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e400f040 con 0x7f27ec14dd00 2023-06-09T11:55:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e400e040 con 0x7f27ec14dd00 2023-06-09T11:55:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec157770 con 0x7f27ec14dd00 2023-06-09T11:55:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.590+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec157d30 con 0x7f27ec14dd00 2023-06-09T11:55:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.591+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec1ead10 con 0x7f27ec14dd00 2023-06-09T11:55:03.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.592+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f27e400fd90 con 0x7f27ec14dd00 2023-06-09T11:55:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.593+0000 7f27d3fff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 0x7f27d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.593+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 8098+0+0 (secure 0 0 0) 0x7f27e4089a30 con 0x7f27ec14dd00 2023-06-09T11:55:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.593+0000 7f27eb7fe700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 0x7f27d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:03.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.594+0000 7f27eb7fe700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 0x7f27d806d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f27dc001b60 tx=0x7f27dc000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:03.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.598+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f27e4057cd0 con 0x7f27ec14dd00 2023-06-09T11:55:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.851+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27ec074cf0 con 0x7f27ec14dd00 2023-06-09T11:55:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.852+0000 7f27d3fff700 1 -- 172.21.15.7:0/2189845973 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 203 v203) v1 ==== 95+0+6221 (secure 0 0 0) 0x7f27e4057860 con 0x7f27ec14dd00 2023-06-09T11:55:03.854 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:03.854 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":203,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7807,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":199}],"filesystems":[{"mdsmap":{"epoch":203,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:54:51.911234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":72,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7405,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7405":{"gid":7405,"name":"d","rank":1,"incarnation":178,"state":"up:active","state_seq":18,"addr":"172.21.15.117:6835/3642322665","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3642322665},{"type":"v1","addr":"172.21.15.117:6835","nonce":3642322665}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 msgr2=0x7f27d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 0x7f27d806d190 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f27dc001b60 tx=0x7f27dc000ed0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 msgr2=0x7f27ec1eaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f27ec06d970 tx=0x7f27e400f490 comp rx=0 tx=0).stop 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 shutdown_connections 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27ec14dd00 0x7f27ec1eaef0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f27d806ace0 0x7f27d806d190 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f27d4004850 0x7f27ec146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 --2- 172.21.15.7:0/2189845973 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f27ec14cb60 0x7f27ec14d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 >> 172.21.15.7:0/2189845973 conn(0x7f27ec06a4d0 msgr2=0x7f27ec0cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 shutdown_connections 2023-06-09T11:55:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:03.854+0000 7f27f1bff700 1 -- 172.21.15.7:0/2189845973 wait complete. 2023-06-09T11:55:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 203 2023-06-09T11:55:03.874 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is laggy/crashed, in failed state, or mds.d is removed from mdsmap 2023-06-09T11:55:05.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.087+0000 7fb6da3ff700 1 Processor -- start 2023-06-09T11:55:06.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.087+0000 7fb6da3ff700 1 -- start start 2023-06-09T11:55:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d4153880 0x7fb6d415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d40c9f40 0x7fb6d415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb6d415b2d0 0x7fb6d40c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d40c5f10 con 0x7fb6d40c9f40 2023-06-09T11:55:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d415a500 con 0x7fb6d415b2d0 2023-06-09T11:55:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6da3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415da00 con 0x7fb6d4153880 2023-06-09T11:55:06.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d40c9f40 0x7fb6d415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6d40c9f40 0x7fb6d415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54992/0 (socket says 172.21.15.7:54992) 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6d37fe700 1 -- 172.21.15.7:0/593402989 learned_addr learned my addr 172.21.15.7:0/593402989 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6d3fff700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d4153880 0x7fb6d415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.089+0000 7fb6d899c700 1 --2- 172.21.15.7:0/593402989 >> v2:172.21.15.117:3300/0 conn(0x7fb6d415b2d0 0x7fb6d40c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 -- 172.21.15.7:0/593402989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d4153880 msgr2=0x7fb6d415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d4153880 0x7fb6d415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 -- 172.21.15.7:0/593402989 >> v2:172.21.15.117:3300/0 conn(0x7fb6d415b2d0 msgr2=0x7fb6d40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:06.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/593402989 >> v2:172.21.15.117:3300/0 conn(0x7fb6d415b2d0 0x7fb6d40c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 -- 172.21.15.7:0/593402989 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d415b750 con 0x7fb6d40c9f40 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c9f40 0x7fb6d415ad90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4071db0 tx=0x7fb6c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c8d97ff55d1cfc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d2ffd700 1 -- 172.21.15.7:0/593402989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011830 con 0x7fb6d40c9f40 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d2ffd700 1 -- 172.21.15.7:0/593402989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8011990 con 0x7fb6d40c9f40 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.090+0000 7fb6d2ffd700 1 -- 172.21.15.7:0/593402989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8011d20 con 0x7fb6d40c9f40 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 -- 172.21.15.7:0/593402989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c9f40 msgr2=0x7fb6d415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c9f40 0x7fb6d415ad90 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4071db0 tx=0x7fb6c8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 -- 172.21.15.7:0/593402989 shutdown_connections 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/593402989 >> v2:172.21.15.117:3300/0 conn(0x7fb6d415b2d0 0x7fb6d40c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d4153880 0x7fb6d415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/593402989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d40c9f40 0x7fb6d415ad90 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 -- 172.21.15.7:0/593402989 >> 172.21.15.7:0/593402989 conn(0x7fb6d40737f0 msgr2=0x7fb6d40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 -- 172.21.15.7:0/593402989 shutdown_connections 2023-06-09T11:55:06.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.091+0000 7fb6da3ff700 1 -- 172.21.15.7:0/593402989 wait complete. 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.093+0000 7fb6da3ff700 1 Processor -- start 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.093+0000 7fb6da3ff700 1 -- start start 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 0x7fb6d406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d415b2d0 0x7fb6d414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d41e3f50 con 0x7fb6d4153880 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d41e3c70 con 0x7fb6d40c9f40 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6da3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40b2fa0 con 0x7fb6d415b2d0 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54994/0 (socket says 172.21.15.7:54994) 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 -- 172.21.15.7:0/3932019620 learned_addr learned my addr 172.21.15.7:0/3932019620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d3fff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 0x7fb6d406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 -- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d415b2d0 msgr2=0x7fb6d414c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d415b2d0 0x7fb6d414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 -- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 msgr2=0x7fb6d406a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 0x7fb6d406a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d37fe700 1 -- 172.21.15.7:0/3932019620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009550 con 0x7fb6d4153880 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.094+0000 7fb6d3fff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 0x7fb6d406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6d37fe700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8004300 con 0x7fb6d4153880 2023-06-09T11:55:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c801f070 con 0x7fb6d4153880 2023-06-09T11:55:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80036a0 con 0x7fb6d4153880 2023-06-09T11:55:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4159980 con 0x7fb6d4153880 2023-06-09T11:55:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.095+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d41596e0 con 0x7fb6d4153880 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.096+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d4159c60 con 0x7fb6d4153880 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.097+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb6c8021030 con 0x7fb6d4153880 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.098+0000 7fb6d0ff9700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 0x7fb6c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.099+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fb6c808af20 con 0x7fb6d4153880 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.099+0000 7fb6d3fff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 0x7fb6c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.099+0000 7fb6d3fff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 0x7fb6c006d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009ae0 tx=0x7fb6c4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.103+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb6c8055760 con 0x7fb6d4153880 2023-06-09T11:55:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.360+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d40b2e00 con 0x7fb6d4153880 2023-06-09T11:55:06.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.361+0000 7fb6d0ff9700 1 -- 172.21.15.7:0/3932019620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) v1 ==== 95+0+5458 (secure 0 0 0) 0x7fb6d40b2e00 con 0x7fb6d4153880 2023-06-09T11:55:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:06.364 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":204,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:04.730024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.363+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 msgr2=0x7fb6c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.363+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 0x7fb6c006d190 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4009ae0 tx=0x7fb6c4008040 comp rx=0 tx=0).stop 2023-06-09T11:55:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.363+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 msgr2=0x7fb6d41ef3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.363+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8000c00 tx=0x7fb6c8004750 comp rx=0 tx=0).stop 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 shutdown_connections 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb6c006ace0 0x7fb6c006d190 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6d415b2d0 0x7fb6d414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6d40c9f40 0x7fb6d406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 --2- 172.21.15.7:0/3932019620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6d4153880 0x7fb6d41ef3d0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 >> 172.21.15.7:0/3932019620 conn(0x7fb6d40737f0 msgr2=0x7fb6d414dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 shutdown_connections 2023-06-09T11:55:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.364+0000 7fb6da3ff700 1 -- 172.21.15.7:0/3932019620 wait complete. 2023-06-09T11:55:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 204 2023-06-09T11:55:06.384 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d down, removed from mdsmap 2023-06-09T11:55:06.384 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:55:06.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:06.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.581+0000 7f458c9ff700 1 Processor -- start 2023-06-09T11:55:06.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.582+0000 7f458c9ff700 1 -- start start 2023-06-09T11:55:06.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f458c9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4588071230 0x7f4588158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f458c9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45881580a0 con 0x7f45880c5b40 2023-06-09T11:55:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f458c9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45880c6180 con 0x7f4588151510 2023-06-09T11:55:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f458c9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45881518e0 con 0x7f4588071230 2023-06-09T11:55:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f457ffff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f45880c5b40 0x7f4588070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47284/0 (socket says 172.21.15.7:47284) 2023-06-09T11:55:06.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f457ffff700 1 -- 172.21.15.7:0/1414692881 learned_addr learned my addr 172.21.15.7:0/1414692881 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.583+0000 7f4586d9d700 1 --2- 172.21.15.7:0/1414692881 >> v2:172.21.15.117:3300/0 conn(0x7f4588071230 0x7f4588158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4585d9b700 1 -- 172.21.15.7:0/1414692881 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366009535 0 0) 0x7f45881580a0 con 0x7f45880c5b40 2023-06-09T11:55:06.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4585d9b700 1 -- 172.21.15.7:0/1414692881 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45740035b0 con 0x7f45880c5b40 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4586d9d700 1 -- 172.21.15.7:0/1414692881 >> v1:172.21.15.117:6789/0 conn(0x7f4588151510 legacy=0x7f458815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4586d9d700 1 -- 172.21.15.7:0/1414692881 >> v1:172.21.15.7:6789/0 conn(0x7f45880c5b40 legacy=0x7f4588070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4586d9d700 1 -- 172.21.15.7:0/1414692881 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4588157490 con 0x7f4588071230 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4586d9d700 1 --2- 172.21.15.7:0/1414692881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4588071230 0x7f4588158870 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4580009d90 tx=0x7f458000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f74b6dcf9502bc47 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4585d9b700 1 -- 172.21.15.7:0/1414692881 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458000e070 con 0x7f4588071230 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4585d9b700 1 -- 172.21.15.7:0/1414692881 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f458000bb20 con 0x7f4588071230 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.584+0000 7f4585d9b700 1 -- 172.21.15.7:0/1414692881 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4580012430 con 0x7f4588071230 2023-06-09T11:55:06.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 -- 172.21.15.7:0/1414692881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4588071230 msgr2=0x7f4588158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 --2- 172.21.15.7:0/1414692881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4588071230 0x7f4588158870 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4580009d90 tx=0x7f458000b330 comp rx=0 tx=0).stop 2023-06-09T11:55:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 -- 172.21.15.7:0/1414692881 shutdown_connections 2023-06-09T11:55:06.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 --2- 172.21.15.7:0/1414692881 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4588071230 0x7f4588158870 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 -- 172.21.15.7:0/1414692881 >> 172.21.15.7:0/1414692881 conn(0x7f458806a4d0 msgr2=0x7f458806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 -- 172.21.15.7:0/1414692881 shutdown_connections 2023-06-09T11:55:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.585+0000 7f458c9ff700 1 -- 172.21.15.7:0/1414692881 wait complete. 2023-06-09T11:55:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.587+0000 7f458c9ff700 1 Processor -- start 2023-06-09T11:55:06.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.587+0000 7f458c9ff700 1 -- start start 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45880c5b40 0x7f4588156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4588151510 0x7f45881ed440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45880c3e00 con 0x7f4588071230 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45880c4ff0 con 0x7f45880c5b40 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458c9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45880c4750 con 0x7f4588151510 2023-06-09T11:55:06.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55006/0 (socket says 172.21.15.7:55006) 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.588+0000 7f458659c700 1 -- 172.21.15.7:0/684399560 learned_addr learned my addr 172.21.15.7:0/684399560 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f457ffff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45880c5b40 0x7f4588156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f4586d9d700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4588151510 0x7f45881ed440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 -- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4588151510 msgr2=0x7f45881ed440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4588151510 0x7f45881ed440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 -- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45880c5b40 msgr2=0x7f4588156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45880c5b40 0x7f4588156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 -- 172.21.15.7:0/684399560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580009a70 con 0x7f4588071230 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458659c700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f457800c4e0 tx=0x7f457800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45780052d0 con 0x7f4588071230 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4578005430 con 0x7f4588071230 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4588157a50 con 0x7f4588071230 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45780038c0 con 0x7f4588071230 2023-06-09T11:55:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.589+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4588157770 con 0x7f4588071230 2023-06-09T11:55:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.590+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4588157d30 con 0x7f4588071230 2023-06-09T11:55:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.591+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4578003a20 con 0x7f4588071230 2023-06-09T11:55:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.592+0000 7f457effd700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 0x7f456806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.592+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f45780891a0 con 0x7f4588071230 2023-06-09T11:55:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.593+0000 7f457ffff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 0x7f456806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.593+0000 7f457ffff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 0x7f456806d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4588159a00 tx=0x7f4570009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.596+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f45780539e0 con 0x7f4588071230 2023-06-09T11:55:06.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.849+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4588150c30 con 0x7f4588071230 2023-06-09T11:55:06.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.850+0000 7f457effd700 1 -- 172.21.15.7:0/684399560 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) v1 ==== 95+0+5458 (secure 0 0 0) 0x7f4578057000 con 0x7f4588071230 2023-06-09T11:55:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:06.853 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":204,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:04.730024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:06.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.852+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 msgr2=0x7f456806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.852+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 0x7f456806d190 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4588159a00 tx=0x7f4570009040 comp rx=0 tx=0).stop 2023-06-09T11:55:06.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.852+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 msgr2=0x7f458815a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:06.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.852+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f457800c4e0 tx=0x7f457800c8a0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 shutdown_connections 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f456806ace0 0x7f456806d190 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4588151510 0x7f45881ed440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4588071230 0x7f458815a1c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 --2- 172.21.15.7:0/684399560 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f45880c5b40 0x7f4588156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 >> 172.21.15.7:0/684399560 conn(0x7f458806a4d0 msgr2=0x7f458814adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 shutdown_connections 2023-06-09T11:55:06.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:06.853+0000 7f458c9ff700 1 -- 172.21.15.7:0/684399560 wait complete. 2023-06-09T11:55:06.857 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 204 2023-06-09T11:55:06.873 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 204, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 204, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:55:04.730024+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:55:08.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:09.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.084+0000 7f99911ff700 1 Processor -- start 2023-06-09T11:55:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.084+0000 7f99911ff700 1 -- start start 2023-06-09T11:55:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.085+0000 7f99911ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f998c153880 0x7f998c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f99911ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f998c15b4a0 0x7f998c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f99911ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c15a500 con 0x7f998c0c9f40 2023-06-09T11:55:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f99911ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c0c5f10 con 0x7f998c15b4a0 2023-06-09T11:55:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f99911ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c15da00 con 0x7f998c153880 2023-06-09T11:55:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998b59e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f998c15b4a0 0x7f998c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f998c0c9f40 0x7f998c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47296/0 (socket says 172.21.15.7:47296) 2023-06-09T11:55:09.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998a59c700 1 -- 172.21.15.7:0/961702304 learned_addr learned my addr 172.21.15.7:0/961702304 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998ad9d700 1 --2- 172.21.15.7:0/961702304 >> v2:172.21.15.117:3300/0 conn(0x7f998c153880 0x7f998c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998b59e700 1 -- 172.21.15.7:0/961702304 >> v2:172.21.15.117:3300/0 conn(0x7f998c153880 msgr2=0x7f998c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998b59e700 1 --2- 172.21.15.7:0/961702304 >> v2:172.21.15.117:3300/0 conn(0x7f998c153880 0x7f998c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998b59e700 1 -- 172.21.15.7:0/961702304 >> v1:172.21.15.7:6789/0 conn(0x7f998c0c9f40 legacy=0x7f998c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f998b59e700 1 -- 172.21.15.7:0/961702304 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c1596e0 con 0x7f998c15b4a0 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.086+0000 7f9989d9b700 1 -- 172.21.15.7:0/961702304 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341922681 0 0) 0x7f9980001110 con 0x7f998c0c9f40 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f998b59e700 1 --2- 172.21.15.7:0/961702304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c15b4a0 0x7f998c0c4e30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f998c08d340 tx=0x7f998400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8150fe69d717bd server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f9989d9b700 1 -- 172.21.15.7:0/961702304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998400e070 con 0x7f998c15b4a0 2023-06-09T11:55:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f9989d9b700 1 -- 172.21.15.7:0/961702304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998400bb20 con 0x7f998c15b4a0 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f9989d9b700 1 -- 172.21.15.7:0/961702304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9984012430 con 0x7f998c15b4a0 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 -- 172.21.15.7:0/961702304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c15b4a0 msgr2=0x7f998c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 --2- 172.21.15.7:0/961702304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c15b4a0 0x7f998c0c4e30 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f998c08d340 tx=0x7f998400b330 comp rx=0 tx=0).stop 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 -- 172.21.15.7:0/961702304 shutdown_connections 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 --2- 172.21.15.7:0/961702304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c15b4a0 0x7f998c0c4e30 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 --2- 172.21.15.7:0/961702304 >> v2:172.21.15.117:3300/0 conn(0x7f998c153880 0x7f998c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.087+0000 7f99911ff700 1 -- 172.21.15.7:0/961702304 >> 172.21.15.7:0/961702304 conn(0x7f998c0737f0 msgr2=0x7f998c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:09.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.088+0000 7f99911ff700 1 -- 172.21.15.7:0/961702304 shutdown_connections 2023-06-09T11:55:09.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.088+0000 7f99911ff700 1 -- 172.21.15.7:0/961702304 wait complete. 2023-06-09T11:55:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.089+0000 7f99911ff700 1 Processor -- start 2023-06-09T11:55:09.094 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.089+0000 7f99911ff700 1 -- start start 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f998c0c9f40 0x7f998c1e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f998c15b4a0 0x7f998c14a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c152830 con 0x7f998c153880 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1511d0 con 0x7f998c0c9f40 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f99911ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c1effa0 con 0x7f998c15b4a0 2023-06-09T11:55:09.095 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.090+0000 7f998a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55024/0 (socket says 172.21.15.7:55024) 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 -- 172.21.15.7:0/2057219005 learned_addr learned my addr 172.21.15.7:0/2057219005 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998ad9d700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f998c0c9f40 0x7f998c1e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998b59e700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f998c15b4a0 0x7f998c14a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 -- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f998c15b4a0 msgr2=0x7f998c14a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f998c15b4a0 0x7f998c14a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 -- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f998c0c9f40 msgr2=0x7f998c1e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f998c0c9f40 0x7f998c1e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 -- 172.21.15.7:0/2057219005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9984009a70 con 0x7f998c153880 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.091+0000 7f998a59c700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f998000c730 tx=0x7f998000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.092+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980004c80 con 0x7f998c153880 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.092+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9980004e00 con 0x7f998c153880 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.092+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c159c60 con 0x7f998c153880 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.092+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9980007310 con 0x7f998c153880 2023-06-09T11:55:09.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.092+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c159980 con 0x7f998c153880 2023-06-09T11:55:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.093+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c159f40 con 0x7f998c153880 2023-06-09T11:55:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.094+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9980007470 con 0x7f998c153880 2023-06-09T11:55:09.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.095+0000 7f99737fe700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 0x7f997806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.095+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f99800897f0 con 0x7f998c153880 2023-06-09T11:55:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.095+0000 7f998ad9d700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 0x7f997806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:09.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.095+0000 7f998ad9d700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 0x7f997806d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f997c009be0 tx=0x7f997c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:09.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.099+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9980054000 con 0x7f998c153880 2023-06-09T11:55:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.352+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f998c151da0 con 0x7f998c153880 2023-06-09T11:55:09.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.353+0000 7f99737fe700 1 -- 172.21.15.7:0/2057219005 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) v1 ==== 95+0+5458 (secure 0 0 0) 0x7f9980057620 con 0x7f998c153880 2023-06-09T11:55:09.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:09.356 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":204,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:04.730024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.355+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 msgr2=0x7f997806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.355+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 0x7f997806d190 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f997c009be0 tx=0x7f997c009210 comp rx=0 tx=0).stop 2023-06-09T11:55:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 msgr2=0x7f998c149d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:09.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f998000c730 tx=0x7f998000ca40 comp rx=0 tx=0).stop 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 shutdown_connections 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f997806ace0 0x7f997806d190 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f998c15b4a0 0x7f998c14a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f998c0c9f40 0x7f998c1e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 --2- 172.21.15.7:0/2057219005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f998c153880 0x7f998c149d70 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 >> 172.21.15.7:0/2057219005 conn(0x7f998c0737f0 msgr2=0x7f998c14bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:09.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 shutdown_connections 2023-06-09T11:55:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:09.356+0000 7f99911ff700 1 -- 172.21.15.7:0/2057219005 wait complete. 2023-06-09T11:55:09.360 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 204 2023-06-09T11:55:09.376 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:11.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.588+0000 7f2cef3ff700 1 Processor -- start 2023-06-09T11:55:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.588+0000 7f2cef3ff700 1 -- start start 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ce8153880 0x7f2ce815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce80c9f40 0x7f2ce815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ce815b2d0 0x7f2ce80c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce80c5f10 con 0x7f2ce815b2d0 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce815a500 con 0x7f2ce8153880 2023-06-09T11:55:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2cef3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce815da00 con 0x7f2ce80c9f40 2023-06-09T11:55:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.589+0000 7f2ced99c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ce815b2d0 0x7f2ce80c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ce815b2d0 0x7f2ce80c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55026/0 (socket says 172.21.15.7:55026) 2023-06-09T11:55:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 -- 172.21.15.7:0/3635869300 learned_addr learned my addr 172.21.15.7:0/3635869300 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2cec99a700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce80c9f40 0x7f2ce815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced19b700 1 --2- 172.21.15.7:0/3635869300 >> v2:172.21.15.117:3300/0 conn(0x7f2ce8153880 0x7f2ce815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 -- 172.21.15.7:0/3635869300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce80c9f40 msgr2=0x7f2ce815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce80c9f40 0x7f2ce815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 -- 172.21.15.7:0/3635869300 >> v2:172.21.15.117:3300/0 conn(0x7f2ce8153880 msgr2=0x7f2ce815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 --2- 172.21.15.7:0/3635869300 >> v2:172.21.15.117:3300/0 conn(0x7f2ce8153880 0x7f2ce815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 -- 172.21.15.7:0/3635869300 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce815b750 con 0x7f2ce815b2d0 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2ced99c700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce815b2d0 0x7f2ce80c4e30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ce808d340 tx=0x7f2ce000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e15df9be3fe283 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2cdffff700 1 -- 172.21.15.7:0/3635869300 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000e070 con 0x7f2ce815b2d0 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.590+0000 7f2cdffff700 1 -- 172.21.15.7:0/3635869300 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce000b910 con 0x7f2ce815b2d0 2023-06-09T11:55:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cdffff700 1 -- 172.21.15.7:0/3635869300 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000bcc0 con 0x7f2ce815b2d0 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 -- 172.21.15.7:0/3635869300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce815b2d0 msgr2=0x7f2ce80c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce815b2d0 0x7f2ce80c4e30 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2ce808d340 tx=0x7f2ce000b330 comp rx=0 tx=0).stop 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 -- 172.21.15.7:0/3635869300 shutdown_connections 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/3635869300 >> v2:172.21.15.117:3300/0 conn(0x7f2ce8153880 0x7f2ce815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce815b2d0 0x7f2ce80c4e30 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/3635869300 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce80c9f40 0x7f2ce815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 -- 172.21.15.7:0/3635869300 >> 172.21.15.7:0/3635869300 conn(0x7f2ce80737f0 msgr2=0x7f2ce80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 -- 172.21.15.7:0/3635869300 shutdown_connections 2023-06-09T11:55:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.591+0000 7f2cef3ff700 1 -- 172.21.15.7:0/3635869300 wait complete. 2023-06-09T11:55:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.593+0000 7f2cef3ff700 1 Processor -- start 2023-06-09T11:55:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.593+0000 7f2cef3ff700 1 -- start start 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce8153880 0x7f2ce81ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 0x7f2ce81e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81593e0 con 0x7f2ce80c9f40 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81f3520 con 0x7f2ce815b2d0 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cef3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80cd9c0 con 0x7f2ce8153880 2023-06-09T11:55:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2ced19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2ced19b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55042/0 (socket says 172.21.15.7:55042) 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2ced19b700 1 -- 172.21.15.7:0/342546553 learned_addr learned my addr 172.21.15.7:0/342546553 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2ced99c700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 0x7f2ce81e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.594+0000 7f2cec99a700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce8153880 0x7f2ce81ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 -- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce8153880 msgr2=0x7f2ce81ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce8153880 0x7f2ce81ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 -- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 msgr2=0x7f2ce81e4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 0x7f2ce81e4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 -- 172.21.15.7:0/342546553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce0009b30 con 0x7f2ce80c9f40 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced99c700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 0x7f2ce81e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2ced19b700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9b0 tx=0x7f2ce400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4009710 con 0x7f2ce80c9f40 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8159980 con 0x7f2ce80c9f40 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce400c850 con 0x7f2ce80c9f40 2023-06-09T11:55:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce400cbc0 con 0x7f2ce80c9f40 2023-06-09T11:55:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.595+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81596e0 con 0x7f2ce80c9f40 2023-06-09T11:55:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.596+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8159c60 con 0x7f2ce80c9f40 2023-06-09T11:55:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.597+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2ce400cd20 con 0x7f2ce80c9f40 2023-06-09T11:55:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.598+0000 7f2cddffb700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.598+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f2ce408a240 con 0x7f2ce80c9f40 2023-06-09T11:55:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.599+0000 7f2cec99a700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.599+0000 7f2cec99a700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 0x7f2ccc06d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2ce814c800 tx=0x7f2cd8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.603+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2ce4054970 con 0x7f2ce80c9f40 2023-06-09T11:55:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.861+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ce8074cf0 con 0x7f2ce80c9f40 2023-06-09T11:55:11.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.861+0000 7f2cddffb700 1 -- 172.21.15.7:0/342546553 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 204 v204) v1 ==== 95+0+5458 (secure 0 0 0) 0x7f2ce4057f90 con 0x7f2ce80c9f40 2023-06-09T11:55:11.864 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:11.864 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":204,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":204,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:04.730024+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:11.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 msgr2=0x7f2ccc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 0x7f2ccc06d190 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2ce814c800 tx=0x7f2cd8009210 comp rx=0 tx=0).stop 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 msgr2=0x7f2ce81e64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400e9b0 tx=0x7f2ce400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 shutdown_connections 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ce815b2d0 0x7f2ce81e4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ce80c9f40 0x7f2ce81e64e0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ccc06ace0 0x7f2ccc06d190 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 --2- 172.21.15.7:0/342546553 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ce8153880 0x7f2ce81ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 >> 172.21.15.7:0/342546553 conn(0x7f2ce80737f0 msgr2=0x7f2ce814c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 shutdown_connections 2023-06-09T11:55:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:11.864+0000 7f2cef3ff700 1 -- 172.21.15.7:0/342546553 wait complete. 2023-06-09T11:55:11.868 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 204 2023-06-09T11:55:11.884 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:13.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:14.096 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.094+0000 7fc2881ff700 1 Processor -- start 2023-06-09T11:55:14.097 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.094+0000 7fc2881ff700 1 -- start start 2023-06-09T11:55:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.095+0000 7fc2881ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc28015b4a0 0x7fc28015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.095+0000 7fc2881ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2800c9f40 0x7fc2800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.098 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc2881ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28015a500 con 0x7fc280153880 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc2881ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2800c5f10 con 0x7fc2800c9f40 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc2881ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28015da00 con 0x7fc28015b4a0 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28679c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc2800c9f40 0x7fc2800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc285f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc280153880 0x7fc28015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42686/0 (socket says 172.21.15.7:42686) 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc285f9b700 1 -- 172.21.15.7:0/2700980631 learned_addr learned my addr 172.21.15.7:0/2700980631 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:14.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28579a700 1 --2- 172.21.15.7:0/2700980631 >> v2:172.21.15.117:3300/0 conn(0x7fc28015b4a0 0x7fc28015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28679c700 1 -- 172.21.15.7:0/2700980631 >> v2:172.21.15.117:3300/0 conn(0x7fc28015b4a0 msgr2=0x7fc28015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc284f99700 1 -- 172.21.15.7:0/2700980631 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809288417 0 0) 0x7fc27c001110 con 0x7fc280153880 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28679c700 1 --2- 172.21.15.7:0/2700980631 >> v2:172.21.15.117:3300/0 conn(0x7fc28015b4a0 0x7fc28015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28679c700 1 -- 172.21.15.7:0/2700980631 >> v1:172.21.15.7:6789/0 conn(0x7fc280153880 legacy=0x7fc28015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.096+0000 7fc28679c700 1 -- 172.21.15.7:0/2700980631 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2801596e0 con 0x7fc2800c9f40 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc28679c700 1 --2- 172.21.15.7:0/2700980631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2800c9f40 0x7fc2800c4e30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc2800cce30 tx=0x7fc27800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d251225250078b92 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:14.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc284f99700 1 -- 172.21.15.7:0/2700980631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800e070 con 0x7fc2800c9f40 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc284f99700 1 -- 172.21.15.7:0/2700980631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27800bbd0 con 0x7fc2800c9f40 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc284f99700 1 -- 172.21.15.7:0/2700980631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278012430 con 0x7fc2800c9f40 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 -- 172.21.15.7:0/2700980631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2800c9f40 msgr2=0x7fc2800c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 --2- 172.21.15.7:0/2700980631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2800c9f40 0x7fc2800c4e30 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc2800cce30 tx=0x7fc27800b330 comp rx=0 tx=0).stop 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 -- 172.21.15.7:0/2700980631 shutdown_connections 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 --2- 172.21.15.7:0/2700980631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2800c9f40 0x7fc2800c4e30 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 --2- 172.21.15.7:0/2700980631 >> v2:172.21.15.117:3300/0 conn(0x7fc28015b4a0 0x7fc28015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.097+0000 7fc2881ff700 1 -- 172.21.15.7:0/2700980631 >> 172.21.15.7:0/2700980631 conn(0x7fc2800737f0 msgr2=0x7fc2800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.098+0000 7fc2881ff700 1 -- 172.21.15.7:0/2700980631 shutdown_connections 2023-06-09T11:55:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.098+0000 7fc2881ff700 1 -- 172.21.15.7:0/2700980631 wait complete. 2023-06-09T11:55:14.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.099+0000 7fc2881ff700 1 Processor -- start 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.099+0000 7fc2881ff700 1 -- start start 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2800c9f40 0x7fc2801f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc280153880 0x7fc2801f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2801f3360 con 0x7fc28015b4a0 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28014f8c0 con 0x7fc280153880 2023-06-09T11:55:14.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc2881ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2800cd990 con 0x7fc2800c9f40 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.100+0000 7fc28679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34564/0 (socket says 172.21.15.7:34564) 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 -- 172.21.15.7:0/3570155464 learned_addr learned my addr 172.21.15.7:0/3570155464 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28579a700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc280153880 0x7fc2801f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc285f9b700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2800c9f40 0x7fc2801f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 -- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2800c9f40 msgr2=0x7fc2801f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2800c9f40 0x7fc2801f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 -- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc280153880 msgr2=0x7fc2801f2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc280153880 0x7fc2801f2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 -- 172.21.15.7:0/3570155464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc278009a70 con 0x7fc28015b4a0 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc28679c700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc28006b580 tx=0x7fc278004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278012430 con 0x7fc28015b4a0 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc278020070 con 0x7fc28015b4a0 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278004280 con 0x7fc28015b4a0 2023-06-09T11:55:14.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.101+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc280159c60 con 0x7fc28015b4a0 2023-06-09T11:55:14.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.102+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc280159980 con 0x7fc28015b4a0 2023-06-09T11:55:14.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.103+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc280159f40 con 0x7fc28015b4a0 2023-06-09T11:55:14.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.103+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2780043e0 con 0x7fc28015b4a0 2023-06-09T11:55:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.104+0000 7fc2767fc700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 0x7fc26406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.105+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fc278022070 con 0x7fc28015b4a0 2023-06-09T11:55:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.105+0000 7fc285f9b700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 0x7fc26406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:14.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.105+0000 7fc285f9b700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 0x7fc26406d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000f60 tx=0x7fc27c007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:14.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.109+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc278056340 con 0x7fc28015b4a0 2023-06-09T11:55:14.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.361+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc280074cf0 con 0x7fc28015b4a0 2023-06-09T11:55:14.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.362+0000 7fc2767fc700 1 -- 172.21.15.7:0/3570155464 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5460 (secure 0 0 0) 0x7fc27802c020 con 0x7fc28015b4a0 2023-06-09T11:55:14.364 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:14.364 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":205,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:12.933831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:resolve","state_seq":31,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:14.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 msgr2=0x7fc26406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 0x7fc26406d190 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fc27c000f60 tx=0x7fc27c007a40 comp rx=0 tx=0).stop 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 msgr2=0x7fc28014fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc28006b580 tx=0x7fc278004ad0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 shutdown_connections 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc28015b4a0 0x7fc28014fd50 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc26406ace0 0x7fc26406d190 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc2800c9f40 0x7fc2801f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 --2- 172.21.15.7:0/3570155464 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc280153880 0x7fc2801f2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:14.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 >> 172.21.15.7:0/3570155464 conn(0x7fc2800737f0 msgr2=0x7fc2800720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:14.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 shutdown_connections 2023-06-09T11:55:14.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:14.364+0000 7fc2881ff700 1 -- 172.21.15.7:0/3570155464 wait complete. 2023-06-09T11:55:14.368 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 205 2023-06-09T11:55:14.384 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:16.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.593+0000 7ff6f63ff700 1 Processor -- start 2023-06-09T11:55:16.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.593+0000 7ff6f63ff700 1 -- start start 2023-06-09T11:55:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.594+0000 7ff6f63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6f015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.594+0000 7ff6f63ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f015a500 con 0x7ff6f0153880 2023-06-09T11:55:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.594+0000 7ff6f63ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f015da00 con 0x7ff6f00c9f40 2023-06-09T11:55:16.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.594+0000 7ff6f63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00c5f10 con 0x7ff6f015b4a0 2023-06-09T11:55:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6effff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff6f0153880 0x7ff6f015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42692/0 (socket says 172.21.15.7:42692) 2023-06-09T11:55:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6effff700 1 -- 172.21.15.7:0/1790635616 learned_addr learned my addr 172.21.15.7:0/1790635616 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6ef7fe700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6f015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080208679 0 0) 0x7ff6e0001110 con 0x7ff6f0153880 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d80035b0 con 0x7ff6f0153880 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708800090 0 0) 0x7ff6e8001490 con 0x7ff6f00c9f40 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e0001110 con 0x7ff6f00c9f40 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 307768277 0 0) 0x7ff6d80035b0 con 0x7ff6f0153880 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 msgr2=0x7ff6f015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6f015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 >> v1:172.21.15.117:6789/0 conn(0x7ff6f00c9f40 legacy=0x7ff6f00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f01596e0 con 0x7ff6f0153880 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.595+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7ff6e0001800 con 0x7ff6f0153880 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 >> v1:172.21.15.7:6789/0 conn(0x7ff6f0153880 legacy=0x7ff6f015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d8003bc0 0x7ff6d8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d8004850 0x7ff6d8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6d8005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00c5f10 con 0x7ff6d8003bc0 2023-06-09T11:55:16.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800bba0 con 0x7ff6d8004850 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800bd60 con 0x7ff6f015b4a0 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d8003bc0 0x7ff6d8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6ef7fe700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d8004850 0x7ff6d8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6f499c700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6d8005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 -- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 msgr2=0x7ff6d8005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6d8005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 -- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d8004850 msgr2=0x7ff6d8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6d8004850 0x7ff6d8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6effff700 1 -- 172.21.15.7:0/1790635616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d80035b0 con 0x7ff6d8003bc0 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.596+0000 7ff6f499c700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6f015b4a0 0x7ff6d8005260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6effff700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d8003bc0 0x7ff6d8003fb0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c110 tx=0x7ff6d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8775ba65666c2855 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000ccb0 con 0x7ff6d8003bc0 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000ce10 con 0x7ff6d8003bc0 2023-06-09T11:55:16.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6eeffd700 1 -- 172.21.15.7:0/1790635616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0005ba0 con 0x7ff6d8003bc0 2023-06-09T11:55:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 -- 172.21.15.7:0/1790635616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d8003bc0 msgr2=0x7ff6d8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/1790635616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6d8003bc0 0x7ff6d8003fb0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000c110 tx=0x7ff6d8005880 comp rx=0 tx=0).stop 2023-06-09T11:55:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f499c700 1 -- 172.21.15.7:0/1790635616 reap_dead start 2023-06-09T11:55:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 -- 172.21.15.7:0/1790635616 shutdown_connections 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 -- 172.21.15.7:0/1790635616 >> 172.21.15.7:0/1790635616 conn(0x7ff6f00737f0 msgr2=0x7ff6f00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 -- 172.21.15.7:0/1790635616 shutdown_connections 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.597+0000 7ff6f63ff700 1 -- 172.21.15.7:0/1790635616 wait complete. 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.599+0000 7ff6f63ff700 1 Processor -- start 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.599+0000 7ff6f63ff700 1 -- start start 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6d8003bc0 0x7ff6f01e65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f00cf070 0x7ff6f014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00cca30 con 0x7ff6f0072720 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00cad80 con 0x7ff6f00cf070 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.600+0000 7ff6f63ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f00cdc70 con 0x7ff6d8003bc0 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34582/0 (socket says 172.21.15.7:34582) 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 -- 172.21.15.7:0/108171235 learned_addr learned my addr 172.21.15.7:0/108171235 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6effff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6d8003bc0 0x7ff6f01e65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6f499c700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f00cf070 0x7ff6f014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 -- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6d8003bc0 msgr2=0x7ff6f01e65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6d8003bc0 0x7ff6f01e65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 -- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f00cf070 msgr2=0x7ff6f014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f00cf070 0x7ff6f014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 -- 172.21.15.7:0/108171235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0159c60 con 0x7ff6f0072720 2023-06-09T11:55:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ef7fe700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4011aa0 tx=0x7ff6e4011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400f980 con 0x7ff6f0072720 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e400fae0 con 0x7ff6f0072720 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400fe90 con 0x7ff6f0072720 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.601+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f0159980 con 0x7ff6f0072720 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.602+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f0159f40 con 0x7ff6f0072720 2023-06-09T11:55:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.603+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f0074cf0 con 0x7ff6f0072720 2023-06-09T11:55:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.604+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff6e4014440 con 0x7ff6f0072720 2023-06-09T11:55:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.605+0000 7ff6ecff9700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.605+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7ff6e408a5c0 con 0x7ff6f0072720 2023-06-09T11:55:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.605+0000 7ff6effff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.606+0000 7ff6effff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 0x7ff6dc06d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0000c00 tx=0x7ff6e0007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:16.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.610+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff6e40588c0 con 0x7ff6f0072720 2023-06-09T11:55:16.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.862+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6f014fb20 con 0x7ff6f0072720 2023-06-09T11:55:16.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.863+0000 7ff6ecff9700 1 -- 172.21.15.7:0/108171235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 205 v205) v1 ==== 95+0+5460 (secure 0 0 0) 0x7ff6e4022090 con 0x7ff6f0072720 2023-06-09T11:55:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:16.865 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":205,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":205,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:12.933831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:resolve","state_seq":31,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:16.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 msgr2=0x7ff6dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 0x7ff6dc06d190 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0000c00 tx=0x7ff6e0007000 comp rx=0 tx=0).stop 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 msgr2=0x7ff6f00ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4011aa0 tx=0x7ff6e4011e60 comp rx=0 tx=0).stop 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 shutdown_connections 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff6f00cf070 0x7ff6f014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff6dc06ace0 0x7ff6dc06d190 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff6d8003bc0 0x7ff6f01e65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 --2- 172.21.15.7:0/108171235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6f0072720 0x7ff6f00ceb30 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 >> 172.21.15.7:0/108171235 conn(0x7ff6f00737f0 msgr2=0x7ff6f014e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 shutdown_connections 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:16.865+0000 7ff6f63ff700 1 -- 172.21.15.7:0/108171235 wait complete. 2023-06-09T11:55:16.869 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 205 2023-06-09T11:55:16.885 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:18.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.099+0000 7f4accdff700 1 Processor -- start 2023-06-09T11:55:19.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.100+0000 7f4accdff700 1 -- start start 2023-06-09T11:55:19.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4accdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4accdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ac80c9f40 0x7f4ac815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4accdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac815a500 con 0x7f4ac815b2d0 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4accdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80c5f10 con 0x7f4ac80c9f40 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4accdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac815da00 con 0x7f4ac8153880 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4ac5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ac80c9f40 0x7f4ac815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4ac5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ac80c9f40 0x7f4ac815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34584/0 (socket says 172.21.15.7:34584) 2023-06-09T11:55:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4ac5d9b700 1 -- 172.21.15.7:0/875949071 learned_addr learned my addr 172.21.15.7:0/875949071 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:19.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.101+0000 7f4ac659c700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac5d9b700 1 -- 172.21.15.7:0/875949071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 msgr2=0x7f4ac815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac5d9b700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac5d9b700 1 -- 172.21.15.7:0/875949071 >> v1:172.21.15.7:6789/0 conn(0x7f4ac815b2d0 legacy=0x7f4ac80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac5d9b700 1 -- 172.21.15.7:0/875949071 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac815b750 con 0x7f4ac80c9f40 2023-06-09T11:55:19.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac559a700 1 -- 172.21.15.7:0/875949071 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745537781 0 0) 0x7f4ac0001490 con 0x7f4ac815b2d0 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac5d9b700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac815ad90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098f0 tx=0x7f4abc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b4c30bb00feed3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac559a700 1 -- 172.21.15.7:0/875949071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc011830 con 0x7f4ac80c9f40 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac559a700 1 -- 172.21.15.7:0/875949071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc011990 con 0x7f4ac80c9f40 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.102+0000 7f4ac559a700 1 -- 172.21.15.7:0/875949071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc011d20 con 0x7f4ac80c9f40 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 -- 172.21.15.7:0/875949071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 msgr2=0x7f4ac815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac815ad90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098f0 tx=0x7f4abc009c00 comp rx=0 tx=0).stop 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 -- 172.21.15.7:0/875949071 shutdown_connections 2023-06-09T11:55:19.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 --2- 172.21.15.7:0/875949071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac815ad90 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 -- 172.21.15.7:0/875949071 >> 172.21.15.7:0/875949071 conn(0x7f4ac80737f0 msgr2=0x7f4ac80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 -- 172.21.15.7:0/875949071 shutdown_connections 2023-06-09T11:55:19.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.103+0000 7f4accdff700 1 -- 172.21.15.7:0/875949071 wait complete. 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.105+0000 7f4accdff700 1 Processor -- start 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.105+0000 7f4accdff700 1 -- start start 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac80b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 0x7f4ac806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac81518c0 con 0x7f4ac80c9f40 2023-06-09T11:55:19.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8152f40 con 0x7f4ac815b2d0 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4accdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80cbbb0 con 0x7f4ac8153880 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34600/0 (socket says 172.21.15.7:34600) 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 -- 172.21.15.7:0/2713967754 learned_addr learned my addr 172.21.15.7:0/2713967754 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac5d9b700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac80b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac6d9d700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 0x7f4ac806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 -- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 msgr2=0x7f4ac80b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac80b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 -- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 msgr2=0x7f4ac806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 0x7f4ac806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac659c700 1 -- 172.21.15.7:0/2713967754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009550 con 0x7f4ac80c9f40 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.106+0000 7f4ac6d9d700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 0x7f4ac806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4ac659c700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9e0 tx=0x7f4ab800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8009710 con 0x7f4ac80c9f40 2023-06-09T11:55:19.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab800c850 con 0x7f4ac80c9f40 2023-06-09T11:55:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac8159980 con 0x7f4ac80c9f40 2023-06-09T11:55:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800cbc0 con 0x7f4ac80c9f40 2023-06-09T11:55:19.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.107+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac81596e0 con 0x7f4ac80c9f40 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.108+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac8159c60 con 0x7f4ac80c9f40 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.109+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4ab800cd20 con 0x7f4ac80c9f40 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.110+0000 7f4ab6ffd700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 0x7f4aa806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.110+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f4ab8089ce0 con 0x7f4ac80c9f40 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.110+0000 7f4ac5d9b700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 0x7f4aa806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:19.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.111+0000 7f4ac5d9b700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 0x7f4aa806d1c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009130 tx=0x7f4abc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:19.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.114+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4ab8054520 con 0x7f4ac80c9f40 2023-06-09T11:55:19.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.370+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ac8151b30 con 0x7f4ac80c9f40 2023-06-09T11:55:19.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.371+0000 7f4ab6ffd700 1 -- 172.21.15.7:0/2713967754 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f4ab8017090 con 0x7f4ac80c9f40 2023-06-09T11:55:19.373 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:19.373 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":206,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:18.950445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:19.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 msgr2=0x7f4aa806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 0x7f4aa806d1c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009130 tx=0x7f4abc008030 comp rx=0 tx=0).stop 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 msgr2=0x7f4ac8151ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800e9e0 tx=0x7f4ab800ecf0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 shutdown_connections 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ac815b2d0 0x7f4ac806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ac80c9f40 0x7f4ac8151ce0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4aa806ad10 0x7f4aa806d1c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 --2- 172.21.15.7:0/2713967754 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ac8153880 0x7f4ac80b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 >> 172.21.15.7:0/2713967754 conn(0x7f4ac80737f0 msgr2=0x7f4ac814dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 shutdown_connections 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:19.373+0000 7f4accdff700 1 -- 172.21.15.7:0/2713967754 wait complete. 2023-06-09T11:55:19.377 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 206 2023-06-09T11:55:19.394 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 206, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 206, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:active", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:reconnect", "state_seq": 33 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 3, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:55:18.950445+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:55:21.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:21.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.603+0000 7f9c51dff700 1 Processor -- start 2023-06-09T11:55:21.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.603+0000 7f9c51dff700 1 -- start start 2023-06-09T11:55:21.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.604+0000 7f9c51dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c4c153880 0x7f9c4c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.604+0000 7f9c51dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 0x7f9c4c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.604+0000 7f9c51dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 0x7f9c4c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c51dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c5f10 con 0x7f9c4c153880 2023-06-09T11:55:21.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c51dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c15a500 con 0x7f9c4c15b2d0 2023-06-09T11:55:21.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c51dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c15da00 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c4c153880 0x7f9c4c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c4c153880 0x7f9c4c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34608/0 (socket says 172.21.15.7:34608) 2023-06-09T11:55:21.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/2642954373 learned_addr learned my addr 172.21.15.7:0/2642954373 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4bfff700 1 --2- 172.21.15.7:0/2642954373 >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 0x7f9c4c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4affd700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 0x7f9c4c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/2642954373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 msgr2=0x7f9c4c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 0x7f9c4c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/2642954373 >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 msgr2=0x7f9c4c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:21.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/2642954373 >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 0x7f9c4c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/2642954373 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c15b750 con 0x7f9c4c153880 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4affd700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 0x7f9c4c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.605+0000 7f9c4bfff700 1 --2- 172.21.15.7:0/2642954373 >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 0x7f9c4c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c153880 0x7f9c4c15d4c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c071db0 tx=0x7f9c3c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b55f26941e52139 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2642954373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00b870 con 0x7f9c4c153880 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2642954373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00b9d0 con 0x7f9c4c153880 2023-06-09T11:55:21.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c4a7fc700 1 -- 172.21.15.7:0/2642954373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bd60 con 0x7f9c4c153880 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 -- 172.21.15.7:0/2642954373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c153880 msgr2=0x7f9c4c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c153880 0x7f9c4c15d4c0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c071db0 tx=0x7f9c3c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 -- 172.21.15.7:0/2642954373 shutdown_connections 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 --2- 172.21.15.7:0/2642954373 >> v2:172.21.15.117:3300/0 conn(0x7f9c4c15b2d0 0x7f9c4c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c153880 0x7f9c4c15d4c0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 --2- 172.21.15.7:0/2642954373 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c0c9f40 0x7f9c4c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.606+0000 7f9c51dff700 1 -- 172.21.15.7:0/2642954373 >> 172.21.15.7:0/2642954373 conn(0x7f9c4c0737f0 msgr2=0x7f9c4c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:21.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.607+0000 7f9c51dff700 1 -- 172.21.15.7:0/2642954373 shutdown_connections 2023-06-09T11:55:21.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.607+0000 7f9c51dff700 1 -- 172.21.15.7:0/2642954373 wait complete. 2023-06-09T11:55:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.608+0000 7f9c51dff700 1 Processor -- start 2023-06-09T11:55:21.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.608+0000 7f9c51dff700 1 -- start start 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c153880 0x7f9c4c1e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c4c15b2d0 0x7f9c4c1f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0b3750 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c1e49a0 con 0x7f9c4c15b2d0 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.609+0000 7f9c51dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c1f0240 con 0x7f9c4c153880 2023-06-09T11:55:21.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34624/0 (socket says 172.21.15.7:34624) 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/575954633 learned_addr learned my addr 172.21.15.7:0/575954633 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4affd700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c153880 0x7f9c4c1e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4bfff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c4c15b2d0 0x7f9c4c1f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c153880 msgr2=0x7f9c4c1e61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c153880 0x7f9c4c1e61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c4c15b2d0 msgr2=0x7f9c4c1f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c4c15b2d0 0x7f9c4c1f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 -- 172.21.15.7:0/575954633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009550 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c4b7fe700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009fd0 tx=0x7f9c3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c004310 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.610+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c01f070 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.611+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c159980 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.611+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c0036a0 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.611+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c1596e0 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.612+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c159c60 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.612+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c3c021030 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.614+0000 7f9c33fff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 0x7f9c3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:21.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.614+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f9c3c08b210 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.614+0000 7f9c4affd700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 0x7f9c3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:21.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.615+0000 7f9c4affd700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 0x7f9c3806d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0b5290 tx=0x7f9c40008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:21.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.619+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9c3c055b70 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.870+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c4c14e4f0 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.870+0000 7f9c33fff700 1 -- 172.21.15.7:0/575954633 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 206 v206) v1 ==== 95+0+5462 (secure 0 0 0) 0x7f9c3c059190 con 0x7f9c4c0c9f40 2023-06-09T11:55:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:21.873 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":206,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":206,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:18.950445+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:reconnect","state_seq":33,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 msgr2=0x7f9c3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 0x7f9c3806d190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0b5290 tx=0x7f9c40008040 comp rx=0 tx=0).stop 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 msgr2=0x7f9c4c1efd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009fd0 tx=0x7f9c3c004750 comp rx=0 tx=0).stop 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 shutdown_connections 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c4c15b2d0 0x7f9c4c1f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c4c0c9f40 0x7f9c4c1efd00 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c3806ace0 0x7f9c3806d190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 --2- 172.21.15.7:0/575954633 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c4c153880 0x7f9c4c1e61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 >> 172.21.15.7:0/575954633 conn(0x7f9c4c0737f0 msgr2=0x7f9c4c15c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 shutdown_connections 2023-06-09T11:55:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:21.873+0000 7f9c51dff700 1 -- 172.21.15.7:0/575954633 wait complete. 2023-06-09T11:55:21.877 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 206 2023-06-09T11:55:21.894 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:23.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:23.942 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:23.946 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:24.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.108+0000 7f0fe53ff700 1 Processor -- start 2023-06-09T11:55:24.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.108+0000 7f0fe53ff700 1 -- start start 2023-06-09T11:55:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.109+0000 7f0fe53ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fe0153880 0x7f0fe015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.109+0000 7f0fe53ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0fe00c9f40 0x7f0fe015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.109+0000 7f0fe53ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe015a500 con 0x7f0fe015b2d0 2023-06-09T11:55:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.109+0000 7f0fe53ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe00c5f10 con 0x7f0fe0153880 2023-06-09T11:55:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.109+0000 7f0fe53ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe015da00 con 0x7f0fe00c9f40 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0fe0153880 0x7f0fe015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdf7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0fe015b2d0 0x7f0fe00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38486/0 (socket says 172.21.15.7:38486) 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdf7fe700 1 -- 172.21.15.7:0/2311328506 learned_addr learned my addr 172.21.15.7:0/2311328506 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fde7fc700 1 --2- 172.21.15.7:0/2311328506 >> v2:172.21.15.117:3300/0 conn(0x7f0fe00c9f40 0x7f0fe015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 -- 172.21.15.7:0/2311328506 >> v2:172.21.15.117:3300/0 conn(0x7f0fe00c9f40 msgr2=0x7f0fe015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:24.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 --2- 172.21.15.7:0/2311328506 >> v2:172.21.15.117:3300/0 conn(0x7f0fe00c9f40 0x7f0fe015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 -- 172.21.15.7:0/2311328506 >> v1:172.21.15.7:6789/0 conn(0x7f0fe015b2d0 legacy=0x7f0fe00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 -- 172.21.15.7:0/2311328506 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe01596e0 con 0x7f0fe0153880 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fddffb700 1 -- 172.21.15.7:0/2311328506 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295308395 0 0) 0x7f0fd8001490 con 0x7f0fe015b2d0 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.110+0000 7f0fdeffd700 1 --2- 172.21.15.7:0/2311328506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe015d4c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00cce30 tx=0x7f0fd0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e51d8fb25b3d924 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fddffb700 1 -- 172.21.15.7:0/2311328506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011830 con 0x7f0fe0153880 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fddffb700 1 -- 172.21.15.7:0/2311328506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd0011990 con 0x7f0fe0153880 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fddffb700 1 -- 172.21.15.7:0/2311328506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011d20 con 0x7f0fe0153880 2023-06-09T11:55:24.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 -- 172.21.15.7:0/2311328506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 msgr2=0x7f0fe015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/2311328506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe015d4c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00cce30 tx=0x7f0fd0009b20 comp rx=0 tx=0).stop 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 -- 172.21.15.7:0/2311328506 shutdown_connections 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/2311328506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe015d4c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/2311328506 >> v2:172.21.15.117:3300/0 conn(0x7f0fe00c9f40 0x7f0fe015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 -- 172.21.15.7:0/2311328506 >> 172.21.15.7:0/2311328506 conn(0x7f0fe00737f0 msgr2=0x7f0fe00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 -- 172.21.15.7:0/2311328506 shutdown_connections 2023-06-09T11:55:24.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.111+0000 7f0fe53ff700 1 -- 172.21.15.7:0/2311328506 wait complete. 2023-06-09T11:55:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.113+0000 7f0fe53ff700 1 Processor -- start 2023-06-09T11:55:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.113+0000 7f0fe53ff700 1 -- start start 2023-06-09T11:55:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe00c9f40 0x7f0fe01587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.118 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe015b2d0 0x7f0fe01ef7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe01593c0 con 0x7f0fe0153880 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe014bc40 con 0x7f0fe015b2d0 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fe53ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe01f24c0 con 0x7f0fe00c9f40 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fde7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fde7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45822/0 (socket says 172.21.15.7:45822) 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fde7fc700 1 -- 172.21.15.7:0/4216550096 learned_addr learned my addr 172.21.15.7:0/4216550096 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:24.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fdeffd700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe00c9f40 0x7f0fe01587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.114+0000 7f0fdf7fe700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe015b2d0 0x7f0fe01ef7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 -- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe00c9f40 msgr2=0x7f0fe01587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe00c9f40 0x7f0fe01587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 -- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe015b2d0 msgr2=0x7f0fe01ef7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe015b2d0 0x7f0fe01ef7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 -- 172.21.15.7:0/4216550096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00094a0 con 0x7f0fe0153880 2023-06-09T11:55:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fde7fc700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400e810 tx=0x7f0fd400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd400c070 con 0x7f0fe0153880 2023-06-09T11:55:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd4013610 con 0x7f0fe0153880 2023-06-09T11:55:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe0159c60 con 0x7f0fe0153880 2023-06-09T11:55:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd40139c0 con 0x7f0fe0153880 2023-06-09T11:55:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.115+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe0159980 con 0x7f0fe0153880 2023-06-09T11:55:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.116+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe0159f40 con 0x7f0fe0153880 2023-06-09T11:55:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.117+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fd4013b20 con 0x7f0fe0153880 2023-06-09T11:55:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.118+0000 7f0fc77fe700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 0x7f0fcc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:24.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.118+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f0fd4089b60 con 0x7f0fe0153880 2023-06-09T11:55:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.119+0000 7f0fdeffd700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 0x7f0fcc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:24.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.119+0000 7f0fdeffd700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 0x7f0fcc06d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00090b0 tx=0x7f0fd0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.123+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0fd4054290 con 0x7f0fe0153880 2023-06-09T11:55:24.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.373+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe0074cf0 con 0x7f0fe0153880 2023-06-09T11:55:24.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.373+0000 7f0fc77fe700 1 -- 172.21.15.7:0/4216550096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f0fd405d020 con 0x7f0fe0153880 2023-06-09T11:55:24.376 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:24.376 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:24.013353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 msgr2=0x7f0fcc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 0x7f0fcc06d190 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0fd00090b0 tx=0x7f0fd0008030 comp rx=0 tx=0).stop 2023-06-09T11:55:24.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 msgr2=0x7f0fe01ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400e810 tx=0x7f0fd400eb20 comp rx=0 tx=0).stop 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 shutdown_connections 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0fe015b2d0 0x7f0fe01ef7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0fcc06ace0 0x7f0fcc06d190 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0fe00c9f40 0x7f0fe01587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 --2- 172.21.15.7:0/4216550096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe0153880 0x7f0fe01ef2a0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:24.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 >> 172.21.15.7:0/4216550096 conn(0x7f0fe00737f0 msgr2=0x7f0fe014e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 shutdown_connections 2023-06-09T11:55:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:24.376+0000 7f0fe53ff700 1 -- 172.21.15.7:0/4216550096 wait complete. 2023-06-09T11:55:24.380 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 207 2023-06-09T11:55:24.396 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:26.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:26.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.604+0000 7fa3467ff700 1 Processor -- start 2023-06-09T11:55:26.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.604+0000 7fa3467ff700 1 -- start start 2023-06-09T11:55:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 0x7fa3401599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3400c0140 0x7fa340159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa3400c1f90 0x7fa3401501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340070110 con 0x7fa3400c0140 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa340071300 con 0x7fa3400c1f90 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa3467ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400cbd20 con 0x7fa3400c1bc0 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3400c0140 0x7fa340159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3400c0140 0x7fa340159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45828/0 (socket says 172.21.15.7:45828) 2023-06-09T11:55:26.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 -- 172.21.15.7:0/3356079034 learned_addr learned my addr 172.21.15.7:0/3356079034 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:26.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa344d9c700 1 --2- 172.21.15.7:0/3356079034 >> v2:172.21.15.117:3300/0 conn(0x7fa3400c1f90 0x7fa3401501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33ffff700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 0x7fa3401599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 -- 172.21.15.7:0/3356079034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 msgr2=0x7fa3401599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 0x7fa3401599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 -- 172.21.15.7:0/3356079034 >> v2:172.21.15.117:3300/0 conn(0x7fa3400c1f90 msgr2=0x7fa3401501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3356079034 >> v2:172.21.15.117:3300/0 conn(0x7fa3400c1f90 0x7fa3401501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33f7fe700 1 -- 172.21.15.7:0/3356079034 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3400c0bf0 con 0x7fa3400c0140 2023-06-09T11:55:26.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.606+0000 7fa33ffff700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 0x7fa3401599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa340159ef0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa3400cc250 tx=0x7fa334009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fe2e26fb1ab323 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa33effd700 1 -- 172.21.15.7:0/3356079034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400b860 con 0x7fa3400c0140 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa33effd700 1 -- 172.21.15.7:0/3356079034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33400b9c0 con 0x7fa3400c0140 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa33effd700 1 -- 172.21.15.7:0/3356079034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400bd50 con 0x7fa3400c0140 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa3467ff700 1 -- 172.21.15.7:0/3356079034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 msgr2=0x7fa340159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:26.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.607+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa340159ef0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa3400cc250 tx=0x7fa334009c00 comp rx=0 tx=0).stop 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 -- 172.21.15.7:0/3356079034 shutdown_connections 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3356079034 >> v2:172.21.15.117:3300/0 conn(0x7fa3400c1f90 0x7fa3401501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1bc0 0x7fa3401599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3356079034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa340159ef0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 -- 172.21.15.7:0/3356079034 >> 172.21.15.7:0/3356079034 conn(0x7fa34006a4d0 msgr2=0x7fa34006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 -- 172.21.15.7:0/3356079034 shutdown_connections 2023-06-09T11:55:26.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.608+0000 7fa3467ff700 1 -- 172.21.15.7:0/3356079034 wait complete. 2023-06-09T11:55:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.609+0000 7fa3467ff700 1 Processor -- start 2023-06-09T11:55:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.609+0000 7fa3467ff700 1 -- start start 2023-06-09T11:55:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3400c1bc0 0x7fa3400ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 0x7fa3401e1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3400ca8e0 con 0x7fa3400c0140 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3401e1690 con 0x7fa3400c1bc0 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.610+0000 7fa3467ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3401eb640 con 0x7fa3400c1f90 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45836/0 (socket says 172.21.15.7:45836) 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 -- 172.21.15.7:0/3239111047 learned_addr learned my addr 172.21.15.7:0/3239111047 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:26.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3400c1bc0 0x7fa3400ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa344d9c700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 0x7fa3401e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 -- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 msgr2=0x7fa3401e1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 0x7fa3401e1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 -- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3400c1bc0 msgr2=0x7fa3400ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3400c1bc0 0x7fa3400ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 -- 172.21.15.7:0/3239111047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334009550 con 0x7fa3400c0140 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa344d9c700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 0x7fa3401e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:26.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33ffff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa33000ed70 tx=0x7fa330009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330009ce0 con 0x7fa3400c0140 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330009e40 con 0x7fa3400c0140 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330010640 con 0x7fa3400c0140 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.611+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3400c0e90 con 0x7fa3400c0140 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.612+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3400702f0 con 0x7fa3400c0140 2023-06-09T11:55:26.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.613+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3400c1170 con 0x7fa3400c0140 2023-06-09T11:55:26.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.614+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3300107a0 con 0x7fa3400c0140 2023-06-09T11:55:26.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.614+0000 7fa33cff9700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 0x7fa32c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:26.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.615+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fa330014070 con 0x7fa3400c0140 2023-06-09T11:55:26.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.615+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 0x7fa32c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:26.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.615+0000 7fa33f7fe700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 0x7fa32c06d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa334000c00 tx=0x7fa334008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:26.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.619+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa33005a910 con 0x7fa3400c0140 2023-06-09T11:55:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.878+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa340074cf0 con 0x7fa3400c0140 2023-06-09T11:55:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.879+0000 7fa33cff9700 1 -- 172.21.15.7:0/3239111047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5459 (secure 0 0 0) 0x7fa33005a4a0 con 0x7fa3400c0140 2023-06-09T11:55:26.882 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:26.882 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:24.013353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 msgr2=0x7fa32c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 0x7fa32c06d190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa334000c00 tx=0x7fa334008030 comp rx=0 tx=0).stop 2023-06-09T11:55:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 msgr2=0x7fa3401df3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa33000ed70 tx=0x7fa330009530 comp rx=0 tx=0).stop 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 shutdown_connections 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa32c06ace0 0x7fa32c06d190 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3400c1f90 0x7fa3401e1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3400c0140 0x7fa3401df3c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 --2- 172.21.15.7:0/3239111047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3400c1bc0 0x7fa3400ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.881+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 >> 172.21.15.7:0/3239111047 conn(0x7fa34006a4d0 msgr2=0x7fa34014ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.882+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 shutdown_connections 2023-06-09T11:55:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:26.882+0000 7fa3467ff700 1 -- 172.21.15.7:0/3239111047 wait complete. 2023-06-09T11:55:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 207 2023-06-09T11:55:26.901 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:28.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:29.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.117+0000 7fdcf9bff700 1 Processor -- start 2023-06-09T11:55:29.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.118+0000 7fdcf9bff700 1 -- start start 2023-06-09T11:55:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf9bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcf40ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf9bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdcf4071fa0 0x7fdcf40723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf9bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf415a390 con 0x7fdcf4153b30 2023-06-09T11:55:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf9bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf40c5f40 con 0x7fdcf4071fa0 2023-06-09T11:55:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf9bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40ca720 con 0x7fdcf4153760 2023-06-09T11:55:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdcf4153b30 0x7fdcf40b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38492/0 (socket says 172.21.15.7:38492) 2023-06-09T11:55:29.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf2ffd700 1 -- 172.21.15.7:0/4026379183 learned_addr learned my addr 172.21.15.7:0/4026379183 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.119+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcf40ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf3fff700 1 --2- 172.21.15.7:0/4026379183 >> v2:172.21.15.117:3300/0 conn(0x7fdcf4071fa0 0x7fdcf40723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960960680 0 0) 0x7fdcf415a390 con 0x7fdcf4153b30 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcdc0035b0 con 0x7fdcf4153b30 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041538577 0 0) 0x7fdcdc0035b0 con 0x7fdcf4153b30 2023-06-09T11:55:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 msgr2=0x7fdcf40ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcf40ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 >> v2:172.21.15.117:3300/0 conn(0x7fdcf4071fa0 msgr2=0x7fdcf40723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 --2- 172.21.15.7:0/4026379183 >> v2:172.21.15.117:3300/0 conn(0x7fdcf4071fa0 0x7fdcf40723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf40c5a20 con 0x7fdcf4153b30 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fdce8001670 con 0x7fdcf4153b30 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 >> v1:172.21.15.7:6789/0 conn(0x7fdcf4153b30 legacy=0x7fdcf40b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcdc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcdc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc004d90 0x7fdcdc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40c5f40 con 0x7fdcdc003a00 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf40ca720 con 0x7fdcdc004d90 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc00ba10 con 0x7fdcf4153760 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf3fff700 1 --2- 172.21.15.7:0/4026379183 >> v2:172.21.15.117:3300/0 conn(0x7fdcf4071fa0 0x7fdcf40723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:55:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.120+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcdc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf2ffd700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcdc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf3fff700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc004d90 0x7fdcdc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 -- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 msgr2=0x7fdcdc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf4153760 0x7fdcdc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 -- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc004d90 msgr2=0x7fdcdc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc004d90 0x7fdcdc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 -- 172.21.15.7:0/4026379183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0035b0 con 0x7fdcdc003a00 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf3fff700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcdc004d90 0x7fdcdc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcdc003e10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009b90 tx=0x7fdcdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647c5b1d2e93dcb3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400e660 con 0x7fdcdc003a00 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce400e7c0 con 0x7fdcdc003a00 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.121+0000 7fdcf27fc700 1 -- 172.21.15.7:0/4026379183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400eb30 con 0x7fdcdc003a00 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 -- 172.21.15.7:0/4026379183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 msgr2=0x7fdcdc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/4026379183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcdc003e10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009b90 tx=0x7fdcdc005880 comp rx=0 tx=0).stop 2023-06-09T11:55:29.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf3fff700 1 -- 172.21.15.7:0/4026379183 reap_dead start 2023-06-09T11:55:29.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 -- 172.21.15.7:0/4026379183 shutdown_connections 2023-06-09T11:55:29.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 -- 172.21.15.7:0/4026379183 >> 172.21.15.7:0/4026379183 conn(0x7fdcf40737f0 msgr2=0x7fdcf4073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:29.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 -- 172.21.15.7:0/4026379183 shutdown_connections 2023-06-09T11:55:29.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.122+0000 7fdcf9bff700 1 -- 172.21.15.7:0/4026379183 wait complete. 2023-06-09T11:55:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.124+0000 7fdcf9bff700 1 Processor -- start 2023-06-09T11:55:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.124+0000 7fdcf9bff700 1 -- start start 2023-06-09T11:55:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf41f20a0 0x7fdcf4151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41517b0 0x7fdcf414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4154b00 con 0x7fdcdc003a00 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4151c20 con 0x7fdcf41517b0 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf9bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4159470 con 0x7fdcf41f20a0 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45854/0 (socket says 172.21.15.7:45854) 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 -- 172.21.15.7:0/1777404286 learned_addr learned my addr 172.21.15.7:0/1777404286 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 -- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf41f20a0 msgr2=0x7fdcf4151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf41f20a0 0x7fdcf4151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf3fff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41517b0 0x7fdcf414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 -- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41517b0 msgr2=0x7fdcf414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41517b0 0x7fdcf414ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.125+0000 7fdcf37fe700 1 -- 172.21.15.7:0/1777404286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4159a60 con 0x7fdcdc003a00 2023-06-09T11:55:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcf37fe700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc004760 tx=0x7fdce40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4021070 con 0x7fdcdc003a00 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce40045d0 con 0x7fdcdc003a00 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400e660 con 0x7fdcdc003a00 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf4159780 con 0x7fdcdc003a00 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.126+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf4159d40 con 0x7fdcdc003a00 2023-06-09T11:55:29.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.127+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf4074cf0 con 0x7fdcdc003a00 2023-06-09T11:55:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.128+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdce4025070 con 0x7fdcdc003a00 2023-06-09T11:55:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.129+0000 7fdcdbfff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 0x7fdce006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.129+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fdce408b110 con 0x7fdcdc003a00 2023-06-09T11:55:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.129+0000 7fdcf2ffd700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 0x7fdce006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.130+0000 7fdcf2ffd700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 0x7fdce006d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40cf440 tx=0x7fdce8000ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:29.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.134+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdce4059500 con 0x7fdcdc003a00 2023-06-09T11:55:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.383+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdcf40ca970 con 0x7fdcdc003a00 2023-06-09T11:55:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.384+0000 7fdcdbfff700 1 -- 172.21.15.7:0/1777404286 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 207 v207) v1 ==== 95+0+5459 (secure 0 0 0) 0x7fdce4059090 con 0x7fdcdc003a00 2023-06-09T11:55:29.388 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:29.388 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":207,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":207,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:24.013353+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:rejoin","state_seq":35,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 msgr2=0x7fdce006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 0x7fdce006d190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40cf440 tx=0x7fdce8000ed0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 msgr2=0x7fdcf41587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc004760 tx=0x7fdce40047c0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 shutdown_connections 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdcf41517b0 0x7fdcf414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdcdc003a00 0x7fdcf41587a0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdce006ace0 0x7fdce006d190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 --2- 172.21.15.7:0/1777404286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdcf41f20a0 0x7fdcf4151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 >> 172.21.15.7:0/1777404286 conn(0x7fdcf40737f0 msgr2=0x7fdcf414f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.386+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 shutdown_connections 2023-06-09T11:55:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:29.387+0000 7fdcf9bff700 1 -- 172.21.15.7:0/1777404286 wait complete. 2023-06-09T11:55:29.392 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 207 2023-06-09T11:55:29.408 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:55:31.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.616+0000 7f0c11dff700 1 Processor -- start 2023-06-09T11:55:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.616+0000 7f0c11dff700 1 -- start start 2023-06-09T11:55:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 0x7f0c0c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 0x7f0c0c0cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c15cec0 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c15d160 con 0x7f0c0c0c9090 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c11dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c15d440 con 0x7f0c0c0cad00 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c0bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c0bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45868/0 (socket says 172.21.15.7:45868) 2023-06-09T11:55:31.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.617+0000 7f0c0bfff700 1 -- 172.21.15.7:0/4208256555 learned_addr learned my addr 172.21.15.7:0/4208256555 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0b7fe700 1 --2- 172.21.15.7:0/4208256555 >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 0x7f0c0c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c03fff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 0x7f0c0c0cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 -- 172.21.15.7:0/4208256555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 msgr2=0x7f0c0c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 0x7f0c0c0cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 -- 172.21.15.7:0/4208256555 >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 msgr2=0x7f0c0c0c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 --2- 172.21.15.7:0/4208256555 >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 0x7f0c0c0c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 -- 172.21.15.7:0/4208256555 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c15d720 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0b7fe700 1 --2- 172.21.15.7:0/4208256555 >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 0x7f0c0c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c03fff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 0x7f0c0c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0bfff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c066ee0 tx=0x7f0c0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8220458a234fa5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0affd700 1 -- 172.21.15.7:0/4208256555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400e070 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0affd700 1 -- 172.21.15.7:0/4208256555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0400b9c0 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.618+0000 7f0c0affd700 1 -- 172.21.15.7:0/4208256555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400bd70 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 -- 172.21.15.7:0/4208256555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cfaf0 msgr2=0x7f0c0c1598b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c066ee0 tx=0x7f0c0400b330 comp rx=0 tx=0).stop 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 -- 172.21.15.7:0/4208256555 shutdown_connections 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 --2- 172.21.15.7:0/4208256555 >> v2:172.21.15.117:3300/0 conn(0x7f0c0c0c9090 0x7f0c0c0c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cfaf0 0x7f0c0c1598b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 --2- 172.21.15.7:0/4208256555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cad00 0x7f0c0c0cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 -- 172.21.15.7:0/4208256555 >> 172.21.15.7:0/4208256555 conn(0x7f0c0c070af0 msgr2=0x7f0c0c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 -- 172.21.15.7:0/4208256555 shutdown_connections 2023-06-09T11:55:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.619+0000 7f0c11dff700 1 -- 172.21.15.7:0/4208256555 wait complete. 2023-06-09T11:55:31.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.621+0000 7f0c11dff700 1 Processor -- start 2023-06-09T11:55:31.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.621+0000 7f0c11dff700 1 -- start start 2023-06-09T11:55:31.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c0c0c9090 0x7f0c0c1e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cfaf0 0x7f0c0c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c155f40 con 0x7f0c0c0cad00 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1590f0 con 0x7f0c0c0c9090 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c11dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c1e58a0 con 0x7f0c0c0cfaf0 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c03fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45882/0 (socket says 172.21.15.7:45882) 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c0bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cfaf0 0x7f0c0c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c03fff700 1 -- 172.21.15.7:0/1784340538 learned_addr learned my addr 172.21.15.7:0/1784340538 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c0b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c0c0c9090 0x7f0c0c1e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c03fff700 1 -- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cfaf0 msgr2=0x7f0c0c157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.622+0000 7f0c03fff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cfaf0 0x7f0c0c157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c03fff700 1 -- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c0c0c9090 msgr2=0x7f0c0c1e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c03fff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c0c0c9090 0x7f0c0c1e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c03fff700 1 -- 172.21.15.7:0/1784340538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04009b30 con 0x7f0c0c0cad00 2023-06-09T11:55:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c03fff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c820 tx=0x7f0bf400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:31.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4011800 con 0x7f0c0c0cad00 2023-06-09T11:55:31.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf4011960 con 0x7f0c0c0cad00 2023-06-09T11:55:31.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4011cf0 con 0x7f0c0c0cad00 2023-06-09T11:55:31.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0cb130 con 0x7f0c0c0cad00 2023-06-09T11:55:31.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.623+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0ca620 con 0x7f0c0c0cad00 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.624+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c0cb2c0 con 0x7f0c0c0cad00 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.625+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bf4018360 con 0x7f0c0c0cad00 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.626+0000 7f0c08ff9700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 0x7f0bec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.626+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f0bf4089c20 con 0x7f0c0c0cad00 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.626+0000 7f0c0b7fe700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 0x7f0bec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:31.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.627+0000 7f0c0b7fe700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 0x7f0bec06d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00ab70 tx=0x7f0bfc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:31.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.631+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0bf40543d0 con 0x7f0c0c0cad00 2023-06-09T11:55:31.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.881+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c0c1e5a50 con 0x7f0c0c0cad00 2023-06-09T11:55:31.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.882+0000 7f0c08ff9700 1 -- 172.21.15.7:0/1784340538 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+5458 (secure 0 0 0) 0x7f0c0c1e5a50 con 0x7f0c0c0cad00 2023-06-09T11:55:31.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:31.886 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":208,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:30.028030+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 msgr2=0x7f0bec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 0x7f0bec06d190 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00ab70 tx=0x7f0bfc008040 comp rx=0 tx=0).stop 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 msgr2=0x7f0c0c155a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c820 tx=0x7f0bf400cb30 comp rx=0 tx=0).stop 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 shutdown_connections 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0bec06ace0 0x7f0bec06d190 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0c0c0cfaf0 0x7f0c0c157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0c0c0c9090 0x7f0c0c1e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 --2- 172.21.15.7:0/1784340538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0c0cad00 0x7f0c0c155a00 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 >> 172.21.15.7:0/1784340538 conn(0x7f0c0c070af0 msgr2=0x7f0c0c0c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 shutdown_connections 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:31.884+0000 7f0c11dff700 1 -- 172.21.15.7:0/1784340538 wait complete. 2023-06-09T11:55:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 208 2023-06-09T11:55:31.906 INFO:tasks.mds_thrash.fs.[cephfs]:mds.f has gained rank=1, replacing gid=7405 2023-06-09T11:55:31.906 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.d 2023-06-09T11:55:36.907 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.d 2023-06-09T11:55:36.908 INFO:tasks.ceph.mds.d:Restarting daemon 2023-06-09T11:55:36.908 DEBUG:teuthology.orchestra.run.smithi117:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-06-09T11:55:36.912 INFO:tasks.ceph.mds.d:Started 2023-06-09T11:55:36.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:37.066 INFO:tasks.ceph.mds.d.smithi117.stdout:starting mds.d at 2023-06-09T11:55:37.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.136+0000 7fa305dff700 1 Processor -- start 2023-06-09T11:55:37.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.136+0000 7fa305dff700 1 -- start start 2023-06-09T11:55:37.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa305dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3001593d0 0x7fa3001597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa305dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa300072020 0x7fa300159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa305dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3000cc140 con 0x7fa3000c08d0 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa305dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3000714f0 con 0x7fa3001593d0 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa305dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3000c1830 con 0x7fa300072020 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa2ff7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3001593d0 0x7fa3001597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa2ff7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa3001593d0 0x7fa3001597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47274/0 (socket says 172.21.15.7:47274) 2023-06-09T11:55:37.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/2764756549 learned_addr learned my addr 172.21.15.7:0/2764756549 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:37.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.138+0000 7fa2fffff700 1 --2- 172.21.15.7:0/2764756549 >> v2:172.21.15.117:3300/0 conn(0x7fa300072020 0x7fa300159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2fe7fc700 1 -- 172.21.15.7:0/2764756549 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268451229 0 0) 0x7fa3000cc140 con 0x7fa3000c08d0 2023-06-09T11:55:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/2764756549 >> v2:172.21.15.117:3300/0 conn(0x7fa300072020 msgr2=0x7fa300159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2ff7fe700 1 --2- 172.21.15.7:0/2764756549 >> v2:172.21.15.117:3300/0 conn(0x7fa300072020 0x7fa300159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/2764756549 >> v1:172.21.15.117:6789/0 conn(0x7fa3000c08d0 legacy=0x7fa300071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/2764756549 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000c0560 con 0x7fa3001593d0 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2ff7fe700 1 --2- 172.21.15.7:0/2764756549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3001593d0 0x7fa3001597a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa3000c0dd0 tx=0x7fa2f000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d266b82561efd3a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2fe7fc700 1 -- 172.21.15.7:0/2764756549 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f000dc70 con 0x7fa3001593d0 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2fe7fc700 1 -- 172.21.15.7:0/2764756549 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f000ddd0 con 0x7fa3001593d0 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.139+0000 7fa2fe7fc700 1 -- 172.21.15.7:0/2764756549 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0010690 con 0x7fa3001593d0 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 -- 172.21.15.7:0/2764756549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3001593d0 msgr2=0x7fa3001597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 --2- 172.21.15.7:0/2764756549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3001593d0 0x7fa3001597a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fa3000c0dd0 tx=0x7fa2f000be70 comp rx=0 tx=0).stop 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 -- 172.21.15.7:0/2764756549 shutdown_connections 2023-06-09T11:55:37.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 --2- 172.21.15.7:0/2764756549 >> v2:172.21.15.117:3300/0 conn(0x7fa300072020 0x7fa300159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 --2- 172.21.15.7:0/2764756549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3001593d0 0x7fa3001597a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 -- 172.21.15.7:0/2764756549 >> 172.21.15.7:0/2764756549 conn(0x7fa30006a4d0 msgr2=0x7fa30006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 -- 172.21.15.7:0/2764756549 shutdown_connections 2023-06-09T11:55:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.140+0000 7fa305dff700 1 -- 172.21.15.7:0/2764756549 wait complete. 2023-06-09T11:55:37.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.142+0000 7fa305dff700 1 Processor -- start 2023-06-09T11:55:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.143+0000 7fa305dff700 1 -- start start 2023-06-09T11:55:37.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 0x7fa3001ee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3001593d0 0x7fa3001e18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300155780 con 0x7fa3000c08d0 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b03d0 con 0x7fa3001593d0 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa305dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001ef370 con 0x7fa300072020 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2feffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 0x7fa3001ee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2fffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3001593d0 0x7fa3001e18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2feffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47286/0 (socket says 172.21.15.7:47286) 2023-06-09T11:55:37.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2feffd700 1 -- 172.21.15.7:0/3288504505 learned_addr learned my addr 172.21.15.7:0/3288504505 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3001593d0 msgr2=0x7fa3001e18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2ff7fe700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3001593d0 0x7fa3001e18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.144+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 msgr2=0x7fa3001e1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2ff7fe700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2ff7fe700 1 -- 172.21.15.7:0/3288504505 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f000b7f0 con 0x7fa300072020 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2feffd700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:55:37.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2ff7fe700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 0x7fa3001ee3d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0018000 tx=0x7fa2f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0003ab0 con 0x7fa300072020 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0003c10 con 0x7fa300072020 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0010470 con 0x7fa300072020 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3000c1d60 con 0x7fa300072020 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.145+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000c1a80 con 0x7fa300072020 2023-06-09T11:55:37.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.146+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3000c2040 con 0x7fa300072020 2023-06-09T11:55:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.147+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2f0019400 con 0x7fa300072020 2023-06-09T11:55:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.148+0000 7fa2e7fff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 0x7fa2ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.148+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fa2f008b1f0 con 0x7fa300072020 2023-06-09T11:55:37.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.149+0000 7fa2feffd700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 0x7fa2ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:37.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.149+0000 7fa2feffd700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 0x7fa2ec06d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa3001e1f40 tx=0x7fa2f4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:37.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.154+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa2f0055a10 con 0x7fa300072020 2023-06-09T11:55:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.452+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa300074cf0 con 0x7fa300072020 2023-06-09T11:55:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.453+0000 7fa2e7fff700 1 -- 172.21.15.7:0/3288504505 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 208 v208) v1 ==== 95+0+5458 (secure 0 0 0) 0x7fa2f0059030 con 0x7fa300072020 2023-06-09T11:55:37.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:37.457 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":208,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:30.028030+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 msgr2=0x7fa2ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 0x7fa2ec06d190 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa3001e1f40 tx=0x7fa2f4009040 comp rx=0 tx=0).stop 2023-06-09T11:55:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 msgr2=0x7fa3001ee3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 0x7fa3001ee3d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0018000 tx=0x7fa2f0004750 comp rx=0 tx=0).stop 2023-06-09T11:55:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 shutdown_connections 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3001593d0 0x7fa3001e18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa2ec06ace0 0x7fa2ec06d190 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa300072020 0x7fa3001ee3d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 --2- 172.21.15.7:0/3288504505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3000c08d0 0x7fa3001e1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.455+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 >> 172.21.15.7:0/3288504505 conn(0x7fa30006a4d0 msgr2=0x7fa3001597a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.456+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 shutdown_connections 2023-06-09T11:55:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:37.456+0000 7fa305dff700 1 -- 172.21.15.7:0/3288504505 wait complete. 2023-06-09T11:55:37.461 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 208 2023-06-09T11:55:37.477 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.d is in active, standby or standby-replay 2023-06-09T11:55:39.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:55:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.688+0000 7f0aa43ff700 1 Processor -- start 2023-06-09T11:55:39.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.689+0000 7f0aa43ff700 1 -- start start 2023-06-09T11:55:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa43ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa43ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 0x7f0a9c072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa43ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c15a390 con 0x7f0a9c153760 2023-06-09T11:55:39.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa43ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0c5f40 con 0x7f0a9c071fa0 2023-06-09T11:55:39.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa43ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0ca720 con 0x7f0a9c153b30 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa199a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa299c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 0x7f0a9c072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa219b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f0a9c153760 0x7f0a9c0b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:33514/0 (socket says 172.21.15.7:33514) 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.691+0000 7f0aa219b700 1 -- 172.21.15.7:0/2253129069 learned_addr learned my addr 172.21.15.7:0/2253129069 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa199a700 1 -- 172.21.15.7:0/2253129069 >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 msgr2=0x7f0a9c072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:55:39.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa199a700 1 --2- 172.21.15.7:0/2253129069 >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 0x7f0a9c072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa199a700 1 -- 172.21.15.7:0/2253129069 >> v1:172.21.15.117:6789/0 conn(0x7f0a9c153760 legacy=0x7f0a9c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa199a700 1 -- 172.21.15.7:0/2253129069 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0c5a20 con 0x7f0a9c153b30 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa1199700 1 -- 172.21.15.7:0/2253129069 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783302317 0 0) 0x7f0a9c15a390 con 0x7f0a9c153760 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa299c700 1 --2- 172.21.15.7:0/2253129069 >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 0x7f0a9c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa199a700 1 --2- 172.21.15.7:0/2253129069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c0ca070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009840 tx=0x7f0a8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a85c1385c8b89c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa1199700 1 -- 172.21.15.7:0/2253129069 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011830 con 0x7f0a9c153b30 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa1199700 1 -- 172.21.15.7:0/2253129069 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c011990 con 0x7f0a9c153b30 2023-06-09T11:55:39.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.692+0000 7f0aa1199700 1 -- 172.21.15.7:0/2253129069 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011d20 con 0x7f0a9c153b30 2023-06-09T11:55:39.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 -- 172.21.15.7:0/2253129069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 msgr2=0x7f0a9c0ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/2253129069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c0ca070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c009840 tx=0x7f0a8c009b50 comp rx=0 tx=0).stop 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 -- 172.21.15.7:0/2253129069 shutdown_connections 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/2253129069 >> v2:172.21.15.117:3300/0 conn(0x7f0a9c071fa0 0x7f0a9c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/2253129069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c0ca070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 -- 172.21.15.7:0/2253129069 >> 172.21.15.7:0/2253129069 conn(0x7f0a9c0737f0 msgr2=0x7f0a9c073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 -- 172.21.15.7:0/2253129069 shutdown_connections 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.693+0000 7f0aa43ff700 1 -- 172.21.15.7:0/2253129069 wait complete. 2023-06-09T11:55:39.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.695+0000 7f0aa43ff700 1 Processor -- start 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.695+0000 7f0aa43ff700 1 -- start start 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a9c071fa0 0x7f0a9c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c1f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1f2790 con 0x7f0a9c153760 2023-06-09T11:55:39.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1f2490 con 0x7f0a9c071fa0 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa43ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0cc750 con 0x7f0a9c153b30 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.696+0000 7f0aa199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47290/0 (socket says 172.21.15.7:47290) 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 -- 172.21.15.7:0/1706520269 learned_addr learned my addr 172.21.15.7:0/1706520269 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa219b700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a9c071fa0 0x7f0a9c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa299c700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c1f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 -- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 msgr2=0x7f0a9c1f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c1f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 -- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a9c071fa0 msgr2=0x7f0a9c154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a9c071fa0 0x7f0a9c154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 -- 172.21.15.7:0/1706520269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a94009a70 con 0x7f0a9c153760 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa299c700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0aa199a700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0090b0 tx=0x7f0a8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0036a0 con 0x7f0a9c153760 2023-06-09T11:55:39.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.697+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c01f070 con 0x7f0a9c153760 2023-06-09T11:55:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.698+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c0094a0 con 0x7f0a9c153760 2023-06-09T11:55:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.698+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003a00 con 0x7f0a9c153760 2023-06-09T11:55:39.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.698+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c159a60 con 0x7f0a9c153760 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.699+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c159780 con 0x7f0a9c153760 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.700+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0a8c003b60 con 0x7f0a9c153760 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.701+0000 7f0a92ffd700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.701+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f0a8c021070 con 0x7f0a9c153760 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.701+0000 7f0aa219b700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:55:39.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.701+0000 7f0aa219b700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 0x7f0a8006d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0a98000f60 tx=0x7f0a98007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:55:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.705+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0a8c055ef0 con 0x7f0a9c153760 2023-06-09T11:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.986+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a9c159d40 con 0x7f0a9c153760 2023-06-09T11:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.986+0000 7f0a92ffd700 1 -- 172.21.15.7:0/1706520269 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 209 v209) v1 ==== 95+0+6219 (secure 0 0 0) 0x7f0a8c059510 con 0x7f0a9c153760 2023-06-09T11:55:39.990 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:55:39.990 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":209,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":208,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:55:30.028030+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 msgr2=0x7f0a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 0x7f0a8006d190 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0a98000f60 tx=0x7f0a98007be0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 msgr2=0x7f0a9c154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0090b0 tx=0x7f0a8c004750 comp rx=0 tx=0).stop 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 shutdown_connections 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0a8006ace0 0x7f0a8006d190 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0a9c153b30 0x7f0a9c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0a9c071fa0 0x7f0a9c154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 --2- 172.21.15.7:0/1706520269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0a9c153760 0x7f0a9c154460 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 >> 172.21.15.7:0/1706520269 conn(0x7f0a9c0737f0 msgr2=0x7f0a9c15cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 shutdown_connections 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:55:39.989+0000 7f0aa43ff700 1 -- 172.21.15.7:0/1706520269 wait complete. 2023-06-09T11:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 209 2023-06-09T11:55:40.012 INFO:tasks.mds_thrash.fs.[cephfs]:mds.d reported in up:standby state 2023-06-09T11:55:40.012 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 87 secs before thrashing 2023-06-09T11:55:46.664 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 19% (38/195) Updating files: 20% (39/195) Updating files: 21% (41/195) Updating files: 22% (43/195) Updating files: 23% (45/195) Updating files: 24% (47/195) Updating files: 25% (49/195) Updating files: 26% (51/195) Updating files: 27% (53/195) Updating files: 28% (55/195) Updating files: 29% (57/195) Updating files: 30% (59/195) Updating files: 31% (61/195) Updating files: 32% (63/195) Updating files: 33% (65/195) Updating files: 34% (67/195) Updating files: 35% (69/195) Updating files: 36% (71/195) Updating files: 37% (73/195) Updating files: 38% (75/195) Updating files: 39% (77/195) Updating files: 40% (78/195) Updating files: 41% (80/195) Updating files: 42% (82/195) Updating files: 43% (84/195) Updating files: 44% (86/195) Updating files: 45% (88/195) Updating files: 46% (90/195) Updating files: 47% (92/195) Updating files: 48% (94/195) Updating files: 49% (96/195) Updating files: 50% (98/195) Updating files: 51% (100/195) Updating files: 52% (102/195) Updating files: 53% (104/195) Updating files: 54% (106/195) Updating files: 55% (108/195) Updating files: 56% (110/195) Updating files: 57% (112/195) Updating files: 58% (114/195) Updating files: 59% (116/195) Updating files: 60% (117/195) Updating files: 61% (119/195) Updating files: 62% (121/195) Updating files: 63% (123/195) Updating files: 64% (125/195) Updating files: 65% (127/195) Updating files: 66% (129/195) Updating files: 67% (131/195) Updating files: 68% (133/195) Updating files: 69% (135/195) Updating files: 70% (137/195) Updating files: 71% (139/195) Updating files: 71% (140/195) Updating files: 72% (141/195) Updating files: 73% (143/195) Updating files: 74% (145/195) Updating files: 75% (147/195) Updating files: 76% (149/195) Updating files: 77% (151/195) Updating files: 78% (153/195) Updating files: 78% (154/195) Updating files: 79% (155/195) Updating files: 80% (156/195) Updating files: 81% (158/195) Updating files: 82% (160/195) Updating files: 83% (162/195) Updating files: 84% (164/195) Updating files: 85% (166/195) Updating files: 86% (168/195) Updating files: 87% (170/195) Updating files: 88% (172/195) Updating files: 89% (174/195) Updating files: 90% (176/195) Updating files: 91% (178/195) Updating files: 92% (180/195) Updating files: 93% (182/195) Updating files: 94% (184/195) Updating files: 95% (186/195) Updating files: 96% (188/195) Updating files: 97% (190/195) Updating files: 98% (192/195) Updating files: 99% (194/195) Updating files: 100% (195/195) Updating files: 100% (195/195), done. 2023-06-09T11:55:46.919 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at c08d08baa6a v0.32 2023-06-09T11:55:46.924 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.32 2023-06-09T11:55:46.973 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:55:46.974 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 33 -eq 48 ']' 2023-06-09T11:55:46.974 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.33 2023-06-09T11:55:46.974 INFO:tasks.workunit.client.0.smithi007.stdout:v0.33 2023-06-09T11:55:46.974 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.33 2023-06-09T11:55:46.974 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.33 2023-06-09T11:55:54.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:55:54.108 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:21.191 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 14% (23/160) Updating files: 15% (24/160) Updating files: 16% (26/160) Updating files: 17% (28/160) Updating files: 18% (29/160) Updating files: 19% (31/160) Updating files: 20% (32/160) Updating files: 21% (34/160) Updating files: 22% (36/160) Updating files: 23% (37/160) Updating files: 24% (39/160) Updating files: 25% (40/160) Updating files: 26% (42/160) Updating files: 27% (44/160) Updating files: 28% (45/160) Updating files: 29% (47/160) Updating files: 30% (48/160) Updating files: 31% (50/160) Updating files: 32% (52/160) Updating files: 33% (53/160) Updating files: 33% (54/160) Updating files: 34% (55/160) Updating files: 35% (56/160) Updating files: 36% (58/160) Updating files: 37% (60/160) Updating files: 38% (61/160) Updating files: 39% (63/160) Updating files: 40% (64/160) Updating files: 41% (66/160) Updating files: 42% (68/160) Updating files: 43% (69/160) Updating files: 44% (71/160) Updating files: 45% (72/160) Updating files: 46% (74/160) Updating files: 47% (76/160) Updating files: 48% (77/160) Updating files: 49% (79/160) Updating files: 50% (80/160) Updating files: 50% (81/160) Updating files: 51% (82/160) Updating files: 52% (84/160) Updating files: 53% (85/160) Updating files: 54% (87/160) Updating files: 55% (88/160) Updating files: 56% (90/160) Updating files: 57% (92/160) Updating files: 58% (93/160) Updating files: 59% (95/160) Updating files: 60% (96/160) Updating files: 60% (97/160) Updating files: 61% (98/160) Updating files: 62% (100/160) Updating files: 63% (101/160) Updating files: 64% (103/160) Updating files: 65% (104/160) Updating files: 66% (106/160) Updating files: 67% (108/160) Updating files: 68% (109/160) Updating files: 69% (111/160) Updating files: 70% (112/160) Updating files: 71% (114/160) Updating files: 72% (116/160) Updating files: 73% (117/160) Updating files: 74% (119/160) Updating files: 75% (120/160) Updating files: 76% (122/160) Updating files: 77% (124/160) Updating files: 78% (125/160) Updating files: 79% (127/160) Updating files: 80% (128/160) Updating files: 81% (130/160) Updating files: 82% (132/160) Updating files: 83% (133/160) Updating files: 84% (135/160) Updating files: 85% (136/160) Updating files: 85% (137/160) Updating files: 86% (138/160) Updating files: 87% (140/160) Updating files: 88% (141/160) Updating files: 89% (143/160) Updating files: 90% (144/160) Updating files: 91% (146/160) Updating files: 92% (148/160) Updating files: 93% (149/160) Updating files: 94% (151/160) Updating files: 95% (152/160) Updating files: 96% (154/160) Updating files: 97% (156/160) Updating files: 98% (157/160) Updating files: 99% (159/160) Updating files: 100% (160/160) Updating files: 100% (160/160), done. 2023-06-09T11:56:21.925 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 340b4348bf6 v0.33 2023-06-09T11:56:21.929 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.33 2023-06-09T11:56:21.968 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:56:21.968 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 34 -eq 48 ']' 2023-06-09T11:56:21.968 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.34 2023-06-09T11:56:21.969 INFO:tasks.workunit.client.0.smithi007.stdout:v0.34 2023-06-09T11:56:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.34 2023-06-09T11:56:21.969 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.34 2023-06-09T11:56:24.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:24.270 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:35.990 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 14% (19/129) Updating files: 15% (20/129) Updating files: 16% (21/129) Updating files: 17% (22/129) Updating files: 18% (24/129) Updating files: 19% (25/129) Updating files: 20% (26/129) Updating files: 21% (28/129) Updating files: 22% (29/129) Updating files: 23% (30/129) Updating files: 24% (31/129) Updating files: 25% (33/129) Updating files: 26% (34/129) Updating files: 27% (35/129) Updating files: 28% (37/129) Updating files: 29% (38/129) Updating files: 30% (39/129) Updating files: 31% (40/129) Updating files: 32% (42/129) Updating files: 33% (43/129) Updating files: 34% (44/129) Updating files: 35% (46/129) Updating files: 36% (47/129) Updating files: 37% (48/129) Updating files: 38% (50/129) Updating files: 39% (51/129) Updating files: 40% (52/129) Updating files: 41% (53/129) Updating files: 42% (55/129) Updating files: 43% (56/129) Updating files: 44% (57/129) Updating files: 45% (59/129) Updating files: 46% (60/129) Updating files: 47% (61/129) Updating files: 48% (62/129) Updating files: 49% (64/129) Updating files: 50% (65/129) Updating files: 51% (66/129) Updating files: 52% (68/129) Updating files: 53% (69/129) Updating files: 54% (70/129) Updating files: 55% (71/129) Updating files: 56% (73/129) Updating files: 57% (74/129) Updating files: 58% (75/129) Updating files: 59% (77/129) Updating files: 60% (78/129) Updating files: 61% (79/129) Updating files: 62% (80/129) Updating files: 63% (82/129) Updating files: 64% (83/129) Updating files: 65% (84/129) Updating files: 66% (86/129) Updating files: 67% (87/129) Updating files: 68% (88/129) Updating files: 69% (90/129) Updating files: 70% (91/129) Updating files: 71% (92/129) Updating files: 72% (93/129) Updating files: 73% (95/129) Updating files: 74% (96/129) Updating files: 75% (97/129) Updating files: 76% (99/129) Updating files: 77% (100/129) Updating files: 78% (101/129) Updating files: 79% (102/129) Updating files: 80% (104/129) Updating files: 81% (105/129) Updating files: 82% (106/129) Updating files: 83% (108/129) Updating files: 84% (109/129) Updating files: 85% (110/129) Updating files: 86% (111/129) Updating files: 87% (113/129) Updating files: 88% (114/129) Updating files: 89% (115/129) Updating files: 90% (117/129) Updating files: 91% (118/129) Updating files: 92% (119/129) Updating files: 93% (120/129) Updating files: 94% (122/129) Updating files: 95% (123/129) Updating files: 96% (124/129) Updating files: 97% (126/129) Updating files: 98% (127/129) Updating files: 99% (128/129) Updating files: 100% (129/129) Updating files: 100% (129/129), done. 2023-06-09T11:56:37.135 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 2f039eeeb74 v0.34 2023-06-09T11:56:37.139 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.34 2023-06-09T11:56:37.184 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:56:37.185 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 35 -eq 48 ']' 2023-06-09T11:56:37.185 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.35 2023-06-09T11:56:37.185 INFO:tasks.workunit.client.0.smithi007.stdout:v0.35 2023-06-09T11:56:37.185 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.35 2023-06-09T11:56:37.186 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.35 2023-06-09T11:56:54.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:54.456 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:56:58.382 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 2% (9/334) Updating files: 3% (11/334) Updating files: 4% (14/334) Updating files: 5% (17/334) Updating files: 6% (21/334) Updating files: 7% (24/334) Updating files: 8% (27/334) Updating files: 9% (31/334) Updating files: 10% (34/334) Updating files: 11% (37/334) Updating files: 12% (41/334) Updating files: 13% (44/334) Updating files: 14% (47/334) Updating files: 15% (51/334) Updating files: 16% (54/334) Updating files: 17% (57/334) Updating files: 18% (61/334) Updating files: 19% (64/334) Updating files: 20% (67/334) Updating files: 21% (71/334) Updating files: 22% (74/334) Updating files: 23% (77/334) Updating files: 24% (81/334) Updating files: 25% (84/334) Updating files: 26% (87/334) Updating files: 27% (91/334) Updating files: 28% (94/334) Updating files: 29% (97/334) Updating files: 30% (101/334) Updating files: 31% (104/334) Updating files: 32% (107/334) Updating files: 33% (111/334) Updating files: 34% (114/334) Updating files: 35% (117/334) Updating files: 36% (121/334) Updating files: 37% (124/334) Updating files: 37% (126/334) Updating files: 38% (127/334) Updating files: 39% (131/334) Updating files: 40% (134/334) Updating files: 41% (137/334) Updating files: 42% (141/334) Updating files: 43% (144/334) Updating files: 44% (147/334) Updating files: 45% (151/334) Updating files: 46% (154/334) Updating files: 47% (157/334) Updating files: 48% (161/334) Updating files: 49% (164/334) Updating files: 50% (167/334) Updating files: 51% (171/334) Updating files: 52% (174/334) Updating files: 52% (176/334) Updating files: 53% (178/334) Updating files: 54% (181/334) Updating files: 55% (184/334) Updating files: 56% (188/334) Updating files: 56% (190/334) Updating files: 57% (191/334) Updating files: 58% (194/334) Updating files: 59% (198/334) Updating files: 60% (201/334) Updating files: 61% (204/334) Updating files: 62% (208/334) Updating files: 62% (209/334) Updating files: 63% (211/334) Updating files: 64% (214/334) Updating files: 65% (218/334) Updating files: 66% (221/334) Updating files: 67% (224/334) Updating files: 68% (228/334) Updating files: 69% (231/334) Updating files: 70% (234/334) Updating files: 71% (238/334) Updating files: 72% (241/334) Updating files: 73% (244/334) Updating files: 74% (248/334) Updating files: 75% (251/334) Updating files: 76% (254/334) Updating files: 77% (258/334) Updating files: 78% (261/334) Updating files: 79% (264/334) Updating files: 80% (268/334) Updating files: 81% (271/334) Updating files: 82% (274/334) Updating files: 83% (278/334) Updating files: 84% (281/334) Updating files: 85% (284/334) Updating files: 86% (288/334) Updating files: 87% (291/334) Updating files: 88% (294/334) Updating files: 89% (298/334) Updating files: 90% (301/334) Updating files: 91% (304/334) Updating files: 92% (308/334) Updating files: 93% (311/334) Updating files: 93% (312/334) Updating files: 94% (314/334) Updating files: 95% (318/334) Updating files: 96% (321/334) Updating files: 97% (324/334) Updating files: 98% (328/334) Updating files: 99% (331/334) Updating files: 100% (334/334) Updating files: 100% (334/334), done. 2023-06-09T11:57:00.341 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 0afda378124 v0.35 2023-06-09T11:57:00.345 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.35 2023-06-09T11:57:00.405 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:57:00.405 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 36 -eq 48 ']' 2023-06-09T11:57:00.406 INFO:tasks.workunit.client.0.smithi007.stdout:v0.36 2023-06-09T11:57:00.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.36 2023-06-09T11:57:00.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.36 2023-06-09T11:57:00.406 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.36 2023-06-09T11:57:07.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.224+0000 7f5ca87ff700 1 Processor -- start 2023-06-09T11:57:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.224+0000 7f5ca87ff700 1 -- start start 2023-06-09T11:57:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca87ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca87ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5ca00cb9a0 0x7f5ca0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca87ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca0159b20 con 0x7f5ca015cd40 2023-06-09T11:57:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca87ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca00cac80 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca87ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca01597d0 con 0x7f5ca00cb9a0 2023-06-09T11:57:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca659b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca659b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34482/0 (socket says 172.21.15.7:34482) 2023-06-09T11:57:07.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca659b700 1 -- 172.21.15.7:0/3804088969 learned_addr learned my addr 172.21.15.7:0/3804088969 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.225+0000 7f5ca5d9a700 1 --2- 172.21.15.7:0/3804088969 >> v2:172.21.15.117:3300/0 conn(0x7f5ca00cb9a0 0x7f5ca0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca659b700 1 -- 172.21.15.7:0/3804088969 >> v2:172.21.15.117:3300/0 conn(0x7f5ca00cb9a0 msgr2=0x7f5ca0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca659b700 1 --2- 172.21.15.7:0/3804088969 >> v2:172.21.15.117:3300/0 conn(0x7f5ca00cb9a0 0x7f5ca0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca659b700 1 -- 172.21.15.7:0/3804088969 >> v1:172.21.15.117:6789/0 conn(0x7f5ca015cd40 legacy=0x7f5ca00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca659b700 1 -- 172.21.15.7:0/3804088969 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca0071ab0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca5599700 1 -- 172.21.15.7:0/3804088969 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151082401 0 0) 0x7f5ca0159b20 con 0x7f5ca015cd40 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca659b700 1 --2- 172.21.15.7:0/3804088969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00c9c60 tx=0x7f5c9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e155086fa6a365 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca5599700 1 -- 172.21.15.7:0/3804088969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c011830 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca5599700 1 -- 172.21.15.7:0/3804088969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c011990 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.226+0000 7f5ca5599700 1 -- 172.21.15.7:0/3804088969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c011d20 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 -- 172.21.15.7:0/3804088969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 msgr2=0x7f5ca00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/3804088969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5ca00c9c60 tx=0x7f5c9c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 -- 172.21.15.7:0/3804088969 shutdown_connections 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/3804088969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00b52b0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/3804088969 >> v2:172.21.15.117:3300/0 conn(0x7f5ca00cb9a0 0x7f5ca0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 -- 172.21.15.7:0/3804088969 >> 172.21.15.7:0/3804088969 conn(0x7f5ca0070af0 msgr2=0x7f5ca0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 -- 172.21.15.7:0/3804088969 shutdown_connections 2023-06-09T11:57:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.227+0000 7f5ca87ff700 1 -- 172.21.15.7:0/3804088969 wait complete. 2023-06-09T11:57:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.228+0000 7f5ca87ff700 1 Processor -- start 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.229+0000 7f5ca87ff700 1 -- start start 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.229+0000 7f5ca87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.229+0000 7f5ca87ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ca00cb9a0 0x7f5ca00c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ca015cd40 0x7f5ca01e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c3d40 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca87ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca0155860 con 0x7f5ca00cb9a0 2023-06-09T11:57:07.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c3190 con 0x7f5ca015cd40 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34498/0 (socket says 172.21.15.7:34498) 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 -- 172.21.15.7:0/2126291449 learned_addr learned my addr 172.21.15.7:0/2126291449 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca5d9a700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ca00cb9a0 0x7f5ca00c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca6d9c700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ca015cd40 0x7f5ca01e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 -- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ca015cd40 msgr2=0x7f5ca01e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ca015cd40 0x7f5ca01e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 -- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ca00cb9a0 msgr2=0x7f5ca00c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ca00cb9a0 0x7f5ca00c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 -- 172.21.15.7:0/2126291449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009550 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.230+0000 7f5ca659b700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000c00 tx=0x7f5c9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.231+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0045d0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.231+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c01f070 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.231+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca015d4f0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.231+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c0038a0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.231+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca015d210 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.232+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca015d7d0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.233+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c9c003a00 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.234+0000 7f5c96ffd700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 0x7f5c8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.234+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f5c9c021070 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.234+0000 7f5ca5d9a700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 0x7f5c8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.235+0000 7f5ca5d9a700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 0x7f5c8406d190 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015bfb0 tx=0x7f5c90009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.238+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5c9c055dc0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.494+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ca0074cf0 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.495+0000 7f5c96ffd700 1 -- 172.21.15.7:0/2126291449 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 214 v214) v1 ==== 95+0+6226 (secure 0 0 0) 0x7f5c9c02b090 con 0x7f5ca00cb5d0 2023-06-09T11:57:07.498 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:07.498 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":214,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":214,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:56:46.936490+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:active","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 msgr2=0x7f5c8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 0x7f5c8406d190 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f5ca015bfb0 tx=0x7f5c90009210 comp rx=0 tx=0).stop 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 msgr2=0x7f5ca00c4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000c00 tx=0x7f5c9c004750 comp rx=0 tx=0).stop 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 shutdown_connections 2023-06-09T11:57:07.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5c8406ace0 0x7f5c8406d190 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5ca015cd40 0x7f5ca01e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ca00cb5d0 0x7f5ca00c4160 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 --2- 172.21.15.7:0/2126291449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5ca00cb9a0 0x7f5ca00c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 >> 172.21.15.7:0/2126291449 conn(0x7f5ca0070af0 msgr2=0x7f5ca00c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.497+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 shutdown_connections 2023-06-09T11:57:07.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.498+0000 7f5ca87ff700 1 -- 172.21.15.7:0/2126291449 wait complete. 2023-06-09T11:57:07.503 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 214 2023-06-09T11:57:07.518 INFO:tasks.mds_thrash.fs.[cephfs]:thrashing max_mds: 3 -> 2 2023-06-09T11:57:07.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-06-09T11:57:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.721+0000 7fef0e3ff700 1 Processor -- start 2023-06-09T11:57:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.721+0000 7fef0e3ff700 1 -- start start 2023-06-09T11:57:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef0e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 0x7fef080cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef0e3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef080b40e0 con 0x7fef08153850 2023-06-09T11:57:07.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef0e3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0806b110 con 0x7fef0806ac20 2023-06-09T11:57:07.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef0e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef080b4330 con 0x7fef080c6590 2023-06-09T11:57:07.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef07fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fef08153850 0x7fef080ca890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52236/0 (socket says 172.21.15.7:52236) 2023-06-09T11:57:07.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7fef07fff700 1 -- 172.21.15.7:0/3325841293 learned_addr learned my addr 172.21.15.7:0/3325841293 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:07.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.722+0000 7feefffff700 1 --2- 172.21.15.7:0/3325841293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 0x7fef080cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773147543 0 0) 0x7fef080b40e0 con 0x7fef08153850 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feef40035b0 con 0x7fef08153850 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061956520 0 0) 0x7fef0806b110 con 0x7fef0806ac20 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef080b40e0 con 0x7fef0806ac20 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7feefffff700 1 -- 172.21.15.7:0/3325841293 >> v1:172.21.15.117:6789/0 conn(0x7fef0806ac20 legacy=0x7fef080cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921885077 0 0) 0x7feef40035b0 con 0x7fef08153850 2023-06-09T11:57:07.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7feefffff700 1 -- 172.21.15.7:0/3325841293 >> v1:172.21.15.7:6789/0 conn(0x7fef08153850 legacy=0x7fef080ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7feefffff700 1 -- 172.21.15.7:0/3325841293 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef080b4610 con 0x7fef080c6590 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.723+0000 7feefffff700 1 --2- 172.21.15.7:0/3325841293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 0x7fef080cb4b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feef0009870 tx=0x7feef0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df7f17282399e0f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8001670 con 0x7fef080c6590 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0004030 con 0x7fef080c6590 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef077fe700 1 -- 172.21.15.7:0/3325841293 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef00043c0 con 0x7fef080c6590 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3325841293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 msgr2=0x7fef080cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3325841293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 0x7fef080cb4b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feef0009870 tx=0x7feef0009b80 comp rx=0 tx=0).stop 2023-06-09T11:57:07.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3325841293 shutdown_connections 2023-06-09T11:57:07.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3325841293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef080c6590 0x7fef080cb4b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3325841293 >> 172.21.15.7:0/3325841293 conn(0x7fef080737f0 msgr2=0x7fef08072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:07.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3325841293 shutdown_connections 2023-06-09T11:57:07.730 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.724+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3325841293 wait complete. 2023-06-09T11:57:07.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.727+0000 7fef0e3ff700 1 Processor -- start 2023-06-09T11:57:07.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.727+0000 7fef0e3ff700 1 -- start start 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0806ac20 0x7fef081f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef08153850 0x7fef0814fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef081ef430 con 0x7fef080c6590 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef081ef960 con 0x7fef08153850 2023-06-09T11:57:07.732 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7fef0e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef08159fa0 con 0x7fef0806ac20 2023-06-09T11:57:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7feefffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7feefffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34512/0 (socket says 172.21.15.7:34512) 2023-06-09T11:57:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.728+0000 7feefffff700 1 -- 172.21.15.7:0/3221214607 learned_addr learned my addr 172.21.15.7:0/3221214607 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef07fff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0806ac20 0x7fef081f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef0c99c700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef08153850 0x7fef0814fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 -- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0806ac20 msgr2=0x7fef081f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0806ac20 0x7fef081f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 -- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef08153850 msgr2=0x7fef0814fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef08153850 0x7fef0814fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 -- 172.21.15.7:0/3221214607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef00094a0 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7feefffff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7feef00090b0 tx=0x7feef0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003ea0 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef001f070 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011870 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.729+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef08153c20 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.730+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef080b48f0 con 0x7fef080c6590 2023-06-09T11:57:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.731+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef080cc100 con 0x7fef080c6590 2023-06-09T11:57:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.732+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feef00119d0 con 0x7fef080c6590 2023-06-09T11:57:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.733+0000 7fef057fa700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 0x7feee806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.733+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7feef0021070 con 0x7fef080c6590 2023-06-09T11:57:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.733+0000 7fef07fff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 0x7feee806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.734+0000 7fef07fff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 0x7feee806d190 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feef8001380 tx=0x7feef8007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:07.737+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feef0055c80 con 0x7fef080c6590 2023-06-09T11:57:08.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.008+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fef08074cf0 con 0x7fef080c6590 2023-06-09T11:57:08.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.949+0000 7fef057fa700 1 -- 172.21.15.7:0/3221214607 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v215) v1 ==== 105+0+0 (secure 0 0 0) 0x7feef002b0b0 con 0x7fef080c6590 2023-06-09T11:57:08.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.952+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 msgr2=0x7feee806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:08.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.952+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 0x7feee806d190 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7feef8001380 tx=0x7feef8007be0 comp rx=0 tx=0).stop 2023-06-09T11:57:08.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.952+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 msgr2=0x7fef08159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:08.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.952+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7feef00090b0 tx=0x7feef0004750 comp rx=0 tx=0).stop 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 shutdown_connections 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fef08153850 0x7fef0814fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feee806ace0 0x7feee806d190 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fef0806ac20 0x7fef081f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 --2- 172.21.15.7:0/3221214607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef080c6590 0x7fef08159420 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:08.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 >> 172.21.15.7:0/3221214607 conn(0x7fef080737f0 msgr2=0x7fef080cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:08.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 shutdown_connections 2023-06-09T11:57:08.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:08.953+0000 7fef0e3ff700 1 -- 172.21.15.7:0/3221214607 wait complete. 2023-06-09T11:57:08.979 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:57:08.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:09.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.181+0000 7f8a241ff700 1 Processor -- start 2023-06-09T11:57:09.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.181+0000 7f8a241ff700 1 -- start start 2023-06-09T11:57:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a241ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a241ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c0c5a80 con 0x7f8a1c153880 2023-06-09T11:57:09.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a241ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1c15a500 con 0x7f8a1c15b4a0 2023-06-09T11:57:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a241ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1c0c5f10 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a2279c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a21f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a1c153880 0x7f8a1c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52246/0 (socket says 172.21.15.7:52246) 2023-06-09T11:57:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a2279c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34514/0 (socket says 172.21.15.7:34514) 2023-06-09T11:57:09.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a21f9b700 1 -- 172.21.15.7:0/1582532465 learned_addr learned my addr 172.21.15.7:0/1582532465 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:09.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a2279c700 1 -- 172.21.15.7:0/1582532465 >> v1:172.21.15.117:6789/0 conn(0x7f8a1c15b4a0 legacy=0x7f8a1c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a2279c700 1 -- 172.21.15.7:0/1582532465 >> v1:172.21.15.7:6789/0 conn(0x7f8a1c153880 legacy=0x7f8a1c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a2279c700 1 -- 172.21.15.7:0/1582532465 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1c1596e0 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.183+0000 7f8a20f99700 1 -- 172.21.15.7:0/1582532465 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211380781 0 0) 0x7f8a0c001110 con 0x7f8a1c15b4a0 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a2279c700 1 --2- 172.21.15.7:0/1582532465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009d90 tx=0x7f8a1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fda877948a3c63 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a20f99700 1 -- 172.21.15.7:0/1582532465 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400e070 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a20f99700 1 -- 172.21.15.7:0/1582532465 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400bad0 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a20f99700 1 -- 172.21.15.7:0/1582532465 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400be80 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a241ff700 1 -- 172.21.15.7:0/1582532465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 msgr2=0x7f8a1c0c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.184+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1582532465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f8a14009d90 tx=0x7f8a1400b330 comp rx=0 tx=0).stop 2023-06-09T11:57:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.185+0000 7f8a241ff700 1 -- 172.21.15.7:0/1582532465 shutdown_connections 2023-06-09T11:57:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.185+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1582532465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c0c5540 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.185+0000 7f8a241ff700 1 -- 172.21.15.7:0/1582532465 >> 172.21.15.7:0/1582532465 conn(0x7f8a1c0737f0 msgr2=0x7f8a1c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.185+0000 7f8a241ff700 1 -- 172.21.15.7:0/1582532465 shutdown_connections 2023-06-09T11:57:09.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.185+0000 7f8a241ff700 1 -- 172.21.15.7:0/1582532465 wait complete. 2023-06-09T11:57:09.191 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.186+0000 7f8a241ff700 1 Processor -- start 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 -- start start 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 0x7f8a1c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 0x7f8a1c0cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c0cda90 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c14ac60 con 0x7f8a1c153880 2023-06-09T11:57:09.192 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.187+0000 7f8a241ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1c14af40 con 0x7f8a1c15b4a0 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34530/0 (socket says 172.21.15.7:34530) 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 -- 172.21.15.7:0/1217179166 learned_addr learned my addr 172.21.15.7:0/1217179166 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a2179a700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 0x7f8a1c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a2279c700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 0x7f8a1c0cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 -- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 msgr2=0x7f8a1c0cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 0x7f8a1c0cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 -- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 msgr2=0x7f8a1c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 0x7f8a1c0cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 -- 172.21.15.7:0/1217179166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009a70 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a2179a700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 0x7f8a1c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a2279c700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 0x7f8a1c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:09.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.188+0000 7f8a21f9b700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c780 tx=0x7f8a1800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.189+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18004c80 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.189+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a18004de0 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.189+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a18007310 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.189+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1c159c60 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.189+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c159980 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.190+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a1c159f40 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.191+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a18007470 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.192+0000 7f8a127fc700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 0x7f8a0006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.192+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f8a18088e90 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.192+0000 7f8a2179a700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 0x7f8a0006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.193+0000 7f8a2179a700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 0x7f8a0006d1c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0cf310 tx=0x7f8a0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:09.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.196+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8a180536d0 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.449+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a1c150ee0 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.450+0000 7f8a127fc700 1 -- 172.21.15.7:0/1217179166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f8a18018020 con 0x7f8a1c0c9f40 2023-06-09T11:57:09.453 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:09.453 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 msgr2=0x7f8a0006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 0x7f8a0006d1c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0cf310 tx=0x7f8a0c009040 comp rx=0 tx=0).stop 2023-06-09T11:57:09.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 msgr2=0x7f8a1c149fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800c780 tx=0x7f8a1800cb40 comp rx=0 tx=0).stop 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 shutdown_connections 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a0006ad10 0x7f8a0006d1c0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a1c15b4a0 0x7f8a1c0cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a1c0c9f40 0x7f8a1c149fa0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 --2- 172.21.15.7:0/1217179166 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a1c153880 0x7f8a1c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:09.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 >> 172.21.15.7:0/1217179166 conn(0x7f8a1c0737f0 msgr2=0x7f8a1c14fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 shutdown_connections 2023-06-09T11:57:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:09.452+0000 7f8a241ff700 1 -- 172.21.15.7:0/1217179166 wait complete. 2023-06-09T11:57:09.457 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:09.473 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-06-09T11:57:09.474 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 216, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 216, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:stopping", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:57:08.951234+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:57:11.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.683+0000 7f10451ff700 1 Processor -- start 2023-06-09T11:57:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.683+0000 7f10451ff700 1 -- start start 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.684+0000 7f10451ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f104015b4a0 0x7f104015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.684+0000 7f10451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f10400c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.684+0000 7f10451ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104015a500 con 0x7f1040153880 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.684+0000 7f10451ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10400c5f10 con 0x7f104015b4a0 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.684+0000 7f10451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104015da00 con 0x7f10400c9f40 2023-06-09T11:57:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103ed9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1040153880 0x7f104015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:52254/0 (socket says 172.21.15.7:52254) 2023-06-09T11:57:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103ed9d700 1 -- 172.21.15.7:0/1961988228 learned_addr learned my addr 172.21.15.7:0/1961988228 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103f59e700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f10400c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103e59c700 1 --2- 172.21.15.7:0/1961988228 >> v2:172.21.15.117:3300/0 conn(0x7f104015b4a0 0x7f104015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396388847 0 0) 0x7f1030001110 con 0x7f1040153880 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.685+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10280035b0 con 0x7f1040153880 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508932454 0 0) 0x7f10280035b0 con 0x7f1040153880 2023-06-09T11:57:11.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 msgr2=0x7f10400c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f10400c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 >> v2:172.21.15.117:3300/0 conn(0x7f104015b4a0 msgr2=0x7f104015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 --2- 172.21.15.7:0/1961988228 >> v2:172.21.15.117:3300/0 conn(0x7f104015b4a0 0x7f104015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10401596e0 con 0x7f1040153880 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f1030001800 con 0x7f1040153880 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 >> v1:172.21.15.7:6789/0 conn(0x7f1040153880 legacy=0x7f104015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f1028003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1028004430 0x7f10280048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f1028005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c5f10 con 0x7f1028003a00 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104015da00 con 0x7f1028004430 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102800ba10 con 0x7f10400c9f40 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f1028003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103e59c700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1028004430 0x7f10280048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103f59e700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f1028005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 -- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 msgr2=0x7f1028005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f1028005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 -- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1028004430 msgr2=0x7f10280048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1028004430 0x7f10280048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103ed9d700 1 -- 172.21.15.7:0/1961988228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280035b0 con 0x7f1028003a00 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.686+0000 7f103f59e700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10400c9f40 0x7f1028005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f1028003e10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f103000c320 tx=0x7f1028005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6505f6da410c98e2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:11.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000ce20 con 0x7f1028003a00 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1030005960 con 0x7f1028003a00 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f103dd9b700 1 -- 172.21.15.7:0/1961988228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030005d10 con 0x7f1028003a00 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f10451ff700 1 -- 172.21.15.7:0/1961988228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 msgr2=0x7f1028003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f10451ff700 1 --2- 172.21.15.7:0/1961988228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f1028003e10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f103000c320 tx=0x7f1028005880 comp rx=0 tx=0).stop 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.687+0000 7f103f59e700 1 -- 172.21.15.7:0/1961988228 reap_dead start 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.688+0000 7f10451ff700 1 -- 172.21.15.7:0/1961988228 shutdown_connections 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.688+0000 7f10451ff700 1 -- 172.21.15.7:0/1961988228 >> 172.21.15.7:0/1961988228 conn(0x7f10400737f0 msgr2=0x7f10400720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:11.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.688+0000 7f10451ff700 1 -- 172.21.15.7:0/1961988228 shutdown_connections 2023-06-09T11:57:11.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.688+0000 7f10451ff700 1 -- 172.21.15.7:0/1961988228 wait complete. 2023-06-09T11:57:11.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.689+0000 7f10451ff700 1 Processor -- start 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 -- start start 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10401efb10 0x7f104014ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 0x7f10400ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104014d3c0 con 0x7f1028003a00 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1040156f70 con 0x7f10401efb10 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.690+0000 7f10451ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10401eff80 con 0x7f10401f2f40 2023-06-09T11:57:11.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34562/0 (socket says 172.21.15.7:34562) 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 -- 172.21.15.7:0/1823522361 learned_addr learned my addr 172.21.15.7:0/1823522361 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103e59c700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10401efb10 0x7f104014ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103f59e700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 0x7f10400ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 -- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 msgr2=0x7f10400ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 0x7f10400ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 -- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10401efb10 msgr2=0x7f104014ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10401efb10 0x7f104014ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 -- 172.21.15.7:0/1823522361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040159c60 con 0x7f1028003a00 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103f59e700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 0x7f10400ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.691+0000 7f103ed9d700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1028005770 tx=0x7f1030000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.692+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030004870 con 0x7f1028003a00 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.692+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10300049d0 con 0x7f1028003a00 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.692+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030004d80 con 0x7f1028003a00 2023-06-09T11:57:11.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.692+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1040159980 con 0x7f1028003a00 2023-06-09T11:57:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.692+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1040159f40 con 0x7f1028003a00 2023-06-09T11:57:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.693+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1040074cf0 con 0x7f1028003a00 2023-06-09T11:57:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.694+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1030005070 con 0x7f1028003a00 2023-06-09T11:57:11.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.695+0000 7f10277fe700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 0x7f102c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.696+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f103008b170 con 0x7f1028003a00 2023-06-09T11:57:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.696+0000 7f103e59c700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 0x7f102c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:11.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.696+0000 7f103e59c700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 0x7f102c06d190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f10400cf0d0 tx=0x7f1034009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:11.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.700+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f10300559b0 con 0x7f1028003a00 2023-06-09T11:57:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.960+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f104015a500 con 0x7f1028003a00 2023-06-09T11:57:11.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.960+0000 7f10277fe700 1 -- 172.21.15.7:0/1823522361 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f104015a500 con 0x7f1028003a00 2023-06-09T11:57:11.964 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:11.964 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.962+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 msgr2=0x7f102c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.962+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 0x7f102c06d190 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f10400cf0d0 tx=0x7f1034009210 comp rx=0 tx=0).stop 2023-06-09T11:57:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.962+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 msgr2=0x7f10400b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:11.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.962+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f1028005770 tx=0x7f1030000fe0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 shutdown_connections 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f102c06ace0 0x7f102c06d190 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10401f2f40 0x7f10400ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1028003a00 0x7f10400b3fc0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 --2- 172.21.15.7:0/1823522361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10401efb10 0x7f104014ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 >> 172.21.15.7:0/1823522361 conn(0x7f10400737f0 msgr2=0x7f104014eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 shutdown_connections 2023-06-09T11:57:11.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:11.963+0000 7f10451ff700 1 -- 172.21.15.7:0/1823522361 wait complete. 2023-06-09T11:57:11.968 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:11.984 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:13.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.274+0000 7f9a251ff700 1 Processor -- start 2023-06-09T11:57:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.274+0000 7f9a251ff700 1 -- start start 2023-06-09T11:57:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.276+0000 7f9a251ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9a200c9090 0x7f9a200c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.276+0000 7f9a251ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a200cad00 0x7f9a200cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.276+0000 7f9a251ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2015cec0 con 0x7f9a200cfaf0 2023-06-09T11:57:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.276+0000 7f9a251ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2015d160 con 0x7f9a200cad00 2023-06-09T11:57:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.276+0000 7f9a251ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2015d440 con 0x7f9a200c9090 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1f59e700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f9a200cfaf0 0x7f9a201598b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:57648/0 (socket says 172.21.15.7:57648) 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1e59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9a200cad00 0x7f9a200cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1f59e700 1 -- 172.21.15.7:0/852222580 learned_addr learned my addr 172.21.15.7:0/852222580 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1ed9d700 1 --2- 172.21.15.7:0/852222580 >> v2:172.21.15.117:3300/0 conn(0x7f9a200c9090 0x7f9a200c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1e59c700 1 -- 172.21.15.7:0/852222580 >> v2:172.21.15.117:3300/0 conn(0x7f9a200c9090 msgr2=0x7f9a200c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/852222580 >> v2:172.21.15.117:3300/0 conn(0x7f9a200c9090 0x7f9a200c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1e59c700 1 -- 172.21.15.7:0/852222580 >> v1:172.21.15.117:6789/0 conn(0x7f9a200cfaf0 legacy=0x7f9a201598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.277+0000 7f9a1e59c700 1 -- 172.21.15.7:0/852222580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2015d720 con 0x7f9a200cad00 2023-06-09T11:57:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.278+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/852222580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200cad00 0x7f9a200cf5b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9a20066ee0 tx=0x7f9a14009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dcc04934389b8e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.278+0000 7f9a1dd9b700 1 -- 172.21.15.7:0/852222580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14011830 con 0x7f9a200cad00 2023-06-09T11:57:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.278+0000 7f9a1dd9b700 1 -- 172.21.15.7:0/852222580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a14011990 con 0x7f9a200cad00 2023-06-09T11:57:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.278+0000 7f9a1dd9b700 1 -- 172.21.15.7:0/852222580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a14011d20 con 0x7f9a200cad00 2023-06-09T11:57:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 -- 172.21.15.7:0/852222580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200cad00 msgr2=0x7f9a200cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 --2- 172.21.15.7:0/852222580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200cad00 0x7f9a200cf5b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f9a20066ee0 tx=0x7f9a14009b20 comp rx=0 tx=0).stop 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 -- 172.21.15.7:0/852222580 shutdown_connections 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 --2- 172.21.15.7:0/852222580 >> v2:172.21.15.117:3300/0 conn(0x7f9a200c9090 0x7f9a200c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 --2- 172.21.15.7:0/852222580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200cad00 0x7f9a200cf5b0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.279+0000 7f9a251ff700 1 -- 172.21.15.7:0/852222580 >> 172.21.15.7:0/852222580 conn(0x7f9a20070af0 msgr2=0x7f9a20070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.280+0000 7f9a251ff700 1 -- 172.21.15.7:0/852222580 shutdown_connections 2023-06-09T11:57:14.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.280+0000 7f9a251ff700 1 -- 172.21.15.7:0/852222580 wait complete. 2023-06-09T11:57:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.282+0000 7f9a251ff700 1 Processor -- start 2023-06-09T11:57:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.282+0000 7f9a251ff700 1 -- start start 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.283+0000 7f9a251ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.283+0000 7f9a251ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 0x7f9a200c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a251ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a200cfaf0 0x7f9a2006c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a251ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a20070610 con 0x7f9a200c9090 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a251ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200be9e0 con 0x7f9a200cad00 2023-06-09T11:57:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a251ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a201f5b10 con 0x7f9a200cfaf0 2023-06-09T11:57:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a1ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a1ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58484/0 (socket says 172.21.15.7:58484) 2023-06-09T11:57:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a1ed9d700 1 -- 172.21.15.7:0/1259991496 learned_addr learned my addr 172.21.15.7:0/1259991496 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 0x7f9a200c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.284+0000 7f9a1f59e700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a200cfaf0 0x7f9a2006c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 -- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a200cfaf0 msgr2=0x7f9a2006c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a200cfaf0 0x7f9a2006c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 -- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 msgr2=0x7f9a200c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 0x7f9a200c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 -- 172.21.15.7:0/1259991496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140094a0 con 0x7f9a200c9090 2023-06-09T11:57:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 0x7f9a200c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:57:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.285+0000 7f9a1ed9d700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000e870 tx=0x7f9a1000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.286+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10009590 con 0x7f9a200c9090 2023-06-09T11:57:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.286+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10013680 con 0x7f9a200c9090 2023-06-09T11:57:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.286+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a200cb160 con 0x7f9a200c9090 2023-06-09T11:57:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.286+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100139f0 con 0x7f9a200c9090 2023-06-09T11:57:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.286+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a200ca620 con 0x7f9a200c9090 2023-06-09T11:57:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.287+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a200b53f0 con 0x7f9a200c9090 2023-06-09T11:57:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.289+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a10013b50 con 0x7f9a200c9090 2023-06-09T11:57:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.290+0000 7f9a077fe700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 0x7f9a0c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.291+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f9a10089a60 con 0x7f9a200c9090 2023-06-09T11:57:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.292+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 0x7f9a0c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.293+0000 7f9a1e59c700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 0x7f9a0c06d1c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9a140090b0 tx=0x7f9a1401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:14.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.297+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9a10057d90 con 0x7f9a200c9090 2023-06-09T11:57:14.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.662+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a201f4f70 con 0x7f9a200c9090 2023-06-09T11:57:14.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.663+0000 7f9a077fe700 1 -- 172.21.15.7:0/1259991496 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f9a10021020 con 0x7f9a200c9090 2023-06-09T11:57:14.666 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:14.666 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:14.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 msgr2=0x7f9a0c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 0x7f9a0c06d1c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9a140090b0 tx=0x7f9a1401c040 comp rx=0 tx=0).stop 2023-06-09T11:57:14.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 msgr2=0x7f9a2015c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000e870 tx=0x7f9a1000eb80 comp rx=0 tx=0).stop 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 shutdown_connections 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9a0c06ad10 0x7f9a0c06d1c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9a200cfaf0 0x7f9a2006c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a200c9090 0x7f9a2015c290 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 --2- 172.21.15.7:0/1259991496 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9a200cad00 0x7f9a200c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:14.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 >> 172.21.15.7:0/1259991496 conn(0x7f9a20070af0 msgr2=0x7f9a200c6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:14.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 shutdown_connections 2023-06-09T11:57:14.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:14.666+0000 7f9a251ff700 1 -- 172.21.15.7:0/1259991496 wait complete. 2023-06-09T11:57:14.671 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:14.697 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:16.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:16.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.900+0000 7fc3083ff700 1 Processor -- start 2023-06-09T11:57:16.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.901+0000 7fc3083ff700 1 -- start start 2023-06-09T11:57:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc3083ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc30006a4d0 0x7fc3000c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc3083ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc30015cac0 con 0x7fc30006e460 2023-06-09T11:57:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc3083ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3000c37d0 con 0x7fc3000c6280 2023-06-09T11:57:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc3083ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3000c8870 con 0x7fc30006a4d0 2023-06-09T11:57:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc30599a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc30006a4d0 0x7fc3000c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc30599a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc30006a4d0 0x7fc3000c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58492/0 (socket says 172.21.15.7:58492) 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.902+0000 7fc30599a700 1 -- 172.21.15.7:0/1748062385 learned_addr learned my addr 172.21.15.7:0/1748062385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc30599a700 1 -- 172.21.15.7:0/1748062385 >> v1:172.21.15.117:6789/0 conn(0x7fc3000c6280 legacy=0x7fc3000737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc30599a700 1 -- 172.21.15.7:0/1748062385 >> v1:172.21.15.7:6789/0 conn(0x7fc30006e460 legacy=0x7fc30006cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc30599a700 1 -- 172.21.15.7:0/1748062385 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30015cd30 con 0x7fc30006a4d0 2023-06-09T11:57:16.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc305199700 1 -- 172.21.15.7:0/1748062385 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519707560 0 0) 0x7fc30015cac0 con 0x7fc30006e460 2023-06-09T11:57:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc30599a700 1 --2- 172.21.15.7:0/1748062385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30006a4d0 0x7fc3000c76b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098f0 tx=0x7fc2f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1c67914938e73d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc305199700 1 -- 172.21.15.7:0/1748062385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000b8a0 con 0x7fc30006a4d0 2023-06-09T11:57:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc305199700 1 -- 172.21.15.7:0/1748062385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f000ba00 con 0x7fc30006a4d0 2023-06-09T11:57:16.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.903+0000 7fc305199700 1 -- 172.21.15.7:0/1748062385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f000bd90 con 0x7fc30006a4d0 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 -- 172.21.15.7:0/1748062385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30006a4d0 msgr2=0x7fc3000c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 --2- 172.21.15.7:0/1748062385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30006a4d0 0x7fc3000c76b0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00098f0 tx=0x7fc2f0009c00 comp rx=0 tx=0).stop 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 -- 172.21.15.7:0/1748062385 shutdown_connections 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 --2- 172.21.15.7:0/1748062385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc30006a4d0 0x7fc3000c76b0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 -- 172.21.15.7:0/1748062385 >> 172.21.15.7:0/1748062385 conn(0x7fc30006f980 msgr2=0x7fc3000cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 -- 172.21.15.7:0/1748062385 shutdown_connections 2023-06-09T11:57:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.904+0000 7fc3083ff700 1 -- 172.21.15.7:0/1748062385 wait complete. 2023-06-09T11:57:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.907+0000 7fc3083ff700 1 Processor -- start 2023-06-09T11:57:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.907+0000 7fc3083ff700 1 -- start start 2023-06-09T11:57:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30006e460 0x7fc3001e5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 0x7fc3001eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001ef510 con 0x7fc3000c6280 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001e49b0 con 0x7fc30006a4d0 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc3083ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3001e3280 con 0x7fc30006e460 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30599a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30006e460 0x7fc3001e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:33008/0 (socket says 172.21.15.7:33008) 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 0x7fc3001eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 -- 172.21.15.7:0/2413197341 learned_addr learned my addr 172.21.15.7:0/2413197341 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 -- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30006e460 msgr2=0x7fc3001e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30006e460 0x7fc3001e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.908+0000 7fc30619b700 1 -- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 msgr2=0x7fc3001eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc30619b700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 0x7fc3001eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc30619b700 1 -- 172.21.15.7:0/2413197341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009550 con 0x7fc30006a4d0 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc30699c700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 0x7fc3001eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc30619b700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c450 tx=0x7fc2fc00c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0052d0 con 0x7fc30006a4d0 2023-06-09T11:57:16.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2fc005430 con 0x7fc30006a4d0 2023-06-09T11:57:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0038c0 con 0x7fc30006a4d0 2023-06-09T11:57:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30015d2f0 con 0x7fc30006a4d0 2023-06-09T11:57:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.909+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30015d010 con 0x7fc30006a4d0 2023-06-09T11:57:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.911+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30015d5d0 con 0x7fc30006a4d0 2023-06-09T11:57:16.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.912+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc2fc003a20 con 0x7fc30006a4d0 2023-06-09T11:57:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.913+0000 7fc2f6ffd700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 0x7fc2e406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.913+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fc2fc0890d0 con 0x7fc30006a4d0 2023-06-09T11:57:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.913+0000 7fc30599a700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 0x7fc2e406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:16.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.913+0000 7fc30599a700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 0x7fc2e406d1c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:16.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:16.918+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc2fc053850 con 0x7fc30006a4d0 2023-06-09T11:57:17.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.202+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc300074cf0 con 0x7fc30006a4d0 2023-06-09T11:57:17.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.202+0000 7fc2f6ffd700 1 -- 172.21.15.7:0/2413197341 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fc2fc017020 con 0x7fc30006a4d0 2023-06-09T11:57:17.205 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:17.205 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:17.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 msgr2=0x7fc2e406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 0x7fc2e406d1c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0008030 comp rx=0 tx=0).stop 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 msgr2=0x7fc3001407b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c450 tx=0x7fc2fc00c760 comp rx=0 tx=0).stop 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 shutdown_connections 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3000c6280 0x7fc3001eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc30006a4d0 0x7fc3001407b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:17.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc2e406ad10 0x7fc2e406d1c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 --2- 172.21.15.7:0/2413197341 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc30006e460 0x7fc3001e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 >> 172.21.15.7:0/2413197341 conn(0x7fc30006f980 msgr2=0x7fc30006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 shutdown_connections 2023-06-09T11:57:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:17.205+0000 7fc3083ff700 1 -- 172.21.15.7:0/2413197341 wait complete. 2023-06-09T11:57:17.209 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:17.225 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:18.585 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 21% (52/244) Updating files: 22% (54/244) Updating files: 23% (57/244) Updating files: 24% (59/244) Updating files: 25% (61/244) Updating files: 26% (64/244) Updating files: 27% (66/244) Updating files: 28% (69/244) Updating files: 29% (71/244) Updating files: 30% (74/244) Updating files: 31% (76/244) Updating files: 32% (79/244) Updating files: 33% (81/244) Updating files: 34% (83/244) Updating files: 35% (86/244) Updating files: 36% (88/244) Updating files: 37% (91/244) Updating files: 38% (93/244) Updating files: 39% (96/244) Updating files: 40% (98/244) Updating files: 41% (101/244) Updating files: 42% (103/244) Updating files: 43% (105/244) Updating files: 44% (108/244) Updating files: 45% (110/244) Updating files: 46% (113/244) Updating files: 47% (115/244) Updating files: 48% (118/244) Updating files: 49% (120/244) Updating files: 50% (122/244) Updating files: 51% (125/244) Updating files: 52% (127/244) Updating files: 53% (130/244) Updating files: 54% (132/244) Updating files: 55% (135/244) Updating files: 56% (137/244) Updating files: 57% (140/244) Updating files: 57% (141/244) Updating files: 58% (142/244) Updating files: 59% (144/244) Updating files: 59% (145/244) Updating files: 60% (147/244) Updating files: 61% (149/244) Updating files: 62% (152/244) Updating files: 62% (153/244) Updating files: 63% (154/244) Updating files: 64% (157/244) Updating files: 65% (159/244) Updating files: 66% (162/244) Updating files: 67% (164/244) Updating files: 68% (166/244) Updating files: 68% (168/244) Updating files: 69% (169/244) Updating files: 69% (170/244) Updating files: 70% (171/244) Updating files: 71% (174/244) Updating files: 71% (175/244) Updating files: 72% (176/244) Updating files: 73% (179/244) Updating files: 74% (181/244) Updating files: 75% (183/244) Updating files: 76% (186/244) Updating files: 77% (188/244) Updating files: 78% (191/244) Updating files: 79% (193/244) Updating files: 80% (196/244) Updating files: 81% (198/244) Updating files: 82% (201/244) Updating files: 83% (203/244) Updating files: 84% (205/244) Updating files: 85% (208/244) Updating files: 86% (210/244) Updating files: 87% (213/244) Updating files: 88% (215/244) Updating files: 89% (218/244) Updating files: 90% (220/244) Updating files: 91% (223/244) Updating files: 92% (225/244) Updating files: 93% (227/244) Updating files: 94% (230/244) Updating files: 95% (232/244) Updating files: 96% (235/244) Updating files: 97% (237/244) Updating files: 98% (240/244) Updating files: 99% (242/244) Updating files: 100% (244/244) Updating files: 100% (244/244), done. 2023-06-09T11:57:19.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:19.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.451+0000 7f1f26bff700 1 Processor -- start 2023-06-09T11:57:19.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.451+0000 7f1f26bff700 1 -- start start 2023-06-09T11:57:19.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f26bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 0x7f1f201501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f26bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200c0bf0 con 0x7f1f200c0140 2023-06-09T11:57:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f26bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f200702f0 con 0x7f1f200c1bc0 2023-06-09T11:57:19.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f26bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f20070110 con 0x7f1f2015a0c0 2023-06-09T11:57:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f2519c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f200c0140 0x7f1f20160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46450/0 (socket says 172.21.15.7:46450) 2023-06-09T11:57:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.452+0000 7f1f2519c700 1 -- 172.21.15.7:0/930731742 learned_addr learned my addr 172.21.15.7:0/930731742 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1ffff700 1 --2- 172.21.15.7:0/930731742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 0x7f1f201501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130502699 0 0) 0x7f1f200c0bf0 con 0x7f1f200c0140 2023-06-09T11:57:19.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f100035b0 con 0x7f1f200c0140 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971244045 0 0) 0x7f1f200702f0 con 0x7f1f200c1bc0 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f200c0bf0 con 0x7f1f200c1bc0 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1ffff700 1 -- 172.21.15.7:0/930731742 >> v1:172.21.15.117:6789/0 conn(0x7f1f200c1bc0 legacy=0x7f1f201599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1ffff700 1 -- 172.21.15.7:0/930731742 >> v1:172.21.15.7:6789/0 conn(0x7f1f200c0140 legacy=0x7f1f20160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1ffff700 1 -- 172.21.15.7:0/930731742 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f200c0e90 con 0x7f1f2015a0c0 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.453+0000 7f1f1ffff700 1 --2- 172.21.15.7:0/930731742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 0x7f1f201501b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ceba0020efdf7f4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:19.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c011830 con 0x7f1f2015a0c0 2023-06-09T11:57:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c011990 con 0x7f1f2015a0c0 2023-06-09T11:57:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f1f7fe700 1 -- 172.21.15.7:0/930731742 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c011d00 con 0x7f1f2015a0c0 2023-06-09T11:57:19.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 -- 172.21.15.7:0/930731742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 msgr2=0x7f1f201501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 --2- 172.21.15.7:0/930731742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 0x7f1f201501b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009870 tx=0x7f1f0c009b80 comp rx=0 tx=0).stop 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 -- 172.21.15.7:0/930731742 shutdown_connections 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 --2- 172.21.15.7:0/930731742 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f2015a0c0 0x7f1f201501b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 -- 172.21.15.7:0/930731742 >> 172.21.15.7:0/930731742 conn(0x7f1f2006a4d0 msgr2=0x7f1f2006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 -- 172.21.15.7:0/930731742 shutdown_connections 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.454+0000 7f1f26bff700 1 -- 172.21.15.7:0/930731742 wait complete. 2023-06-09T11:57:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.456+0000 7f1f26bff700 1 Processor -- start 2023-06-09T11:57:19.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.456+0000 7f1f26bff700 1 -- start start 2023-06-09T11:57:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 0x7f1f2014b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c1bc0 0x7f1f20158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 0x7f1f201592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c4690 con 0x7f1f200c0140 2023-06-09T11:57:19.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c2f30 con 0x7f1f2015a0c0 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f26bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f200c3ab0 con 0x7f1f200c1bc0 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f2499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 0x7f1f2014b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f2519c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 0x7f1f201592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f1ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c1bc0 0x7f1f20158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f2499b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 0x7f1f2014b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58516/0 (socket says 172.21.15.7:58516) 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.457+0000 7f1f2499b700 1 -- 172.21.15.7:0/2223610264 learned_addr learned my addr 172.21.15.7:0/2223610264 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 -- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c1bc0 msgr2=0x7f1f20158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c1bc0 0x7f1f20158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 -- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 msgr2=0x7f1f2014b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 0x7f1f2014b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 -- 172.21.15.7:0/2223610264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0094a0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f2519c700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 0x7f1f201592e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800cad0 tx=0x7f1f1800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800a960 con 0x7f1f2015a0c0 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1800aac0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1800ae30 con 0x7f1f2015a0c0 2023-06-09T11:57:19.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f200c1450 con 0x7f1f2015a0c0 2023-06-09T11:57:19.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.458+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f200c1170 con 0x7f1f2015a0c0 2023-06-09T11:57:19.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.459+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f200705d0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.460+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f1801b3b0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.462+0000 7f1f1d7fa700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 0x7f1f0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.462+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f1f180897a0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.462+0000 7f1f2499b700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 0x7f1f0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.462+0000 7f1f2499b700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 0x7f1f0406d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001010 tx=0x7f1f14009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.466+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f18054000 con 0x7f1f2015a0c0 2023-06-09T11:57:19.660 INFO:tasks.workunit.client.0.smithi007.stdout:HEAD is now at 877cacbbdbb v0.36 2023-06-09T11:57:19.663 INFO:tasks.workunit.client.0.smithi007.stderr:+ mkdir .snap/v0.36 2023-06-09T11:57:19.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.746+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f20074cf0 con 0x7f1f2015a0c0 2023-06-09T11:57:19.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.747+0000 7f1f1d7fa700 1 -- 172.21.15.7:0/2223610264 <== mon.1 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f1f18057620 con 0x7f1f2015a0c0 2023-06-09T11:57:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:19.749 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:19.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 msgr2=0x7f1f0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 0x7f1f0406d190 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1f14001010 tx=0x7f1f14009040 comp rx=0 tx=0).stop 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 msgr2=0x7f1f201592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 0x7f1f201592e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1f1800cad0 tx=0x7f1f1800cde0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 shutdown_connections 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f2015a0c0 0x7f1f201592e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f200c0140 0x7f1f2014b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f0406ace0 0x7f1f0406d190 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 --2- 172.21.15.7:0/2223610264 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f200c1bc0 0x7f1f20158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 >> 172.21.15.7:0/2223610264 conn(0x7f1f2006a4d0 msgr2=0x7f1f200c6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 shutdown_connections 2023-06-09T11:57:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:19.749+0000 7f1f26bff700 1 -- 172.21.15.7:0/2223610264 wait complete. 2023-06-09T11:57:19.753 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:19.769 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:20.015 INFO:tasks.workunit.client.0.smithi007.stderr:+ for v in $versions 2023-06-09T11:57:20.016 INFO:tasks.workunit.client.0.smithi007.stderr:+ '[' 37 -eq 48 ']' 2023-06-09T11:57:20.016 INFO:tasks.workunit.client.0.smithi007.stdout:v0.37 2023-06-09T11:57:20.016 INFO:tasks.workunit.client.0.smithi007.stderr:+ ver=v0.37 2023-06-09T11:57:20.016 INFO:tasks.workunit.client.0.smithi007.stderr:+ echo v0.37 2023-06-09T11:57:20.017 INFO:tasks.workunit.client.0.smithi007.stderr:+ git reset --hard v0.37 2023-06-09T11:57:21.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.972+0000 7fde68fff700 1 Processor -- start 2023-06-09T11:57:21.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.972+0000 7fde68fff700 1 -- start start 2023-06-09T11:57:21.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde68fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde640c8e90 0x7fde6415d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde68fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde640b5370 0x7fde640c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde68fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6415dc90 con 0x7fde640c9260 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde68fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde641597d0 con 0x7fde640b5370 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde68fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640cfaf0 con 0x7fde640c8e90 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde62d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde640b5370 0x7fde640c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde62d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde640b5370 0x7fde640c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58526/0 (socket says 172.21.15.7:58526) 2023-06-09T11:57:21.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.973+0000 7fde62d9d700 1 -- 172.21.15.7:0/421361882 learned_addr learned my addr 172.21.15.7:0/421361882 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde6259c700 1 --2- 172.21.15.7:0/421361882 >> v2:172.21.15.117:3300/0 conn(0x7fde640c8e90 0x7fde6415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde62d9d700 1 -- 172.21.15.7:0/421361882 >> v2:172.21.15.117:3300/0 conn(0x7fde640c8e90 msgr2=0x7fde6415d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde62d9d700 1 --2- 172.21.15.7:0/421361882 >> v2:172.21.15.117:3300/0 conn(0x7fde640c8e90 0x7fde6415d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde62d9d700 1 -- 172.21.15.7:0/421361882 >> v1:172.21.15.117:6789/0 conn(0x7fde640c9260 legacy=0x7fde640cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde62d9d700 1 -- 172.21.15.7:0/421361882 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640cfdd0 con 0x7fde640b5370 2023-06-09T11:57:21.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde62d9d700 1 --2- 172.21.15.7:0/421361882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640b5370 0x7fde640c9fe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fde640d0760 tx=0x7fde5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5132c9f7760961ee server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde61d9b700 1 -- 172.21.15.7:0/421361882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00e070 con 0x7fde640b5370 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde61d9b700 1 -- 172.21.15.7:0/421361882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c003a20 con 0x7fde640b5370 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.974+0000 7fde61d9b700 1 -- 172.21.15.7:0/421361882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c003dd0 con 0x7fde640b5370 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 -- 172.21.15.7:0/421361882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640b5370 msgr2=0x7fde640c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 --2- 172.21.15.7:0/421361882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640b5370 0x7fde640c9fe0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fde640d0760 tx=0x7fde5c00b330 comp rx=0 tx=0).stop 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 -- 172.21.15.7:0/421361882 shutdown_connections 2023-06-09T11:57:21.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 --2- 172.21.15.7:0/421361882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640b5370 0x7fde640c9fe0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 --2- 172.21.15.7:0/421361882 >> v2:172.21.15.117:3300/0 conn(0x7fde640c8e90 0x7fde6415d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 -- 172.21.15.7:0/421361882 >> 172.21.15.7:0/421361882 conn(0x7fde64070af0 msgr2=0x7fde64070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 -- 172.21.15.7:0/421361882 shutdown_connections 2023-06-09T11:57:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.975+0000 7fde68fff700 1 -- 172.21.15.7:0/421361882 wait complete. 2023-06-09T11:57:21.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.977+0000 7fde68fff700 1 Processor -- start 2023-06-09T11:57:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.978+0000 7fde68fff700 1 -- start start 2023-06-09T11:57:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde640b5370 0x7fde640aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde640c8e90 0x7fde640c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415c930 con 0x7fde640c9260 2023-06-09T11:57:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640ac240 con 0x7fde640c8e90 2023-06-09T11:57:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde68fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6415d280 con 0x7fde640b5370 2023-06-09T11:57:21.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde6259c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde640b5370 0x7fde640aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde62d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58534/0 (socket says 172.21.15.7:58534) 2023-06-09T11:57:21.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde62d9d700 1 -- 172.21.15.7:0/1737030101 learned_addr learned my addr 172.21.15.7:0/1737030101 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.979+0000 7fde5bfff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde640c8e90 0x7fde640c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 -- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde640b5370 msgr2=0x7fde640aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde640b5370 0x7fde640aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 -- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde640c8e90 msgr2=0x7fde640c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde640c8e90 0x7fde640c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 -- 172.21.15.7:0/1737030101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c009b30 con 0x7fde640c9260 2023-06-09T11:57:21.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde62d9d700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fde5c004000 tx=0x7fde5c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004440 con 0x7fde640c9260 2023-06-09T11:57:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c0092e0 con 0x7fde640c9260 2023-06-09T11:57:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.980+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde64159ca0 con 0x7fde640c9260 2023-06-09T11:57:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.981+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c004800 con 0x7fde640c9260 2023-06-09T11:57:21.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.981+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde641599c0 con 0x7fde640c9260 2023-06-09T11:57:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.982+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde64159f80 con 0x7fde640c9260 2023-06-09T11:57:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.983+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde5c01c440 con 0x7fde640c9260 2023-06-09T11:57:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.983+0000 7fde5affd700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 0x7fde4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.984+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fde5c08ad70 con 0x7fde640c9260 2023-06-09T11:57:21.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.984+0000 7fde6259c700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 0x7fde4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:21.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.984+0000 7fde6259c700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 0x7fde4406d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fde54006b20 tx=0x7fde54008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:21.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:21.988+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde5c0555b0 con 0x7fde640c9260 2023-06-09T11:57:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.244+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde6415dc90 con 0x7fde640c9260 2023-06-09T11:57:22.247 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.245+0000 7fde5affd700 1 -- 172.21.15.7:0/1737030101 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fde6415dc90 con 0x7fde640c9260 2023-06-09T11:57:22.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:22.248 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.247+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 msgr2=0x7fde4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.247+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 0x7fde4406d190 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fde54006b20 tx=0x7fde54008040 comp rx=0 tx=0).stop 2023-06-09T11:57:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.247+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 msgr2=0x7fde640b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.247+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fde5c004000 tx=0x7fde5c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 shutdown_connections 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde640c9260 0x7fde640b1c30 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde4406ace0 0x7fde4406d190 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde640b5370 0x7fde640aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 --2- 172.21.15.7:0/1737030101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde640c8e90 0x7fde640c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 >> 172.21.15.7:0/1737030101 conn(0x7fde64070af0 msgr2=0x7fde640b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 shutdown_connections 2023-06-09T11:57:22.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:22.248+0000 7fde68fff700 1 -- 172.21.15.7:0/1737030101 wait complete. 2023-06-09T11:57:22.252 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:22.268 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-06-09T11:57:22.269 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 216, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 216, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:stopping", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:57:08.951234+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:57:23.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: In function 'xlist::~xlist() [with T = Inode*]' thread 7f5969ffb700 time 2023-06-09T11:57:23.986081+0000 2023-06-09T11:57:23.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: 81: FAILED ceph_assert(_size == 0) 2023-06-09T11:57:23.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:23.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f598f2a9dbf] 2023-06-09T11:57:23.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:23.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:23.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:23.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:23.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: clone() 2023-06-09T11:57:23.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:*** Caught signal (Aborted) ** 2023-06-09T11:57:23.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: in thread 7f5969ffb700 thread_name:ms_dispatch 2023-06-09T11:57:23.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:23.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f598de12ce0] 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: gsignal() 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: abort() 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f598f2a9e19] 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:23.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 15: clone() 2023-06-09T11:57:23.993 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-06-09T11:57:23.987+0000 7f5969ffb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: In function 'xlist::~xlist() [with T = Inode*]' thread 7f5969ffb700 time 2023-06-09T11:57:23.986081+0000 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: 81: FAILED ceph_assert(_size == 0) 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f598f2a9dbf] 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:23.994 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:23.995 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: clone() 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:2023-06-09T11:57:23.989+0000 7f5969ffb700 -1 *** Caught signal (Aborted) ** 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: in thread 7f5969ffb700 thread_name:ms_dispatch 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f598de12ce0] 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: gsignal() 2023-06-09T11:57:23.996 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: abort() 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f598f2a9e19] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:23.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 15: clone() 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T11:57:23.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: -1> 2023-06-09T11:57:23.987+0000 7f5969ffb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: In function 'xlist::~xlist() [with T = Inode*]' thread 7f5969ffb700 time 2023-06-09T11:57:23.986081+0000 2023-06-09T11:57:24.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: 81: FAILED ceph_assert(_size == 0) 2023-06-09T11:57:24.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:24.048 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f598f2a9dbf] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:24.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: clone() 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 0> 2023-06-09T11:57:23.989+0000 7f5969ffb700 -1 *** Caught signal (Aborted) ** 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: in thread 7f5969ffb700 thread_name:ms_dispatch 2023-06-09T11:57:24.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f598de12ce0] 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: gsignal() 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: abort() 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f598f2a9e19] 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:24.051 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:24.052 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 15: clone() 2023-06-09T11:57:24.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T11:57:24.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: -1> 2023-06-09T11:57:23.987+0000 7f5969ffb700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: In function 'xlist::~xlist() [with T = Inode*]' thread 7f5969ffb700 time 2023-06-09T11:57:23.986081+0000 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3924-g03410882/rpm/el8/BUILD/ceph-18.0.0-3924-g03410882/src/include/xlist.h: 81: FAILED ceph_assert(_size == 0) 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f598f2a9dbf] 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:24.074 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:24.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: clone() 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 0> 2023-06-09T11:57:23.989+0000 7f5969ffb700 -1 *** Caught signal (Aborted) ** 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: in thread 7f5969ffb700 thread_name:ms_dispatch 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: ceph version 18.0.0-3924-g03410882 (034108821cde95ca1c56d458dfccc7ef9e0d964d) reef (dev) 2023-06-09T11:57:24.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f598de12ce0] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2: gsignal() 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 3: abort() 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f598f2a9e19] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2a9f85) [0x7f598f2a9f85] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 6: (MetaSession::~MetaSession()+0x1eb) [0x55b978d7a1cb] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 7: (std::_Sp_counted_ptr::_M_dispose()+0x16) [0x55b978d7a246] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 8: (std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x3a) [0x55b978d5ccea] 2023-06-09T11:57:24.077 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 9: (Client::handle_client_session(boost::intrusive_ptr const&)+0xf8) [0x55b978d43a38] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 10: (Client::ms_dispatch2(boost::intrusive_ptr const&)+0x555) [0x55b978d468b5] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 11: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x478) [0x7f598f522e48] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 12: (DispatchQueue::entry()+0x50f) [0x7f598f51ffef] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f598f5e62e1] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 14: /lib64/libpthread.so.0(+0x81cf) [0x7f598de081cf] 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 15: clone() 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-06-09T11:57:24.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr: 2023-06-09T11:57:24.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.472+0000 7fd9af5ff700 1 Processor -- start 2023-06-09T11:57:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.472+0000 7fd9af5ff700 1 -- start start 2023-06-09T11:57:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd9a8151510 0x7fd9a80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9a8071060 0x7fd9a8158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a80714e0 con 0x7fd9a8071060 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a81518e0 con 0x7fd9a8151510 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.473+0000 7fd9af5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81580a0 con 0x7fd9a815b380 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9a8071060 0x7fd9a8158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd9a8071060 0x7fd9a8158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48558/0 (socket says 172.21.15.7:48558) 2023-06-09T11:57:24.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 -- 172.21.15.7:0/213600317 learned_addr learned my addr 172.21.15.7:0/213600317 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:24.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9ad39b700 1 --2- 172.21.15.7:0/213600317 >> v2:172.21.15.117:3300/0 conn(0x7fd9a8151510 0x7fd9a80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9acb9a700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 -- 172.21.15.7:0/213600317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 msgr2=0x7fd9a8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:24.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 -- 172.21.15.7:0/213600317 >> v2:172.21.15.117:3300/0 conn(0x7fd9a8151510 msgr2=0x7fd9a80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 --2- 172.21.15.7:0/213600317 >> v2:172.21.15.117:3300/0 conn(0x7fd9a8151510 0x7fd9a80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.474+0000 7fd9adb9c700 1 -- 172.21.15.7:0/213600317 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80c6180 con 0x7fd9a8071060 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.475+0000 7fd9adb9c700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8071060 0x7fd9a8158870 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd9a806ff10 tx=0x7fd9a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351f8d450069debc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.475+0000 7fd99ffff700 1 -- 172.21.15.7:0/213600317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000e070 con 0x7fd9a8071060 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.475+0000 7fd99ffff700 1 -- 172.21.15.7:0/213600317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a000b910 con 0x7fd9a8071060 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.475+0000 7fd99ffff700 1 -- 172.21.15.7:0/213600317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000bcc0 con 0x7fd9a8071060 2023-06-09T11:57:24.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 -- 172.21.15.7:0/213600317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8071060 msgr2=0x7fd9a8158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8071060 0x7fd9a8158870 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd9a806ff10 tx=0x7fd9a000b330 comp rx=0 tx=0).stop 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 -- 172.21.15.7:0/213600317 shutdown_connections 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/213600317 >> v2:172.21.15.117:3300/0 conn(0x7fd9a8151510 0x7fd9a80c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8071060 0x7fd9a8158870 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/213600317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 -- 172.21.15.7:0/213600317 >> 172.21.15.7:0/213600317 conn(0x7fd9a806a4d0 msgr2=0x7fd9a806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:24.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 -- 172.21.15.7:0/213600317 shutdown_connections 2023-06-09T11:57:24.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.476+0000 7fd9af5ff700 1 -- 172.21.15.7:0/213600317 wait complete. 2023-06-09T11:57:24.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.478+0000 7fd9af5ff700 1 Processor -- start 2023-06-09T11:57:24.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.478+0000 7fd9af5ff700 1 -- start start 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a8071060 0x7fd9a814da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a814e5a0 con 0x7fd9a8151510 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81ef520 con 0x7fd9a8071060 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9af5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a81ef800 con 0x7fd9a815b380 2023-06-09T11:57:24.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9acb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9acb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48572/0 (socket says 172.21.15.7:48572) 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9acb9a700 1 -- 172.21.15.7:0/1271612933 learned_addr learned my addr 172.21.15.7:0/1271612933 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9ad39b700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a8071060 0x7fd9a814da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.479+0000 7fd9adb9c700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 -- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 msgr2=0x7fd9a8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 -- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a8071060 msgr2=0x7fd9a814da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:24.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a8071060 0x7fd9a814da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 -- 172.21.15.7:0/1271612933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009b30 con 0x7fd9a8151510 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9acb9a700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e870 tx=0x7fd99800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd998009590 con 0x7fd9a8151510 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd998013680 con 0x7fd9a8151510 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a8157770 con 0x7fd9a8151510 2023-06-09T11:57:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9980139f0 con 0x7fd9a8151510 2023-06-09T11:57:24.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.480+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a8157490 con 0x7fd9a8151510 2023-06-09T11:57:24.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.481+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a8157a50 con 0x7fd9a8151510 2023-06-09T11:57:24.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.482+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd998013b50 con 0x7fd9a8151510 2023-06-09T11:57:24.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.484+0000 7fd99dffb700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 0x7fd98c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:24.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.484+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fd998089d00 con 0x7fd9a8151510 2023-06-09T11:57:24.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.484+0000 7fd9ad39b700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 0x7fd98c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:24.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.485+0000 7fd9ad39b700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 0x7fd98c06d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009b60 tx=0x7fd9a4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.488+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd9980544b0 con 0x7fd9a8151510 2023-06-09T11:57:24.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:24.644 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:24.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.745+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9a8074cf0 con 0x7fd9a8151510 2023-06-09T11:57:24.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.746+0000 7fd99dffb700 1 -- 172.21.15.7:0/1271612933 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fd998021090 con 0x7fd9a8151510 2023-06-09T11:57:24.748 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:24.748 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 msgr2=0x7fd98c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 0x7fd98c06d190 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009b60 tx=0x7fd9a4009210 comp rx=0 tx=0).stop 2023-06-09T11:57:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 msgr2=0x7fd9a8159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd99800e870 tx=0x7fd99800eb80 comp rx=0 tx=0).stop 2023-06-09T11:57:24.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 shutdown_connections 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd98c06ace0 0x7fd98c06d190 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd9a815b380 0x7fd9a8159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd9a8071060 0x7fd9a814da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 --2- 172.21.15.7:0/1271612933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd9a8151510 0x7fd9a8159120 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 >> 172.21.15.7:0/1271612933 conn(0x7fd9a806a4d0 msgr2=0x7fd9a814adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 shutdown_connections 2023-06-09T11:57:24.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:24.748+0000 7fd9af5ff700 1 -- 172.21.15.7:0/1271612933 wait complete. 2023-06-09T11:57:24.753 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:24.769 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:25.964 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 4% (8/181) Updating files: 5% (10/181) Updating files: 6% (11/181) Updating files: 7% (13/181) Updating files: 8% (15/181) Updating files: 9% (17/181) Updating files: 10% (19/181) Updating files: 11% (20/181) Updating files: 12% (22/181) Updating files: 13% (24/181) Updating files: 14% (26/181) Updating files: 15% (28/181) Updating files: 16% (29/181) Updating files: 16% (30/181) Updating files: 17% (31/181) Updating files: 18% (33/181) Updating files: 19% (35/181) Updating files: 20% (37/181) error: unable to create file src/ceph_mds.cc: Transport endpoint is not connected 2023-06-09T11:57:25.965 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 20% (38/181) error: unable to create file src/ceph_osd.cc: Transport endpoint is not connected 2023-06-09T11:57:25.965 INFO:tasks.workunit.client.0.smithi007.stderr:Updating files: 21% (39/181) fatal: cannot create directory at 'src/client': Transport endpoint is not connected 2023-06-09T11:57:25.966 INFO:tasks.workunit.client.0.smithi007.stderr:warning: unable to unlink '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ceph/.git/index.lock': Transport endpoint is not connected 2023-06-09T11:57:25.968 DEBUG:teuthology.orchestra.run:got remote process result: 128 2023-06-09T11:57:25.969 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0... 2023-06-09T11:57:25.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-06-09T11:57:26.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:26.895 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi007 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-06-09T11:57:26.961 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4a265fdf331448508ad0c0be8859880f Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 84, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_034108821cde95ca1c56d458dfccc7ef9e0d964d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi007 with status 128: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh' 2023-06-09T11:57:26.964 DEBUG:teuthology.run_tasks:Unwinding manager mds_thrash 2023-06-09T11:57:26.973 INFO:tasks.mds_thrash:joining mds_thrasher 2023-06-09T11:57:26.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.985+0000 7fc0541ff700 1 Processor -- start 2023-06-09T11:57:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.986+0000 7fc0541ff700 1 -- start start 2023-06-09T11:57:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc0541ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:26.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc0541ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc0541ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc04c15a500 con 0x7fc04c0c9f40 2023-06-09T11:57:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc0541ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc04c0c5f10 con 0x7fc04c153880 2023-06-09T11:57:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc0541ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c15da00 con 0x7fc04c15b4a0 2023-06-09T11:57:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc051f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:26.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc051f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48586/0 (socket says 172.21.15.7:48586) 2023-06-09T11:57:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.987+0000 7fc051f9b700 1 -- 172.21.15.7:0/3291823007 learned_addr learned my addr 172.21.15.7:0/3291823007 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:27.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc05279c700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc051f9b700 1 -- 172.21.15.7:0/3291823007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 msgr2=0x7fc04c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc051f9b700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc051f9b700 1 -- 172.21.15.7:0/3291823007 >> v1:172.21.15.117:6789/0 conn(0x7fc04c0c9f40 legacy=0x7fc04c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc051f9b700 1 -- 172.21.15.7:0/3291823007 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c1596e0 con 0x7fc04c153880 2023-06-09T11:57:27.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc051f9b700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc0480098f0 tx=0x7fc048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b84caff48447ec5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc050f99700 1 -- 172.21.15.7:0/3291823007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800b8a0 con 0x7fc04c153880 2023-06-09T11:57:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc050f99700 1 -- 172.21.15.7:0/3291823007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800ba00 con 0x7fc04c153880 2023-06-09T11:57:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.988+0000 7fc050f99700 1 -- 172.21.15.7:0/3291823007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04800bd90 con 0x7fc04c153880 2023-06-09T11:57:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 -- 172.21.15.7:0/3291823007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 msgr2=0x7fc04c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fc0480098f0 tx=0x7fc048009c00 comp rx=0 tx=0).stop 2023-06-09T11:57:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 -- 172.21.15.7:0/3291823007 shutdown_connections 2023-06-09T11:57:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 --2- 172.21.15.7:0/3291823007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c15d4c0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 -- 172.21.15.7:0/3291823007 >> 172.21.15.7:0/3291823007 conn(0x7fc04c0737f0 msgr2=0x7fc04c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:27.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 -- 172.21.15.7:0/3291823007 shutdown_connections 2023-06-09T11:57:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.989+0000 7fc0541ff700 1 -- 172.21.15.7:0/3291823007 wait complete. 2023-06-09T11:57:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.991+0000 7fc0541ff700 1 Processor -- start 2023-06-09T11:57:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.991+0000 7fc0541ff700 1 -- start start 2023-06-09T11:57:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c0c9f40 0x7fc04c14b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:27.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c1efc90 con 0x7fc04c153880 2023-06-09T11:57:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c14b5c0 con 0x7fc04c0c9f40 2023-06-09T11:57:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc0541ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04c1f00b0 con 0x7fc04c15b4a0 2023-06-09T11:57:27.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48594/0 (socket says 172.21.15.7:48594) 2023-06-09T11:57:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 -- 172.21.15.7:0/4014069631 learned_addr learned my addr 172.21.15.7:0/4014069631 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05279c700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc051f9b700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c0c9f40 0x7fc04c14b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:27.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 -- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 msgr2=0x7fc04c0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 -- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c0c9f40 msgr2=0x7fc04c14b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c0c9f40 0x7fc04c14b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.992+0000 7fc05179a700 1 -- 172.21.15.7:0/4014069631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc048009550 con 0x7fc04c153880 2023-06-09T11:57:27.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc05179a700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c4e0 tx=0x7fc03c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c0052d0 con 0x7fc04c153880 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c00ce90 con 0x7fc04c153880 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c003680 con 0x7fc04c153880 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04c159c60 con 0x7fc04c153880 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.993+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04c159980 con 0x7fc04c153880 2023-06-09T11:57:27.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.994+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04c159f40 con 0x7fc04c153880 2023-06-09T11:57:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.995+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc03c0037e0 con 0x7fc04c153880 2023-06-09T11:57:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.997+0000 7fc0427fc700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.997+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fc03c088df0 con 0x7fc04c153880 2023-06-09T11:57:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.997+0000 7fc051f9b700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:27.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:26.998+0000 7fc051f9b700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 0x7fc03006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc048009c40 tx=0x7fc048008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:27.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.002+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc03c053690 con 0x7fc04c153880 2023-06-09T11:57:27.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.260+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc04c15a500 con 0x7fc04c153880 2023-06-09T11:57:27.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.261+0000 7fc0427fc700 1 -- 172.21.15.7:0/4014069631 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fc04c15a500 con 0x7fc04c153880 2023-06-09T11:57:27.264 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:27.264 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 msgr2=0x7fc03006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 0x7fc03006d190 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc048009c40 tx=0x7fc048008030 comp rx=0 tx=0).stop 2023-06-09T11:57:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 msgr2=0x7fc04c0ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc03c00c4e0 tx=0x7fc03c00c7f0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 shutdown_connections 2023-06-09T11:57:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc03006ace0 0x7fc03006d190 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc04c15b4a0 0x7fc04c0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc04c0c9f40 0x7fc04c14b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 --2- 172.21.15.7:0/4014069631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc04c153880 0x7fc04c0ce850 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.263+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 >> 172.21.15.7:0/4014069631 conn(0x7fc04c0737f0 msgr2=0x7fc04c14dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.264+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 shutdown_connections 2023-06-09T11:57:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:27.264+0000 7fc0541ff700 1 -- 172.21.15.7:0/4014069631 wait complete. 2023-06-09T11:57:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:27.285 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:29.286 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:29.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.495+0000 7fa184bff700 1 Processor -- start 2023-06-09T11:57:29.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.496+0000 7fa184bff700 1 -- start start 2023-06-09T11:57:29.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa184bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa18015b4a0 0x7fa18015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa184bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 0x7fa1800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa184bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18015a500 con 0x7fa180153880 2023-06-09T11:57:29.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa184bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1800c5f10 con 0x7fa18015b4a0 2023-06-09T11:57:29.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa184bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18015da00 con 0x7fa1800c9f40 2023-06-09T11:57:29.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa18015b4a0 0x7fa18015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa18015b4a0 0x7fa18015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48602/0 (socket says 172.21.15.7:48602) 2023-06-09T11:57:29.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 -- 172.21.15.7:0/3679840503 learned_addr learned my addr 172.21.15.7:0/3679840503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:29.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17ed9d700 1 --2- 172.21.15.7:0/3679840503 >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 0x7fa1800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 -- 172.21.15.7:0/3679840503 >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 msgr2=0x7fa1800c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:29.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 --2- 172.21.15.7:0/3679840503 >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 0x7fa1800c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 -- 172.21.15.7:0/3679840503 >> v1:172.21.15.117:6789/0 conn(0x7fa180153880 legacy=0x7fa18015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:29.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.497+0000 7fa17dd9b700 1 -- 172.21.15.7:0/3679840503 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1801596e0 con 0x7fa18015b4a0 2023-06-09T11:57:29.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa17ed9d700 1 --2- 172.21.15.7:0/3679840503 >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 0x7fa1800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:29.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa17dd9b700 1 --2- 172.21.15.7:0/3679840503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa18015d4c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa1800749f0 tx=0x7fa174009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34eb68c3c2aad83 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa17d59a700 1 -- 172.21.15.7:0/3679840503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400b870 con 0x7fa18015b4a0 2023-06-09T11:57:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa17d59a700 1 -- 172.21.15.7:0/3679840503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17400b9d0 con 0x7fa18015b4a0 2023-06-09T11:57:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa17d59a700 1 -- 172.21.15.7:0/3679840503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17400bd60 con 0x7fa18015b4a0 2023-06-09T11:57:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa184bff700 1 -- 172.21.15.7:0/3679840503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 msgr2=0x7fa18015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.498+0000 7fa184bff700 1 --2- 172.21.15.7:0/3679840503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa18015d4c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fa1800749f0 tx=0x7fa174009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 -- 172.21.15.7:0/3679840503 shutdown_connections 2023-06-09T11:57:29.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 --2- 172.21.15.7:0/3679840503 >> v2:172.21.15.117:3300/0 conn(0x7fa1800c9f40 0x7fa1800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 --2- 172.21.15.7:0/3679840503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa18015d4c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 -- 172.21.15.7:0/3679840503 >> 172.21.15.7:0/3679840503 conn(0x7fa1800737f0 msgr2=0x7fa1800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:29.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 -- 172.21.15.7:0/3679840503 shutdown_connections 2023-06-09T11:57:29.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.499+0000 7fa184bff700 1 -- 172.21.15.7:0/3679840503 wait complete. 2023-06-09T11:57:29.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.501+0000 7fa184bff700 1 Processor -- start 2023-06-09T11:57:29.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.501+0000 7fa184bff700 1 -- start start 2023-06-09T11:57:29.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.502+0000 7fa184bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 0x7fa180146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.502+0000 7fa184bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 0x7fa1800caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa184bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa184bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1800cca40 con 0x7fa18015b4a0 2023-06-09T11:57:29.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa184bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801ef4b0 con 0x7fa180153880 2023-06-09T11:57:29.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa184bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1801efd10 con 0x7fa1800c9f40 2023-06-09T11:57:29.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48614/0 (socket says 172.21.15.7:48614) 2023-06-09T11:57:29.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 -- 172.21.15.7:0/3926425329 learned_addr learned my addr 172.21.15.7:0/3926425329 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:29.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17dd9b700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 0x7fa1800caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 -- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 msgr2=0x7fa180146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:29.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17e59c700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 0x7fa180146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 0x7fa180146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 -- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 msgr2=0x7fa1800caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:29.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 0x7fa1800caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:29.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17ed9d700 1 -- 172.21.15.7:0/3926425329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa174009550 con 0x7fa18015b4a0 2023-06-09T11:57:29.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17dd9b700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 0x7fa1800caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:29.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.503+0000 7fa17e59c700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 0x7fa180146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:29.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa17ed9d700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa178007f20 tx=0x7fa17800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:29.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17800de90 con 0x7fa18015b4a0 2023-06-09T11:57:29.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17800f040 con 0x7fa18015b4a0 2023-06-09T11:57:29.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa180159c60 con 0x7fa18015b4a0 2023-06-09T11:57:29.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178003c60 con 0x7fa18015b4a0 2023-06-09T11:57:29.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.504+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa180159980 con 0x7fa18015b4a0 2023-06-09T11:57:29.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.505+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180159f40 con 0x7fa18015b4a0 2023-06-09T11:57:29.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.506+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa178004030 con 0x7fa18015b4a0 2023-06-09T11:57:29.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.507+0000 7fa16effd700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 0x7fa16006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:29.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.507+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fa178089ff0 con 0x7fa18015b4a0 2023-06-09T11:57:29.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.507+0000 7fa17e59c700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 0x7fa16006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:29.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.508+0000 7fa17e59c700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 0x7fa16006d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa170000f60 tx=0x7fa170009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:29.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.511+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa1780bc2d0 con 0x7fa18015b4a0 2023-06-09T11:57:29.783 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.781+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa180074cf0 con 0x7fa18015b4a0 2023-06-09T11:57:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:29.784 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.781+0000 7fa16effd700 1 -- 172.21.15.7:0/3926425329 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fa17801a070 con 0x7fa18015b4a0 2023-06-09T11:57:30.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 msgr2=0x7fa16006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 0x7fa16006d190 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa170000f60 tx=0x7fa170009040 comp rx=0 tx=0).stop 2023-06-09T11:57:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 msgr2=0x7fa180145970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:30.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fa178007f20 tx=0x7fa17800d3b0 comp rx=0 tx=0).stop 2023-06-09T11:57:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 shutdown_connections 2023-06-09T11:57:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa18015b4a0 0x7fa180145970 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:30.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa16006ace0 0x7fa16006d190 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1800c9f40 0x7fa180146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:30.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 --2- 172.21.15.7:0/3926425329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa180153880 0x7fa1800caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 >> 172.21.15.7:0/3926425329 conn(0x7fa1800737f0 msgr2=0x7fa1801e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 shutdown_connections 2023-06-09T11:57:30.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:29.784+0000 7fa184bff700 1 -- 172.21.15.7:0/3926425329 wait complete. 2023-06-09T11:57:30.106 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:30.123 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:30.170 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi007.stderr:daemon-helper: command crashed with signal 6 2023-06-09T11:57:32.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.332+0000 7f694c1ff700 1 Processor -- start 2023-06-09T11:57:32.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.333+0000 7f694c1ff700 1 -- start start 2023-06-09T11:57:32.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694c1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69440b52b0 0x7f69440cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694c1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69440cb1c0 0x7f69440cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694c1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f694415ccd0 con 0x7f69440cadf0 2023-06-09T11:57:32.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694c1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440cfac0 con 0x7f69440cb1c0 2023-06-09T11:57:32.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694c1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440cfce0 con 0x7f69440b52b0 2023-06-09T11:57:32.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69440cb1c0 0x7f69440cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69440cb1c0 0x7f69440cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48630/0 (socket says 172.21.15.7:48630) 2023-06-09T11:57:32.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 -- 172.21.15.7:0/3801389569 learned_addr learned my addr 172.21.15.7:0/3801389569 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:32.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694979a700 1 --2- 172.21.15.7:0/3801389569 >> v2:172.21.15.117:3300/0 conn(0x7f69440b52b0 0x7f69440cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 -- 172.21.15.7:0/3801389569 >> v2:172.21.15.117:3300/0 conn(0x7f69440b52b0 msgr2=0x7f69440cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:32.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 --2- 172.21.15.7:0/3801389569 >> v2:172.21.15.117:3300/0 conn(0x7f69440b52b0 0x7f69440cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.334+0000 7f694a79c700 1 -- 172.21.15.7:0/3801389569 >> v1:172.21.15.117:6789/0 conn(0x7f69440cadf0 legacy=0x7f69440c8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.335+0000 7f694a79c700 1 -- 172.21.15.7:0/3801389569 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694415cf20 con 0x7f69440cb1c0 2023-06-09T11:57:32.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.335+0000 7f694a79c700 1 --2- 172.21.15.7:0/3801389569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440cf580 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f694406bce0 tx=0x7f693c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5253f642c34ad36a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:32.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.335+0000 7f6948f99700 1 -- 172.21.15.7:0/3801389569 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00e070 con 0x7f69440cb1c0 2023-06-09T11:57:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.335+0000 7f6948f99700 1 -- 172.21.15.7:0/3801389569 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c00b910 con 0x7f69440cb1c0 2023-06-09T11:57:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.335+0000 7f6948f99700 1 -- 172.21.15.7:0/3801389569 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00bcc0 con 0x7f69440cb1c0 2023-06-09T11:57:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 -- 172.21.15.7:0/3801389569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 msgr2=0x7f69440cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 --2- 172.21.15.7:0/3801389569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440cf580 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f694406bce0 tx=0x7f693c00b330 comp rx=0 tx=0).stop 2023-06-09T11:57:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 -- 172.21.15.7:0/3801389569 shutdown_connections 2023-06-09T11:57:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 --2- 172.21.15.7:0/3801389569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440cf580 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 --2- 172.21.15.7:0/3801389569 >> v2:172.21.15.117:3300/0 conn(0x7f69440b52b0 0x7f69440cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 -- 172.21.15.7:0/3801389569 >> 172.21.15.7:0/3801389569 conn(0x7f6944070af0 msgr2=0x7f6944070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 -- 172.21.15.7:0/3801389569 shutdown_connections 2023-06-09T11:57:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.336+0000 7f694c1ff700 1 -- 172.21.15.7:0/3801389569 wait complete. 2023-06-09T11:57:32.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.337+0000 7f694c1ff700 1 Processor -- start 2023-06-09T11:57:32.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.338+0000 7f694c1ff700 1 -- start start 2023-06-09T11:57:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.338+0000 7f694c1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69440b52b0 0x7f69440c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69440cadf0 0x7f694414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694c1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694c1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6944152ec0 con 0x7f69440cb1c0 2023-06-09T11:57:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c32a0 con 0x7f69440cadf0 2023-06-09T11:57:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694c1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c3580 con 0x7f69440b52b0 2023-06-09T11:57:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48642/0 (socket says 172.21.15.7:48642) 2023-06-09T11:57:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 -- 172.21.15.7:0/1451471873 learned_addr learned my addr 172.21.15.7:0/1451471873 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694979a700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69440cadf0 0x7f694414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f6949f9b700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69440b52b0 0x7f69440c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 -- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69440b52b0 msgr2=0x7f69440c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69440b52b0 0x7f69440c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 -- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69440cadf0 msgr2=0x7f694414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69440cadf0 0x7f694414f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 -- 172.21.15.7:0/1451471873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693c009b30 con 0x7f69440cb1c0 2023-06-09T11:57:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.339+0000 7f694a79c700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f69440cd170 tx=0x7f693c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.340+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c004930 con 0x7f69440cb1c0 2023-06-09T11:57:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.340+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c0092e0 con 0x7f69440cb1c0 2023-06-09T11:57:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.340+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c01b650 con 0x7f69440cb1c0 2023-06-09T11:57:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.340+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694415d4e0 con 0x7f69440cb1c0 2023-06-09T11:57:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.340+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694415d200 con 0x7f69440cb1c0 2023-06-09T11:57:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.341+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694415d7c0 con 0x7f69440cb1c0 2023-06-09T11:57:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.342+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f693c01b7b0 con 0x7f69440cb1c0 2023-06-09T11:57:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.343+0000 7f693a7fc700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 0x7f692806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.343+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f693c08b580 con 0x7f69440cb1c0 2023-06-09T11:57:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.344+0000 7f6949f9b700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 0x7f692806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.344+0000 7f6949f9b700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 0x7f692806d1c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6940000f60 tx=0x7f6940009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.348+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f693c055dc0 con 0x7f69440cb1c0 2023-06-09T11:57:32.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.599+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f694408cba0 con 0x7f69440cb1c0 2023-06-09T11:57:32.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.600+0000 7f693a7fc700 1 -- 172.21.15.7:0/1451471873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f693c0593e0 con 0x7f69440cb1c0 2023-06-09T11:57:32.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:32.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:32.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 msgr2=0x7f692806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 0x7f692806d1c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6940000f60 tx=0x7f6940009040 comp rx=0 tx=0).stop 2023-06-09T11:57:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 msgr2=0x7f69440c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f69440cd170 tx=0x7f693c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 shutdown_connections 2023-06-09T11:57:32.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69440cb1c0 0x7f69440c4f30 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692806ad10 0x7f692806d1c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69440b52b0 0x7f69440c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 --2- 172.21.15.7:0/1451471873 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69440cadf0 0x7f694414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 >> 172.21.15.7:0/1451471873 conn(0x7f6944070af0 msgr2=0x7f69440ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:32.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 shutdown_connections 2023-06-09T11:57:32.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:32.602+0000 7f694c1ff700 1 -- 172.21.15.7:0/1451471873 wait complete. 2023-06-09T11:57:32.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:32.622 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:34.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.832+0000 7f83effff700 1 Processor -- start 2023-06-09T11:57:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.832+0000 7f83effff700 1 -- start start 2023-06-09T11:57:34.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.833+0000 7f83effff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e8153880 0x7f83e815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.833+0000 7f83effff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83e80c9f40 0x7f83e815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.833+0000 7f83effff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e815a500 con 0x7f83e815b2d0 2023-06-09T11:57:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.833+0000 7f83effff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80c5f10 con 0x7f83e80c9f40 2023-06-09T11:57:34.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.833+0000 7f83effff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e815da00 con 0x7f83e8153880 2023-06-09T11:57:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83e80c9f40 0x7f83e815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83edd9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e8153880 0x7f83e815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f83e80c9f40 0x7f83e815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:33220/0 (socket says 172.21.15.7:33220) 2023-06-09T11:57:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ee59c700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f83e815b2d0 0x7f83e80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:51086/0 (socket says 172.21.15.7:51086) 2023-06-09T11:57:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 -- 172.21.15.7:0/3316822466 learned_addr learned my addr 172.21.15.7:0/3316822466 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 -- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e8153880 msgr2=0x7f83e815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:34.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 --2- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e8153880 0x7f83e815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 -- 172.21.15.7:0/3316822466 >> v1:172.21.15.117:6789/0 conn(0x7f83e815b2d0 legacy=0x7f83e80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.834+0000 7f83ed59a700 1 -- 172.21.15.7:0/3316822466 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e815b750 con 0x7f83e80c9f40 2023-06-09T11:57:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83ed59a700 1 --2- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e80c9f40 0x7f83e815ad90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098f0 tx=0x7f83d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ea62c8d64d6b116 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83ecd99700 1 -- 172.21.15.7:0/3316822466 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8011830 con 0x7f83e80c9f40 2023-06-09T11:57:34.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83ecd99700 1 -- 172.21.15.7:0/3316822466 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8011990 con 0x7f83e80c9f40 2023-06-09T11:57:34.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83ecd99700 1 -- 172.21.15.7:0/3316822466 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8011d20 con 0x7f83e80c9f40 2023-06-09T11:57:34.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 -- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e80c9f40 msgr2=0x7f83e815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 --2- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e80c9f40 0x7f83e815ad90 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f83d80098f0 tx=0x7f83d8009c00 comp rx=0 tx=0).stop 2023-06-09T11:57:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 -- 172.21.15.7:0/3316822466 shutdown_connections 2023-06-09T11:57:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 --2- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e8153880 0x7f83e815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 --2- 172.21.15.7:0/3316822466 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e80c9f40 0x7f83e815ad90 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.835+0000 7f83effff700 1 -- 172.21.15.7:0/3316822466 >> 172.21.15.7:0/3316822466 conn(0x7f83e80737f0 msgr2=0x7f83e80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.836+0000 7f83effff700 1 -- 172.21.15.7:0/3316822466 shutdown_connections 2023-06-09T11:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.836+0000 7f83effff700 1 -- 172.21.15.7:0/3316822466 wait complete. 2023-06-09T11:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.837+0000 7f83effff700 1 Processor -- start 2023-06-09T11:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.837+0000 7f83effff700 1 -- start start 2023-06-09T11:57:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e8153880 0x7f83e80cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e815b2d0 0x7f83e81ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8154540 con 0x7f83e80c9f40 2023-06-09T11:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e81f28b0 con 0x7f83e8153880 2023-06-09T11:57:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83effff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80b53f0 con 0x7f83e815b2d0 2023-06-09T11:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.838+0000 7f83edd9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40078/0 (socket says 172.21.15.7:40078) 2023-06-09T11:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 -- 172.21.15.7:0/1613994620 learned_addr learned my addr 172.21.15.7:0/1613994620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83ee59c700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e815b2d0 0x7f83e81ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83ed59a700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e8153880 0x7f83e80cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 -- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e815b2d0 msgr2=0x7f83e81ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e815b2d0 0x7f83e81ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 -- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e8153880 msgr2=0x7f83e80cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e8153880 0x7f83e80cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 -- 172.21.15.7:0/1613994620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d8009550 con 0x7f83e80c9f40 2023-06-09T11:57:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.839+0000 7f83edd9b700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e9b0 tx=0x7f83e400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.840+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4009710 con 0x7f83e80c9f40 2023-06-09T11:57:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.840+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400ee50 con 0x7f83e80c9f40 2023-06-09T11:57:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.840+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400ca50 con 0x7f83e80c9f40 2023-06-09T11:57:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.840+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e8159980 con 0x7f83e80c9f40 2023-06-09T11:57:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.840+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e81596e0 con 0x7f83e80c9f40 2023-06-09T11:57:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.841+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e8159c60 con 0x7f83e80c9f40 2023-06-09T11:57:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.842+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83e400cbb0 con 0x7f83e80c9f40 2023-06-09T11:57:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.843+0000 7f83de7fc700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 0x7f83cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.843+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f83e4089d10 con 0x7f83e80c9f40 2023-06-09T11:57:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.843+0000 7f83ed59a700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 0x7f83cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.844+0000 7f83ed59a700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 0x7f83cc06d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009c40 tx=0x7f83d8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:34.848+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f83e408f020 con 0x7f83e80c9f40 2023-06-09T11:57:35.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.101+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83e8074cf0 con 0x7f83e80c9f40 2023-06-09T11:57:35.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.102+0000 7f83de7fc700 1 -- 172.21.15.7:0/1613994620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f83e4021090 con 0x7f83e80c9f40 2023-06-09T11:57:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:35.104 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 msgr2=0x7f83cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:35.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 0x7f83cc06d190 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009c40 tx=0x7f83d8008030 comp rx=0 tx=0).stop 2023-06-09T11:57:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 msgr2=0x7f83e80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e9b0 tx=0x7f83e400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:57:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 shutdown_connections 2023-06-09T11:57:35.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f83cc06ace0 0x7f83cc06d190 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f83e815b2d0 0x7f83e81ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83e80c9f40 0x7f83e80b4eb0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 --2- 172.21.15.7:0/1613994620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f83e8153880 0x7f83e80cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 >> 172.21.15.7:0/1613994620 conn(0x7f83e80737f0 msgr2=0x7f83e80720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:35.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 shutdown_connections 2023-06-09T11:57:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:35.104+0000 7f83effff700 1 -- 172.21.15.7:0/1613994620 wait complete. 2023-06-09T11:57:35.109 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:35.125 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-06-09T11:57:35.126 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 216, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 216, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:stopping", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:57:08.951234+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:57:37.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:37.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.336+0000 7f9c1cbff700 1 Processor -- start 2023-06-09T11:57:37.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.336+0000 7f9c1cbff700 1 -- start start 2023-06-09T11:57:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.337+0000 7f9c1cbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c181539b0 0x7f9c18159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.337+0000 7f9c1cbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 0x7f9c180c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.337+0000 7f9c1cbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1815a420 con 0x7f9c181535e0 2023-06-09T11:57:37.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.337+0000 7f9c1cbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c180ca670 con 0x7f9c181539b0 2023-06-09T11:57:37.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.337+0000 7f9c1cbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c180c5a00 con 0x7f9c180c4ce0 2023-06-09T11:57:37.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c181539b0 0x7f9c18159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9c181539b0 0x7f9c18159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40080/0 (socket says 172.21.15.7:40080) 2023-06-09T11:57:37.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 -- 172.21.15.7:0/3055265906 learned_addr learned my addr 172.21.15.7:0/3055265906 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c16d9d700 1 --2- 172.21.15.7:0/3055265906 >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 0x7f9c180c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 -- 172.21.15.7:0/3055265906 >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 msgr2=0x7f9c180c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 --2- 172.21.15.7:0/3055265906 >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 0x7f9c180c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 -- 172.21.15.7:0/3055265906 >> v1:172.21.15.7:6789/0 conn(0x7f9c181535e0 legacy=0x7f9c180c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 -- 172.21.15.7:0/3055265906 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1806b460 con 0x7f9c181539b0 2023-06-09T11:57:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c16d9d700 1 --2- 172.21.15.7:0/3055265906 >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 0x7f9c180c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.338+0000 7f9c15d9b700 1 --2- 172.21.15.7:0/3055265906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c181539b0 0x7f9c18159ee0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9c180749f0 tx=0x7f9c0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc395c4bc309c45 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1559a700 1 -- 172.21.15.7:0/3055265906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00b870 con 0x7f9c181539b0 2023-06-09T11:57:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1559a700 1 -- 172.21.15.7:0/3055265906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c00b9d0 con 0x7f9c181539b0 2023-06-09T11:57:37.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1559a700 1 -- 172.21.15.7:0/3055265906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c00bd60 con 0x7f9c181539b0 2023-06-09T11:57:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 -- 172.21.15.7:0/3055265906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c181539b0 msgr2=0x7f9c18159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/3055265906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c181539b0 0x7f9c18159ee0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f9c180749f0 tx=0x7f9c0c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 -- 172.21.15.7:0/3055265906 shutdown_connections 2023-06-09T11:57:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/3055265906 >> v2:172.21.15.117:3300/0 conn(0x7f9c180c4ce0 0x7f9c180c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/3055265906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c181539b0 0x7f9c18159ee0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 -- 172.21.15.7:0/3055265906 >> 172.21.15.7:0/3055265906 conn(0x7f9c180737f0 msgr2=0x7f9c18072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 -- 172.21.15.7:0/3055265906 shutdown_connections 2023-06-09T11:57:37.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.339+0000 7f9c1cbff700 1 -- 172.21.15.7:0/3055265906 wait complete. 2023-06-09T11:57:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.342+0000 7f9c1cbff700 1 Processor -- start 2023-06-09T11:57:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.342+0000 7f9c1cbff700 1 -- start start 2023-06-09T11:57:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c181535e0 0x7f9c181e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 0x7f9c180b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180cdeb0 con 0x7f9c180c4ce0 2023-06-09T11:57:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c181efe70 con 0x7f9c181535e0 2023-06-09T11:57:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c181ef010 con 0x7f9c181539b0 2023-06-09T11:57:37.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40094/0 (socket says 172.21.15.7:40094) 2023-06-09T11:57:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 -- 172.21.15.7:0/4210249645 learned_addr learned my addr 172.21.15.7:0/4210249645 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c15d9b700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c181535e0 0x7f9c181e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c16d9d700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 0x7f9c180b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 -- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 msgr2=0x7f9c180b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 0x7f9c180b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 -- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c181535e0 msgr2=0x7f9c181e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c181535e0 0x7f9c181e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c1659c700 1 -- 172.21.15.7:0/4210249645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c009550 con 0x7f9c180c4ce0 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.343+0000 7f9c16d9d700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 0x7f9c180b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:37.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c1659c700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4e0 tx=0x7f9c0800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:37.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080052d0 con 0x7f9c180c4ce0 2023-06-09T11:57:37.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c08005430 con 0x7f9c180c4ce0 2023-06-09T11:57:37.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1806a850 con 0x7f9c180c4ce0 2023-06-09T11:57:37.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080038c0 con 0x7f9c180c4ce0 2023-06-09T11:57:37.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.344+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1806a570 con 0x7f9c180c4ce0 2023-06-09T11:57:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.345+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1806ab30 con 0x7f9c180c4ce0 2023-06-09T11:57:37.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.346+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c08003a20 con 0x7f9c180c4ce0 2023-06-09T11:57:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.347+0000 7f9c06ffd700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 0x7f9bf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.347+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f9c08089060 con 0x7f9c180c4ce0 2023-06-09T11:57:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.347+0000 7f9c15d9b700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 0x7f9bf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:37.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.348+0000 7f9c15d9b700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 0x7f9bf806d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c18153da0 tx=0x7f9c0c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:37.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.351+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9c080bb2d0 con 0x7f9c180c4ce0 2023-06-09T11:57:37.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.604+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c1815a420 con 0x7f9c180c4ce0 2023-06-09T11:57:37.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.605+0000 7f9c06ffd700 1 -- 172.21.15.7:0/4210249645 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f9c1815a420 con 0x7f9c180c4ce0 2023-06-09T11:57:37.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:37.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 msgr2=0x7f9bf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 0x7f9bf806d190 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c18153da0 tx=0x7f9c0c008030 comp rx=0 tx=0).stop 2023-06-09T11:57:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 msgr2=0x7f9c180cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c4e0 tx=0x7f9c0800c7f0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 shutdown_connections 2023-06-09T11:57:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bf806ace0 0x7f9bf806d190 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9c181539b0 0x7f9c180b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c180c4ce0 0x7f9c180cd3c0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 --2- 172.21.15.7:0/4210249645 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9c181535e0 0x7f9c181e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 >> 172.21.15.7:0/4210249645 conn(0x7f9c180737f0 msgr2=0x7f9c18158610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 shutdown_connections 2023-06-09T11:57:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:37.607+0000 7f9c1cbff700 1 -- 172.21.15.7:0/4210249645 wait complete. 2023-06-09T11:57:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:37.628 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:39.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.839+0000 7f89a87ff700 1 Processor -- start 2023-06-09T11:57:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.839+0000 7f89a87ff700 1 -- start start 2023-06-09T11:57:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 0x7f89a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89a00c9f40 0x7f89a015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a015b2d0 0x7f89a00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a00c5f10 con 0x7f89a00c9f40 2023-06-09T11:57:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a015a500 con 0x7f89a0153880 2023-06-09T11:57:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a015da00 con 0x7f89a015b2d0 2023-06-09T11:57:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a5d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89a00c9f40 0x7f89a015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a5d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f89a00c9f40 0x7f89a015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40096/0 (socket says 172.21.15.7:40096) 2023-06-09T11:57:39.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a5d9a700 1 -- 172.21.15.7:0/491230462 learned_addr learned my addr 172.21.15.7:0/491230462 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a659b700 1 --2- 172.21.15.7:0/491230462 >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 0x7f89a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.840+0000 7f89a6d9c700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a015b2d0 0x7f89a00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 -- 172.21.15.7:0/491230462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a015b2d0 msgr2=0x7f89a00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a015b2d0 0x7f89a00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 -- 172.21.15.7:0/491230462 >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 msgr2=0x7f89a015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 --2- 172.21.15.7:0/491230462 >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 0x7f89a015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 -- 172.21.15.7:0/491230462 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a01596e0 con 0x7f89a00c9f40 2023-06-09T11:57:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a659b700 1 --2- 172.21.15.7:0/491230462 >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 0x7f89a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5d9a700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a00c9f40 0x7f89a015ad90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f89a008d340 tx=0x7f8990009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf204b82c618e37 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5599700 1 -- 172.21.15.7:0/491230462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000b870 con 0x7f89a00c9f40 2023-06-09T11:57:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5599700 1 -- 172.21.15.7:0/491230462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000b9d0 con 0x7f89a00c9f40 2023-06-09T11:57:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.841+0000 7f89a5599700 1 -- 172.21.15.7:0/491230462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899000bd60 con 0x7f89a00c9f40 2023-06-09T11:57:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 -- 172.21.15.7:0/491230462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a00c9f40 msgr2=0x7f89a015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a00c9f40 0x7f89a015ad90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f89a008d340 tx=0x7f8990009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 -- 172.21.15.7:0/491230462 shutdown_connections 2023-06-09T11:57:39.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 --2- 172.21.15.7:0/491230462 >> v2:172.21.15.117:3300/0 conn(0x7f89a0153880 0x7f89a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a015b2d0 0x7f89a00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 --2- 172.21.15.7:0/491230462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a00c9f40 0x7f89a015ad90 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 -- 172.21.15.7:0/491230462 >> 172.21.15.7:0/491230462 conn(0x7f89a00737f0 msgr2=0x7f89a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 -- 172.21.15.7:0/491230462 shutdown_connections 2023-06-09T11:57:39.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.842+0000 7f89a87ff700 1 -- 172.21.15.7:0/491230462 wait complete. 2023-06-09T11:57:39.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.844+0000 7f89a87ff700 1 Processor -- start 2023-06-09T11:57:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.844+0000 7f89a87ff700 1 -- start start 2023-06-09T11:57:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a00c9f40 0x7f89a01523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 0x7f89a0151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01594a0 con 0x7f89a015b2d0 2023-06-09T11:57:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00c62c0 con 0x7f89a0153880 2023-06-09T11:57:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a87ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a0157500 con 0x7f89a00c9f40 2023-06-09T11:57:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40104/0 (socket says 172.21.15.7:40104) 2023-06-09T11:57:39.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 -- 172.21.15.7:0/1527026503 learned_addr learned my addr 172.21.15.7:0/1527026503 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:39.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a5d9a700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 0x7f89a0151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a659b700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a00c9f40 0x7f89a01523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 -- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a00c9f40 msgr2=0x7f89a01523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:39.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a00c9f40 0x7f89a01523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 -- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 msgr2=0x7f89a0151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 0x7f89a0151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a6d9c700 1 -- 172.21.15.7:0/1527026503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990009550 con 0x7f89a015b2d0 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.845+0000 7f89a5d9a700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 0x7f89a0151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f89a6d9c700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8998007e60 tx=0x7f899800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:39.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800ede0 con 0x7f89a015b2d0 2023-06-09T11:57:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8998014420 con 0x7f89a015b2d0 2023-06-09T11:57:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a0159980 con 0x7f89a015b2d0 2023-06-09T11:57:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980147d0 con 0x7f89a015b2d0 2023-06-09T11:57:39.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.846+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a015b730 con 0x7f89a015b2d0 2023-06-09T11:57:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.847+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a0159c60 con 0x7f89a015b2d0 2023-06-09T11:57:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.848+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8998014930 con 0x7f89a015b2d0 2023-06-09T11:57:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.849+0000 7f8996ffd700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 0x7f898406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.849+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f899808a520 con 0x7f89a015b2d0 2023-06-09T11:57:39.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.849+0000 7f89a659b700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 0x7f898406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.850+0000 7f89a659b700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 0x7f898406d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f899c0045f0 tx=0x7f899c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:39.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:39.854+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8998054d50 con 0x7f89a015b2d0 2023-06-09T11:57:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.109+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89a0074cf0 con 0x7f89a015b2d0 2023-06-09T11:57:40.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.109+0000 7f8996ffd700 1 -- 172.21.15.7:0/1527026503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f8998058370 con 0x7f89a015b2d0 2023-06-09T11:57:40.112 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:40.112 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 msgr2=0x7f898406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 0x7f898406d190 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f899c0045f0 tx=0x7f899c008040 comp rx=0 tx=0).stop 2023-06-09T11:57:40.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 msgr2=0x7f89a015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8998007e60 tx=0x7f899800e3c0 comp rx=0 tx=0).stop 2023-06-09T11:57:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 shutdown_connections 2023-06-09T11:57:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f89a015b2d0 0x7f89a015bb30 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f898406ace0 0x7f898406d190 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:40.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f89a00c9f40 0x7f89a01523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 --2- 172.21.15.7:0/1527026503 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f89a0153880 0x7f89a0151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 >> 172.21.15.7:0/1527026503 conn(0x7f89a00737f0 msgr2=0x7f89a006ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 shutdown_connections 2023-06-09T11:57:40.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:40.112+0000 7f89a87ff700 1 -- 172.21.15.7:0/1527026503 wait complete. 2023-06-09T11:57:40.117 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:40.132 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:42.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:42.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.340+0000 7f863d5ff700 1 Processor -- start 2023-06-09T11:57:42.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.340+0000 7f863d5ff700 1 -- start start 2023-06-09T11:57:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8638153880 0x7f863815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f863815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 0x7f86380c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86380c5f10 con 0x7f8638153880 2023-06-09T11:57:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863815a500 con 0x7f863815b2d0 2023-06-09T11:57:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.341+0000 7f863d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863815da00 con 0x7f86380c9f40 2023-06-09T11:57:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8638153880 0x7f863815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8638153880 0x7f863815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40112/0 (socket says 172.21.15.7:40112) 2023-06-09T11:57:42.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 -- 172.21.15.7:0/3422726727 learned_addr learned my addr 172.21.15.7:0/3422726727 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f86377fe700 1 --2- 172.21.15.7:0/3422726727 >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 0x7f86380c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f86367fc700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f863815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 -- 172.21.15.7:0/3422726727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 msgr2=0x7f863815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f863815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 -- 172.21.15.7:0/3422726727 >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 msgr2=0x7f86380c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 --2- 172.21.15.7:0/3422726727 >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 0x7f86380c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f8636ffd700 1 -- 172.21.15.7:0/3422726727 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863815b750 con 0x7f8638153880 2023-06-09T11:57:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f86377fe700 1 --2- 172.21.15.7:0/3422726727 >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 0x7f86380c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.342+0000 7f86367fc700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f863815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f8636ffd700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8638153880 0x7f863815d4c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8638071db0 tx=0x7f8628009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8174b9b826c92785 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f8635ffb700 1 -- 172.21.15.7:0/3422726727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800b870 con 0x7f8638153880 2023-06-09T11:57:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f8635ffb700 1 -- 172.21.15.7:0/3422726727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862800b9d0 con 0x7f8638153880 2023-06-09T11:57:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f8635ffb700 1 -- 172.21.15.7:0/3422726727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862800bd60 con 0x7f8638153880 2023-06-09T11:57:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 -- 172.21.15.7:0/3422726727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8638153880 msgr2=0x7f863815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8638153880 0x7f863815d4c0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8638071db0 tx=0x7f8628009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 -- 172.21.15.7:0/3422726727 shutdown_connections 2023-06-09T11:57:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 --2- 172.21.15.7:0/3422726727 >> v2:172.21.15.117:3300/0 conn(0x7f863815b2d0 0x7f86380c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8638153880 0x7f863815d4c0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 --2- 172.21.15.7:0/3422726727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f863815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.343+0000 7f863d5ff700 1 -- 172.21.15.7:0/3422726727 >> 172.21.15.7:0/3422726727 conn(0x7f86380737f0 msgr2=0x7f86380720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.344+0000 7f863d5ff700 1 -- 172.21.15.7:0/3422726727 shutdown_connections 2023-06-09T11:57:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.344+0000 7f863d5ff700 1 -- 172.21.15.7:0/3422726727 wait complete. 2023-06-09T11:57:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.345+0000 7f863d5ff700 1 Processor -- start 2023-06-09T11:57:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.345+0000 7f863d5ff700 1 -- start start 2023-06-09T11:57:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f86381540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 0x7f8638157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86381e2dd0 con 0x7f863815b2d0 2023-06-09T11:57:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8638155120 con 0x7f8638153880 2023-06-09T11:57:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.346+0000 7f863d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8638155400 con 0x7f86380c9f40 2023-06-09T11:57:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40118/0 (socket says 172.21.15.7:40118) 2023-06-09T11:57:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 -- 172.21.15.7:0/109842247 learned_addr learned my addr 172.21.15.7:0/109842247 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f8636ffd700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f86381540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86367fc700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 0x7f8638157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 -- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 msgr2=0x7f86381540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f86381540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 -- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 msgr2=0x7f8638157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 0x7f8638157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 -- 172.21.15.7:0/109842247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628009550 con 0x7f863815b2d0 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86367fc700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 0x7f8638157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f86377fe700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f863000cd50 tx=0x7f863000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000f480 con 0x7f863815b2d0 2023-06-09T11:57:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863000f600 con 0x7f863815b2d0 2023-06-09T11:57:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000f9b0 con 0x7f863815b2d0 2023-06-09T11:57:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.347+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8638159980 con 0x7f863815b2d0 2023-06-09T11:57:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.348+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86381596e0 con 0x7f863815b2d0 2023-06-09T11:57:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.349+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8638159c60 con 0x7f863815b2d0 2023-06-09T11:57:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.350+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f863000b040 con 0x7f863815b2d0 2023-06-09T11:57:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.350+0000 7f861f7fe700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.351+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f863008a620 con 0x7f863815b2d0 2023-06-09T11:57:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.351+0000 7f8636ffd700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.351+0000 7f8636ffd700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 0x7f862406d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8628000c00 tx=0x7f8628008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.355+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8630054da0 con 0x7f863815b2d0 2023-06-09T11:57:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.609+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8638074cf0 con 0x7f863815b2d0 2023-06-09T11:57:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.610+0000 7f861f7fe700 1 -- 172.21.15.7:0/109842247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f863001d070 con 0x7f863815b2d0 2023-06-09T11:57:42.614 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:42.614 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 msgr2=0x7f862406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:42.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 0x7f862406d190 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8628000c00 tx=0x7f8628008030 comp rx=0 tx=0).stop 2023-06-09T11:57:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 msgr2=0x7f8638158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f863000cd50 tx=0x7f863000a560 comp rx=0 tx=0).stop 2023-06-09T11:57:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 shutdown_connections 2023-06-09T11:57:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f863815b2d0 0x7f8638158140 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f862406ace0 0x7f862406d190 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86380c9f40 0x7f86381540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 --2- 172.21.15.7:0/109842247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8638153880 0x7f8638157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 >> 172.21.15.7:0/109842247 conn(0x7f86380737f0 msgr2=0x7f863814dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:42.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 shutdown_connections 2023-06-09T11:57:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:42.613+0000 7f863d5ff700 1 -- 172.21.15.7:0/109842247 wait complete. 2023-06-09T11:57:42.619 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:42.634 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:44.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.843+0000 7fc3249ff700 1 Processor -- start 2023-06-09T11:57:44.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.844+0000 7fc3249ff700 1 -- start start 2023-06-09T11:57:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc3249ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 0x7fc32015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc3249ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3200c9f40 0x7fc3200c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc3249ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32015a500 con 0x7fc320153880 2023-06-09T11:57:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc3249ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3200c5f10 con 0x7fc3200c9f40 2023-06-09T11:57:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc3249ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32015da00 con 0x7fc32015b4a0 2023-06-09T11:57:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3200c9f40 0x7fc3200c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3200c9f40 0x7fc3200c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58442/0 (socket says 172.21.15.7:58442) 2023-06-09T11:57:44.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 -- 172.21.15.7:0/2885920475 learned_addr learned my addr 172.21.15.7:0/2885920475 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/2885920475 >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 0x7fc32015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 -- 172.21.15.7:0/2885920475 >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 msgr2=0x7fc32015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:44.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 --2- 172.21.15.7:0/2885920475 >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 0x7fc32015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.845+0000 7fc31ed9d700 1 -- 172.21.15.7:0/2885920475 >> v1:172.21.15.7:6789/0 conn(0x7fc320153880 legacy=0x7fc32015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31ed9d700 1 -- 172.21.15.7:0/2885920475 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3201596e0 con 0x7fc3200c9f40 2023-06-09T11:57:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/2885920475 >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 0x7fc32015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31ed9d700 1 --2- 172.21.15.7:0/2885920475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3200c4e30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cce30 tx=0x7fc31800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a4ff8bf279783 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:44.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31d59a700 1 -- 172.21.15.7:0/2885920475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800e070 con 0x7fc3200c9f40 2023-06-09T11:57:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31d59a700 1 -- 172.21.15.7:0/2885920475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800b9c0 con 0x7fc3200c9f40 2023-06-09T11:57:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.846+0000 7fc31d59a700 1 -- 172.21.15.7:0/2885920475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800bd70 con 0x7fc3200c9f40 2023-06-09T11:57:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 -- 172.21.15.7:0/2885920475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 msgr2=0x7fc3200c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 --2- 172.21.15.7:0/2885920475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3200c4e30 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fc3200cce30 tx=0x7fc31800b330 comp rx=0 tx=0).stop 2023-06-09T11:57:44.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 -- 172.21.15.7:0/2885920475 shutdown_connections 2023-06-09T11:57:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 --2- 172.21.15.7:0/2885920475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3200c4e30 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 --2- 172.21.15.7:0/2885920475 >> v2:172.21.15.117:3300/0 conn(0x7fc32015b4a0 0x7fc32015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 -- 172.21.15.7:0/2885920475 >> 172.21.15.7:0/2885920475 conn(0x7fc3200737f0 msgr2=0x7fc3200720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 -- 172.21.15.7:0/2885920475 shutdown_connections 2023-06-09T11:57:44.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.847+0000 7fc3249ff700 1 -- 172.21.15.7:0/2885920475 wait complete. 2023-06-09T11:57:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.848+0000 7fc3249ff700 1 Processor -- start 2023-06-09T11:57:44.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 -- start start 2023-06-09T11:57:44.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 0x7fc3200b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 0x7fc3201521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320151d20 con 0x7fc3200c9f40 2023-06-09T11:57:44.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201f0080 con 0x7fc320153880 2023-06-09T11:57:44.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.849+0000 7fc3249ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201513b0 con 0x7fc32015b4a0 2023-06-09T11:57:44.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58450/0 (socket says 172.21.15.7:58450) 2023-06-09T11:57:44.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 -- 172.21.15.7:0/3552822130 learned_addr learned my addr 172.21.15.7:0/3552822130 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:44.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 0x7fc3200b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31ed9d700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 0x7fc3201521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 -- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 msgr2=0x7fc3201521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:44.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 0x7fc3201521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:44.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 -- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 msgr2=0x7fc3200b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:44.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 0x7fc3200b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:44.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 -- 172.21.15.7:0/3552822130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318009b30 con 0x7fc3200c9f40 2023-06-09T11:57:44.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31ed9d700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 0x7fc3201521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:44.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 0x7fc3200b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:44.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.850+0000 7fc31e59c700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c4e0 tx=0x7fc31000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:44.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.851+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100052d0 con 0x7fc3200c9f40 2023-06-09T11:57:44.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.851+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc310005430 con 0x7fc3200c9f40 2023-06-09T11:57:44.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.851+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc320159c60 con 0x7fc3200c9f40 2023-06-09T11:57:44.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.851+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100038c0 con 0x7fc3200c9f40 2023-06-09T11:57:44.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.851+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc320159980 con 0x7fc3200c9f40 2023-06-09T11:57:44.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.852+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc320159f40 con 0x7fc3200c9f40 2023-06-09T11:57:44.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.853+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc310003a20 con 0x7fc3200c9f40 2023-06-09T11:57:44.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.854+0000 7fc30effd700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 0x7fc30006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:44.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.854+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fc310089180 con 0x7fc3200c9f40 2023-06-09T11:57:44.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.854+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 0x7fc30006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:44.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.855+0000 7fc31dd9b700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 0x7fc30006d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc32014bd90 tx=0x7fc314008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:44.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:44.859+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc3100538a0 con 0x7fc3200c9f40 2023-06-09T11:57:45.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.116+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc32015a500 con 0x7fc3200c9f40 2023-06-09T11:57:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.117+0000 7fc30effd700 1 -- 172.21.15.7:0/3552822130 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7fc32015a500 con 0x7fc3200c9f40 2023-06-09T11:57:45.120 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:45.120 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 msgr2=0x7fc30006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 0x7fc30006d190 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc32014bd90 tx=0x7fc314008040 comp rx=0 tx=0).stop 2023-06-09T11:57:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 msgr2=0x7fc3201efb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:45.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc31000c4e0 tx=0x7fc31000c8a0 comp rx=0 tx=0).stop 2023-06-09T11:57:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 shutdown_connections 2023-06-09T11:57:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc30006ace0 0x7fc30006d190 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc32015b4a0 0x7fc3201521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3200c9f40 0x7fc3201efb40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:45.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 --2- 172.21.15.7:0/3552822130 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc320153880 0x7fc3200b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 >> 172.21.15.7:0/3552822130 conn(0x7fc3200737f0 msgr2=0x7fc32014d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 shutdown_connections 2023-06-09T11:57:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:45.119+0000 7fc3249ff700 1 -- 172.21.15.7:0/3552822130 wait complete. 2023-06-09T11:57:45.125 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:45.141 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:57:47.142 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.350+0000 7f168bbff700 1 Processor -- start 2023-06-09T11:57:47.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.351+0000 7f168bbff700 1 -- start start 2023-06-09T11:57:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168bbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 0x7f168415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168bbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16840c9f40 0x7f168415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168bbff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f168415a500 con 0x7f168415b2d0 2023-06-09T11:57:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168bbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16840c5f10 con 0x7f16840c9f40 2023-06-09T11:57:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168bbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f168415da00 con 0x7f1684153880 2023-06-09T11:57:47.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16840c9f40 0x7f168415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f16840c9f40 0x7f168415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58458/0 (socket says 172.21.15.7:58458) 2023-06-09T11:57:47.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 -- 172.21.15.7:0/2067317669 learned_addr learned my addr 172.21.15.7:0/2067317669 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:47.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168999b700 1 --2- 172.21.15.7:0/2067317669 >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 0x7f168415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 -- 172.21.15.7:0/2067317669 >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 msgr2=0x7f168415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:47.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 --2- 172.21.15.7:0/2067317669 >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 0x7f168415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.352+0000 7f168919a700 1 -- 172.21.15.7:0/2067317669 >> v1:172.21.15.117:6789/0 conn(0x7f168415b2d0 legacy=0x7f16840c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f168919a700 1 -- 172.21.15.7:0/2067317669 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16841596e0 con 0x7f16840c9f40 2023-06-09T11:57:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f168999b700 1 --2- 172.21.15.7:0/2067317669 >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 0x7f168415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f168919a700 1 --2- 172.21.15.7:0/2067317669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f168415ad90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f168408d340 tx=0x7f1674009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43fe6e0eda0f82f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f1688999700 1 -- 172.21.15.7:0/2067317669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674004230 con 0x7f16840c9f40 2023-06-09T11:57:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f1688999700 1 -- 172.21.15.7:0/2067317669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674004390 con 0x7f16840c9f40 2023-06-09T11:57:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.353+0000 7f1688999700 1 -- 172.21.15.7:0/2067317669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740036a0 con 0x7f16840c9f40 2023-06-09T11:57:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 -- 172.21.15.7:0/2067317669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 msgr2=0x7f168415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 --2- 172.21.15.7:0/2067317669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f168415ad90 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f168408d340 tx=0x7f1674009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 -- 172.21.15.7:0/2067317669 shutdown_connections 2023-06-09T11:57:47.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 --2- 172.21.15.7:0/2067317669 >> v2:172.21.15.117:3300/0 conn(0x7f1684153880 0x7f168415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 --2- 172.21.15.7:0/2067317669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f168415ad90 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 -- 172.21.15.7:0/2067317669 >> 172.21.15.7:0/2067317669 conn(0x7f16840737f0 msgr2=0x7f16840720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:47.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 -- 172.21.15.7:0/2067317669 shutdown_connections 2023-06-09T11:57:47.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.354+0000 7f168bbff700 1 -- 172.21.15.7:0/2067317669 wait complete. 2023-06-09T11:57:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.356+0000 7f168bbff700 1 Processor -- start 2023-06-09T11:57:47.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.356+0000 7f168bbff700 1 -- start start 2023-06-09T11:57:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 0x7f16840b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 0x7f16840b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16840b30b0 con 0x7f16840c9f40 2023-06-09T11:57:47.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16841ef800 con 0x7f1684153880 2023-06-09T11:57:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168bbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16841eebf0 con 0x7f168415b2d0 2023-06-09T11:57:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58468/0 (socket says 172.21.15.7:58468) 2023-06-09T11:57:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 -- 172.21.15.7:0/64747118 learned_addr learned my addr 172.21.15.7:0/64747118 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:47.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168919a700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 0x7f16840b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168a19c700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 0x7f16840b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 -- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 msgr2=0x7f16840b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 0x7f16840b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 -- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 msgr2=0x7f16840b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:47.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 0x7f16840b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168999b700 1 -- 172.21.15.7:0/64747118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1674009550 con 0x7f16840c9f40 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168919a700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 0x7f16840b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.357+0000 7f168a19c700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 0x7f16840b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f168999b700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f168000c750 tx=0x7f168000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680011800 con 0x7f16840c9f40 2023-06-09T11:57:47.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1680011960 con 0x7f16840c9f40 2023-06-09T11:57:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1680011cd0 con 0x7f16840c9f40 2023-06-09T11:57:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1684159c60 con 0x7f16840c9f40 2023-06-09T11:57:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.358+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1684159980 con 0x7f16840c9f40 2023-06-09T11:57:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.359+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1684159f40 con 0x7f16840c9f40 2023-06-09T11:57:47.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.360+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1680021350 con 0x7f16840c9f40 2023-06-09T11:57:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.361+0000 7f167a7fc700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 0x7f166806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:47.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.361+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f1680089d20 con 0x7f16840c9f40 2023-06-09T11:57:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.361+0000 7f168919a700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 0x7f166806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.362+0000 7f168919a700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 0x7f166806d1c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16841e5ca0 tx=0x7f1674008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:47.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.365+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f16800544d0 con 0x7f16840c9f40 2023-06-09T11:57:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.617+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1684074cf0 con 0x7f16840c9f40 2023-06-09T11:57:47.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.617+0000 7f167a7fc700 1 -- 172.21.15.7:0/64747118 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 216 v216) v1 ==== 95+0+6228 (secure 0 0 0) 0x7f168001e020 con 0x7f16840c9f40 2023-06-09T11:57:47.621 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:47.621 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":216,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":216,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:08.951234+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1,2],"up":{"mds_0":7437,"mds_1":7807,"mds_2":7666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7666":{"gid":7666,"name":"b","rank":2,"incarnation":194,"state":"up:stopping","state_seq":44,"addr":"172.21.15.117:6833/180047226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":180047226},{"type":"v1","addr":"172.21.15.117:6833","nonce":180047226}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:47.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 msgr2=0x7f166806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:47.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 0x7f166806d1c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f16841e5ca0 tx=0x7f1674008030 comp rx=0 tx=0).stop 2023-06-09T11:57:47.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 msgr2=0x7f16841e4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f168000c750 tx=0x7f168000cb10 comp rx=0 tx=0).stop 2023-06-09T11:57:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 shutdown_connections 2023-06-09T11:57:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f166806ad10 0x7f166806d1c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f168415b2d0 0x7f16840b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f16840c9f40 0x7f16841e4fc0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 --2- 172.21.15.7:0/64747118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1684153880 0x7f16840b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:47.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 >> 172.21.15.7:0/64747118 conn(0x7f16840737f0 msgr2=0x7f168414e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:47.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 shutdown_connections 2023-06-09T11:57:47.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:47.620+0000 7f168bbff700 1 -- 172.21.15.7:0/64747118 wait complete. 2023-06-09T11:57:47.626 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 216 2023-06-09T11:57:47.641 INFO:tasks.mds_thrash.fs.[cephfs]:cluster is considered unstable while MDS are in up:stopping (!thrash_while_stopping) 2023-06-09T11:57:47.642 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 216, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 216, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1, 2 ], "info": { "gid_7437": { "addr": "172.21.15.7:6839/1569034808", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6838", "nonce": 1569034808, "type": "v2" }, { "addr": "172.21.15.7:6839", "nonce": 1569034808, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7437, "incarnation": 152, "join_fscid": -1, "name": "e", "rank": 0, "state": "up:active", "state_seq": 32 }, "gid_7666": { "addr": "172.21.15.117:6833/180047226", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 180047226, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 180047226, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 1 ], "features": 4540138322906710015, "flags": 0, "gid": 7666, "incarnation": 194, "join_fscid": -1, "name": "b", "rank": 2, "state": "up:stopping", "state_seq": 44 }, "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 } }, "last_failure": 0, "last_failure_osd_epoch": 73, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:57:08.951234+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7437, "mds_1": 7807, "mds_2": 7666 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:57:49.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:49.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.850+0000 7fd3947ff700 1 Processor -- start 2023-06-09T11:57:49.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.850+0000 7fd3947ff700 1 -- start start 2023-06-09T11:57:49.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.851+0000 7fd3947ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd38c1535e0 0x7fd38c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.851+0000 7fd3947ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd38c1539b0 0x7fd38c0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.851+0000 7fd3947ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c15a420 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.851+0000 7fd3947ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0c5a00 con 0x7fd38c1535e0 2023-06-09T11:57:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.851+0000 7fd3947ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c06b460 con 0x7fd38c1539b0 2023-06-09T11:57:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd38c1535e0 0x7fd38c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd38c1535e0 0x7fd38c159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58472/0 (socket says 172.21.15.7:58472) 2023-06-09T11:57:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 -- 172.21.15.7:0/32468263 learned_addr learned my addr 172.21.15.7:0/32468263 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd391d9a700 1 --2- 172.21.15.7:0/32468263 >> v2:172.21.15.117:3300/0 conn(0x7fd38c1539b0 0x7fd38c0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 -- 172.21.15.7:0/32468263 >> v2:172.21.15.117:3300/0 conn(0x7fd38c1539b0 msgr2=0x7fd38c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 --2- 172.21.15.7:0/32468263 >> v2:172.21.15.117:3300/0 conn(0x7fd38c1539b0 0x7fd38c0c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 -- 172.21.15.7:0/32468263 >> v1:172.21.15.7:6789/0 conn(0x7fd38c0c4ce0 legacy=0x7fd38c15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:49.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 -- 172.21.15.7:0/32468263 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c06a570 con 0x7fd38c1535e0 2023-06-09T11:57:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.852+0000 7fd39259b700 1 --2- 172.21.15.7:0/32468263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c1535e0 0x7fd38c159ee0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd38c08d1e0 tx=0x7fd388009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8308bb87d87edd9e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd391599700 1 -- 172.21.15.7:0/32468263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388011830 con 0x7fd38c1535e0 2023-06-09T11:57:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd391599700 1 -- 172.21.15.7:0/32468263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd388011990 con 0x7fd38c1535e0 2023-06-09T11:57:49.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd391599700 1 -- 172.21.15.7:0/32468263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388011d20 con 0x7fd38c1535e0 2023-06-09T11:57:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 -- 172.21.15.7:0/32468263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c1535e0 msgr2=0x7fd38c159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 --2- 172.21.15.7:0/32468263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c1535e0 0x7fd38c159ee0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fd38c08d1e0 tx=0x7fd388009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 -- 172.21.15.7:0/32468263 shutdown_connections 2023-06-09T11:57:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 --2- 172.21.15.7:0/32468263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c1535e0 0x7fd38c159ee0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 --2- 172.21.15.7:0/32468263 >> v2:172.21.15.117:3300/0 conn(0x7fd38c1539b0 0x7fd38c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 -- 172.21.15.7:0/32468263 >> 172.21.15.7:0/32468263 conn(0x7fd38c0737f0 msgr2=0x7fd38c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 -- 172.21.15.7:0/32468263 shutdown_connections 2023-06-09T11:57:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.853+0000 7fd3947ff700 1 -- 172.21.15.7:0/32468263 wait complete. 2023-06-09T11:57:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.855+0000 7fd3947ff700 1 Processor -- start 2023-06-09T11:57:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.855+0000 7fd3947ff700 1 -- start start 2023-06-09T11:57:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 0x7fd38c0b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 0x7fd38c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c1587a0 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c158ee0 con 0x7fd38c1539b0 2023-06-09T11:57:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd3947ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c159180 con 0x7fd38c1535e0 2023-06-09T11:57:49.864 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd39259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd39259b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58478/0 (socket says 172.21.15.7:58478) 2023-06-09T11:57:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.856+0000 7fd39259b700 1 -- 172.21.15.7:0/3226291155 learned_addr learned my addr 172.21.15.7:0/3226291155 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 -- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 msgr2=0x7fd38c0b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd391d9a700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 0x7fd38c0b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.865 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd392d9c700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 0x7fd38c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 0x7fd38c0b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 -- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 msgr2=0x7fd38c0b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 0x7fd38c0b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 -- 172.21.15.7:0/3226291155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd388009550 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd391d9a700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 0x7fd38c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd392d9c700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 0x7fd38c0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:57:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd39259b700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd388000c00 tx=0x7fd388004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388004570 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38801f070 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c06ab30 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.857+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3880038e0 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.858+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c06a850 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.858+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c06ae10 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.860+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd388021030 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.860+0000 7fd382ffd700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 0x7fd37006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.861+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fd38808b210 con 0x7fd38c0c4ce0 2023-06-09T11:57:49.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.861+0000 7fd391d9a700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 0x7fd37006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.861+0000 7fd391d9a700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 0x7fd37006d190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0d05c0 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:49.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:49.865+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd3880559c0 con 0x7fd38c0c4ce0 2023-06-09T11:57:50.041 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T11:57:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.119+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd38c1585f0 con 0x7fd38c0c4ce0 2023-06-09T11:57:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.119+0000 7fd382ffd700 1 -- 172.21.15.7:0/3226291155 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+5451 (secure 0 0 0) 0x7fd388015090 con 0x7fd38c0c4ce0 2023-06-09T11:57:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:50.123 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":217,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:50.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 msgr2=0x7fd37006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 0x7fd37006d190 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd38c0d05c0 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2023-06-09T11:57:50.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 msgr2=0x7fd38c1e5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fd388000c00 tx=0x7fd388004750 comp rx=0 tx=0).stop 2023-06-09T11:57:50.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 shutdown_connections 2023-06-09T11:57:50.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd38c1539b0 0x7fd38c0b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd38c0c4ce0 0x7fd38c1e5f30 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd37006ace0 0x7fd37006d190 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 --2- 172.21.15.7:0/3226291155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd38c1535e0 0x7fd38c0b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 >> 172.21.15.7:0/3226291155 conn(0x7fd38c0737f0 msgr2=0x7fd38c151910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:50.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 shutdown_connections 2023-06-09T11:57:50.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.122+0000 7fd3947ff700 1 -- 172.21.15.7:0/3226291155 wait complete. 2023-06-09T11:57:50.128 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 217 2023-06-09T11:57:50.143 INFO:tasks.mds_thrash.fs.[cephfs]:mds cluster has 2 alive and active, now stable! 2023-06-09T11:57:50.144 INFO:tasks.mds_thrash.fs.[cephfs]:kill mds.e (rank=0) 2023-06-09T11:57:50.144 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-06-09T11:57:50.144 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T11:57:50.169 INFO:tasks.ceph.mds.e:Stopped 2023-06-09T11:57:50.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.406+0000 7fd22cfff700 1 Processor -- start 2023-06-09T11:57:50.410 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.406+0000 7fd22cfff700 1 -- start start 2023-06-09T11:57:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 0x7fd22815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22cfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd2280c9f40 0x7fd22815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22cfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22815a500 con 0x7fd22815b2d0 2023-06-09T11:57:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22cfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280c5f10 con 0x7fd2280c9f40 2023-06-09T11:57:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22815da00 con 0x7fd228153880 2023-06-09T11:57:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd226d9d700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd22815b2d0 0x7fd2280c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57278/0 (socket says 172.21.15.7:57278) 2023-06-09T11:57:50.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd226d9d700 1 -- 172.21.15.7:0/523709415 learned_addr learned my addr 172.21.15.7:0/523709415 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:57:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd22659c700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 0x7fd22815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd21ffff700 1 --2- 172.21.15.7:0/523709415 >> v2:172.21.15.117:3300/0 conn(0x7fd2280c9f40 0x7fd22815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266921698 0 0) 0x7fd220001490 con 0x7fd22815b2d0 2023-06-09T11:57:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.408+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2140035b0 con 0x7fd22815b2d0 2023-06-09T11:57:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1710365161 0 0) 0x7fd2140035b0 con 0x7fd22815b2d0 2023-06-09T11:57:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd21ffff700 1 -- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 msgr2=0x7fd22815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd21ffff700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 0x7fd22815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd21ffff700 1 -- 172.21.15.7:0/523709415 >> v1:172.21.15.7:6789/0 conn(0x7fd22815b2d0 legacy=0x7fd2280c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd21ffff700 1 -- 172.21.15.7:0/523709415 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22815b750 con 0x7fd2280c9f40 2023-06-09T11:57:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd22659c700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 0x7fd22815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:57:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd21ffff700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2280c9f40 0x7fd22815ad90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd210009840 tx=0x7fd210009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dcc45b444b7f44e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220001b80 con 0x7fd2280c9f40 2023-06-09T11:57:50.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd210004030 con 0x7fd2280c9f40 2023-06-09T11:57:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.409+0000 7fd225d9b700 1 -- 172.21.15.7:0/523709415 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2100043c0 con 0x7fd2280c9f40 2023-06-09T11:57:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 -- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2280c9f40 msgr2=0x7fd22815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2280c9f40 0x7fd22815ad90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd210009840 tx=0x7fd210009b50 comp rx=0 tx=0).stop 2023-06-09T11:57:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 -- 172.21.15.7:0/523709415 shutdown_connections 2023-06-09T11:57:50.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd228153880 0x7fd22815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 --2- 172.21.15.7:0/523709415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd2280c9f40 0x7fd22815ad90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 -- 172.21.15.7:0/523709415 >> 172.21.15.7:0/523709415 conn(0x7fd2280737f0 msgr2=0x7fd2280720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:50.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 -- 172.21.15.7:0/523709415 shutdown_connections 2023-06-09T11:57:50.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.410+0000 7fd22cfff700 1 -- 172.21.15.7:0/523709415 wait complete. 2023-06-09T11:57:50.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.412+0000 7fd22cfff700 1 Processor -- start 2023-06-09T11:57:50.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.412+0000 7fd22cfff700 1 -- start start 2023-06-09T11:57:50.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228153880 0x7fd22814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd22815b2d0 0x7fd2280cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281561c0 con 0x7fd2280c9f40 2023-06-09T11:57:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228155bb0 con 0x7fd228153880 2023-06-09T11:57:50.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd228153eb0 con 0x7fd22815b2d0 2023-06-09T11:57:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58494/0 (socket says 172.21.15.7:58494) 2023-06-09T11:57:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 -- 172.21.15.7:0/2127794233 learned_addr learned my addr 172.21.15.7:0/2127794233 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd226d9d700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd22815b2d0 0x7fd2280cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd21ffff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228153880 0x7fd22814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 -- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd22815b2d0 msgr2=0x7fd2280cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd22815b2d0 0x7fd2280cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 -- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228153880 msgr2=0x7fd22814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228153880 0x7fd22814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.413+0000 7fd22659c700 1 -- 172.21.15.7:0/2127794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180094a0 con 0x7fd2280c9f40 2023-06-09T11:57:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd22659c700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ebe0 tx=0x7fd21800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800c850 con 0x7fd2280c9f40 2023-06-09T11:57:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21800c9b0 con 0x7fd2280c9f40 2023-06-09T11:57:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800cd40 con 0x7fd2280c9f40 2023-06-09T11:57:50.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2100094a0 con 0x7fd2280c9f40 2023-06-09T11:57:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.414+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228159980 con 0x7fd2280c9f40 2023-06-09T11:57:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.415+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2281596e0 con 0x7fd2280c9f40 2023-06-09T11:57:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.416+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd218013470 con 0x7fd2280c9f40 2023-06-09T11:57:50.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.417+0000 7fd21effd700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 0x7fd20806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.417+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fd218089eb0 con 0x7fd2280c9f40 2023-06-09T11:57:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.417+0000 7fd21ffff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 0x7fd20806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.418+0000 7fd21ffff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 0x7fd20806d190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd2100090b0 tx=0x7fd210008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.421+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd2180545e0 con 0x7fd2280c9f40 2023-06-09T11:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.672+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd228159c60 con 0x7fd2280c9f40 2023-06-09T11:57:50.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.673+0000 7fd21effd700 1 -- 172.21.15.7:0/2127794233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 217 v217) v1 ==== 95+0+5451 (secure 0 0 0) 0x7fd21800cea0 con 0x7fd2280c9f40 2023-06-09T11:57:50.677 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:50.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":217,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 msgr2=0x7fd20806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 0x7fd20806d190 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd2100090b0 tx=0x7fd210008030 comp rx=0 tx=0).stop 2023-06-09T11:57:50.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 msgr2=0x7fd2280b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd21800ebe0 tx=0x7fd21800eef0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 shutdown_connections 2023-06-09T11:57:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd20806ace0 0x7fd20806d190 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd22815b2d0 0x7fd2280cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2280c9f40 0x7fd2280b4c00 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 --2- 172.21.15.7:0/2127794233 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd228153880 0x7fd22814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.675+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 >> 172.21.15.7:0/2127794233 conn(0x7fd2280737f0 msgr2=0x7fd2280cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.676+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 shutdown_connections 2023-06-09T11:57:50.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:50.676+0000 7fd22cfff700 1 -- 172.21.15.7:0/2127794233 wait complete. 2023-06-09T11:57:50.682 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 217 2023-06-09T11:57:50.697 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:57:52.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.907+0000 7f47ae3ff700 1 Processor -- start 2023-06-09T11:57:52.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.908+0000 7f47ae3ff700 1 -- start start 2023-06-09T11:57:52.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ae3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a80c1bc0 0x7f47a81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ae3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f47a80c1f90 0x7f47a8159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ae3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80c0bf0 con 0x7f47a80c0140 2023-06-09T11:57:52.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ae3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8070110 con 0x7f47a80c1bc0 2023-06-09T11:57:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ae3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a8071300 con 0x7f47a80c1f90 2023-06-09T11:57:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a80c1bc0 0x7f47a81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47a7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f47a80c1bc0 0x7f47a81599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58500/0 (socket says 172.21.15.7:58500) 2023-06-09T11:57:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47a7fff700 1 -- 172.21.15.7:0/3158163232 learned_addr learned my addr 172.21.15.7:0/3158163232 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.909+0000 7f47ac99c700 1 --2- 172.21.15.7:0/3158163232 >> v2:172.21.15.117:3300/0 conn(0x7f47a80c1f90 0x7f47a8159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a7fff700 1 -- 172.21.15.7:0/3158163232 >> v2:172.21.15.117:3300/0 conn(0x7f47a80c1f90 msgr2=0x7f47a8159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a7fff700 1 --2- 172.21.15.7:0/3158163232 >> v2:172.21.15.117:3300/0 conn(0x7f47a80c1f90 0x7f47a8159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a7fff700 1 -- 172.21.15.7:0/3158163232 >> v1:172.21.15.117:6789/0 conn(0x7f47a80c0140 legacy=0x7f47a81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a7fff700 1 -- 172.21.15.7:0/3158163232 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80702f0 con 0x7f47a80c1bc0 2023-06-09T11:57:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a7fff700 1 --2- 172.21.15.7:0/3158163232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c1bc0 0x7f47a81599b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f47a8071ae0 tx=0x7f4798009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf9f6e5df9c7234 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:52.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a6ffd700 1 -- 172.21.15.7:0/3158163232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798004230 con 0x7f47a80c1bc0 2023-06-09T11:57:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a6ffd700 1 -- 172.21.15.7:0/3158163232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798004390 con 0x7f47a80c1bc0 2023-06-09T11:57:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.910+0000 7f47a6ffd700 1 -- 172.21.15.7:0/3158163232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47980036a0 con 0x7f47a80c1bc0 2023-06-09T11:57:52.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 -- 172.21.15.7:0/3158163232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c1bc0 msgr2=0x7f47a81599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/3158163232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c1bc0 0x7f47a81599b0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f47a8071ae0 tx=0x7f4798009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 -- 172.21.15.7:0/3158163232 shutdown_connections 2023-06-09T11:57:52.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/3158163232 >> v2:172.21.15.117:3300/0 conn(0x7f47a80c1f90 0x7f47a8159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/3158163232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c1bc0 0x7f47a81599b0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 -- 172.21.15.7:0/3158163232 >> 172.21.15.7:0/3158163232 conn(0x7f47a806a4d0 msgr2=0x7f47a806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 -- 172.21.15.7:0/3158163232 shutdown_connections 2023-06-09T11:57:52.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.911+0000 7f47ae3ff700 1 -- 172.21.15.7:0/3158163232 wait complete. 2023-06-09T11:57:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.913+0000 7f47ae3ff700 1 Processor -- start 2023-06-09T11:57:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.913+0000 7f47ae3ff700 1 -- start start 2023-06-09T11:57:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.914+0000 7f47ae3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.914+0000 7f47ae3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47a80c1bc0 0x7f47a814e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47ae3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a80c1f90 0x7f47a814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47ae3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81f0060 con 0x7f47a80c0140 2023-06-09T11:57:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47ae3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81639f0 con 0x7f47a80c1f90 2023-06-09T11:57:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47ae3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81efae0 con 0x7f47a80c1bc0 2023-06-09T11:57:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58514/0 (socket says 172.21.15.7:58514) 2023-06-09T11:57:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 -- 172.21.15.7:0/843109823 learned_addr learned my addr 172.21.15.7:0/843109823 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47ac99c700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a80c1f90 0x7f47a814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a77fe700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47a80c1bc0 0x7f47a814e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 -- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47a80c1bc0 msgr2=0x7f47a814e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47a80c1bc0 0x7f47a814e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 -- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a80c1f90 msgr2=0x7f47a814e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a80c1f90 0x7f47a814e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.915+0000 7f47a7fff700 1 -- 172.21.15.7:0/843109823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798009550 con 0x7f47a80c0140 2023-06-09T11:57:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47a7fff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4798009130 tx=0x7f4798004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798003b90 con 0x7f47a80c0140 2023-06-09T11:57:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479801f070 con 0x7f47a80c0140 2023-06-09T11:57:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4798011630 con 0x7f47a80c0140 2023-06-09T11:57:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a80c1170 con 0x7f47a80c0140 2023-06-09T11:57:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.916+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a80c0e90 con 0x7f47a80c0140 2023-06-09T11:57:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.917+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a80c1450 con 0x7f47a80c0140 2023-06-09T11:57:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.918+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4798021030 con 0x7f47a80c0140 2023-06-09T11:57:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.919+0000 7f47a4ff9700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 0x7f479406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.919+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f479808b1b0 con 0x7f47a80c0140 2023-06-09T11:57:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.919+0000 7f47a77fe700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 0x7f479406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.920+0000 7f47a77fe700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 0x7f479406d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f47a80c5f20 tx=0x7f479c007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:52.923+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4798055970 con 0x7f47a80c0140 2023-06-09T11:57:53.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.175+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47a80cb890 con 0x7f47a80c0140 2023-06-09T11:57:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.176+0000 7f47a4ff9700 1 -- 172.21.15.7:0/843109823 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7f4798058f90 con 0x7f47a80c0140 2023-06-09T11:57:53.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:53.181 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:53.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 msgr2=0x7f479406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:53.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 0x7f479406d190 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f47a80c5f20 tx=0x7f479c007ec0 comp rx=0 tx=0).stop 2023-06-09T11:57:53.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 msgr2=0x7f47a8154600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:53.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f4798009130 tx=0x7f4798004750 comp rx=0 tx=0).stop 2023-06-09T11:57:53.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 shutdown_connections 2023-06-09T11:57:53.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47a80c1f90 0x7f47a814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:53.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47a80c0140 0x7f47a8154600 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f479406ace0 0x7f479406d190 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 --2- 172.21.15.7:0/843109823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47a80c1bc0 0x7f47a814e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:53.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.178+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 >> 172.21.15.7:0/843109823 conn(0x7f47a806a4d0 msgr2=0x7f47a80ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.179+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 shutdown_connections 2023-06-09T11:57:53.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:53.179+0000 7f47ae3ff700 1 -- 172.21.15.7:0/843109823 wait complete. 2023-06-09T11:57:53.187 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:57:53.200 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:57:54.985 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:54.990 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:57:55.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.405+0000 7f24261ff700 1 Processor -- start 2023-06-09T11:57:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.405+0000 7f24261ff700 1 -- start start 2023-06-09T11:57:55.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.406+0000 7f24261ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 0x7f24200c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.406+0000 7f24261ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.406+0000 7f24261ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242015cec0 con 0x7f24200cfaf0 2023-06-09T11:57:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.406+0000 7f24261ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242015d160 con 0x7f24200c9090 2023-06-09T11:57:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.406+0000 7f24261ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242015d440 con 0x7f24200cad00 2023-06-09T11:57:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 0x7f24200c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:48546/0 (socket says 172.21.15.7:48546) 2023-06-09T11:57:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 -- 172.21.15.7:0/3650037260 learned_addr learned my addr 172.21.15.7:0/3650037260 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 -- 172.21.15.7:0/3650037260 >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 msgr2=0x7f24200c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 --2- 172.21.15.7:0/3650037260 >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 0x7f24200c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 -- 172.21.15.7:0/3650037260 >> v1:172.21.15.117:6789/0 conn(0x7f24200cfaf0 legacy=0x7f24201598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 -- 172.21.15.7:0/3650037260 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242015d720 con 0x7f24200cad00 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3650037260 >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 0x7f24200c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.407+0000 7f241effd700 1 --2- 172.21.15.7:0/3650037260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f24140098f0 tx=0x7f2414009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f843b2bc69856c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f241e7fc700 1 -- 172.21.15.7:0/3650037260 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414011830 con 0x7f24200cad00 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f241e7fc700 1 -- 172.21.15.7:0/3650037260 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2414011990 con 0x7f24200cad00 2023-06-09T11:57:55.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f241e7fc700 1 -- 172.21.15.7:0/3650037260 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414011d20 con 0x7f24200cad00 2023-06-09T11:57:55.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 -- 172.21.15.7:0/3650037260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 msgr2=0x7f24200cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 --2- 172.21.15.7:0/3650037260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f24140098f0 tx=0x7f2414009c00 comp rx=0 tx=0).stop 2023-06-09T11:57:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 -- 172.21.15.7:0/3650037260 shutdown_connections 2023-06-09T11:57:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 --2- 172.21.15.7:0/3650037260 >> v2:172.21.15.117:3300/0 conn(0x7f24200c9090 0x7f24200c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 --2- 172.21.15.7:0/3650037260 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200cad00 0x7f24200cf5b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 -- 172.21.15.7:0/3650037260 >> 172.21.15.7:0/3650037260 conn(0x7f2420070af0 msgr2=0x7f2420070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 -- 172.21.15.7:0/3650037260 shutdown_connections 2023-06-09T11:57:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.408+0000 7f24261ff700 1 -- 172.21.15.7:0/3650037260 wait complete. 2023-06-09T11:57:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.410+0000 7f24261ff700 1 Processor -- start 2023-06-09T11:57:55.418 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.410+0000 7f24261ff700 1 -- start start 2023-06-09T11:57:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c9090 0x7f2420152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200cad00 0x7f24200c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c4120 con 0x7f24200cfaf0 2023-06-09T11:57:55.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c4400 con 0x7f24200cad00 2023-06-09T11:57:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.411+0000 7f24261ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c46e0 con 0x7f24200c9090 2023-06-09T11:57:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241f7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c9090 0x7f2420152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48218/0 (socket says 172.21.15.7:48218) 2023-06-09T11:57:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 -- 172.21.15.7:0/3959433679 learned_addr learned my addr 172.21.15.7:0/3959433679 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200cad00 0x7f24200c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 -- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c9090 msgr2=0x7f2420152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c9090 0x7f2420152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 -- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200cad00 msgr2=0x7f24200c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200cad00 0x7f24200c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 -- 172.21.15.7:0/3959433679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100094a0 con 0x7f24200cfaf0 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.412+0000 7f241ffff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f241800cb50 tx=0x7f241800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.413+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418003c00 con 0x7f24200cfaf0 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.413+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2414009550 con 0x7f24200cfaf0 2023-06-09T11:57:55.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.413+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418003d60 con 0x7f24200cfaf0 2023-06-09T11:57:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.413+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241800aa20 con 0x7f24200cfaf0 2023-06-09T11:57:55.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.413+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200cb130 con 0x7f24200cfaf0 2023-06-09T11:57:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.414+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24200ca620 con 0x7f24200cfaf0 2023-06-09T11:57:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.415+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f241800ab80 con 0x7f24200cfaf0 2023-06-09T11:57:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.416+0000 7f2407fff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 0x7f240c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:55.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.416+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f2418089e70 con 0x7f24200cfaf0 2023-06-09T11:57:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.416+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 0x7f240c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.417+0000 7f241f7fe700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 0x7f240c06d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2410009cd0 tx=0x7f2410009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:55.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.421+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2418054510 con 0x7f24200cfaf0 2023-06-09T11:57:55.675 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.672+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24200cb2c0 con 0x7f24200cfaf0 2023-06-09T11:57:55.676 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.673+0000 7f2407fff700 1 -- 172.21.15.7:0/3959433679 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7f2418057b30 con 0x7f24200cfaf0 2023-06-09T11:57:55.677 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:55.677 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 msgr2=0x7f240c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 0x7f240c06d190 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f2410009cd0 tx=0x7f2410009210 comp rx=0 tx=0).stop 2023-06-09T11:57:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 msgr2=0x7f24200c5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:55.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f241800cb50 tx=0x7f241800ce60 comp rx=0 tx=0).stop 2023-06-09T11:57:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 shutdown_connections 2023-06-09T11:57:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24200cfaf0 0x7f24200c5f80 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.675+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f240c06ace0 0x7f240c06d190 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.676+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24200c9090 0x7f2420152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.680 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.676+0000 7f24261ff700 1 --2- 172.21.15.7:0/3959433679 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24200cad00 0x7f24200c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.676+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 >> 172.21.15.7:0/3959433679 conn(0x7f2420070af0 msgr2=0x7f242006ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.676+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 shutdown_connections 2023-06-09T11:57:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:55.676+0000 7f24261ff700 1 -- 172.21.15.7:0/3959433679 wait complete. 2023-06-09T11:57:55.681 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:57:55.697 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:57:57.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:57:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.905+0000 7fbf8e3ff700 1 Processor -- start 2023-06-09T11:57:57.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.905+0000 7fbf8e3ff700 1 -- start start 2023-06-09T11:57:57.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf8e3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 0x7fbf88159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf8e3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf881539b0 0x7fbf880c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf8e3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8815a420 con 0x7fbf880c4ce0 2023-06-09T11:57:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf8e3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880c5a00 con 0x7fbf881539b0 2023-06-09T11:57:57.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf8e3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf8806b460 con 0x7fbf881535e0 2023-06-09T11:57:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf881539b0 0x7fbf880c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf87fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 0x7fbf88159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf877fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbf881539b0 0x7fbf880c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48230/0 (socket says 172.21.15.7:48230) 2023-06-09T11:57:57.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf87fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 0x7fbf88159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:48362/0 (socket says 172.21.15.7:48362) 2023-06-09T11:57:57.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.906+0000 7fbf877fe700 1 -- 172.21.15.7:0/868198620 learned_addr learned my addr 172.21.15.7:0/868198620 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf877fe700 1 -- 172.21.15.7:0/868198620 >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 msgr2=0x7fbf88159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:57:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf877fe700 1 --2- 172.21.15.7:0/868198620 >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 0x7fbf88159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf877fe700 1 -- 172.21.15.7:0/868198620 >> v1:172.21.15.7:6789/0 conn(0x7fbf880c4ce0 legacy=0x7fbf8815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:57.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf877fe700 1 -- 172.21.15.7:0/868198620 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8806a570 con 0x7fbf881539b0 2023-06-09T11:57:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf877fe700 1 --2- 172.21.15.7:0/868198620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf880c9f60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fbf88073d90 tx=0x7fbf7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36320c67470c842 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf86ffd700 1 -- 172.21.15.7:0/868198620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c011830 con 0x7fbf881539b0 2023-06-09T11:57:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf86ffd700 1 -- 172.21.15.7:0/868198620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c011990 con 0x7fbf881539b0 2023-06-09T11:57:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.907+0000 7fbf86ffd700 1 -- 172.21.15.7:0/868198620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c011d20 con 0x7fbf881539b0 2023-06-09T11:57:57.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/868198620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 msgr2=0x7fbf880c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/868198620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf880c9f60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fbf88073d90 tx=0x7fbf7c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/868198620 shutdown_connections 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/868198620 >> v2:172.21.15.117:3300/0 conn(0x7fbf881535e0 0x7fbf88159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/868198620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf880c9f60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/868198620 >> 172.21.15.7:0/868198620 conn(0x7fbf880737f0 msgr2=0x7fbf88072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:57.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/868198620 shutdown_connections 2023-06-09T11:57:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.908+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/868198620 wait complete. 2023-06-09T11:57:57.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.910+0000 7fbf8e3ff700 1 Processor -- start 2023-06-09T11:57:57.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.910+0000 7fbf8e3ff700 1 -- start start 2023-06-09T11:57:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf880c4ce0 0x7fbf881584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf881535e0 0x7fbf881ef860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881ef440 con 0x7fbf881539b0 2023-06-09T11:57:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf881e4d30 con 0x7fbf881535e0 2023-06-09T11:57:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880b2e70 con 0x7fbf880c4ce0 2023-06-09T11:57:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8c99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8c99c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48236/0 (socket says 172.21.15.7:48236) 2023-06-09T11:57:57.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf8c99c700 1 -- 172.21.15.7:0/3185088864 learned_addr learned my addr 172.21.15.7:0/3185088864 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:57:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf87fff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf880c4ce0 0x7fbf881584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.911+0000 7fbf877fe700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf881535e0 0x7fbf881ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 -- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf880c4ce0 msgr2=0x7fbf881584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf880c4ce0 0x7fbf881584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.920 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 -- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf881535e0 msgr2=0x7fbf881ef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf881535e0 0x7fbf881ef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 -- 172.21.15.7:0/3185088864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009550 con 0x7fbf881539b0 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8c99c700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000cc80 tx=0x7fbf8000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf80003c00 con 0x7fbf881539b0 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf80003d60 con 0x7fbf881539b0 2023-06-09T11:57:57.921 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000aaa0 con 0x7fbf881539b0 2023-06-09T11:57:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8806ab30 con 0x7fbf881539b0 2023-06-09T11:57:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.912+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8806a850 con 0x7fbf881539b0 2023-06-09T11:57:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.913+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8806ae10 con 0x7fbf881539b0 2023-06-09T11:57:57.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.914+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf8000ac00 con 0x7fbf881539b0 2023-06-09T11:57:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.915+0000 7fbf84ff9700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 0x7fbf7406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:57:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.915+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fbf80089a60 con 0x7fbf881539b0 2023-06-09T11:57:57.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.915+0000 7fbf87fff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 0x7fbf7406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:57:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.916+0000 7fbf87fff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 0x7fbf7406d1c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009be0 tx=0x7fbf78009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:57:57.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:57.920+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbf800542a0 con 0x7fbf881539b0 2023-06-09T11:57:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.180+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf881ef580 con 0x7fbf881539b0 2023-06-09T11:57:58.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.180+0000 7fbf84ff9700 1 -- 172.21.15.7:0/3185088864 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7fbf881ef580 con 0x7fbf881539b0 2023-06-09T11:57:58.184 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:57:58.184 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:57:58.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 msgr2=0x7fbf7406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 0x7fbf7406d1c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009be0 tx=0x7fbf78009210 comp rx=0 tx=0).stop 2023-06-09T11:57:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 msgr2=0x7fbf8814ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:57:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000cc80 tx=0x7fbf8000cf90 comp rx=0 tx=0).stop 2023-06-09T11:57:58.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 shutdown_connections 2023-06-09T11:57:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf881539b0 0x7fbf8814ce50 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbf7406ad10 0x7fbf7406d1c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbf880c4ce0 0x7fbf881584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 --2- 172.21.15.7:0/3185088864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf881535e0 0x7fbf881ef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:57:58.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 >> 172.21.15.7:0/3185088864 conn(0x7fbf880737f0 msgr2=0x7fbf8815aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:57:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 shutdown_connections 2023-06-09T11:57:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:57:58.183+0000 7fbf8e3ff700 1 -- 172.21.15.7:0/3185088864 wait complete. 2023-06-09T11:57:58.189 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:57:58.204 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:58:00.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.413+0000 7fb9da7ff700 1 Processor -- start 2023-06-09T11:58:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.414+0000 7fb9da7ff700 1 -- start start 2023-06-09T11:58:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9da7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d40c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9da7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d40cad00 0x7fb9d40cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9da7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d415cec0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9da7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d415d160 con 0x7fb9d40cad00 2023-06-09T11:58:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9da7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d415d440 con 0x7fb9d40c9090 2023-06-09T11:58:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d40cad00 0x7fb9d40cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9d37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d40cad00 0x7fb9d40cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48248/0 (socket says 172.21.15.7:48248) 2023-06-09T11:58:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9d37fe700 1 -- 172.21.15.7:0/959396274 learned_addr learned my addr 172.21.15.7:0/959396274 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9d8d9c700 1 --1- 172.21.15.7:0/959396274 >> v1:172.21.15.7:6789/0 conn(0x7fb9d40cfaf0 0x7fb9d41598b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55974/0 (socket says 172.21.15.7:55974) 2023-06-09T11:58:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.415+0000 7fb9d3fff700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d40c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d37fe700 1 -- 172.21.15.7:0/959396274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 msgr2=0x7fb9d40c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d37fe700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d40c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d37fe700 1 -- 172.21.15.7:0/959396274 >> v1:172.21.15.7:6789/0 conn(0x7fb9d40cfaf0 legacy=0x7fb9d41598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d37fe700 1 -- 172.21.15.7:0/959396274 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d415d720 con 0x7fb9d40cad00 2023-06-09T11:58:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d37fe700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cad00 0x7fb9d40cf5b0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcecfc1c61a1f6c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d2ffd700 1 -- 172.21.15.7:0/959396274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8011830 con 0x7fb9d40cad00 2023-06-09T11:58:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d2ffd700 1 -- 172.21.15.7:0/959396274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c8011990 con 0x7fb9d40cad00 2023-06-09T11:58:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.416+0000 7fb9d2ffd700 1 -- 172.21.15.7:0/959396274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8011d20 con 0x7fb9d40cad00 2023-06-09T11:58:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 -- 172.21.15.7:0/959396274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cad00 msgr2=0x7fb9d40cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cad00 0x7fb9d40cf5b0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8009c00 comp rx=0 tx=0).stop 2023-06-09T11:58:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 -- 172.21.15.7:0/959396274 shutdown_connections 2023-06-09T11:58:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d40c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/959396274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cad00 0x7fb9d40cf5b0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 -- 172.21.15.7:0/959396274 >> 172.21.15.7:0/959396274 conn(0x7fb9d4070af0 msgr2=0x7fb9d4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 -- 172.21.15.7:0/959396274 shutdown_connections 2023-06-09T11:58:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.417+0000 7fb9da7ff700 1 -- 172.21.15.7:0/959396274 wait complete. 2023-06-09T11:58:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.419+0000 7fb9da7ff700 1 Processor -- start 2023-06-09T11:58:00.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.419+0000 7fb9da7ff700 1 -- start start 2023-06-09T11:58:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d4154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 0x7fb9d41ef180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c4a90 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c64b0 con 0x7fb9d40cad00 2023-06-09T11:58:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9da7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d41e3290 con 0x7fb9d40c9090 2023-06-09T11:58:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d3fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d4154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48250/0 (socket says 172.21.15.7:48250) 2023-06-09T11:58:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 -- 172.21.15.7:0/1944793823 learned_addr learned my addr 172.21.15.7:0/1944793823 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d37fe700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 0x7fb9d41ef180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 -- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 msgr2=0x7fb9d4154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d4154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 -- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 msgr2=0x7fb9d41ef180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 0x7fb9d41ef180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d8d9c700 1 -- 172.21.15.7:0/1944793823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009550 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.421+0000 7fb9d37fe700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 0x7fb9d41ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:00.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9d8d9c700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00cbc0 tx=0x7fb9cc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc00aaa0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc00ac00 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d40cb130 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc007070 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.422+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d40ca620 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.423+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40c4bd0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.424+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9cc0071d0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.425+0000 7fb9d0ff9700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 0x7fb9c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.425+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fb9cc089910 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.425+0000 7fb9d3fff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 0x7fb9c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:00.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.426+0000 7fb9d3fff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 0x7fb9c006d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009b40 tx=0x7fb9c4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.430+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb9d40c4bd0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.688+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9d40c4bd0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.689+0000 7fb9d0ff9700 1 -- 172.21.15.7:0/1944793823 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7fb9d40c4bd0 con 0x7fb9d40cfaf0 2023-06-09T11:58:00.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:00.693 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:00.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 msgr2=0x7fb9c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 0x7fb9c006d190 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009b40 tx=0x7fb9c4009210 comp rx=0 tx=0).stop 2023-06-09T11:58:00.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 msgr2=0x7fb9d40c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:00.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00cbc0 tx=0x7fb9cc00ced0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 shutdown_connections 2023-06-09T11:58:00.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cfaf0 0x7fb9d40c4eb0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9c006ace0 0x7fb9c006d190 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d40c9090 0x7fb9d4154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 --2- 172.21.15.7:0/1944793823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d40cad00 0x7fb9d41ef180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:00.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 >> 172.21.15.7:0/1944793823 conn(0x7fb9d4070af0 msgr2=0x7fb9d40cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:00.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 shutdown_connections 2023-06-09T11:58:00.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:00.691+0000 7fb9da7ff700 1 -- 172.21.15.7:0/1944793823 wait complete. 2023-06-09T11:58:00.697 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:58:00.713 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:58:02.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:02.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.921+0000 7f6f85bff700 1 Processor -- start 2023-06-09T11:58:02.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.921+0000 7f6f85bff700 1 -- start start 2023-06-09T11:58:02.925 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f85bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6f80153880 0x7f6f8015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f85bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8015b4a0 0x7f6f800c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f85bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f8015a500 con 0x7f6f800c9f40 2023-06-09T11:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f85bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f800c5f10 con 0x7f6f80153880 2023-06-09T11:58:02.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f85bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8015da00 con 0x7f6f8015b4a0 2023-06-09T11:58:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f7effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6f800c9f40 0x7f6f8015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:55988/0 (socket says 172.21.15.7:55988) 2023-06-09T11:58:02.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.922+0000 7f6f7effd700 1 -- 172.21.15.7:0/3860674791 learned_addr learned my addr 172.21.15.7:0/3860674791 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:02.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/3860674791 >> v2:172.21.15.117:3300/0 conn(0x7f6f80153880 0x7f6f8015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7ffff700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8015b4a0 0x7f6f800c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7e7fc700 1 -- 172.21.15.7:0/3860674791 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319743430 0 0) 0x7f6f74001110 con 0x7f6f800c9f40 2023-06-09T11:58:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7e7fc700 1 -- 172.21.15.7:0/3860674791 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f680035b0 con 0x7f6f800c9f40 2023-06-09T11:58:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8015b4a0 msgr2=0x7f6f800c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8015b4a0 0x7f6f800c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:02.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/3860674791 >> v1:172.21.15.7:6789/0 conn(0x7f6f800c9f40 legacy=0x7f6f8015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/3860674791 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f801596e0 con 0x7f6f80153880 2023-06-09T11:58:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.923+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80153880 0x7f6f8015d4c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009840 tx=0x7f6f70009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7fcef183d8bb60 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f7e7fc700 1 -- 172.21.15.7:0/3860674791 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70011830 con 0x7f6f80153880 2023-06-09T11:58:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f7e7fc700 1 -- 172.21.15.7:0/3860674791 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70011990 con 0x7f6f80153880 2023-06-09T11:58:02.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f7e7fc700 1 -- 172.21.15.7:0/3860674791 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70011d20 con 0x7f6f80153880 2023-06-09T11:58:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 -- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80153880 msgr2=0x7f6f8015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80153880 0x7f6f8015d4c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009840 tx=0x7f6f70009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 -- 172.21.15.7:0/3860674791 shutdown_connections 2023-06-09T11:58:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f8015b4a0 0x7f6f800c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:02.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 --2- 172.21.15.7:0/3860674791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f80153880 0x7f6f8015d4c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 -- 172.21.15.7:0/3860674791 >> 172.21.15.7:0/3860674791 conn(0x7f6f800737f0 msgr2=0x7f6f800720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.924+0000 7f6f85bff700 1 -- 172.21.15.7:0/3860674791 shutdown_connections 2023-06-09T11:58:02.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.925+0000 7f6f85bff700 1 -- 172.21.15.7:0/3860674791 wait complete. 2023-06-09T11:58:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.926+0000 7f6f85bff700 1 Processor -- start 2023-06-09T11:58:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.926+0000 7f6f85bff700 1 -- start start 2023-06-09T11:58:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f80153880 0x7f6f801f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8015b4a0 0x7f6f801f2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f800ce870 con 0x7f6f800c9f40 2023-06-09T11:58:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f800cb620 con 0x7f6f8015b4a0 2023-06-09T11:58:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f85bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f800cab10 con 0x7f6f80153880 2023-06-09T11:58:02.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.927+0000 7f6f7f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48258/0 (socket says 172.21.15.7:48258) 2023-06-09T11:58:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/165316730 learned_addr learned my addr 172.21.15.7:0/165316730 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7effd700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f80153880 0x7f6f801f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7ffff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8015b4a0 0x7f6f801f2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f80153880 msgr2=0x7f6f801f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:02.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f80153880 0x7f6f801f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:02.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8015b4a0 msgr2=0x7f6f801f2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:02.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8015b4a0 0x7f6f801f2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:02.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 -- 172.21.15.7:0/165316730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700094a0 con 0x7f6f800c9f40 2023-06-09T11:58:02.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f7f7fe700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000c00 tx=0x7f6f70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.928+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70004590 con 0x7f6f800c9f40 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.929+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7001f070 con 0x7f6f800c9f40 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.929+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f700038e0 con 0x7f6f800c9f40 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.929+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f80159c60 con 0x7f6f800c9f40 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.929+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f80159980 con 0x7f6f800c9f40 2023-06-09T11:58:02.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.930+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f80159f40 con 0x7f6f800c9f40 2023-06-09T11:58:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.930+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6f70003a40 con 0x7f6f800c9f40 2023-06-09T11:58:02.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.931+0000 7f6f67fff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 0x7f6f6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:02.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.931+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7f6f70021070 con 0x7f6f800c9f40 2023-06-09T11:58:02.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.932+0000 7f6f7effd700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 0x7f6f6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:02.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.932+0000 7f6f7effd700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 0x7f6f6c06d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006b4b0 tx=0x7f6f74007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:02.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:02.936+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6f70055c60 con 0x7f6f800c9f40 2023-06-09T11:58:03.193 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.190+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f80074cf0 con 0x7f6f800c9f40 2023-06-09T11:58:03.194 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.190+0000 7f6f67fff700 1 -- 172.21.15.7:0/165316730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7f6f70059280 con 0x7f6f800c9f40 2023-06-09T11:58:03.194 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:03.194 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:03.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 msgr2=0x7f6f6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:03.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 0x7f6f6c06d190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6f8006b4b0 tx=0x7f6f74007c40 comp rx=0 tx=0).stop 2023-06-09T11:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 msgr2=0x7f6f800cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f6f70000c00 tx=0x7f6f70004750 comp rx=0 tx=0).stop 2023-06-09T11:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 shutdown_connections 2023-06-09T11:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6f8015b4a0 0x7f6f801f2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:03.197 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6f800c9f40 0x7f6f800cbab0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6f6c06ace0 0x7f6f6c06d190 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 --2- 172.21.15.7:0/165316730 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6f80153880 0x7f6f801f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 >> 172.21.15.7:0/165316730 conn(0x7f6f800737f0 msgr2=0x7f6f8014e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 shutdown_connections 2023-06-09T11:58:03.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:03.193+0000 7f6f85bff700 1 -- 172.21.15.7:0/165316730 wait complete. 2023-06-09T11:58:03.199 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:58:03.215 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:58:05.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.423+0000 7fc4493ff700 1 Processor -- start 2023-06-09T11:58:05.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.423+0000 7fc4493ff700 1 -- start start 2023-06-09T11:58:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.424+0000 7fc4493ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc4440cb5d0 0x7fc4440b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.424+0000 7fc4493ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4440cb9a0 0x7fc444159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.424+0000 7fc4493ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc444159b20 con 0x7fc44415cd40 2023-06-09T11:58:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.424+0000 7fc4493ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4440cac80 con 0x7fc4440cb9a0 2023-06-09T11:58:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.424+0000 7fc4493ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4441597d0 con 0x7fc4440cb5d0 2023-06-09T11:58:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4440cb9a0 0x7fc444159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc4440cb9a0 0x7fc444159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40258/0 (socket says 172.21.15.7:40258) 2023-06-09T11:58:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 -- 172.21.15.7:0/1659981977 learned_addr learned my addr 172.21.15.7:0/1659981977 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc442ffd700 1 --2- 172.21.15.7:0/1659981977 >> v2:172.21.15.117:3300/0 conn(0x7fc4440cb5d0 0x7fc4440b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 -- 172.21.15.7:0/1659981977 >> v2:172.21.15.117:3300/0 conn(0x7fc4440cb5d0 msgr2=0x7fc4440b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 --2- 172.21.15.7:0/1659981977 >> v2:172.21.15.117:3300/0 conn(0x7fc4440cb5d0 0x7fc4440b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 -- 172.21.15.7:0/1659981977 >> v1:172.21.15.117:6789/0 conn(0x7fc44415cd40 legacy=0x7fc4440cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 -- 172.21.15.7:0/1659981977 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444071ab0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc441ffb700 1 -- 172.21.15.7:0/1659981977 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090555339 0 0) 0x7fc444159b20 con 0x7fc44415cd40 2023-06-09T11:58:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.425+0000 7fc4427fc700 1 --2- 172.21.15.7:0/1659981977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc444159fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fc44408d420 tx=0x7fc438009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca09021502aeb8cc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc441ffb700 1 -- 172.21.15.7:0/1659981977 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011830 con 0x7fc4440cb9a0 2023-06-09T11:58:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc441ffb700 1 -- 172.21.15.7:0/1659981977 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438011990 con 0x7fc4440cb9a0 2023-06-09T11:58:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc441ffb700 1 -- 172.21.15.7:0/1659981977 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438011d20 con 0x7fc4440cb9a0 2023-06-09T11:58:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 -- 172.21.15.7:0/1659981977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 msgr2=0x7fc444159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 --2- 172.21.15.7:0/1659981977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc444159fb0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fc44408d420 tx=0x7fc438009bd0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 -- 172.21.15.7:0/1659981977 shutdown_connections 2023-06-09T11:58:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 --2- 172.21.15.7:0/1659981977 >> v2:172.21.15.117:3300/0 conn(0x7fc4440cb5d0 0x7fc4440b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 --2- 172.21.15.7:0/1659981977 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc444159fb0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 -- 172.21.15.7:0/1659981977 >> 172.21.15.7:0/1659981977 conn(0x7fc444070af0 msgr2=0x7fc444070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 -- 172.21.15.7:0/1659981977 shutdown_connections 2023-06-09T11:58:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.426+0000 7fc4493ff700 1 -- 172.21.15.7:0/1659981977 wait complete. 2023-06-09T11:58:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.428+0000 7fc4493ff700 1 Processor -- start 2023-06-09T11:58:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.428+0000 7fc4493ff700 1 -- start start 2023-06-09T11:58:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4440cb5d0 0x7fc4440c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc44415cd40 0x7fc4440ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4441545b0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4440cf8f0 con 0x7fc4440cb5d0 2023-06-09T11:58:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4493ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc444152f50 con 0x7fc44415cd40 2023-06-09T11:58:05.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4427fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40268/0 (socket says 172.21.15.7:40268) 2023-06-09T11:58:05.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4427fc700 1 -- 172.21.15.7:0/76023732 learned_addr learned my addr 172.21.15.7:0/76023732 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:05.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc442ffd700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4440cb5d0 0x7fc4440c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.429+0000 7fc4437fe700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc44415cd40 0x7fc4440ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 -- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc44415cd40 msgr2=0x7fc4440ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc44415cd40 0x7fc4440ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 -- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4440cb5d0 msgr2=0x7fc4440c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4440cb5d0 0x7fc4440c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 -- 172.21.15.7:0/76023732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438009550 con 0x7fc4440cb9a0 2023-06-09T11:58:05.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4427fc700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc438009c10 tx=0x7fc438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:05.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438004510 con 0x7fc4440cb9a0 2023-06-09T11:58:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4380036a0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44415d4f0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.430+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438003a10 con 0x7fc4440cb9a0 2023-06-09T11:58:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.431+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44415d210 con 0x7fc4440cb9a0 2023-06-09T11:58:05.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.431+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44415d7d0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.432+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc438003b70 con 0x7fc4440cb9a0 2023-06-09T11:58:05.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.434+0000 7fc42b7fe700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 0x7fc43006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.434+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fc438021030 con 0x7fc4440cb9a0 2023-06-09T11:58:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.434+0000 7fc442ffd700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 0x7fc43006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.435+0000 7fc442ffd700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 0x7fc43006d160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc434009be0 tx=0x7fc434009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:05.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.439+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc438055b10 con 0x7fc4440cb9a0 2023-06-09T11:58:05.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.691+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc444074cf0 con 0x7fc4440cb9a0 2023-06-09T11:58:05.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.692+0000 7fc42b7fe700 1 -- 172.21.15.7:0/76023732 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7fc438059130 con 0x7fc4440cb9a0 2023-06-09T11:58:05.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:05.696 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:05.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 msgr2=0x7fc43006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 0x7fc43006d160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fc434009be0 tx=0x7fc434009210 comp rx=0 tx=0).stop 2023-06-09T11:58:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 msgr2=0x7fc4440cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc438009c10 tx=0x7fc438004750 comp rx=0 tx=0).stop 2023-06-09T11:58:05.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 shutdown_connections 2023-06-09T11:58:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc43006acb0 0x7fc43006d160 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc44415cd40 0x7fc4440ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc4440cb5d0 0x7fc4440c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 --2- 172.21.15.7:0/76023732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4440cb9a0 0x7fc4440cdb20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:05.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 >> 172.21.15.7:0/76023732 conn(0x7fc444070af0 msgr2=0x7fc4440cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 shutdown_connections 2023-06-09T11:58:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:05.694+0000 7fc4493ff700 1 -- 172.21.15.7:0/76023732 wait complete. 2023-06-09T11:58:05.700 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:58:05.716 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:58:07.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.924+0000 7fdc517ff700 1 Processor -- start 2023-06-09T11:58:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.924+0000 7fdc517ff700 1 -- start start 2023-06-09T11:58:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.925+0000 7fdc517ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.925+0000 7fdc517ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c15cd40 0x7fdc4c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.925+0000 7fdc517ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4c159b20 con 0x7fdc4c0cb9a0 2023-06-09T11:58:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc517ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4c0cac80 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc517ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1597d0 con 0x7fdc4c15cd40 2023-06-09T11:58:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40280/0 (socket says 172.21.15.7:40280) 2023-06-09T11:58:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 -- 172.21.15.7:0/3282260158 learned_addr learned my addr 172.21.15.7:0/3282260158 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4b7fe700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c15cd40 0x7fdc4c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 -- 172.21.15.7:0/3282260158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c15cd40 msgr2=0x7fdc4c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c15cd40 0x7fdc4c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 -- 172.21.15.7:0/3282260158 >> v1:172.21.15.117:6789/0 conn(0x7fdc4c0cb9a0 legacy=0x7fdc4c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:07.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 -- 172.21.15.7:0/3282260158 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c071ab0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.926+0000 7fdc4affd700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098f0 tx=0x7fdc3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e945d69e2dabe7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc49ffb700 1 -- 172.21.15.7:0/3282260158 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00b8a0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc49ffb700 1 -- 172.21.15.7:0/3282260158 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00ba00 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc49ffb700 1 -- 172.21.15.7:0/3282260158 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00bd90 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 -- 172.21.15.7:0/3282260158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 msgr2=0x7fdc4c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0098f0 tx=0x7fdc3c009c00 comp rx=0 tx=0).stop 2023-06-09T11:58:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 -- 172.21.15.7:0/3282260158 shutdown_connections 2023-06-09T11:58:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c15cd40 0x7fdc4c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 --2- 172.21.15.7:0/3282260158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0b52b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:07.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 -- 172.21.15.7:0/3282260158 >> 172.21.15.7:0/3282260158 conn(0x7fdc4c070af0 msgr2=0x7fdc4c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 -- 172.21.15.7:0/3282260158 shutdown_connections 2023-06-09T11:58:07.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.927+0000 7fdc517ff700 1 -- 172.21.15.7:0/3282260158 wait complete. 2023-06-09T11:58:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.929+0000 7fdc517ff700 1 Processor -- start 2023-06-09T11:58:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.929+0000 7fdc517ff700 1 -- start start 2023-06-09T11:58:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 0x7fdc4c0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc4c15cd40 0x7fdc4c1f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c0c4af0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1ee700 con 0x7fdc4c15cd40 2023-06-09T11:58:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc517ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c0c3190 con 0x7fdc4c0cb9a0 2023-06-09T11:58:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.930+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40290/0 (socket says 172.21.15.7:40290) 2023-06-09T11:58:07.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 -- 172.21.15.7:0/2509695626 learned_addr learned my addr 172.21.15.7:0/2509695626 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:07.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4b7fe700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc4c15cd40 0x7fdc4c1f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4a7fc700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 0x7fdc4c0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 -- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 msgr2=0x7fdc4c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:07.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 0x7fdc4c0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 -- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc4c15cd40 msgr2=0x7fdc4c1f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc4c15cd40 0x7fdc4c1f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 -- 172.21.15.7:0/2509695626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c009550 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4a7fc700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 0x7fdc4c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:07.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc4affd700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.931+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c004480 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.932+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c01f070 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.932+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c15d4f0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.932+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0038c0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.932+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c15d210 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.933+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c15d7d0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.934+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc3c021030 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.934+0000 7fdc337fe700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 0x7fdc3806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.935+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 8184+0+0 (secure 0 0 0) 0x7fdc3c08b280 con 0x7fdc4c0cb5d0 2023-06-09T11:58:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.935+0000 7fdc4a7fc700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 0x7fdc3806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.935+0000 7fdc4a7fc700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 0x7fdc3806d1c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15cce0 tx=0x7fdc40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:07.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:07.939+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdc3c055be0 con 0x7fdc4c0cb5d0 2023-06-09T11:58:08.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.193+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc4c1ef240 con 0x7fdc4c0cb5d0 2023-06-09T11:58:08.196 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.193+0000 7fdc337fe700 1 -- 172.21.15.7:0/2509695626 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 218 v218) v1 ==== 95+0+6212 (secure 0 0 0) 0x7fdc3c059200 con 0x7fdc4c0cb5d0 2023-06-09T11:58:08.197 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:08.197 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":218,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":218}],"filesystems":[{"mdsmap":{"epoch":217,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:57:49.953252+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":73,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7437,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7437":{"gid":7437,"name":"e","rank":0,"incarnation":152,"state":"up:active","state_seq":32,"addr":"172.21.15.7:6839/1569034808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1569034808},{"type":"v1","addr":"172.21.15.7:6839","nonce":1569034808}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 msgr2=0x7fdc3806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:08.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 0x7fdc3806d1c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15cce0 tx=0x7fdc40009040 comp rx=0 tx=0).stop 2023-06-09T11:58:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 msgr2=0x7fdc4c0cfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c00 tx=0x7fdc3c004750 comp rx=0 tx=0).stop 2023-06-09T11:58:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 shutdown_connections 2023-06-09T11:58:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdc4c15cd40 0x7fdc4c1f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:08.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc4c0cb5d0 0x7fdc4c0cfad0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdc3806ad10 0x7fdc3806d1c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 --2- 172.21.15.7:0/2509695626 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdc4c0cb9a0 0x7fdc4c0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 >> 172.21.15.7:0/2509695626 conn(0x7fdc4c070af0 msgr2=0x7fdc4c158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 shutdown_connections 2023-06-09T11:58:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:08.196+0000 7fdc517ff700 1 -- 172.21.15.7:0/2509695626 wait complete. 2023-06-09T11:58:08.202 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 218 2023-06-09T11:58:08.218 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is laggy/crashed, in failed state, or mds.e is removed from mdsmap 2023-06-09T11:58:10.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:10.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.429+0000 7fe88bfff700 1 Processor -- start 2023-06-09T11:58:10.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.430+0000 7fe88bfff700 1 -- start start 2023-06-09T11:58:10.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe88415b4a0 0x7fe88415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8840c9f40 0x7fe8840c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe88415a500 con 0x7fe884153880 2023-06-09T11:58:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8840c5f10 con 0x7fe8840c9f40 2023-06-09T11:58:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88415da00 con 0x7fe88415b4a0 2023-06-09T11:58:10.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88a59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe8840c9f40 0x7fe8840c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe889d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe884153880 0x7fe88415ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48882/0 (socket says 172.21.15.7:48882) 2023-06-09T11:58:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe889d9b700 1 -- 172.21.15.7:0/3899083432 learned_addr learned my addr 172.21.15.7:0/3899083432 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:10.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.431+0000 7fe88959a700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe88415b4a0 0x7fe88415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe88a59c700 1 -- 172.21.15.7:0/3899083432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe88415b4a0 msgr2=0x7fe88415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe88a59c700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe88415b4a0 0x7fe88415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe88a59c700 1 -- 172.21.15.7:0/3899083432 >> v1:172.21.15.7:6789/0 conn(0x7fe884153880 legacy=0x7fe88415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe88a59c700 1 -- 172.21.15.7:0/3899083432 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8841596e0 con 0x7fe8840c9f40 2023-06-09T11:58:10.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe88a59c700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8840c9f40 0x7fe8840c4e30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009d90 tx=0x7fe87c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cec3918bd81b224 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe888d99700 1 -- 172.21.15.7:0/3899083432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c00e070 con 0x7fe8840c9f40 2023-06-09T11:58:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe888d99700 1 -- 172.21.15.7:0/3899083432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87c00bb20 con 0x7fe8840c9f40 2023-06-09T11:58:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.432+0000 7fe888d99700 1 -- 172.21.15.7:0/3899083432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87c012430 con 0x7fe8840c9f40 2023-06-09T11:58:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 -- 172.21.15.7:0/3899083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8840c9f40 msgr2=0x7fe8840c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8840c9f40 0x7fe8840c4e30 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009d90 tx=0x7fe87c00b330 comp rx=0 tx=0).stop 2023-06-09T11:58:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 -- 172.21.15.7:0/3899083432 shutdown_connections 2023-06-09T11:58:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8840c9f40 0x7fe8840c4e30 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3899083432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe88415b4a0 0x7fe88415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 -- 172.21.15.7:0/3899083432 >> 172.21.15.7:0/3899083432 conn(0x7fe8840737f0 msgr2=0x7fe8840720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:10.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 -- 172.21.15.7:0/3899083432 shutdown_connections 2023-06-09T11:58:10.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.433+0000 7fe88bfff700 1 -- 172.21.15.7:0/3899083432 wait complete. 2023-06-09T11:58:10.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.434+0000 7fe88bfff700 1 Processor -- start 2023-06-09T11:58:10.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.435+0000 7fe88bfff700 1 -- start start 2023-06-09T11:58:10.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.435+0000 7fe88bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 0x7fe884158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.435+0000 7fe88bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.435+0000 7fe88bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 0x7fe8840b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840b31c0 con 0x7fe884153880 2023-06-09T11:58:10.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8841582c0 con 0x7fe88415b4a0 2023-06-09T11:58:10.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe884158ba0 con 0x7fe8840c9f40 2023-06-09T11:58:10.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40298/0 (socket says 172.21.15.7:40298) 2023-06-09T11:58:10.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 -- 172.21.15.7:0/3414536344 learned_addr learned my addr 172.21.15.7:0/3414536344 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:10.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88a59c700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 0x7fe8840b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe889d9b700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 0x7fe884158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 -- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 msgr2=0x7fe884158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 0x7fe884158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 -- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 msgr2=0x7fe8840b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 0x7fe8840b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88959a700 1 -- 172.21.15.7:0/3414536344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c009a70 con 0x7fe884153880 2023-06-09T11:58:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe889d9b700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 0x7fe884158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:10.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.436+0000 7fe88a59c700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 0x7fe8840b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe88959a700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e8a0 tx=0x7fe87400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874009590 con 0x7fe884153880 2023-06-09T11:58:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400eea0 con 0x7fe884153880 2023-06-09T11:58:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe884159c60 con 0x7fe884153880 2023-06-09T11:58:10.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400c9f0 con 0x7fe884153880 2023-06-09T11:58:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.437+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe884159980 con 0x7fe884153880 2023-06-09T11:58:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.438+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe884159f40 con 0x7fe884153880 2023-06-09T11:58:10.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.439+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe87400cb50 con 0x7fe884153880 2023-06-09T11:58:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.440+0000 7fe87a7fc700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 0x7fe86806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.440+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fe874089ce0 con 0x7fe884153880 2023-06-09T11:58:10.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.440+0000 7fe889d9b700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 0x7fe86806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.441+0000 7fe889d9b700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 0x7fe86806d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe880007fd0 tx=0x7fe880009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.445+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe874057ed0 con 0x7fe884153880 2023-06-09T11:58:10.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.696+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8840b3430 con 0x7fe884153880 2023-06-09T11:58:10.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.697+0000 7fe87a7fc700 1 -- 172.21.15.7:0/3414536344 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+5455 (secure 0 0 0) 0x7fe874021090 con 0x7fe884153880 2023-06-09T11:58:10.700 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:10.701 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":219,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:09.773612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 msgr2=0x7fe86806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 0x7fe86806d190 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe880007fd0 tx=0x7fe880009040 comp rx=0 tx=0).stop 2023-06-09T11:58:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 msgr2=0x7fe8840cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe87400e8a0 tx=0x7fe87400ec60 comp rx=0 tx=0).stop 2023-06-09T11:58:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 shutdown_connections 2023-06-09T11:58:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe88415b4a0 0x7fe8840b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe86806ace0 0x7fe86806d190 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe8840c9f40 0x7fe884158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 --2- 172.21.15.7:0/3414536344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe884153880 0x7fe8840cc150 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 >> 172.21.15.7:0/3414536344 conn(0x7fe8840737f0 msgr2=0x7fe88414d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 shutdown_connections 2023-06-09T11:58:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.699+0000 7fe88bfff700 1 -- 172.21.15.7:0/3414536344 wait complete. 2023-06-09T11:58:10.705 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 219 2023-06-09T11:58:10.721 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e down, removed from mdsmap 2023-06-09T11:58:10.721 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for mds cluster to stabilize... 2023-06-09T11:58:10.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.921+0000 7fdf7f5ff700 1 Processor -- start 2023-06-09T11:58:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.921+0000 7fdf7f5ff700 1 -- start start 2023-06-09T11:58:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.922+0000 7fdf7f5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf780c5b40 0x7fdf78070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.922+0000 7fdf7f5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf781580a0 con 0x7fdf78151510 2023-06-09T11:58:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7f5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780c6180 con 0x7fdf78071060 2023-06-09T11:58:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7f5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf781518e0 con 0x7fdf780c5b40 2023-06-09T11:58:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf780c5b40 0x7fdf78070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdf780c5b40 0x7fdf78070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40302/0 (socket says 172.21.15.7:40302) 2023-06-09T11:58:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/2258689407 learned_addr learned my addr 172.21.15.7:0/2258689407 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/2258689407 >> v1:172.21.15.117:6789/0 conn(0x7fdf78071060 legacy=0x7fdf78158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/2258689407 >> v1:172.21.15.7:6789/0 conn(0x7fdf78151510 legacy=0x7fdf7815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/2258689407 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78157490 con 0x7fdf780c5b40 2023-06-09T11:58:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.923+0000 7fdf7cb9a700 1 --2- 172.21.15.7:0/2258689407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf78070b20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fdf680098f0 tx=0x7fdf68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8464f6bbefa036e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf6ffff700 1 -- 172.21.15.7:0/2258689407 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800b8a0 con 0x7fdf780c5b40 2023-06-09T11:58:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf6ffff700 1 -- 172.21.15.7:0/2258689407 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6800ba00 con 0x7fdf780c5b40 2023-06-09T11:58:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf6ffff700 1 -- 172.21.15.7:0/2258689407 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800bd90 con 0x7fdf780c5b40 2023-06-09T11:58:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/2258689407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 msgr2=0x7fdf78070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/2258689407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf78070b20 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fdf680098f0 tx=0x7fdf68009c00 comp rx=0 tx=0).stop 2023-06-09T11:58:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/2258689407 shutdown_connections 2023-06-09T11:58:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/2258689407 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf78070b20 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/2258689407 >> 172.21.15.7:0/2258689407 conn(0x7fdf7806a4d0 msgr2=0x7fdf7806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/2258689407 shutdown_connections 2023-06-09T11:58:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.924+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/2258689407 wait complete. 2023-06-09T11:58:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.926+0000 7fdf7f5ff700 1 Processor -- start 2023-06-09T11:58:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.926+0000 7fdf7f5ff700 1 -- start start 2023-06-09T11:58:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf78071060 0x7fdf781522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf78151510 0x7fdf781457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780c46e0 con 0x7fdf780c5b40 2023-06-09T11:58:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78146390 con 0x7fdf78151510 2023-06-09T11:58:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7f5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78152830 con 0x7fdf78071060 2023-06-09T11:58:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7cb9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40312/0 (socket says 172.21.15.7:40312) 2023-06-09T11:58:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/3872095542 learned_addr learned my addr 172.21.15.7:0/3872095542 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7d39b700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf78071060 0x7fdf781522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.927+0000 7fdf7db9c700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf78151510 0x7fdf781457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf78071060 msgr2=0x7fdf781522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf78071060 0x7fdf781522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf78151510 msgr2=0x7fdf781457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf78151510 0x7fdf781457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 -- 172.21.15.7:0/3872095542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68009550 con 0x7fdf780c5b40 2023-06-09T11:58:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7cb9a700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009c40 tx=0x7fdf68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf68004380 con 0x7fdf780c5b40 2023-06-09T11:58:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6801f070 con 0x7fdf780c5b40 2023-06-09T11:58:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf680036a0 con 0x7fdf780c5b40 2023-06-09T11:58:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf78157a50 con 0x7fdf780c5b40 2023-06-09T11:58:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.928+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf78157770 con 0x7fdf780c5b40 2023-06-09T11:58:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.929+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78157d30 con 0x7fdf780c5b40 2023-06-09T11:58:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.930+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf68021030 con 0x7fdf780c5b40 2023-06-09T11:58:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.931+0000 7fdf6dffb700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 0x7fdf5c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.931+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fdf6808b1a0 con 0x7fdf780c5b40 2023-06-09T11:58:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.932+0000 7fdf7d39b700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 0x7fdf5c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.932+0000 7fdf7d39b700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 0x7fdf5c06d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007fd0 tx=0x7fdf74009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:10.936+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdf68055900 con 0x7fdf780c5b40 2023-06-09T11:58:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.192+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf78074cf0 con 0x7fdf780c5b40 2023-06-09T11:58:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.192+0000 7fdf6dffb700 1 -- 172.21.15.7:0/3872095542 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+5455 (secure 0 0 0) 0x7fdf68058f20 con 0x7fdf780c5b40 2023-06-09T11:58:11.196 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:11.196 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":219,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:09.773612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:11.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 msgr2=0x7fdf5c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:11.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 0x7fdf5c06d190 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007fd0 tx=0x7fdf74009040 comp rx=0 tx=0).stop 2023-06-09T11:58:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 msgr2=0x7fdf7814ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fdf68009c40 tx=0x7fdf68004750 comp rx=0 tx=0).stop 2023-06-09T11:58:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 shutdown_connections 2023-06-09T11:58:11.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf78151510 0x7fdf781457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:11.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf5c06ace0 0x7fdf5c06d190 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:11.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf78071060 0x7fdf781522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:11.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 --2- 172.21.15.7:0/3872095542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdf780c5b40 0x7fdf7814ddf0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:11.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 >> 172.21.15.7:0/3872095542 conn(0x7fdf7806a4d0 msgr2=0x7fdf781558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:11.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 shutdown_connections 2023-06-09T11:58:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:11.195+0000 7fdf7f5ff700 1 -- 172.21.15.7:0/3872095542 wait complete. 2023-06-09T11:58:11.201 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 219 2023-06-09T11:58:11.217 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 219, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 219, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:replay", "state_seq": 1 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:09.773612+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:58:13.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:13.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.425+0000 7f693a3ff700 1 Processor -- start 2023-06-09T11:58:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.425+0000 7f693a3ff700 1 -- start start 2023-06-09T11:58:13.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693a3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 0x7f69340b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:13.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693a3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6934159b20 con 0x7f693415cd40 2023-06-09T11:58:13.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693a3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69340cac80 con 0x7f69340cb9a0 2023-06-09T11:58:13.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693a3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69341597d0 con 0x7f69340cb5d0 2023-06-09T11:58:13.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f6933fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 0x7f69340b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:13.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693899c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f693415cd40 0x7f69340cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36530/0 (socket says 172.21.15.7:36530) 2023-06-09T11:58:13.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f693899c700 1 -- 172.21.15.7:0/1226674 learned_addr learned my addr 172.21.15.7:0/1226674 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:13.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f6933fff700 1 -- 172.21.15.7:0/1226674 >> v1:172.21.15.117:6789/0 conn(0x7f69340cb9a0 legacy=0x7f6934159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f6932ffd700 1 -- 172.21.15.7:0/1226674 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247999766 0 0) 0x7f6934159b20 con 0x7f693415cd40 2023-06-09T11:58:13.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f6933fff700 1 -- 172.21.15.7:0/1226674 >> v1:172.21.15.7:6789/0 conn(0x7f693415cd40 legacy=0x7f69340cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.427+0000 7f6933fff700 1 -- 172.21.15.7:0/1226674 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6934071ab0 con 0x7f69340cb5d0 2023-06-09T11:58:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.428+0000 7f6933fff700 1 --2- 172.21.15.7:0/1226674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 0x7f69340b52b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6924009870 tx=0x7f6924009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=167f654931a83c7a server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.428+0000 7f6932ffd700 1 -- 172.21.15.7:0/1226674 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924011830 con 0x7f69340cb5d0 2023-06-09T11:58:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.428+0000 7f6932ffd700 1 -- 172.21.15.7:0/1226674 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924011990 con 0x7f69340cb5d0 2023-06-09T11:58:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.428+0000 7f6932ffd700 1 -- 172.21.15.7:0/1226674 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924011d20 con 0x7f69340cb5d0 2023-06-09T11:58:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 -- 172.21.15.7:0/1226674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 msgr2=0x7f69340b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 --2- 172.21.15.7:0/1226674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 0x7f69340b52b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6924009870 tx=0x7f6924009b80 comp rx=0 tx=0).stop 2023-06-09T11:58:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 -- 172.21.15.7:0/1226674 shutdown_connections 2023-06-09T11:58:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 --2- 172.21.15.7:0/1226674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69340cb5d0 0x7f69340b52b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 -- 172.21.15.7:0/1226674 >> 172.21.15.7:0/1226674 conn(0x7f6934070af0 msgr2=0x7f6934070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:13.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 -- 172.21.15.7:0/1226674 shutdown_connections 2023-06-09T11:58:13.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.429+0000 7f693a3ff700 1 -- 172.21.15.7:0/1226674 wait complete. 2023-06-09T11:58:13.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.430+0000 7f693a3ff700 1 Processor -- start 2023-06-09T11:58:13.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.431+0000 7f693a3ff700 1 -- start start 2023-06-09T11:58:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69340cb5d0 0x7f693415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693415cd40 0x7f69340c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69340acbf0 con 0x7f69340cb9a0 2023-06-09T11:58:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69340c8990 con 0x7f69340cb5d0 2023-06-09T11:58:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693a3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340b1e60 con 0x7f693415cd40 2023-06-09T11:58:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57780/0 (socket says 172.21.15.7:57780) 2023-06-09T11:58:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 -- 172.21.15.7:0/3489600069 learned_addr learned my addr 172.21.15.7:0/3489600069 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f6933fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69340cb5d0 0x7f693415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f693899c700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693415cd40 0x7f69340c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 -- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693415cd40 msgr2=0x7f69340c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693415cd40 0x7f69340c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 -- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69340cb5d0 msgr2=0x7f693415be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69340cb5d0 0x7f693415be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.432+0000 7f69337fe700 1 -- 172.21.15.7:0/3489600069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240094a0 con 0x7f69340cb9a0 2023-06-09T11:58:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f69337fe700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f692800c4b0 tx=0x7f692800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:13.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280052d0 con 0x7f69340cb9a0 2023-06-09T11:58:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928005430 con 0x7f69340cb9a0 2023-06-09T11:58:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280038c0 con 0x7f69340cb9a0 2023-06-09T11:58:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693415d4f0 con 0x7f69340cb9a0 2023-06-09T11:58:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.433+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693415d210 con 0x7f69340cb9a0 2023-06-09T11:58:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.434+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f693415d7d0 con 0x7f69340cb9a0 2023-06-09T11:58:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.435+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6928003a20 con 0x7f69340cb9a0 2023-06-09T11:58:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.436+0000 7f6930ff9700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 0x7f692006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.436+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f6928089190 con 0x7f69340cb9a0 2023-06-09T11:58:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.436+0000 7f6933fff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 0x7f692006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.437+0000 7f6933fff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 0x7f692006d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f69240090b0 tx=0x7f6924008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.440+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6928053860 con 0x7f69340cb9a0 2023-06-09T11:58:13.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.692+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69340caa60 con 0x7f69340cb9a0 2023-06-09T11:58:13.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.693+0000 7f6930ff9700 1 -- 172.21.15.7:0/3489600069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+5455 (secure 0 0 0) 0x7f6928017020 con 0x7f69340cb9a0 2023-06-09T11:58:13.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:13.697 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":219,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:09.773612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:13.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 msgr2=0x7f692006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 0x7f692006d190 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f69240090b0 tx=0x7f6924008030 comp rx=0 tx=0).stop 2023-06-09T11:58:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 msgr2=0x7f69340c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f692800c4b0 tx=0x7f692800c7c0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 shutdown_connections 2023-06-09T11:58:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f692006ace0 0x7f692006d190 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f693415cd40 0x7f69340c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69340cb5d0 0x7f693415be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 --2- 172.21.15.7:0/3489600069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69340cb9a0 0x7f69340c6f00 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:13.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 >> 172.21.15.7:0/3489600069 conn(0x7f6934070af0 msgr2=0x7f69340c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 shutdown_connections 2023-06-09T11:58:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:13.695+0000 7f693a3ff700 1 -- 172.21.15.7:0/3489600069 wait complete. 2023-06-09T11:58:13.701 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 219 2023-06-09T11:58:13.717 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:15.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.927+0000 7fbd057ff700 1 Processor -- start 2023-06-09T11:58:15.930 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.927+0000 7fbd057ff700 1 -- start start 2023-06-09T11:58:15.931 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.928+0000 7fbd057ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd000c5b40 0x7fbd00070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.928+0000 7fbd057ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd00071060 0x7fbd00158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.928+0000 7fbd057ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd001580a0 con 0x7fbd00151510 2023-06-09T11:58:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.928+0000 7fbd057ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd001518e0 con 0x7fbd00071060 2023-06-09T11:58:15.932 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.928+0000 7fbd057ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd000c5f70 con 0x7fbd000c5b40 2023-06-09T11:58:15.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd00071060 0x7fbd00158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbd00071060 0x7fbd00158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57790/0 (socket says 172.21.15.7:57790) 2023-06-09T11:58:15.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2337843866 learned_addr learned my addr 172.21.15.7:0/2337843866 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcfe7fc700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbd000c5b40 0x7fbd00070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcfeffd700 1 --1- 172.21.15.7:0/2337843866 >> v1:172.21.15.117:6789/0 conn(0x7fbd00151510 0x7fbd0015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:42594/0 (socket says 172.21.15.7:42594) 2023-06-09T11:58:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2337843866 >> v2:172.21.15.117:3300/0 conn(0x7fbd000c5b40 msgr2=0x7fbd00070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 --2- 172.21.15.7:0/2337843866 >> v2:172.21.15.117:3300/0 conn(0x7fbd000c5b40 0x7fbd00070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:15.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2337843866 >> v1:172.21.15.117:6789/0 conn(0x7fbd00151510 legacy=0x7fbd0015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcfdffb700 1 -- 172.21.15.7:0/2337843866 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512102799 0 0) 0x7fbd001580a0 con 0x7fbd00151510 2023-06-09T11:58:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2337843866 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd000c6180 con 0x7fbd00071060 2023-06-09T11:58:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.929+0000 7fbcff7fe700 1 --2- 172.21.15.7:0/2337843866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00071060 0x7fbd00158870 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbd000703a0 tx=0x7fbcf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad7a31c7ad01c4b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbcfdffb700 1 -- 172.21.15.7:0/2337843866 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800e070 con 0x7fbd00071060 2023-06-09T11:58:15.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbcfdffb700 1 -- 172.21.15.7:0/2337843866 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf800b910 con 0x7fbd00071060 2023-06-09T11:58:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbcfdffb700 1 -- 172.21.15.7:0/2337843866 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf800bcc0 con 0x7fbd00071060 2023-06-09T11:58:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 -- 172.21.15.7:0/2337843866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00071060 msgr2=0x7fbd00158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2337843866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00071060 0x7fbd00158870 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fbd000703a0 tx=0x7fbcf800b330 comp rx=0 tx=0).stop 2023-06-09T11:58:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 -- 172.21.15.7:0/2337843866 shutdown_connections 2023-06-09T11:58:15.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2337843866 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00071060 0x7fbd00158870 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:15.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2337843866 >> v2:172.21.15.117:3300/0 conn(0x7fbd000c5b40 0x7fbd00070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:15.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 -- 172.21.15.7:0/2337843866 >> 172.21.15.7:0/2337843866 conn(0x7fbd0006a4d0 msgr2=0x7fbd0006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:15.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 -- 172.21.15.7:0/2337843866 shutdown_connections 2023-06-09T11:58:15.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.930+0000 7fbd057ff700 1 -- 172.21.15.7:0/2337843866 wait complete. 2023-06-09T11:58:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.932+0000 7fbd057ff700 1 Processor -- start 2023-06-09T11:58:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.932+0000 7fbd057ff700 1 -- start start 2023-06-09T11:58:15.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd00071060 0x7fbd0014a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 0x7fbd0014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001efe30 con 0x7fbd00151510 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001eef40 con 0x7fbd000c5b40 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbd057ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd001ef220 con 0x7fbd00071060 2023-06-09T11:58:15.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.933+0000 7fbcff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcfe7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 0x7fbd0014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57796/0 (socket says 172.21.15.7:57796) 2023-06-09T11:58:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcfeffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd00071060 0x7fbd0014a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcfe7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 0x7fbd0014ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:35090/0 (socket says 172.21.15.7:35090) 2023-06-09T11:58:15.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2706763274 learned_addr learned my addr 172.21.15.7:0/2706763274 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd00071060 msgr2=0x7fbd0014a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd00071060 0x7fbd0014a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 msgr2=0x7fbd0014ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 0x7fbd0014ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:15.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 -- 172.21.15.7:0/2706763274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf8009b30 con 0x7fbd00151510 2023-06-09T11:58:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbcff7fe700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbd000cc2c0 tx=0x7fbcf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8004840 con 0x7fbd00151510 2023-06-09T11:58:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf80099b0 con 0x7fbd00151510 2023-06-09T11:58:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.934+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf801b670 con 0x7fbd00151510 2023-06-09T11:58:15.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.935+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd00157770 con 0x7fbd00151510 2023-06-09T11:58:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.935+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd00157490 con 0x7fbd00151510 2023-06-09T11:58:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.936+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd00157a50 con 0x7fbd00151510 2023-06-09T11:58:15.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.937+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbcf801b7d0 con 0x7fbd00151510 2023-06-09T11:58:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.938+0000 7fbce77fe700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 0x7fbcec06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.938+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fbcf808b500 con 0x7fbd00151510 2023-06-09T11:58:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.938+0000 7fbcfeffd700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 0x7fbcec06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:15.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.939+0000 7fbcfeffd700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 0x7fbcec06d1c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007fd0 tx=0x7fbcf0007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:15.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:15.942+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbcf8055cf0 con 0x7fbd00151510 2023-06-09T11:58:16.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.203+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd00074cf0 con 0x7fbd00151510 2023-06-09T11:58:16.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.204+0000 7fbce77fe700 1 -- 172.21.15.7:0/2706763274 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+5455 (secure 0 0 0) 0x7fbcf8059310 con 0x7fbd00151510 2023-06-09T11:58:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:16.207 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":219,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:09.773612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 msgr2=0x7fbcec06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:16.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 0x7fbcec06d1c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007fd0 tx=0x7fbcf0007de0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 msgr2=0x7fbd001597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbd000cc2c0 tx=0x7fbcf8004ad0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 shutdown_connections 2023-06-09T11:58:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd00151510 0x7fbd001597b0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbcec06ad10 0x7fbcec06d1c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbd00071060 0x7fbd0014a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 --2- 172.21.15.7:0/2706763274 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbd000c5b40 0x7fbd0014ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 >> 172.21.15.7:0/2706763274 conn(0x7fbd0006a4d0 msgr2=0x7fbd0014f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 shutdown_connections 2023-06-09T11:58:16.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:16.206+0000 7fbd057ff700 1 -- 172.21.15.7:0/2706763274 wait complete. 2023-06-09T11:58:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 219 2023-06-09T11:58:16.228 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:18.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:18.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.438+0000 7fb7585ff700 1 Processor -- start 2023-06-09T11:58:18.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.439+0000 7fb7585ff700 1 -- start start 2023-06-09T11:58:18.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.440+0000 7fb7585ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb750153880 0x7fb75015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:18.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb7585ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75015a500 con 0x7fb7500c9f40 2023-06-09T11:58:18.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb7585ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75015da00 con 0x7fb75015b4a0 2023-06-09T11:58:18.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb7585ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7500c5f10 con 0x7fb750153880 2023-06-09T11:58:18.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb750153880 0x7fb75015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:18.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb750153880 0x7fb75015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57798/0 (socket says 172.21.15.7:57798) 2023-06-09T11:58:18.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 -- 172.21.15.7:0/3007989107 learned_addr learned my addr 172.21.15.7:0/3007989107 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:18.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 -- 172.21.15.7:0/3007989107 >> v1:172.21.15.117:6789/0 conn(0x7fb75015b4a0 legacy=0x7fb7500c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 -- 172.21.15.7:0/3007989107 >> v1:172.21.15.7:6789/0 conn(0x7fb7500c9f40 legacy=0x7fb75015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.441+0000 7fb75639b700 1 -- 172.21.15.7:0/3007989107 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7501596e0 con 0x7fb750153880 2023-06-09T11:58:18.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.442+0000 7fb75639b700 1 --2- 172.21.15.7:0/3007989107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb750153880 0x7fb75015d4c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0098f0 tx=0x7fb74c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f86e12ba244527 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:18.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.442+0000 7fb755399700 1 -- 172.21.15.7:0/3007989107 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00b8a0 con 0x7fb750153880 2023-06-09T11:58:18.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.442+0000 7fb755399700 1 -- 172.21.15.7:0/3007989107 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c00ba00 con 0x7fb750153880 2023-06-09T11:58:18.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.442+0000 7fb755399700 1 -- 172.21.15.7:0/3007989107 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00bd90 con 0x7fb750153880 2023-06-09T11:58:18.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 -- 172.21.15.7:0/3007989107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb750153880 msgr2=0x7fb75015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 --2- 172.21.15.7:0/3007989107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb750153880 0x7fb75015d4c0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0098f0 tx=0x7fb74c009c00 comp rx=0 tx=0).stop 2023-06-09T11:58:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 -- 172.21.15.7:0/3007989107 shutdown_connections 2023-06-09T11:58:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 --2- 172.21.15.7:0/3007989107 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb750153880 0x7fb75015d4c0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 -- 172.21.15.7:0/3007989107 >> 172.21.15.7:0/3007989107 conn(0x7fb7500737f0 msgr2=0x7fb7500720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 -- 172.21.15.7:0/3007989107 shutdown_connections 2023-06-09T11:58:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.443+0000 7fb7585ff700 1 -- 172.21.15.7:0/3007989107 wait complete. 2023-06-09T11:58:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.445+0000 7fb7585ff700 1 Processor -- start 2023-06-09T11:58:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.445+0000 7fb7585ff700 1 -- start start 2023-06-09T11:58:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 0x7fb750157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb75015b4a0 0x7fb750157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750158010 con 0x7fb7500c9f40 2023-06-09T11:58:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7500ce050 con 0x7fb750153880 2023-06-09T11:58:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb7585ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb750151b40 con 0x7fb75015b4a0 2023-06-09T11:58:18.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.446+0000 7fb75639b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57800/0 (socket says 172.21.15.7:57800) 2023-06-09T11:58:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 -- 172.21.15.7:0/2529761701 learned_addr learned my addr 172.21.15.7:0/2529761701 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb755b9a700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 0x7fb750157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 -- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb75015b4a0 msgr2=0x7fb750157ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb75015b4a0 0x7fb750157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 -- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 msgr2=0x7fb750157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 0x7fb750157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 -- 172.21.15.7:0/2529761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb74c009550 con 0x7fb7500c9f40 2023-06-09T11:58:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb755b9a700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 0x7fb750157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb75639b700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009c40 tx=0x7fb74c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.447+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c004530 con 0x7fb7500c9f40 2023-06-09T11:58:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.448+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c01f070 con 0x7fb7500c9f40 2023-06-09T11:58:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.448+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb750159c60 con 0x7fb7500c9f40 2023-06-09T11:58:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.448+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0038e0 con 0x7fb7500c9f40 2023-06-09T11:58:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.448+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb750159980 con 0x7fb7500c9f40 2023-06-09T11:58:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.449+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb750159f40 con 0x7fb7500c9f40 2023-06-09T11:58:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.450+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb74c021030 con 0x7fb7500c9f40 2023-06-09T11:58:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.451+0000 7fb746ffd700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 0x7fb73406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.451+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fb74c08b250 con 0x7fb7500c9f40 2023-06-09T11:58:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.451+0000 7fb755b9a700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 0x7fb73406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.452+0000 7fb755b9a700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 0x7fb73406d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb75006b4b0 tx=0x7fb740009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.455+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb74c055b60 con 0x7fb7500c9f40 2023-06-09T11:58:18.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.710+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb750074cf0 con 0x7fb7500c9f40 2023-06-09T11:58:18.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.711+0000 7fb746ffd700 1 -- 172.21.15.7:0/2529761701 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 219 v219) v1 ==== 95+0+5455 (secure 0 0 0) 0x7fb74c015090 con 0x7fb7500c9f40 2023-06-09T11:58:18.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:18.715 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":219,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":219,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:09.773612+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:replay","state_seq":1,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 msgr2=0x7fb73406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 0x7fb73406d190 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb75006b4b0 tx=0x7fb740009040 comp rx=0 tx=0).stop 2023-06-09T11:58:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 msgr2=0x7fb75014b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fb74c009c40 tx=0x7fb74c004750 comp rx=0 tx=0).stop 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 shutdown_connections 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb73406ace0 0x7fb73406d190 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb75015b4a0 0x7fb750157ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7500c9f40 0x7fb75014b400 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 --2- 172.21.15.7:0/2529761701 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb750153880 0x7fb750157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:18.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 >> 172.21.15.7:0/2529761701 conn(0x7fb7500737f0 msgr2=0x7fb7500cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 shutdown_connections 2023-06-09T11:58:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:18.713+0000 7fb7585ff700 1 -- 172.21.15.7:0/2529761701 wait complete. 2023-06-09T11:58:18.719 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 219 2023-06-09T11:58:18.735 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:20.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.944+0000 7f51b77ff700 1 Processor -- start 2023-06-09T11:58:20.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.945+0000 7f51b77ff700 1 -- start start 2023-06-09T11:58:20.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.946+0000 7f51b77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.946+0000 7f51b77ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 0x7f51b0158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.946+0000 7f51b77ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b01580a0 con 0x7f51b015b380 2023-06-09T11:58:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.946+0000 7f51b77ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b01518e0 con 0x7f51b0071230 2023-06-09T11:58:20.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.946+0000 7f51b77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b00c6180 con 0x7f51b0151510 2023-06-09T11:58:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b5d9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 0x7f51b0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b5d9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 0x7f51b0158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:35112/0 (socket says 172.21.15.7:35112) 2023-06-09T11:58:20.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:53320/0 (socket says 172.21.15.7:53320) 2023-06-09T11:58:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b5d9c700 1 -- 172.21.15.7:0/3895721290 learned_addr learned my addr 172.21.15.7:0/3895721290 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 -- 172.21.15.7:0/3895721290 >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 msgr2=0x7f51b0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 --2- 172.21.15.7:0/3895721290 >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 0x7f51b0158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 -- 172.21.15.7:0/3895721290 >> v1:172.21.15.117:6789/0 conn(0x7f51b015b380 legacy=0x7f51b0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:20.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 -- 172.21.15.7:0/3895721290 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b0157490 con 0x7f51b0151510 2023-06-09T11:58:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.947+0000 7f51b559b700 1 --2- 172.21.15.7:0/3895721290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009840 tx=0x7f51ac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb8594560fabd8e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51a7fff700 1 -- 172.21.15.7:0/3895721290 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac011830 con 0x7f51b0151510 2023-06-09T11:58:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51a7fff700 1 -- 172.21.15.7:0/3895721290 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac011990 con 0x7f51b0151510 2023-06-09T11:58:20.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51a7fff700 1 -- 172.21.15.7:0/3895721290 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac011d20 con 0x7f51b0151510 2023-06-09T11:58:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 -- 172.21.15.7:0/3895721290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 msgr2=0x7f51b00c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 --2- 172.21.15.7:0/3895721290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009840 tx=0x7f51ac009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 -- 172.21.15.7:0/3895721290 shutdown_connections 2023-06-09T11:58:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 --2- 172.21.15.7:0/3895721290 >> v2:172.21.15.117:3300/0 conn(0x7f51b0071230 0x7f51b0158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:20.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 --2- 172.21.15.7:0/3895721290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b0151510 0x7f51b00c5b40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 -- 172.21.15.7:0/3895721290 >> 172.21.15.7:0/3895721290 conn(0x7f51b006a4d0 msgr2=0x7f51b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 -- 172.21.15.7:0/3895721290 shutdown_connections 2023-06-09T11:58:20.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.948+0000 7f51b77ff700 1 -- 172.21.15.7:0/3895721290 wait complete. 2023-06-09T11:58:20.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.951+0000 7f51b77ff700 1 Processor -- start 2023-06-09T11:58:20.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.951+0000 7f51b77ff700 1 -- start start 2023-06-09T11:58:20.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b0151510 0x7f51b014e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b015b380 0x7f51b00c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0158380 con 0x7f51b0071230 2023-06-09T11:58:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b015a300 con 0x7f51b0151510 2023-06-09T11:58:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b77ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b014f960 con 0x7f51b015b380 2023-06-09T11:58:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b559b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b559b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57804/0 (socket says 172.21.15.7:57804) 2023-06-09T11:58:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.952+0000 7f51b559b700 1 -- 172.21.15.7:0/479454127 learned_addr learned my addr 172.21.15.7:0/479454127 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b4d9a700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b0151510 0x7f51b014e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b5d9c700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b015b380 0x7f51b00c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 -- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b015b380 msgr2=0x7f51b00c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b015b380 0x7f51b00c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 -- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b0151510 msgr2=0x7f51b014e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b0151510 0x7f51b014e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 -- 172.21.15.7:0/479454127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0094a0 con 0x7f51b0071230 2023-06-09T11:58:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b559b700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0090b0 tx=0x7f51ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac0036a0 con 0x7f51b0071230 2023-06-09T11:58:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac01f070 con 0x7f51b0071230 2023-06-09T11:58:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0157a50 con 0x7f51b0071230 2023-06-09T11:58:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.953+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac003a60 con 0x7f51b0071230 2023-06-09T11:58:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.954+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b0157770 con 0x7f51b0071230 2023-06-09T11:58:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.955+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b0157d30 con 0x7f51b0071230 2023-06-09T11:58:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.955+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51ac003bc0 con 0x7f51b0071230 2023-06-09T11:58:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.956+0000 7f51a5ffb700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.956+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f51ac021070 con 0x7f51b0071230 2023-06-09T11:58:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.957+0000 7f51b4d9a700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.957+0000 7f51b4d9a700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 0x7f519406d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51b01efc40 tx=0x7f51a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:20.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:20.961+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f51ac055d20 con 0x7f51b0071230 2023-06-09T11:58:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.211+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51b00c21a0 con 0x7f51b0071230 2023-06-09T11:58:21.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.212+0000 7f51a5ffb700 1 -- 172.21.15.7:0/479454127 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f51ac059340 con 0x7f51b0071230 2023-06-09T11:58:21.215 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:21.215 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":220,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:20.795436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:resolve","state_seq":9,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 msgr2=0x7f519406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 0x7f519406d190 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f51b01efc40 tx=0x7f51a0009040 comp rx=0 tx=0).stop 2023-06-09T11:58:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 msgr2=0x7f51b014db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:21.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0090b0 tx=0x7f51ac004750 comp rx=0 tx=0).stop 2023-06-09T11:58:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 shutdown_connections 2023-06-09T11:58:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f519406ace0 0x7f519406d190 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f51b015b380 0x7f51b00c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:21.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51b0071230 0x7f51b014db20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 --2- 172.21.15.7:0/479454127 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f51b0151510 0x7f51b014e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 >> 172.21.15.7:0/479454127 conn(0x7f51b006a4d0 msgr2=0x7f51b00c6670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.214+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 shutdown_connections 2023-06-09T11:58:21.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:21.215+0000 7f51b77ff700 1 -- 172.21.15.7:0/479454127 wait complete. 2023-06-09T11:58:21.220 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 220 2023-06-09T11:58:21.235 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:23.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:23.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.444+0000 7f662cdff700 1 Processor -- start 2023-06-09T11:58:23.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.445+0000 7f662cdff700 1 -- start start 2023-06-09T11:58:23.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662cdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66280cb5d0 0x7f66280b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662cdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f662815cd40 0x7f66280cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662cdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6628159b20 con 0x7f66280cb9a0 2023-06-09T11:58:23.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662cdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66280cac80 con 0x7f66280cb5d0 2023-06-09T11:58:23.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662cdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66281597d0 con 0x7f662815cd40 2023-06-09T11:58:23.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66280cb5d0 0x7f66280b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f66280cb5d0 0x7f66280b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33100/0 (socket says 172.21.15.7:33100) 2023-06-09T11:58:23.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 -- 172.21.15.7:0/2993453884 learned_addr learned my addr 172.21.15.7:0/2993453884 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f6626d9d700 1 --2- 172.21.15.7:0/2993453884 >> v2:172.21.15.117:3300/0 conn(0x7f662815cd40 0x7f66280cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 -- 172.21.15.7:0/2993453884 >> v2:172.21.15.117:3300/0 conn(0x7f662815cd40 msgr2=0x7f66280cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:23.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 --2- 172.21.15.7:0/2993453884 >> v2:172.21.15.117:3300/0 conn(0x7f662815cd40 0x7f66280cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.446+0000 7f662659c700 1 -- 172.21.15.7:0/2993453884 >> v1:172.21.15.7:6789/0 conn(0x7f66280cb9a0 legacy=0x7f6628159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f662659c700 1 -- 172.21.15.7:0/2993453884 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628071ab0 con 0x7f66280cb5d0 2023-06-09T11:58:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f662659c700 1 --2- 172.21.15.7:0/2993453884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f66280b52b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f66280c9b80 tx=0x7f6618009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893c268b731edf23 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f6625d9b700 1 -- 172.21.15.7:0/2993453884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618011830 con 0x7f66280cb5d0 2023-06-09T11:58:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f6625d9b700 1 -- 172.21.15.7:0/2993453884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6618011990 con 0x7f66280cb5d0 2023-06-09T11:58:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f6625d9b700 1 -- 172.21.15.7:0/2993453884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618011d20 con 0x7f66280cb5d0 2023-06-09T11:58:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f662cdff700 1 -- 172.21.15.7:0/2993453884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 msgr2=0x7f66280b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.447+0000 7f662cdff700 1 --2- 172.21.15.7:0/2993453884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f66280b52b0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f66280c9b80 tx=0x7f6618009bd0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 -- 172.21.15.7:0/2993453884 shutdown_connections 2023-06-09T11:58:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 --2- 172.21.15.7:0/2993453884 >> v2:172.21.15.117:3300/0 conn(0x7f662815cd40 0x7f66280cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 --2- 172.21.15.7:0/2993453884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f66280b52b0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 -- 172.21.15.7:0/2993453884 >> 172.21.15.7:0/2993453884 conn(0x7f6628070af0 msgr2=0x7f6628070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 -- 172.21.15.7:0/2993453884 shutdown_connections 2023-06-09T11:58:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.448+0000 7f662cdff700 1 -- 172.21.15.7:0/2993453884 wait complete. 2023-06-09T11:58:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.449+0000 7f662cdff700 1 Processor -- start 2023-06-09T11:58:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 -- start start 2023-06-09T11:58:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280cb9a0 0x7f6628154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 0x7f66280cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c45d0 con 0x7f66280cb5d0 2023-06-09T11:58:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c47e0 con 0x7f66280cb9a0 2023-06-09T11:58:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.450+0000 7f662cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66280c4ac0 con 0x7f662815cd40 2023-06-09T11:58:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33114/0 (socket says 172.21.15.7:33114) 2023-06-09T11:58:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 -- 172.21.15.7:0/2639209415 learned_addr learned my addr 172.21.15.7:0/2639209415 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f661ffff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280cb9a0 0x7f6628154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f6626d9d700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 0x7f66280cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 -- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 msgr2=0x7f66280cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 0x7f66280cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 -- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280cb9a0 msgr2=0x7f6628154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280cb9a0 0x7f6628154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 -- 172.21.15.7:0/2639209415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009550 con 0x7f66280cb5d0 2023-06-09T11:58:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f6626d9d700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 0x7f66280cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.451+0000 7f662659c700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6618000c00 tx=0x7f6618004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.452+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6618004570 con 0x7f66280cb5d0 2023-06-09T11:58:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.452+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661801f070 con 0x7f66280cb5d0 2023-06-09T11:58:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.452+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662815d4f0 con 0x7f66280cb5d0 2023-06-09T11:58:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.452+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180038e0 con 0x7f66280cb5d0 2023-06-09T11:58:23.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.452+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662815d210 con 0x7f66280cb5d0 2023-06-09T11:58:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.453+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662815d7d0 con 0x7f66280cb5d0 2023-06-09T11:58:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.454+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6618021030 con 0x7f66280cb5d0 2023-06-09T11:58:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.454+0000 7f661effd700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 0x7f660806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.455+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f661808b210 con 0x7f66280cb5d0 2023-06-09T11:58:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.455+0000 7f661ffff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 0x7f660806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.455+0000 7f661ffff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 0x7f660806d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6628158400 tx=0x7f6610009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.459+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6618055970 con 0x7f66280cb5d0 2023-06-09T11:58:23.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.712+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6628074cf0 con 0x7f66280cb5d0 2023-06-09T11:58:23.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.713+0000 7f661effd700 1 -- 172.21.15.7:0/2639209415 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f6618015090 con 0x7f66280cb5d0 2023-06-09T11:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:23.716 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":220,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:20.795436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:resolve","state_seq":9,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:23.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 msgr2=0x7f660806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 0x7f660806d190 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f6628158400 tx=0x7f6610009040 comp rx=0 tx=0).stop 2023-06-09T11:58:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 msgr2=0x7f6628153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f6618000c00 tx=0x7f6618004750 comp rx=0 tx=0).stop 2023-06-09T11:58:23.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 shutdown_connections 2023-06-09T11:58:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f660806ace0 0x7f660806d190 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f662815cd40 0x7f66280cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66280cb5d0 0x7f6628153e00 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 --2- 172.21.15.7:0/2639209415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f66280cb9a0 0x7f6628154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:23.719 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 >> 172.21.15.7:0/2639209415 conn(0x7f6628070af0 msgr2=0x7f662815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 shutdown_connections 2023-06-09T11:58:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:23.715+0000 7f662cdff700 1 -- 172.21.15.7:0/2639209415 wait complete. 2023-06-09T11:58:23.720 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 220 2023-06-09T11:58:23.736 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 220, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 220, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 0, 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:resolve", "state_seq": 9 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:20.795436+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:58:25.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:25.109 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:25.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.948+0000 7f79e41ff700 1 Processor -- start 2023-06-09T11:58:25.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.948+0000 7f79e41ff700 1 -- start start 2023-06-09T11:58:25.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e41ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 0x7f79dc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e41ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79dc0cb9a0 0x7f79dc159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e41ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc159b20 con 0x7f79dc15cd40 2023-06-09T11:58:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e41ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc0cac80 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e41ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc1597d0 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e179a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79dc0cb9a0 0x7f79dc159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e179a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f79dc0cb9a0 0x7f79dc159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33126/0 (socket says 172.21.15.7:33126) 2023-06-09T11:58:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.949+0000 7f79e179a700 1 -- 172.21.15.7:0/1701695498 learned_addr learned my addr 172.21.15.7:0/1701695498 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e1f9b700 1 --2- 172.21.15.7:0/1701695498 >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 0x7f79dc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e179a700 1 -- 172.21.15.7:0/1701695498 >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 msgr2=0x7f79dc0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e179a700 1 --2- 172.21.15.7:0/1701695498 >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 0x7f79dc0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e179a700 1 -- 172.21.15.7:0/1701695498 >> v1:172.21.15.7:6789/0 conn(0x7f79dc15cd40 legacy=0x7f79dc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e179a700 1 -- 172.21.15.7:0/1701695498 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc071ab0 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e1f9b700 1 --2- 172.21.15.7:0/1701695498 >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 0x7f79dc0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:25.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e179a700 1 --2- 172.21.15.7:0/1701695498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc159fb0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f79dc08d420 tx=0x7f79cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3496249608200e99 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e0f99700 1 -- 172.21.15.7:0/1701695498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00b870 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e0f99700 1 -- 172.21.15.7:0/1701695498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc00b9d0 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.950+0000 7f79e0f99700 1 -- 172.21.15.7:0/1701695498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00bd60 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 -- 172.21.15.7:0/1701695498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb9a0 msgr2=0x7f79dc159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:25.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 --2- 172.21.15.7:0/1701695498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc159fb0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f79dc08d420 tx=0x7f79cc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 -- 172.21.15.7:0/1701695498 shutdown_connections 2023-06-09T11:58:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 --2- 172.21.15.7:0/1701695498 >> v2:172.21.15.117:3300/0 conn(0x7f79dc0cb5d0 0x7f79dc0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 --2- 172.21.15.7:0/1701695498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc159fb0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 -- 172.21.15.7:0/1701695498 >> 172.21.15.7:0/1701695498 conn(0x7f79dc070af0 msgr2=0x7f79dc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:25.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 -- 172.21.15.7:0/1701695498 shutdown_connections 2023-06-09T11:58:25.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.951+0000 7f79e41ff700 1 -- 172.21.15.7:0/1701695498 wait complete. 2023-06-09T11:58:25.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.953+0000 7f79e41ff700 1 Processor -- start 2023-06-09T11:58:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.953+0000 7f79e41ff700 1 -- start start 2023-06-09T11:58:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc1508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 0x7f79dc150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc154020 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc152120 con 0x7f79dc0cb9a0 2023-06-09T11:58:25.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e41ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0ce210 con 0x7f79dc15cd40 2023-06-09T11:58:25.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e1f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33138/0 (socket says 172.21.15.7:33138) 2023-06-09T11:58:25.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e1f9b700 1 -- 172.21.15.7:0/3816747506 learned_addr learned my addr 172.21.15.7:0/3816747506 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:25.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e179a700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc1508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.954+0000 7f79e279c700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 0x7f79dc150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 -- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 msgr2=0x7f79dc150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:25.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 0x7f79dc150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 -- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79dc0cb9a0 msgr2=0x7f79dc1508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:25.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc1508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:25.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 -- 172.21.15.7:0/3816747506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc009550 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e279c700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 0x7f79dc150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:25.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e1f9b700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e810 tx=0x7f79d800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8004230 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d80043b0 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80038e0 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc15d4f0 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.955+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc15d210 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.956+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc15d7d0 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.957+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79d8003a40 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.958+0000 7f79d27fc700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 0x7f79c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:25.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.958+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f79d8089910 con 0x7f79dc0cb5d0 2023-06-09T11:58:25.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.958+0000 7f79e179a700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 0x7f79c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:25.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.959+0000 7f79e179a700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 0x7f79c006d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0cbd70 tx=0x7f79cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:25.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:25.963+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f79d8057a70 con 0x7f79dc0cb5d0 2023-06-09T11:58:26.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.214+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79dc074cf0 con 0x7f79dc0cb5d0 2023-06-09T11:58:26.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.215+0000 7f79d27fc700 1 -- 172.21.15.7:0/3816747506 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 220 v220) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f79d8020070 con 0x7f79dc0cb5d0 2023-06-09T11:58:26.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:26.218 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":220,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":220,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:20.795436+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:resolve","state_seq":9,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 msgr2=0x7f79c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 0x7f79c006d190 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0cbd70 tx=0x7f79cc008030 comp rx=0 tx=0).stop 2023-06-09T11:58:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 msgr2=0x7f79dc0ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:26.220 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e810 tx=0x7f79d800eb20 comp rx=0 tx=0).stop 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 shutdown_connections 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f79c006ace0 0x7f79c006d190 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f79dc15cd40 0x7f79dc150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79dc0cb5d0 0x7f79dc0ce6a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 --2- 172.21.15.7:0/3816747506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f79dc0cb9a0 0x7f79dc1508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:26.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 >> 172.21.15.7:0/3816747506 conn(0x7f79dc070af0 msgr2=0x7f79dc158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.217+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 shutdown_connections 2023-06-09T11:58:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:26.218+0000 7f79e41ff700 1 -- 172.21.15.7:0/3816747506 wait complete. 2023-06-09T11:58:26.222 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 220 2023-06-09T11:58:26.238 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:28.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:28.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.447+0000 7ff29c5ff700 1 Processor -- start 2023-06-09T11:58:28.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.448+0000 7ff29c5ff700 1 -- start start 2023-06-09T11:58:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29c5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 0x7ff2940b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29c5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff2940cb9a0 0x7ff294159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29c5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff294159b20 con 0x7ff29415cd40 2023-06-09T11:58:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29c5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2940cac80 con 0x7ff2940cb9a0 2023-06-09T11:58:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29c5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941597d0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29ab9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff29415cd40 0x7ff2940cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50220/0 (socket says 172.21.15.7:50220) 2023-06-09T11:58:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29ab9c700 1 -- 172.21.15.7:0/236828674 learned_addr learned my addr 172.21.15.7:0/236828674 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:28.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff299b9a700 1 --2- 172.21.15.7:0/236828674 >> v2:172.21.15.117:3300/0 conn(0x7ff2940cb9a0 0x7ff294159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.449+0000 7ff29a39b700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 0x7ff2940b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299399700 1 -- 172.21.15.7:0/236828674 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819795635 0 0) 0x7ff294159b20 con 0x7ff29415cd40 2023-06-09T11:58:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299399700 1 -- 172.21.15.7:0/236828674 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2800035b0 con 0x7ff29415cd40 2023-06-09T11:58:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299b9a700 1 -- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 msgr2=0x7ff2940b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:28.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299b9a700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 0x7ff2940b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299b9a700 1 -- 172.21.15.7:0/236828674 >> v1:172.21.15.7:6789/0 conn(0x7ff29415cd40 legacy=0x7ff2940cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299b9a700 1 -- 172.21.15.7:0/236828674 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff294071ab0 con 0x7ff2940cb9a0 2023-06-09T11:58:28.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff29a39b700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 0x7ff2940b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:58:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299b9a700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff294159fb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff284009840 tx=0x7ff284009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ae0c51349d24f5c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:28.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299399700 1 -- 172.21.15.7:0/236828674 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011830 con 0x7ff2940cb9a0 2023-06-09T11:58:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299399700 1 -- 172.21.15.7:0/236828674 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff284011990 con 0x7ff2940cb9a0 2023-06-09T11:58:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.450+0000 7ff299399700 1 -- 172.21.15.7:0/236828674 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff284011d00 con 0x7ff2940cb9a0 2023-06-09T11:58:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 -- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 msgr2=0x7ff294159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff294159fb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff284009840 tx=0x7ff284009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 -- 172.21.15.7:0/236828674 shutdown_connections 2023-06-09T11:58:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff2940cb5d0 0x7ff2940b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/236828674 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff294159fb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 -- 172.21.15.7:0/236828674 >> 172.21.15.7:0/236828674 conn(0x7ff294070af0 msgr2=0x7ff294070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 -- 172.21.15.7:0/236828674 shutdown_connections 2023-06-09T11:58:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.451+0000 7ff29c5ff700 1 -- 172.21.15.7:0/236828674 wait complete. 2023-06-09T11:58:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.453+0000 7ff29c5ff700 1 Processor -- start 2023-06-09T11:58:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.453+0000 7ff29c5ff700 1 -- start start 2023-06-09T11:58:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff2940c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff29415cd40 0x7ff2940cf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940cf310 con 0x7ff2940cb5d0 2023-06-09T11:58:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941526e0 con 0x7ff2940cb9a0 2023-06-09T11:58:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29c5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941f34d0 con 0x7ff29415cd40 2023-06-09T11:58:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33140/0 (socket says 172.21.15.7:33140) 2023-06-09T11:58:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 -- 172.21.15.7:0/1647521905 learned_addr learned my addr 172.21.15.7:0/1647521905 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff299b9a700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff2940c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29ab9c700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff29415cd40 0x7ff2940cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 -- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff29415cd40 msgr2=0x7ff2940cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff29415cd40 0x7ff2940cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 -- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 msgr2=0x7ff2940c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.454+0000 7ff29a39b700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff2940c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff29a39b700 1 -- 172.21.15.7:0/1647521905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900094a0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff299b9a700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff2940c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff29a39b700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff29000ebe0 tx=0x7ff290009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29000c8b0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29000ca10 con 0x7ff2940cb5d0 2023-06-09T11:58:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2840094a0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29000cd80 con 0x7ff2940cb5d0 2023-06-09T11:58:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.455+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29415d4f0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.456+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29415d210 con 0x7ff2940cb5d0 2023-06-09T11:58:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.457+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff290013470 con 0x7ff2940cb5d0 2023-06-09T11:58:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.458+0000 7ff28affd700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 0x7ff27806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.458+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7ff290089ef0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.458+0000 7ff299b9a700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 0x7ff27806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.459+0000 7ff299b9a700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 0x7ff27806d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff2940cbd70 tx=0x7ff284008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.463+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff290054650 con 0x7ff2940cb5d0 2023-06-09T11:58:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.714+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff29415d7d0 con 0x7ff2940cb5d0 2023-06-09T11:58:28.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.715+0000 7ff28affd700 1 -- 172.21.15.7:0/1647521905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) v1 ==== 95+0+5459 (secure 0 0 0) 0x7ff290017090 con 0x7ff2940cb5d0 2023-06-09T11:58:28.718 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:28.718 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":221,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":221,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:26.788273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 msgr2=0x7ff27806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 0x7ff27806d190 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff2940cbd70 tx=0x7ff284008030 comp rx=0 tx=0).stop 2023-06-09T11:58:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 msgr2=0x7ff294152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:28.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7ff29000ebe0 tx=0x7ff290009ec0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 shutdown_connections 2023-06-09T11:58:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff27806ace0 0x7ff27806d190 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff29415cd40 0x7ff2940cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2940cb5d0 0x7ff294152c30 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 --2- 172.21.15.7:0/1647521905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff2940cb9a0 0x7ff2940c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.717+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 >> 172.21.15.7:0/1647521905 conn(0x7ff294070af0 msgr2=0x7ff294158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.718+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 shutdown_connections 2023-06-09T11:58:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:28.718+0000 7ff29c5ff700 1 -- 172.21.15.7:0/1647521905 wait complete. 2023-06-09T11:58:28.722 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 221 2023-06-09T11:58:28.738 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:30.740 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.949+0000 7f56345ff700 1 Processor -- start 2023-06-09T11:58:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.949+0000 7f56345ff700 1 -- start start 2023-06-09T11:58:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.950+0000 7f56345ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f562c0cb5d0 0x7f562c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f56345ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c159b20 con 0x7f562c0cb9a0 2023-06-09T11:58:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f56345ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c0cac80 con 0x7f562c15cd40 2023-06-09T11:58:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f56345ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562c1597d0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f562c0cb9a0 0x7f562c159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50232/0 (socket says 172.21.15.7:50232) 2023-06-09T11:58:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631b9a700 1 -- 172.21.15.7:0/3076858518 learned_addr learned my addr 172.21.15.7:0/3076858518 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f563239b700 1 --2- 172.21.15.7:0/3076858518 >> v2:172.21.15.117:3300/0 conn(0x7f562c0cb5d0 0x7f562c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255480870 0 0) 0x7f562c159b20 con 0x7f562c0cb9a0 2023-06-09T11:58:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56180035b0 con 0x7f562c0cb9a0 2023-06-09T11:58:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045576566 0 0) 0x7f562c0cac80 con 0x7f562c15cd40 2023-06-09T11:58:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.951+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f562c159b20 con 0x7f562c15cd40 2023-06-09T11:58:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f563239b700 1 -- 172.21.15.7:0/3076858518 >> v1:172.21.15.117:6789/0 conn(0x7f562c15cd40 legacy=0x7f562c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f563239b700 1 -- 172.21.15.7:0/3076858518 >> v1:172.21.15.7:6789/0 conn(0x7f562c0cb9a0 legacy=0x7f562c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f563239b700 1 -- 172.21.15.7:0/3076858518 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c071ab0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f563239b700 1 --2- 172.21.15.7:0/3076858518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb5d0 0x7f562c0b52b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5628009840 tx=0x7f5628009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c448eb8b0d9f9102 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628011830 con 0x7f562c0cb5d0 2023-06-09T11:58:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5628011990 con 0x7f562c0cb5d0 2023-06-09T11:58:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.952+0000 7f5631399700 1 -- 172.21.15.7:0/3076858518 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628011d20 con 0x7f562c0cb5d0 2023-06-09T11:58:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 -- 172.21.15.7:0/3076858518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb5d0 msgr2=0x7f562c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 --2- 172.21.15.7:0/3076858518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb5d0 0x7f562c0b52b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5628009840 tx=0x7f5628009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 -- 172.21.15.7:0/3076858518 shutdown_connections 2023-06-09T11:58:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 --2- 172.21.15.7:0/3076858518 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb5d0 0x7f562c0b52b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 -- 172.21.15.7:0/3076858518 >> 172.21.15.7:0/3076858518 conn(0x7f562c070af0 msgr2=0x7f562c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 -- 172.21.15.7:0/3076858518 shutdown_connections 2023-06-09T11:58:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.953+0000 7f56345ff700 1 -- 172.21.15.7:0/3076858518 wait complete. 2023-06-09T11:58:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.954+0000 7f56345ff700 1 Processor -- start 2023-06-09T11:58:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.955+0000 7f56345ff700 1 -- start start 2023-06-09T11:58:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.955+0000 7f56345ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.955+0000 7f56345ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 0x7f562c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.955+0000 7f56345ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c15cd40 0x7f562c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.955+0000 7f56345ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c0b2240 con 0x7f562c0cb5d0 2023-06-09T11:58:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f56345ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1ef900 con 0x7f562c0cb9a0 2023-06-09T11:58:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f56345ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c1efbe0 con 0x7f562c15cd40 2023-06-09T11:58:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33152/0 (socket says 172.21.15.7:33152) 2023-06-09T11:58:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 -- 172.21.15.7:0/1184235404 learned_addr learned my addr 172.21.15.7:0/1184235404 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f5631b9a700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 0x7f562c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 -- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c15cd40 msgr2=0x7f562c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c15cd40 0x7f562c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 -- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 msgr2=0x7f562c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 0x7f562c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 -- 172.21.15.7:0/1184235404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56280094a0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f5631b9a700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 0x7f562c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.956+0000 7f563239b700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f56280090b0 tx=0x7f5628004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.957+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562801a860 con 0x7f562c0cb5d0 2023-06-09T11:58:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.957+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562801a9c0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.957+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562801ad30 con 0x7f562c0cb5d0 2023-06-09T11:58:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.957+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c15d4f0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.957+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c15d210 con 0x7f562c0cb5d0 2023-06-09T11:58:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.958+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c15d7d0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.959+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5628022070 con 0x7f562c0cb5d0 2023-06-09T11:58:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.960+0000 7f5622ffd700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.960+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f562808aad0 con 0x7f562c0cb5d0 2023-06-09T11:58:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.960+0000 7f5631b9a700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.961+0000 7f5631b9a700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f562c0c97c0 tx=0x7f561c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:30.964+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f56280552e0 con 0x7f562c0cb5d0 2023-06-09T11:58:31.218 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.216+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f562c074cf0 con 0x7f562c0cb5d0 2023-06-09T11:58:31.219 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.217+0000 7f5622ffd700 1 -- 172.21.15.7:0/1184235404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) v1 ==== 95+0+5459 (secure 0 0 0) 0x7f5628029070 con 0x7f562c0cb5d0 2023-06-09T11:58:31.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:31.219 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":221,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":221,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:26.788273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 msgr2=0x7f561006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:31.221 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 0x7f561006d190 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f562c0c97c0 tx=0x7f561c009040 comp rx=0 tx=0).stop 2023-06-09T11:58:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 msgr2=0x7f562c0ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f56280090b0 tx=0x7f5628004750 comp rx=0 tx=0).stop 2023-06-09T11:58:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 shutdown_connections 2023-06-09T11:58:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f561006ace0 0x7f561006d190 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:31.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f562c15cd40 0x7f562c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f562c0cb5d0 0x7f562c0ce320 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 --2- 172.21.15.7:0/1184235404 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f562c0cb9a0 0x7f562c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 >> 172.21.15.7:0/1184235404 conn(0x7f562c070af0 msgr2=0x7f562c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 shutdown_connections 2023-06-09T11:58:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:31.219+0000 7f56345ff700 1 -- 172.21.15.7:0/1184235404 wait complete. 2023-06-09T11:58:31.224 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 221 2023-06-09T11:58:31.240 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:33.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:33.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.448+0000 7fdfb5bff700 1 Processor -- start 2023-06-09T11:58:33.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.448+0000 7fdfb5bff700 1 -- start start 2023-06-09T11:58:33.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.449+0000 7fdfb5bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdfb015b4a0 0x7fdfb015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:33.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfb5bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb015a500 con 0x7fdfb00c9f40 2023-06-09T11:58:33.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfb5bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb015da00 con 0x7fdfb0153880 2023-06-09T11:58:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfb5bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb00c5f10 con 0x7fdfb015b4a0 2023-06-09T11:58:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdfb015b4a0 0x7fdfb015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdfb015b4a0 0x7fdfb015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48160/0 (socket says 172.21.15.7:48160) 2023-06-09T11:58:33.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 -- 172.21.15.7:0/2551037714 learned_addr learned my addr 172.21.15.7:0/2551037714 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:33.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 -- 172.21.15.7:0/2551037714 >> v1:172.21.15.117:6789/0 conn(0x7fdfb0153880 legacy=0x7fdfb015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 -- 172.21.15.7:0/2551037714 >> v1:172.21.15.7:6789/0 conn(0x7fdfb00c9f40 legacy=0x7fdfb00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.450+0000 7fdfaeffd700 1 -- 172.21.15.7:0/2551037714 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb01596e0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfaeffd700 1 --2- 172.21.15.7:0/2551037714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb015d4c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40098f0 tx=0x7fdfa4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7fcc0022569f20 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:33.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfae7fc700 1 -- 172.21.15.7:0/2551037714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400b8a0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfae7fc700 1 -- 172.21.15.7:0/2551037714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa400ba00 con 0x7fdfb015b4a0 2023-06-09T11:58:33.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfae7fc700 1 -- 172.21.15.7:0/2551037714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400bd90 con 0x7fdfb015b4a0 2023-06-09T11:58:33.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2551037714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 msgr2=0x7fdfb015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2551037714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb015d4c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40098f0 tx=0x7fdfa4009c00 comp rx=0 tx=0).stop 2023-06-09T11:58:33.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2551037714 shutdown_connections 2023-06-09T11:58:33.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2551037714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb015d4c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.451+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2551037714 >> 172.21.15.7:0/2551037714 conn(0x7fdfb00737f0 msgr2=0x7fdfb00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:33.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.452+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2551037714 shutdown_connections 2023-06-09T11:58:33.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.452+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2551037714 wait complete. 2023-06-09T11:58:33.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.453+0000 7fdfb5bff700 1 Processor -- start 2023-06-09T11:58:33.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.453+0000 7fdfb5bff700 1 -- start start 2023-06-09T11:58:33.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb00c9f40 0x7fdfb01e6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:33.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfb0153880 0x7fdfb00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:33.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00cd3a0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00cc060 con 0x7fdfb00c9f40 2023-06-09T11:58:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.454+0000 7fdfb5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb00cf100 con 0x7fdfb0153880 2023-06-09T11:58:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48168/0 (socket says 172.21.15.7:48168) 2023-06-09T11:58:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 -- 172.21.15.7:0/2913063652 learned_addr learned my addr 172.21.15.7:0/2913063652 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaf7fe700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb00c9f40 0x7fdfb01e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaeffd700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfb0153880 0x7fdfb00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 -- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfb0153880 msgr2=0x7fdfb00b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfb0153880 0x7fdfb00b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 -- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb00c9f40 msgr2=0x7fdfb01e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb00c9f40 0x7fdfb01e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 -- 172.21.15.7:0/2913063652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4009550 con 0x7fdfb015b4a0 2023-06-09T11:58:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.455+0000 7fdfaffff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800cc50 tx=0x7fdfa800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.456+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8003c00 con 0x7fdfb015b4a0 2023-06-09T11:58:33.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.456+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa8003d60 con 0x7fdfb015b4a0 2023-06-09T11:58:33.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.456+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800aab0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.456+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb0159c60 con 0x7fdfb015b4a0 2023-06-09T11:58:33.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.456+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb0159980 con 0x7fdfb015b4a0 2023-06-09T11:58:33.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.457+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb0159f40 con 0x7fdfb015b4a0 2023-06-09T11:58:33.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.458+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfa800ac10 con 0x7fdfb015b4a0 2023-06-09T11:58:33.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.459+0000 7fdf97fff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 0x7fdf9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.459+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fdfa8089ce0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.459+0000 7fdfaf7fe700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 0x7fdf9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.459+0000 7fdfaf7fe700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 0x7fdf9c06d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0007fd0 tx=0x7fdfa0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.463+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdfa80543b0 con 0x7fdfb015b4a0 2023-06-09T11:58:33.720 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.717+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfb015a500 con 0x7fdfb015b4a0 2023-06-09T11:58:33.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.718+0000 7fdf97fff700 1 -- 172.21.15.7:0/2913063652 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 221 v221) v1 ==== 95+0+5459 (secure 0 0 0) 0x7fdfb015a500 con 0x7fdfb015b4a0 2023-06-09T11:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:33.721 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":221,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":221,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:26.788273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.720+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 msgr2=0x7fdf9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.720+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 0x7fdf9c06d190 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0007fd0 tx=0x7fdfa0009040 comp rx=0 tx=0).stop 2023-06-09T11:58:33.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.720+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 msgr2=0x7fdfb0158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.720+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800cc50 tx=0x7fdfa800cf60 comp rx=0 tx=0).stop 2023-06-09T11:58:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 shutdown_connections 2023-06-09T11:58:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfb015b4a0 0x7fdfb0158d80 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdfb00c9f40 0x7fdfb01e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdf9c06ace0 0x7fdf9c06d190 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 --2- 172.21.15.7:0/2913063652 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdfb0153880 0x7fdfb00b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 >> 172.21.15.7:0/2913063652 conn(0x7fdfb00737f0 msgr2=0x7fdfb014f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 shutdown_connections 2023-06-09T11:58:33.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:33.721+0000 7fdfb5bff700 1 -- 172.21.15.7:0/2913063652 wait complete. 2023-06-09T11:58:33.726 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 221 2023-06-09T11:58:33.741 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:35.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:35.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.952+0000 7fb629fff700 1 Processor -- start 2023-06-09T11:58:35.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.952+0000 7fb629fff700 1 -- start start 2023-06-09T11:58:35.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.953+0000 7fb629fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb6240c0140 0x7fb624161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:35.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.953+0000 7fb629fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6240c0bf0 con 0x7fb62415a0c0 2023-06-09T11:58:35.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.953+0000 7fb629fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6240702f0 con 0x7fb6240c1bc0 2023-06-09T11:58:35.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.953+0000 7fb629fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb624070110 con 0x7fb6240c0140 2023-06-09T11:58:35.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.953+0000 7fb61bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb62415a0c0 0x7fb6241501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47788/0 (socket says 172.21.15.7:47788) 2023-06-09T11:58:35.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb61bfff700 1 -- 172.21.15.7:0/3180761716 learned_addr learned my addr 172.21.15.7:0/3180761716 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:35.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb623fff700 1 --2- 172.21.15.7:0/3180761716 >> v2:172.21.15.7:3300/0 conn(0x7fb6240c0140 0x7fb624161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:35.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb623fff700 1 -- 172.21.15.7:0/3180761716 >> v1:172.21.15.117:6789/0 conn(0x7fb6240c1bc0 legacy=0x7fb6241599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:35.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb623fff700 1 -- 172.21.15.7:0/3180761716 >> v1:172.21.15.7:6789/0 conn(0x7fb62415a0c0 legacy=0x7fb6241501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:35.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb623fff700 1 -- 172.21.15.7:0/3180761716 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6240c0e90 con 0x7fb6240c0140 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb623fff700 1 --2- 172.21.15.7:0/3180761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6240c0140 0x7fb624161cf0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4f90f5a6319e0e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.954+0000 7fb622ffd700 1 -- 172.21.15.7:0/3180761716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb6240c0140 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb622ffd700 1 -- 172.21.15.7:0/3180761716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00bbd0 con 0x7fb6240c0140 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb622ffd700 1 -- 172.21.15.7:0/3180761716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012430 con 0x7fb6240c0140 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 -- 172.21.15.7:0/3180761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6240c0140 msgr2=0x7fb624161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:35.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 --2- 172.21.15.7:0/3180761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6240c0140 0x7fb624161cf0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).stop 2023-06-09T11:58:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 -- 172.21.15.7:0/3180761716 shutdown_connections 2023-06-09T11:58:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 --2- 172.21.15.7:0/3180761716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6240c0140 0x7fb624161cf0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 -- 172.21.15.7:0/3180761716 >> 172.21.15.7:0/3180761716 conn(0x7fb62406a4d0 msgr2=0x7fb62406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 -- 172.21.15.7:0/3180761716 shutdown_connections 2023-06-09T11:58:35.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.955+0000 7fb629fff700 1 -- 172.21.15.7:0/3180761716 wait complete. 2023-06-09T11:58:35.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.957+0000 7fb629fff700 1 Processor -- start 2023-06-09T11:58:35.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.957+0000 7fb629fff700 1 -- start start 2023-06-09T11:58:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 0x7fb6240cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6240c1bc0 0x7fb624157810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624158b00 con 0x7fb62415a0c0 2023-06-09T11:58:35.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241442a0 con 0x7fb6240c0140 2023-06-09T11:58:35.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb629fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624159860 con 0x7fb6240c1bc0 2023-06-09T11:58:35.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb623fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:35.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb623fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48182/0 (socket says 172.21.15.7:48182) 2023-06-09T11:58:35.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb623fff700 1 -- 172.21.15.7:0/1525088860 learned_addr learned my addr 172.21.15.7:0/1525088860 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:35.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb61bfff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6240c1bc0 0x7fb624157810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.958+0000 7fb6237fe700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 0x7fb6240cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 -- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6240c1bc0 msgr2=0x7fb624157810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6240c1bc0 0x7fb624157810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 -- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 msgr2=0x7fb6240cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:35.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 0x7fb6240cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 -- 172.21.15.7:0/1525088860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009a70 con 0x7fb62415a0c0 2023-06-09T11:58:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb6237fe700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 0x7fb6240cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb623fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb61c014040 tx=0x7fb61c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012430 con 0x7fb62415a0c0 2023-06-09T11:58:35.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c020070 con 0x7fb62415a0c0 2023-06-09T11:58:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6240c1450 con 0x7fb62415a0c0 2023-06-09T11:58:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.959+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c004280 con 0x7fb62415a0c0 2023-06-09T11:58:35.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.960+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240c1170 con 0x7fb62415a0c0 2023-06-09T11:58:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.961+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6240705d0 con 0x7fb62415a0c0 2023-06-09T11:58:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.961+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb61c0043e0 con 0x7fb62415a0c0 2023-06-09T11:58:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.962+0000 7fb620ff9700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 0x7fb60406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:35.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.962+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fb61c022070 con 0x7fb62415a0c0 2023-06-09T11:58:35.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.963+0000 7fb6237fe700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 0x7fb60406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:35.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.963+0000 7fb6237fe700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 0x7fb60406d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f60 tx=0x7fb614009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:35.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:35.967+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb61c0563c0 con 0x7fb62415a0c0 2023-06-09T11:58:36.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.220+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb624074cf0 con 0x7fb62415a0c0 2023-06-09T11:58:36.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.221+0000 7fb620ff9700 1 -- 172.21.15.7:0/1525088860 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 222 v222) v1 ==== 95+0+5457 (secure 0 0 0) 0x7fb61c02c070 con 0x7fb62415a0c0 2023-06-09T11:58:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:36.223 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":222,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:35.783132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 msgr2=0x7fb60406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 0x7fb60406d190 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb614000f60 tx=0x7fb614009040 comp rx=0 tx=0).stop 2023-06-09T11:58:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 msgr2=0x7fb624143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb61c014040 tx=0x7fb61c004ad0 comp rx=0 tx=0).stop 2023-06-09T11:58:36.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 shutdown_connections 2023-06-09T11:58:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb62415a0c0 0x7fb624143550 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb6240c0140 0x7fb6240cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb60406ace0 0x7fb60406d190 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 --2- 172.21.15.7:0/1525088860 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb6240c1bc0 0x7fb624157810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:36.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 >> 172.21.15.7:0/1525088860 conn(0x7fb62406a4d0 msgr2=0x7fb6240c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:36.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.223+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 shutdown_connections 2023-06-09T11:58:36.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:36.224+0000 7fb629fff700 1 -- 172.21.15.7:0/1525088860 wait complete. 2023-06-09T11:58:36.228 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 222 2023-06-09T11:58:36.244 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 222, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 222, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [ 2 ], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:35.783132+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:58:38.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:38.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.453+0000 7f5dec9ff700 1 Processor -- start 2023-06-09T11:58:38.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.453+0000 7f5dec9ff700 1 -- start start 2023-06-09T11:58:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 0x7f5de80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5de80cad00 0x7f5de80cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de815cec0 con 0x7f5de80cad00 2023-06-09T11:58:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de815d160 con 0x7f5de80c9090 2023-06-09T11:58:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5dec9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de815d440 con 0x7f5de80cfaf0 2023-06-09T11:58:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5de80cad00 0x7f5de80cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5de80cad00 0x7f5de80cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48198/0 (socket says 172.21.15.7:48198) 2023-06-09T11:58:38.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 -- 172.21.15.7:0/1739136412 learned_addr learned my addr 172.21.15.7:0/1739136412 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5de6d9d700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5de659c700 1 --2- 172.21.15.7:0/1739136412 >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 0x7f5de80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 -- 172.21.15.7:0/1739136412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 msgr2=0x7f5de81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 -- 172.21.15.7:0/1739136412 >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 msgr2=0x7f5de80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1739136412 >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 0x7f5de80c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5ddffff700 1 -- 172.21.15.7:0/1739136412 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de815d720 con 0x7f5de80cad00 2023-06-09T11:58:38.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.455+0000 7f5de659c700 1 --2- 172.21.15.7:0/1739136412 >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 0x7f5de80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80cf5b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5de8066ee0 tx=0x7f5dd0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4c3b50ae6617c5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5de5d9b700 1 -- 172.21.15.7:0/1739136412 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000b870 con 0x7f5de80cad00 2023-06-09T11:58:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5de5d9b700 1 -- 172.21.15.7:0/1739136412 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000b9d0 con 0x7f5de80cad00 2023-06-09T11:58:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5de5d9b700 1 -- 172.21.15.7:0/1739136412 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000bd60 con 0x7f5de80cad00 2023-06-09T11:58:38.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1739136412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 msgr2=0x7f5de80cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.456+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80cf5b0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f5de8066ee0 tx=0x7f5dd0009bd0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1739136412 shutdown_connections 2023-06-09T11:58:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1739136412 >> v2:172.21.15.117:3300/0 conn(0x7f5de80c9090 0x7f5de80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1739136412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80cf5b0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1739136412 >> 172.21.15.7:0/1739136412 conn(0x7f5de8070af0 msgr2=0x7f5de8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1739136412 shutdown_connections 2023-06-09T11:58:38.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.457+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1739136412 wait complete. 2023-06-09T11:58:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.458+0000 7f5dec9ff700 1 Processor -- start 2023-06-09T11:58:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.458+0000 7f5dec9ff700 1 -- start start 2023-06-09T11:58:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5de80c9090 0x7f5de8153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81e5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80c3190 con 0x7f5de80cad00 2023-06-09T11:58:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80c4160 con 0x7f5de80c9090 2023-06-09T11:58:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.459+0000 7f5dec9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de81e5360 con 0x7f5de80cfaf0 2023-06-09T11:58:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48204/0 (socket says 172.21.15.7:48204) 2023-06-09T11:58:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 -- 172.21.15.7:0/1328681448 learned_addr learned my addr 172.21.15.7:0/1328681448 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5de659c700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5de80c9090 0x7f5de8153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5de6d9d700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 -- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 msgr2=0x7f5de81e5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:38.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81e5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 -- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5de80c9090 msgr2=0x7f5de8153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5de80c9090 0x7f5de8153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 -- 172.21.15.7:0/1328681448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd0009550 con 0x7f5de80cad00 2023-06-09T11:58:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.460+0000 7f5ddffff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0000c00 tx=0x7f5dd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:38.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.461+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0004310 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.461+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd001e070 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.461+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00036a0 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.461+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de80cb160 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.461+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de80ca620 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.462+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de80b53f0 con 0x7f5de80cad00 2023-06-09T11:58:38.470 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.463+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5dd0020030 con 0x7f5de80cad00 2023-06-09T11:58:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.464+0000 7f5ddeffd700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 0x7f5dc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.464+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f5dd008acb0 con 0x7f5de80cad00 2023-06-09T11:58:38.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.464+0000 7f5de659c700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 0x7f5dc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:38.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.465+0000 7f5de659c700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 0x7f5dc806d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800ab70 tx=0x7f5dd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:38.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.469+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5dd0055410 con 0x7f5de80cad00 2023-06-09T11:58:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.721+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5de80cdba0 con 0x7f5de80cad00 2023-06-09T11:58:38.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.722+0000 7f5ddeffd700 1 -- 172.21.15.7:0/1328681448 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 222 v222) v1 ==== 95+0+5457 (secure 0 0 0) 0x7f5dd0015090 con 0x7f5de80cad00 2023-06-09T11:58:38.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:38.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":222,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":222,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:35.783132+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:38.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 msgr2=0x7f5dc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 0x7f5dc806d190 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800ab70 tx=0x7f5dd8008040 comp rx=0 tx=0).stop 2023-06-09T11:58:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 msgr2=0x7f5de80c4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0000c00 tx=0x7f5dd0004750 comp rx=0 tx=0).stop 2023-06-09T11:58:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 shutdown_connections 2023-06-09T11:58:38.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5dc806ace0 0x7f5dc806d190 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5de80cfaf0 0x7f5de81e5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5de80c9090 0x7f5de8153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 --2- 172.21.15.7:0/1328681448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5de80cad00 0x7f5de80c4310 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 >> 172.21.15.7:0/1328681448 conn(0x7f5de8070af0 msgr2=0x7f5de80c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:38.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 shutdown_connections 2023-06-09T11:58:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:38.724+0000 7f5dec9ff700 1 -- 172.21.15.7:0/1328681448 wait complete. 2023-06-09T11:58:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 222 2023-06-09T11:58:38.751 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:40.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.961+0000 7fbe281ff700 1 Processor -- start 2023-06-09T11:58:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.961+0000 7fbe281ff700 1 -- start start 2023-06-09T11:58:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe200c9f40 0x7fbe2015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 0x7fbe200c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe200c5f10 con 0x7fbe20153880 2023-06-09T11:58:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe2015a500 con 0x7fbe200c9f40 2023-06-09T11:58:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe281ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2015da00 con 0x7fbe2015b2d0 2023-06-09T11:58:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe25f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.962+0000 7fbe2679c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 0x7fbe200c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe25f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48210/0 (socket says 172.21.15.7:48210) 2023-06-09T11:58:40.967 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2579a700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbe200c9f40 0x7fbe2015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe25f9b700 1 -- 172.21.15.7:0/1877199557 learned_addr learned my addr 172.21.15.7:0/1877199557 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 -- 172.21.15.7:0/1877199557 >> v2:172.21.15.117:3300/0 conn(0x7fbe200c9f40 msgr2=0x7fbe2015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 --2- 172.21.15.7:0/1877199557 >> v2:172.21.15.117:3300/0 conn(0x7fbe200c9f40 0x7fbe2015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 -- 172.21.15.7:0/1877199557 >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 msgr2=0x7fbe2015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:40.968 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 --2- 172.21.15.7:0/1877199557 >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 -- 172.21.15.7:0/1877199557 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe201596e0 con 0x7fbe2015b2d0 2023-06-09T11:58:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe25f9b700 1 --2- 172.21.15.7:0/1877199557 >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:58:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe2679c700 1 --2- 172.21.15.7:0/1877199557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 0x7fbe200c4e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe200cce30 tx=0x7fbe1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4503aa315327928d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.963+0000 7fbe24f99700 1 -- 172.21.15.7:0/1877199557 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1800e070 con 0x7fbe2015b2d0 2023-06-09T11:58:40.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe24f99700 1 -- 172.21.15.7:0/1877199557 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1800bb20 con 0x7fbe2015b2d0 2023-06-09T11:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe24f99700 1 -- 172.21.15.7:0/1877199557 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18012430 con 0x7fbe2015b2d0 2023-06-09T11:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 -- 172.21.15.7:0/1877199557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 msgr2=0x7fbe200c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 --2- 172.21.15.7:0/1877199557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 0x7fbe200c4e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe200cce30 tx=0x7fbe1800b330 comp rx=0 tx=0).stop 2023-06-09T11:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 -- 172.21.15.7:0/1877199557 shutdown_connections 2023-06-09T11:58:40.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 --2- 172.21.15.7:0/1877199557 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe2015b2d0 0x7fbe200c4e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 --2- 172.21.15.7:0/1877199557 >> v2:172.21.15.7:3300/0 conn(0x7fbe20153880 0x7fbe2015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 --2- 172.21.15.7:0/1877199557 >> v2:172.21.15.117:3300/0 conn(0x7fbe200c9f40 0x7fbe2015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 -- 172.21.15.7:0/1877199557 >> 172.21.15.7:0/1877199557 conn(0x7fbe200737f0 msgr2=0x7fbe200720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 -- 172.21.15.7:0/1877199557 shutdown_connections 2023-06-09T11:58:40.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.964+0000 7fbe281ff700 1 -- 172.21.15.7:0/1877199557 wait complete. 2023-06-09T11:58:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.967+0000 7fbe281ff700 1 Processor -- start 2023-06-09T11:58:40.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.967+0000 7fbe281ff700 1 -- start start 2023-06-09T11:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe200c9f40 0x7fbe20158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe20153880 0x7fbe20158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe20157c70 con 0x7fbe2015b2d0 2023-06-09T11:58:40.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200b4780 con 0x7fbe20153880 2023-06-09T11:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe281ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe200b3540 con 0x7fbe200c9f40 2023-06-09T11:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe2679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe2679c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48222/0 (socket says 172.21.15.7:48222) 2023-06-09T11:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.968+0000 7fbe2679c700 1 -- 172.21.15.7:0/624726070 learned_addr learned my addr 172.21.15.7:0/624726070 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:40.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe25f9b700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe200c9f40 0x7fbe20158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2579a700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe20153880 0x7fbe20158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 -- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe200c9f40 msgr2=0x7fbe20158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe200c9f40 0x7fbe20158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 -- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe20153880 msgr2=0x7fbe20158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:40.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe20153880 0x7fbe20158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 -- 172.21.15.7:0/624726070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c009550 con 0x7fbe2015b2d0 2023-06-09T11:58:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe2679c700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fbe2015a670 tx=0x7fbe18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe18004800 con 0x7fbe2015b2d0 2023-06-09T11:58:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe18020070 con 0x7fbe2015b2d0 2023-06-09T11:58:40.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18009a70 con 0x7fbe2015b2d0 2023-06-09T11:58:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.969+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe180276c0 con 0x7fbe2015b2d0 2023-06-09T11:58:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.970+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe20159980 con 0x7fbe2015b2d0 2023-06-09T11:58:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.971+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe2015b730 con 0x7fbe2015b2d0 2023-06-09T11:58:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.972+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe18012430 con 0x7fbe2015b2d0 2023-06-09T11:58:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.972+0000 7fbe167fc700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.973+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fbe18022070 con 0x7fbe2015b2d0 2023-06-09T11:58:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.973+0000 7fbe25f9b700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:40.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.973+0000 7fbe25f9b700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006d30 tx=0x7fbe1c0092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:40.977+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbe180563e0 con 0x7fbe2015b2d0 2023-06-09T11:58:41.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.234+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe20159c60 con 0x7fbe2015b2d0 2023-06-09T11:58:41.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.235+0000 7fbe167fc700 1 -- 172.21.15.7:0/624726070 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7fbe1802c070 con 0x7fbe2015b2d0 2023-06-09T11:58:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:41.237 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:41.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 msgr2=0x7fbe0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:41.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c006d30 tx=0x7fbe1c0092c0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 msgr2=0x7fbe2014ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:41.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fbe2015a670 tx=0x7fbe18004ad0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 shutdown_connections 2023-06-09T11:58:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe2015b2d0 0x7fbe2014ce10 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbe0406ace0 0x7fbe0406d190 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbe200c9f40 0x7fbe20158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 --2- 172.21.15.7:0/624726070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbe20153880 0x7fbe20158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 >> 172.21.15.7:0/624726070 conn(0x7fbe200737f0 msgr2=0x7fbe2006ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 shutdown_connections 2023-06-09T11:58:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:41.237+0000 7fbe281ff700 1 -- 172.21.15.7:0/624726070 wait complete. 2023-06-09T11:58:41.242 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:41.258 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:43.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.468+0000 7f42e79ff700 1 Processor -- start 2023-06-09T11:58:43.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.469+0000 7f42e79ff700 1 -- start start 2023-06-09T11:58:43.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e79ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f42e015b4a0 0x7f42e015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e79ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e015a500 con 0x7f42e00c9f40 2023-06-09T11:58:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e79ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e015da00 con 0x7f42e0153880 2023-06-09T11:58:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e79ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e00c5f10 con 0x7f42e015b4a0 2023-06-09T11:58:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e5f9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f42e00c9f40 0x7f42e00c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56878/0 (socket says 172.21.15.7:56878) 2023-06-09T11:58:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e5f9c700 1 -- 172.21.15.7:0/3742973243 learned_addr learned my addr 172.21.15.7:0/3742973243 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:43.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.470+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/3742973243 >> v2:172.21.15.117:3300/0 conn(0x7f42e015b4a0 0x7f42e015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754699238 0 0) 0x7f42d8001490 con 0x7f42e00c9f40 2023-06-09T11:58:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42cc0035b0 con 0x7f42e00c9f40 2023-06-09T11:58:43.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258147702 0 0) 0x7f42dc001110 con 0x7f42e0153880 2023-06-09T11:58:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8001490 con 0x7f42e0153880 2023-06-09T11:58:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42e4f9a700 1 -- 172.21.15.7:0/3742973243 >> v1:172.21.15.117:6789/0 conn(0x7f42e0153880 legacy=0x7f42e015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42e4f9a700 1 -- 172.21.15.7:0/3742973243 >> v1:172.21.15.7:6789/0 conn(0x7f42e00c9f40 legacy=0x7f42e00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42e4f9a700 1 -- 172.21.15.7:0/3742973243 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e01596e0 con 0x7f42e015b4a0 2023-06-09T11:58:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.471+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/3742973243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e015d4c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009840 tx=0x7f42d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90274351c2eeda server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0011830 con 0x7f42e015b4a0 2023-06-09T11:58:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d0011990 con 0x7f42e015b4a0 2023-06-09T11:58:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42d7fff700 1 -- 172.21.15.7:0/3742973243 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0011d20 con 0x7f42e015b4a0 2023-06-09T11:58:43.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 -- 172.21.15.7:0/3742973243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 msgr2=0x7f42e015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 --2- 172.21.15.7:0/3742973243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e015d4c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009840 tx=0x7f42d0009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 -- 172.21.15.7:0/3742973243 shutdown_connections 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 --2- 172.21.15.7:0/3742973243 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e015d4c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 -- 172.21.15.7:0/3742973243 >> 172.21.15.7:0/3742973243 conn(0x7f42e00737f0 msgr2=0x7f42e00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 -- 172.21.15.7:0/3742973243 shutdown_connections 2023-06-09T11:58:43.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.472+0000 7f42e79ff700 1 -- 172.21.15.7:0/3742973243 wait complete. 2023-06-09T11:58:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.474+0000 7f42e79ff700 1 Processor -- start 2023-06-09T11:58:43.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.474+0000 7f42e79ff700 1 -- start start 2023-06-09T11:58:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 0x7f42e01f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e01f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01542e0 con 0x7f42e00c9f40 2023-06-09T11:58:43.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e0154e50 con 0x7f42e015b4a0 2023-06-09T11:58:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e79ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e01f02f0 con 0x7f42e0153880 2023-06-09T11:58:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e579b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33692/0 (socket says 172.21.15.7:33692) 2023-06-09T11:58:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.475+0000 7f42e579b700 1 -- 172.21.15.7:0/1839324377 learned_addr learned my addr 172.21.15.7:0/1839324377 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:43.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 -- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 msgr2=0x7f42e01f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e5f9c700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e01f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 0x7f42e01f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 0x7f42e01f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 -- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 msgr2=0x7f42e01f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e01f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 -- 172.21.15.7:0/1839324377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00094a0 con 0x7f42e00c9f40 2023-06-09T11:58:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e5f9c700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e01f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 0x7f42e01f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e579b700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c700 tx=0x7f42dc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0052d0 con 0x7f42e00c9f40 2023-06-09T11:58:43.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc005450 con 0x7f42e00c9f40 2023-06-09T11:58:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0038c0 con 0x7f42e00c9f40 2023-06-09T11:58:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e0159c60 con 0x7f42e00c9f40 2023-06-09T11:58:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.476+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e0159980 con 0x7f42e00c9f40 2023-06-09T11:58:43.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.478+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e0159f40 con 0x7f42e00c9f40 2023-06-09T11:58:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.479+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42dc003a20 con 0x7f42e00c9f40 2023-06-09T11:58:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.479+0000 7f42d5ffb700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 0x7f42c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:43.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.479+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f42dc089240 con 0x7f42e00c9f40 2023-06-09T11:58:43.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.480+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 0x7f42c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:43.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.480+0000 7f42e4f9a700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 0x7f42c406d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f42e00d0580 tx=0x7f42d001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:43.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.484+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f42dc053920 con 0x7f42e00c9f40 2023-06-09T11:58:43.738 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.736+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42e015da00 con 0x7f42e00c9f40 2023-06-09T11:58:43.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.737+0000 7f42d5ffb700 1 -- 172.21.15.7:0/1839324377 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f42e015da00 con 0x7f42e00c9f40 2023-06-09T11:58:43.740 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:43.740 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 msgr2=0x7f42c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 0x7f42c406d190 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f42e00d0580 tx=0x7f42d001c040 comp rx=0 tx=0).stop 2023-06-09T11:58:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 msgr2=0x7f42e00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00c700 tx=0x7f42dc00cac0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 shutdown_connections 2023-06-09T11:58:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f42e015b4a0 0x7f42e01f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42e00c9f40 0x7f42e00b4440 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f42c406ace0 0x7f42c406d190 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 --2- 172.21.15.7:0/1839324377 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f42e0153880 0x7f42e01f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 >> 172.21.15.7:0/1839324377 conn(0x7f42e00737f0 msgr2=0x7f42e014d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 shutdown_connections 2023-06-09T11:58:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:43.739+0000 7f42e79ff700 1 -- 172.21.15.7:0/1839324377 wait complete. 2023-06-09T11:58:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:43.760 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:45.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:45.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.970+0000 7fa742dff700 1 Processor -- start 2023-06-09T11:58:45.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.970+0000 7fa742dff700 1 -- start start 2023-06-09T11:58:45.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa742dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 0x7fa73c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa742dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa73c0c9f40 0x7fa73c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa742dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c15a500 con 0x7fa73c153880 2023-06-09T11:58:45.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa742dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0c5f10 con 0x7fa73c0c9f40 2023-06-09T11:58:45.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa742dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c15da00 con 0x7fa73c15b4a0 2023-06-09T11:58:45.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa74139c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa73c0c9f40 0x7fa73c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa74139c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa73c0c9f40 0x7fa73c0c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33706/0 (socket says 172.21.15.7:33706) 2023-06-09T11:58:45.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.971+0000 7fa74139c700 1 -- 172.21.15.7:0/3510218013 learned_addr learned my addr 172.21.15.7:0/3510218013 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:45.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa73bfff700 1 --2- 172.21.15.7:0/3510218013 >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 0x7fa73c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa74139c700 1 -- 172.21.15.7:0/3510218013 >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 msgr2=0x7fa73c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa74139c700 1 --2- 172.21.15.7:0/3510218013 >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 0x7fa73c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa74139c700 1 -- 172.21.15.7:0/3510218013 >> v1:172.21.15.117:6789/0 conn(0x7fa73c153880 legacy=0x7fa73c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa74139c700 1 -- 172.21.15.7:0/3510218013 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c1596e0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa73bfff700 1 --2- 172.21.15.7:0/3510218013 >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 0x7fa73c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa74139c700 1 --2- 172.21.15.7:0/3510218013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c0c4e30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0cce30 tx=0x7fa73400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01f8d006e497a07 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa73b7fe700 1 -- 172.21.15.7:0/3510218013 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400e070 con 0x7fa73c0c9f40 2023-06-09T11:58:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa73b7fe700 1 -- 172.21.15.7:0/3510218013 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400b9c0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.972+0000 7fa73b7fe700 1 -- 172.21.15.7:0/3510218013 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400bd70 con 0x7fa73c0c9f40 2023-06-09T11:58:45.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 -- 172.21.15.7:0/3510218013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 msgr2=0x7fa73c0c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 --2- 172.21.15.7:0/3510218013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c0c4e30 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0cce30 tx=0x7fa73400b330 comp rx=0 tx=0).stop 2023-06-09T11:58:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 -- 172.21.15.7:0/3510218013 shutdown_connections 2023-06-09T11:58:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 --2- 172.21.15.7:0/3510218013 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c0c4e30 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 --2- 172.21.15.7:0/3510218013 >> v2:172.21.15.117:3300/0 conn(0x7fa73c15b4a0 0x7fa73c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:45.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 -- 172.21.15.7:0/3510218013 >> 172.21.15.7:0/3510218013 conn(0x7fa73c0737f0 msgr2=0x7fa73c0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:45.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 -- 172.21.15.7:0/3510218013 shutdown_connections 2023-06-09T11:58:45.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.973+0000 7fa742dff700 1 -- 172.21.15.7:0/3510218013 wait complete. 2023-06-09T11:58:45.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.975+0000 7fa742dff700 1 Processor -- start 2023-06-09T11:58:45.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.975+0000 7fa742dff700 1 -- start start 2023-06-09T11:58:45.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa73c153880 0x7fa73c1e4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c15b4a0 0x7fa73c1e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0cd9f0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0cf140 con 0x7fa73c15b4a0 2023-06-09T11:58:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa742dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0cde80 con 0x7fa73c153880 2023-06-09T11:58:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa740b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa740b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33710/0 (socket says 172.21.15.7:33710) 2023-06-09T11:58:45.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa740b9b700 1 -- 172.21.15.7:0/1380470697 learned_addr learned my addr 172.21.15.7:0/1380470697 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa74139c700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c15b4a0 0x7fa73c1e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.976+0000 7fa73bfff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa73c153880 0x7fa73c1e4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 -- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa73c153880 msgr2=0x7fa73c1e4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa73c153880 0x7fa73c1e4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:45.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 -- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c15b4a0 msgr2=0x7fa73c1e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c15b4a0 0x7fa73c1e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 -- 172.21.15.7:0/1380470697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa734009b30 con 0x7fa73c0c9f40 2023-06-09T11:58:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa740b9b700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c4e0 tx=0x7fa73000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7300052d0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa730005430 con 0x7fa73c0c9f40 2023-06-09T11:58:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c159c60 con 0x7fa73c0c9f40 2023-06-09T11:58:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7300038c0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.977+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c159980 con 0x7fa73c0c9f40 2023-06-09T11:58:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.978+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c159f40 con 0x7fa73c0c9f40 2023-06-09T11:58:45.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.979+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa730003a20 con 0x7fa73c0c9f40 2023-06-09T11:58:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.980+0000 7fa7397fa700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 0x7fa72006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.980+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fa730088ff0 con 0x7fa73c0c9f40 2023-06-09T11:58:45.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.981+0000 7fa73bfff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 0x7fa72006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.981+0000 7fa73bfff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 0x7fa72006d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0cbca0 tx=0x7fa728008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:45.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:45.985+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa7300537b0 con 0x7fa73c0c9f40 2023-06-09T11:58:46.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.237+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa73c15a500 con 0x7fa73c0c9f40 2023-06-09T11:58:46.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.238+0000 7fa7397fa700 1 -- 172.21.15.7:0/1380470697 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7fa73c15a500 con 0x7fa73c0c9f40 2023-06-09T11:58:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:46.242 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.240+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 msgr2=0x7fa72006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.240+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 0x7fa72006d190 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0cbca0 tx=0x7fa728008040 comp rx=0 tx=0).stop 2023-06-09T11:58:46.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 msgr2=0x7fa73c06a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c4e0 tx=0x7fa73000c7f0 comp rx=0 tx=0).stop 2023-06-09T11:58:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 shutdown_connections 2023-06-09T11:58:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa73c15b4a0 0x7fa73c1e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa73c0c9f40 0x7fa73c06a740 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:46.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa72006ace0 0x7fa72006d190 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 --2- 172.21.15.7:0/1380470697 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa73c153880 0x7fa73c1e4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 >> 172.21.15.7:0/1380470697 conn(0x7fa73c0737f0 msgr2=0x7fa73c14daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 shutdown_connections 2023-06-09T11:58:46.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:46.241+0000 7fa742dff700 1 -- 172.21.15.7:0/1380470697 wait complete. 2023-06-09T11:58:46.247 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:46.262 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:48.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.470+0000 7f26469ff700 1 Processor -- start 2023-06-09T11:58:48.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.470+0000 7f26469ff700 1 -- start start 2023-06-09T11:58:48.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.471+0000 7f26469ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.471+0000 7f26469ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.471+0000 7f26469ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26400c9bd0 con 0x7f26400c7cb0 2023-06-09T11:58:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.471+0000 7f26469ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2640159360 con 0x7f26400c88e0 2023-06-09T11:58:48.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.471+0000 7f26469ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26400c8e20 con 0x7f26400c2ae0 2023-06-09T11:58:48.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263f7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:34578/0 (socket says 172.21.15.7:34578) 2023-06-09T11:58:48.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263f7fe700 1 -- 172.21.15.7:0/1130303156 learned_addr learned my addr 172.21.15.7:0/1130303156 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:48.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 --2- 172.21.15.7:0/1130303156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:42056/0 (socket says 172.21.15.7:42056) 2023-06-09T11:58:48.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f2644f9c700 1 --1- 172.21.15.7:0/1130303156 >> v1:172.21.15.117:6789/0 conn(0x7f26400c7cb0 0x7f264015c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:38896/0 (socket says 172.21.15.7:38896) 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 -- 172.21.15.7:0/1130303156 >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 msgr2=0x7f264015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 --2- 172.21.15.7:0/1130303156 >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 -- 172.21.15.7:0/1130303156 >> v1:172.21.15.117:6789/0 conn(0x7f26400c7cb0 legacy=0x7f264015c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263ffff700 1 -- 172.21.15.7:0/1130303156 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2640159570 con 0x7f26400c2ae0 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263f7fe700 1 --2- 172.21.15.7:0/1130303156 >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:58:48.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.472+0000 7f263effd700 1 -- 172.21.15.7:0/1130303156 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044216475 0 0) 0x7f26400c9bd0 con 0x7f26400c7cb0 2023-06-09T11:58:48.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.473+0000 7f263ffff700 1 --2- 172.21.15.7:0/1130303156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2630009840 tx=0x7f2630009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f7709ff6287fa77 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:48.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.473+0000 7f263effd700 1 -- 172.21.15.7:0/1130303156 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630011830 con 0x7f26400c2ae0 2023-06-09T11:58:48.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.473+0000 7f263effd700 1 -- 172.21.15.7:0/1130303156 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630011990 con 0x7f26400c2ae0 2023-06-09T11:58:48.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.473+0000 7f263effd700 1 -- 172.21.15.7:0/1130303156 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2630011d20 con 0x7f26400c2ae0 2023-06-09T11:58:48.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 -- 172.21.15.7:0/1130303156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 msgr2=0x7f26400c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 --2- 172.21.15.7:0/1130303156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2630009840 tx=0x7f2630009b50 comp rx=0 tx=0).stop 2023-06-09T11:58:48.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 -- 172.21.15.7:0/1130303156 shutdown_connections 2023-06-09T11:58:48.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 --2- 172.21.15.7:0/1130303156 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f26400c2eb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 --2- 172.21.15.7:0/1130303156 >> v2:172.21.15.117:3300/0 conn(0x7f26400c88e0 0x7f264015a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 -- 172.21.15.7:0/1130303156 >> 172.21.15.7:0/1130303156 conn(0x7f26400cd330 msgr2=0x7f26400cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:48.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 -- 172.21.15.7:0/1130303156 shutdown_connections 2023-06-09T11:58:48.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.474+0000 7f26469ff700 1 -- 172.21.15.7:0/1130303156 wait complete. 2023-06-09T11:58:48.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.475+0000 7f26469ff700 1 Processor -- start 2023-06-09T11:58:48.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.476+0000 7f26469ff700 1 -- start start 2023-06-09T11:58:48.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f264014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26400c88e0 0x7f2640150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26400c5e30 con 0x7f26400c7cb0 2023-06-09T11:58:48.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264014f940 con 0x7f26400c88e0 2023-06-09T11:58:48.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f26469ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26400c60d0 con 0x7f26400c2ae0 2023-06-09T11:58:48.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33722/0 (socket says 172.21.15.7:33722) 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 -- 172.21.15.7:0/862504216 learned_addr learned my addr 172.21.15.7:0/862504216 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f2644f9c700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26400c88e0 0x7f2640150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263ffff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f264014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 -- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 msgr2=0x7f264014dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f264014dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 -- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26400c88e0 msgr2=0x7f2640150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26400c88e0 0x7f2640150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.477+0000 7f263f7fe700 1 -- 172.21.15.7:0/862504216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634009550 con 0x7f26400c7cb0 2023-06-09T11:58:48.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f263f7fe700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9b0 tx=0x7f263400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:48.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400ee50 con 0x7f26400c7cb0 2023-06-09T11:58:48.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263400c850 con 0x7f26400c7cb0 2023-06-09T11:58:48.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400cbc0 con 0x7f26400c7cb0 2023-06-09T11:58:48.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26300094a0 con 0x7f26400c7cb0 2023-06-09T11:58:48.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.478+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2640159af0 con 0x7f26400c7cb0 2023-06-09T11:58:48.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.479+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2640159810 con 0x7f26400c7cb0 2023-06-09T11:58:48.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.480+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f263400cd20 con 0x7f26400c7cb0 2023-06-09T11:58:48.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.481+0000 7f263cff9700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 0x7f262c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:48.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.481+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f2634089d90 con 0x7f26400c7cb0 2023-06-09T11:58:48.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.482+0000 7f263ffff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 0x7f262c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:48.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.482+0000 7f263ffff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 0x7f262c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f26300090b0 tx=0x7f2630008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:48.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.486+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f26340544f0 con 0x7f26400c7cb0 2023-06-09T11:58:48.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.736+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2640159dd0 con 0x7f26400c7cb0 2023-06-09T11:58:48.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.736+0000 7f263cff9700 1 -- 172.21.15.7:0/862504216 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f2634021090 con 0x7f26400c7cb0 2023-06-09T11:58:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:48.740 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:48.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 msgr2=0x7f262c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 0x7f262c06d190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f26300090b0 tx=0x7f2630008030 comp rx=0 tx=0).stop 2023-06-09T11:58:48.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 msgr2=0x7f26400c4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:48.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f263400e9b0 tx=0x7f263400ecc0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 shutdown_connections 2023-06-09T11:58:48.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f26400c88e0 0x7f2640150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f262c06ace0 0x7f262c06d190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f26400c2ae0 0x7f264014dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 --2- 172.21.15.7:0/862504216 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f26400c7cb0 0x7f26400c4d50 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:48.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 >> 172.21.15.7:0/862504216 conn(0x7f26400cd330 msgr2=0x7f2640158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:48.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 shutdown_connections 2023-06-09T11:58:48.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:48.739+0000 7f26469ff700 1 -- 172.21.15.7:0/862504216 wait complete. 2023-06-09T11:58:48.745 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:48.761 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 223, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 223, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:40.804214+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:58:50.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:50.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.970+0000 7f8c577ff700 1 Processor -- start 2023-06-09T11:58:50.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.970+0000 7f8c577ff700 1 -- start start 2023-06-09T11:58:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.971+0000 7f8c577ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c501501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.971+0000 7f8c577ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c500c0140 0x7f8c501506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.971+0000 7f8c577ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500c0bf0 con 0x7f8c500c1bc0 2023-06-09T11:58:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.971+0000 7f8c577ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c50070110 con 0x7f8c500c0140 2023-06-09T11:58:50.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.971+0000 7f8c577ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50071300 con 0x7f8c5015a0c0 2023-06-09T11:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c500c0140 0x7f8c501506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8c500c0140 0x7f8c501506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33726/0 (socket says 172.21.15.7:33726) 2023-06-09T11:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 -- 172.21.15.7:0/5091378 learned_addr learned my addr 172.21.15.7:0/5091378 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c54d9a700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c501501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 -- 172.21.15.7:0/5091378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 msgr2=0x7f8c501501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c501501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 -- 172.21.15.7:0/5091378 >> v1:172.21.15.117:6789/0 conn(0x7f8c500c1bc0 legacy=0x7f8c501599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 -- 172.21.15.7:0/5091378 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500702f0 con 0x7f8c500c0140 2023-06-09T11:58:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.972+0000 7f8c55d9c700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c0140 0x7f8c501506f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009e50 tx=0x7f8c4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b9e88681bbc747 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:50.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c47fff700 1 -- 172.21.15.7:0/5091378 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800bbc0 con 0x7f8c500c0140 2023-06-09T11:58:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c47fff700 1 -- 172.21.15.7:0/5091378 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4800bd20 con 0x7f8c500c0140 2023-06-09T11:58:50.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c47fff700 1 -- 172.21.15.7:0/5091378 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c480116a0 con 0x7f8c500c0140 2023-06-09T11:58:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 -- 172.21.15.7:0/5091378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c0140 msgr2=0x7f8c501506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c0140 0x7f8c501506f0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009e50 tx=0x7f8c4800b330 comp rx=0 tx=0).stop 2023-06-09T11:58:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 -- 172.21.15.7:0/5091378 shutdown_connections 2023-06-09T11:58:50.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c0140 0x7f8c501506f0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 --2- 172.21.15.7:0/5091378 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c501501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 -- 172.21.15.7:0/5091378 >> 172.21.15.7:0/5091378 conn(0x7f8c5006a4d0 msgr2=0x7f8c5006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 -- 172.21.15.7:0/5091378 shutdown_connections 2023-06-09T11:58:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.973+0000 7f8c577ff700 1 -- 172.21.15.7:0/5091378 wait complete. 2023-06-09T11:58:50.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.975+0000 7f8c577ff700 1 Processor -- start 2023-06-09T11:58:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.975+0000 7f8c577ff700 1 -- start start 2023-06-09T11:58:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500c0140 0x7f8c501eb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c50158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50157bf0 con 0x7f8c500c1bc0 2023-06-09T11:58:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c50157370 con 0x7f8c500c0140 2023-06-09T11:58:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c577ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501591b0 con 0x7f8c5015a0c0 2023-06-09T11:58:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.976+0000 7f8c54d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33728/0 (socket says 172.21.15.7:33728) 2023-06-09T11:58:50.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 -- 172.21.15.7:0/3346856247 learned_addr learned my addr 172.21.15.7:0/3346856247 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:50.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c55d9c700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c50158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c5559b700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500c0140 0x7f8c501eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 -- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 msgr2=0x7f8c50158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:50.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c50158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:50.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 -- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500c0140 msgr2=0x7f8c501eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500c0140 0x7f8c501eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 -- 172.21.15.7:0/3346856247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009b30 con 0x7f8c500c1bc0 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c54d9a700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000e810 tx=0x7f8c4000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40013610 con 0x7f8c500c1bc0 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c40013770 con 0x7f8c500c1bc0 2023-06-09T11:58:50.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c500c1170 con 0x7f8c500c1bc0 2023-06-09T11:58:50.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.977+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c40013b20 con 0x7f8c500c1bc0 2023-06-09T11:58:50.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.978+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c500c0e90 con 0x7f8c500c1bc0 2023-06-09T11:58:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.979+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c500c1450 con 0x7f8c500c1bc0 2023-06-09T11:58:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.980+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c40013c80 con 0x7f8c500c1bc0 2023-06-09T11:58:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.981+0000 7f8c45ffb700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:50.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.981+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f8c40089b00 con 0x7f8c500c1bc0 2023-06-09T11:58:50.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.981+0000 7f8c5559b700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:50.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.982+0000 7f8c5559b700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:50.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:50.986+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c4008be60 con 0x7f8c500c1bc0 2023-06-09T11:58:51.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.236+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c500c1f90 con 0x7f8c500c1bc0 2023-06-09T11:58:51.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.236+0000 7f8c45ffb700 1 -- 172.21.15.7:0/3346856247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f8c40021090 con 0x7f8c500c1bc0 2023-06-09T11:58:51.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:51.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.238+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 msgr2=0x7f8c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.238+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c000f60 tx=0x7f8c4c009040 comp rx=0 tx=0).stop 2023-06-09T11:58:51.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.238+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 msgr2=0x7f8c5014b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.238+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000e810 tx=0x7f8c4000ebd0 comp rx=0 tx=0).stop 2023-06-09T11:58:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 shutdown_connections 2023-06-09T11:58:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c5015a0c0 0x7f8c50158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:51.243 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500c0140 0x7f8c501eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 --2- 172.21.15.7:0/3346856247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500c1bc0 0x7f8c5014b610 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 >> 172.21.15.7:0/3346856247 conn(0x7f8c5006a4d0 msgr2=0x7f8c500c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 shutdown_connections 2023-06-09T11:58:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:51.239+0000 7f8c577ff700 1 -- 172.21.15.7:0/3346856247 wait complete. 2023-06-09T11:58:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:51.260 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:53.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:53.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.469+0000 7f9ca5bff700 1 Processor -- start 2023-06-09T11:58:53.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.469+0000 7f9ca5bff700 1 -- start start 2023-06-09T11:58:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca0153880 0x7f9ca015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ca00c9f40 0x7f9ca015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ca015b2d0 0x7f9ca00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca00c5f10 con 0x7f9ca00c9f40 2023-06-09T11:58:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca015a500 con 0x7f9ca015b2d0 2023-06-09T11:58:53.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9ca5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca015da00 con 0x7f9ca0153880 2023-06-09T11:58:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9c9effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ca00c9f40 0x7f9ca015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9c9effd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ca00c9f40 0x7f9ca015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37834/0 (socket says 172.21.15.7:37834) 2023-06-09T11:58:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.470+0000 7f9c9effd700 1 -- 172.21.15.7:0/2129523022 learned_addr learned my addr 172.21.15.7:0/2129523022 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:53.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9ffff700 1 --2- 172.21.15.7:0/2129523022 >> v2:172.21.15.117:3300/0 conn(0x7f9ca015b2d0 0x7f9ca00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9f7fe700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca0153880 0x7f9ca015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 -- 172.21.15.7:0/2129523022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca0153880 msgr2=0x7f9ca015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca0153880 0x7f9ca015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 -- 172.21.15.7:0/2129523022 >> v2:172.21.15.117:3300/0 conn(0x7f9ca015b2d0 msgr2=0x7f9ca00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:53.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 --2- 172.21.15.7:0/2129523022 >> v2:172.21.15.117:3300/0 conn(0x7f9ca015b2d0 0x7f9ca00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 -- 172.21.15.7:0/2129523022 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca015b750 con 0x7f9ca00c9f40 2023-06-09T11:58:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9effd700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca015ad90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0071db0 tx=0x7f9c94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa491e9ea3f55cf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9e7fc700 1 -- 172.21.15.7:0/2129523022 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400b870 con 0x7f9ca00c9f40 2023-06-09T11:58:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9e7fc700 1 -- 172.21.15.7:0/2129523022 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9400b9d0 con 0x7f9ca00c9f40 2023-06-09T11:58:53.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.471+0000 7f9c9e7fc700 1 -- 172.21.15.7:0/2129523022 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9400bd60 con 0x7f9ca00c9f40 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 -- 172.21.15.7:0/2129523022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 msgr2=0x7f9ca015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca015ad90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0071db0 tx=0x7f9c94009bd0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 -- 172.21.15.7:0/2129523022 shutdown_connections 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/2129523022 >> v2:172.21.15.117:3300/0 conn(0x7f9ca015b2d0 0x7f9ca00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca0153880 0x7f9ca015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/2129523022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca015ad90 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 -- 172.21.15.7:0/2129523022 >> 172.21.15.7:0/2129523022 conn(0x7f9ca00737f0 msgr2=0x7f9ca00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 -- 172.21.15.7:0/2129523022 shutdown_connections 2023-06-09T11:58:53.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.472+0000 7f9ca5bff700 1 -- 172.21.15.7:0/2129523022 wait complete. 2023-06-09T11:58:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.474+0000 7f9ca5bff700 1 Processor -- start 2023-06-09T11:58:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.474+0000 7f9ca5bff700 1 -- start start 2023-06-09T11:58:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 0x7f9ca01ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca015b2d0 0x7f9ca014c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca01e3f50 con 0x7f9ca00c9f40 2023-06-09T11:58:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca01e3c70 con 0x7f9ca0153880 2023-06-09T11:58:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9ca5bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00b2fa0 con 0x7f9ca015b2d0 2023-06-09T11:58:53.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9c9f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9c9f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37838/0 (socket says 172.21.15.7:37838) 2023-06-09T11:58:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.475+0000 7f9c9f7fe700 1 -- 172.21.15.7:0/3969392226 learned_addr learned my addr 172.21.15.7:0/3969392226 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9effd700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 0x7f9ca01ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9ffff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca015b2d0 0x7f9ca014c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 -- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca015b2d0 msgr2=0x7f9ca014c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca015b2d0 0x7f9ca014c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 -- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 msgr2=0x7f9ca01ef3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 0x7f9ca01ef3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 -- 172.21.15.7:0/3969392226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009550 con 0x7f9ca00c9f40 2023-06-09T11:58:53.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9effd700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 0x7f9ca01ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:58:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c9f7fe700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e810 tx=0x7f9c9000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90013610 con 0x7f9ca00c9f40 2023-06-09T11:58:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c90013770 con 0x7f9ca00c9f40 2023-06-09T11:58:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca0159980 con 0x7f9ca00c9f40 2023-06-09T11:58:53.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.476+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90013b00 con 0x7f9ca00c9f40 2023-06-09T11:58:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.477+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca01596e0 con 0x7f9ca00c9f40 2023-06-09T11:58:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.477+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca0159c60 con 0x7f9ca00c9f40 2023-06-09T11:58:53.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.478+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c90013c60 con 0x7f9ca00c9f40 2023-06-09T11:58:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.479+0000 7f9c87fff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 0x7f9c8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.479+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f9c90089b10 con 0x7f9ca00c9f40 2023-06-09T11:58:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.480+0000 7f9c9effd700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 0x7f9c8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:53.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.480+0000 7f9c9effd700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 0x7f9c8c06d190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c94008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.484+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9c900541e0 con 0x7f9ca00c9f40 2023-06-09T11:58:53.735 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.732+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ca00b2e00 con 0x7f9ca00c9f40 2023-06-09T11:58:53.736 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.733+0000 7f9c87fff700 1 -- 172.21.15.7:0/3969392226 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f9ca00b2e00 con 0x7f9ca00c9f40 2023-06-09T11:58:53.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:53.737 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 msgr2=0x7f9c8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 0x7f9c8c06d190 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c94000c00 tx=0x7f9c94008030 comp rx=0 tx=0).stop 2023-06-09T11:58:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 msgr2=0x7f9ca006a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f9c9000e810 tx=0x7f9c9000ebd0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 shutdown_connections 2023-06-09T11:58:53.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9c8c06ace0 0x7f9c8c06d190 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ca015b2d0 0x7f9ca014c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ca00c9f40 0x7f9ca006a740 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 --2- 172.21.15.7:0/3969392226 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ca0153880 0x7f9ca01ef3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:53.740 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.735+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 >> 172.21.15.7:0/3969392226 conn(0x7f9ca00737f0 msgr2=0x7f9ca014dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:53.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.736+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 shutdown_connections 2023-06-09T11:58:53.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:53.736+0000 7f9ca5bff700 1 -- 172.21.15.7:0/3969392226 wait complete. 2023-06-09T11:58:53.741 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:53.757 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:55.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:55.271 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:58:55.758 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.966+0000 7f34575ff700 1 Processor -- start 2023-06-09T11:58:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.966+0000 7f34575ff700 1 -- start start 2023-06-09T11:58:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f34575ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f345015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f34575ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34500c9f40 0x7f34500c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f34575ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345015a500 con 0x7f3450153880 2023-06-09T11:58:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f34575ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34500c5f10 con 0x7f34500c9f40 2023-06-09T11:58:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f34575ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345015da00 con 0x7f345015b4a0 2023-06-09T11:58:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f3455b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34500c9f40 0x7f34500c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f3455b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f34500c9f40 0x7f34500c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37850/0 (socket says 172.21.15.7:37850) 2023-06-09T11:58:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f3455b9c700 1 -- 172.21.15.7:0/2472449663 learned_addr learned my addr 172.21.15.7:0/2472449663 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f3454b9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f345015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.973 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.967+0000 7f345539b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f3450153880 0x7f345015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:59978/0 (socket says 172.21.15.7:59978) 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3455b9c700 1 -- 172.21.15.7:0/2472449663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 msgr2=0x7f345015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3455b9c700 1 --2- 172.21.15.7:0/2472449663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f345015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3455b9c700 1 -- 172.21.15.7:0/2472449663 >> v1:172.21.15.117:6789/0 conn(0x7f3450153880 legacy=0x7f345015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3447fff700 1 -- 172.21.15.7:0/2472449663 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1733888075 0 0) 0x7f344c001110 con 0x7f3450153880 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3455b9c700 1 -- 172.21.15.7:0/2472449663 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34501596e0 con 0x7f34500c9f40 2023-06-09T11:58:55.974 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3455b9c700 1 --2- 172.21.15.7:0/2472449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500c9f40 0x7f34500c4e30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc448bb4ff51d638 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3447fff700 1 -- 172.21.15.7:0/2472449663 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800e070 con 0x7f34500c9f40 2023-06-09T11:58:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3447fff700 1 -- 172.21.15.7:0/2472449663 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344800b910 con 0x7f34500c9f40 2023-06-09T11:58:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.968+0000 7f3447fff700 1 -- 172.21.15.7:0/2472449663 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800bcc0 con 0x7f34500c9f40 2023-06-09T11:58:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 -- 172.21.15.7:0/2472449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500c9f40 msgr2=0x7f34500c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:55.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 --2- 172.21.15.7:0/2472449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500c9f40 0x7f34500c4e30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f3448009e50 tx=0x7f344800b330 comp rx=0 tx=0).stop 2023-06-09T11:58:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 -- 172.21.15.7:0/2472449663 shutdown_connections 2023-06-09T11:58:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 --2- 172.21.15.7:0/2472449663 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34500c9f40 0x7f34500c4e30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 --2- 172.21.15.7:0/2472449663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f345015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 -- 172.21.15.7:0/2472449663 >> 172.21.15.7:0/2472449663 conn(0x7f34500737f0 msgr2=0x7f34500720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:55.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 -- 172.21.15.7:0/2472449663 shutdown_connections 2023-06-09T11:58:55.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.969+0000 7f34575ff700 1 -- 172.21.15.7:0/2472449663 wait complete. 2023-06-09T11:58:55.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.971+0000 7f34575ff700 1 Processor -- start 2023-06-09T11:58:55.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.971+0000 7f34575ff700 1 -- start start 2023-06-09T11:58:55.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 0x7f34500ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f3450152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3450150b60 con 0x7f3450153880 2023-06-09T11:58:55.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34500cd720 con 0x7f34500c9f40 2023-06-09T11:58:55.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f34575ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3450158bb0 con 0x7f345015b4a0 2023-06-09T11:58:55.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f3454b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f3454b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37852/0 (socket says 172.21.15.7:37852) 2023-06-09T11:58:55.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.972+0000 7f3454b9a700 1 -- 172.21.15.7:0/4102676842 learned_addr learned my addr 172.21.15.7:0/4102676842 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f345539b700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 0x7f34500ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3455b9c700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f3450152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 -- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 msgr2=0x7f3450152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f3450152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:55.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 -- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 msgr2=0x7f34500ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 0x7f34500ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 -- 172.21.15.7:0/4102676842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3448009b30 con 0x7f3450153880 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f345539b700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 0x7f34500ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3454b9a700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8a0 tx=0x7f344000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440009590 con 0x7f3450153880 2023-06-09T11:58:55.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344000eea0 con 0x7f3450153880 2023-06-09T11:58:55.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344000ca50 con 0x7f3450153880 2023-06-09T11:58:55.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3450159c60 con 0x7f3450153880 2023-06-09T11:58:55.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.973+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3450159980 con 0x7f3450153880 2023-06-09T11:58:55.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.974+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3450159f40 con 0x7f3450153880 2023-06-09T11:58:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.976+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f344000cbb0 con 0x7f3450153880 2023-06-09T11:58:55.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.976+0000 7f3445ffb700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 0x7f343406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:55.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.977+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f3440089d30 con 0x7f3450153880 2023-06-09T11:58:55.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.977+0000 7f345539b700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 0x7f343406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:55.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.977+0000 7f345539b700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 0x7f343406d1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f344c007fd0 tx=0x7f344c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:55.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:55.981+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3440054460 con 0x7f3450153880 2023-06-09T11:58:56.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.232+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3450152100 con 0x7f3450153880 2023-06-09T11:58:56.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.232+0000 7f3445ffb700 1 -- 172.21.15.7:0/4102676842 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f3450152100 con 0x7f3450153880 2023-06-09T11:58:56.236 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:56.236 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:56.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 msgr2=0x7f343406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 0x7f343406d1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f344c007fd0 tx=0x7f344c009040 comp rx=0 tx=0).stop 2023-06-09T11:58:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 msgr2=0x7f3450152450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f344000e8a0 tx=0x7f344000ec60 comp rx=0 tx=0).stop 2023-06-09T11:58:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 shutdown_connections 2023-06-09T11:58:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f343406ad10 0x7f343406d1c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f345015b4a0 0x7f3450152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f34500c9f40 0x7f34500ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 --2- 172.21.15.7:0/4102676842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3450153880 0x7f3450152450 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 >> 172.21.15.7:0/4102676842 conn(0x7f34500737f0 msgr2=0x7f345014bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:56.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 shutdown_connections 2023-06-09T11:58:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:56.235+0000 7f34575ff700 1 -- 172.21.15.7:0/4102676842 wait complete. 2023-06-09T11:58:56.241 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:56.257 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:58:58.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:58:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.468+0000 7f3f555ff700 1 Processor -- start 2023-06-09T11:58:58.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.468+0000 7f3f555ff700 1 -- start start 2023-06-09T11:58:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f555ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 0x7f3f5015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:58.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f555ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5015a500 con 0x7f3f50153880 2023-06-09T11:58:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f555ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5015da00 con 0x7f3f500c9f40 2023-06-09T11:58:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f555ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500c5f10 con 0x7f3f5015b4a0 2023-06-09T11:58:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f4effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f50153880 0x7f3f5015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56396/0 (socket says 172.21.15.7:56396) 2023-06-09T11:58:58.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.469+0000 7f3f4effd700 1 -- 172.21.15.7:0/2839007278 learned_addr learned my addr 172.21.15.7:0/2839007278 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:58:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/2839007278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 0x7f3f5015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601415183 0 0) 0x7f3f40001110 con 0x7f3f50153880 2023-06-09T11:58:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f380035b0 con 0x7f3f50153880 2023-06-09T11:58:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/2839007278 >> v1:172.21.15.117:6789/0 conn(0x7f3f500c9f40 legacy=0x7f3f500c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3015067799 0 0) 0x7f3f380035b0 con 0x7f3f50153880 2023-06-09T11:58:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/2839007278 >> v1:172.21.15.7:6789/0 conn(0x7f3f50153880 legacy=0x7f3f5015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4e7fc700 1 -- 172.21.15.7:0/2839007278 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f501596e0 con 0x7f3f5015b4a0 2023-06-09T11:58:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.470+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/2839007278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 0x7f3f5015d4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f44009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72eb963345f78383 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40001800 con 0x7f3f5015b4a0 2023-06-09T11:58:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f44004030 con 0x7f3f5015b4a0 2023-06-09T11:58:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f4dffb700 1 -- 172.21.15.7:0/2839007278 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440043c0 con 0x7f3f5015b4a0 2023-06-09T11:58:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 -- 172.21.15.7:0/2839007278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 msgr2=0x7f3f5015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 --2- 172.21.15.7:0/2839007278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 0x7f3f5015d4c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009870 tx=0x7f3f44009b80 comp rx=0 tx=0).stop 2023-06-09T11:58:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 -- 172.21.15.7:0/2839007278 shutdown_connections 2023-06-09T11:58:58.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 --2- 172.21.15.7:0/2839007278 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f5015b4a0 0x7f3f5015d4c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 -- 172.21.15.7:0/2839007278 >> 172.21.15.7:0/2839007278 conn(0x7f3f500737f0 msgr2=0x7f3f500720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 -- 172.21.15.7:0/2839007278 shutdown_connections 2023-06-09T11:58:58.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.471+0000 7f3f555ff700 1 -- 172.21.15.7:0/2839007278 wait complete. 2023-06-09T11:58:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.473+0000 7f3f555ff700 1 Processor -- start 2023-06-09T11:58:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.473+0000 7f3f555ff700 1 -- start start 2023-06-09T11:58:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f500c9f40 0x7f3f5014b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f50153880 0x7f3f501ef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:58.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500caa80 con 0x7f3f5015b4a0 2023-06-09T11:58:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f500cacb0 con 0x7f3f50153880 2023-06-09T11:58:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f555ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5014a040 con 0x7f3f500c9f40 2023-06-09T11:58:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:58.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f4f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37866/0 (socket says 172.21.15.7:37866) 2023-06-09T11:58:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/1590036198 learned_addr learned my addr 172.21.15.7:0/1590036198 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:58:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f4e7fc700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f50153880 0x7f3f501ef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.474+0000 7f3f4effd700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f500c9f40 0x7f3f5014b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f500c9f40 msgr2=0x7f3f5014b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f500c9f40 0x7f3f5014b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f50153880 msgr2=0x7f3f501ef0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f50153880 0x7f3f501ef0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 -- 172.21.15.7:0/1590036198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440094a0 con 0x7f3f5015b4a0 2023-06-09T11:58:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f4f7fe700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800caf0 tx=0x7f3f4800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:58.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48003e30 con 0x7f3f5015b4a0 2023-06-09T11:58:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48005070 con 0x7f3f5015b4a0 2023-06-09T11:58:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48005420 con 0x7f3f5015b4a0 2023-06-09T11:58:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.475+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f50159c60 con 0x7f3f5015b4a0 2023-06-09T11:58:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.476+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f50159980 con 0x7f3f5015b4a0 2023-06-09T11:58:58.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.477+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f50159f40 con 0x7f3f5015b4a0 2023-06-09T11:58:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.477+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f48005580 con 0x7f3f5015b4a0 2023-06-09T11:58:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.479+0000 7f3f377fe700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 0x7f3f3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:58:58.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.479+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f3f48089070 con 0x7f3f5015b4a0 2023-06-09T11:58:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.479+0000 7f3f4effd700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 0x7f3f3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:58:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.480+0000 7f3f4effd700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 0x7f3f3c06d190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001930 tx=0x7f3f40001760 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:58:58.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.483+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3f48057230 con 0x7f3f5015b4a0 2023-06-09T11:58:58.742 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.739+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f5014ac50 con 0x7f3f5015b4a0 2023-06-09T11:58:58.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.740+0000 7f3f377fe700 1 -- 172.21.15.7:0/1590036198 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f3f48017020 con 0x7f3f5015b4a0 2023-06-09T11:58:58.743 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:58:58.744 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:58:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 msgr2=0x7f3f3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 0x7f3f3c06d190 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3f40001930 tx=0x7f3f40001760 comp rx=0 tx=0).stop 2023-06-09T11:58:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 msgr2=0x7f3f501ef630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:58:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800caf0 tx=0x7f3f4800ce00 comp rx=0 tx=0).stop 2023-06-09T11:58:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 shutdown_connections 2023-06-09T11:58:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f5015b4a0 0x7f3f501ef630 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f3c06ace0 0x7f3f3c06d190 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f500c9f40 0x7f3f5014b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 --2- 172.21.15.7:0/1590036198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f50153880 0x7f3f501ef0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:58:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 >> 172.21.15.7:0/1590036198 conn(0x7f3f500737f0 msgr2=0x7f3f5014c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:58:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 shutdown_connections 2023-06-09T11:58:58.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:58:58.742+0000 7f3f555ff700 1 -- 172.21.15.7:0/1590036198 wait complete. 2023-06-09T11:58:58.748 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:58:58.763 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:00.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.971+0000 7f03a59ff700 1 Processor -- start 2023-06-09T11:59:00.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.972+0000 7f03a59ff700 1 -- start start 2023-06-09T11:59:00.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f03a59ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f03a015b4a0 0x7f03a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:00.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f03a59ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a015a500 con 0x7f03a0153880 2023-06-09T11:59:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f03a59ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a015da00 con 0x7f03a00c9f40 2023-06-09T11:59:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f03a59ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00c5f10 con 0x7f03a015b4a0 2023-06-09T11:59:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f039effd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f03a0153880 0x7f03a015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56408/0 (socket says 172.21.15.7:56408) 2023-06-09T11:59:00.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f039effd700 1 -- 172.21.15.7:0/1962557744 learned_addr learned my addr 172.21.15.7:0/1962557744 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:00.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.973+0000 7f039e7fc700 1 --2- 172.21.15.7:0/1962557744 >> v2:172.21.15.7:3300/0 conn(0x7f03a015b4a0 0x7f03a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039e7fc700 1 -- 172.21.15.7:0/1962557744 >> v1:172.21.15.117:6789/0 conn(0x7f03a00c9f40 legacy=0x7f03a00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039e7fc700 1 -- 172.21.15.7:0/1962557744 >> v1:172.21.15.7:6789/0 conn(0x7f03a0153880 legacy=0x7f03a015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039e7fc700 1 -- 172.21.15.7:0/1962557744 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a01596e0 con 0x7f03a015b4a0 2023-06-09T11:59:00.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039e7fc700 1 --2- 172.21.15.7:0/1962557744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a015d4c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0394009840 tx=0x7f0394009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc272ef2705c77c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039dffb700 1 -- 172.21.15.7:0/1962557744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0394004230 con 0x7f03a015b4a0 2023-06-09T11:59:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039dffb700 1 -- 172.21.15.7:0/1962557744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0394004390 con 0x7f03a015b4a0 2023-06-09T11:59:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.974+0000 7f039dffb700 1 -- 172.21.15.7:0/1962557744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940036a0 con 0x7f03a015b4a0 2023-06-09T11:59:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 -- 172.21.15.7:0/1962557744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 msgr2=0x7f03a015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:00.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 --2- 172.21.15.7:0/1962557744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a015d4c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0394009840 tx=0x7f0394009b50 comp rx=0 tx=0).stop 2023-06-09T11:59:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 -- 172.21.15.7:0/1962557744 shutdown_connections 2023-06-09T11:59:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 --2- 172.21.15.7:0/1962557744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a015d4c0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 -- 172.21.15.7:0/1962557744 >> 172.21.15.7:0/1962557744 conn(0x7f03a00737f0 msgr2=0x7f03a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 -- 172.21.15.7:0/1962557744 shutdown_connections 2023-06-09T11:59:00.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.975+0000 7f03a59ff700 1 -- 172.21.15.7:0/1962557744 wait complete. 2023-06-09T11:59:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.977+0000 7f03a59ff700 1 Processor -- start 2023-06-09T11:59:00.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.977+0000 7f03a59ff700 1 -- start start 2023-06-09T11:59:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a00c9f40 0x7f03a0150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03a0153880 0x7f03a00cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a01526f0 con 0x7f03a015b4a0 2023-06-09T11:59:00.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a006aca0 con 0x7f03a00c9f40 2023-06-09T11:59:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f03a59ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a006af80 con 0x7f03a0153880 2023-06-09T11:59:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37876/0 (socket says 172.21.15.7:37876) 2023-06-09T11:59:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 -- 172.21.15.7:0/316534175 learned_addr learned my addr 172.21.15.7:0/316534175 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:00.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039effd700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a00c9f40 0x7f03a0150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039e7fc700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03a0153880 0x7f03a00cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 -- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03a0153880 msgr2=0x7f03a00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03a0153880 0x7f03a00cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 -- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a00c9f40 msgr2=0x7f03a0150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:00.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a00c9f40 0x7f03a0150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.979+0000 7f039f7fe700 1 -- 172.21.15.7:0/316534175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940094a0 con 0x7f03a015b4a0 2023-06-09T11:59:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f039f7fe700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f039800cc10 tx=0x7f039800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039800abb0 con 0x7f03a015b4a0 2023-06-09T11:59:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039800ad10 con 0x7f03a015b4a0 2023-06-09T11:59:00.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a0159c60 con 0x7f03a015b4a0 2023-06-09T11:59:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0398012660 con 0x7f03a015b4a0 2023-06-09T11:59:00.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.980+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0159980 con 0x7f03a015b4a0 2023-06-09T11:59:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.981+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0159f40 con 0x7f03a015b4a0 2023-06-09T11:59:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.982+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03980127c0 con 0x7f03a015b4a0 2023-06-09T11:59:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.984+0000 7f0387fff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 0x7f038c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.984+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f039808a600 con 0x7f03a015b4a0 2023-06-09T11:59:00.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.984+0000 7f039effd700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 0x7f038c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:00.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.984+0000 7f039effd700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 0x7f038c06d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03a015a960 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:00.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:00.988+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0398054d60 con 0x7f03a015b4a0 2023-06-09T11:59:01.245 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.242+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03a0074cf0 con 0x7f03a015b4a0 2023-06-09T11:59:01.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.243+0000 7f0387fff700 1 -- 172.21.15.7:0/316534175 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f0398016020 con 0x7f03a015b4a0 2023-06-09T11:59:01.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:01.247 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:01.248 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 msgr2=0x7f038c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 0x7f038c06d190 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f03a015a960 tx=0x7f0390009040 comp rx=0 tx=0).stop 2023-06-09T11:59:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 msgr2=0x7f03a014c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f039800cc10 tx=0x7f039800a060 comp rx=0 tx=0).stop 2023-06-09T11:59:01.249 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 shutdown_connections 2023-06-09T11:59:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f03a015b4a0 0x7f03a014c5a0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f03a00c9f40 0x7f03a0150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f038c06ace0 0x7f038c06d190 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 --2- 172.21.15.7:0/316534175 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f03a0153880 0x7f03a00cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:01.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 >> 172.21.15.7:0/316534175 conn(0x7f03a00737f0 msgr2=0x7f03a015b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 shutdown_connections 2023-06-09T11:59:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:01.245+0000 7f03a59ff700 1 -- 172.21.15.7:0/316534175 wait complete. 2023-06-09T11:59:01.251 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:01.267 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 223, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 223, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:40.804214+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:59:03.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.476+0000 7fed7e9ff700 1 Processor -- start 2023-06-09T11:59:03.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.476+0000 7fed7e9ff700 1 -- start start 2023-06-09T11:59:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.477+0000 7fed7e9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed78153880 0x7fed7815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.477+0000 7fed7e9ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7815a500 con 0x7fed7815b4a0 2023-06-09T11:59:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.477+0000 7fed7e9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7815da00 con 0x7fed780c9f40 2023-06-09T11:59:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.477+0000 7fed7e9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed780c5f10 con 0x7fed78153880 2023-06-09T11:59:03.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.477+0000 7fed77fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed78153880 0x7fed7815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed78153880 0x7fed7815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39392/0 (socket says 172.21.15.7:39392) 2023-06-09T11:59:03.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 -- 172.21.15.7:0/2552803782 learned_addr learned my addr 172.21.15.7:0/2552803782 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 -- 172.21.15.7:0/2552803782 >> v1:172.21.15.117:6789/0 conn(0x7fed780c9f40 legacy=0x7fed7815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 -- 172.21.15.7:0/2552803782 >> v1:172.21.15.7:6789/0 conn(0x7fed7815b4a0 legacy=0x7fed780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 -- 172.21.15.7:0/2552803782 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed781596e0 con 0x7fed78153880 2023-06-09T11:59:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed77fff700 1 --2- 172.21.15.7:0/2552803782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78153880 0x7fed7815d4c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fed680098f0 tx=0x7fed68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67777d020d83daf4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed76ffd700 1 -- 172.21.15.7:0/2552803782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800b8a0 con 0x7fed78153880 2023-06-09T11:59:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed76ffd700 1 -- 172.21.15.7:0/2552803782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6800ba00 con 0x7fed78153880 2023-06-09T11:59:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.478+0000 7fed76ffd700 1 -- 172.21.15.7:0/2552803782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800bd90 con 0x7fed78153880 2023-06-09T11:59:03.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 -- 172.21.15.7:0/2552803782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78153880 msgr2=0x7fed7815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/2552803782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78153880 0x7fed7815d4c0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fed680098f0 tx=0x7fed68009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 -- 172.21.15.7:0/2552803782 shutdown_connections 2023-06-09T11:59:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/2552803782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed78153880 0x7fed7815d4c0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 -- 172.21.15.7:0/2552803782 >> 172.21.15.7:0/2552803782 conn(0x7fed780737f0 msgr2=0x7fed780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:03.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 -- 172.21.15.7:0/2552803782 shutdown_connections 2023-06-09T11:59:03.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.479+0000 7fed7e9ff700 1 -- 172.21.15.7:0/2552803782 wait complete. 2023-06-09T11:59:03.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.481+0000 7fed7e9ff700 1 Processor -- start 2023-06-09T11:59:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.481+0000 7fed7e9ff700 1 -- start start 2023-06-09T11:59:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed78153880 0x7fed78157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed7815b4a0 0x7fed78158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:03.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781efe20 con 0x7fed780c9f40 2023-06-09T11:59:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed781eefb0 con 0x7fed7815b4a0 2023-06-09T11:59:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7e9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7815c9b0 con 0x7fed78153880 2023-06-09T11:59:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39394/0 (socket says 172.21.15.7:39394) 2023-06-09T11:59:03.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed77fff700 1 -- 172.21.15.7:0/1380308307 learned_addr learned my addr 172.21.15.7:0/1380308307 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed7cf9c700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed7815b4a0 0x7fed78158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.482+0000 7fed777fe700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed78153880 0x7fed78157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 -- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed78153880 msgr2=0x7fed78157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed78153880 0x7fed78157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 -- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed7815b4a0 msgr2=0x7fed78158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed7815b4a0 0x7fed78158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 -- 172.21.15.7:0/1380308307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed68009550 con 0x7fed780c9f40 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed77fff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fed7806b580 tx=0x7fed68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed68004530 con 0x7fed780c9f40 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6801f070 con 0x7fed780c9f40 2023-06-09T11:59:03.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed680038a0 con 0x7fed780c9f40 2023-06-09T11:59:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed78159c60 con 0x7fed780c9f40 2023-06-09T11:59:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.483+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed78159980 con 0x7fed780c9f40 2023-06-09T11:59:03.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.484+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed78159f40 con 0x7fed780c9f40 2023-06-09T11:59:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.485+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed68021030 con 0x7fed780c9f40 2023-06-09T11:59:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.486+0000 7fed74ff9700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 0x7fed6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.486+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fed6808b200 con 0x7fed780c9f40 2023-06-09T11:59:03.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.486+0000 7fed777fe700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 0x7fed6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.487+0000 7fed777fe700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 0x7fed6406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fed7814dd20 tx=0x7fed6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:03.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.490+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fed68055b10 con 0x7fed780c9f40 2023-06-09T11:59:03.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.749+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed78074cf0 con 0x7fed780c9f40 2023-06-09T11:59:03.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.749+0000 7fed74ff9700 1 -- 172.21.15.7:0/1380308307 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7fed68015090 con 0x7fed780c9f40 2023-06-09T11:59:03.753 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:03.754 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:03.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 msgr2=0x7fed6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 0x7fed6406d190 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fed7814dd20 tx=0x7fed6c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 msgr2=0x7fed780b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fed7806b580 tx=0x7fed68004750 comp rx=0 tx=0).stop 2023-06-09T11:59:03.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 shutdown_connections 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed7815b4a0 0x7fed78158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed780c9f40 0x7fed780b1ea0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fed6406ace0 0x7fed6406d190 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 --2- 172.21.15.7:0/1380308307 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed78153880 0x7fed78157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 >> 172.21.15.7:0/1380308307 conn(0x7fed780737f0 msgr2=0x7fed7814b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:03.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 shutdown_connections 2023-06-09T11:59:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:03.752+0000 7fed7e9ff700 1 -- 172.21.15.7:0/1380308307 wait complete. 2023-06-09T11:59:03.758 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:03.774 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:05.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.984+0000 7f29c67ff700 1 Processor -- start 2023-06-09T11:59:05.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.984+0000 7f29c67ff700 1 -- start start 2023-06-09T11:59:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29c00cb5d0 0x7f29c00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c00cb9a0 0x7f29c0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29c015cd40 0x7f29c00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c0159b20 con 0x7f29c015cd40 2023-06-09T11:59:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00cac80 con 0x7f29c00cb5d0 2023-06-09T11:59:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c67ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01597d0 con 0x7f29c00cb9a0 2023-06-09T11:59:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29c015cd40 0x7f29c00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:05.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29c015cd40 0x7f29c00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39398/0 (socket says 172.21.15.7:39398) 2023-06-09T11:59:05.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 -- 172.21.15.7:0/1918361545 learned_addr learned my addr 172.21.15.7:0/1918361545 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29bffff700 1 --2- 172.21.15.7:0/1918361545 >> v2:172.21.15.117:3300/0 conn(0x7f29c00cb5d0 0x7f29c00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29bf7fe700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c00cb9a0 0x7f29c0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 -- 172.21.15.7:0/1918361545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c00cb9a0 msgr2=0x7f29c0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:05.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c00cb9a0 0x7f29c0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 -- 172.21.15.7:0/1918361545 >> v2:172.21.15.117:3300/0 conn(0x7f29c00cb5d0 msgr2=0x7f29c00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 --2- 172.21.15.7:0/1918361545 >> v2:172.21.15.117:3300/0 conn(0x7f29c00cb5d0 0x7f29c00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.986+0000 7f29c4d9c700 1 -- 172.21.15.7:0/1918361545 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0071ab0 con 0x7f29c015cd40 2023-06-09T11:59:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29c4d9c700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c015cd40 0x7f29c00cbea0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f29c008d420 tx=0x7f29b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0d8617d18a7e2c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:05.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29beffd700 1 -- 172.21.15.7:0/1918361545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800e070 con 0x7f29c015cd40 2023-06-09T11:59:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29beffd700 1 -- 172.21.15.7:0/1918361545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800b910 con 0x7f29c015cd40 2023-06-09T11:59:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29beffd700 1 -- 172.21.15.7:0/1918361545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800bcc0 con 0x7f29c015cd40 2023-06-09T11:59:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29c67ff700 1 -- 172.21.15.7:0/1918361545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c015cd40 msgr2=0x7f29c00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.987+0000 7f29c67ff700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c015cd40 0x7f29c00cbea0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f29c008d420 tx=0x7f29b800b330 comp rx=0 tx=0).stop 2023-06-09T11:59:05.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 -- 172.21.15.7:0/1918361545 shutdown_connections 2023-06-09T11:59:05.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 --2- 172.21.15.7:0/1918361545 >> v2:172.21.15.117:3300/0 conn(0x7f29c00cb5d0 0x7f29c00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:05.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c015cd40 0x7f29c00cbea0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:05.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 --2- 172.21.15.7:0/1918361545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c00cb9a0 0x7f29c0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:05.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 -- 172.21.15.7:0/1918361545 >> 172.21.15.7:0/1918361545 conn(0x7f29c0070af0 msgr2=0x7f29c0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:05.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 -- 172.21.15.7:0/1918361545 shutdown_connections 2023-06-09T11:59:05.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.988+0000 7f29c67ff700 1 -- 172.21.15.7:0/1918361545 wait complete. 2023-06-09T11:59:05.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.989+0000 7f29c67ff700 1 Processor -- start 2023-06-09T11:59:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.990+0000 7f29c67ff700 1 -- start start 2023-06-09T11:59:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 0x7f29c01528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c015cd40 0x7f29c00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:05.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c47f0 con 0x7f29c00cb5d0 2023-06-09T11:59:05.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c6050 con 0x7f29c00cb9a0 2023-06-09T11:59:05.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c67ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c00c8e20 con 0x7f29c015cd40 2023-06-09T11:59:05.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:05.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39400/0 (socket says 172.21.15.7:39400) 2023-06-09T11:59:05.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 -- 172.21.15.7:0/2926775325 learned_addr learned my addr 172.21.15.7:0/2926775325 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:06.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29c4d9c700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c015cd40 0x7f29c00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:06.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bf7fe700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 0x7f29c01528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:06.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 -- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c015cd40 msgr2=0x7f29c00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:06.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c015cd40 0x7f29c00c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 -- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 msgr2=0x7f29c01528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 0x7f29c01528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bffff700 1 -- 172.21.15.7:0/2926775325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b8009b30 con 0x7f29c00cb5d0 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.991+0000 7f29bf7fe700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 0x7f29c01528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29bffff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e870 tx=0x7f29b000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0013680 con 0x7f29c00cb5d0 2023-06-09T11:59:06.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00137e0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c015d4f0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0013b90 con 0x7f29c00cb5d0 2023-06-09T11:59:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.992+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c015d210 con 0x7f29c00cb5d0 2023-06-09T11:59:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.993+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c015d7d0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.994+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29b0013cf0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.995+0000 7f29bcff9700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 0x7f29ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:06.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.995+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f29b0089ff0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.995+0000 7f29bf7fe700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 0x7f29ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:06.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:05.996+0000 7f29bf7fe700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 0x7f29ac06d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f29c00b3760 tx=0x7f29b4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:06.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.000+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f29b0054720 con 0x7f29c00cb5d0 2023-06-09T11:59:06.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.261+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c0074cf0 con 0x7f29c00cb5d0 2023-06-09T11:59:06.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.261+0000 7f29bcff9700 1 -- 172.21.15.7:0/2926775325 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f29b0021090 con 0x7f29c00cb5d0 2023-06-09T11:59:06.265 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:06.265 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:06.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 msgr2=0x7f29ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:06.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 0x7f29ac06d190 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f29c00b3760 tx=0x7f29b4008040 comp rx=0 tx=0).stop 2023-06-09T11:59:06.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 msgr2=0x7f29c00c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:06.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f29b000e870 tx=0x7f29b000eb80 comp rx=0 tx=0).stop 2023-06-09T11:59:06.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 shutdown_connections 2023-06-09T11:59:06.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29ac06ace0 0x7f29ac06d190 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29c015cd40 0x7f29c00c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29c00cb5d0 0x7f29c00c3a00 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 --2- 172.21.15.7:0/2926775325 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29c00cb9a0 0x7f29c01528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 >> 172.21.15.7:0/2926775325 conn(0x7f29c0070af0 msgr2=0x7f29c00cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:06.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 shutdown_connections 2023-06-09T11:59:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:06.264+0000 7f29c67ff700 1 -- 172.21.15.7:0/2926775325 wait complete. 2023-06-09T11:59:06.270 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:06.285 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:08.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:08.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.494+0000 7ff731bff700 1 Processor -- start 2023-06-09T11:59:08.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.494+0000 7ff731bff700 1 -- start start 2023-06-09T11:59:08.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff731bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff72c0c1bc0 0x7ff72c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff731bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c0140 0x7ff72c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff731bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c0c0bf0 con 0x7ff72c0c1f90 2023-06-09T11:59:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff731bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c070110 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff731bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c071300 con 0x7ff72c0c0140 2023-06-09T11:59:08.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff72b7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff72c0c1bc0 0x7ff72c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff72bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff72c0c1f90 0x7ff72c1501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35572/0 (socket says 172.21.15.7:35572) 2023-06-09T11:59:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff72bfff700 1 -- 172.21.15.7:0/2424823558 learned_addr learned my addr 172.21.15.7:0/2424823558 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:08.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.496+0000 7ff72affd700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c0140 0x7ff72c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72b7fe700 1 -- 172.21.15.7:0/2424823558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c0140 msgr2=0x7ff72c159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:08.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72b7fe700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c0140 0x7ff72c159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72b7fe700 1 -- 172.21.15.7:0/2424823558 >> v1:172.21.15.7:6789/0 conn(0x7ff72c0c1f90 legacy=0x7ff72c1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72b7fe700 1 -- 172.21.15.7:0/2424823558 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c0702f0 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72b7fe700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c1599b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009840 tx=0x7ff71c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d5c6941e6ce1b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72a7fc700 1 -- 172.21.15.7:0/2424823558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c011830 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72a7fc700 1 -- 172.21.15.7:0/2424823558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c011990 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.497+0000 7ff72a7fc700 1 -- 172.21.15.7:0/2424823558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c011d20 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 -- 172.21.15.7:0/2424823558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 msgr2=0x7ff72c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c1599b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff71c009840 tx=0x7ff71c009b50 comp rx=0 tx=0).stop 2023-06-09T11:59:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 -- 172.21.15.7:0/2424823558 shutdown_connections 2023-06-09T11:59:08.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c1599b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 --2- 172.21.15.7:0/2424823558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c0140 0x7ff72c159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 -- 172.21.15.7:0/2424823558 >> 172.21.15.7:0/2424823558 conn(0x7ff72c06a4d0 msgr2=0x7ff72c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 -- 172.21.15.7:0/2424823558 shutdown_connections 2023-06-09T11:59:08.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.498+0000 7ff731bff700 1 -- 172.21.15.7:0/2424823558 wait complete. 2023-06-09T11:59:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.499+0000 7ff731bff700 1 Processor -- start 2023-06-09T11:59:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.500+0000 7ff731bff700 1 -- start start 2023-06-09T11:59:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.500+0000 7ff731bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c0c0140 0x7ff72c1e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.500+0000 7ff731bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff731bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 0x7ff72c147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff731bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c43e0 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff731bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c39e0 con 0x7ff72c0c0140 2023-06-09T11:59:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff731bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c25c0 con 0x7ff72c0c1f90 2023-06-09T11:59:08.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39410/0 (socket says 172.21.15.7:39410) 2023-06-09T11:59:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 -- 172.21.15.7:0/2738181043 learned_addr learned my addr 172.21.15.7:0/2738181043 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72b7fe700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c0c0140 0x7ff72c1e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72bfff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 0x7ff72c147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 -- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 msgr2=0x7ff72c147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 0x7ff72c147430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 -- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c0c0140 msgr2=0x7ff72c1e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c0c0140 0x7ff72c1e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72affd700 1 -- 172.21.15.7:0/2738181043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0094a0 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.501+0000 7ff72bfff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 0x7ff72c147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff72affd700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e9b0 tx=0x7ff72000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720009710 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000c850 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000cbc0 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c0c1170 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.502+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c0c0e90 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.503+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c0c1450 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.504+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff72000cd20 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.505+0000 7ff713fff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 0x7ff71806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.505+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7ff72008a180 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.505+0000 7ff72b7fe700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 0x7ff71806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.506+0000 7ff72b7fe700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 0x7ff71806d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff71c000c00 tx=0x7ff71c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:08.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.509+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff720054970 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.761+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff72c074cf0 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.762+0000 7ff713fff700 1 -- 172.21.15.7:0/2738181043 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7ff720057f90 con 0x7ff72c0c1bc0 2023-06-09T11:59:08.765 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:08.766 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 msgr2=0x7ff71806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 0x7ff71806d190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff71c000c00 tx=0x7ff71c008030 comp rx=0 tx=0).stop 2023-06-09T11:59:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 msgr2=0x7ff72c146ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e9b0 tx=0x7ff72000ecc0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 shutdown_connections 2023-06-09T11:59:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff71806ace0 0x7ff71806d190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff72c0c1f90 0x7ff72c147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff72c0c0140 0x7ff72c1e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.769 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 --2- 172.21.15.7:0/2738181043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff72c0c1bc0 0x7ff72c146ef0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 >> 172.21.15.7:0/2738181043 conn(0x7ff72c06a4d0 msgr2=0x7ff72c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.764+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 shutdown_connections 2023-06-09T11:59:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:08.765+0000 7ff731bff700 1 -- 172.21.15.7:0/2738181043 wait complete. 2023-06-09T11:59:08.770 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:08.786 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:10.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:10.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.995+0000 7fc3a6fff700 1 Processor -- start 2023-06-09T11:59:10.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.995+0000 7fc3a6fff700 1 -- start start 2023-06-09T11:59:11.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a6fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc3a0153880 0x7fc3a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a6fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a015b4a0 0x7fc3a00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a6fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a015a500 con 0x7fc3a00c9f40 2023-06-09T11:59:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a6fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a00c5f10 con 0x7fc3a015b4a0 2023-06-09T11:59:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a6fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a015da00 con 0x7fc3a0153880 2023-06-09T11:59:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a015b4a0 0x7fc3a00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc3a015b4a0 0x7fc3a00c4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39426/0 (socket says 172.21.15.7:39426) 2023-06-09T11:59:11.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 -- 172.21.15.7:0/3654209482 learned_addr learned my addr 172.21.15.7:0/3654209482 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:11.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a4d9b700 1 --2- 172.21.15.7:0/3654209482 >> v2:172.21.15.117:3300/0 conn(0x7fc3a0153880 0x7fc3a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 -- 172.21.15.7:0/3654209482 >> v2:172.21.15.117:3300/0 conn(0x7fc3a0153880 msgr2=0x7fc3a015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 --2- 172.21.15.7:0/3654209482 >> v2:172.21.15.117:3300/0 conn(0x7fc3a0153880 0x7fc3a015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 -- 172.21.15.7:0/3654209482 >> v1:172.21.15.117:6789/0 conn(0x7fc3a00c9f40 legacy=0x7fc3a015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.997+0000 7fc3a559c700 1 -- 172.21.15.7:0/3654209482 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a01596e0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a559c700 1 --2- 172.21.15.7:0/3654209482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a00c4e30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc3a008d340 tx=0x7fc39800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c90c76e97e93b1b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc39f7fe700 1 -- 172.21.15.7:0/3654209482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800e070 con 0x7fc3a015b4a0 2023-06-09T11:59:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc39f7fe700 1 -- 172.21.15.7:0/3654209482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc39800b9c0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc39f7fe700 1 -- 172.21.15.7:0/3654209482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39800bd70 con 0x7fc3a015b4a0 2023-06-09T11:59:11.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 -- 172.21.15.7:0/3654209482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 msgr2=0x7fc3a00c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/3654209482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a00c4e30 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fc3a008d340 tx=0x7fc39800b330 comp rx=0 tx=0).stop 2023-06-09T11:59:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 -- 172.21.15.7:0/3654209482 shutdown_connections 2023-06-09T11:59:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/3654209482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a00c4e30 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/3654209482 >> v2:172.21.15.117:3300/0 conn(0x7fc3a0153880 0x7fc3a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 -- 172.21.15.7:0/3654209482 >> 172.21.15.7:0/3654209482 conn(0x7fc3a00737f0 msgr2=0x7fc3a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.998+0000 7fc3a6fff700 1 -- 172.21.15.7:0/3654209482 shutdown_connections 2023-06-09T11:59:11.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:10.999+0000 7fc3a6fff700 1 -- 172.21.15.7:0/3654209482 wait complete. 2023-06-09T11:59:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.000+0000 7fc3a6fff700 1 Processor -- start 2023-06-09T11:59:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.000+0000 7fc3a6fff700 1 -- start start 2023-06-09T11:59:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a00c9f40 0x7fc3a00ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 0x7fc3a00cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00ceb40 con 0x7fc3a015b4a0 2023-06-09T11:59:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00cd8e0 con 0x7fc3a00c9f40 2023-06-09T11:59:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a6fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a00cf320 con 0x7fc3a0153880 2023-06-09T11:59:11.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39436/0 (socket says 172.21.15.7:39436) 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.001+0000 7fc3a559c700 1 -- 172.21.15.7:0/4182206392 learned_addr learned my addr 172.21.15.7:0/4182206392 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a4d9b700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a00c9f40 0x7fc3a00ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc39ffff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 0x7fc3a00cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 -- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 msgr2=0x7fc3a00cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 0x7fc3a00cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 -- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a00c9f40 msgr2=0x7fc3a00ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a00c9f40 0x7fc3a00ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 -- 172.21.15.7:0/4182206392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398009b30 con 0x7fc3a015b4a0 2023-06-09T11:59:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc39ffff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 0x7fc3a00cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:11.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a559c700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b730 tx=0x7fc398004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980099b0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3980092e0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a0159c60 con 0x7fc3a015b4a0 2023-06-09T11:59:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.002+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39801b630 con 0x7fc3a015b4a0 2023-06-09T11:59:11.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.003+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a0159980 con 0x7fc3a015b4a0 2023-06-09T11:59:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.003+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0159f40 con 0x7fc3a015b4a0 2023-06-09T11:59:11.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.005+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc39801b790 con 0x7fc3a015b4a0 2023-06-09T11:59:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.006+0000 7fc39d7fa700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 0x7fc38406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.006+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fc39808b6b0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.006+0000 7fc3a4d9b700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 0x7fc38406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.007+0000 7fc3a4d9b700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 0x7fc38406d1c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc394009930 tx=0x7fc394008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:11.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.010+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc398055ea0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.265+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3a0074cf0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.266+0000 7fc39d7fa700 1 -- 172.21.15.7:0/4182206392 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7fc3980594c0 con 0x7fc3a015b4a0 2023-06-09T11:59:11.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:11.269 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 msgr2=0x7fc38406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 0x7fc38406d1c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc394009930 tx=0x7fc394008040 comp rx=0 tx=0).stop 2023-06-09T11:59:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 msgr2=0x7fc3a01e5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b730 tx=0x7fc398004ad0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 shutdown_connections 2023-06-09T11:59:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3a015b4a0 0x7fc3a01e5dd0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3a00c9f40 0x7fc3a00ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc38406ad10 0x7fc38406d1c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 --2- 172.21.15.7:0/4182206392 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc3a0153880 0x7fc3a00cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 >> 172.21.15.7:0/4182206392 conn(0x7fc3a00737f0 msgr2=0x7fc3a014e630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 shutdown_connections 2023-06-09T11:59:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:11.268+0000 7fc3a6fff700 1 -- 172.21.15.7:0/4182206392 wait complete. 2023-06-09T11:59:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:11.290 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:13.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:13.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.497+0000 7f7064fff700 1 Processor -- start 2023-06-09T11:59:13.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.497+0000 7f7064fff700 1 -- start start 2023-06-09T11:59:13.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.498+0000 7f7064fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f70601539b0 0x7f7060159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f7064fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706015a420 con 0x7f70601535e0 2023-06-09T11:59:13.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f7064fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706006b460 con 0x7f70600c4ce0 2023-06-09T11:59:13.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f7064fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600ca670 con 0x7f70601539b0 2023-06-09T11:59:13.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f70601535e0 0x7f70600c9f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33310/0 (socket says 172.21.15.7:33310) 2023-06-09T11:59:13.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705e59c700 1 -- 172.21.15.7:0/1572029159 learned_addr learned my addr 172.21.15.7:0/1572029159 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:13.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f7057fff700 1 --2- 172.21.15.7:0/1572029159 >> v2:172.21.15.117:3300/0 conn(0x7f70601539b0 0x7f7060159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194747163 0 0) 0x7f7050001110 con 0x7f70601535e0 2023-06-09T11:59:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704c0035b0 con 0x7f70601535e0 2023-06-09T11:59:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877263230 0 0) 0x7f706006b460 con 0x7f70600c4ce0 2023-06-09T11:59:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.499+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7050001110 con 0x7f70600c4ce0 2023-06-09T11:59:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3644057514 0 0) 0x7f704c0035b0 con 0x7f70601535e0 2023-06-09T11:59:13.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 >> v2:172.21.15.117:3300/0 conn(0x7f70601539b0 msgr2=0x7f7060159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 --2- 172.21.15.7:0/1572029159 >> v2:172.21.15.117:3300/0 conn(0x7f70601539b0 0x7f7060159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 >> v1:172.21.15.117:6789/0 conn(0x7f70600c4ce0 legacy=0x7f706015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706006a570 con 0x7f70601535e0 2023-06-09T11:59:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f7050001800 con 0x7f70601535e0 2023-06-09T11:59:13.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 >> v1:172.21.15.7:6789/0 conn(0x7f70601535e0 legacy=0x7f70600c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 0x7f704c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c004850 0x7f704c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c005200 0x7f704c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c00c0e0 con 0x7f704c003be0 2023-06-09T11:59:13.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c00c220 con 0x7f704c005200 2023-06-09T11:59:13.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704c00c3b0 con 0x7f704c004850 2023-06-09T11:59:13.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 0x7f704c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f7057fff700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c004850 0x7f704c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705ed9d700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c005200 0x7f704c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 -- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c004850 msgr2=0x7f704c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c004850 0x7f704c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 -- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c005200 msgr2=0x7f704c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c005200 0x7f704c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705e59c700 1 -- 172.21.15.7:0/1572029159 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0035b0 con 0x7f704c003be0 2023-06-09T11:59:13.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f705ed9d700 1 -- 172.21.15.7:0/1572029159 reap_dead start 2023-06-09T11:59:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.500+0000 7f7057fff700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f704c004850 0x7f704c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f705e59c700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 0x7f704c003fd0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f705000c0e0 tx=0x7f704c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb46645da7cba97 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705000cc80 con 0x7f704c003be0 2023-06-09T11:59:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705000cde0 con 0x7f704c003be0 2023-06-09T11:59:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f705dd9b700 1 -- 172.21.15.7:0/1572029159 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050005ba0 con 0x7f704c003be0 2023-06-09T11:59:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f7064fff700 1 -- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 msgr2=0x7f704c003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.501+0000 7f7064fff700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 0x7f704c003fd0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f705000c0e0 tx=0x7f704c005880 comp rx=0 tx=0).stop 2023-06-09T11:59:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.502+0000 7f7064fff700 1 -- 172.21.15.7:0/1572029159 shutdown_connections 2023-06-09T11:59:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.502+0000 7f7064fff700 1 --2- 172.21.15.7:0/1572029159 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704c003be0 0x7f704c003fd0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.502+0000 7f7064fff700 1 -- 172.21.15.7:0/1572029159 >> 172.21.15.7:0/1572029159 conn(0x7f70600737f0 msgr2=0x7f7060072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.502+0000 7f7064fff700 1 -- 172.21.15.7:0/1572029159 shutdown_connections 2023-06-09T11:59:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.502+0000 7f7064fff700 1 -- 172.21.15.7:0/1572029159 wait complete. 2023-06-09T11:59:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.503+0000 7f7064fff700 1 Processor -- start 2023-06-09T11:59:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.504+0000 7f7064fff700 1 -- start start 2023-06-09T11:59:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c003be0 0x7f70600cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70601525a0 0x7f7060152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f3060 con 0x7f70600b31f0 2023-06-09T11:59:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ce420 con 0x7f704c003be0 2023-06-09T11:59:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7064fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600ced40 con 0x7f70601525a0 2023-06-09T11:59:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f705e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c003be0 0x7f70600cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40560/0 (socket says 172.21.15.7:40560) 2023-06-09T11:59:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 -- 172.21.15.7:0/1449490209 learned_addr learned my addr 172.21.15.7:0/1449490209 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f705ed9d700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70601525a0 0x7f7060152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 -- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70601525a0 msgr2=0x7f7060152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70601525a0 0x7f7060152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 -- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c003be0 msgr2=0x7f70600cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c003be0 0x7f70600cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.505+0000 7f7057fff700 1 -- 172.21.15.7:0/1449490209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706006ab30 con 0x7f70600b31f0 2023-06-09T11:59:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7057fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7048011580 tx=0x7f7048011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048004220 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7048004380 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706006a850 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7048011ea0 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.506+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706006ae10 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.507+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7060074cf0 con 0x7f70600b31f0 2023-06-09T11:59:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.508+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f70480036a0 con 0x7f70600b31f0 2023-06-09T11:59:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.509+0000 7f7056ffd700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 0x7f704006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.509+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7f7048089fe0 con 0x7f70600b31f0 2023-06-09T11:59:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.509+0000 7f705e59c700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 0x7f704006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.509+0000 7f705e59c700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 0x7f704006d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7050015040 tx=0x7f7050007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.513+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f70480546d0 con 0x7f70600b31f0 2023-06-09T11:59:13.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.767+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70600c48b0 con 0x7f70600b31f0 2023-06-09T11:59:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.768+0000 7f7056ffd700 1 -- 172.21.15.7:0/1449490209 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f7048022030 con 0x7f70600b31f0 2023-06-09T11:59:13.771 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:13.772 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 msgr2=0x7f704006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 0x7f704006d190 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7050015040 tx=0x7f7050007000 comp rx=0 tx=0).stop 2023-06-09T11:59:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 msgr2=0x7f70601f2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7048011580 tx=0x7f7048011890 comp rx=0 tx=0).stop 2023-06-09T11:59:13.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 shutdown_connections 2023-06-09T11:59:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f704006ace0 0x7f704006d190 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f70601525a0 0x7f7060152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f704c003be0 0x7f70600cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 --2- 172.21.15.7:0/1449490209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70600b31f0 0x7f70601f2b20 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:13.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.770+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 >> 172.21.15.7:0/1449490209 conn(0x7f70600737f0 msgr2=0x7f706014f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.771+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 shutdown_connections 2023-06-09T11:59:13.776 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:13.771+0000 7f7064fff700 1 -- 172.21.15.7:0/1449490209 wait complete. 2023-06-09T11:59:13.777 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:13.793 INFO:tasks.mds_thrash.fs.[cephfs]:mds map: { "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 1, "epoch": 223, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 1, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-06-09T11:10:50.179197+0000", "damaged": [], "data_pools": [ 4 ], "enabled": true, "epoch": 223, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false }, "fs_name": "cephfs", "in": [ 0, 1 ], "info": { "gid_7807": { "addr": "172.21.15.117:6837/3903572761", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6836", "nonce": 3903572761, "type": "v2" }, { "addr": "172.21.15.117:6837", "nonce": 3903572761, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7807, "incarnation": 204, "join_fscid": -1, "name": "f", "rank": 1, "state": "up:active", "state_seq": 37 }, "gid_8056": { "addr": "172.21.15.117:6833/2893966006", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6832", "nonce": 2893966006, "type": "v2" }, { "addr": "172.21.15.117:6833", "nonce": 2893966006, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8056, "incarnation": 219, "join_fscid": -1, "name": "b", "rank": 0, "state": "up:reconnect", "state_seq": 11 } }, "last_failure": 0, "last_failure_osd_epoch": 74, "max_file_size": 1099511627776, "max_mds": 2, "max_xattr_size": 65536, "metadata_pool": 3, "modified": "2023-06-09T11:58:40.804214+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 300, "standby_count_wanted": 0, "stopped": [ 2 ], "tableserver": 0, "up": { "mds_0": 8056, "mds_1": 7807 } } } ], "standbys": [ { "addr": "172.21.15.7:6836/1511608042", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6835", "nonce": 1511608042, "type": "v2" }, { "addr": "172.21.15.7:6836", "nonce": 1511608042, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 2, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4254, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.7:6837/2096062193", "addrs": { "addrvec": [ { "addr": "172.21.15.7:6834", "nonce": 2096062193, "type": "v2" }, { "addr": "172.21.15.7:6837", "nonce": 2096062193, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 3, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4257, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.117:6835/3975520914", "addrs": { "addrvec": [ { "addr": "172.21.15.117:6834", "nonce": 3975520914, "type": "v2" }, { "addr": "172.21.15.117:6835", "nonce": 3975520914, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 209, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7930, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-06-09T11:59:15.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.002+0000 7fe77d1ff700 1 Processor -- start 2023-06-09T11:59:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.002+0000 7fe77d1ff700 1 -- start start 2023-06-09T11:59:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.003+0000 7fe77d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe7780b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.003+0000 7fe77d1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fe7780cb9a0 0x7fe778159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77d1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe778159b20 con 0x7fe77815cd40 2023-06-09T11:59:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77d1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780cac80 con 0x7fe7780cb9a0 2023-06-09T11:59:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781597d0 con 0x7fe7780cb5d0 2023-06-09T11:59:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77759e700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe77815cd40 0x7fe7780cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33326/0 (socket says 172.21.15.7:33326) 2023-06-09T11:59:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77759e700 1 -- 172.21.15.7:0/2018587930 learned_addr learned my addr 172.21.15.7:0/2018587930 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe776d9d700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe7780b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe76ffff700 1 --2- 172.21.15.7:0/2018587930 >> v2:172.21.15.117:3300/0 conn(0x7fe7780cb9a0 0x7fe778159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519230704 0 0) 0x7fe778159b20 con 0x7fe77815cd40 2023-06-09T11:59:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.004+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7640035b0 con 0x7fe77815cd40 2023-06-09T11:59:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608678685 0 0) 0x7fe7640035b0 con 0x7fe77815cd40 2023-06-09T11:59:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 msgr2=0x7fe7780b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe7780b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 >> v2:172.21.15.117:3300/0 conn(0x7fe7780cb9a0 msgr2=0x7fe778159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 --2- 172.21.15.7:0/2018587930 >> v2:172.21.15.117:3300/0 conn(0x7fe7780cb9a0 0x7fe778159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778071ab0 con 0x7fe77815cd40 2023-06-09T11:59:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7fe7700019f0 con 0x7fe77815cd40 2023-06-09T11:59:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 >> v1:172.21.15.7:6789/0 conn(0x7fe77815cd40 legacy=0x7fe7780cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764003a00 0x7fe764003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe764004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe764005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780cac80 con 0x7fe764004d90 2023-06-09T11:59:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781597d0 con 0x7fe764003a00 2023-06-09T11:59:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76400ba10 con 0x7fe7780cb5d0 2023-06-09T11:59:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe77759e700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe764005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe776d9d700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764003a00 0x7fe764003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.005+0000 7fe76ffff700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe764004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 -- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 msgr2=0x7fe764004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7780cb5d0 0x7fe764004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 -- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764003a00 msgr2=0x7fe764003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe764003a00 0x7fe764003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 -- 172.21.15.7:0/2018587930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640035b0 con 0x7fe764004d90 2023-06-09T11:59:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77759e700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe764005200 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe77815d140 tx=0x7fe764005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231cf26176669c00 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700072d0 con 0x7fe764004d90 2023-06-09T11:59:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770007430 con 0x7fe764004d90 2023-06-09T11:59:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.006+0000 7fe77659c700 1 -- 172.21.15.7:0/2018587930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700077e0 con 0x7fe764004d90 2023-06-09T11:59:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 -- 172.21.15.7:0/2018587930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 msgr2=0x7fe764005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/2018587930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe764005200 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fe77815d140 tx=0x7fe764005880 comp rx=0 tx=0).stop 2023-06-09T11:59:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77759e700 1 -- 172.21.15.7:0/2018587930 reap_dead start 2023-06-09T11:59:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 -- 172.21.15.7:0/2018587930 shutdown_connections 2023-06-09T11:59:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 -- 172.21.15.7:0/2018587930 >> 172.21.15.7:0/2018587930 conn(0x7fe778070af0 msgr2=0x7fe778070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 -- 172.21.15.7:0/2018587930 shutdown_connections 2023-06-09T11:59:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.007+0000 7fe77d1ff700 1 -- 172.21.15.7:0/2018587930 wait complete. 2023-06-09T11:59:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.009+0000 7fe77d1ff700 1 Processor -- start 2023-06-09T11:59:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.009+0000 7fe77d1ff700 1 -- start start 2023-06-09T11:59:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7781e61b0 0x7fe7781e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 0x7fe7781e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778153990 con 0x7fe764004d90 2023-06-09T11:59:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781f2e60 con 0x7fe7781e6600 2023-06-09T11:59:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77d1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77815a9d0 con 0x7fe7781e61b0 2023-06-09T11:59:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40580/0 (socket says 172.21.15.7:40580) 2023-06-09T11:59:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 -- 172.21.15.7:0/1536405444 learned_addr learned my addr 172.21.15.7:0/1536405444 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe76ffff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7781e61b0 0x7fe7781e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77759e700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 0x7fe7781e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 -- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7781e61b0 msgr2=0x7fe7781e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7781e61b0 0x7fe7781e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 -- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 msgr2=0x7fe7781e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 0x7fe7781e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe776d9d700 1 -- 172.21.15.7:0/1536405444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77815d4f0 con 0x7fe764004d90 2023-06-09T11:59:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.011+0000 7fe77759e700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 0x7fe7781e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe776d9d700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fe768011aa0 tx=0x7fe768011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800f860 con 0x7fe764004d90 2023-06-09T11:59:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800f9c0 con 0x7fe764004d90 2023-06-09T11:59:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800fd70 con 0x7fe764004d90 2023-06-09T11:59:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77815d210 con 0x7fe764004d90 2023-06-09T11:59:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.012+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77815d7d0 con 0x7fe764004d90 2023-06-09T11:59:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.013+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778074cf0 con 0x7fe764004d90 2023-06-09T11:59:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.014+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe768014440 con 0x7fe764004d90 2023-06-09T11:59:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.015+0000 7fe76f7fe700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 0x7fe75806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.015+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 8270+0+0 (secure 0 0 0) 0x7fe76808a470 con 0x7fe764004d90 2023-06-09T11:59:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.015+0000 7fe76ffff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 0x7fe75806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.016+0000 7fe76ffff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 0x7fe75806d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe7780712f0 tx=0x7fe760009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.020+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fe768054d20 con 0x7fe764004d90 2023-06-09T11:59:16.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.271+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7780caa60 con 0x7fe764004d90 2023-06-09T11:59:16.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.272+0000 7fe76f7fe700 1 -- 172.21.15.7:0/1536405444 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7fe768022090 con 0x7fe764004d90 2023-06-09T11:59:16.275 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:16.276 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:16.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.274+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 msgr2=0x7fe75806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.274+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 0x7fe75806d190 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe7780712f0 tx=0x7fe760009210 comp rx=0 tx=0).stop 2023-06-09T11:59:16.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.274+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 msgr2=0x7fe7781f2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.274+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fe768011aa0 tx=0x7fe768011e60 comp rx=0 tx=0).stop 2023-06-09T11:59:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 shutdown_connections 2023-06-09T11:59:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe7781e6600 0x7fe7781e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe764004d90 0x7fe7781f2920 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe75806ace0 0x7fe75806d190 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 --2- 172.21.15.7:0/1536405444 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe7781e61b0 0x7fe7781e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:16.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 >> 172.21.15.7:0/1536405444 conn(0x7fe778070af0 msgr2=0x7fe7780ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:16.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 shutdown_connections 2023-06-09T11:59:16.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:16.275+0000 7fe77d1ff700 1 -- 172.21.15.7:0/1536405444 wait complete. 2023-06-09T11:59:16.281 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:16.296 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:18.297 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.504+0000 7f64d03ff700 1 Processor -- start 2023-06-09T11:59:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.505+0000 7f64d03ff700 1 -- start start 2023-06-09T11:59:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.506+0000 7f64d03ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64c80c5d20 0x7f64c815e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.506+0000 7f64d03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c81579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.506+0000 7f64d03ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c80c0810 con 0x7f64c815e880 2023-06-09T11:59:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.506+0000 7f64d03ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c815db60 con 0x7f64c80c5d20 2023-06-09T11:59:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.506+0000 7f64d03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c81615a0 con 0x7f64c81610d0 2023-06-09T11:59:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce19b700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f64c80c5d20 0x7f64c815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64cd99a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c81579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64cd99a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c81579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:53580/0 (socket says 172.21.15.7:53580) 2023-06-09T11:59:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64cd99a700 1 -- 172.21.15.7:0/2470476103 learned_addr learned my addr 172.21.15.7:0/2470476103 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce99c700 1 --1- 172.21.15.7:0/2470476103 >> v1:172.21.15.117:6789/0 conn(0x7f64c815e880 0x7f64c80c7bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:54392/0 (socket says 172.21.15.7:54392) 2023-06-09T11:59:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce19b700 1 -- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 msgr2=0x7f64c81579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce19b700 1 --2- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c81579b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce19b700 1 -- 172.21.15.7:0/2470476103 >> v1:172.21.15.117:6789/0 conn(0x7f64c815e880 legacy=0x7f64c80c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64ce19b700 1 -- 172.21.15.7:0/2470476103 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8161880 con 0x7f64c80c5d20 2023-06-09T11:59:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.507+0000 7f64cd199700 1 -- 172.21.15.7:0/2470476103 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116513947 0 0) 0x7f64c80c0810 con 0x7f64c815e880 2023-06-09T11:59:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64ce19b700 1 --2- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c815e340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009840 tx=0x7f64c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3467c37641c9eb server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64cd199700 1 -- 172.21.15.7:0/2470476103 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011830 con 0x7f64c80c5d20 2023-06-09T11:59:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64cd199700 1 -- 172.21.15.7:0/2470476103 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4011990 con 0x7f64c80c5d20 2023-06-09T11:59:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64cd199700 1 -- 172.21.15.7:0/2470476103 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011d20 con 0x7f64c80c5d20 2023-06-09T11:59:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64d03ff700 1 -- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 msgr2=0x7f64c815e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.508+0000 7f64d03ff700 1 --2- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c815e340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009840 tx=0x7f64c4009b50 comp rx=0 tx=0).stop 2023-06-09T11:59:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 -- 172.21.15.7:0/2470476103 shutdown_connections 2023-06-09T11:59:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 --2- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c815e340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 --2- 172.21.15.7:0/2470476103 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c81579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 -- 172.21.15.7:0/2470476103 >> 172.21.15.7:0/2470476103 conn(0x7f64c806a4d0 msgr2=0x7f64c806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 -- 172.21.15.7:0/2470476103 shutdown_connections 2023-06-09T11:59:18.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.509+0000 7f64d03ff700 1 -- 172.21.15.7:0/2470476103 wait complete. 2023-06-09T11:59:18.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.511+0000 7f64d03ff700 1 Processor -- start 2023-06-09T11:59:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.511+0000 7f64d03ff700 1 -- start start 2023-06-09T11:59:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.512+0000 7f64d03ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.512+0000 7f64d03ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64d03ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c8072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64d03ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c8158a80 con 0x7f64c815e880 2023-06-09T11:59:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64d03ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80cac70 con 0x7f64c80c5d20 2023-06-09T11:59:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64d03ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c80c8fa0 con 0x7f64c81610d0 2023-06-09T11:59:18.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40588/0 (socket says 172.21.15.7:40588) 2023-06-09T11:59:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 -- 172.21.15.7:0/3158490069 learned_addr learned my addr 172.21.15.7:0/3158490069 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64ce19b700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64ce99c700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c8072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 -- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 msgr2=0x7f64c8072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c8072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 -- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 msgr2=0x7f64c8158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c8158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.513+0000 7f64cd99a700 1 -- 172.21.15.7:0/3158490069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40094a0 con 0x7f64c815e880 2023-06-09T11:59:18.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64cd99a700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f64b800e8e0 tx=0x7f64b800ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8009600 con 0x7f64c815e880 2023-06-09T11:59:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8013680 con 0x7f64c815e880 2023-06-09T11:59:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8013a30 con 0x7f64c815e880 2023-06-09T11:59:18.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c8161e40 con 0x7f64c815e880 2023-06-09T11:59:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.514+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c8161b60 con 0x7f64c815e880 2023-06-09T11:59:18.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.515+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c815de20 con 0x7f64c815e880 2023-06-09T11:59:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.516+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f64b8013b90 con 0x7f64c815e880 2023-06-09T11:59:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.517+0000 7f64beffd700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 0x7f64ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:18.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.517+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f64b8089fa0 con 0x7f64c815e880 2023-06-09T11:59:18.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.517+0000 7f64ce19b700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 0x7f64ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:18.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.518+0000 7f64ce19b700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 0x7f64ac06d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f64c40090b0 tx=0x7f64c4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:18.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.521+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f64b80546e0 con 0x7f64c815e880 2023-06-09T11:59:18.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.777+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64c804e8c0 con 0x7f64c815e880 2023-06-09T11:59:18.781 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.778+0000 7f64beffd700 1 -- 172.21.15.7:0/3158490069 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 223 v223) v1 ==== 95+0+5456 (secure 0 0 0) 0x7f64b8057d00 con 0x7f64c815e880 2023-06-09T11:59:18.781 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:18.782 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":223,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":223,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:58:40.804214+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:reconnect","state_seq":11,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 msgr2=0x7f64ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 0x7f64ac06d190 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f64c40090b0 tx=0x7f64c4008030 comp rx=0 tx=0).stop 2023-06-09T11:59:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 msgr2=0x7f64c80ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f64b800e8e0 tx=0x7f64b800ebf0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 shutdown_connections 2023-06-09T11:59:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f64ac06ace0 0x7f64ac06d190 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f64c81610d0 0x7f64c8072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f64c80c5d20 0x7f64c8158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 --2- 172.21.15.7:0/3158490069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64c815e880 0x7f64c80ca730 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 >> 172.21.15.7:0/3158490069 conn(0x7f64c806a4d0 msgr2=0x7f64c80c7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 shutdown_connections 2023-06-09T11:59:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:18.780+0000 7f64d03ff700 1 -- 172.21.15.7:0/3158490069 wait complete. 2023-06-09T11:59:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 223 2023-06-09T11:59:18.802 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:20.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:21.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.010+0000 7fd007fff700 1 Processor -- start 2023-06-09T11:59:21.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.010+0000 7fd007fff700 1 -- start start 2023-06-09T11:59:21.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0000c1bc0 0x7fd0001599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0000c0140 0x7fd000159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c1f90 0x7fd0001501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd000070110 con 0x7fd0000c0140 2023-06-09T11:59:21.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd000071300 con 0x7fd0000c1bc0 2023-06-09T11:59:21.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd007fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000cbd20 con 0x7fd0000c1f90 2023-06-09T11:59:21.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0000c0140 0x7fd000159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd0000c0140 0x7fd000159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40590/0 (socket says 172.21.15.7:40590) 2023-06-09T11:59:21.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 -- 172.21.15.7:0/37039717 learned_addr learned my addr 172.21.15.7:0/37039717 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:21.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00659c700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c1f90 0x7fd0001501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd005d9b700 1 --2- 172.21.15.7:0/37039717 >> v2:172.21.15.117:3300/0 conn(0x7fd0000c1bc0 0x7fd0001599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 -- 172.21.15.7:0/37039717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c1f90 msgr2=0x7fd0001501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c1f90 0x7fd0001501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 -- 172.21.15.7:0/37039717 >> v2:172.21.15.117:3300/0 conn(0x7fd0000c1bc0 msgr2=0x7fd0001599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 --2- 172.21.15.7:0/37039717 >> v2:172.21.15.117:3300/0 conn(0x7fd0000c1bc0 0x7fd0001599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.012+0000 7fd00559a700 1 -- 172.21.15.7:0/37039717 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0000c0bf0 con 0x7fd0000c0140 2023-06-09T11:59:21.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.013+0000 7fd00559a700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c0140 0x7fd000159ef0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd0000cc0c0 tx=0x7fcff0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08ddb33002f279f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:21.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.013+0000 7fd004d99700 1 -- 172.21.15.7:0/37039717 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000b860 con 0x7fd0000c0140 2023-06-09T11:59:21.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.013+0000 7fd004d99700 1 -- 172.21.15.7:0/37039717 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000b9c0 con 0x7fd0000c0140 2023-06-09T11:59:21.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.013+0000 7fd004d99700 1 -- 172.21.15.7:0/37039717 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000bd50 con 0x7fd0000c0140 2023-06-09T11:59:21.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 -- 172.21.15.7:0/37039717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c0140 msgr2=0x7fd000159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:21.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c0140 0x7fd000159ef0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fd0000cc0c0 tx=0x7fcff0009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 -- 172.21.15.7:0/37039717 shutdown_connections 2023-06-09T11:59:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 --2- 172.21.15.7:0/37039717 >> v2:172.21.15.117:3300/0 conn(0x7fd0000c1bc0 0x7fd0001599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c1f90 0x7fd0001501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 --2- 172.21.15.7:0/37039717 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c0140 0x7fd000159ef0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 -- 172.21.15.7:0/37039717 >> 172.21.15.7:0/37039717 conn(0x7fd00006a4d0 msgr2=0x7fd00006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 -- 172.21.15.7:0/37039717 shutdown_connections 2023-06-09T11:59:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.014+0000 7fd007fff700 1 -- 172.21.15.7:0/37039717 wait complete. 2023-06-09T11:59:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.016+0000 7fd007fff700 1 Processor -- start 2023-06-09T11:59:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.016+0000 7fd007fff700 1 -- start start 2023-06-09T11:59:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 0x7fd0000c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 0x7fd0000c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd000156690 con 0x7fd0000c1f90 2023-06-09T11:59:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0000c2b80 con 0x7fd0000c1bc0 2023-06-09T11:59:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd007fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00014d450 con 0x7fd0000c0140 2023-06-09T11:59:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40606/0 (socket says 172.21.15.7:40606) 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 -- 172.21.15.7:0/1689183170 learned_addr learned my addr 172.21.15.7:0/1689183170 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd005d9b700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 0x7fd0000c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00559a700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 0x7fd0000c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 -- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 msgr2=0x7fd0000c2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 0x7fd0000c2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 -- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 msgr2=0x7fd0000c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 0x7fd0000c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00659c700 1 -- 172.21.15.7:0/1689183170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0009550 con 0x7fd0000c1f90 2023-06-09T11:59:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd005d9b700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 0x7fd0000c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.018+0000 7fd00559a700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 0x7fd0000c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fd00659c700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcff8007e90 tx=0x7fcff800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff800ede0 con 0x7fd0000c1f90 2023-06-09T11:59:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8014420 con 0x7fd0000c1f90 2023-06-09T11:59:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0000c0e90 con 0x7fd0000c1f90 2023-06-09T11:59:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80147d0 con 0x7fd0000c1f90 2023-06-09T11:59:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.019+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0000702f0 con 0x7fd0000c1f90 2023-06-09T11:59:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.020+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0000c1170 con 0x7fd0000c1f90 2023-06-09T11:59:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.021+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcff8014930 con 0x7fd0000c1f90 2023-06-09T11:59:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.022+0000 7fcff67fc700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.022+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fcff808a540 con 0x7fd0000c1f90 2023-06-09T11:59:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.022+0000 7fd005d9b700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:21.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.023+0000 7fd005d9b700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 0x7fcfe406d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009970 tx=0x7fcffc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:21.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.027+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fcff8054c70 con 0x7fd0000c1f90 2023-06-09T11:59:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.277+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0001563f0 con 0x7fd0000c1f90 2023-06-09T11:59:21.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.278+0000 7fcff67fc700 1 -- 172.21.15.7:0/1689183170 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 224 v224) v1 ==== 95+0+5453 (secure 0 0 0) 0x7fcff8058290 con 0x7fd0000c1f90 2023-06-09T11:59:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:21.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":224,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":224,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:59:18.807113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 msgr2=0x7fcfe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:21.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 0x7fcfe406d190 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fcffc009970 tx=0x7fcffc008040 comp rx=0 tx=0).stop 2023-06-09T11:59:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 msgr2=0x7fd000147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fcff8007e90 tx=0x7fcff800e3c0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 shutdown_connections 2023-06-09T11:59:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0000c1f90 0x7fd000147f00 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fcfe406ace0 0x7fcfe406d190 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0000c0140 0x7fd0000c2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 --2- 172.21.15.7:0/1689183170 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0000c1bc0 0x7fd0000c2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:21.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 >> 172.21.15.7:0/1689183170 conn(0x7fd00006a4d0 msgr2=0x7fd00014ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:21.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 shutdown_connections 2023-06-09T11:59:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:21.280+0000 7fd007fff700 1 -- 172.21.15.7:0/1689183170 wait complete. 2023-06-09T11:59:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 224 2023-06-09T11:59:21.301 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:23.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.509+0000 7f4ba65ff700 1 Processor -- start 2023-06-09T11:59:23.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.509+0000 7f4ba65ff700 1 -- start start 2023-06-09T11:59:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.510+0000 7f4ba65ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba00b5360 0x7f4ba015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.510+0000 7f4ba65ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 0x7f4ba00c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.510+0000 7f4ba65ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba015dcf0 con 0x7f4ba00ca1b0 2023-06-09T11:59:23.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.510+0000 7f4ba65ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba00c5e00 con 0x7f4ba00b5360 2023-06-09T11:59:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.510+0000 7f4ba65ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba015d450 con 0x7f4ba00ca580 2023-06-09T11:59:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba00b5360 0x7f4ba015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4ba00b5360 0x7f4ba015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53668/0 (socket says 172.21.15.7:53668) 2023-06-09T11:59:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 -- 172.21.15.7:0/321107747 learned_addr learned my addr 172.21.15.7:0/321107747 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:23.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9ffff700 1 --1- 172.21.15.7:0/321107747 >> v1:172.21.15.7:6789/0 conn(0x7f4ba00ca1b0 0x7f4ba01536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58592/0 (socket says 172.21.15.7:58592) 2023-06-09T11:59:23.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4ba4b9c700 1 --2- 172.21.15.7:0/321107747 >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 0x7f4ba00c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 -- 172.21.15.7:0/321107747 >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 msgr2=0x7f4ba00c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 --2- 172.21.15.7:0/321107747 >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 0x7f4ba00c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 -- 172.21.15.7:0/321107747 >> v1:172.21.15.7:6789/0 conn(0x7f4ba00ca1b0 legacy=0x7f4ba01536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 -- 172.21.15.7:0/321107747 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba015d730 con 0x7f4ba00b5360 2023-06-09T11:59:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4ba4b9c700 1 --2- 172.21.15.7:0/321107747 >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 0x7f4ba00c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.511+0000 7f4b9f7fe700 1 --2- 172.21.15.7:0/321107747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00b5360 0x7f4ba015cdc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4ba006bb50 tx=0x7f4b94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb702073ef08812b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4b9effd700 1 -- 172.21.15.7:0/321107747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400b870 con 0x7f4ba00b5360 2023-06-09T11:59:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4b9effd700 1 -- 172.21.15.7:0/321107747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9400b9d0 con 0x7f4ba00b5360 2023-06-09T11:59:23.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4b9effd700 1 -- 172.21.15.7:0/321107747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9400bd60 con 0x7f4ba00b5360 2023-06-09T11:59:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 -- 172.21.15.7:0/321107747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00b5360 msgr2=0x7f4ba015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/321107747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00b5360 0x7f4ba015cdc0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f4ba006bb50 tx=0x7f4b94009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 -- 172.21.15.7:0/321107747 shutdown_connections 2023-06-09T11:59:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/321107747 >> v2:172.21.15.117:3300/0 conn(0x7f4ba00ca580 0x7f4ba00c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/321107747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00b5360 0x7f4ba015cdc0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 -- 172.21.15.7:0/321107747 >> 172.21.15.7:0/321107747 conn(0x7f4ba0072090 msgr2=0x7f4ba0072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 -- 172.21.15.7:0/321107747 shutdown_connections 2023-06-09T11:59:23.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.512+0000 7f4ba65ff700 1 -- 172.21.15.7:0/321107747 wait complete. 2023-06-09T11:59:23.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.515+0000 7f4ba65ff700 1 Processor -- start 2023-06-09T11:59:23.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.515+0000 7f4ba65ff700 1 -- start start 2023-06-09T11:59:23.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba00b5360 0x7f4ba01f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba00ca1b0 0x7f4ba00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00ce500 con 0x7f4ba00ca580 2023-06-09T11:59:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba01f3340 con 0x7f4ba00ca1b0 2023-06-09T11:59:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba65ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba00cd5b0 con 0x7f4ba00b5360 2023-06-09T11:59:23.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba4b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba4b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53672/0 (socket says 172.21.15.7:53672) 2023-06-09T11:59:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4ba4b9c700 1 -- 172.21.15.7:0/4027929625 learned_addr learned my addr 172.21.15.7:0/4027929625 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.516+0000 7f4b9ffff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba00b5360 0x7f4ba01f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4b9f7fe700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba00ca1b0 0x7f4ba00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 -- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba00b5360 msgr2=0x7f4ba01f2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba00b5360 0x7f4ba01f2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 -- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba00ca1b0 msgr2=0x7f4ba00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba00ca1b0 0x7f4ba00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 -- 172.21.15.7:0/4027929625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94009550 con 0x7f4ba00ca580 2023-06-09T11:59:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba4b9c700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800cd50 tx=0x7f4b9800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800cea0 con 0x7f4ba00ca580 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800f480 con 0x7f4ba00ca580 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba006a570 con 0x7f4ba00ca580 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.517+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800f830 con 0x7f4ba00ca580 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.518+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba015da10 con 0x7f4ba00ca580 2023-06-09T11:59:23.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.519+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba006a850 con 0x7f4ba00ca580 2023-06-09T11:59:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.520+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b9800b040 con 0x7f4ba00ca580 2023-06-09T11:59:23.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.520+0000 7f4b9cff9700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 0x7f4b8c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:23.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.520+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f4b9808a5b0 con 0x7f4ba00ca580 2023-06-09T11:59:23.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.521+0000 7f4b9ffff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 0x7f4b8c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:23.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.521+0000 7f4b9ffff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 0x7f4b8c06d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f60 tx=0x7f4b90007ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:23.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.525+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4b98054c50 con 0x7f4ba00ca580 2023-06-09T11:59:23.784 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.781+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ba00ce6b0 con 0x7f4ba00ca580 2023-06-09T11:59:23.785 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.781+0000 7f4b9cff9700 1 -- 172.21.15.7:0/4027929625 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 224 v224) v1 ==== 95+0+5453 (secure 0 0 0) 0x7f4ba00ce6b0 con 0x7f4ba00ca580 2023-06-09T11:59:23.785 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:23.786 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":224,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":224,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:59:18.807113+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:rejoin","state_seq":24,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 msgr2=0x7f4b8c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 0x7f4b8c06d1c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4b90000f60 tx=0x7f4b90007ed0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 msgr2=0x7f4ba00ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800cd50 tx=0x7f4b9800a560 comp rx=0 tx=0).stop 2023-06-09T11:59:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 shutdown_connections 2023-06-09T11:59:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ba00ca580 0x7f4ba00ceed0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4b8c06ad10 0x7f4b8c06d1c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4ba00b5360 0x7f4ba01f2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 --2- 172.21.15.7:0/4027929625 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4ba00ca1b0 0x7f4ba00ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 >> 172.21.15.7:0/4027929625 conn(0x7f4ba0072090 msgr2=0x7f4ba01517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 shutdown_connections 2023-06-09T11:59:23.789 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:23.784+0000 7f4ba65ff700 1 -- 172.21.15.7:0/4027929625 wait complete. 2023-06-09T11:59:23.790 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 224 2023-06-09T11:59:23.806 INFO:tasks.mds_thrash.fs.[cephfs]:no change 2023-06-09T11:59:25.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:25.483 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:25.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:26.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.010+0000 7f297a5ff700 1 Processor -- start 2023-06-09T11:59:26.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.010+0000 7f297a5ff700 1 -- start start 2023-06-09T11:59:26.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29740c1bc0 0x7f29741599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f2974159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 0x7f29741501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2974070110 con 0x7f29740c1bc0 2023-06-09T11:59:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2974071300 con 0x7f29740c1f90 2023-06-09T11:59:26.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f297a5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29740cbd20 con 0x7f29740c0140 2023-06-09T11:59:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f2973fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29740c1bc0 0x7f29741599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f2973fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29740c1bc0 0x7f29741599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53682/0 (socket says 172.21.15.7:53682) 2023-06-09T11:59:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f2973fff700 1 -- 172.21.15.7:0/1944182864 learned_addr learned my addr 172.21.15.7:0/1944182864 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f29737fe700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f2974159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.012+0000 7f2978b9c700 1 --2- 172.21.15.7:0/1944182864 >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 0x7f29741501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 -- 172.21.15.7:0/1944182864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 msgr2=0x7f2974159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f2974159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 -- 172.21.15.7:0/1944182864 >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 msgr2=0x7f29741501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:26.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 --2- 172.21.15.7:0/1944182864 >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 0x7f29741501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 -- 172.21.15.7:0/1944182864 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29740c0bf0 con 0x7f29740c1bc0 2023-06-09T11:59:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2978b9c700 1 --2- 172.21.15.7:0/1944182864 >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 0x7f29741501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2973fff700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1bc0 0x7f29741599b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f297406b700 tx=0x7f2964009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f875a718068ed5c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2972ffd700 1 -- 172.21.15.7:0/1944182864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964011830 con 0x7f29740c1bc0 2023-06-09T11:59:26.019 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2972ffd700 1 -- 172.21.15.7:0/1944182864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2964011990 con 0x7f29740c1bc0 2023-06-09T11:59:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.013+0000 7f2972ffd700 1 -- 172.21.15.7:0/1944182864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2964011d20 con 0x7f29740c1bc0 2023-06-09T11:59:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 -- 172.21.15.7:0/1944182864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1bc0 msgr2=0x7f29741599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1bc0 0x7f29741599b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f297406b700 tx=0x7f2964009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 -- 172.21.15.7:0/1944182864 shutdown_connections 2023-06-09T11:59:26.020 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1944182864 >> v2:172.21.15.117:3300/0 conn(0x7f29740c1f90 0x7f29741501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1bc0 0x7f29741599b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1944182864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f2974159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 -- 172.21.15.7:0/1944182864 >> 172.21.15.7:0/1944182864 conn(0x7f297406a4d0 msgr2=0x7f297406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 -- 172.21.15.7:0/1944182864 shutdown_connections 2023-06-09T11:59:26.021 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.014+0000 7f297a5ff700 1 -- 172.21.15.7:0/1944182864 wait complete. 2023-06-09T11:59:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.015+0000 7f297a5ff700 1 Processor -- start 2023-06-09T11:59:26.022 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.016+0000 7f297a5ff700 1 -- start start 2023-06-09T11:59:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29740c1bc0 0x7f29740c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 0x7f2974148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974158df0 con 0x7f29740c1f90 2023-06-09T11:59:26.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297414f020 con 0x7f29740c1bc0 2023-06-09T11:59:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f297a5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29741595a0 con 0x7f29740c0140 2023-06-09T11:59:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 0x7f2974148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2973fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2973fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46894/0 (socket says 172.21.15.7:46894) 2023-06-09T11:59:26.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2973fff700 1 -- 172.21.15.7:0/1771997287 learned_addr learned my addr 172.21.15.7:0/1771997287 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f29737fe700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29740c1bc0 0x7f29740c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 -- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 msgr2=0x7f29740c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 -- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29740c1bc0 msgr2=0x7f29740c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:26.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29740c1bc0 0x7f29740c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2978b9c700 1 -- 172.21.15.7:0/1771997287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2964009550 con 0x7f29740c1f90 2023-06-09T11:59:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.017+0000 7f2973fff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f2978b9c700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 0x7f2974148b10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f296c007d40 tx=0x7f296c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c00dd20 con 0x7f29740c1f90 2023-06-09T11:59:26.027 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c00f040 con 0x7f29740c1f90 2023-06-09T11:59:26.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29740c0e90 con 0x7f29740c1f90 2023-06-09T11:59:26.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c003c60 con 0x7f29740c1f90 2023-06-09T11:59:26.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.018+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29740702f0 con 0x7f29740c1f90 2023-06-09T11:59:26.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.019+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29740c1170 con 0x7f29740c1f90 2023-06-09T11:59:26.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.020+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f296c004030 con 0x7f29740c1f90 2023-06-09T11:59:26.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.021+0000 7f2970ff9700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 0x7f296006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:26.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.021+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f296c089dd0 con 0x7f29740c1f90 2023-06-09T11:59:26.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.021+0000 7f2973fff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 0x7f296006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:26.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.022+0000 7f2973fff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 0x7f296006d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2964009130 tx=0x7f2964008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:26.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.026+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f296c054470 con 0x7f29740c1f90 2023-06-09T11:59:26.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.278+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29740c3d90 con 0x7f29740c1f90 2023-06-09T11:59:26.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.279+0000 7f2970ff9700 1 -- 172.21.15.7:0/1771997287 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) v1 ==== 95+0+5453 (secure 0 0 0) 0x7f296c01a070 con 0x7f29740c1f90 2023-06-09T11:59:26.282 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:26.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":225,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:59:23.977988+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 msgr2=0x7f296006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 0x7f296006d190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2964009130 tx=0x7f2964008030 comp rx=0 tx=0).stop 2023-06-09T11:59:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 msgr2=0x7f2974148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:26.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 0x7f2974148b10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f296c007d40 tx=0x7f296c00d3b0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 shutdown_connections 2023-06-09T11:59:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29740c1f90 0x7f2974148b10 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f296006ace0 0x7f296006d190 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29740c0140 0x7f29740c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 --2- 172.21.15.7:0/1771997287 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29740c1bc0 0x7f29740c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.281+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 >> 172.21.15.7:0/1771997287 conn(0x7f297406a4d0 msgr2=0x7f29740c85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.282+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 shutdown_connections 2023-06-09T11:59:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:26.282+0000 7f297a5ff700 1 -- 172.21.15.7:0/1771997287 wait complete. 2023-06-09T11:59:26.286 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 225 2023-06-09T11:59:26.303 INFO:tasks.mds_thrash.fs.[cephfs]:mds.b has gained rank=0, replacing gid=7437 2023-06-09T11:59:26.303 INFO:tasks.mds_thrash.fs.[cephfs]:waiting for 5 secs before reviving mds.e 2023-06-09T11:59:31.304 INFO:tasks.mds_thrash.fs.[cephfs]:reviving mds.e 2023-06-09T11:59:31.304 INFO:tasks.ceph.mds.e:Restarting daemon 2023-06-09T11:59:31.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2023-06-09T11:59:31.309 INFO:tasks.ceph.mds.e:Started 2023-06-09T11:59:31.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:31.489 INFO:tasks.ceph.mds.e.smithi007.stdout:starting mds.e at 2023-06-09T11:59:31.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.523+0000 7f7ea0bff700 1 Processor -- start 2023-06-09T11:59:31.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.523+0000 7f7ea0bff700 1 -- start start 2023-06-09T11:59:31.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7ea0bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 0x7f7e9c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7ea0bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7ea0bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c15a420 con 0x7f7e9c1535e0 2023-06-09T11:59:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7ea0bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0ca670 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7ea0bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0c5a00 con 0x7f7e9c1539b0 2023-06-09T11:59:31.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7e9ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7e9ad9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53726/0 (socket says 172.21.15.7:53726) 2023-06-09T11:59:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.525+0000 7f7e9ad9d700 1 -- 172.21.15.7:0/1163925744 learned_addr learned my addr 172.21.15.7:0/1163925744 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e99d9b700 1 --2- 172.21.15.7:0/1163925744 >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 0x7f7e9c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9ad9d700 1 -- 172.21.15.7:0/1163925744 >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 msgr2=0x7f7e9c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9ad9d700 1 --2- 172.21.15.7:0/1163925744 >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 0x7f7e9c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9ad9d700 1 -- 172.21.15.7:0/1163925744 >> v1:172.21.15.117:6789/0 conn(0x7f7e9c1535e0 legacy=0x7f7e9c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9ad9d700 1 -- 172.21.15.7:0/1163925744 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c06b460 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e99d9b700 1 --2- 172.21.15.7:0/1163925744 >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 0x7f7e9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:31.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9ad9d700 1 --2- 172.21.15.7:0/1163925744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c08d1e0 tx=0x7f7e9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f327efdcb94d3870 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:31.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9959a700 1 -- 172.21.15.7:0/1163925744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400e070 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9959a700 1 -- 172.21.15.7:0/1163925744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400b9c0 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.526+0000 7f7e9959a700 1 -- 172.21.15.7:0/1163925744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400bd70 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 -- 172.21.15.7:0/1163925744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c0c4ce0 msgr2=0x7f7e9c0c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/1163925744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c08d1e0 tx=0x7f7e9400b330 comp rx=0 tx=0).stop 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 -- 172.21.15.7:0/1163925744 shutdown_connections 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/1163925744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0c5140 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/1163925744 >> v2:172.21.15.117:3300/0 conn(0x7f7e9c1539b0 0x7f7e9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 -- 172.21.15.7:0/1163925744 >> 172.21.15.7:0/1163925744 conn(0x7f7e9c0737f0 msgr2=0x7f7e9c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 -- 172.21.15.7:0/1163925744 shutdown_connections 2023-06-09T11:59:31.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.527+0000 7f7ea0bff700 1 -- 172.21.15.7:0/1163925744 wait complete. 2023-06-09T11:59:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.529+0000 7f7ea0bff700 1 Processor -- start 2023-06-09T11:59:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.529+0000 7f7ea0bff700 1 -- start start 2023-06-09T11:59:31.535 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e9c1539b0 0x7f7e9c1e1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c14ebe0 con 0x7f7e9c1535e0 2023-06-09T11:59:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1e1600 con 0x7f7e9c1539b0 2023-06-09T11:59:31.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7ea0bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c14e620 con 0x7f7e9c0c4ce0 2023-06-09T11:59:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7e99d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7e99d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53730/0 (socket says 172.21.15.7:53730) 2023-06-09T11:59:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7e99d9b700 1 -- 172.21.15.7:0/3305831599 learned_addr learned my addr 172.21.15.7:0/3305831599 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7e9ad9d700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e9c1539b0 0x7f7e9c1e1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.530+0000 7f7e9a59c700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 -- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e9c0c4ce0 msgr2=0x7f7e9c0b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 -- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e9c1539b0 msgr2=0x7f7e9c1e1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e9c1539b0 0x7f7e9c1e1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 -- 172.21.15.7:0/3305831599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e94009b30 con 0x7f7e9c1535e0 2023-06-09T11:59:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e99d9b700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000e7e0 tx=0x7f7e9000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000c030 con 0x7f7e9c1535e0 2023-06-09T11:59:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90004220 con 0x7f7e9c1535e0 2023-06-09T11:59:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c06a850 con 0x7f7e9c1535e0 2023-06-09T11:59:31.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e90004590 con 0x7f7e9c1535e0 2023-06-09T11:59:31.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.531+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c06a570 con 0x7f7e9c1535e0 2023-06-09T11:59:31.540 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.532+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c06ab30 con 0x7f7e9c1535e0 2023-06-09T11:59:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.533+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e900036a0 con 0x7f7e9c1535e0 2023-06-09T11:59:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.534+0000 7f7e8affd700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 0x7f7e7c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.534+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f7e90089a10 con 0x7f7e9c1535e0 2023-06-09T11:59:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.535+0000 7f7e9a59c700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 0x7f7e7c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:31.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.535+0000 7f7e9a59c700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 0x7f7e7c06d1c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c007fd0 tx=0x7f7e8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:31.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.539+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7e90054150 con 0x7f7e9c1535e0 2023-06-09T11:59:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.789+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c1e1770 con 0x7f7e9c1535e0 2023-06-09T11:59:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.790+0000 7f7e8affd700 1 -- 172.21.15.7:0/3305831599 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 225 v225) v1 ==== 95+0+5453 (secure 0 0 0) 0x7f7e90057770 con 0x7f7e9c1535e0 2023-06-09T11:59:31.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:31.794 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":225,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:59:23.977988+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 msgr2=0x7f7e7c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 0x7f7e7c06d1c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c007fd0 tx=0x7f7e8c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 msgr2=0x7f7e9c15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000e7e0 tx=0x7f7e9000eaf0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 shutdown_connections 2023-06-09T11:59:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7e9c1539b0 0x7f7e9c1e1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7e7c06ad10 0x7f7e7c06d1c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7e9c0c4ce0 0x7f7e9c0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 --2- 172.21.15.7:0/3305831599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9c1535e0 0x7f7e9c15be10 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.792+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 >> 172.21.15.7:0/3305831599 conn(0x7f7e9c0737f0 msgr2=0x7f7e9c1587b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.793+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 shutdown_connections 2023-06-09T11:59:31.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:31.793+0000 7f7ea0bff700 1 -- 172.21.15.7:0/3305831599 wait complete. 2023-06-09T11:59:31.799 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 225 2023-06-09T11:59:31.813 INFO:tasks.mds_thrash.fs.[cephfs]:waiting till mds map indicates mds.e is in active, standby or standby-replay 2023-06-09T11:59:33.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T11:59:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.024+0000 7f7fcabff700 1 Processor -- start 2023-06-09T11:59:34.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.024+0000 7f7fcabff700 1 -- start start 2023-06-09T11:59:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.025+0000 7f7fcabff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.025+0000 7f7fcabff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7fc40cb9a0 0x7f7fc4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fcabff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc4159b20 con 0x7f7fc415cd40 2023-06-09T11:59:34.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fcabff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc40cac80 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fcabff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc41597d0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38690/0 (socket says 172.21.15.7:38690) 2023-06-09T11:59:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 -- 172.21.15.7:0/1601284721 learned_addr learned my addr 172.21.15.7:0/1601284721 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:34.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc3fff700 1 --2- 172.21.15.7:0/1601284721 >> v2:172.21.15.117:3300/0 conn(0x7f7fc40cb9a0 0x7f7fc4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 -- 172.21.15.7:0/1601284721 >> v2:172.21.15.117:3300/0 conn(0x7f7fc40cb9a0 msgr2=0x7f7fc4159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 --2- 172.21.15.7:0/1601284721 >> v2:172.21.15.117:3300/0 conn(0x7f7fc40cb9a0 0x7f7fc4159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 -- 172.21.15.7:0/1601284721 >> v1:172.21.15.117:6789/0 conn(0x7f7fc415cd40 legacy=0x7f7fc40cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:34.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 -- 172.21.15.7:0/1601284721 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4071ab0 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.026+0000 7f7fc899b700 1 --2- 172.21.15.7:0/1601284721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40c9c60 tx=0x7f7fb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa2fcf1fa90544c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fc37fe700 1 -- 172.21.15.7:0/1601284721 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8004230 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fc37fe700 1 -- 172.21.15.7:0/1601284721 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8004390 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fc37fe700 1 -- 172.21.15.7:0/1601284721 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80036a0 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 -- 172.21.15.7:0/1601284721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb5d0 msgr2=0x7f7fc40b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 --2- 172.21.15.7:0/1601284721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40c9c60 tx=0x7f7fb8009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 -- 172.21.15.7:0/1601284721 shutdown_connections 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 --2- 172.21.15.7:0/1601284721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb5d0 0x7f7fc40b52b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 --2- 172.21.15.7:0/1601284721 >> v2:172.21.15.117:3300/0 conn(0x7f7fc40cb9a0 0x7f7fc4159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 -- 172.21.15.7:0/1601284721 >> 172.21.15.7:0/1601284721 conn(0x7f7fc4070af0 msgr2=0x7f7fc4070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:34.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 -- 172.21.15.7:0/1601284721 shutdown_connections 2023-06-09T11:59:34.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.027+0000 7f7fcabff700 1 -- 172.21.15.7:0/1601284721 wait complete. 2023-06-09T11:59:34.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.029+0000 7f7fcabff700 1 Processor -- start 2023-06-09T11:59:34.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.029+0000 7f7fcabff700 1 -- start start 2023-06-09T11:59:34.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc40cb5d0 0x7f7fc40b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc415cd40 0x7f7fc415b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40cf620 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc4154600 con 0x7f7fc415cd40 2023-06-09T11:59:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fcabff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc40ce730 con 0x7f7fc40cb5d0 2023-06-09T11:59:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fc3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fc3fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38696/0 (socket says 172.21.15.7:38696) 2023-06-09T11:59:34.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.030+0000 7f7fc3fff700 1 -- 172.21.15.7:0/436173381 learned_addr learned my addr 172.21.15.7:0/436173381 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:34.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc899b700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc40cb5d0 0x7f7fc40b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc919c700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc415cd40 0x7f7fc415b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 -- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc40cb5d0 msgr2=0x7f7fc40b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc40cb5d0 0x7f7fc40b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 -- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc415cd40 msgr2=0x7f7fc415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc415cd40 0x7f7fc415b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 -- 172.21.15.7:0/436173381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009550 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc3fff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c7f0 tx=0x7f7fb400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4011800 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb4011960 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc415d4f0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.031+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4011cf0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.032+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc415d210 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.032+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc415d7d0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.033+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fb4018360 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.034+0000 7f7fc17fa700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 0x7f7fb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.034+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f7fb4089eb0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.035+0000 7f7fc899b700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 0x7f7fb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.035+0000 7f7fc899b700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 0x7f7fb006d190 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009130 tx=0x7f7fb8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.039+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7fb4057fe0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.297+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fc4074cf0 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.298+0000 7f7fc17fa700 1 -- 172.21.15.7:0/436173381 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 226 v226) v1 ==== 95+0+6208 (secure 0 0 0) 0x7f7fb401f030 con 0x7f7fc40cb9a0 2023-06-09T11:59:34.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":226,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8886,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1757884923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1757884923},{"type":"v1","addr":"172.21.15.7:6839","nonce":1757884923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":225,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T11:59:23.977988+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T11:59:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 msgr2=0x7f7fb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 0x7f7fb006d190 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8009130 tx=0x7f7fb8008030 comp rx=0 tx=0).stop 2023-06-09T11:59:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 msgr2=0x7f7fc41eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c7f0 tx=0x7f7fb400cbb0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 shutdown_connections 2023-06-09T11:59:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7fc415cd40 0x7f7fc415b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7fb006ace0 0x7f7fb006d190 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7fc40cb5d0 0x7f7fc40b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 --2- 172.21.15.7:0/436173381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fc40cb9a0 0x7f7fc41eff50 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 >> 172.21.15.7:0/436173381 conn(0x7f7fc4070af0 msgr2=0x7f7fc4158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 shutdown_connections 2023-06-09T11:59:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.301+0000 7f7fcabff700 1 -- 172.21.15.7:0/436173381 wait complete. 2023-06-09T11:59:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 226 2023-06-09T11:59:34.322 INFO:tasks.mds_thrash.fs.[cephfs]:mds.e reported in up:standby state 2023-06-09T11:59:34.322 INFO:tasks.mds_thrash.fs.[cephfs]:stat['max_mds'] = 1 2023-06-09T11:59:34.322 INFO:tasks.mds_thrash.fs.[cephfs]:stat['deactivate'] = 0 2023-06-09T11:59:34.322 INFO:tasks.mds_thrash.fs.[cephfs]:stat['kill'] = 25 2023-06-09T11:59:34.323 INFO:tasks.mds_thrash:done joining 2023-06-09T11:59:34.323 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-06-09T11:59:34.332 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-06-09T11:59:34.333 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:59:34.333 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T11:59:34.376 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2023-06-09T11:59:34.376 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-09T11:59:34.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:59:34.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.695+0000 7f224ebff700 1 Processor -- start 2023-06-09T11:59:34.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.696+0000 7f224ebff700 1 -- start start 2023-06-09T11:59:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224ebff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f224815b4a0 0x7f224815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224ebff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f22480c9f40 0x7f22480c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224ebff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224815a500 con 0x7f2248153880 2023-06-09T11:59:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224ebff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22480c5f10 con 0x7f22480c9f40 2023-06-09T11:59:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224ebff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224815da00 con 0x7f224815b4a0 2023-06-09T11:59:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224c99b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2248153880 0x7f224815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2023-06-09T11:59:34.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224c99b700 1 -- 172.21.15.7:0/3189365424 learned_addr learned my addr 172.21.15.7:0/3189365424 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:34.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224d19c700 1 --2- 172.21.15.7:0/3189365424 >> v2:172.21.15.7:3300/0 conn(0x7f22480c9f40 0x7f22480c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f2247fff700 1 --2- 172.21.15.7:0/3189365424 >> v2:172.21.15.117:3300/0 conn(0x7f224815b4a0 0x7f224815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224d19c700 1 -- 172.21.15.7:0/3189365424 >> v2:172.21.15.117:3300/0 conn(0x7f224815b4a0 msgr2=0x7f224815d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224d19c700 1 --2- 172.21.15.7:0/3189365424 >> v2:172.21.15.117:3300/0 conn(0x7f224815b4a0 0x7f224815d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.697+0000 7f224d19c700 1 -- 172.21.15.7:0/3189365424 >> v1:172.21.15.7:6789/0 conn(0x7f2248153880 legacy=0x7f224815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.698+0000 7f224d19c700 1 -- 172.21.15.7:0/3189365424 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22481596e0 con 0x7f22480c9f40 2023-06-09T11:59:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.698+0000 7f224d19c700 1 --2- 172.21.15.7:0/3189365424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22480c9f40 0x7f22480c4e30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f22480cce30 tx=0x7f224000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c49c70e6e6776d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.698+0000 7f22477fe700 1 -- 172.21.15.7:0/3189365424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224000e070 con 0x7f22480c9f40 2023-06-09T11:59:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.698+0000 7f22477fe700 1 -- 172.21.15.7:0/3189365424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224000bbd0 con 0x7f22480c9f40 2023-06-09T11:59:34.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.698+0000 7f22477fe700 1 -- 172.21.15.7:0/3189365424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240012430 con 0x7f22480c9f40 2023-06-09T11:59:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 -- 172.21.15.7:0/3189365424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22480c9f40 msgr2=0x7f22480c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 --2- 172.21.15.7:0/3189365424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22480c9f40 0x7f22480c4e30 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f22480cce30 tx=0x7f224000b330 comp rx=0 tx=0).stop 2023-06-09T11:59:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 -- 172.21.15.7:0/3189365424 shutdown_connections 2023-06-09T11:59:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 --2- 172.21.15.7:0/3189365424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22480c9f40 0x7f22480c4e30 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 --2- 172.21.15.7:0/3189365424 >> v2:172.21.15.117:3300/0 conn(0x7f224815b4a0 0x7f224815d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 -- 172.21.15.7:0/3189365424 >> 172.21.15.7:0/3189365424 conn(0x7f22480737f0 msgr2=0x7f22480720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 -- 172.21.15.7:0/3189365424 shutdown_connections 2023-06-09T11:59:34.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.699+0000 7f224ebff700 1 -- 172.21.15.7:0/3189365424 wait complete. 2023-06-09T11:59:34.706 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.700+0000 7f224ebff700 1 Processor -- start 2023-06-09T11:59:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 -- start start 2023-06-09T11:59:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480c9f40 0x7f22481519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224815b4a0 0x7f224806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480ced00 con 0x7f2248153880 2023-06-09T11:59:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2248151540 con 0x7f22480c9f40 2023-06-09T11:59:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.701+0000 7f224ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22480cd970 con 0x7f224815b4a0 2023-06-09T11:59:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38726/0 (socket says 172.21.15.7:38726) 2023-06-09T11:59:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 -- 172.21.15.7:0/2621528317 learned_addr learned my addr 172.21.15.7:0/2621528317 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f224c99b700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480c9f40 0x7f22481519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f224d19c700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224815b4a0 0x7f224806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 -- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224815b4a0 msgr2=0x7f224806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224815b4a0 0x7f224806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 -- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480c9f40 msgr2=0x7f22481519d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480c9f40 0x7f22481519d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 -- 172.21.15.7:0/2621528317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2240009a70 con 0x7f2248153880 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.702+0000 7f2247fff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f223400d800 tx=0x7f223400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.703+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22340125f0 con 0x7f2248153880 2023-06-09T11:59:34.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.703+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2234012750 con 0x7f2248153880 2023-06-09T11:59:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.703+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2248159c60 con 0x7f2248153880 2023-06-09T11:59:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.703+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234012b00 con 0x7f2248153880 2023-06-09T11:59:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.703+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2248159980 con 0x7f2248153880 2023-06-09T11:59:34.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.704+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248159f40 con 0x7f2248153880 2023-06-09T11:59:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.705+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2234012c60 con 0x7f2248153880 2023-06-09T11:59:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.706+0000 7f22457fa700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 0x7f222c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:34.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.706+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f2234089e40 con 0x7f2248153880 2023-06-09T11:59:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.707+0000 7f224c99b700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 0x7f222c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.707+0000 7f224c99b700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 0x7f222c06d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f223c005910 tx=0x7f223c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:34.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.710+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2234054570 con 0x7f2248153880 2023-06-09T11:59:34.922 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.920+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f224814e350 con 0x7f2248153880 2023-06-09T11:59:34.923 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.921+0000 7f22457fa700 1 -- 172.21.15.7:0/2621528317 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+21641 (secure 0 0 0) 0x7f2234057b90 con 0x7f2248153880 2023-06-09T11:59:34.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:34.925 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:59:17.800788+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6833/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6835/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.117:6834/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.7:6838/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6836/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6833/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6832/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6834/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.117:6834/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6833/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6833/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6832/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6835/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.117:6833/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.117:6837/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6832/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6837/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6836/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.117:6834/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.7:6839/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6835/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.7:6839/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.7:6838/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6837/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6834/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6836/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6836/1520141131":"2023-06-10T11:44:19.573452+0000","172.21.15.117:6833/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6835/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6833/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6834/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6835/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6834/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.117:6832/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.7:6838/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.117:6836/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6832/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.7:6839/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6834/1343666954":"2023-06-10T11:40:29.517640+0000","172.21.15.117:6835/1343666954":"2023-06-10T11:40:29.517640+0000","192.168.0.1:0/1078574456":"2023-06-09T12:59:16.804971+0000","172.21.15.7:6838/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6837/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.7:6839/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6836/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6837/1520141131":"2023-06-10T11:44:19.573452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:59:34.926 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 msgr2=0x7f222c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 0x7f222c06d190 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f223c005910 tx=0x7f223c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 msgr2=0x7f22480caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f223400d800 tx=0x7f223400db10 comp rx=0 tx=0).stop 2023-06-09T11:59:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 shutdown_connections 2023-06-09T11:59:34.927 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f222c06ace0 0x7f222c06d190 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f224815b4a0 0x7f224806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f22480c9f40 0x7f22481519d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 --2- 172.21.15.7:0/2621528317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2248153880 0x7f22480caa80 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 >> 172.21.15.7:0/2621528317 conn(0x7f22480737f0 msgr2=0x7f224814d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 shutdown_connections 2023-06-09T11:59:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:34.924+0000 7f224ebff700 1 -- 172.21.15.7:0/2621528317 wait complete. 2023-06-09T11:59:34.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-09T11:10:42.244017+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-06-09T11:10:42.285603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-09T11:10:47.777397+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-06-09T11:10:49.153345+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 505, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-09T11:59:34.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-09T11:59:35.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.145+0000 7f7c73bff700 1 Processor -- start 2023-06-09T11:59:35.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.145+0000 7f7c73bff700 1 -- start start 2023-06-09T11:59:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.146+0000 7f7c73bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c6c14d070 0x7f7c6c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c73bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7c6c070110 0x7f7c6c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c73bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c0cbc90 con 0x7f7c6c14d440 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c73bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c1710 con 0x7f7c6c14d070 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c73bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c0b90 con 0x7f7c6c070110 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c6c14d070 0x7f7c6c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f7c6c14d070 0x7f7c6c0cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38734/0 (socket says 172.21.15.7:38734) 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 -- 172.21.15.7:0/604386304 learned_addr learned my addr 172.21.15.7:0/604386304 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:35.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7219c700 1 --2- 172.21.15.7:0/604386304 >> v2:172.21.15.117:3300/0 conn(0x7f7c6c070110 0x7f7c6c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 -- 172.21.15.7:0/604386304 >> v2:172.21.15.117:3300/0 conn(0x7f7c6c070110 msgr2=0x7f7c6c0cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 --2- 172.21.15.7:0/604386304 >> v2:172.21.15.117:3300/0 conn(0x7f7c6c070110 0x7f7c6c0cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 -- 172.21.15.7:0/604386304 >> v1:172.21.15.117:6789/0 conn(0x7f7c6c14d440 legacy=0x7f7c6c0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.147+0000 7f7c7199b700 1 -- 172.21.15.7:0/604386304 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c0c0e70 con 0x7f7c6c14d070 2023-06-09T11:59:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c7199b700 1 --2- 172.21.15.7:0/604386304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb170 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c06f770 tx=0x7f7c6800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cec5d83502eb86d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c70999700 1 -- 172.21.15.7:0/604386304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6800dd30 con 0x7f7c6c14d070 2023-06-09T11:59:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c70999700 1 -- 172.21.15.7:0/604386304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6800de90 con 0x7f7c6c14d070 2023-06-09T11:59:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c70999700 1 -- 172.21.15.7:0/604386304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68010690 con 0x7f7c6c14d070 2023-06-09T11:59:35.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 -- 172.21.15.7:0/604386304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c14d070 msgr2=0x7f7c6c0cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 --2- 172.21.15.7:0/604386304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb170 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c06f770 tx=0x7f7c6800be70 comp rx=0 tx=0).stop 2023-06-09T11:59:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 -- 172.21.15.7:0/604386304 shutdown_connections 2023-06-09T11:59:35.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 --2- 172.21.15.7:0/604386304 >> v2:172.21.15.117:3300/0 conn(0x7f7c6c070110 0x7f7c6c0cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 --2- 172.21.15.7:0/604386304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb170 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 -- 172.21.15.7:0/604386304 >> 172.21.15.7:0/604386304 conn(0x7f7c6c06a4d0 msgr2=0x7f7c6c06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.148+0000 7f7c73bff700 1 -- 172.21.15.7:0/604386304 shutdown_connections 2023-06-09T11:59:35.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.149+0000 7f7c73bff700 1 -- 172.21.15.7:0/604386304 wait complete. 2023-06-09T11:59:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.150+0000 7f7c73bff700 1 Processor -- start 2023-06-09T11:59:35.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.150+0000 7f7c73bff700 1 -- start start 2023-06-09T11:59:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c6c14d440 0x7f7c6c0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c3420 con 0x7f7c6c070110 2023-06-09T11:59:35.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c4580 con 0x7f7c6c14d070 2023-06-09T11:59:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.151+0000 7f7c73bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c1ee3e0 con 0x7f7c6c14d440 2023-06-09T11:59:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38744/0 (socket says 172.21.15.7:38744) 2023-06-09T11:59:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 -- 172.21.15.7:0/958415534 learned_addr learned my addr 172.21.15.7:0/958415534 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:35.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7119a700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7219c700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c6c14d440 0x7f7c6c0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 -- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c6c14d440 msgr2=0x7f7c6c0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c6c14d440 0x7f7c6c0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 -- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c6c14d070 msgr2=0x7f7c6c0cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 -- 172.21.15.7:0/958415534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6800b7f0 con 0x7f7c6c070110 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.152+0000 7f7c7199b700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.153+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6801f070 con 0x7f7c6c070110 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.153+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c680039f0 con 0x7f7c6c070110 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.153+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003d80 con 0x7f7c6c070110 2023-06-09T11:59:35.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.153+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c0c1430 con 0x7f7c6c070110 2023-06-09T11:59:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.153+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c0c1150 con 0x7f7c6c070110 2023-06-09T11:59:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.154+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c0c2060 con 0x7f7c6c070110 2023-06-09T11:59:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.155+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7c6802c430 con 0x7f7c6c070110 2023-06-09T11:59:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.156+0000 7f7c627fc700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 0x7f7c5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.156+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f7c6808cef0 con 0x7f7c6c070110 2023-06-09T11:59:35.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.156+0000 7f7c7119a700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 0x7f7c5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.157+0000 7f7c7119a700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 0x7f7c5006d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c072380 tx=0x7f7c5c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.161+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7c68057710 con 0x7f7c6c070110 2023-06-09T11:59:35.369 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.367+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c6c074cf0 con 0x7f7c6c070110 2023-06-09T11:59:35.370 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.368+0000 7f7c627fc700 1 -- 172.21.15.7:0/958415534 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c6805ad30 con 0x7f7c6c070110 2023-06-09T11:59:35.371 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2023-06-09T11:59:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 msgr2=0x7f7c5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 0x7f7c5006d190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c072380 tx=0x7f7c5c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:35.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 msgr2=0x7f7c6c0c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7c68000c00 tx=0x7f7c68004750 comp rx=0 tx=0).stop 2023-06-09T11:59:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 shutdown_connections 2023-06-09T11:59:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7c5006ace0 0x7f7c5006d190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7c6c14d440 0x7f7c6c0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c6c070110 0x7f7c6c0c3e30 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 --2- 172.21.15.7:0/958415534 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7c6c14d070 0x7f7c6c0cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 >> 172.21.15.7:0/958415534 conn(0x7f7c6c06a4d0 msgr2=0x7f7c6c071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 shutdown_connections 2023-06-09T11:59:35.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.370+0000 7f7c73bff700 1 -- 172.21.15.7:0/958415534 wait complete. 2023-06-09T11:59:35.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-06-09T11:59:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.594+0000 7f50b97ff700 1 Processor -- start 2023-06-09T11:59:35.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.594+0000 7f50b97ff700 1 -- start start 2023-06-09T11:59:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b97ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50b40c64e0 0x7f50b406b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b97ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f50b406b740 0x7f50b415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b97ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b40c6220 con 0x7f50b41537a0 2023-06-09T11:59:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b97ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b40c4f70 con 0x7f50b406b740 2023-06-09T11:59:35.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b97ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b40c5c20 con 0x7f50b40c64e0 2023-06-09T11:59:35.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b2ffd700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f50b41537a0 0x7f50b40cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34022/0 (socket says 172.21.15.7:34022) 2023-06-09T11:59:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b2ffd700 1 -- 172.21.15.7:0/4026651628 learned_addr learned my addr 172.21.15.7:0/4026651628 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.595+0000 7f50b37fe700 1 --2- 172.21.15.7:0/4026651628 >> v2:172.21.15.7:3300/0 conn(0x7f50b406b740 0x7f50b415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b27fc700 1 --2- 172.21.15.7:0/4026651628 >> v2:172.21.15.117:3300/0 conn(0x7f50b40c64e0 0x7f50b406b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b37fe700 1 -- 172.21.15.7:0/4026651628 >> v2:172.21.15.117:3300/0 conn(0x7f50b40c64e0 msgr2=0x7f50b406b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:35.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b37fe700 1 --2- 172.21.15.7:0/4026651628 >> v2:172.21.15.117:3300/0 conn(0x7f50b40c64e0 0x7f50b406b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b37fe700 1 -- 172.21.15.7:0/4026651628 >> v1:172.21.15.7:6789/0 conn(0x7f50b41537a0 legacy=0x7f50b40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b37fe700 1 -- 172.21.15.7:0/4026651628 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40c4480 con 0x7f50b406b740 2023-06-09T11:59:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b1ffb700 1 -- 172.21.15.7:0/4026651628 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884168893 0 0) 0x7f50a4001110 con 0x7f50b41537a0 2023-06-09T11:59:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b37fe700 1 --2- 172.21.15.7:0/4026651628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b406b740 0x7f50b415b2b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f50b40cd0b0 tx=0x7f50ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3beaccc67ae20d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b1ffb700 1 -- 172.21.15.7:0/4026651628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00e070 con 0x7f50b406b740 2023-06-09T11:59:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b1ffb700 1 -- 172.21.15.7:0/4026651628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac00bb20 con 0x7f50b406b740 2023-06-09T11:59:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.596+0000 7f50b1ffb700 1 -- 172.21.15.7:0/4026651628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac012430 con 0x7f50b406b740 2023-06-09T11:59:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 -- 172.21.15.7:0/4026651628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b406b740 msgr2=0x7f50b415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 --2- 172.21.15.7:0/4026651628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b406b740 0x7f50b415b2b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f50b40cd0b0 tx=0x7f50ac00b330 comp rx=0 tx=0).stop 2023-06-09T11:59:35.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 -- 172.21.15.7:0/4026651628 shutdown_connections 2023-06-09T11:59:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 --2- 172.21.15.7:0/4026651628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b406b740 0x7f50b415b2b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 --2- 172.21.15.7:0/4026651628 >> v2:172.21.15.117:3300/0 conn(0x7f50b40c64e0 0x7f50b406b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 -- 172.21.15.7:0/4026651628 >> 172.21.15.7:0/4026651628 conn(0x7f50b4072040 msgr2=0x7f50b4072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 -- 172.21.15.7:0/4026651628 shutdown_connections 2023-06-09T11:59:35.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.597+0000 7f50b97ff700 1 -- 172.21.15.7:0/4026651628 wait complete. 2023-06-09T11:59:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.599+0000 7f50b97ff700 1 Processor -- start 2023-06-09T11:59:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.599+0000 7f50b97ff700 1 -- start start 2023-06-09T11:59:35.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b406b740 0x7f50b41eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b41537a0 0x7f50b4150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b40c8a50 con 0x7f50b40c64e0 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41ef160 con 0x7f50b41537a0 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b97ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41ef420 con 0x7f50b406b740 2023-06-09T11:59:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38762/0 (socket says 172.21.15.7:38762) 2023-06-09T11:59:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 -- 172.21.15.7:0/1545487695 learned_addr learned my addr 172.21.15.7:0/1545487695 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b2ffd700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b406b740 0x7f50b41eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b37fe700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b41537a0 0x7f50b4150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 -- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b406b740 msgr2=0x7f50b41eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b406b740 0x7f50b41eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 -- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b41537a0 msgr2=0x7f50b4150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b41537a0 0x7f50b4150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.600+0000 7f50b27fc700 1 -- 172.21.15.7:0/1545487695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac009a70 con 0x7f50b40c64e0 2023-06-09T11:59:35.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f50b27fc700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9b0 tx=0x7f50a800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8009710 con 0x7f50b40c64e0 2023-06-09T11:59:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a800ee50 con 0x7f50b40c64e0 2023-06-09T11:59:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a800ca50 con 0x7f50b40c64e0 2023-06-09T11:59:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b40c46d0 con 0x7f50b40c64e0 2023-06-09T11:59:35.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.601+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b415ae20 con 0x7f50b40c64e0 2023-06-09T11:59:35.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.602+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b40c49b0 con 0x7f50b40c64e0 2023-06-09T11:59:35.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.603+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50a800cbb0 con 0x7f50b40c64e0 2023-06-09T11:59:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.604+0000 7f509b7fe700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.604+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f50a8089df0 con 0x7f50b40c64e0 2023-06-09T11:59:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.604+0000 7f50b2ffd700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.605+0000 7f50b2ffd700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 0x7f50a006d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50a4000f60 tx=0x7f50a4007c20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:35.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.609+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f50a8057fb0 con 0x7f50b40c64e0 2023-06-09T11:59:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.817+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f50b41e3800 con 0x7f50b40c64e0 2023-06-09T11:59:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.818+0000 7f509b7fe700 1 -- 172.21.15.7:0/1545487695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v75) v1 ==== 92+0+10 (secure 0 0 0) 0x7f50a80210a0 con 0x7f50b40c64e0 2023-06-09T11:59:35.821 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2023-06-09T11:59:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 msgr2=0x7f50a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 0x7f50a006d190 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f50a4000f60 tx=0x7f50a4007c20 comp rx=0 tx=0).stop 2023-06-09T11:59:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 msgr2=0x7f50b414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f50a800e9b0 tx=0x7f50a800ecc0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 shutdown_connections 2023-06-09T11:59:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50b41537a0 0x7f50b4150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50a006ace0 0x7f50a006d190 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50b406b740 0x7f50b41eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 --2- 172.21.15.7:0/1545487695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50b40c64e0 0x7f50b414fe10 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.820+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 >> 172.21.15.7:0/1545487695 conn(0x7f50b4072040 msgr2=0x7f50b40b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.821+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 shutdown_connections 2023-06-09T11:59:35.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:35.821+0000 7f50b97ff700 1 -- 172.21.15.7:0/1545487695 wait complete. 2023-06-09T11:59:35.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-09T11:59:36.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.042+0000 7fd21edff700 1 Processor -- start 2023-06-09T11:59:36.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.043+0000 7fd21edff700 1 -- start start 2023-06-09T11:59:36.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd21806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21edff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2180c64e0 0x7fd2180cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21edff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2180c6220 con 0x7fd21806b740 2023-06-09T11:59:36.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21edff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180c4f70 con 0x7fd2180c64e0 2023-06-09T11:59:36.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180c5c20 con 0x7fd2181537a0 2023-06-09T11:59:36.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2180c64e0 0x7fd2180cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd2180c64e0 0x7fd2180cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38768/0 (socket says 172.21.15.7:38768) 2023-06-09T11:59:36.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 -- 172.21.15.7:0/800095576 learned_addr learned my addr 172.21.15.7:0/800095576 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:36.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd21cb9b700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd21806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 -- 172.21.15.7:0/800095576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 msgr2=0x7fd21806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd21806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.044+0000 7fd217fff700 1 -- 172.21.15.7:0/800095576 >> v1:172.21.15.117:6789/0 conn(0x7fd21806b740 legacy=0x7fd21815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.045+0000 7fd217fff700 1 -- 172.21.15.7:0/800095576 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180c4480 con 0x7fd2180c64e0 2023-06-09T11:59:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.045+0000 7fd217fff700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2180cf590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098f0 tx=0x7fd204009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b300ca3725cb5e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.045+0000 7fd2177fe700 1 -- 172.21.15.7:0/800095576 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204004230 con 0x7fd2180c64e0 2023-06-09T11:59:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.045+0000 7fd2177fe700 1 -- 172.21.15.7:0/800095576 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd204004390 con 0x7fd2180c64e0 2023-06-09T11:59:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.045+0000 7fd2177fe700 1 -- 172.21.15.7:0/800095576 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2040036a0 con 0x7fd2180c64e0 2023-06-09T11:59:36.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 -- 172.21.15.7:0/800095576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 msgr2=0x7fd2180cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2180cf590 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fd2040098f0 tx=0x7fd204009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:36.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 -- 172.21.15.7:0/800095576 shutdown_connections 2023-06-09T11:59:36.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd21806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 --2- 172.21.15.7:0/800095576 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2180cf590 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 -- 172.21.15.7:0/800095576 >> 172.21.15.7:0/800095576 conn(0x7fd218072040 msgr2=0x7fd218072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:36.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 -- 172.21.15.7:0/800095576 shutdown_connections 2023-06-09T11:59:36.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.046+0000 7fd21edff700 1 -- 172.21.15.7:0/800095576 wait complete. 2023-06-09T11:59:36.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.047+0000 7fd21edff700 1 Processor -- start 2023-06-09T11:59:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 -- start start 2023-06-09T11:59:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 0x7fd21815b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd218153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2180c7110 con 0x7fd2180c64e0 2023-06-09T11:59:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181eec10 con 0x7fd21806b740 2023-06-09T11:59:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.048+0000 7fd21edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2181f2670 con 0x7fd2181537a0 2023-06-09T11:59:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38774/0 (socket says 172.21.15.7:38774) 2023-06-09T11:59:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 -- 172.21.15.7:0/1577746304 learned_addr learned my addr 172.21.15.7:0/1577746304 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd21cb9b700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 0x7fd21815b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd21d39c700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd218153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 -- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 msgr2=0x7fd218153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd218153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 -- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 msgr2=0x7fd21815b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 0x7fd21815b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 -- 172.21.15.7:0/1577746304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd204009550 con 0x7fd2180c64e0 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd21cb9b700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 0x7fd21815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd21d39c700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd218153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:36.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.049+0000 7fd217fff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd204009130 tx=0x7fd204004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.050+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd204003c40 con 0x7fd2180c64e0 2023-06-09T11:59:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.050+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20401f070 con 0x7fd2180c64e0 2023-06-09T11:59:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.050+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20400b660 con 0x7fd2180c64e0 2023-06-09T11:59:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.050+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2180c46d0 con 0x7fd2180c64e0 2023-06-09T11:59:36.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.050+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21815ae20 con 0x7fd2180c64e0 2023-06-09T11:59:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.051+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2180c49b0 con 0x7fd2180c64e0 2023-06-09T11:59:36.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.052+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd204021030 con 0x7fd2180c64e0 2023-06-09T11:59:36.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.053+0000 7fd2157fa700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 0x7fd1fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.053+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fd20408aba0 con 0x7fd2180c64e0 2023-06-09T11:59:36.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.053+0000 7fd21cb9b700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 0x7fd1fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.054+0000 7fd21cb9b700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 0x7fd1fc06d190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009820 tx=0x7fd20c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.057+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd204055240 con 0x7fd2180c64e0 2023-06-09T11:59:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.261+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2180c6220 con 0x7fd2180c64e0 2023-06-09T11:59:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.262+0000 7fd2157fa700 1 -- 172.21.15.7:0/1577746304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2180c6220 con 0x7fd2180c64e0 2023-06-09T11:59:36.264 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2023-06-09T11:59:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 msgr2=0x7fd1fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 0x7fd1fc06d190 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd20c009820 tx=0x7fd20c008040 comp rx=0 tx=0).stop 2023-06-09T11:59:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 msgr2=0x7fd2181ef3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fd204009130 tx=0x7fd204004750 comp rx=0 tx=0).stop 2023-06-09T11:59:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 shutdown_connections 2023-06-09T11:59:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd1fc06ace0 0x7fd1fc06d190 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd2181537a0 0x7fd218153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd21806b740 0x7fd21815b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.264+0000 7fd21edff700 1 --2- 172.21.15.7:0/1577746304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2180c64e0 0x7fd2181ef3f0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.265+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 >> 172.21.15.7:0/1577746304 conn(0x7fd218072040 msgr2=0x7fd2180ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:36.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.265+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 shutdown_connections 2023-06-09T11:59:36.269 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.265+0000 7fd21edff700 1 -- 172.21.15.7:0/1577746304 wait complete. 2023-06-09T11:59:36.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-09T11:59:36.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.486+0000 7fd6723ff700 1 Processor -- start 2023-06-09T11:59:36.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.486+0000 7fd6723ff700 1 -- start start 2023-06-09T11:59:36.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd6723ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd6723ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66c0c6220 con 0x7fd66c06b740 2023-06-09T11:59:36.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd6723ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66c0c4480 con 0x7fd66c0c64e0 2023-06-09T11:59:36.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd6723ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0c4f70 con 0x7fd66c1537a0 2023-06-09T11:59:36.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd67099c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd66c06b740 0x7fd66c15b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34034/0 (socket says 172.21.15.7:34034) 2023-06-09T11:59:36.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd67099c700 1 -- 172.21.15.7:0/1790944659 learned_addr learned my addr 172.21.15.7:0/1790944659 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:36.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd66bfff700 1 --2- 172.21.15.7:0/1790944659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084059255 0 0) 0x7fd664001490 con 0x7fd66c06b740 2023-06-09T11:59:36.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.488+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6540035b0 con 0x7fd66c06b740 2023-06-09T11:59:36.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145614995 0 0) 0x7fd66c0c4480 con 0x7fd66c0c64e0 2023-06-09T11:59:36.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd664001490 con 0x7fd66c0c64e0 2023-06-09T11:59:36.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66bfff700 1 -- 172.21.15.7:0/1790944659 >> v1:172.21.15.117:6789/0 conn(0x7fd66c0c64e0 legacy=0x7fd66c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66bfff700 1 -- 172.21.15.7:0/1790944659 >> v1:172.21.15.7:6789/0 conn(0x7fd66c06b740 legacy=0x7fd66c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66bfff700 1 -- 172.21.15.7:0/1790944659 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c15ae20 con 0x7fd66c1537a0 2023-06-09T11:59:36.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66bfff700 1 --2- 172.21.15.7:0/1790944659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c06b1b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2467c1c99e64063 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011830 con 0x7fd66c1537a0 2023-06-09T11:59:36.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65c011990 con 0x7fd66c1537a0 2023-06-09T11:59:36.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.489+0000 7fd66affd700 1 -- 172.21.15.7:0/1790944659 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011d00 con 0x7fd66c1537a0 2023-06-09T11:59:36.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 -- 172.21.15.7:0/1790944659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 msgr2=0x7fd66c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1790944659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c06b1b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009870 tx=0x7fd65c009b80 comp rx=0 tx=0).stop 2023-06-09T11:59:36.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 -- 172.21.15.7:0/1790944659 shutdown_connections 2023-06-09T11:59:36.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1790944659 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c06b1b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 -- 172.21.15.7:0/1790944659 >> 172.21.15.7:0/1790944659 conn(0x7fd66c072040 msgr2=0x7fd66c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:36.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 -- 172.21.15.7:0/1790944659 shutdown_connections 2023-06-09T11:59:36.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.490+0000 7fd6723ff700 1 -- 172.21.15.7:0/1790944659 wait complete. 2023-06-09T11:59:36.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.492+0000 7fd6723ff700 1 Processor -- start 2023-06-09T11:59:36.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.493+0000 7fd6723ff700 1 -- start start 2023-06-09T11:59:36.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66c06b740 0x7fd66c1f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c150aa0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c14a920 con 0x7fd66c06b740 2023-06-09T11:59:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd6723ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd66c0c6f50 con 0x7fd66c1537a0 2023-06-09T11:59:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd66b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38776/0 (socket says 172.21.15.7:38776) 2023-06-09T11:59:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd66b7fe700 1 -- 172.21.15.7:0/1258335806 learned_addr learned my addr 172.21.15.7:0/1258335806 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:36.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd66bfff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66c06b740 0x7fd66c1f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.494+0000 7fd67099c700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 -- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 msgr2=0x7fd66c0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c0cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 -- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66c06b740 msgr2=0x7fd66c1f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66c06b740 0x7fd66c1f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 -- 172.21.15.7:0/1258335806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0094a0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd67099c700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd66b7fe700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c4c0 tx=0x7fd66000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600052d0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd660005450 con 0x7fd66c0c64e0 2023-06-09T11:59:36.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6600038c0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66c0c49b0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.495+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66c0c46d0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.496+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66c0c4c90 con 0x7fd66c0c64e0 2023-06-09T11:59:36.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.497+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd660003a20 con 0x7fd66c0c64e0 2023-06-09T11:59:36.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.498+0000 7fd668ff9700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 0x7fd65806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.498+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fd6600893d0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.498+0000 7fd66bfff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 0x7fd65806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.499+0000 7fd66bfff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 0x7fd65806d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0090b0 tx=0x7fd65c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.503+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd660053b00 con 0x7fd66c0c64e0 2023-06-09T11:59:36.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.710+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd66c074cf0 con 0x7fd66c0c64e0 2023-06-09T11:59:36.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.711+0000 7fd668ff9700 1 -- 172.21.15.7:0/1258335806 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd660017020 con 0x7fd66c0c64e0 2023-06-09T11:59:36.714 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2023-06-09T11:59:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 msgr2=0x7fd65806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 0x7fd65806d190 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd65c0090b0 tx=0x7fd65c008030 comp rx=0 tx=0).stop 2023-06-09T11:59:36.716 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 msgr2=0x7fd66c14a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fd66000c4c0 tx=0x7fd66000c880 comp rx=0 tx=0).stop 2023-06-09T11:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 shutdown_connections 2023-06-09T11:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd65806ace0 0x7fd65806d190 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd66c1537a0 0x7fd66c0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd66c06b740 0x7fd66c1f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.713+0000 7fd6723ff700 1 --2- 172.21.15.7:0/1258335806 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd66c0c64e0 0x7fd66c14a3e0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.714+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 >> 172.21.15.7:0/1258335806 conn(0x7fd66c072040 msgr2=0x7fd66c14c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.714+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 shutdown_connections 2023-06-09T11:59:36.718 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.714+0000 7fd6723ff700 1 -- 172.21.15.7:0/1258335806 wait complete. 2023-06-09T11:59:36.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-09T11:59:36.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.933+0000 7f463c1ff700 1 Processor -- start 2023-06-09T11:59:36.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.933+0000 7f463c1ff700 1 -- start start 2023-06-09T11:59:36.936 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463c1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4634151510 0x7f46340c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463c1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4634158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463c1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46341580a0 con 0x7f463415b380 2023-06-09T11:59:36.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463c1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46341518e0 con 0x7f4634151510 2023-06-09T11:59:36.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463c1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c6180 con 0x7f4634071230 2023-06-09T11:59:36.937 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463979a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f463415b380 0x7f4634070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34044/0 (socket says 172.21.15.7:34044) 2023-06-09T11:59:36.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.934+0000 7f463979a700 1 -- 172.21.15.7:0/2780760703 learned_addr learned my addr 172.21.15.7:0/2780760703 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4639f9b700 1 --2- 172.21.15.7:0/2780760703 >> v2:172.21.15.117:3300/0 conn(0x7f4634151510 0x7f46340c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f463a79c700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4634158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458109830 0 0) 0x7f46341580a0 con 0x7f463415b380 2023-06-09T11:59:36.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46200035b0 con 0x7f463415b380 2023-06-09T11:59:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 968625950 0 0) 0x7f46200035b0 con 0x7f463415b380 2023-06-09T11:59:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 msgr2=0x7f4634158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4634158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 >> v2:172.21.15.117:3300/0 conn(0x7f4634151510 msgr2=0x7f46340c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.941 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 --2- 172.21.15.7:0/2780760703 >> v2:172.21.15.117:3300/0 conn(0x7f4634151510 0x7f46340c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634157490 con 0x7f463415b380 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f4624001670 con 0x7f463415b380 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 >> v1:172.21.15.7:6789/0 conn(0x7f463415b380 legacy=0x7f4634070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620003a00 0x7f4620003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620004430 0x7f46200048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4620004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46341518e0 con 0x7f4620004430 2023-06-09T11:59:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c6180 con 0x7f4620003a00 2023-06-09T11:59:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.935+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462000ba10 con 0x7f4634071230 2023-06-09T11:59:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620004430 0x7f46200048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f4639f9b700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620003a00 0x7f4620003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463a79c700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4620004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 -- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 msgr2=0x7f4620004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4634071230 0x7f4620004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 -- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620003a00 msgr2=0x7f4620003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:36.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620003a00 0x7f4620003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 -- 172.21.15.7:0/2780760703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46200035b0 con 0x7f4620004430 2023-06-09T11:59:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f463979a700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620004430 0x7f46200048f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f462400c320 tx=0x7f4620005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eab73cf3a04569 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462400ce60 con 0x7f4620004430 2023-06-09T11:59:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4624005960 con 0x7f4620004430 2023-06-09T11:59:36.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.936+0000 7f4638f99700 1 -- 172.21.15.7:0/2780760703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624005cf0 con 0x7f4620004430 2023-06-09T11:59:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 -- 172.21.15.7:0/2780760703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620004430 msgr2=0x7f46200048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 --2- 172.21.15.7:0/2780760703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4620004430 0x7f46200048f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f462400c320 tx=0x7f4620005880 comp rx=0 tx=0).stop 2023-06-09T11:59:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463a79c700 1 -- 172.21.15.7:0/2780760703 reap_dead start 2023-06-09T11:59:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 -- 172.21.15.7:0/2780760703 shutdown_connections 2023-06-09T11:59:36.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 -- 172.21.15.7:0/2780760703 >> 172.21.15.7:0/2780760703 conn(0x7f463406a4d0 msgr2=0x7f463406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 -- 172.21.15.7:0/2780760703 shutdown_connections 2023-06-09T11:59:36.947 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.937+0000 7f463c1ff700 1 -- 172.21.15.7:0/2780760703 wait complete. 2023-06-09T11:59:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.939+0000 7f463c1ff700 1 Processor -- start 2023-06-09T11:59:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.939+0000 7f463c1ff700 1 -- start start 2023-06-09T11:59:36.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 0x7f46340ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 0x7f46340c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c0860 con 0x7f46340c1d10 2023-06-09T11:59:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c8fa0 con 0x7f4620004430 2023-06-09T11:59:36.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463c1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46340c02e0 con 0x7f46340c1020 2023-06-09T11:59:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463979a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38794/0 (socket says 172.21.15.7:38794) 2023-06-09T11:59:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.940+0000 7f463979a700 1 -- 172.21.15.7:0/867519329 learned_addr learned my addr 172.21.15.7:0/867519329 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f4639f9b700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 0x7f46340ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463a79c700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 0x7f46340c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 -- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 msgr2=0x7f46340c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 0x7f46340c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 -- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 msgr2=0x7f46340ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 0x7f46340ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:36.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 -- 172.21.15.7:0/867519329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4634157a50 con 0x7f46340c1d10 2023-06-09T11:59:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f4639f9b700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 0x7f46340ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463a79c700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 0x7f46340c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463979a700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f46200056d0 tx=0x7f4624001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624001680 con 0x7f46340c1d10 2023-06-09T11:59:36.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4624004a80 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4624004e30 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4634157770 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.941+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4634157d30 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.942+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4634074cf0 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.943+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f46240070d0 con 0x7f46340c1d10 2023-06-09T11:59:36.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.944+0000 7f462a7fc700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.945+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f462408b200 con 0x7f46340c1d10 2023-06-09T11:59:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.945+0000 7f4639f9b700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.945+0000 7f4639f9b700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 0x7f461806d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4630009b50 tx=0x7f4630008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:36.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:36.949+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f4624055a80 con 0x7f46340c1d10 2023-06-09T11:59:37.172 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.170+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46340b10b0 con 0x7f46340c1d10 2023-06-09T11:59:37.173 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.170+0000 7f462a7fc700 1 -- 172.21.15.7:0/867519329 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 51 entries v75) v1 ==== 81+0+3135 (secure 0 0 0) 0x7f4624007da0 con 0x7f46340c1d10 2023-06-09T11:59:37.173 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/3744293936 2023-06-10T11:52:39.695648+0000 2023-06-09T11:59:37.174 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/3302752268 2023-06-10T11:49:49.651187+0000 2023-06-09T11:59:37.174 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/477143596 2023-06-10T11:46:44.607439+0000 2023-06-09T11:59:37.174 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/477143596 2023-06-10T11:46:44.607439+0000 2023-06-09T11:59:37.174 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6838/1569034808 2023-06-10T11:58:09.772592+0000 2023-06-09T11:59:37.174 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/3174501100 2023-06-10T11:24:19.294959+0000 2023-06-09T11:59:37.175 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/1239341654 2023-06-10T11:22:09.265354+0000 2023-06-09T11:59:37.175 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/1018690909 2023-06-10T11:33:04.415919+0000 2023-06-09T11:59:37.175 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/1617501683 2023-06-10T11:20:01.246951+0000 2023-06-09T11:59:37.175 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/4051500902 2023-06-10T11:15:19.165964+0000 2023-06-09T11:59:37.175 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/3302752268 2023-06-10T11:49:49.651187+0000 2023-06-09T11:59:37.176 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/1018690909 2023-06-10T11:33:04.415919+0000 2023-06-09T11:59:37.176 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/1113843600 2023-06-10T11:12:14.125539+0000 2023-06-09T11:59:37.176 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/1239341654 2023-06-10T11:22:09.265354+0000 2023-06-09T11:59:37.176 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/3642322665 2023-06-10T11:55:04.729418+0000 2023-06-09T11:59:37.176 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/557223025 2023-06-10T11:38:14.486645+0000 2023-06-09T11:59:37.177 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/3727664816 2023-06-10T11:47:54.623961+0000 2023-06-09T11:59:37.177 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/1113843600 2023-06-10T11:12:14.125539+0000 2023-06-09T11:59:37.177 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/3311310614 2023-06-10T11:36:19.460437+0000 2023-06-09T11:59:37.177 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/2733808306 2023-06-10T11:17:14.192970+0000 2023-06-09T11:59:37.177 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/3642322665 2023-06-10T11:55:04.729418+0000 2023-06-09T11:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6839/967742498 2023-06-10T11:13:19.140088+0000 2023-06-09T11:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/1617501683 2023-06-10T11:20:01.246951+0000 2023-06-09T11:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6839/2631794280 2023-06-10T11:25:04.306342+0000 2023-06-09T11:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6838/967742498 2023-06-10T11:13:19.140088+0000 2023-06-09T11:59:37.178 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/3174501100 2023-06-10T11:24:19.294959+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/2166536228 2023-06-10T11:27:44.342982+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/3727664816 2023-06-10T11:47:54.623961+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/1520141131 2023-06-10T11:44:19.573452+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/1900293448 2023-06-10T11:45:24.588596+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/4051500902 2023-06-10T11:15:19.165964+0000 2023-06-09T11:59:37.179 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/2112619043 2023-06-10T11:29:34.367463+0000 2023-06-09T11:59:37.180 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/2166536228 2023-06-10T11:27:44.342982+0000 2023-06-09T11:59:37.180 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6833/2112619043 2023-06-10T11:29:34.367463+0000 2023-06-09T11:59:37.180 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/3983837537 2023-06-10T11:30:34.382664+0000 2023-06-09T11:59:37.180 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/3983837537 2023-06-10T11:30:34.382664+0000 2023-06-09T11:59:37.180 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/3162263716 2023-06-10T11:34:49.438691+0000 2023-06-09T11:59:37.181 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/1900293448 2023-06-10T11:45:24.588596+0000 2023-06-09T11:59:37.181 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/3162263716 2023-06-10T11:34:49.438691+0000 2023-06-09T11:59:37.181 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6838/2631794280 2023-06-10T11:25:04.306342+0000 2023-06-09T11:59:37.181 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/3311310614 2023-06-10T11:36:19.460437+0000 2023-06-09T11:59:37.182 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6832/557223025 2023-06-10T11:38:14.486645+0000 2023-06-09T11:59:37.182 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6839/1569034808 2023-06-10T11:58:09.772592+0000 2023-06-09T11:59:37.182 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6834/1343666954 2023-06-10T11:40:29.517640+0000 2023-06-09T11:59:37.182 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6835/1343666954 2023-06-10T11:40:29.517640+0000 2023-06-09T11:59:37.182 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.1:0/1078574456 2023-06-09T12:59:16.804971+0000 2023-06-09T11:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6838/2829758995 2023-06-10T11:42:19.540762+0000 2023-06-09T11:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/2733808306 2023-06-10T11:17:14.192970+0000 2023-06-09T11:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6839/2829758995 2023-06-10T11:42:19.540762+0000 2023-06-09T11:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6836/3744293936 2023-06-10T11:52:39.695648+0000 2023-06-09T11:59:37.183 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.117:6837/1520141131 2023-06-10T11:44:19.573452+0000 2023-06-09T11:59:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 msgr2=0x7f461806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:37.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 0x7f461806d190 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4630009b50 tx=0x7f4630008040 comp rx=0 tx=0).stop 2023-06-09T11:59:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 msgr2=0x7f4634159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f46200056d0 tx=0x7f4624001a00 comp rx=0 tx=0).stop 2023-06-09T11:59:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 shutdown_connections 2023-06-09T11:59:37.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f461806ace0 0x7f461806d190 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f46340c1020 0x7f46340c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4620004430 0x7f46340ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 --2- 172.21.15.7:0/867519329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46340c1d10 0x7f4634159b30 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 >> 172.21.15.7:0/867519329 conn(0x7f463406a4d0 msgr2=0x7f46341e06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 shutdown_connections 2023-06-09T11:59:37.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.173+0000 7f463c1ff700 1 -- 172.21.15.7:0/867519329 wait complete. 2023-06-09T11:59:37.187 INFO:teuthology.orchestra.run.smithi007.stderr:listed 51 entries 2023-06-09T11:59:37.194 DEBUG:tasks.cephfs.mount:Force/lazy unmounting on client.0 2023-06-09T11:59:37.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:59:37.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount --lazy --force /home/ubuntu/cephtest/mnt.0 2023-06-09T11:59:37.427 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T11:59:37.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:59:37.428 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-06-09T11:59:37.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:59:37.471 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2023-06-09T11:59:37.553 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-06-09T11:59:37.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:59:37.555 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-06-09T11:59:37.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-09T11:59:37.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2023-06-09T11:59:37.781 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-06-09T11:59:37.790 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-06-09T11:59:37.791 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:59:37.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.993+0000 7f8a977ff700 1 Processor -- start 2023-06-09T11:59:37.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.993+0000 7f8a977ff700 1 -- start start 2023-06-09T11:59:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a977ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 0x7f8a90070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a977ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a901580a0 con 0x7f8a90151510 2023-06-09T11:59:37.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a977ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a900c6180 con 0x7f8a90071060 2023-06-09T11:59:37.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a977ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900714e0 con 0x7f8a900c5b40 2023-06-09T11:59:37.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a9559b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8a90151510 0x7f8a9015b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34052/0 (socket says 172.21.15.7:34052) 2023-06-09T11:59:37.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a9559b700 1 -- 172.21.15.7:0/755491412 learned_addr learned my addr 172.21.15.7:0/755491412 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:37.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a94d9a700 1 --2- 172.21.15.7:0/755491412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 0x7f8a90070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540952084 0 0) 0x7f8a901580a0 con 0x7f8a90151510 2023-06-09T11:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.995+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a7c0035b0 con 0x7f8a90151510 2023-06-09T11:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737981654 0 0) 0x7f8a900c6180 con 0x7f8a90071060 2023-06-09T11:59:38.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a94d9a700 1 -- 172.21.15.7:0/755491412 >> v1:172.21.15.117:6789/0 conn(0x7f8a90071060 legacy=0x7f8a90158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a94d9a700 1 -- 172.21.15.7:0/755491412 >> v1:172.21.15.7:6789/0 conn(0x7f8a90151510 legacy=0x7f8a9015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a94d9a700 1 -- 172.21.15.7:0/755491412 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a90157490 con 0x7f8a900c5b40 2023-06-09T11:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a94d9a700 1 --2- 172.21.15.7:0/755491412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 0x7f8a90070b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009870 tx=0x7f8a80009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=191c6f55cf318c45 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80011830 con 0x7f8a900c5b40 2023-06-09T11:59:38.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80011990 con 0x7f8a900c5b40 2023-06-09T11:59:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.996+0000 7f8a87fff700 1 -- 172.21.15.7:0/755491412 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80011d20 con 0x7f8a900c5b40 2023-06-09T11:59:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 -- 172.21.15.7:0/755491412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 msgr2=0x7f8a90070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 --2- 172.21.15.7:0/755491412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 0x7f8a90070b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009870 tx=0x7f8a80009b80 comp rx=0 tx=0).stop 2023-06-09T11:59:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 -- 172.21.15.7:0/755491412 shutdown_connections 2023-06-09T11:59:38.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 --2- 172.21.15.7:0/755491412 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a900c5b40 0x7f8a90070b20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 -- 172.21.15.7:0/755491412 >> 172.21.15.7:0/755491412 conn(0x7f8a9006a4d0 msgr2=0x7f8a9006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 -- 172.21.15.7:0/755491412 shutdown_connections 2023-06-09T11:59:38.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.997+0000 7f8a977ff700 1 -- 172.21.15.7:0/755491412 wait complete. 2023-06-09T11:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.999+0000 7f8a977ff700 1 Processor -- start 2023-06-09T11:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.999+0000 7f8a977ff700 1 -- start start 2023-06-09T11:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:37.999+0000 7f8a977ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 0x7f8a900c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a977ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a977ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 0x7f8a900c1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a977ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a900c7b80 con 0x7f8a900c5b40 2023-06-09T11:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a977ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901505d0 con 0x7f8a90151510 2023-06-09T11:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a977ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a901f2c90 con 0x7f8a90071060 2023-06-09T11:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38798/0 (socket says 172.21.15.7:38798) 2023-06-09T11:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 -- 172.21.15.7:0/836973961 learned_addr learned my addr 172.21.15.7:0/836973961 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a9559b700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 0x7f8a900c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a95d9c700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 0x7f8a900c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 -- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 msgr2=0x7f8a900c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 0x7f8a900c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 -- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 msgr2=0x7f8a900c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 0x7f8a900c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 -- 172.21.15.7:0/836973961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800094a0 con 0x7f8a900c5b40 2023-06-09T11:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a9559b700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 0x7f8a900c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:38.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a95d9c700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 0x7f8a900c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.000+0000 7f8a94d9a700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009bc0 tx=0x7f8a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.001+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8001a910 con 0x7f8a900c5b40 2023-06-09T11:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.001+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a8001aa70 con 0x7f8a900c5b40 2023-06-09T11:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.001+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a90157a50 con 0x7f8a900c5b40 2023-06-09T11:59:38.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.001+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8001ade0 con 0x7f8a900c5b40 2023-06-09T11:59:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.001+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a90157770 con 0x7f8a900c5b40 2023-06-09T11:59:38.010 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.002+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90157d30 con 0x7f8a900c5b40 2023-06-09T11:59:38.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.003+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a80022070 con 0x7f8a900c5b40 2023-06-09T11:59:38.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.004+0000 7f8a85ffb700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 0x7f8a7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.004+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8a8008af70 con 0x7f8a900c5b40 2023-06-09T11:59:38.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.004+0000 7f8a9559b700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 0x7f8a7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.005+0000 7f8a9559b700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 0x7f8a7406d190 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005910 tx=0x7f8a8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.008+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8a800bd2d0 con 0x7f8a900c5b40 2023-06-09T11:59:38.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.209+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a901f2e00 con 0x7f8a7406ace0 2023-06-09T11:59:38.222 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.220+0000 7f8a85ffb700 1 -- 172.21.15.7:0/836973961 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316295 (secure 0 0 0) 0x7f8a901f2e00 con 0x7f8a7406ace0 2023-06-09T11:59:38.226 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:38.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 msgr2=0x7f8a7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 0x7f8a7406d190 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c005910 tx=0x7f8a8c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 msgr2=0x7f8a901eed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009bc0 tx=0x7f8a80004750 comp rx=0 tx=0).stop 2023-06-09T11:59:38.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 shutdown_connections 2023-06-09T11:59:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a90151510 0x7f8a900c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a7406ace0 0x7f8a7406d190 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a90071060 0x7f8a900c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 --2- 172.21.15.7:0/836973961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a900c5b40 0x7f8a901eed50 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 >> 172.21.15.7:0/836973961 conn(0x7f8a9006a4d0 msgr2=0x7f8a9014b840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 shutdown_connections 2023-06-09T11:59:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.225+0000 7f8a977ff700 1 -- 172.21.15.7:0/836973961 wait complete. 2023-06-09T11:59:38.231 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:59:38.246 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1535,"stamp":"2023-06-09T11:59:36.925439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1371589134,"num_objects":16489,"num_object_clones":7273,"num_object_copies":32978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16489,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76374,"num_write_kb":1827736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":330,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88670,"ondisk_log_size":88670,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2545384,"kb_used_data":1881704,"kb_used_omap":0,"kb_used_meta":663616,"kb_avail":752429336,"statfs":{"total":773094113280,"available":770487640064,"internally_reserved":0,"allocated":1926864896,"data_stored":2747528768,"data_compressed":268411584,"data_compressed_allocated":324517888,"data_compressed_original":1187856384,"omap_allocated":0,"internal_metadata":679542784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,8,100,58],"upper_bound":256},"perf_stat":{"commit_latency_ms":173,"apply_latency_ms":173,"commit_latency_ns":173000000,"apply_latency_ns":173000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2121907,"num_objects":18,"num_object_clones":22,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":22,"num_read":264,"num_read_kb":1003,"num_write":137,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.176802+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:29.176802+0000","last_peered":"2023-06-09T11:59:29.176802+0000","last_clean":"2023-06-09T11:59:29.176802+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:29.176802+0000","last_undegraded":"2023-06-09T11:59:29.176802+0000","last_fullsized":"2023-06-09T11:59:29.176802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:41.997596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138064+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138064+0000","last_peered":"2023-06-09T11:59:29.138064+0000","last_clean":"2023-06-09T11:59:29.138064+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138064+0000","last_undegraded":"2023-06-09T11:59:29.138064+0000","last_fullsized":"2023-06-09T11:59:29.138064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:43.538089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184905+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.184905+0000","last_peered":"2023-06-09T11:59:29.184905+0000","last_clean":"2023-06-09T11:59:29.184905+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.184905+0000","last_undegraded":"2023-06-09T11:59:29.184905+0000","last_fullsized":"2023-06-09T11:59:29.184905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:54.053113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183976+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:29.183976+0000","last_peered":"2023-06-09T11:59:29.183976+0000","last_clean":"2023-06-09T11:59:29.183976+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:29.183976+0000","last_undegraded":"2023-06-09T11:59:29.183976+0000","last_fullsized":"2023-06-09T11:59:29.183976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:12.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179117+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179117+0000","last_peered":"2023-06-09T11:59:29.179117+0000","last_clean":"2023-06-09T11:59:29.179117+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179117+0000","last_undegraded":"2023-06-09T11:59:29.179117+0000","last_fullsized":"2023-06-09T11:59:29.179117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:24:43.823091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'864","reported_seq":3341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174723+0000","last_change":"2023-06-09T11:18:26.314501+0000","last_active":"2023-06-09T11:59:29.174723+0000","last_peered":"2023-06-09T11:59:29.174723+0000","last_clean":"2023-06-09T11:59:29.174723+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:59:29.174723+0000","last_undegraded":"2023-06-09T11:59:29.174723+0000","last_fullsized":"2023-06-09T11:59:29.174723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:01:52.942030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134658+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.134658+0000","last_peered":"2023-06-09T11:59:29.134658+0000","last_clean":"2023-06-09T11:59:29.134658+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.134658+0000","last_undegraded":"2023-06-09T11:59:29.134658+0000","last_fullsized":"2023-06-09T11:59:29.134658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:08.996053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3243,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164252+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164252+0000","last_peered":"2023-06-09T11:59:29.164252+0000","last_clean":"2023-06-09T11:59:29.164252+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164252+0000","last_undegraded":"2023-06-09T11:59:29.164252+0000","last_fullsized":"2023-06-09T11:59:29.164252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:39:08.386702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4197,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183305+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183305+0000","last_peered":"2023-06-09T11:59:29.183305+0000","last_clean":"2023-06-09T11:59:29.183305+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183305+0000","last_undegraded":"2023-06-09T11:59:29.183305+0000","last_fullsized":"2023-06-09T11:59:29.183305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:42:35.512902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163514+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163514+0000","last_peered":"2023-06-09T11:59:29.163514+0000","last_clean":"2023-06-09T11:59:29.163514+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163514+0000","last_undegraded":"2023-06-09T11:59:29.163514+0000","last_fullsized":"2023-06-09T11:59:29.163514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:40.374816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184625+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184625+0000","last_peered":"2023-06-09T11:59:29.184625+0000","last_clean":"2023-06-09T11:59:29.184625+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184625+0000","last_undegraded":"2023-06-09T11:59:29.184625+0000","last_fullsized":"2023-06-09T11:59:29.184625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:26:32.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.166730+0000","last_change":"2023-06-09T11:18:26.271005+0000","last_active":"2023-06-09T11:59:29.166730+0000","last_peered":"2023-06-09T11:59:29.166730+0000","last_clean":"2023-06-09T11:59:29.166730+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:29.166730+0000","last_undegraded":"2023-06-09T11:59:29.166730+0000","last_fullsized":"2023-06-09T11:59:29.166730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:55.416203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108516+0000","last_change":"2023-06-09T11:18:26.253105+0000","last_active":"2023-06-09T11:59:30.108516+0000","last_peered":"2023-06-09T11:59:30.108516+0000","last_clean":"2023-06-09T11:59:30.108516+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:30.108516+0000","last_undegraded":"2023-06-09T11:59:30.108516+0000","last_fullsized":"2023-06-09T11:59:30.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:36:08.467688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108534+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:29.108534+0000","last_peered":"2023-06-09T11:59:29.108534+0000","last_clean":"2023-06-09T11:59:29.108534+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:29.108534+0000","last_undegraded":"2023-06-09T11:59:29.108534+0000","last_fullsized":"2023-06-09T11:59:29.108534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:08:27.844416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134761+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:29.134761+0000","last_peered":"2023-06-09T11:59:29.134761+0000","last_clean":"2023-06-09T11:59:29.134761+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:29.134761+0000","last_undegraded":"2023-06-09T11:59:29.134761+0000","last_fullsized":"2023-06-09T11:59:29.134761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:43:06.239662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3425,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174867+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:29.174867+0000","last_peered":"2023-06-09T11:59:29.174867+0000","last_clean":"2023-06-09T11:59:29.174867+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:29.174867+0000","last_undegraded":"2023-06-09T11:59:29.174867+0000","last_fullsized":"2023-06-09T11:59:29.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:35.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183203+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:29.183203+0000","last_peered":"2023-06-09T11:59:29.183203+0000","last_clean":"2023-06-09T11:59:29.183203+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:29.183203+0000","last_undegraded":"2023-06-09T11:59:29.183203+0000","last_fullsized":"2023-06-09T11:59:29.183203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:37.819209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174315+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:29.174315+0000","last_peered":"2023-06-09T11:59:29.174315+0000","last_clean":"2023-06-09T11:59:29.174315+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:29.174315+0000","last_undegraded":"2023-06-09T11:59:29.174315+0000","last_fullsized":"2023-06-09T11:59:29.174315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:47:55.986286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3553,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108428+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.108428+0000","last_peered":"2023-06-09T11:59:29.108428+0000","last_clean":"2023-06-09T11:59:29.108428+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.108428+0000","last_undegraded":"2023-06-09T11:59:29.108428+0000","last_fullsized":"2023-06-09T11:59:29.108428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:53:59.032131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186654+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:29.186654+0000","last_peered":"2023-06-09T11:59:29.186654+0000","last_clean":"2023-06-09T11:59:29.186654+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:29.186654+0000","last_undegraded":"2023-06-09T11:59:29.186654+0000","last_fullsized":"2023-06-09T11:59:29.186654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:31.924577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.127903+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:29.127903+0000","last_peered":"2023-06-09T11:59:29.127903+0000","last_clean":"2023-06-09T11:59:29.127903+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:29.127903+0000","last_undegraded":"2023-06-09T11:59:29.127903+0000","last_fullsized":"2023-06-09T11:59:29.127903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:53.061847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3479,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.148430+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:29.148430+0000","last_peered":"2023-06-09T11:59:29.148430+0000","last_clean":"2023-06-09T11:59:29.148430+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:29.148430+0000","last_undegraded":"2023-06-09T11:59:29.148430+0000","last_fullsized":"2023-06-09T11:59:29.148430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:33:23.877739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206328+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:29.206328+0000","last_peered":"2023-06-09T11:59:29.206328+0000","last_clean":"2023-06-09T11:59:29.206328+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:29.206328+0000","last_undegraded":"2023-06-09T11:59:29.206328+0000","last_fullsized":"2023-06-09T11:59:29.206328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:31:00.135897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181162+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181162+0000","last_peered":"2023-06-09T11:59:29.181162+0000","last_clean":"2023-06-09T11:59:29.181162+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181162+0000","last_undegraded":"2023-06-09T11:59:29.181162+0000","last_fullsized":"2023-06-09T11:59:29.181162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:46:27.956223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3016,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105429+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.105429+0000","last_peered":"2023-06-09T11:59:30.105429+0000","last_clean":"2023-06-09T11:59:30.105429+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.105429+0000","last_undegraded":"2023-06-09T11:59:30.105429+0000","last_fullsized":"2023-06-09T11:59:30.105429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:07.769640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171191+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171191+0000","last_peered":"2023-06-09T11:59:29.171191+0000","last_clean":"2023-06-09T11:59:29.171191+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171191+0000","last_undegraded":"2023-06-09T11:59:29.171191+0000","last_fullsized":"2023-06-09T11:59:29.171191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:38:31.443299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116586+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116586+0000","last_peered":"2023-06-09T11:59:29.116586+0000","last_clean":"2023-06-09T11:59:29.116586+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116586+0000","last_undegraded":"2023-06-09T11:59:29.116586+0000","last_fullsized":"2023-06-09T11:59:29.116586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:28.791273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.608768+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:29.608768+0000","last_peered":"2023-06-09T11:59:29.608768+0000","last_clean":"2023-06-09T11:59:29.608768+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:29.608768+0000","last_undegraded":"2023-06-09T11:59:29.608768+0000","last_fullsized":"2023-06-09T11:59:29.608768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:54:04.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.159320+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:29.159320+0000","last_peered":"2023-06-09T11:59:29.159320+0000","last_clean":"2023-06-09T11:59:29.159320+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:29.159320+0000","last_undegraded":"2023-06-09T11:59:29.159320+0000","last_fullsized":"2023-06-09T11:59:29.159320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:04:01.139122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604309+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:29.604309+0000","last_peered":"2023-06-09T11:59:29.604309+0000","last_clean":"2023-06-09T11:59:29.604309+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:29.604309+0000","last_undegraded":"2023-06-09T11:59:29.604309+0000","last_fullsized":"2023-06-09T11:59:29.604309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:19.987020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099238+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.099238+0000","last_peered":"2023-06-09T11:59:30.099238+0000","last_clean":"2023-06-09T11:59:30.099238+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.099238+0000","last_undegraded":"2023-06-09T11:59:30.099238+0000","last_fullsized":"2023-06-09T11:59:30.099238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:03.591541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826279+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:23.826279+0000","last_peered":"2023-06-09T11:59:23.826279+0000","last_clean":"2023-06-09T11:59:23.826279+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:23.826279+0000","last_undegraded":"2023-06-09T11:59:23.826279+0000","last_fullsized":"2023-06-09T11:59:23.826279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:19:22.681147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120527+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120527+0000","last_peered":"2023-06-09T11:59:29.120527+0000","last_clean":"2023-06-09T11:59:29.120527+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120527+0000","last_undegraded":"2023-06-09T11:59:29.120527+0000","last_fullsized":"2023-06-09T11:59:29.120527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:15.836484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'180","reported_seq":343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825404+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:59:23.825404+0000","last_peered":"2023-06-09T11:59:23.825404+0000","last_clean":"2023-06-09T11:59:23.825404+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:23.825404+0000","last_undegraded":"2023-06-09T11:59:23.825404+0000","last_fullsized":"2023-06-09T11:59:23.825404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:22:06.545450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":124,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2960,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.204530+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:29.204530+0000","last_peered":"2023-06-09T11:59:29.204530+0000","last_clean":"2023-06-09T11:59:29.204530+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:29.204530+0000","last_undegraded":"2023-06-09T11:59:29.204530+0000","last_fullsized":"2023-06-09T11:59:29.204530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:51:52.478717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813587+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:59:17.813587+0000","last_peered":"2023-06-09T11:59:17.813587+0000","last_clean":"2023-06-09T11:59:17.813587+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:59:17.813587+0000","last_undegraded":"2023-06-09T11:59:17.813587+0000","last_fullsized":"2023-06-09T11:59:17.813587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:40.338566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208409+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:29.208409+0000","last_peered":"2023-06-09T11:59:29.208409+0000","last_clean":"2023-06-09T11:59:29.208409+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:29.208409+0000","last_undegraded":"2023-06-09T11:59:29.208409+0000","last_fullsized":"2023-06-09T11:59:29.208409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:32:56.568860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086513+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:59:30.086513+0000","last_peered":"2023-06-09T11:59:30.086513+0000","last_clean":"2023-06-09T11:59:30.086513+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:30.086513+0000","last_undegraded":"2023-06-09T11:59:30.086513+0000","last_fullsized":"2023-06-09T11:59:30.086513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:11:48.144420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088093+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:30.088093+0000","last_peered":"2023-06-09T11:59:30.088093+0000","last_clean":"2023-06-09T11:59:30.088093+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:30.088093+0000","last_undegraded":"2023-06-09T11:59:30.088093+0000","last_fullsized":"2023-06-09T11:59:30.088093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:43:21.546026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'822","reported_seq":1030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.085204+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:30.085204+0000","last_peered":"2023-06-09T11:59:30.085204+0000","last_clean":"2023-06-09T11:59:30.085204+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:30.085204+0000","last_undegraded":"2023-06-09T11:59:30.085204+0000","last_fullsized":"2023-06-09T11:59:30.085204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:15:45.170741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":767,"num_write_kb":25247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3094,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186545+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:29.186545+0000","last_peered":"2023-06-09T11:59:29.186545+0000","last_clean":"2023-06-09T11:59:29.186545+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:29.186545+0000","last_undegraded":"2023-06-09T11:59:29.186545+0000","last_fullsized":"2023-06-09T11:59:29.186545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:43.603410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'857","reported_seq":1071,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.080210+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:29.080210+0000","last_peered":"2023-06-09T11:59:29.080210+0000","last_clean":"2023-06-09T11:59:29.080210+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:29.080210+0000","last_undegraded":"2023-06-09T11:59:29.080210+0000","last_fullsized":"2023-06-09T11:59:29.080210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:48.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14073316,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":799,"num_write_kb":14332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3556,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.161492+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:29.161492+0000","last_peered":"2023-06-09T11:59:29.161492+0000","last_clean":"2023-06-09T11:59:29.161492+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:29.161492+0000","last_undegraded":"2023-06-09T11:59:29.161492+0000","last_fullsized":"2023-06-09T11:59:29.161492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:48.346810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'413","reported_seq":575,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814193+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:59:17.814193+0000","last_peered":"2023-06-09T11:59:17.814193+0000","last_clean":"2023-06-09T11:59:17.814193+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:17.814193+0000","last_undegraded":"2023-06-09T11:59:17.814193+0000","last_fullsized":"2023-06-09T11:59:17.814193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:03:12.461060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116041+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116041+0000","last_peered":"2023-06-09T11:59:29.116041+0000","last_clean":"2023-06-09T11:59:29.116041+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116041+0000","last_undegraded":"2023-06-09T11:59:29.116041+0000","last_fullsized":"2023-06-09T11:59:29.116041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:24:03.501267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"73'960","reported_seq":1143,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.820047+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:59:23.820047+0000","last_peered":"2023-06-09T11:59:23.820047+0000","last_clean":"2023-06-09T11:59:23.820047+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:59:23.820047+0000","last_undegraded":"2023-06-09T11:59:23.820047+0000","last_fullsized":"2023-06-09T11:59:23.820047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:53:42.964479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":890,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3022,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185110+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185110+0000","last_peered":"2023-06-09T11:59:29.185110+0000","last_clean":"2023-06-09T11:59:29.185110+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185110+0000","last_undegraded":"2023-06-09T11:59:29.185110+0000","last_fullsized":"2023-06-09T11:59:29.185110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:02:58.361435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1054","reported_seq":1302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086379+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:30.086379+0000","last_peered":"2023-06-09T11:59:30.086379+0000","last_clean":"2023-06-09T11:59:30.086379+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:30.086379+0000","last_undegraded":"2023-06-09T11:59:30.086379+0000","last_fullsized":"2023-06-09T11:59:30.086379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:30.247597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1013,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203830+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203830+0000","last_peered":"2023-06-09T11:59:29.203830+0000","last_clean":"2023-06-09T11:59:29.203830+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203830+0000","last_undegraded":"2023-06-09T11:59:29.203830+0000","last_fullsized":"2023-06-09T11:59:29.203830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:33:19.947716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825423+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.825423+0000","last_peered":"2023-06-09T11:59:23.825423+0000","last_clean":"2023-06-09T11:59:23.825423+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.825423+0000","last_undegraded":"2023-06-09T11:59:23.825423+0000","last_fullsized":"2023-06-09T11:59:23.825423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:22:06.716248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113291+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113291+0000","last_peered":"2023-06-09T11:59:29.113291+0000","last_clean":"2023-06-09T11:59:29.113291+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113291+0000","last_undegraded":"2023-06-09T11:59:29.113291+0000","last_fullsized":"2023-06-09T11:59:29.113291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:25:53.990234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.607238+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:29.607238+0000","last_peered":"2023-06-09T11:59:29.607238+0000","last_clean":"2023-06-09T11:59:29.607238+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:29.607238+0000","last_undegraded":"2023-06-09T11:59:29.607238+0000","last_fullsized":"2023-06-09T11:59:29.607238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:10:40.976689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178950+0000","last_change":"2023-06-09T11:18:26.315365+0000","last_active":"2023-06-09T11:59:29.178950+0000","last_peered":"2023-06-09T11:59:29.178950+0000","last_clean":"2023-06-09T11:59:29.178950+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:29.178950+0000","last_undegraded":"2023-06-09T11:59:29.178950+0000","last_fullsized":"2023-06-09T11:59:29.178950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:48:14.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'432","reported_seq":599,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134735+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:29.134735+0000","last_peered":"2023-06-09T11:59:29.134735+0000","last_clean":"2023-06-09T11:59:29.134735+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:29.134735+0000","last_undegraded":"2023-06-09T11:59:29.134735+0000","last_fullsized":"2023-06-09T11:59:29.134735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:17.698189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":370,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088582+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.088582+0000","last_peered":"2023-06-09T11:59:30.088582+0000","last_clean":"2023-06-09T11:59:30.088582+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.088582+0000","last_undegraded":"2023-06-09T11:59:30.088582+0000","last_fullsized":"2023-06-09T11:59:30.088582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:35:52.399332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"73'517","reported_seq":687,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.819694+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:59:23.819694+0000","last_peered":"2023-06-09T11:59:23.819694+0000","last_clean":"2023-06-09T11:59:23.819694+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:59:23.819694+0000","last_undegraded":"2023-06-09T11:59:23.819694+0000","last_fullsized":"2023-06-09T11:59:23.819694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:58.048595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":471,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.149039+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:29.149039+0000","last_peered":"2023-06-09T11:59:29.149039+0000","last_clean":"2023-06-09T11:59:29.149039+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:29.149039+0000","last_undegraded":"2023-06-09T11:59:29.149039+0000","last_fullsized":"2023-06-09T11:59:29.149039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:45.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.812549+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:17.812549+0000","last_peered":"2023-06-09T11:59:17.812549+0000","last_clean":"2023-06-09T11:59:17.812549+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:17.812549+0000","last_undegraded":"2023-06-09T11:59:17.812549+0000","last_fullsized":"2023-06-09T11:59:17.812549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:59.524461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105462+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:30.105462+0000","last_peered":"2023-06-09T11:59:30.105462+0000","last_clean":"2023-06-09T11:59:30.105462+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:30.105462+0000","last_undegraded":"2023-06-09T11:59:30.105462+0000","last_fullsized":"2023-06-09T11:59:30.105462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:52:08.337474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1064","reported_seq":1239,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822915+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:59:23.822915+0000","last_peered":"2023-06-09T11:59:23.822915+0000","last_clean":"2023-06-09T11:59:23.822915+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:59:23.822915+0000","last_undegraded":"2023-06-09T11:59:23.822915+0000","last_fullsized":"2023-06-09T11:59:23.822915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1064,"log_dups_size":0,"ondisk_log_size":1064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:47:25.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180294+0000","last_change":"2023-06-09T11:18:26.342753+0000","last_active":"2023-06-09T11:59:29.180294+0000","last_peered":"2023-06-09T11:59:29.180294+0000","last_clean":"2023-06-09T11:59:29.180294+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:29.180294+0000","last_undegraded":"2023-06-09T11:59:29.180294+0000","last_fullsized":"2023-06-09T11:59:29.180294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:36.852596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824479+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:23.824479+0000","last_peered":"2023-06-09T11:59:23.824479+0000","last_clean":"2023-06-09T11:59:23.824479+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:23.824479+0000","last_undegraded":"2023-06-09T11:59:23.824479+0000","last_fullsized":"2023-06-09T11:59:23.824479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:00:20.027849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3202,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.182579+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:29.182579+0000","last_peered":"2023-06-09T11:59:29.182579+0000","last_clean":"2023-06-09T11:59:29.182579+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:29.182579+0000","last_undegraded":"2023-06-09T11:59:29.182579+0000","last_fullsized":"2023-06-09T11:59:29.182579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:14.768104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.811217+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:17.811217+0000","last_peered":"2023-06-09T11:59:17.811217+0000","last_clean":"2023-06-09T11:59:17.811217+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:17.811217+0000","last_undegraded":"2023-06-09T11:59:17.811217+0000","last_fullsized":"2023-06-09T11:59:17.811217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:02:12.084788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198203+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:29.198203+0000","last_peered":"2023-06-09T11:59:29.198203+0000","last_clean":"2023-06-09T11:59:29.198203+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:29.198203+0000","last_undegraded":"2023-06-09T11:59:29.198203+0000","last_fullsized":"2023-06-09T11:59:29.198203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:18:57.401435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.836981+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:59:17.836981+0000","last_peered":"2023-06-09T11:59:17.836981+0000","last_clean":"2023-06-09T11:59:17.836981+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:17.836981+0000","last_undegraded":"2023-06-09T11:59:17.836981+0000","last_fullsized":"2023-06-09T11:59:17.836981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:00.401114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813812+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:17.813812+0000","last_peered":"2023-06-09T11:59:17.813812+0000","last_clean":"2023-06-09T11:59:17.813812+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:17.813812+0000","last_undegraded":"2023-06-09T11:59:17.813812+0000","last_fullsized":"2023-06-09T11:59:17.813812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:04.925798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198730+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198730+0000","last_peered":"2023-06-09T11:59:29.198730+0000","last_clean":"2023-06-09T11:59:29.198730+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198730+0000","last_undegraded":"2023-06-09T11:59:29.198730+0000","last_fullsized":"2023-06-09T11:59:29.198730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:20:09.665610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813891+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:59:17.813891+0000","last_peered":"2023-06-09T11:59:17.813891+0000","last_clean":"2023-06-09T11:59:17.813891+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:17.813891+0000","last_undegraded":"2023-06-09T11:59:17.813891+0000","last_fullsized":"2023-06-09T11:59:17.813891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:44.339799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":895,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826018+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:23.826018+0000","last_peered":"2023-06-09T11:59:23.826018+0000","last_clean":"2023-06-09T11:59:23.826018+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:23.826018+0000","last_undegraded":"2023-06-09T11:59:23.826018+0000","last_fullsized":"2023-06-09T11:59:23.826018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:11.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3887,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203580+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203580+0000","last_peered":"2023-06-09T11:59:29.203580+0000","last_clean":"2023-06-09T11:59:29.203580+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203580+0000","last_undegraded":"2023-06-09T11:59:29.203580+0000","last_fullsized":"2023-06-09T11:59:29.203580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:03:36.679974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.847101+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:59:17.847101+0000","last_peered":"2023-06-09T11:59:17.847101+0000","last_clean":"2023-06-09T11:59:17.847101+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:17.847101+0000","last_undegraded":"2023-06-09T11:59:17.847101+0000","last_fullsized":"2023-06-09T11:59:17.847101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:05:14.595589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.821723+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.821723+0000","last_peered":"2023-06-09T11:59:23.821723+0000","last_clean":"2023-06-09T11:59:23.821723+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.821723+0000","last_undegraded":"2023-06-09T11:59:23.821723+0000","last_fullsized":"2023-06-09T11:59:23.821723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:56.504880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094605+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.094605+0000","last_peered":"2023-06-09T11:59:30.094605+0000","last_clean":"2023-06-09T11:59:30.094605+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.094605+0000","last_undegraded":"2023-06-09T11:59:30.094605+0000","last_fullsized":"2023-06-09T11:59:30.094605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:11:45.496865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814030+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:59:17.814030+0000","last_peered":"2023-06-09T11:59:17.814030+0000","last_clean":"2023-06-09T11:59:17.814030+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:17.814030+0000","last_undegraded":"2023-06-09T11:59:17.814030+0000","last_fullsized":"2023-06-09T11:59:17.814030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:17.409970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810193+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:59:17.810193+0000","last_peered":"2023-06-09T11:59:17.810193+0000","last_clean":"2023-06-09T11:59:17.810193+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:17.810193+0000","last_undegraded":"2023-06-09T11:59:17.810193+0000","last_fullsized":"2023-06-09T11:59:17.810193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:47:29.671673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1140","reported_seq":1323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824017+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:59:23.824017+0000","last_peered":"2023-06-09T11:59:23.824017+0000","last_clean":"2023-06-09T11:59:23.824017+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:59:23.824017+0000","last_undegraded":"2023-06-09T11:59:23.824017+0000","last_fullsized":"2023-06-09T11:59:23.824017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:27:21.609537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3295,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142633+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142633+0000","last_peered":"2023-06-09T11:59:29.142633+0000","last_clean":"2023-06-09T11:59:29.142633+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142633+0000","last_undegraded":"2023-06-09T11:59:29.142633+0000","last_fullsized":"2023-06-09T11:59:29.142633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:51:16.604757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810429+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:59:17.810429+0000","last_peered":"2023-06-09T11:59:17.810429+0000","last_clean":"2023-06-09T11:59:17.810429+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:17.810429+0000","last_undegraded":"2023-06-09T11:59:17.810429+0000","last_fullsized":"2023-06-09T11:59:17.810429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:44:14.620706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2984,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099630+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.099630+0000","last_peered":"2023-06-09T11:59:30.099630+0000","last_clean":"2023-06-09T11:59:30.099630+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.099630+0000","last_undegraded":"2023-06-09T11:59:30.099630+0000","last_fullsized":"2023-06-09T11:59:30.099630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:27:13.002268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185314+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185314+0000","last_peered":"2023-06-09T11:59:29.185314+0000","last_clean":"2023-06-09T11:59:29.185314+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185314+0000","last_undegraded":"2023-06-09T11:59:29.185314+0000","last_fullsized":"2023-06-09T11:59:29.185314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:05.482688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810999+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:59:17.810999+0000","last_peered":"2023-06-09T11:59:17.810999+0000","last_clean":"2023-06-09T11:59:17.810999+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:59:17.810999+0000","last_undegraded":"2023-06-09T11:59:17.810999+0000","last_fullsized":"2023-06-09T11:59:17.810999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:10.967583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823044+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823044+0000","last_peered":"2023-06-09T11:59:23.823044+0000","last_clean":"2023-06-09T11:59:23.823044+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823044+0000","last_undegraded":"2023-06-09T11:59:23.823044+0000","last_fullsized":"2023-06-09T11:59:23.823044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:37.692810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4110,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142122+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:29.142122+0000","last_peered":"2023-06-09T11:59:29.142122+0000","last_clean":"2023-06-09T11:59:29.142122+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:29.142122+0000","last_undegraded":"2023-06-09T11:59:29.142122+0000","last_fullsized":"2023-06-09T11:59:29.142122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:00:06.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105019+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.105019+0000","last_peered":"2023-06-09T11:59:30.105019+0000","last_clean":"2023-06-09T11:59:30.105019+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.105019+0000","last_undegraded":"2023-06-09T11:59:30.105019+0000","last_fullsized":"2023-06-09T11:59:30.105019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:59:20.477510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.839493+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:59:17.839493+0000","last_peered":"2023-06-09T11:59:17.839493+0000","last_clean":"2023-06-09T11:59:17.839493+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:17.839493+0000","last_undegraded":"2023-06-09T11:59:17.839493+0000","last_fullsized":"2023-06-09T11:59:17.839493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:09:23.586799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208331+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208331+0000","last_peered":"2023-06-09T11:59:29.208331+0000","last_clean":"2023-06-09T11:59:29.208331+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208331+0000","last_undegraded":"2023-06-09T11:59:29.208331+0000","last_fullsized":"2023-06-09T11:59:29.208331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:58.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.819642+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:59:17.819642+0000","last_peered":"2023-06-09T11:59:17.819642+0000","last_clean":"2023-06-09T11:59:17.819642+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:17.819642+0000","last_undegraded":"2023-06-09T11:59:17.819642+0000","last_fullsized":"2023-06-09T11:59:17.819642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:01.435357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'645","reported_seq":824,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086059+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:30.086059+0000","last_peered":"2023-06-09T11:59:30.086059+0000","last_clean":"2023-06-09T11:59:30.086059+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:30.086059+0000","last_undegraded":"2023-06-09T11:59:30.086059+0000","last_fullsized":"2023-06-09T11:59:30.086059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:33:14.865876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1041,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3713,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170033+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170033+0000","last_peered":"2023-06-09T11:59:29.170033+0000","last_clean":"2023-06-09T11:59:29.170033+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170033+0000","last_undegraded":"2023-06-09T11:59:29.170033+0000","last_fullsized":"2023-06-09T11:59:29.170033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:28:31.292243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":879,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086836+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.086836+0000","last_peered":"2023-06-09T11:59:30.086836+0000","last_clean":"2023-06-09T11:59:30.086836+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.086836+0000","last_undegraded":"2023-06-09T11:59:30.086836+0000","last_fullsized":"2023-06-09T11:59:30.086836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:03:06.187539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'955","reported_seq":4519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095034+0000","last_change":"2023-06-09T11:18:26.261803+0000","last_active":"2023-06-09T11:59:30.095034+0000","last_peered":"2023-06-09T11:59:30.095034+0000","last_clean":"2023-06-09T11:59:30.095034+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:59:30.095034+0000","last_undegraded":"2023-06-09T11:59:30.095034+0000","last_fullsized":"2023-06-09T11:59:30.095034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:10:57.891708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":765,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.090216+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.090216+0000","last_peered":"2023-06-09T11:59:30.090216+0000","last_clean":"2023-06-09T11:59:30.090216+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.090216+0000","last_undegraded":"2023-06-09T11:59:30.090216+0000","last_fullsized":"2023-06-09T11:59:30.090216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:31.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178756+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.178756+0000","last_peered":"2023-06-09T11:59:29.178756+0000","last_clean":"2023-06-09T11:59:29.178756+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.178756+0000","last_undegraded":"2023-06-09T11:59:29.178756+0000","last_fullsized":"2023-06-09T11:59:29.178756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:09:06.669625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'554","reported_seq":747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.461722+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:30.461722+0000","last_peered":"2023-06-09T11:59:30.461722+0000","last_clean":"2023-06-09T11:59:30.461722+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:30.461722+0000","last_undegraded":"2023-06-09T11:59:30.461722+0000","last_fullsized":"2023-06-09T11:59:30.461722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:38.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22835015,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":480,"num_write_kb":26733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3673,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179771+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:29.179771+0000","last_peered":"2023-06-09T11:59:29.179771+0000","last_clean":"2023-06-09T11:59:29.179771+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:29.179771+0000","last_undegraded":"2023-06-09T11:59:29.179771+0000","last_fullsized":"2023-06-09T11:59:29.179771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:10:40.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206896+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:59:29.206896+0000","last_peered":"2023-06-09T11:59:29.206896+0000","last_clean":"2023-06-09T11:59:29.206896+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:59:29.206896+0000","last_undegraded":"2023-06-09T11:59:29.206896+0000","last_fullsized":"2023-06-09T11:59:29.206896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:57.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3147,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206536+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.206536+0000","last_peered":"2023-06-09T11:59:29.206536+0000","last_clean":"2023-06-09T11:59:29.206536+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.206536+0000","last_undegraded":"2023-06-09T11:59:29.206536+0000","last_fullsized":"2023-06-09T11:59:29.206536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:49:35.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"73'1383","reported_seq":1600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.817828+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:59:23.817828+0000","last_peered":"2023-06-09T11:59:23.817828+0000","last_clean":"2023-06-09T11:59:23.817828+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:59:23.817828+0000","last_undegraded":"2023-06-09T11:59:23.817828+0000","last_fullsized":"2023-06-09T11:59:23.817828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:25:09.505525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187254+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:29.187254+0000","last_peered":"2023-06-09T11:59:29.187254+0000","last_clean":"2023-06-09T11:59:29.187254+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:29.187254+0000","last_undegraded":"2023-06-09T11:59:29.187254+0000","last_fullsized":"2023-06-09T11:59:29.187254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:17:10.649718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120227+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120227+0000","last_peered":"2023-06-09T11:59:29.120227+0000","last_clean":"2023-06-09T11:59:29.120227+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120227+0000","last_undegraded":"2023-06-09T11:59:29.120227+0000","last_fullsized":"2023-06-09T11:59:29.120227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:29:14.112446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207330+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.207330+0000","last_peered":"2023-06-09T11:59:29.207330+0000","last_clean":"2023-06-09T11:59:29.207330+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.207330+0000","last_undegraded":"2023-06-09T11:59:29.207330+0000","last_fullsized":"2023-06-09T11:59:29.207330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:56:35.301385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187085+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187085+0000","last_peered":"2023-06-09T11:59:29.187085+0000","last_clean":"2023-06-09T11:59:29.187085+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187085+0000","last_undegraded":"2023-06-09T11:59:29.187085+0000","last_fullsized":"2023-06-09T11:59:29.187085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:25.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2977,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180708+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.180708+0000","last_peered":"2023-06-09T11:59:29.180708+0000","last_clean":"2023-06-09T11:59:29.180708+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.180708+0000","last_undegraded":"2023-06-09T11:59:29.180708+0000","last_fullsized":"2023-06-09T11:59:29.180708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:36:34.824605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2991,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207551+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207551+0000","last_peered":"2023-06-09T11:59:29.207551+0000","last_clean":"2023-06-09T11:59:29.207551+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207551+0000","last_undegraded":"2023-06-09T11:59:29.207551+0000","last_fullsized":"2023-06-09T11:59:29.207551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:28.927370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58085,"ondisk_log_size":58085,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":507052263,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29923,"num_write_kb":668620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":330,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":288800768,"data_stored":1014104526,"data_compressed":126302064,"data_compressed_allocated":166912000,"data_compressed_original":892370944,"omap_allocated":0,"internal_metadata":0},"log_size":30468,"ondisk_log_size":30468,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575437,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253804,"kb_used_data":179492,"kb_used_omap":0,"kb_used_meta":74304,"kb_avail":94118036,"statfs":{"total":96636764160,"available":96376868864,"internally_reserved":0,"allocated":183799808,"data_stored":309369991,"data_compressed":31721492,"data_compressed_allocated":39276544,"data_compressed_original":169447424,"omap_allocated":0,"internal_metadata":76087296},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.379,"15min":0.379},"max":{"1min":0.84999999999999998,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.504}]}]},{"osd":6,"up_from":13,"seq":55834575437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.83299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575437,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396016,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89600,"kb_avail":93975824,"statfs":{"total":96636764160,"available":96231243776,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91750400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":1.014}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.97399999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.85899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575437,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327400,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99584,"kb_avail":94044440,"statfs":{"total":96636764160,"available":96301506560,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101974016},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.85599999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":1.0900000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.30599999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.308},"max":{"1min":0.871,"5min":0.871,"15min":1.2909999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.1060000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97499999999999998,"5min":1.21,"15min":1.21},"last":0.745}]},{"osd":5,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0429999999999999,"5min":1.05,"15min":1.236},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.165},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.3700000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":1.159,"15min":1.228},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83299999999999996,"15min":0.79600000000000004},"min":{"1min":0.64200000000000002,"5min":0.52400000000000002,"15min":0.46999999999999997},"max":{"1min":1.0860000000000001,"5min":1.111,"15min":1.29},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232480,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52672,"kb_avail":94139360,"statfs":{"total":96636764160,"available":96398704640,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53936128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.76300000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.501},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.38800000000000001,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":1.367,"15min":1.367},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.546,"15min":1.546},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63300000000000001,"15min":0.59399999999999997},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.55500000000000005,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.0109999999999999,"5min":1.601,"15min":1.601},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84799999999999998,"15min":0.82199999999999995},"min":{"1min":0.71599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.747},"min":{"1min":0.58999999999999997,"5min":0.496,"15min":0.438},"max":{"1min":0.878,"5min":1.2090000000000001,"15min":1.3100000000000001},"last":0.746},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":1.121,"15min":1.3540000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.66100000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.208,"15min":1.248},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.82899999999999996,"15min":0.81000000000000005},"min":{"1min":0.73899999999999999,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.3080000000000001,"15min":1.337},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575437,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.60799999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.80600000000000005,"5min":1.355,"15min":1.355},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76700000000000002,"5min":1.198,"15min":1.198},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":1.635,"15min":1.635},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":2.351,"15min":2.351},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.61899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95399999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.64400000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":0.97599999999999998,"5min":1.367,"15min":1.367},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":1.286,"15min":1.286},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.98499999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.99399999999999999,"5min":1.413,"15min":1.413},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.45,"15min":1.45},"last":0.95999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575436,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412228,"kb_used_data":291964,"kb_used_omap":0,"kb_used_meta":120256,"kb_avail":93959612,"statfs":{"total":96636764160,"available":96214642688,"internally_reserved":0,"allocated":298971136,"data_stored":478232100,"data_compressed":51106947,"data_compressed_allocated":62287872,"data_compressed_original":247705600,"omap_allocated":0,"internal_metadata":123142144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.81999999999999995,"5min":1.137,"15min":1.137},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.83299999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.77900000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.41199999999999998,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.0640000000000001,"15min":1.103},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1279999999999999,"15min":1.2},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":1.609,"15min":1.609},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.47999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.871,"5min":1.159,"15min":1.1659999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.407,"15min":1.407},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.0620000000000001,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":1.046,"5min":1.361,"15min":1.3740000000000001},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575436,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287340,"kb_used_data":216932,"kb_used_omap":0,"kb_used_meta":70400,"kb_avail":94084500,"statfs":{"total":96636764160,"available":96342528000,"internally_reserved":0,"allocated":222138368,"data_stored":324680473,"data_compressed":32076821,"data_compressed_allocated":38850560,"data_compressed_original":145833984,"omap_allocated":0,"internal_metadata":72089600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,8,100,58],"upper_bound":256},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":5.4969999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":5.6269999999999998}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":19.715},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":7.4089999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":5.5620000000000003},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":14.439}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":12.130000000000001},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":12.218999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":67.590000000000003},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":67.521000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":12.163},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":7.6040000000000001}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":3.8039999999999998},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":19.834}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":63250432,"data_stored":227580890,"data_compressed":28660926,"data_compressed_allocated":37535744,"data_compressed_original":201904128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35291136,"data_stored":127693961,"data_compressed":16055263,"data_compressed_allocated":21200896,"data_compressed_original":113623040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42078208,"data_stored":158395181,"data_compressed":20420954,"data_compressed_allocated":26488832,"data_compressed_original":142835712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:59:38.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:59:38.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.485+0000 7fde9e3ff700 1 Processor -- start 2023-06-09T11:59:38.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.486+0000 7fde9e3ff700 1 -- start start 2023-06-09T11:59:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9e3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 0x7fde980b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 0x7fde98159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9e3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde98159b20 con 0x7fde9815cd40 2023-06-09T11:59:38.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9e3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde980cac80 con 0x7fde980cb5d0 2023-06-09T11:59:38.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde981597d0 con 0x7fde980cb9a0 2023-06-09T11:59:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9c99c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fde9815cd40 0x7fde980cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34064/0 (socket says 172.21.15.7:34064) 2023-06-09T11:59:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde9c99c700 1 -- 172.21.15.7:0/1206607039 learned_addr learned my addr 172.21.15.7:0/1206607039 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde977fe700 1 --2- 172.21.15.7:0/1206607039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 0x7fde98159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.487+0000 7fde97fff700 1 --2- 172.21.15.7:0/1206607039 >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 0x7fde980b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181359264 0 0) 0x7fde98159b20 con 0x7fde9815cd40 2023-06-09T11:59:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde800035b0 con 0x7fde9815cd40 2023-06-09T11:59:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde977fe700 1 -- 172.21.15.7:0/1206607039 >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 msgr2=0x7fde980b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde977fe700 1 --2- 172.21.15.7:0/1206607039 >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 0x7fde980b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde977fe700 1 -- 172.21.15.7:0/1206607039 >> v1:172.21.15.7:6789/0 conn(0x7fde9815cd40 legacy=0x7fde980cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259441717 0 0) 0x7fde800035b0 con 0x7fde9815cd40 2023-06-09T11:59:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde977fe700 1 -- 172.21.15.7:0/1206607039 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98071ab0 con 0x7fde980cb9a0 2023-06-09T11:59:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde97fff700 1 --2- 172.21.15.7:0/1206607039 >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 0x7fde980b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.488+0000 7fde977fe700 1 --2- 172.21.15.7:0/1206607039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 0x7fde98159fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009840 tx=0x7fde8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=605258af7b30daa1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900019f0 con 0x7fde980cb9a0 2023-06-09T11:59:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c004030 con 0x7fde980cb9a0 2023-06-09T11:59:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde96ffd700 1 -- 172.21.15.7:0/1206607039 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0043a0 con 0x7fde980cb9a0 2023-06-09T11:59:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 -- 172.21.15.7:0/1206607039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 msgr2=0x7fde98159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/1206607039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 0x7fde98159fb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009840 tx=0x7fde8c009b50 comp rx=0 tx=0).stop 2023-06-09T11:59:38.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 -- 172.21.15.7:0/1206607039 shutdown_connections 2023-06-09T11:59:38.496 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/1206607039 >> v2:172.21.15.117:3300/0 conn(0x7fde980cb5d0 0x7fde980b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/1206607039 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb9a0 0x7fde98159fb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 -- 172.21.15.7:0/1206607039 >> 172.21.15.7:0/1206607039 conn(0x7fde98070af0 msgr2=0x7fde98070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 -- 172.21.15.7:0/1206607039 shutdown_connections 2023-06-09T11:59:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.489+0000 7fde9e3ff700 1 -- 172.21.15.7:0/1206607039 wait complete. 2023-06-09T11:59:38.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.491+0000 7fde9e3ff700 1 Processor -- start 2023-06-09T11:59:38.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.491+0000 7fde9e3ff700 1 -- start start 2023-06-09T11:59:38.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 0x7fde981f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 0x7fde980cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980c6210 con 0x7fde980cb9a0 2023-06-09T11:59:38.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde980cfea0 con 0x7fde9815cd40 2023-06-09T11:59:38.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde9e3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde980ce650 con 0x7fde980cb5d0 2023-06-09T11:59:38.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde977fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38800/0 (socket says 172.21.15.7:38800) 2023-06-09T11:59:38.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.492+0000 7fde977fe700 1 -- 172.21.15.7:0/2051232101 learned_addr learned my addr 172.21.15.7:0/2051232101 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde9c99c700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 0x7fde980cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde97fff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 0x7fde981f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 -- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 msgr2=0x7fde981f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 0x7fde981f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 -- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 msgr2=0x7fde980cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 0x7fde980cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 -- 172.21.15.7:0/2051232101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880094a0 con 0x7fde980cb9a0 2023-06-09T11:59:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde97fff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 0x7fde981f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde9c99c700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 0x7fde980cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde977fe700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fde8c004000 tx=0x7fde8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c004590 con 0x7fde980cb9a0 2023-06-09T11:59:38.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c01f070 con 0x7fde980cb9a0 2023-06-09T11:59:38.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c0094a0 con 0x7fde980cb9a0 2023-06-09T11:59:38.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c011870 con 0x7fde980cb9a0 2023-06-09T11:59:38.502 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.493+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9815d4f0 con 0x7fde980cb9a0 2023-06-09T11:59:38.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.494+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9815d210 con 0x7fde980cb9a0 2023-06-09T11:59:38.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.495+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde8c0119d0 con 0x7fde980cb9a0 2023-06-09T11:59:38.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.496+0000 7fde94ff9700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 0x7fde8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.496+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fde8c021070 con 0x7fde980cb9a0 2023-06-09T11:59:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.496+0000 7fde97fff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 0x7fde8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.497+0000 7fde97fff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 0x7fde8406d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde88009cf0 tx=0x7fde88008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.500+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde8c056400 con 0x7fde980cb9a0 2023-06-09T11:59:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.706+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde9815d7d0 con 0x7fde8406ace0 2023-06-09T11:59:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.715+0000 7fde94ff9700 1 -- 172.21.15.7:0/2051232101 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316295 (secure 0 0 0) 0x7fde9815d7d0 con 0x7fde8406ace0 2023-06-09T11:59:38.721 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 msgr2=0x7fde8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 0x7fde8406d190 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fde88009cf0 tx=0x7fde88008040 comp rx=0 tx=0).stop 2023-06-09T11:59:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 msgr2=0x7fde981f0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fde8c004000 tx=0x7fde8c004750 comp rx=0 tx=0).stop 2023-06-09T11:59:38.723 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 shutdown_connections 2023-06-09T11:59:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde9815cd40 0x7fde980cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde8406ace0 0x7fde8406d190 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde980cb5d0 0x7fde981f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 --2- 172.21.15.7:0/2051232101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde980cb9a0 0x7fde981f0080 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.724 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 >> 172.21.15.7:0/2051232101 conn(0x7fde98070af0 msgr2=0x7fde980b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 shutdown_connections 2023-06-09T11:59:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.719+0000 7fde9e3ff700 1 -- 172.21.15.7:0/2051232101 wait complete. 2023-06-09T11:59:38.725 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:59:38.740 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1535,"stamp":"2023-06-09T11:59:36.925439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1371589134,"num_objects":16489,"num_object_clones":7273,"num_object_copies":32978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16489,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76374,"num_write_kb":1827736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":330,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88670,"ondisk_log_size":88670,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2545384,"kb_used_data":1881704,"kb_used_omap":0,"kb_used_meta":663616,"kb_avail":752429336,"statfs":{"total":773094113280,"available":770487640064,"internally_reserved":0,"allocated":1926864896,"data_stored":2747528768,"data_compressed":268411584,"data_compressed_allocated":324517888,"data_compressed_original":1187856384,"omap_allocated":0,"internal_metadata":679542784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,8,100,58],"upper_bound":256},"perf_stat":{"commit_latency_ms":173,"apply_latency_ms":173,"commit_latency_ns":173000000,"apply_latency_ns":173000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2121907,"num_objects":18,"num_object_clones":22,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":22,"num_read":264,"num_read_kb":1003,"num_write":137,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.176802+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:29.176802+0000","last_peered":"2023-06-09T11:59:29.176802+0000","last_clean":"2023-06-09T11:59:29.176802+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:29.176802+0000","last_undegraded":"2023-06-09T11:59:29.176802+0000","last_fullsized":"2023-06-09T11:59:29.176802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:41.997596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138064+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138064+0000","last_peered":"2023-06-09T11:59:29.138064+0000","last_clean":"2023-06-09T11:59:29.138064+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138064+0000","last_undegraded":"2023-06-09T11:59:29.138064+0000","last_fullsized":"2023-06-09T11:59:29.138064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:43.538089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184905+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.184905+0000","last_peered":"2023-06-09T11:59:29.184905+0000","last_clean":"2023-06-09T11:59:29.184905+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.184905+0000","last_undegraded":"2023-06-09T11:59:29.184905+0000","last_fullsized":"2023-06-09T11:59:29.184905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:54.053113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183976+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:29.183976+0000","last_peered":"2023-06-09T11:59:29.183976+0000","last_clean":"2023-06-09T11:59:29.183976+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:29.183976+0000","last_undegraded":"2023-06-09T11:59:29.183976+0000","last_fullsized":"2023-06-09T11:59:29.183976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:12.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179117+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179117+0000","last_peered":"2023-06-09T11:59:29.179117+0000","last_clean":"2023-06-09T11:59:29.179117+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179117+0000","last_undegraded":"2023-06-09T11:59:29.179117+0000","last_fullsized":"2023-06-09T11:59:29.179117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:24:43.823091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'864","reported_seq":3341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174723+0000","last_change":"2023-06-09T11:18:26.314501+0000","last_active":"2023-06-09T11:59:29.174723+0000","last_peered":"2023-06-09T11:59:29.174723+0000","last_clean":"2023-06-09T11:59:29.174723+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:59:29.174723+0000","last_undegraded":"2023-06-09T11:59:29.174723+0000","last_fullsized":"2023-06-09T11:59:29.174723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:01:52.942030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134658+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.134658+0000","last_peered":"2023-06-09T11:59:29.134658+0000","last_clean":"2023-06-09T11:59:29.134658+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.134658+0000","last_undegraded":"2023-06-09T11:59:29.134658+0000","last_fullsized":"2023-06-09T11:59:29.134658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:08.996053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3243,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164252+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164252+0000","last_peered":"2023-06-09T11:59:29.164252+0000","last_clean":"2023-06-09T11:59:29.164252+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164252+0000","last_undegraded":"2023-06-09T11:59:29.164252+0000","last_fullsized":"2023-06-09T11:59:29.164252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:39:08.386702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4197,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183305+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183305+0000","last_peered":"2023-06-09T11:59:29.183305+0000","last_clean":"2023-06-09T11:59:29.183305+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183305+0000","last_undegraded":"2023-06-09T11:59:29.183305+0000","last_fullsized":"2023-06-09T11:59:29.183305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:42:35.512902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163514+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163514+0000","last_peered":"2023-06-09T11:59:29.163514+0000","last_clean":"2023-06-09T11:59:29.163514+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163514+0000","last_undegraded":"2023-06-09T11:59:29.163514+0000","last_fullsized":"2023-06-09T11:59:29.163514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:40.374816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184625+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184625+0000","last_peered":"2023-06-09T11:59:29.184625+0000","last_clean":"2023-06-09T11:59:29.184625+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184625+0000","last_undegraded":"2023-06-09T11:59:29.184625+0000","last_fullsized":"2023-06-09T11:59:29.184625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:26:32.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.166730+0000","last_change":"2023-06-09T11:18:26.271005+0000","last_active":"2023-06-09T11:59:29.166730+0000","last_peered":"2023-06-09T11:59:29.166730+0000","last_clean":"2023-06-09T11:59:29.166730+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:29.166730+0000","last_undegraded":"2023-06-09T11:59:29.166730+0000","last_fullsized":"2023-06-09T11:59:29.166730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:55.416203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108516+0000","last_change":"2023-06-09T11:18:26.253105+0000","last_active":"2023-06-09T11:59:30.108516+0000","last_peered":"2023-06-09T11:59:30.108516+0000","last_clean":"2023-06-09T11:59:30.108516+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:30.108516+0000","last_undegraded":"2023-06-09T11:59:30.108516+0000","last_fullsized":"2023-06-09T11:59:30.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:36:08.467688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108534+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:29.108534+0000","last_peered":"2023-06-09T11:59:29.108534+0000","last_clean":"2023-06-09T11:59:29.108534+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:29.108534+0000","last_undegraded":"2023-06-09T11:59:29.108534+0000","last_fullsized":"2023-06-09T11:59:29.108534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:08:27.844416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134761+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:29.134761+0000","last_peered":"2023-06-09T11:59:29.134761+0000","last_clean":"2023-06-09T11:59:29.134761+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:29.134761+0000","last_undegraded":"2023-06-09T11:59:29.134761+0000","last_fullsized":"2023-06-09T11:59:29.134761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:43:06.239662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3425,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174867+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:29.174867+0000","last_peered":"2023-06-09T11:59:29.174867+0000","last_clean":"2023-06-09T11:59:29.174867+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:29.174867+0000","last_undegraded":"2023-06-09T11:59:29.174867+0000","last_fullsized":"2023-06-09T11:59:29.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:35.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183203+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:29.183203+0000","last_peered":"2023-06-09T11:59:29.183203+0000","last_clean":"2023-06-09T11:59:29.183203+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:29.183203+0000","last_undegraded":"2023-06-09T11:59:29.183203+0000","last_fullsized":"2023-06-09T11:59:29.183203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:37.819209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174315+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:29.174315+0000","last_peered":"2023-06-09T11:59:29.174315+0000","last_clean":"2023-06-09T11:59:29.174315+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:29.174315+0000","last_undegraded":"2023-06-09T11:59:29.174315+0000","last_fullsized":"2023-06-09T11:59:29.174315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:47:55.986286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3553,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108428+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.108428+0000","last_peered":"2023-06-09T11:59:29.108428+0000","last_clean":"2023-06-09T11:59:29.108428+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.108428+0000","last_undegraded":"2023-06-09T11:59:29.108428+0000","last_fullsized":"2023-06-09T11:59:29.108428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:53:59.032131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186654+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:29.186654+0000","last_peered":"2023-06-09T11:59:29.186654+0000","last_clean":"2023-06-09T11:59:29.186654+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:29.186654+0000","last_undegraded":"2023-06-09T11:59:29.186654+0000","last_fullsized":"2023-06-09T11:59:29.186654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:31.924577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.127903+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:29.127903+0000","last_peered":"2023-06-09T11:59:29.127903+0000","last_clean":"2023-06-09T11:59:29.127903+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:29.127903+0000","last_undegraded":"2023-06-09T11:59:29.127903+0000","last_fullsized":"2023-06-09T11:59:29.127903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:53.061847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3479,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.148430+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:29.148430+0000","last_peered":"2023-06-09T11:59:29.148430+0000","last_clean":"2023-06-09T11:59:29.148430+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:29.148430+0000","last_undegraded":"2023-06-09T11:59:29.148430+0000","last_fullsized":"2023-06-09T11:59:29.148430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:33:23.877739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206328+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:29.206328+0000","last_peered":"2023-06-09T11:59:29.206328+0000","last_clean":"2023-06-09T11:59:29.206328+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:29.206328+0000","last_undegraded":"2023-06-09T11:59:29.206328+0000","last_fullsized":"2023-06-09T11:59:29.206328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:31:00.135897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181162+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181162+0000","last_peered":"2023-06-09T11:59:29.181162+0000","last_clean":"2023-06-09T11:59:29.181162+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181162+0000","last_undegraded":"2023-06-09T11:59:29.181162+0000","last_fullsized":"2023-06-09T11:59:29.181162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:46:27.956223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3016,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105429+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.105429+0000","last_peered":"2023-06-09T11:59:30.105429+0000","last_clean":"2023-06-09T11:59:30.105429+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.105429+0000","last_undegraded":"2023-06-09T11:59:30.105429+0000","last_fullsized":"2023-06-09T11:59:30.105429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:07.769640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171191+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171191+0000","last_peered":"2023-06-09T11:59:29.171191+0000","last_clean":"2023-06-09T11:59:29.171191+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171191+0000","last_undegraded":"2023-06-09T11:59:29.171191+0000","last_fullsized":"2023-06-09T11:59:29.171191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:38:31.443299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116586+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116586+0000","last_peered":"2023-06-09T11:59:29.116586+0000","last_clean":"2023-06-09T11:59:29.116586+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116586+0000","last_undegraded":"2023-06-09T11:59:29.116586+0000","last_fullsized":"2023-06-09T11:59:29.116586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:28.791273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.608768+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:29.608768+0000","last_peered":"2023-06-09T11:59:29.608768+0000","last_clean":"2023-06-09T11:59:29.608768+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:29.608768+0000","last_undegraded":"2023-06-09T11:59:29.608768+0000","last_fullsized":"2023-06-09T11:59:29.608768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:54:04.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.159320+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:29.159320+0000","last_peered":"2023-06-09T11:59:29.159320+0000","last_clean":"2023-06-09T11:59:29.159320+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:29.159320+0000","last_undegraded":"2023-06-09T11:59:29.159320+0000","last_fullsized":"2023-06-09T11:59:29.159320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:04:01.139122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604309+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:29.604309+0000","last_peered":"2023-06-09T11:59:29.604309+0000","last_clean":"2023-06-09T11:59:29.604309+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:29.604309+0000","last_undegraded":"2023-06-09T11:59:29.604309+0000","last_fullsized":"2023-06-09T11:59:29.604309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:19.987020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099238+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.099238+0000","last_peered":"2023-06-09T11:59:30.099238+0000","last_clean":"2023-06-09T11:59:30.099238+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.099238+0000","last_undegraded":"2023-06-09T11:59:30.099238+0000","last_fullsized":"2023-06-09T11:59:30.099238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:03.591541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826279+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:23.826279+0000","last_peered":"2023-06-09T11:59:23.826279+0000","last_clean":"2023-06-09T11:59:23.826279+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:23.826279+0000","last_undegraded":"2023-06-09T11:59:23.826279+0000","last_fullsized":"2023-06-09T11:59:23.826279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:19:22.681147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120527+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120527+0000","last_peered":"2023-06-09T11:59:29.120527+0000","last_clean":"2023-06-09T11:59:29.120527+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120527+0000","last_undegraded":"2023-06-09T11:59:29.120527+0000","last_fullsized":"2023-06-09T11:59:29.120527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:15.836484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'180","reported_seq":343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825404+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:59:23.825404+0000","last_peered":"2023-06-09T11:59:23.825404+0000","last_clean":"2023-06-09T11:59:23.825404+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:23.825404+0000","last_undegraded":"2023-06-09T11:59:23.825404+0000","last_fullsized":"2023-06-09T11:59:23.825404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:22:06.545450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":124,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2960,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.204530+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:29.204530+0000","last_peered":"2023-06-09T11:59:29.204530+0000","last_clean":"2023-06-09T11:59:29.204530+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:29.204530+0000","last_undegraded":"2023-06-09T11:59:29.204530+0000","last_fullsized":"2023-06-09T11:59:29.204530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:51:52.478717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813587+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:59:17.813587+0000","last_peered":"2023-06-09T11:59:17.813587+0000","last_clean":"2023-06-09T11:59:17.813587+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:59:17.813587+0000","last_undegraded":"2023-06-09T11:59:17.813587+0000","last_fullsized":"2023-06-09T11:59:17.813587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:40.338566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208409+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:29.208409+0000","last_peered":"2023-06-09T11:59:29.208409+0000","last_clean":"2023-06-09T11:59:29.208409+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:29.208409+0000","last_undegraded":"2023-06-09T11:59:29.208409+0000","last_fullsized":"2023-06-09T11:59:29.208409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:32:56.568860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086513+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:59:30.086513+0000","last_peered":"2023-06-09T11:59:30.086513+0000","last_clean":"2023-06-09T11:59:30.086513+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:30.086513+0000","last_undegraded":"2023-06-09T11:59:30.086513+0000","last_fullsized":"2023-06-09T11:59:30.086513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:11:48.144420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088093+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:30.088093+0000","last_peered":"2023-06-09T11:59:30.088093+0000","last_clean":"2023-06-09T11:59:30.088093+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:30.088093+0000","last_undegraded":"2023-06-09T11:59:30.088093+0000","last_fullsized":"2023-06-09T11:59:30.088093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:43:21.546026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'822","reported_seq":1030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.085204+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:30.085204+0000","last_peered":"2023-06-09T11:59:30.085204+0000","last_clean":"2023-06-09T11:59:30.085204+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:30.085204+0000","last_undegraded":"2023-06-09T11:59:30.085204+0000","last_fullsized":"2023-06-09T11:59:30.085204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:15:45.170741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":767,"num_write_kb":25247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3094,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186545+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:29.186545+0000","last_peered":"2023-06-09T11:59:29.186545+0000","last_clean":"2023-06-09T11:59:29.186545+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:29.186545+0000","last_undegraded":"2023-06-09T11:59:29.186545+0000","last_fullsized":"2023-06-09T11:59:29.186545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:43.603410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'857","reported_seq":1071,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.080210+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:29.080210+0000","last_peered":"2023-06-09T11:59:29.080210+0000","last_clean":"2023-06-09T11:59:29.080210+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:29.080210+0000","last_undegraded":"2023-06-09T11:59:29.080210+0000","last_fullsized":"2023-06-09T11:59:29.080210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:48.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14073316,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":799,"num_write_kb":14332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3556,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.161492+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:29.161492+0000","last_peered":"2023-06-09T11:59:29.161492+0000","last_clean":"2023-06-09T11:59:29.161492+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:29.161492+0000","last_undegraded":"2023-06-09T11:59:29.161492+0000","last_fullsized":"2023-06-09T11:59:29.161492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:48.346810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'413","reported_seq":575,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814193+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:59:17.814193+0000","last_peered":"2023-06-09T11:59:17.814193+0000","last_clean":"2023-06-09T11:59:17.814193+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:17.814193+0000","last_undegraded":"2023-06-09T11:59:17.814193+0000","last_fullsized":"2023-06-09T11:59:17.814193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:03:12.461060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116041+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116041+0000","last_peered":"2023-06-09T11:59:29.116041+0000","last_clean":"2023-06-09T11:59:29.116041+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116041+0000","last_undegraded":"2023-06-09T11:59:29.116041+0000","last_fullsized":"2023-06-09T11:59:29.116041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:24:03.501267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"73'960","reported_seq":1143,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.820047+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:59:23.820047+0000","last_peered":"2023-06-09T11:59:23.820047+0000","last_clean":"2023-06-09T11:59:23.820047+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:59:23.820047+0000","last_undegraded":"2023-06-09T11:59:23.820047+0000","last_fullsized":"2023-06-09T11:59:23.820047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:53:42.964479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":890,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3022,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185110+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185110+0000","last_peered":"2023-06-09T11:59:29.185110+0000","last_clean":"2023-06-09T11:59:29.185110+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185110+0000","last_undegraded":"2023-06-09T11:59:29.185110+0000","last_fullsized":"2023-06-09T11:59:29.185110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:02:58.361435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1054","reported_seq":1302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086379+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:30.086379+0000","last_peered":"2023-06-09T11:59:30.086379+0000","last_clean":"2023-06-09T11:59:30.086379+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:30.086379+0000","last_undegraded":"2023-06-09T11:59:30.086379+0000","last_fullsized":"2023-06-09T11:59:30.086379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:30.247597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1013,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203830+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203830+0000","last_peered":"2023-06-09T11:59:29.203830+0000","last_clean":"2023-06-09T11:59:29.203830+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203830+0000","last_undegraded":"2023-06-09T11:59:29.203830+0000","last_fullsized":"2023-06-09T11:59:29.203830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:33:19.947716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825423+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.825423+0000","last_peered":"2023-06-09T11:59:23.825423+0000","last_clean":"2023-06-09T11:59:23.825423+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.825423+0000","last_undegraded":"2023-06-09T11:59:23.825423+0000","last_fullsized":"2023-06-09T11:59:23.825423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:22:06.716248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113291+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113291+0000","last_peered":"2023-06-09T11:59:29.113291+0000","last_clean":"2023-06-09T11:59:29.113291+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113291+0000","last_undegraded":"2023-06-09T11:59:29.113291+0000","last_fullsized":"2023-06-09T11:59:29.113291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:25:53.990234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.607238+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:29.607238+0000","last_peered":"2023-06-09T11:59:29.607238+0000","last_clean":"2023-06-09T11:59:29.607238+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:29.607238+0000","last_undegraded":"2023-06-09T11:59:29.607238+0000","last_fullsized":"2023-06-09T11:59:29.607238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:10:40.976689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178950+0000","last_change":"2023-06-09T11:18:26.315365+0000","last_active":"2023-06-09T11:59:29.178950+0000","last_peered":"2023-06-09T11:59:29.178950+0000","last_clean":"2023-06-09T11:59:29.178950+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:29.178950+0000","last_undegraded":"2023-06-09T11:59:29.178950+0000","last_fullsized":"2023-06-09T11:59:29.178950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:48:14.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'432","reported_seq":599,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134735+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:29.134735+0000","last_peered":"2023-06-09T11:59:29.134735+0000","last_clean":"2023-06-09T11:59:29.134735+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:29.134735+0000","last_undegraded":"2023-06-09T11:59:29.134735+0000","last_fullsized":"2023-06-09T11:59:29.134735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":432,"log_dups_size":0,"ondisk_log_size":432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:17.698189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":370,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088582+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.088582+0000","last_peered":"2023-06-09T11:59:30.088582+0000","last_clean":"2023-06-09T11:59:30.088582+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.088582+0000","last_undegraded":"2023-06-09T11:59:30.088582+0000","last_fullsized":"2023-06-09T11:59:30.088582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:35:52.399332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"73'517","reported_seq":687,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.819694+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:59:23.819694+0000","last_peered":"2023-06-09T11:59:23.819694+0000","last_clean":"2023-06-09T11:59:23.819694+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:59:23.819694+0000","last_undegraded":"2023-06-09T11:59:23.819694+0000","last_fullsized":"2023-06-09T11:59:23.819694+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:58.048595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":471,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.149039+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:29.149039+0000","last_peered":"2023-06-09T11:59:29.149039+0000","last_clean":"2023-06-09T11:59:29.149039+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:29.149039+0000","last_undegraded":"2023-06-09T11:59:29.149039+0000","last_fullsized":"2023-06-09T11:59:29.149039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:45.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.812549+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:17.812549+0000","last_peered":"2023-06-09T11:59:17.812549+0000","last_clean":"2023-06-09T11:59:17.812549+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:17.812549+0000","last_undegraded":"2023-06-09T11:59:17.812549+0000","last_fullsized":"2023-06-09T11:59:17.812549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:59.524461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105462+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:30.105462+0000","last_peered":"2023-06-09T11:59:30.105462+0000","last_clean":"2023-06-09T11:59:30.105462+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:30.105462+0000","last_undegraded":"2023-06-09T11:59:30.105462+0000","last_fullsized":"2023-06-09T11:59:30.105462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:52:08.337474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1064","reported_seq":1239,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822915+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:59:23.822915+0000","last_peered":"2023-06-09T11:59:23.822915+0000","last_clean":"2023-06-09T11:59:23.822915+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:59:23.822915+0000","last_undegraded":"2023-06-09T11:59:23.822915+0000","last_fullsized":"2023-06-09T11:59:23.822915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1064,"log_dups_size":0,"ondisk_log_size":1064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:47:25.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180294+0000","last_change":"2023-06-09T11:18:26.342753+0000","last_active":"2023-06-09T11:59:29.180294+0000","last_peered":"2023-06-09T11:59:29.180294+0000","last_clean":"2023-06-09T11:59:29.180294+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:29.180294+0000","last_undegraded":"2023-06-09T11:59:29.180294+0000","last_fullsized":"2023-06-09T11:59:29.180294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:36.852596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824479+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:23.824479+0000","last_peered":"2023-06-09T11:59:23.824479+0000","last_clean":"2023-06-09T11:59:23.824479+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:23.824479+0000","last_undegraded":"2023-06-09T11:59:23.824479+0000","last_fullsized":"2023-06-09T11:59:23.824479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:00:20.027849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3202,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.182579+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:29.182579+0000","last_peered":"2023-06-09T11:59:29.182579+0000","last_clean":"2023-06-09T11:59:29.182579+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:29.182579+0000","last_undegraded":"2023-06-09T11:59:29.182579+0000","last_fullsized":"2023-06-09T11:59:29.182579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:14.768104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.811217+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:17.811217+0000","last_peered":"2023-06-09T11:59:17.811217+0000","last_clean":"2023-06-09T11:59:17.811217+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:17.811217+0000","last_undegraded":"2023-06-09T11:59:17.811217+0000","last_fullsized":"2023-06-09T11:59:17.811217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:02:12.084788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198203+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:29.198203+0000","last_peered":"2023-06-09T11:59:29.198203+0000","last_clean":"2023-06-09T11:59:29.198203+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:29.198203+0000","last_undegraded":"2023-06-09T11:59:29.198203+0000","last_fullsized":"2023-06-09T11:59:29.198203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:18:57.401435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.836981+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:59:17.836981+0000","last_peered":"2023-06-09T11:59:17.836981+0000","last_clean":"2023-06-09T11:59:17.836981+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:17.836981+0000","last_undegraded":"2023-06-09T11:59:17.836981+0000","last_fullsized":"2023-06-09T11:59:17.836981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:00.401114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813812+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:17.813812+0000","last_peered":"2023-06-09T11:59:17.813812+0000","last_clean":"2023-06-09T11:59:17.813812+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:17.813812+0000","last_undegraded":"2023-06-09T11:59:17.813812+0000","last_fullsized":"2023-06-09T11:59:17.813812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:04.925798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198730+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198730+0000","last_peered":"2023-06-09T11:59:29.198730+0000","last_clean":"2023-06-09T11:59:29.198730+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198730+0000","last_undegraded":"2023-06-09T11:59:29.198730+0000","last_fullsized":"2023-06-09T11:59:29.198730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:20:09.665610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813891+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:59:17.813891+0000","last_peered":"2023-06-09T11:59:17.813891+0000","last_clean":"2023-06-09T11:59:17.813891+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:17.813891+0000","last_undegraded":"2023-06-09T11:59:17.813891+0000","last_fullsized":"2023-06-09T11:59:17.813891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:44.339799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":895,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826018+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:23.826018+0000","last_peered":"2023-06-09T11:59:23.826018+0000","last_clean":"2023-06-09T11:59:23.826018+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:23.826018+0000","last_undegraded":"2023-06-09T11:59:23.826018+0000","last_fullsized":"2023-06-09T11:59:23.826018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:11.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3887,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203580+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203580+0000","last_peered":"2023-06-09T11:59:29.203580+0000","last_clean":"2023-06-09T11:59:29.203580+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203580+0000","last_undegraded":"2023-06-09T11:59:29.203580+0000","last_fullsized":"2023-06-09T11:59:29.203580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:03:36.679974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.847101+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:59:17.847101+0000","last_peered":"2023-06-09T11:59:17.847101+0000","last_clean":"2023-06-09T11:59:17.847101+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:17.847101+0000","last_undegraded":"2023-06-09T11:59:17.847101+0000","last_fullsized":"2023-06-09T11:59:17.847101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:05:14.595589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.821723+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.821723+0000","last_peered":"2023-06-09T11:59:23.821723+0000","last_clean":"2023-06-09T11:59:23.821723+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.821723+0000","last_undegraded":"2023-06-09T11:59:23.821723+0000","last_fullsized":"2023-06-09T11:59:23.821723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:56.504880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094605+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.094605+0000","last_peered":"2023-06-09T11:59:30.094605+0000","last_clean":"2023-06-09T11:59:30.094605+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.094605+0000","last_undegraded":"2023-06-09T11:59:30.094605+0000","last_fullsized":"2023-06-09T11:59:30.094605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:11:45.496865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814030+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:59:17.814030+0000","last_peered":"2023-06-09T11:59:17.814030+0000","last_clean":"2023-06-09T11:59:17.814030+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:17.814030+0000","last_undegraded":"2023-06-09T11:59:17.814030+0000","last_fullsized":"2023-06-09T11:59:17.814030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:17.409970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810193+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:59:17.810193+0000","last_peered":"2023-06-09T11:59:17.810193+0000","last_clean":"2023-06-09T11:59:17.810193+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:17.810193+0000","last_undegraded":"2023-06-09T11:59:17.810193+0000","last_fullsized":"2023-06-09T11:59:17.810193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:47:29.671673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1140","reported_seq":1323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824017+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:59:23.824017+0000","last_peered":"2023-06-09T11:59:23.824017+0000","last_clean":"2023-06-09T11:59:23.824017+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:59:23.824017+0000","last_undegraded":"2023-06-09T11:59:23.824017+0000","last_fullsized":"2023-06-09T11:59:23.824017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:27:21.609537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3295,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142633+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142633+0000","last_peered":"2023-06-09T11:59:29.142633+0000","last_clean":"2023-06-09T11:59:29.142633+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142633+0000","last_undegraded":"2023-06-09T11:59:29.142633+0000","last_fullsized":"2023-06-09T11:59:29.142633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:51:16.604757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810429+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:59:17.810429+0000","last_peered":"2023-06-09T11:59:17.810429+0000","last_clean":"2023-06-09T11:59:17.810429+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:17.810429+0000","last_undegraded":"2023-06-09T11:59:17.810429+0000","last_fullsized":"2023-06-09T11:59:17.810429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:44:14.620706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2984,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099630+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.099630+0000","last_peered":"2023-06-09T11:59:30.099630+0000","last_clean":"2023-06-09T11:59:30.099630+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.099630+0000","last_undegraded":"2023-06-09T11:59:30.099630+0000","last_fullsized":"2023-06-09T11:59:30.099630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:27:13.002268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185314+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185314+0000","last_peered":"2023-06-09T11:59:29.185314+0000","last_clean":"2023-06-09T11:59:29.185314+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185314+0000","last_undegraded":"2023-06-09T11:59:29.185314+0000","last_fullsized":"2023-06-09T11:59:29.185314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:05.482688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810999+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:59:17.810999+0000","last_peered":"2023-06-09T11:59:17.810999+0000","last_clean":"2023-06-09T11:59:17.810999+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:59:17.810999+0000","last_undegraded":"2023-06-09T11:59:17.810999+0000","last_fullsized":"2023-06-09T11:59:17.810999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:10.967583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823044+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823044+0000","last_peered":"2023-06-09T11:59:23.823044+0000","last_clean":"2023-06-09T11:59:23.823044+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823044+0000","last_undegraded":"2023-06-09T11:59:23.823044+0000","last_fullsized":"2023-06-09T11:59:23.823044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:37.692810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4110,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142122+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:29.142122+0000","last_peered":"2023-06-09T11:59:29.142122+0000","last_clean":"2023-06-09T11:59:29.142122+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:29.142122+0000","last_undegraded":"2023-06-09T11:59:29.142122+0000","last_fullsized":"2023-06-09T11:59:29.142122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:00:06.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105019+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.105019+0000","last_peered":"2023-06-09T11:59:30.105019+0000","last_clean":"2023-06-09T11:59:30.105019+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.105019+0000","last_undegraded":"2023-06-09T11:59:30.105019+0000","last_fullsized":"2023-06-09T11:59:30.105019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:59:20.477510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.839493+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:59:17.839493+0000","last_peered":"2023-06-09T11:59:17.839493+0000","last_clean":"2023-06-09T11:59:17.839493+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:17.839493+0000","last_undegraded":"2023-06-09T11:59:17.839493+0000","last_fullsized":"2023-06-09T11:59:17.839493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:09:23.586799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208331+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208331+0000","last_peered":"2023-06-09T11:59:29.208331+0000","last_clean":"2023-06-09T11:59:29.208331+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208331+0000","last_undegraded":"2023-06-09T11:59:29.208331+0000","last_fullsized":"2023-06-09T11:59:29.208331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:58.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.819642+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:59:17.819642+0000","last_peered":"2023-06-09T11:59:17.819642+0000","last_clean":"2023-06-09T11:59:17.819642+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:17.819642+0000","last_undegraded":"2023-06-09T11:59:17.819642+0000","last_fullsized":"2023-06-09T11:59:17.819642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:01.435357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'645","reported_seq":824,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086059+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:30.086059+0000","last_peered":"2023-06-09T11:59:30.086059+0000","last_clean":"2023-06-09T11:59:30.086059+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:30.086059+0000","last_undegraded":"2023-06-09T11:59:30.086059+0000","last_fullsized":"2023-06-09T11:59:30.086059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":645,"log_dups_size":0,"ondisk_log_size":645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:33:14.865876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1041,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3713,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170033+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170033+0000","last_peered":"2023-06-09T11:59:29.170033+0000","last_clean":"2023-06-09T11:59:29.170033+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170033+0000","last_undegraded":"2023-06-09T11:59:29.170033+0000","last_fullsized":"2023-06-09T11:59:29.170033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:28:31.292243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":879,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086836+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.086836+0000","last_peered":"2023-06-09T11:59:30.086836+0000","last_clean":"2023-06-09T11:59:30.086836+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.086836+0000","last_undegraded":"2023-06-09T11:59:30.086836+0000","last_fullsized":"2023-06-09T11:59:30.086836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:03:06.187539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'955","reported_seq":4519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095034+0000","last_change":"2023-06-09T11:18:26.261803+0000","last_active":"2023-06-09T11:59:30.095034+0000","last_peered":"2023-06-09T11:59:30.095034+0000","last_clean":"2023-06-09T11:59:30.095034+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:59:30.095034+0000","last_undegraded":"2023-06-09T11:59:30.095034+0000","last_fullsized":"2023-06-09T11:59:30.095034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:10:57.891708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":765,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.090216+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.090216+0000","last_peered":"2023-06-09T11:59:30.090216+0000","last_clean":"2023-06-09T11:59:30.090216+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.090216+0000","last_undegraded":"2023-06-09T11:59:30.090216+0000","last_fullsized":"2023-06-09T11:59:30.090216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:31.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178756+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.178756+0000","last_peered":"2023-06-09T11:59:29.178756+0000","last_clean":"2023-06-09T11:59:29.178756+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.178756+0000","last_undegraded":"2023-06-09T11:59:29.178756+0000","last_fullsized":"2023-06-09T11:59:29.178756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:09:06.669625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'554","reported_seq":747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.461722+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:30.461722+0000","last_peered":"2023-06-09T11:59:30.461722+0000","last_clean":"2023-06-09T11:59:30.461722+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:30.461722+0000","last_undegraded":"2023-06-09T11:59:30.461722+0000","last_fullsized":"2023-06-09T11:59:30.461722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:38.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22835015,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":480,"num_write_kb":26733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3673,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179771+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:29.179771+0000","last_peered":"2023-06-09T11:59:29.179771+0000","last_clean":"2023-06-09T11:59:29.179771+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:29.179771+0000","last_undegraded":"2023-06-09T11:59:29.179771+0000","last_fullsized":"2023-06-09T11:59:29.179771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:10:40.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206896+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:59:29.206896+0000","last_peered":"2023-06-09T11:59:29.206896+0000","last_clean":"2023-06-09T11:59:29.206896+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:59:29.206896+0000","last_undegraded":"2023-06-09T11:59:29.206896+0000","last_fullsized":"2023-06-09T11:59:29.206896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:57.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3147,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206536+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.206536+0000","last_peered":"2023-06-09T11:59:29.206536+0000","last_clean":"2023-06-09T11:59:29.206536+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.206536+0000","last_undegraded":"2023-06-09T11:59:29.206536+0000","last_fullsized":"2023-06-09T11:59:29.206536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:49:35.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"73'1383","reported_seq":1600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.817828+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:59:23.817828+0000","last_peered":"2023-06-09T11:59:23.817828+0000","last_clean":"2023-06-09T11:59:23.817828+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:59:23.817828+0000","last_undegraded":"2023-06-09T11:59:23.817828+0000","last_fullsized":"2023-06-09T11:59:23.817828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:25:09.505525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187254+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:29.187254+0000","last_peered":"2023-06-09T11:59:29.187254+0000","last_clean":"2023-06-09T11:59:29.187254+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:29.187254+0000","last_undegraded":"2023-06-09T11:59:29.187254+0000","last_fullsized":"2023-06-09T11:59:29.187254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:17:10.649718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120227+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120227+0000","last_peered":"2023-06-09T11:59:29.120227+0000","last_clean":"2023-06-09T11:59:29.120227+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120227+0000","last_undegraded":"2023-06-09T11:59:29.120227+0000","last_fullsized":"2023-06-09T11:59:29.120227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:29:14.112446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207330+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.207330+0000","last_peered":"2023-06-09T11:59:29.207330+0000","last_clean":"2023-06-09T11:59:29.207330+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.207330+0000","last_undegraded":"2023-06-09T11:59:29.207330+0000","last_fullsized":"2023-06-09T11:59:29.207330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:56:35.301385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187085+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187085+0000","last_peered":"2023-06-09T11:59:29.187085+0000","last_clean":"2023-06-09T11:59:29.187085+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187085+0000","last_undegraded":"2023-06-09T11:59:29.187085+0000","last_fullsized":"2023-06-09T11:59:29.187085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:25.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2977,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180708+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.180708+0000","last_peered":"2023-06-09T11:59:29.180708+0000","last_clean":"2023-06-09T11:59:29.180708+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.180708+0000","last_undegraded":"2023-06-09T11:59:29.180708+0000","last_fullsized":"2023-06-09T11:59:29.180708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:36:34.824605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2991,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207551+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207551+0000","last_peered":"2023-06-09T11:59:29.207551+0000","last_clean":"2023-06-09T11:59:29.207551+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207551+0000","last_undegraded":"2023-06-09T11:59:29.207551+0000","last_fullsized":"2023-06-09T11:59:29.207551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:28.927370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58085,"ondisk_log_size":58085,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":507052263,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29923,"num_write_kb":668620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":330,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":288800768,"data_stored":1014104526,"data_compressed":126302064,"data_compressed_allocated":166912000,"data_compressed_original":892370944,"omap_allocated":0,"internal_metadata":0},"log_size":30468,"ondisk_log_size":30468,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575437,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253804,"kb_used_data":179492,"kb_used_omap":0,"kb_used_meta":74304,"kb_avail":94118036,"statfs":{"total":96636764160,"available":96376868864,"internally_reserved":0,"allocated":183799808,"data_stored":309369991,"data_compressed":31721492,"data_compressed_allocated":39276544,"data_compressed_original":169447424,"omap_allocated":0,"internal_metadata":76087296},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.67400000000000004}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:58:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.379,"15min":0.379},"max":{"1min":0.84999999999999998,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.55000000000000004,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.93400000000000005,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.61899999999999999}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.40100000000000002}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.504}]}]},{"osd":6,"up_from":13,"seq":55834575437,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.83299999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575437,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396016,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89600,"kb_avail":93975824,"statfs":{"total":96636764160,"available":96231243776,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91750400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":1.014}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.97399999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.85899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575437,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327400,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99584,"kb_avail":94044440,"statfs":{"total":96636764160,"available":96301506560,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101974016},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.85599999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":1.0900000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.30599999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.308},"max":{"1min":0.871,"5min":0.871,"15min":1.2909999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.1060000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97499999999999998,"5min":1.21,"15min":1.21},"last":0.745}]},{"osd":5,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0429999999999999,"5min":1.05,"15min":1.236},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.165},"last":0.72099999999999997}]},{"osd":6,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.3700000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":1.159,"15min":1.228},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83299999999999996,"15min":0.79600000000000004},"min":{"1min":0.64200000000000002,"5min":0.52400000000000002,"15min":0.46999999999999997},"max":{"1min":1.0860000000000001,"5min":1.111,"15min":1.29},"last":0.79100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232480,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52672,"kb_avail":94139360,"statfs":{"total":96636764160,"available":96398704640,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53936128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.76300000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.501},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.38800000000000001,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":1.367,"15min":1.367},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.546,"15min":1.546},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63300000000000001,"15min":0.59399999999999997},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.55500000000000005,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.0109999999999999,"5min":1.601,"15min":1.601},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84799999999999998,"15min":0.82199999999999995},"min":{"1min":0.71599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.88900000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.747},"min":{"1min":0.58999999999999997,"5min":0.496,"15min":0.438},"max":{"1min":0.878,"5min":1.2090000000000001,"15min":1.3100000000000001},"last":0.746},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":1.121,"15min":1.3540000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.66100000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.208,"15min":1.248},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.872,"5min":0.82899999999999996,"15min":0.81000000000000005},"min":{"1min":0.73899999999999999,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.3080000000000001,"15min":1.337},"last":0.85099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575437,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.60799999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.80600000000000005,"5min":1.355,"15min":1.355},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76700000000000002,"5min":1.198,"15min":1.198},"last":0.72999999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":1.635,"15min":1.635},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":2.351,"15min":2.351},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.61899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95399999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.64400000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":0.97599999999999998,"5min":1.367,"15min":1.367},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":1.286,"15min":1.286},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.98499999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.99399999999999999,"5min":1.413,"15min":1.413},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.45,"15min":1.45},"last":0.95999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575436,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412228,"kb_used_data":291964,"kb_used_omap":0,"kb_used_meta":120256,"kb_avail":93959612,"statfs":{"total":96636764160,"available":96214642688,"internally_reserved":0,"allocated":298971136,"data_stored":478232100,"data_compressed":51106947,"data_compressed_allocated":62287872,"data_compressed_original":247705600,"omap_allocated":0,"internal_metadata":123142144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.81999999999999995,"5min":1.137,"15min":1.137},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.83299999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.77900000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.41199999999999998,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.0640000000000001,"15min":1.103},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1279999999999999,"15min":1.2},"last":0.73799999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":1.609,"15min":1.609},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.47999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.871,"5min":1.159,"15min":1.1659999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.407,"15min":1.407},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.0620000000000001,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.628},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":1.046,"5min":1.361,"15min":1.3740000000000001},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575436,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287340,"kb_used_data":216932,"kb_used_omap":0,"kb_used_meta":70400,"kb_avail":94084500,"statfs":{"total":96636764160,"available":96342528000,"internally_reserved":0,"allocated":222138368,"data_stored":324680473,"data_compressed":32076821,"data_compressed_allocated":38850560,"data_compressed_original":145833984,"omap_allocated":0,"internal_metadata":72089600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2,8,100,58],"upper_bound":256},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":5.4969999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":5.6269999999999998}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":19.715},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":7.4089999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":5.5620000000000003},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":14.439}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":12.130000000000001},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":12.218999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":67.590000000000003},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":67.521000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":12.163},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":7.6040000000000001}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":3.8039999999999998},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":19.834}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":63250432,"data_stored":227580890,"data_compressed":28660926,"data_compressed_allocated":37535744,"data_compressed_original":201904128,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35291136,"data_stored":127693961,"data_compressed":16055263,"data_compressed_allocated":21200896,"data_compressed_original":113623040,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42078208,"data_stored":158395181,"data_compressed":20420954,"data_compressed_allocated":26488832,"data_compressed_original":142835712,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:59:38.747 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-06-09T11:59:38.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:59:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.977+0000 7f019ebff700 1 Processor -- start 2023-06-09T11:59:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.977+0000 7f019ebff700 1 -- start start 2023-06-09T11:59:38.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.978+0000 7f019ebff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f019815b4a0 0x7f019815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.978+0000 7f019ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01980c9f40 0x7f01980c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.978+0000 7f019ebff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019815a500 con 0x7f0198153880 2023-06-09T11:59:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.978+0000 7f019ebff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01980c5f10 con 0x7f019815b4a0 2023-06-09T11:59:38.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.978+0000 7f019ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019815da00 con 0x7f01980c9f40 2023-06-09T11:59:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f019815b4a0 0x7f019815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f019815b4a0 0x7f019815d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38814/0 (socket says 172.21.15.7:38814) 2023-06-09T11:59:38.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 -- 172.21.15.7:0/807319038 learned_addr learned my addr 172.21.15.7:0/807319038 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f019d19c700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01980c9f40 0x7f01980c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 -- 172.21.15.7:0/807319038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01980c9f40 msgr2=0x7f01980c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01980c9f40 0x7f01980c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 -- 172.21.15.7:0/807319038 >> v1:172.21.15.117:6789/0 conn(0x7f0198153880 legacy=0x7f019815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 -- 172.21.15.7:0/807319038 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01981596e0 con 0x7f019815b4a0 2023-06-09T11:59:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f0197fff700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f019815b4a0 0x7f019815d4c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f01840098f0 tx=0x7f0184009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308bfbedaa62103f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.979+0000 7f01977fe700 1 -- 172.21.15.7:0/807319038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018400b8a0 con 0x7f019815b4a0 2023-06-09T11:59:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f01977fe700 1 -- 172.21.15.7:0/807319038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018400ba00 con 0x7f019815b4a0 2023-06-09T11:59:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f01977fe700 1 -- 172.21.15.7:0/807319038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018400bd90 con 0x7f019815b4a0 2023-06-09T11:59:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 -- 172.21.15.7:0/807319038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f019815b4a0 msgr2=0x7f019815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f019815b4a0 0x7f019815d4c0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f01840098f0 tx=0x7f0184009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:38.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 -- 172.21.15.7:0/807319038 shutdown_connections 2023-06-09T11:59:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f01980c9f40 0x7f01980c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 --2- 172.21.15.7:0/807319038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f019815b4a0 0x7f019815d4c0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 -- 172.21.15.7:0/807319038 >> 172.21.15.7:0/807319038 conn(0x7f01980737f0 msgr2=0x7f01980720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 -- 172.21.15.7:0/807319038 shutdown_connections 2023-06-09T11:59:38.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.980+0000 7f019ebff700 1 -- 172.21.15.7:0/807319038 wait complete. 2023-06-09T11:59:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.982+0000 7f019ebff700 1 Processor -- start 2023-06-09T11:59:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.982+0000 7f019ebff700 1 -- start start 2023-06-09T11:59:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0198153880 0x7f0198149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f019815b4a0 0x7f019814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0198151670 con 0x7f01980c9f40 2023-06-09T11:59:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01981529e0 con 0x7f019815b4a0 2023-06-09T11:59:38.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019ebff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019814b660 con 0x7f0198153880 2023-06-09T11:59:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019c99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38818/0 (socket says 172.21.15.7:38818) 2023-06-09T11:59:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.983+0000 7f019c99b700 1 -- 172.21.15.7:0/2242290037 learned_addr learned my addr 172.21.15.7:0/2242290037 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f0197fff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0198153880 0x7f0198149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019d19c700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f019815b4a0 0x7f019814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 -- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0198153880 msgr2=0x7f0198149e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0198153880 0x7f0198149e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 -- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f019815b4a0 msgr2=0x7f019814eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f019815b4a0 0x7f019814eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 -- 172.21.15.7:0/2242290037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184009550 con 0x7f01980c9f40 2023-06-09T11:59:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019c99b700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c4e0 tx=0x7f018c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0052d0 con 0x7f01980c9f40 2023-06-09T11:59:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c005430 con 0x7f01980c9f40 2023-06-09T11:59:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0198159c60 con 0x7f01980c9f40 2023-06-09T11:59:38.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c0038c0 con 0x7f01980c9f40 2023-06-09T11:59:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.984+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0198159980 con 0x7f01980c9f40 2023-06-09T11:59:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.985+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0198159f40 con 0x7f01980c9f40 2023-06-09T11:59:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.986+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f018c003a20 con 0x7f01980c9f40 2023-06-09T11:59:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.988+0000 7f01957fa700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.988+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f018c089210 con 0x7f01980c9f40 2023-06-09T11:59:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.989+0000 7f0197fff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.989+0000 7f0197fff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 0x7f017c06d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0184009c40 tx=0x7f0184008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:38.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:38.993+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f018c0538b0 con 0x7f01980c9f40 2023-06-09T11:59:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.196+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f019815a500 con 0x7f017c06ace0 2023-06-09T11:59:39.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.206+0000 7f01957fa700 1 -- 172.21.15.7:0/2242290037 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316087 (secure 0 0 0) 0x7f019815a500 con 0x7f017c06ace0 2023-06-09T11:59:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 msgr2=0x7f017c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 0x7f017c06d190 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f0184009c40 tx=0x7f0184008030 comp rx=0 tx=0).stop 2023-06-09T11:59:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 msgr2=0x7f0198151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c4e0 tx=0x7f018c00c8a0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 shutdown_connections 2023-06-09T11:59:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f019815b4a0 0x7f019814eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01980c9f40 0x7f0198151130 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f017c06ace0 0x7f017c06d190 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 --2- 172.21.15.7:0/2242290037 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0198153880 0x7f0198149e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 >> 172.21.15.7:0/2242290037 conn(0x7f01980737f0 msgr2=0x7f019815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 shutdown_connections 2023-06-09T11:59:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.210+0000 7f019ebff700 1 -- 172.21.15.7:0/2242290037 wait complete. 2023-06-09T11:59:39.216 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:59:39.231 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1536,"stamp":"2023-06-09T11:59:38.925840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1373306950,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76390,"num_write_kb":1829420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88685,"ondisk_log_size":88685,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2546224,"kb_used_data":1882480,"kb_used_omap":0,"kb_used_meta":663680,"kb_avail":752428496,"statfs":{"total":773094113280,"available":770486779904,"internally_reserved":0,"allocated":1927659520,"data_stored":2750964400,"data_compressed":269027180,"data_compressed_allocated":325271552,"data_compressed_original":1191256064,"omap_allocated":0,"internal_metadata":679608320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3839723,"num_objects":13,"num_object_clones":22,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":22,"num_read":264,"num_read_kb":1003,"num_write":153,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.176802+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:29.176802+0000","last_peered":"2023-06-09T11:59:29.176802+0000","last_clean":"2023-06-09T11:59:29.176802+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:29.176802+0000","last_undegraded":"2023-06-09T11:59:29.176802+0000","last_fullsized":"2023-06-09T11:59:29.176802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:41.997596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138064+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138064+0000","last_peered":"2023-06-09T11:59:29.138064+0000","last_clean":"2023-06-09T11:59:29.138064+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138064+0000","last_undegraded":"2023-06-09T11:59:29.138064+0000","last_fullsized":"2023-06-09T11:59:29.138064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:43.538089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184905+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.184905+0000","last_peered":"2023-06-09T11:59:29.184905+0000","last_clean":"2023-06-09T11:59:29.184905+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.184905+0000","last_undegraded":"2023-06-09T11:59:29.184905+0000","last_fullsized":"2023-06-09T11:59:29.184905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:54.053113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183976+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:29.183976+0000","last_peered":"2023-06-09T11:59:29.183976+0000","last_clean":"2023-06-09T11:59:29.183976+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:29.183976+0000","last_undegraded":"2023-06-09T11:59:29.183976+0000","last_fullsized":"2023-06-09T11:59:29.183976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:12.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179117+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179117+0000","last_peered":"2023-06-09T11:59:29.179117+0000","last_clean":"2023-06-09T11:59:29.179117+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179117+0000","last_undegraded":"2023-06-09T11:59:29.179117+0000","last_fullsized":"2023-06-09T11:59:29.179117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:24:43.823091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'864","reported_seq":3341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174723+0000","last_change":"2023-06-09T11:18:26.314501+0000","last_active":"2023-06-09T11:59:29.174723+0000","last_peered":"2023-06-09T11:59:29.174723+0000","last_clean":"2023-06-09T11:59:29.174723+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:59:29.174723+0000","last_undegraded":"2023-06-09T11:59:29.174723+0000","last_fullsized":"2023-06-09T11:59:29.174723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:01:52.942030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134658+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.134658+0000","last_peered":"2023-06-09T11:59:29.134658+0000","last_clean":"2023-06-09T11:59:29.134658+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.134658+0000","last_undegraded":"2023-06-09T11:59:29.134658+0000","last_fullsized":"2023-06-09T11:59:29.134658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:08.996053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3243,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164252+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164252+0000","last_peered":"2023-06-09T11:59:29.164252+0000","last_clean":"2023-06-09T11:59:29.164252+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164252+0000","last_undegraded":"2023-06-09T11:59:29.164252+0000","last_fullsized":"2023-06-09T11:59:29.164252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:39:08.386702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4197,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183305+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183305+0000","last_peered":"2023-06-09T11:59:29.183305+0000","last_clean":"2023-06-09T11:59:29.183305+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183305+0000","last_undegraded":"2023-06-09T11:59:29.183305+0000","last_fullsized":"2023-06-09T11:59:29.183305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:42:35.512902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163514+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163514+0000","last_peered":"2023-06-09T11:59:29.163514+0000","last_clean":"2023-06-09T11:59:29.163514+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163514+0000","last_undegraded":"2023-06-09T11:59:29.163514+0000","last_fullsized":"2023-06-09T11:59:29.163514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:40.374816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184625+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184625+0000","last_peered":"2023-06-09T11:59:29.184625+0000","last_clean":"2023-06-09T11:59:29.184625+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184625+0000","last_undegraded":"2023-06-09T11:59:29.184625+0000","last_fullsized":"2023-06-09T11:59:29.184625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:26:32.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.166730+0000","last_change":"2023-06-09T11:18:26.271005+0000","last_active":"2023-06-09T11:59:29.166730+0000","last_peered":"2023-06-09T11:59:29.166730+0000","last_clean":"2023-06-09T11:59:29.166730+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:29.166730+0000","last_undegraded":"2023-06-09T11:59:29.166730+0000","last_fullsized":"2023-06-09T11:59:29.166730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:55.416203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108516+0000","last_change":"2023-06-09T11:18:26.253105+0000","last_active":"2023-06-09T11:59:30.108516+0000","last_peered":"2023-06-09T11:59:30.108516+0000","last_clean":"2023-06-09T11:59:30.108516+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:30.108516+0000","last_undegraded":"2023-06-09T11:59:30.108516+0000","last_fullsized":"2023-06-09T11:59:30.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:36:08.467688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108534+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:29.108534+0000","last_peered":"2023-06-09T11:59:29.108534+0000","last_clean":"2023-06-09T11:59:29.108534+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:29.108534+0000","last_undegraded":"2023-06-09T11:59:29.108534+0000","last_fullsized":"2023-06-09T11:59:29.108534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:08:27.844416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134761+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:29.134761+0000","last_peered":"2023-06-09T11:59:29.134761+0000","last_clean":"2023-06-09T11:59:29.134761+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:29.134761+0000","last_undegraded":"2023-06-09T11:59:29.134761+0000","last_fullsized":"2023-06-09T11:59:29.134761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:43:06.239662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3425,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174867+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:29.174867+0000","last_peered":"2023-06-09T11:59:29.174867+0000","last_clean":"2023-06-09T11:59:29.174867+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:29.174867+0000","last_undegraded":"2023-06-09T11:59:29.174867+0000","last_fullsized":"2023-06-09T11:59:29.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:35.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183203+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:29.183203+0000","last_peered":"2023-06-09T11:59:29.183203+0000","last_clean":"2023-06-09T11:59:29.183203+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:29.183203+0000","last_undegraded":"2023-06-09T11:59:29.183203+0000","last_fullsized":"2023-06-09T11:59:29.183203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:37.819209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174315+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:29.174315+0000","last_peered":"2023-06-09T11:59:29.174315+0000","last_clean":"2023-06-09T11:59:29.174315+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:29.174315+0000","last_undegraded":"2023-06-09T11:59:29.174315+0000","last_fullsized":"2023-06-09T11:59:29.174315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:47:55.986286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3553,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108428+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.108428+0000","last_peered":"2023-06-09T11:59:29.108428+0000","last_clean":"2023-06-09T11:59:29.108428+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.108428+0000","last_undegraded":"2023-06-09T11:59:29.108428+0000","last_fullsized":"2023-06-09T11:59:29.108428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:53:59.032131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186654+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:29.186654+0000","last_peered":"2023-06-09T11:59:29.186654+0000","last_clean":"2023-06-09T11:59:29.186654+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:29.186654+0000","last_undegraded":"2023-06-09T11:59:29.186654+0000","last_fullsized":"2023-06-09T11:59:29.186654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:31.924577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.127903+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:29.127903+0000","last_peered":"2023-06-09T11:59:29.127903+0000","last_clean":"2023-06-09T11:59:29.127903+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:29.127903+0000","last_undegraded":"2023-06-09T11:59:29.127903+0000","last_fullsized":"2023-06-09T11:59:29.127903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:53.061847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3479,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.148430+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:29.148430+0000","last_peered":"2023-06-09T11:59:29.148430+0000","last_clean":"2023-06-09T11:59:29.148430+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:29.148430+0000","last_undegraded":"2023-06-09T11:59:29.148430+0000","last_fullsized":"2023-06-09T11:59:29.148430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:33:23.877739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206328+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:29.206328+0000","last_peered":"2023-06-09T11:59:29.206328+0000","last_clean":"2023-06-09T11:59:29.206328+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:29.206328+0000","last_undegraded":"2023-06-09T11:59:29.206328+0000","last_fullsized":"2023-06-09T11:59:29.206328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:31:00.135897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181162+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181162+0000","last_peered":"2023-06-09T11:59:29.181162+0000","last_clean":"2023-06-09T11:59:29.181162+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181162+0000","last_undegraded":"2023-06-09T11:59:29.181162+0000","last_fullsized":"2023-06-09T11:59:29.181162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:46:27.956223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3016,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105429+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.105429+0000","last_peered":"2023-06-09T11:59:30.105429+0000","last_clean":"2023-06-09T11:59:30.105429+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.105429+0000","last_undegraded":"2023-06-09T11:59:30.105429+0000","last_fullsized":"2023-06-09T11:59:30.105429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:07.769640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171191+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171191+0000","last_peered":"2023-06-09T11:59:29.171191+0000","last_clean":"2023-06-09T11:59:29.171191+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171191+0000","last_undegraded":"2023-06-09T11:59:29.171191+0000","last_fullsized":"2023-06-09T11:59:29.171191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:38:31.443299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116586+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116586+0000","last_peered":"2023-06-09T11:59:29.116586+0000","last_clean":"2023-06-09T11:59:29.116586+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116586+0000","last_undegraded":"2023-06-09T11:59:29.116586+0000","last_fullsized":"2023-06-09T11:59:29.116586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:28.791273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.608768+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:29.608768+0000","last_peered":"2023-06-09T11:59:29.608768+0000","last_clean":"2023-06-09T11:59:29.608768+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:29.608768+0000","last_undegraded":"2023-06-09T11:59:29.608768+0000","last_fullsized":"2023-06-09T11:59:29.608768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:54:04.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.159320+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:29.159320+0000","last_peered":"2023-06-09T11:59:29.159320+0000","last_clean":"2023-06-09T11:59:29.159320+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:29.159320+0000","last_undegraded":"2023-06-09T11:59:29.159320+0000","last_fullsized":"2023-06-09T11:59:29.159320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:04:01.139122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2240,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604309+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:29.604309+0000","last_peered":"2023-06-09T11:59:29.604309+0000","last_clean":"2023-06-09T11:59:29.604309+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:29.604309+0000","last_undegraded":"2023-06-09T11:59:29.604309+0000","last_fullsized":"2023-06-09T11:59:29.604309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:19.987020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3133,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099238+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.099238+0000","last_peered":"2023-06-09T11:59:30.099238+0000","last_clean":"2023-06-09T11:59:30.099238+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.099238+0000","last_undegraded":"2023-06-09T11:59:30.099238+0000","last_fullsized":"2023-06-09T11:59:30.099238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:03.591541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826279+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:23.826279+0000","last_peered":"2023-06-09T11:59:23.826279+0000","last_clean":"2023-06-09T11:59:23.826279+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:23.826279+0000","last_undegraded":"2023-06-09T11:59:23.826279+0000","last_fullsized":"2023-06-09T11:59:23.826279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:19:22.681147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120527+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120527+0000","last_peered":"2023-06-09T11:59:29.120527+0000","last_clean":"2023-06-09T11:59:29.120527+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120527+0000","last_undegraded":"2023-06-09T11:59:29.120527+0000","last_fullsized":"2023-06-09T11:59:29.120527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:15.836484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'181","reported_seq":344,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.080996+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:59:30.080996+0000","last_peered":"2023-06-09T11:59:30.080996+0000","last_clean":"2023-06-09T11:59:30.080996+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:30.080996+0000","last_undegraded":"2023-06-09T11:59:30.080996+0000","last_fullsized":"2023-06-09T11:59:30.080996+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:22:06.545450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":125,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2960,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.204530+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:29.204530+0000","last_peered":"2023-06-09T11:59:29.204530+0000","last_clean":"2023-06-09T11:59:29.204530+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:29.204530+0000","last_undegraded":"2023-06-09T11:59:29.204530+0000","last_fullsized":"2023-06-09T11:59:29.204530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:51:52.478717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813587+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:59:17.813587+0000","last_peered":"2023-06-09T11:59:17.813587+0000","last_clean":"2023-06-09T11:59:17.813587+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:59:17.813587+0000","last_undegraded":"2023-06-09T11:59:17.813587+0000","last_fullsized":"2023-06-09T11:59:17.813587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:40.338566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208409+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:29.208409+0000","last_peered":"2023-06-09T11:59:29.208409+0000","last_clean":"2023-06-09T11:59:29.208409+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:29.208409+0000","last_undegraded":"2023-06-09T11:59:29.208409+0000","last_fullsized":"2023-06-09T11:59:29.208409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:32:56.568860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086513+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:59:30.086513+0000","last_peered":"2023-06-09T11:59:30.086513+0000","last_clean":"2023-06-09T11:59:30.086513+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:30.086513+0000","last_undegraded":"2023-06-09T11:59:30.086513+0000","last_fullsized":"2023-06-09T11:59:30.086513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:11:48.144420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088093+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:30.088093+0000","last_peered":"2023-06-09T11:59:30.088093+0000","last_clean":"2023-06-09T11:59:30.088093+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:30.088093+0000","last_undegraded":"2023-06-09T11:59:30.088093+0000","last_fullsized":"2023-06-09T11:59:30.088093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:43:21.546026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'823","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.465432+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:30.465432+0000","last_peered":"2023-06-09T11:59:30.465432+0000","last_clean":"2023-06-09T11:59:30.465432+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:30.465432+0000","last_undegraded":"2023-06-09T11:59:30.465432+0000","last_fullsized":"2023-06-09T11:59:30.465432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:15:45.170741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":768,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3094,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186545+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:29.186545+0000","last_peered":"2023-06-09T11:59:29.186545+0000","last_clean":"2023-06-09T11:59:29.186545+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:29.186545+0000","last_undegraded":"2023-06-09T11:59:29.186545+0000","last_fullsized":"2023-06-09T11:59:29.186545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:43.603410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'861","reported_seq":1075,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.596869+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:34.596869+0000","last_peered":"2023-06-09T11:59:34.596869+0000","last_clean":"2023-06-09T11:59:34.596869+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:34.596869+0000","last_undegraded":"2023-06-09T11:59:34.596869+0000","last_fullsized":"2023-06-09T11:59:34.596869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":861,"log_dups_size":0,"ondisk_log_size":861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:48.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15597013,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":803,"num_write_kb":15822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3556,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.161492+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:29.161492+0000","last_peered":"2023-06-09T11:59:29.161492+0000","last_clean":"2023-06-09T11:59:29.161492+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:29.161492+0000","last_undegraded":"2023-06-09T11:59:29.161492+0000","last_fullsized":"2023-06-09T11:59:29.161492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:48.346810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'413","reported_seq":575,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814193+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:59:17.814193+0000","last_peered":"2023-06-09T11:59:17.814193+0000","last_clean":"2023-06-09T11:59:17.814193+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:17.814193+0000","last_undegraded":"2023-06-09T11:59:17.814193+0000","last_fullsized":"2023-06-09T11:59:17.814193+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:03:12.461060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116041+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116041+0000","last_peered":"2023-06-09T11:59:29.116041+0000","last_clean":"2023-06-09T11:59:29.116041+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116041+0000","last_undegraded":"2023-06-09T11:59:29.116041+0000","last_fullsized":"2023-06-09T11:59:29.116041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:24:03.501267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'961","reported_seq":1144,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824131+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:59:23.824131+0000","last_peered":"2023-06-09T11:59:23.824131+0000","last_clean":"2023-06-09T11:59:23.824131+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:59:23.824131+0000","last_undegraded":"2023-06-09T11:59:23.824131+0000","last_fullsized":"2023-06-09T11:59:23.824131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:53:42.964479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3022,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185110+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185110+0000","last_peered":"2023-06-09T11:59:29.185110+0000","last_clean":"2023-06-09T11:59:29.185110+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185110+0000","last_undegraded":"2023-06-09T11:59:29.185110+0000","last_fullsized":"2023-06-09T11:59:29.185110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:02:58.361435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466196+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:30.466196+0000","last_peered":"2023-06-09T11:59:30.466196+0000","last_clean":"2023-06-09T11:59:30.466196+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:30.466196+0000","last_undegraded":"2023-06-09T11:59:30.466196+0000","last_fullsized":"2023-06-09T11:59:30.466196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:30.247597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203830+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203830+0000","last_peered":"2023-06-09T11:59:29.203830+0000","last_clean":"2023-06-09T11:59:29.203830+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203830+0000","last_undegraded":"2023-06-09T11:59:29.203830+0000","last_fullsized":"2023-06-09T11:59:29.203830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:33:19.947716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825423+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.825423+0000","last_peered":"2023-06-09T11:59:23.825423+0000","last_clean":"2023-06-09T11:59:23.825423+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.825423+0000","last_undegraded":"2023-06-09T11:59:23.825423+0000","last_fullsized":"2023-06-09T11:59:23.825423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:22:06.716248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113291+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113291+0000","last_peered":"2023-06-09T11:59:29.113291+0000","last_clean":"2023-06-09T11:59:29.113291+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113291+0000","last_undegraded":"2023-06-09T11:59:29.113291+0000","last_fullsized":"2023-06-09T11:59:29.113291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:25:53.990234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.607238+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:29.607238+0000","last_peered":"2023-06-09T11:59:29.607238+0000","last_clean":"2023-06-09T11:59:29.607238+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:29.607238+0000","last_undegraded":"2023-06-09T11:59:29.607238+0000","last_fullsized":"2023-06-09T11:59:29.607238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:10:40.976689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178950+0000","last_change":"2023-06-09T11:18:26.315365+0000","last_active":"2023-06-09T11:59:29.178950+0000","last_peered":"2023-06-09T11:59:29.178950+0000","last_clean":"2023-06-09T11:59:29.178950+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:29.178950+0000","last_undegraded":"2023-06-09T11:59:29.178950+0000","last_fullsized":"2023-06-09T11:59:29.178950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:48:14.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086762+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:30.086762+0000","last_peered":"2023-06-09T11:59:30.086762+0000","last_clean":"2023-06-09T11:59:30.086762+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:30.086762+0000","last_undegraded":"2023-06-09T11:59:30.086762+0000","last_fullsized":"2023-06-09T11:59:30.086762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:17.698189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088582+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.088582+0000","last_peered":"2023-06-09T11:59:30.088582+0000","last_clean":"2023-06-09T11:59:30.088582+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.088582+0000","last_undegraded":"2023-06-09T11:59:30.088582+0000","last_fullsized":"2023-06-09T11:59:30.088582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:35:52.399332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825965+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:59:23.825965+0000","last_peered":"2023-06-09T11:59:23.825965+0000","last_clean":"2023-06-09T11:59:23.825965+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:59:23.825965+0000","last_undegraded":"2023-06-09T11:59:23.825965+0000","last_fullsized":"2023-06-09T11:59:23.825965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:58.048595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.149039+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:29.149039+0000","last_peered":"2023-06-09T11:59:29.149039+0000","last_clean":"2023-06-09T11:59:29.149039+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:29.149039+0000","last_undegraded":"2023-06-09T11:59:29.149039+0000","last_fullsized":"2023-06-09T11:59:29.149039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:22:45.713305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.812549+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:17.812549+0000","last_peered":"2023-06-09T11:59:17.812549+0000","last_clean":"2023-06-09T11:59:17.812549+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:17.812549+0000","last_undegraded":"2023-06-09T11:59:17.812549+0000","last_fullsized":"2023-06-09T11:59:17.812549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:59.524461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105462+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:30.105462+0000","last_peered":"2023-06-09T11:59:30.105462+0000","last_clean":"2023-06-09T11:59:30.105462+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:30.105462+0000","last_undegraded":"2023-06-09T11:59:30.105462+0000","last_fullsized":"2023-06-09T11:59:30.105462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:52:08.337474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1064","reported_seq":1239,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822915+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:59:23.822915+0000","last_peered":"2023-06-09T11:59:23.822915+0000","last_clean":"2023-06-09T11:59:23.822915+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:59:23.822915+0000","last_undegraded":"2023-06-09T11:59:23.822915+0000","last_fullsized":"2023-06-09T11:59:23.822915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1064,"log_dups_size":0,"ondisk_log_size":1064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:47:25.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180294+0000","last_change":"2023-06-09T11:18:26.342753+0000","last_active":"2023-06-09T11:59:29.180294+0000","last_peered":"2023-06-09T11:59:29.180294+0000","last_clean":"2023-06-09T11:59:29.180294+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:29.180294+0000","last_undegraded":"2023-06-09T11:59:29.180294+0000","last_fullsized":"2023-06-09T11:59:29.180294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:36.852596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824479+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:23.824479+0000","last_peered":"2023-06-09T11:59:23.824479+0000","last_clean":"2023-06-09T11:59:23.824479+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:23.824479+0000","last_undegraded":"2023-06-09T11:59:23.824479+0000","last_fullsized":"2023-06-09T11:59:23.824479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:00:20.027849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3202,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.182579+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:29.182579+0000","last_peered":"2023-06-09T11:59:29.182579+0000","last_clean":"2023-06-09T11:59:29.182579+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:29.182579+0000","last_undegraded":"2023-06-09T11:59:29.182579+0000","last_fullsized":"2023-06-09T11:59:29.182579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:14.768104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.811217+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:17.811217+0000","last_peered":"2023-06-09T11:59:17.811217+0000","last_clean":"2023-06-09T11:59:17.811217+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:17.811217+0000","last_undegraded":"2023-06-09T11:59:17.811217+0000","last_fullsized":"2023-06-09T11:59:17.811217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:02:12.084788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198203+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:29.198203+0000","last_peered":"2023-06-09T11:59:29.198203+0000","last_clean":"2023-06-09T11:59:29.198203+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:29.198203+0000","last_undegraded":"2023-06-09T11:59:29.198203+0000","last_fullsized":"2023-06-09T11:59:29.198203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:18:57.401435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.836981+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:59:17.836981+0000","last_peered":"2023-06-09T11:59:17.836981+0000","last_clean":"2023-06-09T11:59:17.836981+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:17.836981+0000","last_undegraded":"2023-06-09T11:59:17.836981+0000","last_fullsized":"2023-06-09T11:59:17.836981+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:00.401114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813812+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:17.813812+0000","last_peered":"2023-06-09T11:59:17.813812+0000","last_clean":"2023-06-09T11:59:17.813812+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:17.813812+0000","last_undegraded":"2023-06-09T11:59:17.813812+0000","last_fullsized":"2023-06-09T11:59:17.813812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:04.925798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198730+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198730+0000","last_peered":"2023-06-09T11:59:29.198730+0000","last_clean":"2023-06-09T11:59:29.198730+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198730+0000","last_undegraded":"2023-06-09T11:59:29.198730+0000","last_fullsized":"2023-06-09T11:59:29.198730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:20:09.665610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813891+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:59:17.813891+0000","last_peered":"2023-06-09T11:59:17.813891+0000","last_clean":"2023-06-09T11:59:17.813891+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:17.813891+0000","last_undegraded":"2023-06-09T11:59:17.813891+0000","last_fullsized":"2023-06-09T11:59:17.813891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:44.339799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":895,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826018+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:23.826018+0000","last_peered":"2023-06-09T11:59:23.826018+0000","last_clean":"2023-06-09T11:59:23.826018+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:23.826018+0000","last_undegraded":"2023-06-09T11:59:23.826018+0000","last_fullsized":"2023-06-09T11:59:23.826018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:11.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3887,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203580+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203580+0000","last_peered":"2023-06-09T11:59:29.203580+0000","last_clean":"2023-06-09T11:59:29.203580+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203580+0000","last_undegraded":"2023-06-09T11:59:29.203580+0000","last_fullsized":"2023-06-09T11:59:29.203580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:03:36.679974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.847101+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:59:17.847101+0000","last_peered":"2023-06-09T11:59:17.847101+0000","last_clean":"2023-06-09T11:59:17.847101+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:17.847101+0000","last_undegraded":"2023-06-09T11:59:17.847101+0000","last_fullsized":"2023-06-09T11:59:17.847101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:05:14.595589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.821723+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.821723+0000","last_peered":"2023-06-09T11:59:23.821723+0000","last_clean":"2023-06-09T11:59:23.821723+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.821723+0000","last_undegraded":"2023-06-09T11:59:23.821723+0000","last_fullsized":"2023-06-09T11:59:23.821723+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:56.504880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094605+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.094605+0000","last_peered":"2023-06-09T11:59:30.094605+0000","last_clean":"2023-06-09T11:59:30.094605+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.094605+0000","last_undegraded":"2023-06-09T11:59:30.094605+0000","last_fullsized":"2023-06-09T11:59:30.094605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:11:45.496865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814030+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:59:17.814030+0000","last_peered":"2023-06-09T11:59:17.814030+0000","last_clean":"2023-06-09T11:59:17.814030+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:17.814030+0000","last_undegraded":"2023-06-09T11:59:17.814030+0000","last_fullsized":"2023-06-09T11:59:17.814030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:17.409970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810193+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:59:17.810193+0000","last_peered":"2023-06-09T11:59:17.810193+0000","last_clean":"2023-06-09T11:59:17.810193+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:17.810193+0000","last_undegraded":"2023-06-09T11:59:17.810193+0000","last_fullsized":"2023-06-09T11:59:17.810193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:47:29.671673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1140","reported_seq":1323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824017+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:59:23.824017+0000","last_peered":"2023-06-09T11:59:23.824017+0000","last_clean":"2023-06-09T11:59:23.824017+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:59:23.824017+0000","last_undegraded":"2023-06-09T11:59:23.824017+0000","last_fullsized":"2023-06-09T11:59:23.824017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:27:21.609537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3295,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142633+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142633+0000","last_peered":"2023-06-09T11:59:29.142633+0000","last_clean":"2023-06-09T11:59:29.142633+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142633+0000","last_undegraded":"2023-06-09T11:59:29.142633+0000","last_fullsized":"2023-06-09T11:59:29.142633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:51:16.604757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810429+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:59:17.810429+0000","last_peered":"2023-06-09T11:59:17.810429+0000","last_clean":"2023-06-09T11:59:17.810429+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:17.810429+0000","last_undegraded":"2023-06-09T11:59:17.810429+0000","last_fullsized":"2023-06-09T11:59:17.810429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:44:14.620706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2984,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.099630+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.099630+0000","last_peered":"2023-06-09T11:59:30.099630+0000","last_clean":"2023-06-09T11:59:30.099630+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.099630+0000","last_undegraded":"2023-06-09T11:59:30.099630+0000","last_fullsized":"2023-06-09T11:59:30.099630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:27:13.002268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185314+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185314+0000","last_peered":"2023-06-09T11:59:29.185314+0000","last_clean":"2023-06-09T11:59:29.185314+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185314+0000","last_undegraded":"2023-06-09T11:59:29.185314+0000","last_fullsized":"2023-06-09T11:59:29.185314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:05.482688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810999+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:59:17.810999+0000","last_peered":"2023-06-09T11:59:17.810999+0000","last_clean":"2023-06-09T11:59:17.810999+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:59:17.810999+0000","last_undegraded":"2023-06-09T11:59:17.810999+0000","last_fullsized":"2023-06-09T11:59:17.810999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:10.967583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823044+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823044+0000","last_peered":"2023-06-09T11:59:23.823044+0000","last_clean":"2023-06-09T11:59:23.823044+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823044+0000","last_undegraded":"2023-06-09T11:59:23.823044+0000","last_fullsized":"2023-06-09T11:59:23.823044+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:53:37.692810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4110,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142122+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:29.142122+0000","last_peered":"2023-06-09T11:59:29.142122+0000","last_clean":"2023-06-09T11:59:29.142122+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:29.142122+0000","last_undegraded":"2023-06-09T11:59:29.142122+0000","last_fullsized":"2023-06-09T11:59:29.142122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:00:06.952928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105019+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.105019+0000","last_peered":"2023-06-09T11:59:30.105019+0000","last_clean":"2023-06-09T11:59:30.105019+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.105019+0000","last_undegraded":"2023-06-09T11:59:30.105019+0000","last_fullsized":"2023-06-09T11:59:30.105019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:59:20.477510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.839493+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:59:17.839493+0000","last_peered":"2023-06-09T11:59:17.839493+0000","last_clean":"2023-06-09T11:59:17.839493+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:17.839493+0000","last_undegraded":"2023-06-09T11:59:17.839493+0000","last_fullsized":"2023-06-09T11:59:17.839493+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:09:23.586799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208331+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208331+0000","last_peered":"2023-06-09T11:59:29.208331+0000","last_clean":"2023-06-09T11:59:29.208331+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208331+0000","last_undegraded":"2023-06-09T11:59:29.208331+0000","last_fullsized":"2023-06-09T11:59:29.208331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:58.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.819642+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:59:17.819642+0000","last_peered":"2023-06-09T11:59:17.819642+0000","last_clean":"2023-06-09T11:59:17.819642+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:17.819642+0000","last_undegraded":"2023-06-09T11:59:17.819642+0000","last_fullsized":"2023-06-09T11:59:17.819642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:01.435357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":825,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.465836+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:30.465836+0000","last_peered":"2023-06-09T11:59:30.465836+0000","last_clean":"2023-06-09T11:59:30.465836+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:30.465836+0000","last_undegraded":"2023-06-09T11:59:30.465836+0000","last_fullsized":"2023-06-09T11:59:30.465836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:33:14.865876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3713,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170033+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170033+0000","last_peered":"2023-06-09T11:59:29.170033+0000","last_clean":"2023-06-09T11:59:29.170033+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170033+0000","last_undegraded":"2023-06-09T11:59:29.170033+0000","last_fullsized":"2023-06-09T11:59:29.170033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:28:31.292243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":879,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086836+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.086836+0000","last_peered":"2023-06-09T11:59:30.086836+0000","last_clean":"2023-06-09T11:59:30.086836+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.086836+0000","last_undegraded":"2023-06-09T11:59:30.086836+0000","last_fullsized":"2023-06-09T11:59:30.086836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:03:06.187539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'955","reported_seq":4519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095034+0000","last_change":"2023-06-09T11:18:26.261803+0000","last_active":"2023-06-09T11:59:30.095034+0000","last_peered":"2023-06-09T11:59:30.095034+0000","last_clean":"2023-06-09T11:59:30.095034+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:59:30.095034+0000","last_undegraded":"2023-06-09T11:59:30.095034+0000","last_fullsized":"2023-06-09T11:59:30.095034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:10:57.891708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":765,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.090216+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.090216+0000","last_peered":"2023-06-09T11:59:30.090216+0000","last_clean":"2023-06-09T11:59:30.090216+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.090216+0000","last_undegraded":"2023-06-09T11:59:30.090216+0000","last_fullsized":"2023-06-09T11:59:30.090216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:31.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178756+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.178756+0000","last_peered":"2023-06-09T11:59:29.178756+0000","last_clean":"2023-06-09T11:59:29.178756+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.178756+0000","last_undegraded":"2023-06-09T11:59:29.178756+0000","last_fullsized":"2023-06-09T11:59:29.178756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:09:06.669625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'558","reported_seq":751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.586797+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:34.586797+0000","last_peered":"2023-06-09T11:59:34.586797+0000","last_clean":"2023-06-09T11:59:34.586797+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:34.586797+0000","last_undegraded":"2023-06-09T11:59:34.586797+0000","last_fullsized":"2023-06-09T11:59:34.586797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:38.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23029134,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":484,"num_write_kb":26925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3673,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179771+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:29.179771+0000","last_peered":"2023-06-09T11:59:29.179771+0000","last_clean":"2023-06-09T11:59:29.179771+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:29.179771+0000","last_undegraded":"2023-06-09T11:59:29.179771+0000","last_fullsized":"2023-06-09T11:59:29.179771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:10:40.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206896+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:59:29.206896+0000","last_peered":"2023-06-09T11:59:29.206896+0000","last_clean":"2023-06-09T11:59:29.206896+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:59:29.206896+0000","last_undegraded":"2023-06-09T11:59:29.206896+0000","last_fullsized":"2023-06-09T11:59:29.206896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:57.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3147,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206536+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.206536+0000","last_peered":"2023-06-09T11:59:29.206536+0000","last_clean":"2023-06-09T11:59:29.206536+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.206536+0000","last_undegraded":"2023-06-09T11:59:29.206536+0000","last_fullsized":"2023-06-09T11:59:29.206536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:49:35.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"73'1383","reported_seq":1600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.817828+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:59:23.817828+0000","last_peered":"2023-06-09T11:59:23.817828+0000","last_clean":"2023-06-09T11:59:23.817828+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:59:23.817828+0000","last_undegraded":"2023-06-09T11:59:23.817828+0000","last_fullsized":"2023-06-09T11:59:23.817828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:25:09.505525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187254+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:29.187254+0000","last_peered":"2023-06-09T11:59:29.187254+0000","last_clean":"2023-06-09T11:59:29.187254+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:29.187254+0000","last_undegraded":"2023-06-09T11:59:29.187254+0000","last_fullsized":"2023-06-09T11:59:29.187254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:17:10.649718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120227+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120227+0000","last_peered":"2023-06-09T11:59:29.120227+0000","last_clean":"2023-06-09T11:59:29.120227+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120227+0000","last_undegraded":"2023-06-09T11:59:29.120227+0000","last_fullsized":"2023-06-09T11:59:29.120227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:29:14.112446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207330+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.207330+0000","last_peered":"2023-06-09T11:59:29.207330+0000","last_clean":"2023-06-09T11:59:29.207330+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.207330+0000","last_undegraded":"2023-06-09T11:59:29.207330+0000","last_fullsized":"2023-06-09T11:59:29.207330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:56:35.301385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187085+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187085+0000","last_peered":"2023-06-09T11:59:29.187085+0000","last_clean":"2023-06-09T11:59:29.187085+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187085+0000","last_undegraded":"2023-06-09T11:59:29.187085+0000","last_fullsized":"2023-06-09T11:59:29.187085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:25.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2977,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180708+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.180708+0000","last_peered":"2023-06-09T11:59:29.180708+0000","last_clean":"2023-06-09T11:59:29.180708+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.180708+0000","last_undegraded":"2023-06-09T11:59:29.180708+0000","last_fullsized":"2023-06-09T11:59:29.180708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:36:34.824605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2991,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207551+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207551+0000","last_peered":"2023-06-09T11:59:29.207551+0000","last_clean":"2023-06-09T11:59:29.207551+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207551+0000","last_undegraded":"2023-06-09T11:59:29.207551+0000","last_fullsized":"2023-06-09T11:59:29.207551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:15:28.927370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58085,"ondisk_log_size":58085,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":508770079,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29939,"num_write_kb":670304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":289595392,"data_stored":1017540158,"data_compressed":126917660,"data_compressed_allocated":167665664,"data_compressed_original":895770624,"omap_allocated":0,"internal_metadata":0},"log_size":30483,"ondisk_log_size":30483,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575438,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254204,"kb_used_data":179828,"kb_used_omap":0,"kb_used_meta":74368,"kb_avail":94117636,"statfs":{"total":96636764160,"available":96376459264,"internally_reserved":0,"allocated":184143872,"data_stored":310893688,"data_compressed":32000116,"data_compressed_allocated":39612416,"data_compressed_original":170962944,"omap_allocated":0,"internal_metadata":76152832},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.63}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Jun 9 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.86899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.94199999999999995,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.502},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.747},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.85399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575438,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.47099999999999997}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.623}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.53400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575438,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396016,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89600,"kb_avail":93975824,"statfs":{"total":96636764160,"available":96231243776,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91750400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.373},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.498}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":0.88600000000000001}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.745},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.78700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575438,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327400,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99584,"kb_avail":94044440,"statfs":{"total":96636764160,"available":96301506560,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101974016},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.85599999999999998},"last":0.60399999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":1.0900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.499}]},{"osd":3,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.308},"max":{"1min":0.871,"5min":0.871,"15min":1.2909999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.1060000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97499999999999998,"5min":1.21,"15min":1.21},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0429999999999999,"5min":1.05,"15min":1.236},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.165},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.3700000000000001},"last":0.88300000000000001}]},{"osd":7,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":1.159,"15min":1.228},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83299999999999996,"15min":0.79600000000000004},"min":{"1min":0.64200000000000002,"5min":0.52400000000000002,"15min":0.46999999999999997},"max":{"1min":1.0860000000000001,"5min":1.111,"15min":1.29},"last":0.86499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232480,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52672,"kb_avail":94139360,"statfs":{"total":96636764160,"available":96398704640,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53936128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.44400000000000001}]},{"osd":1,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.76300000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.38800000000000001,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":1.367,"15min":1.367},"last":0.42199999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.546,"15min":1.546},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63300000000000001,"15min":0.59399999999999997},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.55400000000000005}]},{"osd":4,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.55500000000000005,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.0109999999999999,"5min":1.601,"15min":1.601},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84799999999999998,"15min":0.82199999999999995},"min":{"1min":0.71599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.63}]},{"osd":6,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.747},"min":{"1min":0.58999999999999997,"5min":0.496,"15min":0.438},"max":{"1min":0.878,"5min":1.2090000000000001,"15min":1.3100000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":1.121,"15min":1.3540000000000001},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.66100000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.208,"15min":1.248},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.82899999999999996,"15min":0.81000000000000005},"min":{"1min":0.73899999999999999,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.3080000000000001,"15min":1.337},"last":0.59699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575438,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.78100000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.60799999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.65800000000000003}]},{"osd":2,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.80600000000000005,"5min":1.355,"15min":1.355},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76700000000000002,"5min":1.198,"15min":1.198},"last":0.90000000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":1.635,"15min":1.635},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":2.351,"15min":2.351},"last":1.012}]},{"osd":5,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.61899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95399999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.64400000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":0.97599999999999998,"5min":1.367,"15min":1.367},"last":1.093}]},{"osd":6,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":1.286,"15min":1.286},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.98499999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":1.0469999999999999}]},{"osd":7,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.99399999999999999,"5min":1.413,"15min":1.413},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.45,"15min":1.45},"last":1.111}]}]},{"osd":4,"up_from":13,"seq":55834575437,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412616,"kb_used_data":292352,"kb_used_omap":0,"kb_used_meta":120256,"kb_avail":93959224,"statfs":{"total":96636764160,"available":96214245376,"internally_reserved":0,"allocated":299368448,"data_stored":479949916,"data_compressed":51414745,"data_compressed_allocated":62664704,"data_compressed_original":249405440,"omap_allocated":0,"internal_metadata":123142144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.81999999999999995,"5min":1.137,"15min":1.137},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.83299999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.748}]},{"osd":1,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.41199999999999998,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.0640000000000001,"15min":1.103},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1279999999999999,"15min":1.2},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":1.609,"15min":1.609},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.47999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.871,"5min":1.159,"15min":1.1659999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.407,"15min":1.407},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.0620000000000001,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":1.014}]},{"osd":6,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.92000000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":1.046,"5min":1.361,"15min":1.3740000000000001},"last":0.97299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575437,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287392,"kb_used_data":216984,"kb_used_omap":0,"kb_used_meta":70400,"kb_avail":94084448,"statfs":{"total":96636764160,"available":96342474752,"internally_reserved":0,"allocated":222191616,"data_stored":324874592,"data_compressed":32105995,"data_compressed_allocated":38891520,"data_compressed_original":146018304,"omap_allocated":0,"internal_metadata":72089600},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.55100000000000005}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.439}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":0.99299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":63647744,"data_stored":229298706,"data_compressed":28968724,"data_compressed_allocated":37912576,"data_compressed_original":203603968,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35344384,"data_stored":127888080,"data_compressed":16084437,"data_compressed_allocated":21241856,"data_compressed_original":113807360,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42422272,"data_stored":159918878,"data_compressed":20699578,"data_compressed_allocated":26824704,"data_compressed_original":144351232,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:59:39.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T11:59:39.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.471+0000 7f635cdff700 1 Processor -- start 2023-06-09T11:59:39.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.471+0000 7f635cdff700 1 -- start start 2023-06-09T11:59:39.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.473+0000 7f635cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.473+0000 7f635cdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 0x7f63580cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.473+0000 7f635cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6358159b20 con 0x7f63580cb9a0 2023-06-09T11:59:39.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.473+0000 7f635cdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63580cac80 con 0x7f635815cd40 2023-06-09T11:59:39.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.473+0000 7f635cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63581597d0 con 0x7f63580cb5d0 2023-06-09T11:59:39.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f6356d9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 0x7f63580cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:59482/0 (socket says 172.21.15.7:59482) 2023-06-09T11:59:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 -- 172.21.15.7:0/3524228214 learned_addr learned my addr 172.21.15.7:0/3524228214 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f634ffff700 1 --1- 172.21.15.7:0/3524228214 >> v1:172.21.15.117:6789/0 conn(0x7f63580cb9a0 0x7f6358159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:33932/0 (socket says 172.21.15.7:33932) 2023-06-09T11:59:39.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 -- 172.21.15.7:0/3524228214 >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 msgr2=0x7f63580cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:39.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 --2- 172.21.15.7:0/3524228214 >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 0x7f63580cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 -- 172.21.15.7:0/3524228214 >> v1:172.21.15.117:6789/0 conn(0x7f63580cb9a0 legacy=0x7f6358159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f635659c700 1 -- 172.21.15.7:0/3524228214 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358071ab0 con 0x7f63580cb5d0 2023-06-09T11:59:39.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.474+0000 7f6356d9d700 1 --2- 172.21.15.7:0/3524228214 >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 0x7f63580cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:39.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f635659c700 1 --2- 172.21.15.7:0/3524228214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63480098f0 tx=0x7f6348009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=725b954c6be10b29 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:39.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f6355d9b700 1 -- 172.21.15.7:0/3524228214 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348011830 con 0x7f63580cb5d0 2023-06-09T11:59:39.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f6355d9b700 1 -- 172.21.15.7:0/3524228214 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6348011990 con 0x7f63580cb5d0 2023-06-09T11:59:39.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f6355d9b700 1 -- 172.21.15.7:0/3524228214 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348011d20 con 0x7f63580cb5d0 2023-06-09T11:59:39.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f635cdff700 1 -- 172.21.15.7:0/3524228214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 msgr2=0x7f63580b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.475+0000 7f635cdff700 1 --2- 172.21.15.7:0/3524228214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63480098f0 tx=0x7f6348009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:39.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 -- 172.21.15.7:0/3524228214 shutdown_connections 2023-06-09T11:59:39.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 --2- 172.21.15.7:0/3524228214 >> v2:172.21.15.117:3300/0 conn(0x7f635815cd40 0x7f63580cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 --2- 172.21.15.7:0/3524228214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63580cb5d0 0x7f63580b52b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 -- 172.21.15.7:0/3524228214 >> 172.21.15.7:0/3524228214 conn(0x7f6358070af0 msgr2=0x7f6358070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 -- 172.21.15.7:0/3524228214 shutdown_connections 2023-06-09T11:59:39.483 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.476+0000 7f635cdff700 1 -- 172.21.15.7:0/3524228214 wait complete. 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.477+0000 7f635cdff700 1 Processor -- start 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.477+0000 7f635cdff700 1 -- start start 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580cb9a0 0x7f6358150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f635815cd40 0x7f63581510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.484 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358152d90 con 0x7f63580cb5d0 2023-06-09T11:59:39.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63581546d0 con 0x7f63580cb9a0 2023-06-09T11:59:39.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.478+0000 7f635cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6358152180 con 0x7f635815cd40 2023-06-09T11:59:39.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38820/0 (socket says 172.21.15.7:38820) 2023-06-09T11:59:39.485 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 -- 172.21.15.7:0/1753716680 learned_addr learned my addr 172.21.15.7:0/1753716680 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f634ffff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580cb9a0 0x7f6358150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f6356d9d700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f635815cd40 0x7f63581510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 -- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f635815cd40 msgr2=0x7f63581510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f635815cd40 0x7f63581510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.486 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 -- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580cb9a0 msgr2=0x7f6358150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580cb9a0 0x7f6358150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 -- 172.21.15.7:0/1753716680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009a70 con 0x7f63580cb5d0 2023-06-09T11:59:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.479+0000 7f635659c700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6348000c00 tx=0x7f6348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.480+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6348004480 con 0x7f63580cb5d0 2023-06-09T11:59:39.487 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.480+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634801f070 con 0x7f63580cb5d0 2023-06-09T11:59:39.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.480+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6348009550 con 0x7f63580cb5d0 2023-06-09T11:59:39.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.480+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63480038c0 con 0x7f63580cb5d0 2023-06-09T11:59:39.488 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.480+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635815d4f0 con 0x7f63580cb5d0 2023-06-09T11:59:39.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.481+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635815d210 con 0x7f63580cb5d0 2023-06-09T11:59:39.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.482+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6348021030 con 0x7f63580cb5d0 2023-06-09T11:59:39.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.483+0000 7f634effd700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 0x7f633806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:39.489 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.483+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f634808b310 con 0x7f63580cb5d0 2023-06-09T11:59:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.483+0000 7f634ffff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 0x7f633806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:39.490 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.484+0000 7f634ffff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 0x7f633806d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f63580b3d20 tx=0x7f6340007ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:39.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.491+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6348055b60 con 0x7f63580cb5d0 2023-06-09T11:59:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.706+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f635815d7d0 con 0x7f63580cb5d0 2023-06-09T11:59:39.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.707+0000 7f634effd700 1 -- 172.21.15.7:0/1753716680 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+21641 (secure 0 0 0) 0x7f6348059180 con 0x7f63580cb5d0 2023-06-09T11:59:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:39.710 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:59:17.800788+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6833/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6835/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.117:6834/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.7:6838/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6836/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6833/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6832/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6834/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.117:6834/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6833/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6833/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6832/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6835/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.117:6833/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.117:6837/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6832/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6837/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6836/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.117:6834/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.7:6839/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6835/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.7:6839/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.7:6838/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6837/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6834/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6836/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6836/1520141131":"2023-06-10T11:44:19.573452+0000","172.21.15.117:6833/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6835/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6833/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6834/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6835/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6834/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.117:6832/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.7:6838/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.117:6836/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6832/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.7:6839/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6834/1343666954":"2023-06-10T11:40:29.517640+0000","172.21.15.117:6835/1343666954":"2023-06-10T11:40:29.517640+0000","192.168.0.1:0/1078574456":"2023-06-09T12:59:16.804971+0000","172.21.15.7:6838/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6837/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.7:6839/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6836/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6837/1520141131":"2023-06-10T11:44:19.573452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T11:59:39.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 msgr2=0x7f633806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 0x7f633806d190 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f63580b3d20 tx=0x7f6340007ed0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 msgr2=0x7f63580c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:39.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f6348000c00 tx=0x7f6348004750 comp rx=0 tx=0).stop 2023-06-09T11:59:39.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 shutdown_connections 2023-06-09T11:59:39.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f633806ace0 0x7f633806d190 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f635815cd40 0x7f63581510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63580cb5d0 0x7f63580c7c00 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 --2- 172.21.15.7:0/1753716680 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63580cb9a0 0x7f6358150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:39.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 >> 172.21.15.7:0/1753716680 conn(0x7f6358070af0 msgr2=0x7f635815b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:39.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 shutdown_connections 2023-06-09T11:59:39.715 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:39.710+0000 7f635cdff700 1 -- 172.21.15.7:0/1753716680 wait complete. 2023-06-09T11:59:40.734 INFO:tasks.ceph:Scrubbing osd.0 2023-06-09T11:59:40.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:40.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.946+0000 7f4f825ff700 1 Processor -- start 2023-06-09T11:59:40.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.946+0000 7f4f825ff700 1 -- start start 2023-06-09T11:59:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.947+0000 7f4f825ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f4f7c1508b0 0x7f4f7c0c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.947+0000 7f4f825ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.947+0000 7f4f825ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f7c0b1120 con 0x7f4f7c1504e0 2023-06-09T11:59:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.947+0000 7f4f825ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c06bd40 con 0x7f4f7c0c1660 2023-06-09T11:59:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.947+0000 7f4f825ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f7c070650 con 0x7f4f7c1508b0 2023-06-09T11:59:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38830/0 (socket says 172.21.15.7:38830) 2023-06-09T11:59:40.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 -- 172.21.15.7:0/293176240 learned_addr learned my addr 172.21.15.7:0/293176240 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:40.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f7b7fe700 1 --2- 172.21.15.7:0/293176240 >> v2:172.21.15.117:3300/0 conn(0x7f4f7c1508b0 0x7f4f7c0c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 -- 172.21.15.7:0/293176240 >> v2:172.21.15.117:3300/0 conn(0x7f4f7c1508b0 msgr2=0x7f4f7c0c10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/293176240 >> v2:172.21.15.117:3300/0 conn(0x7f4f7c1508b0 0x7f4f7c0c10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 -- 172.21.15.7:0/293176240 >> v1:172.21.15.117:6789/0 conn(0x7f4f7c1504e0 legacy=0x7f4f7c06ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.948+0000 7f4f80b9c700 1 -- 172.21.15.7:0/293176240 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7c0b0b60 con 0x7f4f7c0c1660 2023-06-09T11:59:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.949+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/293176240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0c7b10 tx=0x7f4f7400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d7b8af75dfe575 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.949+0000 7f4f7affd700 1 -- 172.21.15.7:0/293176240 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74010070 con 0x7f4f7c0c1660 2023-06-09T11:59:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.949+0000 7f4f7affd700 1 -- 172.21.15.7:0/293176240 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7400dcd0 con 0x7f4f7c0c1660 2023-06-09T11:59:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.949+0000 7f4f7affd700 1 -- 172.21.15.7:0/293176240 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400b310 con 0x7f4f7c0c1660 2023-06-09T11:59:40.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 -- 172.21.15.7:0/293176240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c0c1660 msgr2=0x7f4f7c0c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 --2- 172.21.15.7:0/293176240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0c7b10 tx=0x7f4f7400d460 comp rx=0 tx=0).stop 2023-06-09T11:59:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 -- 172.21.15.7:0/293176240 shutdown_connections 2023-06-09T11:59:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 --2- 172.21.15.7:0/293176240 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c0c1a60 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 --2- 172.21.15.7:0/293176240 >> v2:172.21.15.117:3300/0 conn(0x7f4f7c1508b0 0x7f4f7c0c10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 -- 172.21.15.7:0/293176240 >> 172.21.15.7:0/293176240 conn(0x7f4f7c0cbd60 msgr2=0x7f4f7c0c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 -- 172.21.15.7:0/293176240 shutdown_connections 2023-06-09T11:59:40.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.950+0000 7f4f825ff700 1 -- 172.21.15.7:0/293176240 wait complete. 2023-06-09T11:59:40.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.952+0000 7f4f825ff700 1 Processor -- start 2023-06-09T11:59:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.952+0000 7f4f825ff700 1 -- start start 2023-06-09T11:59:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 0x7f4f7c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f7c1f1f80 0x7f4f7c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.958 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c0c4e20 con 0x7f4f7c1504e0 2023-06-09T11:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1eeca0 con 0x7f4f7c0c1660 2023-06-09T11:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f825ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1eef80 con 0x7f4f7c1f1f80 2023-06-09T11:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 0x7f4f7c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 0x7f4f7c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38836/0 (socket says 172.21.15.7:38836) 2023-06-09T11:59:40.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f7b7fe700 1 -- 172.21.15.7:0/3761309998 learned_addr learned my addr 172.21.15.7:0/3761309998 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f7bfff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.953+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f7c1f1f80 0x7f4f7c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 -- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f7c1f1f80 msgr2=0x7f4f7c156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f7c1f1f80 0x7f4f7c156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 -- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c0c1660 msgr2=0x7f4f7c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4f7c0c1660 0x7f4f7c06de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f7400bd40 con 0x7f4f7c1504e0 2023-06-09T11:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4f7c1f1f80 0x7f4f7c156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f7b7fe700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 0x7f4f7c1568e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e840 tx=0x7f4f7000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:40.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70009590 con 0x7f4f7c1504e0 2023-06-09T11:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f70013680 con 0x7f4f7c1504e0 2023-06-09T11:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70013a30 con 0x7f4f7c1504e0 2023-06-09T11:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.954+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c06b680 con 0x7f4f7c1504e0 2023-06-09T11:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.955+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c0b0e40 con 0x7f4f7c1504e0 2023-06-09T11:59:40.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.955+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f7c06b960 con 0x7f4f7c1504e0 2023-06-09T11:59:40.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.958+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4f70013b90 con 0x7f4f7c1504e0 2023-06-09T11:59:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f78ff9700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4f6806ace0 0x7f4f6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f4f7008a410 con 0x7f4f7c1504e0 2023-06-09T11:59:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f7bfff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4f6806ace0 0x7f4f6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f78ff9700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f4f68074970 0x7f4f68076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4f68077470 con 0x7f4f68074970 2023-06-09T11:59:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f7008a780 con 0x7f4f7c1504e0 2023-06-09T11:59:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.959+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f4f68074970 0x7f4f68076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.960+0000 7f4f7bfff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4f6806ace0 0x7f4f6806d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c007fd0 tx=0x7f4f6c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:40.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.960+0000 7f4f80b9c700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f4f68074970 0x7f4f68076da0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:40.966 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.961+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== osd.0 v2:172.21.15.7:6808/1473672210 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4f68077470 con 0x7f4f68074970 2023-06-09T11:59:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.976+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 --> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4f7c0c27b0 con 0x7f4f68074970 2023-06-09T11:59:40.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.976+0000 7f4f78ff9700 1 -- 172.21.15.7:0/3761309998 <== osd.0 v2:172.21.15.7:6808/1473672210 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4f7c0c27b0 con 0x7f4f68074970 2023-06-09T11:59:40.979 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:40.979 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.976+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f4f68074970 msgr2=0x7f4f68076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.976+0000 7f4f825ff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6808/1473672210,v1:172.21.15.7:6809/1473672210] conn(0x7f4f68074970 0x7f4f68076da0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4f6806ace0 msgr2=0x7f4f6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f4f6806ace0 0x7f4f6806d190 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c007fd0 tx=0x7f4f6c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 msgr2=0x7f4f7c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 --2- 172.21.15.7:0/3761309998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f7c1504e0 0x7f4f7c1568e0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f4f7000e840 tx=0x7f4f7000eb50 comp rx=0 tx=0).stop 2023-06-09T11:59:40.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f80b9c700 1 -- 172.21.15.7:0/3761309998 reap_dead start 2023-06-09T11:59:40.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 shutdown_connections 2023-06-09T11:59:40.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 >> 172.21.15.7:0/3761309998 conn(0x7f4f7c0cbd60 msgr2=0x7f4f7c0c5760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:40.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 shutdown_connections 2023-06-09T11:59:40.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:40.977+0000 7f4f825ff700 1 -- 172.21.15.7:0/3761309998 wait complete. 2023-06-09T11:59:40.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-06-09T11:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.196+0000 7fb94bdff700 1 Processor -- start 2023-06-09T11:59:41.199 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.197+0000 7fb94bdff700 1 -- start start 2023-06-09T11:59:41.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 0x7fb94415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.200 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94bdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94415a500 con 0x7fb94415b4a0 2023-06-09T11:59:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94bdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94415da00 con 0x7fb9440c9f40 2023-06-09T11:59:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440c5f10 con 0x7fb944153880 2023-06-09T11:59:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94a39c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb94415b4a0 0x7fb9440c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34080/0 (socket says 172.21.15.7:34080) 2023-06-09T11:59:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.198+0000 7fb94a39c700 1 -- 172.21.15.7:0/174121991 learned_addr learned my addr 172.21.15.7:0/174121991 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb949b9b700 1 --2- 172.21.15.7:0/174121991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 0x7fb94415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080360238 0 0) 0x7fb93c001490 con 0x7fb94415b4a0 2023-06-09T11:59:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9300035b0 con 0x7fb94415b4a0 2023-06-09T11:59:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550387361 0 0) 0x7fb934001110 con 0x7fb9440c9f40 2023-06-09T11:59:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb93c001490 con 0x7fb9440c9f40 2023-06-09T11:59:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb949b9b700 1 -- 172.21.15.7:0/174121991 >> v1:172.21.15.117:6789/0 conn(0x7fb9440c9f40 legacy=0x7fb94415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb949b9b700 1 -- 172.21.15.7:0/174121991 >> v1:172.21.15.7:6789/0 conn(0x7fb94415b4a0 legacy=0x7fb9440c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb949b9b700 1 -- 172.21.15.7:0/174121991 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9441596e0 con 0x7fb944153880 2023-06-09T11:59:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.199+0000 7fb949b9b700 1 --2- 172.21.15.7:0/174121991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 0x7fb94415d4c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb940009870 tx=0x7fb940009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f0e0edf8baf0963 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.200+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940011830 con 0x7fb944153880 2023-06-09T11:59:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.200+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb940011990 con 0x7fb944153880 2023-06-09T11:59:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.200+0000 7fb948b99700 1 -- 172.21.15.7:0/174121991 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940011d00 con 0x7fb944153880 2023-06-09T11:59:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.200+0000 7fb94bdff700 1 -- 172.21.15.7:0/174121991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 msgr2=0x7fb94415d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.200+0000 7fb94bdff700 1 --2- 172.21.15.7:0/174121991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 0x7fb94415d4c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb940009870 tx=0x7fb940009b80 comp rx=0 tx=0).stop 2023-06-09T11:59:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.201+0000 7fb94bdff700 1 -- 172.21.15.7:0/174121991 shutdown_connections 2023-06-09T11:59:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.201+0000 7fb94bdff700 1 --2- 172.21.15.7:0/174121991 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb944153880 0x7fb94415d4c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.201+0000 7fb94bdff700 1 -- 172.21.15.7:0/174121991 >> 172.21.15.7:0/174121991 conn(0x7fb9440737f0 msgr2=0x7fb9440720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.201+0000 7fb94bdff700 1 -- 172.21.15.7:0/174121991 shutdown_connections 2023-06-09T11:59:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.201+0000 7fb94bdff700 1 -- 172.21.15.7:0/174121991 wait complete. 2023-06-09T11:59:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.203+0000 7fb94bdff700 1 Processor -- start 2023-06-09T11:59:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.203+0000 7fb94bdff700 1 -- start start 2023-06-09T11:59:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.204+0000 7fb94bdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.204+0000 7fb94bdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 0x7fb94406ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94bdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94bdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94414d0e0 con 0x7fb94415b4a0 2023-06-09T11:59:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94bdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9441f2a20 con 0x7fb944153880 2023-06-09T11:59:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94bdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9440cc290 con 0x7fb9440c9f40 2023-06-09T11:59:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb949b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb949b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:59514/0 (socket says 172.21.15.7:59514) 2023-06-09T11:59:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38838/0 (socket says 172.21.15.7:38838) 2023-06-09T11:59:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb949b9b700 1 -- 172.21.15.7:0/3562751121 learned_addr learned my addr 172.21.15.7:0/3562751121 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94939a700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 0x7fb94406ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 -- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 msgr2=0x7fb94406a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 -- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 msgr2=0x7fb94406ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 0x7fb94406ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94a39c700 1 -- 172.21.15.7:0/3562751121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400094a0 con 0x7fb94415b4a0 2023-06-09T11:59:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb949b9b700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.205+0000 7fb94939a700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 0x7fb94406ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb94a39c700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb94415b900 tx=0x7fb93c00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00aaa0 con 0x7fb94415b4a0 2023-06-09T11:59:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c00ac00 con 0x7fb94415b4a0 2023-06-09T11:59:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c012400 con 0x7fb94415b4a0 2023-06-09T11:59:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb944159c60 con 0x7fb94415b4a0 2023-06-09T11:59:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.206+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb944159980 con 0x7fb94415b4a0 2023-06-09T11:59:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.207+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb944159f40 con 0x7fb94415b4a0 2023-06-09T11:59:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.208+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb93c01b3b0 con 0x7fb94415b4a0 2023-06-09T11:59:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.209+0000 7fb93a7fc700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 0x7fb92806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.209+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb93c089950 con 0x7fb94415b4a0 2023-06-09T11:59:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.209+0000 7fb949b9b700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 0x7fb92806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.210+0000 7fb949b9b700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 0x7fb92806d1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb940000c00 tx=0x7fb94001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.214+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb93c0540e0 con 0x7fb94415b4a0 2023-06-09T11:59:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.422+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb94415a500 con 0x7fb92806ad10 2023-06-09T11:59:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.422+0000 7fb93a7fc700 1 -- 172.21.15.7:0/3562751121 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb94415a500 con 0x7fb92806ad10 2023-06-09T11:59:41.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 msgr2=0x7fb92806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 0x7fb92806d1c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb940000c00 tx=0x7fb94001c040 comp rx=0 tx=0).stop 2023-06-09T11:59:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 msgr2=0x7fb9441f2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fb94415b900 tx=0x7fb93c00cee0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 shutdown_connections 2023-06-09T11:59:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb94415b4a0 0x7fb9441f2eb0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb92806ad10 0x7fb92806d1c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9440c9f40 0x7fb94406a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 --2- 172.21.15.7:0/3562751121 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb944153880 0x7fb94406ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 >> 172.21.15.7:0/3562751121 conn(0x7fb9440737f0 msgr2=0x7fb94415b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 shutdown_connections 2023-06-09T11:59:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.425+0000 7fb94bdff700 1 -- 172.21.15.7:0/3562751121 wait complete. 2023-06-09T11:59:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2023-06-09T11:59:41.446 INFO:tasks.ceph:Scrubbing osd.1 2023-06-09T11:59:41.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:41.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.648+0000 7fb9d6fff700 1 Processor -- start 2023-06-09T11:59:41.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.648+0000 7fb9d6fff700 1 -- start start 2023-06-09T11:59:41.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.649+0000 7fb9d6fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d00cc100 0x7fb9d015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.649+0000 7fb9d6fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9d00b5270 0x7fb9d00c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.649+0000 7fb9d6fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d015dd00 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.649+0000 7fb9d6fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d0071790 con 0x7fb9d00cc100 2023-06-09T11:59:41.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d6fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d00ca3f0 con 0x7fb9d00b5270 2023-06-09T11:59:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d00cc100 0x7fb9d015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d559c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb9d00b5270 0x7fb9d00c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb9d00cc100 0x7fb9d015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38848/0 (socket says 172.21.15.7:38848) 2023-06-09T11:59:41.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 -- 172.21.15.7:0/4184711387 learned_addr learned my addr 172.21.15.7:0/4184711387 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9cffff700 1 --1- 172.21.15.7:0/4184711387 >> v1:172.21.15.117:6789/0 conn(0x7fb9d00cc4d0 0x7fb9d00cac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:33962/0 (socket says 172.21.15.7:33962) 2023-06-09T11:59:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 -- 172.21.15.7:0/4184711387 >> v2:172.21.15.117:3300/0 conn(0x7fb9d00b5270 msgr2=0x7fb9d00c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 --2- 172.21.15.7:0/4184711387 >> v2:172.21.15.117:3300/0 conn(0x7fb9d00b5270 0x7fb9d00c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 -- 172.21.15.7:0/4184711387 >> v1:172.21.15.117:6789/0 conn(0x7fb9d00cc4d0 legacy=0x7fb9d00cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.650+0000 7fb9d4d9b700 1 -- 172.21.15.7:0/4184711387 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0159840 con 0x7fb9d00cc100 2023-06-09T11:59:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d4d9b700 1 --2- 172.21.15.7:0/4184711387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc100 0x7fb9d015d7c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0159a30 tx=0x7fb9c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fbabd13bac95a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9cf7fe700 1 -- 172.21.15.7:0/4184711387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4004230 con 0x7fb9d00cc100 2023-06-09T11:59:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9cf7fe700 1 -- 172.21.15.7:0/4184711387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4004390 con 0x7fb9d00cc100 2023-06-09T11:59:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9cf7fe700 1 -- 172.21.15.7:0/4184711387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c40036a0 con 0x7fb9d00cc100 2023-06-09T11:59:41.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 -- 172.21.15.7:0/4184711387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc100 msgr2=0x7fb9d015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/4184711387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc100 0x7fb9d015d7c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0159a30 tx=0x7fb9c4009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 -- 172.21.15.7:0/4184711387 shutdown_connections 2023-06-09T11:59:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/4184711387 >> v2:172.21.15.117:3300/0 conn(0x7fb9d00b5270 0x7fb9d00c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/4184711387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc100 0x7fb9d015d7c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.651+0000 7fb9d6fff700 1 -- 172.21.15.7:0/4184711387 >> 172.21.15.7:0/4184711387 conn(0x7fb9d0070af0 msgr2=0x7fb9d0070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:41.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.652+0000 7fb9d6fff700 1 -- 172.21.15.7:0/4184711387 shutdown_connections 2023-06-09T11:59:41.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.652+0000 7fb9d6fff700 1 -- 172.21.15.7:0/4184711387 wait complete. 2023-06-09T11:59:41.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.653+0000 7fb9d6fff700 1 Processor -- start 2023-06-09T11:59:41.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.653+0000 7fb9d6fff700 1 -- start start 2023-06-09T11:59:41.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d00b5270 0x7fb9d00c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d00cc100 0x7fb9d01e38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 0x7fb9d0155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d01f03c0 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00d0050 con 0x7fb9d00cc100 2023-06-09T11:59:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.654+0000 7fb9d6fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00d02f0 con 0x7fb9d00b5270 2023-06-09T11:59:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 0x7fb9d0155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 0x7fb9d0155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38862/0 (socket says 172.21.15.7:38862) 2023-06-09T11:59:41.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 -- 172.21.15.7:0/1145209118 learned_addr learned my addr 172.21.15.7:0/1145209118 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9cffff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d00cc100 0x7fb9d01e38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d4d9b700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d00b5270 0x7fb9d00c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 -- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d00b5270 msgr2=0x7fb9d00c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb9d00b5270 0x7fb9d00c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 -- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d00cc100 msgr2=0x7fb9d01e38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb9d00cc100 0x7fb9d01e38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.655+0000 7fb9d559c700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009550 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9d559c700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 0x7fb9d0155940 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800efc0 tx=0x7fb9c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8009ea0 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c800f040 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c800ced0 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d00ca6b0 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.656+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d00ca120 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.657+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9d0159b00 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.658+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9c8014480 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.659+0000 7fb9cd7fa700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9b406ace0 0x7fb9b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.659+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb9c808a860 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.659+0000 7fb9d4d9b700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9b406ace0 0x7fb9b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.659+0000 7fb9cd7fa700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7fb9b4074800 0x7fb9b4076c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.660+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9b4077300 con 0x7fb9b4074800 2023-06-09T11:59:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.660+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c808ce40 con 0x7fb9d00cc4d0 2023-06-09T11:59:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.660+0000 7fb9cffff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7fb9b4074800 0x7fb9b4076c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.660+0000 7fb9d4d9b700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9b406ace0 0x7fb9b406d190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009130 tx=0x7fb9c4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.660+0000 7fb9cffff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7fb9b4074800 0x7fb9b4076c30 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.661+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== osd.1 v2:172.21.15.7:6800/48537347 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb9b4077300 con 0x7fb9b4074800 2023-06-09T11:59:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.676+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 --> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb9d00c9c40 con 0x7fb9b4074800 2023-06-09T11:59:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9cd7fa700 1 -- 172.21.15.7:0/1145209118 <== osd.1 v2:172.21.15.7:6800/48537347 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb9d00c9c40 con 0x7fb9b4074800 2023-06-09T11:59:41.680 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:41.680 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:41.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7fb9b4074800 msgr2=0x7fb9b4076c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6800/48537347,v1:172.21.15.7:6801/48537347] conn(0x7fb9b4074800 0x7fb9b4076c30 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9b406ace0 msgr2=0x7fb9b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb9b406ace0 0x7fb9b406d190 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009130 tx=0x7fb9c4008030 comp rx=0 tx=0).stop 2023-06-09T11:59:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 msgr2=0x7fb9d0155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 --2- 172.21.15.7:0/1145209118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d00cc4d0 0x7fb9d0155940 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb9c800efc0 tx=0x7fb9c800c3a0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d559c700 1 -- 172.21.15.7:0/1145209118 reap_dead start 2023-06-09T11:59:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 shutdown_connections 2023-06-09T11:59:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.677+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 >> 172.21.15.7:0/1145209118 conn(0x7fb9d0070af0 msgr2=0x7fb9d015cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.678+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 shutdown_connections 2023-06-09T11:59:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.678+0000 7fb9d6fff700 1 -- 172.21.15.7:0/1145209118 wait complete. 2023-06-09T11:59:41.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-06-09T11:59:41.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.896+0000 7fc1211ff700 1 Processor -- start 2023-06-09T11:59:41.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.896+0000 7fc1211ff700 1 -- start start 2023-06-09T11:59:41.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc1211ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 0x7fc11c06a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc1211ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c06a4d0 con 0x7fc11c0bf5d0 2023-06-09T11:59:41.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc1211ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0cc280 con 0x7fc11c073d80 2023-06-09T11:59:41.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc1211ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c073a10 con 0x7fc11c074150 2023-06-09T11:59:41.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc11a59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc11c0bf5d0 0x7fc11c1534b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34084/0 (socket says 172.21.15.7:34084) 2023-06-09T11:59:41.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc11a59c700 1 -- 172.21.15.7:0/1521790607 learned_addr learned my addr 172.21.15.7:0/1521790607 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:41.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc11b59e700 1 --2- 172.21.15.7:0/1521790607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 0x7fc11c06a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.898+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082469288 0 0) 0x7fc11c06a4d0 con 0x7fc11c0bf5d0 2023-06-09T11:59:41.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1040035b0 con 0x7fc11c0bf5d0 2023-06-09T11:59:41.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257989637 0 0) 0x7fc11c0cc280 con 0x7fc11c073d80 2023-06-09T11:59:41.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc11c06a4d0 con 0x7fc11c073d80 2023-06-09T11:59:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc11b59e700 1 -- 172.21.15.7:0/1521790607 >> v1:172.21.15.117:6789/0 conn(0x7fc11c073d80 legacy=0x7fc11c06b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc11b59e700 1 -- 172.21.15.7:0/1521790607 >> v1:172.21.15.7:6789/0 conn(0x7fc11c0bf5d0 legacy=0x7fc11c1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc11b59e700 1 -- 172.21.15.7:0/1521790607 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c06bdc0 con 0x7fc11c074150 2023-06-09T11:59:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc11b59e700 1 --2- 172.21.15.7:0/1521790607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 0x7fc11c06a960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc114009dc0 tx=0x7fc11400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee4e03f34e968577 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11400e070 con 0x7fc11c074150 2023-06-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11400bb20 con 0x7fc11c074150 2023-06-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.899+0000 7fc119d9b700 1 -- 172.21.15.7:0/1521790607 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114012430 con 0x7fc11c074150 2023-06-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 -- 172.21.15.7:0/1521790607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 msgr2=0x7fc11c06a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1521790607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 0x7fc11c06a960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc114009dc0 tx=0x7fc11400b330 comp rx=0 tx=0).stop 2023-06-09T11:59:41.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 -- 172.21.15.7:0/1521790607 shutdown_connections 2023-06-09T11:59:41.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1521790607 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c074150 0x7fc11c06a960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 -- 172.21.15.7:0/1521790607 >> 172.21.15.7:0/1521790607 conn(0x7fc11c06d150 msgr2=0x7fc11c06d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:41.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 -- 172.21.15.7:0/1521790607 shutdown_connections 2023-06-09T11:59:41.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.900+0000 7fc1211ff700 1 -- 172.21.15.7:0/1521790607 wait complete. 2023-06-09T11:59:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.902+0000 7fc1211ff700 1 Processor -- start 2023-06-09T11:59:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.902+0000 7fc1211ff700 1 -- start start 2023-06-09T11:59:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc11c074150 0x7fc11c0bd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c0bf5d0 0x7fc11c14cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c0c66c0 con 0x7fc11c073d80 2023-06-09T11:59:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c159980 con 0x7fc11c074150 2023-06-09T11:59:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc1211ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c159e10 con 0x7fc11c0bf5d0 2023-06-09T11:59:41.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc11ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc11ad9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38864/0 (socket says 172.21.15.7:38864) 2023-06-09T11:59:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc11ad9d700 1 -- 172.21.15.7:0/1330730220 learned_addr learned my addr 172.21.15.7:0/1330730220 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc11a59c700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc11c074150 0x7fc11c0bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.903+0000 7fc11b59e700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c0bf5d0 0x7fc11c14cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 -- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c0bf5d0 msgr2=0x7fc11c14cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c0bf5d0 0x7fc11c14cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 -- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc11c074150 msgr2=0x7fc11c0bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc11c074150 0x7fc11c0bd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 -- 172.21.15.7:0/1330730220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc114009a70 con 0x7fc11c073d80 2023-06-09T11:59:41.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc11ad9d700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c520 tx=0x7fc10c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0052d0 con 0x7fc11c073d80 2023-06-09T11:59:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c005430 con 0x7fc11c073d80 2023-06-09T11:59:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10c0038c0 con 0x7fc11c073d80 2023-06-09T11:59:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c0cc4d0 con 0x7fc11c073d80 2023-06-09T11:59:41.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.904+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c0c8700 con 0x7fc11c073d80 2023-06-09T11:59:41.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.905+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c0cc7b0 con 0x7fc11c073d80 2023-06-09T11:59:41.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.906+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc10c003a20 con 0x7fc11c073d80 2023-06-09T11:59:41.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.908+0000 7fc1037fe700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 0x7fc10806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:41.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.908+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fc10c0892f0 con 0x7fc11c073d80 2023-06-09T11:59:41.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.908+0000 7fc11a59c700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 0x7fc10806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:41.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.908+0000 7fc11a59c700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 0x7fc10806d190 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0c68c0 tx=0x7fc110007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:41.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:41.912+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc10c053990 con 0x7fc11c073d80 2023-06-09T11:59:42.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.120+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c151d60 con 0x7fc10806ace0 2023-06-09T11:59:42.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.120+0000 7fc1037fe700 1 -- 172.21.15.7:0/1330730220 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc11c151d60 con 0x7fc10806ace0 2023-06-09T11:59:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 msgr2=0x7fc10806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 0x7fc10806d190 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0c68c0 tx=0x7fc110007dc0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 msgr2=0x7fc11c158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00c520 tx=0x7fc10c00c830 comp rx=0 tx=0).stop 2023-06-09T11:59:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 shutdown_connections 2023-06-09T11:59:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc10806ace0 0x7fc10806d190 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc11c0bf5d0 0x7fc11c14cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc11c073d80 0x7fc11c158430 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 --2- 172.21.15.7:0/1330730220 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc11c074150 0x7fc11c0bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 >> 172.21.15.7:0/1330730220 conn(0x7fc11c06d150 msgr2=0x7fc11c1e4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 shutdown_connections 2023-06-09T11:59:42.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.123+0000 7fc1211ff700 1 -- 172.21.15.7:0/1330730220 wait complete. 2023-06-09T11:59:42.128 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2023-06-09T11:59:42.143 INFO:tasks.ceph:Scrubbing osd.2 2023-06-09T11:59:42.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:42.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.346+0000 7fe68cbff700 1 Processor -- start 2023-06-09T11:59:42.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.347+0000 7fe68cbff700 1 -- start start 2023-06-09T11:59:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6881513d0 0x7fe6880c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68cbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6880b0e70 0x7fe6880c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68cbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe688072520 con 0x7fe6880c1aa0 2023-06-09T11:59:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68cbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe688157580 con 0x7fe6880b0e70 2023-06-09T11:59:42.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe688157860 con 0x7fe6881513d0 2023-06-09T11:59:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe686d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6880b0e70 0x7fe6880c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe686d9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fe6880b0e70 0x7fe6880c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38874/0 (socket says 172.21.15.7:38874) 2023-06-09T11:59:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe686d9d700 1 -- 172.21.15.7:0/999512638 learned_addr learned my addr 172.21.15.7:0/999512638 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.348+0000 7fe68659c700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6881513d0 0x7fe6880c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe686d9d700 1 -- 172.21.15.7:0/999512638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6881513d0 msgr2=0x7fe6880c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe686d9d700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6881513d0 0x7fe6880c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe686d9d700 1 -- 172.21.15.7:0/999512638 >> v1:172.21.15.7:6789/0 conn(0x7fe6880c1aa0 legacy=0x7fe6880bffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe686d9d700 1 -- 172.21.15.7:0/999512638 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe688157b40 con 0x7fe6880b0e70 2023-06-09T11:59:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe686d9d700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880b0e70 0x7fe6880c6e30 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe680009e50 tx=0x7fe68000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b778c5ecd63446e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe685d9b700 1 -- 172.21.15.7:0/999512638 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68000e070 con 0x7fe6880b0e70 2023-06-09T11:59:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe685d9b700 1 -- 172.21.15.7:0/999512638 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68000b9c0 con 0x7fe6880b0e70 2023-06-09T11:59:42.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.349+0000 7fe685d9b700 1 -- 172.21.15.7:0/999512638 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68000bd70 con 0x7fe6880b0e70 2023-06-09T11:59:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 -- 172.21.15.7:0/999512638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880b0e70 msgr2=0x7fe6880c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880b0e70 0x7fe6880c6e30 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fe680009e50 tx=0x7fe68000b330 comp rx=0 tx=0).stop 2023-06-09T11:59:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 -- 172.21.15.7:0/999512638 shutdown_connections 2023-06-09T11:59:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880b0e70 0x7fe6880c6e30 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 --2- 172.21.15.7:0/999512638 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6881513d0 0x7fe6880c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 -- 172.21.15.7:0/999512638 >> 172.21.15.7:0/999512638 conn(0x7fe6880cbd60 msgr2=0x7fe6880c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 -- 172.21.15.7:0/999512638 shutdown_connections 2023-06-09T11:59:42.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.350+0000 7fe68cbff700 1 -- 172.21.15.7:0/999512638 wait complete. 2023-06-09T11:59:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.352+0000 7fe68cbff700 1 Processor -- start 2023-06-09T11:59:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.353+0000 7fe68cbff700 1 -- start start 2023-06-09T11:59:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6880b0e70 0x7fe6880c2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 0x7fe6880c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6881513d0 0x7fe6880c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880c3090 con 0x7fe6880c1aa0 2023-06-09T11:59:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881efa80 con 0x7fe6881513d0 2023-06-09T11:59:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68cbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881efce0 con 0x7fe6880b0e70 2023-06-09T11:59:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 0x7fe6880c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 0x7fe6880c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38888/0 (socket says 172.21.15.7:38888) 2023-06-09T11:59:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 -- 172.21.15.7:0/682780379 learned_addr learned my addr 172.21.15.7:0/682780379 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe68659c700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6880b0e70 0x7fe6880c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe686d9d700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6881513d0 0x7fe6880c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 -- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6880b0e70 msgr2=0x7fe6880c2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe6880b0e70 0x7fe6880c2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.354+0000 7fe67ffff700 1 -- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6881513d0 msgr2=0x7fe6880c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67ffff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe6881513d0 0x7fe6880c4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67ffff700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe680009b30 con 0x7fe6880c1aa0 2023-06-09T11:59:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67ffff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 0x7fe6880c42d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c4e0 tx=0x7fe67000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700052d0 con 0x7fe6880c1aa0 2023-06-09T11:59:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670005430 con 0x7fe6880c1aa0 2023-06-09T11:59:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700038c0 con 0x7fe6880c1aa0 2023-06-09T11:59:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe688158100 con 0x7fe6880c1aa0 2023-06-09T11:59:42.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.355+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe688157e20 con 0x7fe6880c1aa0 2023-06-09T11:59:42.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.356+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe688072230 con 0x7fe6880c1aa0 2023-06-09T11:59:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.357+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe670003a20 con 0x7fe6880c1aa0 2023-06-09T11:59:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.358+0000 7fe67effd700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe66806ace0 0x7fe66806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.358+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fe6700890b0 con 0x7fe6880c1aa0 2023-06-09T11:59:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.358+0000 7fe68659c700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe66806ace0 0x7fe66806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe67effd700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7fe668074970 0x7fe668076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe668077470 con 0x7fe668074970 2023-06-09T11:59:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67008b710 con 0x7fe6880c1aa0 2023-06-09T11:59:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe686d9d700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7fe668074970 0x7fe668076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe68659c700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe66806ace0 0x7fe66806d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe678006b20 tx=0x7fe678008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.359+0000 7fe686d9d700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7fe668074970 0x7fe668076da0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.360+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== osd.2 v2:172.21.15.7:6816/843671507 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe668077470 con 0x7fe668074970 2023-06-09T11:59:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 --> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe688074cf0 con 0x7fe668074970 2023-06-09T11:59:42.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe67effd700 1 -- 172.21.15.7:0/682780379 <== osd.2 v2:172.21.15.7:6816/843671507 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe688074cf0 con 0x7fe668074970 2023-06-09T11:59:42.379 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:42.379 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:42.379 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7fe668074970 msgr2=0x7fe668076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6816/843671507,v1:172.21.15.7:6817/843671507] conn(0x7fe668074970 0x7fe668076da0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe66806ace0 msgr2=0x7fe66806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fe66806ace0 0x7fe66806d190 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe678006b20 tx=0x7fe678008040 comp rx=0 tx=0).stop 2023-06-09T11:59:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 msgr2=0x7fe6880c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe68cbff700 1 --2- 172.21.15.7:0/682780379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6880c1aa0 0x7fe6880c42d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c4e0 tx=0x7fe67000c7f0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.376+0000 7fe686d9d700 1 -- 172.21.15.7:0/682780379 reap_dead start 2023-06-09T11:59:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.377+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 shutdown_connections 2023-06-09T11:59:42.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.377+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 >> 172.21.15.7:0/682780379 conn(0x7fe6880cbd60 msgr2=0x7fe6881552e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.377+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 shutdown_connections 2023-06-09T11:59:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.377+0000 7fe68cbff700 1 -- 172.21.15.7:0/682780379 wait complete. 2023-06-09T11:59:42.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-06-09T11:59:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.595+0000 7f071e5ff700 1 Processor -- start 2023-06-09T11:59:42.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.595+0000 7f071e5ff700 1 -- start start 2023-06-09T11:59:42.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.596+0000 7f071e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07180cb9a0 0x7f07180b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.596+0000 7f071e5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071815cd40 0x7f07180cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.596+0000 7f071e5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0718159b20 con 0x7f07180cb5d0 2023-06-09T11:59:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.596+0000 7f071e5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180cac80 con 0x7f071815cd40 2023-06-09T11:59:42.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.596+0000 7f071e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07181597d0 con 0x7f07180cb9a0 2023-06-09T11:59:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071815cd40 0x7f07180cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f071815cd40 0x7f07180cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38898/0 (socket says 172.21.15.7:38898) 2023-06-09T11:59:42.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 -- 172.21.15.7:0/2180140177 learned_addr learned my addr 172.21.15.7:0/2180140177 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:42.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f07177fe700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07180cb9a0 0x7f07180b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 -- 172.21.15.7:0/2180140177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07180cb9a0 msgr2=0x7f07180b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07180cb9a0 0x7f07180b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 -- 172.21.15.7:0/2180140177 >> v1:172.21.15.117:6789/0 conn(0x7f07180cb5d0 legacy=0x7f0718159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 -- 172.21.15.7:0/2180140177 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718071ab0 con 0x7f071815cd40 2023-06-09T11:59:42.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.597+0000 7f071cb9c700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071815cd40 0x7f07180cbea0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0710009e50 tx=0x7f071000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108330ae67d5b088 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f0716ffd700 1 -- 172.21.15.7:0/2180140177 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000e070 con 0x7f071815cd40 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f0716ffd700 1 -- 172.21.15.7:0/2180140177 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071000b9c0 con 0x7f071815cd40 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f0716ffd700 1 -- 172.21.15.7:0/2180140177 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071000bd70 con 0x7f071815cd40 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 -- 172.21.15.7:0/2180140177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071815cd40 msgr2=0x7f07180cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071815cd40 0x7f07180cbea0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f0710009e50 tx=0x7f071000b330 comp rx=0 tx=0).stop 2023-06-09T11:59:42.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 -- 172.21.15.7:0/2180140177 shutdown_connections 2023-06-09T11:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f071815cd40 0x7f07180cbea0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 --2- 172.21.15.7:0/2180140177 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f07180cb9a0 0x7f07180b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 -- 172.21.15.7:0/2180140177 >> 172.21.15.7:0/2180140177 conn(0x7f0718070af0 msgr2=0x7f0718070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 -- 172.21.15.7:0/2180140177 shutdown_connections 2023-06-09T11:59:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.598+0000 7f071e5ff700 1 -- 172.21.15.7:0/2180140177 wait complete. 2023-06-09T11:59:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.600+0000 7f071e5ff700 1 Processor -- start 2023-06-09T11:59:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.600+0000 7f071e5ff700 1 -- start start 2023-06-09T11:59:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07180cb5d0 0x7f07181f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f071815cd40 0x7f07180ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180cf920 con 0x7f07180cb9a0 2023-06-09T11:59:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07181f2bc0 con 0x7f07180cb5d0 2023-06-09T11:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.601+0000 7f071e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c81d0 con 0x7f071815cd40 2023-06-09T11:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38902/0 (socket says 172.21.15.7:38902) 2023-06-09T11:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 -- 172.21.15.7:0/3399862864 learned_addr learned my addr 172.21.15.7:0/3399862864 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f0717fff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07180cb5d0 0x7f07181f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f071cb9c700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f071815cd40 0x7f07180ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 -- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f071815cd40 msgr2=0x7f07180ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f071815cd40 0x7f07180ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 -- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07180cb5d0 msgr2=0x7f07181f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07180cb5d0 0x7f07181f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 -- 172.21.15.7:0/3399862864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0710009b30 con 0x7f07180cb9a0 2023-06-09T11:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.602+0000 7f07177fe700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f070c00e7e0 tx=0x7f070c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.603+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c00c070 con 0x7f07180cb9a0 2023-06-09T11:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.603+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c013650 con 0x7f07180cb9a0 2023-06-09T11:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.603+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c013a00 con 0x7f07180cb9a0 2023-06-09T11:59:42.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.603+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071815d4f0 con 0x7f07180cb9a0 2023-06-09T11:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.603+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071815d210 con 0x7f07180cb9a0 2023-06-09T11:59:42.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.604+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071815d7d0 con 0x7f07180cb9a0 2023-06-09T11:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.605+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f070c013b60 con 0x7f07180cb9a0 2023-06-09T11:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.605+0000 7f0714ff9700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 0x7f070406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.606+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f070c089b50 con 0x7f07180cb9a0 2023-06-09T11:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.606+0000 7f0717fff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 0x7f070406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:42.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.606+0000 7f0717fff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 0x7f070406d1c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0708007fd0 tx=0x7f0708009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:42.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.610+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f070c054260 con 0x7f07180cb9a0 2023-06-09T11:59:42.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.820+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0718074cf0 con 0x7f070406ad10 2023-06-09T11:59:42.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.821+0000 7f0714ff9700 1 -- 172.21.15.7:0/3399862864 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0718074cf0 con 0x7f070406ad10 2023-06-09T11:59:42.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 msgr2=0x7f070406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 0x7f070406d1c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f0708007fd0 tx=0x7f0708009040 comp rx=0 tx=0).stop 2023-06-09T11:59:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 msgr2=0x7f0718158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f070c00e7e0 tx=0x7f070c00eaf0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 shutdown_connections 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f070406ad10 0x7f070406d1c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f071815cd40 0x7f07180ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07180cb5d0 0x7f07181f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 --2- 172.21.15.7:0/3399862864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07180cb9a0 0x7f0718158430 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 >> 172.21.15.7:0/3399862864 conn(0x7f0718070af0 msgr2=0x7f0718158e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:42.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 shutdown_connections 2023-06-09T11:59:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:42.823+0000 7f071e5ff700 1 -- 172.21.15.7:0/3399862864 wait complete. 2023-06-09T11:59:42.828 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2023-06-09T11:59:42.843 INFO:tasks.ceph:Scrubbing osd.3 2023-06-09T11:59:42.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.045+0000 7f24b13ff700 1 Processor -- start 2023-06-09T11:59:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.045+0000 7f24b13ff700 1 -- start start 2023-06-09T11:59:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24b13ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24b13ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24b13ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ac15a4e0 con 0x7f24ac0cf850 2023-06-09T11:59:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24b13ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ac0b2360 con 0x7f24ac0cfc20 2023-06-09T11:59:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24b13ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0b2c00 con 0x7f24ac0c8e50 2023-06-09T11:59:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24ab7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24aa7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24aa7fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:48062/0 (socket says 172.21.15.7:48062) 2023-06-09T11:59:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1156714746 learned_addr learned my addr 172.21.15.7:0/1156714746 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.047+0000 7f24ab7fe700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:54256/0 (socket says 172.21.15.7:54256) 2023-06-09T11:59:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1156714746 >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 msgr2=0x7f24ac156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24aa7fc700 1 --2- 172.21.15.7:0/1156714746 >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1156714746 >> v1:172.21.15.117:6789/0 conn(0x7f24ac0cf850 legacy=0x7f24ac0b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1156714746 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ac06b4b0 con 0x7f24ac0c8e50 2023-06-09T11:59:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24ab7fe700 1 --2- 172.21.15.7:0/1156714746 >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T11:59:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24aa7fc700 1 --2- 172.21.15.7:0/1156714746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098f0 tx=0x7f24a0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10722f432f3144e3 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24a9ffb700 1 -- 172.21.15.7:0/1156714746 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0011830 con 0x7f24ac0c8e50 2023-06-09T11:59:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24a9ffb700 1 -- 172.21.15.7:0/1156714746 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a0011990 con 0x7f24ac0c8e50 2023-06-09T11:59:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.048+0000 7f24a9ffb700 1 -- 172.21.15.7:0/1156714746 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a0011d20 con 0x7f24ac0c8e50 2023-06-09T11:59:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 -- 172.21.15.7:0/1156714746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 msgr2=0x7f24ac159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1156714746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098f0 tx=0x7f24a0009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 -- 172.21.15.7:0/1156714746 shutdown_connections 2023-06-09T11:59:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1156714746 >> v2:172.21.15.117:3300/0 conn(0x7f24ac0cfc20 0x7f24ac156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1156714746 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac159fa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 -- 172.21.15.7:0/1156714746 >> 172.21.15.7:0/1156714746 conn(0x7f24ac072060 msgr2=0x7f24ac072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 -- 172.21.15.7:0/1156714746 shutdown_connections 2023-06-09T11:59:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.049+0000 7f24b13ff700 1 -- 172.21.15.7:0/1156714746 wait complete. 2023-06-09T11:59:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.051+0000 7f24b13ff700 1 Processor -- start 2023-06-09T11:59:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.051+0000 7f24b13ff700 1 -- start start 2023-06-09T11:59:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac1f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 0x7f24ac153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac0cfc20 0x7f24ac153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0c8650 con 0x7f24ac0cf850 2023-06-09T11:59:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac0c72b0 con 0x7f24ac0cfc20 2023-06-09T11:59:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24b13ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ac1f22f0 con 0x7f24ac0c8e50 2023-06-09T11:59:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24aa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 0x7f24ac153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24aa7fc700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 0x7f24ac153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52522/0 (socket says 172.21.15.7:52522) 2023-06-09T11:59:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1777827361 learned_addr learned my addr 172.21.15.7:0/1777827361 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24aaffd700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac1f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.052+0000 7f24ab7fe700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac0cfc20 0x7f24ac153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 msgr2=0x7f24ac1f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f24ac0c8e50 0x7f24ac1f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac0cfc20 msgr2=0x7f24ac153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f24ac0cfc20 0x7f24ac153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a4009a70 con 0x7f24ac0cf850 2023-06-09T11:59:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24aa7fc700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 0x7f24ac153820 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f24a0000c00 tx=0x7f24a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00043c0 con 0x7f24ac0cf850 2023-06-09T11:59:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a001f070 con 0x7f24ac0cf850 2023-06-09T11:59:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a0009550 con 0x7f24ac0cf850 2023-06-09T11:59:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a00038c0 con 0x7f24ac0cf850 2023-06-09T11:59:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.053+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ac0b4b70 con 0x7f24ac0cf850 2023-06-09T11:59:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.054+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f24ac0b5150 con 0x7f24ac0cf850 2023-06-09T11:59:43.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.055+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f24a0021030 con 0x7f24ac0cf850 2023-06-09T11:59:43.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.056+0000 7f24937fe700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f249806ad10 0x7f249806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.056+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f24a008b210 con 0x7f24ac0cf850 2023-06-09T11:59:43.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24aaffd700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f249806ad10 0x7f249806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24937fe700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7f24980749a0 0x7f2498076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f24980774a0 con 0x7f24980749a0 2023-06-09T11:59:43.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24a008d7f0 con 0x7f24ac0cf850 2023-06-09T11:59:43.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24ab7fe700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7f24980749a0 0x7f2498076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24aaffd700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f249806ad10 0x7f249806d1c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f249c007fd0 tx=0x7f249c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.057+0000 7f24ab7fe700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7f24980749a0 0x7f2498076dd0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.058+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== osd.3 v2:172.21.15.7:6824/353233282 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f24980774a0 con 0x7f24980749a0 2023-06-09T11:59:43.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.073+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 --> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f24ac0b53f0 con 0x7f24980749a0 2023-06-09T11:59:43.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.073+0000 7f24937fe700 1 -- 172.21.15.7:0/1777827361 <== osd.3 v2:172.21.15.7:6824/353233282 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f24ac0b53f0 con 0x7f24980749a0 2023-06-09T11:59:43.076 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:43.076 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:43.077 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:43.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7f24980749a0 msgr2=0x7f2498076dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6824/353233282,v1:172.21.15.7:6825/353233282] conn(0x7f24980749a0 0x7f2498076dd0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f249806ad10 msgr2=0x7f249806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f249806ad10 0x7f249806d1c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f249c007fd0 tx=0x7f249c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:43.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 msgr2=0x7f24ac153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 --2- 172.21.15.7:0/1777827361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24ac0cf850 0x7f24ac153820 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f24a0000c00 tx=0x7f24a0004750 comp rx=0 tx=0).stop 2023-06-09T11:59:43.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24ab7fe700 1 -- 172.21.15.7:0/1777827361 reap_dead start 2023-06-09T11:59:43.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 shutdown_connections 2023-06-09T11:59:43.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 >> 172.21.15.7:0/1777827361 conn(0x7f24ac072060 msgr2=0x7f24ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 shutdown_connections 2023-06-09T11:59:43.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.074+0000 7f24b13ff700 1 -- 172.21.15.7:0/1777827361 wait complete. 2023-06-09T11:59:43.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-06-09T11:59:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.290+0000 7fa9169ff700 1 Processor -- start 2023-06-09T11:59:43.293 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.291+0000 7fa9169ff700 1 -- start start 2023-06-09T11:59:43.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa9169ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa91015a0c0 0x7fa9101501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa9169ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa9100c0140 0x7fa9101506f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.294 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa9169ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100c0bf0 con 0x7fa9100c1bc0 2023-06-09T11:59:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa9169ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa910070110 con 0x7fa9100c0140 2023-06-09T11:59:43.295 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa9169ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa910071300 con 0x7fa91015a0c0 2023-06-09T11:59:43.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa90ffff700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa9100c1bc0 0x7fa9101599b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:46172/0 (socket says 172.21.15.7:46172) 2023-06-09T11:59:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa90ffff700 1 -- 172.21.15.7:0/763818336 learned_addr learned my addr 172.21.15.7:0/763818336 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa90f7fe700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa91015a0c0 0x7fa9101501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.292+0000 7fa914f9c700 1 --2- 172.21.15.7:0/763818336 >> v2:172.21.15.117:3300/0 conn(0x7fa9100c0140 0x7fa9101506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa90effd700 1 -- 172.21.15.7:0/763818336 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972249550 0 0) 0x7fa9100c0bf0 con 0x7fa9100c1bc0 2023-06-09T11:59:43.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa90effd700 1 -- 172.21.15.7:0/763818336 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8f80035b0 con 0x7fa9100c1bc0 2023-06-09T11:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa914f9c700 1 -- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa91015a0c0 msgr2=0x7fa9101501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa914f9c700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa91015a0c0 0x7fa9101501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa914f9c700 1 -- 172.21.15.7:0/763818336 >> v1:172.21.15.117:6789/0 conn(0x7fa9100c1bc0 legacy=0x7fa9101599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa914f9c700 1 -- 172.21.15.7:0/763818336 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100702f0 con 0x7fa9100c0140 2023-06-09T11:59:43.298 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa914f9c700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c0140 0x7fa9101506f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d9c72743ded809 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.293+0000 7fa90effd700 1 -- 172.21.15.7:0/763818336 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800e070 con 0x7fa9100c0140 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa90effd700 1 -- 172.21.15.7:0/763818336 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800bb20 con 0x7fa9100c0140 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa90effd700 1 -- 172.21.15.7:0/763818336 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908012430 con 0x7fa9100c0140 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 -- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c0140 msgr2=0x7fa9101506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c0140 0x7fa9101506f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90800b330 comp rx=0 tx=0).stop 2023-06-09T11:59:43.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 -- 172.21.15.7:0/763818336 shutdown_connections 2023-06-09T11:59:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c0140 0x7fa9101506f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 --2- 172.21.15.7:0/763818336 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa91015a0c0 0x7fa9101501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 -- 172.21.15.7:0/763818336 >> 172.21.15.7:0/763818336 conn(0x7fa91006a4d0 msgr2=0x7fa91006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 -- 172.21.15.7:0/763818336 shutdown_connections 2023-06-09T11:59:43.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.294+0000 7fa9169ff700 1 -- 172.21.15.7:0/763818336 wait complete. 2023-06-09T11:59:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.296+0000 7fa9169ff700 1 Processor -- start 2023-06-09T11:59:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.297+0000 7fa9169ff700 1 -- start start 2023-06-09T11:59:43.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9100c0140 0x7fa9101eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c1bc0 0x7fa910154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa910156040 con 0x7fa91015a0c0 2023-06-09T11:59:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101565c0 con 0x7fa9100c1bc0 2023-06-09T11:59:43.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa9169ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101543d0 con 0x7fa9100c0140 2023-06-09T11:59:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa914f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa914f9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52532/0 (socket says 172.21.15.7:52532) 2023-06-09T11:59:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa914f9c700 1 -- 172.21.15.7:0/4210302555 learned_addr learned my addr 172.21.15.7:0/4210302555 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa90ffff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9100c0140 0x7fa9101eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.304 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.298+0000 7fa90f7fe700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c1bc0 0x7fa910154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 -- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9100c0140 msgr2=0x7fa9101eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9100c0140 0x7fa9101eb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 -- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c1bc0 msgr2=0x7fa910154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c1bc0 0x7fa910154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.305 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 -- 172.21.15.7:0/4210302555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa908009a70 con 0x7fa91015a0c0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa914f9c700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa908014040 tx=0x7fa908004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9080129f0 con 0x7fa91015a0c0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908020070 con 0x7fa91015a0c0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100c1170 con 0x7fa91015a0c0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.299+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908012dd0 con 0x7fa91015a0c0 2023-06-09T11:59:43.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.300+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9100c0e90 con 0x7fa91015a0c0 2023-06-09T11:59:43.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.300+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9100c1450 con 0x7fa91015a0c0 2023-06-09T11:59:43.307 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.301+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa908004040 con 0x7fa91015a0c0 2023-06-09T11:59:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.302+0000 7fa90cff9700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 0x7fa8fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.302+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fa908022070 con 0x7fa91015a0c0 2023-06-09T11:59:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.303+0000 7fa90ffff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 0x7fa8fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.303+0000 7fa90ffff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 0x7fa8fc06d190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa900007fd0 tx=0x7fa900007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.309 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.307+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa908056540 con 0x7fa91015a0c0 2023-06-09T11:59:43.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.515+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa910074cf0 con 0x7fa8fc06ace0 2023-06-09T11:59:43.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.516+0000 7fa90cff9700 1 -- 172.21.15.7:0/4210302555 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa910074cf0 con 0x7fa8fc06ace0 2023-06-09T11:59:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 msgr2=0x7fa8fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 0x7fa8fc06d190 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa900007fd0 tx=0x7fa900007dc0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 msgr2=0x7fa910156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fa908014040 tx=0x7fa908004ad0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 shutdown_connections 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa91015a0c0 0x7fa910156a50 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa8fc06ace0 0x7fa8fc06d190 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa9100c0140 0x7fa9101eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 --2- 172.21.15.7:0/4210302555 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9100c1bc0 0x7fa910154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.518+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 >> 172.21.15.7:0/4210302555 conn(0x7fa91006a4d0 msgr2=0x7fa91014f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.519+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 shutdown_connections 2023-06-09T11:59:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.519+0000 7fa9169ff700 1 -- 172.21.15.7:0/4210302555 wait complete. 2023-06-09T11:59:43.523 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2023-06-09T11:59:43.539 INFO:tasks.ceph:Scrubbing osd.4 2023-06-09T11:59:43.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:43.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.741+0000 7fc10b7ff700 1 Processor -- start 2023-06-09T11:59:43.744 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.741+0000 7fc10b7ff700 1 -- start start 2023-06-09T11:59:43.745 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc10b7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040cc100 0x7fc10415d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc10b7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1040cc4d0 0x7fc1040cac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc10b7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10415dd00 con 0x7fc1040b5270 2023-06-09T11:59:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc10b7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc104071790 con 0x7fc1040cc4d0 2023-06-09T11:59:43.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc10b7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1040ca3f0 con 0x7fc1040cc100 2023-06-09T11:59:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc108d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1040cc4d0 0x7fc1040cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc108d9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc1040cc4d0 0x7fc1040cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52546/0 (socket says 172.21.15.7:52546) 2023-06-09T11:59:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.742+0000 7fc108d9a700 1 -- 172.21.15.7:0/1196991617 learned_addr learned my addr 172.21.15.7:0/1196991617 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc10959b700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040cc100 0x7fc10415d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc108d9a700 1 -- 172.21.15.7:0/1196991617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040cc100 msgr2=0x7fc10415d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc108d9a700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040cc100 0x7fc10415d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc108d9a700 1 -- 172.21.15.7:0/1196991617 >> v1:172.21.15.117:6789/0 conn(0x7fc1040b5270 legacy=0x7fc1040c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc108d9a700 1 -- 172.21.15.7:0/1196991617 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104159840 con 0x7fc1040cc4d0 2023-06-09T11:59:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc108d9a700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1040cac90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40098f0 tx=0x7fc0f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e20ec7c2ea09d7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc0fbfff700 1 -- 172.21.15.7:0/1196991617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400b8a0 con 0x7fc1040cc4d0 2023-06-09T11:59:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc0fbfff700 1 -- 172.21.15.7:0/1196991617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f400ba00 con 0x7fc1040cc4d0 2023-06-09T11:59:43.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.743+0000 7fc0fbfff700 1 -- 172.21.15.7:0/1196991617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f400bd90 con 0x7fc1040cc4d0 2023-06-09T11:59:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1196991617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 msgr2=0x7fc1040cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1040cac90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40098f0 tx=0x7fc0f4009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:43.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1196991617 shutdown_connections 2023-06-09T11:59:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040cc100 0x7fc10415d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1196991617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1040cac90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1196991617 >> 172.21.15.7:0/1196991617 conn(0x7fc104070af0 msgr2=0x7fc104070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1196991617 shutdown_connections 2023-06-09T11:59:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.744+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1196991617 wait complete. 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.746+0000 7fc10b7ff700 1 Processor -- start 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.746+0000 7fc10b7ff700 1 -- start start 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040b5270 0x7fc10414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1040cc100 0x7fc104156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1041f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10414faa0 con 0x7fc1040cc4d0 2023-06-09T11:59:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10414fd20 con 0x7fc1040cc100 2023-06-09T11:59:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10b7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc104150000 con 0x7fc1040b5270 2023-06-09T11:59:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc109d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1041f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc109d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1041f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52548/0 (socket says 172.21.15.7:52548) 2023-06-09T11:59:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc109d9c700 1 -- 172.21.15.7:0/1432249095 learned_addr learned my addr 172.21.15.7:0/1432249095 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc10959b700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040b5270 0x7fc10414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.747+0000 7fc108d9a700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1040cc100 0x7fc104156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 -- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040b5270 msgr2=0x7fc10414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1040b5270 0x7fc10414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 -- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1040cc100 msgr2=0x7fc104156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1040cc100 0x7fc104156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009550 con 0x7fc1040cc4d0 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc109d9c700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1041f30d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00cb50 tx=0x7fc0fc00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc003c00 con 0x7fc1040cc4d0 2023-06-09T11:59:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc003d60 con 0x7fc1040cc4d0 2023-06-09T11:59:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1040ca6b0 con 0x7fc1040cc4d0 2023-06-09T11:59:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00ab70 con 0x7fc1040cc4d0 2023-06-09T11:59:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.748+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1040ca120 con 0x7fc1040cc4d0 2023-06-09T11:59:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.749+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc104159b00 con 0x7fc1040cc4d0 2023-06-09T11:59:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.750+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc0fc00acd0 con 0x7fc1040cc4d0 2023-06-09T11:59:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc0f9ffb700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc0e806ace0 0x7fc0e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fc0fc08a810 con 0x7fc1040cc4d0 2023-06-09T11:59:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc10959b700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc0e806ace0 0x7fc0e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc0f9ffb700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7fc0e8074970 0x7fc0e8076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.759 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0e8077470 con 0x7fc0e8074970 2023-06-09T11:59:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.751+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0fc08ab80 con 0x7fc1040cc4d0 2023-06-09T11:59:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.752+0000 7fc108d9a700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7fc0e8074970 0x7fc0e8076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.752+0000 7fc10959b700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc0e806ace0 0x7fc0e806d190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc100009930 tx=0x7fc100008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.752+0000 7fc108d9a700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7fc0e8074970 0x7fc0e8076da0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.753+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== osd.4 v2:172.21.15.117:6800/3721658998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc0e8077470 con 0x7fc0e8074970 2023-06-09T11:59:43.770 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.768+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 --> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc10415dd00 con 0x7fc0e8074970 2023-06-09T11:59:43.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.769+0000 7fc0f9ffb700 1 -- 172.21.15.7:0/1432249095 <== osd.4 v2:172.21.15.117:6800/3721658998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc10415dd00 con 0x7fc0e8074970 2023-06-09T11:59:43.772 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:43.772 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:43.772 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.769+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7fc0e8074970 msgr2=0x7fc0e8076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.769+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.117:6800/3721658998,v1:172.21.15.117:6801/3721658998] conn(0x7fc0e8074970 0x7fc0e8076da0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc0e806ace0 msgr2=0x7fc0e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc0e806ace0 0x7fc0e806d190 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc100009930 tx=0x7fc100008040 comp rx=0 tx=0).stop 2023-06-09T11:59:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 msgr2=0x7fc1041f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 --2- 172.21.15.7:0/1432249095 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1040cc4d0 0x7fc1041f30d0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00cb50 tx=0x7fc0fc00cf10 comp rx=0 tx=0).stop 2023-06-09T11:59:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc109d9c700 1 -- 172.21.15.7:0/1432249095 reap_dead start 2023-06-09T11:59:43.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 shutdown_connections 2023-06-09T11:59:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 >> 172.21.15.7:0/1432249095 conn(0x7fc104070af0 msgr2=0x7fc1040661b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 shutdown_connections 2023-06-09T11:59:43.775 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.770+0000 7fc10b7ff700 1 -- 172.21.15.7:0/1432249095 wait complete. 2023-06-09T11:59:43.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-06-09T11:59:43.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.987+0000 7f8aa33ff700 1 Processor -- start 2023-06-09T11:59:43.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.987+0000 7f8aa33ff700 1 -- start start 2023-06-09T11:59:43.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.988+0000 7f8aa33ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8a9c1535e0 0x7f8a9c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.988+0000 7f8aa33ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.988+0000 7f8aa33ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a9c15a420 con 0x7f8a9c1539b0 2023-06-09T11:59:43.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.988+0000 7f8aa33ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0ca670 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.988+0000 7f8aa33ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9c0c5a00 con 0x7f8a9c1535e0 2023-06-09T11:59:43.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52564/0 (socket says 172.21.15.7:52564) 2023-06-09T11:59:43.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 -- 172.21.15.7:0/3742380755 learned_addr learned my addr 172.21.15.7:0/3742380755 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:43.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa119b700 1 --2- 172.21.15.7:0/3742380755 >> v2:172.21.15.117:3300/0 conn(0x7f8a9c1535e0 0x7f8a9c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:43.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 -- 172.21.15.7:0/3742380755 >> v2:172.21.15.117:3300/0 conn(0x7f8a9c1535e0 msgr2=0x7f8a9c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 --2- 172.21.15.7:0/3742380755 >> v2:172.21.15.117:3300/0 conn(0x7f8a9c1535e0 0x7f8a9c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 -- 172.21.15.7:0/3742380755 >> v1:172.21.15.117:6789/0 conn(0x7f8a9c1539b0 legacy=0x7f8a9c0c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 -- 172.21.15.7:0/3742380755 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c06b460 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8aa199c700 1 --2- 172.21.15.7:0/3742380755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c073d90 tx=0x7f8a9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b08f39545429ff server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:43.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.989+0000 7f8a93fff700 1 -- 172.21.15.7:0/3742380755 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400e070 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8a93fff700 1 -- 172.21.15.7:0/3742380755 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9400b910 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8a93fff700 1 -- 172.21.15.7:0/3742380755 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9400bcc0 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 -- 172.21.15.7:0/3742380755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 msgr2=0x7f8a9c0c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/3742380755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c073d90 tx=0x7f8a9400b330 comp rx=0 tx=0).stop 2023-06-09T11:59:43.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 -- 172.21.15.7:0/3742380755 shutdown_connections 2023-06-09T11:59:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/3742380755 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0c5140 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/3742380755 >> v2:172.21.15.117:3300/0 conn(0x7f8a9c1535e0 0x7f8a9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 -- 172.21.15.7:0/3742380755 >> 172.21.15.7:0/3742380755 conn(0x7f8a9c0737f0 msgr2=0x7f8a9c072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 -- 172.21.15.7:0/3742380755 shutdown_connections 2023-06-09T11:59:43.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.990+0000 7f8aa33ff700 1 -- 172.21.15.7:0/3742380755 wait complete. 2023-06-09T11:59:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.992+0000 7f8aa33ff700 1 Processor -- start 2023-06-09T11:59:43.998 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.992+0000 7f8aa33ff700 1 -- start start 2023-06-09T11:59:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 0x7f8a9c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 0x7f8a9c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1eeca0 con 0x7f8a9c0c4ce0 2023-06-09T11:59:43.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1efc70 con 0x7f8a9c1539b0 2023-06-09T11:59:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c1ef7e0 con 0x7f8a9c1535e0 2023-06-09T11:59:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52566/0 (socket says 172.21.15.7:52566) 2023-06-09T11:59:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa119b700 1 -- 172.21.15.7:0/195427115 learned_addr learned my addr 172.21.15.7:0/195427115 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:44.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa099a700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 0x7f8a9c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.993+0000 7f8aa199c700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 0x7f8a9c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 -- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 msgr2=0x7f8a9c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 0x7f8a9c0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 -- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 msgr2=0x7f8a9c14b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 0x7f8a9c14b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 -- 172.21.15.7:0/195427115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a94009b30 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa199c700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 0x7f8a9c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:44.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa099a700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 0x7f8a9c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:44.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa119b700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800e9e0 tx=0x7f8a9800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98009710 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9800c850 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800cbc0 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c06a850 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.994+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c06a570 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.995+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9c06ab30 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.996+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a9800cd20 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.997+0000 7f8a91ffb700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 0x7f8a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.997+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8a98089fb0 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.997+0000 7f8aa099a700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 0x7f8a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:43.998+0000 7f8aa099a700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 0x7f8a8006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c15c4f0 tx=0x7f8a8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.002+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8a980580e0 con 0x7f8a9c0c4ce0 2023-06-09T11:59:44.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.206+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a9c074cf0 con 0x7f8a8006ace0 2023-06-09T11:59:44.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.207+0000 7f8a91ffb700 1 -- 172.21.15.7:0/195427115 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a9c074cf0 con 0x7f8a8006ace0 2023-06-09T11:59:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 msgr2=0x7f8a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 0x7f8a8006d190 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c15c4f0 tx=0x7f8a8c009040 comp rx=0 tx=0).stop 2023-06-09T11:59:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 msgr2=0x7f8a9c0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800e9e0 tx=0x7f8a9800eda0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 shutdown_connections 2023-06-09T11:59:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a9c1539b0 0x7f8a9c14b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.212 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9c0c4ce0 0x7f8a9c0b4d00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8a8006ace0 0x7f8a8006d190 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 --2- 172.21.15.7:0/195427115 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8a9c1535e0 0x7f8a9c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 >> 172.21.15.7:0/195427115 conn(0x7f8a9c0737f0 msgr2=0x7f8a9c1519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:44.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 shutdown_connections 2023-06-09T11:59:44.213 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.209+0000 7f8aa33ff700 1 -- 172.21.15.7:0/195427115 wait complete. 2023-06-09T11:59:44.214 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2023-06-09T11:59:44.229 INFO:tasks.ceph:Scrubbing osd.5 2023-06-09T11:59:44.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:44.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.431+0000 7f0b687ff700 1 Processor -- start 2023-06-09T11:59:44.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.432+0000 7f0b687ff700 1 -- start start 2023-06-09T11:59:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b601513d0 0x7f0b600c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0b600c1aa0 0x7f0b600c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600b0e70 0x7f0b600bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b60072520 con 0x7f0b601513d0 2023-06-09T11:59:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b60157580 con 0x7f0b600c1aa0 2023-06-09T11:59:44.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b687ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60157860 con 0x7f0b600b0e70 2023-06-09T11:59:44.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b6659b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b601513d0 0x7f0b600c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b6659b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0b601513d0 0x7f0b600c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52580/0 (socket says 172.21.15.7:52580) 2023-06-09T11:59:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b6659b700 1 -- 172.21.15.7:0/1342213656 learned_addr learned my addr 172.21.15.7:0/1342213656 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b65d9a700 1 --2- 172.21.15.7:0/1342213656 >> v2:172.21.15.117:3300/0 conn(0x7f0b600c1aa0 0x7f0b600c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.433+0000 7f0b66d9c700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600b0e70 0x7f0b600bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 -- 172.21.15.7:0/1342213656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600b0e70 msgr2=0x7f0b600bffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600b0e70 0x7f0b600bffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 -- 172.21.15.7:0/1342213656 >> v2:172.21.15.117:3300/0 conn(0x7f0b600c1aa0 msgr2=0x7f0b600c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1342213656 >> v2:172.21.15.117:3300/0 conn(0x7f0b600c1aa0 0x7f0b600c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 -- 172.21.15.7:0/1342213656 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60157b40 con 0x7f0b601513d0 2023-06-09T11:59:44.439 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b601513d0 0x7f0b600c68f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0b60068d10 tx=0x7f0b5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198e45d530dccc15 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b65599700 1 -- 172.21.15.7:0/1342213656 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c011830 con 0x7f0b601513d0 2023-06-09T11:59:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b65599700 1 -- 172.21.15.7:0/1342213656 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c011990 con 0x7f0b601513d0 2023-06-09T11:59:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.434+0000 7f0b65599700 1 -- 172.21.15.7:0/1342213656 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c011d20 con 0x7f0b601513d0 2023-06-09T11:59:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 -- 172.21.15.7:0/1342213656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b601513d0 msgr2=0x7f0b600c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.440 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b601513d0 0x7f0b600c68f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f0b60068d10 tx=0x7f0b5c009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 -- 172.21.15.7:0/1342213656 shutdown_connections 2023-06-09T11:59:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b600b0e70 0x7f0b600bffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1342213656 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b601513d0 0x7f0b600c68f0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1342213656 >> v2:172.21.15.117:3300/0 conn(0x7f0b600c1aa0 0x7f0b600c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.441 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 -- 172.21.15.7:0/1342213656 >> 172.21.15.7:0/1342213656 conn(0x7f0b600cbd60 msgr2=0x7f0b600c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 -- 172.21.15.7:0/1342213656 shutdown_connections 2023-06-09T11:59:44.442 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.435+0000 7f0b687ff700 1 -- 172.21.15.7:0/1342213656 wait complete. 2023-06-09T11:59:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.437+0000 7f0b687ff700 1 Processor -- start 2023-06-09T11:59:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.437+0000 7f0b687ff700 1 -- start start 2023-06-09T11:59:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 0x7f0b600ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b600c1aa0 0x7f0b6014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b601513d0 0x7f0b6014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60156ae0 con 0x7f0b600b0e70 2023-06-09T11:59:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6014f040 con 0x7f0b600c1aa0 2023-06-09T11:59:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b687ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b60156e60 con 0x7f0b601513d0 2023-06-09T11:59:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 0x7f0b600ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 0x7f0b600ca1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52594/0 (socket says 172.21.15.7:52594) 2023-06-09T11:59:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 -- 172.21.15.7:0/1042051089 learned_addr learned my addr 172.21.15.7:0/1042051089 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b65d9a700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b600c1aa0 0x7f0b6014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b66d9c700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b601513d0 0x7f0b6014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 -- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b601513d0 msgr2=0x7f0b6014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.446 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0b601513d0 0x7f0b6014fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 -- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b600c1aa0 msgr2=0x7f0b6014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0b600c1aa0 0x7f0b6014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.438+0000 7f0b6659b700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c009550 con 0x7f0b600b0e70 2023-06-09T11:59:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b6659b700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 0x7f0b600ca1b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c014040 tx=0x7f0b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.447 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c004450 con 0x7f0b600b0e70 2023-06-09T11:59:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5c01e070 con 0x7f0b600b0e70 2023-06-09T11:59:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5c0038e0 con 0x7f0b600b0e70 2023-06-09T11:59:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b60158100 con 0x7f0b600b0e70 2023-06-09T11:59:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.439+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b60157e20 con 0x7f0b600b0e70 2023-06-09T11:59:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.440+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b60072230 con 0x7f0b600b0e70 2023-06-09T11:59:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.441+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0b5c003a40 con 0x7f0b600b0e70 2023-06-09T11:59:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b56ffd700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4406ad10 0x7f0b4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f0b5c08b160 con 0x7f0b600b0e70 2023-06-09T11:59:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b65d9a700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4406ad10 0x7f0b4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b56ffd700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7f0b440749a0 0x7f0b44076dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0b440774a0 con 0x7f0b440749a0 2023-06-09T11:59:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.442+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b5c08b4d0 con 0x7f0b600b0e70 2023-06-09T11:59:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.443+0000 7f0b66d9c700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7f0b440749a0 0x7f0b44076dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.443+0000 7f0b65d9a700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4406ad10 0x7f0b4406d1c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0b60072200 tx=0x7f0b50009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.443+0000 7f0b66d9c700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7f0b440749a0 0x7f0b44076dd0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.444+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== osd.5 v2:172.21.15.117:6824/644345713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0b440774a0 con 0x7f0b440749a0 2023-06-09T11:59:44.461 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.459+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 --> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b60157310 con 0x7f0b440749a0 2023-06-09T11:59:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.460+0000 7f0b56ffd700 1 -- 172.21.15.7:0/1042051089 <== osd.5 v2:172.21.15.117:6824/644345713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0b60157310 con 0x7f0b440749a0 2023-06-09T11:59:44.463 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:44.464 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7f0b440749a0 msgr2=0x7f0b44076dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.117:6824/644345713,v1:172.21.15.117:6825/644345713] conn(0x7f0b440749a0 0x7f0b44076dd0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4406ad10 msgr2=0x7f0b4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0b4406ad10 0x7f0b4406d1c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0b60072200 tx=0x7f0b50009210 comp rx=0 tx=0).stop 2023-06-09T11:59:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 msgr2=0x7f0b600ca1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 --2- 172.21.15.7:0/1042051089 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b600b0e70 0x7f0b600ca1b0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c014040 tx=0x7f0b5c004750 comp rx=0 tx=0).stop 2023-06-09T11:59:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b66d9c700 1 -- 172.21.15.7:0/1042051089 reap_dead start 2023-06-09T11:59:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 shutdown_connections 2023-06-09T11:59:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 >> 172.21.15.7:0/1042051089 conn(0x7f0b600cbd60 msgr2=0x7f0b600c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 shutdown_connections 2023-06-09T11:59:44.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.461+0000 7f0b687ff700 1 -- 172.21.15.7:0/1042051089 wait complete. 2023-06-09T11:59:44.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-06-09T11:59:44.681 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.679+0000 7f6729bff700 1 Processor -- start 2023-06-09T11:59:44.682 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.679+0000 7f6729bff700 1 -- start start 2023-06-09T11:59:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 0x7f67240b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67240cb9a0 0x7f6724159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67240cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6724159b20 con 0x7f67240cb9a0 2023-06-09T11:59:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240cac80 con 0x7f67240cb5d0 2023-06-09T11:59:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.680+0000 7f6729bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67241597d0 con 0x7f672415cd40 2023-06-09T11:59:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67240cb9a0 0x7f6724159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f67240cb9a0 0x7f6724159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52606/0 (socket says 172.21.15.7:52606) 2023-06-09T11:59:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 -- 172.21.15.7:0/4238034139 learned_addr learned my addr 172.21.15.7:0/4238034139 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6723fff700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67240cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f67237fe700 1 --2- 172.21.15.7:0/4238034139 >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 0x7f67240b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 -- 172.21.15.7:0/4238034139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 msgr2=0x7f67240cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67240cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 -- 172.21.15.7:0/4238034139 >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 msgr2=0x7f67240b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 --2- 172.21.15.7:0/4238034139 >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 0x7f67240b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 -- 172.21.15.7:0/4238034139 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724071ab0 con 0x7f67240cb9a0 2023-06-09T11:59:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f67237fe700 1 --2- 172.21.15.7:0/4238034139 >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 0x7f67240b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T11:59:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.681+0000 7f6722ffd700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f6724159fb0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f672408d420 tx=0x7f6718009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13221fb3fe169b4f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f67227fc700 1 -- 172.21.15.7:0/4238034139 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718011830 con 0x7f67240cb9a0 2023-06-09T11:59:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f67227fc700 1 -- 172.21.15.7:0/4238034139 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6718011990 con 0x7f67240cb9a0 2023-06-09T11:59:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f67227fc700 1 -- 172.21.15.7:0/4238034139 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718011d20 con 0x7f67240cb9a0 2023-06-09T11:59:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 -- 172.21.15.7:0/4238034139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 msgr2=0x7f6724159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f6724159fb0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f672408d420 tx=0x7f6718009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 -- 172.21.15.7:0/4238034139 shutdown_connections 2023-06-09T11:59:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 --2- 172.21.15.7:0/4238034139 >> v2:172.21.15.117:3300/0 conn(0x7f67240cb5d0 0x7f67240b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67240cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 --2- 172.21.15.7:0/4238034139 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f6724159fb0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 -- 172.21.15.7:0/4238034139 >> 172.21.15.7:0/4238034139 conn(0x7f6724070af0 msgr2=0x7f6724070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 -- 172.21.15.7:0/4238034139 shutdown_connections 2023-06-09T11:59:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.682+0000 7f6729bff700 1 -- 172.21.15.7:0/4238034139 wait complete. 2023-06-09T11:59:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.684+0000 7f6729bff700 1 Processor -- start 2023-06-09T11:59:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.684+0000 7f6729bff700 1 -- start start 2023-06-09T11:59:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67240cb5d0 0x7f67241f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67241ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67241efe70 con 0x7f67240cb9a0 2023-06-09T11:59:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67241542f0 con 0x7f67240cb5d0 2023-06-09T11:59:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6729bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6724152170 con 0x7f672415cd40 2023-06-09T11:59:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6722ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6722ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52618/0 (socket says 172.21.15.7:52618) 2023-06-09T11:59:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6722ffd700 1 -- 172.21.15.7:0/2110890699 learned_addr learned my addr 172.21.15.7:0/2110890699 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f6723fff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67241ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.685+0000 7f67237fe700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67240cb5d0 0x7f67241f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 -- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 msgr2=0x7f67241ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67241ef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 -- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67240cb5d0 msgr2=0x7f67241f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67240cb5d0 0x7f67241f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 -- 172.21.15.7:0/2110890699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718009550 con 0x7f67240cb9a0 2023-06-09T11:59:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6722ffd700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6718000c00 tx=0x7f6718004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718004240 con 0x7f67240cb9a0 2023-06-09T11:59:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671801f070 con 0x7f67240cb9a0 2023-06-09T11:59:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180036a0 con 0x7f67240cb9a0 2023-06-09T11:59:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.686+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672415d4f0 con 0x7f67240cb9a0 2023-06-09T11:59:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.687+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672415d210 con 0x7f67240cb9a0 2023-06-09T11:59:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.688+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672415d7d0 con 0x7f67240cb9a0 2023-06-09T11:59:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.689+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6718021030 con 0x7f67240cb9a0 2023-06-09T11:59:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.690+0000 7f670bfff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 0x7f671006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.690+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f671808aae0 con 0x7f67240cb9a0 2023-06-09T11:59:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.690+0000 7f67237fe700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 0x7f671006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.690+0000 7f67237fe700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 0x7f671006d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f67140045f0 tx=0x7f6714008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.694+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f67180551e0 con 0x7f67240cb9a0 2023-06-09T11:59:44.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.904+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6724074cf0 con 0x7f671006ace0 2023-06-09T11:59:44.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.904+0000 7f670bfff700 1 -- 172.21.15.7:0/2110890699 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6724074cf0 con 0x7f671006ace0 2023-06-09T11:59:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 msgr2=0x7f671006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 0x7f671006d190 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f67140045f0 tx=0x7f6714008040 comp rx=0 tx=0).stop 2023-06-09T11:59:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 msgr2=0x7f67241ef310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:44.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f6718000c00 tx=0x7f6718004750 comp rx=0 tx=0).stop 2023-06-09T11:59:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 shutdown_connections 2023-06-09T11:59:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f671006ace0 0x7f671006d190 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f672415cd40 0x7f67241ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67240cb5d0 0x7f67241f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 --2- 172.21.15.7:0/2110890699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67240cb9a0 0x7f67241ef310 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.906+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 >> 172.21.15.7:0/2110890699 conn(0x7f6724070af0 msgr2=0x7f672415b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.907+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 shutdown_connections 2023-06-09T11:59:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:44.907+0000 7f6729bff700 1 -- 172.21.15.7:0/2110890699 wait complete. 2023-06-09T11:59:44.911 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2023-06-09T11:59:44.927 INFO:tasks.ceph:Scrubbing osd.6 2023-06-09T11:59:44.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.129+0000 7ff36cdff700 1 Processor -- start 2023-06-09T11:59:45.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.129+0000 7ff36cdff700 1 -- start start 2023-06-09T11:59:45.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.130+0000 7ff36cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 0x7ff368156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.130+0000 7ff36cdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3680b2360 con 0x7ff3680cf850 2023-06-09T11:59:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.130+0000 7ff36cdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3680b2c00 con 0x7ff3680c8e50 2023-06-09T11:59:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.130+0000 7ff36cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36806b4b0 con 0x7ff3680cfc20 2023-06-09T11:59:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36659c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3680cf850 0x7ff3680b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34636/0 (socket says 172.21.15.7:34636) 2023-06-09T11:59:45.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36659c700 1 -- 172.21.15.7:0/3448572739 learned_addr learned my addr 172.21.15.7:0/3448572739 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff366d9d700 1 --2- 172.21.15.7:0/3448572739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 0x7ff368156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772318474 0 0) 0x7ff3680b2360 con 0x7ff3680cf850 2023-06-09T11:59:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3500035b0 con 0x7ff3680cf850 2023-06-09T11:59:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317524930 0 0) 0x7ff3680b2c00 con 0x7ff3680c8e50 2023-06-09T11:59:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3680b2360 con 0x7ff3680c8e50 2023-06-09T11:59:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff366d9d700 1 -- 172.21.15.7:0/3448572739 >> v1:172.21.15.117:6789/0 conn(0x7ff3680c8e50 legacy=0x7ff368156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850262840 0 0) 0x7ff3500035b0 con 0x7ff3680cf850 2023-06-09T11:59:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff366d9d700 1 -- 172.21.15.7:0/3448572739 >> v1:172.21.15.7:6789/0 conn(0x7ff3680cf850 legacy=0x7ff3680b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.131+0000 7ff366d9d700 1 -- 172.21.15.7:0/3448572739 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680b5150 con 0x7ff3680cfc20 2023-06-09T11:59:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff366d9d700 1 --2- 172.21.15.7:0/3448572739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 0x7ff368156e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff360009dc0 tx=0x7ff36000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fdb9bc33632910 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000e070 con 0x7ff3680cfc20 2023-06-09T11:59:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff360003a20 con 0x7ff3680cfc20 2023-06-09T11:59:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff36559a700 1 -- 172.21.15.7:0/3448572739 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff358001670 con 0x7ff3680cfc20 2023-06-09T11:59:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff36cdff700 1 -- 172.21.15.7:0/3448572739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 msgr2=0x7ff368156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.132+0000 7ff36cdff700 1 --2- 172.21.15.7:0/3448572739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 0x7ff368156e50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff360009dc0 tx=0x7ff36000b330 comp rx=0 tx=0).stop 2023-06-09T11:59:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.133+0000 7ff36cdff700 1 -- 172.21.15.7:0/3448572739 shutdown_connections 2023-06-09T11:59:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.133+0000 7ff36cdff700 1 --2- 172.21.15.7:0/3448572739 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680cfc20 0x7ff368156e50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.133+0000 7ff36cdff700 1 -- 172.21.15.7:0/3448572739 >> 172.21.15.7:0/3448572739 conn(0x7ff368072060 msgr2=0x7ff368072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.133+0000 7ff36cdff700 1 -- 172.21.15.7:0/3448572739 shutdown_connections 2023-06-09T11:59:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.133+0000 7ff36cdff700 1 -- 172.21.15.7:0/3448572739 wait complete. 2023-06-09T11:59:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.134+0000 7ff36cdff700 1 Processor -- start 2023-06-09T11:59:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.134+0000 7ff36cdff700 1 -- start start 2023-06-09T11:59:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680c8e50 0x7ff3681e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 0x7ff3681e2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3680cfc20 0x7ff368151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680ca2a0 con 0x7ff3680cf850 2023-06-09T11:59:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681ee330 con 0x7ff3680cfc20 2023-06-09T11:59:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.135+0000 7ff36cdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3681f2830 con 0x7ff3680c8e50 2023-06-09T11:59:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 0x7ff3681e2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 0x7ff3681e2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52628/0 (socket says 172.21.15.7:52628) 2023-06-09T11:59:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 -- 172.21.15.7:0/544268329 learned_addr learned my addr 172.21.15.7:0/544268329 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff366d9d700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3680cfc20 0x7ff368151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff36659c700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680c8e50 0x7ff3681e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 -- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680c8e50 msgr2=0x7ff3681e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff3680c8e50 0x7ff3681e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 -- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3680cfc20 msgr2=0x7ff368151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3680cfc20 0x7ff368151df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360009a70 con 0x7ff3680cf850 2023-06-09T11:59:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff365d9b700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 0x7ff3681e2f50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c520 tx=0x7ff35c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.136+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0052d0 con 0x7ff3680cf850 2023-06-09T11:59:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.137+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c005430 con 0x7ff3680cf850 2023-06-09T11:59:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.137+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c0038c0 con 0x7ff3680cf850 2023-06-09T11:59:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.137+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3680b53f0 con 0x7ff3680cf850 2023-06-09T11:59:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.137+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3680b4b70 con 0x7ff3680cf850 2023-06-09T11:59:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.137+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff3680b25b0 con 0x7ff3680cf850 2023-06-09T11:59:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.139+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff35c003a20 con 0x7ff3680cf850 2023-06-09T11:59:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff356ffd700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff34806ace0 0x7ff34806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7ff35c089450 con 0x7ff3680cf850 2023-06-09T11:59:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff356ffd700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7ff348074970 0x7ff348076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff348077470 con 0x7ff348074970 2023-06-09T11:59:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff35c08ba30 con 0x7ff3680cf850 2023-06-09T11:59:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff36659c700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff34806ace0 0x7ff34806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.140+0000 7ff366d9d700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7ff348074970 0x7ff348076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.141+0000 7ff36659c700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff34806ace0 0x7ff34806d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff358001380 tx=0x7ff3580015f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.141+0000 7ff366d9d700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7ff348074970 0x7ff348076da0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.142+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== osd.6 v2:172.21.15.117:6808/2098282767 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff348077470 con 0x7ff348074970 2023-06-09T11:59:45.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.157+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 --> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff3681ee820 con 0x7ff348074970 2023-06-09T11:59:45.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff356ffd700 1 -- 172.21.15.7:0/544268329 <== osd.6 v2:172.21.15.117:6808/2098282767 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff3681ee820 con 0x7ff348074970 2023-06-09T11:59:45.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:45.161 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:45.161 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7ff348074970 msgr2=0x7ff348076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.117:6808/2098282767,v1:172.21.15.117:6809/2098282767] conn(0x7ff348074970 0x7ff348076da0 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff34806ace0 msgr2=0x7ff34806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff34806ace0 0x7ff34806d190 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff358001380 tx=0x7ff3580015f0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 msgr2=0x7ff3681e2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.158+0000 7ff36cdff700 1 --2- 172.21.15.7:0/544268329 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff3680cf850 0x7ff3681e2f50 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00c520 tx=0x7ff35c00c8e0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.159+0000 7ff366d9d700 1 -- 172.21.15.7:0/544268329 reap_dead start 2023-06-09T11:59:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.159+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 shutdown_connections 2023-06-09T11:59:45.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.159+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 >> 172.21.15.7:0/544268329 conn(0x7ff368072060 msgr2=0x7ff368157590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.159+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 shutdown_connections 2023-06-09T11:59:45.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.159+0000 7ff36cdff700 1 -- 172.21.15.7:0/544268329 wait complete. 2023-06-09T11:59:45.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-06-09T11:59:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.376+0000 7f3baa7ff700 1 Processor -- start 2023-06-09T11:59:45.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.376+0000 7f3baa7ff700 1 -- start start 2023-06-09T11:59:45.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3baa7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba4153880 0x7f3ba415d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3baa7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ba40c9f40 0x7f3ba415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3baa7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba415a500 con 0x7f3ba415b2d0 2023-06-09T11:59:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3baa7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40c5f10 con 0x7f3ba40c9f40 2023-06-09T11:59:45.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3baa7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba415da00 con 0x7f3ba4153880 2023-06-09T11:59:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.377+0000 7f3ba37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ba40c9f40 0x7f3ba415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3ba40c9f40 0x7f3ba415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52636/0 (socket says 172.21.15.7:52636) 2023-06-09T11:59:45.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 -- 172.21.15.7:0/3253531248 learned_addr learned my addr 172.21.15.7:0/3253531248 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba3fff700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba4153880 0x7f3ba415d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 -- 172.21.15.7:0/3253531248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba4153880 msgr2=0x7f3ba415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba4153880 0x7f3ba415d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 -- 172.21.15.7:0/3253531248 >> v1:172.21.15.7:6789/0 conn(0x7f3ba415b2d0 legacy=0x7f3ba40c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 -- 172.21.15.7:0/3253531248 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba415b750 con 0x7f3ba40c9f40 2023-06-09T11:59:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.378+0000 7f3ba37fe700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba40c9f40 0x7f3ba415ad90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3b980098f0 tx=0x7f3b98009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbd9489a7cffc69 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3ba2ffd700 1 -- 172.21.15.7:0/3253531248 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800b8a0 con 0x7f3ba40c9f40 2023-06-09T11:59:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3ba2ffd700 1 -- 172.21.15.7:0/3253531248 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9800ba00 con 0x7f3ba40c9f40 2023-06-09T11:59:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3ba2ffd700 1 -- 172.21.15.7:0/3253531248 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800bd90 con 0x7f3ba40c9f40 2023-06-09T11:59:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 -- 172.21.15.7:0/3253531248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba40c9f40 msgr2=0x7f3ba415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba40c9f40 0x7f3ba415ad90 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f3b980098f0 tx=0x7f3b98009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 -- 172.21.15.7:0/3253531248 shutdown_connections 2023-06-09T11:59:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba4153880 0x7f3ba415d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/3253531248 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba40c9f40 0x7f3ba415ad90 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 -- 172.21.15.7:0/3253531248 >> 172.21.15.7:0/3253531248 conn(0x7f3ba40737f0 msgr2=0x7f3ba40720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 -- 172.21.15.7:0/3253531248 shutdown_connections 2023-06-09T11:59:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.379+0000 7f3baa7ff700 1 -- 172.21.15.7:0/3253531248 wait complete. 2023-06-09T11:59:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.381+0000 7f3baa7ff700 1 Processor -- start 2023-06-09T11:59:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.381+0000 7f3baa7ff700 1 -- start start 2023-06-09T11:59:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba40c9f40 0x7f3ba40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba4153880 0x7f3ba40cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba4154540 con 0x7f3ba415b2d0 2023-06-09T11:59:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41f28b0 con 0x7f3ba4153880 2023-06-09T11:59:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3baa7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40b53f0 con 0x7f3ba40c9f40 2023-06-09T11:59:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3ba8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3ba8d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52638/0 (socket says 172.21.15.7:52638) 2023-06-09T11:59:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3ba8d9c700 1 -- 172.21.15.7:0/870671293 learned_addr learned my addr 172.21.15.7:0/870671293 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3ba37fe700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba4153880 0x7f3ba40cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.382+0000 7f3ba3fff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba40c9f40 0x7f3ba40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 -- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba40c9f40 msgr2=0x7f3ba40b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba40c9f40 0x7f3ba40b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 -- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba4153880 msgr2=0x7f3ba40cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba4153880 0x7f3ba40cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 -- 172.21.15.7:0/870671293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009550 con 0x7f3ba415b2d0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba8d9c700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00cb50 tx=0x7f3b9c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c003c00 con 0x7f3ba415b2d0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c003d60 con 0x7f3ba415b2d0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00aa20 con 0x7f3ba415b2d0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba4159980 con 0x7f3ba415b2d0 2023-06-09T11:59:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.383+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba41596e0 con 0x7f3ba415b2d0 2023-06-09T11:59:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.384+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba4159c60 con 0x7f3ba415b2d0 2023-06-09T11:59:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.385+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b9c00ab80 con 0x7f3ba415b2d0 2023-06-09T11:59:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.386+0000 7f3ba0ff9700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 0x7f3b9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.386+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f3b9c089c00 con 0x7f3ba415b2d0 2023-06-09T11:59:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.387+0000 7f3ba3fff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 0x7f3b9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.387+0000 7f3ba3fff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 0x7f3b9006d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006b20 tx=0x7f3b94008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.391+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3b9c054330 con 0x7f3ba415b2d0 2023-06-09T11:59:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.600+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ba415a500 con 0x7f3b9006ace0 2023-06-09T11:59:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.600+0000 7f3ba0ff9700 1 -- 172.21.15.7:0/870671293 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ba415a500 con 0x7f3b9006ace0 2023-06-09T11:59:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 msgr2=0x7f3b9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 0x7f3b9006d190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006b20 tx=0x7f3b94008040 comp rx=0 tx=0).stop 2023-06-09T11:59:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 msgr2=0x7f3ba41ef340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00cb50 tx=0x7f3b9c00ce60 comp rx=0 tx=0).stop 2023-06-09T11:59:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 shutdown_connections 2023-06-09T11:59:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ba415b2d0 0x7f3ba41ef340 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3b9006ace0 0x7f3b9006d190 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3ba40c9f40 0x7f3ba40b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 --2- 172.21.15.7:0/870671293 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ba4153880 0x7f3ba40cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 >> 172.21.15.7:0/870671293 conn(0x7f3ba40737f0 msgr2=0x7f3ba40720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 shutdown_connections 2023-06-09T11:59:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.603+0000 7f3baa7ff700 1 -- 172.21.15.7:0/870671293 wait complete. 2023-06-09T11:59:45.608 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2023-06-09T11:59:45.623 INFO:tasks.ceph:Scrubbing osd.7 2023-06-09T11:59:45.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-06-09T11:59:45.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.827+0000 7f29d7fff700 1 Processor -- start 2023-06-09T11:59:45.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.827+0000 7f29d7fff700 1 -- start start 2023-06-09T11:59:45.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.828+0000 7f29d7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29d00cf850 0x7f29d0159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.828+0000 7f29d7fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f29d00cfc20 0x7f29d0156740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.828+0000 7f29d7fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d015a4e0 con 0x7f29d00c8e50 2023-06-09T11:59:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.828+0000 7f29d7fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00b2360 con 0x7f29d00cf850 2023-06-09T11:59:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.828+0000 7f29d7fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d00b2c00 con 0x7f29d00cfc20 2023-06-09T11:59:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29d00cf850 0x7f29d0159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f29d00cf850 0x7f29d0159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52640/0 (socket says 172.21.15.7:52640) 2023-06-09T11:59:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 -- 172.21.15.7:0/519383324 learned_addr learned my addr 172.21.15.7:0/519383324 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:45.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d659c700 1 --2- 172.21.15.7:0/519383324 >> v2:172.21.15.117:3300/0 conn(0x7f29d00cfc20 0x7f29d0156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 -- 172.21.15.7:0/519383324 >> v2:172.21.15.117:3300/0 conn(0x7f29d00cfc20 msgr2=0x7f29d0156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/519383324 >> v2:172.21.15.117:3300/0 conn(0x7f29d00cfc20 0x7f29d0156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 -- 172.21.15.7:0/519383324 >> v1:172.21.15.117:6789/0 conn(0x7f29d00c8e50 legacy=0x7f29d00b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 -- 172.21.15.7:0/519383324 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d006b4b0 con 0x7f29d00cf850 2023-06-09T11:59:45.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.829+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/519383324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0159fa0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f29d00c6620 tx=0x7f29cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d483c0eedb2704 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d4d99700 1 -- 172.21.15.7:0/519383324 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011830 con 0x7f29d00cf850 2023-06-09T11:59:45.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d4d99700 1 -- 172.21.15.7:0/519383324 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc011990 con 0x7f29d00cf850 2023-06-09T11:59:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d4d99700 1 -- 172.21.15.7:0/519383324 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011d20 con 0x7f29d00cf850 2023-06-09T11:59:45.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 -- 172.21.15.7:0/519383324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 msgr2=0x7f29d0159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 --2- 172.21.15.7:0/519383324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0159fa0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f29d00c6620 tx=0x7f29cc009bd0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 -- 172.21.15.7:0/519383324 shutdown_connections 2023-06-09T11:59:45.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 --2- 172.21.15.7:0/519383324 >> v2:172.21.15.117:3300/0 conn(0x7f29d00cfc20 0x7f29d0156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 --2- 172.21.15.7:0/519383324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0159fa0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 -- 172.21.15.7:0/519383324 >> 172.21.15.7:0/519383324 conn(0x7f29d0072060 msgr2=0x7f29d0072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 -- 172.21.15.7:0/519383324 shutdown_connections 2023-06-09T11:59:45.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.830+0000 7f29d7fff700 1 -- 172.21.15.7:0/519383324 wait complete. 2023-06-09T11:59:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.833+0000 7f29d7fff700 1 Processor -- start 2023-06-09T11:59:45.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.833+0000 7f29d7fff700 1 -- start start 2023-06-09T11:59:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29d00c8e50 0x7f29d01ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.850 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d00cfc20 0x7f29d00cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d00c8530 con 0x7f29d00cf850 2023-06-09T11:59:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01ef830 con 0x7f29d00cfc20 2023-06-09T11:59:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d7fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d01526d0 con 0x7f29d00c8e50 2023-06-09T11:59:45.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d559a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52654/0 (socket says 172.21.15.7:52654) 2023-06-09T11:59:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.834+0000 7f29d559a700 1 -- 172.21.15.7:0/2256743642 learned_addr learned my addr 172.21.15.7:0/2256743642 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 -- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29d00c8e50 msgr2=0x7f29d01ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d659c700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d00cfc20 0x7f29d00cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29d00c8e50 0x7f29d01ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29d00c8e50 0x7f29d01ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 -- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d00cfc20 msgr2=0x7f29d00cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d00cfc20 0x7f29d00cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc009550 con 0x7f29d00cf850 2023-06-09T11:59:45.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f29d00c8e50 0x7f29d01ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d659c700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f29d00cfc20 0x7f29d00cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T11:59:45.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d559a700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0151bb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c480 tx=0x7f29c000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00052d0 con 0x7f29d00cf850 2023-06-09T11:59:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c0005430 con 0x7f29d00cf850 2023-06-09T11:59:45.855 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d00b4b70 con 0x7f29d00cf850 2023-06-09T11:59:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.835+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00038c0 con 0x7f29d00cf850 2023-06-09T11:59:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.836+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d00b5150 con 0x7f29d00cf850 2023-06-09T11:59:45.856 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.836+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29d00b53f0 con 0x7f29d00cf850 2023-06-09T11:59:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.837+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f29c0003a20 con 0x7f29d00cf850 2023-06-09T11:59:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.838+0000 7f29c67fc700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29b406ace0 0x7f29b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.838+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f29c0088ff0 con 0x7f29d00cf850 2023-06-09T11:59:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29b406ace0 0x7f29b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29c67fc700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7f29b4074970 0x7f29b4076da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29b4077470 con 0x7f29b4074970 2023-06-09T11:59:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29c008b5f0 con 0x7f29d00cf850 2023-06-09T11:59:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29d5d9b700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29b406ace0 0x7f29b406d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009fd0 tx=0x7f29cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.839+0000 7f29d659c700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7f29b4074970 0x7f29b4076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:45.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.840+0000 7f29d659c700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7f29b4074970 0x7f29b4076da0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.841+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== osd.7 v2:172.21.15.117:6816/1383862628 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f29b4077470 con 0x7f29b4074970 2023-06-09T11:59:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.856+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 --> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f29d01535e0 con 0x7f29b4074970 2023-06-09T11:59:45.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.856+0000 7f29c67fc700 1 -- 172.21.15.7:0/2256743642 <== osd.7 v2:172.21.15.117:6816/1383862628 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f29d01535e0 con 0x7f29b4074970 2023-06-09T11:59:45.860 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2023-06-09T11:59:45.860 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-06-09T11:59:45.860 INFO:teuthology.orchestra.run.smithi007.stdout:} 2023-06-09T11:59:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7f29b4074970 msgr2=0x7f29b4076da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.117:6816/1383862628,v1:172.21.15.117:6817/1383862628] conn(0x7f29b4074970 0x7f29b4076da0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29b406ace0 msgr2=0x7f29b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f29b406ace0 0x7f29b406d190 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009fd0 tx=0x7f29cc008030 comp rx=0 tx=0).stop 2023-06-09T11:59:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 msgr2=0x7f29d0151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 --2- 172.21.15.7:0/2256743642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f29d00cf850 0x7f29d0151bb0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f29c000c480 tx=0x7f29c000c840 comp rx=0 tx=0).stop 2023-06-09T11:59:45.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d659c700 1 -- 172.21.15.7:0/2256743642 reap_dead start 2023-06-09T11:59:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 shutdown_connections 2023-06-09T11:59:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 >> 172.21.15.7:0/2256743642 conn(0x7f29d0072060 msgr2=0x7f29d00cea80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 shutdown_connections 2023-06-09T11:59:45.863 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:45.857+0000 7f29d7fff700 1 -- 172.21.15.7:0/2256743642 wait complete. 2023-06-09T11:59:45.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-06-09T11:59:46.076 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.074+0000 7f3f219ff700 1 Processor -- start 2023-06-09T11:59:46.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.074+0000 7f3f219ff700 1 -- start start 2023-06-09T11:59:46.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f219ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f219ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c1598b0 0x7f3f1c159d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f219ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c15cec0 con 0x7f3f1c0cad00 2023-06-09T11:59:46.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f219ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c15d160 con 0x7f3f1c0c9090 2023-06-09T11:59:46.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f219ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c15d440 con 0x7f3f1c1598b0 2023-06-09T11:59:46.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c1598b0 0x7f3f1c159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:54418/0 (socket says 172.21.15.7:54418) 2023-06-09T11:59:46.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 -- 172.21.15.7:0/1541271026 learned_addr learned my addr 172.21.15.7:0/1541271026 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:46.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 -- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c1598b0 msgr2=0x7f3f1c159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T11:59:46.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 --2- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c1598b0 0x7f3f1c159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 -- 172.21.15.7:0/1541271026 >> v1:172.21.15.117:6789/0 conn(0x7f3f1c0cad00 legacy=0x7f3f1c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.076+0000 7f3f1affd700 1 -- 172.21.15.7:0/1541271026 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c15d720 con 0x7f3f1c0c9090 2023-06-09T11:59:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.077+0000 7f3f1affd700 1 --2- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0098f0 tx=0x7f3f0c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b412bc61a3d31e95 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.077+0000 7f3f1a7fc700 1 -- 172.21.15.7:0/1541271026 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c011830 con 0x7f3f1c0c9090 2023-06-09T11:59:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.077+0000 7f3f1a7fc700 1 -- 172.21.15.7:0/1541271026 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c011990 con 0x7f3f1c0c9090 2023-06-09T11:59:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.077+0000 7f3f1a7fc700 1 -- 172.21.15.7:0/1541271026 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c011d20 con 0x7f3f1c0c9090 2023-06-09T11:59:46.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 -- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 msgr2=0x7f3f1c0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0098f0 tx=0x7f3f0c009c00 comp rx=0 tx=0).stop 2023-06-09T11:59:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 -- 172.21.15.7:0/1541271026 shutdown_connections 2023-06-09T11:59:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c1598b0 0x7f3f1c159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1541271026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0c9fe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 -- 172.21.15.7:0/1541271026 >> 172.21.15.7:0/1541271026 conn(0x7f3f1c070af0 msgr2=0x7f3f1c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:46.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 -- 172.21.15.7:0/1541271026 shutdown_connections 2023-06-09T11:59:46.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.078+0000 7f3f219ff700 1 -- 172.21.15.7:0/1541271026 wait complete. 2023-06-09T11:59:46.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.079+0000 7f3f219ff700 1 Processor -- start 2023-06-09T11:59:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.080+0000 7f3f219ff700 1 -- start start 2023-06-09T11:59:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.080+0000 7f3f219ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.080+0000 7f3f219ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c0cad00 0x7f3f1c1f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f219ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f219ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c1f2860 con 0x7f3f1c1598b0 2023-06-09T11:59:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f219ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c3240 con 0x7f3f1c0c9090 2023-06-09T11:59:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f219ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c152010 con 0x7f3f1c0cad00 2023-06-09T11:59:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52658/0 (socket says 172.21.15.7:52658) 2023-06-09T11:59:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 -- 172.21.15.7:0/1887511195 learned_addr learned my addr 172.21.15.7:0/1887511195 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1affd700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f13fff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c0cad00 0x7f3f1c1f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 -- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c0cad00 msgr2=0x7f3f1c1f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c0cad00 0x7f3f1c1f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 -- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 msgr2=0x7f3f1c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.081+0000 7f3f1b7fe700 1 -- 172.21.15.7:0/1887511195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c009550 con 0x7f3f1c1598b0 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f1b7fe700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400efc0 tx=0x7f3f1400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400cad0 con 0x7f3f1c1598b0 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1400f040 con 0x7f3f1c1598b0 2023-06-09T11:59:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c15a250 con 0x7f3f1c1598b0 2023-06-09T11:59:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400ce70 con 0x7f3f1c1598b0 2023-06-09T11:59:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.082+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c0ca620 con 0x7f3f1c1598b0 2023-06-09T11:59:46.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.083+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0cb160 con 0x7f3f1c1598b0 2023-06-09T11:59:46.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.084+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f14014480 con 0x7f3f1c1598b0 2023-06-09T11:59:46.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.085+0000 7f3f18ff9700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 0x7f3efc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.085+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f3f1408a8c0 con 0x7f3f1c1598b0 2023-06-09T11:59:46.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.085+0000 7f3f1affd700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 0x7f3efc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.086+0000 7f3f1affd700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 0x7f3efc06d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c014040 tx=0x7f3f0c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.090+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3f14054f60 con 0x7f3f1c1598b0 2023-06-09T11:59:46.296 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.294+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f1c1f2a40 con 0x7f3efc06ace0 2023-06-09T11:59:46.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.295+0000 7f3f18ff9700 1 -- 172.21.15.7:0/1887511195 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3f1c1f2a40 con 0x7f3efc06ace0 2023-06-09T11:59:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 msgr2=0x7f3efc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 0x7f3efc06d190 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c014040 tx=0x7f3f0c008030 comp rx=0 tx=0).stop 2023-06-09T11:59:46.300 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 msgr2=0x7f3f1c150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400efc0 tx=0x7f3f1400c0c0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 shutdown_connections 2023-06-09T11:59:46.301 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f1c1598b0 0x7f3f1c150980 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f1c0c9090 0x7f3f1c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3efc06ace0 0x7f3efc06d190 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 --2- 172.21.15.7:0/1887511195 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f1c0cad00 0x7f3f1c1f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.298+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 >> 172.21.15.7:0/1887511195 conn(0x7f3f1c070af0 msgr2=0x7f3f1c0cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:46.302 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.299+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 shutdown_connections 2023-06-09T11:59:46.303 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.299+0000 7f3f219ff700 1 -- 172.21.15.7:0/1887511195 wait complete. 2023-06-09T11:59:46.303 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2023-06-09T11:59:46.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T11:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.516+0000 7fedd1bff700 1 Processor -- start 2023-06-09T11:59:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.517+0000 7fedd1bff700 1 -- start start 2023-06-09T11:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedd1bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fedcc15d4e0 0x7fedcc066a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedd1bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc071730 con 0x7fedcc1534c0 2023-06-09T11:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedd1bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc15d050 con 0x7fedcc1592e0 2023-06-09T11:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedd1bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc071ec0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedcb7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fedcc1534c0 0x7fedcc070950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34650/0 (socket says 172.21.15.7:34650) 2023-06-09T11:59:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedcb7fe700 1 -- 172.21.15.7:0/1562718909 learned_addr learned my addr 172.21.15.7:0/1562718909 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T11:59:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.518+0000 7fedcaffd700 1 --2- 172.21.15.7:0/1562718909 >> v2:172.21.15.117:3300/0 conn(0x7fedcc15d4e0 0x7fedcc066a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedca7fc700 1 -- 172.21.15.7:0/1562718909 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977798586 0 0) 0x7fedcc071730 con 0x7fedcc1534c0 2023-06-09T11:59:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedca7fc700 1 -- 172.21.15.7:0/1562718909 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb40035b0 con 0x7fedcc1534c0 2023-06-09T11:59:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedcaffd700 1 -- 172.21.15.7:0/1562718909 >> v1:172.21.15.117:6789/0 conn(0x7fedcc1592e0 legacy=0x7fedcc06e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedcaffd700 1 -- 172.21.15.7:0/1562718909 >> v1:172.21.15.7:6789/0 conn(0x7fedcc1534c0 legacy=0x7fedcc070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedcaffd700 1 -- 172.21.15.7:0/1562718909 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc071170 con 0x7fedcc15d4e0 2023-06-09T11:59:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.519+0000 7fedcaffd700 1 --2- 172.21.15.7:0/1562718909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc15d4e0 0x7fedcc066a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009840 tx=0x7fedc0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=850e5837cd4c3a91 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedca7fc700 1 -- 172.21.15.7:0/1562718909 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011830 con 0x7fedcc15d4e0 2023-06-09T11:59:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedca7fc700 1 -- 172.21.15.7:0/1562718909 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc0011990 con 0x7fedcc15d4e0 2023-06-09T11:59:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedca7fc700 1 -- 172.21.15.7:0/1562718909 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011d00 con 0x7fedcc15d4e0 2023-06-09T11:59:46.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 -- 172.21.15.7:0/1562718909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc15d4e0 msgr2=0x7fedcc066a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 --2- 172.21.15.7:0/1562718909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc15d4e0 0x7fedcc066a40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009840 tx=0x7fedc0009b50 comp rx=0 tx=0).stop 2023-06-09T11:59:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 -- 172.21.15.7:0/1562718909 shutdown_connections 2023-06-09T11:59:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 --2- 172.21.15.7:0/1562718909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc15d4e0 0x7fedcc066a40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 -- 172.21.15.7:0/1562718909 >> 172.21.15.7:0/1562718909 conn(0x7fedcc0cba20 msgr2=0x7fedcc0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 -- 172.21.15.7:0/1562718909 shutdown_connections 2023-06-09T11:59:46.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.520+0000 7fedd1bff700 1 -- 172.21.15.7:0/1562718909 wait complete. 2023-06-09T11:59:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.522+0000 7fedd1bff700 1 Processor -- start 2023-06-09T11:59:46.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.522+0000 7fedd1bff700 1 -- start start 2023-06-09T11:59:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc1534c0 0x7fedcc0c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedcc1592e0 0x7fedcc1ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1405d0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1408b0 con 0x7fedcc1534c0 2023-06-09T11:59:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.523+0000 7fedd1bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc140b90 con 0x7fedcc1592e0 2023-06-09T11:59:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52662/0 (socket says 172.21.15.7:52662) 2023-06-09T11:59:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 -- 172.21.15.7:0/336599383 learned_addr learned my addr 172.21.15.7:0/336599383 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T11:59:46.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcb7fe700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc1534c0 0x7fedcc0c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcaffd700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedcc1592e0 0x7fedcc1ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 -- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedcc1592e0 msgr2=0x7fedcc1ef1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedcc1592e0 0x7fedcc1ef1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 -- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc1534c0 msgr2=0x7fedcc0c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc1534c0 0x7fedcc0c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 -- 172.21.15.7:0/336599383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00094a0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.524+0000 7fedcbfff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fedc400cc50 tx=0x7fedc400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.525+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4003c00 con 0x7fedcc15d4e0 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.525+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4003d60 con 0x7fedcc15d4e0 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.525+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400aab0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.525+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc15ca00 con 0x7fedcc15d4e0 2023-06-09T11:59:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.525+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc071450 con 0x7fedcc15d4e0 2023-06-09T11:59:46.532 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.526+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc15cce0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.527+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fedc400ac10 con 0x7fedcc15d4e0 2023-06-09T11:59:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.528+0000 7fedb3fff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 0x7fedb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T11:59:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.528+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fedc4089d00 con 0x7fedcc15d4e0 2023-06-09T11:59:46.533 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.528+0000 7fedcb7fe700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 0x7fedb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T11:59:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.529+0000 7fedcb7fe700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 0x7fedb806d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fedbc007fd0 tx=0x7fedbc007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T11:59:46.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.532+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fedc40544c0 con 0x7fedcc15d4e0 2023-06-09T11:59:46.739 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.737+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fedcc140a20 con 0x7fedb806ace0 2023-06-09T11:59:46.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.746+0000 7fedb3fff700 1 -- 172.21.15.7:0/336599383 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315058 (secure 0 0 0) 0x7fedcc140a20 con 0x7fedb806ace0 2023-06-09T11:59:46.751 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T11:59:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 msgr2=0x7fedb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.753 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 0x7fedb806d190 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fedbc007fd0 tx=0x7fedbc007dc0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 msgr2=0x7fedcc1ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T11:59:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fedc400cc50 tx=0x7fedc400cf60 comp rx=0 tx=0).stop 2023-06-09T11:59:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 shutdown_connections 2023-06-09T11:59:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fedcc15d4e0 0x7fedcc1ef6e0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fedcc1534c0 0x7fedcc0c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fedb806ace0 0x7fedb806d190 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 --2- 172.21.15.7:0/336599383 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fedcc1592e0 0x7fedcc1ef1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T11:59:46.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.750+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 >> 172.21.15.7:0/336599383 conn(0x7fedcc0cba20 msgr2=0x7fedcc141ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T11:59:46.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.751+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 shutdown_connections 2023-06-09T11:59:46.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T11:59:46.751+0000 7fedd1bff700 1 -- 172.21.15.7:0/336599383 wait complete. 2023-06-09T11:59:46.756 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T11:59:46.771 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1539,"stamp":"2023-06-09T11:59:44.927240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374151130,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76415,"num_write_kb":1830255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88710,"ondisk_log_size":88710,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2547128,"kb_used_data":1883000,"kb_used_omap":0,"kb_used_meta":664064,"kb_avail":752427592,"statfs":{"total":773094113280,"available":770485854208,"internally_reserved":0,"allocated":1928192000,"data_stored":2752652760,"data_compressed":269255480,"data_compressed_allocated":325599232,"data_compressed_original":1192738816,"omap_allocated":0,"internal_metadata":680001536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2561996,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":2519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002881"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.176802+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:29.176802+0000","last_peered":"2023-06-09T11:59:29.176802+0000","last_clean":"2023-06-09T11:59:29.176802+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:29.176802+0000","last_undegraded":"2023-06-09T11:59:29.176802+0000","last_fullsized":"2023-06-09T11:59:29.176802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:29:41.997596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3217,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138064+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138064+0000","last_peered":"2023-06-09T11:59:29.138064+0000","last_clean":"2023-06-09T11:59:29.138064+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138064+0000","last_undegraded":"2023-06-09T11:59:29.138064+0000","last_fullsized":"2023-06-09T11:59:29.138064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:43.538089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184905+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.184905+0000","last_peered":"2023-06-09T11:59:29.184905+0000","last_clean":"2023-06-09T11:59:29.184905+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.184905+0000","last_undegraded":"2023-06-09T11:59:29.184905+0000","last_fullsized":"2023-06-09T11:59:29.184905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:54.053113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183976+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:29.183976+0000","last_peered":"2023-06-09T11:59:29.183976+0000","last_clean":"2023-06-09T11:59:29.183976+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:29.183976+0000","last_undegraded":"2023-06-09T11:59:29.183976+0000","last_fullsized":"2023-06-09T11:59:29.183976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:21:12.443270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179222+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179222+0000","last_peered":"2023-06-09T11:59:29.179222+0000","last_clean":"2023-06-09T11:59:29.179222+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179222+0000","last_undegraded":"2023-06-09T11:59:29.179222+0000","last_fullsized":"2023-06-09T11:59:29.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'864","reported_seq":3341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174723+0000","last_change":"2023-06-09T11:18:26.314501+0000","last_active":"2023-06-09T11:59:29.174723+0000","last_peered":"2023-06-09T11:59:29.174723+0000","last_clean":"2023-06-09T11:59:29.174723+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:59:29.174723+0000","last_undegraded":"2023-06-09T11:59:29.174723+0000","last_fullsized":"2023-06-09T11:59:29.174723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:01:52.942030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134658+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.134658+0000","last_peered":"2023-06-09T11:59:29.134658+0000","last_clean":"2023-06-09T11:59:29.134658+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.134658+0000","last_undegraded":"2023-06-09T11:59:29.134658+0000","last_fullsized":"2023-06-09T11:59:29.134658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:38:08.996053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3243,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164252+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164252+0000","last_peered":"2023-06-09T11:59:29.164252+0000","last_clean":"2023-06-09T11:59:29.164252+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164252+0000","last_undegraded":"2023-06-09T11:59:29.164252+0000","last_fullsized":"2023-06-09T11:59:29.164252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:39:08.386702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163514+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163514+0000","last_peered":"2023-06-09T11:59:29.163514+0000","last_clean":"2023-06-09T11:59:29.163514+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163514+0000","last_undegraded":"2023-06-09T11:59:29.163514+0000","last_fullsized":"2023-06-09T11:59:29.163514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:40.374816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184625+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184625+0000","last_peered":"2023-06-09T11:59:29.184625+0000","last_clean":"2023-06-09T11:59:29.184625+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184625+0000","last_undegraded":"2023-06-09T11:59:29.184625+0000","last_fullsized":"2023-06-09T11:59:29.184625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:26:32.298494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.166730+0000","last_change":"2023-06-09T11:18:26.271005+0000","last_active":"2023-06-09T11:59:29.166730+0000","last_peered":"2023-06-09T11:59:29.166730+0000","last_clean":"2023-06-09T11:59:29.166730+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:29.166730+0000","last_undegraded":"2023-06-09T11:59:29.166730+0000","last_fullsized":"2023-06-09T11:59:29.166730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:55.416203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3229,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108516+0000","last_change":"2023-06-09T11:18:26.253105+0000","last_active":"2023-06-09T11:59:30.108516+0000","last_peered":"2023-06-09T11:59:30.108516+0000","last_clean":"2023-06-09T11:59:30.108516+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:30.108516+0000","last_undegraded":"2023-06-09T11:59:30.108516+0000","last_fullsized":"2023-06-09T11:59:30.108516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:36:08.467688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108534+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:29.108534+0000","last_peered":"2023-06-09T11:59:29.108534+0000","last_clean":"2023-06-09T11:59:29.108534+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:29.108534+0000","last_undegraded":"2023-06-09T11:59:29.108534+0000","last_fullsized":"2023-06-09T11:59:29.108534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:08:27.844416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.134761+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:29.134761+0000","last_peered":"2023-06-09T11:59:29.134761+0000","last_clean":"2023-06-09T11:59:29.134761+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:29.134761+0000","last_undegraded":"2023-06-09T11:59:29.134761+0000","last_fullsized":"2023-06-09T11:59:29.134761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:43:06.239662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3425,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174867+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:29.174867+0000","last_peered":"2023-06-09T11:59:29.174867+0000","last_clean":"2023-06-09T11:59:29.174867+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:29.174867+0000","last_undegraded":"2023-06-09T11:59:29.174867+0000","last_fullsized":"2023-06-09T11:59:29.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:35.459206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3733,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.100281+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:30.100281+0000","last_peered":"2023-06-09T11:59:30.100281+0000","last_clean":"2023-06-09T11:59:30.100281+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:30.100281+0000","last_undegraded":"2023-06-09T11:59:30.100281+0000","last_fullsized":"2023-06-09T11:59:30.100281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.174315+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:29.174315+0000","last_peered":"2023-06-09T11:59:29.174315+0000","last_clean":"2023-06-09T11:59:29.174315+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:29.174315+0000","last_undegraded":"2023-06-09T11:59:29.174315+0000","last_fullsized":"2023-06-09T11:59:29.174315+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:47:55.986286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3553,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.108428+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.108428+0000","last_peered":"2023-06-09T11:59:29.108428+0000","last_clean":"2023-06-09T11:59:29.108428+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.108428+0000","last_undegraded":"2023-06-09T11:59:29.108428+0000","last_fullsized":"2023-06-09T11:59:29.108428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:53:59.032131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186654+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:29.186654+0000","last_peered":"2023-06-09T11:59:29.186654+0000","last_clean":"2023-06-09T11:59:29.186654+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:29.186654+0000","last_undegraded":"2023-06-09T11:59:29.186654+0000","last_fullsized":"2023-06-09T11:59:29.186654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:31.924577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.127903+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:29.127903+0000","last_peered":"2023-06-09T11:59:29.127903+0000","last_clean":"2023-06-09T11:59:29.127903+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:29.127903+0000","last_undegraded":"2023-06-09T11:59:29.127903+0000","last_fullsized":"2023-06-09T11:59:29.127903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:53.061847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.148527+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:29.148527+0000","last_peered":"2023-06-09T11:59:29.148527+0000","last_clean":"2023-06-09T11:59:29.148527+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:29.148527+0000","last_undegraded":"2023-06-09T11:59:29.148527+0000","last_fullsized":"2023-06-09T11:59:29.148527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206328+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:29.206328+0000","last_peered":"2023-06-09T11:59:29.206328+0000","last_clean":"2023-06-09T11:59:29.206328+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:29.206328+0000","last_undegraded":"2023-06-09T11:59:29.206328+0000","last_fullsized":"2023-06-09T11:59:29.206328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:31:00.135897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181162+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181162+0000","last_peered":"2023-06-09T11:59:29.181162+0000","last_clean":"2023-06-09T11:59:29.181162+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181162+0000","last_undegraded":"2023-06-09T11:59:29.181162+0000","last_fullsized":"2023-06-09T11:59:29.181162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:46:27.956223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3017,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108855+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.108855+0000","last_peered":"2023-06-09T11:59:30.108855+0000","last_clean":"2023-06-09T11:59:30.108855+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.108855+0000","last_undegraded":"2023-06-09T11:59:30.108855+0000","last_fullsized":"2023-06-09T11:59:30.108855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171290+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171290+0000","last_peered":"2023-06-09T11:59:29.171290+0000","last_clean":"2023-06-09T11:59:29.171290+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171290+0000","last_undegraded":"2023-06-09T11:59:29.171290+0000","last_fullsized":"2023-06-09T11:59:29.171290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116586+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116586+0000","last_peered":"2023-06-09T11:59:29.116586+0000","last_clean":"2023-06-09T11:59:29.116586+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116586+0000","last_undegraded":"2023-06-09T11:59:29.116586+0000","last_fullsized":"2023-06-09T11:59:29.116586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:56:28.791273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.608768+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:29.608768+0000","last_peered":"2023-06-09T11:59:29.608768+0000","last_clean":"2023-06-09T11:59:29.608768+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:29.608768+0000","last_undegraded":"2023-06-09T11:59:29.608768+0000","last_fullsized":"2023-06-09T11:59:29.608768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:54:04.925729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.159320+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:29.159320+0000","last_peered":"2023-06-09T11:59:29.159320+0000","last_clean":"2023-06-09T11:59:29.159320+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:29.159320+0000","last_undegraded":"2023-06-09T11:59:29.159320+0000","last_fullsized":"2023-06-09T11:59:29.159320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:04:01.139122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2241,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.087049+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:30.087049+0000","last_peered":"2023-06-09T11:59:30.087049+0000","last_clean":"2023-06-09T11:59:30.087049+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:30.087049+0000","last_undegraded":"2023-06-09T11:59:30.087049+0000","last_fullsized":"2023-06-09T11:59:30.087049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3134,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105820+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.105820+0000","last_peered":"2023-06-09T11:59:30.105820+0000","last_clean":"2023-06-09T11:59:30.105820+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.105820+0000","last_undegraded":"2023-06-09T11:59:30.105820+0000","last_fullsized":"2023-06-09T11:59:30.105820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826279+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:23.826279+0000","last_peered":"2023-06-09T11:59:23.826279+0000","last_clean":"2023-06-09T11:59:23.826279+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:23.826279+0000","last_undegraded":"2023-06-09T11:59:23.826279+0000","last_fullsized":"2023-06-09T11:59:23.826279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:19:22.681147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120527+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120527+0000","last_peered":"2023-06-09T11:59:29.120527+0000","last_clean":"2023-06-09T11:59:29.120527+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120527+0000","last_undegraded":"2023-06-09T11:59:29.120527+0000","last_fullsized":"2023-06-09T11:59:29.120527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:31:15.836484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'181","reported_seq":345,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.547291+0000","last_change":"2023-06-09T11:10:49.677528+0000","last_active":"2023-06-09T11:59:34.547291+0000","last_peered":"2023-06-09T11:59:34.547291+0000","last_clean":"2023-06-09T11:59:34.547291+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:34.547291+0000","last_undegraded":"2023-06-09T11:59:34.547291+0000","last_fullsized":"2023-06-09T11:59:34.547291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":125,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2960,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.204530+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:29.204530+0000","last_peered":"2023-06-09T11:59:29.204530+0000","last_clean":"2023-06-09T11:59:29.204530+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:29.204530+0000","last_undegraded":"2023-06-09T11:59:29.204530+0000","last_fullsized":"2023-06-09T11:59:29.204530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:51:52.478717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813587+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:59:17.813587+0000","last_peered":"2023-06-09T11:59:17.813587+0000","last_clean":"2023-06-09T11:59:17.813587+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:59:17.813587+0000","last_undegraded":"2023-06-09T11:59:17.813587+0000","last_fullsized":"2023-06-09T11:59:17.813587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:40.338566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208409+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:29.208409+0000","last_peered":"2023-06-09T11:59:29.208409+0000","last_clean":"2023-06-09T11:59:29.208409+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:29.208409+0000","last_undegraded":"2023-06-09T11:59:29.208409+0000","last_fullsized":"2023-06-09T11:59:29.208409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:32:56.568860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086513+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:59:30.086513+0000","last_peered":"2023-06-09T11:59:30.086513+0000","last_clean":"2023-06-09T11:59:30.086513+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:30.086513+0000","last_undegraded":"2023-06-09T11:59:30.086513+0000","last_fullsized":"2023-06-09T11:59:30.086513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:11:48.144420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105979+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:30.105979+0000","last_peered":"2023-06-09T11:59:30.105979+0000","last_clean":"2023-06-09T11:59:30.105979+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:30.105979+0000","last_undegraded":"2023-06-09T11:59:30.105979+0000","last_fullsized":"2023-06-09T11:59:30.105979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'823","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.465432+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:30.465432+0000","last_peered":"2023-06-09T11:59:30.465432+0000","last_clean":"2023-06-09T11:59:30.465432+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:30.465432+0000","last_undegraded":"2023-06-09T11:59:30.465432+0000","last_fullsized":"2023-06-09T11:59:30.465432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:15:45.170741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":768,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3094,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186545+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:29.186545+0000","last_peered":"2023-06-09T11:59:29.186545+0000","last_clean":"2023-06-09T11:59:29.186545+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:29.186545+0000","last_undegraded":"2023-06-09T11:59:29.186545+0000","last_fullsized":"2023-06-09T11:59:29.186545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:43.603410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'863","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:40.068817+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:40.068817+0000","last_peered":"2023-06-09T11:59:40.068817+0000","last_clean":"2023-06-09T11:59:40.068817+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:40.068817+0000","last_undegraded":"2023-06-09T11:59:40.068817+0000","last_fullsized":"2023-06-09T11:59:40.068817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":863,"log_dups_size":0,"ondisk_log_size":863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:55:48.089456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15646594,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":805,"num_write_kb":15871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3556,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.161492+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:29.161492+0000","last_peered":"2023-06-09T11:59:29.161492+0000","last_clean":"2023-06-09T11:59:29.161492+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:29.161492+0000","last_undegraded":"2023-06-09T11:59:29.161492+0000","last_fullsized":"2023-06-09T11:59:29.161492+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:40:48.346810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'413","reported_seq":576,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.603714+0000","last_change":"2023-06-09T11:10:49.674741+0000","last_active":"2023-06-09T11:59:29.603714+0000","last_peered":"2023-06-09T11:59:29.603714+0000","last_clean":"2023-06-09T11:59:29.603714+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:29.603714+0000","last_undegraded":"2023-06-09T11:59:29.603714+0000","last_fullsized":"2023-06-09T11:59:29.603714+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116041+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116041+0000","last_peered":"2023-06-09T11:59:29.116041+0000","last_clean":"2023-06-09T11:59:29.116041+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116041+0000","last_undegraded":"2023-06-09T11:59:29.116041+0000","last_fullsized":"2023-06-09T11:59:29.116041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:24:03.501267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'961","reported_seq":1144,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824131+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:59:23.824131+0000","last_peered":"2023-06-09T11:59:23.824131+0000","last_clean":"2023-06-09T11:59:23.824131+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:59:23.824131+0000","last_undegraded":"2023-06-09T11:59:23.824131+0000","last_fullsized":"2023-06-09T11:59:23.824131+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:53:42.964479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466196+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:30.466196+0000","last_peered":"2023-06-09T11:59:30.466196+0000","last_clean":"2023-06-09T11:59:30.466196+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:30.466196+0000","last_undegraded":"2023-06-09T11:59:30.466196+0000","last_fullsized":"2023-06-09T11:59:30.466196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:35:30.247597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203830+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203830+0000","last_peered":"2023-06-09T11:59:29.203830+0000","last_clean":"2023-06-09T11:59:29.203830+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203830+0000","last_undegraded":"2023-06-09T11:59:29.203830+0000","last_fullsized":"2023-06-09T11:59:29.203830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:33:19.947716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825423+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.825423+0000","last_peered":"2023-06-09T11:59:23.825423+0000","last_clean":"2023-06-09T11:59:23.825423+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.825423+0000","last_undegraded":"2023-06-09T11:59:23.825423+0000","last_fullsized":"2023-06-09T11:59:23.825423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:22:06.716248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113291+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113291+0000","last_peered":"2023-06-09T11:59:29.113291+0000","last_clean":"2023-06-09T11:59:29.113291+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113291+0000","last_undegraded":"2023-06-09T11:59:29.113291+0000","last_fullsized":"2023-06-09T11:59:29.113291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:25:53.990234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.607238+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:29.607238+0000","last_peered":"2023-06-09T11:59:29.607238+0000","last_clean":"2023-06-09T11:59:29.607238+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:29.607238+0000","last_undegraded":"2023-06-09T11:59:29.607238+0000","last_fullsized":"2023-06-09T11:59:29.607238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:10:40.976689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178950+0000","last_change":"2023-06-09T11:18:26.315365+0000","last_active":"2023-06-09T11:59:29.178950+0000","last_peered":"2023-06-09T11:59:29.178950+0000","last_clean":"2023-06-09T11:59:29.178950+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:29.178950+0000","last_undegraded":"2023-06-09T11:59:29.178950+0000","last_fullsized":"2023-06-09T11:59:29.178950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:48:14.978014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086762+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:30.086762+0000","last_peered":"2023-06-09T11:59:30.086762+0000","last_clean":"2023-06-09T11:59:30.086762+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:30.086762+0000","last_undegraded":"2023-06-09T11:59:30.086762+0000","last_fullsized":"2023-06-09T11:59:30.086762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:20:17.698189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088582+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.088582+0000","last_peered":"2023-06-09T11:59:30.088582+0000","last_clean":"2023-06-09T11:59:30.088582+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.088582+0000","last_undegraded":"2023-06-09T11:59:30.088582+0000","last_fullsized":"2023-06-09T11:59:30.088582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:35:52.399332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":688,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.825965+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:59:23.825965+0000","last_peered":"2023-06-09T11:59:23.825965+0000","last_clean":"2023-06-09T11:59:23.825965+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:59:23.825965+0000","last_undegraded":"2023-06-09T11:59:23.825965+0000","last_fullsized":"2023-06-09T11:59:23.825965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:30:58.048595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.155241+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:29.155241+0000","last_peered":"2023-06-09T11:59:29.155241+0000","last_clean":"2023-06-09T11:59:29.155241+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:29.155241+0000","last_undegraded":"2023-06-09T11:59:29.155241+0000","last_fullsized":"2023-06-09T11:59:29.155241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.812549+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:17.812549+0000","last_peered":"2023-06-09T11:59:17.812549+0000","last_clean":"2023-06-09T11:59:17.812549+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:17.812549+0000","last_undegraded":"2023-06-09T11:59:17.812549+0000","last_fullsized":"2023-06-09T11:59:17.812549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:13:59.524461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105462+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:30.105462+0000","last_peered":"2023-06-09T11:59:30.105462+0000","last_clean":"2023-06-09T11:59:30.105462+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:30.105462+0000","last_undegraded":"2023-06-09T11:59:30.105462+0000","last_fullsized":"2023-06-09T11:59:30.105462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:52:08.337474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1064","reported_seq":1239,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822915+0000","last_change":"2023-06-09T11:10:49.682706+0000","last_active":"2023-06-09T11:59:23.822915+0000","last_peered":"2023-06-09T11:59:23.822915+0000","last_clean":"2023-06-09T11:59:23.822915+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:59:23.822915+0000","last_undegraded":"2023-06-09T11:59:23.822915+0000","last_fullsized":"2023-06-09T11:59:23.822915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1064,"log_dups_size":0,"ondisk_log_size":1064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:47:25.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180294+0000","last_change":"2023-06-09T11:18:26.342753+0000","last_active":"2023-06-09T11:59:29.180294+0000","last_peered":"2023-06-09T11:59:29.180294+0000","last_clean":"2023-06-09T11:59:29.180294+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:29.180294+0000","last_undegraded":"2023-06-09T11:59:29.180294+0000","last_fullsized":"2023-06-09T11:59:29.180294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:54:36.852596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824479+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:23.824479+0000","last_peered":"2023-06-09T11:59:23.824479+0000","last_clean":"2023-06-09T11:59:23.824479+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:23.824479+0000","last_undegraded":"2023-06-09T11:59:23.824479+0000","last_fullsized":"2023-06-09T11:59:23.824479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:00:20.027849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3202,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.182579+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:29.182579+0000","last_peered":"2023-06-09T11:59:29.182579+0000","last_clean":"2023-06-09T11:59:29.182579+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:29.182579+0000","last_undegraded":"2023-06-09T11:59:29.182579+0000","last_fullsized":"2023-06-09T11:59:29.182579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:36:14.768104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.811217+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:17.811217+0000","last_peered":"2023-06-09T11:59:17.811217+0000","last_clean":"2023-06-09T11:59:17.811217+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:17.811217+0000","last_undegraded":"2023-06-09T11:59:17.811217+0000","last_fullsized":"2023-06-09T11:59:17.811217+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:02:12.084788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3206,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.093008+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:30.093008+0000","last_peered":"2023-06-09T11:59:30.093008+0000","last_clean":"2023-06-09T11:59:30.093008+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:30.093008+0000","last_undegraded":"2023-06-09T11:59:30.093008+0000","last_fullsized":"2023-06-09T11:59:30.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.837038+0000","last_change":"2023-06-09T11:10:43.862299+0000","last_active":"2023-06-09T11:59:17.837038+0000","last_peered":"2023-06-09T11:59:17.837038+0000","last_clean":"2023-06-09T11:59:17.837038+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:17.837038+0000","last_undegraded":"2023-06-09T11:59:17.837038+0000","last_fullsized":"2023-06-09T11:59:17.837038+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813812+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:17.813812+0000","last_peered":"2023-06-09T11:59:17.813812+0000","last_clean":"2023-06-09T11:59:17.813812+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:17.813812+0000","last_undegraded":"2023-06-09T11:59:17.813812+0000","last_fullsized":"2023-06-09T11:59:17.813812+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:21:04.925798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198957+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198957+0000","last_peered":"2023-06-09T11:59:29.198957+0000","last_clean":"2023-06-09T11:59:29.198957+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198957+0000","last_undegraded":"2023-06-09T11:59:29.198957+0000","last_fullsized":"2023-06-09T11:59:29.198957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813891+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:59:17.813891+0000","last_peered":"2023-06-09T11:59:17.813891+0000","last_clean":"2023-06-09T11:59:17.813891+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:17.813891+0000","last_undegraded":"2023-06-09T11:59:17.813891+0000","last_fullsized":"2023-06-09T11:59:17.813891+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:05:44.339799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":895,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826018+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:23.826018+0000","last_peered":"2023-06-09T11:59:23.826018+0000","last_clean":"2023-06-09T11:59:23.826018+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:23.826018+0000","last_undegraded":"2023-06-09T11:59:23.826018+0000","last_fullsized":"2023-06-09T11:59:23.826018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:42:11.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3888,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203700+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203700+0000","last_peered":"2023-06-09T11:59:29.203700+0000","last_clean":"2023-06-09T11:59:29.203700+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203700+0000","last_undegraded":"2023-06-09T11:59:29.203700+0000","last_fullsized":"2023-06-09T11:59:29.203700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.847101+0000","last_change":"2023-06-09T11:10:43.558471+0000","last_active":"2023-06-09T11:59:17.847101+0000","last_peered":"2023-06-09T11:59:17.847101+0000","last_clean":"2023-06-09T11:59:17.847101+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:17.847101+0000","last_undegraded":"2023-06-09T11:59:17.847101+0000","last_fullsized":"2023-06-09T11:59:17.847101+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:05:14.595589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822857+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.822857+0000","last_peered":"2023-06-09T11:59:23.822857+0000","last_clean":"2023-06-09T11:59:23.822857+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.822857+0000","last_undegraded":"2023-06-09T11:59:23.822857+0000","last_fullsized":"2023-06-09T11:59:23.822857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105068+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.105068+0000","last_peered":"2023-06-09T11:59:30.105068+0000","last_clean":"2023-06-09T11:59:30.105068+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.105068+0000","last_undegraded":"2023-06-09T11:59:30.105068+0000","last_fullsized":"2023-06-09T11:59:30.105068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814030+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:59:17.814030+0000","last_peered":"2023-06-09T11:59:17.814030+0000","last_clean":"2023-06-09T11:59:17.814030+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:17.814030+0000","last_undegraded":"2023-06-09T11:59:17.814030+0000","last_fullsized":"2023-06-09T11:59:17.814030+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:17.409970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810193+0000","last_change":"2023-06-09T11:10:43.863036+0000","last_active":"2023-06-09T11:59:17.810193+0000","last_peered":"2023-06-09T11:59:17.810193+0000","last_clean":"2023-06-09T11:59:17.810193+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:17.810193+0000","last_undegraded":"2023-06-09T11:59:17.810193+0000","last_fullsized":"2023-06-09T11:59:17.810193+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:47:29.671673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1140","reported_seq":1323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.824017+0000","last_change":"2023-06-09T11:10:49.674253+0000","last_active":"2023-06-09T11:59:23.824017+0000","last_peered":"2023-06-09T11:59:23.824017+0000","last_clean":"2023-06-09T11:59:23.824017+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:59:23.824017+0000","last_undegraded":"2023-06-09T11:59:23.824017+0000","last_fullsized":"2023-06-09T11:59:23.824017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:27:21.609537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3295,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142633+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142633+0000","last_peered":"2023-06-09T11:59:29.142633+0000","last_clean":"2023-06-09T11:59:29.142633+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142633+0000","last_undegraded":"2023-06-09T11:59:29.142633+0000","last_fullsized":"2023-06-09T11:59:29.142633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:51:16.604757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810429+0000","last_change":"2023-06-09T11:10:43.556935+0000","last_active":"2023-06-09T11:59:17.810429+0000","last_peered":"2023-06-09T11:59:17.810429+0000","last_clean":"2023-06-09T11:59:17.810429+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:17.810429+0000","last_undegraded":"2023-06-09T11:59:17.810429+0000","last_fullsized":"2023-06-09T11:59:17.810429+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T11:44:14.620706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.467905+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.467905+0000","last_peered":"2023-06-09T11:59:30.467905+0000","last_clean":"2023-06-09T11:59:30.467905+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.467905+0000","last_undegraded":"2023-06-09T11:59:30.467905+0000","last_fullsized":"2023-06-09T11:59:30.467905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185984+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185984+0000","last_peered":"2023-06-09T11:59:29.185984+0000","last_clean":"2023-06-09T11:59:29.185984+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185984+0000","last_undegraded":"2023-06-09T11:59:29.185984+0000","last_fullsized":"2023-06-09T11:59:29.185984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.810999+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:59:17.810999+0000","last_peered":"2023-06-09T11:59:17.810999+0000","last_clean":"2023-06-09T11:59:17.810999+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:59:17.810999+0000","last_undegraded":"2023-06-09T11:59:17.810999+0000","last_fullsized":"2023-06-09T11:59:17.810999+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:24:10.967583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":820,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823970+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823970+0000","last_peered":"2023-06-09T11:59:23.823970+0000","last_clean":"2023-06-09T11:59:23.823970+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823970+0000","last_undegraded":"2023-06-09T11:59:23.823970+0000","last_fullsized":"2023-06-09T11:59:23.823970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.303700+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:30.303700+0000","last_peered":"2023-06-09T11:59:30.303700+0000","last_clean":"2023-06-09T11:59:30.303700+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:30.303700+0000","last_undegraded":"2023-06-09T11:59:30.303700+0000","last_fullsized":"2023-06-09T11:59:30.303700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":518,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105019+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.105019+0000","last_peered":"2023-06-09T11:59:30.105019+0000","last_clean":"2023-06-09T11:59:30.105019+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.105019+0000","last_undegraded":"2023-06-09T11:59:30.105019+0000","last_fullsized":"2023-06-09T11:59:30.105019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:59:20.477510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.839538+0000","last_change":"2023-06-09T11:10:43.561512+0000","last_active":"2023-06-09T11:59:17.839538+0000","last_peered":"2023-06-09T11:59:17.839538+0000","last_clean":"2023-06-09T11:59:17.839538+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:17.839538+0000","last_undegraded":"2023-06-09T11:59:17.839538+0000","last_fullsized":"2023-06-09T11:59:17.839538+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208331+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208331+0000","last_peered":"2023-06-09T11:59:29.208331+0000","last_clean":"2023-06-09T11:59:29.208331+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208331+0000","last_undegraded":"2023-06-09T11:59:29.208331+0000","last_fullsized":"2023-06-09T11:59:29.208331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:58:58.992911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.819642+0000","last_change":"2023-06-09T11:10:43.561204+0000","last_active":"2023-06-09T11:59:17.819642+0000","last_peered":"2023-06-09T11:59:17.819642+0000","last_clean":"2023-06-09T11:59:17.819642+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:17.819642+0000","last_undegraded":"2023-06-09T11:59:17.819642+0000","last_fullsized":"2023-06-09T11:59:17.819642+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:01:01.435357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":826,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.052133+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:34.052133+0000","last_peered":"2023-06-09T11:59:34.052133+0000","last_clean":"2023-06-09T11:59:34.052133+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:34.052133+0000","last_undegraded":"2023-06-09T11:59:34.052133+0000","last_fullsized":"2023-06-09T11:59:34.052133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3713,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170033+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170033+0000","last_peered":"2023-06-09T11:59:29.170033+0000","last_clean":"2023-06-09T11:59:29.170033+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170033+0000","last_undegraded":"2023-06-09T11:59:29.170033+0000","last_fullsized":"2023-06-09T11:59:29.170033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:28:31.292243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":880,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088435+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.088435+0000","last_peered":"2023-06-09T11:59:30.088435+0000","last_clean":"2023-06-09T11:59:30.088435+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.088435+0000","last_undegraded":"2023-06-09T11:59:30.088435+0000","last_fullsized":"2023-06-09T11:59:30.088435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'955","reported_seq":4519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095034+0000","last_change":"2023-06-09T11:18:26.261803+0000","last_active":"2023-06-09T11:59:30.095034+0000","last_peered":"2023-06-09T11:59:30.095034+0000","last_clean":"2023-06-09T11:59:30.095034+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T11:59:30.095034+0000","last_undegraded":"2023-06-09T11:59:30.095034+0000","last_fullsized":"2023-06-09T11:59:30.095034+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:10:57.891708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":765,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.090216+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.090216+0000","last_peered":"2023-06-09T11:59:30.090216+0000","last_clean":"2023-06-09T11:59:30.090216+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.090216+0000","last_undegraded":"2023-06-09T11:59:30.090216+0000","last_fullsized":"2023-06-09T11:59:30.090216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:58:31.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.178756+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.178756+0000","last_peered":"2023-06-09T11:59:29.178756+0000","last_clean":"2023-06-09T11:59:29.178756+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.178756+0000","last_undegraded":"2023-06-09T11:59:29.178756+0000","last_fullsized":"2023-06-09T11:59:29.178756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:09:06.669625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'581","reported_seq":774,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:39.099995+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:39.099995+0000","last_peered":"2023-06-09T11:59:39.099995+0000","last_clean":"2023-06-09T11:59:39.099995+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:39.099995+0000","last_undegraded":"2023-06-09T11:59:39.099995+0000","last_fullsized":"2023-06-09T11:59:39.099995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:53:38.980281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23823733,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":507,"num_write_kb":27711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3673,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179771+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:29.179771+0000","last_peered":"2023-06-09T11:59:29.179771+0000","last_clean":"2023-06-09T11:59:29.179771+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:29.179771+0000","last_undegraded":"2023-06-09T11:59:29.179771+0000","last_fullsized":"2023-06-09T11:59:29.179771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:10:40.529511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1031,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206896+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:59:29.206896+0000","last_peered":"2023-06-09T11:59:29.206896+0000","last_clean":"2023-06-09T11:59:29.206896+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:59:29.206896+0000","last_undegraded":"2023-06-09T11:59:29.206896+0000","last_fullsized":"2023-06-09T11:59:29.206896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:57.664891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3147,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206536+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.206536+0000","last_peered":"2023-06-09T11:59:29.206536+0000","last_clean":"2023-06-09T11:59:29.206536+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.206536+0000","last_undegraded":"2023-06-09T11:59:29.206536+0000","last_fullsized":"2023-06-09T11:59:29.206536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:49:35.680592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"73'1383","reported_seq":1601,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.821320+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:59:23.821320+0000","last_peered":"2023-06-09T11:59:23.821320+0000","last_clean":"2023-06-09T11:59:23.821320+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:59:23.821320+0000","last_undegraded":"2023-06-09T11:59:23.821320+0000","last_fullsized":"2023-06-09T11:59:23.821320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187254+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:29.187254+0000","last_peered":"2023-06-09T11:59:29.187254+0000","last_clean":"2023-06-09T11:59:29.187254+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:29.187254+0000","last_undegraded":"2023-06-09T11:59:29.187254+0000","last_fullsized":"2023-06-09T11:59:29.187254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:17:10.649718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120227+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120227+0000","last_peered":"2023-06-09T11:59:29.120227+0000","last_clean":"2023-06-09T11:59:29.120227+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120227+0000","last_undegraded":"2023-06-09T11:59:29.120227+0000","last_fullsized":"2023-06-09T11:59:29.120227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:29:14.112446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207330+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.207330+0000","last_peered":"2023-06-09T11:59:29.207330+0000","last_clean":"2023-06-09T11:59:29.207330+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.207330+0000","last_undegraded":"2023-06-09T11:59:29.207330+0000","last_fullsized":"2023-06-09T11:59:29.207330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:56:35.301385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187085+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187085+0000","last_peered":"2023-06-09T11:59:29.187085+0000","last_clean":"2023-06-09T11:59:29.187085+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187085+0000","last_undegraded":"2023-06-09T11:59:29.187085+0000","last_fullsized":"2023-06-09T11:59:29.187085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:10:25.264503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2977,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.180708+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.180708+0000","last_peered":"2023-06-09T11:59:29.180708+0000","last_clean":"2023-06-09T11:59:29.180708+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.180708+0000","last_undegraded":"2023-06-09T11:59:29.180708+0000","last_fullsized":"2023-06-09T11:59:29.180708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:36:34.824605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207673+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207673+0000","last_peered":"2023-06-09T11:59:29.207673+0000","last_clean":"2023-06-09T11:59:29.207673+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207673+0000","last_undegraded":"2023-06-09T11:59:29.207673+0000","last_fullsized":"2023-06-09T11:59:29.207673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58085,"ondisk_log_size":58085,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":509614259,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29964,"num_write_kb":671139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290127872,"data_stored":1019228518,"data_compressed":127145960,"data_compressed_allocated":167993344,"data_compressed_original":897253376,"omap_allocated":0,"internal_metadata":0},"log_size":30508,"ondisk_log_size":30508,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575439,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254220,"kb_used_data":179844,"kb_used_omap":0,"kb_used_meta":74368,"kb_avail":94117620,"statfs":{"total":96636764160,"available":96376442880,"internally_reserved":0,"allocated":184160256,"data_stored":310943269,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76152832},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.496}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jun 9 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.86899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.755},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.94199999999999995,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.74099999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575439,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.502},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.63700000000000001}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.40799999999999997}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.625},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.57999999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575439,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396016,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89600,"kb_avail":93975824,"statfs":{"total":96636764160,"available":96231243776,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91750400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.495},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":0.54100000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.69699999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575439,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327400,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99584,"kb_avail":94044440,"statfs":{"total":96636764160,"available":96301506560,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101974016},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.443},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.85599999999999998},"last":0.35799999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":1.0900000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.308},"max":{"1min":0.871,"5min":0.871,"15min":1.2909999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.1060000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97499999999999998,"5min":1.21,"15min":1.21},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0429999999999999,"5min":1.05,"15min":1.236},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.165},"last":0.754}]},{"osd":6,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.3700000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":1.159,"15min":1.228},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83299999999999996,"15min":0.79600000000000004},"min":{"1min":0.64200000000000002,"5min":0.52400000000000002,"15min":0.46999999999999997},"max":{"1min":1.0860000000000001,"5min":1.111,"15min":1.29},"last":0.84299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":232480,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52672,"kb_avail":94139360,"statfs":{"total":96636764160,"available":96398704640,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53936128},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.432},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.76300000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.38800000000000001,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":1.367,"15min":1.367},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.546,"15min":1.546},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63300000000000001,"15min":0.59399999999999997},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.48699999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.55500000000000005,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.877}]},{"osd":5,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.0109999999999999,"5min":1.601,"15min":1.601},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84799999999999998,"15min":0.82199999999999995},"min":{"1min":0.71599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.0660000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.747},"min":{"1min":0.58999999999999997,"5min":0.496,"15min":0.438},"max":{"1min":0.878,"5min":1.2090000000000001,"15min":1.3100000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":1.121,"15min":1.3540000000000001},"last":0.92700000000000005}]},{"osd":7,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.66100000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.208,"15min":1.248},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.82899999999999996,"15min":0.81000000000000005},"min":{"1min":0.73899999999999999,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.3080000000000001,"15min":1.337},"last":1.014}]}]},{"osd":3,"up_from":13,"seq":55834575439,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.443}]},{"osd":1,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.60799999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.80600000000000005,"5min":1.355,"15min":1.355},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76700000000000002,"5min":1.198,"15min":1.198},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":1.635,"15min":1.635},"last":0.754},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":2.351,"15min":2.351},"last":3.081}]},{"osd":5,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.61899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95399999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.64400000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":0.97599999999999998,"5min":1.367,"15min":1.367},"last":0.89300000000000002}]},{"osd":6,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":1.286,"15min":1.286},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.98499999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.99399999999999999,"5min":1.413,"15min":1.413},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.45,"15min":1.45},"last":3.1160000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575438,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413068,"kb_used_data":292612,"kb_used_omap":0,"kb_used_meta":120448,"kb_avail":93958772,"statfs":{"total":96636764160,"available":96213782528,"internally_reserved":0,"allocated":299634688,"data_stored":480794096,"data_compressed":51528895,"data_compressed_allocated":62828544,"data_compressed_original":250146816,"omap_allocated":0,"internal_metadata":123338752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.81999999999999995,"5min":1.137,"15min":1.137},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.83299999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.41199999999999998,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.0640000000000001,"15min":1.103},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1279999999999999,"15min":1.2},"last":0.72699999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":1.609,"15min":1.609},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.47999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.871,"5min":1.159,"15min":1.1659999999999999},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.407,"15min":1.407},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.0620000000000001,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":1.046,"5min":1.361,"15min":1.3740000000000001},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575438,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287828,"kb_used_data":217228,"kb_used_omap":0,"kb_used_meta":70592,"kb_avail":94084012,"statfs":{"total":96636764160,"available":96342028288,"internally_reserved":0,"allocated":222441472,"data_stored":325669191,"data_compressed":32213196,"data_compressed_allocated":39047168,"data_compressed_original":146718720,"omap_allocated":0,"internal_metadata":72286208},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":0.71899999999999997},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":0.73199999999999998}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.749}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":0.68899999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":0.79800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":63913984,"data_stored":230142886,"data_compressed":29082874,"data_compressed_allocated":38076416,"data_compressed_original":204345344,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35594240,"data_stored":128682679,"data_compressed":16191638,"data_compressed_allocated":21397504,"data_compressed_original":114507776,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42438656,"data_stored":159968459,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T11:59:46.782 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.783 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.783 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.783 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.783 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.784 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.784 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.784 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.785 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.785 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.785 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.785 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.786 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.786 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.786 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.787 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.787 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.787 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.787 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.788 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.788 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.788 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.789 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.789 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.789 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.790 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.790 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.790 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.790 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.791 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.791 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.791 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.792 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.792 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.792 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.792 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.793 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.793 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.793 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.794 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.794 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.794 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.794 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.795 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.795 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.795 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.796 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.796 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.796 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.796 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.797 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.797 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.797 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.798 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.798 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.798 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.799 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.799 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.799 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.799 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.800 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.800 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.800 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.801 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.801 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.801 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.801 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.802 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.802 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.802 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.803 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.803 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.803 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.803 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.804 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.804 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.804 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.805 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.805 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.805 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.805 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.806 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.806 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.806 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.807 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.807 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.807 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.807 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.808 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.808 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.808 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.808 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.809 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.809 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.809 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.810 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.810 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.810 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.810 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.811 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.811 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.811 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.812 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.812 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.812 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T11:59:46.812 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T11:59:55.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T11:59:55.686 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:06.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:00:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.025+0000 7f8c581ff700 1 Processor -- start 2023-06-09T12:00:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.025+0000 7f8c581ff700 1 -- start start 2023-06-09T12:00:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.026+0000 7f8c581ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.026+0000 7f8c581ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 0x7f8c500c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.026+0000 7f8c581ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5015dcf0 con 0x7f8c500b5360 2023-06-09T12:00:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.026+0000 7f8c581ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c500c5e00 con 0x7f8c500ca580 2023-06-09T12:00:07.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.026+0000 7f8c581ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5015d450 con 0x7f8c500ca1b0 2023-06-09T12:00:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c5679c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 0x7f8c500c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39588/0 (socket says 172.21.15.7:39588) 2023-06-09T12:00:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c5679c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 0x7f8c500c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:60938/0 (socket says 172.21.15.7:60938) 2023-06-09T12:00:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c5579a700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8c500b5360 0x7f8c501536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:49442/0 (socket says 172.21.15.7:49442) 2023-06-09T12:00:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 -- 172.21.15.7:0/2359112918 learned_addr learned my addr 172.21.15.7:0/2359112918 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c54f99700 1 -- 172.21.15.7:0/2359112918 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139852857 0 0) 0x7f8c5015dcf0 con 0x7f8c500b5360 2023-06-09T12:00:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 -- 172.21.15.7:0/2359112918 >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 msgr2=0x7f8c500c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:00:07.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 --2- 172.21.15.7:0/2359112918 >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 0x7f8c500c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 -- 172.21.15.7:0/2359112918 >> v1:172.21.15.117:6789/0 conn(0x7f8c500b5360 legacy=0x7f8c501536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.027+0000 7f8c55f9b700 1 -- 172.21.15.7:0/2359112918 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5015d730 con 0x7f8c500ca1b0 2023-06-09T12:00:07.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c55f9b700 1 --2- 172.21.15.7:0/2359112918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0098f0 tx=0x7f8c4c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4010cfb89f28bd25 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:07.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c54f99700 1 -- 172.21.15.7:0/2359112918 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c011830 con 0x7f8c500ca1b0 2023-06-09T12:00:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c54f99700 1 -- 172.21.15.7:0/2359112918 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c011990 con 0x7f8c500ca1b0 2023-06-09T12:00:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c54f99700 1 -- 172.21.15.7:0/2359112918 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c011d20 con 0x7f8c500ca1b0 2023-06-09T12:00:07.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c581ff700 1 -- 172.21.15.7:0/2359112918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 msgr2=0x7f8c5015cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.028+0000 7f8c581ff700 1 --2- 172.21.15.7:0/2359112918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0098f0 tx=0x7f8c4c009c00 comp rx=0 tx=0).stop 2023-06-09T12:00:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 -- 172.21.15.7:0/2359112918 shutdown_connections 2023-06-09T12:00:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 --2- 172.21.15.7:0/2359112918 >> v2:172.21.15.117:3300/0 conn(0x7f8c500ca580 0x7f8c500c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 --2- 172.21.15.7:0/2359112918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500ca1b0 0x7f8c5015cdc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 -- 172.21.15.7:0/2359112918 >> 172.21.15.7:0/2359112918 conn(0x7f8c50072090 msgr2=0x7f8c50072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 -- 172.21.15.7:0/2359112918 shutdown_connections 2023-06-09T12:00:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.029+0000 7f8c581ff700 1 -- 172.21.15.7:0/2359112918 wait complete. 2023-06-09T12:00:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.030+0000 7f8c581ff700 1 Processor -- start 2023-06-09T12:00:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.031+0000 7f8c581ff700 1 -- start start 2023-06-09T12:00:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.031+0000 7f8c581ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c581ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c581ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500ca580 0x7f8c5015be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c581ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c501e3cc0 con 0x7f8c500ca1b0 2023-06-09T12:00:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c581ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5015a990 con 0x7f8c500ca580 2023-06-09T12:00:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c581ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5015b400 con 0x7f8c500b5360 2023-06-09T12:00:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c5579a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39604/0 (socket says 172.21.15.7:39604) 2023-06-09T12:00:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c5579a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36962/0 (socket says 172.21.15.7:36962) 2023-06-09T12:00:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 -- 172.21.15.7:0/1978649547 learned_addr learned my addr 172.21.15.7:0/1978649547 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c5679c700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500ca580 0x7f8c5015be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 -- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500ca580 msgr2=0x7f8c5015be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500ca580 0x7f8c5015be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 -- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 msgr2=0x7f8c501e5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c55f9b700 1 -- 172.21.15.7:0/1978649547 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c009550 con 0x7f8c500b5360 2023-06-09T12:00:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.032+0000 7f8c5579a700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:00:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c55f9b700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009c40 tx=0x7f8c4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004300 con 0x7f8c500b5360 2023-06-09T12:00:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4c01f070 con 0x7f8c500b5360 2023-06-09T12:00:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5006a570 con 0x7f8c500b5360 2023-06-09T12:00:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0036a0 con 0x7f8c500b5360 2023-06-09T12:00:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.033+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5015da10 con 0x7f8c500b5360 2023-06-09T12:00:07.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.034+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5006a850 con 0x7f8c500b5360 2023-06-09T12:00:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.035+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c4c021030 con 0x7f8c500b5360 2023-06-09T12:00:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.036+0000 7f8c467fc700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.036+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8c4c08b150 con 0x7f8c500b5360 2023-06-09T12:00:07.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.036+0000 7f8c5579a700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:07.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.037+0000 7f8c5579a700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c50149210 tx=0x7f8c40007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:07.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.041+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8c4c055890 con 0x7f8c500b5360 2023-06-09T12:00:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.247+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c5015bca0 con 0x7f8c3406ace0 2023-06-09T12:00:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.256+0000 7f8c467fc700 1 -- 172.21.15.7:0/1978649547 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312678 (secure 0 0 0) 0x7f8c5015bca0 con 0x7f8c3406ace0 2023-06-09T12:00:07.263 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:00:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 msgr2=0x7f8c3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8c50149210 tx=0x7f8c40007de0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 msgr2=0x7f8c501e3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009c40 tx=0x7f8c4c004750 comp rx=0 tx=0).stop 2023-06-09T12:00:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 shutdown_connections 2023-06-09T12:00:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8c500ca580 0x7f8c5015be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8c3406ace0 0x7f8c3406d190 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8c500b5360 0x7f8c501e3630 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 --2- 172.21.15.7:0/1978649547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c500ca1b0 0x7f8c501e5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 >> 172.21.15.7:0/1978649547 conn(0x7f8c50072090 msgr2=0x7f8c5014da60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 shutdown_connections 2023-06-09T12:00:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:07.260+0000 7f8c581ff700 1 -- 172.21.15.7:0/1978649547 wait complete. 2023-06-09T12:00:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:00:07.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1550,"stamp":"2023-06-09T12:00:06.932881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88801,"ondisk_log_size":88801,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2551880,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":668544,"kb_avail":752422840,"statfs":{"total":773094113280,"available":770480988160,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":684589056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":79601,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002922"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3462,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094000+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:30.094000+0000","last_peered":"2023-06-09T11:59:30.094000+0000","last_clean":"2023-06-09T11:59:30.094000+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:30.094000+0000","last_undegraded":"2023-06-09T11:59:30.094000+0000","last_fullsized":"2023-06-09T11:59:30.094000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3218,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138170+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138170+0000","last_peered":"2023-06-09T11:59:29.138170+0000","last_clean":"2023-06-09T11:59:29.138170+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138170+0000","last_undegraded":"2023-06-09T11:59:29.138170+0000","last_fullsized":"2023-06-09T11:59:29.138170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186026+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.186026+0000","last_peered":"2023-06-09T11:59:29.186026+0000","last_clean":"2023-06-09T11:59:29.186026+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.186026+0000","last_undegraded":"2023-06-09T11:59:29.186026+0000","last_fullsized":"2023-06-09T11:59:29.186026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095399+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:30.095399+0000","last_peered":"2023-06-09T11:59:30.095399+0000","last_clean":"2023-06-09T11:59:30.095399+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:30.095399+0000","last_undegraded":"2023-06-09T11:59:30.095399+0000","last_fullsized":"2023-06-09T11:59:30.095399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179222+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179222+0000","last_peered":"2023-06-09T11:59:29.179222+0000","last_clean":"2023-06-09T11:59:29.179222+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179222+0000","last_undegraded":"2023-06-09T11:59:29.179222+0000","last_fullsized":"2023-06-09T11:59:29.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'864","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.109605+0000","last_change":"2023-06-09T11:18:26.314501+0000","last_active":"2023-06-09T11:59:30.109605+0000","last_peered":"2023-06-09T11:59:30.109605+0000","last_clean":"2023-06-09T11:59:30.109605+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T11:59:30.109605+0000","last_undegraded":"2023-06-09T11:59:30.109605+0000","last_fullsized":"2023-06-09T11:59:30.109605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":864,"log_dups_size":0,"ondisk_log_size":864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3089,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.135048+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.135048+0000","last_peered":"2023-06-09T11:59:29.135048+0000","last_clean":"2023-06-09T11:59:29.135048+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.135048+0000","last_undegraded":"2023-06-09T11:59:29.135048+0000","last_fullsized":"2023-06-09T11:59:29.135048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3244,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164360+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164360+0000","last_peered":"2023-06-09T11:59:29.164360+0000","last_clean":"2023-06-09T11:59:29.164360+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164360+0000","last_undegraded":"2023-06-09T11:59:29.164360+0000","last_fullsized":"2023-06-09T11:59:29.164360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163620+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163620+0000","last_peered":"2023-06-09T11:59:29.163620+0000","last_clean":"2023-06-09T11:59:29.163620+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163620+0000","last_undegraded":"2023-06-09T11:59:29.163620+0000","last_fullsized":"2023-06-09T11:59:29.163620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184811+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184811+0000","last_peered":"2023-06-09T11:59:29.184811+0000","last_clean":"2023-06-09T11:59:29.184811+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184811+0000","last_undegraded":"2023-06-09T11:59:29.184811+0000","last_fullsized":"2023-06-09T11:59:29.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3401,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821600+0000","last_change":"2023-06-09T11:59:49.712515+0000","last_active":"2023-06-09T11:59:42.821600+0000","last_peered":"2023-06-09T11:59:42.821600+0000","last_clean":"2023-06-09T11:59:42.821600+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:42.821600+0000","last_undegraded":"2023-06-09T11:59:42.821600+0000","last_fullsized":"2023-06-09T11:59:42.821600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3231,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821561+0000","last_change":"2023-06-09T11:59:48.700631+0000","last_active":"2023-06-09T11:59:42.821561+0000","last_peered":"2023-06-09T11:59:42.821561+0000","last_clean":"2023-06-09T11:59:42.821561+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:42.821561+0000","last_undegraded":"2023-06-09T11:59:42.821561+0000","last_fullsized":"2023-06-09T11:59:42.821561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3554,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.110992+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.110992+0000","last_peered":"2023-06-09T11:59:29.110992+0000","last_clean":"2023-06-09T11:59:29.110992+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.110992+0000","last_undegraded":"2023-06-09T11:59:29.110992+0000","last_fullsized":"2023-06-09T11:59:29.110992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105214+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:30.105214+0000","last_peered":"2023-06-09T11:59:30.105214+0000","last_clean":"2023-06-09T11:59:30.105214+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:30.105214+0000","last_undegraded":"2023-06-09T11:59:30.105214+0000","last_fullsized":"2023-06-09T11:59:30.105214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.107316+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:30.107316+0000","last_peered":"2023-06-09T11:59:30.107316+0000","last_clean":"2023-06-09T11:59:30.107316+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:30.107316+0000","last_undegraded":"2023-06-09T11:59:30.107316+0000","last_fullsized":"2023-06-09T11:59:30.107316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3480,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.148527+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:29.148527+0000","last_peered":"2023-06-09T11:59:29.148527+0000","last_clean":"2023-06-09T11:59:29.148527+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:29.148527+0000","last_undegraded":"2023-06-09T11:59:29.148527+0000","last_fullsized":"2023-06-09T11:59:29.148527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.206471+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:29.206471+0000","last_peered":"2023-06-09T11:59:29.206471+0000","last_clean":"2023-06-09T11:59:29.206471+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:29.206471+0000","last_undegraded":"2023-06-09T11:59:29.206471+0000","last_fullsized":"2023-06-09T11:59:29.206471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181720+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181720+0000","last_peered":"2023-06-09T11:59:29.181720+0000","last_clean":"2023-06-09T11:59:29.181720+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181720+0000","last_undegraded":"2023-06-09T11:59:29.181720+0000","last_fullsized":"2023-06-09T11:59:29.181720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3017,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108855+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.108855+0000","last_peered":"2023-06-09T11:59:30.108855+0000","last_clean":"2023-06-09T11:59:30.108855+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.108855+0000","last_undegraded":"2023-06-09T11:59:30.108855+0000","last_fullsized":"2023-06-09T11:59:30.108855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171290+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171290+0000","last_peered":"2023-06-09T11:59:29.171290+0000","last_clean":"2023-06-09T11:59:29.171290+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171290+0000","last_undegraded":"2023-06-09T11:59:29.171290+0000","last_fullsized":"2023-06-09T11:59:29.171290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116713+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116713+0000","last_peered":"2023-06-09T11:59:29.116713+0000","last_clean":"2023-06-09T11:59:29.116713+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116713+0000","last_undegraded":"2023-06-09T11:59:29.116713+0000","last_fullsized":"2023-06-09T11:59:29.116713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.470082+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:30.470082+0000","last_peered":"2023-06-09T11:59:30.470082+0000","last_clean":"2023-06-09T11:59:30.470082+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:30.470082+0000","last_undegraded":"2023-06-09T11:59:30.470082+0000","last_fullsized":"2023-06-09T11:59:30.470082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.177938+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:30.177938+0000","last_peered":"2023-06-09T11:59:30.177938+0000","last_clean":"2023-06-09T11:59:30.177938+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:30.177938+0000","last_undegraded":"2023-06-09T11:59:30.177938+0000","last_fullsized":"2023-06-09T11:59:30.177938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2241,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.087049+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:30.087049+0000","last_peered":"2023-06-09T11:59:30.087049+0000","last_clean":"2023-06-09T11:59:30.087049+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:30.087049+0000","last_undegraded":"2023-06-09T11:59:30.087049+0000","last_fullsized":"2023-06-09T11:59:30.087049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3134,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105820+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.105820+0000","last_peered":"2023-06-09T11:59:30.105820+0000","last_clean":"2023-06-09T11:59:30.105820+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.105820+0000","last_undegraded":"2023-06-09T11:59:30.105820+0000","last_fullsized":"2023-06-09T11:59:30.105820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466839+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:30.466839+0000","last_peered":"2023-06-09T11:59:30.466839+0000","last_clean":"2023-06-09T11:59:30.466839+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:30.466839+0000","last_undegraded":"2023-06-09T11:59:30.466839+0000","last_fullsized":"2023-06-09T11:59:30.466839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120625+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120625+0000","last_peered":"2023-06-09T11:59:29.120625+0000","last_clean":"2023-06-09T11:59:29.120625+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120625+0000","last_undegraded":"2023-06-09T11:59:29.120625+0000","last_fullsized":"2023-06-09T11:59:29.120625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":347,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:50.067505+0000","last_change":"2023-06-09T11:59:59.471359+0000","last_active":"2023-06-09T11:59:50.067505+0000","last_peered":"2023-06-09T11:59:50.067505+0000","last_clean":"2023-06-09T11:59:50.067505+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:50.067505+0000","last_undegraded":"2023-06-09T11:59:50.067505+0000","last_fullsized":"2023-06-09T11:59:50.067505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2962,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516988+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:43.516988+0000","last_peered":"2023-06-09T11:59:43.516988+0000","last_clean":"2023-06-09T11:59:43.516988+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:43.516988+0000","last_undegraded":"2023-06-09T11:59:43.516988+0000","last_fullsized":"2023-06-09T11:59:43.516988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814385+0000","last_change":"2023-06-09T11:10:49.676841+0000","last_active":"2023-06-09T11:59:17.814385+0000","last_peered":"2023-06-09T11:59:17.814385+0000","last_clean":"2023-06-09T11:59:17.814385+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T11:59:17.814385+0000","last_undegraded":"2023-06-09T11:59:17.814385+0000","last_fullsized":"2023-06-09T11:59:17.814385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.367732+0000","last_change":"2023-06-09T11:10:49.685979+0000","last_active":"2023-06-09T11:59:30.367732+0000","last_peered":"2023-06-09T11:59:30.367732+0000","last_clean":"2023-06-09T11:59:30.367732+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:30.367732+0000","last_undegraded":"2023-06-09T11:59:30.367732+0000","last_fullsized":"2023-06-09T11:59:30.367732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1034,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:49.057672+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:49.057672+0000","last_peered":"2023-06-09T11:59:49.057672+0000","last_clean":"2023-06-09T11:59:49.057672+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:49.057672+0000","last_undegraded":"2023-06-09T11:59:49.057672+0000","last_fullsized":"2023-06-09T11:59:49.057672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.390934+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:52.390934+0000","last_peered":"2023-06-09T11:59:52.390934+0000","last_clean":"2023-06-09T11:59:52.390934+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:52.390934+0000","last_undegraded":"2023-06-09T11:59:52.390934+0000","last_fullsized":"2023-06-09T11:59:52.390934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516984+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:43.516984+0000","last_peered":"2023-06-09T11:59:43.516984+0000","last_clean":"2023-06-09T11:59:43.516984+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:43.516984+0000","last_undegraded":"2023-06-09T11:59:43.516984+0000","last_fullsized":"2023-06-09T11:59:43.516984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3586,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116172+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116172+0000","last_peered":"2023-06-09T11:59:29.116172+0000","last_clean":"2023-06-09T11:59:29.116172+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116172+0000","last_undegraded":"2023-06-09T11:59:29.116172+0000","last_fullsized":"2023-06-09T11:59:29.116172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'961","reported_seq":1146,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296228+0000","last_change":"2023-06-09T11:10:49.679307+0000","last_active":"2023-06-09T11:59:46.296228+0000","last_peered":"2023-06-09T11:59:46.296228+0000","last_clean":"2023-06-09T11:59:46.296228+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T11:59:46.296228+0000","last_undegraded":"2023-06-09T11:59:46.296228+0000","last_fullsized":"2023-06-09T11:59:46.296228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.601385+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:45.601385+0000","last_peered":"2023-06-09T11:59:45.601385+0000","last_clean":"2023-06-09T11:59:45.601385+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:45.601385+0000","last_undegraded":"2023-06-09T11:59:45.601385+0000","last_fullsized":"2023-06-09T11:59:45.601385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3763,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203922+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203922+0000","last_peered":"2023-06-09T11:59:29.203922+0000","last_clean":"2023-06-09T11:59:29.203922+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203922+0000","last_undegraded":"2023-06-09T11:59:29.203922+0000","last_fullsized":"2023-06-09T11:59:29.203922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":726,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826142+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.826142+0000","last_peered":"2023-06-09T11:59:23.826142+0000","last_clean":"2023-06-09T11:59:23.826142+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.826142+0000","last_undegraded":"2023-06-09T11:59:23.826142+0000","last_fullsized":"2023-06-09T11:59:23.826142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2997,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113419+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113419+0000","last_peered":"2023-06-09T11:59:29.113419+0000","last_clean":"2023-06-09T11:59:29.113419+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113419+0000","last_undegraded":"2023-06-09T11:59:29.113419+0000","last_fullsized":"2023-06-09T11:59:29.113419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":689,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.250552+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:30.250552+0000","last_peered":"2023-06-09T11:59:30.250552+0000","last_clean":"2023-06-09T11:59:30.250552+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:30.250552+0000","last_undegraded":"2023-06-09T11:59:30.250552+0000","last_fullsized":"2023-06-09T11:59:30.250552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.106856+0000","last_change":"2023-06-09T11:18:26.315365+0000","last_active":"2023-06-09T11:59:30.106856+0000","last_peered":"2023-06-09T11:59:30.106856+0000","last_clean":"2023-06-09T11:59:30.106856+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:30.106856+0000","last_undegraded":"2023-06-09T11:59:30.106856+0000","last_fullsized":"2023-06-09T11:59:30.106856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":601,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.056497+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:34.056497+0000","last_peered":"2023-06-09T11:59:34.056497+0000","last_clean":"2023-06-09T11:59:34.056497+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:34.056497+0000","last_undegraded":"2023-06-09T11:59:34.056497+0000","last_fullsized":"2023-06-09T11:59:34.056497+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.106223+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.106223+0000","last_peered":"2023-06-09T11:59:30.106223+0000","last_clean":"2023-06-09T11:59:30.106223+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.106223+0000","last_undegraded":"2023-06-09T11:59:30.106223+0000","last_fullsized":"2023-06-09T11:59:30.106223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":689,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.055537+0000","last_change":"2023-06-09T11:10:49.673050+0000","last_active":"2023-06-09T11:59:34.055537+0000","last_peered":"2023-06-09T11:59:34.055537+0000","last_clean":"2023-06-09T11:59:34.055537+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T11:59:34.055537+0000","last_undegraded":"2023-06-09T11:59:34.055537+0000","last_fullsized":"2023-06-09T11:59:34.055537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.155241+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:29.155241+0000","last_peered":"2023-06-09T11:59:29.155241+0000","last_clean":"2023-06-09T11:59:29.155241+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:29.155241+0000","last_undegraded":"2023-06-09T11:59:29.155241+0000","last_fullsized":"2023-06-09T11:59:29.155241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.086324+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:30.086324+0000","last_peered":"2023-06-09T11:59:30.086324+0000","last_clean":"2023-06-09T11:59:30.086324+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:30.086324+0000","last_undegraded":"2023-06-09T11:59:30.086324+0000","last_fullsized":"2023-06-09T11:59:30.086324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3838,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.107081+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:30.107081+0000","last_peered":"2023-06-09T11:59:30.107081+0000","last_clean":"2023-06-09T11:59:30.107081+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:30.107081+0000","last_undegraded":"2023-06-09T11:59:30.107081+0000","last_fullsized":"2023-06-09T11:59:30.107081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1064","reported_seq":1244,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:49.933616+0000","last_change":"2023-06-09T11:59:49.561245+0000","last_active":"2023-06-09T11:59:49.933616+0000","last_peered":"2023-06-09T11:59:49.933616+0000","last_clean":"2023-06-09T11:59:49.933616+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T11:59:49.933616+0000","last_undegraded":"2023-06-09T11:59:49.933616+0000","last_fullsized":"2023-06-09T11:59:49.933616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1064,"log_dups_size":0,"ondisk_log_size":1064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2926,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.207821+0000","last_change":"2023-06-09T11:59:44.497062+0000","last_active":"2023-06-09T11:59:44.207821+0000","last_peered":"2023-06-09T11:59:44.207821+0000","last_clean":"2023-06-09T11:59:44.207821+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.207821+0000","last_undegraded":"2023-06-09T11:59:44.207821+0000","last_fullsized":"2023-06-09T11:59:44.207821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466208+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:30.466208+0000","last_peered":"2023-06-09T11:59:30.466208+0000","last_clean":"2023-06-09T11:59:30.466208+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:30.466208+0000","last_undegraded":"2023-06-09T11:59:30.466208+0000","last_fullsized":"2023-06-09T11:59:30.466208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185925+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:29.185925+0000","last_peered":"2023-06-09T11:59:29.185925+0000","last_clean":"2023-06-09T11:59:29.185925+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:29.185925+0000","last_undegraded":"2023-06-09T11:59:29.185925+0000","last_fullsized":"2023-06-09T11:59:29.185925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822272+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:23.822272+0000","last_peered":"2023-06-09T11:59:23.822272+0000","last_clean":"2023-06-09T11:59:23.822272+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:23.822272+0000","last_undegraded":"2023-06-09T11:59:23.822272+0000","last_fullsized":"2023-06-09T11:59:23.822272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3206,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.093008+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:30.093008+0000","last_peered":"2023-06-09T11:59:30.093008+0000","last_clean":"2023-06-09T11:59:30.093008+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:30.093008+0000","last_undegraded":"2023-06-09T11:59:30.093008+0000","last_fullsized":"2023-06-09T11:59:30.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525010+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525010+0000","last_peered":"2023-06-09T11:59:44.525010+0000","last_clean":"2023-06-09T11:59:44.525010+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525010+0000","last_undegraded":"2023-06-09T11:59:44.525010+0000","last_fullsized":"2023-06-09T11:59:44.525010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822281+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:23.822281+0000","last_peered":"2023-06-09T11:59:23.822281+0000","last_clean":"2023-06-09T11:59:23.822281+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:23.822281+0000","last_undegraded":"2023-06-09T11:59:23.822281+0000","last_fullsized":"2023-06-09T11:59:23.822281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198957+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198957+0000","last_peered":"2023-06-09T11:59:29.198957+0000","last_clean":"2023-06-09T11:59:29.198957+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198957+0000","last_undegraded":"2023-06-09T11:59:29.198957+0000","last_fullsized":"2023-06-09T11:59:29.198957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.813938+0000","last_change":"2023-06-09T11:10:43.559739+0000","last_active":"2023-06-09T11:59:17.813938+0000","last_peered":"2023-06-09T11:59:17.813938+0000","last_clean":"2023-06-09T11:59:17.813938+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:17.813938+0000","last_undegraded":"2023-06-09T11:59:17.813938+0000","last_fullsized":"2023-06-09T11:59:17.813938+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":896,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604134+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:29.604134+0000","last_peered":"2023-06-09T11:59:29.604134+0000","last_clean":"2023-06-09T11:59:29.604134+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:29.604134+0000","last_undegraded":"2023-06-09T11:59:29.604134+0000","last_fullsized":"2023-06-09T11:59:29.604134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3888,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203700+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203700+0000","last_peered":"2023-06-09T11:59:29.203700+0000","last_clean":"2023-06-09T11:59:29.203700+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203700+0000","last_undegraded":"2023-06-09T11:59:29.203700+0000","last_fullsized":"2023-06-09T11:59:29.203700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937169+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937169+0000","last_peered":"2023-06-09T11:59:45.937169+0000","last_clean":"2023-06-09T11:59:45.937169+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937169+0000","last_undegraded":"2023-06-09T11:59:45.937169+0000","last_fullsized":"2023-06-09T11:59:45.937169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822857+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.822857+0000","last_peered":"2023-06-09T11:59:23.822857+0000","last_clean":"2023-06-09T11:59:23.822857+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.822857+0000","last_undegraded":"2023-06-09T11:59:23.822857+0000","last_fullsized":"2023-06-09T11:59:23.822857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105068+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.105068+0000","last_peered":"2023-06-09T11:59:30.105068+0000","last_clean":"2023-06-09T11:59:30.105068+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.105068+0000","last_undegraded":"2023-06-09T11:59:30.105068+0000","last_fullsized":"2023-06-09T11:59:30.105068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.814078+0000","last_change":"2023-06-09T11:10:43.558511+0000","last_active":"2023-06-09T11:59:17.814078+0000","last_peered":"2023-06-09T11:59:17.814078+0000","last_clean":"2023-06-09T11:59:17.814078+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:17.814078+0000","last_undegraded":"2023-06-09T11:59:17.814078+0000","last_fullsized":"2023-06-09T11:59:17.814078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1140","reported_seq":1329,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:47.292638+0000","last_change":"2023-06-09T11:59:46.009973+0000","last_active":"2023-06-09T11:59:47.292638+0000","last_peered":"2023-06-09T11:59:47.292638+0000","last_clean":"2023-06-09T11:59:47.292638+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T11:59:47.292638+0000","last_undegraded":"2023-06-09T11:59:47.292638+0000","last_fullsized":"2023-06-09T11:59:47.292638+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1140,"log_dups_size":0,"ondisk_log_size":1140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142726+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142726+0000","last_peered":"2023-06-09T11:59:29.142726+0000","last_clean":"2023-06-09T11:59:29.142726+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142726+0000","last_undegraded":"2023-06-09T11:59:29.142726+0000","last_fullsized":"2023-06-09T11:59:29.142726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296161+0000","last_change":"2023-06-09T11:59:47.289624+0000","last_active":"2023-06-09T11:59:46.296161+0000","last_peered":"2023-06-09T11:59:46.296161+0000","last_clean":"2023-06-09T11:59:46.296161+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:46.296161+0000","last_undegraded":"2023-06-09T11:59:46.296161+0000","last_fullsized":"2023-06-09T11:59:46.296161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.467905+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.467905+0000","last_peered":"2023-06-09T11:59:30.467905+0000","last_clean":"2023-06-09T11:59:30.467905+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.467905+0000","last_undegraded":"2023-06-09T11:59:30.467905+0000","last_fullsized":"2023-06-09T11:59:30.467905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185984+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185984+0000","last_peered":"2023-06-09T11:59:29.185984+0000","last_clean":"2023-06-09T11:59:29.185984+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185984+0000","last_undegraded":"2023-06-09T11:59:29.185984+0000","last_fullsized":"2023-06-09T11:59:29.185984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:17.811073+0000","last_change":"2023-06-09T11:10:43.862203+0000","last_active":"2023-06-09T11:59:17.811073+0000","last_peered":"2023-06-09T11:59:17.811073+0000","last_clean":"2023-06-09T11:59:17.811073+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T11:59:17.811073+0000","last_undegraded":"2023-06-09T11:59:17.811073+0000","last_fullsized":"2023-06-09T11:59:17.811073+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":820,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823970+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823970+0000","last_peered":"2023-06-09T11:59:23.823970+0000","last_clean":"2023-06-09T11:59:23.823970+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823970+0000","last_undegraded":"2023-06-09T11:59:23.823970+0000","last_fullsized":"2023-06-09T11:59:23.823970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.303700+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:30.303700+0000","last_peered":"2023-06-09T11:59:30.303700+0000","last_clean":"2023-06-09T11:59:30.303700+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:30.303700+0000","last_undegraded":"2023-06-09T11:59:30.303700+0000","last_fullsized":"2023-06-09T11:59:30.303700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.251702+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.251702+0000","last_peered":"2023-06-09T11:59:30.251702+0000","last_clean":"2023-06-09T11:59:30.251702+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.251702+0000","last_undegraded":"2023-06-09T11:59:30.251702+0000","last_fullsized":"2023-06-09T11:59:30.251702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208432+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208432+0000","last_peered":"2023-06-09T11:59:29.208432+0000","last_clean":"2023-06-09T11:59:29.208432+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208432+0000","last_undegraded":"2023-06-09T11:59:29.208432+0000","last_fullsized":"2023-06-09T11:59:29.208432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601330+0000","last_change":"2023-06-09T11:59:48.603490+0000","last_active":"2023-06-09T11:59:45.601330+0000","last_peered":"2023-06-09T11:59:45.601330+0000","last_clean":"2023-06-09T11:59:45.601330+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:45.601330+0000","last_undegraded":"2023-06-09T11:59:45.601330+0000","last_fullsized":"2023-06-09T11:59:45.601330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":826,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.052133+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:34.052133+0000","last_peered":"2023-06-09T11:59:34.052133+0000","last_clean":"2023-06-09T11:59:34.052133+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:34.052133+0000","last_undegraded":"2023-06-09T11:59:34.052133+0000","last_fullsized":"2023-06-09T11:59:34.052133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3714,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170249+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170249+0000","last_peered":"2023-06-09T11:59:29.170249+0000","last_clean":"2023-06-09T11:59:29.170249+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170249+0000","last_undegraded":"2023-06-09T11:59:29.170249+0000","last_fullsized":"2023-06-09T11:59:29.170249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":880,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088435+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.088435+0000","last_peered":"2023-06-09T11:59:30.088435+0000","last_clean":"2023-06-09T11:59:30.088435+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.088435+0000","last_undegraded":"2023-06-09T11:59:30.088435+0000","last_fullsized":"2023-06-09T11:59:30.088435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1027","reported_seq":4599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:01.576654+0000","last_change":"2023-06-09T11:59:50.761964+0000","last_active":"2023-06-09T12:00:01.576654+0000","last_peered":"2023-06-09T12:00:01.576654+0000","last_clean":"2023-06-09T12:00:01.576654+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:01.576654+0000","last_undegraded":"2023-06-09T12:00:01.576654+0000","last_fullsized":"2023-06-09T12:00:01.576654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":72,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":766,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.247850+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.247850+0000","last_peered":"2023-06-09T11:59:30.247850+0000","last_clean":"2023-06-09T11:59:30.247850+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.247850+0000","last_undegraded":"2023-06-09T11:59:30.247850+0000","last_fullsized":"2023-06-09T11:59:30.247850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3559,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179988+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.179988+0000","last_peered":"2023-06-09T11:59:29.179988+0000","last_clean":"2023-06-09T11:59:29.179988+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.179988+0000","last_undegraded":"2023-06-09T11:59:29.179988+0000","last_fullsized":"2023-06-09T11:59:29.179988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'588","reported_seq":782,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:50.080648+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:50.080648+0000","last_peered":"2023-06-09T11:59:50.080648+0000","last_clean":"2023-06-09T11:59:50.080648+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:50.080648+0000","last_undegraded":"2023-06-09T11:59:50.080648+0000","last_fullsized":"2023-06-09T11:59:50.080648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3674,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.109212+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:30.109212+0000","last_peered":"2023-06-09T11:59:30.109212+0000","last_clean":"2023-06-09T11:59:30.109212+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:30.109212+0000","last_undegraded":"2023-06-09T11:59:30.109212+0000","last_fullsized":"2023-06-09T11:59:30.109212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1032,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.274861+0000","last_change":"2023-06-09T11:10:49.677572+0000","last_active":"2023-06-09T11:59:29.274861+0000","last_peered":"2023-06-09T11:59:29.274861+0000","last_clean":"2023-06-09T11:59:29.274861+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T11:59:29.274861+0000","last_undegraded":"2023-06-09T11:59:29.274861+0000","last_fullsized":"2023-06-09T11:59:29.274861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3148,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207251+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.207251+0000","last_peered":"2023-06-09T11:59:29.207251+0000","last_clean":"2023-06-09T11:59:29.207251+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.207251+0000","last_undegraded":"2023-06-09T11:59:29.207251+0000","last_fullsized":"2023-06-09T11:59:29.207251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"73'1383","reported_seq":1601,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.821320+0000","last_change":"2023-06-09T11:10:49.674323+0000","last_active":"2023-06-09T11:59:23.821320+0000","last_peered":"2023-06-09T11:59:23.821320+0000","last_clean":"2023-06-09T11:59:23.821320+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T11:59:23.821320+0000","last_undegraded":"2023-06-09T11:59:23.821320+0000","last_fullsized":"2023-06-09T11:59:23.821320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1383,"log_dups_size":0,"ondisk_log_size":1383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.364947+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:30.364947+0000","last_peered":"2023-06-09T11:59:30.364947+0000","last_clean":"2023-06-09T11:59:30.364947+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:30.364947+0000","last_undegraded":"2023-06-09T11:59:30.364947+0000","last_fullsized":"2023-06-09T11:59:30.364947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120737+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120737+0000","last_peered":"2023-06-09T11:59:29.120737+0000","last_clean":"2023-06-09T11:59:29.120737+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120737+0000","last_undegraded":"2023-06-09T11:59:29.120737+0000","last_fullsized":"2023-06-09T11:59:29.120737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.575027+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.575027+0000","last_peered":"2023-06-09T11:59:29.575027+0000","last_clean":"2023-06-09T11:59:29.575027+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.575027+0000","last_undegraded":"2023-06-09T11:59:29.575027+0000","last_fullsized":"2023-06-09T11:59:29.575027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187181+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187181+0000","last_peered":"2023-06-09T11:59:29.187181+0000","last_clean":"2023-06-09T11:59:29.187181+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187181+0000","last_undegraded":"2023-06-09T11:59:29.187181+0000","last_fullsized":"2023-06-09T11:59:29.187181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2978,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181067+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.181067+0000","last_peered":"2023-06-09T11:59:29.181067+0000","last_clean":"2023-06-09T11:59:29.181067+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.181067+0000","last_undegraded":"2023-06-09T11:59:29.181067+0000","last_fullsized":"2023-06-09T11:59:29.181067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207673+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207673+0000","last_peered":"2023-06-09T11:59:29.207673+0000","last_clean":"2023-06-09T11:59:29.207673+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207673+0000","last_undegraded":"2023-06-09T11:59:29.207673+0000","last_fullsized":"2023-06-09T11:59:29.207673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58157,"ondisk_log_size":58157,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30525,"ondisk_log_size":30525,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575443,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254224,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74368,"kb_avail":94117616,"statfs":{"total":96636764160,"available":96376438784,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76152832},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.72799999999999998}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.79600000000000004}]},{"osd":3,"last update":"Fri Jun 9 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.86899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.94199999999999995,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.54700000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.50800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575443,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.749},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":1.0169999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.877},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.97299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575443,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396080,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89664,"kb_avail":93975760,"statfs":{"total":96636764160,"available":96231178240,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91815936},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.502},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":0.873}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834575443,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327464,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99648,"kb_avail":94044376,"statfs":{"total":96636764160,"available":96301441024,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":102039552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.30199999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.63200000000000001,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.82699999999999996,"15min":0.85599999999999998},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.52600000000000002},"min":{"1min":0.40500000000000003,"5min":0.28999999999999998,"15min":0.28399999999999997},"max":{"1min":0.83399999999999996,"5min":0.89700000000000002,"15min":1.0900000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48899999999999999,"15min":0.47999999999999998},"min":{"1min":0.35199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56299999999999994,"15min":0.57599999999999996},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.308},"max":{"1min":0.871,"5min":0.871,"15min":1.2909999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.52600000000000002,"5min":0.46400000000000002,"15min":0.436},"max":{"1min":0.90600000000000003,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.73799999999999999}]},{"osd":4,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.432,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.96199999999999997,"5min":1.016,"15min":1.1060000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.47699999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97499999999999998,"5min":1.21,"15min":1.21},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.46100000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0429999999999999,"5min":1.05,"15min":1.236},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73599999999999999,"15min":0.746},"min":{"1min":0.53900000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.165},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85099999999999998,"15min":0.80300000000000005},"min":{"1min":0.53300000000000003,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":0.91700000000000004,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.83099999999999996,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0249999999999999,"5min":1.1599999999999999,"15min":1.3700000000000001},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Jun 9 11:59:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":0.995,"5min":1.159,"15min":1.228},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83299999999999996,"15min":0.79600000000000004},"min":{"1min":0.64200000000000002,"5min":0.52400000000000002,"15min":0.46999999999999997},"max":{"1min":1.0860000000000001,"5min":1.111,"15min":1.29},"last":0.83299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575444,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236704,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":56896,"kb_avail":94135136,"statfs":{"total":96636764160,"available":96394379264,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58261504},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44700000000000001,"15min":0.442},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.76300000000000001,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.59399999999999997,"15min":0.57799999999999996},"min":{"1min":0.48599999999999999,"5min":0.38800000000000001,"15min":0.36099999999999999},"max":{"1min":0.83699999999999997,"5min":1.367,"15min":1.367},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58299999999999996,"15min":0.57399999999999995},"min":{"1min":0.45000000000000001,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.82499999999999996,"5min":1.546,"15min":1.546},"last":0.496},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63300000000000001,"15min":0.59399999999999997},"min":{"1min":0.48899999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.84799999999999998,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.753,"15min":0.73799999999999999},"min":{"1min":0.58899999999999997,"5min":0.373,"15min":0.373},"max":{"1min":1.042,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.76600000000000001,"15min":0.751},"min":{"1min":0.55500000000000005,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.016,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80400000000000005,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.0109999999999999,"5min":1.601,"15min":1.601},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84799999999999998,"15min":0.82199999999999995},"min":{"1min":0.71599999999999997,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97199999999999998,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76700000000000002,"15min":0.747},"min":{"1min":0.58999999999999997,"5min":0.496,"15min":0.438},"max":{"1min":0.878,"5min":1.2090000000000001,"15min":1.3100000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.60199999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.99099999999999999,"5min":1.121,"15min":1.3540000000000001},"last":0.62}]},{"osd":7,"last update":"Fri Jun 9 11:59:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.78500000000000003,"15min":0.78100000000000003},"min":{"1min":0.66100000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.0189999999999999,"5min":1.208,"15min":1.248},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.872,"5min":0.82899999999999996,"15min":0.81000000000000005},"min":{"1min":0.73899999999999999,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.0900000000000001,"5min":1.3080000000000001,"15min":1.337},"last":0.747}]}]},{"osd":3,"up_from":13,"seq":55834575443,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48099999999999998,"15min":0.47999999999999998},"min":{"1min":0.34399999999999997,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.60799999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50600000000000001,"15min":0.52100000000000002},"min":{"1min":0.33800000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66200000000000003,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.497}]},{"osd":2,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.372},"max":{"1min":0.80600000000000005,"5min":1.355,"15min":1.355},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.379,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.76700000000000002,"5min":1.198,"15min":1.198},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94399999999999995,"5min":1.635,"15min":1.635},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.96399999999999997,"5min":2.351,"15min":2.351},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.746,"15min":0.73999999999999999},"min":{"1min":0.61899999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95399999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.78100000000000003,"15min":0.78200000000000003},"min":{"1min":0.64400000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":0.97599999999999998,"5min":1.367,"15min":1.367},"last":0.90700000000000003}]},{"osd":6,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.76800000000000002,"15min":0.78400000000000003},"min":{"1min":0.54600000000000004,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":1.286,"15min":1.286},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77400000000000002,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.434,"15min":0.434},"max":{"1min":0.98499999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.78200000000000003}]},{"osd":7,"last update":"Fri Jun 9 11:59:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.80700000000000005,"15min":0.81999999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.99399999999999999,"5min":1.413,"15min":1.413},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.85799999999999998,"15min":0.86699999999999999},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.45,"15min":1.45},"last":0.76400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575442,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413268,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120512,"kb_avail":93958572,"statfs":{"total":96636764160,"available":96213577728,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123404288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.81999999999999995,"5min":1.137,"15min":1.137},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74299999999999999,"15min":0.72499999999999998},"min":{"1min":0.51200000000000001,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.83299999999999996,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.41199999999999998,"15min":0.37},"max":{"1min":0.89700000000000002,"5min":1.0640000000000001,"15min":1.103},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.60199999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.96299999999999997,"5min":1.1279999999999999,"15min":1.2},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.628},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.77000000000000002,"5min":1.609,"15min":1.609},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.47999999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.871,"5min":1.159,"15min":1.1659999999999999},"last":0.84799999999999998}]},{"osd":3,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.67700000000000005},"min":{"1min":0.51300000000000001,"5min":0.44400000000000001,"15min":0.35899999999999999},"max":{"1min":0.94299999999999995,"5min":1.407,"15min":1.407},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.747,"15min":0.73299999999999998},"min":{"1min":0.64100000000000001,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.53600000000000003,"15min":0.52700000000000002},"min":{"1min":0.34599999999999997,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.41699999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.0620000000000001,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.55800000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.86699999999999999,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.51500000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.98499999999999999,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:02 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.47099999999999997,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.501},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.66100000000000003,"15min":0.64600000000000002},"min":{"1min":0.54200000000000004,"5min":0.39800000000000002,"15min":0.36399999999999999},"max":{"1min":1.046,"5min":1.361,"15min":1.3740000000000001},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575442,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288024,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70656,"kb_avail":94083816,"statfs":{"total":96636764160,"available":96341827584,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72351744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":0.745},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.86899999999999999}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.78300000000000003},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:00:07.350 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.350 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.351 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.351 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.351 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.352 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.352 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.352 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.352 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.353 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.353 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.353 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.354 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.354 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.354 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.355 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.355 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.355 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.355 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.356 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.356 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.356 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.357 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.357 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.357 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.358 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.358 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.358 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.358 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.359 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.359 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.359 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.360 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.360 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.360 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.360 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.361 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.361 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.361 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.362 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.362 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.362 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.362 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.363 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.363 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.363 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.364 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.364 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.364 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.365 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.365 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.365 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.365 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.366 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.366 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.366 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.367 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.367 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.367 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.367 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.368 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.368 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.368 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.369 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.369 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.369 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.370 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.370 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.370 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.370 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.371 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.371 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.371 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.372 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.372 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.372 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.373 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.373 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.373 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.373 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.374 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.374 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.374 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.375 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.375 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.375 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.376 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.376 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.376 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.376 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.377 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.377 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.377 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.378 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.378 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.378 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.378 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.379 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.379 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.379 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:07.380 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:00:25.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:25.816 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:27.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:00:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.589+0000 7f32b57ff700 1 Processor -- start 2023-06-09T12:00:27.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.589+0000 7f32b57ff700 1 -- start start 2023-06-09T12:00:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.590+0000 7f32b57ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32b0151510 0x7f32b00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.590+0000 7f32b57ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 0x7f32b0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.590+0000 7f32b57ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b01580a0 con 0x7f32b0071060 2023-06-09T12:00:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.590+0000 7f32b57ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b00714e0 con 0x7f32b0151510 2023-06-09T12:00:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.590+0000 7f32b57ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01518e0 con 0x7f32b015b380 2023-06-09T12:00:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32b0151510 0x7f32b00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f32b0151510 0x7f32b00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43554/0 (socket says 172.21.15.7:43554) 2023-06-09T12:00:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 -- 172.21.15.7:0/155146480 learned_addr learned my addr 172.21.15.7:0/155146480 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 -- 172.21.15.7:0/155146480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 msgr2=0x7f32b0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:00:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32ae7fc700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 0x7f32b0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 0x7f32b0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 -- 172.21.15.7:0/155146480 >> v1:172.21.15.117:6789/0 conn(0x7f32b0071060 legacy=0x7f32b0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:00:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 -- 172.21.15.7:0/155146480 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00c6180 con 0x7f32b0151510 2023-06-09T12:00:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32ae7fc700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 0x7f32b0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.591+0000 7f32aeffd700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0151510 0x7f32b00c5b40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f32a00098f0 tx=0x7f32a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ff4c1b1fc18edc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32adffb700 1 -- 172.21.15.7:0/155146480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0004230 con 0x7f32b0151510 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32adffb700 1 -- 172.21.15.7:0/155146480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a0004390 con 0x7f32b0151510 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32adffb700 1 -- 172.21.15.7:0/155146480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a00036a0 con 0x7f32b0151510 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 -- 172.21.15.7:0/155146480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0151510 msgr2=0x7f32b00c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0151510 0x7f32b00c5b40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f32a00098f0 tx=0x7f32a0009c00 comp rx=0 tx=0).stop 2023-06-09T12:00:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 -- 172.21.15.7:0/155146480 shutdown_connections 2023-06-09T12:00:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0151510 0x7f32b00c5b40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 --2- 172.21.15.7:0/155146480 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b015b380 0x7f32b0070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 -- 172.21.15.7:0/155146480 >> 172.21.15.7:0/155146480 conn(0x7f32b006a4d0 msgr2=0x7f32b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 -- 172.21.15.7:0/155146480 shutdown_connections 2023-06-09T12:00:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.592+0000 7f32b57ff700 1 -- 172.21.15.7:0/155146480 wait complete. 2023-06-09T12:00:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.594+0000 7f32b57ff700 1 Processor -- start 2023-06-09T12:00:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.594+0000 7f32b57ff700 1 -- start start 2023-06-09T12:00:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b0151510 0x7f32b01456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32b015b380 0x7f32b00c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00c0650 con 0x7f32b0071060 2023-06-09T12:00:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b0145fc0 con 0x7f32b015b380 2023-06-09T12:00:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32b57ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b01462a0 con 0x7f32b0151510 2023-06-09T12:00:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32aeffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43562/0 (socket says 172.21.15.7:43562) 2023-06-09T12:00:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32aeffd700 1 -- 172.21.15.7:0/3213955457 learned_addr learned my addr 172.21.15.7:0/3213955457 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32af7fe700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32b015b380 0x7f32b00c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.595+0000 7f32ae7fc700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b0151510 0x7f32b01456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 -- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b0151510 msgr2=0x7f32b01456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b0151510 0x7f32b01456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 -- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32b015b380 msgr2=0x7f32b00c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32b015b380 0x7f32b00c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 -- 172.21.15.7:0/3213955457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a0009550 con 0x7f32b0071060 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32aeffd700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009130 tx=0x7f32a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0003cd0 con 0x7f32b0071060 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a001f070 con 0x7f32b0071060 2023-06-09T12:00:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a000b8a0 con 0x7f32b0071060 2023-06-09T12:00:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.596+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b0157770 con 0x7f32b0071060 2023-06-09T12:00:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.597+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b0157490 con 0x7f32b0071060 2023-06-09T12:00:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.598+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b0157a50 con 0x7f32b0071060 2023-06-09T12:00:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.598+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f32a0021030 con 0x7f32b0071060 2023-06-09T12:00:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.599+0000 7f32977fe700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 0x7f329c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.599+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f32a008b3f0 con 0x7f32b0071060 2023-06-09T12:00:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.600+0000 7f32ae7fc700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 0x7f329c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.600+0000 7f32ae7fc700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 0x7f329c06d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f32b01eff20 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.604+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f32a0055b20 con 0x7f32b0071060 2023-06-09T12:00:27.816 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.813+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32b00c0790 con 0x7f329c06ace0 2023-06-09T12:00:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.822+0000 7f32977fe700 1 -- 172.21.15.7:0/3213955457 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+312984 (secure 0 0 0) 0x7f32a400c040 con 0x7f329c06ace0 2023-06-09T12:00:27.829 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:00:27.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 msgr2=0x7f329c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:27.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 0x7f329c06d190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f32b01eff20 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2023-06-09T12:00:27.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 msgr2=0x7f32b00cb940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:27.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009130 tx=0x7f32a0004750 comp rx=0 tx=0).stop 2023-06-09T12:00:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 shutdown_connections 2023-06-09T12:00:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f32b015b380 0x7f32b00c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b0071060 0x7f32b00cb940 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f329c06ace0 0x7f329c06d190 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 --2- 172.21.15.7:0/3213955457 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f32b0151510 0x7f32b01456a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 >> 172.21.15.7:0/3213955457 conn(0x7f32b006a4d0 msgr2=0x7f32b00c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 shutdown_connections 2023-06-09T12:00:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:27.827+0000 7f32b57ff700 1 -- 172.21.15.7:0/3213955457 wait complete. 2023-06-09T12:00:27.833 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:00:27.849 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1560,"stamp":"2023-06-09T12:00:26.938491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":316498,"num_omap_keys":584,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88978,"ondisk_log_size":88978,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2552200,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":668864,"kb_avail":752422520,"statfs":{"total":773094113280,"available":770480660480,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":684916736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92403,"num_omap_keys":175,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003403"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3462,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094000+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:30.094000+0000","last_peered":"2023-06-09T11:59:30.094000+0000","last_clean":"2023-06-09T11:59:30.094000+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:30.094000+0000","last_undegraded":"2023-06-09T11:59:30.094000+0000","last_fullsized":"2023-06-09T11:59:30.094000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3218,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138170+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138170+0000","last_peered":"2023-06-09T11:59:29.138170+0000","last_clean":"2023-06-09T11:59:29.138170+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138170+0000","last_undegraded":"2023-06-09T11:59:29.138170+0000","last_fullsized":"2023-06-09T11:59:29.138170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186026+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.186026+0000","last_peered":"2023-06-09T11:59:29.186026+0000","last_clean":"2023-06-09T11:59:29.186026+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.186026+0000","last_undegraded":"2023-06-09T11:59:29.186026+0000","last_fullsized":"2023-06-09T11:59:29.186026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.095399+0000","last_change":"2023-06-09T11:18:26.644882+0000","last_active":"2023-06-09T11:59:30.095399+0000","last_peered":"2023-06-09T11:59:30.095399+0000","last_clean":"2023-06-09T11:59:30.095399+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:30.095399+0000","last_undegraded":"2023-06-09T11:59:30.095399+0000","last_fullsized":"2023-06-09T11:59:30.095399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179222+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179222+0000","last_peered":"2023-06-09T11:59:29.179222+0000","last_clean":"2023-06-09T11:59:29.179222+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179222+0000","last_undegraded":"2023-06-09T11:59:29.179222+0000","last_fullsized":"2023-06-09T11:59:29.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'888","reported_seq":3371,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:20.530765+0000","last_change":"2023-06-09T12:00:18.653707+0000","last_active":"2023-06-09T12:00:20.530765+0000","last_peered":"2023-06-09T12:00:20.530765+0000","last_clean":"2023-06-09T12:00:20.530765+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:20.530765+0000","last_undegraded":"2023-06-09T12:00:20.530765+0000","last_fullsized":"2023-06-09T12:00:20.530765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3089,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.135048+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.135048+0000","last_peered":"2023-06-09T11:59:29.135048+0000","last_clean":"2023-06-09T11:59:29.135048+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.135048+0000","last_undegraded":"2023-06-09T11:59:29.135048+0000","last_fullsized":"2023-06-09T11:59:29.135048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3244,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164360+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164360+0000","last_peered":"2023-06-09T11:59:29.164360+0000","last_clean":"2023-06-09T11:59:29.164360+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164360+0000","last_undegraded":"2023-06-09T11:59:29.164360+0000","last_fullsized":"2023-06-09T11:59:29.164360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163620+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163620+0000","last_peered":"2023-06-09T11:59:29.163620+0000","last_clean":"2023-06-09T11:59:29.163620+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163620+0000","last_undegraded":"2023-06-09T11:59:29.163620+0000","last_fullsized":"2023-06-09T11:59:29.163620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184811+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184811+0000","last_peered":"2023-06-09T11:59:29.184811+0000","last_clean":"2023-06-09T11:59:29.184811+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184811+0000","last_undegraded":"2023-06-09T11:59:29.184811+0000","last_fullsized":"2023-06-09T11:59:29.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3401,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821600+0000","last_change":"2023-06-09T11:59:49.712515+0000","last_active":"2023-06-09T11:59:42.821600+0000","last_peered":"2023-06-09T11:59:42.821600+0000","last_clean":"2023-06-09T11:59:42.821600+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:42.821600+0000","last_undegraded":"2023-06-09T11:59:42.821600+0000","last_fullsized":"2023-06-09T11:59:42.821600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3231,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821561+0000","last_change":"2023-06-09T11:59:48.700631+0000","last_active":"2023-06-09T11:59:42.821561+0000","last_peered":"2023-06-09T11:59:42.821561+0000","last_clean":"2023-06-09T11:59:42.821561+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:42.821561+0000","last_undegraded":"2023-06-09T11:59:42.821561+0000","last_fullsized":"2023-06-09T11:59:42.821561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3554,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.110992+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:29.110992+0000","last_peered":"2023-06-09T11:59:29.110992+0000","last_clean":"2023-06-09T11:59:29.110992+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:29.110992+0000","last_undegraded":"2023-06-09T11:59:29.110992+0000","last_fullsized":"2023-06-09T11:59:29.110992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296222+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:46.296222+0000","last_peered":"2023-06-09T11:59:46.296222+0000","last_clean":"2023-06-09T11:59:46.296222+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:46.296222+0000","last_undegraded":"2023-06-09T11:59:46.296222+0000","last_fullsized":"2023-06-09T11:59:46.296222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423589+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:41.423589+0000","last_peered":"2023-06-09T11:59:41.423589+0000","last_clean":"2023-06-09T11:59:41.423589+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:41.423589+0000","last_undegraded":"2023-06-09T11:59:41.423589+0000","last_fullsized":"2023-06-09T11:59:41.423589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517081+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:43.517081+0000","last_peered":"2023-06-09T11:59:43.517081+0000","last_clean":"2023-06-09T11:59:43.517081+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:43.517081+0000","last_undegraded":"2023-06-09T11:59:43.517081+0000","last_fullsized":"2023-06-09T11:59:43.517081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181720+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181720+0000","last_peered":"2023-06-09T11:59:29.181720+0000","last_clean":"2023-06-09T11:59:29.181720+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181720+0000","last_undegraded":"2023-06-09T11:59:29.181720+0000","last_fullsized":"2023-06-09T11:59:29.181720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3017,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.108855+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:30.108855+0000","last_peered":"2023-06-09T11:59:30.108855+0000","last_clean":"2023-06-09T11:59:30.108855+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:30.108855+0000","last_undegraded":"2023-06-09T11:59:30.108855+0000","last_fullsized":"2023-06-09T11:59:30.108855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.171290+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:29.171290+0000","last_peered":"2023-06-09T11:59:29.171290+0000","last_clean":"2023-06-09T11:59:29.171290+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:29.171290+0000","last_undegraded":"2023-06-09T11:59:29.171290+0000","last_fullsized":"2023-06-09T11:59:29.171290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116713+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:29.116713+0000","last_peered":"2023-06-09T11:59:29.116713+0000","last_clean":"2023-06-09T11:59:29.116713+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:29.116713+0000","last_undegraded":"2023-06-09T11:59:29.116713+0000","last_fullsized":"2023-06-09T11:59:29.116713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.470082+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:30.470082+0000","last_peered":"2023-06-09T11:59:30.470082+0000","last_clean":"2023-06-09T11:59:30.470082+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:30.470082+0000","last_undegraded":"2023-06-09T11:59:30.470082+0000","last_fullsized":"2023-06-09T11:59:30.470082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.177938+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:30.177938+0000","last_peered":"2023-06-09T11:59:30.177938+0000","last_clean":"2023-06-09T11:59:30.177938+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:30.177938+0000","last_undegraded":"2023-06-09T11:59:30.177938+0000","last_fullsized":"2023-06-09T11:59:30.177938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2241,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.087049+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:30.087049+0000","last_peered":"2023-06-09T11:59:30.087049+0000","last_clean":"2023-06-09T11:59:30.087049+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:30.087049+0000","last_undegraded":"2023-06-09T11:59:30.087049+0000","last_fullsized":"2023-06-09T11:59:30.087049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3134,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105820+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.105820+0000","last_peered":"2023-06-09T11:59:30.105820+0000","last_clean":"2023-06-09T11:59:30.105820+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.105820+0000","last_undegraded":"2023-06-09T11:59:30.105820+0000","last_fullsized":"2023-06-09T11:59:30.105820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466839+0000","last_change":"2023-06-09T11:10:49.677403+0000","last_active":"2023-06-09T11:59:30.466839+0000","last_peered":"2023-06-09T11:59:30.466839+0000","last_clean":"2023-06-09T11:59:30.466839+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:30.466839+0000","last_undegraded":"2023-06-09T11:59:30.466839+0000","last_fullsized":"2023-06-09T11:59:30.466839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120625+0000","last_change":"2023-06-09T11:18:26.644824+0000","last_active":"2023-06-09T11:59:29.120625+0000","last_peered":"2023-06-09T11:59:29.120625+0000","last_clean":"2023-06-09T11:59:29.120625+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:29.120625+0000","last_undegraded":"2023-06-09T11:59:29.120625+0000","last_fullsized":"2023-06-09T11:59:29.120625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":347,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:50.067505+0000","last_change":"2023-06-09T11:59:59.471359+0000","last_active":"2023-06-09T11:59:50.067505+0000","last_peered":"2023-06-09T11:59:50.067505+0000","last_clean":"2023-06-09T11:59:50.067505+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:50.067505+0000","last_undegraded":"2023-06-09T11:59:50.067505+0000","last_fullsized":"2023-06-09T11:59:50.067505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2962,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516988+0000","last_change":"2023-06-09T11:18:26.280277+0000","last_active":"2023-06-09T11:59:43.516988+0000","last_peered":"2023-06-09T11:59:43.516988+0000","last_clean":"2023-06-09T11:59:43.516988+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:43.516988+0000","last_undegraded":"2023-06-09T11:59:43.516988+0000","last_fullsized":"2023-06-09T11:59:43.516988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1541,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:20.643819+0000","last_change":"2023-06-09T12:00:20.271439+0000","last_active":"2023-06-09T12:00:20.643819+0000","last_peered":"2023-06-09T12:00:20.643819+0000","last_clean":"2023-06-09T12:00:20.643819+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:20.643819+0000","last_undegraded":"2023-06-09T12:00:20.643819+0000","last_fullsized":"2023-06-09T12:00:20.643819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1264,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296204+0000","last_change":"2023-06-09T12:00:19.292684+0000","last_active":"2023-06-09T11:59:46.296204+0000","last_peered":"2023-06-09T11:59:46.296204+0000","last_clean":"2023-06-09T11:59:46.296204+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:46.296204+0000","last_undegraded":"2023-06-09T11:59:46.296204+0000","last_fullsized":"2023-06-09T11:59:46.296204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1034,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:49.057672+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:49.057672+0000","last_peered":"2023-06-09T11:59:49.057672+0000","last_clean":"2023-06-09T11:59:49.057672+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:49.057672+0000","last_undegraded":"2023-06-09T11:59:49.057672+0000","last_fullsized":"2023-06-09T11:59:49.057672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.390934+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:52.390934+0000","last_peered":"2023-06-09T11:59:52.390934+0000","last_clean":"2023-06-09T11:59:52.390934+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:52.390934+0000","last_undegraded":"2023-06-09T11:59:52.390934+0000","last_fullsized":"2023-06-09T11:59:52.390934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3558,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516984+0000","last_change":"2023-06-09T11:18:26.302811+0000","last_active":"2023-06-09T11:59:43.516984+0000","last_peered":"2023-06-09T11:59:43.516984+0000","last_clean":"2023-06-09T11:59:43.516984+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T11:59:43.516984+0000","last_undegraded":"2023-06-09T11:59:43.516984+0000","last_fullsized":"2023-06-09T11:59:43.516984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3586,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116172+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116172+0000","last_peered":"2023-06-09T11:59:29.116172+0000","last_clean":"2023-06-09T11:59:29.116172+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116172+0000","last_undegraded":"2023-06-09T11:59:29.116172+0000","last_fullsized":"2023-06-09T11:59:29.116172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.601385+0000","last_change":"2023-06-09T11:10:49.677261+0000","last_active":"2023-06-09T11:59:45.601385+0000","last_peered":"2023-06-09T11:59:45.601385+0000","last_clean":"2023-06-09T11:59:45.601385+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T11:59:45.601385+0000","last_undegraded":"2023-06-09T11:59:45.601385+0000","last_fullsized":"2023-06-09T11:59:45.601385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3763,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203922+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203922+0000","last_peered":"2023-06-09T11:59:29.203922+0000","last_clean":"2023-06-09T11:59:29.203922+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203922+0000","last_undegraded":"2023-06-09T11:59:29.203922+0000","last_fullsized":"2023-06-09T11:59:29.203922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":726,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.826142+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:23.826142+0000","last_peered":"2023-06-09T11:59:23.826142+0000","last_clean":"2023-06-09T11:59:23.826142+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:23.826142+0000","last_undegraded":"2023-06-09T11:59:23.826142+0000","last_fullsized":"2023-06-09T11:59:23.826142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2997,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.113419+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:29.113419+0000","last_peered":"2023-06-09T11:59:29.113419+0000","last_clean":"2023-06-09T11:59:29.113419+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:29.113419+0000","last_undegraded":"2023-06-09T11:59:29.113419+0000","last_fullsized":"2023-06-09T11:59:29.113419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296113+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:46.296113+0000","last_peered":"2023-06-09T11:59:46.296113+0000","last_clean":"2023-06-09T11:59:46.296113+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:46.296113+0000","last_undegraded":"2023-06-09T11:59:46.296113+0000","last_fullsized":"2023-06-09T11:59:46.296113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3650,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.207890+0000","last_change":"2023-06-09T12:00:04.594865+0000","last_active":"2023-06-09T11:59:44.207890+0000","last_peered":"2023-06-09T11:59:44.207890+0000","last_clean":"2023-06-09T11:59:44.207890+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T11:59:44.207890+0000","last_undegraded":"2023-06-09T11:59:44.207890+0000","last_fullsized":"2023-06-09T11:59:44.207890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296114+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:46.296114+0000","last_peered":"2023-06-09T11:59:46.296114+0000","last_clean":"2023-06-09T11:59:46.296114+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:46.296114+0000","last_undegraded":"2023-06-09T11:59:46.296114+0000","last_fullsized":"2023-06-09T11:59:46.296114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.106223+0000","last_change":"2023-06-09T11:18:26.645218+0000","last_active":"2023-06-09T11:59:30.106223+0000","last_peered":"2023-06-09T11:59:30.106223+0000","last_clean":"2023-06-09T11:59:30.106223+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:30.106223+0000","last_undegraded":"2023-06-09T11:59:30.106223+0000","last_fullsized":"2023-06-09T11:59:30.106223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516969+0000","last_change":"2023-06-09T11:10:49.675332+0000","last_active":"2023-06-09T11:59:43.516969+0000","last_peered":"2023-06-09T11:59:43.516969+0000","last_clean":"2023-06-09T11:59:43.516969+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T11:59:43.516969+0000","last_undegraded":"2023-06-09T11:59:43.516969+0000","last_fullsized":"2023-06-09T11:59:43.516969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445722+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445722+0000","last_peered":"2023-06-09T12:00:04.445722+0000","last_clean":"2023-06-09T12:00:04.445722+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445722+0000","last_undegraded":"2023-06-09T12:00:04.445722+0000","last_fullsized":"2023-06-09T12:00:04.445722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.466208+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:30.466208+0000","last_peered":"2023-06-09T11:59:30.466208+0000","last_clean":"2023-06-09T11:59:30.466208+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:30.466208+0000","last_undegraded":"2023-06-09T11:59:30.466208+0000","last_fullsized":"2023-06-09T11:59:30.466208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822272+0000","last_change":"2023-06-09T11:10:49.683492+0000","last_active":"2023-06-09T11:59:23.822272+0000","last_peered":"2023-06-09T11:59:23.822272+0000","last_clean":"2023-06-09T11:59:23.822272+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T11:59:23.822272+0000","last_undegraded":"2023-06-09T11:59:23.822272+0000","last_fullsized":"2023-06-09T11:59:23.822272+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3206,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.093008+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:30.093008+0000","last_peered":"2023-06-09T11:59:30.093008+0000","last_clean":"2023-06-09T11:59:30.093008+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:30.093008+0000","last_undegraded":"2023-06-09T11:59:30.093008+0000","last_fullsized":"2023-06-09T11:59:30.093008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525010+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525010+0000","last_peered":"2023-06-09T11:59:44.525010+0000","last_clean":"2023-06-09T11:59:44.525010+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525010+0000","last_undegraded":"2023-06-09T11:59:44.525010+0000","last_fullsized":"2023-06-09T11:59:44.525010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822281+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:23.822281+0000","last_peered":"2023-06-09T11:59:23.822281+0000","last_clean":"2023-06-09T11:59:23.822281+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:23.822281+0000","last_undegraded":"2023-06-09T11:59:23.822281+0000","last_fullsized":"2023-06-09T11:59:23.822281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198957+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198957+0000","last_peered":"2023-06-09T11:59:29.198957+0000","last_clean":"2023-06-09T11:59:29.198957+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198957+0000","last_undegraded":"2023-06-09T11:59:29.198957+0000","last_fullsized":"2023-06-09T11:59:29.198957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":154,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905097+0000","last_change":"2023-06-09T12:00:17.246824+0000","last_active":"2023-06-09T11:59:44.905097+0000","last_peered":"2023-06-09T11:59:44.905097+0000","last_clean":"2023-06-09T11:59:44.905097+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:44.905097+0000","last_undegraded":"2023-06-09T11:59:44.905097+0000","last_fullsized":"2023-06-09T11:59:44.905097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":896,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604134+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:29.604134+0000","last_peered":"2023-06-09T11:59:29.604134+0000","last_clean":"2023-06-09T11:59:29.604134+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:29.604134+0000","last_undegraded":"2023-06-09T11:59:29.604134+0000","last_fullsized":"2023-06-09T11:59:29.604134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3888,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203700+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203700+0000","last_peered":"2023-06-09T11:59:29.203700+0000","last_clean":"2023-06-09T11:59:29.203700+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203700+0000","last_undegraded":"2023-06-09T11:59:29.203700+0000","last_fullsized":"2023-06-09T11:59:29.203700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937169+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937169+0000","last_peered":"2023-06-09T11:59:45.937169+0000","last_clean":"2023-06-09T11:59:45.937169+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937169+0000","last_undegraded":"2023-06-09T11:59:45.937169+0000","last_fullsized":"2023-06-09T11:59:45.937169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822857+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.822857+0000","last_peered":"2023-06-09T11:59:23.822857+0000","last_clean":"2023-06-09T11:59:23.822857+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.822857+0000","last_undegraded":"2023-06-09T11:59:23.822857+0000","last_fullsized":"2023-06-09T11:59:23.822857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105068+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:30.105068+0000","last_peered":"2023-06-09T11:59:30.105068+0000","last_clean":"2023-06-09T11:59:30.105068+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:30.105068+0000","last_undegraded":"2023-06-09T11:59:30.105068+0000","last_fullsized":"2023-06-09T11:59:30.105068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":154,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905109+0000","last_change":"2023-06-09T12:00:15.338006+0000","last_active":"2023-06-09T11:59:44.905109+0000","last_peered":"2023-06-09T11:59:44.905109+0000","last_clean":"2023-06-09T11:59:44.905109+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:44.905109+0000","last_undegraded":"2023-06-09T11:59:44.905109+0000","last_fullsized":"2023-06-09T11:59:44.905109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1347,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602275+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602275+0000","last_peered":"2023-06-09T12:00:03.602275+0000","last_clean":"2023-06-09T12:00:03.602275+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602275+0000","last_undegraded":"2023-06-09T12:00:03.602275+0000","last_fullsized":"2023-06-09T12:00:03.602275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3296,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.142726+0000","last_change":"2023-06-09T11:18:26.644599+0000","last_active":"2023-06-09T11:59:29.142726+0000","last_peered":"2023-06-09T11:59:29.142726+0000","last_clean":"2023-06-09T11:59:29.142726+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:29.142726+0000","last_undegraded":"2023-06-09T11:59:29.142726+0000","last_fullsized":"2023-06-09T11:59:29.142726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296161+0000","last_change":"2023-06-09T11:59:47.289624+0000","last_active":"2023-06-09T11:59:46.296161+0000","last_peered":"2023-06-09T11:59:46.296161+0000","last_clean":"2023-06-09T11:59:46.296161+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:46.296161+0000","last_undegraded":"2023-06-09T11:59:46.296161+0000","last_fullsized":"2023-06-09T11:59:46.296161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.467905+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.467905+0000","last_peered":"2023-06-09T11:59:30.467905+0000","last_clean":"2023-06-09T11:59:30.467905+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.467905+0000","last_undegraded":"2023-06-09T11:59:30.467905+0000","last_fullsized":"2023-06-09T11:59:30.467905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185984+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:29.185984+0000","last_peered":"2023-06-09T11:59:29.185984+0000","last_clean":"2023-06-09T11:59:29.185984+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:29.185984+0000","last_undegraded":"2023-06-09T11:59:29.185984+0000","last_fullsized":"2023-06-09T11:59:29.185984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637615+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637615+0000","last_peered":"2023-06-09T12:00:04.637615+0000","last_clean":"2023-06-09T12:00:04.637615+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637615+0000","last_undegraded":"2023-06-09T12:00:04.637615+0000","last_fullsized":"2023-06-09T12:00:04.637615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":820,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823970+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823970+0000","last_peered":"2023-06-09T11:59:23.823970+0000","last_clean":"2023-06-09T11:59:23.823970+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823970+0000","last_undegraded":"2023-06-09T11:59:23.823970+0000","last_fullsized":"2023-06-09T11:59:23.823970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.303700+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:30.303700+0000","last_peered":"2023-06-09T11:59:30.303700+0000","last_clean":"2023-06-09T11:59:30.303700+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:30.303700+0000","last_undegraded":"2023-06-09T11:59:30.303700+0000","last_fullsized":"2023-06-09T11:59:30.303700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.251702+0000","last_change":"2023-06-09T11:10:49.682304+0000","last_active":"2023-06-09T11:59:30.251702+0000","last_peered":"2023-06-09T11:59:30.251702+0000","last_clean":"2023-06-09T11:59:30.251702+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T11:59:30.251702+0000","last_undegraded":"2023-06-09T11:59:30.251702+0000","last_fullsized":"2023-06-09T11:59:30.251702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.208432+0000","last_change":"2023-06-09T11:18:26.268918+0000","last_active":"2023-06-09T11:59:29.208432+0000","last_peered":"2023-06-09T11:59:29.208432+0000","last_clean":"2023-06-09T11:59:29.208432+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:29.208432+0000","last_undegraded":"2023-06-09T11:59:29.208432+0000","last_fullsized":"2023-06-09T11:59:29.208432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601330+0000","last_change":"2023-06-09T11:59:48.603490+0000","last_active":"2023-06-09T11:59:45.601330+0000","last_peered":"2023-06-09T11:59:45.601330+0000","last_clean":"2023-06-09T11:59:45.601330+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T11:59:45.601330+0000","last_undegraded":"2023-06-09T11:59:45.601330+0000","last_fullsized":"2023-06-09T11:59:45.601330+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":826,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:34.052133+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:34.052133+0000","last_peered":"2023-06-09T11:59:34.052133+0000","last_clean":"2023-06-09T11:59:34.052133+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:34.052133+0000","last_undegraded":"2023-06-09T11:59:34.052133+0000","last_fullsized":"2023-06-09T11:59:34.052133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3714,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170249+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170249+0000","last_peered":"2023-06-09T11:59:29.170249+0000","last_clean":"2023-06-09T11:59:29.170249+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170249+0000","last_undegraded":"2023-06-09T11:59:29.170249+0000","last_fullsized":"2023-06-09T11:59:29.170249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":880,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088435+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.088435+0000","last_peered":"2023-06-09T11:59:30.088435+0000","last_clean":"2023-06-09T11:59:30.088435+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.088435+0000","last_undegraded":"2023-06-09T11:59:30.088435+0000","last_fullsized":"2023-06-09T11:59:30.088435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1126","reported_seq":4704,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:19.440817+0000","last_change":"2023-06-09T11:59:50.761964+0000","last_active":"2023-06-09T12:00:19.440817+0000","last_peered":"2023-06-09T12:00:19.440817+0000","last_clean":"2023-06-09T12:00:19.440817+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:19.440817+0000","last_undegraded":"2023-06-09T12:00:19.440817+0000","last_fullsized":"2023-06-09T12:00:19.440817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":171,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":766,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.247850+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.247850+0000","last_peered":"2023-06-09T11:59:30.247850+0000","last_clean":"2023-06-09T11:59:30.247850+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.247850+0000","last_undegraded":"2023-06-09T11:59:30.247850+0000","last_fullsized":"2023-06-09T11:59:30.247850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3559,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179988+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.179988+0000","last_peered":"2023-06-09T11:59:29.179988+0000","last_clean":"2023-06-09T11:59:29.179988+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.179988+0000","last_undegraded":"2023-06-09T11:59:29.179988+0000","last_fullsized":"2023-06-09T11:59:29.179988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'588","reported_seq":782,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:50.080648+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:50.080648+0000","last_peered":"2023-06-09T11:59:50.080648+0000","last_clean":"2023-06-09T11:59:50.080648+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:50.080648+0000","last_undegraded":"2023-06-09T11:59:50.080648+0000","last_fullsized":"2023-06-09T11:59:50.080648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3674,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.109212+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:30.109212+0000","last_peered":"2023-06-09T11:59:30.109212+0000","last_clean":"2023-06-09T11:59:30.109212+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:30.109212+0000","last_undegraded":"2023-06-09T11:59:30.109212+0000","last_fullsized":"2023-06-09T11:59:30.109212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'858","reported_seq":1037,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:07.295175+0000","last_change":"2023-06-09T12:00:04.859802+0000","last_active":"2023-06-09T12:00:07.295175+0000","last_peered":"2023-06-09T12:00:07.295175+0000","last_clean":"2023-06-09T12:00:07.295175+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:07.295175+0000","last_undegraded":"2023-06-09T12:00:07.295175+0000","last_fullsized":"2023-06-09T12:00:07.295175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 17s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3148,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207251+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.207251+0000","last_peered":"2023-06-09T11:59:29.207251+0000","last_clean":"2023-06-09T11:59:29.207251+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.207251+0000","last_undegraded":"2023-06-09T11:59:29.207251+0000","last_fullsized":"2023-06-09T11:59:29.207251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.364947+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:30.364947+0000","last_peered":"2023-06-09T11:59:30.364947+0000","last_clean":"2023-06-09T11:59:30.364947+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:30.364947+0000","last_undegraded":"2023-06-09T11:59:30.364947+0000","last_fullsized":"2023-06-09T11:59:30.364947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.120737+0000","last_change":"2023-06-09T11:18:26.644564+0000","last_active":"2023-06-09T11:59:29.120737+0000","last_peered":"2023-06-09T11:59:29.120737+0000","last_clean":"2023-06-09T11:59:29.120737+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:29.120737+0000","last_undegraded":"2023-06-09T11:59:29.120737+0000","last_fullsized":"2023-06-09T11:59:29.120737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.575027+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.575027+0000","last_peered":"2023-06-09T11:59:29.575027+0000","last_clean":"2023-06-09T11:59:29.575027+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.575027+0000","last_undegraded":"2023-06-09T11:59:29.575027+0000","last_fullsized":"2023-06-09T11:59:29.575027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187181+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187181+0000","last_peered":"2023-06-09T11:59:29.187181+0000","last_clean":"2023-06-09T11:59:29.187181+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187181+0000","last_undegraded":"2023-06-09T11:59:29.187181+0000","last_fullsized":"2023-06-09T11:59:29.187181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2978,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181067+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.181067+0000","last_peered":"2023-06-09T11:59:29.181067+0000","last_clean":"2023-06-09T11:59:29.181067+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.181067+0000","last_undegraded":"2023-06-09T11:59:29.181067+0000","last_fullsized":"2023-06-09T11:59:29.181067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207673+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:29.207673+0000","last_peered":"2023-06-09T11:59:29.207673+0000","last_clean":"2023-06-09T11:59:29.207673+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:29.207673+0000","last_undegraded":"2023-06-09T11:59:29.207673+0000","last_fullsized":"2023-06-09T11:59:29.207673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58280,"ondisk_log_size":58280,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":316498,"num_omap_keys":584,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30579,"ondisk_log_size":30579,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254288,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74432,"kb_avail":94117552,"statfs":{"total":96636764160,"available":96376373248,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76218368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.65400000000000003,"15min":0.64000000000000001},"min":{"1min":0.45600000000000002,"5min":0.29699999999999999,"15min":0.28699999999999998},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.75,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.52900000000000003,"5min":0.38,"15min":0.35299999999999998},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.69199999999999995,"15min":0.67400000000000004},"min":{"1min":0.54800000000000004,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.25},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68999999999999995,"15min":0.67800000000000005},"min":{"1min":0.438,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.503,"5min":0.48899999999999999,"15min":0.38400000000000001},"max":{"1min":0.93000000000000005,"5min":1.099,"15min":1.099},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72599999999999998,"15min":0.71999999999999997},"min":{"1min":0.48299999999999998,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.93000000000000005,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jun 9 11:59:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.48599999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.86899999999999999,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.56399999999999995,"5min":0.441,"15min":0.41499999999999998},"max":{"1min":0.94199999999999995,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.54400000000000004,"15min":0.53000000000000003},"min":{"1min":0.40699999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.73799999999999999,"5min":0.81999999999999995,"15min":1.1200000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65700000000000003,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91500000000000004,"5min":0.96399999999999997,"15min":1.1579999999999999},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.622,"15min":0.59999999999999998},"min":{"1min":0.46200000000000002,"5min":0.39500000000000002,"15min":0.32200000000000001},"max":{"1min":0.81599999999999995,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.622},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.624},"min":{"1min":0.442,"5min":0.34399999999999997,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.871}]},{"osd":6,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.64500000000000002},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":1.0920000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.56899999999999995,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":0.94399999999999995,"5min":1.0209999999999999,"15min":1.206},"last":0.88500000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575447,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202840,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46720,"kb_avail":94169000,"statfs":{"total":96636764160,"available":96429056000,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47841280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.63200000000000001,"15min":0.64400000000000002},"min":{"1min":0.46100000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.64800000000000002,"15min":0.63700000000000001},"min":{"1min":0.52200000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.91800000000000004,"5min":1.069,"15min":1.069},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Jun 9 11:59:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.66500000000000004},"min":{"1min":0.46400000000000002,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.96599999999999997,"5min":0.99299999999999999,"15min":1.081},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.71499999999999997,"15min":0.68899999999999995},"min":{"1min":0.61599999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":1.0149999999999999,"5min":1.022,"15min":1.1319999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.88600000000000001,"5min":1.024,"15min":1.125},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57999999999999996,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68799999999999994,"15min":0.69799999999999995},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0369999999999999,"15min":1.1719999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73999999999999999,"15min":0.73799999999999999},"min":{"1min":0.53500000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.002,"5min":1.034,"15min":1.0549999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.438,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":1.107},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.44600000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":0.97699999999999998,"5min":1,"15min":1.0700000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.498,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.052,"5min":1.052,"15min":1.0589999999999999},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.70699999999999996,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.101,"15min":1.101},"last":0.745},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65500000000000003,"15min":0.64700000000000002},"min":{"1min":0.51400000000000001,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.91800000000000004,"5min":1.071,"15min":1.071},"last":0.63700000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575447,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396208,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89792,"kb_avail":93975632,"statfs":{"total":96636764160,"available":96231047168,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91947008},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.39300000000000002,"15min":0.40600000000000003},"min":{"1min":0.29899999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":1.7270000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.53800000000000003},"min":{"1min":0.40699999999999997,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.72899999999999998,"5min":0.89400000000000002,"15min":1.877},"last":0.55300000000000005}]},{"osd":2,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.443,"15min":0.45900000000000002},"min":{"1min":0.318,"5min":0.28000000000000003,"15min":0.27600000000000002},"max":{"1min":0.63900000000000001,"5min":0.82699999999999996,"15min":1.982},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52100000000000002,"15min":0.52600000000000002},"min":{"1min":0.39200000000000002,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.749,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.374,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77200000000000002,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.627,"15min":0.63900000000000001},"min":{"1min":0.44,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.92900000000000005,"5min":1.091,"15min":1.9790000000000001},"last":0.627}]},{"osd":4,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77900000000000003,"5min":1.0649999999999999,"15min":1.609},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.54400000000000004,"15min":0.435},"max":{"1min":1.0940000000000001,"5min":1.1459999999999999,"15min":1.774},"last":0.745}]},{"osd":5,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.77300000000000002,"15min":0.77700000000000002},"min":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0880000000000001,"15min":1.774},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.81200000000000006,"15min":0.80500000000000005},"min":{"1min":0.63100000000000001,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.82199999999999995,"15min":0.83199999999999996},"min":{"1min":0.59199999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.147,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0389999999999999,"5min":1.117,"15min":1.921},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jun 9 11:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.84799999999999998,"15min":0.85899999999999999},"min":{"1min":0.64600000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2010000000000001,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.877},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.751,"15min":0.77600000000000002},"min":{"1min":0.53300000000000003,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0760000000000001,"5min":1.226,"15min":1.827},"last":0.68500000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327464,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99648,"kb_avail":94044376,"statfs":{"total":96636764160,"available":96301441024,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":102039552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.435},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.64100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.82699999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":0.86399999999999999,"15min":1.0900000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65100000000000002,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.57099999999999995,"5min":0.871,"15min":1.2909999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.59499999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.1060000000000001},"last":0.498},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.21},"last":0.70099999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.61899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.236},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.747},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.165},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85799999999999998,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.1240000000000001,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.153,"15min":1.3700000000000001},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.159,"15min":1.228},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84299999999999997,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.58199999999999996,"15min":0.46999999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.29},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575448,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236768,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":56960,"kb_avail":94135072,"statfs":{"total":96636764160,"available":96394313728,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58327040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.27300000000000002,"15min":0.22},"max":{"1min":0.56000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50700000000000001,"15min":0.5},"min":{"1min":0.38400000000000001,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.65400000000000003,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60499999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.92300000000000004,"5min":1.367,"15min":1.367},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.96799999999999997,"5min":1.546,"15min":1.546},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":1.171,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.57999999999999996,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.1180000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.2450000000000001,"5min":1.601,"15min":1.601},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.871,"5min":0.84899999999999998,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.96899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.438},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":1.105,"5min":1.121,"15min":1.3540000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.208,"15min":1.248},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83599999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.262,"5min":1.262,"15min":1.337},"last":0.73999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575447,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433276,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110080,"kb_avail":93938564,"statfs":{"total":96636764160,"available":96193089536,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112721920},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.32000000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.65500000000000003,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"min":{"1min":0.39500000000000002,"5min":0.33600000000000002,"15min":0.29499999999999998},"max":{"1min":0.624,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82899999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77000000000000002,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.50800000000000001,"5min":0.42299999999999999,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.355,"15min":1.355},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90000000000000002,"5min":1.198,"15min":1.198},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.59399999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":1.635,"15min":1.635},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.751,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.61499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.198,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.234,"5min":1.367,"15min":1.367},"last":0.66600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.503,"15min":0.48499999999999999},"max":{"1min":1.0569999999999999,"5min":1.286,"15min":1.286},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.434,"15min":0.434},"max":{"1min":1.0469999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.629}]},{"osd":7,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.66700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.0780000000000001,"5min":1.413,"15min":1.413},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88200000000000001,"15min":0.873},"min":{"1min":0.71399999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.79600000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575446,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413332,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120576,"kb_avail":93958508,"statfs":{"total":96636764160,"available":96213512192,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123469824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.95799999999999996,"5min":1.137,"15min":1.137},"last":0.753},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.63,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":0.96799999999999997,"5min":1.0640000000000001,"15min":1.103},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.64500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.044,"5min":1.1279999999999999,"15min":1.2},"last":0.88600000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.52700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":1.609,"15min":1.609},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.997,"5min":0.997,"15min":1.1659999999999999},"last":0.80900000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.89000000000000001,"5min":1.407,"15min":1.407},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.96199999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.67100000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.014,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.92000000000000004,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.94199999999999995,"5min":1.032,"15min":1.032},"last":0.626},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.97299999999999998,"5min":1.361,"15min":1.3740000000000001},"last":0.77100000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575446,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288024,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70656,"kb_avail":94083816,"statfs":{"total":96636764160,"available":96341827584,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72351744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.72599999999999998,"15min":0.67700000000000005},"min":{"1min":0.52700000000000002,"5min":0.36599999999999999,"15min":0.28999999999999998},"max":{"1min":5.4969999999999999,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.748},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.749,"15min":0.70399999999999996},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.318},"max":{"1min":5.6269999999999998,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.89400000000000002}]},{"osd":1,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.754,"5min":0.86899999999999999,"15min":0.70899999999999996},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":19.715,"5min":19.715,"15min":19.715},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.137,"5min":0.77000000000000002,"15min":0.69499999999999995},"min":{"1min":0.53700000000000003,"5min":0.42599999999999999,"15min":0.33400000000000002},"max":{"1min":7.4089999999999998,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.53600000000000003,"5min":0.38,"15min":0.373},"max":{"1min":5.5620000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.423,"5min":0.86199999999999999,"15min":0.75800000000000001},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":14.439,"5min":14.439,"15min":14.439},"last":0.82899999999999996}]},{"osd":3,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":0.85099999999999998,"15min":0.749},"min":{"1min":0.51300000000000001,"5min":0.433,"15min":0.375},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.401,"5min":0.89100000000000001,"15min":0.80700000000000005},"min":{"1min":0.59599999999999997,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":12.218999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":4.1680000000000001,"5min":1.306,"15min":0.82399999999999995},"min":{"1min":0.46600000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":67.590000000000003,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":4.1959999999999997,"5min":1.3160000000000001,"15min":0.83999999999999997},"min":{"1min":0.45700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":67.521000000000001,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":0.77600000000000002,"15min":0.68799999999999994},"min":{"1min":0.54100000000000004,"5min":0.40400000000000003,"15min":0.36799999999999999},"max":{"1min":12.163,"5min":12.163,"15min":12.163},"last":0.58799999999999997},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.69899999999999995,"15min":0.64700000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35699999999999998},"max":{"1min":7.6040000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Jun 9 11:59:29 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.55800000000000005,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":3.8039999999999998,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.504},{"interface":"front","average":{"1min":1.883,"5min":0.97499999999999998,"15min":0.81299999999999994},"min":{"1min":0.65300000000000002,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":19.834,"5min":19.834,"15min":19.834},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:00:27.856 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.857 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.857 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.857 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.858 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.858 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.858 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.859 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.859 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.859 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.860 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.860 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.861 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.861 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.861 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.862 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.862 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.862 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.863 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.863 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.863 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.864 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.864 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.864 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.865 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.865 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.865 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.866 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.866 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.866 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.867 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.867 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.867 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.868 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.868 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.868 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.869 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.869 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.869 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.870 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.870 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.870 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.871 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.871 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.871 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.872 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.872 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.872 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.873 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.873 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.874 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.874 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.874 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.875 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.875 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.875 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.876 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.876 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.876 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.877 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.877 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.877 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.878 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.878 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.878 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.879 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.879 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.879 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.880 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.880 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.880 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.881 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.881 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.881 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.882 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.882 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.882 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.883 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.883 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.883 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.884 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.884 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.884 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.885 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.885 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.885 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.886 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.886 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.886 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.887 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.887 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.887 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.888 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.888 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.888 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:27.889 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:00:47.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:00:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.098+0000 7ff326bff700 1 Processor -- start 2023-06-09T12:00:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.098+0000 7ff326bff700 1 -- start start 2023-06-09T12:00:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff326bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff320153880 0x7ff32015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff326bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7ff3200c9f40 0x7ff32015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff326bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32015a500 con 0x7ff32015b2d0 2023-06-09T12:00:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff326bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3200c5f10 con 0x7ff320153880 2023-06-09T12:00:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff326bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32015da00 con 0x7ff3200c9f40 2023-06-09T12:00:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff320153880 0x7ff32015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff320153880 0x7ff32015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53156/0 (socket says 172.21.15.7:53156) 2023-06-09T12:00:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 -- 172.21.15.7:0/4088453498 learned_addr learned my addr 172.21.15.7:0/4088453498 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff31ffff700 1 --2- 172.21.15.7:0/4088453498 >> v2:172.21.15.117:3300/0 conn(0x7ff3200c9f40 0x7ff32015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 -- 172.21.15.7:0/4088453498 >> v2:172.21.15.117:3300/0 conn(0x7ff3200c9f40 msgr2=0x7ff32015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:00:48.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 --2- 172.21.15.7:0/4088453498 >> v2:172.21.15.117:3300/0 conn(0x7ff3200c9f40 0x7ff32015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 -- 172.21.15.7:0/4088453498 >> v1:172.21.15.7:6789/0 conn(0x7ff32015b2d0 legacy=0x7ff3200c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.100+0000 7ff32499b700 1 -- 172.21.15.7:0/4088453498 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3201596e0 con 0x7ff320153880 2023-06-09T12:00:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff32499b700 1 --2- 172.21.15.7:0/4088453498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff320153880 0x7ff32015d4c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff3200cce30 tx=0x7ff314009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3610f27c4e5365 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff31f7fe700 1 -- 172.21.15.7:0/4088453498 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400b870 con 0x7ff320153880 2023-06-09T12:00:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff31f7fe700 1 -- 172.21.15.7:0/4088453498 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400b9d0 con 0x7ff320153880 2023-06-09T12:00:48.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff31f7fe700 1 -- 172.21.15.7:0/4088453498 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400bd60 con 0x7ff320153880 2023-06-09T12:00:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff326bff700 1 -- 172.21.15.7:0/4088453498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff320153880 msgr2=0x7ff32015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.101+0000 7ff326bff700 1 --2- 172.21.15.7:0/4088453498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff320153880 0x7ff32015d4c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff3200cce30 tx=0x7ff314009bd0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 -- 172.21.15.7:0/4088453498 shutdown_connections 2023-06-09T12:00:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 --2- 172.21.15.7:0/4088453498 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff320153880 0x7ff32015d4c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 --2- 172.21.15.7:0/4088453498 >> v2:172.21.15.117:3300/0 conn(0x7ff3200c9f40 0x7ff32015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 -- 172.21.15.7:0/4088453498 >> 172.21.15.7:0/4088453498 conn(0x7ff3200737f0 msgr2=0x7ff3200720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:48.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 -- 172.21.15.7:0/4088453498 shutdown_connections 2023-06-09T12:00:48.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.102+0000 7ff326bff700 1 -- 172.21.15.7:0/4088453498 wait complete. 2023-06-09T12:00:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.104+0000 7ff326bff700 1 Processor -- start 2023-06-09T12:00:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.104+0000 7ff326bff700 1 -- start start 2023-06-09T12:00:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3200c9f40 0x7ff3200b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff320153880 0x7ff32014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3200b3560 con 0x7ff32015b2d0 2023-06-09T12:00:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201f3470 con 0x7ff3200c9f40 2023-06-09T12:00:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.105+0000 7ff326bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3201f2860 con 0x7ff320153880 2023-06-09T12:00:48.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53164/0 (socket says 172.21.15.7:53164) 2023-06-09T12:00:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 -- 172.21.15.7:0/2939657453 learned_addr learned my addr 172.21.15.7:0/2939657453 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:00:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32499b700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3200c9f40 0x7ff3200b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff31ffff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff320153880 0x7ff32014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 -- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff320153880 msgr2=0x7ff32014b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff320153880 0x7ff32014b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 -- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3200c9f40 msgr2=0x7ff3200b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3200c9f40 0x7ff3200b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 -- 172.21.15.7:0/2939657453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314009550 con 0x7ff32015b2d0 2023-06-09T12:00:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.106+0000 7ff32519c700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cb50 tx=0x7ff31800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.107+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318003c00 con 0x7ff32015b2d0 2023-06-09T12:00:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.107+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff318003d60 con 0x7ff32015b2d0 2023-06-09T12:00:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.107+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff320159c60 con 0x7ff32015b2d0 2023-06-09T12:00:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.107+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800ab70 con 0x7ff32015b2d0 2023-06-09T12:00:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.107+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff320159980 con 0x7ff32015b2d0 2023-06-09T12:00:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.108+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff320159f40 con 0x7ff32015b2d0 2023-06-09T12:00:48.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.109+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff31800acd0 con 0x7ff32015b2d0 2023-06-09T12:00:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.110+0000 7ff31d7fa700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 0x7ff30406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:00:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.110+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7ff318089dd0 con 0x7ff32015b2d0 2023-06-09T12:00:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.110+0000 7ff32499b700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 0x7ff30406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:00:48.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.111+0000 7ff32499b700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 0x7ff30406d1c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:00:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.115+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff318054470 con 0x7ff32015b2d0 2023-06-09T12:00:48.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.320+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3201f35e0 con 0x7ff30406ad10 2023-06-09T12:00:48.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.329+0000 7ff31d7fa700 1 -- 172.21.15.7:0/2939657453 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+313569 (secure 0 0 0) 0x7ff3201f35e0 con 0x7ff30406ad10 2023-06-09T12:00:48.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:00:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 msgr2=0x7ff30406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 0x7ff30406d1c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff314000c00 tx=0x7ff314008030 comp rx=0 tx=0).stop 2023-06-09T12:00:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 msgr2=0x7ff3201ef370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:00:48.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cb50 tx=0x7ff31800cf10 comp rx=0 tx=0).stop 2023-06-09T12:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 shutdown_connections 2023-06-09T12:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff32015b2d0 0x7ff3201ef370 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff3200c9f40 0x7ff3200b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff30406ad10 0x7ff30406d1c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.338 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 --2- 172.21.15.7:0/2939657453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff320153880 0x7ff32014b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 >> 172.21.15.7:0/2939657453 conn(0x7ff3200737f0 msgr2=0x7ff32015b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 shutdown_connections 2023-06-09T12:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:00:48.334+0000 7ff326bff700 1 -- 172.21.15.7:0/2939657453 wait complete. 2023-06-09T12:00:48.339 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:00:48.355 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1570,"stamp":"2023-06-09T12:00:46.943887+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":434116,"num_omap_keys":816,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89219,"ondisk_log_size":89219,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2556872,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":673536,"kb_avail":752417848,"statfs":{"total":773094113280,"available":770475876352,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":689700864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003158"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3462,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.094000+0000","last_change":"2023-06-09T11:18:26.253407+0000","last_active":"2023-06-09T11:59:30.094000+0000","last_peered":"2023-06-09T11:59:30.094000+0000","last_clean":"2023-06-09T11:59:30.094000+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T11:59:30.094000+0000","last_undegraded":"2023-06-09T11:59:30.094000+0000","last_fullsized":"2023-06-09T11:59:30.094000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3218,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.138170+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:29.138170+0000","last_peered":"2023-06-09T11:59:29.138170+0000","last_clean":"2023-06-09T11:59:29.138170+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:29.138170+0000","last_undegraded":"2023-06-09T11:59:29.138170+0000","last_fullsized":"2023-06-09T11:59:29.138170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.186026+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:29.186026+0000","last_peered":"2023-06-09T11:59:29.186026+0000","last_clean":"2023-06-09T11:59:29.186026+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:29.186026+0000","last_undegraded":"2023-06-09T11:59:29.186026+0000","last_fullsized":"2023-06-09T11:59:29.186026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3450,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601391+0000","last_change":"2023-06-09T12:00:26.578211+0000","last_active":"2023-06-09T11:59:45.601391+0000","last_peered":"2023-06-09T11:59:45.601391+0000","last_clean":"2023-06-09T11:59:45.601391+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T11:59:45.601391+0000","last_undegraded":"2023-06-09T11:59:45.601391+0000","last_fullsized":"2023-06-09T11:59:45.601391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3603,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179222+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:29.179222+0000","last_peered":"2023-06-09T11:59:29.179222+0000","last_clean":"2023-06-09T11:59:29.179222+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:29.179222+0000","last_undegraded":"2023-06-09T11:59:29.179222+0000","last_fullsized":"2023-06-09T11:59:29.179222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1036","reported_seq":3526,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:41.375100+0000","last_change":"2023-06-09T12:00:18.653707+0000","last_active":"2023-06-09T12:00:41.375100+0000","last_peered":"2023-06-09T12:00:41.375100+0000","last_clean":"2023-06-09T12:00:41.375100+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:41.375100+0000","last_undegraded":"2023-06-09T12:00:41.375100+0000","last_fullsized":"2023-06-09T12:00:41.375100+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":172,"log_size":1036,"log_dups_size":0,"ondisk_log_size":1036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'873","reported_seq":3089,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.135048+0000","last_change":"2023-06-09T11:18:26.316980+0000","last_active":"2023-06-09T11:59:29.135048+0000","last_peered":"2023-06-09T11:59:29.135048+0000","last_clean":"2023-06-09T11:59:29.135048+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T11:59:29.135048+0000","last_undegraded":"2023-06-09T11:59:29.135048+0000","last_fullsized":"2023-06-09T11:59:29.135048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":873,"log_dups_size":0,"ondisk_log_size":873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3244,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.164360+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:29.164360+0000","last_peered":"2023-06-09T11:59:29.164360+0000","last_clean":"2023-06-09T11:59:29.164360+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:29.164360+0000","last_undegraded":"2023-06-09T11:59:29.164360+0000","last_fullsized":"2023-06-09T11:59:29.164360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163620+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163620+0000","last_peered":"2023-06-09T11:59:29.163620+0000","last_clean":"2023-06-09T11:59:29.163620+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163620+0000","last_undegraded":"2023-06-09T11:59:29.163620+0000","last_fullsized":"2023-06-09T11:59:29.163620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184811+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184811+0000","last_peered":"2023-06-09T11:59:29.184811+0000","last_clean":"2023-06-09T11:59:29.184811+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184811+0000","last_undegraded":"2023-06-09T11:59:29.184811+0000","last_fullsized":"2023-06-09T11:59:29.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3401,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821600+0000","last_change":"2023-06-09T11:59:49.712515+0000","last_active":"2023-06-09T11:59:42.821600+0000","last_peered":"2023-06-09T11:59:42.821600+0000","last_clean":"2023-06-09T11:59:42.821600+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:42.821600+0000","last_undegraded":"2023-06-09T11:59:42.821600+0000","last_fullsized":"2023-06-09T11:59:42.821600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3231,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821561+0000","last_change":"2023-06-09T11:59:48.700631+0000","last_active":"2023-06-09T11:59:42.821561+0000","last_peered":"2023-06-09T11:59:42.821561+0000","last_clean":"2023-06-09T11:59:42.821561+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:42.821561+0000","last_undegraded":"2023-06-09T11:59:42.821561+0000","last_fullsized":"2023-06-09T11:59:42.821561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'880","reported_seq":3555,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.821547+0000","last_change":"2023-06-09T11:18:26.252839+0000","last_active":"2023-06-09T11:59:42.821547+0000","last_peered":"2023-06-09T11:59:42.821547+0000","last_clean":"2023-06-09T11:59:42.821547+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T11:59:42.821547+0000","last_undegraded":"2023-06-09T11:59:42.821547+0000","last_fullsized":"2023-06-09T11:59:42.821547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296222+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:46.296222+0000","last_peered":"2023-06-09T11:59:46.296222+0000","last_clean":"2023-06-09T11:59:46.296222+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:46.296222+0000","last_undegraded":"2023-06-09T11:59:46.296222+0000","last_fullsized":"2023-06-09T11:59:46.296222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423589+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:41.423589+0000","last_peered":"2023-06-09T11:59:41.423589+0000","last_clean":"2023-06-09T11:59:41.423589+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:41.423589+0000","last_undegraded":"2023-06-09T11:59:41.423589+0000","last_fullsized":"2023-06-09T11:59:41.423589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'909","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517081+0000","last_change":"2023-06-09T11:18:26.313696+0000","last_active":"2023-06-09T11:59:43.517081+0000","last_peered":"2023-06-09T11:59:43.517081+0000","last_clean":"2023-06-09T11:59:43.517081+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T11:59:43.517081+0000","last_undegraded":"2023-06-09T11:59:43.517081+0000","last_fullsized":"2023-06-09T11:59:43.517081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":909,"log_dups_size":0,"ondisk_log_size":909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181720+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:29.181720+0000","last_peered":"2023-06-09T11:59:29.181720+0000","last_clean":"2023-06-09T11:59:29.181720+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:29.181720+0000","last_undegraded":"2023-06-09T11:59:29.181720+0000","last_fullsized":"2023-06-09T11:59:29.181720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3018,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121483+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:42.121483+0000","last_peered":"2023-06-09T11:59:42.121483+0000","last_clean":"2023-06-09T11:59:42.121483+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:42.121483+0000","last_undegraded":"2023-06-09T11:59:42.121483+0000","last_fullsized":"2023-06-09T11:59:42.121483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905252+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:44.905252+0000","last_peered":"2023-06-09T11:59:44.905252+0000","last_clean":"2023-06-09T11:59:44.905252+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:44.905252+0000","last_undegraded":"2023-06-09T11:59:44.905252+0000","last_fullsized":"2023-06-09T11:59:44.905252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.470082+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:30.470082+0000","last_peered":"2023-06-09T11:59:30.470082+0000","last_clean":"2023-06-09T11:59:30.470082+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:30.470082+0000","last_undegraded":"2023-06-09T11:59:30.470082+0000","last_fullsized":"2023-06-09T11:59:30.470082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.177938+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:30.177938+0000","last_peered":"2023-06-09T11:59:30.177938+0000","last_clean":"2023-06-09T11:59:30.177938+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:30.177938+0000","last_undegraded":"2023-06-09T11:59:30.177938+0000","last_fullsized":"2023-06-09T11:59:30.177938+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1238","reported_seq":2241,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.087049+0000","last_change":"2023-06-09T11:10:49.679016+0000","last_active":"2023-06-09T11:59:30.087049+0000","last_peered":"2023-06-09T11:59:30.087049+0000","last_clean":"2023-06-09T11:59:30.087049+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T11:59:30.087049+0000","last_undegraded":"2023-06-09T11:59:30.087049+0000","last_fullsized":"2023-06-09T11:59:30.087049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1238,"log_dups_size":0,"ondisk_log_size":1238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3134,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.105820+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:30.105820+0000","last_peered":"2023-06-09T11:59:30.105820+0000","last_clean":"2023-06-09T11:59:30.105820+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:30.105820+0000","last_undegraded":"2023-06-09T11:59:30.105820+0000","last_fullsized":"2023-06-09T11:59:30.105820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1997,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:43.516919+0000","last_change":"2023-06-09T12:00:33.899239+0000","last_active":"2023-06-09T11:59:43.516919+0000","last_peered":"2023-06-09T11:59:43.516919+0000","last_clean":"2023-06-09T11:59:43.516919+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:43.516919+0000","last_undegraded":"2023-06-09T11:59:43.516919+0000","last_fullsized":"2023-06-09T11:59:43.516919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3304,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601328+0000","last_change":"2023-06-09T12:00:28.569063+0000","last_active":"2023-06-09T11:59:45.601328+0000","last_peered":"2023-06-09T11:59:45.601328+0000","last_clean":"2023-06-09T11:59:45.601328+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T11:59:45.601328+0000","last_undegraded":"2023-06-09T11:59:45.601328+0000","last_fullsized":"2023-06-09T11:59:45.601328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":347,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:50.067505+0000","last_change":"2023-06-09T11:59:59.471359+0000","last_active":"2023-06-09T11:59:50.067505+0000","last_peered":"2023-06-09T11:59:50.067505+0000","last_clean":"2023-06-09T11:59:50.067505+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:50.067505+0000","last_undegraded":"2023-06-09T11:59:50.067505+0000","last_fullsized":"2023-06-09T11:59:50.067505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2963,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:52.293286+0000","last_change":"2023-06-09T12:00:36.885523+0000","last_active":"2023-06-09T11:59:52.293286+0000","last_peered":"2023-06-09T11:59:52.293286+0000","last_clean":"2023-06-09T11:59:52.293286+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:52.293286+0000","last_undegraded":"2023-06-09T11:59:52.293286+0000","last_fullsized":"2023-06-09T11:59:52.293286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"73'1365","reported_seq":1542,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.287114+0000","last_change":"2023-06-09T12:00:20.271439+0000","last_active":"2023-06-09T12:00:22.287114+0000","last_peered":"2023-06-09T12:00:22.287114+0000","last_clean":"2023-06-09T12:00:22.287114+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:22.287114+0000","last_undegraded":"2023-06-09T12:00:22.287114+0000","last_fullsized":"2023-06-09T12:00:22.287114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1365,"log_dups_size":0,"ondisk_log_size":1365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 17s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1264,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296204+0000","last_change":"2023-06-09T12:00:19.292684+0000","last_active":"2023-06-09T11:59:46.296204+0000","last_peered":"2023-06-09T11:59:46.296204+0000","last_clean":"2023-06-09T11:59:46.296204+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:46.296204+0000","last_undegraded":"2023-06-09T11:59:46.296204+0000","last_fullsized":"2023-06-09T11:59:46.296204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1034,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:49.057672+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:49.057672+0000","last_peered":"2023-06-09T11:59:49.057672+0000","last_clean":"2023-06-09T11:59:49.057672+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:49.057672+0000","last_undegraded":"2023-06-09T11:59:49.057672+0000","last_fullsized":"2023-06-09T11:59:49.057672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.390934+0000","last_change":"2023-06-09T11:10:49.679535+0000","last_active":"2023-06-09T11:59:52.390934+0000","last_peered":"2023-06-09T11:59:52.390934+0000","last_clean":"2023-06-09T11:59:52.390934+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:52.390934+0000","last_undegraded":"2023-06-09T11:59:52.390934+0000","last_fullsized":"2023-06-09T11:59:52.390934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3559,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.294703+0000","last_change":"2023-06-09T12:00:41.875998+0000","last_active":"2023-06-09T12:00:02.294703+0000","last_peered":"2023-06-09T12:00:02.294703+0000","last_clean":"2023-06-09T12:00:02.294703+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:02.294703+0000","last_undegraded":"2023-06-09T12:00:02.294703+0000","last_fullsized":"2023-06-09T12:00:02.294703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3586,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116172+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116172+0000","last_peered":"2023-06-09T11:59:29.116172+0000","last_clean":"2023-06-09T11:59:29.116172+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116172+0000","last_undegraded":"2023-06-09T11:59:29.116172+0000","last_fullsized":"2023-06-09T11:59:29.116172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:25.604660+0000","last_change":"2023-06-09T12:00:25.604660+0000","last_active":"2023-06-09T12:00:25.604660+0000","last_peered":"2023-06-09T12:00:25.604660+0000","last_clean":"2023-06-09T12:00:25.604660+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:00:25.604660+0000","last_undegraded":"2023-06-09T12:00:25.604660+0000","last_fullsized":"2023-06-09T12:00:25.604660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3763,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203922+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203922+0000","last_peered":"2023-06-09T11:59:29.203922+0000","last_clean":"2023-06-09T11:59:29.203922+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203922+0000","last_undegraded":"2023-06-09T11:59:29.203922+0000","last_fullsized":"2023-06-09T11:59:29.203922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":727,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296114+0000","last_change":"2023-06-09T11:10:49.680810+0000","last_active":"2023-06-09T11:59:46.296114+0000","last_peered":"2023-06-09T11:59:46.296114+0000","last_clean":"2023-06-09T11:59:46.296114+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T11:59:46.296114+0000","last_undegraded":"2023-06-09T11:59:46.296114+0000","last_fullsized":"2023-06-09T11:59:46.296114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905230+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:44.905230+0000","last_peered":"2023-06-09T11:59:44.905230+0000","last_clean":"2023-06-09T11:59:44.905230+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:44.905230+0000","last_undegraded":"2023-06-09T11:59:44.905230+0000","last_fullsized":"2023-06-09T11:59:44.905230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":690,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296113+0000","last_change":"2023-06-09T11:10:49.682075+0000","last_active":"2023-06-09T11:59:46.296113+0000","last_peered":"2023-06-09T11:59:46.296113+0000","last_clean":"2023-06-09T11:59:46.296113+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T11:59:46.296113+0000","last_undegraded":"2023-06-09T11:59:46.296113+0000","last_fullsized":"2023-06-09T11:59:46.296113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296114+0000","last_change":"2023-06-09T11:10:49.681678+0000","last_active":"2023-06-09T11:59:46.296114+0000","last_peered":"2023-06-09T11:59:46.296114+0000","last_clean":"2023-06-09T11:59:46.296114+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T11:59:46.296114+0000","last_undegraded":"2023-06-09T11:59:46.296114+0000","last_fullsized":"2023-06-09T11:59:46.296114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3538,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601328+0000","last_change":"2023-06-09T12:00:35.635944+0000","last_active":"2023-06-09T11:59:45.601328+0000","last_peered":"2023-06-09T11:59:45.601328+0000","last_clean":"2023-06-09T11:59:45.601328+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T11:59:45.601328+0000","last_undegraded":"2023-06-09T11:59:45.601328+0000","last_fullsized":"2023-06-09T11:59:45.601328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445722+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445722+0000","last_peered":"2023-06-09T12:00:04.445722+0000","last_clean":"2023-06-09T12:00:04.445722+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445722+0000","last_undegraded":"2023-06-09T12:00:04.445722+0000","last_fullsized":"2023-06-09T12:00:04.445722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207708+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:44.207708+0000","last_peered":"2023-06-09T11:59:44.207708+0000","last_clean":"2023-06-09T11:59:44.207708+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:44.207708+0000","last_undegraded":"2023-06-09T11:59:44.207708+0000","last_fullsized":"2023-06-09T11:59:44.207708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121371+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:42.121371+0000","last_peered":"2023-06-09T11:59:42.121371+0000","last_clean":"2023-06-09T11:59:42.121371+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:42.121371+0000","last_undegraded":"2023-06-09T11:59:42.121371+0000","last_fullsized":"2023-06-09T11:59:42.121371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525010+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525010+0000","last_peered":"2023-06-09T11:59:44.525010+0000","last_clean":"2023-06-09T11:59:44.525010+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525010+0000","last_undegraded":"2023-06-09T11:59:44.525010+0000","last_fullsized":"2023-06-09T11:59:44.525010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822281+0000","last_change":"2023-06-09T11:10:49.673773+0000","last_active":"2023-06-09T11:59:23.822281+0000","last_peered":"2023-06-09T11:59:23.822281+0000","last_clean":"2023-06-09T11:59:23.822281+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T11:59:23.822281+0000","last_undegraded":"2023-06-09T11:59:23.822281+0000","last_fullsized":"2023-06-09T11:59:23.822281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.198957+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:29.198957+0000","last_peered":"2023-06-09T11:59:29.198957+0000","last_clean":"2023-06-09T11:59:29.198957+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:29.198957+0000","last_undegraded":"2023-06-09T11:59:29.198957+0000","last_fullsized":"2023-06-09T11:59:29.198957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":154,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905097+0000","last_change":"2023-06-09T12:00:17.246824+0000","last_active":"2023-06-09T11:59:44.905097+0000","last_peered":"2023-06-09T11:59:44.905097+0000","last_clean":"2023-06-09T11:59:44.905097+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T11:59:44.905097+0000","last_undegraded":"2023-06-09T11:59:44.905097+0000","last_fullsized":"2023-06-09T11:59:44.905097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":896,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.604134+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:29.604134+0000","last_peered":"2023-06-09T11:59:29.604134+0000","last_clean":"2023-06-09T11:59:29.604134+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:29.604134+0000","last_undegraded":"2023-06-09T11:59:29.604134+0000","last_fullsized":"2023-06-09T11:59:29.604134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3888,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203700+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:29.203700+0000","last_peered":"2023-06-09T11:59:29.203700+0000","last_clean":"2023-06-09T11:59:29.203700+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:29.203700+0000","last_undegraded":"2023-06-09T11:59:29.203700+0000","last_fullsized":"2023-06-09T11:59:29.203700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937169+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937169+0000","last_peered":"2023-06-09T11:59:45.937169+0000","last_clean":"2023-06-09T11:59:45.937169+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937169+0000","last_undegraded":"2023-06-09T11:59:45.937169+0000","last_fullsized":"2023-06-09T11:59:45.937169+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.822857+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:23.822857+0000","last_peered":"2023-06-09T11:59:23.822857+0000","last_clean":"2023-06-09T11:59:23.822857+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:23.822857+0000","last_undegraded":"2023-06-09T11:59:23.822857+0000","last_fullsized":"2023-06-09T11:59:23.822857+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121335+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:42.121335+0000","last_peered":"2023-06-09T11:59:42.121335+0000","last_clean":"2023-06-09T11:59:42.121335+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:42.121335+0000","last_undegraded":"2023-06-09T11:59:42.121335+0000","last_fullsized":"2023-06-09T11:59:42.121335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":154,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905109+0000","last_change":"2023-06-09T12:00:15.338006+0000","last_active":"2023-06-09T11:59:44.905109+0000","last_peered":"2023-06-09T11:59:44.905109+0000","last_clean":"2023-06-09T11:59:44.905109+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T11:59:44.905109+0000","last_undegraded":"2023-06-09T11:59:44.905109+0000","last_fullsized":"2023-06-09T11:59:44.905109+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1347,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602275+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602275+0000","last_peered":"2023-06-09T12:00:03.602275+0000","last_clean":"2023-06-09T12:00:03.602275+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602275+0000","last_undegraded":"2023-06-09T12:00:03.602275+0000","last_fullsized":"2023-06-09T12:00:03.602275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3297,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601367+0000","last_change":"2023-06-09T12:00:32.542755+0000","last_active":"2023-06-09T11:59:45.601367+0000","last_peered":"2023-06-09T11:59:45.601367+0000","last_clean":"2023-06-09T11:59:45.601367+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T11:59:45.601367+0000","last_undegraded":"2023-06-09T11:59:45.601367+0000","last_fullsized":"2023-06-09T11:59:45.601367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296161+0000","last_change":"2023-06-09T11:59:47.289624+0000","last_active":"2023-06-09T11:59:46.296161+0000","last_peered":"2023-06-09T11:59:46.296161+0000","last_clean":"2023-06-09T11:59:46.296161+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:46.296161+0000","last_undegraded":"2023-06-09T11:59:46.296161+0000","last_fullsized":"2023-06-09T11:59:46.296161+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.467905+0000","last_change":"2023-06-09T11:10:49.684817+0000","last_active":"2023-06-09T11:59:30.467905+0000","last_peered":"2023-06-09T11:59:30.467905+0000","last_clean":"2023-06-09T11:59:30.467905+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T11:59:30.467905+0000","last_undegraded":"2023-06-09T11:59:30.467905+0000","last_fullsized":"2023-06-09T11:59:30.467905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423608+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:41.423608+0000","last_peered":"2023-06-09T11:59:41.423608+0000","last_clean":"2023-06-09T11:59:41.423608+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:41.423608+0000","last_undegraded":"2023-06-09T11:59:41.423608+0000","last_fullsized":"2023-06-09T11:59:41.423608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637615+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637615+0000","last_peered":"2023-06-09T12:00:04.637615+0000","last_clean":"2023-06-09T12:00:04.637615+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637615+0000","last_undegraded":"2023-06-09T12:00:04.637615+0000","last_fullsized":"2023-06-09T12:00:04.637615+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":820,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:23.823970+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:23.823970+0000","last_peered":"2023-06-09T11:59:23.823970+0000","last_clean":"2023-06-09T11:59:23.823970+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:23.823970+0000","last_undegraded":"2023-06-09T11:59:23.823970+0000","last_fullsized":"2023-06-09T11:59:23.823970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4111,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.303700+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:30.303700+0000","last_peered":"2023-06-09T11:59:30.303700+0000","last_clean":"2023-06-09T11:59:30.303700+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:30.303700+0000","last_undegraded":"2023-06-09T11:59:30.303700+0000","last_fullsized":"2023-06-09T11:59:30.303700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3368,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:43.516945+0000","last_change":"2023-06-09T12:00:38.899563+0000","last_active":"2023-06-09T11:59:43.516945+0000","last_peered":"2023-06-09T11:59:43.516945+0000","last_clean":"2023-06-09T11:59:43.516945+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T11:59:43.516945+0000","last_undegraded":"2023-06-09T11:59:43.516945+0000","last_fullsized":"2023-06-09T11:59:43.516945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.652148+0000","last_change":"2023-06-09T12:00:37.490252+0000","last_active":"2023-06-09T12:00:36.652148+0000","last_peered":"2023-06-09T12:00:36.652148+0000","last_clean":"2023-06-09T12:00:36.652148+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:36.652148+0000","last_undegraded":"2023-06-09T12:00:36.652148+0000","last_fullsized":"2023-06-09T12:00:36.652148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423433+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:41.423433+0000","last_peered":"2023-06-09T11:59:41.423433+0000","last_clean":"2023-06-09T11:59:41.423433+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:41.423433+0000","last_undegraded":"2023-06-09T11:59:41.423433+0000","last_fullsized":"2023-06-09T11:59:41.423433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3714,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.170249+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:29.170249+0000","last_peered":"2023-06-09T11:59:29.170249+0000","last_clean":"2023-06-09T11:59:29.170249+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:29.170249+0000","last_undegraded":"2023-06-09T11:59:29.170249+0000","last_fullsized":"2023-06-09T11:59:29.170249+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":880,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.088435+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:30.088435+0000","last_peered":"2023-06-09T11:59:30.088435+0000","last_clean":"2023-06-09T11:59:30.088435+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:30.088435+0000","last_undegraded":"2023-06-09T11:59:30.088435+0000","last_fullsized":"2023-06-09T11:59:30.088435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1200","reported_seq":4782,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:40.283808+0000","last_change":"2023-06-09T11:59:50.761964+0000","last_active":"2023-06-09T12:00:40.283808+0000","last_peered":"2023-06-09T12:00:40.283808+0000","last_clean":"2023-06-09T12:00:40.283808+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:40.283808+0000","last_undegraded":"2023-06-09T12:00:40.283808+0000","last_fullsized":"2023-06-09T12:00:40.283808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":245,"log_size":1200,"log_dups_size":0,"ondisk_log_size":1200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 49s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":766,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.247850+0000","last_change":"2023-06-09T11:10:49.682035+0000","last_active":"2023-06-09T11:59:30.247850+0000","last_peered":"2023-06-09T11:59:30.247850+0000","last_clean":"2023-06-09T11:59:30.247850+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T11:59:30.247850+0000","last_undegraded":"2023-06-09T11:59:30.247850+0000","last_fullsized":"2023-06-09T11:59:30.247850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3559,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.179988+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:29.179988+0000","last_peered":"2023-06-09T11:59:29.179988+0000","last_clean":"2023-06-09T11:59:29.179988+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:29.179988+0000","last_undegraded":"2023-06-09T11:59:29.179988+0000","last_fullsized":"2023-06-09T11:59:29.179988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'588","reported_seq":782,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:50.080648+0000","last_change":"2023-06-09T11:10:49.684059+0000","last_active":"2023-06-09T11:59:50.080648+0000","last_peered":"2023-06-09T11:59:50.080648+0000","last_clean":"2023-06-09T11:59:50.080648+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T11:59:50.080648+0000","last_undegraded":"2023-06-09T11:59:50.080648+0000","last_fullsized":"2023-06-09T11:59:50.080648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3674,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.109212+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:30.109212+0000","last_peered":"2023-06-09T11:59:30.109212+0000","last_clean":"2023-06-09T11:59:30.109212+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:30.109212+0000","last_undegraded":"2023-06-09T11:59:30.109212+0000","last_fullsized":"2023-06-09T11:59:30.109212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1061,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446248+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446248+0000","last_peered":"2023-06-09T12:00:25.446248+0000","last_clean":"2023-06-09T12:00:25.446248+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446248+0000","last_undegraded":"2023-06-09T12:00:25.446248+0000","last_fullsized":"2023-06-09T12:00:25.446248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3148,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207251+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.207251+0000","last_peered":"2023-06-09T11:59:29.207251+0000","last_clean":"2023-06-09T11:59:29.207251+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.207251+0000","last_undegraded":"2023-06-09T11:59:29.207251+0000","last_fullsized":"2023-06-09T11:59:29.207251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.364947+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:30.364947+0000","last_peered":"2023-06-09T11:59:30.364947+0000","last_clean":"2023-06-09T11:59:30.364947+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:30.364947+0000","last_undegraded":"2023-06-09T11:59:30.364947+0000","last_fullsized":"2023-06-09T11:59:30.364947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3425,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:45.601459+0000","last_change":"2023-06-09T12:00:30.528446+0000","last_active":"2023-06-09T11:59:45.601459+0000","last_peered":"2023-06-09T11:59:45.601459+0000","last_clean":"2023-06-09T11:59:45.601459+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T11:59:45.601459+0000","last_undegraded":"2023-06-09T11:59:45.601459+0000","last_fullsized":"2023-06-09T11:59:45.601459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.575027+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:29.575027+0000","last_peered":"2023-06-09T11:59:29.575027+0000","last_clean":"2023-06-09T11:59:29.575027+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:29.575027+0000","last_undegraded":"2023-06-09T11:59:29.575027+0000","last_fullsized":"2023-06-09T11:59:29.575027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.187181+0000","last_change":"2023-06-09T11:18:26.323550+0000","last_active":"2023-06-09T11:59:29.187181+0000","last_peered":"2023-06-09T11:59:29.187181+0000","last_clean":"2023-06-09T11:59:29.187181+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T11:59:29.187181+0000","last_undegraded":"2023-06-09T11:59:29.187181+0000","last_fullsized":"2023-06-09T11:59:29.187181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2978,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.181067+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:29.181067+0000","last_peered":"2023-06-09T11:59:29.181067+0000","last_clean":"2023-06-09T11:59:29.181067+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:29.181067+0000","last_undegraded":"2023-06-09T11:59:29.181067+0000","last_fullsized":"2023-06-09T11:59:29.181067+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121452+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:42.121452+0000","last_peered":"2023-06-09T11:59:42.121452+0000","last_clean":"2023-06-09T11:59:42.121452+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:42.121452+0000","last_undegraded":"2023-06-09T11:59:42.121452+0000","last_fullsized":"2023-06-09T11:59:42.121452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58502,"ondisk_log_size":58502,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":434116,"num_omap_keys":816,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30598,"ondisk_log_size":30598,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575451,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254288,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74432,"kb_avail":94117552,"statfs":{"total":96636764160,"available":96376373248,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76218368},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":0.90800000000000003,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.628},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":1.0089999999999999,"15min":1.25},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.53200000000000003,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":1.05,"15min":1.05},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.98999999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.66200000000000003},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.65900000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.54700000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.76800000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":1.044,"5min":1.044,"15min":1.0920000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.504,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.206},"last":0.73399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575451,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":202904,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46784,"kb_avail":94168936,"statfs":{"total":96636764160,"available":96428990464,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":47906816},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81599999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92200000000000004,"5min":1.069,"15min":1.069},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.53200000000000003,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99299999999999999,"15min":1.081},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.125},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.54900000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.94499999999999995,"5min":1.036,"15min":1.1719999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.0549999999999999},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.877,"5min":0.95299999999999996,"15min":1.107},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.78500000000000003,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.40799999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89800000000000002,"5min":1.052,"15min":1.0589999999999999},"last":0.505}]},{"osd":7,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.071},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834575451,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":396272,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":89856,"kb_avail":93975568,"statfs":{"total":96636764160,"available":96230981632,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":92012544},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":1.877},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.67200000000000004,"5min":0.82699999999999996,"15min":1.982},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.74399999999999999,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.501}]},{"osd":3,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70199999999999996,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.64200000000000002},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.091,"15min":1.9790000000000001},"last":0.58899999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.0649999999999999,"15min":1.609},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.081,"5min":1.1459999999999999,"15min":1.774},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.77700000000000002},"min":{"1min":0.627,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":0.96199999999999997,"5min":1.0880000000000001,"15min":1.774},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":0.99199999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.996,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.621,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.921},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0349999999999999,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.60599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95299999999999996,"5min":1.226,"15min":1.827},"last":0.72299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575451,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327656,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99840,"kb_avail":94044184,"statfs":{"total":96636764160,"available":96301244416,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":102236160},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.64100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":0.86399999999999999,"15min":1.0900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65100000000000002,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.41199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.57099999999999995,"5min":0.871,"15min":1.2909999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.59499999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.1060000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.21},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.61899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.236},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.747},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.165},"last":0.63200000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85799999999999998,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.1240000000000001,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.153,"15min":1.3700000000000001},"last":0.745}]},{"osd":7,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.159,"15min":1.228},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84299999999999997,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.58199999999999996,"15min":0.46999999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.29},"last":0.81599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575452,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236832,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57024,"kb_avail":94135008,"statfs":{"total":96636764160,"available":96394248192,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58392576},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.92300000000000004,"5min":1.367,"15min":1.367},"last":0.64600000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.96799999999999997,"5min":1.546,"15min":1.546},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":1.171,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.57999999999999996,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.1180000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.2450000000000001,"5min":1.601,"15min":1.601},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.84899999999999998,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.438},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.622},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":1.105,"5min":1.121,"15min":1.3540000000000001},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.208,"15min":1.248},"last":0.755},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83599999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.262,"5min":1.262,"15min":1.337},"last":0.83299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575451,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433340,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110144,"kb_avail":93938500,"statfs":{"total":96636764160,"available":96193024000,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112787456},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.66700000000000004,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54100000000000004,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33600000000000002,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82899999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77000000000000002,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.50800000000000001,"5min":0.42299999999999999,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.355,"15min":1.355},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90000000000000002,"5min":1.198,"15min":1.198},"last":0.623}]},{"osd":4,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.59399999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":1.635,"15min":1.635},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.751,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":0.80900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.61499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.198,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.234,"5min":1.367,"15min":1.367},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.503,"15min":0.48499999999999999},"max":{"1min":1.0569999999999999,"5min":1.286,"15min":1.286},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.434,"15min":0.434},"max":{"1min":1.0469999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.88300000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.66700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.0780000000000001,"5min":1.413,"15min":1.413},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88200000000000001,"15min":0.873},"min":{"1min":0.71399999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.046}]}]},{"osd":4,"up_from":13,"seq":55834575450,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417556,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":124800,"kb_avail":93954284,"statfs":{"total":96636764160,"available":96209186816,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":127795200},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.95799999999999996,"5min":1.137,"15min":1.137},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.63,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.874}]},{"osd":1,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":0.96799999999999997,"5min":1.0640000000000001,"15min":1.103},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.64500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.044,"5min":1.1279999999999999,"15min":1.2},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.52700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":1.609,"15min":1.609},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.997,"5min":0.997,"15min":1.1659999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.89000000000000001,"5min":1.407,"15min":1.407},"last":0.749},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.96199999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81699999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.505},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.014,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.92000000000000004,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.94199999999999995,"5min":1.032,"15min":1.032},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.97299999999999998,"5min":1.361,"15min":1.3740000000000001},"last":0.53800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575450,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288024,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70656,"kb_avail":94083816,"statfs":{"total":96636764160,"available":96341827584,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72351744},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.753,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.314},"max":{"1min":3.468,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.70499999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.68300000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.89000000000000001,"15min":0.71599999999999997},"min":{"1min":0.377,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":0.89600000000000002,"5min":19.715,"15min":19.715},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.94099999999999995,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.68600000000000005},"min":{"1min":0.53700000000000003,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.751},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.85899999999999999,"15min":0.75700000000000001},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.91300000000000003,"5min":14.439,"15min":14.439},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85999999999999999,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.433,"15min":0.375},"max":{"1min":0.94999999999999996,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88800000000000001,"15min":0.80700000000000005},"min":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":0.875,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.298,"15min":0.82599999999999996},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.3180000000000001,"15min":0.83799999999999997},"min":{"1min":0.45200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.78000000000000003,"15min":0.68700000000000006},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84999999999999998,"5min":12.163,"15min":12.163},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.64900000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.62}]},{"osd":7,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.97199999999999998,"15min":0.81499999999999995},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":19.834,"15min":19.834},"last":0.73799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:00:48.362 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.362 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.363 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.363 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.363 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.364 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.364 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.364 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.365 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.365 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.366 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.366 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.366 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.367 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.367 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.367 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.368 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.368 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.368 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.369 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.369 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.369 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.370 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.370 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.370 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.371 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.371 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.371 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.372 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.372 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.373 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.373 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.373 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.374 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.374 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.374 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.375 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.375 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.375 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.376 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.376 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.376 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.377 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.377 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.377 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.378 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.378 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.378 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.379 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.379 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.379 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.380 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.380 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.380 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.381 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.381 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.381 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.382 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.382 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.382 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.383 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.383 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.384 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.384 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.384 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.385 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.385 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.385 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.386 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.386 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.386 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.387 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.387 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.387 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.388 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.388 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.388 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.389 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.389 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.389 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.390 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.390 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.390 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.391 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.391 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.392 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.392 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.392 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.393 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.393 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.393 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.394 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.394 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.394 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:00:48.395 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:00:55.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:00:55.951 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:08.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:01:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.611+0000 7f9edfbff700 1 Processor -- start 2023-06-09T12:01:08.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.611+0000 7f9edfbff700 1 -- start start 2023-06-09T12:01:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.612+0000 7f9edfbff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c2ae0 0x7f9ed80c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.612+0000 7f9edfbff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b060 0x7f9ed8163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.612+0000 7f9edfbff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed80c9bd0 con 0x7f9ed80c88e0 2023-06-09T12:01:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.612+0000 7f9edfbff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed8159360 con 0x7f9ed815b060 2023-06-09T12:01:08.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.612+0000 7f9edfbff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed80c8e20 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b060 0x7f9ed8163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9ed815b060 0x7f9ed8163ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39154/0 (socket says 172.21.15.7:39154) 2023-06-09T12:01:08.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 -- 172.21.15.7:0/2687579418 learned_addr learned my addr 172.21.15.7:0/2687579418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:01:08.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9edd99b700 1 --2- 172.21.15.7:0/2687579418 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c2ae0 0x7f9ed80c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 -- 172.21.15.7:0/2687579418 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c2ae0 msgr2=0x7f9ed80c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:01:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 --2- 172.21.15.7:0/2687579418 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c2ae0 0x7f9ed80c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 -- 172.21.15.7:0/2687579418 >> v1:172.21.15.7:6789/0 conn(0x7f9ed80c88e0 legacy=0x7f9ed815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 -- 172.21.15.7:0/2687579418 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8159570 con 0x7f9ed815b060 2023-06-09T12:01:08.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.613+0000 7f9ede19c700 1 --2- 172.21.15.7:0/2687579418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b060 0x7f9ed8163ce0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8071720 tx=0x7f9ed000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952a1426a67beb2d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edc999700 1 -- 172.21.15.7:0/2687579418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000e070 con 0x7f9ed815b060 2023-06-09T12:01:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edc999700 1 -- 172.21.15.7:0/2687579418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed000b9c0 con 0x7f9ed815b060 2023-06-09T12:01:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edc999700 1 -- 172.21.15.7:0/2687579418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000bd70 con 0x7f9ed815b060 2023-06-09T12:01:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 -- 172.21.15.7:0/2687579418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b060 msgr2=0x7f9ed8163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 --2- 172.21.15.7:0/2687579418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b060 0x7f9ed8163ce0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8071720 tx=0x7f9ed000b330 comp rx=0 tx=0).stop 2023-06-09T12:01:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 -- 172.21.15.7:0/2687579418 shutdown_connections 2023-06-09T12:01:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 --2- 172.21.15.7:0/2687579418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed815b060 0x7f9ed8163ce0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 --2- 172.21.15.7:0/2687579418 >> v2:172.21.15.117:3300/0 conn(0x7f9ed80c2ae0 0x7f9ed80c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 -- 172.21.15.7:0/2687579418 >> 172.21.15.7:0/2687579418 conn(0x7f9ed80cd330 msgr2=0x7f9ed80cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:08.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 -- 172.21.15.7:0/2687579418 shutdown_connections 2023-06-09T12:01:08.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.614+0000 7f9edfbff700 1 -- 172.21.15.7:0/2687579418 wait complete. 2023-06-09T12:01:08.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.617+0000 7f9edfbff700 1 Processor -- start 2023-06-09T12:01:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.617+0000 7f9edfbff700 1 -- start start 2023-06-09T12:01:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed80c88e0 0x7f9ed81f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 0x7f9ed814c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81f11a0 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed81f0b70 con 0x7f9ed815b060 2023-06-09T12:01:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edfbff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8152f10 con 0x7f9ed80c88e0 2023-06-09T12:01:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edd99b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39156/0 (socket says 172.21.15.7:39156) 2023-06-09T12:01:08.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.618+0000 7f9edd99b700 1 -- 172.21.15.7:0/3331647460 learned_addr learned my addr 172.21.15.7:0/3331647460 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9ede19c700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 0x7f9ed814c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd19a700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed80c88e0 0x7f9ed81f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 -- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed80c88e0 msgr2=0x7f9ed81f1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed80c88e0 0x7f9ed81f1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 -- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 msgr2=0x7f9ed814c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 0x7f9ed814c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 -- 172.21.15.7:0/3331647460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed0009b30 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9ede19c700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 0x7f9ed814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9edd99b700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e810 tx=0x7f9ed400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.619+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4013610 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.620+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4013770 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.620+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8159af0 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.620+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4013b20 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.620+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed8159810 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.621+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed8159dd0 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.622+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ed4013c80 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.623+0000 7f9ece7fc700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 0x7f9ebc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:08.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.623+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f9ed4089a80 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.623+0000 7f9edd19a700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 0x7f9ebc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.623+0000 7f9edd19a700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 0x7f9ebc06d1c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9ed814d630 tx=0x7f9ec8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:08.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.627+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9ed40541c0 con 0x7f9ed80c2ae0 2023-06-09T12:01:08.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.834+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ed81f1480 con 0x7f9ebc06ad10 2023-06-09T12:01:08.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.843+0000 7f9ece7fc700 1 -- 172.21.15.7:0/3331647460 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+313841 (secure 0 0 0) 0x7f9ed81f1480 con 0x7f9ebc06ad10 2023-06-09T12:01:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:01:08.851 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 msgr2=0x7f9ebc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 0x7f9ebc06d1c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9ed814d630 tx=0x7f9ec8009040 comp rx=0 tx=0).stop 2023-06-09T12:01:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 msgr2=0x7f9ed81f1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e810 tx=0x7f9ed400ebd0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 shutdown_connections 2023-06-09T12:01:08.852 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9ed815b060 0x7f9ed814c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9ed80c2ae0 0x7f9ed81f1630 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9ebc06ad10 0x7f9ebc06d1c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 --2- 172.21.15.7:0/3331647460 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9ed80c88e0 0x7f9ed81f1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.847+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 >> 172.21.15.7:0/3331647460 conn(0x7f9ed80cd330 msgr2=0x7f9ed80cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:08.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.848+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 shutdown_connections 2023-06-09T12:01:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:08.848+0000 7f9edfbff700 1 -- 172.21.15.7:0/3331647460 wait complete. 2023-06-09T12:01:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:01:08.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1580,"stamp":"2023-06-09T12:01:06.949309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568122,"num_omap_keys":1070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89563,"ondisk_log_size":89563,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2561800,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":678464,"kb_avail":752412920,"statfs":{"total":773094113280,"available":770470830080,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":694747136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003292"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3464,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.954869+0000","last_change":"2023-06-09T12:00:52.376091+0000","last_active":"2023-06-09T12:00:46.954869+0000","last_peered":"2023-06-09T12:00:46.954869+0000","last_clean":"2023-06-09T12:00:46.954869+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:46.954869+0000","last_undegraded":"2023-06-09T12:00:46.954869+0000","last_fullsized":"2023-06-09T12:00:46.954869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905267+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:44.905267+0000","last_peered":"2023-06-09T11:59:44.905267+0000","last_clean":"2023-06-09T11:59:44.905267+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:44.905267+0000","last_undegraded":"2023-06-09T11:59:44.905267+0000","last_fullsized":"2023-06-09T11:59:44.905267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3452,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:45.478589+0000","last_change":"2023-06-09T12:01:02.493236+0000","last_active":"2023-06-09T12:00:45.478589+0000","last_peered":"2023-06-09T12:00:45.478589+0000","last_clean":"2023-06-09T12:00:45.478589+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:00:45.478589+0000","last_undegraded":"2023-06-09T12:00:45.478589+0000","last_fullsized":"2023-06-09T12:00:45.478589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'921","reported_seq":3143,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:00.842633+0000","last_change":"2023-06-09T12:00:55.982660+0000","last_active":"2023-06-09T12:01:00.842633+0000","last_peered":"2023-06-09T12:01:00.842633+0000","last_clean":"2023-06-09T12:01:00.842633+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:00.842633+0000","last_undegraded":"2023-06-09T12:01:00.842633+0000","last_fullsized":"2023-06-09T12:01:00.842633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":48,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'868","reported_seq":3245,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905258+0000","last_change":"2023-06-09T11:18:26.262768+0000","last_active":"2023-06-09T11:59:44.905258+0000","last_peered":"2023-06-09T11:59:44.905258+0000","last_clean":"2023-06-09T11:59:44.905258+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T11:59:44.905258+0000","last_undegraded":"2023-06-09T11:59:44.905258+0000","last_fullsized":"2023-06-09T11:59:44.905258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.163620+0000","last_change":"2023-06-09T11:18:26.272389+0000","last_active":"2023-06-09T11:59:29.163620+0000","last_peered":"2023-06-09T11:59:29.163620+0000","last_clean":"2023-06-09T11:59:29.163620+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:29.163620+0000","last_undegraded":"2023-06-09T11:59:29.163620+0000","last_fullsized":"2023-06-09T11:59:29.163620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184811+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184811+0000","last_peered":"2023-06-09T11:59:29.184811+0000","last_clean":"2023-06-09T11:59:29.184811+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184811+0000","last_undegraded":"2023-06-09T11:59:29.184811+0000","last_fullsized":"2023-06-09T11:59:29.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3401,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821600+0000","last_change":"2023-06-09T11:59:49.712515+0000","last_active":"2023-06-09T11:59:42.821600+0000","last_peered":"2023-06-09T11:59:42.821600+0000","last_clean":"2023-06-09T11:59:42.821600+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:42.821600+0000","last_undegraded":"2023-06-09T11:59:42.821600+0000","last_fullsized":"2023-06-09T11:59:42.821600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3231,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821561+0000","last_change":"2023-06-09T11:59:48.700631+0000","last_active":"2023-06-09T11:59:42.821561+0000","last_peered":"2023-06-09T11:59:42.821561+0000","last_clean":"2023-06-09T11:59:42.821561+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:42.821561+0000","last_undegraded":"2023-06-09T11:59:42.821561+0000","last_fullsized":"2023-06-09T11:59:42.821561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1001","reported_seq":3685,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:01.762801+0000","last_change":"2023-06-09T12:00:47.970957+0000","last_active":"2023-06-09T12:01:01.762801+0000","last_peered":"2023-06-09T12:01:01.762801+0000","last_clean":"2023-06-09T12:01:01.762801+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:01.762801+0000","last_undegraded":"2023-06-09T12:01:01.762801+0000","last_fullsized":"2023-06-09T12:01:01.762801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":121,"log_size":1001,"log_dups_size":0,"ondisk_log_size":1001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296222+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:46.296222+0000","last_peered":"2023-06-09T11:59:46.296222+0000","last_clean":"2023-06-09T11:59:46.296222+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:46.296222+0000","last_undegraded":"2023-06-09T11:59:46.296222+0000","last_fullsized":"2023-06-09T11:59:46.296222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423589+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:41.423589+0000","last_peered":"2023-06-09T11:59:41.423589+0000","last_clean":"2023-06-09T11:59:41.423589+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:41.423589+0000","last_undegraded":"2023-06-09T11:59:41.423589+0000","last_fullsized":"2023-06-09T11:59:41.423589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'958","reported_seq":3586,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:59.563982+0000","last_change":"2023-06-09T12:00:54.677013+0000","last_active":"2023-06-09T12:00:59.563982+0000","last_peered":"2023-06-09T12:00:59.563982+0000","last_clean":"2023-06-09T12:00:59.563982+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:00:59.563982+0000","last_undegraded":"2023-06-09T12:00:59.563982+0000","last_fullsized":"2023-06-09T12:00:59.563982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":49,"log_size":958,"log_dups_size":0,"ondisk_log_size":958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3018,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121483+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:42.121483+0000","last_peered":"2023-06-09T11:59:42.121483+0000","last_clean":"2023-06-09T11:59:42.121483+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:42.121483+0000","last_undegraded":"2023-06-09T11:59:42.121483+0000","last_fullsized":"2023-06-09T11:59:42.121483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905252+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:44.905252+0000","last_peered":"2023-06-09T11:59:44.905252+0000","last_clean":"2023-06-09T11:59:44.905252+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:44.905252+0000","last_undegraded":"2023-06-09T11:59:44.905252+0000","last_fullsized":"2023-06-09T11:59:44.905252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207728+0000","last_change":"2023-06-09T11:10:49.684108+0000","last_active":"2023-06-09T11:59:44.207728+0000","last_peered":"2023-06-09T11:59:44.207728+0000","last_clean":"2023-06-09T11:59:44.207728+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T11:59:44.207728+0000","last_undegraded":"2023-06-09T11:59:44.207728+0000","last_fullsized":"2023-06-09T11:59:44.207728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517078+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:43.517078+0000","last_peered":"2023-06-09T11:59:43.517078+0000","last_clean":"2023-06-09T11:59:43.517078+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:43.517078+0000","last_undegraded":"2023-06-09T11:59:43.517078+0000","last_fullsized":"2023-06-09T11:59:43.517078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3135,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423662+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:41.423662+0000","last_peered":"2023-06-09T11:59:41.423662+0000","last_clean":"2023-06-09T11:59:41.423662+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:41.423662+0000","last_undegraded":"2023-06-09T11:59:41.423662+0000","last_fullsized":"2023-06-09T11:59:41.423662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1997,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:43.516919+0000","last_change":"2023-06-09T12:00:33.899239+0000","last_active":"2023-06-09T11:59:43.516919+0000","last_peered":"2023-06-09T11:59:43.516919+0000","last_clean":"2023-06-09T11:59:43.516919+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:43.516919+0000","last_undegraded":"2023-06-09T11:59:43.516919+0000","last_fullsized":"2023-06-09T11:59:43.516919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:28.569063+0000","last_change":"2023-06-09T12:00:52.491920+0000","last_active":"2023-06-09T12:00:28.569063+0000","last_peered":"2023-06-09T12:00:28.569063+0000","last_clean":"2023-06-09T12:00:28.569063+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:00:28.569063+0000","last_undegraded":"2023-06-09T12:00:28.569063+0000","last_fullsized":"2023-06-09T12:00:28.569063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":347,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:50.067505+0000","last_change":"2023-06-09T11:59:59.471359+0000","last_active":"2023-06-09T11:59:50.067505+0000","last_peered":"2023-06-09T11:59:50.067505+0000","last_clean":"2023-06-09T11:59:50.067505+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:50.067505+0000","last_undegraded":"2023-06-09T11:59:50.067505+0000","last_fullsized":"2023-06-09T11:59:50.067505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2963,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:52.293286+0000","last_change":"2023-06-09T12:00:36.885523+0000","last_active":"2023-06-09T11:59:52.293286+0000","last_peered":"2023-06-09T11:59:52.293286+0000","last_clean":"2023-06-09T11:59:52.293286+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:52.293286+0000","last_undegraded":"2023-06-09T11:59:52.293286+0000","last_fullsized":"2023-06-09T11:59:52.293286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1264,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296204+0000","last_change":"2023-06-09T12:00:19.292684+0000","last_active":"2023-06-09T11:59:46.296204+0000","last_peered":"2023-06-09T11:59:46.296204+0000","last_clean":"2023-06-09T11:59:46.296204+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:46.296204+0000","last_undegraded":"2023-06-09T11:59:46.296204+0000","last_fullsized":"2023-06-09T11:59:46.296204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1034,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:49.057672+0000","last_change":"2023-06-09T11:10:49.677578+0000","last_active":"2023-06-09T11:59:49.057672+0000","last_peered":"2023-06-09T11:59:49.057672+0000","last_clean":"2023-06-09T11:59:49.057672+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T11:59:49.057672+0000","last_undegraded":"2023-06-09T11:59:49.057672+0000","last_fullsized":"2023-06-09T11:59:49.057672+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1082,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:55.067574+0000","last_change":"2023-06-09T12:00:45.031530+0000","last_active":"2023-06-09T11:59:55.067574+0000","last_peered":"2023-06-09T11:59:55.067574+0000","last_clean":"2023-06-09T11:59:55.067574+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:55.067574+0000","last_undegraded":"2023-06-09T11:59:55.067574+0000","last_fullsized":"2023-06-09T11:59:55.067574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3559,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.294703+0000","last_change":"2023-06-09T12:00:41.875998+0000","last_active":"2023-06-09T12:00:02.294703+0000","last_peered":"2023-06-09T12:00:02.294703+0000","last_clean":"2023-06-09T12:00:02.294703+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:02.294703+0000","last_undegraded":"2023-06-09T12:00:02.294703+0000","last_fullsized":"2023-06-09T12:00:02.294703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3586,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.116172+0000","last_change":"2023-06-09T11:18:26.272670+0000","last_active":"2023-06-09T11:59:29.116172+0000","last_peered":"2023-06-09T11:59:29.116172+0000","last_clean":"2023-06-09T11:59:29.116172+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:29.116172+0000","last_undegraded":"2023-06-09T11:59:29.116172+0000","last_fullsized":"2023-06-09T11:59:29.116172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:25.604660+0000","last_change":"2023-06-09T12:00:25.604660+0000","last_active":"2023-06-09T12:00:25.604660+0000","last_peered":"2023-06-09T12:00:25.604660+0000","last_clean":"2023-06-09T12:00:25.604660+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:00:25.604660+0000","last_undegraded":"2023-06-09T12:00:25.604660+0000","last_fullsized":"2023-06-09T12:00:25.604660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3763,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203922+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203922+0000","last_peered":"2023-06-09T11:59:29.203922+0000","last_clean":"2023-06-09T11:59:29.203922+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203922+0000","last_undegraded":"2023-06-09T11:59:29.203922+0000","last_fullsized":"2023-06-09T11:59:29.203922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":728,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:27.395619+0000","last_change":"2023-06-09T12:00:46.032468+0000","last_active":"2023-06-09T12:00:27.395619+0000","last_peered":"2023-06-09T12:00:27.395619+0000","last_clean":"2023-06-09T12:00:27.395619+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:27.395619+0000","last_undegraded":"2023-06-09T12:00:27.395619+0000","last_fullsized":"2023-06-09T12:00:27.395619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905230+0000","last_change":"2023-06-09T11:18:26.271872+0000","last_active":"2023-06-09T11:59:44.905230+0000","last_peered":"2023-06-09T11:59:44.905230+0000","last_clean":"2023-06-09T11:59:44.905230+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T11:59:44.905230+0000","last_undegraded":"2023-06-09T11:59:44.905230+0000","last_fullsized":"2023-06-09T11:59:44.905230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394854+0000","last_change":"2023-06-09T12:00:47.993494+0000","last_active":"2023-06-09T12:00:22.394854+0000","last_peered":"2023-06-09T12:00:22.394854+0000","last_clean":"2023-06-09T12:00:22.394854+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:22.394854+0000","last_undegraded":"2023-06-09T12:00:22.394854+0000","last_fullsized":"2023-06-09T12:00:22.394854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394838+0000","last_change":"2023-06-09T12:00:53.983699+0000","last_active":"2023-06-09T12:00:22.394838+0000","last_peered":"2023-06-09T12:00:22.394838+0000","last_clean":"2023-06-09T12:00:22.394838+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:22.394838+0000","last_undegraded":"2023-06-09T12:00:22.394838+0000","last_fullsized":"2023-06-09T12:00:22.394838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:35.635944+0000","last_change":"2023-06-09T12:00:52.491865+0000","last_active":"2023-06-09T12:00:35.635944+0000","last_peered":"2023-06-09T12:00:35.635944+0000","last_clean":"2023-06-09T12:00:35.635944+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:00:35.635944+0000","last_undegraded":"2023-06-09T12:00:35.635944+0000","last_fullsized":"2023-06-09T12:00:35.635944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445722+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445722+0000","last_peered":"2023-06-09T12:00:04.445722+0000","last_clean":"2023-06-09T12:00:04.445722+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445722+0000","last_undegraded":"2023-06-09T12:00:04.445722+0000","last_fullsized":"2023-06-09T12:00:04.445722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207708+0000","last_change":"2023-06-09T11:10:49.685742+0000","last_active":"2023-06-09T11:59:44.207708+0000","last_peered":"2023-06-09T11:59:44.207708+0000","last_clean":"2023-06-09T11:59:44.207708+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T11:59:44.207708+0000","last_undegraded":"2023-06-09T11:59:44.207708+0000","last_fullsized":"2023-06-09T11:59:44.207708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121371+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:42.121371+0000","last_peered":"2023-06-09T11:59:42.121371+0000","last_clean":"2023-06-09T11:59:42.121371+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:42.121371+0000","last_undegraded":"2023-06-09T11:59:42.121371+0000","last_fullsized":"2023-06-09T11:59:42.121371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.246824+0000","last_change":"2023-06-09T12:00:47.290422+0000","last_active":"2023-06-09T12:00:17.246824+0000","last_peered":"2023-06-09T12:00:17.246824+0000","last_clean":"2023-06-09T12:00:17.246824+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:17.246824+0000","last_undegraded":"2023-06-09T12:00:17.246824+0000","last_fullsized":"2023-06-09T12:00:17.246824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":897,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207703+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:44.207703+0000","last_peered":"2023-06-09T11:59:44.207703+0000","last_clean":"2023-06-09T11:59:44.207703+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:44.207703+0000","last_undegraded":"2023-06-09T11:59:44.207703+0000","last_fullsized":"2023-06-09T11:59:44.207703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3889,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121324+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:42.121324+0000","last_peered":"2023-06-09T11:59:42.121324+0000","last_clean":"2023-06-09T11:59:42.121324+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:42.121324+0000","last_undegraded":"2023-06-09T11:59:42.121324+0000","last_fullsized":"2023-06-09T11:59:42.121324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121240+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:42.121240+0000","last_peered":"2023-06-09T11:59:42.121240+0000","last_clean":"2023-06-09T11:59:42.121240+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:42.121240+0000","last_undegraded":"2023-06-09T11:59:42.121240+0000","last_fullsized":"2023-06-09T11:59:42.121240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121335+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:42.121335+0000","last_peered":"2023-06-09T11:59:42.121335+0000","last_clean":"2023-06-09T11:59:42.121335+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:42.121335+0000","last_undegraded":"2023-06-09T11:59:42.121335+0000","last_fullsized":"2023-06-09T11:59:42.121335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.338006+0000","last_change":"2023-06-09T12:00:47.290438+0000","last_active":"2023-06-09T12:00:15.338006+0000","last_peered":"2023-06-09T12:00:15.338006+0000","last_clean":"2023-06-09T12:00:15.338006+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:00:15.338006+0000","last_undegraded":"2023-06-09T12:00:15.338006+0000","last_fullsized":"2023-06-09T12:00:15.338006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1347,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602275+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602275+0000","last_peered":"2023-06-09T12:00:03.602275+0000","last_clean":"2023-06-09T12:00:03.602275+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602275+0000","last_undegraded":"2023-06-09T12:00:03.602275+0000","last_fullsized":"2023-06-09T12:00:03.602275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3298,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.542755+0000","last_change":"2023-06-09T12:01:02.493261+0000","last_active":"2023-06-09T12:00:32.542755+0000","last_peered":"2023-06-09T12:00:32.542755+0000","last_clean":"2023-06-09T12:00:32.542755+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:00:32.542755+0000","last_undegraded":"2023-06-09T12:00:32.542755+0000","last_fullsized":"2023-06-09T12:00:32.542755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:47.289624+0000","last_change":"2023-06-09T12:00:52.398167+0000","last_active":"2023-06-09T11:59:47.289624+0000","last_peered":"2023-06-09T11:59:47.289624+0000","last_clean":"2023-06-09T11:59:47.289624+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:47.289624+0000","last_undegraded":"2023-06-09T11:59:47.289624+0000","last_fullsized":"2023-06-09T11:59:47.289624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423608+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:41.423608+0000","last_peered":"2023-06-09T11:59:41.423608+0000","last_clean":"2023-06-09T11:59:41.423608+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:41.423608+0000","last_undegraded":"2023-06-09T11:59:41.423608+0000","last_fullsized":"2023-06-09T11:59:41.423608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":821,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121243+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:42.121243+0000","last_peered":"2023-06-09T11:59:42.121243+0000","last_clean":"2023-06-09T11:59:42.121243+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:42.121243+0000","last_undegraded":"2023-06-09T11:59:42.121243+0000","last_fullsized":"2023-06-09T11:59:42.121243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4112,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423567+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:41.423567+0000","last_peered":"2023-06-09T11:59:41.423567+0000","last_clean":"2023-06-09T11:59:41.423567+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:41.423567+0000","last_undegraded":"2023-06-09T11:59:41.423567+0000","last_fullsized":"2023-06-09T11:59:41.423567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.652148+0000","last_change":"2023-06-09T12:00:37.490252+0000","last_active":"2023-06-09T12:00:36.652148+0000","last_peered":"2023-06-09T12:00:36.652148+0000","last_clean":"2023-06-09T12:00:36.652148+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:36.652148+0000","last_undegraded":"2023-06-09T12:00:36.652148+0000","last_fullsized":"2023-06-09T12:00:36.652148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423433+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:41.423433+0000","last_peered":"2023-06-09T11:59:41.423433+0000","last_clean":"2023-06-09T11:59:41.423433+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:41.423433+0000","last_undegraded":"2023-06-09T11:59:41.423433+0000","last_fullsized":"2023-06-09T11:59:41.423433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":881,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423490+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:41.423490+0000","last_peered":"2023-06-09T11:59:41.423490+0000","last_clean":"2023-06-09T11:59:41.423490+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:41.423490+0000","last_undegraded":"2023-06-09T11:59:41.423490+0000","last_fullsized":"2023-06-09T11:59:41.423490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'588","reported_seq":786,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:53.580843+0000","last_change":"2023-06-09T12:00:53.208468+0000","last_active":"2023-06-09T12:00:53.580843+0000","last_peered":"2023-06-09T12:00:53.580843+0000","last_clean":"2023-06-09T12:00:53.580843+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:00:53.580843+0000","last_undegraded":"2023-06-09T12:00:53.580843+0000","last_fullsized":"2023-06-09T12:00:53.580843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":588,"log_dups_size":0,"ondisk_log_size":588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1061,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446248+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446248+0000","last_peered":"2023-06-09T12:00:25.446248+0000","last_clean":"2023-06-09T12:00:25.446248+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446248+0000","last_undegraded":"2023-06-09T12:00:25.446248+0000","last_fullsized":"2023-06-09T12:00:25.446248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3148,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207251+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.207251+0000","last_peered":"2023-06-09T11:59:29.207251+0000","last_clean":"2023-06-09T11:59:29.207251+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.207251+0000","last_undegraded":"2023-06-09T11:59:29.207251+0000","last_fullsized":"2023-06-09T11:59:29.207251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.364947+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:30.364947+0000","last_peered":"2023-06-09T11:59:30.364947+0000","last_clean":"2023-06-09T11:59:30.364947+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:30.364947+0000","last_undegraded":"2023-06-09T11:59:30.364947+0000","last_fullsized":"2023-06-09T11:59:30.364947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3426,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:30.528446+0000","last_change":"2023-06-09T12:00:52.491822+0000","last_active":"2023-06-09T12:00:30.528446+0000","last_peered":"2023-06-09T12:00:30.528446+0000","last_clean":"2023-06-09T12:00:30.528446+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:00:30.528446+0000","last_undegraded":"2023-06-09T12:00:30.528446+0000","last_fullsized":"2023-06-09T12:00:30.528446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121452+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:42.121452+0000","last_peered":"2023-06-09T11:59:42.121452+0000","last_clean":"2023-06-09T11:59:42.121452+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:42.121452+0000","last_undegraded":"2023-06-09T11:59:42.121452+0000","last_fullsized":"2023-06-09T11:59:42.121452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":58816,"ondisk_log_size":58816,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":568122,"num_omap_keys":1070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30628,"ondisk_log_size":30628,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575455,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254352,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74496,"kb_avail":94117488,"statfs":{"total":96636764160,"available":96376307712,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76283904},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":0.90800000000000003,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":1.256}]},{"osd":1,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":1.0089999999999999,"15min":1.25},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.53200000000000003,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":1.05,"15min":1.05},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.879},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.98999999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":1.3089999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.66200000000000003},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.80700000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.54700000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.872}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.76800000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":1.044,"5min":1.044,"15min":1.0920000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.504,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.206},"last":1.2869999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203032,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46912,"kb_avail":94168808,"statfs":{"total":96636764160,"available":96428859392,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48037888},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81599999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92200000000000004,"5min":1.069,"15min":1.069},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.53200000000000003,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99299999999999999,"15min":1.081},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.628}]},{"osd":2,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.125},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.54900000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.94499999999999995,"5min":1.036,"15min":1.1719999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.0549999999999999},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.877,"5min":0.95299999999999996,"15min":1.107},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.78500000000000003,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.40799999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89800000000000002,"5min":1.052,"15min":1.0589999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.071},"last":0.753}]}]},{"osd":1,"up_from":13,"seq":55834575455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400496,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94080,"kb_avail":93971344,"statfs":{"total":96636764160,"available":96226656256,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96337920},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":1.877},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.67200000000000004,"5min":0.82699999999999996,"15min":1.982},"last":0.505},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.74399999999999999,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70199999999999996,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.64200000000000002},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.091,"15min":1.9790000000000001},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.0649999999999999,"15min":1.609},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.081,"5min":1.1459999999999999,"15min":1.774},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.77700000000000002},"min":{"1min":0.627,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":0.96199999999999997,"5min":1.0880000000000001,"15min":1.774},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":0.99199999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.996,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.621,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.921},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0349999999999999,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.60599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95299999999999996,"5min":1.226,"15min":1.827},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327784,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99968,"kb_avail":94044056,"statfs":{"total":96636764160,"available":96301113344,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":102367232},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.433,"15min":0.441},"min":{"1min":0.28199999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.66100000000000003,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.64100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55700000000000005,"15min":0.52900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":0.70799999999999996,"5min":0.86399999999999999,"15min":1.0900000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48099999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65100000000000002,"5min":0.76100000000000001,"15min":1.0289999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.54600000000000004,"15min":0.57199999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.57099999999999995,"5min":0.871,"15min":1.2909999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.68899999999999995,"15min":0.70199999999999996},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.82099999999999995,"5min":1.0109999999999999,"15min":1.3280000000000001},"last":0.71099999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.67900000000000005},"min":{"1min":0.59499999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.1060000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77400000000000002,"15min":0.75800000000000001},"min":{"1min":0.64800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.0029999999999999,"5min":1.0600000000000001,"15min":1.21},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.61899999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.236},"last":0.748},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.747,"15min":0.747},"min":{"1min":0.64200000000000002,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":0.97999999999999998,"5min":1.0369999999999999,"15min":1.165},"last":0.84499999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85799999999999998,"15min":0.80400000000000005},"min":{"1min":0.505,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.1240000000000001,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.83699999999999997},"min":{"1min":0.625,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0449999999999999,"5min":1.153,"15min":1.3700000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:00:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.0900000000000001,"5min":1.159,"15min":1.228},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.88700000000000001,"5min":0.84299999999999997,"15min":0.80300000000000005},"min":{"1min":0.60999999999999999,"5min":0.58199999999999996,"15min":0.46999999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.29},"last":0.876}]}]},{"osd":2,"up_from":13,"seq":55834575456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237024,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57216,"kb_avail":94134816,"statfs":{"total":96636764160,"available":96394051584,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58589184},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.92300000000000004,"5min":1.367,"15min":1.367},"last":0.52000000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.437,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.96799999999999997,"5min":1.546,"15min":1.546},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64100000000000001,"15min":0.60099999999999998},"min":{"1min":0.48699999999999999,"5min":0.44,"15min":0.378},"max":{"1min":0.98899999999999999,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.51200000000000001,"5min":0.42299999999999999,"15min":0.373},"max":{"1min":1.171,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.57999999999999996,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":1.1180000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80100000000000005,"15min":0.78100000000000003},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":1.2450000000000001,"5min":1.601,"15min":1.601},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.84899999999999998,"15min":0.82199999999999995},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.288,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.438},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.59999999999999998,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":1.105,"5min":1.121,"15min":1.3540000000000001},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:00:18 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.53600000000000003,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":1.149,"5min":1.208,"15min":1.248},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.83599999999999997,"15min":0.81299999999999994},"min":{"1min":0.59699999999999998,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.262,"5min":1.262,"15min":1.337},"last":0.67500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575455,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433404,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110208,"kb_avail":93938436,"statfs":{"total":96636764160,"available":96192958464,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112852992},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:22 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47199999999999998,"15min":0.45200000000000001},"min":{"1min":0.32600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.66700000000000004,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":0.376},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54100000000000004,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33600000000000002,"15min":0.30599999999999999},"max":{"1min":0.78100000000000003,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.48499999999999999,"15min":0.48199999999999998},"min":{"1min":0.378,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82899999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.49399999999999999,"15min":0.51600000000000001},"min":{"1min":0.35899999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.77000000000000002,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.48699999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.50800000000000001,"5min":0.42299999999999999,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.355,"15min":1.355},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.54600000000000004},"min":{"1min":0.34100000000000003,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90000000000000002,"5min":1.198,"15min":1.198},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.59399999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.97199999999999998,"5min":1.635,"15min":1.635},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.751,"15min":0.73399999999999999},"min":{"1min":0.57599999999999996,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":3.081,"5min":3.081,"15min":3.081},"last":0.81399999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.77300000000000002,"15min":0.748},"min":{"1min":0.61499999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.198,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.79700000000000004,"15min":0.78800000000000003},"min":{"1min":0.52500000000000002,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.234,"5min":1.367,"15min":1.367},"last":0.77900000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77800000000000002,"15min":0.78700000000000003},"min":{"1min":0.63400000000000001,"5min":0.503,"15min":0.48499999999999999},"max":{"1min":1.0569999999999999,"5min":1.286,"15min":1.286},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.60099999999999998,"5min":0.434,"15min":0.434},"max":{"1min":1.0469999999999999,"5min":3.3490000000000002,"15min":3.3490000000000002},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:00:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.66700000000000004,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.0780000000000001,"5min":1.413,"15min":1.413},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.88200000000000001,"15min":0.873},"min":{"1min":0.71399999999999997,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.79200000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575454,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417620,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":124864,"kb_avail":93954220,"statfs":{"total":96636764160,"available":96209121280,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":127860736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.65500000000000003},"min":{"1min":0.46400000000000002,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.95799999999999996,"5min":1.137,"15min":1.137},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.63,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":1.0009999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.75700000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.501,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":0.96799999999999997,"5min":1.0640000000000001,"15min":1.103},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76000000000000001,"15min":0.749},"min":{"1min":0.64500000000000002,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.044,"5min":1.1279999999999999,"15min":1.2},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64700000000000002,"15min":0.63200000000000001},"min":{"1min":0.52700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.93500000000000005,"5min":1.609,"15min":1.609},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.997,"5min":0.997,"15min":1.1659999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.67700000000000005},"min":{"1min":0.47999999999999998,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.89000000000000001,"5min":1.407,"15min":1.407},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.504,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.96199999999999997,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.41899999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.95999999999999996,"5min":1.0189999999999999,"15min":1.1439999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.627,"15min":0.61199999999999999},"min":{"1min":0.48299999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":1.014,"5min":1.1719999999999999,"15min":1.2569999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.57299999999999995,"15min":0.56399999999999995},"min":{"1min":0.439,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.70499999999999996,"15min":0.69999999999999996},"min":{"1min":0.58899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.92000000000000004,"5min":1.3100000000000001,"15min":1.3260000000000001},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:00:02 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.45000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.94199999999999995,"5min":1.032,"15min":1.032},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.65700000000000003},"min":{"1min":0.50600000000000001,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.97299999999999998,"5min":1.361,"15min":1.3740000000000001},"last":0.53300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575454,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288088,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70720,"kb_avail":94083752,"statfs":{"total":96636764160,"available":96341762048,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72417280},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.753,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.314},"max":{"1min":3.468,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.70499999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.89000000000000001,"15min":0.71599999999999997},"min":{"1min":0.377,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":0.89600000000000002,"5min":19.715,"15min":19.715},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.94099999999999995,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.629}]},{"osd":2,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.68600000000000005},"min":{"1min":0.53700000000000003,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.85899999999999999,"15min":0.75700000000000001},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.91300000000000003,"5min":14.439,"15min":14.439},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85999999999999999,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.433,"15min":0.375},"max":{"1min":0.94999999999999996,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88800000000000001,"15min":0.80700000000000005},"min":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":0.875,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.298,"15min":0.82599999999999996},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.3180000000000001,"15min":0.83799999999999997},"min":{"1min":0.45200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.78000000000000003,"15min":0.68700000000000006},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84999999999999998,"5min":12.163,"15min":12.163},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.64900000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.97199999999999998,"15min":0.81499999999999995},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":19.834,"15min":19.834},"last":0.65700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:01:08.876 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.877 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.877 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.877 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.878 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.878 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.878 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.879 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.879 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.879 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.880 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.880 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.880 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.881 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.881 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.882 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.882 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.882 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.883 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.883 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.883 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.884 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.884 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.884 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.885 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.885 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.885 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.886 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.886 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.886 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.887 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.887 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.887 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.888 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.888 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.888 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.889 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.889 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.889 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.890 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.890 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.890 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.891 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.891 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.891 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.892 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.892 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.892 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.893 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.894 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.895 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.895 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.896 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.896 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.897 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.897 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.898 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.898 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.898 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.899 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.899 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.899 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.900 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.900 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.900 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.901 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.901 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.901 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.902 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.902 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.902 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.903 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.903 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.903 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.904 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.904 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.905 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.905 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.905 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.906 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.906 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.906 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.907 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.907 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.907 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.908 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.908 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.908 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.909 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.909 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:08.909 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:01:26.126 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:26.130 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:28.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:01:29.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.123+0000 7fdb53bff700 1 Processor -- start 2023-06-09T12:01:29.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.124+0000 7fdb53bff700 1 -- start start 2023-06-09T12:01:29.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.125+0000 7fdb53bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb53bff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c15cec0 con 0x7fdb4c0c9fe0 2023-06-09T12:01:29.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb53bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4c15d160 con 0x7fdb4c0c9090 2023-06-09T12:01:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb53bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0ca440 con 0x7fdb4c1598b0 2023-06-09T12:01:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb5119a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdb4c0c9fe0 0x7fdb4c0cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46476/0 (socket says 172.21.15.7:46476) 2023-06-09T12:01:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb5119a700 1 -- 172.21.15.7:0/87745514 learned_addr learned my addr 172.21.15.7:0/87745514 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:01:29.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.126+0000 7fdb5219c700 1 --2- 172.21.15.7:0/87745514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:29.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb50999700 1 -- 172.21.15.7:0/87745514 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137709389 0 0) 0x7fdb4c15cec0 con 0x7fdb4c0c9fe0 2023-06-09T12:01:29.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb50999700 1 -- 172.21.15.7:0/87745514 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb380035b0 con 0x7fdb4c0c9fe0 2023-06-09T12:01:29.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb5219c700 1 -- 172.21.15.7:0/87745514 >> v1:172.21.15.117:6789/0 conn(0x7fdb4c0c9090 legacy=0x7fdb4c0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb5219c700 1 -- 172.21.15.7:0/87745514 >> v1:172.21.15.7:6789/0 conn(0x7fdb4c0c9fe0 legacy=0x7fdb4c0cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb5219c700 1 -- 172.21.15.7:0/87745514 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4c15d440 con 0x7fdb4c1598b0 2023-06-09T12:01:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.127+0000 7fdb5219c700 1 --2- 172.21.15.7:0/87745514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c159d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009dc0 tx=0x7fdb4400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d69fc957ce3ba814 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.128+0000 7fdb50999700 1 -- 172.21.15.7:0/87745514 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400e070 con 0x7fdb4c1598b0 2023-06-09T12:01:29.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.128+0000 7fdb50999700 1 -- 172.21.15.7:0/87745514 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4400bae0 con 0x7fdb4c1598b0 2023-06-09T12:01:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.128+0000 7fdb50999700 1 -- 172.21.15.7:0/87745514 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400be90 con 0x7fdb4c1598b0 2023-06-09T12:01:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.128+0000 7fdb53bff700 1 -- 172.21.15.7:0/87745514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 msgr2=0x7fdb4c159d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.128+0000 7fdb53bff700 1 --2- 172.21.15.7:0/87745514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c159d30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009dc0 tx=0x7fdb4400b330 comp rx=0 tx=0).stop 2023-06-09T12:01:29.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.129+0000 7fdb53bff700 1 -- 172.21.15.7:0/87745514 shutdown_connections 2023-06-09T12:01:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.129+0000 7fdb53bff700 1 --2- 172.21.15.7:0/87745514 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c159d30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.129+0000 7fdb53bff700 1 -- 172.21.15.7:0/87745514 >> 172.21.15.7:0/87745514 conn(0x7fdb4c070af0 msgr2=0x7fdb4c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.129+0000 7fdb53bff700 1 -- 172.21.15.7:0/87745514 shutdown_connections 2023-06-09T12:01:29.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.129+0000 7fdb53bff700 1 -- 172.21.15.7:0/87745514 wait complete. 2023-06-09T12:01:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.130+0000 7fdb53bff700 1 Processor -- start 2023-06-09T12:01:29.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.130+0000 7fdb53bff700 1 -- start start 2023-06-09T12:01:29.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:29.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c0c9fe0 0x7fdb4c15bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:29.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c1f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:29.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c15c3a0 con 0x7fdb4c0c9090 2023-06-09T12:01:29.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0cd560 con 0x7fdb4c0c9fe0 2023-06-09T12:01:29.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.131+0000 7fdb53bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4c0ce520 con 0x7fdb4c1598b0 2023-06-09T12:01:29.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:29.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36442/0 (socket says 172.21.15.7:36442) 2023-06-09T12:01:29.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 -- 172.21.15.7:0/1715118499 learned_addr learned my addr 172.21.15.7:0/1715118499 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:01:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5219c700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c1f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5119a700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c0c9fe0 0x7fdb4c15bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 -- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 msgr2=0x7fdb4c1f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c1f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 -- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c0c9fe0 msgr2=0x7fdb4c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c0c9fe0 0x7fdb4c15bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 -- 172.21.15.7:0/1715118499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009a70 con 0x7fdb4c0c9090 2023-06-09T12:01:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.132+0000 7fdb5199b700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c4d0 tx=0x7fdb4800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.133+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb480052d0 con 0x7fdb4c0c9090 2023-06-09T12:01:29.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.133+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb48005430 con 0x7fdb4c0c9090 2023-06-09T12:01:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.133+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb48003880 con 0x7fdb4c0c9090 2023-06-09T12:01:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.133+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb4c15a270 con 0x7fdb4c0c9090 2023-06-09T12:01:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.133+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4c15d720 con 0x7fdb4c0c9090 2023-06-09T12:01:29.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.134+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4c0ca620 con 0x7fdb4c0c9090 2023-06-09T12:01:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.135+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb480039e0 con 0x7fdb4c0c9090 2023-06-09T12:01:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.136+0000 7fdb427fc700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:29.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.136+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fdb48088ff0 con 0x7fdb4c0c9090 2023-06-09T12:01:29.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.136+0000 7fdb5119a700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:29.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.137+0000 7fdb5119a700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 0x7fdb3006d190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0ce450 tx=0x7fdb3c007d90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:29.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.141+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdb480bb2d0 con 0x7fdb4c0c9090 2023-06-09T12:01:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.344+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb4c074cf0 con 0x7fdb3006ace0 2023-06-09T12:01:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.353+0000 7fdb427fc700 1 -- 172.21.15.7:0/1715118499 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+313955 (secure 0 0 0) 0x7fdb4c074cf0 con 0x7fdb3006ace0 2023-06-09T12:01:29.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:01:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 msgr2=0x7fdb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 0x7fdb3006d190 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0ce450 tx=0x7fdb3c007d90 comp rx=0 tx=0).stop 2023-06-09T12:01:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 msgr2=0x7fdb4c15b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800c4d0 tx=0x7fdb4800c7e0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 shutdown_connections 2023-06-09T12:01:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb3006ace0 0x7fdb3006d190 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb4c1598b0 0x7fdb4c1f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb4c0c9090 0x7fdb4c15b750 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 --2- 172.21.15.7:0/1715118499 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb4c0c9fe0 0x7fdb4c15bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:29.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 >> 172.21.15.7:0/1715118499 conn(0x7fdb4c070af0 msgr2=0x7fdb4c158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 shutdown_connections 2023-06-09T12:01:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:29.358+0000 7fdb53bff700 1 -- 172.21.15.7:0/1715118499 wait complete. 2023-06-09T12:01:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:01:29.380 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1591,"stamp":"2023-06-09T12:01:28.955421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":631268,"num_omap_keys":1195,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90069,"ondisk_log_size":90069,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2566856,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":683520,"kb_avail":752407864,"statfs":{"total":773094113280,"available":770465652736,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":699924480},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003092"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3464,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.954869+0000","last_change":"2023-06-09T12:00:52.376091+0000","last_active":"2023-06-09T12:00:46.954869+0000","last_peered":"2023-06-09T12:00:46.954869+0000","last_clean":"2023-06-09T12:00:46.954869+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:46.954869+0000","last_undegraded":"2023-06-09T12:00:46.954869+0000","last_fullsized":"2023-06-09T12:00:46.954869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905267+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:44.905267+0000","last_peered":"2023-06-09T11:59:44.905267+0000","last_clean":"2023-06-09T11:59:44.905267+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:44.905267+0000","last_undegraded":"2023-06-09T11:59:44.905267+0000","last_fullsized":"2023-06-09T11:59:44.905267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3452,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:45.478589+0000","last_change":"2023-06-09T12:01:02.493236+0000","last_active":"2023-06-09T12:00:45.478589+0000","last_peered":"2023-06-09T12:00:45.478589+0000","last_clean":"2023-06-09T12:00:45.478589+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:00:45.478589+0000","last_undegraded":"2023-06-09T12:00:45.478589+0000","last_fullsized":"2023-06-09T12:00:45.478589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1091","reported_seq":3322,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:23.915902+0000","last_change":"2023-06-09T12:00:55.982660+0000","last_active":"2023-06-09T12:01:23.915902+0000","last_peered":"2023-06-09T12:01:23.915902+0000","last_clean":"2023-06-09T12:01:23.915902+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:23.915902+0000","last_undegraded":"2023-06-09T12:01:23.915902+0000","last_fullsized":"2023-06-09T12:01:23.915902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":218,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 28s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'917","reported_seq":3300,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:25.041475+0000","last_change":"2023-06-09T12:01:20.178264+0000","last_active":"2023-06-09T12:01:25.041475+0000","last_peered":"2023-06-09T12:01:25.041475+0000","last_clean":"2023-06-09T12:01:25.041475+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:25.041475+0000","last_undegraded":"2023-06-09T12:01:25.041475+0000","last_fullsized":"2023-06-09T12:01:25.041475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":49,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4198,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.183908+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:29.183908+0000","last_peered":"2023-06-09T11:59:29.183908+0000","last_clean":"2023-06-09T11:59:29.183908+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:29.183908+0000","last_undegraded":"2023-06-09T11:59:29.183908+0000","last_fullsized":"2023-06-09T11:59:29.183908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3692,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905258+0000","last_change":"2023-06-09T12:01:17.232199+0000","last_active":"2023-06-09T11:59:44.905258+0000","last_peered":"2023-06-09T11:59:44.905258+0000","last_clean":"2023-06-09T11:59:44.905258+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T11:59:44.905258+0000","last_undegraded":"2023-06-09T11:59:44.905258+0000","last_fullsized":"2023-06-09T11:59:44.905258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.184811+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:29.184811+0000","last_peered":"2023-06-09T11:59:29.184811+0000","last_clean":"2023-06-09T11:59:29.184811+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:29.184811+0000","last_undegraded":"2023-06-09T11:59:29.184811+0000","last_fullsized":"2023-06-09T11:59:29.184811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3401,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821600+0000","last_change":"2023-06-09T11:59:49.712515+0000","last_active":"2023-06-09T11:59:42.821600+0000","last_peered":"2023-06-09T11:59:42.821600+0000","last_clean":"2023-06-09T11:59:42.821600+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:42.821600+0000","last_undegraded":"2023-06-09T11:59:42.821600+0000","last_fullsized":"2023-06-09T11:59:42.821600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3231,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:42.821561+0000","last_change":"2023-06-09T11:59:48.700631+0000","last_active":"2023-06-09T11:59:42.821561+0000","last_peered":"2023-06-09T11:59:42.821561+0000","last_clean":"2023-06-09T11:59:42.821561+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T11:59:42.821561+0000","last_undegraded":"2023-06-09T11:59:42.821561+0000","last_fullsized":"2023-06-09T11:59:42.821561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1099","reported_seq":3789,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:22.612106+0000","last_change":"2023-06-09T12:00:47.970957+0000","last_active":"2023-06-09T12:01:22.612106+0000","last_peered":"2023-06-09T12:01:22.612106+0000","last_clean":"2023-06-09T12:01:22.612106+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:22.612106+0000","last_undegraded":"2023-06-09T12:01:22.612106+0000","last_fullsized":"2023-06-09T12:01:22.612106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":219,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296222+0000","last_change":"2023-06-09T11:18:26.308045+0000","last_active":"2023-06-09T11:59:46.296222+0000","last_peered":"2023-06-09T11:59:46.296222+0000","last_clean":"2023-06-09T11:59:46.296222+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T11:59:46.296222+0000","last_undegraded":"2023-06-09T11:59:46.296222+0000","last_fullsized":"2023-06-09T11:59:46.296222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423589+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:41.423589+0000","last_peered":"2023-06-09T11:59:41.423589+0000","last_clean":"2023-06-09T11:59:41.423589+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:41.423589+0000","last_undegraded":"2023-06-09T11:59:41.423589+0000","last_fullsized":"2023-06-09T11:59:41.423589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1132","reported_seq":3769,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:23.519334+0000","last_change":"2023-06-09T12:00:54.677013+0000","last_active":"2023-06-09T12:01:23.519334+0000","last_peered":"2023-06-09T12:01:23.519334+0000","last_clean":"2023-06-09T12:01:23.519334+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:23.519334+0000","last_undegraded":"2023-06-09T12:01:23.519334+0000","last_fullsized":"2023-06-09T12:01:23.519334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":223,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 32s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.308308+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3018,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121483+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:42.121483+0000","last_peered":"2023-06-09T11:59:42.121483+0000","last_clean":"2023-06-09T11:59:42.121483+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:42.121483+0000","last_undegraded":"2023-06-09T11:59:42.121483+0000","last_fullsized":"2023-06-09T11:59:42.121483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905252+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:44.905252+0000","last_peered":"2023-06-09T11:59:44.905252+0000","last_clean":"2023-06-09T11:59:44.905252+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:44.905252+0000","last_undegraded":"2023-06-09T11:59:44.905252+0000","last_fullsized":"2023-06-09T11:59:44.905252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517078+0000","last_change":"2023-06-09T11:18:26.303047+0000","last_active":"2023-06-09T11:59:43.517078+0000","last_peered":"2023-06-09T11:59:43.517078+0000","last_clean":"2023-06-09T11:59:43.517078+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T11:59:43.517078+0000","last_undegraded":"2023-06-09T11:59:43.517078+0000","last_fullsized":"2023-06-09T11:59:43.517078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3135,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423662+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:41.423662+0000","last_peered":"2023-06-09T11:59:41.423662+0000","last_clean":"2023-06-09T11:59:41.423662+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:41.423662+0000","last_undegraded":"2023-06-09T11:59:41.423662+0000","last_fullsized":"2023-06-09T11:59:41.423662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1997,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:43.516919+0000","last_change":"2023-06-09T12:00:33.899239+0000","last_active":"2023-06-09T11:59:43.516919+0000","last_peered":"2023-06-09T11:59:43.516919+0000","last_clean":"2023-06-09T11:59:43.516919+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T11:59:43.516919+0000","last_undegraded":"2023-06-09T11:59:43.516919+0000","last_fullsized":"2023-06-09T11:59:43.516919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:28.569063+0000","last_change":"2023-06-09T12:00:52.491920+0000","last_active":"2023-06-09T12:00:28.569063+0000","last_peered":"2023-06-09T12:00:28.569063+0000","last_clean":"2023-06-09T12:00:28.569063+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:00:28.569063+0000","last_undegraded":"2023-06-09T12:00:28.569063+0000","last_fullsized":"2023-06-09T12:00:28.569063+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":347,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:50.067505+0000","last_change":"2023-06-09T11:59:59.471359+0000","last_active":"2023-06-09T11:59:50.067505+0000","last_peered":"2023-06-09T11:59:50.067505+0000","last_clean":"2023-06-09T11:59:50.067505+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:50.067505+0000","last_undegraded":"2023-06-09T11:59:50.067505+0000","last_fullsized":"2023-06-09T11:59:50.067505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2963,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:52.293286+0000","last_change":"2023-06-09T12:00:36.885523+0000","last_active":"2023-06-09T11:59:52.293286+0000","last_peered":"2023-06-09T11:59:52.293286+0000","last_clean":"2023-06-09T11:59:52.293286+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:52.293286+0000","last_undegraded":"2023-06-09T11:59:52.293286+0000","last_fullsized":"2023-06-09T11:59:52.293286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1264,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:46.296204+0000","last_change":"2023-06-09T12:00:19.292684+0000","last_active":"2023-06-09T11:59:46.296204+0000","last_peered":"2023-06-09T11:59:46.296204+0000","last_clean":"2023-06-09T11:59:46.296204+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T11:59:46.296204+0000","last_undegraded":"2023-06-09T11:59:46.296204+0000","last_fullsized":"2023-06-09T11:59:46.296204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1035,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.404606+0000","last_change":"2023-06-09T12:01:13.693733+0000","last_active":"2023-06-09T12:00:02.404606+0000","last_peered":"2023-06-09T12:00:02.404606+0000","last_clean":"2023-06-09T12:00:02.404606+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:00:02.404606+0000","last_undegraded":"2023-06-09T12:00:02.404606+0000","last_fullsized":"2023-06-09T12:00:02.404606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1082,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:55.067574+0000","last_change":"2023-06-09T12:00:45.031530+0000","last_active":"2023-06-09T11:59:55.067574+0000","last_peered":"2023-06-09T11:59:55.067574+0000","last_clean":"2023-06-09T11:59:55.067574+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:55.067574+0000","last_undegraded":"2023-06-09T11:59:55.067574+0000","last_fullsized":"2023-06-09T11:59:55.067574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3559,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.294703+0000","last_change":"2023-06-09T12:00:41.875998+0000","last_active":"2023-06-09T12:00:02.294703+0000","last_peered":"2023-06-09T12:00:02.294703+0000","last_clean":"2023-06-09T12:00:02.294703+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:02.294703+0000","last_undegraded":"2023-06-09T12:00:02.294703+0000","last_fullsized":"2023-06-09T12:00:02.294703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3587,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:44.905212+0000","last_change":"2023-06-09T12:01:18.198297+0000","last_active":"2023-06-09T11:59:44.905212+0000","last_peered":"2023-06-09T11:59:44.905212+0000","last_clean":"2023-06-09T11:59:44.905212+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T11:59:44.905212+0000","last_undegraded":"2023-06-09T11:59:44.905212+0000","last_fullsized":"2023-06-09T11:59:44.905212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3023,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.185216+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:29.185216+0000","last_peered":"2023-06-09T11:59:29.185216+0000","last_clean":"2023-06-09T11:59:29.185216+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:29.185216+0000","last_undegraded":"2023-06-09T11:59:29.185216+0000","last_fullsized":"2023-06-09T11:59:29.185216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:25.604660+0000","last_change":"2023-06-09T12:00:25.604660+0000","last_active":"2023-06-09T12:00:25.604660+0000","last_peered":"2023-06-09T12:00:25.604660+0000","last_clean":"2023-06-09T12:00:25.604660+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:00:25.604660+0000","last_undegraded":"2023-06-09T12:00:25.604660+0000","last_fullsized":"2023-06-09T12:00:25.604660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'887","reported_seq":3763,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.203922+0000","last_change":"2023-06-09T11:18:26.313954+0000","last_active":"2023-06-09T11:59:29.203922+0000","last_peered":"2023-06-09T11:59:29.203922+0000","last_clean":"2023-06-09T11:59:29.203922+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T11:59:29.203922+0000","last_undegraded":"2023-06-09T11:59:29.203922+0000","last_fullsized":"2023-06-09T11:59:29.203922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":887,"log_dups_size":0,"ondisk_log_size":887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":728,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:27.395619+0000","last_change":"2023-06-09T12:00:46.032468+0000","last_active":"2023-06-09T12:00:27.395619+0000","last_peered":"2023-06-09T12:00:27.395619+0000","last_clean":"2023-06-09T12:00:27.395619+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:27.395619+0000","last_undegraded":"2023-06-09T12:00:27.395619+0000","last_fullsized":"2023-06-09T12:00:27.395619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289712+0000","last_change":"2023-06-09T12:01:16.188471+0000","last_active":"2023-06-09T12:00:42.289712+0000","last_peered":"2023-06-09T12:00:42.289712+0000","last_clean":"2023-06-09T12:00:42.289712+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:00:42.289712+0000","last_undegraded":"2023-06-09T12:00:42.289712+0000","last_fullsized":"2023-06-09T12:00:42.289712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394854+0000","last_change":"2023-06-09T12:00:47.993494+0000","last_active":"2023-06-09T12:00:22.394854+0000","last_peered":"2023-06-09T12:00:22.394854+0000","last_clean":"2023-06-09T12:00:22.394854+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:22.394854+0000","last_undegraded":"2023-06-09T12:00:22.394854+0000","last_fullsized":"2023-06-09T12:00:22.394854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394838+0000","last_change":"2023-06-09T12:00:53.983699+0000","last_active":"2023-06-09T12:00:22.394838+0000","last_peered":"2023-06-09T12:00:22.394838+0000","last_clean":"2023-06-09T12:00:22.394838+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:22.394838+0000","last_undegraded":"2023-06-09T12:00:22.394838+0000","last_fullsized":"2023-06-09T12:00:22.394838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:35.635944+0000","last_change":"2023-06-09T12:00:52.491865+0000","last_active":"2023-06-09T12:00:35.635944+0000","last_peered":"2023-06-09T12:00:35.635944+0000","last_clean":"2023-06-09T12:00:35.635944+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:00:35.635944+0000","last_undegraded":"2023-06-09T12:00:35.635944+0000","last_fullsized":"2023-06-09T12:00:35.635944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445722+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445722+0000","last_peered":"2023-06-09T12:00:04.445722+0000","last_clean":"2023-06-09T12:00:04.445722+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445722+0000","last_undegraded":"2023-06-09T12:00:04.445722+0000","last_fullsized":"2023-06-09T12:00:04.445722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1636,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.410160+0000","last_change":"2023-06-09T12:01:14.658585+0000","last_active":"2023-06-09T12:00:42.410160+0000","last_peered":"2023-06-09T12:00:42.410160+0000","last_clean":"2023-06-09T12:00:42.410160+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:00:42.410160+0000","last_undegraded":"2023-06-09T12:00:42.410160+0000","last_fullsized":"2023-06-09T12:00:42.410160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121371+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:42.121371+0000","last_peered":"2023-06-09T11:59:42.121371+0000","last_clean":"2023-06-09T11:59:42.121371+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:42.121371+0000","last_undegraded":"2023-06-09T11:59:42.121371+0000","last_fullsized":"2023-06-09T11:59:42.121371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.246824+0000","last_change":"2023-06-09T12:00:47.290422+0000","last_active":"2023-06-09T12:00:17.246824+0000","last_peered":"2023-06-09T12:00:17.246824+0000","last_clean":"2023-06-09T12:00:17.246824+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:17.246824+0000","last_undegraded":"2023-06-09T12:00:17.246824+0000","last_fullsized":"2023-06-09T12:00:17.246824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":897,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207703+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:44.207703+0000","last_peered":"2023-06-09T11:59:44.207703+0000","last_clean":"2023-06-09T11:59:44.207703+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:44.207703+0000","last_undegraded":"2023-06-09T11:59:44.207703+0000","last_fullsized":"2023-06-09T11:59:44.207703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3889,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121324+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:42.121324+0000","last_peered":"2023-06-09T11:59:42.121324+0000","last_clean":"2023-06-09T11:59:42.121324+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:42.121324+0000","last_undegraded":"2023-06-09T11:59:42.121324+0000","last_fullsized":"2023-06-09T11:59:42.121324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121240+0000","last_change":"2023-06-09T11:10:49.676697+0000","last_active":"2023-06-09T11:59:42.121240+0000","last_peered":"2023-06-09T11:59:42.121240+0000","last_clean":"2023-06-09T11:59:42.121240+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T11:59:42.121240+0000","last_undegraded":"2023-06-09T11:59:42.121240+0000","last_fullsized":"2023-06-09T11:59:42.121240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121335+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:42.121335+0000","last_peered":"2023-06-09T11:59:42.121335+0000","last_clean":"2023-06-09T11:59:42.121335+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:42.121335+0000","last_undegraded":"2023-06-09T11:59:42.121335+0000","last_fullsized":"2023-06-09T11:59:42.121335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.338006+0000","last_change":"2023-06-09T12:00:47.290438+0000","last_active":"2023-06-09T12:00:15.338006+0000","last_peered":"2023-06-09T12:00:15.338006+0000","last_clean":"2023-06-09T12:00:15.338006+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:00:15.338006+0000","last_undegraded":"2023-06-09T12:00:15.338006+0000","last_fullsized":"2023-06-09T12:00:15.338006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1347,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602275+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602275+0000","last_peered":"2023-06-09T12:00:03.602275+0000","last_clean":"2023-06-09T12:00:03.602275+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602275+0000","last_undegraded":"2023-06-09T12:00:03.602275+0000","last_fullsized":"2023-06-09T12:00:03.602275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3298,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.542755+0000","last_change":"2023-06-09T12:01:02.493261+0000","last_active":"2023-06-09T12:00:32.542755+0000","last_peered":"2023-06-09T12:00:32.542755+0000","last_clean":"2023-06-09T12:00:32.542755+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:00:32.542755+0000","last_undegraded":"2023-06-09T12:00:32.542755+0000","last_fullsized":"2023-06-09T12:00:32.542755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:47.289624+0000","last_change":"2023-06-09T12:00:52.398167+0000","last_active":"2023-06-09T11:59:47.289624+0000","last_peered":"2023-06-09T11:59:47.289624+0000","last_clean":"2023-06-09T11:59:47.289624+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:47.289624+0000","last_undegraded":"2023-06-09T11:59:47.289624+0000","last_fullsized":"2023-06-09T11:59:47.289624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423608+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:41.423608+0000","last_peered":"2023-06-09T11:59:41.423608+0000","last_clean":"2023-06-09T11:59:41.423608+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:41.423608+0000","last_undegraded":"2023-06-09T11:59:41.423608+0000","last_fullsized":"2023-06-09T11:59:41.423608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":821,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121243+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:42.121243+0000","last_peered":"2023-06-09T11:59:42.121243+0000","last_clean":"2023-06-09T11:59:42.121243+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:42.121243+0000","last_undegraded":"2023-06-09T11:59:42.121243+0000","last_fullsized":"2023-06-09T11:59:42.121243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4112,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423567+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:41.423567+0000","last_peered":"2023-06-09T11:59:41.423567+0000","last_clean":"2023-06-09T11:59:41.423567+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:41.423567+0000","last_undegraded":"2023-06-09T11:59:41.423567+0000","last_fullsized":"2023-06-09T11:59:41.423567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.652148+0000","last_change":"2023-06-09T12:00:37.490252+0000","last_active":"2023-06-09T12:00:36.652148+0000","last_peered":"2023-06-09T12:00:36.652148+0000","last_clean":"2023-06-09T12:00:36.652148+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:36.652148+0000","last_undegraded":"2023-06-09T12:00:36.652148+0000","last_fullsized":"2023-06-09T12:00:36.652148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423433+0000","last_change":"2023-06-09T11:10:49.681480+0000","last_active":"2023-06-09T11:59:41.423433+0000","last_peered":"2023-06-09T11:59:41.423433+0000","last_clean":"2023-06-09T11:59:41.423433+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T11:59:41.423433+0000","last_undegraded":"2023-06-09T11:59:41.423433+0000","last_fullsized":"2023-06-09T11:59:41.423433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":881,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423490+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:41.423490+0000","last_peered":"2023-06-09T11:59:41.423490+0000","last_clean":"2023-06-09T11:59:41.423490+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:41.423490+0000","last_undegraded":"2023-06-09T11:59:41.423490+0000","last_fullsized":"2023-06-09T11:59:41.423490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1061,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446248+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446248+0000","last_peered":"2023-06-09T12:00:25.446248+0000","last_clean":"2023-06-09T12:00:25.446248+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446248+0000","last_undegraded":"2023-06-09T12:00:25.446248+0000","last_fullsized":"2023-06-09T12:00:25.446248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3148,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:29.207251+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:29.207251+0000","last_peered":"2023-06-09T11:59:29.207251+0000","last_clean":"2023-06-09T11:59:29.207251+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:29.207251+0000","last_undegraded":"2023-06-09T11:59:29.207251+0000","last_fullsized":"2023-06-09T11:59:29.207251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:30.364947+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:30.364947+0000","last_peered":"2023-06-09T11:59:30.364947+0000","last_clean":"2023-06-09T11:59:30.364947+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:30.364947+0000","last_undegraded":"2023-06-09T11:59:30.364947+0000","last_fullsized":"2023-06-09T11:59:30.364947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3426,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:30.528446+0000","last_change":"2023-06-09T12:00:52.491822+0000","last_active":"2023-06-09T12:00:30.528446+0000","last_peered":"2023-06-09T12:00:30.528446+0000","last_clean":"2023-06-09T12:00:30.528446+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:00:30.528446+0000","last_undegraded":"2023-06-09T12:00:30.528446+0000","last_fullsized":"2023-06-09T12:00:30.528446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121452+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:42.121452+0000","last_peered":"2023-06-09T11:59:42.121452+0000","last_clean":"2023-06-09T11:59:42.121452+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:42.121452+0000","last_undegraded":"2023-06-09T11:59:42.121452+0000","last_fullsized":"2023-06-09T11:59:42.121452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":59307,"ondisk_log_size":59307,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":631268,"num_omap_keys":1195,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30643,"ondisk_log_size":30643,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254480,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74624,"kb_avail":94117360,"statfs":{"total":96636764160,"available":96376176640,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76414976},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.67500000000000004,"15min":0.64700000000000002},"min":{"1min":0.40600000000000003,"5min":0.32200000000000001,"15min":0.28699999999999998},"max":{"1min":0.90800000000000003,"5min":1.0640000000000001,"15min":1.1599999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.46500000000000002,"5min":0.438,"15min":0.35299999999999998},"max":{"1min":0.79700000000000004,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.433}]},{"osd":1,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.68300000000000005,"15min":0.67300000000000004},"min":{"1min":0.41999999999999998,"5min":0.371,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":1.0089999999999999,"15min":1.25},"last":0.626},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69699999999999995,"15min":0.68100000000000005},"min":{"1min":0.53200000000000003,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.84199999999999997,"5min":1.05,"15min":1.05},"last":0.502}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.73599999999999999,"15min":0.72899999999999998},"min":{"1min":0.51000000000000001,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.93700000000000006,"5min":1.099,"15min":1.099},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.71999999999999997},"min":{"1min":0.54600000000000004,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.98999999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:00:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68100000000000005,"15min":0.66200000000000003},"min":{"1min":0.48499999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.90700000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.69899999999999995},"min":{"1min":0.57699999999999996,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.88500000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.54600000000000004,"15min":0.52900000000000003},"min":{"1min":0.32900000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78000000000000003,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.54700000000000004,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.59799999999999998},"min":{"1min":0.44800000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.76800000000000002,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63700000000000001,"15min":0.623},"min":{"1min":0.40100000000000002,"5min":0.38200000000000001,"15min":0.32400000000000001},"max":{"1min":0.871,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68100000000000005,"15min":0.64600000000000002},"min":{"1min":0.46000000000000002,"5min":0.40300000000000002,"15min":0.372},"max":{"1min":1.044,"5min":1.044,"15min":1.0920000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.71799999999999997},"min":{"1min":0.504,"5min":0.44,"15min":0.42599999999999999},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.206},"last":0.64100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575460,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203096,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":46976,"kb_avail":94168744,"statfs":{"total":96636764160,"available":96428793856,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48103424},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.49299999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81599999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64500000000000002,"15min":0.63700000000000001},"min":{"1min":0.42799999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.92200000000000004,"5min":1.069,"15min":1.069},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.53200000000000003,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.84999999999999998,"5min":0.99299999999999999,"15min":1.081},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.56499999999999995,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92300000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.79700000000000004,"5min":1.024,"15min":1.125},"last":0.749},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72299999999999998,"15min":0.71499999999999997},"min":{"1min":0.54900000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.86599999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69399999999999995,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.94499999999999995,"5min":1.036,"15min":1.1719999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.56499999999999995,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":1.0169999999999999,"5min":1.034,"15min":1.0549999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.877,"5min":0.95299999999999996,"15min":1.107},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.44700000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.78500000000000003,"5min":0.85899999999999999,"15min":0.90900000000000003},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69199999999999995},"min":{"1min":0.44600000000000001,"5min":0.44500000000000001,"15min":0.35199999999999998},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64900000000000002,"15min":0.63500000000000001},"min":{"1min":0.40799999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89800000000000002,"5min":1.052,"15min":1.0589999999999999},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.071},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575460,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400688,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94272,"kb_avail":93971152,"statfs":{"total":96636764160,"available":96226459648,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96534528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.39600000000000002,"15min":0.40400000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.56299999999999994,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.53600000000000003,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":1.877},"last":0.81399999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.45300000000000001,"15min":0.45900000000000002},"min":{"1min":0.33500000000000002,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.67200000000000004,"5min":0.82699999999999996,"15min":1.982},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53100000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.32700000000000001,"15min":0.317},"max":{"1min":0.74399999999999999,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42199999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70199999999999996,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.64200000000000002},"min":{"1min":0.49299999999999999,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.091,"15min":1.9790000000000001},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.67400000000000004},"min":{"1min":0.45700000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.0649999999999999,"15min":1.609},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.081,"5min":1.1459999999999999,"15min":1.774},"last":1.077}]},{"osd":5,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78100000000000003,"15min":0.77700000000000002},"min":{"1min":0.627,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":0.96199999999999997,"5min":1.0880000000000001,"15min":1.774},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.82599999999999996,"15min":0.81000000000000005},"min":{"1min":0.67500000000000004,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":0.99199999999999999,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":1.0449999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82299999999999995,"15min":0.83499999999999996},"min":{"1min":0.58399999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.996,"5min":1.2549999999999999,"15min":1.9059999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.621,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.004,"5min":1.117,"15min":1.921},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:00:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.84999999999999998,"15min":0.85899999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0349999999999999,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75700000000000001,"15min":0.77100000000000002},"min":{"1min":0.60599999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.95299999999999996,"5min":1.226,"15min":1.827},"last":1.0609999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332072,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104256,"kb_avail":94039768,"statfs":{"total":96636764160,"available":96296722432,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106758144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.621}]},{"osd":2,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.0900000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.48499999999999999},"min":{"1min":0.38300000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.88,"5min":0.88,"15min":1.2909999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.75}]},{"osd":4,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.1060000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.76200000000000001},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.21},"last":1.0940000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":1.05,"15min":1.236},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.63200000000000001,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":1.002,"5min":1.0369999999999999,"15min":1.165},"last":1.0089999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.0309999999999999,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":1.123},{"interface":"front","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.73799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":1.038}]},{"osd":7,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.68100000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.1299999999999999,"5min":1.159,"15min":1.228},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.71599999999999997,"5min":0.58199999999999996,"15min":0.48599999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.29},"last":1.0620000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575461,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237088,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94134752,"statfs":{"total":96636764160,"available":96393986048,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42699999999999999,"15min":0.41199999999999998},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51300000000000001,"15min":0.501},"min":{"1min":0.41099999999999998,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.999,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.56299999999999994,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":1.367},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.629,"15min":0.60699999999999998},"min":{"1min":0.498,"5min":0.44,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.73599999999999999},"min":{"1min":0.63900000000000001,"5min":0.443,"15min":0.373},"max":{"1min":1.034,"5min":1.179,"15min":1.8220000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":0.97199999999999998,"5min":1.2450000000000001,"15min":1.601},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83799999999999997,"15min":0.82099999999999995},"min":{"1min":0.70199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96999999999999997,"5min":1.288,"15min":1.6479999999999999},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.496,"15min":0.438},"max":{"1min":1.089,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.91400000000000003,"5min":1.121,"15min":1.3540000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.58899999999999997,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":0.98399999999999999,"5min":1.208,"15min":1.248},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.026,"5min":1.262,"15min":1.337},"last":0.75900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575460,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433596,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110400,"kb_avail":93938244,"statfs":{"total":96636764160,"available":96192761856,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":113049600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.34300000000000003,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":0.71299999999999997,"15min":1.7969999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.30599999999999999},"max":{"1min":0.90900000000000003,"5min":1.2130000000000001,"15min":1.5669999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.40999999999999998,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.496,"15min":0.51500000000000001},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.50700000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.878,"5min":1.355,"15min":1.355},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.78400000000000003,"5min":1.091,"15min":1.198},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54800000000000004,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":1.073,"5min":1.073,"15min":1.635},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.90600000000000003,"5min":3.081,"15min":3.081},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.754},"min":{"1min":0.66400000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.165,"5min":1.198,"15min":1.2110000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.66600000000000004,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.1970000000000001,"5min":1.234,"15min":1.367},"last":0.86499999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.503,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.286,"15min":1.286},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.61899999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.103,"5min":1.103,"15min":3.3490000000000002},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65700000000000003,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.413},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.88,"15min":0.871},"min":{"1min":0.73099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1659999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575459,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417684,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":124928,"kb_avail":93954156,"statfs":{"total":96636764160,"available":96209055744,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":127926272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.65700000000000003},"min":{"1min":0.53600000000000003,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.96899999999999997,"5min":1.137,"15min":1.137},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.64200000000000002,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.84699999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.88600000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67500000000000004,"15min":0.63900000000000001},"min":{"1min":0.64600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94399999999999995,"5min":1.609,"15min":1.609},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.59499999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94899999999999995,"5min":0.997,"15min":1.159},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.58499999999999996,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.98699999999999999,"5min":1.407,"15min":1.407},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73099999999999998},"min":{"1min":0.57299999999999995,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.47599999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83999999999999997,"5min":1.032,"15min":1.032},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67400000000000004,"15min":0.65300000000000002},"min":{"1min":0.47999999999999998,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.89600000000000002,"5min":1.361,"15min":1.361},"last":0.753}]}]},{"osd":5,"up_from":13,"seq":55834575459,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288152,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70784,"kb_avail":94083688,"statfs":{"total":96636764160,"available":96341696512,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72482816},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.753,"15min":0.69099999999999995},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.314},"max":{"1min":3.468,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74299999999999999,"15min":0.70499999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89400000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":1.177}]},{"osd":1,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.89000000000000001,"15min":0.71599999999999997},"min":{"1min":0.377,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":0.89600000000000002,"5min":19.715,"15min":19.715},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.78500000000000003,"15min":0.70399999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.94099999999999995,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.872}]},{"osd":2,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.68600000000000005},"min":{"1min":0.53700000000000003,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.91100000000000003,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.85899999999999999,"15min":0.75700000000000001},"min":{"1min":0.49099999999999999,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.91300000000000003,"5min":14.439,"15min":14.439},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.85999999999999999,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.433,"15min":0.375},"max":{"1min":0.94999999999999996,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.88800000000000001,"15min":0.80700000000000005},"min":{"1min":0.55900000000000005,"5min":0.52300000000000002,"15min":0.50800000000000001},"max":{"1min":0.875,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.96999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":1.298,"15min":0.82599999999999996},"min":{"1min":0.40799999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.80200000000000005,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.3180000000000001,"15min":0.83799999999999997},"min":{"1min":0.45200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.84299999999999997,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.78000000000000003,"15min":0.68700000000000006},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84999999999999998,"5min":12.163,"15min":12.163},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.70799999999999996,"15min":0.64900000000000002},"min":{"1min":0.48499999999999999,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.57499999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:00:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.71699999999999997,"15min":0.69099999999999995},"min":{"1min":0.44600000000000001,"5min":0.42399999999999999,"15min":0.39300000000000002},"max":{"1min":0.96299999999999997,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.97199999999999998,"15min":0.81499999999999995},"min":{"1min":0.58299999999999996,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":19.834,"15min":19.834},"last":0.68700000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:01:29.387 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.387 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.388 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.388 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.388 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.389 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.389 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.389 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.390 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.390 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.390 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.391 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.391 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.392 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.392 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.392 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.393 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.393 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.393 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.394 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.394 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.394 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.395 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.395 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.395 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.396 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.396 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.396 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.397 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.397 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.397 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.398 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.398 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.398 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.399 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.399 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.399 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.400 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.400 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.400 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.401 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.401 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.401 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.402 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.402 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.402 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.403 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.403 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.404 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.404 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.404 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.405 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.405 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.405 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.406 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.406 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.406 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.407 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.407 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.407 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.408 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.408 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.408 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.409 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.409 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.409 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.410 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.410 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.410 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.411 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.411 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.411 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.412 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.412 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.413 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.413 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.413 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.414 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.414 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.414 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.415 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.415 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.415 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.416 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.416 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.416 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.417 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.417 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.417 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:29.418 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:01:49.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:01:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.627+0000 7f8cd85ff700 1 Processor -- start 2023-06-09T12:01:49.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.627+0000 7f8cd85ff700 1 -- start start 2023-06-09T12:01:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 0x7f8cd00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd85ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cd015cd40 0x7f8cd00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd85ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd0159b20 con 0x7f8cd00cb5d0 2023-06-09T12:01:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd85ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd00cac80 con 0x7f8cd015cd40 2023-06-09T12:01:49.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01597d0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.632 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd6b9c700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8cd015cd40 0x7f8cd00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.628+0000 7f8cd5b9a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 0x7f8cd00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd639b700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f8cd00cb5d0 0x7f8cd0159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:35492/0 (socket says 172.21.15.7:35492) 2023-06-09T12:01:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd639b700 1 -- 172.21.15.7:0/866429023 learned_addr learned my addr 172.21.15.7:0/866429023 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:01:49.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd5399700 1 -- 172.21.15.7:0/866429023 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410505719 0 0) 0x7f8cd0159b20 con 0x7f8cd00cb5d0 2023-06-09T12:01:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd5399700 1 -- 172.21.15.7:0/866429023 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cbc0035b0 con 0x7f8cd00cb5d0 2023-06-09T12:01:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd6b9c700 1 -- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 msgr2=0x7f8cd00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:01:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd6b9c700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 0x7f8cd00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd6b9c700 1 -- 172.21.15.7:0/866429023 >> v1:172.21.15.117:6789/0 conn(0x7f8cd00cb5d0 legacy=0x7f8cd0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd6b9c700 1 -- 172.21.15.7:0/866429023 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd0071ab0 con 0x7f8cd015cd40 2023-06-09T12:01:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd5b9a700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 0x7f8cd00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:01:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.629+0000 7f8cd6b9c700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd015cd40 0x7f8cd00cbea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009d90 tx=0x7f8cc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ec39e0bfe7eed1 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd5399700 1 -- 172.21.15.7:0/866429023 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800e070 con 0x7f8cd015cd40 2023-06-09T12:01:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd5399700 1 -- 172.21.15.7:0/866429023 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc800bb20 con 0x7f8cd015cd40 2023-06-09T12:01:49.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd5399700 1 -- 172.21.15.7:0/866429023 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800be90 con 0x7f8cd015cd40 2023-06-09T12:01:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 -- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd015cd40 msgr2=0x7f8cd00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd015cd40 0x7f8cd00cbea0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009d90 tx=0x7f8cc800b330 comp rx=0 tx=0).stop 2023-06-09T12:01:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 -- 172.21.15.7:0/866429023 shutdown_connections 2023-06-09T12:01:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd015cd40 0x7f8cd00cbea0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/866429023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd00cb9a0 0x7f8cd00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 -- 172.21.15.7:0/866429023 >> 172.21.15.7:0/866429023 conn(0x7f8cd0070af0 msgr2=0x7f8cd0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 -- 172.21.15.7:0/866429023 shutdown_connections 2023-06-09T12:01:49.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.630+0000 7f8cd85ff700 1 -- 172.21.15.7:0/866429023 wait complete. 2023-06-09T12:01:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.632+0000 7f8cd85ff700 1 Processor -- start 2023-06-09T12:01:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.632+0000 7f8cd85ff700 1 -- start start 2023-06-09T12:01:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 0x7f8cd01f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd015cd40 0x7f8cd0150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01f3410 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01ef060 con 0x7f8cd00cb5d0 2023-06-09T12:01:49.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01ef300 con 0x7f8cd015cd40 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40140/0 (socket says 172.21.15.7:40140) 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.633+0000 7f8cd5b9a700 1 -- 172.21.15.7:0/3375483969 learned_addr learned my addr 172.21.15.7:0/3375483969 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd639b700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 0x7f8cd01f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd6b9c700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd015cd40 0x7f8cd0150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 -- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd015cd40 msgr2=0x7f8cd0150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd015cd40 0x7f8cd0150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 -- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 msgr2=0x7f8cd01f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 0x7f8cd01f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 -- 172.21.15.7:0/3375483969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00094a0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd639b700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 0x7f8cd01f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:01:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd5b9a700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000ebe0 tx=0x7f8cc0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000c890 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc000c9f0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc8009a70 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc000cda0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.634+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd015d4f0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.636+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd015d210 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.637+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8cc0013470 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.638+0000 7f8cc6ffd700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 0x7f8cb406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:01:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.638+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8cc0089c90 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.638+0000 7f8cd639b700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 0x7f8cb406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:01:49.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.638+0000 7f8cd639b700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 0x7f8cb406d1c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005910 tx=0x7f8ccc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:01:49.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.642+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8cc00543c0 con 0x7f8cd00cb9a0 2023-06-09T12:01:49.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.843+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8cd015d7d0 con 0x7f8cb406ad10 2023-06-09T12:01:49.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.852+0000 7f8cc6ffd700 1 -- 172.21.15.7:0/3375483969 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+314189 (secure 0 0 0) 0x7f8cd015d7d0 con 0x7f8cb406ad10 2023-06-09T12:01:49.858 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:01:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.856+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 msgr2=0x7f8cb406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.856+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 0x7f8cb406d1c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc005910 tx=0x7f8ccc009040 comp rx=0 tx=0).stop 2023-06-09T12:01:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 msgr2=0x7f8cd00b2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:01:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000ebe0 tx=0x7f8cc0009e60 comp rx=0 tx=0).stop 2023-06-09T12:01:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 shutdown_connections 2023-06-09T12:01:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8cb406ad10 0x7f8cb406d1c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8cd015cd40 0x7f8cd0150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8cd00cb5d0 0x7f8cd01f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 --2- 172.21.15.7:0/3375483969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8cd00cb9a0 0x7f8cd00b2780 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:01:49.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 >> 172.21.15.7:0/3375483969 conn(0x7f8cd0070af0 msgr2=0x7f8cd0071050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:01:49.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 shutdown_connections 2023-06-09T12:01:49.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:01:49.857+0000 7f8cd85ff700 1 -- 172.21.15.7:0/3375483969 wait complete. 2023-06-09T12:01:49.862 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:01:49.878 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1601,"stamp":"2023-06-09T12:01:48.960340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":631268,"num_omap_keys":1195,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90333,"ondisk_log_size":90333,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2567432,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":684096,"kb_avail":752407288,"statfs":{"total":773094113280,"available":770465062912,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":700514304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002873"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3464,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.954869+0000","last_change":"2023-06-09T12:00:52.376091+0000","last_active":"2023-06-09T12:00:46.954869+0000","last_peered":"2023-06-09T12:00:46.954869+0000","last_clean":"2023-06-09T12:00:46.954869+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:46.954869+0000","last_undegraded":"2023-06-09T12:00:46.954869+0000","last_fullsized":"2023-06-09T12:00:46.954869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905267+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:44.905267+0000","last_peered":"2023-06-09T11:59:44.905267+0000","last_clean":"2023-06-09T11:59:44.905267+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:44.905267+0000","last_undegraded":"2023-06-09T11:59:44.905267+0000","last_fullsized":"2023-06-09T11:59:44.905267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3452,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:45.478589+0000","last_change":"2023-06-09T12:01:02.493236+0000","last_active":"2023-06-09T12:00:45.478589+0000","last_peered":"2023-06-09T12:00:45.478589+0000","last_clean":"2023-06-09T12:00:45.478589+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:00:45.478589+0000","last_undegraded":"2023-06-09T12:00:45.478589+0000","last_fullsized":"2023-06-09T12:00:45.478589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1040","reported_seq":3429,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:45.879934+0000","last_change":"2023-06-09T12:01:20.178264+0000","last_active":"2023-06-09T12:01:45.879934+0000","last_peered":"2023-06-09T12:01:45.879934+0000","last_clean":"2023-06-09T12:01:45.879934+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:45.879934+0000","last_undegraded":"2023-06-09T12:01:45.879934+0000","last_fullsized":"2023-06-09T12:01:45.879934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":172,"log_size":1040,"log_dups_size":0,"ondisk_log_size":1040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207940+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:44.207940+0000","last_peered":"2023-06-09T11:59:44.207940+0000","last_clean":"2023-06-09T11:59:44.207940+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:44.207940+0000","last_undegraded":"2023-06-09T11:59:44.207940+0000","last_fullsized":"2023-06-09T11:59:44.207940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3402,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:49.712515+0000","last_change":"2023-06-09T12:01:37.381048+0000","last_active":"2023-06-09T11:59:49.712515+0000","last_peered":"2023-06-09T11:59:49.712515+0000","last_clean":"2023-06-09T11:59:49.712515+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T11:59:49.712515+0000","last_undegraded":"2023-06-09T11:59:49.712515+0000","last_fullsized":"2023-06-09T11:59:49.712515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3233,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:31.007812+0000","last_change":"2023-06-09T12:01:37.381026+0000","last_active":"2023-06-09T12:01:31.007812+0000","last_peered":"2023-06-09T12:01:31.007812+0000","last_clean":"2023-06-09T12:01:31.007812+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:31.007812+0000","last_undegraded":"2023-06-09T12:01:31.007812+0000","last_fullsized":"2023-06-09T12:01:31.007812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'859","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905177+0000","last_change":"2023-06-09T11:18:26.254621+0000","last_active":"2023-06-09T11:59:44.905177+0000","last_peered":"2023-06-09T11:59:44.905177+0000","last_clean":"2023-06-09T11:59:44.905177+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T11:59:44.905177+0000","last_undegraded":"2023-06-09T11:59:44.905177+0000","last_fullsized":"2023-06-09T11:59:44.905177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":859,"log_dups_size":0,"ondisk_log_size":859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3525,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394812+0000","last_change":"2023-06-09T12:01:33.911517+0000","last_active":"2023-06-09T12:00:22.394812+0000","last_peered":"2023-06-09T12:00:22.394812+0000","last_clean":"2023-06-09T12:00:22.394812+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:00:22.394812+0000","last_undegraded":"2023-06-09T12:00:22.394812+0000","last_fullsized":"2023-06-09T12:00:22.394812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3481,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423589+0000","last_change":"2023-06-09T11:18:26.292910+0000","last_active":"2023-06-09T11:59:41.423589+0000","last_peered":"2023-06-09T11:59:41.423589+0000","last_clean":"2023-06-09T11:59:41.423589+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T11:59:41.423589+0000","last_undegraded":"2023-06-09T11:59:41.423589+0000","last_fullsized":"2023-06-09T11:59:41.423589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3357,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:47.397661+0000","last_change":"2023-06-09T12:01:30.955115+0000","last_active":"2023-06-09T12:00:47.397661+0000","last_peered":"2023-06-09T12:00:47.397661+0000","last_clean":"2023-06-09T12:00:47.397661+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:00:47.397661+0000","last_undegraded":"2023-06-09T12:00:47.397661+0000","last_fullsized":"2023-06-09T12:00:47.397661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3018,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121483+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:42.121483+0000","last_peered":"2023-06-09T11:59:42.121483+0000","last_clean":"2023-06-09T11:59:42.121483+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:42.121483+0000","last_undegraded":"2023-06-09T11:59:42.121483+0000","last_fullsized":"2023-06-09T11:59:42.121483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905252+0000","last_change":"2023-06-09T11:18:26.263525+0000","last_active":"2023-06-09T11:59:44.905252+0000","last_peered":"2023-06-09T11:59:44.905252+0000","last_clean":"2023-06-09T11:59:44.905252+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T11:59:44.905252+0000","last_undegraded":"2023-06-09T11:59:44.905252+0000","last_fullsized":"2023-06-09T11:59:44.905252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3318,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.301288+0000","last_change":"2023-06-09T12:01:35.754891+0000","last_active":"2023-06-09T12:00:52.301288+0000","last_peered":"2023-06-09T12:00:52.301288+0000","last_clean":"2023-06-09T12:00:52.301288+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:00:52.301288+0000","last_undegraded":"2023-06-09T12:00:52.301288+0000","last_fullsized":"2023-06-09T12:00:52.301288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3135,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423662+0000","last_change":"2023-06-09T11:18:26.320325+0000","last_active":"2023-06-09T11:59:41.423662+0000","last_peered":"2023-06-09T11:59:41.423662+0000","last_clean":"2023-06-09T11:59:41.423662+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T11:59:41.423662+0000","last_undegraded":"2023-06-09T11:59:41.423662+0000","last_fullsized":"2023-06-09T11:59:41.423662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3306,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.491920+0000","last_change":"2023-06-09T12:01:31.393774+0000","last_active":"2023-06-09T12:00:52.491920+0000","last_peered":"2023-06-09T12:00:52.491920+0000","last_clean":"2023-06-09T12:00:52.491920+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:00:52.491920+0000","last_undegraded":"2023-06-09T12:00:52.491920+0000","last_fullsized":"2023-06-09T12:00:52.491920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2963,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:52.293286+0000","last_change":"2023-06-09T12:00:36.885523+0000","last_active":"2023-06-09T11:59:52.293286+0000","last_peered":"2023-06-09T11:59:52.293286+0000","last_clean":"2023-06-09T11:59:52.293286+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:52.293286+0000","last_undegraded":"2023-06-09T11:59:52.293286+0000","last_fullsized":"2023-06-09T11:59:52.293286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:19.292684+0000","last_change":"2023-06-09T12:01:37.403288+0000","last_active":"2023-06-09T12:00:19.292684+0000","last_peered":"2023-06-09T12:00:19.292684+0000","last_clean":"2023-06-09T12:00:19.292684+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:00:19.292684+0000","last_undegraded":"2023-06-09T12:00:19.292684+0000","last_fullsized":"2023-06-09T12:00:19.292684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1035,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.404606+0000","last_change":"2023-06-09T12:01:13.693733+0000","last_active":"2023-06-09T12:00:02.404606+0000","last_peered":"2023-06-09T12:00:02.404606+0000","last_clean":"2023-06-09T12:00:02.404606+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:00:02.404606+0000","last_undegraded":"2023-06-09T12:00:02.404606+0000","last_fullsized":"2023-06-09T12:00:02.404606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1082,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:55.067574+0000","last_change":"2023-06-09T12:00:45.031530+0000","last_active":"2023-06-09T11:59:55.067574+0000","last_peered":"2023-06-09T11:59:55.067574+0000","last_clean":"2023-06-09T11:59:55.067574+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:55.067574+0000","last_undegraded":"2023-06-09T11:59:55.067574+0000","last_fullsized":"2023-06-09T11:59:55.067574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3559,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.294703+0000","last_change":"2023-06-09T12:00:41.875998+0000","last_active":"2023-06-09T12:00:02.294703+0000","last_peered":"2023-06-09T12:00:02.294703+0000","last_clean":"2023-06-09T12:00:02.294703+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:02.294703+0000","last_undegraded":"2023-06-09T12:00:02.294703+0000","last_fullsized":"2023-06-09T12:00:02.294703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3024,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423569+0000","last_change":"2023-06-09T11:18:26.311102+0000","last_active":"2023-06-09T11:59:41.423569+0000","last_peered":"2023-06-09T11:59:41.423569+0000","last_clean":"2023-06-09T11:59:41.423569+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T11:59:41.423569+0000","last_undegraded":"2023-06-09T11:59:41.423569+0000","last_fullsized":"2023-06-09T11:59:41.423569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:25.604660+0000","last_change":"2023-06-09T12:00:25.604660+0000","last_active":"2023-06-09T12:00:25.604660+0000","last_peered":"2023-06-09T12:00:25.604660+0000","last_clean":"2023-06-09T12:00:25.604660+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:00:25.604660+0000","last_undegraded":"2023-06-09T12:00:25.604660+0000","last_fullsized":"2023-06-09T12:00:25.604660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'960","reported_seq":3843,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:45.619486+0000","last_change":"2023-06-09T12:01:37.725220+0000","last_active":"2023-06-09T12:01:45.619486+0000","last_peered":"2023-06-09T12:01:45.619486+0000","last_clean":"2023-06-09T12:01:45.619486+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:01:45.619486+0000","last_undegraded":"2023-06-09T12:01:45.619486+0000","last_fullsized":"2023-06-09T12:01:45.619486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":960,"log_dups_size":0,"ondisk_log_size":960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":728,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:27.395619+0000","last_change":"2023-06-09T12:00:46.032468+0000","last_active":"2023-06-09T12:00:27.395619+0000","last_peered":"2023-06-09T12:00:27.395619+0000","last_clean":"2023-06-09T12:00:27.395619+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:27.395619+0000","last_undegraded":"2023-06-09T12:00:27.395619+0000","last_fullsized":"2023-06-09T12:00:27.395619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289712+0000","last_change":"2023-06-09T12:01:16.188471+0000","last_active":"2023-06-09T12:00:42.289712+0000","last_peered":"2023-06-09T12:00:42.289712+0000","last_clean":"2023-06-09T12:00:42.289712+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:00:42.289712+0000","last_undegraded":"2023-06-09T12:00:42.289712+0000","last_fullsized":"2023-06-09T12:00:42.289712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394854+0000","last_change":"2023-06-09T12:00:47.993494+0000","last_active":"2023-06-09T12:00:22.394854+0000","last_peered":"2023-06-09T12:00:22.394854+0000","last_clean":"2023-06-09T12:00:22.394854+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:22.394854+0000","last_undegraded":"2023-06-09T12:00:22.394854+0000","last_fullsized":"2023-06-09T12:00:22.394854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394838+0000","last_change":"2023-06-09T12:00:53.983699+0000","last_active":"2023-06-09T12:00:22.394838+0000","last_peered":"2023-06-09T12:00:22.394838+0000","last_clean":"2023-06-09T12:00:22.394838+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:22.394838+0000","last_undegraded":"2023-06-09T12:00:22.394838+0000","last_fullsized":"2023-06-09T12:00:22.394838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3540,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.491865+0000","last_change":"2023-06-09T12:01:47.287680+0000","last_active":"2023-06-09T12:00:52.491865+0000","last_peered":"2023-06-09T12:00:52.491865+0000","last_clean":"2023-06-09T12:00:52.491865+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:00:52.491865+0000","last_undegraded":"2023-06-09T12:00:52.491865+0000","last_fullsized":"2023-06-09T12:00:52.491865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445722+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445722+0000","last_peered":"2023-06-09T12:00:04.445722+0000","last_clean":"2023-06-09T12:00:04.445722+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445722+0000","last_undegraded":"2023-06-09T12:00:04.445722+0000","last_fullsized":"2023-06-09T12:00:04.445722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1636,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.410160+0000","last_change":"2023-06-09T12:01:14.658585+0000","last_active":"2023-06-09T12:00:42.410160+0000","last_peered":"2023-06-09T12:00:42.410160+0000","last_clean":"2023-06-09T12:00:42.410160+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:00:42.410160+0000","last_undegraded":"2023-06-09T12:00:42.410160+0000","last_fullsized":"2023-06-09T12:00:42.410160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121371+0000","last_change":"2023-06-09T11:18:26.853311+0000","last_active":"2023-06-09T11:59:42.121371+0000","last_peered":"2023-06-09T11:59:42.121371+0000","last_clean":"2023-06-09T11:59:42.121371+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T11:59:42.121371+0000","last_undegraded":"2023-06-09T11:59:42.121371+0000","last_fullsized":"2023-06-09T11:59:42.121371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.246824+0000","last_change":"2023-06-09T12:00:47.290422+0000","last_active":"2023-06-09T12:00:17.246824+0000","last_peered":"2023-06-09T12:00:17.246824+0000","last_clean":"2023-06-09T12:00:17.246824+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:17.246824+0000","last_undegraded":"2023-06-09T12:00:17.246824+0000","last_fullsized":"2023-06-09T12:00:17.246824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":897,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207703+0000","last_change":"2023-06-09T11:10:49.687769+0000","last_active":"2023-06-09T11:59:44.207703+0000","last_peered":"2023-06-09T11:59:44.207703+0000","last_clean":"2023-06-09T11:59:44.207703+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T11:59:44.207703+0000","last_undegraded":"2023-06-09T11:59:44.207703+0000","last_fullsized":"2023-06-09T11:59:44.207703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3889,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121324+0000","last_change":"2023-06-09T11:18:26.844396+0000","last_active":"2023-06-09T11:59:42.121324+0000","last_peered":"2023-06-09T11:59:42.121324+0000","last_clean":"2023-06-09T11:59:42.121324+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T11:59:42.121324+0000","last_undegraded":"2023-06-09T11:59:42.121324+0000","last_fullsized":"2023-06-09T11:59:42.121324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"74'1053","reported_seq":1263,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:33.295018+0000","last_change":"2023-06-09T12:01:32.922433+0000","last_active":"2023-06-09T12:01:33.295018+0000","last_peered":"2023-06-09T12:01:33.295018+0000","last_clean":"2023-06-09T12:01:33.295018+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:33.295018+0000","last_undegraded":"2023-06-09T12:01:33.295018+0000","last_fullsized":"2023-06-09T12:01:33.295018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121335+0000","last_change":"2023-06-09T11:18:26.844138+0000","last_active":"2023-06-09T11:59:42.121335+0000","last_peered":"2023-06-09T11:59:42.121335+0000","last_clean":"2023-06-09T11:59:42.121335+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T11:59:42.121335+0000","last_undegraded":"2023-06-09T11:59:42.121335+0000","last_fullsized":"2023-06-09T11:59:42.121335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.338006+0000","last_change":"2023-06-09T12:00:47.290438+0000","last_active":"2023-06-09T12:00:15.338006+0000","last_peered":"2023-06-09T12:00:15.338006+0000","last_clean":"2023-06-09T12:00:15.338006+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:00:15.338006+0000","last_undegraded":"2023-06-09T12:00:15.338006+0000","last_fullsized":"2023-06-09T12:00:15.338006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":315,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218669+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218669+0000","last_peered":"2023-06-09T11:59:58.218669+0000","last_clean":"2023-06-09T11:59:58.218669+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218669+0000","last_undegraded":"2023-06-09T11:59:58.218669+0000","last_fullsized":"2023-06-09T11:59:58.218669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1347,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602275+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602275+0000","last_peered":"2023-06-09T12:00:03.602275+0000","last_clean":"2023-06-09T12:00:03.602275+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602275+0000","last_undegraded":"2023-06-09T12:00:03.602275+0000","last_fullsized":"2023-06-09T12:00:03.602275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3299,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493261+0000","last_change":"2023-06-09T12:01:39.395266+0000","last_active":"2023-06-09T12:01:02.493261+0000","last_peered":"2023-06-09T12:01:02.493261+0000","last_clean":"2023-06-09T12:01:02.493261+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:01:02.493261+0000","last_undegraded":"2023-06-09T12:01:02.493261+0000","last_fullsized":"2023-06-09T12:01:02.493261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:47.289624+0000","last_change":"2023-06-09T12:00:52.398167+0000","last_active":"2023-06-09T11:59:47.289624+0000","last_peered":"2023-06-09T11:59:47.289624+0000","last_clean":"2023-06-09T11:59:47.289624+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:47.289624+0000","last_undegraded":"2023-06-09T11:59:47.289624+0000","last_fullsized":"2023-06-09T11:59:47.289624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423608+0000","last_change":"2023-06-09T11:18:26.329526+0000","last_active":"2023-06-09T11:59:41.423608+0000","last_peered":"2023-06-09T11:59:41.423608+0000","last_clean":"2023-06-09T11:59:41.423608+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T11:59:41.423608+0000","last_undegraded":"2023-06-09T11:59:41.423608+0000","last_fullsized":"2023-06-09T11:59:41.423608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":821,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121243+0000","last_change":"2023-06-09T11:10:49.682497+0000","last_active":"2023-06-09T11:59:42.121243+0000","last_peered":"2023-06-09T11:59:42.121243+0000","last_clean":"2023-06-09T11:59:42.121243+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T11:59:42.121243+0000","last_undegraded":"2023-06-09T11:59:42.121243+0000","last_fullsized":"2023-06-09T11:59:42.121243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4112,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423567+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:41.423567+0000","last_peered":"2023-06-09T11:59:41.423567+0000","last_clean":"2023-06-09T11:59:41.423567+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:41.423567+0000","last_undegraded":"2023-06-09T11:59:41.423567+0000","last_fullsized":"2023-06-09T11:59:41.423567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465532+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465532+0000","last_peered":"2023-06-09T11:59:48.465532+0000","last_clean":"2023-06-09T11:59:48.465532+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465532+0000","last_undegraded":"2023-06-09T11:59:48.465532+0000","last_fullsized":"2023-06-09T11:59:48.465532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.652148+0000","last_change":"2023-06-09T12:00:37.490252+0000","last_active":"2023-06-09T12:00:36.652148+0000","last_peered":"2023-06-09T12:00:36.652148+0000","last_clean":"2023-06-09T12:00:36.652148+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:36.652148+0000","last_undegraded":"2023-06-09T12:00:36.652148+0000","last_fullsized":"2023-06-09T12:00:36.652148+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":828,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293122+0000","last_change":"2023-06-09T12:01:34.883218+0000","last_active":"2023-06-09T12:00:42.293122+0000","last_peered":"2023-06-09T12:00:42.293122+0000","last_clean":"2023-06-09T12:00:42.293122+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:00:42.293122+0000","last_undegraded":"2023-06-09T12:00:42.293122+0000","last_fullsized":"2023-06-09T12:00:42.293122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":881,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423490+0000","last_change":"2023-06-09T11:10:49.684215+0000","last_active":"2023-06-09T11:59:41.423490+0000","last_peered":"2023-06-09T11:59:41.423490+0000","last_clean":"2023-06-09T11:59:41.423490+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T11:59:41.423490+0000","last_undegraded":"2023-06-09T11:59:41.423490+0000","last_fullsized":"2023-06-09T11:59:41.423490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1061,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446248+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446248+0000","last_peered":"2023-06-09T12:00:25.446248+0000","last_clean":"2023-06-09T12:00:25.446248+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446248+0000","last_undegraded":"2023-06-09T12:00:25.446248+0000","last_fullsized":"2023-06-09T12:00:25.446248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516956+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:43.516956+0000","last_peered":"2023-06-09T11:59:43.516956+0000","last_clean":"2023-06-09T11:59:43.516956+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:43.516956+0000","last_undegraded":"2023-06-09T11:59:43.516956+0000","last_fullsized":"2023-06-09T11:59:43.516956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3427,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.491822+0000","last_change":"2023-06-09T12:01:37.414157+0000","last_active":"2023-06-09T12:00:52.491822+0000","last_peered":"2023-06-09T12:00:52.491822+0000","last_clean":"2023-06-09T12:00:52.491822+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:00:52.491822+0000","last_undegraded":"2023-06-09T12:00:52.491822+0000","last_fullsized":"2023-06-09T12:00:52.491822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121452+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:42.121452+0000","last_peered":"2023-06-09T11:59:42.121452+0000","last_clean":"2023-06-09T11:59:42.121452+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:42.121452+0000","last_undegraded":"2023-06-09T11:59:42.121452+0000","last_fullsized":"2023-06-09T11:59:42.121452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":59571,"ondisk_log_size":59571,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":631268,"num_omap_keys":1195,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30643,"ondisk_log_size":30643,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254608,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74752,"kb_avail":94117232,"statfs":{"total":96636764160,"available":96376045568,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76546048},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69599999999999995,"15min":0.65500000000000003},"min":{"1min":0.55500000000000005,"5min":0.40600000000000003,"15min":0.28699999999999998},"max":{"1min":0.96099999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.752},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.93700000000000006,"5min":1.05,"15min":1.05},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89000000000000001,"5min":1.099,"15min":1.099},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.78000000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.41799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.56200000000000006,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.47199999999999998,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.94999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.65200000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.84099999999999997,"5min":1.044,"15min":1.0920000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.621,"5min":0.496,"15min":0.42599999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.84799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575464,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203160,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47040,"kb_avail":94168680,"statfs":{"total":96636764160,"available":96428728320,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48168960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.30599999999999999},"max":{"1min":0.86799999999999999,"5min":1.069,"15min":1.069},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67000000000000004,"15min":0.66300000000000003},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1.081},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72699999999999998,"15min":0.69599999999999995},"min":{"1min":0.61799999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.86099999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.57899999999999996,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":1.024,"15min":1.125},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.96499999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.59799999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.036,"15min":1.1719999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.73599999999999999},"min":{"1min":0.65700000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":1.034,"15min":1.0549999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.95799999999999996,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.624},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":1.052,"15min":1.0589999999999999},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.69699999999999995},"min":{"1min":0.53200000000000003,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.64500000000000002},"min":{"1min":0.499,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.97299999999999998,"15min":1.071},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575464,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400688,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94272,"kb_avail":93971152,"statfs":{"total":96636764160,"available":96226459648,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96534528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.495},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.51500000000000001,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.81399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.091,"15min":1.9790000000000001},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.61099999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":1.0649999999999999,"15min":1.609},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81000000000000005,"15min":0.79200000000000004},"min":{"1min":0.64900000000000002,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.077,"5min":1.1459999999999999,"15min":1.774},"last":0.96899999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80200000000000005,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.774},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85099999999999998,"15min":0.81699999999999995},"min":{"1min":0.73499999999999999,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.92200000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83099999999999996,"15min":0.83799999999999997},"min":{"1min":0.71499999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.9059999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":0.99399999999999999,"5min":1.117,"15min":1.921},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.109,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.226,"15min":1.827},"last":0.84799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332072,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104256,"kb_avail":94039768,"statfs":{"total":96636764160,"available":96296722432,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106758144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.441},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.0900000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.48499999999999999},"min":{"1min":0.38300000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.88,"5min":0.88,"15min":1.2909999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.1060000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.76200000000000001},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.21},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":1.05,"15min":1.236},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.63200000000000001,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":1.002,"5min":1.0369999999999999,"15min":1.165},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.0309999999999999,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.73799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.68100000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.1299999999999999,"5min":1.159,"15min":1.228},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.71599999999999997,"5min":0.58199999999999996,"15min":0.48599999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.29},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237152,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94134688,"statfs":{"total":96636764160,"available":96393920512,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41399999999999998},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.55900000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.441},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.434,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.999,"15min":1.0940000000000001},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.56299999999999994,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":1.367},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.629,"15min":0.60699999999999998},"min":{"1min":0.498,"5min":0.44,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.73599999999999999},"min":{"1min":0.63900000000000001,"5min":0.443,"15min":0.373},"max":{"1min":1.034,"5min":1.179,"15min":1.8220000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":0.97199999999999998,"5min":1.2450000000000001,"15min":1.601},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83799999999999997,"15min":0.82099999999999995},"min":{"1min":0.70199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96999999999999997,"5min":1.288,"15min":1.6479999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.496,"15min":0.438},"max":{"1min":1.089,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.625},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.91400000000000003,"5min":1.121,"15min":1.3540000000000001},"last":0.83399999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.58899999999999997,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":0.98399999999999999,"5min":1.208,"15min":1.248},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.026,"5min":1.262,"15min":1.337},"last":0.91500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575464,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433660,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110464,"kb_avail":93938180,"statfs":{"total":96636764160,"available":96192696320,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":113115136},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.34300000000000003,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":0.71299999999999997,"15min":1.7969999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.30599999999999999},"max":{"1min":0.90900000000000003,"5min":1.2130000000000001,"15min":1.5669999999999999},"last":0.47099999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.40999999999999998,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.496,"15min":0.51500000000000001},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.50700000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.878,"5min":1.355,"15min":1.355},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.78400000000000003,"5min":1.091,"15min":1.198},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54800000000000004,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":1.073,"5min":1.073,"15min":1.635},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.90600000000000003,"5min":3.081,"15min":3.081},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.754},"min":{"1min":0.66400000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.165,"5min":1.198,"15min":1.2110000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.66600000000000004,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.1970000000000001,"5min":1.234,"15min":1.367},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.503,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.286,"15min":1.286},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.61899999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.103,"5min":1.103,"15min":3.3490000000000002},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65700000000000003,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.413},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.88,"15min":0.871},"min":{"1min":0.73099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1659999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.94699999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575463,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417812,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125056,"kb_avail":93954028,"statfs":{"total":96636764160,"available":96208924672,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128057344},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.69199999999999995,"15min":0.65700000000000003},"min":{"1min":0.53600000000000003,"5min":0.39400000000000002,"15min":0.28199999999999997},"max":{"1min":0.96899999999999997,"5min":1.137,"15min":1.137},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76300000000000001,"15min":0.73099999999999998},"min":{"1min":0.64200000000000002,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.93100000000000005,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73399999999999999,"15min":0.70099999999999996},"min":{"1min":0.48899999999999999,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.90500000000000003,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77800000000000002,"15min":0.746},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.88600000000000001,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.91300000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67500000000000004,"15min":0.63900000000000001},"min":{"1min":0.64600000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94399999999999995,"5min":1.609,"15min":1.609},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.67100000000000004,"15min":0.65200000000000002},"min":{"1min":0.59499999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.94899999999999995,"5min":0.997,"15min":1.159},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.68899999999999995,"15min":0.67300000000000004},"min":{"1min":0.58499999999999996,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.98699999999999999,"5min":1.407,"15min":1.407},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.749,"15min":0.73099999999999998},"min":{"1min":0.57299999999999995,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.98799999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.42399999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.746,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63400000000000001,"15min":0.60799999999999998},"min":{"1min":0.47599999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.92000000000000004,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.753,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.69699999999999995},"min":{"1min":0.54800000000000004,"5min":0.44,"15min":0.44},"max":{"1min":0.95499999999999996,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:01:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.59799999999999998,"15min":0.58499999999999996},"min":{"1min":0.441,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.83999999999999997,"5min":1.032,"15min":1.032},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.67400000000000004,"15min":0.65300000000000002},"min":{"1min":0.47999999999999998,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.89600000000000002,"5min":1.361,"15min":1.361},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575463,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288280,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70912,"kb_avail":94083560,"statfs":{"total":96636764160,"available":96341565440,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72613888},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.69399999999999995},"min":{"1min":0.57699999999999996,"5min":0.40100000000000002,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":1.177,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.91300000000000003,"15min":0.72199999999999998},"min":{"1min":0.52400000000000002,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":1.018,"5min":19.715,"15min":19.715},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.70799999999999996},"min":{"1min":0.46000000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.88900000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74199999999999999,"15min":0.69099999999999995},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.76000000000000001},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":14.439,"15min":14.439},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.88500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.375},"max":{"1min":1.006,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.89600000000000002,"15min":0.81100000000000005},"min":{"1min":0.624,"5min":0.55900000000000005,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.82299999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.2889999999999999,"15min":0.82499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.76700000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59799999999999998,"5min":1.3280000000000001,"15min":0.83699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78200000000000003,"15min":0.68600000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.40400000000000003},"max":{"1min":0.78400000000000003,"5min":12.163,"15min":12.163},"last":0.753},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.76400000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.97199999999999998,"15min":0.81599999999999995},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":19.834,"15min":19.834},"last":0.89500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:01:49.885 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.885 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.885 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.886 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.886 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.886 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.887 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.887 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.888 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.888 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.888 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.889 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.889 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.889 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.890 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.890 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.890 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.891 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.891 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.891 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.892 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.892 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.892 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.893 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.893 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.893 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.894 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.894 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.894 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.895 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.895 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.895 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.896 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.896 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.897 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.897 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.897 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.898 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.898 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.898 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.899 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.899 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.899 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.900 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.900 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.900 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.901 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.901 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.901 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.902 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.902 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.902 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.903 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.903 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.903 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.904 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.904 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.905 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.905 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.905 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.906 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.906 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.906 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.907 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.907 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.907 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.908 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.908 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.909 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.909 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.909 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.910 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.910 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.910 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.911 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.911 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.911 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.912 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.912 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.912 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.913 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.913 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.913 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.914 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.914 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.914 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:01:49.915 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:01:56.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:01:56.266 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:09.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:02:10.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.125+0000 7f10fb7ff700 1 Processor -- start 2023-06-09T12:02:10.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.125+0000 7f10fb7ff700 1 -- start start 2023-06-09T12:02:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.126+0000 7f10fb7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.126+0000 7f10fb7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f41580a0 con 0x7f10f4151510 2023-06-09T12:02:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.126+0000 7f10fb7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f40c6180 con 0x7f10f4071060 2023-06-09T12:02:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.126+0000 7f10fb7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f40714e0 con 0x7f10f40c5b40 2023-06-09T12:02:10.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f959b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f10f4151510 0x7f10f415b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35874/0 (socket says 172.21.15.7:35874) 2023-06-09T12:02:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f959b700 1 -- 172.21.15.7:0/1679486447 learned_addr learned my addr 172.21.15.7:0/1679486447 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:02:10.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f8d9a700 1 --2- 172.21.15.7:0/1679486447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662495421 0 0) 0x7f10f41580a0 con 0x7f10f4151510 2023-06-09T12:02:10.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e00035b0 con 0x7f10f4151510 2023-06-09T12:02:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307332809 0 0) 0x7f10f40c6180 con 0x7f10f4071060 2023-06-09T12:02:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f41580a0 con 0x7f10f4071060 2023-06-09T12:02:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f8d9a700 1 -- 172.21.15.7:0/1679486447 >> v1:172.21.15.117:6789/0 conn(0x7f10f4071060 legacy=0x7f10f4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f8d9a700 1 -- 172.21.15.7:0/1679486447 >> v1:172.21.15.7:6789/0 conn(0x7f10f4151510 legacy=0x7f10f415b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f8d9a700 1 -- 172.21.15.7:0/1679486447 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f4157490 con 0x7f10f40c5b40 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.127+0000 7f10f8d9a700 1 --2- 172.21.15.7:0/1679486447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4070b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009870 tx=0x7f10e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c23f45da87232b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4011830 con 0x7f10f40c5b40 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e4011990 con 0x7f10f40c5b40 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10ebfff700 1 -- 172.21.15.7:0/1679486447 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e4011d20 con 0x7f10f40c5b40 2023-06-09T12:02:10.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10fb7ff700 1 -- 172.21.15.7:0/1679486447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 msgr2=0x7f10f4070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/1679486447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4070b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009870 tx=0x7f10e4009b80 comp rx=0 tx=0).stop 2023-06-09T12:02:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10fb7ff700 1 -- 172.21.15.7:0/1679486447 shutdown_connections 2023-06-09T12:02:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/1679486447 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4070b20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.128+0000 7f10fb7ff700 1 -- 172.21.15.7:0/1679486447 >> 172.21.15.7:0/1679486447 conn(0x7f10f406a4d0 msgr2=0x7f10f406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:10.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.129+0000 7f10fb7ff700 1 -- 172.21.15.7:0/1679486447 shutdown_connections 2023-06-09T12:02:10.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.129+0000 7f10fb7ff700 1 -- 172.21.15.7:0/1679486447 wait complete. 2023-06-09T12:02:10.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.130+0000 7f10fb7ff700 1 Processor -- start 2023-06-09T12:02:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.131+0000 7f10fb7ff700 1 -- start start 2023-06-09T12:02:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.131+0000 7f10fb7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f4071060 0x7f10f414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.131+0000 7f10fb7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.131+0000 7f10fb7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:10.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10fb7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41535d0 con 0x7f10f4151510 2023-06-09T12:02:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10fb7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41efbe0 con 0x7f10f4071060 2023-06-09T12:02:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10fb7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f414c600 con 0x7f10f40c5b40 2023-06-09T12:02:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44648/0 (socket says 172.21.15.7:44648) 2023-06-09T12:02:10.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 -- 172.21.15.7:0/851296780 learned_addr learned my addr 172.21.15.7:0/851296780 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:02:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f8d9a700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f959b700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f4071060 0x7f10f414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 -- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 msgr2=0x7f10f4153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 -- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f4071060 msgr2=0x7f10f414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f4071060 0x7f10f414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 -- 172.21.15.7:0/851296780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40094a0 con 0x7f10f4151510 2023-06-09T12:02:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f8d9a700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:02:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.132+0000 7f10f9d9c700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00cc20 tx=0x7f10ec00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:10.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.133+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec00ac20 con 0x7f10f4151510 2023-06-09T12:02:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.133+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ec00ad80 con 0x7f10f4151510 2023-06-09T12:02:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.133+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ec003c20 con 0x7f10f4151510 2023-06-09T12:02:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.133+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f4157a50 con 0x7f10f4151510 2023-06-09T12:02:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.133+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f4157770 con 0x7f10f4151510 2023-06-09T12:02:10.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.134+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f4157d30 con 0x7f10f4151510 2023-06-09T12:02:10.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.135+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f10ec0050d0 con 0x7f10f4151510 2023-06-09T12:02:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.136+0000 7f10e9ffb700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 0x7f10d806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.136+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f10ec089f90 con 0x7f10f4151510 2023-06-09T12:02:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.136+0000 7f10f959b700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 0x7f10d806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.137+0000 7f10f959b700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 0x7f10d806d1c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f10f00011e0 tx=0x7f10f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:10.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.141+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f10ec0546c0 con 0x7f10f4151510 2023-06-09T12:02:10.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.346+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10f41efd50 con 0x7f10d806ad10 2023-06-09T12:02:10.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.356+0000 7f10e9ffb700 1 -- 172.21.15.7:0/851296780 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+314456 (secure 0 0 0) 0x7f10f41efd50 con 0x7f10d806ad10 2023-06-09T12:02:10.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:02:10.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 msgr2=0x7f10d806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 0x7f10d806d1c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f10f00011e0 tx=0x7f10f0009040 comp rx=0 tx=0).stop 2023-06-09T12:02:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 msgr2=0x7f10f4158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00cc20 tx=0x7f10ec00a060 comp rx=0 tx=0).stop 2023-06-09T12:02:10.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 shutdown_connections 2023-06-09T12:02:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10f4151510 0x7f10f4158530 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10f4071060 0x7f10f414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f10d806ad10 0x7f10d806d1c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 --2- 172.21.15.7:0/851296780 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10f40c5b40 0x7f10f4153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:10.366 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 >> 172.21.15.7:0/851296780 conn(0x7f10f406a4d0 msgr2=0x7f10f406a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 shutdown_connections 2023-06-09T12:02:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:10.360+0000 7f10fb7ff700 1 -- 172.21.15.7:0/851296780 wait complete. 2023-06-09T12:02:10.367 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:02:10.382 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1611,"stamp":"2023-06-09T12:02:08.966464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":664252,"num_omap_keys":1259,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90502,"ondisk_log_size":90502,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2567816,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":684480,"kb_avail":752406904,"statfs":{"total":773094113280,"available":770464669696,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":700907520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003391"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3465,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.376091+0000","last_change":"2023-06-09T12:01:56.911297+0000","last_active":"2023-06-09T12:00:52.376091+0000","last_peered":"2023-06-09T12:00:52.376091+0000","last_clean":"2023-06-09T12:00:52.376091+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:52.376091+0000","last_undegraded":"2023-06-09T12:00:52.376091+0000","last_fullsized":"2023-06-09T12:00:52.376091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905267+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:44.905267+0000","last_peered":"2023-06-09T11:59:44.905267+0000","last_clean":"2023-06-09T11:59:44.905267+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:44.905267+0000","last_undegraded":"2023-06-09T11:59:44.905267+0000","last_fullsized":"2023-06-09T11:59:44.905267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3453,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493236+0000","last_change":"2023-06-09T12:02:02.302498+0000","last_active":"2023-06-09T12:01:02.493236+0000","last_peered":"2023-06-09T12:01:02.493236+0000","last_clean":"2023-06-09T12:01:02.493236+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:01:02.493236+0000","last_undegraded":"2023-06-09T12:01:02.493236+0000","last_fullsized":"2023-06-09T12:01:02.493236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207940+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:44.207940+0000","last_peered":"2023-06-09T11:59:44.207940+0000","last_clean":"2023-06-09T11:59:44.207940+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:44.207940+0000","last_undegraded":"2023-06-09T11:59:44.207940+0000","last_fullsized":"2023-06-09T11:59:44.207940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3403,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381048+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:01:37.381048+0000","last_peered":"2023-06-09T12:01:37.381048+0000","last_clean":"2023-06-09T12:01:37.381048+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:01:37.381048+0000","last_undegraded":"2023-06-09T12:01:37.381048+0000","last_fullsized":"2023-06-09T12:01:37.381048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3234,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381026+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:01:37.381026+0000","last_peered":"2023-06-09T12:01:37.381026+0000","last_clean":"2023-06-09T12:01:37.381026+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:37.381026+0000","last_undegraded":"2023-06-09T12:01:37.381026+0000","last_fullsized":"2023-06-09T12:01:37.381026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'883","reported_seq":3337,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:03.876720+0000","last_change":"2023-06-09T12:01:59.021680+0000","last_active":"2023-06-09T12:02:03.876720+0000","last_peered":"2023-06-09T12:02:03.876720+0000","last_clean":"2023-06-09T12:02:03.876720+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:03.876720+0000","last_undegraded":"2023-06-09T12:02:03.876720+0000","last_fullsized":"2023-06-09T12:02:03.876720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:46.296255+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:46.296255+0000","last_peered":"2023-06-09T11:59:46.296255+0000","last_clean":"2023-06-09T11:59:46.296255+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:46.296255+0000","last_undegraded":"2023-06-09T11:59:46.296255+0000","last_fullsized":"2023-06-09T11:59:46.296255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'953","reported_seq":3427,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207945+0000","last_change":"2023-06-09T11:18:26.333231+0000","last_active":"2023-06-09T11:59:44.207945+0000","last_peered":"2023-06-09T11:59:44.207945+0000","last_clean":"2023-06-09T11:59:44.207945+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T11:59:44.207945+0000","last_undegraded":"2023-06-09T11:59:44.207945+0000","last_fullsized":"2023-06-09T11:59:44.207945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3734,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423588+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:41.423588+0000","last_peered":"2023-06-09T11:59:41.423588+0000","last_clean":"2023-06-09T11:59:41.423588+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:41.423588+0000","last_undegraded":"2023-06-09T11:59:41.423588+0000","last_fullsized":"2023-06-09T11:59:41.423588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3490,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3525,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394812+0000","last_change":"2023-06-09T12:01:33.911517+0000","last_active":"2023-06-09T12:00:22.394812+0000","last_peered":"2023-06-09T12:00:22.394812+0000","last_clean":"2023-06-09T12:00:22.394812+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:00:22.394812+0000","last_undegraded":"2023-06-09T12:00:22.394812+0000","last_fullsized":"2023-06-09T12:00:22.394812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3482,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.290633+0000","last_change":"2023-06-09T12:01:51.942578+0000","last_active":"2023-06-09T12:00:22.290633+0000","last_peered":"2023-06-09T12:00:22.290633+0000","last_clean":"2023-06-09T12:00:22.290633+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:00:22.290633+0000","last_undegraded":"2023-06-09T12:00:22.290633+0000","last_fullsized":"2023-06-09T12:00:22.290633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3357,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:47.397661+0000","last_change":"2023-06-09T12:01:30.955115+0000","last_active":"2023-06-09T12:00:47.397661+0000","last_peered":"2023-06-09T12:00:47.397661+0000","last_clean":"2023-06-09T12:00:47.397661+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:00:47.397661+0000","last_undegraded":"2023-06-09T12:00:47.397661+0000","last_fullsized":"2023-06-09T12:00:47.397661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'854","reported_seq":3018,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121483+0000","last_change":"2023-06-09T11:18:26.834926+0000","last_active":"2023-06-09T11:59:42.121483+0000","last_peered":"2023-06-09T11:59:42.121483+0000","last_clean":"2023-06-09T11:59:42.121483+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T11:59:42.121483+0000","last_undegraded":"2023-06-09T11:59:42.121483+0000","last_fullsized":"2023-06-09T11:59:42.121483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3606,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289784+0000","last_change":"2023-06-09T12:01:58.002647+0000","last_active":"2023-06-09T12:00:42.289784+0000","last_peered":"2023-06-09T12:00:42.289784+0000","last_clean":"2023-06-09T12:00:42.289784+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:00:42.289784+0000","last_undegraded":"2023-06-09T12:00:42.289784+0000","last_fullsized":"2023-06-09T12:00:42.289784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3318,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.301288+0000","last_change":"2023-06-09T12:01:35.754891+0000","last_active":"2023-06-09T12:00:52.301288+0000","last_peered":"2023-06-09T12:00:52.301288+0000","last_clean":"2023-06-09T12:00:52.301288+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:00:52.301288+0000","last_undegraded":"2023-06-09T12:00:52.301288+0000","last_fullsized":"2023-06-09T12:00:52.301288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3136,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.295697+0000","last_change":"2023-06-09T12:01:52.964134+0000","last_active":"2023-06-09T12:01:02.295697+0000","last_peered":"2023-06-09T12:01:02.295697+0000","last_clean":"2023-06-09T12:01:02.295697+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:02.295697+0000","last_undegraded":"2023-06-09T12:01:02.295697+0000","last_fullsized":"2023-06-09T12:01:02.295697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:31.393774+0000","last_change":"2023-06-09T12:01:31.393774+0000","last_active":"2023-06-09T12:01:31.393774+0000","last_peered":"2023-06-09T12:01:31.393774+0000","last_clean":"2023-06-09T12:01:31.393774+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:01:31.393774+0000","last_undegraded":"2023-06-09T12:01:31.393774+0000","last_fullsized":"2023-06-09T12:01:31.393774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2963,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:52.293286+0000","last_change":"2023-06-09T12:00:36.885523+0000","last_active":"2023-06-09T11:59:52.293286+0000","last_peered":"2023-06-09T11:59:52.293286+0000","last_clean":"2023-06-09T11:59:52.293286+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T11:59:52.293286+0000","last_undegraded":"2023-06-09T11:59:52.293286+0000","last_fullsized":"2023-06-09T11:59:52.293286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3276,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516931+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:43.516931+0000","last_peered":"2023-06-09T11:59:43.516931+0000","last_clean":"2023-06-09T11:59:43.516931+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:43.516931+0000","last_undegraded":"2023-06-09T11:59:43.516931+0000","last_fullsized":"2023-06-09T11:59:43.516931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:19.292684+0000","last_change":"2023-06-09T12:01:37.403288+0000","last_active":"2023-06-09T12:00:19.292684+0000","last_peered":"2023-06-09T12:00:19.292684+0000","last_clean":"2023-06-09T12:00:19.292684+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:00:19.292684+0000","last_undegraded":"2023-06-09T12:00:19.292684+0000","last_fullsized":"2023-06-09T12:00:19.292684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121328+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:42.121328+0000","last_peered":"2023-06-09T11:59:42.121328+0000","last_clean":"2023-06-09T11:59:42.121328+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:42.121328+0000","last_undegraded":"2023-06-09T11:59:42.121328+0000","last_fullsized":"2023-06-09T11:59:42.121328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1035,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.404606+0000","last_change":"2023-06-09T12:01:13.693733+0000","last_active":"2023-06-09T12:00:02.404606+0000","last_peered":"2023-06-09T12:00:02.404606+0000","last_clean":"2023-06-09T12:00:02.404606+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:00:02.404606+0000","last_undegraded":"2023-06-09T12:00:02.404606+0000","last_fullsized":"2023-06-09T12:00:02.404606+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3096,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1082,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T11:59:55.067574+0000","last_change":"2023-06-09T12:00:45.031530+0000","last_active":"2023-06-09T11:59:55.067574+0000","last_peered":"2023-06-09T11:59:55.067574+0000","last_clean":"2023-06-09T11:59:55.067574+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T11:59:55.067574+0000","last_undegraded":"2023-06-09T11:59:55.067574+0000","last_fullsized":"2023-06-09T11:59:55.067574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3559,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:02.294703+0000","last_change":"2023-06-09T12:00:41.875998+0000","last_active":"2023-06-09T12:00:02.294703+0000","last_peered":"2023-06-09T12:00:02.294703+0000","last_clean":"2023-06-09T12:00:02.294703+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:02.294703+0000","last_undegraded":"2023-06-09T12:00:02.294703+0000","last_fullsized":"2023-06-09T12:00:02.294703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393598+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393598+0000","last_peered":"2023-06-09T11:59:54.393598+0000","last_clean":"2023-06-09T11:59:54.393598+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393598+0000","last_undegraded":"2023-06-09T11:59:54.393598+0000","last_fullsized":"2023-06-09T11:59:54.393598+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302556+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302556+0000","last_peered":"2023-06-09T12:00:17.302556+0000","last_clean":"2023-06-09T12:00:17.302556+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302556+0000","last_undegraded":"2023-06-09T12:00:17.302556+0000","last_fullsized":"2023-06-09T12:00:17.302556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1308,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:40.533538+0000","last_change":"2023-06-09T12:00:25.604660+0000","last_active":"2023-06-09T12:00:40.533538+0000","last_peered":"2023-06-09T12:00:40.533538+0000","last_clean":"2023-06-09T12:00:40.533538+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:00:40.533538+0000","last_undegraded":"2023-06-09T12:00:40.533538+0000","last_fullsized":"2023-06-09T12:00:40.533538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1034","reported_seq":3922,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:06.584835+0000","last_change":"2023-06-09T12:01:37.725220+0000","last_active":"2023-06-09T12:02:06.584835+0000","last_peered":"2023-06-09T12:02:06.584835+0000","last_clean":"2023-06-09T12:02:06.584835+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:06.584835+0000","last_undegraded":"2023-06-09T12:02:06.584835+0000","last_fullsized":"2023-06-09T12:02:06.584835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":147,"log_size":1034,"log_dups_size":0,"ondisk_log_size":1034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 29s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":728,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:27.395619+0000","last_change":"2023-06-09T12:00:46.032468+0000","last_active":"2023-06-09T12:00:27.395619+0000","last_peered":"2023-06-09T12:00:27.395619+0000","last_clean":"2023-06-09T12:00:27.395619+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:27.395619+0000","last_undegraded":"2023-06-09T12:00:27.395619+0000","last_fullsized":"2023-06-09T12:00:27.395619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289712+0000","last_change":"2023-06-09T12:01:16.188471+0000","last_active":"2023-06-09T12:00:42.289712+0000","last_peered":"2023-06-09T12:00:42.289712+0000","last_clean":"2023-06-09T12:00:42.289712+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:00:42.289712+0000","last_undegraded":"2023-06-09T12:00:42.289712+0000","last_fullsized":"2023-06-09T12:00:42.289712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394854+0000","last_change":"2023-06-09T12:00:47.993494+0000","last_active":"2023-06-09T12:00:22.394854+0000","last_peered":"2023-06-09T12:00:22.394854+0000","last_clean":"2023-06-09T12:00:22.394854+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:22.394854+0000","last_undegraded":"2023-06-09T12:00:22.394854+0000","last_fullsized":"2023-06-09T12:00:22.394854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394838+0000","last_change":"2023-06-09T12:00:53.983699+0000","last_active":"2023-06-09T12:00:22.394838+0000","last_peered":"2023-06-09T12:00:22.394838+0000","last_clean":"2023-06-09T12:00:22.394838+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:22.394838+0000","last_undegraded":"2023-06-09T12:00:22.394838+0000","last_fullsized":"2023-06-09T12:00:22.394838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3540,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.491865+0000","last_change":"2023-06-09T12:01:47.287680+0000","last_active":"2023-06-09T12:00:52.491865+0000","last_peered":"2023-06-09T12:00:52.491865+0000","last_clean":"2023-06-09T12:00:52.491865+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:00:52.491865+0000","last_undegraded":"2023-06-09T12:00:52.491865+0000","last_fullsized":"2023-06-09T12:00:52.491865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1636,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.410160+0000","last_change":"2023-06-09T12:01:14.658585+0000","last_active":"2023-06-09T12:00:42.410160+0000","last_peered":"2023-06-09T12:00:42.410160+0000","last_clean":"2023-06-09T12:00:42.410160+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:00:42.410160+0000","last_undegraded":"2023-06-09T12:00:42.410160+0000","last_fullsized":"2023-06-09T12:00:42.410160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3208,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:32.287450+0000","last_change":"2023-06-09T12:01:56.966920+0000","last_active":"2023-06-09T12:00:32.287450+0000","last_peered":"2023-06-09T12:00:32.287450+0000","last_clean":"2023-06-09T12:00:32.287450+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:00:32.287450+0000","last_undegraded":"2023-06-09T12:00:32.287450+0000","last_fullsized":"2023-06-09T12:00:32.287450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.246824+0000","last_change":"2023-06-09T12:00:47.290422+0000","last_active":"2023-06-09T12:00:17.246824+0000","last_peered":"2023-06-09T12:00:17.246824+0000","last_clean":"2023-06-09T12:00:17.246824+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:17.246824+0000","last_undegraded":"2023-06-09T12:00:17.246824+0000","last_fullsized":"2023-06-09T12:00:17.246824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'714","reported_seq":901,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:56.076433+0000","last_change":"2023-06-09T12:01:55.703944+0000","last_active":"2023-06-09T12:01:56.076433+0000","last_peered":"2023-06-09T12:01:56.076433+0000","last_clean":"2023-06-09T12:01:56.076433+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:01:56.076433+0000","last_undegraded":"2023-06-09T12:01:56.076433+0000","last_fullsized":"2023-06-09T12:01:56.076433+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3890,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289830+0000","last_change":"2023-06-09T12:02:05.102378+0000","last_active":"2023-06-09T12:00:52.289830+0000","last_peered":"2023-06-09T12:00:52.289830+0000","last_clean":"2023-06-09T12:00:52.289830+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:00:52.289830+0000","last_undegraded":"2023-06-09T12:00:52.289830+0000","last_fullsized":"2023-06-09T12:00:52.289830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3850,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.288758+0000","last_change":"2023-06-09T12:02:06.123925+0000","last_active":"2023-06-09T12:00:42.288758+0000","last_peered":"2023-06-09T12:00:42.288758+0000","last_clean":"2023-06-09T12:00:42.288758+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:00:42.288758+0000","last_undegraded":"2023-06-09T12:00:42.288758+0000","last_fullsized":"2023-06-09T12:00:42.288758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.338006+0000","last_change":"2023-06-09T12:00:47.290438+0000","last_active":"2023-06-09T12:00:15.338006+0000","last_peered":"2023-06-09T12:00:15.338006+0000","last_clean":"2023-06-09T12:00:15.338006+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:00:15.338006+0000","last_undegraded":"2023-06-09T12:00:15.338006+0000","last_fullsized":"2023-06-09T12:00:15.338006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3299,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493261+0000","last_change":"2023-06-09T12:01:39.395266+0000","last_active":"2023-06-09T12:01:02.493261+0000","last_peered":"2023-06-09T12:01:02.493261+0000","last_clean":"2023-06-09T12:01:02.493261+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:01:02.493261+0000","last_undegraded":"2023-06-09T12:01:02.493261+0000","last_fullsized":"2023-06-09T12:01:02.493261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:47.289624+0000","last_change":"2023-06-09T12:00:52.398167+0000","last_active":"2023-06-09T11:59:47.289624+0000","last_peered":"2023-06-09T11:59:47.289624+0000","last_clean":"2023-06-09T11:59:47.289624+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:47.289624+0000","last_undegraded":"2023-06-09T11:59:47.289624+0000","last_fullsized":"2023-06-09T11:59:47.289624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293138+0000","last_change":"2023-06-09T12:01:50.985146+0000","last_active":"2023-06-09T12:00:42.293138+0000","last_peered":"2023-06-09T12:00:42.293138+0000","last_clean":"2023-06-09T12:00:42.293138+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:00:42.293138+0000","last_undegraded":"2023-06-09T12:00:42.293138+0000","last_fullsized":"2023-06-09T12:00:42.293138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":822,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289843+0000","last_change":"2023-06-09T12:01:54.044290+0000","last_active":"2023-06-09T12:00:52.289843+0000","last_peered":"2023-06-09T12:00:52.289843+0000","last_clean":"2023-06-09T12:00:52.289843+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:00:52.289843+0000","last_undegraded":"2023-06-09T12:00:52.289843+0000","last_fullsized":"2023-06-09T12:00:52.289843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4112,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423567+0000","last_change":"2023-06-09T11:18:26.339271+0000","last_active":"2023-06-09T11:59:41.423567+0000","last_peered":"2023-06-09T11:59:41.423567+0000","last_clean":"2023-06-09T11:59:41.423567+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T11:59:41.423567+0000","last_undegraded":"2023-06-09T11:59:41.423567+0000","last_fullsized":"2023-06-09T11:59:41.423567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:37.490252+0000","last_change":"2023-06-09T12:01:53.400629+0000","last_active":"2023-06-09T12:00:37.490252+0000","last_peered":"2023-06-09T12:00:37.490252+0000","last_clean":"2023-06-09T12:00:37.490252+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:37.490252+0000","last_undegraded":"2023-06-09T12:00:37.490252+0000","last_fullsized":"2023-06-09T12:00:37.490252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":828,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293122+0000","last_change":"2023-06-09T12:01:34.883218+0000","last_active":"2023-06-09T12:00:42.293122+0000","last_peered":"2023-06-09T12:00:42.293122+0000","last_clean":"2023-06-09T12:00:42.293122+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:00:42.293122+0000","last_undegraded":"2023-06-09T12:00:42.293122+0000","last_fullsized":"2023-06-09T12:00:42.293122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":882,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294360+0000","last_change":"2023-06-09T12:01:48.982495+0000","last_active":"2023-06-09T12:00:52.294360+0000","last_peered":"2023-06-09T12:00:52.294360+0000","last_clean":"2023-06-09T12:00:52.294360+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:00:52.294360+0000","last_undegraded":"2023-06-09T12:00:52.294360+0000","last_fullsized":"2023-06-09T12:00:52.294360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1061,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446248+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446248+0000","last_peered":"2023-06-09T12:00:25.446248+0000","last_clean":"2023-06-09T12:00:25.446248+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446248+0000","last_undegraded":"2023-06-09T12:00:25.446248+0000","last_fullsized":"2023-06-09T12:00:25.446248+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516956+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:43.516956+0000","last_peered":"2023-06-09T11:59:43.516956+0000","last_clean":"2023-06-09T11:59:43.516956+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:43.516956+0000","last_undegraded":"2023-06-09T11:59:43.516956+0000","last_fullsized":"2023-06-09T11:59:43.516956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1622,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282073+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282073+0000","last_peered":"2023-06-09T12:00:15.282073+0000","last_clean":"2023-06-09T12:00:15.282073+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282073+0000","last_undegraded":"2023-06-09T12:00:15.282073+0000","last_fullsized":"2023-06-09T12:00:15.282073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3428,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.414157+0000","last_change":"2023-06-09T12:01:37.414157+0000","last_active":"2023-06-09T12:01:37.414157+0000","last_peered":"2023-06-09T12:01:37.414157+0000","last_clean":"2023-06-09T12:01:37.414157+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:01:37.414157+0000","last_undegraded":"2023-06-09T12:01:37.414157+0000","last_fullsized":"2023-06-09T12:01:37.414157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121452+0000","last_change":"2023-06-09T11:18:26.835283+0000","last_active":"2023-06-09T11:59:42.121452+0000","last_peered":"2023-06-09T11:59:42.121452+0000","last_clean":"2023-06-09T11:59:42.121452+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T11:59:42.121452+0000","last_undegraded":"2023-06-09T11:59:42.121452+0000","last_fullsized":"2023-06-09T11:59:42.121452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":59723,"ondisk_log_size":59723,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":664252,"num_omap_keys":1259,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30660,"ondisk_log_size":30660,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254672,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74816,"kb_avail":94117168,"statfs":{"total":96636764160,"available":96375980032,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76611584},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69599999999999995,"15min":0.65500000000000003},"min":{"1min":0.55500000000000005,"5min":0.40600000000000003,"15min":0.28699999999999998},"max":{"1min":0.96099999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.92800000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.93700000000000006,"5min":1.05,"15min":1.05},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89000000000000001,"5min":1.099,"15min":1.099},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.84899999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.41799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.56200000000000006,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.81000000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.75},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.47199999999999998,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.94999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.65200000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.84099999999999997,"5min":1.044,"15min":1.0920000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.621,"5min":0.496,"15min":0.42599999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.871}]}]},{"osd":6,"up_from":13,"seq":55834575468,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203160,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47040,"kb_avail":94168680,"statfs":{"total":96636764160,"available":96428728320,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48168960},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.30599999999999999},"max":{"1min":0.86799999999999999,"5min":1.069,"15min":1.069},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67000000000000004,"15min":0.66300000000000003},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1.081},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72699999999999998,"15min":0.69599999999999995},"min":{"1min":0.61799999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.57899999999999996,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":1.024,"15min":1.125},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.96499999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.59799999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.036,"15min":1.1719999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.73599999999999999},"min":{"1min":0.65700000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":1.034,"15min":1.0549999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.95799999999999996,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":1.052,"15min":1.0589999999999999},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.69699999999999995},"min":{"1min":0.53200000000000003,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.64500000000000002},"min":{"1min":0.499,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.97299999999999998,"15min":1.071},"last":0.624}]}]},{"osd":1,"up_from":13,"seq":55834575468,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400688,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94272,"kb_avail":93971152,"statfs":{"total":96636764160,"available":96226459648,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96534528},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.51500000000000001,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.81399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.091,"15min":1.9790000000000001},"last":0.879}]},{"osd":4,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.61099999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":1.0649999999999999,"15min":1.609},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81000000000000005,"15min":0.79200000000000004},"min":{"1min":0.64900000000000002,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.077,"5min":1.1459999999999999,"15min":1.774},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80200000000000005,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.774},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85099999999999998,"15min":0.81699999999999995},"min":{"1min":0.73499999999999999,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":0.98799999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83099999999999996,"15min":0.83799999999999997},"min":{"1min":0.71499999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.9059999999999999},"last":1.056},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":0.99399999999999999,"5min":1.117,"15min":1.921},"last":0.93400000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.109,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.226,"15min":1.827},"last":1.01}]}]},{"osd":0,"up_from":13,"seq":55834575468,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332072,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104256,"kb_avail":94039768,"statfs":{"total":96636764160,"available":96296722432,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106758144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68300000000000005,"5min":0.70799999999999996,"15min":0.86199999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.31},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.502}]},{"osd":2,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.58399999999999996,"15min":0.53600000000000003},"min":{"1min":0.45200000000000001,"5min":0.34999999999999998,"15min":0.28399999999999997},"max":{"1min":1.032,"5min":1.032,"15min":1.0900000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.50600000000000001,"15min":0.48499999999999999},"min":{"1min":0.38300000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.52900000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.57199999999999995},"min":{"1min":0.44400000000000001,"5min":0.34599999999999997,"15min":0.308},"max":{"1min":0.88,"5min":0.88,"15min":1.2909999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.68999999999999995,"15min":0.70499999999999996},"min":{"1min":0.59899999999999998,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.68200000000000005},"min":{"1min":0.498,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":1.1060000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.77900000000000003,"15min":0.76200000000000001},"min":{"1min":0.58799999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.21},"last":0.78900000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.74099999999999999},"min":{"1min":0.59999999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":1.05,"15min":1.236},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.63200000000000001,"5min":0.501,"15min":0.46600000000000003},"max":{"1min":1.002,"5min":1.0369999999999999,"15min":1.165},"last":0.70899999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85299999999999998,"15min":0.80000000000000004},"min":{"1min":0.56499999999999995,"5min":0.48099999999999998,"15min":0.46899999999999997},"max":{"1min":1.0309999999999999,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.82399999999999995,"15min":0.83799999999999997},"min":{"1min":0.73799999999999999,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:01:12 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.68100000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.1299999999999999,"5min":1.159,"15min":1.228},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.85899999999999999,"15min":0.81000000000000005},"min":{"1min":0.71599999999999997,"5min":0.58199999999999996,"15min":0.48599999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.29},"last":0.84899999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237152,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57344,"kb_avail":94134688,"statfs":{"total":96636764160,"available":96393920512,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58720256},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41399999999999998},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.55900000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.434,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.999,"15min":1.0940000000000001},"last":0.56200000000000006}]},{"osd":1,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44900000000000001,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.56299999999999994,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59599999999999997,"15min":0.58399999999999996},"min":{"1min":0.42199999999999999,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.76100000000000001,"5min":0.94799999999999995,"15min":1.367},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.57499999999999996},"min":{"1min":0.379,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.79700000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.629,"15min":0.60699999999999998},"min":{"1min":0.498,"5min":0.44,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.75800000000000001,"15min":0.73599999999999999},"min":{"1min":0.63900000000000001,"5min":0.443,"15min":0.373},"max":{"1min":1.034,"5min":1.179,"15min":1.8220000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.76100000000000001,"15min":0.751},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.30399999999999999},"max":{"1min":0.88700000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.51000000000000001,"15min":0.47099999999999997},"max":{"1min":0.97199999999999998,"5min":1.2450000000000001,"15min":1.601},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83799999999999997,"15min":0.82099999999999995},"min":{"1min":0.70199999999999996,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.96999999999999997,"5min":1.288,"15min":1.6479999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77900000000000003,"15min":0.752},"min":{"1min":0.57599999999999996,"5min":0.496,"15min":0.438},"max":{"1min":1.089,"5min":1.3029999999999999,"15min":1.3100000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.55400000000000005,"5min":0.495,"15min":0.46100000000000002},"max":{"1min":0.91400000000000003,"5min":1.121,"15min":1.3540000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:01:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79200000000000004,"15min":0.78000000000000003},"min":{"1min":0.58899999999999997,"5min":0.497,"15min":0.46800000000000003},"max":{"1min":0.98399999999999999,"5min":1.208,"15min":1.248},"last":0.753},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.626,"5min":0.52800000000000002,"15min":0.495},"max":{"1min":1.026,"5min":1.262,"15min":1.337},"last":0.69799999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575468,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433852,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":110656,"kb_avail":93937988,"statfs":{"total":96636764160,"available":96192499712,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":113311744},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.34300000000000003,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":0.71299999999999997,"15min":1.7969999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.30599999999999999},"max":{"1min":0.90900000000000003,"5min":1.2130000000000001,"15min":1.5669999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.496,"15min":0.48599999999999999},"min":{"1min":0.40999999999999998,"5min":0.27100000000000002,"15min":0.26900000000000002},"max":{"1min":0.82299999999999995,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.496,"15min":0.51500000000000001},"min":{"1min":0.39400000000000002,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.50700000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.878,"5min":1.355,"15min":1.355},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.54800000000000004},"min":{"1min":0.36699999999999999,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.78400000000000003,"5min":1.091,"15min":1.198},"last":0.53800000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.54800000000000004,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":1.073,"5min":1.073,"15min":1.635},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.75700000000000001,"15min":0.73799999999999999},"min":{"1min":0.58999999999999997,"5min":0.42999999999999999,"15min":0.41899999999999998},"max":{"1min":0.90600000000000003,"5min":3.081,"15min":3.081},"last":0.76800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.78500000000000003,"15min":0.754},"min":{"1min":0.66400000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.165,"5min":1.198,"15min":1.2110000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.66600000000000004,"5min":0.47899999999999998,"15min":0.47599999999999998},"max":{"1min":1.1970000000000001,"5min":1.234,"15min":1.367},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.78700000000000003,"15min":0.78800000000000003},"min":{"1min":0.54400000000000004,"5min":0.503,"15min":0.496},"max":{"1min":1.2649999999999999,"5min":1.286,"15min":1.286},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.76500000000000001,"15min":0.76900000000000002},"min":{"1min":0.61899999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.103,"5min":1.103,"15min":3.3490000000000002},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:01:16 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.81200000000000006,"15min":0.81999999999999995},"min":{"1min":0.65700000000000003,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.306,"5min":1.306,"15min":1.413},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.88,"15min":0.871},"min":{"1min":0.73099999999999998,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.1659999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.73799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575467,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417876,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125120,"kb_avail":93953964,"statfs":{"total":96636764160,"available":96208859136,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128122880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":1.137,"15min":1.137},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.044,"15min":1.1279999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.875,"5min":0.94399999999999995,"15min":1.609},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.64800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.997,"15min":1.159},"last":0.65000000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.98699999999999999,"15min":1.407},"last":0.628},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.59899999999999998,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.93300000000000005,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.61599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.67700000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77100000000000002,"5min":1.032,"15min":1.032},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.046,"15min":1.361},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575467,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288344,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":70976,"kb_avail":94083496,"statfs":{"total":96636764160,"available":96341499904,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72679424},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.69399999999999995},"min":{"1min":0.57699999999999996,"5min":0.40100000000000002,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":1.177,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.91300000000000003,"15min":0.72199999999999998},"min":{"1min":0.52400000000000002,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":1.018,"5min":19.715,"15min":19.715},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.70799999999999996},"min":{"1min":0.46000000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.88900000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.755}]},{"osd":2,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74199999999999999,"15min":0.69099999999999995},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.76000000000000001},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":14.439,"15min":14.439},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.88500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.375},"max":{"1min":1.006,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.89600000000000002,"15min":0.81100000000000005},"min":{"1min":0.624,"5min":0.55900000000000005,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.71099999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.2889999999999999,"15min":0.82499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.76700000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":1.3280000000000001,"15min":0.83699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78200000000000003,"15min":0.68600000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.40400000000000003},"max":{"1min":0.78400000000000003,"5min":12.163,"15min":12.163},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.97199999999999998,"15min":0.81599999999999995},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":19.834,"15min":19.834},"last":0.78700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:02:10.393 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.393 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.394 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.394 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.395 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.395 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.395 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.396 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.396 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.396 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.397 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.397 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.397 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.398 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.398 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.398 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.399 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.399 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.399 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.400 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.400 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.400 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.401 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.401 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.401 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.402 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.402 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.402 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.403 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.403 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.404 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.404 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.404 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.405 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.405 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.405 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.406 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.406 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.406 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.407 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.407 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.407 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.408 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.408 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.408 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.409 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.409 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.410 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.410 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.410 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.411 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.411 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.411 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.412 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.412 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.412 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.413 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.413 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.413 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.414 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.414 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.414 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.415 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.415 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.415 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.416 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.416 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.417 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.417 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.417 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.418 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.418 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.418 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.419 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.419 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.419 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.420 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.420 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.420 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.421 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.421 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.421 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.422 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.422 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:10.422 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:02:26.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:26.453 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:30.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:02:30.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.636+0000 7f0f549ff700 1 Processor -- start 2023-06-09T12:02:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.637+0000 7f0f549ff700 1 -- start start 2023-06-09T12:02:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f549ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f500c9fe0 0x7f0f500cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f549ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0f500cfaf0 0x7f0f501598b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f549ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5015cec0 con 0x7f0f500c9090 2023-06-09T12:02:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f549ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500ca440 con 0x7f0f500c9fe0 2023-06-09T12:02:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f549ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5015d160 con 0x7f0f500cfaf0 2023-06-09T12:02:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f500c9fe0 0x7f0f500cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0f500c9fe0 0x7f0f500cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40056/0 (socket says 172.21.15.7:40056) 2023-06-09T12:02:30.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 -- 172.21.15.7:0/2234396892 learned_addr learned my addr 172.21.15.7:0/2234396892 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:02:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f4ed9d700 1 --2- 172.21.15.7:0/2234396892 >> v2:172.21.15.117:3300/0 conn(0x7f0f500cfaf0 0x7f0f501598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 -- 172.21.15.7:0/2234396892 >> v2:172.21.15.117:3300/0 conn(0x7f0f500cfaf0 msgr2=0x7f0f501598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:02:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 --2- 172.21.15.7:0/2234396892 >> v2:172.21.15.117:3300/0 conn(0x7f0f500cfaf0 0x7f0f501598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 -- 172.21.15.7:0/2234396892 >> v1:172.21.15.7:6789/0 conn(0x7f0f500c9090 legacy=0x7f0f500cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.638+0000 7f0f47fff700 1 -- 172.21.15.7:0/2234396892 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5015d440 con 0x7f0f500c9fe0 2023-06-09T12:02:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f47fff700 1 --2- 172.21.15.7:0/2234396892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500c9fe0 0x7f0f500cf5b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ca9c0 tx=0x7f0f38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a36abee0defd3f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f4dd9b700 1 -- 172.21.15.7:0/2234396892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800b870 con 0x7f0f500c9fe0 2023-06-09T12:02:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f4dd9b700 1 -- 172.21.15.7:0/2234396892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3800b9d0 con 0x7f0f500c9fe0 2023-06-09T12:02:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f4dd9b700 1 -- 172.21.15.7:0/2234396892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800bd60 con 0x7f0f500c9fe0 2023-06-09T12:02:30.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f549ff700 1 -- 172.21.15.7:0/2234396892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500c9fe0 msgr2=0x7f0f500cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.639+0000 7f0f549ff700 1 --2- 172.21.15.7:0/2234396892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500c9fe0 0x7f0f500cf5b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f0f500ca9c0 tx=0x7f0f38009bd0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 -- 172.21.15.7:0/2234396892 shutdown_connections 2023-06-09T12:02:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 --2- 172.21.15.7:0/2234396892 >> v2:172.21.15.117:3300/0 conn(0x7f0f500cfaf0 0x7f0f501598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 --2- 172.21.15.7:0/2234396892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500c9fe0 0x7f0f500cf5b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 -- 172.21.15.7:0/2234396892 >> 172.21.15.7:0/2234396892 conn(0x7f0f50070af0 msgr2=0x7f0f50070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 -- 172.21.15.7:0/2234396892 shutdown_connections 2023-06-09T12:02:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.640+0000 7f0f549ff700 1 -- 172.21.15.7:0/2234396892 wait complete. 2023-06-09T12:02:30.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.641+0000 7f0f549ff700 1 Processor -- start 2023-06-09T12:02:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.641+0000 7f0f549ff700 1 -- start start 2023-06-09T12:02:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 0x7f0f501ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f500c9fe0 0x7f0f501f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501e5390 con 0x7f0f500cfaf0 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501f1ad0 con 0x7f0f500c9090 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.642+0000 7f0f549ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f501f0ee0 con 0x7f0f500c9fe0 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40062/0 (socket says 172.21.15.7:40062) 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 -- 172.21.15.7:0/433799450 learned_addr learned my addr 172.21.15.7:0/433799450 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:02:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f47fff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f500c9fe0 0x7f0f501f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4e59c700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 0x7f0f501ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 -- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f500c9fe0 msgr2=0x7f0f501f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f500c9fe0 0x7f0f501f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 -- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 msgr2=0x7f0f501ecfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 0x7f0f501ecfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 -- 172.21.15.7:0/433799450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009550 con 0x7f0f500cfaf0 2023-06-09T12:02:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4e59c700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 0x7f0f501ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:02:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.643+0000 7f0f4ed9d700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007e90 tx=0x7f0f4800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.644+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ede0 con 0x7f0f500cfaf0 2023-06-09T12:02:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.644+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48014420 con 0x7f0f500cfaf0 2023-06-09T12:02:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.644+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480147d0 con 0x7f0f500cfaf0 2023-06-09T12:02:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.644+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500ca620 con 0x7f0f500cfaf0 2023-06-09T12:02:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.644+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5015d720 con 0x7f0f500cfaf0 2023-06-09T12:02:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.645+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f500b53f0 con 0x7f0f500cfaf0 2023-06-09T12:02:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.646+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f48014930 con 0x7f0f500cfaf0 2023-06-09T12:02:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.647+0000 7f0f46ffd700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 0x7f0f3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.647+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f0f4808a500 con 0x7f0f500cfaf0 2023-06-09T12:02:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.647+0000 7f0f4e59c700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 0x7f0f3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.648+0000 7f0f4e59c700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 0x7f0f3006d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0f40000f60 tx=0x7f0f40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.652+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0f48054c30 con 0x7f0f500cfaf0 2023-06-09T12:02:30.857 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.854+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f501f1c10 con 0x7f0f3006ace0 2023-06-09T12:02:30.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.864+0000 7f0f46ffd700 1 -- 172.21.15.7:0/433799450 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+314691 (secure 0 0 0) 0x7f0f501f1c10 con 0x7f0f3006ace0 2023-06-09T12:02:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:02:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.868+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 msgr2=0x7f0f3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.868+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 0x7f0f3006d190 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f0f40000f60 tx=0x7f0f40009040 comp rx=0 tx=0).stop 2023-06-09T12:02:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.868+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 msgr2=0x7f0f50151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f0f48007e90 tx=0x7f0f4800e3c0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 shutdown_connections 2023-06-09T12:02:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f500cfaf0 0x7f0f50151440 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0f500c9090 0x7f0f501ecfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0f3006ace0 0x7f0f3006d190 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 --2- 172.21.15.7:0/433799450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0f500c9fe0 0x7f0f501f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 >> 172.21.15.7:0/433799450 conn(0x7f0f50070af0 msgr2=0x7f0f5006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 shutdown_connections 2023-06-09T12:02:30.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:30.869+0000 7f0f549ff700 1 -- 172.21.15.7:0/433799450 wait complete. 2023-06-09T12:02:30.875 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:02:30.890 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1621,"stamp":"2023-06-09T12:02:28.971981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":729881,"num_omap_keys":1379,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":90985,"ondisk_log_size":90985,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2572872,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":689536,"kb_avail":752401848,"statfs":{"total":773094113280,"available":770459492352,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":706084864},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003340"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3465,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.376091+0000","last_change":"2023-06-09T12:01:56.911297+0000","last_active":"2023-06-09T12:00:52.376091+0000","last_peered":"2023-06-09T12:00:52.376091+0000","last_clean":"2023-06-09T12:00:52.376091+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:52.376091+0000","last_undegraded":"2023-06-09T12:00:52.376091+0000","last_fullsized":"2023-06-09T12:00:52.376091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3219,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.905267+0000","last_change":"2023-06-09T11:18:26.263240+0000","last_active":"2023-06-09T11:59:44.905267+0000","last_peered":"2023-06-09T11:59:44.905267+0000","last_clean":"2023-06-09T11:59:44.905267+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T11:59:44.905267+0000","last_undegraded":"2023-06-09T11:59:44.905267+0000","last_fullsized":"2023-06-09T11:59:44.905267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3453,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493236+0000","last_change":"2023-06-09T12:02:02.302498+0000","last_active":"2023-06-09T12:01:02.493236+0000","last_peered":"2023-06-09T12:01:02.493236+0000","last_clean":"2023-06-09T12:01:02.493236+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:01:02.493236+0000","last_undegraded":"2023-06-09T12:01:02.493236+0000","last_fullsized":"2023-06-09T12:01:02.493236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207940+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:44.207940+0000","last_peered":"2023-06-09T11:59:44.207940+0000","last_clean":"2023-06-09T11:59:44.207940+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:44.207940+0000","last_undegraded":"2023-06-09T11:59:44.207940+0000","last_fullsized":"2023-06-09T11:59:44.207940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3403,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381048+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:01:37.381048+0000","last_peered":"2023-06-09T12:01:37.381048+0000","last_clean":"2023-06-09T12:01:37.381048+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:01:37.381048+0000","last_undegraded":"2023-06-09T12:01:37.381048+0000","last_fullsized":"2023-06-09T12:01:37.381048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3234,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381026+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:01:37.381026+0000","last_peered":"2023-06-09T12:01:37.381026+0000","last_clean":"2023-06-09T12:01:37.381026+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:37.381026+0000","last_undegraded":"2023-06-09T12:01:37.381026+0000","last_fullsized":"2023-06-09T12:01:37.381026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1030","reported_seq":3491,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:26.949529+0000","last_change":"2023-06-09T12:01:59.021680+0000","last_active":"2023-06-09T12:02:26.949529+0000","last_peered":"2023-06-09T12:02:26.949529+0000","last_clean":"2023-06-09T12:02:26.949529+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:26.949529+0000","last_undegraded":"2023-06-09T12:02:26.949529+0000","last_fullsized":"2023-06-09T12:02:26.949529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":171,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 27s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.390920+0000","last_change":"2023-06-09T11:18:26.326114+0000","last_active":"2023-06-09T11:59:52.390920+0000","last_peered":"2023-06-09T11:59:52.390920+0000","last_clean":"2023-06-09T11:59:52.390920+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T11:59:52.390920+0000","last_undegraded":"2023-06-09T11:59:52.390920+0000","last_fullsized":"2023-06-09T11:59:52.390920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1051","reported_seq":3534,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:24.603031+0000","last_change":"2023-06-09T12:02:13.786790+0000","last_active":"2023-06-09T12:02:24.603031+0000","last_peered":"2023-06-09T12:02:24.603031+0000","last_clean":"2023-06-09T12:02:24.603031+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:24.603031+0000","last_undegraded":"2023-06-09T12:02:24.603031+0000","last_fullsized":"2023-06-09T12:02:24.603031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":98,"log_size":1051,"log_dups_size":0,"ondisk_log_size":1051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3735,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.286805+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:52.286805+0000","last_peered":"2023-06-09T11:59:52.286805+0000","last_clean":"2023-06-09T11:59:52.286805+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:52.286805+0000","last_undegraded":"2023-06-09T11:59:52.286805+0000","last_fullsized":"2023-06-09T11:59:52.286805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3101,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208004+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T11:59:44.208004+0000","last_peered":"2023-06-09T11:59:44.208004+0000","last_clean":"2023-06-09T11:59:44.208004+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T11:59:44.208004+0000","last_undegraded":"2023-06-09T11:59:44.208004+0000","last_fullsized":"2023-06-09T11:59:44.208004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3525,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394812+0000","last_change":"2023-06-09T12:01:33.911517+0000","last_active":"2023-06-09T12:00:22.394812+0000","last_peered":"2023-06-09T12:00:22.394812+0000","last_clean":"2023-06-09T12:00:22.394812+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:00:22.394812+0000","last_undegraded":"2023-06-09T12:00:22.394812+0000","last_fullsized":"2023-06-09T12:00:22.394812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3482,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.290633+0000","last_change":"2023-06-09T12:01:51.942578+0000","last_active":"2023-06-09T12:00:22.290633+0000","last_peered":"2023-06-09T12:00:22.290633+0000","last_clean":"2023-06-09T12:00:22.290633+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:00:22.290633+0000","last_undegraded":"2023-06-09T12:00:22.290633+0000","last_fullsized":"2023-06-09T12:00:22.290633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3357,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:47.397661+0000","last_change":"2023-06-09T12:01:30.955115+0000","last_active":"2023-06-09T12:00:47.397661+0000","last_peered":"2023-06-09T12:00:47.397661+0000","last_clean":"2023-06-09T12:00:47.397661+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:00:47.397661+0000","last_undegraded":"2023-06-09T12:00:47.397661+0000","last_fullsized":"2023-06-09T12:00:47.397661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'977","reported_seq":3151,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:22.951216+0000","last_change":"2023-06-09T12:02:09.152352+0000","last_active":"2023-06-09T12:02:22.951216+0000","last_peered":"2023-06-09T12:02:22.951216+0000","last_clean":"2023-06-09T12:02:22.951216+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:22.951216+0000","last_undegraded":"2023-06-09T12:02:22.951216+0000","last_fullsized":"2023-06-09T12:02:22.951216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":123,"log_size":977,"log_dups_size":0,"ondisk_log_size":977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'932","reported_seq":3008,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121476+0000","last_change":"2023-06-09T11:18:26.844923+0000","last_active":"2023-06-09T11:59:42.121476+0000","last_peered":"2023-06-09T11:59:42.121476+0000","last_clean":"2023-06-09T11:59:42.121476+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T11:59:42.121476+0000","last_undegraded":"2023-06-09T11:59:42.121476+0000","last_fullsized":"2023-06-09T11:59:42.121476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":932,"log_dups_size":0,"ondisk_log_size":932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3606,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289784+0000","last_change":"2023-06-09T12:01:58.002647+0000","last_active":"2023-06-09T12:00:42.289784+0000","last_peered":"2023-06-09T12:00:42.289784+0000","last_clean":"2023-06-09T12:00:42.289784+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:00:42.289784+0000","last_undegraded":"2023-06-09T12:00:42.289784+0000","last_fullsized":"2023-06-09T12:00:42.289784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3318,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.301288+0000","last_change":"2023-06-09T12:01:35.754891+0000","last_active":"2023-06-09T12:00:52.301288+0000","last_peered":"2023-06-09T12:00:52.301288+0000","last_clean":"2023-06-09T12:00:52.301288+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:00:52.301288+0000","last_undegraded":"2023-06-09T12:00:52.301288+0000","last_fullsized":"2023-06-09T12:00:52.301288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3136,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.295697+0000","last_change":"2023-06-09T12:01:52.964134+0000","last_active":"2023-06-09T12:01:02.295697+0000","last_peered":"2023-06-09T12:01:02.295697+0000","last_clean":"2023-06-09T12:01:02.295697+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:02.295697+0000","last_undegraded":"2023-06-09T12:01:02.295697+0000","last_fullsized":"2023-06-09T12:01:02.295697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:31.393774+0000","last_change":"2023-06-09T12:01:31.393774+0000","last_active":"2023-06-09T12:01:31.393774+0000","last_peered":"2023-06-09T12:01:31.393774+0000","last_clean":"2023-06-09T12:01:31.393774+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:01:31.393774+0000","last_undegraded":"2023-06-09T12:01:31.393774+0000","last_fullsized":"2023-06-09T12:01:31.393774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3277,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.293326+0000","last_change":"2023-06-09T11:18:26.324973+0000","last_active":"2023-06-09T11:59:52.293326+0000","last_peered":"2023-06-09T11:59:52.293326+0000","last_clean":"2023-06-09T11:59:52.293326+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T11:59:52.293326+0000","last_undegraded":"2023-06-09T11:59:52.293326+0000","last_fullsized":"2023-06-09T11:59:52.293326+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:19.292684+0000","last_change":"2023-06-09T12:01:37.403288+0000","last_active":"2023-06-09T12:00:19.292684+0000","last_peered":"2023-06-09T12:00:19.292684+0000","last_clean":"2023-06-09T12:00:19.292684+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:00:19.292684+0000","last_undegraded":"2023-06-09T12:00:19.292684+0000","last_fullsized":"2023-06-09T12:00:19.292684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3103,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.281797+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:52.281797+0000","last_peered":"2023-06-09T11:59:52.281797+0000","last_clean":"2023-06-09T11:59:52.281797+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:52.281797+0000","last_undegraded":"2023-06-09T11:59:52.281797+0000","last_fullsized":"2023-06-09T11:59:52.281797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3097,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403147+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:52.403147+0000","last_peered":"2023-06-09T11:59:52.403147+0000","last_clean":"2023-06-09T11:59:52.403147+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:52.403147+0000","last_undegraded":"2023-06-09T11:59:52.403147+0000","last_fullsized":"2023-06-09T11:59:52.403147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:45.031530+0000","last_change":"2023-06-09T12:02:12.407323+0000","last_active":"2023-06-09T12:00:45.031530+0000","last_peered":"2023-06-09T12:00:45.031530+0000","last_clean":"2023-06-09T12:00:45.031530+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:00:45.031530+0000","last_undegraded":"2023-06-09T12:00:45.031530+0000","last_fullsized":"2023-06-09T12:00:45.031530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:00.367189+0000","last_change":"2023-06-09T12:02:12.500721+0000","last_active":"2023-06-09T12:02:00.367189+0000","last_peered":"2023-06-09T12:02:00.367189+0000","last_clean":"2023-06-09T12:02:00.367189+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:02:00.367189+0000","last_undegraded":"2023-06-09T12:02:00.367189+0000","last_fullsized":"2023-06-09T12:02:00.367189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1132","reported_seq":4025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:24.168373+0000","last_change":"2023-06-09T12:01:37.725220+0000","last_active":"2023-06-09T12:02:24.168373+0000","last_peered":"2023-06-09T12:02:24.168373+0000","last_clean":"2023-06-09T12:02:24.168373+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:24.168373+0000","last_undegraded":"2023-06-09T12:02:24.168373+0000","last_fullsized":"2023-06-09T12:02:24.168373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":245,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 46s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":728,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:27.395619+0000","last_change":"2023-06-09T12:00:46.032468+0000","last_active":"2023-06-09T12:00:27.395619+0000","last_peered":"2023-06-09T12:00:27.395619+0000","last_clean":"2023-06-09T12:00:27.395619+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:27.395619+0000","last_undegraded":"2023-06-09T12:00:27.395619+0000","last_fullsized":"2023-06-09T12:00:27.395619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289712+0000","last_change":"2023-06-09T12:01:16.188471+0000","last_active":"2023-06-09T12:00:42.289712+0000","last_peered":"2023-06-09T12:00:42.289712+0000","last_clean":"2023-06-09T12:00:42.289712+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:00:42.289712+0000","last_undegraded":"2023-06-09T12:00:42.289712+0000","last_fullsized":"2023-06-09T12:00:42.289712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394854+0000","last_change":"2023-06-09T12:00:47.993494+0000","last_active":"2023-06-09T12:00:22.394854+0000","last_peered":"2023-06-09T12:00:22.394854+0000","last_clean":"2023-06-09T12:00:22.394854+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:22.394854+0000","last_undegraded":"2023-06-09T12:00:22.394854+0000","last_fullsized":"2023-06-09T12:00:22.394854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.394838+0000","last_change":"2023-06-09T12:00:53.983699+0000","last_active":"2023-06-09T12:00:22.394838+0000","last_peered":"2023-06-09T12:00:22.394838+0000","last_clean":"2023-06-09T12:00:22.394838+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:22.394838+0000","last_undegraded":"2023-06-09T12:00:22.394838+0000","last_fullsized":"2023-06-09T12:00:22.394838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3540,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.491865+0000","last_change":"2023-06-09T12:01:47.287680+0000","last_active":"2023-06-09T12:00:52.491865+0000","last_peered":"2023-06-09T12:00:52.491865+0000","last_clean":"2023-06-09T12:00:52.491865+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:00:52.491865+0000","last_undegraded":"2023-06-09T12:00:52.491865+0000","last_fullsized":"2023-06-09T12:00:52.491865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":691,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:18.241925+0000","last_change":"2023-06-09T12:00:22.287061+0000","last_active":"2023-06-09T12:00:18.241925+0000","last_peered":"2023-06-09T12:00:18.241925+0000","last_clean":"2023-06-09T12:00:18.241925+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:00:18.241925+0000","last_undegraded":"2023-06-09T12:00:18.241925+0000","last_fullsized":"2023-06-09T12:00:18.241925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423630+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T11:59:41.423630+0000","last_peered":"2023-06-09T11:59:41.423630+0000","last_clean":"2023-06-09T11:59:41.423630+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T11:59:41.423630+0000","last_undegraded":"2023-06-09T11:59:41.423630+0000","last_fullsized":"2023-06-09T11:59:41.423630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1354,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:22.297334+0000","last_change":"2023-06-09T12:00:32.896322+0000","last_active":"2023-06-09T12:00:22.297334+0000","last_peered":"2023-06-09T12:00:22.297334+0000","last_clean":"2023-06-09T12:00:22.297334+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:22.297334+0000","last_undegraded":"2023-06-09T12:00:22.297334+0000","last_fullsized":"2023-06-09T12:00:22.297334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3839,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207758+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T11:59:44.207758+0000","last_peered":"2023-06-09T11:59:44.207758+0000","last_clean":"2023-06-09T11:59:44.207758+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T11:59:44.207758+0000","last_undegraded":"2023-06-09T11:59:44.207758+0000","last_fullsized":"2023-06-09T11:59:44.207758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.497062+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T11:59:44.497062+0000","last_peered":"2023-06-09T11:59:44.497062+0000","last_clean":"2023-06-09T11:59:44.497062+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T11:59:44.497062+0000","last_undegraded":"2023-06-09T11:59:44.497062+0000","last_fullsized":"2023-06-09T11:59:44.497062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1636,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.410160+0000","last_change":"2023-06-09T12:01:14.658585+0000","last_active":"2023-06-09T12:00:42.410160+0000","last_peered":"2023-06-09T12:00:42.410160+0000","last_clean":"2023-06-09T12:00:42.410160+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:00:42.410160+0000","last_undegraded":"2023-06-09T12:00:42.410160+0000","last_fullsized":"2023-06-09T12:00:42.410160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3204,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207829+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T11:59:44.207829+0000","last_peered":"2023-06-09T11:59:44.207829+0000","last_clean":"2023-06-09T11:59:44.207829+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T11:59:44.207829+0000","last_undegraded":"2023-06-09T11:59:44.207829+0000","last_fullsized":"2023-06-09T11:59:44.207829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3208,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:32.287450+0000","last_change":"2023-06-09T12:01:56.966920+0000","last_active":"2023-06-09T12:00:32.287450+0000","last_peered":"2023-06-09T12:00:32.287450+0000","last_clean":"2023-06-09T12:00:32.287450+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:00:32.287450+0000","last_undegraded":"2023-06-09T12:00:32.287450+0000","last_fullsized":"2023-06-09T12:00:32.287450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.246824+0000","last_change":"2023-06-09T12:00:47.290422+0000","last_active":"2023-06-09T12:00:17.246824+0000","last_peered":"2023-06-09T12:00:17.246824+0000","last_clean":"2023-06-09T12:00:17.246824+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:17.246824+0000","last_undegraded":"2023-06-09T12:00:17.246824+0000","last_fullsized":"2023-06-09T12:00:17.246824+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":923,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590413+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590413+0000","last_peered":"2023-06-09T12:02:10.590413+0000","last_clean":"2023-06-09T12:02:10.590413+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590413+0000","last_undegraded":"2023-06-09T12:02:10.590413+0000","last_fullsized":"2023-06-09T12:02:10.590413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3890,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289830+0000","last_change":"2023-06-09T12:02:05.102378+0000","last_active":"2023-06-09T12:00:52.289830+0000","last_peered":"2023-06-09T12:00:52.289830+0000","last_clean":"2023-06-09T12:00:52.289830+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:00:52.289830+0000","last_undegraded":"2023-06-09T12:00:52.289830+0000","last_fullsized":"2023-06-09T12:00:52.289830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3850,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.288758+0000","last_change":"2023-06-09T12:02:06.123925+0000","last_active":"2023-06-09T12:00:42.288758+0000","last_peered":"2023-06-09T12:00:42.288758+0000","last_clean":"2023-06-09T12:00:42.288758+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:00:42.288758+0000","last_undegraded":"2023-06-09T12:00:42.288758+0000","last_fullsized":"2023-06-09T12:00:42.288758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.338006+0000","last_change":"2023-06-09T12:00:47.290438+0000","last_active":"2023-06-09T12:00:15.338006+0000","last_peered":"2023-06-09T12:00:15.338006+0000","last_clean":"2023-06-09T12:00:15.338006+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:00:15.338006+0000","last_undegraded":"2023-06-09T12:00:15.338006+0000","last_fullsized":"2023-06-09T12:00:15.338006+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3299,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493261+0000","last_change":"2023-06-09T12:01:39.395266+0000","last_active":"2023-06-09T12:01:02.493261+0000","last_peered":"2023-06-09T12:01:02.493261+0000","last_clean":"2023-06-09T12:01:02.493261+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:01:02.493261+0000","last_undegraded":"2023-06-09T12:01:02.493261+0000","last_fullsized":"2023-06-09T12:01:02.493261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:47.289624+0000","last_change":"2023-06-09T12:00:52.398167+0000","last_active":"2023-06-09T11:59:47.289624+0000","last_peered":"2023-06-09T11:59:47.289624+0000","last_clean":"2023-06-09T11:59:47.289624+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T11:59:47.289624+0000","last_undegraded":"2023-06-09T11:59:47.289624+0000","last_fullsized":"2023-06-09T11:59:47.289624+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293138+0000","last_change":"2023-06-09T12:01:50.985146+0000","last_active":"2023-06-09T12:00:42.293138+0000","last_peered":"2023-06-09T12:00:42.293138+0000","last_clean":"2023-06-09T12:00:42.293138+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:00:42.293138+0000","last_undegraded":"2023-06-09T12:00:42.293138+0000","last_fullsized":"2023-06-09T12:00:42.293138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":822,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289843+0000","last_change":"2023-06-09T12:01:54.044290+0000","last_active":"2023-06-09T12:00:52.289843+0000","last_peered":"2023-06-09T12:00:52.289843+0000","last_clean":"2023-06-09T12:00:52.289843+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:00:52.289843+0000","last_undegraded":"2023-06-09T12:00:52.289843+0000","last_fullsized":"2023-06-09T12:00:52.289843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4113,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294399+0000","last_change":"2023-06-09T12:02:12.870372+0000","last_active":"2023-06-09T12:00:52.294399+0000","last_peered":"2023-06-09T12:00:52.294399+0000","last_clean":"2023-06-09T12:00:52.294399+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:00:52.294399+0000","last_undegraded":"2023-06-09T12:00:52.294399+0000","last_fullsized":"2023-06-09T12:00:52.294399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:37.490252+0000","last_change":"2023-06-09T12:01:53.400629+0000","last_active":"2023-06-09T12:00:37.490252+0000","last_peered":"2023-06-09T12:00:37.490252+0000","last_clean":"2023-06-09T12:00:37.490252+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:00:37.490252+0000","last_undegraded":"2023-06-09T12:00:37.490252+0000","last_fullsized":"2023-06-09T12:00:37.490252+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":828,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293122+0000","last_change":"2023-06-09T12:01:34.883218+0000","last_active":"2023-06-09T12:00:42.293122+0000","last_peered":"2023-06-09T12:00:42.293122+0000","last_clean":"2023-06-09T12:00:42.293122+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:00:42.293122+0000","last_undegraded":"2023-06-09T12:00:42.293122+0000","last_fullsized":"2023-06-09T12:00:42.293122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":882,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294360+0000","last_change":"2023-06-09T12:01:48.982495+0000","last_active":"2023-06-09T12:00:52.294360+0000","last_peered":"2023-06-09T12:00:52.294360+0000","last_clean":"2023-06-09T12:00:52.294360+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:00:52.294360+0000","last_undegraded":"2023-06-09T12:00:52.294360+0000","last_fullsized":"2023-06-09T12:00:52.294360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516956+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:43.516956+0000","last_peered":"2023-06-09T11:59:43.516956+0000","last_clean":"2023-06-09T11:59:43.516956+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:43.516956+0000","last_undegraded":"2023-06-09T11:59:43.516956+0000","last_fullsized":"2023-06-09T11:59:43.516956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3428,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.414157+0000","last_change":"2023-06-09T12:01:37.414157+0000","last_active":"2023-06-09T12:01:37.414157+0000","last_peered":"2023-06-09T12:01:37.414157+0000","last_clean":"2023-06-09T12:01:37.414157+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:01:37.414157+0000","last_undegraded":"2023-06-09T12:01:37.414157+0000","last_fullsized":"2023-06-09T12:01:37.414157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2994,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.288645+0000","last_change":"2023-06-09T12:02:08.152707+0000","last_active":"2023-06-09T12:00:42.288645+0000","last_peered":"2023-06-09T12:00:42.288645+0000","last_clean":"2023-06-09T12:00:42.288645+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:00:42.288645+0000","last_undegraded":"2023-06-09T12:00:42.288645+0000","last_fullsized":"2023-06-09T12:00:42.288645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":60189,"ondisk_log_size":60189,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":729881,"num_omap_keys":1379,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30677,"ondisk_log_size":30677,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575472,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254800,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74944,"kb_avail":94117040,"statfs":{"total":96636764160,"available":96375848960,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76742656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.69599999999999995,"15min":0.65500000000000003},"min":{"1min":0.55500000000000005,"5min":0.40600000000000003,"15min":0.28699999999999998},"max":{"1min":0.96099999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.67500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.256,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.60399999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.46600000000000003,"5min":0.38200000000000001,"15min":0.28199999999999997},"max":{"1min":0.91900000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.32900000000000001},"max":{"1min":0.93700000000000006,"5min":1.05,"15min":1.05},"last":0.504}]},{"osd":2,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.72799999999999998},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89000000000000001,"5min":1.099,"15min":1.099},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.71999999999999997},"min":{"1min":0.50900000000000001,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:01:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.90500000000000003,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.755},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.70199999999999996},"min":{"1min":0.60299999999999998,"5min":0.45100000000000001,"15min":0.41499999999999998},"max":{"1min":0.93100000000000005,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.52900000000000003},"min":{"1min":0.41799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68000000000000005,"15min":0.68400000000000005},"min":{"1min":0.56200000000000006,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63200000000000001,"15min":0.60199999999999998},"min":{"1min":0.51100000000000001,"5min":0.41599999999999998,"15min":0.34999999999999998},"max":{"1min":0.85799999999999998,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65000000000000002,"15min":0.627},"min":{"1min":0.47199999999999998,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.94999999999999996,"5min":0.97999999999999998,"15min":1.0269999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68899999999999995,"15min":0.65200000000000002},"min":{"1min":0.53600000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.84099999999999997,"5min":1.044,"15min":1.0920000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.76100000000000001,"15min":0.72499999999999998},"min":{"1min":0.621,"5min":0.496,"15min":0.42599999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575472,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203288,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47168,"kb_avail":94168552,"statfs":{"total":96636764160,"available":96428597248,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48300032},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.63800000000000001,"15min":0.63500000000000001},"min":{"1min":0.47299999999999998,"5min":0.35499999999999998,"15min":0.28299999999999997},"max":{"1min":0.78600000000000003,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.33500000000000002,"15min":0.30599999999999999},"max":{"1min":0.86799999999999999,"5min":1.069,"15min":1.069},"last":0.80900000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.67000000000000004,"15min":0.66300000000000003},"min":{"1min":0.42399999999999999,"5min":0.312,"15min":0.29999999999999999},"max":{"1min":0.91800000000000004,"5min":0.99299999999999999,"15min":1.081},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72699999999999998,"15min":0.69599999999999995},"min":{"1min":0.61799999999999999,"5min":0.43099999999999999,"15min":0.40699999999999997},"max":{"1min":0.92500000000000004,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.93500000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.69199999999999995,"15min":0.66800000000000004},"min":{"1min":0.57899999999999996,"5min":0.38,"15min":0.36799999999999999},"max":{"1min":0.88,"5min":1.024,"15min":1.125},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73899999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.46400000000000002,"15min":0.44800000000000001},"max":{"1min":0.96499999999999997,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71999999999999997,"15min":0.69399999999999995},"min":{"1min":0.59799999999999998,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.98899999999999999,"5min":1.036,"15min":1.1719999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.76500000000000001,"15min":0.73599999999999999},"min":{"1min":0.65700000000000003,"5min":0.53100000000000003,"15min":0.39500000000000002},"max":{"1min":0.94899999999999995,"5min":1.034,"15min":1.0549999999999999},"last":0.88700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.59199999999999997,"15min":0.57299999999999995},"min":{"1min":0.47899999999999998,"5min":0.36899999999999999,"15min":0.27400000000000002},"max":{"1min":0.93300000000000005,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.628},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57899999999999996,"15min":0.56100000000000005},"min":{"1min":0.44900000000000001,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.72499999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70399999999999996,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.95799999999999996,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66300000000000003,"15min":0.63500000000000001},"min":{"1min":0.40600000000000003,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.83499999999999996,"5min":1.052,"15min":1.0589999999999999},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:01:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72699999999999998,"15min":0.69699999999999995},"min":{"1min":0.53200000000000003,"5min":0.45800000000000002,"15min":0.32800000000000001},"max":{"1min":0.82399999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.64500000000000002},"min":{"1min":0.499,"5min":0.36299999999999999,"15min":0.318},"max":{"1min":0.79500000000000004,"5min":0.97299999999999998,"15min":1.071},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575472,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400816,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94400,"kb_avail":93971024,"statfs":{"total":96636764160,"available":96226328576,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96665600},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.40400000000000003,"15min":0.40899999999999997},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.61099999999999999,"5min":0.61199999999999999,"15min":1.7270000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.54900000000000004,"15min":0.54800000000000004},"min":{"1min":0.51500000000000001,"5min":0.36899999999999999,"15min":0.33700000000000002},"max":{"1min":0.81399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.59199999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.47099999999999997,"15min":0.46600000000000003},"min":{"1min":0.36199999999999999,"5min":0.28699999999999998,"15min":0.27600000000000002},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.436,"5min":0.34000000000000002,"15min":0.317},"max":{"1min":0.77600000000000002,"5min":0.84099999999999997,"15min":1.9399999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78900000000000003,"5min":0.90200000000000002,"15min":2.1909999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.83599999999999997,"5min":1.091,"15min":1.9790000000000001},"last":0.748}]},{"osd":4,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.68300000000000005,"15min":0.68000000000000005},"min":{"1min":0.61099999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.96499999999999997,"5min":1.0649999999999999,"15min":1.609},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.81000000000000005,"15min":0.79200000000000004},"min":{"1min":0.64900000000000002,"5min":0.54100000000000004,"15min":0.435},"max":{"1min":1.077,"5min":1.1459999999999999,"15min":1.774},"last":0.93899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.80200000000000005,"15min":0.78300000000000003},"min":{"1min":0.64300000000000002,"5min":0.54500000000000004,"15min":0.51700000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.774},"last":1.137},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.85099999999999998,"15min":0.81699999999999995},"min":{"1min":0.73499999999999999,"5min":0.57599999999999996,"15min":0.53500000000000003},"max":{"1min":1.077,"5min":1.1419999999999999,"15min":1.8180000000000001},"last":1.0980000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83099999999999996,"15min":0.83799999999999997},"min":{"1min":0.71499999999999997,"5min":0.504,"15min":0.504},"max":{"1min":1.27,"5min":1.27,"15min":1.9059999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.747,"15min":0.75800000000000001},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":0.99399999999999999,"5min":1.117,"15min":1.921},"last":1.0780000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:01:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.68999999999999995,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.109,"5min":1.3109999999999999,"15min":1.8660000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.77400000000000002},"min":{"1min":0.51400000000000001,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0609999999999999,"5min":1.226,"15min":1.827},"last":0.995}]}]},{"osd":0,"up_from":13,"seq":55834575472,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332200,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104384,"kb_avail":94039640,"statfs":{"total":96636764160,"available":96296591360,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106889216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.442,"15min":0.44},"min":{"1min":0.34200000000000003,"5min":0.253,"15min":0.247},"max":{"1min":0.59599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.438},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":1.032,"15min":1.0900000000000001},"last":0.505},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":0.88,"15min":1.2909999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.69199999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.747,"15min":0.69499999999999995},"min":{"1min":0.70799999999999996,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.77000000000000002},"min":{"1min":0.63100000000000001,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59699999999999998,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0429999999999999,"15min":1.236},"last":1.034},{"interface":"front","average":{"1min":0.873,"5min":0.79100000000000004,"15min":0.75800000000000001},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.097}]},{"osd":6,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.86499999999999999,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.123,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":1.1850000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.67300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.77700000000000002},"min":{"1min":0.66800000000000004,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":1.212},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.88500000000000001,"15min":0.82299999999999995},"min":{"1min":0.74199999999999999,"5min":0.59499999999999997,"15min":0.52400000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.1299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575473,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237280,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57472,"kb_avail":94134560,"statfs":{"total":96636764160,"available":96393789440,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58851328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41399999999999998},"min":{"1min":0.34399999999999997,"5min":0.27800000000000002,"15min":0.22},"max":{"1min":0.55900000000000005,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.502},"min":{"1min":0.434,"5min":0.33000000000000002,"15min":0.32100000000000001},"max":{"1min":0.66800000000000004,"5min":0.999,"15min":1.0940000000000001},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.38400000000000001,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.625,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.495},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.51200000000000001,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.96799999999999997,"15min":1.546},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.54200000000000004,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.752}]},{"osd":4,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77600000000000002,"15min":0.746},"min":{"1min":0.57699999999999996,"5min":0.51200000000000001,"15min":0.373},"max":{"1min":1.0509999999999999,"5min":1.171,"15min":1.8220000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":1.1100000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.66300000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0229999999999999,"5min":1.2450000000000001,"15min":1.601},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.84899999999999998,"15min":0.82999999999999996},"min":{"1min":0.76600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.079,"5min":1.288,"15min":1.6479999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.75900000000000001},"min":{"1min":0.625,"5min":0.496,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.46100000000000002},"max":{"1min":1.056,"5min":1.105,"15min":1.3540000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81000000000000005,"15min":0.79000000000000004},"min":{"1min":0.72799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.149,"15min":1.248},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.81999999999999995},"min":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0289999999999999,"5min":1.262,"15min":1.337},"last":0.84099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575472,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438012,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114816,"kb_avail":93933828,"statfs":{"total":96636764160,"available":96188239872,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117571584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:24 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.45200000000000001},"min":{"1min":0.34300000000000003,"5min":0.223,"15min":0.223},"max":{"1min":0.59299999999999997,"5min":0.71299999999999997,"15min":1.7969999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.54100000000000004,"15min":0.53000000000000003},"min":{"1min":0.46600000000000003,"5min":0.33800000000000002,"15min":0.30599999999999999},"max":{"1min":0.90900000000000003,"5min":1.2130000000000001,"15min":1.5669999999999999},"last":0.47999999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.26900000000000002},"max":{"1min":0.80200000000000005,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.51900000000000002,"5min":0.44600000000000001,"15min":0.42299999999999999},"max":{"1min":0.77600000000000002,"5min":1.355,"15min":1.355},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":1.091,"15min":1.198},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.72899999999999998},"min":{"1min":0.58799999999999997,"5min":0.52500000000000002,"15min":0.38},"max":{"1min":1.01,"5min":1.073,"15min":1.635},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78300000000000003,"15min":0.73899999999999999},"min":{"1min":0.52500000000000002,"5min":0.47399999999999998,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":3.081,"15min":3.081},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81399999999999995,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.46800000000000003,"15min":0.39200000000000002},"max":{"1min":1.042,"5min":1.198,"15min":1.2110000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84299999999999997,"15min":0.79600000000000004},"min":{"1min":0.66700000000000004,"5min":0.52500000000000002,"15min":0.47599999999999998},"max":{"1min":0.97699999999999998,"5min":1.234,"15min":1.367},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0309999999999999,"5min":1.286,"15min":1.286},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.76900000000000002},"min":{"1min":0.69399999999999995,"5min":0.52500000000000002,"15min":0.434},"max":{"1min":1.0249999999999999,"5min":1.103,"15min":3.3490000000000002},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82499999999999996,"15min":0.81599999999999995},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.306,"15min":1.413},"last":0.751},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.54700000000000004},"max":{"1min":0.98899999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.82699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575471,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418004,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125248,"kb_avail":93953836,"statfs":{"total":96636764160,"available":96208728064,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128253952},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":1.137,"15min":1.137},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.044,"15min":1.1279999999999999},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.875,"5min":0.94399999999999995,"15min":1.609},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.64800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.997,"15min":1.159},"last":0.71499999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.98699999999999999,"15min":1.407},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.59899999999999998,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.93300000000000005,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.61599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77100000000000002,"5min":1.032,"15min":1.032},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.046,"15min":1.361},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834575471,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288472,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":71104,"kb_avail":94083368,"statfs":{"total":96636764160,"available":96341368832,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72810496},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76400000000000001,"15min":0.69399999999999995},"min":{"1min":0.57699999999999996,"5min":0.40100000000000002,"15min":0.314},"max":{"1min":0.89400000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.624},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.751,"15min":0.70899999999999996},"min":{"1min":0.53200000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":1.177,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.91300000000000003,"15min":0.72199999999999998},"min":{"1min":0.52400000000000002,"5min":0.35599999999999998,"15min":0.29899999999999999},"max":{"1min":1.018,"5min":19.715,"15min":19.715},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.70799999999999996},"min":{"1min":0.46000000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.88900000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74199999999999999,"15min":0.69099999999999995},"min":{"1min":0.54900000000000004,"5min":0.44400000000000001,"15min":0.373},"max":{"1min":0.97699999999999998,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.86499999999999999,"15min":0.76000000000000001},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":1.0649999999999999,"5min":14.439,"15min":14.439},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.88500000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.497,"15min":0.375},"max":{"1min":1.006,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.505},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.89600000000000002,"15min":0.81100000000000005},"min":{"1min":0.624,"5min":0.55900000000000005,"15min":0.50800000000000001},"max":{"1min":0.99399999999999999,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":1.2889999999999999,"15min":0.82499999999999996},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.76700000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":1.3280000000000001,"15min":0.83699999999999997},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.35199999999999998},"max":{"1min":0.81599999999999995,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.52300000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.78200000000000003,"15min":0.68600000000000005},"min":{"1min":0.48399999999999999,"5min":0.43099999999999999,"15min":0.40400000000000003},"max":{"1min":0.78400000000000003,"5min":12.163,"15min":12.163},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70199999999999996,"15min":0.64800000000000002},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.35899999999999999},"max":{"1min":0.93000000000000005,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:01:33 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71299999999999997,"15min":0.69099999999999995},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.83399999999999996,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.97199999999999998,"15min":0.81599999999999995},"min":{"1min":0.55200000000000005,"5min":0.46000000000000002,"15min":0.41999999999999998},"max":{"1min":0.95499999999999996,"5min":19.834,"15min":19.834},"last":0.73399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:02:30.897 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.898 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.898 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.898 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.899 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.899 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.899 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.900 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.900 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.900 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.901 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.901 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.902 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.902 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.902 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.903 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.903 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.903 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.904 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.904 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.904 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.905 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.905 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.905 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.906 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.906 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.906 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.907 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.907 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.907 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.908 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.908 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.908 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.909 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.909 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.909 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.910 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.910 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.910 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.911 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.911 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.911 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.912 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.912 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.912 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.913 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.913 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.913 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.914 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.914 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.915 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.915 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.915 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.916 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.916 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.916 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.917 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.917 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.917 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.918 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.918 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.918 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.919 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.919 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.919 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.920 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.920 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.920 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.921 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.921 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.921 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.922 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.922 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.922 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.923 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.923 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.923 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.924 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.924 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.924 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.925 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.925 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.926 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:30.926 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:02:50.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:02:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.142+0000 7f10ab3ff700 1 Processor -- start 2023-06-09T12:02:51.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.143+0000 7f10ab3ff700 1 -- start start 2023-06-09T12:02:51.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10ab3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10a40c6280 0x7f10a40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10ab3ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f10a406e460 0x7f10a406cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10ab3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a415cac0 con 0x7f10a406a4d0 2023-06-09T12:02:51.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10ab3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40c8870 con 0x7f10a40c6280 2023-06-09T12:02:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10ab3ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40c37d0 con 0x7f10a406e460 2023-06-09T12:02:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10a919b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10a40c6280 0x7f10a40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10a919b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f10a40c6280 0x7f10a40c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53328/0 (socket says 172.21.15.7:53328) 2023-06-09T12:02:51.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.144+0000 7f10a919b700 1 -- 172.21.15.7:0/90396525 learned_addr learned my addr 172.21.15.7:0/90396525 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:02:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a999c700 1 --2- 172.21.15.7:0/90396525 >> v2:172.21.15.117:3300/0 conn(0x7f10a406e460 0x7f10a406cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a919b700 1 -- 172.21.15.7:0/90396525 >> v2:172.21.15.117:3300/0 conn(0x7f10a406e460 msgr2=0x7f10a406cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:02:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a919b700 1 --2- 172.21.15.7:0/90396525 >> v2:172.21.15.117:3300/0 conn(0x7f10a406e460 0x7f10a406cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a919b700 1 -- 172.21.15.7:0/90396525 >> v1:172.21.15.117:6789/0 conn(0x7f10a406a4d0 legacy=0x7f10a40737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:02:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a919b700 1 -- 172.21.15.7:0/90396525 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a415cd30 con 0x7f10a40c6280 2023-06-09T12:02:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f10a919b700 1 --2- 172.21.15.7:0/90396525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a40c6280 0x7f10a40c76b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f10a413f730 tx=0x7f10a0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e45996b5da2c977 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:51.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f109bfff700 1 -- 172.21.15.7:0/90396525 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004230 con 0x7f10a40c6280 2023-06-09T12:02:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f109bfff700 1 -- 172.21.15.7:0/90396525 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a0004390 con 0x7f10a40c6280 2023-06-09T12:02:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.145+0000 7f109bfff700 1 -- 172.21.15.7:0/90396525 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a00036a0 con 0x7f10a40c6280 2023-06-09T12:02:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 -- 172.21.15.7:0/90396525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a40c6280 msgr2=0x7f10a40c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/90396525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a40c6280 0x7f10a40c76b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f10a413f730 tx=0x7f10a0009bd0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 -- 172.21.15.7:0/90396525 shutdown_connections 2023-06-09T12:02:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/90396525 >> v2:172.21.15.117:3300/0 conn(0x7f10a406e460 0x7f10a406cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/90396525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a40c6280 0x7f10a40c76b0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 -- 172.21.15.7:0/90396525 >> 172.21.15.7:0/90396525 conn(0x7f10a406f980 msgr2=0x7f10a40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 -- 172.21.15.7:0/90396525 shutdown_connections 2023-06-09T12:02:51.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.146+0000 7f10ab3ff700 1 -- 172.21.15.7:0/90396525 wait complete. 2023-06-09T12:02:51.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.148+0000 7f10ab3ff700 1 Processor -- start 2023-06-09T12:02:51.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.148+0000 7f10ab3ff700 1 -- start start 2023-06-09T12:02:51.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 0x7f10a413b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10a40c6280 0x7f10a413a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a413f780 con 0x7f10a406e460 2023-06-09T12:02:51.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cc880 con 0x7f10a406a4d0 2023-06-09T12:02:51.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10ab3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4152f90 con 0x7f10a40c6280 2023-06-09T12:02:51.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53344/0 (socket says 172.21.15.7:53344) 2023-06-09T12:02:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 -- 172.21.15.7:0/3011516524 learned_addr learned my addr 172.21.15.7:0/3011516524 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:02:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a919b700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 0x7f10a413b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a999c700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10a40c6280 0x7f10a413a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 -- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10a40c6280 msgr2=0x7f10a413a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:51.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10a40c6280 0x7f10a413a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 -- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 msgr2=0x7f10a413b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:51.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 0x7f10a413b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a899a700 1 -- 172.21.15.7:0/3011516524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a0009550 con 0x7f10a406e460 2023-06-09T12:02:51.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.149+0000 7f10a919b700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 0x7f10a413b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:02:51.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f10a899a700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f109400c4d0 tx=0x7f109400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:51.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094011790 con 0x7f10a406e460 2023-06-09T12:02:51.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10940118f0 con 0x7f10a406e460 2023-06-09T12:02:51.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a415d2f0 con 0x7f10a406e460 2023-06-09T12:02:51.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094011ca0 con 0x7f10a406e460 2023-06-09T12:02:51.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.150+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a415d010 con 0x7f10a406e460 2023-06-09T12:02:51.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.151+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a415d5d0 con 0x7f10a406e460 2023-06-09T12:02:51.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.152+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f109401c460 con 0x7f10a406e460 2023-06-09T12:02:51.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.153+0000 7f1099ffb700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 0x7f108806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:02:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.154+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f1094089bc0 con 0x7f10a406e460 2023-06-09T12:02:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.154+0000 7f10a919b700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 0x7f108806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:02:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.154+0000 7f10a919b700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 0x7f108806d1c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009130 tx=0x7f10a0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:02:51.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.158+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f10940542f0 con 0x7f10a406e460 2023-06-09T12:02:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.362+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10a4074cf0 con 0x7f108806ad10 2023-06-09T12:02:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.371+0000 7f1099ffb700 1 -- 172.21.15.7:0/3011516524 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+314758 (secure 0 0 0) 0x7f10a4074cf0 con 0x7f108806ad10 2023-06-09T12:02:51.377 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:02:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.375+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 msgr2=0x7f108806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.375+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 0x7f108806d1c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009130 tx=0x7f10a0008030 comp rx=0 tx=0).stop 2023-06-09T12:02:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.375+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 msgr2=0x7f10a4139ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:02:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.375+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f109400c4d0 tx=0x7f109400c890 comp rx=0 tx=0).stop 2023-06-09T12:02:51.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 shutdown_connections 2023-06-09T12:02:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f108806ad10 0x7f108806d1c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f10a40c6280 0x7f10a413a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f10a406a4d0 0x7f10a413b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 --2- 172.21.15.7:0/3011516524 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f10a406e460 0x7f10a4139ff0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:02:51.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 >> 172.21.15.7:0/3011516524 conn(0x7f10a406f980 msgr2=0x7f10a406b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:02:51.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 shutdown_connections 2023-06-09T12:02:51.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:02:51.376+0000 7f10ab3ff700 1 -- 172.21.15.7:0/3011516524 wait complete. 2023-06-09T12:02:51.381 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:02:51.397 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1632,"stamp":"2023-06-09T12:02:50.977969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":729881,"num_omap_keys":1379,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91300,"ondisk_log_size":91300,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2573576,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":690240,"kb_avail":752401144,"statfs":{"total":773094113280,"available":770458771456,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":706805760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003252"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3465,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.376091+0000","last_change":"2023-06-09T12:01:56.911297+0000","last_active":"2023-06-09T12:00:52.376091+0000","last_peered":"2023-06-09T12:00:52.376091+0000","last_clean":"2023-06-09T12:00:52.376091+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:00:52.376091+0000","last_undegraded":"2023-06-09T12:00:52.376091+0000","last_fullsized":"2023-06-09T12:00:52.376091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3220,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.291020+0000","last_change":"2023-06-09T12:02:35.073409+0000","last_active":"2023-06-09T12:00:52.291020+0000","last_peered":"2023-06-09T12:00:52.291020+0000","last_clean":"2023-06-09T12:00:52.291020+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:00:52.291020+0000","last_undegraded":"2023-06-09T12:00:52.291020+0000","last_fullsized":"2023-06-09T12:00:52.291020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.208043+0000","last_change":"2023-06-09T11:18:26.324325+0000","last_active":"2023-06-09T11:59:44.208043+0000","last_peered":"2023-06-09T11:59:44.208043+0000","last_clean":"2023-06-09T11:59:44.208043+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T11:59:44.208043+0000","last_undegraded":"2023-06-09T11:59:44.208043+0000","last_fullsized":"2023-06-09T11:59:44.208043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'884","reported_seq":3453,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.493236+0000","last_change":"2023-06-09T12:02:02.302498+0000","last_active":"2023-06-09T12:01:02.493236+0000","last_peered":"2023-06-09T12:01:02.493236+0000","last_clean":"2023-06-09T12:01:02.493236+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:01:02.493236+0000","last_undegraded":"2023-06-09T12:01:02.493236+0000","last_fullsized":"2023-06-09T12:01:02.493236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423699+0000","last_change":"2023-06-09T11:18:26.330019+0000","last_active":"2023-06-09T11:59:41.423699+0000","last_peered":"2023-06-09T11:59:41.423699+0000","last_clean":"2023-06-09T11:59:41.423699+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T11:59:41.423699+0000","last_undegraded":"2023-06-09T11:59:41.423699+0000","last_fullsized":"2023-06-09T11:59:41.423699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403215+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403215+0000","last_peered":"2023-06-09T12:00:51.403215+0000","last_clean":"2023-06-09T12:00:51.403215+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403215+0000","last_undegraded":"2023-06-09T12:00:51.403215+0000","last_fullsized":"2023-06-09T12:00:51.403215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207940+0000","last_change":"2023-06-09T11:18:26.333538+0000","last_active":"2023-06-09T11:59:44.207940+0000","last_peered":"2023-06-09T11:59:44.207940+0000","last_clean":"2023-06-09T11:59:44.207940+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T11:59:44.207940+0000","last_undegraded":"2023-06-09T11:59:44.207940+0000","last_fullsized":"2023-06-09T11:59:44.207940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3404,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:53.917872+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:01:53.917872+0000","last_peered":"2023-06-09T12:01:53.917872+0000","last_clean":"2023-06-09T12:01:53.917872+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:01:53.917872+0000","last_undegraded":"2023-06-09T12:01:53.917872+0000","last_fullsized":"2023-06-09T12:01:53.917872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3234,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381026+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:01:37.381026+0000","last_peered":"2023-06-09T12:01:37.381026+0000","last_clean":"2023-06-09T12:01:37.381026+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:37.381026+0000","last_undegraded":"2023-06-09T12:01:37.381026+0000","last_fullsized":"2023-06-09T12:01:37.381026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3344,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407306+0000","last_change":"2023-06-09T12:02:28.972507+0000","last_active":"2023-06-09T12:02:12.407306+0000","last_peered":"2023-06-09T12:02:12.407306+0000","last_clean":"2023-06-09T12:02:12.407306+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:12.407306+0000","last_undegraded":"2023-06-09T12:02:12.407306+0000","last_fullsized":"2023-06-09T12:02:12.407306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1172","reported_seq":3661,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:45.436845+0000","last_change":"2023-06-09T12:02:13.786790+0000","last_active":"2023-06-09T12:02:45.436845+0000","last_peered":"2023-06-09T12:02:45.436845+0000","last_clean":"2023-06-09T12:02:45.436845+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:45.436845+0000","last_undegraded":"2023-06-09T12:02:45.436845+0000","last_fullsized":"2023-06-09T12:02:45.436845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":219,"log_size":1172,"log_dups_size":0,"ondisk_log_size":1172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3735,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.286805+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:52.286805+0000","last_peered":"2023-06-09T11:59:52.286805+0000","last_clean":"2023-06-09T11:59:52.286805+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:52.286805+0000","last_undegraded":"2023-06-09T11:59:52.286805+0000","last_fullsized":"2023-06-09T11:59:52.286805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407536+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T12:00:22.407536+0000","last_peered":"2023-06-09T12:00:22.407536+0000","last_clean":"2023-06-09T12:00:22.407536+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:00:22.407536+0000","last_undegraded":"2023-06-09T12:00:22.407536+0000","last_fullsized":"2023-06-09T12:00:22.407536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.911517+0000","last_change":"2023-06-09T12:02:37.410426+0000","last_active":"2023-06-09T12:01:33.911517+0000","last_peered":"2023-06-09T12:01:33.911517+0000","last_clean":"2023-06-09T12:01:33.911517+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:01:33.911517+0000","last_undegraded":"2023-06-09T12:01:33.911517+0000","last_fullsized":"2023-06-09T12:01:33.911517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3357,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:47.397661+0000","last_change":"2023-06-09T12:01:30.955115+0000","last_active":"2023-06-09T12:00:47.397661+0000","last_peered":"2023-06-09T12:00:47.397661+0000","last_clean":"2023-06-09T12:00:47.397661+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:00:47.397661+0000","last_undegraded":"2023-06-09T12:00:47.397661+0000","last_fullsized":"2023-06-09T12:00:47.397661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'981","reported_seq":3063,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:44.892734+0000","last_change":"2023-06-09T12:02:40.031786+0000","last_active":"2023-06-09T12:02:44.892734+0000","last_peered":"2023-06-09T12:02:44.892734+0000","last_clean":"2023-06-09T12:02:44.892734+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:02:44.892734+0000","last_undegraded":"2023-06-09T12:02:44.892734+0000","last_fullsized":"2023-06-09T12:02:44.892734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":49,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3606,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289784+0000","last_change":"2023-06-09T12:01:58.002647+0000","last_active":"2023-06-09T12:00:42.289784+0000","last_peered":"2023-06-09T12:00:42.289784+0000","last_clean":"2023-06-09T12:00:42.289784+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:00:42.289784+0000","last_undegraded":"2023-06-09T12:00:42.289784+0000","last_fullsized":"2023-06-09T12:00:42.289784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3318,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.301288+0000","last_change":"2023-06-09T12:01:35.754891+0000","last_active":"2023-06-09T12:00:52.301288+0000","last_peered":"2023-06-09T12:00:52.301288+0000","last_clean":"2023-06-09T12:00:52.301288+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:00:52.301288+0000","last_undegraded":"2023-06-09T12:00:52.301288+0000","last_fullsized":"2023-06-09T12:00:52.301288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3136,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:02.295697+0000","last_change":"2023-06-09T12:01:52.964134+0000","last_active":"2023-06-09T12:01:02.295697+0000","last_peered":"2023-06-09T12:01:02.295697+0000","last_clean":"2023-06-09T12:01:02.295697+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:02.295697+0000","last_undegraded":"2023-06-09T12:01:02.295697+0000","last_fullsized":"2023-06-09T12:01:02.295697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3307,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:31.393774+0000","last_change":"2023-06-09T12:01:31.393774+0000","last_active":"2023-06-09T12:01:31.393774+0000","last_peered":"2023-06-09T12:01:31.393774+0000","last_clean":"2023-06-09T12:01:31.393774+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:01:31.393774+0000","last_undegraded":"2023-06-09T12:01:31.393774+0000","last_fullsized":"2023-06-09T12:01:31.393774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087129+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087129+0000","last_peered":"2023-06-09T12:00:44.087129+0000","last_clean":"2023-06-09T12:00:44.087129+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087129+0000","last_undegraded":"2023-06-09T12:00:44.087129+0000","last_fullsized":"2023-06-09T12:00:44.087129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3278,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.312444+0000","last_change":"2023-06-09T12:02:29.148543+0000","last_active":"2023-06-09T12:02:12.312444+0000","last_peered":"2023-06-09T12:02:12.312444+0000","last_clean":"2023-06-09T12:02:12.312444+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:12.312444+0000","last_undegraded":"2023-06-09T12:02:12.312444+0000","last_fullsized":"2023-06-09T12:02:12.312444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1073","reported_seq":1269,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:33.438034+0000","last_change":"2023-06-09T12:02:33.065572+0000","last_active":"2023-06-09T12:02:33.438034+0000","last_peered":"2023-06-09T12:02:33.438034+0000","last_clean":"2023-06-09T12:02:33.438034+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:33.438034+0000","last_undegraded":"2023-06-09T12:02:33.438034+0000","last_fullsized":"2023-06-09T12:02:33.438034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3103,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.281797+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:52.281797+0000","last_peered":"2023-06-09T11:59:52.281797+0000","last_clean":"2023-06-09T11:59:52.281797+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:52.281797+0000","last_undegraded":"2023-06-09T11:59:52.281797+0000","last_fullsized":"2023-06-09T11:59:52.281797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3097,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403147+0000","last_change":"2023-06-09T11:18:26.333491+0000","last_active":"2023-06-09T11:59:52.403147+0000","last_peered":"2023-06-09T11:59:52.403147+0000","last_clean":"2023-06-09T11:59:52.403147+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T11:59:52.403147+0000","last_undegraded":"2023-06-09T11:59:52.403147+0000","last_fullsized":"2023-06-09T11:59:52.403147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:45.031530+0000","last_change":"2023-06-09T12:02:12.407323+0000","last_active":"2023-06-09T12:00:45.031530+0000","last_peered":"2023-06-09T12:00:45.031530+0000","last_clean":"2023-06-09T12:00:45.031530+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:00:45.031530+0000","last_undegraded":"2023-06-09T12:00:45.031530+0000","last_fullsized":"2023-06-09T12:00:45.031530+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1055","reported_seq":1309,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:00.367189+0000","last_change":"2023-06-09T12:02:12.500721+0000","last_active":"2023-06-09T12:02:00.367189+0000","last_peered":"2023-06-09T12:02:00.367189+0000","last_clean":"2023-06-09T12:02:00.367189+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:02:00.367189+0000","last_undegraded":"2023-06-09T12:02:00.367189+0000","last_fullsized":"2023-06-09T12:02:00.367189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1055,"log_dups_size":0,"ondisk_log_size":1055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.032468+0000","last_change":"2023-06-09T12:02:42.411173+0000","last_active":"2023-06-09T12:00:46.032468+0000","last_peered":"2023-06-09T12:00:46.032468+0000","last_clean":"2023-06-09T12:00:46.032468+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:00:46.032468+0000","last_undegraded":"2023-06-09T12:00:46.032468+0000","last_fullsized":"2023-06-09T12:00:46.032468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.289712+0000","last_change":"2023-06-09T12:01:16.188471+0000","last_active":"2023-06-09T12:00:42.289712+0000","last_peered":"2023-06-09T12:00:42.289712+0000","last_clean":"2023-06-09T12:00:42.289712+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:00:42.289712+0000","last_undegraded":"2023-06-09T12:00:42.289712+0000","last_fullsized":"2023-06-09T12:00:42.289712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":692,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.993494+0000","last_change":"2023-06-09T12:02:37.410511+0000","last_active":"2023-06-09T12:00:47.993494+0000","last_peered":"2023-06-09T12:00:47.993494+0000","last_clean":"2023-06-09T12:00:47.993494+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:00:47.993494+0000","last_undegraded":"2023-06-09T12:00:47.993494+0000","last_fullsized":"2023-06-09T12:00:47.993494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.594865+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:04.594865+0000","last_peered":"2023-06-09T12:00:04.594865+0000","last_clean":"2023-06-09T12:00:04.594865+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:04.594865+0000","last_undegraded":"2023-06-09T12:00:04.594865+0000","last_fullsized":"2023-06-09T12:00:04.594865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":604,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:53.983699+0000","last_change":"2023-06-09T12:02:37.410488+0000","last_active":"2023-06-09T12:00:53.983699+0000","last_peered":"2023-06-09T12:00:53.983699+0000","last_clean":"2023-06-09T12:00:53.983699+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:00:53.983699+0000","last_undegraded":"2023-06-09T12:00:53.983699+0000","last_fullsized":"2023-06-09T12:00:53.983699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3541,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:47.287680+0000","last_change":"2023-06-09T12:01:47.287680+0000","last_active":"2023-06-09T12:01:47.287680+0000","last_peered":"2023-06-09T12:01:47.287680+0000","last_clean":"2023-06-09T12:01:47.287680+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:01:47.287680+0000","last_undegraded":"2023-06-09T12:01:47.287680+0000","last_fullsized":"2023-06-09T12:01:47.287680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.290653+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T12:00:22.290653+0000","last_peered":"2023-06-09T12:00:22.290653+0000","last_clean":"2023-06-09T12:00:22.290653+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:00:22.290653+0000","last_undegraded":"2023-06-09T12:00:22.290653+0000","last_fullsized":"2023-06-09T12:00:22.290653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.896322+0000","last_change":"2023-06-09T12:02:37.315767+0000","last_active":"2023-06-09T12:00:32.896322+0000","last_peered":"2023-06-09T12:00:32.896322+0000","last_clean":"2023-06-09T12:00:32.896322+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:32.896322+0000","last_undegraded":"2023-06-09T12:00:32.896322+0000","last_fullsized":"2023-06-09T12:00:32.896322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3840,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407551+0000","last_change":"2023-06-09T11:18:26.333176+0000","last_active":"2023-06-09T12:00:22.407551+0000","last_peered":"2023-06-09T12:00:22.407551+0000","last_clean":"2023-06-09T12:00:22.407551+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:00:22.407551+0000","last_undegraded":"2023-06-09T12:00:22.407551+0000","last_fullsized":"2023-06-09T12:00:22.407551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1636,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.410160+0000","last_change":"2023-06-09T12:01:14.658585+0000","last_active":"2023-06-09T12:00:42.410160+0000","last_peered":"2023-06-09T12:00:42.410160+0000","last_clean":"2023-06-09T12:00:42.410160+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:00:42.410160+0000","last_undegraded":"2023-06-09T12:00:42.410160+0000","last_fullsized":"2023-06-09T12:00:42.410160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:34.903475+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:34.903475+0000","last_peered":"2023-06-09T12:00:34.903475+0000","last_clean":"2023-06-09T12:00:34.903475+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:34.903475+0000","last_undegraded":"2023-06-09T12:00:34.903475+0000","last_fullsized":"2023-06-09T12:00:34.903475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3209,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:56.966920+0000","last_change":"2023-06-09T12:02:47.304467+0000","last_active":"2023-06-09T12:01:56.966920+0000","last_peered":"2023-06-09T12:01:56.966920+0000","last_clean":"2023-06-09T12:01:56.966920+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:01:56.966920+0000","last_undegraded":"2023-06-09T12:01:56.966920+0000","last_fullsized":"2023-06-09T12:01:56.966920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.525083+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T11:59:44.525083+0000","last_peered":"2023-06-09T11:59:44.525083+0000","last_clean":"2023-06-09T11:59:44.525083+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T11:59:44.525083+0000","last_undegraded":"2023-06-09T11:59:44.525083+0000","last_fullsized":"2023-06-09T11:59:44.525083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"73'878","reported_seq":1052,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:50.178241+0000","last_change":"2023-06-09T12:01:02.292249+0000","last_active":"2023-06-09T12:00:50.178241+0000","last_peered":"2023-06-09T12:00:50.178241+0000","last_clean":"2023-06-09T12:00:50.178241+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:00:50.178241+0000","last_undegraded":"2023-06-09T12:00:50.178241+0000","last_fullsized":"2023-06-09T12:00:50.178241+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:42.121318+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T11:59:42.121318+0000","last_peered":"2023-06-09T11:59:42.121318+0000","last_clean":"2023-06-09T11:59:42.121318+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T11:59:42.121318+0000","last_undegraded":"2023-06-09T11:59:42.121318+0000","last_fullsized":"2023-06-09T11:59:42.121318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:47.290422+0000","last_change":"2023-06-09T12:02:43.105985+0000","last_active":"2023-06-09T12:00:47.290422+0000","last_peered":"2023-06-09T12:00:47.290422+0000","last_clean":"2023-06-09T12:00:47.290422+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:00:47.290422+0000","last_undegraded":"2023-06-09T12:00:47.290422+0000","last_fullsized":"2023-06-09T12:00:47.290422+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":923,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590413+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590413+0000","last_peered":"2023-06-09T12:02:10.590413+0000","last_clean":"2023-06-09T12:02:10.590413+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590413+0000","last_undegraded":"2023-06-09T12:02:10.590413+0000","last_fullsized":"2023-06-09T12:02:10.590413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3890,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289830+0000","last_change":"2023-06-09T12:02:05.102378+0000","last_active":"2023-06-09T12:00:52.289830+0000","last_peered":"2023-06-09T12:00:52.289830+0000","last_clean":"2023-06-09T12:00:52.289830+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:00:52.289830+0000","last_undegraded":"2023-06-09T12:00:52.289830+0000","last_fullsized":"2023-06-09T12:00:52.289830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:45.937245+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T11:59:45.937245+0000","last_peered":"2023-06-09T11:59:45.937245+0000","last_clean":"2023-06-09T11:59:45.937245+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T11:59:45.937245+0000","last_undegraded":"2023-06-09T11:59:45.937245+0000","last_fullsized":"2023-06-09T11:59:45.937245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3850,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.288758+0000","last_change":"2023-06-09T12:02:06.123925+0000","last_active":"2023-06-09T12:00:42.288758+0000","last_peered":"2023-06-09T12:00:42.288758+0000","last_clean":"2023-06-09T12:00:42.288758+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:00:42.288758+0000","last_undegraded":"2023-06-09T12:00:42.288758+0000","last_fullsized":"2023-06-09T12:00:42.288758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3300,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:39.395266+0000","last_change":"2023-06-09T12:01:39.395266+0000","last_active":"2023-06-09T12:01:39.395266+0000","last_peered":"2023-06-09T12:01:39.395266+0000","last_clean":"2023-06-09T12:01:39.395266+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:01:39.395266+0000","last_undegraded":"2023-06-09T12:01:39.395266+0000","last_fullsized":"2023-06-09T12:01:39.395266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.398167+0000","last_change":"2023-06-09T12:02:31.034843+0000","last_active":"2023-06-09T12:00:52.398167+0000","last_peered":"2023-06-09T12:00:52.398167+0000","last_clean":"2023-06-09T12:00:52.398167+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:00:52.398167+0000","last_undegraded":"2023-06-09T12:00:52.398167+0000","last_fullsized":"2023-06-09T12:00:52.398167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2987,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294317+0000","last_change":"2023-06-09T12:00:53.030143+0000","last_active":"2023-06-09T12:00:52.294317+0000","last_peered":"2023-06-09T12:00:52.294317+0000","last_clean":"2023-06-09T12:00:52.294317+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:52.294317+0000","last_undegraded":"2023-06-09T12:00:52.294317+0000","last_fullsized":"2023-06-09T12:00:52.294317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293138+0000","last_change":"2023-06-09T12:01:50.985146+0000","last_active":"2023-06-09T12:00:42.293138+0000","last_peered":"2023-06-09T12:00:42.293138+0000","last_clean":"2023-06-09T12:00:42.293138+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:00:42.293138+0000","last_undegraded":"2023-06-09T12:00:42.293138+0000","last_fullsized":"2023-06-09T12:00:42.293138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.637668+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:04.637668+0000","last_peered":"2023-06-09T12:00:04.637668+0000","last_clean":"2023-06-09T12:00:04.637668+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:04.637668+0000","last_undegraded":"2023-06-09T12:00:04.637668+0000","last_fullsized":"2023-06-09T12:00:04.637668+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":822,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.289843+0000","last_change":"2023-06-09T12:01:54.044290+0000","last_active":"2023-06-09T12:00:52.289843+0000","last_peered":"2023-06-09T12:00:52.289843+0000","last_clean":"2023-06-09T12:00:52.289843+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:00:52.289843+0000","last_undegraded":"2023-06-09T12:00:52.289843+0000","last_fullsized":"2023-06-09T12:00:52.289843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4113,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294399+0000","last_change":"2023-06-09T12:02:12.870372+0000","last_active":"2023-06-09T12:00:52.294399+0000","last_peered":"2023-06-09T12:00:52.294399+0000","last_clean":"2023-06-09T12:00:52.294399+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:00:52.294399+0000","last_undegraded":"2023-06-09T12:00:52.294399+0000","last_fullsized":"2023-06-09T12:00:52.294399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:31.868642+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:31.868642+0000","last_peered":"2023-06-09T12:00:31.868642+0000","last_clean":"2023-06-09T12:00:31.868642+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:31.868642+0000","last_undegraded":"2023-06-09T12:00:31.868642+0000","last_fullsized":"2023-06-09T12:00:31.868642+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:38.899563+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:00:38.899563+0000","last_peered":"2023-06-09T12:00:38.899563+0000","last_clean":"2023-06-09T12:00:38.899563+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:00:38.899563+0000","last_undegraded":"2023-06-09T12:00:38.899563+0000","last_fullsized":"2023-06-09T12:00:38.899563+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.400629+0000","last_change":"2023-06-09T12:02:47.504948+0000","last_active":"2023-06-09T12:01:53.400629+0000","last_peered":"2023-06-09T12:01:53.400629+0000","last_clean":"2023-06-09T12:01:53.400629+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:01:53.400629+0000","last_undegraded":"2023-06-09T12:01:53.400629+0000","last_fullsized":"2023-06-09T12:01:53.400629+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":828,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.293122+0000","last_change":"2023-06-09T12:01:34.883218+0000","last_active":"2023-06-09T12:00:42.293122+0000","last_peered":"2023-06-09T12:00:42.293122+0000","last_clean":"2023-06-09T12:00:42.293122+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:00:42.293122+0000","last_undegraded":"2023-06-09T12:00:42.293122+0000","last_fullsized":"2023-06-09T12:00:42.293122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3715,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207846+0000","last_change":"2023-06-09T11:18:26.324577+0000","last_active":"2023-06-09T11:59:44.207846+0000","last_peered":"2023-06-09T11:59:44.207846+0000","last_clean":"2023-06-09T11:59:44.207846+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T11:59:44.207846+0000","last_undegraded":"2023-06-09T11:59:44.207846+0000","last_fullsized":"2023-06-09T11:59:44.207846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":882,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294360+0000","last_change":"2023-06-09T12:01:48.982495+0000","last_active":"2023-06-09T12:00:52.294360+0000","last_peered":"2023-06-09T12:00:52.294360+0000","last_clean":"2023-06-09T12:00:52.294360+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:00:52.294360+0000","last_undegraded":"2023-06-09T12:00:52.294360+0000","last_fullsized":"2023-06-09T12:00:52.294360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355633+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355633+0000","last_peered":"2023-06-09T12:00:44.355633+0000","last_clean":"2023-06-09T12:00:44.355633+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355633+0000","last_undegraded":"2023-06-09T12:00:44.355633+0000","last_fullsized":"2023-06-09T12:00:44.355633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":768,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:46.239578+0000","last_change":"2023-06-09T12:00:52.290975+0000","last_active":"2023-06-09T12:00:46.239578+0000","last_peered":"2023-06-09T12:00:46.239578+0000","last_clean":"2023-06-09T12:00:46.239578+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:46.239578+0000","last_undegraded":"2023-06-09T12:00:46.239578+0000","last_fullsized":"2023-06-09T12:00:46.239578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3675,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207868+0000","last_change":"2023-06-09T11:18:26.332979+0000","last_active":"2023-06-09T11:59:44.207868+0000","last_peered":"2023-06-09T11:59:44.207868+0000","last_clean":"2023-06-09T11:59:44.207868+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T11:59:44.207868+0000","last_undegraded":"2023-06-09T11:59:44.207868+0000","last_fullsized":"2023-06-09T11:59:44.207868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516956+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:43.516956+0000","last_peered":"2023-06-09T11:59:43.516956+0000","last_clean":"2023-06-09T11:59:43.516956+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:43.516956+0000","last_undegraded":"2023-06-09T11:59:43.516956+0000","last_fullsized":"2023-06-09T11:59:43.516956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3482,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207826+0000","last_change":"2023-06-09T11:18:26.342514+0000","last_active":"2023-06-09T11:59:44.207826+0000","last_peered":"2023-06-09T11:59:44.207826+0000","last_clean":"2023-06-09T11:59:44.207826+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T11:59:44.207826+0000","last_undegraded":"2023-06-09T11:59:44.207826+0000","last_fullsized":"2023-06-09T11:59:44.207826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3428,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.414157+0000","last_change":"2023-06-09T12:01:37.414157+0000","last_active":"2023-06-09T12:01:37.414157+0000","last_peered":"2023-06-09T12:01:37.414157+0000","last_clean":"2023-06-09T12:01:37.414157+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:01:37.414157+0000","last_undegraded":"2023-06-09T12:01:37.414157+0000","last_fullsized":"2023-06-09T12:01:37.414157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517050+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T11:59:43.517050+0000","last_peered":"2023-06-09T11:59:43.517050+0000","last_clean":"2023-06-09T11:59:43.517050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T11:59:43.517050+0000","last_undegraded":"2023-06-09T11:59:43.517050+0000","last_fullsized":"2023-06-09T11:59:43.517050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3389,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411774+0000","last_change":"2023-06-09T12:00:52.673491+0000","last_active":"2023-06-09T12:00:52.411774+0000","last_peered":"2023-06-09T12:00:52.411774+0000","last_clean":"2023-06-09T12:00:52.411774+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.411774+0000","last_undegraded":"2023-06-09T12:00:52.411774+0000","last_fullsized":"2023-06-09T12:00:52.411774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2979,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.517076+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T11:59:43.517076+0000","last_peered":"2023-06-09T11:59:43.517076+0000","last_clean":"2023-06-09T11:59:43.517076+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T11:59:43.517076+0000","last_undegraded":"2023-06-09T11:59:43.517076+0000","last_fullsized":"2023-06-09T11:59:43.517076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2994,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:42.288645+0000","last_change":"2023-06-09T12:02:08.152707+0000","last_active":"2023-06-09T12:00:42.288645+0000","last_peered":"2023-06-09T12:00:42.288645+0000","last_clean":"2023-06-09T12:00:42.288645+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:00:42.288645+0000","last_undegraded":"2023-06-09T12:00:42.288645+0000","last_fullsized":"2023-06-09T12:00:42.288645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":60504,"ondisk_log_size":60504,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":729881,"num_omap_keys":1379,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30677,"ondisk_log_size":30677,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254800,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74944,"kb_avail":94117040,"statfs":{"total":96636764160,"available":96375848960,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76742656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.86599999999999999,"5min":1.004,"15min":1.0640000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92800000000000005,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.68000000000000005},"min":{"1min":0.45600000000000002,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":1.05,"15min":1.05},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.71999999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.379},"max":{"1min":0.84599999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.754},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.41499999999999998},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.4870000000000001},"last":0.91200000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45600000000000002,"15min":0.32700000000000001},"max":{"1min":0.81000000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.85799999999999998,"15min":1.3959999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.626},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.65000000000000002},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.372},"max":{"1min":0.80200000000000005,"5min":1.044,"15min":1.0920000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.91600000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.63800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575476,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203352,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47232,"kb_avail":94168488,"statfs":{"total":96636764160,"available":96428531712,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48365568},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.94799999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.55300000000000005,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.83699999999999997,"5min":1.069,"15min":1.069},"last":0.93600000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.29999999999999999},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":1.081},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.623,"5min":0.50800000000000001,"15min":0.40699999999999997},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":1.004}]},{"osd":2,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.0369999999999999},"last":1.012},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.73999999999999999},"min":{"1min":0.60799999999999998,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.034,"15min":1.0549999999999999},"last":1.038}]},{"osd":4,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.872}]},{"osd":5,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.55500000000000005,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.91100000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":1.052,"15min":1.0589999999999999},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":0.93500000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.95099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575476,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401008,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94592,"kb_avail":93970832,"statfs":{"total":96636764160,"available":96226131968,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96862208},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.28000000000000003},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.437},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.505,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.879,"5min":0.92900000000000005,"15min":1.635},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.54100000000000004,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.609},"last":0.753},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.80100000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.016,"5min":1.0940000000000001,"15min":1.774},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.78500000000000003},"min":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.7390000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.82399999999999995},"min":{"1min":0.73599999999999999,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.056,"5min":1.27,"15min":1.794},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78100000000000003,"15min":0.76400000000000001},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.921},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.874,"15min":0.86899999999999999},"min":{"1min":0.71899999999999997,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.839},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0409999999999999,"5min":1.0760000000000001,"15min":1.827},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575476,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332328,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104512,"kb_avail":94039512,"statfs":{"total":96636764160,"available":96296460288,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107020288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.442,"15min":0.44},"min":{"1min":0.34200000000000003,"5min":0.253,"15min":0.247},"max":{"1min":0.59599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47499999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":1.032,"15min":1.0900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.376}]},{"osd":3,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":0.88,"15min":1.2909999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.747,"15min":0.69499999999999995},"min":{"1min":0.70799999999999996,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.77000000000000002},"min":{"1min":0.63100000000000001,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59699999999999998,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0429999999999999,"15min":1.236},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.873,"5min":0.79100000000000004,"15min":0.75800000000000001},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.86499999999999999,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.123,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.67300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.755}]},{"osd":7,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.77700000000000002},"min":{"1min":0.66800000000000004,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.88500000000000001,"15min":0.82299999999999995},"min":{"1min":0.74199999999999999,"5min":0.59499999999999997,"15min":0.52400000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.94399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575477,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237408,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57600,"kb_avail":94134432,"statfs":{"total":96636764160,"available":96393658368,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":58982400},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.26800000000000002},"max":{"1min":0.63200000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.33000000000000002},"max":{"1min":0.66300000000000003,"5min":0.999,"15min":1.0940000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.38400000000000001,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.625,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.51200000000000001,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.96799999999999997,"15min":1.546},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.54200000000000004,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77600000000000002,"15min":0.746},"min":{"1min":0.57699999999999996,"5min":0.51200000000000001,"15min":0.373},"max":{"1min":1.0509999999999999,"5min":1.171,"15min":1.8220000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":1.1100000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.66300000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0229999999999999,"5min":1.2450000000000001,"15min":1.601},"last":0.745},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.84899999999999998,"15min":0.82999999999999996},"min":{"1min":0.76600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.079,"5min":1.288,"15min":1.6479999999999999},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.75900000000000001},"min":{"1min":0.625,"5min":0.496,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.46100000000000002},"max":{"1min":1.056,"5min":1.105,"15min":1.3540000000000001},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81000000000000005,"15min":0.79000000000000004},"min":{"1min":0.72799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.149,"15min":1.248},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.81999999999999995},"min":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0289999999999999,"5min":1.262,"15min":1.337},"last":0.70499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575476,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438012,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114816,"kb_avail":93933828,"statfs":{"total":96636764160,"available":96188239872,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117571584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":1.7969999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.33600000000000002},"max":{"1min":0.68200000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.26900000000000002},"max":{"1min":0.80200000000000005,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.51900000000000002,"5min":0.44600000000000001,"15min":0.42299999999999999},"max":{"1min":0.77600000000000002,"5min":1.355,"15min":1.355},"last":0.622},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":1.091,"15min":1.198},"last":0.45400000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.72899999999999998},"min":{"1min":0.58799999999999997,"5min":0.52500000000000002,"15min":0.38},"max":{"1min":1.01,"5min":1.073,"15min":1.635},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78300000000000003,"15min":0.73899999999999999},"min":{"1min":0.52500000000000002,"5min":0.47399999999999998,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":3.081,"15min":3.081},"last":0.79500000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81399999999999995,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.46800000000000003,"15min":0.39200000000000002},"max":{"1min":1.042,"5min":1.198,"15min":1.2110000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84299999999999997,"15min":0.79600000000000004},"min":{"1min":0.66700000000000004,"5min":0.52500000000000002,"15min":0.47599999999999998},"max":{"1min":0.97699999999999998,"5min":1.234,"15min":1.367},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0309999999999999,"5min":1.286,"15min":1.286},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.76900000000000002},"min":{"1min":0.69399999999999995,"5min":0.52500000000000002,"15min":0.434},"max":{"1min":1.0249999999999999,"5min":1.103,"15min":3.3490000000000002},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82499999999999996,"15min":0.81599999999999995},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.306,"15min":1.413},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.54700000000000004},"max":{"1min":0.98899999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.877}]}]},{"osd":4,"up_from":13,"seq":55834575475,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418132,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125376,"kb_avail":93953708,"statfs":{"total":96636764160,"available":96208596992,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128385024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":1.137,"15min":1.137},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.044,"15min":1.1279999999999999},"last":0.86099999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.875,"5min":0.94399999999999995,"15min":1.609},"last":0.745},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.64800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.997,"15min":1.159},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.98699999999999999,"15min":1.407},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.59899999999999998,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.93300000000000005,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.624},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.61599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77100000000000002,"5min":1.032,"15min":1.032},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.046,"15min":1.361},"last":0.57499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575475,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288536,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94083304,"statfs":{"total":96636764160,"available":96341303296,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.44900000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89700000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.92900000000000005,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.879,"5min":19.715,"15min":19.715},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.71699999999999997},"min":{"1min":0.498,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.92600000000000005,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.77700000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86699999999999999,"15min":0.76300000000000001},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":14.439,"15min":14.439},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.89400000000000002,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":0.88500000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":1.29,"15min":0.82399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.3280000000000001,"15min":0.83199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.35199999999999998},"max":{"1min":0.68400000000000005,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.79100000000000004,"15min":0.68500000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":12.163,"15min":12.163},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71499999999999997,"15min":0.64700000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.98599999999999999,"15min":0.81599999999999995},"min":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":19.834,"15min":19.834},"last":0.82999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:02:51.406 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.407 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.407 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.407 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.408 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.408 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.409 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.409 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.409 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.410 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.410 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.410 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.411 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.411 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.411 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.412 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.412 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.412 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.413 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.413 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.413 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.414 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.414 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.414 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.415 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.415 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.416 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.416 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.416 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.417 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.417 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.417 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.418 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.418 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.418 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.419 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.419 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.419 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.420 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.420 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.420 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.421 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.421 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.421 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.422 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.422 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.422 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.423 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.423 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.423 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.424 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.424 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.424 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.425 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.425 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.425 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.426 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.426 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.427 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.427 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.427 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.428 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.428 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.428 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.429 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.429 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.429 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.430 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.430 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.430 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.431 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.431 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.431 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.432 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.432 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.433 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.433 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.433 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.433 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:02:51.434 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:02:56.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:02:56.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:11.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:03:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.642+0000 7f50f71ff700 1 Processor -- start 2023-06-09T12:03:11.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.642+0000 7f50f71ff700 1 -- start start 2023-06-09T12:03:11.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.643+0000 7f50f71ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50f00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.643+0000 7f50f71ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f50f015b380 0x7f50f0070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f71ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f01580a0 con 0x7f50f0071060 2023-06-09T12:03:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f71ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f00714e0 con 0x7f50f015b380 2023-06-09T12:03:11.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f71ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01518e0 con 0x7f50f0151510 2023-06-09T12:03:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f579c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f50f0071060 0x7f50f0158870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50340/0 (socket says 172.21.15.7:50340) 2023-06-09T12:03:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f579c700 1 -- 172.21.15.7:0/889586507 learned_addr learned my addr 172.21.15.7:0/889586507 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:03:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50effff700 1 --2- 172.21.15.7:0/889586507 >> v2:172.21.15.117:3300/0 conn(0x7f50f015b380 0x7f50f0070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50f00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657862078 0 0) 0x7f50f01580a0 con 0x7f50f0071060 2023-06-09T12:03:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.644+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e00035b0 con 0x7f50f0071060 2023-06-09T12:03:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 718932008 0 0) 0x7f50e00035b0 con 0x7f50f0071060 2023-06-09T12:03:11.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 msgr2=0x7f50f00c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50f00c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 >> v2:172.21.15.117:3300/0 conn(0x7f50f015b380 msgr2=0x7f50f0070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 --2- 172.21.15.7:0/889586507 >> v2:172.21.15.117:3300/0 conn(0x7f50f015b380 0x7f50f0070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f00c6180 con 0x7f50f0071060 2023-06-09T12:03:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f50e80019f0 con 0x7f50f0071060 2023-06-09T12:03:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 >> v1:172.21.15.7:6789/0 conn(0x7f50f0071060 legacy=0x7f50f0158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e0004d90 0x7f50e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00714e0 con 0x7f50e0003a00 2023-06-09T12:03:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01518e0 con 0x7f50e0004d90 2023-06-09T12:03:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e000fe20 con 0x7f50f0151510 2023-06-09T12:03:11.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50effff700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f579c700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e0004d90 0x7f50e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 -- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 msgr2=0x7f50e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151510 0x7f50e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 -- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e0004d90 msgr2=0x7f50e0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e0004d90 0x7f50e0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f4f9b700 1 -- 172.21.15.7:0/889586507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00035b0 con 0x7f50e0003a00 2023-06-09T12:03:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.645+0000 7f50f579c700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e0004d90 0x7f50e0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:03:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50e0003e10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009b60 tx=0x7f50e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f297b04fc1f465 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400e660 con 0x7f50e0003a00 2023-06-09T12:03:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e400e7c0 con 0x7f50e0003a00 2023-06-09T12:03:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50ef7fe700 1 -- 172.21.15.7:0/889586507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400eb30 con 0x7f50e0003a00 2023-06-09T12:03:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50f71ff700 1 -- 172.21.15.7:0/889586507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 msgr2=0x7f50e0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50f71ff700 1 --2- 172.21.15.7:0/889586507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50e0003e10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009b60 tx=0x7f50e0005880 comp rx=0 tx=0).stop 2023-06-09T12:03:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.646+0000 7f50f579c700 1 -- 172.21.15.7:0/889586507 reap_dead start 2023-06-09T12:03:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.647+0000 7f50f71ff700 1 -- 172.21.15.7:0/889586507 shutdown_connections 2023-06-09T12:03:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.647+0000 7f50f71ff700 1 -- 172.21.15.7:0/889586507 >> 172.21.15.7:0/889586507 conn(0x7f50f006a4d0 msgr2=0x7f50f006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:11.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.647+0000 7f50f71ff700 1 -- 172.21.15.7:0/889586507 shutdown_connections 2023-06-09T12:03:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.647+0000 7f50f71ff700 1 -- 172.21.15.7:0/889586507 wait complete. 2023-06-09T12:03:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.649+0000 7f50f71ff700 1 Processor -- start 2023-06-09T12:03:11.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.649+0000 7f50f71ff700 1 -- start start 2023-06-09T12:03:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.650+0000 7f50f71ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.650+0000 7f50f71ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50f014d580 0x7f50f015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.650+0000 7f50f71ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151c70 0x7f50f00c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f71ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01edfb0 con 0x7f50e0003a00 2023-06-09T12:03:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f71ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00c1d00 con 0x7f50f014d580 2023-06-09T12:03:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f71ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f00c1460 con 0x7f50f0151c70 2023-06-09T12:03:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42160/0 (socket says 172.21.15.7:42160) 2023-06-09T12:03:11.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 -- 172.21.15.7:0/2101583771 learned_addr learned my addr 172.21.15.7:0/2101583771 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:03:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f579c700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151c70 0x7f50f00c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50effff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50f014d580 0x7f50f015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 -- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151c70 msgr2=0x7f50f00c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151c70 0x7f50f00c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 -- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50f014d580 msgr2=0x7f50f015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50f014d580 0x7f50f015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 -- 172.21.15.7:0/2101583771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0157770 con 0x7f50e0003a00 2023-06-09T12:03:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.651+0000 7f50f4f9b700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f50e0004720 tx=0x7f50e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.652+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e400e660 con 0x7f50e0003a00 2023-06-09T12:03:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.652+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50e4003ab0 con 0x7f50e0003a00 2023-06-09T12:03:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.652+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f0157490 con 0x7f50e0003a00 2023-06-09T12:03:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.652+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e4003e60 con 0x7f50e0003a00 2023-06-09T12:03:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.652+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f0157a50 con 0x7f50e0003a00 2023-06-09T12:03:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.653+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0150a60 con 0x7f50e0003a00 2023-06-09T12:03:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.654+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f50e4013470 con 0x7f50e0003a00 2023-06-09T12:03:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.655+0000 7f50ed7fa700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 0x7f50d406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.655+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f50e408b470 con 0x7f50e0003a00 2023-06-09T12:03:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.655+0000 7f50effff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 0x7f50d406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:11.668 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.656+0000 7f50effff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 0x7f50d406d1c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f50f0150e20 tx=0x7f50dc009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:11.669 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.659+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f50e4055c30 con 0x7f50e0003a00 2023-06-09T12:03:11.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.864+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f50f01edfb0 con 0x7f50d406ad10 2023-06-09T12:03:11.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.873+0000 7f50ed7fa700 1 -- 172.21.15.7:0/2101583771 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+314925 (secure 0 0 0) 0x7f50f01edfb0 con 0x7f50d406ad10 2023-06-09T12:03:11.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:03:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 msgr2=0x7f50d406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 0x7f50d406d1c0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f50f0150e20 tx=0x7f50dc009210 comp rx=0 tx=0).stop 2023-06-09T12:03:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 msgr2=0x7f50f00b0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f50e0004720 tx=0x7f50e40047c0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 shutdown_connections 2023-06-09T12:03:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f50d406ad10 0x7f50d406d1c0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f50f0151c70 0x7f50f00c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f50e0003a00 0x7f50f00b0700 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 --2- 172.21.15.7:0/2101583771 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50f014d580 0x7f50f015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 >> 172.21.15.7:0/2101583771 conn(0x7f50f006a4d0 msgr2=0x7f50f00cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.877+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 shutdown_connections 2023-06-09T12:03:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:11.878+0000 7f50f71ff700 1 -- 172.21.15.7:0/2101583771 wait complete. 2023-06-09T12:03:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:03:11.899 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1642,"stamp":"2023-06-09T12:03:10.983537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":919818,"num_omap_keys":1752,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91579,"ondisk_log_size":91579,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2574216,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":690880,"kb_avail":752400504,"statfs":{"total":773094113280,"available":770458116096,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":707461120},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":137519,"num_omap_keys":269,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003539"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3466,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:56.911297+0000","last_change":"2023-06-09T12:02:57.390046+0000","last_active":"2023-06-09T12:01:56.911297+0000","last_peered":"2023-06-09T12:01:56.911297+0000","last_clean":"2023-06-09T12:01:56.911297+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:01:56.911297+0000","last_undegraded":"2023-06-09T12:01:56.911297+0000","last_fullsized":"2023-06-09T12:01:56.911297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3221,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:35.073409+0000","last_change":"2023-06-09T12:02:57.307464+0000","last_active":"2023-06-09T12:02:35.073409+0000","last_peered":"2023-06-09T12:02:35.073409+0000","last_clean":"2023-06-09T12:02:35.073409+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:35.073409+0000","last_undegraded":"2023-06-09T12:02:35.073409+0000","last_fullsized":"2023-06-09T12:02:35.073409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'907","reported_seq":3481,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.034218+0000","last_change":"2023-06-09T12:03:05.156895+0000","last_active":"2023-06-09T12:03:07.034218+0000","last_peered":"2023-06-09T12:03:07.034218+0000","last_clean":"2023-06-09T12:03:07.034218+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:07.034218+0000","last_undegraded":"2023-06-09T12:03:07.034218+0000","last_fullsized":"2023-06-09T12:03:07.034218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":23,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294373+0000","last_change":"2023-06-09T12:03:03.327337+0000","last_active":"2023-06-09T12:00:52.294373+0000","last_peered":"2023-06-09T12:00:52.294373+0000","last_clean":"2023-06-09T12:00:52.294373+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:00:52.294373+0000","last_undegraded":"2023-06-09T12:00:52.294373+0000","last_fullsized":"2023-06-09T12:00:52.294373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3459,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418150+0000","last_change":"2023-06-09T12:02:59.084217+0000","last_active":"2023-06-09T12:01:37.418150+0000","last_peered":"2023-06-09T12:01:37.418150+0000","last_clean":"2023-06-09T12:01:37.418150+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:01:37.418150+0000","last_undegraded":"2023-06-09T12:01:37.418150+0000","last_fullsized":"2023-06-09T12:01:37.418150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3404,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:53.917872+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:01:53.917872+0000","last_peered":"2023-06-09T12:01:53.917872+0000","last_clean":"2023-06-09T12:01:53.917872+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:01:53.917872+0000","last_undegraded":"2023-06-09T12:01:53.917872+0000","last_fullsized":"2023-06-09T12:01:53.917872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3234,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.381026+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:01:37.381026+0000","last_peered":"2023-06-09T12:01:37.381026+0000","last_clean":"2023-06-09T12:01:37.381026+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:37.381026+0000","last_undegraded":"2023-06-09T12:01:37.381026+0000","last_fullsized":"2023-06-09T12:01:37.381026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3344,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407306+0000","last_change":"2023-06-09T12:02:28.972507+0000","last_active":"2023-06-09T12:02:12.407306+0000","last_peered":"2023-06-09T12:02:12.407306+0000","last_clean":"2023-06-09T12:02:12.407306+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:12.407306+0000","last_undegraded":"2023-06-09T12:02:12.407306+0000","last_fullsized":"2023-06-09T12:02:12.407306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3735,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.286805+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:52.286805+0000","last_peered":"2023-06-09T11:59:52.286805+0000","last_clean":"2023-06-09T11:59:52.286805+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:52.286805+0000","last_undegraded":"2023-06-09T11:59:52.286805+0000","last_fullsized":"2023-06-09T11:59:52.286805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407536+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T12:00:22.407536+0000","last_peered":"2023-06-09T12:00:22.407536+0000","last_clean":"2023-06-09T12:00:22.407536+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:00:22.407536+0000","last_undegraded":"2023-06-09T12:00:22.407536+0000","last_fullsized":"2023-06-09T12:00:22.407536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.911517+0000","last_change":"2023-06-09T12:02:37.410426+0000","last_active":"2023-06-09T12:01:33.911517+0000","last_peered":"2023-06-09T12:01:33.911517+0000","last_clean":"2023-06-09T12:01:33.911517+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:01:33.911517+0000","last_undegraded":"2023-06-09T12:01:33.911517+0000","last_fullsized":"2023-06-09T12:01:33.911517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3358,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.955115+0000","last_change":"2023-06-09T12:02:57.412828+0000","last_active":"2023-06-09T12:01:30.955115+0000","last_peered":"2023-06-09T12:01:30.955115+0000","last_clean":"2023-06-09T12:01:30.955115+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:01:30.955115+0000","last_undegraded":"2023-06-09T12:01:30.955115+0000","last_fullsized":"2023-06-09T12:01:30.955115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1153","reported_seq":3243,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:05.730869+0000","last_change":"2023-06-09T12:02:40.031786+0000","last_active":"2023-06-09T12:03:05.730869+0000","last_peered":"2023-06-09T12:03:05.730869+0000","last_clean":"2023-06-09T12:03:05.730869+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:05.730869+0000","last_undegraded":"2023-06-09T12:03:05.730869+0000","last_fullsized":"2023-06-09T12:03:05.730869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":221,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:58.002647+0000","last_change":"2023-06-09T12:02:52.306849+0000","last_active":"2023-06-09T12:01:58.002647+0000","last_peered":"2023-06-09T12:01:58.002647+0000","last_clean":"2023-06-09T12:01:58.002647+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:01:58.002647+0000","last_undegraded":"2023-06-09T12:01:58.002647+0000","last_fullsized":"2023-06-09T12:01:58.002647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411796+0000","last_change":"2023-06-09T12:01:17.636747+0000","last_active":"2023-06-09T12:00:52.411796+0000","last_peered":"2023-06-09T12:00:52.411796+0000","last_clean":"2023-06-09T12:00:52.411796+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:00:52.411796+0000","last_undegraded":"2023-06-09T12:00:52.411796+0000","last_fullsized":"2023-06-09T12:00:52.411796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3318,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.301288+0000","last_change":"2023-06-09T12:01:35.754891+0000","last_active":"2023-06-09T12:00:52.301288+0000","last_peered":"2023-06-09T12:00:52.301288+0000","last_clean":"2023-06-09T12:00:52.301288+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:00:52.301288+0000","last_undegraded":"2023-06-09T12:00:52.301288+0000","last_fullsized":"2023-06-09T12:00:52.301288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225124+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225124+0000","last_peered":"2023-06-09T12:00:54.225124+0000","last_clean":"2023-06-09T12:00:54.225124+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225124+0000","last_undegraded":"2023-06-09T12:00:54.225124+0000","last_fullsized":"2023-06-09T12:00:54.225124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:03.325248+0000","last_change":"2023-06-09T12:02:57.506260+0000","last_active":"2023-06-09T12:02:03.325248+0000","last_peered":"2023-06-09T12:02:03.325248+0000","last_clean":"2023-06-09T12:02:03.325248+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:02:03.325248+0000","last_undegraded":"2023-06-09T12:02:03.325248+0000","last_fullsized":"2023-06-09T12:02:03.325248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3278,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.312444+0000","last_change":"2023-06-09T12:02:29.148543+0000","last_active":"2023-06-09T12:02:12.312444+0000","last_peered":"2023-06-09T12:02:12.312444+0000","last_clean":"2023-06-09T12:02:12.312444+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:12.312444+0000","last_undegraded":"2023-06-09T12:02:12.312444+0000","last_fullsized":"2023-06-09T12:02:12.312444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3103,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.281797+0000","last_change":"2023-06-09T11:18:26.835514+0000","last_active":"2023-06-09T11:59:52.281797+0000","last_peered":"2023-06-09T11:59:52.281797+0000","last_clean":"2023-06-09T11:59:52.281797+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T11:59:52.281797+0000","last_undegraded":"2023-06-09T11:59:52.281797+0000","last_fullsized":"2023-06-09T11:59:52.281797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3098,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.422668+0000","last_change":"2023-06-09T12:03:02.033567+0000","last_active":"2023-06-09T12:02:12.422668+0000","last_peered":"2023-06-09T12:02:12.422668+0000","last_clean":"2023-06-09T12:02:12.422668+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:02:12.422668+0000","last_undegraded":"2023-06-09T12:02:12.422668+0000","last_fullsized":"2023-06-09T12:02:12.422668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407323+0000","last_change":"2023-06-09T12:02:49.161159+0000","last_active":"2023-06-09T12:02:12.407323+0000","last_peered":"2023-06-09T12:02:12.407323+0000","last_clean":"2023-06-09T12:02:12.407323+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:12.407323+0000","last_undegraded":"2023-06-09T12:02:12.407323+0000","last_fullsized":"2023-06-09T12:02:12.407323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'919","reported_seq":3001,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306780+0000","last_change":"2023-06-09T12:03:07.006759+0000","last_active":"2023-06-09T12:02:52.306780+0000","last_peered":"2023-06-09T12:02:52.306780+0000","last_clean":"2023-06-09T12:02:52.306780+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:02:52.306780+0000","last_undegraded":"2023-06-09T12:02:52.306780+0000","last_fullsized":"2023-06-09T12:02:52.306780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":919,"log_dups_size":0,"ondisk_log_size":919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410511+0000","last_change":"2023-06-09T12:02:51.154631+0000","last_active":"2023-06-09T12:02:37.410511+0000","last_peered":"2023-06-09T12:02:37.410511+0000","last_clean":"2023-06-09T12:02:37.410511+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:37.410511+0000","last_undegraded":"2023-06-09T12:02:37.410511+0000","last_fullsized":"2023-06-09T12:02:37.410511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410488+0000","last_change":"2023-06-09T12:03:02.187861+0000","last_active":"2023-06-09T12:02:37.410488+0000","last_peered":"2023-06-09T12:02:37.410488+0000","last_clean":"2023-06-09T12:02:37.410488+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:02:37.410488+0000","last_undegraded":"2023-06-09T12:02:37.410488+0000","last_fullsized":"2023-06-09T12:02:37.410488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:41.487710+0000","last_change":"2023-06-09T12:03:02.506728+0000","last_active":"2023-06-09T12:02:41.487710+0000","last_peered":"2023-06-09T12:02:41.487710+0000","last_clean":"2023-06-09T12:02:41.487710+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:02:41.487710+0000","last_undegraded":"2023-06-09T12:02:41.487710+0000","last_fullsized":"2023-06-09T12:02:41.487710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.290653+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T12:00:22.290653+0000","last_peered":"2023-06-09T12:00:22.290653+0000","last_clean":"2023-06-09T12:00:22.290653+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:00:22.290653+0000","last_undegraded":"2023-06-09T12:00:22.290653+0000","last_fullsized":"2023-06-09T12:00:22.290653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.896322+0000","last_change":"2023-06-09T12:02:37.315767+0000","last_active":"2023-06-09T12:00:32.896322+0000","last_peered":"2023-06-09T12:00:32.896322+0000","last_clean":"2023-06-09T12:00:32.896322+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:32.896322+0000","last_undegraded":"2023-06-09T12:00:32.896322+0000","last_fullsized":"2023-06-09T12:00:32.896322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3841,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.426512+0000","last_change":"2023-06-09T12:03:03.011006+0000","last_active":"2023-06-09T12:02:37.426512+0000","last_peered":"2023-06-09T12:02:37.426512+0000","last_clean":"2023-06-09T12:02:37.426512+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:02:37.426512+0000","last_undegraded":"2023-06-09T12:02:37.426512+0000","last_fullsized":"2023-06-09T12:02:37.426512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3209,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:56.966920+0000","last_change":"2023-06-09T12:02:47.304467+0000","last_active":"2023-06-09T12:01:56.966920+0000","last_peered":"2023-06-09T12:01:56.966920+0000","last_clean":"2023-06-09T12:01:56.966920+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:01:56.966920+0000","last_undegraded":"2023-06-09T12:01:56.966920+0000","last_fullsized":"2023-06-09T12:01:56.966920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.289796+0000","last_change":"2023-06-09T11:18:26.844727+0000","last_active":"2023-06-09T12:00:52.289796+0000","last_peered":"2023-06-09T12:00:52.289796+0000","last_clean":"2023-06-09T12:00:52.289796+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:00:52.289796+0000","last_undegraded":"2023-06-09T12:00:52.289796+0000","last_fullsized":"2023-06-09T12:00:52.289796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:43.105985+0000","last_change":"2023-06-09T12:02:52.306808+0000","last_active":"2023-06-09T12:02:43.105985+0000","last_peered":"2023-06-09T12:02:43.105985+0000","last_clean":"2023-06-09T12:02:43.105985+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:43.105985+0000","last_undegraded":"2023-06-09T12:02:43.105985+0000","last_fullsized":"2023-06-09T12:02:43.105985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":923,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590413+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590413+0000","last_peered":"2023-06-09T12:02:10.590413+0000","last_clean":"2023-06-09T12:02:10.590413+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590413+0000","last_undegraded":"2023-06-09T12:02:10.590413+0000","last_fullsized":"2023-06-09T12:02:10.590413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3891,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:05.102378+0000","last_change":"2023-06-09T12:02:57.305823+0000","last_active":"2023-06-09T12:02:05.102378+0000","last_peered":"2023-06-09T12:02:05.102378+0000","last_clean":"2023-06-09T12:02:05.102378+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:05.102378+0000","last_undegraded":"2023-06-09T12:02:05.102378+0000","last_fullsized":"2023-06-09T12:02:05.102378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:06.123925+0000","last_change":"2023-06-09T12:02:52.305226+0000","last_active":"2023-06-09T12:02:06.123925+0000","last_peered":"2023-06-09T12:02:06.123925+0000","last_clean":"2023-06-09T12:02:06.123925+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:02:06.123925+0000","last_undegraded":"2023-06-09T12:02:06.123925+0000","last_fullsized":"2023-06-09T12:02:06.123925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:39.465215+0000","last_change":"2023-06-09T12:03:07.507333+0000","last_active":"2023-06-09T12:02:39.465215+0000","last_peered":"2023-06-09T12:02:39.465215+0000","last_clean":"2023-06-09T12:02:39.465215+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:02:39.465215+0000","last_undegraded":"2023-06-09T12:02:39.465215+0000","last_fullsized":"2023-06-09T12:02:39.465215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:31.034843+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:31.034843+0000","last_peered":"2023-06-09T12:02:31.034843+0000","last_clean":"2023-06-09T12:02:31.034843+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:31.034843+0000","last_undegraded":"2023-06-09T12:02:31.034843+0000","last_fullsized":"2023-06-09T12:02:31.034843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2988,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:53.030143+0000","last_change":"2023-06-09T12:02:57.310842+0000","last_active":"2023-06-09T12:00:53.030143+0000","last_peered":"2023-06-09T12:00:53.030143+0000","last_clean":"2023-06-09T12:00:53.030143+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:53.030143+0000","last_undegraded":"2023-06-09T12:00:53.030143+0000","last_fullsized":"2023-06-09T12:00:53.030143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":823,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.044290+0000","last_change":"2023-06-09T12:02:57.305843+0000","last_active":"2023-06-09T12:01:54.044290+0000","last_peered":"2023-06-09T12:01:54.044290+0000","last_clean":"2023-06-09T12:01:54.044290+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:01:54.044290+0000","last_undegraded":"2023-06-09T12:01:54.044290+0000","last_fullsized":"2023-06-09T12:01:54.044290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.300145+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:42.300145+0000","last_peered":"2023-06-09T12:00:42.300145+0000","last_clean":"2023-06-09T12:00:42.300145+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:42.300145+0000","last_undegraded":"2023-06-09T12:00:42.300145+0000","last_fullsized":"2023-06-09T12:00:42.300145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.504948+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:47.504948+0000","last_peered":"2023-06-09T12:02:47.504948+0000","last_clean":"2023-06-09T12:02:47.504948+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:47.504948+0000","last_undegraded":"2023-06-09T12:02:47.504948+0000","last_fullsized":"2023-06-09T12:02:47.504948+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:34.883218+0000","last_change":"2023-06-09T12:02:52.310351+0000","last_active":"2023-06-09T12:01:34.883218+0000","last_peered":"2023-06-09T12:01:34.883218+0000","last_clean":"2023-06-09T12:01:34.883218+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:01:34.883218+0000","last_undegraded":"2023-06-09T12:01:34.883218+0000","last_fullsized":"2023-06-09T12:01:34.883218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":882,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.294360+0000","last_change":"2023-06-09T12:01:48.982495+0000","last_active":"2023-06-09T12:00:52.294360+0000","last_peered":"2023-06-09T12:00:52.294360+0000","last_clean":"2023-06-09T12:00:52.294360+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:00:52.294360+0000","last_undegraded":"2023-06-09T12:00:52.294360+0000","last_fullsized":"2023-06-09T12:00:52.294360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":769,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.290975+0000","last_change":"2023-06-09T12:02:50.225656+0000","last_active":"2023-06-09T12:00:52.290975+0000","last_peered":"2023-06-09T12:00:52.290975+0000","last_clean":"2023-06-09T12:00:52.290975+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:00:52.290975+0000","last_undegraded":"2023-06-09T12:00:52.290975+0000","last_fullsized":"2023-06-09T12:00:52.290975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'866","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:44.207917+0000","last_change":"2023-06-09T11:18:26.324044+0000","last_active":"2023-06-09T11:59:44.207917+0000","last_peered":"2023-06-09T11:59:44.207917+0000","last_clean":"2023-06-09T11:59:44.207917+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T11:59:44.207917+0000","last_undegraded":"2023-06-09T11:59:44.207917+0000","last_fullsized":"2023-06-09T11:59:44.207917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3676,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:00:52.411817+0000","last_change":"2023-06-09T12:02:52.908103+0000","last_active":"2023-06-09T12:00:52.411817+0000","last_peered":"2023-06-09T12:00:52.411817+0000","last_clean":"2023-06-09T12:00:52.411817+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:00:52.411817+0000","last_undegraded":"2023-06-09T12:00:52.411817+0000","last_fullsized":"2023-06-09T12:00:52.411817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3149,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:43.516956+0000","last_change":"2023-06-09T11:18:26.302396+0000","last_active":"2023-06-09T11:59:43.516956+0000","last_peered":"2023-06-09T11:59:43.516956+0000","last_clean":"2023-06-09T11:59:43.516956+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T11:59:43.516956+0000","last_undegraded":"2023-06-09T11:59:43.516956+0000","last_fullsized":"2023-06-09T11:59:43.516956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3483,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418020+0000","last_change":"2023-06-09T12:02:56.996951+0000","last_active":"2023-06-09T12:01:37.418020+0000","last_peered":"2023-06-09T12:01:37.418020+0000","last_clean":"2023-06-09T12:01:37.418020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:01:37.418020+0000","last_undegraded":"2023-06-09T12:01:37.418020+0000","last_fullsized":"2023-06-09T12:01:37.418020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3429,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:04.323552+0000","last_change":"2023-06-09T12:02:57.506183+0000","last_active":"2023-06-09T12:02:04.323552+0000","last_peered":"2023-06-09T12:02:04.323552+0000","last_clean":"2023-06-09T12:02:04.323552+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:02:04.323552+0000","last_undegraded":"2023-06-09T12:02:04.323552+0000","last_fullsized":"2023-06-09T12:02:04.323552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3938,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.301220+0000","last_change":"2023-06-09T11:18:26.291036+0000","last_active":"2023-06-09T12:00:52.301220+0000","last_peered":"2023-06-09T12:00:52.301220+0000","last_clean":"2023-06-09T12:00:52.301220+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:00:52.301220+0000","last_undegraded":"2023-06-09T12:00:52.301220+0000","last_fullsized":"2023-06-09T12:00:52.301220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2980,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302648+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T12:01:02.302648+0000","last_peered":"2023-06-09T12:01:02.302648+0000","last_clean":"2023-06-09T12:01:02.302648+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:01:02.302648+0000","last_undegraded":"2023-06-09T12:01:02.302648+0000","last_fullsized":"2023-06-09T12:01:02.302648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":60734,"ondisk_log_size":60734,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":919818,"num_omap_keys":1752,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30726,"ondisk_log_size":30726,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254800,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":74944,"kb_avail":94117040,"statfs":{"total":96636764160,"available":96375848960,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76742656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.86599999999999999,"5min":1.004,"15min":1.0640000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92800000000000005,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.68000000000000005},"min":{"1min":0.45600000000000002,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":1.05,"15min":1.05},"last":0.746}]},{"osd":2,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.71999999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67400000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.379},"max":{"1min":0.84599999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.41499999999999998},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.4870000000000001},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45600000000000002,"15min":0.32700000000000001},"max":{"1min":0.81000000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.84799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.85799999999999998,"15min":1.3959999999999999},"last":0.872},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.626},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.59499999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.65000000000000002},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.372},"max":{"1min":0.80200000000000005,"5min":1.044,"15min":1.0920000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.91600000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203416,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47296,"kb_avail":94168424,"statfs":{"total":96636764160,"available":96428466176,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48431104},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.94799999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.55300000000000005,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.83699999999999997,"5min":1.069,"15min":1.069},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.29999999999999999},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":1.081},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.623,"5min":0.50800000000000001,"15min":0.40699999999999997},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.77300000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.0369999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.73999999999999999},"min":{"1min":0.60799999999999998,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.034,"15min":1.0549999999999999},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.55500000000000005,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.91100000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":1.052,"15min":1.0589999999999999},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":0.93500000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.63},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575480,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401200,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94784,"kb_avail":93970640,"statfs":{"total":96636764160,"available":96225935360,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":97058816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.28000000000000003},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.505,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.879,"5min":0.92900000000000005,"15min":1.635},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.54100000000000004,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.609},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.80100000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.016,"5min":1.0940000000000001,"15min":1.774},"last":0.745}]},{"osd":5,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.78500000000000003},"min":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.7390000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.82399999999999995},"min":{"1min":0.73599999999999999,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.056,"5min":1.27,"15min":1.794},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78100000000000003,"15min":0.76400000000000001},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.921},"last":0.86699999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.874,"15min":0.86899999999999999},"min":{"1min":0.71899999999999997,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.839},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0409999999999999,"5min":1.0760000000000001,"15min":1.827},"last":0.83399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575480,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332392,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104576,"kb_avail":94039448,"statfs":{"total":96636764160,"available":96296394752,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107085824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.442,"15min":0.44},"min":{"1min":0.34200000000000003,"5min":0.253,"15min":0.247},"max":{"1min":0.59599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.434,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.66200000000000003,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59199999999999997,"15min":0.54200000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.28399999999999997},"max":{"1min":0.73999999999999999,"5min":1.032,"15min":1.0900000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.48999999999999999},"min":{"1min":0.34999999999999998,"5min":0.29899999999999999,"15min":0.27400000000000002},"max":{"1min":0.70499999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.79800000000000004,"5min":0.88,"15min":1.2909999999999999},"last":0.502},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.51600000000000001,"5min":0.48099999999999998,"15min":0.436},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.747,"15min":0.69499999999999995},"min":{"1min":0.70799999999999996,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80800000000000005,"15min":0.77000000000000002},"min":{"1min":0.63100000000000001,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.997}]},{"osd":5,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.77200000000000002,"15min":0.74399999999999999},"min":{"1min":0.59699999999999998,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0429999999999999,"15min":1.236},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.873,"5min":0.79100000000000004,"15min":0.75800000000000001},"min":{"1min":0.69099999999999995,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.94699999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.86499999999999999,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.123,"5min":9.0790000000000006,"15min":9.0790000000000006},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.67300000000000004,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0700000000000001,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.89500000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:12 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.80700000000000005,"15min":0.77700000000000002},"min":{"1min":0.66800000000000004,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.88500000000000001,"15min":0.82299999999999995},"min":{"1min":0.74199999999999999,"5min":0.59499999999999997,"15min":0.52400000000000002},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.1200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575481,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237600,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57792,"kb_avail":94134240,"statfs":{"total":96636764160,"available":96393461760,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":59179008},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.26800000000000002},"max":{"1min":0.63200000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.33000000000000002},"max":{"1min":0.66300000000000003,"5min":0.999,"15min":1.0940000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.45000000000000001},"min":{"1min":0.38400000000000001,"5min":0.27800000000000002,"15min":0.26200000000000001},"max":{"1min":0.625,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.59199999999999997},"min":{"1min":0.51200000000000001,"5min":0.40400000000000003,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.755}]},{"osd":3,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.96799999999999997,"15min":1.546},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.64600000000000002,"15min":0.61899999999999999},"min":{"1min":0.54200000000000004,"5min":0.44,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.77600000000000002,"15min":0.746},"min":{"1min":0.57699999999999996,"5min":0.51200000000000001,"15min":0.373},"max":{"1min":1.0509999999999999,"5min":1.171,"15min":1.8220000000000001},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77500000000000002,"15min":0.75700000000000001},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.40999999999999998},"max":{"1min":1.1100000000000001,"5min":1.1180000000000001,"15min":1.7370000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.80600000000000005,"15min":0.79200000000000004},"min":{"1min":0.66300000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0229999999999999,"5min":1.2450000000000001,"15min":1.601},"last":0.995},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.84899999999999998,"15min":0.82999999999999996},"min":{"1min":0.76600000000000001,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.079,"5min":1.288,"15min":1.6479999999999999},"last":1.056}]},{"osd":6,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.75900000000000001},"min":{"1min":0.625,"5min":0.496,"15min":0.438},"max":{"1min":1.0600000000000001,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.54300000000000004,"5min":0.51400000000000001,"15min":0.46100000000000002},"max":{"1min":1.056,"5min":1.105,"15min":1.3540000000000001},"last":1.0329999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81000000000000005,"15min":0.79000000000000004},"min":{"1min":0.72799999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.0369999999999999,"5min":1.149,"15min":1.248},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.84299999999999997,"15min":0.81999999999999995},"min":{"1min":0.57699999999999996,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0289999999999999,"5min":1.262,"15min":1.337},"last":0.86799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575480,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438076,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114880,"kb_avail":93933764,"statfs":{"total":96636764160,"available":96188174336,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117637120},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":1.7969999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.33600000000000002},"max":{"1min":0.68200000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.504,"15min":0.48999999999999999},"min":{"1min":0.41999999999999998,"5min":0.34000000000000002,"15min":0.26900000000000002},"max":{"1min":0.80200000000000005,"5min":0.95299999999999996,"15min":0.95499999999999996},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.503,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.65700000000000003,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.51900000000000002,"5min":0.44600000000000001,"15min":0.42299999999999999},"max":{"1min":0.77600000000000002,"5min":1.355,"15min":1.355},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.312,"15min":0.29799999999999999},"max":{"1min":0.81399999999999995,"5min":1.091,"15min":1.198},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.75700000000000001,"15min":0.72899999999999998},"min":{"1min":0.58799999999999997,"5min":0.52500000000000002,"15min":0.38},"max":{"1min":1.01,"5min":1.073,"15min":1.635},"last":0.871},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78300000000000003,"15min":0.73899999999999999},"min":{"1min":0.52500000000000002,"5min":0.47399999999999998,"15min":0.41899999999999998},"max":{"1min":0.93200000000000005,"5min":3.081,"15min":3.081},"last":0.78500000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81399999999999995,"15min":0.76000000000000001},"min":{"1min":0.54400000000000004,"5min":0.46800000000000003,"15min":0.39200000000000002},"max":{"1min":1.042,"5min":1.198,"15min":1.2110000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.84299999999999997,"15min":0.79600000000000004},"min":{"1min":0.66700000000000004,"5min":0.52500000000000002,"15min":0.47599999999999998},"max":{"1min":0.97699999999999998,"5min":1.234,"15min":1.367},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80200000000000005,"15min":0.78900000000000003},"min":{"1min":0.60999999999999999,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0309999999999999,"5min":1.286,"15min":1.286},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79400000000000004,"15min":0.76900000000000002},"min":{"1min":0.69399999999999995,"5min":0.52500000000000002,"15min":0.434},"max":{"1min":1.0249999999999999,"5min":1.103,"15min":3.3490000000000002},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:02:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82499999999999996,"15min":0.81599999999999995},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":1.014,"5min":1.306,"15min":1.413},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.89000000000000001,"15min":0.86599999999999999},"min":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.54700000000000004},"max":{"1min":0.98899999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.88200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575479,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418196,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125440,"kb_avail":93953644,"statfs":{"total":96636764160,"available":96208531456,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128450560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.66500000000000004},"min":{"1min":0.436,"5min":0.39400000000000002,"15min":0.314},"max":{"1min":0.81399999999999995,"5min":1.137,"15min":1.137},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.73799999999999999},"min":{"1min":0.57099999999999995,"5min":0.46200000000000002,"15min":0.43099999999999999},"max":{"1min":0.872,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.39600000000000002},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.75},"min":{"1min":0.58099999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.91300000000000003,"5min":1.044,"15min":1.1279999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.875,"5min":0.94399999999999995,"15min":1.609},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.69299999999999995,"15min":0.66100000000000003},"min":{"1min":0.64800000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89900000000000002,"5min":0.997,"15min":1.159},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67900000000000005,"15min":0.67300000000000004},"min":{"1min":0.53300000000000003,"5min":0.45700000000000002,"15min":0.35899999999999999},"max":{"1min":0.81499999999999995,"5min":0.98699999999999999,"15min":1.407},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73999999999999999,"15min":0.73199999999999998},"min":{"1min":0.59899999999999998,"5min":0.499,"15min":0.45500000000000002},"max":{"1min":0.91700000000000004,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54900000000000004,"15min":0.53000000000000003},"min":{"1min":0.39300000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64100000000000001,"15min":0.61099999999999999},"min":{"1min":0.47199999999999998,"5min":0.38400000000000001,"15min":0.34100000000000003},"max":{"1min":0.93300000000000005,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.61899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57599999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76300000000000001,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72599999999999998,"15min":0.69899999999999995},"min":{"1min":0.61599999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:02:07 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61199999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77100000000000002,"5min":1.032,"15min":1.032},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68300000000000005,"15min":0.65600000000000003},"min":{"1min":0.54000000000000004,"5min":0.39800000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.046,"15min":1.361},"last":0.73499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575479,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":288536,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":71168,"kb_avail":94083304,"statfs":{"total":96636764160,"available":96341303296,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":72876032},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.44900000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89700000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.92900000000000005,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.879,"5min":19.715,"15min":19.715},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.71699999999999997},"min":{"1min":0.498,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.92600000000000005,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86699999999999999,"15min":0.76300000000000001},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":14.439,"15min":14.439},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.89400000000000002,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":0.88500000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":1.29,"15min":0.82399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.3280000000000001,"15min":0.83199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.35199999999999998},"max":{"1min":0.68400000000000005,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.79100000000000004,"15min":0.68500000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":12.163,"15min":12.163},"last":0.434},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71499999999999997,"15min":0.64700000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.98599999999999999,"15min":0.81599999999999995},"min":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":19.834,"15min":19.834},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:03:11.906 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.906 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.907 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.907 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.907 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.908 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.908 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.909 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.909 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.909 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.910 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.910 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.910 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.911 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.911 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.911 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.912 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.912 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.912 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.913 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.913 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.913 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.914 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.914 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.914 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.915 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.915 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.915 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.916 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.916 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.917 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.917 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.917 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.918 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.918 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.918 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.919 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.919 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.919 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.920 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.920 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.920 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.921 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.921 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.921 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.922 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.922 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.922 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.923 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.923 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.923 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.924 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.924 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.924 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.925 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.925 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.925 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.926 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.926 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.926 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.927 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.927 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.927 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.928 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.928 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.929 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.929 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.929 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.930 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.930 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.930 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.931 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.931 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.931 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.932 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:11.932 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:03:26.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:26.701 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:31.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:03:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.148+0000 7f9e683ff700 1 Processor -- start 2023-06-09T12:03:32.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.148+0000 7f9e683ff700 1 -- start start 2023-06-09T12:03:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e683ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e60153880 0x7f9e6015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e683ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6015a500 con 0x7f9e6015b4a0 2023-06-09T12:03:32.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e683ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6015da00 con 0x7f9e600c9f40 2023-06-09T12:03:32.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e683ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600c5f10 con 0x7f9e60153880 2023-06-09T12:03:32.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e6699c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9e6015b4a0 0x7f9e600c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:40842/0 (socket says 172.21.15.7:40842) 2023-06-09T12:03:32.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.149+0000 7f9e6699c700 1 -- 172.21.15.7:0/3195874814 learned_addr learned my addr 172.21.15.7:0/3195874814 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:03:32.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e6619b700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e60153880 0x7f9e6015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368553086 0 0) 0x7f9e58001490 con 0x7f9e6015b4a0 2023-06-09T12:03:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4c0035b0 con 0x7f9e6015b4a0 2023-06-09T12:03:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540044305 0 0) 0x7f9e50001110 con 0x7f9e600c9f40 2023-06-09T12:03:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e58001490 con 0x7f9e600c9f40 2023-06-09T12:03:32.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 73805923 0 0) 0x7f9e4c0035b0 con 0x7f9e6015b4a0 2023-06-09T12:03:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e60153880 msgr2=0x7f9e6015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e60153880 0x7f9e6015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 >> v1:172.21.15.117:6789/0 conn(0x7f9e600c9f40 legacy=0x7f9e6015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e601596e0 con 0x7f9e6015b4a0 2023-06-09T12:03:32.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f9e58001b80 con 0x7f9e6015b4a0 2023-06-09T12:03:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 >> v1:172.21.15.7:6789/0 conn(0x7f9e6015b4a0 legacy=0x7f9e600c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c003bc0 0x7f9e4c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c004850 0x7f9e4c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e4c005200 0x7f9e4c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600c5f10 con 0x7f9e4c005200 2023-06-09T12:03:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00c060 con 0x7f9e4c003bc0 2023-06-09T12:03:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00c220 con 0x7f9e4c004850 2023-06-09T12:03:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.150+0000 7f9e6619b700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e60153880 0x7f9e6015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:03:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e4c005200 0x7f9e4c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6599a700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c004850 0x7f9e4c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6619b700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c003bc0 0x7f9e4c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 -- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c004850 msgr2=0x7f9e4c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e4c004850 0x7f9e4c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 -- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c003bc0 msgr2=0x7f9e4c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c003bc0 0x7f9e4c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 -- 172.21.15.7:0/3195874814 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0035b0 con 0x7f9e4c005200 2023-06-09T12:03:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6619b700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c003bc0 0x7f9e4c003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:03:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e6699c700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e4c005200 0x7f9e4c00ba10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9e6015b900 tx=0x7f9e4c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2b68800f0874d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580072d0 con 0x7f9e4c005200 2023-06-09T12:03:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58007430 con 0x7f9e4c005200 2023-06-09T12:03:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.151+0000 7f9e65199700 1 -- 172.21.15.7:0/3195874814 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580077e0 con 0x7f9e4c005200 2023-06-09T12:03:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 -- 172.21.15.7:0/3195874814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e4c005200 msgr2=0x7f9e4c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 --2- 172.21.15.7:0/3195874814 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e4c005200 0x7f9e4c00ba10 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f9e6015b900 tx=0x7f9e4c005880 comp rx=0 tx=0).stop 2023-06-09T12:03:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e6699c700 1 -- 172.21.15.7:0/3195874814 reap_dead start 2023-06-09T12:03:32.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 -- 172.21.15.7:0/3195874814 shutdown_connections 2023-06-09T12:03:32.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 -- 172.21.15.7:0/3195874814 >> 172.21.15.7:0/3195874814 conn(0x7f9e600737f0 msgr2=0x7f9e600720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:32.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 -- 172.21.15.7:0/3195874814 shutdown_connections 2023-06-09T12:03:32.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.152+0000 7f9e683ff700 1 -- 172.21.15.7:0/3195874814 wait complete. 2023-06-09T12:03:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.154+0000 7f9e683ff700 1 Processor -- start 2023-06-09T12:03:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.154+0000 7f9e683ff700 1 -- start start 2023-06-09T12:03:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 0x7f9e6014b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e600b4ab0 0x7f9e601efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600ce850 con 0x7f9e60152190 2023-06-09T12:03:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e600ced80 con 0x7f9e4c005200 2023-06-09T12:03:32.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e683ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e601ef890 con 0x7f9e600b4ab0 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e6699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e6619b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 0x7f9e6014b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e6599a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e600b4ab0 0x7f9e601efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e6699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35082/0 (socket says 172.21.15.7:35082) 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.155+0000 7f9e6699c700 1 -- 172.21.15.7:0/2707216512 learned_addr learned my addr 172.21.15.7:0/2707216512 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:03:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 -- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e600b4ab0 msgr2=0x7f9e601efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e600b4ab0 0x7f9e601efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 -- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 msgr2=0x7f9e6014b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 0x7f9e6014b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 -- 172.21.15.7:0/2707216512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e60159c60 con 0x7f9e60152190 2023-06-09T12:03:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6619b700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 0x7f9e6014b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:03:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e6699c700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001dc0 tx=0x7f9e58001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58024070 con 0x7f9e60152190 2023-06-09T12:03:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e580077b0 con 0x7f9e60152190 2023-06-09T12:03:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58007b60 con 0x7f9e60152190 2023-06-09T12:03:32.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e60159980 con 0x7f9e60152190 2023-06-09T12:03:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.156+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e60159f40 con 0x7f9e60152190 2023-06-09T12:03:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.158+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e60150110 con 0x7f9e60152190 2023-06-09T12:03:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.159+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9e5801f430 con 0x7f9e60152190 2023-06-09T12:03:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.159+0000 7f9e56ffd700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 0x7f9e4406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:32.170 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.160+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f9e5808aad0 con 0x7f9e60152190 2023-06-09T12:03:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.160+0000 7f9e6619b700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 0x7f9e4406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.160+0000 7f9e6619b700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 0x7f9e4406d1c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9e6008d340 tx=0x7f9e5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:32.171 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.164+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9e580553b0 con 0x7f9e60152190 2023-06-09T12:03:32.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.369+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e6015a500 con 0x7f9e4406ad10 2023-06-09T12:03:32.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.378+0000 7f9e56ffd700 1 -- 172.21.15.7:0/2707216512 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315143 (secure 0 0 0) 0x7f9e6015a500 con 0x7f9e4406ad10 2023-06-09T12:03:32.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:03:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 msgr2=0x7f9e4406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 0x7f9e4406d1c0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9e6008d340 tx=0x7f9e5c008040 comp rx=0 tx=0).stop 2023-06-09T12:03:32.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 msgr2=0x7f9e60152600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f9e58001dc0 tx=0x7f9e58001e90 comp rx=0 tx=0).stop 2023-06-09T12:03:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 shutdown_connections 2023-06-09T12:03:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9e60152190 0x7f9e60152600 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9e4c005200 0x7f9e6014b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9e4406ad10 0x7f9e4406d1c0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 --2- 172.21.15.7:0/2707216512 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9e600b4ab0 0x7f9e601efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 >> 172.21.15.7:0/2707216512 conn(0x7f9e600737f0 msgr2=0x7f9e600cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 shutdown_connections 2023-06-09T12:03:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:32.383+0000 7f9e683ff700 1 -- 172.21.15.7:0/2707216512 wait complete. 2023-06-09T12:03:32.389 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:03:32.404 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1652,"stamp":"2023-06-09T12:03:30.989078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":919818,"num_omap_keys":1752,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91976,"ondisk_log_size":91976,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2579208,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":695872,"kb_avail":752395512,"statfs":{"total":773094113280,"available":770453004288,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":712572928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3467,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.390046+0000","last_change":"2023-06-09T12:03:15.925532+0000","last_active":"2023-06-09T12:02:57.390046+0000","last_peered":"2023-06-09T12:02:57.390046+0000","last_clean":"2023-06-09T12:02:57.390046+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:02:57.390046+0000","last_undegraded":"2023-06-09T12:02:57.390046+0000","last_fullsized":"2023-06-09T12:02:57.390046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3221,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:35.073409+0000","last_change":"2023-06-09T12:02:57.307464+0000","last_active":"2023-06-09T12:02:35.073409+0000","last_peered":"2023-06-09T12:02:35.073409+0000","last_clean":"2023-06-09T12:02:35.073409+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:35.073409+0000","last_undegraded":"2023-06-09T12:02:35.073409+0000","last_fullsized":"2023-06-09T12:02:35.073409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1006","reported_seq":3585,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:24.157051+0000","last_change":"2023-06-09T12:03:05.156895+0000","last_active":"2023-06-09T12:03:24.157051+0000","last_peered":"2023-06-09T12:03:24.157051+0000","last_clean":"2023-06-09T12:03:24.157051+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:24.157051+0000","last_undegraded":"2023-06-09T12:03:24.157051+0000","last_fullsized":"2023-06-09T12:03:24.157051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":1006,"log_dups_size":0,"ondisk_log_size":1006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3693,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.232199+0000","last_change":"2023-06-09T12:01:37.296785+0000","last_active":"2023-06-09T12:01:17.232199+0000","last_peered":"2023-06-09T12:01:17.232199+0000","last_clean":"2023-06-09T12:01:17.232199+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:17.232199+0000","last_undegraded":"2023-06-09T12:01:17.232199+0000","last_fullsized":"2023-06-09T12:01:17.232199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3459,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418150+0000","last_change":"2023-06-09T12:02:59.084217+0000","last_active":"2023-06-09T12:01:37.418150+0000","last_peered":"2023-06-09T12:01:37.418150+0000","last_clean":"2023-06-09T12:01:37.418150+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:01:37.418150+0000","last_undegraded":"2023-06-09T12:01:37.418150+0000","last_fullsized":"2023-06-09T12:01:37.418150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3404,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:53.917872+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:01:53.917872+0000","last_peered":"2023-06-09T12:01:53.917872+0000","last_clean":"2023-06-09T12:01:53.917872+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:01:53.917872+0000","last_undegraded":"2023-06-09T12:01:53.917872+0000","last_fullsized":"2023-06-09T12:01:53.917872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3235,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:57.947291+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:01:57.947291+0000","last_peered":"2023-06-09T12:01:57.947291+0000","last_clean":"2023-06-09T12:01:57.947291+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:01:57.947291+0000","last_undegraded":"2023-06-09T12:01:57.947291+0000","last_fullsized":"2023-06-09T12:01:57.947291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3344,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407306+0000","last_change":"2023-06-09T12:02:28.972507+0000","last_active":"2023-06-09T12:02:12.407306+0000","last_peered":"2023-06-09T12:02:12.407306+0000","last_clean":"2023-06-09T12:02:12.407306+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:12.407306+0000","last_undegraded":"2023-06-09T12:02:12.407306+0000","last_fullsized":"2023-06-09T12:02:12.407306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3735,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.286805+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:52.286805+0000","last_peered":"2023-06-09T11:59:52.286805+0000","last_clean":"2023-06-09T11:59:52.286805+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:52.286805+0000","last_undegraded":"2023-06-09T11:59:52.286805+0000","last_fullsized":"2023-06-09T11:59:52.286805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655020+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655020+0000","last_peered":"2023-06-09T12:01:29.655020+0000","last_clean":"2023-06-09T12:01:29.655020+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655020+0000","last_undegraded":"2023-06-09T12:01:29.655020+0000","last_fullsized":"2023-06-09T12:01:29.655020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407536+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T12:00:22.407536+0000","last_peered":"2023-06-09T12:00:22.407536+0000","last_clean":"2023-06-09T12:00:22.407536+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:00:22.407536+0000","last_undegraded":"2023-06-09T12:00:22.407536+0000","last_fullsized":"2023-06-09T12:00:22.407536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'899","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.911517+0000","last_change":"2023-06-09T12:02:37.410426+0000","last_active":"2023-06-09T12:01:33.911517+0000","last_peered":"2023-06-09T12:01:33.911517+0000","last_clean":"2023-06-09T12:01:33.911517+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:01:33.911517+0000","last_undegraded":"2023-06-09T12:01:33.911517+0000","last_fullsized":"2023-06-09T12:01:33.911517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":899,"log_dups_size":0,"ondisk_log_size":899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3806,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603883+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603883+0000","last_peered":"2023-06-09T12:01:33.603883+0000","last_clean":"2023-06-09T12:01:33.603883+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603883+0000","last_undegraded":"2023-06-09T12:01:33.603883+0000","last_fullsized":"2023-06-09T12:01:33.603883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3358,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.955115+0000","last_change":"2023-06-09T12:02:57.412828+0000","last_active":"2023-06-09T12:01:30.955115+0000","last_peered":"2023-06-09T12:01:30.955115+0000","last_clean":"2023-06-09T12:01:30.955115+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:01:30.955115+0000","last_undegraded":"2023-06-09T12:01:30.955115+0000","last_fullsized":"2023-06-09T12:01:30.955115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:58.002647+0000","last_change":"2023-06-09T12:02:52.306849+0000","last_active":"2023-06-09T12:01:58.002647+0000","last_peered":"2023-06-09T12:01:58.002647+0000","last_clean":"2023-06-09T12:01:58.002647+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:01:58.002647+0000","last_undegraded":"2023-06-09T12:01:58.002647+0000","last_fullsized":"2023-06-09T12:01:58.002647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:03.325248+0000","last_change":"2023-06-09T12:02:57.506260+0000","last_active":"2023-06-09T12:02:03.325248+0000","last_peered":"2023-06-09T12:02:03.325248+0000","last_clean":"2023-06-09T12:02:03.325248+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:02:03.325248+0000","last_undegraded":"2023-06-09T12:02:03.325248+0000","last_fullsized":"2023-06-09T12:02:03.325248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3278,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.312444+0000","last_change":"2023-06-09T12:02:29.148543+0000","last_active":"2023-06-09T12:02:12.312444+0000","last_peered":"2023-06-09T12:02:12.312444+0000","last_clean":"2023-06-09T12:02:12.312444+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:12.312444+0000","last_undegraded":"2023-06-09T12:02:12.312444+0000","last_fullsized":"2023-06-09T12:02:12.312444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3104,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.299809+0000","last_change":"2023-06-09T12:03:13.801581+0000","last_active":"2023-06-09T12:02:12.299809+0000","last_peered":"2023-06-09T12:02:12.299809+0000","last_clean":"2023-06-09T12:02:12.299809+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:02:12.299809+0000","last_undegraded":"2023-06-09T12:02:12.299809+0000","last_fullsized":"2023-06-09T12:02:12.299809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3098,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.422668+0000","last_change":"2023-06-09T12:03:02.033567+0000","last_active":"2023-06-09T12:02:12.422668+0000","last_peered":"2023-06-09T12:02:12.422668+0000","last_clean":"2023-06-09T12:02:12.422668+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:02:12.422668+0000","last_undegraded":"2023-06-09T12:02:12.422668+0000","last_fullsized":"2023-06-09T12:02:12.422668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407323+0000","last_change":"2023-06-09T12:02:49.161159+0000","last_active":"2023-06-09T12:02:12.407323+0000","last_peered":"2023-06-09T12:02:12.407323+0000","last_clean":"2023-06-09T12:02:12.407323+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:12.407323+0000","last_undegraded":"2023-06-09T12:02:12.407323+0000","last_fullsized":"2023-06-09T12:02:12.407323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3588,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:18.198297+0000","last_change":"2023-06-09T12:01:37.296720+0000","last_active":"2023-06-09T12:01:18.198297+0000","last_peered":"2023-06-09T12:01:18.198297+0000","last_clean":"2023-06-09T12:01:18.198297+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:18.198297+0000","last_undegraded":"2023-06-09T12:01:18.198297+0000","last_fullsized":"2023-06-09T12:01:18.198297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1090","reported_seq":3183,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:26.386053+0000","last_change":"2023-06-09T12:03:07.006759+0000","last_active":"2023-06-09T12:03:26.386053+0000","last_peered":"2023-06-09T12:03:26.386053+0000","last_clean":"2023-06-09T12:03:26.386053+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:26.386053+0000","last_undegraded":"2023-06-09T12:03:26.386053+0000","last_fullsized":"2023-06-09T12:03:26.386053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":171,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 20s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410511+0000","last_change":"2023-06-09T12:02:51.154631+0000","last_active":"2023-06-09T12:02:37.410511+0000","last_peered":"2023-06-09T12:02:37.410511+0000","last_clean":"2023-06-09T12:02:37.410511+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:37.410511+0000","last_undegraded":"2023-06-09T12:02:37.410511+0000","last_fullsized":"2023-06-09T12:02:37.410511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410488+0000","last_change":"2023-06-09T12:03:02.187861+0000","last_active":"2023-06-09T12:02:37.410488+0000","last_peered":"2023-06-09T12:02:37.410488+0000","last_clean":"2023-06-09T12:02:37.410488+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:02:37.410488+0000","last_undegraded":"2023-06-09T12:02:37.410488+0000","last_fullsized":"2023-06-09T12:02:37.410488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:41.487710+0000","last_change":"2023-06-09T12:03:02.506728+0000","last_active":"2023-06-09T12:02:41.487710+0000","last_peered":"2023-06-09T12:02:41.487710+0000","last_clean":"2023-06-09T12:02:41.487710+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:02:41.487710+0000","last_undegraded":"2023-06-09T12:02:41.487710+0000","last_fullsized":"2023-06-09T12:02:41.487710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.290653+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T12:00:22.290653+0000","last_peered":"2023-06-09T12:00:22.290653+0000","last_clean":"2023-06-09T12:00:22.290653+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:00:22.290653+0000","last_undegraded":"2023-06-09T12:00:22.290653+0000","last_fullsized":"2023-06-09T12:00:22.290653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.896322+0000","last_change":"2023-06-09T12:02:37.315767+0000","last_active":"2023-06-09T12:00:32.896322+0000","last_peered":"2023-06-09T12:00:32.896322+0000","last_clean":"2023-06-09T12:00:32.896322+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:32.896322+0000","last_undegraded":"2023-06-09T12:00:32.896322+0000","last_fullsized":"2023-06-09T12:00:32.896322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3841,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.426512+0000","last_change":"2023-06-09T12:03:03.011006+0000","last_active":"2023-06-09T12:02:37.426512+0000","last_peered":"2023-06-09T12:02:37.426512+0000","last_clean":"2023-06-09T12:02:37.426512+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:02:37.426512+0000","last_undegraded":"2023-06-09T12:02:37.426512+0000","last_fullsized":"2023-06-09T12:02:37.426512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3209,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:56.966920+0000","last_change":"2023-06-09T12:02:47.304467+0000","last_active":"2023-06-09T12:01:56.966920+0000","last_peered":"2023-06-09T12:01:56.966920+0000","last_clean":"2023-06-09T12:01:56.966920+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:01:56.966920+0000","last_undegraded":"2023-06-09T12:01:56.966920+0000","last_fullsized":"2023-06-09T12:01:56.966920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305793+0000","last_change":"2023-06-09T12:03:17.778068+0000","last_active":"2023-06-09T12:02:57.305793+0000","last_peered":"2023-06-09T12:02:57.305793+0000","last_clean":"2023-06-09T12:02:57.305793+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:02:57.305793+0000","last_undegraded":"2023-06-09T12:02:57.305793+0000","last_fullsized":"2023-06-09T12:02:57.305793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:43.105985+0000","last_change":"2023-06-09T12:02:52.306808+0000","last_active":"2023-06-09T12:02:43.105985+0000","last_peered":"2023-06-09T12:02:43.105985+0000","last_clean":"2023-06-09T12:02:43.105985+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:43.105985+0000","last_undegraded":"2023-06-09T12:02:43.105985+0000","last_fullsized":"2023-06-09T12:02:43.105985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":923,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590413+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590413+0000","last_peered":"2023-06-09T12:02:10.590413+0000","last_clean":"2023-06-09T12:02:10.590413+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590413+0000","last_undegraded":"2023-06-09T12:02:10.590413+0000","last_fullsized":"2023-06-09T12:02:10.590413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3891,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:05.102378+0000","last_change":"2023-06-09T12:02:57.305823+0000","last_active":"2023-06-09T12:02:05.102378+0000","last_peered":"2023-06-09T12:02:05.102378+0000","last_clean":"2023-06-09T12:02:05.102378+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:05.102378+0000","last_undegraded":"2023-06-09T12:02:05.102378+0000","last_fullsized":"2023-06-09T12:02:05.102378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:06.123925+0000","last_change":"2023-06-09T12:02:52.305226+0000","last_active":"2023-06-09T12:02:06.123925+0000","last_peered":"2023-06-09T12:02:06.123925+0000","last_clean":"2023-06-09T12:02:06.123925+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:02:06.123925+0000","last_undegraded":"2023-06-09T12:02:06.123925+0000","last_fullsized":"2023-06-09T12:02:06.123925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:39.465215+0000","last_change":"2023-06-09T12:03:07.507333+0000","last_active":"2023-06-09T12:02:39.465215+0000","last_peered":"2023-06-09T12:02:39.465215+0000","last_clean":"2023-06-09T12:02:39.465215+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:02:39.465215+0000","last_undegraded":"2023-06-09T12:02:39.465215+0000","last_fullsized":"2023-06-09T12:02:39.465215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:31.034843+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:31.034843+0000","last_peered":"2023-06-09T12:02:31.034843+0000","last_clean":"2023-06-09T12:02:31.034843+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:31.034843+0000","last_undegraded":"2023-06-09T12:02:31.034843+0000","last_fullsized":"2023-06-09T12:02:31.034843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2988,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:53.030143+0000","last_change":"2023-06-09T12:02:57.310842+0000","last_active":"2023-06-09T12:00:53.030143+0000","last_peered":"2023-06-09T12:00:53.030143+0000","last_clean":"2023-06-09T12:00:53.030143+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:53.030143+0000","last_undegraded":"2023-06-09T12:00:53.030143+0000","last_fullsized":"2023-06-09T12:00:53.030143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"73'654","reported_seq":827,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:20.159588+0000","last_change":"2023-06-09T12:03:19.787041+0000","last_active":"2023-06-09T12:03:20.159588+0000","last_peered":"2023-06-09T12:03:20.159588+0000","last_clean":"2023-06-09T12:03:20.159588+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:20.159588+0000","last_undegraded":"2023-06-09T12:03:20.159588+0000","last_fullsized":"2023-06-09T12:03:20.159588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":654,"log_dups_size":0,"ondisk_log_size":654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.300145+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:42.300145+0000","last_peered":"2023-06-09T12:00:42.300145+0000","last_clean":"2023-06-09T12:00:42.300145+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:42.300145+0000","last_undegraded":"2023-06-09T12:00:42.300145+0000","last_fullsized":"2023-06-09T12:00:42.300145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.504948+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:47.504948+0000","last_peered":"2023-06-09T12:02:47.504948+0000","last_clean":"2023-06-09T12:02:47.504948+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:47.504948+0000","last_undegraded":"2023-06-09T12:02:47.504948+0000","last_fullsized":"2023-06-09T12:02:47.504948+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:34.883218+0000","last_change":"2023-06-09T12:02:52.310351+0000","last_active":"2023-06-09T12:01:34.883218+0000","last_peered":"2023-06-09T12:01:34.883218+0000","last_clean":"2023-06-09T12:01:34.883218+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:01:34.883218+0000","last_undegraded":"2023-06-09T12:01:34.883218+0000","last_fullsized":"2023-06-09T12:01:34.883218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":883,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:48.982495+0000","last_change":"2023-06-09T12:03:22.314299+0000","last_active":"2023-06-09T12:01:48.982495+0000","last_peered":"2023-06-09T12:01:48.982495+0000","last_clean":"2023-06-09T12:01:48.982495+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:01:48.982495+0000","last_undegraded":"2023-06-09T12:01:48.982495+0000","last_fullsized":"2023-06-09T12:01:48.982495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":770,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.225656+0000","last_change":"2023-06-09T12:03:22.310323+0000","last_active":"2023-06-09T12:02:50.225656+0000","last_peered":"2023-06-09T12:02:50.225656+0000","last_clean":"2023-06-09T12:02:50.225656+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:02:50.225656+0000","last_undegraded":"2023-06-09T12:02:50.225656+0000","last_fullsized":"2023-06-09T12:02:50.225656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'963","reported_seq":3666,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:24.856775+0000","last_change":"2023-06-09T12:03:11.068988+0000","last_active":"2023-06-09T12:03:24.856775+0000","last_peered":"2023-06-09T12:03:24.856775+0000","last_clean":"2023-06-09T12:03:24.856775+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:24.856775+0000","last_undegraded":"2023-06-09T12:03:24.856775+0000","last_fullsized":"2023-06-09T12:03:24.856775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":97,"log_size":963,"log_dups_size":0,"ondisk_log_size":963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303199+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303199+0000","last_peered":"2023-06-09T12:01:13.303199+0000","last_clean":"2023-06-09T12:01:13.303199+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303199+0000","last_undegraded":"2023-06-09T12:01:13.303199+0000","last_fullsized":"2023-06-09T12:01:13.303199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3150,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:47.309246+0000","last_change":"2023-06-09T12:03:08.983351+0000","last_active":"2023-06-09T12:01:47.309246+0000","last_peered":"2023-06-09T12:01:47.309246+0000","last_clean":"2023-06-09T12:01:47.309246+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:01:47.309246+0000","last_undegraded":"2023-06-09T12:01:47.309246+0000","last_fullsized":"2023-06-09T12:01:47.309246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3483,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418020+0000","last_change":"2023-06-09T12:02:56.996951+0000","last_active":"2023-06-09T12:01:37.418020+0000","last_peered":"2023-06-09T12:01:37.418020+0000","last_clean":"2023-06-09T12:01:37.418020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:01:37.418020+0000","last_undegraded":"2023-06-09T12:01:37.418020+0000","last_fullsized":"2023-06-09T12:01:37.418020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3429,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:04.323552+0000","last_change":"2023-06-09T12:02:57.506183+0000","last_active":"2023-06-09T12:02:04.323552+0000","last_peered":"2023-06-09T12:02:04.323552+0000","last_clean":"2023-06-09T12:02:04.323552+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:02:04.323552+0000","last_undegraded":"2023-06-09T12:02:04.323552+0000","last_fullsized":"2023-06-09T12:02:04.323552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3939,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.318356+0000","last_change":"2023-06-09T12:03:09.981419+0000","last_active":"2023-06-09T12:02:57.318356+0000","last_peered":"2023-06-09T12:02:57.318356+0000","last_clean":"2023-06-09T12:02:57.318356+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:02:57.318356+0000","last_undegraded":"2023-06-09T12:02:57.318356+0000","last_fullsized":"2023-06-09T12:02:57.318356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2980,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302648+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T12:01:02.302648+0000","last_peered":"2023-06-09T12:01:02.302648+0000","last_clean":"2023-06-09T12:01:02.302648+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:01:02.302648+0000","last_undegraded":"2023-06-09T12:01:02.302648+0000","last_fullsized":"2023-06-09T12:01:02.302648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":61131,"ondisk_log_size":61131,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":919818,"num_omap_keys":1752,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30726,"ondisk_log_size":30726,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575484,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":254992,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":75136,"kb_avail":94116848,"statfs":{"total":96636764160,"available":96375652352,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":76939264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.70499999999999996,"15min":0.66000000000000003},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.86599999999999999,"5min":1.004,"15min":1.0640000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70499999999999996,"15min":0.67800000000000005},"min":{"1min":0.46500000000000002,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92800000000000005,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.68000000000000005},"min":{"1min":0.45600000000000002,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.92000000000000004,"5min":0.96699999999999997,"15min":1.25},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71199999999999997,"15min":0.68500000000000005},"min":{"1min":0.504,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.94299999999999995,"5min":1.05,"15min":1.05},"last":0.85099999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.72599999999999998},"min":{"1min":0.48899999999999999,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.71999999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.88100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77000000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:02:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68999999999999995,"15min":0.66400000000000003},"min":{"1min":0.56799999999999995,"5min":0.441,"15min":0.379},"max":{"1min":0.84599999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72599999999999998,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.55000000000000004,"15min":0.41499999999999998},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.4870000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54900000000000004,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.67400000000000004,"5min":0.81100000000000005,"15min":1.1200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.68300000000000005},"min":{"1min":0.55100000000000005,"5min":0.45600000000000002,"15min":0.32700000000000001},"max":{"1min":0.81000000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61899999999999999,"15min":0.59799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.75,"5min":0.85799999999999998,"15min":1.3959999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.64500000000000002,"15min":0.626},"min":{"1min":0.443,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.89500000000000002,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:02:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.65000000000000002},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.372},"max":{"1min":0.80200000000000005,"5min":1.044,"15min":1.0920000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.76300000000000001,"15min":0.72499999999999998},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.91600000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.627}]}]},{"osd":6,"up_from":13,"seq":55834575484,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203544,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47424,"kb_avail":94168296,"statfs":{"total":96636764160,"available":96428335104,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48562176},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.63600000000000001},"min":{"1min":0.504,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.94799999999999995,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.69399999999999995,"15min":0.64000000000000001},"min":{"1min":0.55300000000000005,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.83699999999999997,"5min":1.069,"15min":1.069},"last":0.71799999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.29999999999999999},"max":{"1min":0.94999999999999996,"5min":0.96599999999999997,"15min":1.081},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.70399999999999996},"min":{"1min":0.623,"5min":0.50800000000000001,"15min":0.40699999999999997},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70799999999999996,"15min":0.67400000000000004},"min":{"1min":0.56100000000000005,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72299999999999998},"min":{"1min":0.60499999999999998,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.82199999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.63600000000000001,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.96599999999999997,"5min":1.036,"15min":1.0369999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.78800000000000003,"15min":0.73999999999999999},"min":{"1min":0.60799999999999998,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":0.998,"5min":1.034,"15min":1.0549999999999999},"last":0.75}]},{"osd":4,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59499999999999997,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.41699999999999998,"15min":0.27400000000000002},"max":{"1min":0.81299999999999994,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57899999999999996,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70799999999999996,"15min":0.68700000000000006},"min":{"1min":0.55500000000000005,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.91100000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.65700000000000003,"15min":0.63300000000000001},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":1.052,"15min":1.0589999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:02:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.502,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":0.93500000000000005,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.627},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.66000000000000003,"15min":0.64000000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.83299999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575484,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401200,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94784,"kb_avail":93970640,"statfs":{"total":96636764160,"available":96225935360,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":97058816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.41299999999999998},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.57299999999999995,"15min":0.55600000000000005},"min":{"1min":0.49399999999999999,"5min":0.38700000000000001,"15min":0.36899999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":1.877},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.39300000000000002,"5min":0.308,"15min":0.28000000000000003},"max":{"1min":0.73799999999999999,"5min":0.82699999999999996,"15min":1.982},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55800000000000005,"15min":0.53700000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.48499999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.505,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.58999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.879,"5min":0.92900000000000005,"15min":1.635},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.70299999999999996,"15min":0.68700000000000006},"min":{"1min":0.54100000000000004,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.609},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82299999999999995,"15min":0.80100000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.503},"max":{"1min":1.016,"5min":1.0940000000000001,"15min":1.774},"last":0.93100000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.81899999999999995,"15min":0.78500000000000003},"min":{"1min":0.67100000000000004,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.137,"5min":1.137,"15min":1.7390000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.82399999999999995},"min":{"1min":0.73599999999999999,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0980000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.875}]},{"osd":6,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82899999999999996,"15min":0.83499999999999996},"min":{"1min":0.64800000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.056,"5min":1.27,"15min":1.794},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.78100000000000003,"15min":0.76400000000000001},"min":{"1min":0.67700000000000005,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.921},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:02:31 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.874,"15min":0.86899999999999999},"min":{"1min":0.71899999999999997,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0720000000000001,"5min":1.2010000000000001,"15min":1.839},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.78500000000000003,"15min":0.77600000000000002},"min":{"1min":0.60099999999999998,"5min":0.48799999999999999,"15min":0.47299999999999998},"max":{"1min":1.0409999999999999,"5min":1.0760000000000001,"15min":1.827},"last":0.89700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575484,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332520,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104704,"kb_avail":94039320,"statfs":{"total":96636764160,"available":96296263680,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107216896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.58899999999999997,"5min":0.68300000000000005,"15min":0.70799999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.41199999999999998,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.67800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.624}]},{"osd":2,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.79500000000000004,"5min":1.032,"15min":1.0900000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.2909999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77200000000000002,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.89500000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.0429999999999999,"15min":1.236},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82199999999999995,"15min":0.76800000000000002},"min":{"1min":0.63900000000000001,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.86599999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.51300000000000001,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0189999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83299999999999996,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.212,"5min":1.288,"15min":1.288},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90900000000000003,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.1299999999999999,"5min":1.351,"15min":1.351},"last":0.91900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575485,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237664,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57856,"kb_avail":94134176,"statfs":{"total":96636764160,"available":96393396224,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":59244544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:28 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.432,"15min":0.41799999999999998},"min":{"1min":0.35199999999999998,"5min":0.30399999999999999,"15min":0.26800000000000002},"max":{"1min":0.63200000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51700000000000002,"15min":0.50600000000000001},"min":{"1min":0.439,"5min":0.35499999999999998,"15min":0.33000000000000002},"max":{"1min":0.66300000000000003,"5min":0.999,"15min":1.0940000000000001},"last":0.497}]},{"osd":1,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.67100000000000004,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.54000000000000004,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.51000000000000001,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.82099999999999995,"5min":0.96799999999999997,"15min":1.546},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.56599999999999995,"5min":0.48699999999999999,"15min":0.42099999999999999},"max":{"1min":0.79100000000000004,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96499999999999997,"5min":1.171,"15min":1.8220000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.75900000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.7370000000000001},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.79600000000000004},"min":{"1min":0.67300000000000004,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.747},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86899999999999999,"15min":0.83499999999999996},"min":{"1min":0.74299999999999999,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.80400000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80000000000000004,"15min":0.76300000000000001},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":1.0569999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.3540000000000001},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.82999999999999996,"15min":0.79100000000000004},"min":{"1min":0.623,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.70499999999999996,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.83799999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575484,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438204,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":115008,"kb_avail":93933636,"statfs":{"total":96636764160,"available":96188043264,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117768192},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:29 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.46500000000000002,"15min":0.45600000000000002},"min":{"1min":0.37,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.64200000000000002,"5min":0.66700000000000004,"15min":1.7969999999999999},"last":0.442},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.53500000000000003},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.33600000000000002},"max":{"1min":0.68200000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.59999999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.495},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.26900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":0.92500000000000004,"15min":1.355},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.073,"15min":1.635},"last":1.006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.74299999999999999},"min":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":3.081,"15min":3.081},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.76200000000000001},"min":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0229999999999999,"5min":1.198,"15min":1.2110000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.85799999999999998,"15min":0.80200000000000005},"min":{"1min":0.70699999999999996,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.0609999999999999,"5min":1.234,"15min":1.367},"last":0.91000000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.78600000000000003},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.77000000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.014,"5min":1.103,"15min":3.3490000000000002},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.60599999999999998,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.306,"15min":1.413},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.878,"15min":0.86399999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0029999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.0309999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575483,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418260,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125504,"kb_avail":93953580,"statfs":{"total":96636764160,"available":96208465920,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128516096},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.74199999999999999},"min":{"1min":0.54400000000000004,"5min":0.51200000000000001,"15min":0.43099999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.1899999999999999},"last":0.78500000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.0640000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":1.03,"5min":1.044,"15min":1.1279999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.34300000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.609},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.63800000000000001,"5min":0.47999999999999998,"15min":0.38100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.159},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.67800000000000005},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.407},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.30299999999999999},"max":{"1min":0.95499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.498},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65200000000000002,"15min":0.61299999999999999},"min":{"1min":0.51100000000000001,"5min":0.41699999999999998,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.59699999999999998},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.91500000000000004,"5min":1.032,"15min":1.032},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.65800000000000003},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":1.046,"15min":1.361},"last":0.61199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575483,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292824,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75456,"kb_avail":94079016,"statfs":{"total":96636764160,"available":96336912384,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77266944},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.69799999999999995},"min":{"1min":0.48299999999999998,"5min":0.44900000000000001,"15min":0.32600000000000001},"max":{"1min":0.89500000000000002,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.74299999999999999,"15min":0.70899999999999996},"min":{"1min":0.44,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89700000000000002,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.92900000000000005,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.879,"5min":19.715,"15min":19.715},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.82199999999999995,"15min":0.71699999999999997},"min":{"1min":0.498,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":0.92600000000000005,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75900000000000001,"15min":0.69599999999999995},"min":{"1min":0.48999999999999999,"5min":0.44400000000000001,"15min":0.38},"max":{"1min":0.86499999999999999,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.86699999999999999,"15min":0.76300000000000001},"min":{"1min":0.58599999999999997,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":14.439,"15min":14.439},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.88600000000000001,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.89400000000000002,"15min":0.81299999999999994},"min":{"1min":0.64700000000000002,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":0.88500000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":1.29,"15min":0.82399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.495},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.3280000000000001,"15min":0.83199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42599999999999999,"15min":0.35199999999999998},"max":{"1min":0.68400000000000005,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.46500000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.79100000000000004,"15min":0.68500000000000005},"min":{"1min":0.48599999999999999,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.76100000000000001,"5min":12.163,"15min":12.163},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.71499999999999997,"15min":0.64700000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.88400000000000001,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.51500000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:02:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.71899999999999997,"15min":0.68500000000000005},"min":{"1min":0.45300000000000001,"5min":0.44600000000000001,"15min":0.39300000000000002},"max":{"1min":0.86899999999999999,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.98599999999999999,"15min":0.81599999999999995},"min":{"1min":0.56899999999999995,"5min":0.55200000000000005,"15min":0.41999999999999998},"max":{"1min":0.89500000000000002,"5min":19.834,"15min":19.834},"last":0.57399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:03:32.412 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.412 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.412 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.413 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.413 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.414 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.414 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.414 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.415 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.415 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.415 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.416 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.416 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.416 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.417 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.417 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.417 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.418 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.418 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.418 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.419 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.419 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.419 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.420 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.420 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.421 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.421 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.421 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.422 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.422 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.422 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.423 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.423 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.423 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.424 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.424 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.425 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.425 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.425 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.426 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.426 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.426 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.427 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.427 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.427 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.428 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.428 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.428 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.429 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.429 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.429 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.430 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.430 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.430 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.431 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.431 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.431 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.432 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.432 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.433 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.433 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.433 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.434 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.434 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.434 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.435 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.435 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.435 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.436 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.436 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.436 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.437 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.437 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.437 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:32.438 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:03:52.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:03:52.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.647+0000 7f8fa21ff700 1 Processor -- start 2023-06-09T12:03:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.647+0000 7f8fa21ff700 1 -- start start 2023-06-09T12:03:52.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.648+0000 7f8fa21ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.648+0000 7f8fa21ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f9c15cd40 0x7f8f9c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.648+0000 7f8fa21ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c159b20 con 0x7f8f9c0cb5d0 2023-06-09T12:03:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.648+0000 7f8fa21ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9c0cac80 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.648+0000 7f8fa21ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9c1597d0 con 0x7f8f9c15cd40 2023-06-09T12:03:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43432/0 (socket says 172.21.15.7:43432) 2023-06-09T12:03:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 -- 172.21.15.7:0/3860174033 learned_addr learned my addr 172.21.15.7:0/3860174033 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:03:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9bfff700 1 --2- 172.21.15.7:0/3860174033 >> v2:172.21.15.117:3300/0 conn(0x7f8f9c15cd40 0x7f8f9c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 -- 172.21.15.7:0/3860174033 >> v2:172.21.15.117:3300/0 conn(0x7f8f9c15cd40 msgr2=0x7f8f9c0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:03:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 --2- 172.21.15.7:0/3860174033 >> v2:172.21.15.117:3300/0 conn(0x7f8f9c15cd40 0x7f8f9c0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 -- 172.21.15.7:0/3860174033 >> v1:172.21.15.117:6789/0 conn(0x7f8f9c0cb5d0 legacy=0x7f8f9c159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 -- 172.21.15.7:0/3860174033 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c071ab0 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.649+0000 7f8f9affd700 1 --2- 172.21.15.7:0/3860174033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0c9b80 tx=0x7f8f90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfda6b55fa137ef7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8f9a7fc700 1 -- 172.21.15.7:0/3860174033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90011830 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8f9a7fc700 1 -- 172.21.15.7:0/3860174033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90011990 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8f9a7fc700 1 -- 172.21.15.7:0/3860174033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90011d20 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 -- 172.21.15.7:0/3860174033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0cb9a0 msgr2=0x7f8f9c0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/3860174033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0c9b80 tx=0x7f8f90009bd0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 -- 172.21.15.7:0/3860174033 shutdown_connections 2023-06-09T12:03:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/3860174033 >> v2:172.21.15.117:3300/0 conn(0x7f8f9c15cd40 0x7f8f9c0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/3860174033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c0b52b0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 -- 172.21.15.7:0/3860174033 >> 172.21.15.7:0/3860174033 conn(0x7f8f9c070af0 msgr2=0x7f8f9c070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 -- 172.21.15.7:0/3860174033 shutdown_connections 2023-06-09T12:03:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.650+0000 7f8fa21ff700 1 -- 172.21.15.7:0/3860174033 wait complete. 2023-06-09T12:03:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.652+0000 7f8fa21ff700 1 Processor -- start 2023-06-09T12:03:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.652+0000 7f8fa21ff700 1 -- start start 2023-06-09T12:03:52.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f9c0cb5d0 0x7f8f9c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c1f1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c153670 con 0x7f8f9c15cd40 2023-06-09T12:03:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c1f2c60 con 0x7f8f9c0cb9a0 2023-06-09T12:03:52.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8fa21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c0cdaa0 con 0x7f8f9c0cb5d0 2023-06-09T12:03:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8f9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8f9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2023-06-09T12:03:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.653+0000 7f8f9bfff700 1 -- 172.21.15.7:0/1861013545 learned_addr learned my addr 172.21.15.7:0/1861013545 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:03:52.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9b7fe700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f9c0cb5d0 0x7f8f9c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9affd700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c1f1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 -- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f9c0cb5d0 msgr2=0x7f8f9c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f9c0cb5d0 0x7f8f9c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 -- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c0cb9a0 msgr2=0x7f8f9c1f1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c1f1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 -- 172.21.15.7:0/1861013545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f90009550 con 0x7f8f9c15cd40 2023-06-09T12:03:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f9bfff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400efc0 tx=0x7f8f94009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94009b20 con 0x7f8f9c15cd40 2023-06-09T12:03:52.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9400f040 con 0x7f8f9c15cd40 2023-06-09T12:03:52.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c15d4f0 con 0x7f8f9c15cd40 2023-06-09T12:03:52.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.654+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400cc90 con 0x7f8f9c15cd40 2023-06-09T12:03:52.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.655+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c15d210 con 0x7f8f9c15cd40 2023-06-09T12:03:52.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.656+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c15d7d0 con 0x7f8f9c15cd40 2023-06-09T12:03:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.657+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8f94014480 con 0x7f8f9c15cd40 2023-06-09T12:03:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.658+0000 7f8f83fff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 0x7f8f8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:03:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.658+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8f9408a820 con 0x7f8f9c15cd40 2023-06-09T12:03:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.658+0000 7f8f9b7fe700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 0x7f8f8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:03:52.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.658+0000 7f8f9b7fe700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 0x7f8f8806d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c007fd0 tx=0x7f8f8c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:03:52.666 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.663+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8f94054f50 con 0x7f8f9c15cd40 2023-06-09T12:03:52.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.870+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f9c074cf0 con 0x7f8f8806ace0 2023-06-09T12:03:52.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.880+0000 7f8f83fff700 1 -- 172.21.15.7:0/1861013545 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315460 (secure 0 0 0) 0x7f8f9c074cf0 con 0x7f8f8806ace0 2023-06-09T12:03:52.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:03:52.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 msgr2=0x7f8f8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 0x7f8f8806d190 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c007fd0 tx=0x7f8f8c009040 comp rx=0 tx=0).stop 2023-06-09T12:03:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 msgr2=0x7f8f9c150b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:03:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400efc0 tx=0x7f8f94009f60 comp rx=0 tx=0).stop 2023-06-09T12:03:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 shutdown_connections 2023-06-09T12:03:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f9c15cd40 0x7f8f9c150b20 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8f8806ace0 0x7f8f8806d190 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f9c0cb5d0 0x7f8f9c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 --2- 172.21.15.7:0/1861013545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f9c0cb9a0 0x7f8f9c1f1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:03:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 >> 172.21.15.7:0/1861013545 conn(0x7f8f9c070af0 msgr2=0x7f8f9c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:03:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 shutdown_connections 2023-06-09T12:03:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:03:52.884+0000 7f8fa21ff700 1 -- 172.21.15.7:0/1861013545 wait complete. 2023-06-09T12:03:52.890 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:03:52.905 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1662,"stamp":"2023-06-09T12:03:50.994748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1005562,"num_omap_keys":1920,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92256,"ondisk_log_size":92256,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2563400,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":680064,"kb_avail":752411320,"statfs":{"total":773094113280,"available":770469191680,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":696385536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003451"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3468,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:15.925532+0000","last_change":"2023-06-09T12:03:15.925532+0000","last_active":"2023-06-09T12:03:15.925532+0000","last_peered":"2023-06-09T12:03:15.925532+0000","last_clean":"2023-06-09T12:03:15.925532+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:03:15.925532+0000","last_undegraded":"2023-06-09T12:03:15.925532+0000","last_fullsized":"2023-06-09T12:03:15.925532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3221,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:35.073409+0000","last_change":"2023-06-09T12:02:57.307464+0000","last_active":"2023-06-09T12:02:35.073409+0000","last_peered":"2023-06-09T12:02:35.073409+0000","last_clean":"2023-06-09T12:02:35.073409+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:35.073409+0000","last_undegraded":"2023-06-09T12:02:35.073409+0000","last_fullsized":"2023-06-09T12:02:35.073409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3699,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088463+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088463+0000","last_peered":"2023-06-09T12:03:46.088463+0000","last_clean":"2023-06-09T12:03:46.088463+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088463+0000","last_undegraded":"2023-06-09T12:03:46.088463+0000","last_fullsized":"2023-06-09T12:03:46.088463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587585+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587585+0000","last_peered":"2023-06-09T12:01:30.587585+0000","last_clean":"2023-06-09T12:01:30.587585+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587585+0000","last_undegraded":"2023-06-09T12:01:30.587585+0000","last_fullsized":"2023-06-09T12:01:30.587585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4199,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:41.423632+0000","last_change":"2023-06-09T11:18:26.321080+0000","last_active":"2023-06-09T11:59:41.423632+0000","last_peered":"2023-06-09T11:59:41.423632+0000","last_clean":"2023-06-09T11:59:41.423632+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T11:59:41.423632+0000","last_undegraded":"2023-06-09T11:59:41.423632+0000","last_fullsized":"2023-06-09T11:59:41.423632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3694,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.296785+0000","last_change":"2023-06-09T12:03:38.180489+0000","last_active":"2023-06-09T12:01:37.296785+0000","last_peered":"2023-06-09T12:01:37.296785+0000","last_clean":"2023-06-09T12:01:37.296785+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:01:37.296785+0000","last_undegraded":"2023-06-09T12:01:37.296785+0000","last_fullsized":"2023-06-09T12:01:37.296785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3459,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418150+0000","last_change":"2023-06-09T12:02:59.084217+0000","last_active":"2023-06-09T12:01:37.418150+0000","last_peered":"2023-06-09T12:01:37.418150+0000","last_clean":"2023-06-09T12:01:37.418150+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:01:37.418150+0000","last_undegraded":"2023-06-09T12:01:37.418150+0000","last_fullsized":"2023-06-09T12:01:37.418150+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3406,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:33.927817+0000","last_change":"2023-06-09T12:01:53.917872+0000","last_active":"2023-06-09T12:03:33.927817+0000","last_peered":"2023-06-09T12:03:33.927817+0000","last_clean":"2023-06-09T12:03:33.927817+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:03:33.927817+0000","last_undegraded":"2023-06-09T12:03:33.927817+0000","last_fullsized":"2023-06-09T12:03:33.927817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3236,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:18.884495+0000","last_change":"2023-06-09T12:01:57.947291+0000","last_active":"2023-06-09T12:03:18.884495+0000","last_peered":"2023-06-09T12:03:18.884495+0000","last_clean":"2023-06-09T12:03:18.884495+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:03:18.884495+0000","last_undegraded":"2023-06-09T12:03:18.884495+0000","last_fullsized":"2023-06-09T12:03:18.884495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3344,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407306+0000","last_change":"2023-06-09T12:02:28.972507+0000","last_active":"2023-06-09T12:02:12.407306+0000","last_peered":"2023-06-09T12:02:12.407306+0000","last_clean":"2023-06-09T12:02:12.407306+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:12.407306+0000","last_undegraded":"2023-06-09T12:02:12.407306+0000","last_fullsized":"2023-06-09T12:02:12.407306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3735,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.286805+0000","last_change":"2023-06-09T11:18:26.330325+0000","last_active":"2023-06-09T11:59:52.286805+0000","last_peered":"2023-06-09T11:59:52.286805+0000","last_clean":"2023-06-09T11:59:52.286805+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T11:59:52.286805+0000","last_undegraded":"2023-06-09T11:59:52.286805+0000","last_fullsized":"2023-06-09T11:59:52.286805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'924","reported_seq":3102,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407536+0000","last_change":"2023-06-09T11:18:26.323774+0000","last_active":"2023-06-09T12:00:22.407536+0000","last_peered":"2023-06-09T12:00:22.407536+0000","last_clean":"2023-06-09T12:00:22.407536+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:00:22.407536+0000","last_undegraded":"2023-06-09T12:00:22.407536+0000","last_fullsized":"2023-06-09T12:00:22.407536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'923","reported_seq":3555,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:46.828380+0000","last_change":"2023-06-09T12:03:41.975490+0000","last_active":"2023-06-09T12:03:46.828380+0000","last_peered":"2023-06-09T12:03:46.828380+0000","last_clean":"2023-06-09T12:03:46.828380+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:03:46.828380+0000","last_undegraded":"2023-06-09T12:03:46.828380+0000","last_fullsized":"2023-06-09T12:03:46.828380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":923,"log_dups_size":0,"ondisk_log_size":923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3359,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.412828+0000","last_change":"2023-06-09T12:03:38.024447+0000","last_active":"2023-06-09T12:02:57.412828+0000","last_peered":"2023-06-09T12:02:57.412828+0000","last_clean":"2023-06-09T12:02:57.412828+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:02:57.412828+0000","last_undegraded":"2023-06-09T12:02:57.412828+0000","last_fullsized":"2023-06-09T12:02:57.412828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306849+0000","last_change":"2023-06-09T12:03:46.278802+0000","last_active":"2023-06-09T12:02:52.306849+0000","last_peered":"2023-06-09T12:02:52.306849+0000","last_clean":"2023-06-09T12:02:52.306849+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:02:52.306849+0000","last_undegraded":"2023-06-09T12:02:52.306849+0000","last_fullsized":"2023-06-09T12:02:52.306849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1998,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:33.899239+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:00:33.899239+0000","last_peered":"2023-06-09T12:00:33.899239+0000","last_clean":"2023-06-09T12:00:33.899239+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:00:33.899239+0000","last_undegraded":"2023-06-09T12:00:33.899239+0000","last_fullsized":"2023-06-09T12:00:33.899239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3308,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:03.325248+0000","last_change":"2023-06-09T12:02:57.506260+0000","last_active":"2023-06-09T12:02:03.325248+0000","last_peered":"2023-06-09T12:02:03.325248+0000","last_clean":"2023-06-09T12:02:03.325248+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:02:03.325248+0000","last_undegraded":"2023-06-09T12:02:03.325248+0000","last_fullsized":"2023-06-09T12:02:03.325248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:59.471359+0000","last_change":"2023-06-09T12:01:37.300265+0000","last_active":"2023-06-09T11:59:59.471359+0000","last_peered":"2023-06-09T11:59:59.471359+0000","last_clean":"2023-06-09T11:59:59.471359+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T11:59:59.471359+0000","last_undegraded":"2023-06-09T11:59:59.471359+0000","last_fullsized":"2023-06-09T11:59:59.471359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3278,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.312444+0000","last_change":"2023-06-09T12:02:29.148543+0000","last_active":"2023-06-09T12:02:12.312444+0000","last_peered":"2023-06-09T12:02:12.312444+0000","last_clean":"2023-06-09T12:02:12.312444+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:12.312444+0000","last_undegraded":"2023-06-09T12:02:12.312444+0000","last_fullsized":"2023-06-09T12:02:12.312444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3104,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.299809+0000","last_change":"2023-06-09T12:03:13.801581+0000","last_active":"2023-06-09T12:02:12.299809+0000","last_peered":"2023-06-09T12:02:12.299809+0000","last_clean":"2023-06-09T12:02:12.299809+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:02:12.299809+0000","last_undegraded":"2023-06-09T12:02:12.299809+0000","last_fullsized":"2023-06-09T12:02:12.299809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3098,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.422668+0000","last_change":"2023-06-09T12:03:02.033567+0000","last_active":"2023-06-09T12:02:12.422668+0000","last_peered":"2023-06-09T12:02:12.422668+0000","last_clean":"2023-06-09T12:02:12.422668+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:02:12.422668+0000","last_undegraded":"2023-06-09T12:02:12.422668+0000","last_fullsized":"2023-06-09T12:02:12.422668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407323+0000","last_change":"2023-06-09T12:02:49.161159+0000","last_active":"2023-06-09T12:02:12.407323+0000","last_peered":"2023-06-09T12:02:12.407323+0000","last_clean":"2023-06-09T12:02:12.407323+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:12.407323+0000","last_undegraded":"2023-06-09T12:02:12.407323+0000","last_fullsized":"2023-06-09T12:02:12.407323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3589,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.296720+0000","last_change":"2023-06-09T12:03:43.259734+0000","last_active":"2023-06-09T12:01:37.296720+0000","last_peered":"2023-06-09T12:01:37.296720+0000","last_clean":"2023-06-09T12:01:37.296720+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:01:37.296720+0000","last_undegraded":"2023-06-09T12:01:37.296720+0000","last_fullsized":"2023-06-09T12:01:37.296720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3025,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.300179+0000","last_change":"2023-06-09T12:01:54.914290+0000","last_active":"2023-06-09T12:01:37.300179+0000","last_peered":"2023-06-09T12:01:37.300179+0000","last_clean":"2023-06-09T12:01:37.300179+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:37.300179+0000","last_undegraded":"2023-06-09T12:01:37.300179+0000","last_fullsized":"2023-06-09T12:01:37.300179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410511+0000","last_change":"2023-06-09T12:02:51.154631+0000","last_active":"2023-06-09T12:02:37.410511+0000","last_peered":"2023-06-09T12:02:37.410511+0000","last_clean":"2023-06-09T12:02:37.410511+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:37.410511+0000","last_undegraded":"2023-06-09T12:02:37.410511+0000","last_fullsized":"2023-06-09T12:02:37.410511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410488+0000","last_change":"2023-06-09T12:03:02.187861+0000","last_active":"2023-06-09T12:02:37.410488+0000","last_peered":"2023-06-09T12:02:37.410488+0000","last_clean":"2023-06-09T12:02:37.410488+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:02:37.410488+0000","last_undegraded":"2023-06-09T12:02:37.410488+0000","last_fullsized":"2023-06-09T12:02:37.410488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:41.487710+0000","last_change":"2023-06-09T12:03:02.506728+0000","last_active":"2023-06-09T12:02:41.487710+0000","last_peered":"2023-06-09T12:02:41.487710+0000","last_clean":"2023-06-09T12:02:41.487710+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:02:41.487710+0000","last_undegraded":"2023-06-09T12:02:41.487710+0000","last_fullsized":"2023-06-09T12:02:41.487710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.290653+0000","last_change":"2023-06-09T11:18:26.348474+0000","last_active":"2023-06-09T12:00:22.290653+0000","last_peered":"2023-06-09T12:00:22.290653+0000","last_clean":"2023-06-09T12:00:22.290653+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:00:22.290653+0000","last_undegraded":"2023-06-09T12:00:22.290653+0000","last_fullsized":"2023-06-09T12:00:22.290653+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:32.896322+0000","last_change":"2023-06-09T12:02:37.315767+0000","last_active":"2023-06-09T12:00:32.896322+0000","last_peered":"2023-06-09T12:00:32.896322+0000","last_clean":"2023-06-09T12:00:32.896322+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:00:32.896322+0000","last_undegraded":"2023-06-09T12:00:32.896322+0000","last_fullsized":"2023-06-09T12:00:32.896322+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3841,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.426512+0000","last_change":"2023-06-09T12:03:03.011006+0000","last_active":"2023-06-09T12:02:37.426512+0000","last_peered":"2023-06-09T12:02:37.426512+0000","last_clean":"2023-06-09T12:02:37.426512+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:02:37.426512+0000","last_undegraded":"2023-06-09T12:02:37.426512+0000","last_fullsized":"2023-06-09T12:02:37.426512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:04.445847+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:00:04.445847+0000","last_peered":"2023-06-09T12:00:04.445847+0000","last_clean":"2023-06-09T12:00:04.445847+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:00:04.445847+0000","last_undegraded":"2023-06-09T12:00:04.445847+0000","last_fullsized":"2023-06-09T12:00:04.445847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.304467+0000","last_change":"2023-06-09T12:03:35.770715+0000","last_active":"2023-06-09T12:02:47.304467+0000","last_peered":"2023-06-09T12:02:47.304467+0000","last_clean":"2023-06-09T12:02:47.304467+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:02:47.304467+0000","last_undegraded":"2023-06-09T12:02:47.304467+0000","last_fullsized":"2023-06-09T12:02:47.304467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305793+0000","last_change":"2023-06-09T12:03:17.778068+0000","last_active":"2023-06-09T12:02:57.305793+0000","last_peered":"2023-06-09T12:02:57.305793+0000","last_clean":"2023-06-09T12:02:57.305793+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:02:57.305793+0000","last_undegraded":"2023-06-09T12:02:57.305793+0000","last_fullsized":"2023-06-09T12:02:57.305793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:43.105985+0000","last_change":"2023-06-09T12:02:52.306808+0000","last_active":"2023-06-09T12:02:43.105985+0000","last_peered":"2023-06-09T12:02:43.105985+0000","last_clean":"2023-06-09T12:02:43.105985+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:43.105985+0000","last_undegraded":"2023-06-09T12:02:43.105985+0000","last_fullsized":"2023-06-09T12:02:43.105985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":923,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590413+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590413+0000","last_peered":"2023-06-09T12:02:10.590413+0000","last_clean":"2023-06-09T12:02:10.590413+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590413+0000","last_undegraded":"2023-06-09T12:02:10.590413+0000","last_fullsized":"2023-06-09T12:02:10.590413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3892,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305823+0000","last_change":"2023-06-09T12:03:46.981824+0000","last_active":"2023-06-09T12:02:57.305823+0000","last_peered":"2023-06-09T12:02:57.305823+0000","last_clean":"2023-06-09T12:02:57.305823+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:57.305823+0000","last_undegraded":"2023-06-09T12:02:57.305823+0000","last_fullsized":"2023-06-09T12:02:57.305823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.762926+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.762926+0000","last_peered":"2023-06-09T12:01:53.762926+0000","last_clean":"2023-06-09T12:01:53.762926+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.762926+0000","last_undegraded":"2023-06-09T12:01:53.762926+0000","last_fullsized":"2023-06-09T12:01:53.762926+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'883","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:06.123925+0000","last_change":"2023-06-09T12:02:52.305226+0000","last_active":"2023-06-09T12:02:06.123925+0000","last_peered":"2023-06-09T12:02:06.123925+0000","last_clean":"2023-06-09T12:02:06.123925+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:02:06.123925+0000","last_undegraded":"2023-06-09T12:02:06.123925+0000","last_fullsized":"2023-06-09T12:02:06.123925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":316,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:58.218793+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T11:59:58.218793+0000","last_peered":"2023-06-09T11:59:58.218793+0000","last_clean":"2023-06-09T11:59:58.218793+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T11:59:58.218793+0000","last_undegraded":"2023-06-09T11:59:58.218793+0000","last_fullsized":"2023-06-09T11:59:58.218793+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1348,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:03.602373+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:00:03.602373+0000","last_peered":"2023-06-09T12:00:03.602373+0000","last_clean":"2023-06-09T12:00:03.602373+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:00:03.602373+0000","last_undegraded":"2023-06-09T12:00:03.602373+0000","last_fullsized":"2023-06-09T12:00:03.602373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:39.465215+0000","last_change":"2023-06-09T12:03:07.507333+0000","last_active":"2023-06-09T12:02:39.465215+0000","last_peered":"2023-06-09T12:02:39.465215+0000","last_clean":"2023-06-09T12:02:39.465215+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:02:39.465215+0000","last_undegraded":"2023-06-09T12:02:39.465215+0000","last_fullsized":"2023-06-09T12:02:39.465215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:31.034843+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:31.034843+0000","last_peered":"2023-06-09T12:02:31.034843+0000","last_clean":"2023-06-09T12:02:31.034843+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:31.034843+0000","last_undegraded":"2023-06-09T12:02:31.034843+0000","last_fullsized":"2023-06-09T12:02:31.034843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2988,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:53.030143+0000","last_change":"2023-06-09T12:02:57.310842+0000","last_active":"2023-06-09T12:00:53.030143+0000","last_peered":"2023-06-09T12:00:53.030143+0000","last_clean":"2023-06-09T12:00:53.030143+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:00:53.030143+0000","last_undegraded":"2023-06-09T12:00:53.030143+0000","last_fullsized":"2023-06-09T12:00:53.030143+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696148+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696148+0000","last_peered":"2023-06-09T12:03:31.696148+0000","last_clean":"2023-06-09T12:03:31.696148+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696148+0000","last_undegraded":"2023-06-09T12:03:31.696148+0000","last_fullsized":"2023-06-09T12:03:31.696148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.300145+0000","last_change":"2023-06-09T12:00:42.300145+0000","last_active":"2023-06-09T12:00:42.300145+0000","last_peered":"2023-06-09T12:00:42.300145+0000","last_clean":"2023-06-09T12:00:42.300145+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:00:42.300145+0000","last_undegraded":"2023-06-09T12:00:42.300145+0000","last_fullsized":"2023-06-09T12:00:42.300145+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:48.465622+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T11:59:48.465622+0000","last_peered":"2023-06-09T11:59:48.465622+0000","last_clean":"2023-06-09T11:59:48.465622+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T11:59:48.465622+0000","last_undegraded":"2023-06-09T11:59:48.465622+0000","last_fullsized":"2023-06-09T11:59:48.465622+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.504948+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:47.504948+0000","last_peered":"2023-06-09T12:02:47.504948+0000","last_clean":"2023-06-09T12:02:47.504948+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:47.504948+0000","last_undegraded":"2023-06-09T12:02:47.504948+0000","last_fullsized":"2023-06-09T12:02:47.504948+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:34.883218+0000","last_change":"2023-06-09T12:02:52.310351+0000","last_active":"2023-06-09T12:01:34.883218+0000","last_peered":"2023-06-09T12:01:34.883218+0000","last_clean":"2023-06-09T12:01:34.883218+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:01:34.883218+0000","last_undegraded":"2023-06-09T12:01:34.883218+0000","last_fullsized":"2023-06-09T12:01:34.883218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":883,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:48.982495+0000","last_change":"2023-06-09T12:03:22.314299+0000","last_active":"2023-06-09T12:01:48.982495+0000","last_peered":"2023-06-09T12:01:48.982495+0000","last_clean":"2023-06-09T12:01:48.982495+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:01:48.982495+0000","last_undegraded":"2023-06-09T12:01:48.982495+0000","last_fullsized":"2023-06-09T12:01:48.982495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":770,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.225656+0000","last_change":"2023-06-09T12:03:22.310323+0000","last_active":"2023-06-09T12:02:50.225656+0000","last_peered":"2023-06-09T12:02:50.225656+0000","last_clean":"2023-06-09T12:02:50.225656+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:02:50.225656+0000","last_undegraded":"2023-06-09T12:02:50.225656+0000","last_fullsized":"2023-06-09T12:02:50.225656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1036","reported_seq":3743,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:39.387195+0000","last_change":"2023-06-09T12:03:11.068988+0000","last_active":"2023-06-09T12:03:39.387195+0000","last_peered":"2023-06-09T12:03:39.387195+0000","last_clean":"2023-06-09T12:03:39.387195+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:39.387195+0000","last_undegraded":"2023-06-09T12:03:39.387195+0000","last_fullsized":"2023-06-09T12:03:39.387195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":170,"log_size":1036,"log_dups_size":0,"ondisk_log_size":1036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3150,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:47.309246+0000","last_change":"2023-06-09T12:03:08.983351+0000","last_active":"2023-06-09T12:01:47.309246+0000","last_peered":"2023-06-09T12:01:47.309246+0000","last_clean":"2023-06-09T12:01:47.309246+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:01:47.309246+0000","last_undegraded":"2023-06-09T12:01:47.309246+0000","last_fullsized":"2023-06-09T12:01:47.309246+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3483,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:01:37.418020+0000","last_change":"2023-06-09T12:02:56.996951+0000","last_active":"2023-06-09T12:01:37.418020+0000","last_peered":"2023-06-09T12:01:37.418020+0000","last_clean":"2023-06-09T12:01:37.418020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:01:37.418020+0000","last_undegraded":"2023-06-09T12:01:37.418020+0000","last_fullsized":"2023-06-09T12:01:37.418020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3429,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:04.323552+0000","last_change":"2023-06-09T12:02:57.506183+0000","last_active":"2023-06-09T12:02:04.323552+0000","last_peered":"2023-06-09T12:02:04.323552+0000","last_clean":"2023-06-09T12:02:04.323552+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:02:04.323552+0000","last_undegraded":"2023-06-09T12:02:04.323552+0000","last_fullsized":"2023-06-09T12:02:04.323552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3939,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.318356+0000","last_change":"2023-06-09T12:03:09.981419+0000","last_active":"2023-06-09T12:02:57.318356+0000","last_peered":"2023-06-09T12:02:57.318356+0000","last_clean":"2023-06-09T12:02:57.318356+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:02:57.318356+0000","last_undegraded":"2023-06-09T12:02:57.318356+0000","last_fullsized":"2023-06-09T12:02:57.318356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2980,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302648+0000","last_change":"2023-06-09T11:18:26.280511+0000","last_active":"2023-06-09T12:01:02.302648+0000","last_peered":"2023-06-09T12:01:02.302648+0000","last_clean":"2023-06-09T12:01:02.302648+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:01:02.302648+0000","last_undegraded":"2023-06-09T12:01:02.302648+0000","last_fullsized":"2023-06-09T12:01:02.302648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":61395,"ondisk_log_size":61395,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1005562,"num_omap_keys":1920,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30742,"ondisk_log_size":30742,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575488,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255120,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":75264,"kb_avail":94116720,"statfs":{"total":96636764160,"available":96375521280,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":77070336},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.66600000000000004},"min":{"1min":0.60599999999999998,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.83499999999999996,"5min":1.004,"15min":1.0640000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.90000000000000002,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.96699999999999997,"15min":1.25},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.55900000000000005,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.05,"15min":1.05},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89800000000000002,"5min":0.97599999999999998,"15min":1.099},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.53300000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.91800000000000004,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.67200000000000004},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.70899999999999996},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.1200000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71799999999999997,"15min":0.68700000000000006},"min":{"1min":0.62,"5min":0.45900000000000002,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.83999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.754,"5min":1.044,"15min":1.0920000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.879}]}]},{"osd":6,"up_from":13,"seq":55834575488,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203608,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47488,"kb_avail":94168232,"statfs":{"total":96636764160,"available":96428269568,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48627712},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.63600000000000001},"min":{"1min":0.53000000000000003,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.95899999999999996,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.64100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.748}]},{"osd":1,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96599999999999997,"15min":1.081},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.85799999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70699999999999996,"15min":0.67600000000000005},"min":{"1min":0.623,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":0.98499999999999999,"15min":1.024},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.86499999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.036,"15min":1.0369999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.74399999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.68799999999999994},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.96999999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":1.0129999999999999,"5min":1.052,"15min":1.052},"last":0.81399999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.69099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":1.0029999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575488,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401200,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94784,"kb_avail":93970640,"statfs":{"total":96636764160,"available":96225935360,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":97058816},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.36899999999999999},"max":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":1.877},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.47399999999999998},"min":{"1min":0.39600000000000002,"5min":0.318,"15min":0.28000000000000003},"max":{"1min":0.995,"5min":0.995,"15min":1.982},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.504}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":1.577},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.38700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92900000000000005,"15min":1.635},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72899999999999998,"15min":0.69399999999999995},"min":{"1min":0.59899999999999998,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.609},"last":0.873},{"interface":"front","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.774},"last":0.98399999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.78900000000000003},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.145,"5min":1.145,"15min":1.7390000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88500000000000001,"15min":0.83099999999999996},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0840000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.89000000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.27,"15min":1.794},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81000000000000005,"15min":0.76900000000000002},"min":{"1min":0.64100000000000001,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.921},"last":0.91800000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89800000000000002,"15min":0.875},"min":{"1min":0.76100000000000001,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.839},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.66200000000000003,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.827},"last":0.70899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575488,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332648,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104832,"kb_avail":94039192,"statfs":{"total":96636764160,"available":96296132608,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107347968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.58899999999999997,"5min":0.68300000000000005,"15min":0.70799999999999996},"last":0.498},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.41199999999999998,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.67800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.79500000000000004,"5min":1.032,"15min":1.0900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.44}]},{"osd":3,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.2909999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77200000000000002,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.0429999999999999,"15min":1.236},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82199999999999995,"15min":0.76800000000000002},"min":{"1min":0.63900000000000001,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.51300000000000001,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0189999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83299999999999996,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.212,"5min":1.288,"15min":1.288},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90900000000000003,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.1299999999999999,"5min":1.351,"15min":1.351},"last":0.86299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237664,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":57856,"kb_avail":94134176,"statfs":{"total":96636764160,"available":96393396224,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":59244544},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.39700000000000002,"5min":0.309,"15min":0.26800000000000002},"max":{"1min":0.54600000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.73699999999999999,"5min":0.999,"15min":1.0940000000000001},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.67100000000000004,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.442},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.54000000000000004,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.51000000000000001,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.82099999999999995,"5min":0.96799999999999997,"15min":1.546},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.56599999999999995,"5min":0.48699999999999999,"15min":0.42099999999999999},"max":{"1min":0.79100000000000004,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96499999999999997,"5min":1.171,"15min":1.8220000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.75900000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.7370000000000001},"last":0.78600000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.79600000000000004},"min":{"1min":0.67300000000000004,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86899999999999999,"15min":0.83499999999999996},"min":{"1min":0.74299999999999999,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80000000000000004,"15min":0.76300000000000001},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":1.0569999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.3540000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.82999999999999996,"15min":0.79100000000000004},"min":{"1min":0.623,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.749},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.70499999999999996,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.70599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575488,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421884,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":98688,"kb_avail":93949956,"statfs":{"total":96636764160,"available":96204754944,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101056512},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.46100000000000002},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.223},"max":{"1min":0.624,"5min":0.66700000000000004,"15min":1.7969999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.49199999999999999,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.495},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.26900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.505},{"interface":"front","average":{"1min":0.502,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.46000000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":0.92500000000000004,"15min":1.355},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.39900000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.073,"15min":1.635},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.74299999999999999},"min":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":3.081,"15min":3.081},"last":0.76900000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.76200000000000001},"min":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0229999999999999,"5min":1.198,"15min":1.2110000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.871,"5min":0.85799999999999998,"15min":0.80200000000000005},"min":{"1min":0.70699999999999996,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.0609999999999999,"5min":1.234,"15min":1.367},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.78600000000000003},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.77000000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.014,"5min":1.103,"15min":3.3490000000000002},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.60599999999999998,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.306,"15min":1.413},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.878,"15min":0.86399999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0029999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.80000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575487,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418388,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125632,"kb_avail":93953452,"statfs":{"total":96636764160,"available":96208334848,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128647168},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.74199999999999999},"min":{"1min":0.54400000000000004,"5min":0.51200000000000001,"15min":0.43099999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.1899999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.0640000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":1.03,"5min":1.044,"15min":1.1279999999999999},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.34300000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.609},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.63800000000000001,"5min":0.47999999999999998,"15min":0.38100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.159},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.67800000000000005},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.407},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.30299999999999999},"max":{"1min":0.95499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65200000000000002,"15min":0.61299999999999999},"min":{"1min":0.51100000000000001,"5min":0.41699999999999998,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.624}]},{"osd":6,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.75700000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.59699999999999998},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.91500000000000004,"5min":1.032,"15min":1.032},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.65800000000000003},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":1.046,"15min":1.361},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575487,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":292888,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":94078952,"statfs":{"total":96636764160,"available":96336846848,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.69999999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.32600000000000001},"max":{"1min":0.94399999999999995,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.86299999999999999,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.91800000000000004,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.90500000000000003,"5min":19.715,"15min":19.715},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.72399999999999998},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.1870000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.76400000000000001,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.86599999999999999,"15min":0.76400000000000001},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.92600000000000005,"5min":14.439,"15min":14.439},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.75900000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.88300000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.90000000000000002,"15min":0.81699999999999995},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.2789999999999999,"15min":0.82399999999999995},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.64900000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.3069999999999999,"15min":0.82999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85799999999999998,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76800000000000002,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.78000000000000003,"5min":12.163,"15min":12.163},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71599999999999997,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.97799999999999998,"15min":0.81899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":19.834,"15min":19.834},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:03:52.912 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.912 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.913 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.913 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.913 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.914 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.914 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.915 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.915 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.915 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.916 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.916 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.916 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.917 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.917 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.918 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.918 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.918 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.919 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.919 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.919 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.920 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.920 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.920 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.921 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.921 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.922 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.922 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.922 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.923 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.923 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.923 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.924 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.924 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.925 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.925 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.925 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.926 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.926 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.926 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.927 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.927 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.927 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.928 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.928 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.929 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.929 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.929 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.930 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.930 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.930 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.931 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.931 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.932 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.932 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.932 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.933 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.933 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.933 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.934 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.934 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.935 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.935 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.935 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.936 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.936 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.936 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.937 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.937 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.937 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.938 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:03:52.938 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:03:56.883 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:03:56.888 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:12.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:04:13.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.148+0000 7fbee1dff700 1 Processor -- start 2023-06-09T12:04:13.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.149+0000 7fbee1dff700 1 -- start start 2023-06-09T12:04:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbee1dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbee1dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbee1dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc159b20 con 0x7fbedc0cb5d0 2023-06-09T12:04:13.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbee1dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbedc0cac80 con 0x7fbedc15cd40 2023-06-09T12:04:13.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbee1dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc1597d0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbedaffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbedbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbedbfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:59930/0 (socket says 172.21.15.7:59930) 2023-06-09T12:04:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbedaffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:46596/0 (socket says 172.21.15.7:46596) 2023-06-09T12:04:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.150+0000 7fbedbfff700 1 -- 172.21.15.7:0/139915165 learned_addr learned my addr 172.21.15.7:0/139915165 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:04:13.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedaffd700 1 -- 172.21.15.7:0/139915165 >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 msgr2=0x7fbedc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:04:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedaffd700 1 --2- 172.21.15.7:0/139915165 >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedaffd700 1 -- 172.21.15.7:0/139915165 >> v1:172.21.15.117:6789/0 conn(0x7fbedc0cb5d0 legacy=0x7fbedc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedaffd700 1 -- 172.21.15.7:0/139915165 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc071ab0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedbfff700 1 --2- 172.21.15.7:0/139915165 >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:04:13.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbedaffd700 1 --2- 172.21.15.7:0/139915165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009840 tx=0x7fbed0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966604b52de8a2c8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbeda7fc700 1 -- 172.21.15.7:0/139915165 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0011830 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbeda7fc700 1 -- 172.21.15.7:0/139915165 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed0011990 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.151+0000 7fbeda7fc700 1 -- 172.21.15.7:0/139915165 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0011d20 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 -- 172.21.15.7:0/139915165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 msgr2=0x7fbedc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 --2- 172.21.15.7:0/139915165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009840 tx=0x7fbed0009b50 comp rx=0 tx=0).stop 2023-06-09T12:04:13.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 -- 172.21.15.7:0/139915165 shutdown_connections 2023-06-09T12:04:13.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 --2- 172.21.15.7:0/139915165 >> v2:172.21.15.117:3300/0 conn(0x7fbedc15cd40 0x7fbedc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 --2- 172.21.15.7:0/139915165 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb9a0 0x7fbedc0b52b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 -- 172.21.15.7:0/139915165 >> 172.21.15.7:0/139915165 conn(0x7fbedc070af0 msgr2=0x7fbedc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:13.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 -- 172.21.15.7:0/139915165 shutdown_connections 2023-06-09T12:04:13.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.152+0000 7fbee1dff700 1 -- 172.21.15.7:0/139915165 wait complete. 2023-06-09T12:04:13.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.154+0000 7fbee1dff700 1 Processor -- start 2023-06-09T12:04:13.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.154+0000 7fbee1dff700 1 -- start start 2023-06-09T12:04:13.162 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb5d0 0x7fbedc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbedc15cd40 0x7fbedc151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc153be0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0cf740 con 0x7fbedc15cd40 2023-06-09T12:04:13.163 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbee1dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbedc0ce5b0 con 0x7fbedc0cb5d0 2023-06-09T12:04:13.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedaffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48194/0 (socket says 172.21.15.7:48194) 2023-06-09T12:04:13.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedaffd700 1 -- 172.21.15.7:0/3760934777 learned_addr learned my addr 172.21.15.7:0/3760934777 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:04:13.164 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedb7fe700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb5d0 0x7fbedc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedbfff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbedc15cd40 0x7fbedc151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.155+0000 7fbedaffd700 1 -- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb5d0 msgr2=0x7fbedc0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbedaffd700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb5d0 0x7fbedc0c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbedaffd700 1 -- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbedc15cd40 msgr2=0x7fbedc151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbedaffd700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbedc15cd40 0x7fbedc151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbedaffd700 1 -- 172.21.15.7:0/3760934777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009b30 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbedaffd700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000c00 tx=0x7fbed0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00036a0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed001f070 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.166 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed00094a0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed0003a40 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.156+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc15d4f0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.157+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc15d210 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.158+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbed0003ba0 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.167 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.159+0000 7fbec3fff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 0x7fbec806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:13.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.159+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fbed0021070 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.168 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.159+0000 7fbedb7fe700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 0x7fbec806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:13.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.160+0000 7fbedb7fe700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 0x7fbec806d1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbecc007fd0 tx=0x7fbecc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:13.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.164+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fbed0055d70 con 0x7fbedc0cb9a0 2023-06-09T12:04:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.369+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbedc15d7d0 con 0x7fbec806ad10 2023-06-09T12:04:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.378+0000 7fbec3fff700 1 -- 172.21.15.7:0/3760934777 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315575 (secure 0 0 0) 0x7fbedc15d7d0 con 0x7fbec806ad10 2023-06-09T12:04:13.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:04:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 msgr2=0x7fbec806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 0x7fbec806d1c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbecc007fd0 tx=0x7fbecc009040 comp rx=0 tx=0).stop 2023-06-09T12:04:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 msgr2=0x7fbedc0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fbed0000c00 tx=0x7fbed0004750 comp rx=0 tx=0).stop 2023-06-09T12:04:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 shutdown_connections 2023-06-09T12:04:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbedc15cd40 0x7fbedc151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fbec806ad10 0x7fbec806d1c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbedc0cb5d0 0x7fbedc0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 --2- 172.21.15.7:0/3760934777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbedc0cb9a0 0x7fbedc0cfd40 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 >> 172.21.15.7:0/3760934777 conn(0x7fbedc070af0 msgr2=0x7fbedc0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 shutdown_connections 2023-06-09T12:04:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:13.383+0000 7fbee1dff700 1 -- 172.21.15.7:0/3760934777 wait complete. 2023-06-09T12:04:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:04:13.404 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1673,"stamp":"2023-06-09T12:04:13.000724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1005562,"num_omap_keys":1920,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92649,"ondisk_log_size":92649,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2564296,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":680960,"kb_avail":752410424,"statfs":{"total":773094113280,"available":770468274176,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":697303040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003250"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3468,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:15.925532+0000","last_change":"2023-06-09T12:03:15.925532+0000","last_active":"2023-06-09T12:03:15.925532+0000","last_peered":"2023-06-09T12:03:15.925532+0000","last_clean":"2023-06-09T12:03:15.925532+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:03:15.925532+0000","last_undegraded":"2023-06-09T12:03:15.925532+0000","last_fullsized":"2023-06-09T12:03:15.925532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3221,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:35.073409+0000","last_change":"2023-06-09T12:02:57.307464+0000","last_active":"2023-06-09T12:02:35.073409+0000","last_peered":"2023-06-09T12:02:35.073409+0000","last_clean":"2023-06-09T12:02:35.073409+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:35.073409+0000","last_undegraded":"2023-06-09T12:02:35.073409+0000","last_fullsized":"2023-06-09T12:02:35.073409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3699,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088463+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088463+0000","last_peered":"2023-06-09T12:03:46.088463+0000","last_clean":"2023-06-09T12:03:46.088463+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088463+0000","last_undegraded":"2023-06-09T12:03:46.088463+0000","last_fullsized":"2023-06-09T12:03:46.088463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529591+0000","last_peered":"2023-06-09T12:01:55.529591+0000","last_clean":"2023-06-09T12:01:55.529591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529591+0000","last_undegraded":"2023-06-09T12:01:55.529591+0000","last_fullsized":"2023-06-09T12:01:55.529591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.180489+0000","last_change":"2023-06-09T12:04:02.343396+0000","last_active":"2023-06-09T12:03:38.180489+0000","last_peered":"2023-06-09T12:03:38.180489+0000","last_clean":"2023-06-09T12:03:38.180489+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:03:38.180489+0000","last_undegraded":"2023-06-09T12:03:38.180489+0000","last_fullsized":"2023-06-09T12:03:38.180489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.044752+0000","last_change":"2023-06-09T12:04:02.396880+0000","last_active":"2023-06-09T12:03:38.044752+0000","last_peered":"2023-06-09T12:03:38.044752+0000","last_clean":"2023-06-09T12:03:38.044752+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:03:38.044752+0000","last_undegraded":"2023-06-09T12:03:38.044752+0000","last_fullsized":"2023-06-09T12:03:38.044752+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3237,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.993038+0000","last_change":"2023-06-09T12:04:02.396860+0000","last_active":"2023-06-09T12:03:35.993038+0000","last_peered":"2023-06-09T12:03:35.993038+0000","last_clean":"2023-06-09T12:03:35.993038+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:03:35.993038+0000","last_undegraded":"2023-06-09T12:03:35.993038+0000","last_fullsized":"2023-06-09T12:03:35.993038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3344,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407306+0000","last_change":"2023-06-09T12:02:28.972507+0000","last_active":"2023-06-09T12:02:12.407306+0000","last_peered":"2023-06-09T12:02:12.407306+0000","last_clean":"2023-06-09T12:02:12.407306+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:12.407306+0000","last_undegraded":"2023-06-09T12:02:12.407306+0000","last_fullsized":"2023-06-09T12:02:12.407306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3736,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.305552+0000","last_change":"2023-06-09T12:03:55.370675+0000","last_active":"2023-06-09T12:02:12.305552+0000","last_peered":"2023-06-09T12:02:12.305552+0000","last_clean":"2023-06-09T12:02:12.305552+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:02:12.305552+0000","last_undegraded":"2023-06-09T12:02:12.305552+0000","last_fullsized":"2023-06-09T12:02:12.305552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3491,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:52.403205+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T11:59:52.403205+0000","last_peered":"2023-06-09T11:59:52.403205+0000","last_clean":"2023-06-09T11:59:52.403205+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T11:59:52.403205+0000","last_undegraded":"2023-06-09T11:59:52.403205+0000","last_fullsized":"2023-06-09T11:59:52.403205+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'997","reported_seq":3182,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:11.176840+0000","last_change":"2023-06-09T12:04:00.349461+0000","last_active":"2023-06-09T12:04:11.176840+0000","last_peered":"2023-06-09T12:04:11.176840+0000","last_clean":"2023-06-09T12:04:11.176840+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:11.176840+0000","last_undegraded":"2023-06-09T12:04:11.176840+0000","last_fullsized":"2023-06-09T12:04:11.176840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":997,"log_dups_size":0,"ondisk_log_size":997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1071","reported_seq":3711,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:09.916431+0000","last_change":"2023-06-09T12:03:41.975490+0000","last_active":"2023-06-09T12:04:09.916431+0000","last_peered":"2023-06-09T12:04:09.916431+0000","last_clean":"2023-06-09T12:04:09.916431+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:09.916431+0000","last_undegraded":"2023-06-09T12:04:09.916431+0000","last_fullsized":"2023-06-09T12:04:09.916431+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":172,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 30s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3359,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.412828+0000","last_change":"2023-06-09T12:03:38.024447+0000","last_active":"2023-06-09T12:02:57.412828+0000","last_peered":"2023-06-09T12:02:57.412828+0000","last_clean":"2023-06-09T12:02:57.412828+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:02:57.412828+0000","last_undegraded":"2023-06-09T12:02:57.412828+0000","last_fullsized":"2023-06-09T12:02:57.412828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306849+0000","last_change":"2023-06-09T12:03:46.278802+0000","last_active":"2023-06-09T12:02:52.306849+0000","last_peered":"2023-06-09T12:02:52.306849+0000","last_clean":"2023-06-09T12:02:52.306849+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:02:52.306849+0000","last_undegraded":"2023-06-09T12:02:52.306849+0000","last_fullsized":"2023-06-09T12:02:52.306849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":1999,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:37.307943+0000","last_change":"2023-06-09T12:01:37.307943+0000","last_active":"2023-06-09T12:01:37.307943+0000","last_peered":"2023-06-09T12:01:37.307943+0000","last_clean":"2023-06-09T12:01:37.307943+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:01:37.307943+0000","last_undegraded":"2023-06-09T12:01:37.307943+0000","last_fullsized":"2023-06-09T12:01:37.307943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3310,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:51.090009+0000","last_change":"2023-06-09T12:03:51.090009+0000","last_active":"2023-06-09T12:03:51.090009+0000","last_peered":"2023-06-09T12:03:51.090009+0000","last_clean":"2023-06-09T12:03:51.090009+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:51.090009+0000","last_undegraded":"2023-06-09T12:03:51.090009+0000","last_fullsized":"2023-06-09T12:03:51.090009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2964,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:36.885523+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:00:36.885523+0000","last_peered":"2023-06-09T12:00:36.885523+0000","last_clean":"2023-06-09T12:00:36.885523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:00:36.885523+0000","last_undegraded":"2023-06-09T12:00:36.885523+0000","last_fullsized":"2023-06-09T12:00:36.885523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3278,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.312444+0000","last_change":"2023-06-09T12:02:29.148543+0000","last_active":"2023-06-09T12:02:12.312444+0000","last_peered":"2023-06-09T12:02:12.312444+0000","last_clean":"2023-06-09T12:02:12.312444+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:12.312444+0000","last_undegraded":"2023-06-09T12:02:12.312444+0000","last_fullsized":"2023-06-09T12:02:12.312444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3104,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.299809+0000","last_change":"2023-06-09T12:03:13.801581+0000","last_active":"2023-06-09T12:02:12.299809+0000","last_peered":"2023-06-09T12:02:12.299809+0000","last_clean":"2023-06-09T12:02:12.299809+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:02:12.299809+0000","last_undegraded":"2023-06-09T12:02:12.299809+0000","last_fullsized":"2023-06-09T12:02:12.299809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3098,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.422668+0000","last_change":"2023-06-09T12:03:02.033567+0000","last_active":"2023-06-09T12:02:12.422668+0000","last_peered":"2023-06-09T12:02:12.422668+0000","last_clean":"2023-06-09T12:02:12.422668+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:02:12.422668+0000","last_undegraded":"2023-06-09T12:02:12.422668+0000","last_fullsized":"2023-06-09T12:02:12.422668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:12.407323+0000","last_change":"2023-06-09T12:02:49.161159+0000","last_active":"2023-06-09T12:02:12.407323+0000","last_peered":"2023-06-09T12:02:12.407323+0000","last_clean":"2023-06-09T12:02:12.407323+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:12.407323+0000","last_undegraded":"2023-06-09T12:02:12.407323+0000","last_fullsized":"2023-06-09T12:02:12.407323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T11:59:54.393683+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T11:59:54.393683+0000","last_peered":"2023-06-09T11:59:54.393683+0000","last_clean":"2023-06-09T11:59:54.393683+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T11:59:54.393683+0000","last_undegraded":"2023-06-09T11:59:54.393683+0000","last_fullsized":"2023-06-09T11:59:54.393683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:43.259734+0000","last_change":"2023-06-09T12:04:02.343332+0000","last_active":"2023-06-09T12:03:43.259734+0000","last_peered":"2023-06-09T12:03:43.259734+0000","last_clean":"2023-06-09T12:03:43.259734+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:03:43.259734+0000","last_undegraded":"2023-06-09T12:03:43.259734+0000","last_fullsized":"2023-06-09T12:03:43.259734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.914290+0000","last_change":"2023-06-09T12:04:02.319111+0000","last_active":"2023-06-09T12:01:54.914290+0000","last_peered":"2023-06-09T12:01:54.914290+0000","last_clean":"2023-06-09T12:01:54.914290+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:54.914290+0000","last_undegraded":"2023-06-09T12:01:54.914290+0000","last_fullsized":"2023-06-09T12:01:54.914290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410511+0000","last_change":"2023-06-09T12:02:51.154631+0000","last_active":"2023-06-09T12:02:37.410511+0000","last_peered":"2023-06-09T12:02:37.410511+0000","last_clean":"2023-06-09T12:02:37.410511+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:37.410511+0000","last_undegraded":"2023-06-09T12:02:37.410511+0000","last_fullsized":"2023-06-09T12:02:37.410511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410488+0000","last_change":"2023-06-09T12:03:02.187861+0000","last_active":"2023-06-09T12:02:37.410488+0000","last_peered":"2023-06-09T12:02:37.410488+0000","last_clean":"2023-06-09T12:02:37.410488+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:02:37.410488+0000","last_undegraded":"2023-06-09T12:02:37.410488+0000","last_fullsized":"2023-06-09T12:02:37.410488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3543,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:02.506728+0000","last_change":"2023-06-09T12:03:55.039481+0000","last_active":"2023-06-09T12:03:02.506728+0000","last_peered":"2023-06-09T12:03:02.506728+0000","last_clean":"2023-06-09T12:03:02.506728+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:02.506728+0000","last_undegraded":"2023-06-09T12:03:02.506728+0000","last_fullsized":"2023-06-09T12:03:02.506728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4069,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.308365+0000","last_change":"2023-06-09T12:03:53.325190+0000","last_active":"2023-06-09T12:02:37.308365+0000","last_peered":"2023-06-09T12:02:37.308365+0000","last_clean":"2023-06-09T12:02:37.308365+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:02:37.308365+0000","last_undegraded":"2023-06-09T12:02:37.308365+0000","last_fullsized":"2023-06-09T12:02:37.308365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1168","reported_seq":1359,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:07.378614+0000","last_change":"2023-06-09T12:04:07.003838+0000","last_active":"2023-06-09T12:04:07.378614+0000","last_peered":"2023-06-09T12:04:07.378614+0000","last_clean":"2023-06-09T12:04:07.378614+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:07.378614+0000","last_undegraded":"2023-06-09T12:04:07.378614+0000","last_fullsized":"2023-06-09T12:04:07.378614+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1168,"log_dups_size":0,"ondisk_log_size":1168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3841,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.426512+0000","last_change":"2023-06-09T12:03:03.011006+0000","last_active":"2023-06-09T12:02:37.426512+0000","last_peered":"2023-06-09T12:02:37.426512+0000","last_clean":"2023-06-09T12:02:37.426512+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:02:37.426512+0000","last_undegraded":"2023-06-09T12:02:37.426512+0000","last_fullsized":"2023-06-09T12:02:37.426512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.304467+0000","last_change":"2023-06-09T12:03:35.770715+0000","last_active":"2023-06-09T12:02:47.304467+0000","last_peered":"2023-06-09T12:02:47.304467+0000","last_clean":"2023-06-09T12:02:47.304467+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:02:47.304467+0000","last_undegraded":"2023-06-09T12:02:47.304467+0000","last_fullsized":"2023-06-09T12:02:47.304467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305793+0000","last_change":"2023-06-09T12:03:17.778068+0000","last_active":"2023-06-09T12:02:57.305793+0000","last_peered":"2023-06-09T12:02:57.305793+0000","last_clean":"2023-06-09T12:02:57.305793+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:02:57.305793+0000","last_undegraded":"2023-06-09T12:02:57.305793+0000","last_fullsized":"2023-06-09T12:02:57.305793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:43.105985+0000","last_change":"2023-06-09T12:02:52.306808+0000","last_active":"2023-06-09T12:02:43.105985+0000","last_peered":"2023-06-09T12:02:43.105985+0000","last_clean":"2023-06-09T12:02:43.105985+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:43.105985+0000","last_undegraded":"2023-06-09T12:02:43.105985+0000","last_fullsized":"2023-06-09T12:02:43.105985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3892,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305823+0000","last_change":"2023-06-09T12:03:46.981824+0000","last_active":"2023-06-09T12:02:57.305823+0000","last_peered":"2023-06-09T12:02:57.305823+0000","last_clean":"2023-06-09T12:02:57.305823+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:57.305823+0000","last_undegraded":"2023-06-09T12:02:57.305823+0000","last_fullsized":"2023-06-09T12:02:57.305823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.763053+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.763053+0000","last_peered":"2023-06-09T12:01:53.763053+0000","last_clean":"2023-06-09T12:01:53.763053+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.763053+0000","last_undegraded":"2023-06-09T12:01:53.763053+0000","last_fullsized":"2023-06-09T12:01:53.763053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'980","reported_seq":3957,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:10.732049+0000","last_change":"2023-06-09T12:03:48.007085+0000","last_active":"2023-06-09T12:04:10.732049+0000","last_peered":"2023-06-09T12:04:10.732049+0000","last_clean":"2023-06-09T12:04:10.732049+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:10.732049+0000","last_undegraded":"2023-06-09T12:04:10.732049+0000","last_fullsized":"2023-06-09T12:04:10.732049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":97,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3302,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.507333+0000","last_change":"2023-06-09T12:03:53.117922+0000","last_active":"2023-06-09T12:03:07.507333+0000","last_peered":"2023-06-09T12:03:07.507333+0000","last_clean":"2023-06-09T12:03:07.507333+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:03:07.507333+0000","last_undegraded":"2023-06-09T12:03:07.507333+0000","last_fullsized":"2023-06-09T12:03:07.507333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:31.034843+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:31.034843+0000","last_peered":"2023-06-09T12:02:31.034843+0000","last_clean":"2023-06-09T12:02:31.034843+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:31.034843+0000","last_undegraded":"2023-06-09T12:02:31.034843+0000","last_fullsized":"2023-06-09T12:02:31.034843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2989,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.310842+0000","last_change":"2023-06-09T12:04:01.358848+0000","last_active":"2023-06-09T12:02:57.310842+0000","last_peered":"2023-06-09T12:02:57.310842+0000","last_clean":"2023-06-09T12:02:57.310842+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:02:57.310842+0000","last_undegraded":"2023-06-09T12:02:57.310842+0000","last_fullsized":"2023-06-09T12:02:57.310842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696148+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696148+0000","last_peered":"2023-06-09T12:03:31.696148+0000","last_clean":"2023-06-09T12:03:31.696148+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696148+0000","last_undegraded":"2023-06-09T12:03:31.696148+0000","last_fullsized":"2023-06-09T12:03:31.696148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":523,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.317695+0000","last_change":"2023-06-09T12:04:02.117422+0000","last_active":"2023-06-09T12:02:52.317695+0000","last_peered":"2023-06-09T12:02:52.317695+0000","last_clean":"2023-06-09T12:02:52.317695+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:02:52.317695+0000","last_undegraded":"2023-06-09T12:02:52.317695+0000","last_fullsized":"2023-06-09T12:02:52.317695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:49.371371+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:49.371371+0000","last_peered":"2023-06-09T12:02:49.371371+0000","last_clean":"2023-06-09T12:02:49.371371+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:49.371371+0000","last_undegraded":"2023-06-09T12:02:49.371371+0000","last_fullsized":"2023-06-09T12:02:49.371371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:34.883218+0000","last_change":"2023-06-09T12:02:52.310351+0000","last_active":"2023-06-09T12:01:34.883218+0000","last_peered":"2023-06-09T12:01:34.883218+0000","last_clean":"2023-06-09T12:01:34.883218+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:01:34.883218+0000","last_undegraded":"2023-06-09T12:01:34.883218+0000","last_fullsized":"2023-06-09T12:01:34.883218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":883,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:48.982495+0000","last_change":"2023-06-09T12:03:22.314299+0000","last_active":"2023-06-09T12:01:48.982495+0000","last_peered":"2023-06-09T12:01:48.982495+0000","last_clean":"2023-06-09T12:01:48.982495+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:01:48.982495+0000","last_undegraded":"2023-06-09T12:01:48.982495+0000","last_fullsized":"2023-06-09T12:01:48.982495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":770,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.225656+0000","last_change":"2023-06-09T12:03:22.310323+0000","last_active":"2023-06-09T12:02:50.225656+0000","last_peered":"2023-06-09T12:02:50.225656+0000","last_clean":"2023-06-09T12:02:50.225656+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:02:50.225656+0000","last_undegraded":"2023-06-09T12:02:50.225656+0000","last_fullsized":"2023-06-09T12:02:50.225656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1062,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:25.446355+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:00:25.446355+0000","last_peered":"2023-06-09T12:00:25.446355+0000","last_clean":"2023-06-09T12:00:25.446355+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:00:25.446355+0000","last_undegraded":"2023-06-09T12:00:25.446355+0000","last_fullsized":"2023-06-09T12:00:25.446355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1623,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:15.282168+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:00:15.282168+0000","last_peered":"2023-06-09T12:00:15.282168+0000","last_clean":"2023-06-09T12:00:15.282168+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:00:15.282168+0000","last_undegraded":"2023-06-09T12:00:15.282168+0000","last_fullsized":"2023-06-09T12:00:15.282168+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3431,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:52.111000+0000","last_change":"2023-06-09T12:03:52.111000+0000","last_active":"2023-06-09T12:03:52.111000+0000","last_peered":"2023-06-09T12:03:52.111000+0000","last_clean":"2023-06-09T12:03:52.111000+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:52.111000+0000","last_undegraded":"2023-06-09T12:03:52.111000+0000","last_fullsized":"2023-06-09T12:03:52.111000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3939,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.318356+0000","last_change":"2023-06-09T12:03:09.981419+0000","last_active":"2023-06-09T12:02:57.318356+0000","last_peered":"2023-06-09T12:02:57.318356+0000","last_clean":"2023-06-09T12:02:57.318356+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:02:57.318356+0000","last_undegraded":"2023-06-09T12:02:57.318356+0000","last_fullsized":"2023-06-09T12:02:57.318356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2981,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.319759+0000","last_change":"2023-06-09T12:03:59.102505+0000","last_active":"2023-06-09T12:03:07.319759+0000","last_peered":"2023-06-09T12:03:07.319759+0000","last_clean":"2023-06-09T12:03:07.319759+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:07.319759+0000","last_undegraded":"2023-06-09T12:03:07.319759+0000","last_fullsized":"2023-06-09T12:03:07.319759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":61788,"ondisk_log_size":61788,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1005562,"num_omap_keys":1920,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30742,"ondisk_log_size":30742,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575493,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255248,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":75392,"kb_avail":94116592,"statfs":{"total":96636764160,"available":96375390208,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":77201408},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.66600000000000004},"min":{"1min":0.60599999999999998,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.83499999999999996,"5min":1.004,"15min":1.0640000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.90000000000000002,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.96699999999999997,"15min":1.25},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.55900000000000005,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.05,"15min":1.05},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89800000000000002,"5min":0.97599999999999998,"15min":1.099},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.53300000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.91800000000000004,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.67200000000000004},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.70899999999999996},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.1200000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71799999999999997,"15min":0.68700000000000006},"min":{"1min":0.62,"5min":0.45900000000000002,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.754,"5min":1.044,"15min":1.0920000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575493,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":203736,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":47616,"kb_avail":94168104,"statfs":{"total":96636764160,"available":96428138496,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":48758784},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.63600000000000001},"min":{"1min":0.53000000000000003,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.95899999999999996,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.64100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.53500000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96599999999999997,"15min":1.081},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.753}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70699999999999996,"15min":0.67600000000000005},"min":{"1min":0.623,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":0.98499999999999999,"15min":1.024},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.036,"15min":1.0369999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.74399999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.628},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.71199999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.68799999999999994},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.96999999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":1.0129999999999999,"5min":1.052,"15min":1.052},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.69099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":1.0029999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.66800000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575493,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401328,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94912,"kb_avail":93970512,"statfs":{"total":96636764160,"available":96225804288,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":97189888},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.36899999999999999},"max":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":1.877},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.47399999999999998},"min":{"1min":0.39600000000000002,"5min":0.318,"15min":0.28000000000000003},"max":{"1min":0.995,"5min":0.995,"15min":1.982},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":1.577},"last":0.621},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.38700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92900000000000005,"15min":1.635},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72899999999999998,"15min":0.69399999999999995},"min":{"1min":0.59899999999999998,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.609},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.774},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.78900000000000003},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.145,"5min":1.145,"15min":1.7390000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88500000000000001,"15min":0.83099999999999996},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0840000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.72599999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.27,"15min":1.794},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81000000000000005,"15min":0.76900000000000002},"min":{"1min":0.64100000000000001,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.921},"last":0.89000000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89800000000000002,"15min":0.875},"min":{"1min":0.76100000000000001,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.839},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.66200000000000003,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.827},"last":0.84199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575493,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332648,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104832,"kb_avail":94039192,"statfs":{"total":96636764160,"available":96296132608,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107347968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.442,"15min":0.443},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.58899999999999997,"5min":0.68300000000000005,"15min":0.70799999999999996},"last":0.375},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.41199999999999998,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.67800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.59099999999999997,"15min":0.54800000000000004},"min":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.79500000000000004,"5min":1.032,"15min":1.0900000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.51500000000000001,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.628}]},{"osd":3,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.57499999999999996},"min":{"1min":0.44800000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":1.2909999999999999},"last":0.436},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68600000000000005,"15min":0.70799999999999996},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.96599999999999997,"5min":0.98899999999999999,"15min":1.3280000000000001},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77200000000000002,"15min":0.70199999999999996},"min":{"1min":0.44600000000000001,"5min":0.432,"15min":0.36899999999999999},"max":{"1min":1.0669999999999999,"5min":1.175,"15min":1.175},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.82299999999999995,"15min":0.78100000000000003},"min":{"1min":0.61699999999999999,"5min":0.47699999999999998,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77100000000000002,"15min":0.746},"min":{"1min":0.52800000000000002,"5min":0.46100000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.0429999999999999,"15min":1.236},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.82199999999999995,"15min":0.76800000000000002},"min":{"1min":0.63900000000000001,"5min":0.53900000000000003,"15min":0.46600000000000003},"max":{"1min":1.097,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.93300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.51300000000000001,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83299999999999996,"15min":0.83999999999999997},"min":{"1min":0.59199999999999997,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0189999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:03:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.83299999999999996,"15min":0.78700000000000003},"min":{"1min":0.55400000000000005,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.212,"5min":1.288,"15min":1.288},"last":1.032},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.90900000000000003,"15min":0.83699999999999997},"min":{"1min":0.66300000000000003,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.1299999999999999,"5min":1.351,"15min":1.351},"last":1.085}]}]},{"osd":2,"up_from":13,"seq":55834575494,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":237856,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":58048,"kb_avail":94133984,"statfs":{"total":96636764160,"available":96393199616,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":59441152},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.39700000000000002,"5min":0.309,"15min":0.26800000000000002},"max":{"1min":0.54600000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.73699999999999999,"5min":0.999,"15min":1.0940000000000001},"last":0.53600000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.45400000000000001},"min":{"1min":0.38600000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.67100000000000004,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.441},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.54000000000000004,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.80000000000000004,"5min":0.92300000000000004,"15min":1.367},"last":0.60599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.58199999999999996},"min":{"1min":0.51000000000000001,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.82099999999999995,"5min":0.96799999999999997,"15min":1.546},"last":0.505},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.66900000000000004,"15min":0.627},"min":{"1min":0.56599999999999995,"5min":0.48699999999999999,"15min":0.42099999999999999},"max":{"1min":0.79100000000000004,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96499999999999997,"5min":1.171,"15min":1.8220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.75900000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.7370000000000001},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.82899999999999996,"15min":0.79600000000000004},"min":{"1min":0.67300000000000004,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.86899999999999999,"15min":0.83499999999999996},"min":{"1min":0.74299999999999999,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80000000000000004,"15min":0.76300000000000001},"min":{"1min":0.60499999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":1.0569999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.874},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78600000000000003,"15min":0.75900000000000001},"min":{"1min":0.54500000000000004,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.105,"15min":1.3540000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:03:26 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.82999999999999996,"15min":0.79100000000000004},"min":{"1min":0.623,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.84799999999999998,"15min":0.81899999999999995},"min":{"1min":0.70499999999999996,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.95099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575493,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421948,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":98752,"kb_avail":93949892,"statfs":{"total":96636764160,"available":96204689408,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101122048},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47299999999999998,"15min":0.46100000000000002},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.223},"max":{"1min":0.624,"5min":0.66700000000000004,"15min":1.7969999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.54300000000000004},"min":{"1min":0.49199999999999999,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.495},"min":{"1min":0.38200000000000001,"5min":0.34399999999999997,"15min":0.26900000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36499999999999999,"5min":0.33800000000000002,"15min":0.28599999999999998},"max":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":1.4970000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64600000000000002,"15min":0.64300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.81499999999999995,"5min":0.92500000000000004,"15min":1.355},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.35899999999999999,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.629}]},{"osd":4,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76900000000000002,"15min":0.72999999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.92800000000000005,"5min":1.073,"15min":1.635},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.74299999999999999},"min":{"1min":0.53300000000000003,"5min":0.52100000000000002,"15min":0.41899999999999998},"max":{"1min":0.995,"5min":3.081,"15min":3.081},"last":0.92000000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.76200000000000001},"min":{"1min":0.57099999999999995,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0229999999999999,"5min":1.198,"15min":1.2110000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.871,"5min":0.85799999999999998,"15min":0.80200000000000005},"min":{"1min":0.70699999999999996,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.0609999999999999,"5min":1.234,"15min":1.367},"last":0.94999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79800000000000004,"15min":0.78600000000000003},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.98799999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.77000000000000002},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.014,"5min":1.103,"15min":3.3490000000000002},"last":1.006}]},{"osd":7,"last update":"Fri Jun 9 12:03:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81799999999999995,"15min":0.81200000000000006},"min":{"1min":0.60599999999999998,"5min":0.51600000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.306,"15min":1.413},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.878,"15min":0.86399999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0029999999999999,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.85099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575492,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418516,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125760,"kb_avail":93953324,"statfs":{"total":96636764160,"available":96208203776,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128778240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70599999999999996,"15min":0.66700000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.77200000000000002,"15min":0.74199999999999999},"min":{"1min":0.54400000000000004,"5min":0.51200000000000001,"15min":0.43099999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.1899999999999999},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.73299999999999998,"15min":0.70699999999999996},"min":{"1min":0.47699999999999998,"5min":0.45600000000000002,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.0640000000000001},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.753},"min":{"1min":0.63400000000000001,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":1.03,"5min":1.044,"15min":1.1279999999999999},"last":0.91800000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.69799999999999995,"15min":0.65100000000000002},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.34300000000000003},"max":{"1min":1.117,"5min":1.117,"15min":1.609},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.63800000000000001,"5min":0.47999999999999998,"15min":0.38100000000000001},"max":{"1min":1.147,"5min":1.147,"15min":1.159},"last":0.97099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69699999999999995,"15min":0.67800000000000005},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.407},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.73099999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.97199999999999998,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.77800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56100000000000005,"15min":0.53100000000000003},"min":{"1min":0.44900000000000001,"5min":0.34599999999999997,"15min":0.30299999999999999},"max":{"1min":0.95499999999999996,"5min":1.0189999999999999,"15min":1.0349999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65200000000000002,"15min":0.61299999999999999},"min":{"1min":0.51100000000000001,"5min":0.41699999999999998,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":1.0620000000000001,"15min":1.1719999999999999},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58999999999999997,"15min":0.56399999999999995},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.33700000000000002},"max":{"1min":0.85199999999999998,"5min":0.86699999999999999,"15min":1.2490000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.93899999999999995,"5min":0.98499999999999999,"15min":1.3100000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:03:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.63900000000000001,"15min":0.59699999999999998},"min":{"1min":0.53000000000000003,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.91500000000000004,"5min":1.032,"15min":1.032},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.68600000000000005,"15min":0.65800000000000003},"min":{"1min":0.54100000000000004,"5min":0.47999999999999998,"15min":0.38400000000000001},"max":{"1min":0.94399999999999995,"5min":1.046,"15min":1.361},"last":0.59299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293016,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75648,"kb_avail":94078824,"statfs":{"total":96636764160,"available":96336715776,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77463552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.69999999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.32600000000000001},"max":{"1min":0.94399999999999995,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.86299999999999999,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.78500000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.91800000000000004,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.90500000000000003,"5min":19.715,"15min":19.715},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.72399999999999998},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.1870000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.81100000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.76400000000000001,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.86599999999999999,"15min":0.76400000000000001},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.92600000000000005,"5min":14.439,"15min":14.439},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.75900000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.88300000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.753},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.90000000000000002,"15min":0.81699999999999995},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.84199999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.2789999999999999,"15min":0.82399999999999995},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.64900000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.3069999999999999,"15min":0.82999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85799999999999998,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.53400000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76800000000000002,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.78000000000000003,"5min":12.163,"15min":12.163},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71599999999999997,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.97799999999999998,"15min":0.81899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":19.834,"15min":19.834},"last":0.70699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:04:13.411 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.412 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.412 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.413 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.413 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.414 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.414 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.414 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.415 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.415 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.415 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.416 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.416 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.417 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.417 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.417 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.418 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.418 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.418 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.419 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.419 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.419 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.420 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.420 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.421 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.421 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.421 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.422 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.422 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.422 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.423 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.423 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.424 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.424 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.424 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.425 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.425 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.426 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.426 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.426 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.427 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.427 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.427 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.428 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.428 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.428 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.429 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.429 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.430 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.430 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.430 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.431 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.431 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.431 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.432 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.432 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.433 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.433 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.433 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.434 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.434 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.434 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.435 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.435 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.435 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.436 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.436 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.437 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.437 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.437 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:13.438 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:04:27.002 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:27.008 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:33.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:04:33.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.645+0000 7fc1cd7ff700 1 Processor -- start 2023-06-09T12:04:33.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.646+0000 7fc1cd7ff700 1 -- start start 2023-06-09T12:04:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1cd7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1cd7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c815a420 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1cd7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c806b460 con 0x7fc1c81535e0 2023-06-09T12:04:33.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1cd7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ca670 con 0x7fc1c81539b0 2023-06-09T12:04:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1c77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc1c80c4ce0 0x7fc1c815abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38502/0 (socket says 172.21.15.7:38502) 2023-06-09T12:04:33.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1c77fe700 1 -- 172.21.15.7:0/2849112818 learned_addr learned my addr 172.21.15.7:0/2849112818 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:04:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.647+0000 7fc1bffff700 1 --2- 172.21.15.7:0/2849112818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327466788 0 0) 0x7fc1c0001490 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b40035b0 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859277526 0 0) 0x7fc1c806b460 con 0x7fc1c81535e0 2023-06-09T12:04:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c0001490 con 0x7fc1c81535e0 2023-06-09T12:04:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1bffff700 1 -- 172.21.15.7:0/2849112818 >> v1:172.21.15.117:6789/0 conn(0x7fc1c81535e0 legacy=0x7fc1c80c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1bffff700 1 -- 172.21.15.7:0/2849112818 >> v1:172.21.15.7:6789/0 conn(0x7fc1c80c4ce0 legacy=0x7fc1c815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1bffff700 1 -- 172.21.15.7:0/2849112818 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c806a570 con 0x7fc1c81539b0 2023-06-09T12:04:33.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.648+0000 7fc1bffff700 1 --2- 172.21.15.7:0/2849112818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c8159ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009870 tx=0x7fc1b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eae31c1a1d1199d server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0011830 con 0x7fc1c81539b0 2023-06-09T12:04:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b0011990 con 0x7fc1c81539b0 2023-06-09T12:04:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1c67fc700 1 -- 172.21.15.7:0/2849112818 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b0011d00 con 0x7fc1c81539b0 2023-06-09T12:04:33.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/2849112818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 msgr2=0x7fc1c8159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/2849112818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c8159ee0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009870 tx=0x7fc1b0009b80 comp rx=0 tx=0).stop 2023-06-09T12:04:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/2849112818 shutdown_connections 2023-06-09T12:04:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/2849112818 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c8159ee0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/2849112818 >> 172.21.15.7:0/2849112818 conn(0x7fc1c80737f0 msgr2=0x7fc1c8072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/2849112818 shutdown_connections 2023-06-09T12:04:33.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.649+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/2849112818 wait complete. 2023-06-09T12:04:33.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.651+0000 7fc1cd7ff700 1 Processor -- start 2023-06-09T12:04:33.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.651+0000 7fc1cd7ff700 1 -- start start 2023-06-09T12:04:33.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:33.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 0x7fc1c81f2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:33.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c81f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80c7700 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ce430 con 0x7fc1c81535e0 2023-06-09T12:04:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1cd7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c80ce710 con 0x7fc1c81539b0 2023-06-09T12:04:33.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.652+0000 7fc1c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54322/0 (socket says 172.21.15.7:54322) 2023-06-09T12:04:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 -- 172.21.15.7:0/1653662250 learned_addr learned my addr 172.21.15.7:0/1653662250 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:04:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 -- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 msgr2=0x7fc1c81f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c81f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 -- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 msgr2=0x7fc1c81f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1bffff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 0x7fc1c81f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 0x7fc1c81f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 -- 172.21.15.7:0/1653662250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b00094a0 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1bffff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 0x7fc1c81f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:04:33.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1c6ffd700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c520 tx=0x7fc1b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80052d0 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b8005430 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.653+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80038c0 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.654+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c806ab30 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.654+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c806a850 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.655+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c806ae10 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.656+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc1b8003a20 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.657+0000 7fc1bf7fe700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 0x7fc1a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:33.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.657+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fc1b8089410 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.657+0000 7fc1bffff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 0x7fc1a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:33.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.658+0000 7fc1bffff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 0x7fc1a806d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8153da0 tx=0x7fc1b0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:33.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.661+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc1b8053b40 con 0x7fc1c80c4ce0 2023-06-09T12:04:33.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.866+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1c81efe80 con 0x7fc1a806ace0 2023-06-09T12:04:33.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.876+0000 7fc1bf7fe700 1 -- 172.21.15.7:0/1653662250 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315800 (secure 0 0 0) 0x7fc1c81efe80 con 0x7fc1a806ace0 2023-06-09T12:04:33.883 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:04:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 msgr2=0x7fc1a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 0x7fc1a806d190 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8153da0 tx=0x7fc1b0008030 comp rx=0 tx=0).stop 2023-06-09T12:04:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 msgr2=0x7fc1c80c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:33.885 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c520 tx=0x7fc1b800c8e0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 shutdown_connections 2023-06-09T12:04:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc1a806ace0 0x7fc1a806d190 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc1c81539b0 0x7fc1c81f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c80c4ce0 0x7fc1c80c6d00 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.886 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 --2- 172.21.15.7:0/1653662250 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc1c81535e0 0x7fc1c81f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:33.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 >> 172.21.15.7:0/1653662250 conn(0x7fc1c80737f0 msgr2=0x7fc1c8151340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:33.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 shutdown_connections 2023-06-09T12:04:33.887 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:33.880+0000 7fc1cd7ff700 1 -- 172.21.15.7:0/1653662250 wait complete. 2023-06-09T12:04:33.887 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:04:33.901 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1683,"stamp":"2023-06-09T12:04:33.006614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1036430,"num_omap_keys":1957,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92974,"ondisk_log_size":92974,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2573064,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":689728,"kb_avail":752401656,"statfs":{"total":773094113280,"available":770459295744,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":706281472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003506"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3469,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:34.950757+0000","last_change":"2023-06-09T12:03:15.925532+0000","last_active":"2023-06-09T12:03:34.950757+0000","last_peered":"2023-06-09T12:03:34.950757+0000","last_clean":"2023-06-09T12:03:34.950757+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:03:34.950757+0000","last_undegraded":"2023-06-09T12:03:34.950757+0000","last_fullsized":"2023-06-09T12:03:34.950757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3221,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:35.073409+0000","last_change":"2023-06-09T12:02:57.307464+0000","last_active":"2023-06-09T12:02:35.073409+0000","last_peered":"2023-06-09T12:02:35.073409+0000","last_clean":"2023-06-09T12:02:35.073409+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:35.073409+0000","last_undegraded":"2023-06-09T12:02:35.073409+0000","last_fullsized":"2023-06-09T12:02:35.073409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3699,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088463+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088463+0000","last_peered":"2023-06-09T12:03:46.088463+0000","last_clean":"2023-06-09T12:03:46.088463+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088463+0000","last_undegraded":"2023-06-09T12:03:46.088463+0000","last_fullsized":"2023-06-09T12:03:46.088463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.180489+0000","last_change":"2023-06-09T12:04:02.343396+0000","last_active":"2023-06-09T12:03:38.180489+0000","last_peered":"2023-06-09T12:03:38.180489+0000","last_clean":"2023-06-09T12:03:38.180489+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:03:38.180489+0000","last_undegraded":"2023-06-09T12:03:38.180489+0000","last_fullsized":"2023-06-09T12:03:38.180489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3408,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:02.396880+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:02.396880+0000","last_peered":"2023-06-09T12:04:02.396880+0000","last_clean":"2023-06-09T12:04:02.396880+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:02.396880+0000","last_undegraded":"2023-06-09T12:04:02.396880+0000","last_fullsized":"2023-06-09T12:04:02.396880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3238,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:02.396860+0000","last_change":"2023-06-09T12:04:30.729758+0000","last_active":"2023-06-09T12:04:02.396860+0000","last_peered":"2023-06-09T12:04:02.396860+0000","last_clean":"2023-06-09T12:04:02.396860+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:04:02.396860+0000","last_undegraded":"2023-06-09T12:04:02.396860+0000","last_fullsized":"2023-06-09T12:04:02.396860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3530,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000708+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000708+0000","last_peered":"2023-06-09T12:02:34.000708+0000","last_clean":"2023-06-09T12:02:34.000708+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000708+0000","last_undegraded":"2023-06-09T12:02:34.000708+0000","last_fullsized":"2023-06-09T12:02:34.000708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'935","reported_seq":3345,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.972507+0000","last_change":"2023-06-09T12:04:32.424029+0000","last_active":"2023-06-09T12:02:28.972507+0000","last_peered":"2023-06-09T12:02:28.972507+0000","last_clean":"2023-06-09T12:02:28.972507+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:02:28.972507+0000","last_undegraded":"2023-06-09T12:02:28.972507+0000","last_fullsized":"2023-06-09T12:02:28.972507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3737,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.370675+0000","last_change":"2023-06-09T12:04:32.323061+0000","last_active":"2023-06-09T12:03:55.370675+0000","last_peered":"2023-06-09T12:03:55.370675+0000","last_clean":"2023-06-09T12:03:55.370675+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:03:55.370675+0000","last_undegraded":"2023-06-09T12:03:55.370675+0000","last_fullsized":"2023-06-09T12:03:55.370675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.422718+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T12:02:12.422718+0000","last_peered":"2023-06-09T12:02:12.422718+0000","last_clean":"2023-06-09T12:02:12.422718+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:02:12.422718+0000","last_undegraded":"2023-06-09T12:02:12.422718+0000","last_fullsized":"2023-06-09T12:02:12.422718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1118","reported_seq":3310,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.017493+0000","last_change":"2023-06-09T12:04:00.349461+0000","last_active":"2023-06-09T12:04:32.017493+0000","last_peered":"2023-06-09T12:04:32.017493+0000","last_clean":"2023-06-09T12:04:32.017493+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:32.017493+0000","last_undegraded":"2023-06-09T12:04:32.017493+0000","last_fullsized":"2023-06-09T12:04:32.017493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":194,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 32s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3483,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:51.942578+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:01:51.942578+0000","last_peered":"2023-06-09T12:01:51.942578+0000","last_clean":"2023-06-09T12:01:51.942578+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:01:51.942578+0000","last_undegraded":"2023-06-09T12:01:51.942578+0000","last_fullsized":"2023-06-09T12:01:51.942578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3359,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.412828+0000","last_change":"2023-06-09T12:03:38.024447+0000","last_active":"2023-06-09T12:02:57.412828+0000","last_peered":"2023-06-09T12:02:57.412828+0000","last_clean":"2023-06-09T12:02:57.412828+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:02:57.412828+0000","last_undegraded":"2023-06-09T12:02:57.412828+0000","last_fullsized":"2023-06-09T12:02:57.412828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924165+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924165+0000","last_peered":"2023-06-09T12:02:38.924165+0000","last_clean":"2023-06-09T12:02:38.924165+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924165+0000","last_undegraded":"2023-06-09T12:02:38.924165+0000","last_fullsized":"2023-06-09T12:02:38.924165+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306849+0000","last_change":"2023-06-09T12:03:46.278802+0000","last_active":"2023-06-09T12:02:52.306849+0000","last_peered":"2023-06-09T12:02:52.306849+0000","last_clean":"2023-06-09T12:02:52.306849+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:02:52.306849+0000","last_undegraded":"2023-06-09T12:02:52.306849+0000","last_fullsized":"2023-06-09T12:02:52.306849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1818","reported_seq":2003,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:29.518768+0000","last_change":"2023-06-09T12:04:29.144078+0000","last_active":"2023-06-09T12:04:29.518768+0000","last_peered":"2023-06-09T12:04:29.518768+0000","last_clean":"2023-06-09T12:04:29.518768+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:29.518768+0000","last_undegraded":"2023-06-09T12:04:29.518768+0000","last_fullsized":"2023-06-09T12:04:29.518768+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1818,"log_dups_size":0,"ondisk_log_size":1818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 3s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3310,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:51.090009+0000","last_change":"2023-06-09T12:03:51.090009+0000","last_active":"2023-06-09T12:03:51.090009+0000","last_peered":"2023-06-09T12:03:51.090009+0000","last_clean":"2023-06-09T12:03:51.090009+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:51.090009+0000","last_undegraded":"2023-06-09T12:03:51.090009+0000","last_fullsized":"2023-06-09T12:03:51.090009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:13.801581+0000","last_change":"2023-06-09T12:04:32.318108+0000","last_active":"2023-06-09T12:03:13.801581+0000","last_peered":"2023-06-09T12:03:13.801581+0000","last_clean":"2023-06-09T12:03:13.801581+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:03:13.801581+0000","last_undegraded":"2023-06-09T12:03:13.801581+0000","last_fullsized":"2023-06-09T12:03:13.801581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.693733+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:01:13.693733+0000","last_peered":"2023-06-09T12:01:13.693733+0000","last_clean":"2023-06-09T12:01:13.693733+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:01:13.693733+0000","last_undegraded":"2023-06-09T12:01:13.693733+0000","last_fullsized":"2023-06-09T12:01:13.693733+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.161159+0000","last_change":"2023-06-09T12:04:32.424054+0000","last_active":"2023-06-09T12:02:49.161159+0000","last_peered":"2023-06-09T12:02:49.161159+0000","last_clean":"2023-06-09T12:02:49.161159+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:49.161159+0000","last_undegraded":"2023-06-09T12:02:49.161159+0000","last_fullsized":"2023-06-09T12:02:49.161159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3560,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:41.875998+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:00:41.875998+0000","last_peered":"2023-06-09T12:00:41.875998+0000","last_clean":"2023-06-09T12:00:41.875998+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:00:41.875998+0000","last_undegraded":"2023-06-09T12:00:41.875998+0000","last_fullsized":"2023-06-09T12:00:41.875998+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:43.259734+0000","last_change":"2023-06-09T12:04:02.343332+0000","last_active":"2023-06-09T12:03:43.259734+0000","last_peered":"2023-06-09T12:03:43.259734+0000","last_clean":"2023-06-09T12:03:43.259734+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:03:43.259734+0000","last_undegraded":"2023-06-09T12:03:43.259734+0000","last_fullsized":"2023-06-09T12:03:43.259734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:17.302647+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:00:17.302647+0000","last_peered":"2023-06-09T12:00:17.302647+0000","last_clean":"2023-06-09T12:00:17.302647+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:00:17.302647+0000","last_undegraded":"2023-06-09T12:00:17.302647+0000","last_fullsized":"2023-06-09T12:00:17.302647+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.914290+0000","last_change":"2023-06-09T12:04:02.319111+0000","last_active":"2023-06-09T12:01:54.914290+0000","last_peered":"2023-06-09T12:01:54.914290+0000","last_clean":"2023-06-09T12:01:54.914290+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:54.914290+0000","last_undegraded":"2023-06-09T12:01:54.914290+0000","last_fullsized":"2023-06-09T12:01:54.914290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629439+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629439+0000","last_peered":"2023-06-09T12:02:28.629439+0000","last_clean":"2023-06-09T12:02:28.629439+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629439+0000","last_undegraded":"2023-06-09T12:02:28.629439+0000","last_fullsized":"2023-06-09T12:02:28.629439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410511+0000","last_change":"2023-06-09T12:02:51.154631+0000","last_active":"2023-06-09T12:02:37.410511+0000","last_peered":"2023-06-09T12:02:37.410511+0000","last_clean":"2023-06-09T12:02:37.410511+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:37.410511+0000","last_undegraded":"2023-06-09T12:02:37.410511+0000","last_fullsized":"2023-06-09T12:02:37.410511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.410488+0000","last_change":"2023-06-09T12:03:02.187861+0000","last_active":"2023-06-09T12:02:37.410488+0000","last_peered":"2023-06-09T12:02:37.410488+0000","last_clean":"2023-06-09T12:02:37.410488+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:02:37.410488+0000","last_undegraded":"2023-06-09T12:02:37.410488+0000","last_fullsized":"2023-06-09T12:02:37.410488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3543,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:02.506728+0000","last_change":"2023-06-09T12:03:55.039481+0000","last_active":"2023-06-09T12:03:02.506728+0000","last_peered":"2023-06-09T12:03:02.506728+0000","last_clean":"2023-06-09T12:03:02.506728+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:02.506728+0000","last_undegraded":"2023-06-09T12:03:02.506728+0000","last_fullsized":"2023-06-09T12:03:02.506728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":693,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.304423+0000","last_change":"2023-06-09T12:02:47.165212+0000","last_active":"2023-06-09T12:02:37.304423+0000","last_peered":"2023-06-09T12:02:37.304423+0000","last_clean":"2023-06-09T12:02:37.304423+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:02:37.304423+0000","last_undegraded":"2023-06-09T12:02:37.304423+0000","last_fullsized":"2023-06-09T12:02:37.304423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4069,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.308365+0000","last_change":"2023-06-09T12:03:53.325190+0000","last_active":"2023-06-09T12:02:37.308365+0000","last_peered":"2023-06-09T12:02:37.308365+0000","last_clean":"2023-06-09T12:02:37.308365+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:02:37.308365+0000","last_undegraded":"2023-06-09T12:02:37.308365+0000","last_fullsized":"2023-06-09T12:02:37.308365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3841,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:37.426512+0000","last_change":"2023-06-09T12:03:03.011006+0000","last_active":"2023-06-09T12:02:37.426512+0000","last_peered":"2023-06-09T12:02:37.426512+0000","last_clean":"2023-06-09T12:02:37.426512+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:02:37.426512+0000","last_undegraded":"2023-06-09T12:02:37.426512+0000","last_fullsized":"2023-06-09T12:02:37.426512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407587+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:00:22.407587+0000","last_peered":"2023-06-09T12:00:22.407587+0000","last_clean":"2023-06-09T12:00:22.407587+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:00:22.407587+0000","last_undegraded":"2023-06-09T12:00:22.407587+0000","last_fullsized":"2023-06-09T12:00:22.407587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'865","reported_seq":3205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:22.407612+0000","last_change":"2023-06-09T11:18:26.342970+0000","last_active":"2023-06-09T12:00:22.407612+0000","last_peered":"2023-06-09T12:00:22.407612+0000","last_clean":"2023-06-09T12:00:22.407612+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:00:22.407612+0000","last_undegraded":"2023-06-09T12:00:22.407612+0000","last_fullsized":"2023-06-09T12:00:22.407612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.304467+0000","last_change":"2023-06-09T12:03:35.770715+0000","last_active":"2023-06-09T12:02:47.304467+0000","last_peered":"2023-06-09T12:02:47.304467+0000","last_clean":"2023-06-09T12:02:47.304467+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:02:47.304467+0000","last_undegraded":"2023-06-09T12:02:47.304467+0000","last_fullsized":"2023-06-09T12:02:47.304467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305793+0000","last_change":"2023-06-09T12:03:17.778068+0000","last_active":"2023-06-09T12:02:57.305793+0000","last_peered":"2023-06-09T12:02:57.305793+0000","last_clean":"2023-06-09T12:02:57.305793+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:02:57.305793+0000","last_undegraded":"2023-06-09T12:02:57.305793+0000","last_fullsized":"2023-06-09T12:02:57.305793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":157,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:43.105985+0000","last_change":"2023-06-09T12:02:52.306808+0000","last_active":"2023-06-09T12:02:43.105985+0000","last_peered":"2023-06-09T12:02:43.105985+0000","last_clean":"2023-06-09T12:02:43.105985+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:43.105985+0000","last_undegraded":"2023-06-09T12:02:43.105985+0000","last_fullsized":"2023-06-09T12:02:43.105985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3892,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305823+0000","last_change":"2023-06-09T12:03:46.981824+0000","last_active":"2023-06-09T12:02:57.305823+0000","last_peered":"2023-06-09T12:02:57.305823+0000","last_clean":"2023-06-09T12:02:57.305823+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:57.305823+0000","last_undegraded":"2023-06-09T12:02:57.305823+0000","last_fullsized":"2023-06-09T12:02:57.305823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.763053+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.763053+0000","last_peered":"2023-06-09T12:01:53.763053+0000","last_clean":"2023-06-09T12:01:53.763053+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.763053+0000","last_undegraded":"2023-06-09T12:01:53.763053+0000","last_fullsized":"2023-06-09T12:01:53.763053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1102","reported_seq":4085,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:31.568490+0000","last_change":"2023-06-09T12:03:48.007085+0000","last_active":"2023-06-09T12:04:31.568490+0000","last_peered":"2023-06-09T12:04:31.568490+0000","last_clean":"2023-06-09T12:04:31.568490+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:31.568490+0000","last_undegraded":"2023-06-09T12:04:31.568490+0000","last_fullsized":"2023-06-09T12:04:31.568490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":219,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 43s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268856+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268856+0000","last_peered":"2023-06-09T12:02:38.268856+0000","last_clean":"2023-06-09T12:02:38.268856+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268856+0000","last_undegraded":"2023-06-09T12:02:38.268856+0000","last_fullsized":"2023-06-09T12:02:38.268856+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3302,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.507333+0000","last_change":"2023-06-09T12:03:53.117922+0000","last_active":"2023-06-09T12:03:07.507333+0000","last_peered":"2023-06-09T12:03:07.507333+0000","last_clean":"2023-06-09T12:03:07.507333+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:03:07.507333+0000","last_undegraded":"2023-06-09T12:03:07.507333+0000","last_fullsized":"2023-06-09T12:03:07.507333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:31.034843+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:31.034843+0000","last_peered":"2023-06-09T12:02:31.034843+0000","last_clean":"2023-06-09T12:02:31.034843+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:31.034843+0000","last_undegraded":"2023-06-09T12:02:31.034843+0000","last_fullsized":"2023-06-09T12:02:31.034843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2989,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.310842+0000","last_change":"2023-06-09T12:04:01.358848+0000","last_active":"2023-06-09T12:02:57.310842+0000","last_peered":"2023-06-09T12:02:57.310842+0000","last_clean":"2023-06-09T12:02:57.310842+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:02:57.310842+0000","last_undegraded":"2023-06-09T12:02:57.310842+0000","last_fullsized":"2023-06-09T12:02:57.310842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696148+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696148+0000","last_peered":"2023-06-09T12:03:31.696148+0000","last_clean":"2023-06-09T12:03:31.696148+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696148+0000","last_undegraded":"2023-06-09T12:03:31.696148+0000","last_fullsized":"2023-06-09T12:03:31.696148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":523,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.317695+0000","last_change":"2023-06-09T12:04:02.117422+0000","last_active":"2023-06-09T12:02:52.317695+0000","last_peered":"2023-06-09T12:02:52.317695+0000","last_clean":"2023-06-09T12:02:52.317695+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:02:52.317695+0000","last_undegraded":"2023-06-09T12:02:52.317695+0000","last_fullsized":"2023-06-09T12:02:52.317695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:49.371371+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:49.371371+0000","last_peered":"2023-06-09T12:02:49.371371+0000","last_clean":"2023-06-09T12:02:49.371371+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:49.371371+0000","last_undegraded":"2023-06-09T12:02:49.371371+0000","last_fullsized":"2023-06-09T12:02:49.371371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":830,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.310351+0000","last_change":"2023-06-09T12:04:28.374029+0000","last_active":"2023-06-09T12:02:52.310351+0000","last_peered":"2023-06-09T12:02:52.310351+0000","last_clean":"2023-06-09T12:02:52.310351+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:02:52.310351+0000","last_undegraded":"2023-06-09T12:02:52.310351+0000","last_fullsized":"2023-06-09T12:02:52.310351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":883,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:48.982495+0000","last_change":"2023-06-09T12:03:22.314299+0000","last_active":"2023-06-09T12:01:48.982495+0000","last_peered":"2023-06-09T12:01:48.982495+0000","last_clean":"2023-06-09T12:01:48.982495+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:01:48.982495+0000","last_undegraded":"2023-06-09T12:01:48.982495+0000","last_fullsized":"2023-06-09T12:01:48.982495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":770,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.225656+0000","last_change":"2023-06-09T12:03:22.310323+0000","last_active":"2023-06-09T12:02:50.225656+0000","last_peered":"2023-06-09T12:02:50.225656+0000","last_clean":"2023-06-09T12:02:50.225656+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:02:50.225656+0000","last_undegraded":"2023-06-09T12:02:50.225656+0000","last_fullsized":"2023-06-09T12:02:50.225656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3431,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:52.111000+0000","last_change":"2023-06-09T12:03:52.111000+0000","last_active":"2023-06-09T12:03:52.111000+0000","last_peered":"2023-06-09T12:03:52.111000+0000","last_clean":"2023-06-09T12:03:52.111000+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:52.111000+0000","last_undegraded":"2023-06-09T12:03:52.111000+0000","last_fullsized":"2023-06-09T12:03:52.111000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3939,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.318356+0000","last_change":"2023-06-09T12:03:09.981419+0000","last_active":"2023-06-09T12:02:57.318356+0000","last_peered":"2023-06-09T12:02:57.318356+0000","last_clean":"2023-06-09T12:02:57.318356+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:02:57.318356+0000","last_undegraded":"2023-06-09T12:02:57.318356+0000","last_fullsized":"2023-06-09T12:02:57.318356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2981,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.319759+0000","last_change":"2023-06-09T12:03:59.102505+0000","last_active":"2023-06-09T12:03:07.319759+0000","last_peered":"2023-06-09T12:03:07.319759+0000","last_clean":"2023-06-09T12:03:07.319759+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:07.319759+0000","last_undegraded":"2023-06-09T12:03:07.319759+0000","last_fullsized":"2023-06-09T12:03:07.319759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":62099,"ondisk_log_size":62099,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1036430,"num_omap_keys":1957,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30756,"ondisk_log_size":30756,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575497,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255312,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":75456,"kb_avail":94116528,"statfs":{"total":96636764160,"available":96375324672,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":77266944},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71599999999999997,"15min":0.66600000000000004},"min":{"1min":0.60599999999999998,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.83499999999999996,"5min":1.004,"15min":1.0640000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70899999999999996,"15min":0.68300000000000005},"min":{"1min":0.51400000000000001,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.90000000000000002,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.66900000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":0.81599999999999995,"5min":0.96699999999999997,"15min":1.25},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.72399999999999998,"15min":0.69099999999999995},"min":{"1min":0.55900000000000005,"5min":0.438,"15min":0.32900000000000001},"max":{"1min":0.90000000000000002,"5min":1.05,"15min":1.05},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.56100000000000005,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.89800000000000002,"5min":0.97599999999999998,"15min":1.099},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.73899999999999999,"15min":0.72299999999999998},"min":{"1min":0.53300000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.91800000000000004,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:03:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69499999999999995,"15min":0.67200000000000004},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.379},"max":{"1min":0.83899999999999997,"5min":0.90700000000000003,"15min":1.4450000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.70899999999999996},"min":{"1min":0.59699999999999998,"5min":0.55100000000000005,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.52900000000000003},"min":{"1min":0.38700000000000001,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":1.1200000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.71799999999999997,"15min":0.68700000000000006},"min":{"1min":0.62,"5min":0.45900000000000002,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.49199999999999999,"5min":0.433,"15min":0.34999999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.44500000000000001,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0269999999999999},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:03:30 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.64800000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":0.754,"5min":1.044,"15min":1.0920000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75,"15min":0.72499999999999998},"min":{"1min":0.57899999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":0.89600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575497,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":207896,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":94163944,"statfs":{"total":96636764160,"available":96423878656,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.66900000000000004,"15min":0.63600000000000001},"min":{"1min":0.53000000000000003,"5min":0.46100000000000002,"15min":0.28299999999999997},"max":{"1min":0.95899999999999996,"5min":1.0129999999999999,"15min":1.0780000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69499999999999995,"15min":0.64100000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.86099999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70399999999999996,"15min":0.66500000000000004},"min":{"1min":0.53100000000000003,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.89900000000000002,"5min":0.96599999999999997,"15min":1.081},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.75600000000000001,"15min":0.70699999999999996},"min":{"1min":0.60499999999999998,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":1.004,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.85199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.70699999999999996,"15min":0.67600000000000005},"min":{"1min":0.623,"5min":0.438,"15min":0.36799999999999999},"max":{"1min":0.92800000000000005,"5min":0.98499999999999999,"15min":1.024},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.76200000000000001,"15min":0.72599999999999998},"min":{"1min":0.59399999999999997,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.98299999999999998,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.746,"15min":0.69599999999999995},"min":{"1min":0.499,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":1.03,"5min":1.036,"15min":1.0369999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.80100000000000005,"15min":0.74399999999999999},"min":{"1min":0.65100000000000002,"5min":0.53500000000000003,"15min":0.39500000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.83499999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58299999999999996,"15min":0.55800000000000005},"min":{"1min":0.38300000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71199999999999997,"15min":0.68799999999999994},"min":{"1min":0.51800000000000002,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.96999999999999997,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.437,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":1.0129999999999999,"5min":1.052,"15min":1.052},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.69099999999999995},"min":{"1min":0.47699999999999998,"5min":0.46999999999999997,"15min":0.32800000000000001},"max":{"1min":1.0029999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.66300000000000003,"15min":0.63900000000000001},"min":{"1min":0.46000000000000002,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":1.071},"last":0.48299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575497,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401456,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":95040,"kb_avail":93970384,"statfs":{"total":96636764160,"available":96225673216,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":97320960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.59199999999999997,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57999999999999996,"15min":0.56000000000000005},"min":{"1min":0.47899999999999998,"5min":0.40699999999999997,"15min":0.36899999999999999},"max":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":1.877},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.47399999999999998},"min":{"1min":0.39600000000000002,"5min":0.318,"15min":0.28000000000000003},"max":{"1min":0.995,"5min":0.995,"15min":1.982},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56200000000000006,"15min":0.54000000000000004},"min":{"1min":0.41099999999999998,"5min":0.35699999999999998,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.57399999999999995,"15min":0.56100000000000005},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.64300000000000002,"5min":0.79800000000000004,"15min":1.577},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.66200000000000003,"15min":0.65100000000000002},"min":{"1min":0.52100000000000002,"5min":0.44,"15min":0.38700000000000001},"max":{"1min":0.68100000000000005,"5min":0.92900000000000005,"15min":1.635},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.72899999999999998,"15min":0.69399999999999995},"min":{"1min":0.59899999999999998,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.609},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.876,"5min":0.84699999999999998,"15min":0.81000000000000005},"min":{"1min":0.69799999999999995,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.774},"last":0.79200000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.82899999999999996,"15min":0.78900000000000003},"min":{"1min":0.68999999999999995,"5min":0.56399999999999995,"15min":0.54500000000000004},"max":{"1min":1.145,"5min":1.145,"15min":1.7390000000000001},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88500000000000001,"15min":0.83099999999999996},"min":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.53500000000000003},"max":{"1min":1.0840000000000001,"5min":1.1419999999999999,"15min":1.6879999999999999},"last":0.83599999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.59999999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":1.0940000000000001,"5min":1.27,"15min":1.794},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.81000000000000005,"15min":0.76900000000000002},"min":{"1min":0.64100000000000001,"5min":0.49399999999999999,"15min":0.42299999999999999},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.921},"last":0.81100000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:03:35 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.89800000000000002,"15min":0.875},"min":{"1min":0.76100000000000001,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.839},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.80500000000000005,"15min":0.77900000000000003},"min":{"1min":0.66200000000000003,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.252,"5min":1.252,"15min":1.827},"last":0.77300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575497,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332648,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104832,"kb_avail":94039192,"statfs":{"total":96636764160,"available":96296132608,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107347968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.442,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.70799999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.46400000000000002,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.629,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.032,"15min":1.0900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.499},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.625}]},{"osd":3,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.97399999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.024},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.71099999999999997},"min":{"1min":0.629,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.0569999999999999,"5min":1.175,"15min":1.175},"last":0.749},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.94399999999999995,"5min":1.034,"15min":1.236},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83799999999999997,"15min":0.77200000000000002},"min":{"1min":0.68700000000000006,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":0.98799999999999999,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0149999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.84599999999999997},"min":{"1min":0.81299999999999994,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.0940000000000001,"5min":1.351,"15min":1.351},"last":0.95399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575498,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242016,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62208,"kb_avail":94129824,"statfs":{"total":96636764160,"available":96388939776,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":63700992},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.39700000000000002,"5min":0.309,"15min":0.26800000000000002},"max":{"1min":0.54600000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53600000000000003,"15min":0.50900000000000001},"min":{"1min":0.48099999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.73699999999999999,"5min":0.999,"15min":1.0940000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60999999999999999,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89200000000000002,"5min":0.92300000000000004,"15min":1.367},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.171,"15min":1.8220000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":1.214,"15min":1.7370000000000001},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.79900000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":0.96799999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.83899999999999997},"min":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.105,"15min":1.3540000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.0029999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83799999999999997,"15min":0.82299999999999995},"min":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.86899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575497,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421948,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":98752,"kb_avail":93949892,"statfs":{"total":96636764160,"available":96204689408,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101122048},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.26900000000000002},"max":{"1min":0.70199999999999996,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.28599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.64300000000000002},"min":{"1min":0.50600000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.355},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.85899999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":1.006,"5min":1.073,"15min":1.635},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0980000000000001,"5min":3.081,"15min":3.081},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.68700000000000006,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0920000000000001,"5min":1.198,"15min":1.2110000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88100000000000001,"15min":0.81000000000000005},"min":{"1min":0.73699999999999999,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.234,"15min":1.367},"last":0.96999999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.65900000000000003,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0229999999999999,"5min":1.103,"15min":3.3490000000000002},"last":1.028}]},{"osd":7,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.0700000000000001,"5min":1.306,"15min":1.413},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.88900000000000001,"15min":0.86199999999999999},"min":{"1min":0.64200000000000002,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.137,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.93600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575496,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418644,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125888,"kb_avail":93953196,"statfs":{"total":96636764160,"available":96208072704,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128909312},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72799999999999998,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0600000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.0640000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.755},"min":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.044,"15min":1.1279999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":1.117,"15min":1.609},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.68100000000000005},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.38100000000000001},"max":{"1min":1.0149999999999999,"5min":1.147,"15min":1.159},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":0.999,"5min":1.014,"15min":1.407},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.86199999999999999,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.63}]},{"osd":5,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.47999999999999998,"5min":0.39300000000000002,"15min":0.30299999999999999},"max":{"1min":0.78900000000000003,"5min":0.95999999999999996,"15min":1.0349999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.47199999999999998,"15min":0.34100000000000003},"max":{"1min":0.98799999999999999,"5min":1.014,"15min":1.1719999999999999},"last":0.53000000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.69999999999999996},"min":{"1min":0.58699999999999997,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.3100000000000001},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60199999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":0.94199999999999995,"15min":1.032},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.361},"last":0.56499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293144,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75776,"kb_avail":94078696,"statfs":{"total":96636764160,"available":96336584704,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77594624},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.69999999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.32600000000000001},"max":{"1min":0.94399999999999995,"5min":5.4969999999999999,"15min":5.4969999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.71199999999999997},"min":{"1min":0.52500000000000002,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.86299999999999999,"5min":5.6269999999999998,"15min":5.6269999999999998},"last":0.71099999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.91800000000000004,"15min":0.72999999999999998},"min":{"1min":0.47599999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":0.90500000000000003,"5min":19.715,"15min":19.715},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.82799999999999996,"15min":0.72399999999999998},"min":{"1min":0.52600000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.1870000000000001,"5min":7.4089999999999998,"15min":7.4089999999999998},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.76400000000000001,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.82499999999999996,"5min":5.5620000000000003,"15min":5.5620000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.86599999999999999,"15min":0.76400000000000001},"min":{"1min":0.53800000000000003,"5min":0.46100000000000002,"15min":0.39200000000000002},"max":{"1min":0.92600000000000005,"5min":14.439,"15min":14.439},"last":0.56899999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.86799999999999999,"15min":0.75900000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.88300000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.90000000000000002,"15min":0.81699999999999995},"min":{"1min":0.63100000000000001,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.2330000000000001,"5min":12.218999999999999,"15min":12.218999999999999},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":1.2789999999999999,"15min":0.82399999999999995},"min":{"1min":0.40899999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.64900000000000002,"5min":67.590000000000003,"15min":67.590000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":1.3069999999999999,"15min":0.82999999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85799999999999998,"5min":67.521000000000001,"15min":67.521000000000001},"last":0.54800000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.76800000000000002,"15min":0.68100000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.78000000000000003,"5min":12.163,"15min":12.163},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71599999999999997,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.84099999999999997,"5min":7.6040000000000001,"15min":7.6040000000000001},"last":0.505}]},{"osd":7,"last update":"Fri Jun 9 12:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.69899999999999995,"15min":0.68400000000000005},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":3.8039999999999998,"15min":3.8039999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.97799999999999998,"15min":0.81899999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.135,"5min":19.834,"15min":19.834},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:04:33.908 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.909 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.909 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.910 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.910 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.910 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.911 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.911 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.911 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.912 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.912 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.912 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.913 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.913 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.913 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.914 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.914 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.914 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.915 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.915 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.916 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.916 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.916 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.917 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.917 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.917 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.918 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.918 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.918 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.919 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.919 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.919 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.920 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.920 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.920 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.921 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.921 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.921 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.922 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.922 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.922 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.923 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.923 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.923 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.924 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.924 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.925 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.925 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.925 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.926 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.926 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.926 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.927 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.927 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.927 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.928 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.928 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.928 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.929 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.929 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.929 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.930 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.930 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.931 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.931 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.931 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.932 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.932 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:33.932 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:04:53.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:04:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.141+0000 7f0ee6fff700 1 Processor -- start 2023-06-09T12:04:54.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.142+0000 7f0ee6fff700 1 -- start start 2023-06-09T12:04:54.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 0x7f0ee01599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f0ee00c0140 0x7f0ee0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ee00c1f90 0x7f0ee01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0070110 con 0x7f0ee00c1f90 2023-06-09T12:04:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0071300 con 0x7f0ee00c0140 2023-06-09T12:04:54.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee6fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00cbd20 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ee00c1f90 0x7f0ee01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee559c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f0ee00c1f90 0x7f0ee01501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40408/0 (socket says 172.21.15.7:40408) 2023-06-09T12:04:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee559c700 1 -- 172.21.15.7:0/3927992339 learned_addr learned my addr 172.21.15.7:0/3927992339 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:04:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0edffff700 1 --2- 172.21.15.7:0/3927992339 >> v2:172.21.15.117:3300/0 conn(0x7f0ee00c0140 0x7f0ee0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.147 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.143+0000 7f0ee4d9b700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 0x7f0ee01599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 -- 172.21.15.7:0/3927992339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 msgr2=0x7f0ee01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:04:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 0x7f0ee01599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 -- 172.21.15.7:0/3927992339 >> v2:172.21.15.117:3300/0 conn(0x7f0ee00c0140 msgr2=0x7f0ee0159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:04:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 --2- 172.21.15.7:0/3927992339 >> v2:172.21.15.117:3300/0 conn(0x7f0ee00c0140 0x7f0ee0159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.148 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 -- 172.21.15.7:0/3927992339 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee00c0bf0 con 0x7f0ee00c1f90 2023-06-09T12:04:54.149 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee4d9b700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 0x7f0ee01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:04:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0ee559c700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1f90 0x7f0ee01501b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f0ee006f740 tx=0x7f0ed800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17120089617add90 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0edf7fe700 1 -- 172.21.15.7:0/3927992339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800e070 con 0x7f0ee00c1f90 2023-06-09T12:04:54.151 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0edf7fe700 1 -- 172.21.15.7:0/3927992339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800b980 con 0x7f0ee00c1f90 2023-06-09T12:04:54.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.144+0000 7f0edf7fe700 1 -- 172.21.15.7:0/3927992339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed800bd30 con 0x7f0ee00c1f90 2023-06-09T12:04:54.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3927992339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1f90 msgr2=0x7f0ee01501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:54.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1f90 0x7f0ee01501b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f0ee006f740 tx=0x7f0ed800b330 comp rx=0 tx=0).stop 2023-06-09T12:04:54.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3927992339 shutdown_connections 2023-06-09T12:04:54.152 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1f90 0x7f0ee01501b0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3927992339 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c1bc0 0x7f0ee01599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3927992339 >> v2:172.21.15.117:3300/0 conn(0x7f0ee00c0140 0x7f0ee0159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3927992339 >> 172.21.15.7:0/3927992339 conn(0x7f0ee006a4d0 msgr2=0x7f0ee006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3927992339 shutdown_connections 2023-06-09T12:04:54.153 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.145+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3927992339 wait complete. 2023-06-09T12:04:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.146+0000 7f0ee6fff700 1 Processor -- start 2023-06-09T12:04:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 -- start start 2023-06-09T12:04:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c0140 0x7f0ee00c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.154 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee00c1f90 0x7f0ee00c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0151f20 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee00c2e10 con 0x7f0ee00c1f90 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.147+0000 7f0ee6fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0152150 con 0x7f0ee00c0140 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40420/0 (socket says 172.21.15.7:40420) 2023-06-09T12:04:54.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 -- 172.21.15.7:0/3340705766 learned_addr learned my addr 172.21.15.7:0/3340705766 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:04:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0ee4d9b700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c0140 0x7f0ee00c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0ee559c700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee00c1f90 0x7f0ee00c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 -- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c0140 msgr2=0x7f0ee00c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c0140 0x7f0ee00c4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 -- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee00c1f90 msgr2=0x7f0ee00c37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee00c1f90 0x7f0ee00c37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 -- 172.21.15.7:0/3340705766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8009b30 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.148+0000 7f0edffff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e810 tx=0x7f0ecc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.149+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00c070 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.157 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.149+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc013610 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.149+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc0139a0 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.149+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee00c0e90 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.149+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee00702f0 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.150+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee00c1170 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.151+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0ecc013b00 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.152+0000 7f0edd7fa700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 0x7f0ec406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:04:54.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.152+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f0ecc089990 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.152+0000 7f0ee4d9b700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 0x7f0ec406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:04:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.153+0000 7f0ee4d9b700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 0x7f0ec406d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40099d0 tx=0x7f0ed4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:04:54.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.156+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f0ecc0541b0 con 0x7f0ee00c1bc0 2023-06-09T12:04:54.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.357+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ee0074cf0 con 0x7f0ec406ace0 2023-06-09T12:04:54.368 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.366+0000 7f0edd7fa700 1 -- 172.21.15.7:0/3340705766 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315712 (secure 0 0 0) 0x7f0ee0074cf0 con 0x7f0ec406ace0 2023-06-09T12:04:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:04:54.373 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.370+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 msgr2=0x7f0ec406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.370+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 0x7f0ec406d190 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0ed40099d0 tx=0x7f0ed4008040 comp rx=0 tx=0).stop 2023-06-09T12:04:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.370+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 msgr2=0x7f0ee00c32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:04:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.370+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00e810 tx=0x7f0ecc00eb20 comp rx=0 tx=0).stop 2023-06-09T12:04:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 shutdown_connections 2023-06-09T12:04:54.374 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0ee00c1f90 0x7f0ee00c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f0ec406ace0 0x7f0ec406d190 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0ee00c0140 0x7f0ee00c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 --2- 172.21.15.7:0/3340705766 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ee00c1bc0 0x7f0ee00c32a0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:04:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 >> 172.21.15.7:0/3340705766 conn(0x7f0ee006a4d0 msgr2=0x7f0ee014daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:04:54.375 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 shutdown_connections 2023-06-09T12:04:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:04:54.371+0000 7f0ee6fff700 1 -- 172.21.15.7:0/3340705766 wait complete. 2023-06-09T12:04:54.376 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:04:54.391 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1693,"stamp":"2023-06-09T12:04:53.012134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93249,"ondisk_log_size":93249,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2577736,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":694400,"kb_avail":752396984,"statfs":{"total":773094113280,"available":770454511616,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":711065600},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003432"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3471,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.710286+0000","last_change":"2023-06-09T12:03:15.925532+0000","last_active":"2023-06-09T12:04:32.710286+0000","last_peered":"2023-06-09T12:04:32.710286+0000","last_clean":"2023-06-09T12:04:32.710286+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:04:32.710286+0000","last_undegraded":"2023-06-09T12:04:32.710286+0000","last_fullsized":"2023-06-09T12:04:32.710286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3222,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.307464+0000","last_change":"2023-06-09T12:04:42.076031+0000","last_active":"2023-06-09T12:02:57.307464+0000","last_peered":"2023-06-09T12:02:57.307464+0000","last_clean":"2023-06-09T12:02:57.307464+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:02:57.307464+0000","last_undegraded":"2023-06-09T12:02:57.307464+0000","last_fullsized":"2023-06-09T12:02:57.307464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3258,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429163+0000","last_change":"2023-06-09T12:03:07.029687+0000","last_active":"2023-06-09T12:02:57.429163+0000","last_peered":"2023-06-09T12:02:57.429163+0000","last_clean":"2023-06-09T12:02:57.429163+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:02:57.429163+0000","last_undegraded":"2023-06-09T12:02:57.429163+0000","last_fullsized":"2023-06-09T12:02:57.429163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3699,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088463+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088463+0000","last_peered":"2023-06-09T12:03:46.088463+0000","last_clean":"2023-06-09T12:03:46.088463+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088463+0000","last_undegraded":"2023-06-09T12:03:46.088463+0000","last_fullsized":"2023-06-09T12:03:46.088463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'862","reported_seq":3695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.180489+0000","last_change":"2023-06-09T12:04:02.343396+0000","last_active":"2023-06-09T12:03:38.180489+0000","last_peered":"2023-06-09T12:03:38.180489+0000","last_clean":"2023-06-09T12:03:38.180489+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:03:38.180489+0000","last_undegraded":"2023-06-09T12:03:38.180489+0000","last_fullsized":"2023-06-09T12:03:38.180489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3409,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:31.748046+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:31.748046+0000","last_peered":"2023-06-09T12:04:31.748046+0000","last_clean":"2023-06-09T12:04:31.748046+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:31.748046+0000","last_undegraded":"2023-06-09T12:04:31.748046+0000","last_fullsized":"2023-06-09T12:04:31.748046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'862","reported_seq":3243,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:51.030864+0000","last_change":"2023-06-09T12:04:30.729758+0000","last_active":"2023-06-09T12:04:51.030864+0000","last_peered":"2023-06-09T12:04:51.030864+0000","last_clean":"2023-06-09T12:04:51.030864+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:04:51.030864+0000","last_undegraded":"2023-06-09T12:04:51.030864+0000","last_fullsized":"2023-06-09T12:04:51.030864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1009","reported_seq":3426,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:49.557308+0000","last_change":"2023-06-09T12:04:38.738579+0000","last_active":"2023-06-09T12:04:49.557308+0000","last_peered":"2023-06-09T12:04:49.557308+0000","last_clean":"2023-06-09T12:04:49.557308+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:04:49.557308+0000","last_undegraded":"2023-06-09T12:04:49.557308+0000","last_fullsized":"2023-06-09T12:04:49.557308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":74,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490734+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490734+0000","last_peered":"2023-06-09T12:02:52.490734+0000","last_clean":"2023-06-09T12:02:52.490734+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490734+0000","last_undegraded":"2023-06-09T12:02:52.490734+0000","last_fullsized":"2023-06-09T12:02:52.490734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3737,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.370675+0000","last_change":"2023-06-09T12:04:32.323061+0000","last_active":"2023-06-09T12:03:55.370675+0000","last_peered":"2023-06-09T12:03:55.370675+0000","last_clean":"2023-06-09T12:03:55.370675+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:03:55.370675+0000","last_undegraded":"2023-06-09T12:03:55.370675+0000","last_fullsized":"2023-06-09T12:03:55.370675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.422718+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T12:02:12.422718+0000","last_peered":"2023-06-09T12:02:12.422718+0000","last_clean":"2023-06-09T12:02:12.422718+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:02:12.422718+0000","last_undegraded":"2023-06-09T12:02:12.422718+0000","last_fullsized":"2023-06-09T12:02:12.422718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.308346+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:02:37.308346+0000","last_peered":"2023-06-09T12:02:37.308346+0000","last_clean":"2023-06-09T12:02:37.308346+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:02:37.308346+0000","last_undegraded":"2023-06-09T12:02:37.308346+0000","last_fullsized":"2023-06-09T12:02:37.308346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3359,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.412828+0000","last_change":"2023-06-09T12:03:38.024447+0000","last_active":"2023-06-09T12:02:57.412828+0000","last_peered":"2023-06-09T12:02:57.412828+0000","last_clean":"2023-06-09T12:02:57.412828+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:02:57.412828+0000","last_undegraded":"2023-06-09T12:02:57.412828+0000","last_fullsized":"2023-06-09T12:02:57.412828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306849+0000","last_change":"2023-06-09T12:03:46.278802+0000","last_active":"2023-06-09T12:02:52.306849+0000","last_peered":"2023-06-09T12:02:52.306849+0000","last_clean":"2023-06-09T12:02:52.306849+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:02:52.306849+0000","last_undegraded":"2023-06-09T12:02:52.306849+0000","last_fullsized":"2023-06-09T12:02:52.306849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:52.964134+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:01:52.964134+0000","last_peered":"2023-06-09T12:01:52.964134+0000","last_clean":"2023-06-09T12:01:52.964134+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:01:52.964134+0000","last_undegraded":"2023-06-09T12:01:52.964134+0000","last_fullsized":"2023-06-09T12:01:52.964134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3310,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:51.090009+0000","last_change":"2023-06-09T12:03:51.090009+0000","last_active":"2023-06-09T12:03:51.090009+0000","last_peered":"2023-06-09T12:03:51.090009+0000","last_clean":"2023-06-09T12:03:51.090009+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:51.090009+0000","last_undegraded":"2023-06-09T12:03:51.090009+0000","last_fullsized":"2023-06-09T12:03:51.090009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.065982+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.065982+0000","last_peered":"2023-06-09T12:02:49.065982+0000","last_clean":"2023-06-09T12:02:49.065982+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.065982+0000","last_undegraded":"2023-06-09T12:02:49.065982+0000","last_fullsized":"2023-06-09T12:02:49.065982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:13.801581+0000","last_change":"2023-06-09T12:04:32.318108+0000","last_active":"2023-06-09T12:03:13.801581+0000","last_peered":"2023-06-09T12:03:13.801581+0000","last_clean":"2023-06-09T12:03:13.801581+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:03:13.801581+0000","last_undegraded":"2023-06-09T12:03:13.801581+0000","last_fullsized":"2023-06-09T12:03:13.801581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.161159+0000","last_change":"2023-06-09T12:04:32.424054+0000","last_active":"2023-06-09T12:02:49.161159+0000","last_peered":"2023-06-09T12:02:49.161159+0000","last_clean":"2023-06-09T12:02:49.161159+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:49.161159+0000","last_undegraded":"2023-06-09T12:02:49.161159+0000","last_fullsized":"2023-06-09T12:02:49.161159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3561,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313831+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:02:22.313831+0000","last_peered":"2023-06-09T12:02:22.313831+0000","last_clean":"2023-06-09T12:02:22.313831+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:02:22.313831+0000","last_undegraded":"2023-06-09T12:02:22.313831+0000","last_fullsized":"2023-06-09T12:02:22.313831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:43.259734+0000","last_change":"2023-06-09T12:04:02.343332+0000","last_active":"2023-06-09T12:03:43.259734+0000","last_peered":"2023-06-09T12:03:43.259734+0000","last_clean":"2023-06-09T12:03:43.259734+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:03:43.259734+0000","last_undegraded":"2023-06-09T12:03:43.259734+0000","last_fullsized":"2023-06-09T12:03:43.259734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.914290+0000","last_change":"2023-06-09T12:04:02.319111+0000","last_active":"2023-06-09T12:01:54.914290+0000","last_peered":"2023-06-09T12:01:54.914290+0000","last_clean":"2023-06-09T12:01:54.914290+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:54.914290+0000","last_undegraded":"2023-06-09T12:01:54.914290+0000","last_fullsized":"2023-06-09T12:01:54.914290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":730,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:42.411173+0000","last_change":"2023-06-09T12:02:50.198865+0000","last_active":"2023-06-09T12:02:42.411173+0000","last_peered":"2023-06-09T12:02:42.411173+0000","last_clean":"2023-06-09T12:02:42.411173+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:42.411173+0000","last_undegraded":"2023-06-09T12:02:42.411173+0000","last_fullsized":"2023-06-09T12:02:42.411173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.410140+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:00:42.410140+0000","last_peered":"2023-06-09T12:00:42.410140+0000","last_clean":"2023-06-09T12:00:42.410140+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:00:42.410140+0000","last_undegraded":"2023-06-09T12:00:42.410140+0000","last_fullsized":"2023-06-09T12:00:42.410140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3543,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:02.506728+0000","last_change":"2023-06-09T12:03:55.039481+0000","last_active":"2023-06-09T12:03:02.506728+0000","last_peered":"2023-06-09T12:03:02.506728+0000","last_clean":"2023-06-09T12:03:02.506728+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:02.506728+0000","last_undegraded":"2023-06-09T12:03:02.506728+0000","last_fullsized":"2023-06-09T12:03:02.506728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:51.058132+0000","last_change":"2023-06-09T12:04:52.349590+0000","last_active":"2023-06-09T12:04:51.058132+0000","last_peered":"2023-06-09T12:04:51.058132+0000","last_clean":"2023-06-09T12:04:51.058132+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:04:51.058132+0000","last_undegraded":"2023-06-09T12:04:51.058132+0000","last_fullsized":"2023-06-09T12:04:51.058132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:53.325190+0000","last_change":"2023-06-09T12:04:52.325633+0000","last_active":"2023-06-09T12:03:53.325190+0000","last_peered":"2023-06-09T12:03:53.325190+0000","last_clean":"2023-06-09T12:03:53.325190+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:03:53.325190+0000","last_undegraded":"2023-06-09T12:03:53.325190+0000","last_fullsized":"2023-06-09T12:03:53.325190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.426543+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:02:37.426543+0000","last_peered":"2023-06-09T12:02:37.426543+0000","last_clean":"2023-06-09T12:02:37.426543+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:02:37.426543+0000","last_undegraded":"2023-06-09T12:02:37.426543+0000","last_fullsized":"2023-06-09T12:02:37.426543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:14.658585+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:01:14.658585+0000","last_peered":"2023-06-09T12:01:14.658585+0000","last_clean":"2023-06-09T12:01:14.658585+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:01:14.658585+0000","last_undegraded":"2023-06-09T12:01:14.658585+0000","last_fullsized":"2023-06-09T12:01:14.658585+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'937","reported_seq":3284,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:50.315825+0000","last_change":"2023-06-09T12:04:42.478870+0000","last_active":"2023-06-09T12:04:50.315825+0000","last_peered":"2023-06-09T12:04:50.315825+0000","last_clean":"2023-06-09T12:04:50.315825+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:04:50.315825+0000","last_undegraded":"2023-06-09T12:04:50.315825+0000","last_fullsized":"2023-06-09T12:04:50.315825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":72,"log_size":937,"log_dups_size":0,"ondisk_log_size":937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:42.299937+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:00:42.299937+0000","last_peered":"2023-06-09T12:00:42.299937+0000","last_clean":"2023-06-09T12:00:42.299937+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:00:42.299937+0000","last_undegraded":"2023-06-09T12:00:42.299937+0000","last_fullsized":"2023-06-09T12:00:42.299937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:47.304467+0000","last_change":"2023-06-09T12:03:35.770715+0000","last_active":"2023-06-09T12:02:47.304467+0000","last_peered":"2023-06-09T12:02:47.304467+0000","last_clean":"2023-06-09T12:02:47.304467+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:02:47.304467+0000","last_undegraded":"2023-06-09T12:02:47.304467+0000","last_fullsized":"2023-06-09T12:02:47.304467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.289301+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:00:47.289301+0000","last_peered":"2023-06-09T12:00:47.289301+0000","last_clean":"2023-06-09T12:00:47.289301+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:00:47.289301+0000","last_undegraded":"2023-06-09T12:00:47.289301+0000","last_fullsized":"2023-06-09T12:00:47.289301+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1077,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064711+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064711+0000","last_peered":"2023-06-09T12:03:06.064711+0000","last_clean":"2023-06-09T12:03:06.064711+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064711+0000","last_undegraded":"2023-06-09T12:03:06.064711+0000","last_fullsized":"2023-06-09T12:03:06.064711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3599,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305793+0000","last_change":"2023-06-09T12:03:17.778068+0000","last_active":"2023-06-09T12:02:57.305793+0000","last_peered":"2023-06-09T12:02:57.305793+0000","last_clean":"2023-06-09T12:02:57.305793+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:02:57.305793+0000","last_undegraded":"2023-06-09T12:02:57.305793+0000","last_fullsized":"2023-06-09T12:02:57.305793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":158,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.306808+0000","last_change":"2023-06-09T12:04:48.020256+0000","last_active":"2023-06-09T12:02:52.306808+0000","last_peered":"2023-06-09T12:02:52.306808+0000","last_clean":"2023-06-09T12:02:52.306808+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:02:52.306808+0000","last_undegraded":"2023-06-09T12:02:52.306808+0000","last_fullsized":"2023-06-09T12:02:52.306808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3892,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.305823+0000","last_change":"2023-06-09T12:03:46.981824+0000","last_active":"2023-06-09T12:02:57.305823+0000","last_peered":"2023-06-09T12:02:57.305823+0000","last_clean":"2023-06-09T12:02:57.305823+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:02:57.305823+0000","last_undegraded":"2023-06-09T12:02:57.305823+0000","last_fullsized":"2023-06-09T12:02:57.305823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:47.375458+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:00:47.375458+0000","last_peered":"2023-06-09T12:00:47.375458+0000","last_clean":"2023-06-09T12:00:47.375458+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:00:47.375458+0000","last_undegraded":"2023-06-09T12:00:47.375458+0000","last_fullsized":"2023-06-09T12:00:47.375458+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.763053+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.763053+0000","last_peered":"2023-06-09T12:01:53.763053+0000","last_clean":"2023-06-09T12:01:53.763053+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.763053+0000","last_undegraded":"2023-06-09T12:01:53.763053+0000","last_fullsized":"2023-06-09T12:01:53.763053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3302,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.507333+0000","last_change":"2023-06-09T12:03:53.117922+0000","last_active":"2023-06-09T12:03:07.507333+0000","last_peered":"2023-06-09T12:03:07.507333+0000","last_clean":"2023-06-09T12:03:07.507333+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:03:07.507333+0000","last_undegraded":"2023-06-09T12:03:07.507333+0000","last_fullsized":"2023-06-09T12:03:07.507333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.412288+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:52.412288+0000","last_peered":"2023-06-09T12:02:52.412288+0000","last_clean":"2023-06-09T12:02:52.412288+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:52.412288+0000","last_undegraded":"2023-06-09T12:02:52.412288+0000","last_fullsized":"2023-06-09T12:02:52.412288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2989,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.310842+0000","last_change":"2023-06-09T12:04:01.358848+0000","last_active":"2023-06-09T12:02:57.310842+0000","last_peered":"2023-06-09T12:02:57.310842+0000","last_clean":"2023-06-09T12:02:57.310842+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:02:57.310842+0000","last_undegraded":"2023-06-09T12:02:57.310842+0000","last_fullsized":"2023-06-09T12:02:57.310842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:50.985146+0000","last_change":"2023-06-09T12:02:52.310367+0000","last_active":"2023-06-09T12:01:50.985146+0000","last_peered":"2023-06-09T12:01:50.985146+0000","last_clean":"2023-06-09T12:01:50.985146+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:01:50.985146+0000","last_undegraded":"2023-06-09T12:01:50.985146+0000","last_fullsized":"2023-06-09T12:01:50.985146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.398228+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:00:52.398228+0000","last_peered":"2023-06-09T12:00:52.398228+0000","last_clean":"2023-06-09T12:00:52.398228+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:00:52.398228+0000","last_undegraded":"2023-06-09T12:00:52.398228+0000","last_fullsized":"2023-06-09T12:00:52.398228+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696148+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696148+0000","last_peered":"2023-06-09T12:03:31.696148+0000","last_clean":"2023-06-09T12:03:31.696148+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696148+0000","last_undegraded":"2023-06-09T12:03:31.696148+0000","last_fullsized":"2023-06-09T12:03:31.696148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4114,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.870372+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:12.870372+0000","last_peered":"2023-06-09T12:02:12.870372+0000","last_clean":"2023-06-09T12:02:12.870372+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:12.870372+0000","last_undegraded":"2023-06-09T12:02:12.870372+0000","last_fullsized":"2023-06-09T12:02:12.870372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":523,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.317695+0000","last_change":"2023-06-09T12:04:02.117422+0000","last_active":"2023-06-09T12:02:52.317695+0000","last_peered":"2023-06-09T12:02:52.317695+0000","last_clean":"2023-06-09T12:02:52.317695+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:02:52.317695+0000","last_undegraded":"2023-06-09T12:02:52.317695+0000","last_fullsized":"2023-06-09T12:02:52.317695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:02.302532+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:01:02.302532+0000","last_peered":"2023-06-09T12:01:02.302532+0000","last_clean":"2023-06-09T12:01:02.302532+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:01:02.302532+0000","last_undegraded":"2023-06-09T12:01:02.302532+0000","last_fullsized":"2023-06-09T12:01:02.302532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:49.371371+0000","last_change":"2023-06-09T12:02:49.371371+0000","last_active":"2023-06-09T12:02:49.371371+0000","last_peered":"2023-06-09T12:02:49.371371+0000","last_clean":"2023-06-09T12:02:49.371371+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:02:49.371371+0000","last_undegraded":"2023-06-09T12:02:49.371371+0000","last_fullsized":"2023-06-09T12:02:49.371371+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":830,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:52.310351+0000","last_change":"2023-06-09T12:04:28.374029+0000","last_active":"2023-06-09T12:02:52.310351+0000","last_peered":"2023-06-09T12:02:52.310351+0000","last_clean":"2023-06-09T12:02:52.310351+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:02:52.310351+0000","last_undegraded":"2023-06-09T12:02:52.310351+0000","last_fullsized":"2023-06-09T12:02:52.310351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.429061+0000","last_change":"2023-06-09T12:02:58.042721+0000","last_active":"2023-06-09T12:02:57.429061+0000","last_peered":"2023-06-09T12:02:57.429061+0000","last_clean":"2023-06-09T12:02:57.429061+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:57.429061+0000","last_undegraded":"2023-06-09T12:02:57.429061+0000","last_fullsized":"2023-06-09T12:02:57.429061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":883,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:48.982495+0000","last_change":"2023-06-09T12:03:22.314299+0000","last_active":"2023-06-09T12:01:48.982495+0000","last_peered":"2023-06-09T12:01:48.982495+0000","last_clean":"2023-06-09T12:01:48.982495+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:01:48.982495+0000","last_undegraded":"2023-06-09T12:01:48.982495+0000","last_fullsized":"2023-06-09T12:01:48.982495+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":770,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.225656+0000","last_change":"2023-06-09T12:03:22.310323+0000","last_active":"2023-06-09T12:02:50.225656+0000","last_peered":"2023-06-09T12:02:50.225656+0000","last_clean":"2023-06-09T12:02:50.225656+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:02:50.225656+0000","last_undegraded":"2023-06-09T12:02:50.225656+0000","last_fullsized":"2023-06-09T12:02:50.225656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3431,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:52.111000+0000","last_change":"2023-06-09T12:03:52.111000+0000","last_active":"2023-06-09T12:03:52.111000+0000","last_peered":"2023-06-09T12:03:52.111000+0000","last_clean":"2023-06-09T12:03:52.111000+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:52.111000+0000","last_undegraded":"2023-06-09T12:03:52.111000+0000","last_fullsized":"2023-06-09T12:03:52.111000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3939,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:02:57.318356+0000","last_change":"2023-06-09T12:03:09.981419+0000","last_active":"2023-06-09T12:02:57.318356+0000","last_peered":"2023-06-09T12:02:57.318356+0000","last_clean":"2023-06-09T12:02:57.318356+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:02:57.318356+0000","last_undegraded":"2023-06-09T12:02:57.318356+0000","last_fullsized":"2023-06-09T12:02:57.318356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:52.673491+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:00:52.673491+0000","last_peered":"2023-06-09T12:00:52.673491+0000","last_clean":"2023-06-09T12:00:52.673491+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:00:52.673491+0000","last_undegraded":"2023-06-09T12:00:52.673491+0000","last_fullsized":"2023-06-09T12:00:52.673491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2981,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:07.319759+0000","last_change":"2023-06-09T12:03:59.102505+0000","last_active":"2023-06-09T12:03:07.319759+0000","last_peered":"2023-06-09T12:03:07.319759+0000","last_clean":"2023-06-09T12:03:07.319759+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:07.319759+0000","last_undegraded":"2023-06-09T12:03:07.319759+0000","last_fullsized":"2023-06-09T12:03:07.319759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:08.152707+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:08.152707+0000","last_peered":"2023-06-09T12:02:08.152707+0000","last_clean":"2023-06-09T12:02:08.152707+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:08.152707+0000","last_undegraded":"2023-06-09T12:02:08.152707+0000","last_fullsized":"2023-06-09T12:02:08.152707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":62358,"ondisk_log_size":62358,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30772,"ondisk_log_size":30772,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575501,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255376,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":75520,"kb_avail":94116464,"statfs":{"total":96636764160,"available":96375259136,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":77332480},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.66700000000000004},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.95999999999999996,"5min":0.96099999999999997,"15min":1.0640000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":1.256,"15min":1.3320000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.25},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.72599999999999998,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.05},"last":0.752}]},{"osd":2,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.624},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.95399999999999996,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.77900000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.441,"15min":0.379},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.1200000000000001},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.32700000000000001},"max":{"1min":0.99299999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.497}]},{"osd":5,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.47799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.044,"15min":1.0920000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.73099999999999998},"min":{"1min":0.57599999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":1.093,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.52800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575501,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208088,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":51968,"kb_avail":94163752,"statfs":{"total":96636764160,"available":96423682048,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53215232},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.746},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96799999999999997,"15min":1.069},"last":0.77800000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.60699999999999998,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.86199999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76600000000000001,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.81999999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.754,"15min":0.70399999999999996},"min":{"1min":0.57499999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0369999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.93799999999999994,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.623},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.498,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0129999999999999,"15min":1.052},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":1.071},"last":0.67300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575501,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405680,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99264,"kb_avail":93966160,"statfs":{"total":96636764160,"available":96221347840,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101646336},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.438,"15min":0.41599999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.57799999999999996,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.48499999999999999,"5min":0.41099999999999998,"15min":0.36899999999999999},"max":{"1min":0.68700000000000006,"5min":0.82399999999999995,"15min":1.877},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.47799999999999998},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.28000000000000003},"max":{"1min":0.67700000000000005,"5min":0.995,"15min":1.982},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.47699999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.502},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.49299999999999999,"15min":0.38700000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.635},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.609},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.81799999999999995},"min":{"1min":0.67800000000000005,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.2370000000000001,"15min":1.774},"last":0.88300000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83299999999999996,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.038,"5min":1.145,"15min":1.7390000000000001},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.878,"15min":0.83599999999999997},"min":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.55600000000000005},"max":{"1min":1.0629999999999999,"5min":1.0980000000000001,"15min":1.6879999999999999},"last":1}]},{"osd":6,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":0.99199999999999999,"5min":1.27,"15min":1.794},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.97799999999999998,"5min":1.1819999999999999,"15min":1.921},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89700000000000002,"15min":0.876},"min":{"1min":0.73699999999999999,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0069999999999999,"5min":1.2629999999999999,"15min":1.839},"last":1.032},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.52100000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.827},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332648,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104832,"kb_avail":94039192,"statfs":{"total":96636764160,"available":96296132608,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107347968},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.442,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.70799999999999996},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.46400000000000002,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.629,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.032,"15min":1.0900000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.499},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.97399999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.024},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.71099999999999997},"min":{"1min":0.629,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.0569999999999999,"5min":1.175,"15min":1.175},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.0449999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.94399999999999995,"5min":1.034,"15min":1.236},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83799999999999997,"15min":0.77200000000000002},"min":{"1min":0.68700000000000006,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":1.073}]},{"osd":6,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":0.98799999999999999,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0149999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.745}]},{"osd":7,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.84599999999999997},"min":{"1min":0.81299999999999994,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.0940000000000001,"5min":1.351,"15min":1.351},"last":1.0069999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242016,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62208,"kb_avail":94129824,"statfs":{"total":96636764160,"available":96388939776,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":63700992},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26800000000000002},"max":{"1min":0.70299999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.999,"15min":1.0940000000000001},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60999999999999999,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89200000000000002,"5min":0.92300000000000004,"15min":1.367},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.504},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.67400000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.171,"15min":1.8220000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":1.214,"15min":1.7370000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.79900000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":0.96799999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.83899999999999997},"min":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.752}]},{"osd":6,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.105,"15min":1.3540000000000001},"last":0.629}]},{"osd":7,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.0029999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83799999999999997,"15min":0.82299999999999995},"min":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.79900000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575501,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422012,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":93949828,"statfs":{"total":96636764160,"available":96204623872,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.26900000000000002},"max":{"1min":0.70199999999999996,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.28599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.64300000000000002},"min":{"1min":0.50600000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.355},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":1.006,"5min":1.073,"15min":1.635},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0980000000000001,"5min":3.081,"15min":3.081},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.68700000000000006,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0920000000000001,"5min":1.198,"15min":1.2110000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88100000000000001,"15min":0.81000000000000005},"min":{"1min":0.73699999999999999,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.234,"15min":1.367},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.65900000000000003,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0229999999999999,"5min":1.103,"15min":3.3490000000000002},"last":0.81599999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.0700000000000001,"5min":1.306,"15min":1.413},"last":0.628},{"interface":"front","average":{"1min":0.875,"5min":0.88900000000000001,"15min":0.86199999999999999},"min":{"1min":0.64200000000000002,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.137,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575500,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418772,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":126016,"kb_avail":93953068,"statfs":{"total":96636764160,"available":96207941632,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":129040384},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72799999999999998,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0600000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.746}]},{"osd":1,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.0640000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.755},"min":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.044,"15min":1.1279999999999999},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":1.117,"15min":1.609},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.68100000000000005},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.38100000000000001},"max":{"1min":1.0149999999999999,"5min":1.147,"15min":1.159},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":0.999,"5min":1.014,"15min":1.407},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.86199999999999999,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.47999999999999998,"5min":0.39300000000000002,"15min":0.30299999999999999},"max":{"1min":0.78900000000000003,"5min":0.95999999999999996,"15min":1.0349999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.47199999999999998,"15min":0.34100000000000003},"max":{"1min":0.98799999999999999,"5min":1.014,"15min":1.1719999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.69999999999999996},"min":{"1min":0.58699999999999997,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.3100000000000001},"last":0.753}]},{"osd":7,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60199999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":0.94199999999999995,"15min":1.032},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.361},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575500,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293144,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75776,"kb_avail":94078696,"statfs":{"total":96636764160,"available":96336584704,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77594624},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.96299999999999997,"5min":3.468,"15min":5.4969999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89200000000000002,"5min":1.177,"15min":5.6269999999999998},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":19.715},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.72699999999999998},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.127,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":1.016,"5min":1.016,"15min":5.5620000000000003},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.49099999999999999,"15min":0.39200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":14.439},"last":0.69999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.61699999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":12.130000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78400000000000003,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.81899999999999995},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.44},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.82899999999999996},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":67.521000000000001},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":1.097,"5min":1.097,"15min":12.163},"last":0.629},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.95599999999999996,"5min":0.96299999999999997,"15min":3.8039999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.81799999999999995},"min":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.135,"15min":19.834},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:04:54.398 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.399 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.399 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.400 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.400 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.400 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.401 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.401 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.401 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.402 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.402 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.402 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.403 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.403 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.404 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.404 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.404 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.405 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.405 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.405 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.406 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.406 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.406 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.407 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.407 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.407 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.408 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.408 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.408 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.409 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.409 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.409 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.410 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.410 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.410 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.411 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.411 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.411 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.412 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.412 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.413 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.413 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.413 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.414 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.414 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.414 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.415 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.415 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.415 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.416 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.416 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.416 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.417 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.417 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.417 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.418 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.418 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.418 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.419 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.419 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.419 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.420 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.420 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.421 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.421 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:04:54.421 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:04:57.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:04:57.186 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:14.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:05:14.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.632+0000 7f33ce1ff700 1 Processor -- start 2023-06-09T12:05:14.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.632+0000 7f33ce1ff700 1 -- start start 2023-06-09T12:05:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33ce1ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33c815a0c0 0x7f33c81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:14.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33ce1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80c0bf0 con 0x7f33c80c0140 2023-06-09T12:05:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33ce1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80702f0 con 0x7f33c80c1bc0 2023-06-09T12:05:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33ce1ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c8070110 con 0x7f33c815a0c0 2023-06-09T12:05:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33c6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33c815a0c0 0x7f33c81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33c6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f33c815a0c0 0x7f33c81501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59858/0 (socket says 172.21.15.7:59858) 2023-06-09T12:05:14.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.633+0000 7f33c6ffd700 1 -- 172.21.15.7:0/718125176 learned_addr learned my addr 172.21.15.7:0/718125176 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:05:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c6ffd700 1 -- 172.21.15.7:0/718125176 >> v1:172.21.15.117:6789/0 conn(0x7f33c80c1bc0 legacy=0x7f33c81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c6ffd700 1 -- 172.21.15.7:0/718125176 >> v1:172.21.15.7:6789/0 conn(0x7f33c80c0140 legacy=0x7f33c8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c6ffd700 1 -- 172.21.15.7:0/718125176 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80c0e90 con 0x7f33c815a0c0 2023-06-09T12:05:14.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c6ffd700 1 --2- 172.21.15.7:0/718125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81501b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0098f0 tx=0x7f33bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643cab5b853b0a03 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c67fc700 1 -- 172.21.15.7:0/718125176 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00b8a0 con 0x7f33c815a0c0 2023-06-09T12:05:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c67fc700 1 -- 172.21.15.7:0/718125176 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc00ba00 con 0x7f33c815a0c0 2023-06-09T12:05:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.634+0000 7f33c67fc700 1 -- 172.21.15.7:0/718125176 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc00bd90 con 0x7f33c815a0c0 2023-06-09T12:05:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 -- 172.21.15.7:0/718125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 msgr2=0x7f33c81501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/718125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81501b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0098f0 tx=0x7f33bc009c00 comp rx=0 tx=0).stop 2023-06-09T12:05:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 -- 172.21.15.7:0/718125176 shutdown_connections 2023-06-09T12:05:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/718125176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81501b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 -- 172.21.15.7:0/718125176 >> 172.21.15.7:0/718125176 conn(0x7f33c806a4d0 msgr2=0x7f33c806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:14.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 -- 172.21.15.7:0/718125176 shutdown_connections 2023-06-09T12:05:14.641 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.635+0000 7f33ce1ff700 1 -- 172.21.15.7:0/718125176 wait complete. 2023-06-09T12:05:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.637+0000 7f33ce1ff700 1 Processor -- start 2023-06-09T12:05:14.642 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.637+0000 7f33ce1ff700 1 -- start start 2023-06-09T12:05:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c80c0140 0x7f33c80c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33c80c1bc0 0x7f33c814b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:14.643 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c814b7c0 con 0x7f33c815a0c0 2023-06-09T12:05:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c814ea40 con 0x7f33c80c0140 2023-06-09T12:05:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33ce1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c81ece70 con 0x7f33c80c1bc0 2023-06-09T12:05:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59860/0 (socket says 172.21.15.7:59860) 2023-06-09T12:05:14.644 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 -- 172.21.15.7:0/3026599958 learned_addr learned my addr 172.21.15.7:0/3026599958 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:05:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c6ffd700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33c80c1bc0 0x7f33c814b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c77fe700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c80c0140 0x7f33c80c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 -- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33c80c1bc0 msgr2=0x7f33c814b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.645 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33c80c1bc0 0x7f33c814b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 -- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c80c0140 msgr2=0x7f33c80c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c80c0140 0x7f33c80c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.638+0000 7f33c7fff700 1 -- 172.21.15.7:0/3026599958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc009550 con 0x7f33c815a0c0 2023-06-09T12:05:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33c7fff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f33c000cc50 tx=0x7f33c000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:14.646 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0003c00 con 0x7f33c815a0c0 2023-06-09T12:05:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c0003d60 con 0x7f33c815a0c0 2023-06-09T12:05:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000aab0 con 0x7f33c815a0c0 2023-06-09T12:05:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c80c1450 con 0x7f33c815a0c0 2023-06-09T12:05:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.639+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c80c1170 con 0x7f33c815a0c0 2023-06-09T12:05:14.647 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.640+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c80705d0 con 0x7f33c815a0c0 2023-06-09T12:05:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.641+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33c000ac10 con 0x7f33c815a0c0 2023-06-09T12:05:14.648 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.643+0000 7f33affff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 0x7f33b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.643+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f33c0089ca0 con 0x7f33c815a0c0 2023-06-09T12:05:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.643+0000 7f33c77fe700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 0x7f33b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.643+0000 7f33c77fe700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 0x7f33b406d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f33b8007fd0 tx=0x7f33b8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:14.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.647+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f33c00544c0 con 0x7f33c815a0c0 2023-06-09T12:05:14.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.851+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33c8074cf0 con 0x7f33b406ace0 2023-06-09T12:05:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.860+0000 7f33affff700 1 -- 172.21.15.7:0/3026599958 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315535 (secure 0 0 0) 0x7f33c8074cf0 con 0x7f33b406ace0 2023-06-09T12:05:14.866 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:05:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 msgr2=0x7f33b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 0x7f33b406d190 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f33b8007fd0 tx=0x7f33b8009040 comp rx=0 tx=0).stop 2023-06-09T12:05:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 msgr2=0x7f33c81ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f33c000cc50 tx=0x7f33c000cf60 comp rx=0 tx=0).stop 2023-06-09T12:05:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 shutdown_connections 2023-06-09T12:05:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33c815a0c0 0x7f33c81ef670 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33c80c0140 0x7f33c80c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f33b406ace0 0x7f33b406d190 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 --2- 172.21.15.7:0/3026599958 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33c80c1bc0 0x7f33c814b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 >> 172.21.15.7:0/3026599958 conn(0x7f33c806a4d0 msgr2=0x7f33c80c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 shutdown_connections 2023-06-09T12:05:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:14.865+0000 7f33ce1ff700 1 -- 172.21.15.7:0/3026599958 wait complete. 2023-06-09T12:05:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:05:14.885 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1703,"stamp":"2023-06-09T12:05:13.018188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93762,"ondisk_log_size":93762,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2587016,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":703680,"kb_avail":752387704,"statfs":{"total":773094113280,"available":770445008896,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":720568320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003589"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3472,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:44.696195+0000","last_change":"2023-06-09T12:05:02.403019+0000","last_active":"2023-06-09T12:04:44.696195+0000","last_peered":"2023-06-09T12:04:44.696195+0000","last_clean":"2023-06-09T12:04:44.696195+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:04:44.696195+0000","last_undegraded":"2023-06-09T12:04:44.696195+0000","last_fullsized":"2023-06-09T12:04:44.696195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3223,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.076031+0000","last_change":"2023-06-09T12:05:02.350871+0000","last_active":"2023-06-09T12:04:42.076031+0000","last_peered":"2023-06-09T12:04:42.076031+0000","last_clean":"2023-06-09T12:04:42.076031+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:04:42.076031+0000","last_undegraded":"2023-06-09T12:04:42.076031+0000","last_fullsized":"2023-06-09T12:04:42.076031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3699,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088463+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088463+0000","last_peered":"2023-06-09T12:03:46.088463+0000","last_clean":"2023-06-09T12:03:46.088463+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088463+0000","last_undegraded":"2023-06-09T12:03:46.088463+0000","last_fullsized":"2023-06-09T12:03:46.088463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'935","reported_seq":3775,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:10.894923+0000","last_change":"2023-06-09T12:04:57.102635+0000","last_active":"2023-06-09T12:05:10.894923+0000","last_peered":"2023-06-09T12:05:10.894923+0000","last_clean":"2023-06-09T12:05:10.894923+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:10.894923+0000","last_undegraded":"2023-06-09T12:05:10.894923+0000","last_fullsized":"2023-06-09T12:05:10.894923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3409,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:31.748046+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:31.748046+0000","last_peered":"2023-06-09T12:04:31.748046+0000","last_clean":"2023-06-09T12:04:31.748046+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:31.748046+0000","last_undegraded":"2023-06-09T12:04:31.748046+0000","last_fullsized":"2023-06-09T12:04:31.748046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1009","reported_seq":3397,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:07.426913+0000","last_change":"2023-06-09T12:04:30.729758+0000","last_active":"2023-06-09T12:05:07.426913+0000","last_peered":"2023-06-09T12:05:07.426913+0000","last_clean":"2023-06-09T12:05:07.426913+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:07.426913+0000","last_undegraded":"2023-06-09T12:05:07.426913+0000","last_fullsized":"2023-06-09T12:05:07.426913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":147,"log_size":1009,"log_dups_size":0,"ondisk_log_size":1009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1158","reported_seq":3582,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:07.040033+0000","last_change":"2023-06-09T12:04:38.738579+0000","last_active":"2023-06-09T12:05:07.040033+0000","last_peered":"2023-06-09T12:05:07.040033+0000","last_clean":"2023-06-09T12:05:07.040033+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:07.040033+0000","last_undegraded":"2023-06-09T12:05:07.040033+0000","last_fullsized":"2023-06-09T12:05:07.040033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":223,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 28s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3737,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.370675+0000","last_change":"2023-06-09T12:04:32.323061+0000","last_active":"2023-06-09T12:03:55.370675+0000","last_peered":"2023-06-09T12:03:55.370675+0000","last_clean":"2023-06-09T12:03:55.370675+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:03:55.370675+0000","last_undegraded":"2023-06-09T12:03:55.370675+0000","last_fullsized":"2023-06-09T12:03:55.370675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.422718+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T12:02:12.422718+0000","last_peered":"2023-06-09T12:02:12.422718+0000","last_clean":"2023-06-09T12:02:12.422718+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:02:12.422718+0000","last_undegraded":"2023-06-09T12:02:12.422718+0000","last_fullsized":"2023-06-09T12:02:12.422718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.308346+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:02:37.308346+0000","last_peered":"2023-06-09T12:02:37.308346+0000","last_clean":"2023-06-09T12:02:37.308346+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:02:37.308346+0000","last_undegraded":"2023-06-09T12:02:37.308346+0000","last_fullsized":"2023-06-09T12:02:37.308346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780827+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780827+0000","last_peered":"2023-06-09T12:03:12.780827+0000","last_clean":"2023-06-09T12:03:12.780827+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780827+0000","last_undegraded":"2023-06-09T12:03:12.780827+0000","last_fullsized":"2023-06-09T12:03:12.780827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.278802+0000","last_change":"2023-06-09T12:05:02.350828+0000","last_active":"2023-06-09T12:03:46.278802+0000","last_peered":"2023-06-09T12:03:46.278802+0000","last_clean":"2023-06-09T12:03:46.278802+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:03:46.278802+0000","last_undegraded":"2023-06-09T12:03:46.278802+0000","last_fullsized":"2023-06-09T12:03:46.278802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.311625+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:03:02.311625+0000","last_peered":"2023-06-09T12:03:02.311625+0000","last_clean":"2023-06-09T12:03:02.311625+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:03:02.311625+0000","last_undegraded":"2023-06-09T12:03:02.311625+0000","last_fullsized":"2023-06-09T12:03:02.311625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3311,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.082714+0000","last_change":"2023-06-09T12:05:02.520050+0000","last_active":"2023-06-09T12:03:58.082714+0000","last_peered":"2023-06-09T12:03:58.082714+0000","last_clean":"2023-06-09T12:03:58.082714+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:58.082714+0000","last_undegraded":"2023-06-09T12:03:58.082714+0000","last_fullsized":"2023-06-09T12:03:58.082714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1564,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.087225+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:00:44.087225+0000","last_peered":"2023-06-09T12:00:44.087225+0000","last_clean":"2023-06-09T12:00:44.087225+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:00:44.087225+0000","last_undegraded":"2023-06-09T12:00:44.087225+0000","last_fullsized":"2023-06-09T12:00:44.087225+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:13.801581+0000","last_change":"2023-06-09T12:04:32.318108+0000","last_active":"2023-06-09T12:03:13.801581+0000","last_peered":"2023-06-09T12:03:13.801581+0000","last_clean":"2023-06-09T12:03:13.801581+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:03:13.801581+0000","last_undegraded":"2023-06-09T12:03:13.801581+0000","last_fullsized":"2023-06-09T12:03:13.801581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.161159+0000","last_change":"2023-06-09T12:04:32.424054+0000","last_active":"2023-06-09T12:02:49.161159+0000","last_peered":"2023-06-09T12:02:49.161159+0000","last_clean":"2023-06-09T12:02:49.161159+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:49.161159+0000","last_undegraded":"2023-06-09T12:02:49.161159+0000","last_fullsized":"2023-06-09T12:02:49.161159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3561,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313831+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:02:22.313831+0000","last_peered":"2023-06-09T12:02:22.313831+0000","last_clean":"2023-06-09T12:02:22.313831+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:02:22.313831+0000","last_undegraded":"2023-06-09T12:02:22.313831+0000","last_fullsized":"2023-06-09T12:02:22.313831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:43.259734+0000","last_change":"2023-06-09T12:04:02.343332+0000","last_active":"2023-06-09T12:03:43.259734+0000","last_peered":"2023-06-09T12:03:43.259734+0000","last_clean":"2023-06-09T12:03:43.259734+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:03:43.259734+0000","last_undegraded":"2023-06-09T12:03:43.259734+0000","last_fullsized":"2023-06-09T12:03:43.259734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.914290+0000","last_change":"2023-06-09T12:04:02.319111+0000","last_active":"2023-06-09T12:01:54.914290+0000","last_peered":"2023-06-09T12:01:54.914290+0000","last_clean":"2023-06-09T12:01:54.914290+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:54.914290+0000","last_undegraded":"2023-06-09T12:01:54.914290+0000","last_fullsized":"2023-06-09T12:01:54.914290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1337,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206326+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206326+0000","last_peered":"2023-06-09T12:03:02.206326+0000","last_clean":"2023-06-09T12:03:02.206326+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206326+0000","last_undegraded":"2023-06-09T12:03:02.206326+0000","last_fullsized":"2023-06-09T12:03:02.206326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.198865+0000","last_change":"2023-06-09T12:04:57.427044+0000","last_active":"2023-06-09T12:02:50.198865+0000","last_peered":"2023-06-09T12:02:50.198865+0000","last_clean":"2023-06-09T12:02:50.198865+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:50.198865+0000","last_undegraded":"2023-06-09T12:02:50.198865+0000","last_fullsized":"2023-06-09T12:02:50.198865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428433+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:02:52.428433+0000","last_peered":"2023-06-09T12:02:52.428433+0000","last_clean":"2023-06-09T12:02:52.428433+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:02:52.428433+0000","last_undegraded":"2023-06-09T12:02:52.428433+0000","last_fullsized":"2023-06-09T12:02:52.428433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3544,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.039481+0000","last_change":"2023-06-09T12:05:12.521142+0000","last_active":"2023-06-09T12:03:55.039481+0000","last_peered":"2023-06-09T12:03:55.039481+0000","last_clean":"2023-06-09T12:03:55.039481+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:55.039481+0000","last_undegraded":"2023-06-09T12:03:55.039481+0000","last_fullsized":"2023-06-09T12:03:55.039481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:51.058132+0000","last_change":"2023-06-09T12:04:52.349590+0000","last_active":"2023-06-09T12:04:51.058132+0000","last_peered":"2023-06-09T12:04:51.058132+0000","last_clean":"2023-06-09T12:04:51.058132+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:04:51.058132+0000","last_undegraded":"2023-06-09T12:04:51.058132+0000","last_fullsized":"2023-06-09T12:04:51.058132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:53.325190+0000","last_change":"2023-06-09T12:04:52.325633+0000","last_active":"2023-06-09T12:03:53.325190+0000","last_peered":"2023-06-09T12:03:53.325190+0000","last_clean":"2023-06-09T12:03:53.325190+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:03:53.325190+0000","last_undegraded":"2023-06-09T12:03:53.325190+0000","last_fullsized":"2023-06-09T12:03:53.325190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.426543+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:02:37.426543+0000","last_peered":"2023-06-09T12:02:37.426543+0000","last_clean":"2023-06-09T12:02:37.426543+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:02:37.426543+0000","last_undegraded":"2023-06-09T12:02:37.426543+0000","last_fullsized":"2023-06-09T12:02:37.426543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1081","reported_seq":3435,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:11.155880+0000","last_change":"2023-06-09T12:04:42.478870+0000","last_active":"2023-06-09T12:05:11.155880+0000","last_peered":"2023-06-09T12:05:11.155880+0000","last_clean":"2023-06-09T12:05:11.155880+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:11.155880+0000","last_undegraded":"2023-06-09T12:05:11.155880+0000","last_fullsized":"2023-06-09T12:05:11.155880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":216,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 28s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.317523+0000","last_change":"2023-06-09T12:00:42.299937+0000","last_active":"2023-06-09T12:02:52.317523+0000","last_peered":"2023-06-09T12:02:52.317523+0000","last_clean":"2023-06-09T12:02:52.317523+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:02:52.317523+0000","last_undegraded":"2023-06-09T12:02:52.317523+0000","last_fullsized":"2023-06-09T12:02:52.317523+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:17.778068+0000","last_change":"2023-06-09T12:05:02.322556+0000","last_active":"2023-06-09T12:03:17.778068+0000","last_peered":"2023-06-09T12:03:17.778068+0000","last_clean":"2023-06-09T12:03:17.778068+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:03:17.778068+0000","last_undegraded":"2023-06-09T12:03:17.778068+0000","last_fullsized":"2023-06-09T12:03:17.778068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:48.020256+0000","last_change":"2023-06-09T12:04:57.350272+0000","last_active":"2023-06-09T12:04:48.020256+0000","last_peered":"2023-06-09T12:04:48.020256+0000","last_clean":"2023-06-09T12:04:48.020256+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:04:48.020256+0000","last_undegraded":"2023-06-09T12:04:48.020256+0000","last_fullsized":"2023-06-09T12:04:48.020256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.763053+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.763053+0000","last_peered":"2023-06-09T12:01:53.763053+0000","last_clean":"2023-06-09T12:01:53.763053+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.763053+0000","last_undegraded":"2023-06-09T12:01:53.763053+0000","last_fullsized":"2023-06-09T12:01:53.763053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'961","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:53.117922+0000","last_change":"2023-06-09T12:05:12.521168+0000","last_active":"2023-06-09T12:03:53.117922+0000","last_peered":"2023-06-09T12:03:53.117922+0000","last_clean":"2023-06-09T12:03:53.117922+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:03:53.117922+0000","last_undegraded":"2023-06-09T12:03:53.117922+0000","last_fullsized":"2023-06-09T12:03:53.117922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.412288+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:52.412288+0000","last_peered":"2023-06-09T12:02:52.412288+0000","last_clean":"2023-06-09T12:02:52.412288+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:52.412288+0000","last_undegraded":"2023-06-09T12:02:52.412288+0000","last_fullsized":"2023-06-09T12:02:52.412288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2990,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:01.358848+0000","last_change":"2023-06-09T12:05:02.326837+0000","last_active":"2023-06-09T12:04:01.358848+0000","last_peered":"2023-06-09T12:04:01.358848+0000","last_clean":"2023-06-09T12:04:01.358848+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:04:01.358848+0000","last_undegraded":"2023-06-09T12:04:01.358848+0000","last_fullsized":"2023-06-09T12:04:01.358848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:54.261006+0000","last_change":"2023-06-09T12:05:02.326991+0000","last_active":"2023-06-09T12:04:54.261006+0000","last_peered":"2023-06-09T12:04:54.261006+0000","last_clean":"2023-06-09T12:04:54.261006+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:04:54.261006+0000","last_undegraded":"2023-06-09T12:04:54.261006+0000","last_fullsized":"2023-06-09T12:04:54.261006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":848,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696148+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696148+0000","last_peered":"2023-06-09T12:03:31.696148+0000","last_clean":"2023-06-09T12:03:31.696148+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696148+0000","last_undegraded":"2023-06-09T12:03:31.696148+0000","last_fullsized":"2023-06-09T12:03:31.696148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.310949+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:57.310949+0000","last_peered":"2023-06-09T12:02:57.310949+0000","last_clean":"2023-06-09T12:02:57.310949+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:57.310949+0000","last_undegraded":"2023-06-09T12:02:57.310949+0000","last_fullsized":"2023-06-09T12:02:57.310949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3371,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.318985+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:03:02.318985+0000","last_peered":"2023-06-09T12:03:02.318985+0000","last_clean":"2023-06-09T12:03:02.318985+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:03:02.318985+0000","last_undegraded":"2023-06-09T12:03:02.318985+0000","last_fullsized":"2023-06-09T12:03:02.318985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.112083+0000","last_change":"2023-06-09T12:04:57.519309+0000","last_active":"2023-06-09T12:03:57.112083+0000","last_peered":"2023-06-09T12:03:57.112083+0000","last_clean":"2023-06-09T12:03:57.112083+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:03:57.112083+0000","last_undegraded":"2023-06-09T12:03:57.112083+0000","last_fullsized":"2023-06-09T12:03:57.112083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:28.374029+0000","last_change":"2023-06-09T12:05:02.326978+0000","last_active":"2023-06-09T12:04:28.374029+0000","last_peered":"2023-06-09T12:04:28.374029+0000","last_clean":"2023-06-09T12:04:28.374029+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:04:28.374029+0000","last_undegraded":"2023-06-09T12:04:28.374029+0000","last_fullsized":"2023-06-09T12:04:28.374029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":884,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:22.314299+0000","last_change":"2023-06-09T12:04:52.346421+0000","last_active":"2023-06-09T12:03:22.314299+0000","last_peered":"2023-06-09T12:03:22.314299+0000","last_clean":"2023-06-09T12:03:22.314299+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:03:22.314299+0000","last_undegraded":"2023-06-09T12:03:22.314299+0000","last_fullsized":"2023-06-09T12:03:22.314299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":771,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:22.310323+0000","last_change":"2023-06-09T12:04:53.107623+0000","last_active":"2023-06-09T12:03:22.310323+0000","last_peered":"2023-06-09T12:03:22.310323+0000","last_clean":"2023-06-09T12:03:22.310323+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:03:22.310323+0000","last_undegraded":"2023-06-09T12:03:22.310323+0000","last_fullsized":"2023-06-09T12:03:22.310323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.124241+0000","last_change":"2023-06-09T12:05:02.519982+0000","last_active":"2023-06-09T12:03:59.124241+0000","last_peered":"2023-06-09T12:03:59.124241+0000","last_clean":"2023-06-09T12:03:59.124241+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:59.124241+0000","last_undegraded":"2023-06-09T12:03:59.124241+0000","last_fullsized":"2023-06-09T12:03:59.124241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'866","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305110+0000","last_change":"2023-06-09T12:02:52.305110+0000","last_active":"2023-06-09T12:02:52.305110+0000","last_peered":"2023-06-09T12:02:52.305110+0000","last_clean":"2023-06-09T12:02:52.305110+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:02:52.305110+0000","last_undegraded":"2023-06-09T12:02:52.305110+0000","last_fullsized":"2023-06-09T12:02:52.305110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":62871,"ondisk_log_size":62871,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30772,"ondisk_log_size":30772,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575505,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259664,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":79808,"kb_avail":94112176,"statfs":{"total":96636764160,"available":96370868224,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":81723392},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.66700000000000004},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.95999999999999996,"5min":0.96099999999999997,"15min":1.0640000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":1.256,"15min":1.3320000000000001},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.25},"last":0.745},{"interface":"front","average":{"1min":0.746,"5min":0.72599999999999998,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.05},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.95399999999999996,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.441,"15min":0.379},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.1200000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.32700000000000001},"max":{"1min":0.99299999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.044,"15min":1.0920000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.73099999999999998},"min":{"1min":0.57599999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":1.093,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.83299999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208216,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52096,"kb_avail":94163624,"statfs":{"total":96636764160,"available":96423550976,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53346304},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96799999999999997,"15min":1.069},"last":0.63}]},{"osd":1,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.60699999999999998,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.86199999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.745},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76600000000000001,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.754,"15min":0.70399999999999996},"min":{"1min":0.57499999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0369999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.93799999999999994,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.498,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0129999999999999,"15min":1.052},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":1.071},"last":0.505}]}]},{"osd":1,"up_from":13,"seq":55834575505,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405808,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99392,"kb_avail":93966032,"statfs":{"total":96636764160,"available":96221216768,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101777408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.438,"15min":0.41599999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.57799999999999996,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.48499999999999999,"5min":0.41099999999999998,"15min":0.36899999999999999},"max":{"1min":0.68700000000000006,"5min":0.82399999999999995,"15min":1.877},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.47799999999999998},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.28000000000000003},"max":{"1min":0.67700000000000005,"5min":0.995,"15min":1.982},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.49299999999999999,"15min":0.38700000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.635},"last":0.625}]},{"osd":4,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.609},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.81799999999999995},"min":{"1min":0.67800000000000005,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.2370000000000001,"15min":1.774},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83299999999999996,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.038,"5min":1.145,"15min":1.7390000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.878,"15min":0.83599999999999997},"min":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.55600000000000005},"max":{"1min":1.0629999999999999,"5min":1.0980000000000001,"15min":1.6879999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":0.99199999999999999,"5min":1.27,"15min":1.794},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.97799999999999998,"5min":1.1819999999999999,"15min":1.921},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89700000000000002,"15min":0.876},"min":{"1min":0.73699999999999999,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0069999999999999,"5min":1.2629999999999999,"15min":1.839},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.52100000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.827},"last":0.64300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575505,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332712,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":104896,"kb_avail":94039128,"statfs":{"total":96636764160,"available":96296067072,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107413504},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.442,"15min":0.441},"min":{"1min":0.32000000000000001,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.70799999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.46400000000000002,"5min":0.35799999999999998,"15min":0.34000000000000002},"max":{"1min":0.629,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56599999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.58699999999999997,"15min":0.54900000000000004},"min":{"1min":0.41199999999999998,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.81899999999999995,"5min":1.032,"15min":1.0900000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.52500000000000002,"15min":0.499},"min":{"1min":0.374,"5min":0.30599999999999999,"15min":0.27400000000000002},"max":{"1min":0.70999999999999996,"5min":0.80000000000000004,"15min":1.0289999999999999},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.38400000000000001,"5min":0.34599999999999997,"15min":0.33000000000000002},"max":{"1min":0.78200000000000003,"5min":0.91700000000000004,"15min":0.97399999999999998},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.70299999999999996},"min":{"1min":0.53100000000000003,"5min":0.48099999999999998,"15min":0.46400000000000002},"max":{"1min":0.85799999999999998,"5min":0.98899999999999999,"15min":1.024},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79400000000000004,"15min":0.71099999999999997},"min":{"1min":0.629,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.0569999999999999,"5min":1.175,"15min":1.175},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.69599999999999995,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0469999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.95799999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.749},"min":{"1min":0.495,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.94399999999999995,"5min":1.034,"15min":1.236},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83799999999999997,"15min":0.77200000000000002},"min":{"1min":0.68700000000000006,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.98899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.505,"15min":0.46899999999999997},"max":{"1min":0.98799999999999999,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83499999999999996},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0149999999999999,"5min":1.1100000000000001,"15min":1.3700000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:04:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84199999999999997,"15min":0.78900000000000003},"min":{"1min":0.59299999999999997,"5min":0.47699999999999998,"15min":0.47299999999999998},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.018},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.92300000000000004,"15min":0.84599999999999997},"min":{"1min":0.81299999999999994,"5min":0.60999999999999999,"15min":0.52400000000000002},"max":{"1min":1.0940000000000001,"5min":1.351,"15min":1.351},"last":1.0449999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575506,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242144,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62336,"kb_avail":94129696,"statfs":{"total":96636764160,"available":96388808704,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":63832064},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26800000000000002},"max":{"1min":0.70299999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.999,"15min":1.0940000000000001},"last":0.76900000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60999999999999999,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89200000000000002,"5min":0.92300000000000004,"15min":1.367},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.171,"15min":1.8220000000000001},"last":1.014},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":1.214,"15min":1.7370000000000001},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.79900000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":0.96799999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.83899999999999997},"min":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.95699999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.747,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.105,"15min":1.3540000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.0029999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.876},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83799999999999997,"15min":0.82299999999999995},"min":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.91100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575505,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422140,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":98944,"kb_avail":93949700,"statfs":{"total":96636764160,"available":96204492800,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101318656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.441},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53400000000000003,"15min":0.499},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.26900000000000002},"max":{"1min":0.70199999999999996,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.28599999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65400000000000003,"15min":0.64300000000000002},"min":{"1min":0.50600000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.355},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56899999999999995,"15min":0.54800000000000004},"min":{"1min":0.377,"5min":0.34100000000000003,"15min":0.29799999999999999},"max":{"1min":0.77900000000000003,"5min":0.91200000000000003,"15min":1.198},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79000000000000004,"15min":0.73199999999999998},"min":{"1min":0.60699999999999998,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":1.006,"5min":1.073,"15min":1.635},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82399999999999995,"15min":0.747},"min":{"1min":0.57199999999999995,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0980000000000001,"5min":3.081,"15min":3.081},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.76500000000000001},"min":{"1min":0.68700000000000006,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0920000000000001,"5min":1.198,"15min":1.2110000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.88100000000000001,"15min":0.81000000000000005},"min":{"1min":0.73699999999999999,"5min":0.52500000000000002,"15min":0.47899999999999998},"max":{"1min":1.133,"5min":1.234,"15min":1.367},"last":0.88}]},{"osd":6,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81499999999999995,"15min":0.78500000000000003},"min":{"1min":0.65900000000000003,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0149999999999999,"5min":1.2649999999999999,"15min":1.286},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.81899999999999995,"15min":0.77500000000000002},"min":{"1min":0.61399999999999999,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0229999999999999,"5min":1.103,"15min":3.3490000000000002},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:04:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.81200000000000006},"min":{"1min":0.63300000000000001,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.0700000000000001,"5min":1.306,"15min":1.413},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.88900000000000001,"15min":0.86199999999999999},"min":{"1min":0.64200000000000002,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.137,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":0.81299999999999994}]}]},{"osd":4,"up_from":13,"seq":55834575504,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423060,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130304,"kb_avail":93948780,"statfs":{"total":96636764160,"available":96203550720,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133431296},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.72799999999999998,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":1.0600000000000001,"5min":1.0640000000000001,"15min":1.137},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.746},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.76700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.0640000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.755},"min":{"1min":0.54400000000000004,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.95399999999999996,"5min":1.044,"15min":1.1279999999999999},"last":0.89500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.73299999999999998,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.34300000000000003},"max":{"1min":0.93899999999999995,"5min":1.117,"15min":1.609},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76500000000000001,"15min":0.68100000000000005},"min":{"1min":0.60099999999999998,"5min":0.56899999999999995,"15min":0.38100000000000001},"max":{"1min":1.0149999999999999,"5min":1.147,"15min":1.159},"last":0.82499999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.70599999999999996,"15min":0.67900000000000005},"min":{"1min":0.52900000000000003,"5min":0.47999999999999998,"15min":0.35899999999999999},"max":{"1min":0.999,"5min":1.014,"15min":1.407},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.73099999999999998},"min":{"1min":0.54700000000000004,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":0.86199999999999999,"5min":0.98799999999999999,"15min":1.4430000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55600000000000005,"15min":0.53300000000000003},"min":{"1min":0.47999999999999998,"5min":0.39300000000000002,"15min":0.30299999999999999},"max":{"1min":0.78900000000000003,"5min":0.95999999999999996,"15min":1.0349999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.47199999999999998,"15min":0.34100000000000003},"max":{"1min":0.98799999999999999,"5min":1.014,"15min":1.1719999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.56899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.69999999999999996},"min":{"1min":0.58699999999999997,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.3100000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:04:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.64900000000000002,"15min":0.60199999999999998},"min":{"1min":0.52800000000000002,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":0.94199999999999995,"15min":1.032},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67900000000000005,"15min":0.65700000000000003},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.38400000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.361},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575504,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293272,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":75904,"kb_avail":94078568,"statfs":{"total":96636764160,"available":96336453632,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77725696},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.96299999999999997,"5min":3.468,"15min":5.4969999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89200000000000002,"5min":1.177,"15min":5.6269999999999998},"last":0.755}]},{"osd":1,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":19.715},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.72699999999999998},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.127,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.621}]},{"osd":2,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":1.016,"5min":1.016,"15min":5.5620000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.49099999999999999,"15min":0.39200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":14.439},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.61699999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":12.130000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78400000000000003,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.81899999999999995},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.82899999999999996},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":67.521000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":1.097,"5min":1.097,"15min":12.163},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.95599999999999996,"5min":0.96299999999999997,"15min":3.8039999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.81799999999999995},"min":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.135,"15min":19.834},"last":0.78000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:05:14.893 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.893 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.894 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.894 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.894 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.895 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.895 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.895 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.896 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.896 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.897 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.897 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.897 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.898 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.898 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.898 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.899 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.899 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.900 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.900 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.900 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.901 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.901 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.901 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.902 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.902 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.902 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.903 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.903 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.904 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.904 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.904 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.905 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.905 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.905 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.906 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.906 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.906 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.907 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.907 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.908 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.908 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.908 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.909 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.909 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.910 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.910 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.910 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.911 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.911 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.911 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.912 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.912 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.912 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.913 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.913 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.914 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.914 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.914 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.915 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.915 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.915 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.916 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.916 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.916 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:14.917 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:05:27.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:27.315 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:34.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:05:35.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.126+0000 7fd0bc1ff700 1 Processor -- start 2023-06-09T12:05:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.126+0000 7fd0bc1ff700 1 -- start start 2023-06-09T12:05:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0bc1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd0b415b060 0x7fd0b40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0bc1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 0x7fd0b4163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0bc1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0b40c9bd0 con 0x7fd0b40c2ae0 2023-06-09T12:05:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0bc1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0b4159360 con 0x7fd0b415b060 2023-06-09T12:05:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0bc1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b40c8e20 con 0x7fd0b40c88e0 2023-06-09T12:05:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0b9f9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd0b40c2ae0 0x7fd0b415a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58234/0 (socket says 172.21.15.7:58234) 2023-06-09T12:05:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0b9f9b700 1 -- 172.21.15.7:0/1655390406 learned_addr learned my addr 172.21.15.7:0/1655390406 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:05:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1655390406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 0x7fd0b4163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0b979a700 1 --2- 172.21.15.7:0/1655390406 >> v2:172.21.15.117:3300/0 conn(0x7fd0b415b060 0x7fd0b40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0b8f99700 1 -- 172.21.15.7:0/1655390406 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635245785 0 0) 0x7fd0b40c9bd0 con 0x7fd0b40c2ae0 2023-06-09T12:05:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.128+0000 7fd0b8f99700 1 -- 172.21.15.7:0/1655390406 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a00035b0 con 0x7fd0b40c2ae0 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1655390406 >> v2:172.21.15.117:3300/0 conn(0x7fd0b415b060 msgr2=0x7fd0b40c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1655390406 >> v2:172.21.15.117:3300/0 conn(0x7fd0b415b060 0x7fd0b40c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1655390406 >> v1:172.21.15.7:6789/0 conn(0x7fd0b40c2ae0 legacy=0x7fd0b415a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1655390406 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b4159570 con 0x7fd0b40c88e0 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1655390406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 0x7fd0b4163ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009d90 tx=0x7fd0ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8fe603484c56736 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0b8f99700 1 -- 172.21.15.7:0/1655390406 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00e070 con 0x7fd0b40c88e0 2023-06-09T12:05:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0b8f99700 1 -- 172.21.15.7:0/1655390406 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac00bb20 con 0x7fd0b40c88e0 2023-06-09T12:05:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.129+0000 7fd0b8f99700 1 -- 172.21.15.7:0/1655390406 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac00be90 con 0x7fd0b40c88e0 2023-06-09T12:05:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1655390406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 msgr2=0x7fd0b4163ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1655390406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 0x7fd0b4163ce0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009d90 tx=0x7fd0ac00b330 comp rx=0 tx=0).stop 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1655390406 shutdown_connections 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1655390406 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c88e0 0x7fd0b4163ce0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1655390406 >> v2:172.21.15.117:3300/0 conn(0x7fd0b415b060 0x7fd0b40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1655390406 >> 172.21.15.7:0/1655390406 conn(0x7fd0b40cd330 msgr2=0x7fd0b40cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1655390406 shutdown_connections 2023-06-09T12:05:35.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.130+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1655390406 wait complete. 2023-06-09T12:05:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.132+0000 7fd0bc1ff700 1 Processor -- start 2023-06-09T12:05:35.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.132+0000 7fd0bc1ff700 1 -- start start 2023-06-09T12:05:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c2ae0 0x7fd0b41f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40c88e0 0x7fd0b4144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b41ee780 con 0x7fd0b415b060 2023-06-09T12:05:35.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b4145520 con 0x7fd0b40c88e0 2023-06-09T12:05:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0bc1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b41e2b20 con 0x7fd0b40c2ae0 2023-06-09T12:05:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0b9f9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c2ae0 0x7fd0b41f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41702/0 (socket says 172.21.15.7:41702) 2023-06-09T12:05:35.141 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1182329201 learned_addr learned my addr 172.21.15.7:0/1182329201 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:05:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0b979a700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40c88e0 0x7fd0b4144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c2ae0 msgr2=0x7fd0b41f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c2ae0 0x7fd0b41f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40c88e0 msgr2=0x7fd0b4144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.142 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40c88e0 0x7fd0b4144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.133+0000 7fd0ba79c700 1 -- 172.21.15.7:0/1182329201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac009a70 con 0x7fd0b415b060 2023-06-09T12:05:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0ba79c700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac014040 tx=0x7fd0ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac004800 con 0x7fd0b415b060 2023-06-09T12:05:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0ac020070 con 0x7fd0b415b060 2023-06-09T12:05:35.143 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b4159af0 con 0x7fd0b415b060 2023-06-09T12:05:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0276c0 con 0x7fd0b415b060 2023-06-09T12:05:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.134+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b4159810 con 0x7fd0b415b060 2023-06-09T12:05:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.135+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b4159dd0 con 0x7fd0b415b060 2023-06-09T12:05:35.144 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.136+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0ac012430 con 0x7fd0b415b060 2023-06-09T12:05:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.137+0000 7fd0aa7fc700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 0x7fd09806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.137+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fd0ac022070 con 0x7fd0b415b060 2023-06-09T12:05:35.145 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.137+0000 7fd0b9f9b700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 0x7fd09806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:35.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.138+0000 7fd0b9f9b700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 0x7fd09806d190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007fd0 tx=0x7fd0b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:35.146 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.142+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd0ac056070 con 0x7fd0b415b060 2023-06-09T12:05:35.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.345+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0b41ee5e0 con 0x7fd09806ace0 2023-06-09T12:05:35.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.355+0000 7fd0aa7fc700 1 -- 172.21.15.7:0/1182329201 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315863 (secure 0 0 0) 0x7fd0b000f690 con 0x7fd09806ace0 2023-06-09T12:05:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:05:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 msgr2=0x7fd09806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 0x7fd09806d190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007fd0 tx=0x7fd0b0009040 comp rx=0 tx=0).stop 2023-06-09T12:05:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 msgr2=0x7fd0b41eec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:35.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac014040 tx=0x7fd0ac004ad0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 shutdown_connections 2023-06-09T12:05:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0b415b060 0x7fd0b41eec10 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd09806ace0 0x7fd09806d190 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0b40c2ae0 0x7fd0b41f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 --2- 172.21.15.7:0/1182329201 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0b40c88e0 0x7fd0b4144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 >> 172.21.15.7:0/1182329201 conn(0x7fd0b40cd330 msgr2=0x7fd0b40cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 shutdown_connections 2023-06-09T12:05:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:35.359+0000 7fd0bc1ff700 1 -- 172.21.15.7:0/1182329201 wait complete. 2023-06-09T12:05:35.365 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:05:35.381 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1714,"stamp":"2023-06-09T12:05:35.023853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94272,"ondisk_log_size":94272,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2588168,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":704832,"kb_avail":752386552,"statfs":{"total":773094113280,"available":770443829248,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":721747968},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003003"},"pg_stats":[{"pgid":"4.3f","version":"75'927","reported_seq":3472,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:44.696195+0000","last_change":"2023-06-09T12:05:02.403019+0000","last_active":"2023-06-09T12:04:44.696195+0000","last_peered":"2023-06-09T12:04:44.696195+0000","last_clean":"2023-06-09T12:04:44.696195+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:04:44.696195+0000","last_undegraded":"2023-06-09T12:04:44.696195+0000","last_fullsized":"2023-06-09T12:04:44.696195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3223,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.076031+0000","last_change":"2023-06-09T12:05:02.350871+0000","last_active":"2023-06-09T12:04:42.076031+0000","last_peered":"2023-06-09T12:04:42.076031+0000","last_clean":"2023-06-09T12:04:42.076031+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:04:42.076031+0000","last_undegraded":"2023-06-09T12:04:42.076031+0000","last_fullsized":"2023-06-09T12:04:42.076031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.327337+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:03.327337+0000","last_peered":"2023-06-09T12:03:03.327337+0000","last_clean":"2023-06-09T12:03:03.327337+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:03.327337+0000","last_undegraded":"2023-06-09T12:03:03.327337+0000","last_fullsized":"2023-06-09T12:03:03.327337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3605,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:51.403386+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:00:51.403386+0000","last_peered":"2023-06-09T12:00:51.403386+0000","last_clean":"2023-06-09T12:00:51.403386+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:00:51.403386+0000","last_undegraded":"2023-06-09T12:00:51.403386+0000","last_fullsized":"2023-06-09T12:00:51.403386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1084","reported_seq":3932,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:31.354661+0000","last_change":"2023-06-09T12:04:57.102635+0000","last_active":"2023-06-09T12:05:31.354661+0000","last_peered":"2023-06-09T12:05:31.354661+0000","last_clean":"2023-06-09T12:05:31.354661+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:31.354661+0000","last_undegraded":"2023-06-09T12:05:31.354661+0000","last_fullsized":"2023-06-09T12:05:31.354661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":222,"log_size":1084,"log_dups_size":0,"ondisk_log_size":1084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 35s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3409,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:31.748046+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:31.748046+0000","last_peered":"2023-06-09T12:04:31.748046+0000","last_clean":"2023-06-09T12:04:31.748046+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:31.748046+0000","last_undegraded":"2023-06-09T12:04:31.748046+0000","last_fullsized":"2023-06-09T12:04:31.748046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3737,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.370675+0000","last_change":"2023-06-09T12:04:32.323061+0000","last_active":"2023-06-09T12:03:55.370675+0000","last_peered":"2023-06-09T12:03:55.370675+0000","last_clean":"2023-06-09T12:03:55.370675+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:03:55.370675+0000","last_undegraded":"2023-06-09T12:03:55.370675+0000","last_fullsized":"2023-06-09T12:03:55.370675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.422718+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T12:02:12.422718+0000","last_peered":"2023-06-09T12:02:12.422718+0000","last_clean":"2023-06-09T12:02:12.422718+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:02:12.422718+0000","last_undegraded":"2023-06-09T12:02:12.422718+0000","last_fullsized":"2023-06-09T12:02:12.422718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:29.655122+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:01:29.655122+0000","last_peered":"2023-06-09T12:01:29.655122+0000","last_clean":"2023-06-09T12:01:29.655122+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:01:29.655122+0000","last_undegraded":"2023-06-09T12:01:29.655122+0000","last_fullsized":"2023-06-09T12:01:29.655122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.308346+0000","last_change":"2023-06-09T12:02:37.308346+0000","last_active":"2023-06-09T12:02:37.308346+0000","last_peered":"2023-06-09T12:02:37.308346+0000","last_clean":"2023-06-09T12:02:37.308346+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:02:37.308346+0000","last_undegraded":"2023-06-09T12:02:37.308346+0000","last_fullsized":"2023-06-09T12:02:37.308346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.278802+0000","last_change":"2023-06-09T12:05:02.350828+0000","last_active":"2023-06-09T12:03:46.278802+0000","last_peered":"2023-06-09T12:03:46.278802+0000","last_clean":"2023-06-09T12:03:46.278802+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:03:46.278802+0000","last_undegraded":"2023-06-09T12:03:46.278802+0000","last_fullsized":"2023-06-09T12:03:46.278802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:17.636747+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:01:17.636747+0000","last_peered":"2023-06-09T12:01:17.636747+0000","last_clean":"2023-06-09T12:01:17.636747+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:01:17.636747+0000","last_undegraded":"2023-06-09T12:01:17.636747+0000","last_fullsized":"2023-06-09T12:01:17.636747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:35.754891+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:01:35.754891+0000","last_peered":"2023-06-09T12:01:35.754891+0000","last_clean":"2023-06-09T12:01:35.754891+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:01:35.754891+0000","last_undegraded":"2023-06-09T12:01:35.754891+0000","last_fullsized":"2023-06-09T12:01:35.754891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2265,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:54.225246+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:00:54.225246+0000","last_peered":"2023-06-09T12:00:54.225246+0000","last_clean":"2023-06-09T12:00:54.225246+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:00:54.225246+0000","last_undegraded":"2023-06-09T12:00:54.225246+0000","last_fullsized":"2023-06-09T12:00:54.225246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.311625+0000","last_change":"2023-06-09T12:03:02.311625+0000","last_active":"2023-06-09T12:03:02.311625+0000","last_peered":"2023-06-09T12:03:02.311625+0000","last_clean":"2023-06-09T12:03:02.311625+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:03:02.311625+0000","last_undegraded":"2023-06-09T12:03:02.311625+0000","last_fullsized":"2023-06-09T12:03:02.311625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3311,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.082714+0000","last_change":"2023-06-09T12:05:02.520050+0000","last_active":"2023-06-09T12:03:58.082714+0000","last_peered":"2023-06-09T12:03:58.082714+0000","last_clean":"2023-06-09T12:03:58.082714+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:58.082714+0000","last_undegraded":"2023-06-09T12:03:58.082714+0000","last_fullsized":"2023-06-09T12:03:58.082714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:13.801581+0000","last_change":"2023-06-09T12:04:32.318108+0000","last_active":"2023-06-09T12:03:13.801581+0000","last_peered":"2023-06-09T12:03:13.801581+0000","last_clean":"2023-06-09T12:03:13.801581+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:03:13.801581+0000","last_undegraded":"2023-06-09T12:03:13.801581+0000","last_fullsized":"2023-06-09T12:03:13.801581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.161159+0000","last_change":"2023-06-09T12:04:32.424054+0000","last_active":"2023-06-09T12:02:49.161159+0000","last_peered":"2023-06-09T12:02:49.161159+0000","last_clean":"2023-06-09T12:02:49.161159+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:49.161159+0000","last_undegraded":"2023-06-09T12:02:49.161159+0000","last_fullsized":"2023-06-09T12:02:49.161159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3561,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313831+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:02:22.313831+0000","last_peered":"2023-06-09T12:02:22.313831+0000","last_clean":"2023-06-09T12:02:22.313831+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:02:22.313831+0000","last_undegraded":"2023-06-09T12:02:22.313831+0000","last_fullsized":"2023-06-09T12:02:22.313831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:43.259734+0000","last_change":"2023-06-09T12:04:02.343332+0000","last_active":"2023-06-09T12:03:43.259734+0000","last_peered":"2023-06-09T12:03:43.259734+0000","last_clean":"2023-06-09T12:03:43.259734+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:03:43.259734+0000","last_undegraded":"2023-06-09T12:03:43.259734+0000","last_fullsized":"2023-06-09T12:03:43.259734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:54.914290+0000","last_change":"2023-06-09T12:04:02.319111+0000","last_active":"2023-06-09T12:01:54.914290+0000","last_peered":"2023-06-09T12:01:54.914290+0000","last_clean":"2023-06-09T12:01:54.914290+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:01:54.914290+0000","last_undegraded":"2023-06-09T12:01:54.914290+0000","last_fullsized":"2023-06-09T12:01:54.914290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.198865+0000","last_change":"2023-06-09T12:04:57.427044+0000","last_active":"2023-06-09T12:02:50.198865+0000","last_peered":"2023-06-09T12:02:50.198865+0000","last_clean":"2023-06-09T12:02:50.198865+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:50.198865+0000","last_undegraded":"2023-06-09T12:02:50.198865+0000","last_fullsized":"2023-06-09T12:02:50.198865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3253,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404017+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404017+0000","last_peered":"2023-06-09T12:03:36.404017+0000","last_clean":"2023-06-09T12:03:36.404017+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404017+0000","last_undegraded":"2023-06-09T12:03:36.404017+0000","last_fullsized":"2023-06-09T12:03:36.404017+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428433+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:02:52.428433+0000","last_peered":"2023-06-09T12:02:52.428433+0000","last_clean":"2023-06-09T12:02:52.428433+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:02:52.428433+0000","last_undegraded":"2023-06-09T12:02:52.428433+0000","last_fullsized":"2023-06-09T12:02:52.428433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3544,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.039481+0000","last_change":"2023-06-09T12:05:12.521142+0000","last_active":"2023-06-09T12:03:55.039481+0000","last_peered":"2023-06-09T12:03:55.039481+0000","last_clean":"2023-06-09T12:03:55.039481+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:55.039481+0000","last_undegraded":"2023-06-09T12:03:55.039481+0000","last_fullsized":"2023-06-09T12:03:55.039481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:51.058132+0000","last_change":"2023-06-09T12:04:52.349590+0000","last_active":"2023-06-09T12:04:51.058132+0000","last_peered":"2023-06-09T12:04:51.058132+0000","last_clean":"2023-06-09T12:04:51.058132+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:04:51.058132+0000","last_undegraded":"2023-06-09T12:04:51.058132+0000","last_fullsized":"2023-06-09T12:04:51.058132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:53.325190+0000","last_change":"2023-06-09T12:04:52.325633+0000","last_active":"2023-06-09T12:03:53.325190+0000","last_peered":"2023-06-09T12:03:53.325190+0000","last_clean":"2023-06-09T12:03:53.325190+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:03:53.325190+0000","last_undegraded":"2023-06-09T12:03:53.325190+0000","last_fullsized":"2023-06-09T12:03:53.325190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.426543+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:02:37.426543+0000","last_peered":"2023-06-09T12:02:37.426543+0000","last_clean":"2023-06-09T12:02:37.426543+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:02:37.426543+0000","last_undegraded":"2023-06-09T12:02:37.426543+0000","last_fullsized":"2023-06-09T12:02:37.426543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":757,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334635+0000","last_change":"2023-06-09T12:05:30.983184+0000","last_active":"2023-06-09T12:05:02.334635+0000","last_peered":"2023-06-09T12:05:02.334635+0000","last_clean":"2023-06-09T12:05:02.334635+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:02.334635+0000","last_undegraded":"2023-06-09T12:05:02.334635+0000","last_fullsized":"2023-06-09T12:05:02.334635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:17.778068+0000","last_change":"2023-06-09T12:05:02.322556+0000","last_active":"2023-06-09T12:03:17.778068+0000","last_peered":"2023-06-09T12:03:17.778068+0000","last_clean":"2023-06-09T12:03:17.778068+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:03:17.778068+0000","last_undegraded":"2023-06-09T12:03:17.778068+0000","last_fullsized":"2023-06-09T12:03:17.778068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:48.020256+0000","last_change":"2023-06-09T12:04:57.350272+0000","last_active":"2023-06-09T12:04:48.020256+0000","last_peered":"2023-06-09T12:04:48.020256+0000","last_clean":"2023-06-09T12:04:48.020256+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:04:48.020256+0000","last_undegraded":"2023-06-09T12:04:48.020256+0000","last_fullsized":"2023-06-09T12:04:48.020256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:53.763053+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:01:53.763053+0000","last_peered":"2023-06-09T12:01:53.763053+0000","last_clean":"2023-06-09T12:01:53.763053+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:01:53.763053+0000","last_undegraded":"2023-06-09T12:01:53.763053+0000","last_fullsized":"2023-06-09T12:01:53.763053+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1057","reported_seq":3407,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:29.840130+0000","last_change":"2023-06-09T12:05:16.047851+0000","last_active":"2023-06-09T12:05:29.840130+0000","last_peered":"2023-06-09T12:05:29.840130+0000","last_clean":"2023-06-09T12:05:29.840130+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:29.840130+0000","last_undegraded":"2023-06-09T12:05:29.840130+0000","last_fullsized":"2023-06-09T12:05:29.840130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":96,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.412288+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:52.412288+0000","last_peered":"2023-06-09T12:02:52.412288+0000","last_clean":"2023-06-09T12:02:52.412288+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:52.412288+0000","last_undegraded":"2023-06-09T12:02:52.412288+0000","last_fullsized":"2023-06-09T12:02:52.412288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2990,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:01.358848+0000","last_change":"2023-06-09T12:05:02.326837+0000","last_active":"2023-06-09T12:04:01.358848+0000","last_peered":"2023-06-09T12:04:01.358848+0000","last_clean":"2023-06-09T12:04:01.358848+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:04:01.358848+0000","last_undegraded":"2023-06-09T12:04:01.358848+0000","last_fullsized":"2023-06-09T12:04:01.358848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:54.261006+0000","last_change":"2023-06-09T12:05:02.326991+0000","last_active":"2023-06-09T12:04:54.261006+0000","last_peered":"2023-06-09T12:04:54.261006+0000","last_clean":"2023-06-09T12:04:54.261006+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:04:54.261006+0000","last_undegraded":"2023-06-09T12:04:54.261006+0000","last_fullsized":"2023-06-09T12:04:54.261006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.310949+0000","last_change":"2023-06-09T12:02:57.310949+0000","last_active":"2023-06-09T12:02:57.310949+0000","last_peered":"2023-06-09T12:02:57.310949+0000","last_clean":"2023-06-09T12:02:57.310949+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:02:57.310949+0000","last_undegraded":"2023-06-09T12:02:57.310949+0000","last_fullsized":"2023-06-09T12:02:57.310949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3371,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.318985+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:03:02.318985+0000","last_peered":"2023-06-09T12:03:02.318985+0000","last_clean":"2023-06-09T12:03:02.318985+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:03:02.318985+0000","last_undegraded":"2023-06-09T12:03:02.318985+0000","last_fullsized":"2023-06-09T12:03:02.318985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.112083+0000","last_change":"2023-06-09T12:04:57.519309+0000","last_active":"2023-06-09T12:03:57.112083+0000","last_peered":"2023-06-09T12:03:57.112083+0000","last_clean":"2023-06-09T12:03:57.112083+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:03:57.112083+0000","last_undegraded":"2023-06-09T12:03:57.112083+0000","last_fullsized":"2023-06-09T12:03:57.112083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:28.374029+0000","last_change":"2023-06-09T12:05:02.326978+0000","last_active":"2023-06-09T12:04:28.374029+0000","last_peered":"2023-06-09T12:04:28.374029+0000","last_clean":"2023-06-09T12:04:28.374029+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:04:28.374029+0000","last_undegraded":"2023-06-09T12:04:28.374029+0000","last_fullsized":"2023-06-09T12:04:28.374029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":884,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:22.314299+0000","last_change":"2023-06-09T12:04:52.346421+0000","last_active":"2023-06-09T12:03:22.314299+0000","last_peered":"2023-06-09T12:03:22.314299+0000","last_clean":"2023-06-09T12:03:22.314299+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:03:22.314299+0000","last_undegraded":"2023-06-09T12:03:22.314299+0000","last_fullsized":"2023-06-09T12:03:22.314299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:00:44.355731+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:00:44.355731+0000","last_peered":"2023-06-09T12:00:44.355731+0000","last_clean":"2023-06-09T12:00:44.355731+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:00:44.355731+0000","last_undegraded":"2023-06-09T12:00:44.355731+0000","last_fullsized":"2023-06-09T12:00:44.355731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":771,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:03:22.310323+0000","last_change":"2023-06-09T12:04:53.107623+0000","last_active":"2023-06-09T12:03:22.310323+0000","last_peered":"2023-06-09T12:03:22.310323+0000","last_clean":"2023-06-09T12:03:22.310323+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:03:22.310323+0000","last_undegraded":"2023-06-09T12:03:22.310323+0000","last_fullsized":"2023-06-09T12:03:22.310323+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3677,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.908103+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:02:52.908103+0000","last_peered":"2023-06-09T12:02:52.908103+0000","last_clean":"2023-06-09T12:02:52.908103+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:02:52.908103+0000","last_undegraded":"2023-06-09T12:02:52.908103+0000","last_fullsized":"2023-06-09T12:02:52.908103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.124241+0000","last_change":"2023-06-09T12:05:02.519982+0000","last_active":"2023-06-09T12:03:59.124241+0000","last_peered":"2023-06-09T12:03:59.124241+0000","last_clean":"2023-06-09T12:03:59.124241+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:59.124241+0000","last_undegraded":"2023-06-09T12:03:59.124241+0000","last_fullsized":"2023-06-09T12:03:59.124241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'964","reported_seq":3102,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:31.021470+0000","last_change":"2023-06-09T12:05:18.721187+0000","last_active":"2023-06-09T12:05:31.021470+0000","last_peered":"2023-06-09T12:05:31.021470+0000","last_clean":"2023-06-09T12:05:31.021470+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:31.021470+0000","last_undegraded":"2023-06-09T12:05:31.021470+0000","last_fullsized":"2023-06-09T12:05:31.021470+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":98,"log_size":964,"log_dups_size":0,"ondisk_log_size":964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 12s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":63381,"ondisk_log_size":63381,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30772,"ondisk_log_size":30772,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575509,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259792,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":79936,"kb_avail":94112048,"statfs":{"total":96636764160,"available":96370737152,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":81854464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.70799999999999996,"15min":0.66700000000000004},"min":{"1min":0.46800000000000003,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.95999999999999996,"5min":0.96099999999999997,"15min":1.0640000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.44,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":0.92700000000000005,"5min":1.256,"15min":1.3320000000000001},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.56299999999999994,"5min":0.41999999999999998,"15min":0.28199999999999997},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.25},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.746,"5min":0.72599999999999998,"15min":0.69399999999999995},"min":{"1min":0.53700000000000003,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.05},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.749,"15min":0.72699999999999998},"min":{"1min":0.57599999999999996,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":0.95399999999999996,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.67500000000000004},"min":{"1min":0.54800000000000004,"5min":0.441,"15min":0.379},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.627}]},{"osd":4,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.32900000000000001,"15min":0.28199999999999997},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":1.1200000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73399999999999999,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.32700000000000001},"max":{"1min":0.99299999999999999,"5min":1.0189999999999999,"15min":1.1579999999999999},"last":0.65100000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.624,"15min":0.60499999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.85399999999999998,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63800000000000001,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.32400000000000001},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:04:33 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66200000000000003,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.0369999999999999,"5min":1.044,"15min":1.0920000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.73099999999999998},"min":{"1min":0.57599999999999996,"5min":0.504,"15min":0.42599999999999999},"max":{"1min":1.093,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.57799999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575509,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208344,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52224,"kb_avail":94163496,"statfs":{"total":96636764160,"available":96423419904,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53477376},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66200000000000003,"15min":0.63600000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.79700000000000004,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68400000000000005,"15min":0.64400000000000002},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.86099999999999999,"5min":0.96799999999999997,"15min":1.069},"last":0.626}]},{"osd":1,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71199999999999997,"15min":0.67000000000000004},"min":{"1min":0.60699999999999998,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.92500000000000004,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.76100000000000001,"15min":0.71999999999999997},"min":{"1min":0.67200000000000004,"5min":0.56499999999999995,"15min":0.40699999999999997},"max":{"1min":0.96899999999999997,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.71699999999999997,"15min":0.68100000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.86199999999999999,"5min":0.98499999999999999,"15min":1.024},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76600000000000001,"15min":0.72899999999999998},"min":{"1min":0.56200000000000006,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.754,"15min":0.70399999999999996},"min":{"1min":0.57499999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"max":{"1min":0.94699999999999995,"5min":1.03,"15min":1.0369999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.80000000000000004,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.93799999999999994,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60199999999999998,"15min":0.57499999999999996},"min":{"1min":0.46800000000000003,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84299999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.438},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58399999999999996,"15min":0.56200000000000006},"min":{"1min":0.47899999999999998,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.76400000000000001,"5min":0.872,"15min":0.872},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.35199999999999998},"max":{"1min":0.78700000000000003,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.67000000000000004,"15min":0.64100000000000001},"min":{"1min":0.498,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.97599999999999998,"5min":1.0129999999999999,"15min":1.052},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.99299999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.65600000000000003,"15min":0.64400000000000002},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.93100000000000005,"5min":0.97299999999999998,"15min":1.071},"last":0.52900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575509,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406000,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99584,"kb_avail":93965840,"statfs":{"total":96636764160,"available":96221020160,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101974016},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.438,"15min":0.41599999999999998},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.248},"max":{"1min":0.57799999999999996,"5min":0.71399999999999997,"15min":1.7270000000000001},"last":0.438},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.48499999999999999,"5min":0.41099999999999998,"15min":0.36899999999999999},"max":{"1min":0.68700000000000006,"5min":0.82399999999999995,"15min":1.877},"last":0.60699999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.503,"15min":0.47799999999999998},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.28000000000000003},"max":{"1min":0.67700000000000005,"5min":0.995,"15min":1.982},"last":0.499},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56100000000000005,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35699999999999998,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":0.86099999999999999,"15min":1.5049999999999999},"last":0.51300000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.45000000000000001,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":1.577},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.49299999999999999,"15min":0.38700000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.635},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.748,"15min":0.69899999999999995},"min":{"1min":0.48599999999999999,"5min":0.45700000000000002,"15min":0.36099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0569999999999999,"15min":1.609},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84699999999999998,"15min":0.81799999999999995},"min":{"1min":0.67800000000000005,"5min":0.54100000000000004,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.2370000000000001,"15min":1.774},"last":0.98199999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.83299999999999996,"15min":0.79400000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.038,"5min":1.145,"15min":1.7390000000000001},"last":1.012},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.878,"15min":0.83599999999999997},"min":{"1min":0.68899999999999995,"5min":0.64600000000000002,"15min":0.55600000000000005},"max":{"1min":1.0629999999999999,"5min":1.0980000000000001,"15min":1.6879999999999999},"last":0.89200000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.82999999999999996},"min":{"1min":0.60599999999999998,"5min":0.58399999999999996,"15min":0.504},"max":{"1min":0.99199999999999999,"5min":1.27,"15min":1.794},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81499999999999995,"15min":0.76900000000000002},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.97799999999999998,"5min":1.1819999999999999,"15min":1.921},"last":0.85799999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:04:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.89700000000000002,"15min":0.876},"min":{"1min":0.73699999999999999,"5min":0.58399999999999996,"15min":0.56299999999999994},"max":{"1min":1.0069999999999999,"5min":1.2629999999999999,"15min":1.839},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.52100000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.355,"5min":1.355,"15min":1.827},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575509,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332904,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":105088,"kb_avail":94038936,"statfs":{"total":96636764160,"available":96295870464,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107610112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.443,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.63900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.72299999999999998,"5min":1.032,"15min":1.032},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":1.024},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80700000000000005,"15min":0.72099999999999997},"min":{"1min":0.68300000000000005,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":0.99299999999999999,"5min":1.175,"15min":1.175},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.86199999999999999,"15min":0.79900000000000004},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.879}]},{"osd":5,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.72999999999999998,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.012,"5min":1.034,"15min":1.236},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.85899999999999999,"15min":0.78000000000000003},"min":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":1.103,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.90700000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.083,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.83199999999999996},"min":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.1100000000000001,"15min":1.3149999999999999},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85899999999999999,"15min":0.79300000000000004},"min":{"1min":0.69799999999999995,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.1100000000000001,"5min":1.298,"15min":1.298},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94099999999999995,"15min":0.85699999999999998},"min":{"1min":0.84599999999999997,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1359999999999999,"5min":1.351,"15min":1.351},"last":0.93600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575510,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242272,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62464,"kb_avail":94129568,"statfs":{"total":96636764160,"available":96388677632,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":63963136},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45200000000000001,"15min":0.42599999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26800000000000002},"max":{"1min":0.70299999999999996,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54800000000000004,"15min":0.51600000000000001},"min":{"1min":0.45400000000000001,"5min":0.41099999999999998,"15min":0.33000000000000002},"max":{"1min":0.71299999999999997,"5min":0.999,"15min":1.0940000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.45700000000000002},"min":{"1min":0.38100000000000001,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.60999999999999999,"5min":0.84799999999999998,"15min":1.3109999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63400000000000001,"15min":0.60399999999999998},"min":{"1min":0.47999999999999998,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89200000000000002,"5min":0.92300000000000004,"15min":1.367},"last":0.47199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.60399999999999998,"15min":0.58399999999999996},"min":{"1min":0.48399999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.79000000000000004,"5min":0.96799999999999997,"15min":1.546},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67900000000000005,"15min":0.63800000000000001},"min":{"1min":0.54800000000000004,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":0.86099999999999999,"5min":0.98899999999999999,"15min":1.4850000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76500000000000001,"15min":0.74199999999999999},"min":{"1min":0.60199999999999998,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.88500000000000001,"5min":1.171,"15min":1.8220000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.78200000000000003,"15min":0.76200000000000001},"min":{"1min":0.56000000000000005,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.95799999999999996,"5min":1.214,"15min":1.7370000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82199999999999995,"15min":0.79900000000000004},"min":{"1min":0.61299999999999999,"5min":0.52800000000000002,"15min":0.51000000000000001},"max":{"1min":0.96799999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86799999999999999,"15min":0.83899999999999997},"min":{"1min":0.69799999999999995,"5min":0.60899999999999999,"15min":0.53400000000000003},"max":{"1min":1.0189999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.77000000000000002},"min":{"1min":0.59899999999999998,"5min":0.53700000000000003,"15min":0.438},"max":{"1min":0.97699999999999998,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.77700000000000002,"15min":0.76100000000000001},"min":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":0.97499999999999998,"5min":1.105,"15min":1.3540000000000001},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82799999999999996,"15min":0.79500000000000004},"min":{"1min":0.60299999999999998,"5min":0.53600000000000003,"15min":0.497},"max":{"1min":1.0029999999999999,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.83799999999999997,"15min":0.82299999999999995},"min":{"1min":0.64500000000000002,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.01,"5min":1.262,"15min":1.3080000000000001},"last":0.72299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575509,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422268,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":99072,"kb_avail":93949572,"statfs":{"total":96636764160,"available":96204361728,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101449728},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:31 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48199999999999998,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.44400000000000001,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.745,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.44}]},{"osd":1,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.26900000000000002},"max":{"1min":0.81999999999999995,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77400000000000002,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.52300000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":0.97099999999999997,"15min":1.355},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.85899999999999999,"5min":0.91200000000000003,"15min":1.198},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79600000000000004,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.073,"15min":1.635},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0369999999999999,"5min":1.0980000000000001,"15min":3.081},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83099999999999996,"15min":0.77400000000000002},"min":{"1min":0.66300000000000003,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.165,"15min":1.2110000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88400000000000001,"15min":0.81399999999999995},"min":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.47899999999999998},"max":{"1min":1.095,"5min":1.1970000000000001,"15min":1.367},"last":0.88100000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.78300000000000003},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.94799999999999995,"5min":1.2649999999999999,"15min":1.286},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":0.77900000000000003},"min":{"1min":0.65800000000000003,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":3.3490000000000002},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.81000000000000005},"min":{"1min":0.628,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.306,"15min":1.413},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.70299999999999996,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":3.1160000000000001},"last":0.78300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575508,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423188,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130432,"kb_avail":93948652,"statfs":{"total":96636764160,"available":96203419648,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133562368},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72399999999999998,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":1.0640000000000001,"15min":1.137},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78700000000000003,"15min":0.749},"min":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":0.85199999999999998,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.79900000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75,"15min":0.72099999999999997},"min":{"1min":0.63800000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.0640000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":1.03,"15min":1.1279999999999999},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73299999999999998,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83299999999999996,"5min":1.117,"15min":1.609},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.68899999999999995},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.38100000000000001},"max":{"1min":1.0129999999999999,"5min":1.147,"15min":1.159},"last":1.107}]},{"osd":3,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":1.014,"15min":1.407},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.1719999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.131,"5min":1.131,"15min":1.3100000000000001},"last":0.95799999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.361},"last":0.68200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575508,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293400,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76032,"kb_avail":94078440,"statfs":{"total":96636764160,"available":96336322560,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77856768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.96299999999999997,"5min":3.468,"15min":5.4969999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70799999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.40999999999999998,"15min":0.318},"max":{"1min":0.89200000000000002,"5min":1.177,"15min":5.6269999999999998},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.41199999999999998,"5min":0.377,"15min":0.29899999999999999},"max":{"1min":1.083,"5min":1.083,"15min":19.715},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.72699999999999998},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.33400000000000002},"max":{"1min":1.127,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.71299999999999997,"15min":0.69799999999999995},"min":{"1min":0.56100000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":1.016,"5min":1.016,"15min":5.5620000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.49099999999999999,"15min":0.39200000000000002},"max":{"1min":1.145,"5min":1.145,"15min":14.439},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.61699999999999999,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.99299999999999999,"5min":1.006,"15min":12.130000000000001},"last":0.5},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78400000000000003,"15min":0.81499999999999995},"min":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55700000000000005,"15min":0.81899999999999995},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59099999999999997,"15min":0.82899999999999996},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.85599999999999998,"5min":0.85799999999999998,"15min":67.521000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.68000000000000005},"min":{"1min":0.46100000000000002,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":1.097,"5min":1.097,"15min":12.163},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.49199999999999999,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.64100000000000001,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.44400000000000001,"15min":0.39300000000000002},"max":{"1min":0.95599999999999996,"5min":0.96299999999999997,"15min":3.8039999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76000000000000001,"15min":0.81799999999999995},"min":{"1min":0.56699999999999995,"5min":0.54900000000000004,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.135,"15min":19.834},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:05:35.388 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.388 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.389 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.389 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.390 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.390 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.390 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.391 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.391 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.391 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.392 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.392 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.392 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.393 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.393 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.393 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.394 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.394 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.395 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.395 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.395 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.396 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.396 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.397 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.397 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.397 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.398 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.398 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.398 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.399 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.399 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.399 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.400 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.400 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.400 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.401 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.401 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.401 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.402 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.402 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.402 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.403 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.403 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.404 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.404 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.404 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.405 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.405 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.405 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.406 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.406 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.406 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.407 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.407 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.408 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.408 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.408 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.409 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.409 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.409 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.410 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.410 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:35.410 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:05:55.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:05:55.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.620+0000 7fed1cfff700 1 Processor -- start 2023-06-09T12:05:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.620+0000 7fed1cfff700 1 -- start start 2023-06-09T12:05:55.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed180cb5d0 0x7fed180b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fed180cb9a0 0x7fed18159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed18159b20 con 0x7fed180cb5d0 2023-06-09T12:05:55.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180cac80 con 0x7fed180cb9a0 2023-06-09T12:05:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.621+0000 7fed1cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181597d0 con 0x7fed1815cd40 2023-06-09T12:05:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed180cb5d0 0x7fed180b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fed180cb5d0 0x7fed180b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52554/0 (socket says 172.21.15.7:52554) 2023-06-09T12:05:55.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 -- 172.21.15.7:0/455419797 learned_addr learned my addr 172.21.15.7:0/455419797 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:05:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed16d9d700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed15d9b700 1 --2- 172.21.15.7:0/455419797 >> v2:172.21.15.117:3300/0 conn(0x7fed180cb9a0 0x7fed18159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 -- 172.21.15.7:0/455419797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 msgr2=0x7fed180cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:05:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 -- 172.21.15.7:0/455419797 >> v2:172.21.15.117:3300/0 conn(0x7fed180cb9a0 msgr2=0x7fed18159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:05:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 --2- 172.21.15.7:0/455419797 >> v2:172.21.15.117:3300/0 conn(0x7fed180cb9a0 0x7fed18159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 -- 172.21.15.7:0/455419797 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18071ab0 con 0x7fed180cb5d0 2023-06-09T12:05:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.622+0000 7fed1659c700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb5d0 0x7fed180b52b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fed180c9c60 tx=0x7fed08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4c3f68087d3bc6 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1559a700 1 -- 172.21.15.7:0/455419797 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800b870 con 0x7fed180cb5d0 2023-06-09T12:05:55.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1559a700 1 -- 172.21.15.7:0/455419797 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800b9d0 con 0x7fed180cb5d0 2023-06-09T12:05:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1559a700 1 -- 172.21.15.7:0/455419797 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0800bd60 con 0x7fed180cb5d0 2023-06-09T12:05:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 -- 172.21.15.7:0/455419797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb5d0 msgr2=0x7fed180b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb5d0 0x7fed180b52b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fed180c9c60 tx=0x7fed08009bd0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 -- 172.21.15.7:0/455419797 shutdown_connections 2023-06-09T12:05:55.630 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 --2- 172.21.15.7:0/455419797 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb5d0 0x7fed180b52b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 --2- 172.21.15.7:0/455419797 >> v2:172.21.15.117:3300/0 conn(0x7fed180cb9a0 0x7fed18159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 -- 172.21.15.7:0/455419797 >> 172.21.15.7:0/455419797 conn(0x7fed18070af0 msgr2=0x7fed18070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.623+0000 7fed1cfff700 1 -- 172.21.15.7:0/455419797 shutdown_connections 2023-06-09T12:05:55.631 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.624+0000 7fed1cfff700 1 -- 172.21.15.7:0/455419797 wait complete. 2023-06-09T12:05:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.625+0000 7fed1cfff700 1 Processor -- start 2023-06-09T12:05:55.633 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.625+0000 7fed1cfff700 1 -- start start 2023-06-09T12:05:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed180cb5d0 0x7fed180c3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ce0a0 con 0x7fed180cb9a0 2023-06-09T12:05:55.634 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cf880 con 0x7fed180cb5d0 2023-06-09T12:05:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cfea0 con 0x7fed1815cd40 2023-06-09T12:05:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed15d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed15d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52562/0 (socket says 172.21.15.7:52562) 2023-06-09T12:05:55.635 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed15d9b700 1 -- 172.21.15.7:0/1863386844 learned_addr learned my addr 172.21.15.7:0/1863386844 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:05:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.626+0000 7fed1659c700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed180cb5d0 0x7fed180c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed16d9d700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 -- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 msgr2=0x7fed180cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.636 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 -- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed180cb5d0 msgr2=0x7fed180c3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed180cb5d0 0x7fed180c3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 -- 172.21.15.7:0/1863386844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed08009550 con 0x7fed180cb9a0 2023-06-09T12:05:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed15d9b700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00e7e0 tx=0x7fed0c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00c070 con 0x7fed180cb9a0 2023-06-09T12:05:55.637 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c013610 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1815d4f0 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.627+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0139a0 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.628+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1815d210 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.629+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1815d7d0 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.629+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed0c013b00 con 0x7fed180cb9a0 2023-06-09T12:05:55.638 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.631+0000 7fed06ffd700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 0x7fecf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:05:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.631+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fed0c089b00 con 0x7fed180cb9a0 2023-06-09T12:05:55.639 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.631+0000 7fed1659c700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 0x7fecf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:05:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.631+0000 7fed1659c700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 0x7fecf806d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:05:55.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.635+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fed0c08be60 con 0x7fed180cb9a0 2023-06-09T12:05:55.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.841+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed18074cf0 con 0x7fecf806ace0 2023-06-09T12:05:55.853 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.850+0000 7fed06ffd700 1 -- 172.21.15.7:0/1863386844 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+315869 (secure 0 0 0) 0x7fed18074cf0 con 0x7fecf806ace0 2023-06-09T12:05:55.857 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:05:55.858 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 msgr2=0x7fecf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:55.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 0x7fecf806d190 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08008030 comp rx=0 tx=0).stop 2023-06-09T12:05:55.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 msgr2=0x7fed180ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:05:55.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00e7e0 tx=0x7fed0c00eaf0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 shutdown_connections 2023-06-09T12:05:55.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fecf806ace0 0x7fecf806d190 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fed1815cd40 0x7fed180cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed180cb5d0 0x7fed180c3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 --2- 172.21.15.7:0/1863386844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed180cb9a0 0x7fed180ce530 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:05:55.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 >> 172.21.15.7:0/1863386844 conn(0x7fed18070af0 msgr2=0x7fed180cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:05:55.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 shutdown_connections 2023-06-09T12:05:55.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:05:55.854+0000 7fed1cfff700 1 -- 172.21.15.7:0/1863386844 wait complete. 2023-06-09T12:05:55.861 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:05:55.876 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1724,"stamp":"2023-06-09T12:05:55.029838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94725,"ondisk_log_size":94725,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589064,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":705728,"kb_avail":752385656,"statfs":{"total":773094113280,"available":770442911744,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":722665472},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003555"},"pg_stats":[{"pgid":"4.3f","version":"75'1048","reported_seq":3603,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:50.302292+0000","last_change":"2023-06-09T12:05:33.531281+0000","last_active":"2023-06-09T12:05:50.302292+0000","last_peered":"2023-06-09T12:05:50.302292+0000","last_clean":"2023-06-09T12:05:50.302292+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:05:50.302292+0000","last_undegraded":"2023-06-09T12:05:50.302292+0000","last_fullsized":"2023-06-09T12:05:50.302292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":121,"log_size":1048,"log_dups_size":0,"ondisk_log_size":1048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3224,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.350871+0000","last_change":"2023-06-09T12:05:52.126507+0000","last_active":"2023-06-09T12:05:02.350871+0000","last_peered":"2023-06-09T12:05:02.350871+0000","last_clean":"2023-06-09T12:05:02.350871+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:05:02.350871+0000","last_undegraded":"2023-06-09T12:05:02.350871+0000","last_fullsized":"2023-06-09T12:05:02.350871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.314313+0000","last_change":"2023-06-09T12:03:22.314313+0000","last_active":"2023-06-09T12:03:22.314313+0000","last_peered":"2023-06-09T12:03:22.314313+0000","last_clean":"2023-06-09T12:03:22.314313+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:03:22.314313+0000","last_undegraded":"2023-06-09T12:03:22.314313+0000","last_fullsized":"2023-06-09T12:03:22.314313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4201,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.341111+0000","last_change":"2023-06-09T12:04:12.320379+0000","last_active":"2023-06-09T12:03:57.341111+0000","last_peered":"2023-06-09T12:03:57.341111+0000","last_clean":"2023-06-09T12:03:57.341111+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:03:57.341111+0000","last_undegraded":"2023-06-09T12:03:57.341111+0000","last_fullsized":"2023-06-09T12:03:57.341111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3410,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:42.781015+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:42.781015+0000","last_peered":"2023-06-09T12:04:42.781015+0000","last_clean":"2023-06-09T12:04:42.781015+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:42.781015+0000","last_undegraded":"2023-06-09T12:04:42.781015+0000","last_fullsized":"2023-06-09T12:04:42.781015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3737,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.370675+0000","last_change":"2023-06-09T12:04:32.323061+0000","last_active":"2023-06-09T12:03:55.370675+0000","last_peered":"2023-06-09T12:03:55.370675+0000","last_clean":"2023-06-09T12:03:55.370675+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:03:55.370675+0000","last_undegraded":"2023-06-09T12:03:55.370675+0000","last_fullsized":"2023-06-09T12:03:55.370675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.422718+0000","last_change":"2023-06-09T11:18:26.333712+0000","last_active":"2023-06-09T12:02:12.422718+0000","last_peered":"2023-06-09T12:02:12.422718+0000","last_clean":"2023-06-09T12:02:12.422718+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:02:12.422718+0000","last_undegraded":"2023-06-09T12:02:12.422718+0000","last_fullsized":"2023-06-09T12:02:12.422718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3485,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.325615+0000","last_change":"2023-06-09T12:05:41.406462+0000","last_active":"2023-06-09T12:04:52.325615+0000","last_peered":"2023-06-09T12:04:52.325615+0000","last_clean":"2023-06-09T12:04:52.325615+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:04:52.325615+0000","last_undegraded":"2023-06-09T12:04:52.325615+0000","last_fullsized":"2023-06-09T12:04:52.325615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3807,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:33.603990+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:01:33.603990+0000","last_peered":"2023-06-09T12:01:33.603990+0000","last_clean":"2023-06-09T12:01:33.603990+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:01:33.603990+0000","last_undegraded":"2023-06-09T12:01:33.603990+0000","last_fullsized":"2023-06-09T12:01:33.603990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3610,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.350828+0000","last_change":"2023-06-09T12:05:51.156280+0000","last_active":"2023-06-09T12:05:02.350828+0000","last_peered":"2023-06-09T12:05:02.350828+0000","last_clean":"2023-06-09T12:05:02.350828+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:05:02.350828+0000","last_undegraded":"2023-06-09T12:05:02.350828+0000","last_fullsized":"2023-06-09T12:05:02.350828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3139,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326950+0000","last_change":"2023-06-09T12:05:44.454243+0000","last_active":"2023-06-09T12:05:02.326950+0000","last_peered":"2023-06-09T12:05:02.326950+0000","last_clean":"2023-06-09T12:05:02.326950+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:05:02.326950+0000","last_undegraded":"2023-06-09T12:05:02.326950+0000","last_fullsized":"2023-06-09T12:05:02.326950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3311,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.082714+0000","last_change":"2023-06-09T12:05:02.520050+0000","last_active":"2023-06-09T12:03:58.082714+0000","last_peered":"2023-06-09T12:03:58.082714+0000","last_clean":"2023-06-09T12:03:58.082714+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:58.082714+0000","last_undegraded":"2023-06-09T12:03:58.082714+0000","last_fullsized":"2023-06-09T12:03:58.082714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'182","reported_seq":350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:00.377061+0000","last_change":"2023-06-09T12:04:02.319159+0000","last_active":"2023-06-09T12:04:00.377061+0000","last_peered":"2023-06-09T12:04:00.377061+0000","last_clean":"2023-06-09T12:04:00.377061+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:04:00.377061+0000","last_undegraded":"2023-06-09T12:04:00.377061+0000","last_fullsized":"2023-06-09T12:04:00.377061+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'957","reported_seq":3105,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:13.801581+0000","last_change":"2023-06-09T12:04:32.318108+0000","last_active":"2023-06-09T12:03:13.801581+0000","last_peered":"2023-06-09T12:03:13.801581+0000","last_clean":"2023-06-09T12:03:13.801581+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:03:13.801581+0000","last_undegraded":"2023-06-09T12:03:13.801581+0000","last_fullsized":"2023-06-09T12:03:13.801581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'865","reported_seq":1085,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.161159+0000","last_change":"2023-06-09T12:04:32.424054+0000","last_active":"2023-06-09T12:02:49.161159+0000","last_peered":"2023-06-09T12:02:49.161159+0000","last_clean":"2023-06-09T12:02:49.161159+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:02:49.161159+0000","last_undegraded":"2023-06-09T12:02:49.161159+0000","last_fullsized":"2023-06-09T12:02:49.161159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3561,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313831+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:02:22.313831+0000","last_peered":"2023-06-09T12:02:22.313831+0000","last_clean":"2023-06-09T12:02:22.313831+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:02:22.313831+0000","last_undegraded":"2023-06-09T12:02:22.313831+0000","last_fullsized":"2023-06-09T12:02:22.313831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3591,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:02.343332+0000","last_change":"2023-06-09T12:05:49.090152+0000","last_active":"2023-06-09T12:04:02.343332+0000","last_peered":"2023-06-09T12:04:02.343332+0000","last_clean":"2023-06-09T12:04:02.343332+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:04:02.343332+0000","last_undegraded":"2023-06-09T12:04:02.343332+0000","last_fullsized":"2023-06-09T12:04:02.343332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3027,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:02.319111+0000","last_change":"2023-06-09T12:05:50.361843+0000","last_active":"2023-06-09T12:04:02.319111+0000","last_peered":"2023-06-09T12:04:02.319111+0000","last_clean":"2023-06-09T12:04:02.319111+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:04:02.319111+0000","last_undegraded":"2023-06-09T12:04:02.319111+0000","last_fullsized":"2023-06-09T12:04:02.319111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:50.198865+0000","last_change":"2023-06-09T12:04:57.427044+0000","last_active":"2023-06-09T12:02:50.198865+0000","last_peered":"2023-06-09T12:02:50.198865+0000","last_clean":"2023-06-09T12:02:50.198865+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:02:50.198865+0000","last_undegraded":"2023-06-09T12:02:50.198865+0000","last_fullsized":"2023-06-09T12:02:50.198865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428433+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:02:52.428433+0000","last_peered":"2023-06-09T12:02:52.428433+0000","last_clean":"2023-06-09T12:02:52.428433+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:02:52.428433+0000","last_undegraded":"2023-06-09T12:02:52.428433+0000","last_fullsized":"2023-06-09T12:02:52.428433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3544,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:55.039481+0000","last_change":"2023-06-09T12:05:12.521142+0000","last_active":"2023-06-09T12:03:55.039481+0000","last_peered":"2023-06-09T12:03:55.039481+0000","last_clean":"2023-06-09T12:03:55.039481+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:03:55.039481+0000","last_undegraded":"2023-06-09T12:03:55.039481+0000","last_fullsized":"2023-06-09T12:03:55.039481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":695,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:51.058132+0000","last_change":"2023-06-09T12:04:52.349590+0000","last_active":"2023-06-09T12:04:51.058132+0000","last_peered":"2023-06-09T12:04:51.058132+0000","last_clean":"2023-06-09T12:04:51.058132+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:04:51.058132+0000","last_undegraded":"2023-06-09T12:04:51.058132+0000","last_fullsized":"2023-06-09T12:04:51.058132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:53.325190+0000","last_change":"2023-06-09T12:04:52.325633+0000","last_active":"2023-06-09T12:03:53.325190+0000","last_peered":"2023-06-09T12:03:53.325190+0000","last_clean":"2023-06-09T12:03:53.325190+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:03:53.325190+0000","last_undegraded":"2023-06-09T12:03:53.325190+0000","last_fullsized":"2023-06-09T12:03:53.325190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.426543+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:02:37.426543+0000","last_peered":"2023-06-09T12:02:37.426543+0000","last_clean":"2023-06-09T12:02:37.426543+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:02:37.426543+0000","last_undegraded":"2023-06-09T12:02:37.426543+0000","last_fullsized":"2023-06-09T12:02:37.426543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":757,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334635+0000","last_change":"2023-06-09T12:05:30.983184+0000","last_active":"2023-06-09T12:05:02.334635+0000","last_peered":"2023-06-09T12:05:02.334635+0000","last_clean":"2023-06-09T12:05:02.334635+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:02.334635+0000","last_undegraded":"2023-06-09T12:05:02.334635+0000","last_fullsized":"2023-06-09T12:05:02.334635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:17.778068+0000","last_change":"2023-06-09T12:05:02.322556+0000","last_active":"2023-06-09T12:03:17.778068+0000","last_peered":"2023-06-09T12:03:17.778068+0000","last_clean":"2023-06-09T12:03:17.778068+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:03:17.778068+0000","last_undegraded":"2023-06-09T12:03:17.778068+0000","last_fullsized":"2023-06-09T12:03:17.778068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:48.020256+0000","last_change":"2023-06-09T12:04:57.350272+0000","last_active":"2023-06-09T12:04:48.020256+0000","last_peered":"2023-06-09T12:04:48.020256+0000","last_clean":"2023-06-09T12:04:48.020256+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:04:48.020256+0000","last_undegraded":"2023-06-09T12:04:48.020256+0000","last_fullsized":"2023-06-09T12:04:48.020256+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":924,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:10.590528+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:02:10.590528+0000","last_peered":"2023-06-09T12:02:10.590528+0000","last_clean":"2023-06-09T12:02:10.590528+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:02:10.590528+0000","last_undegraded":"2023-06-09T12:02:10.590528+0000","last_fullsized":"2023-06-09T12:02:10.590528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":317,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.405695+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:01:57.405695+0000","last_peered":"2023-06-09T12:01:57.405695+0000","last_clean":"2023-06-09T12:01:57.405695+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:01:57.405695+0000","last_undegraded":"2023-06-09T12:01:57.405695+0000","last_fullsized":"2023-06-09T12:01:57.405695+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1204","reported_seq":3561,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:50.672797+0000","last_change":"2023-06-09T12:05:16.047851+0000","last_active":"2023-06-09T12:05:50.672797+0000","last_peered":"2023-06-09T12:05:50.672797+0000","last_clean":"2023-06-09T12:05:50.672797+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:50.672797+0000","last_undegraded":"2023-06-09T12:05:50.672797+0000","last_fullsized":"2023-06-09T12:05:50.672797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":243,"log_size":1204,"log_dups_size":0,"ondisk_log_size":1204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":159,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.412288+0000","last_change":"2023-06-09T12:02:52.412288+0000","last_active":"2023-06-09T12:02:52.412288+0000","last_peered":"2023-06-09T12:02:52.412288+0000","last_clean":"2023-06-09T12:02:52.412288+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:02:52.412288+0000","last_undegraded":"2023-06-09T12:02:52.412288+0000","last_fullsized":"2023-06-09T12:02:52.412288+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2990,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:01.358848+0000","last_change":"2023-06-09T12:05:02.326837+0000","last_active":"2023-06-09T12:04:01.358848+0000","last_peered":"2023-06-09T12:04:01.358848+0000","last_clean":"2023-06-09T12:04:01.358848+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:04:01.358848+0000","last_undegraded":"2023-06-09T12:04:01.358848+0000","last_fullsized":"2023-06-09T12:04:01.358848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:54.261006+0000","last_change":"2023-06-09T12:05:02.326991+0000","last_active":"2023-06-09T12:04:54.261006+0000","last_peered":"2023-06-09T12:04:54.261006+0000","last_clean":"2023-06-09T12:04:54.261006+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:04:54.261006+0000","last_undegraded":"2023-06-09T12:04:54.261006+0000","last_fullsized":"2023-06-09T12:04:54.261006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4116,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326908+0000","last_change":"2023-06-09T12:05:51.360005+0000","last_active":"2023-06-09T12:05:02.326908+0000","last_peered":"2023-06-09T12:05:02.326908+0000","last_clean":"2023-06-09T12:05:02.326908+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:05:02.326908+0000","last_undegraded":"2023-06-09T12:05:02.326908+0000","last_fullsized":"2023-06-09T12:05:02.326908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3371,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.318985+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:03:02.318985+0000","last_peered":"2023-06-09T12:03:02.318985+0000","last_clean":"2023-06-09T12:03:02.318985+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:03:02.318985+0000","last_undegraded":"2023-06-09T12:03:02.318985+0000","last_fullsized":"2023-06-09T12:03:02.318985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.112083+0000","last_change":"2023-06-09T12:04:57.519309+0000","last_active":"2023-06-09T12:03:57.112083+0000","last_peered":"2023-06-09T12:03:57.112083+0000","last_clean":"2023-06-09T12:03:57.112083+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:03:57.112083+0000","last_undegraded":"2023-06-09T12:03:57.112083+0000","last_fullsized":"2023-06-09T12:03:57.112083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:28.374029+0000","last_change":"2023-06-09T12:05:02.326978+0000","last_active":"2023-06-09T12:04:28.374029+0000","last_peered":"2023-06-09T12:04:28.374029+0000","last_clean":"2023-06-09T12:04:28.374029+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:04:28.374029+0000","last_undegraded":"2023-06-09T12:04:28.374029+0000","last_fullsized":"2023-06-09T12:04:28.374029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":885,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.346421+0000","last_change":"2023-06-09T12:05:52.332905+0000","last_active":"2023-06-09T12:04:52.346421+0000","last_peered":"2023-06-09T12:04:52.346421+0000","last_clean":"2023-06-09T12:04:52.346421+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:04:52.346421+0000","last_undegraded":"2023-06-09T12:04:52.346421+0000","last_fullsized":"2023-06-09T12:04:52.346421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":772,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:53.107623+0000","last_change":"2023-06-09T12:05:52.357458+0000","last_active":"2023-06-09T12:04:53.107623+0000","last_peered":"2023-06-09T12:04:53.107623+0000","last_clean":"2023-06-09T12:04:53.107623+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:04:53.107623+0000","last_undegraded":"2023-06-09T12:04:53.107623+0000","last_fullsized":"2023-06-09T12:04:53.107623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:13.303360+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:01:13.303360+0000","last_peered":"2023-06-09T12:01:13.303360+0000","last_clean":"2023-06-09T12:01:13.303360+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:01:13.303360+0000","last_undegraded":"2023-06-09T12:01:13.303360+0000","last_fullsized":"2023-06-09T12:01:13.303360+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.124241+0000","last_change":"2023-06-09T12:05:02.519982+0000","last_active":"2023-06-09T12:03:59.124241+0000","last_peered":"2023-06-09T12:03:59.124241+0000","last_clean":"2023-06-09T12:03:59.124241+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:59.124241+0000","last_undegraded":"2023-06-09T12:03:59.124241+0000","last_fullsized":"2023-06-09T12:03:59.124241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3264,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:51.849509+0000","last_change":"2023-06-09T12:05:18.721187+0000","last_active":"2023-06-09T12:05:51.849509+0000","last_peered":"2023-06-09T12:05:51.849509+0000","last_clean":"2023-06-09T12:05:51.849509+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:51.849509+0000","last_undegraded":"2023-06-09T12:05:51.849509+0000","last_fullsized":"2023-06-09T12:05:51.849509+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 33s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":63834,"ondisk_log_size":63834,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1129158,"num_omap_keys":2142,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30772,"ondisk_log_size":30772,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575513,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259984,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80128,"kb_avail":94111856,"statfs":{"total":96636764160,"available":96370540544,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82051072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71699999999999997,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.0549999999999999,"5min":1.256,"15min":1.3320000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72699999999999998,"15min":0.68999999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.28199999999999997},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":1.081},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.05},"last":0.79900000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.85499999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.72899999999999998},"min":{"1min":0.53500000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.50700000000000001,"5min":0.441,"15min":0.379},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72399999999999998,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.629,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.32400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.505}]},{"osd":6,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.624},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":1.1899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.52900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575513,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208472,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52352,"kb_avail":94163368,"statfs":{"total":96636764160,"available":96423288832,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53608448},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.90100000000000002,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71699999999999997,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.72999999999999998},"min":{"1min":0.621,"5min":0.60499999999999998,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72499999999999998,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.98499999999999999,"15min":1.024},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.64900000000000002,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.871,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.53400000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77200000000000002,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.92200000000000004,"5min":1.03,"15min":1.0369999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80600000000000005,"15min":0.753},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.94999999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.57799999999999996},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.35199999999999998},"max":{"1min":0.86499999999999999,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.58299999999999996,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.052},"last":0.75}]},{"osd":7,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65600000000000003,"15min":0.64700000000000002},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.071},"last":0.503}]}]},{"osd":1,"up_from":13,"seq":55834575514,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406128,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99712,"kb_avail":93965712,"statfs":{"total":96636764160,"available":96220889088,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102105088},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.47299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28000000000000003},"max":{"1min":0.81299999999999994,"5min":0.995,"15min":0.995},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.83799999999999997,"5min":0.86099999999999999,"15min":0.871},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.79800000000000004,"15min":0.90200000000000002},"last":0.501},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.38700000000000001},"max":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":1.091},"last":0.628}]},{"osd":4,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77700000000000002,"15min":0.70499999999999996},"min":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.0569999999999999,"15min":1.0649999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.877,"15min":0.82599999999999996},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.85899999999999999,"15min":0.79800000000000004},"min":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90200000000000002,"15min":0.84099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.879}]},{"osd":6,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.82799999999999996},"min":{"1min":0.71199999999999997,"5min":0.59999999999999998,"15min":0.504},"max":{"1min":1.155,"5min":1.27,"15min":1.4039999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82499999999999996,"15min":0.76500000000000001},"min":{"1min":0.67100000000000004,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.77000000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.92000000000000004,"15min":0.878},"min":{"1min":0.72499999999999998,"5min":0.68999999999999995,"15min":0.56299999999999994},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.748},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82099999999999995,"15min":0.77600000000000002},"min":{"1min":0.64300000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.2150000000000001,"5min":1.355,"15min":1.355},"last":1.0669999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575513,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332904,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":105088,"kb_avail":94038936,"statfs":{"total":96636764160,"available":96295870464,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107610112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.443,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.52500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.72299999999999998,"5min":1.032,"15min":1.032},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.505},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":1.024},"last":0.57199999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80700000000000005,"15min":0.72099999999999997},"min":{"1min":0.68300000000000005,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":0.99299999999999999,"5min":1.175,"15min":1.175},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.86199999999999999,"15min":0.79900000000000004},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.0469999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.72999999999999998,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.012,"5min":1.034,"15min":1.236},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.85899999999999999,"15min":0.78000000000000003},"min":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":1.103,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.96399999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.083,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.83199999999999996},"min":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.1100000000000001,"15min":1.3149999999999999},"last":0.76500000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85899999999999999,"15min":0.79300000000000004},"min":{"1min":0.69799999999999995,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.1100000000000001,"5min":1.298,"15min":1.298},"last":0.878},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94099999999999995,"15min":0.85699999999999998},"min":{"1min":0.84599999999999997,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1359999999999999,"5min":1.351,"15min":1.351},"last":1.006}]}]},{"osd":2,"up_from":13,"seq":55834575514,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242400,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62592,"kb_avail":94129440,"statfs":{"total":96636764160,"available":96388546560,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64094208},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.501},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.57699999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.45800000000000002},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":1.367},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.85299999999999998,"15min":1.546},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.014,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78500000000000003,"15min":0.76400000000000001},"min":{"1min":0.60799999999999998,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":0.78800000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83199999999999996,"15min":0.79900000000000004},"min":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.96299999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.872,"15min":0.83999999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.88500000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.77500000000000002},"min":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.438},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.3029999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77800000000000002,"15min":0.76300000000000001},"min":{"1min":0.629,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.056,"15min":1.3540000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84099999999999997,"15min":0.79800000000000004},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.497},"max":{"1min":1.069,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83999999999999997,"15min":0.82499999999999996},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.3080000000000001},"last":0.89900000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575513,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422396,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":99200,"kb_avail":93949444,"statfs":{"total":96636764160,"available":96204230656,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101580800},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.58099999999999996,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.26900000000000002},"max":{"1min":0.81999999999999995,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77400000000000002,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.52300000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":0.97099999999999997,"15min":1.355},"last":0.623},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.85899999999999999,"5min":0.91200000000000003,"15min":1.198},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79600000000000004,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.073,"15min":1.635},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0369999999999999,"5min":1.0980000000000001,"15min":3.081},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83099999999999996,"15min":0.77400000000000002},"min":{"1min":0.66300000000000003,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.165,"15min":1.2110000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88400000000000001,"15min":0.81399999999999995},"min":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.47899999999999998},"max":{"1min":1.095,"5min":1.1970000000000001,"15min":1.367},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.78300000000000003},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.94799999999999995,"5min":1.2649999999999999,"15min":1.286},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":0.77900000000000003},"min":{"1min":0.65800000000000003,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":3.3490000000000002},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.81000000000000005},"min":{"1min":0.628,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.306,"15min":1.413},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.70299999999999996,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":3.1160000000000001},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575512,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423316,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130560,"kb_avail":93948524,"statfs":{"total":96636764160,"available":96203288576,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133693440},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72399999999999998,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":1.0640000000000001,"15min":1.137},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78700000000000003,"15min":0.749},"min":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":0.85199999999999998,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75,"15min":0.72099999999999997},"min":{"1min":0.63800000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.0640000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":1.03,"15min":1.1279999999999999},"last":0.96999999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73299999999999998,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83299999999999996,"5min":1.117,"15min":1.609},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.68899999999999995},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.38100000000000001},"max":{"1min":1.0129999999999999,"5min":1.147,"15min":1.159},"last":0.90300000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":1.014,"15min":1.407},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.622},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.1719999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.131,"5min":1.131,"15min":1.3100000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.361},"last":0.59599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293464,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76096,"kb_avail":94078376,"statfs":{"total":96636764160,"available":96336257024,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77922304},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56599999999999995,"5min":0.44,"15min":0.318},"max":{"1min":0.90700000000000003,"5min":1.177,"15min":5.6269999999999998},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":1.083,"15min":19.715},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.59299999999999997,"5min":0.45700000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.54500000000000004,"5min":0.53800000000000003,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":1.145,"15min":14.439},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.75700000000000001},"min":{"1min":0.5,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.81399999999999995,"5min":1.006,"15min":12.130000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.624,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.749}]},{"osd":4,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.498},{"interface":"front","average":{"1min":0.624,"5min":0.59499999999999997,"15min":0.83199999999999996},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.623}]},{"osd":6,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63800000000000001,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":12.163},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.81999999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":1.135,"15min":19.834},"last":0.81299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:05:55.883 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.884 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.884 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.885 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.885 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.885 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.886 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.886 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.886 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.887 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.887 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.888 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.888 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.888 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.889 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.889 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.889 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.890 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.890 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.890 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.891 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.891 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.891 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.892 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.892 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.893 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.893 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.893 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.894 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.894 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.894 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.895 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.895 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.895 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.896 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.896 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.896 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.897 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.897 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.897 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.898 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.898 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.899 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.899 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.899 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.900 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.900 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.900 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.901 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.901 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.901 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.902 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.902 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.902 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.903 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.903 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.903 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.904 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.904 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.904 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.905 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:05:55.905 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:05:57.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:05:57.475 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:15.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:06:16.119 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.118+0000 7f05ac9ff700 1 Processor -- start 2023-06-09T12:06:16.120 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.118+0000 7f05ac9ff700 1 -- start start 2023-06-09T12:06:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05ac9ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05a80cb5d0 0x7f05a80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05ac9ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f05a80cb9a0 0x7f05a8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05ac9ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a8159b20 con 0x7f05a815cd40 2023-06-09T12:06:16.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05ac9ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80cac80 con 0x7f05a80cb5d0 2023-06-09T12:06:16.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05ac9ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a81597d0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05a659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05a80cb5d0 0x7f05a80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05a659c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f05a80cb5d0 0x7f05a80b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:33722/0 (socket says 172.21.15.7:33722) 2023-06-09T12:06:16.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.119+0000 7f05a659c700 1 -- 172.21.15.7:0/2122350844 learned_addr learned my addr 172.21.15.7:0/2122350844 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a5d9b700 1 --2- 172.21.15.7:0/2122350844 >> v2:172.21.15.117:3300/0 conn(0x7f05a80cb9a0 0x7f05a8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a659c700 1 -- 172.21.15.7:0/2122350844 >> v2:172.21.15.117:3300/0 conn(0x7f05a80cb9a0 msgr2=0x7f05a8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:06:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a659c700 1 --2- 172.21.15.7:0/2122350844 >> v2:172.21.15.117:3300/0 conn(0x7f05a80cb9a0 0x7f05a8159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a659c700 1 -- 172.21.15.7:0/2122350844 >> v1:172.21.15.117:6789/0 conn(0x7f05a815cd40 legacy=0x7f05a80cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a659c700 1 -- 172.21.15.7:0/2122350844 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8071ab0 con 0x7f05a80cb5d0 2023-06-09T12:06:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a659c700 1 --2- 172.21.15.7:0/2122350844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a80cb5d0 0x7f05a80b52b0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f05a80c9c60 tx=0x7f0598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2a1e80a7cca80f server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a559a700 1 -- 172.21.15.7:0/2122350844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800b870 con 0x7f05a80cb5d0 2023-06-09T12:06:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a559a700 1 -- 172.21.15.7:0/2122350844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800b9d0 con 0x7f05a80cb5d0 2023-06-09T12:06:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.120+0000 7f05a559a700 1 -- 172.21.15.7:0/2122350844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800bd60 con 0x7f05a80cb5d0 2023-06-09T12:06:16.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2122350844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a80cb5d0 msgr2=0x7f05a80b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2122350844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a80cb5d0 0x7f05a80b52b0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f05a80c9c60 tx=0x7f0598009bd0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2122350844 shutdown_connections 2023-06-09T12:06:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2122350844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a80cb5d0 0x7f05a80b52b0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2122350844 >> v2:172.21.15.117:3300/0 conn(0x7f05a80cb9a0 0x7f05a8159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2122350844 >> 172.21.15.7:0/2122350844 conn(0x7f05a8070af0 msgr2=0x7f05a8070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:16.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2122350844 shutdown_connections 2023-06-09T12:06:16.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.121+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2122350844 wait complete. 2023-06-09T12:06:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.123+0000 7f05ac9ff700 1 Processor -- start 2023-06-09T12:06:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.123+0000 7f05ac9ff700 1 -- start start 2023-06-09T12:06:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 0x7f05a8154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 0x7f05a80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c45d0 con 0x7f05a815cd40 2023-06-09T12:06:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c47e0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05ac9ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c4ac0 con 0x7f05a80cb5d0 2023-06-09T12:06:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05a6d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 0x7f05a80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05a659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05a5d9b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 0x7f05a8154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05a659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:35590/0 (socket says 172.21.15.7:35590) 2023-06-09T12:06:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.124+0000 7f05a659c700 1 -- 172.21.15.7:0/2116897384 learned_addr learned my addr 172.21.15.7:0/2116897384 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 -- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 msgr2=0x7f05a8153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 -- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 msgr2=0x7f05a80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 0x7f05a80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 -- 172.21.15.7:0/2116897384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009550 con 0x7f05a80cb9a0 2023-06-09T12:06:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a659c700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:06:16.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a6d9d700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 0x7f05a80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05a5d9b700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 0x7f05a8154340 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f059c00e7e0 tx=0x7f059c00eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c00c070 con 0x7f05a80cb9a0 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c004020 con 0x7f05a80cb9a0 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a815d4f0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c0043d0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.125+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a815d210 con 0x7f05a80cb9a0 2023-06-09T12:06:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.126+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a815d7d0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.127+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f059c0036a0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.128+0000 7f0596ffd700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 0x7f058806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.128+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f059c0896e0 con 0x7f05a80cb9a0 2023-06-09T12:06:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.128+0000 7f05a659c700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 0x7f058806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:16.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.129+0000 7f05a659c700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 0x7f058806d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f059800c010 tx=0x7f0598008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:16.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.134+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mon.1 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f059c053e20 con 0x7f05a80cb9a0 2023-06-09T12:06:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.339+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05a80cd560 con 0x7f058806ace0 2023-06-09T12:06:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.348+0000 7f0596ffd700 1 -- 172.21.15.7:0/2116897384 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316407 (secure 0 0 0) 0x7f05a80cd560 con 0x7f058806ace0 2023-06-09T12:06:16.353 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:06:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 msgr2=0x7f058806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 0x7f058806d190 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f059800c010 tx=0x7f0598008030 comp rx=0 tx=0).stop 2023-06-09T12:06:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 msgr2=0x7f05a8154340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 0x7f05a8154340 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f059c00e7e0 tx=0x7f059c00eaf0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 shutdown_connections 2023-06-09T12:06:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f05a815cd40 0x7f05a80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f058806ace0 0x7f058806d190 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f05a80cb5d0 0x7f05a8153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 --2- 172.21.15.7:0/2116897384 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f05a80cb9a0 0x7f05a8154340 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.352+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 >> 172.21.15.7:0/2116897384 conn(0x7f05a8070af0 msgr2=0x7f05a815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.353+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 shutdown_connections 2023-06-09T12:06:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:16.353+0000 7f05ac9ff700 1 -- 172.21.15.7:0/2116897384 wait complete. 2023-06-09T12:06:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:06:16.373 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1734,"stamp":"2023-06-09T12:06:15.036122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1260437,"num_omap_keys":2391,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95017,"ondisk_log_size":95017,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589768,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":706432,"kb_avail":752384952,"statfs":{"total":773094113280,"available":770442190848,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":723386368},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":131279,"num_omap_keys":249,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003959"},"pg_stats":[{"pgid":"4.3f","version":"75'1170","reported_seq":3731,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:11.142518+0000","last_change":"2023-06-09T12:05:33.531281+0000","last_active":"2023-06-09T12:06:11.142518+0000","last_peered":"2023-06-09T12:06:11.142518+0000","last_clean":"2023-06-09T12:06:11.142518+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:11.142518+0000","last_undegraded":"2023-06-09T12:06:11.142518+0000","last_fullsized":"2023-06-09T12:06:11.142518+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":243,"log_size":1170,"log_dups_size":0,"ondisk_log_size":1170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3224,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.350871+0000","last_change":"2023-06-09T12:05:52.126507+0000","last_active":"2023-06-09T12:05:02.350871+0000","last_peered":"2023-06-09T12:05:02.350871+0000","last_clean":"2023-06-09T12:05:02.350871+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:05:02.350871+0000","last_undegraded":"2023-06-09T12:05:02.350871+0000","last_fullsized":"2023-06-09T12:05:02.350871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332933+0000","last_change":"2023-06-09T12:05:55.319769+0000","last_active":"2023-06-09T12:05:52.332933+0000","last_peered":"2023-06-09T12:05:52.332933+0000","last_clean":"2023-06-09T12:05:52.332933+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:05:52.332933+0000","last_undegraded":"2023-06-09T12:05:52.332933+0000","last_fullsized":"2023-06-09T12:05:52.332933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:30.587711+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:01:30.587711+0000","last_peered":"2023-06-09T12:01:30.587711+0000","last_clean":"2023-06-09T12:01:30.587711+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:01:30.587711+0000","last_undegraded":"2023-06-09T12:01:30.587711+0000","last_fullsized":"2023-06-09T12:01:30.587711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4202,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:12.320379+0000","last_change":"2023-06-09T12:06:04.285410+0000","last_active":"2023-06-09T12:04:12.320379+0000","last_peered":"2023-06-09T12:04:12.320379+0000","last_clean":"2023-06-09T12:04:12.320379+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:04:12.320379+0000","last_undegraded":"2023-06-09T12:04:12.320379+0000","last_fullsized":"2023-06-09T12:04:12.320379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:59.084217+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:02:59.084217+0000","last_peered":"2023-06-09T12:02:59.084217+0000","last_clean":"2023-06-09T12:02:59.084217+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:02:59.084217+0000","last_undegraded":"2023-06-09T12:02:59.084217+0000","last_fullsized":"2023-06-09T12:02:59.084217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3410,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:42.781015+0000","last_change":"2023-06-09T12:04:31.748046+0000","last_active":"2023-06-09T12:04:42.781015+0000","last_peered":"2023-06-09T12:04:42.781015+0000","last_clean":"2023-06-09T12:04:42.781015+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:04:42.781015+0000","last_undegraded":"2023-06-09T12:04:42.781015+0000","last_fullsized":"2023-06-09T12:04:42.781015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3738,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.323061+0000","last_change":"2023-06-09T12:05:59.293378+0000","last_active":"2023-06-09T12:04:32.323061+0000","last_peered":"2023-06-09T12:04:32.323061+0000","last_clean":"2023-06-09T12:04:32.323061+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:04:32.323061+0000","last_undegraded":"2023-06-09T12:04:32.323061+0000","last_fullsized":"2023-06-09T12:04:32.323061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3493,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.441395+0000","last_change":"2023-06-09T12:05:55.201855+0000","last_active":"2023-06-09T12:04:32.441395+0000","last_peered":"2023-06-09T12:04:32.441395+0000","last_clean":"2023-06-09T12:04:32.441395+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:04:32.441395+0000","last_undegraded":"2023-06-09T12:04:32.441395+0000","last_fullsized":"2023-06-09T12:04:32.441395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929267+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929267+0000","last_peered":"2023-06-09T12:04:19.929267+0000","last_clean":"2023-06-09T12:04:19.929267+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929267+0000","last_undegraded":"2023-06-09T12:04:19.929267+0000","last_fullsized":"2023-06-09T12:04:19.929267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3485,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.325615+0000","last_change":"2023-06-09T12:05:41.406462+0000","last_active":"2023-06-09T12:04:52.325615+0000","last_peered":"2023-06-09T12:04:52.325615+0000","last_clean":"2023-06-09T12:04:52.325615+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:04:52.325615+0000","last_undegraded":"2023-06-09T12:04:52.325615+0000","last_fullsized":"2023-06-09T12:04:52.325615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'953","reported_seq":3611,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:51.156280+0000","last_change":"2023-06-09T12:05:51.156280+0000","last_active":"2023-06-09T12:05:51.156280+0000","last_peered":"2023-06-09T12:05:51.156280+0000","last_clean":"2023-06-09T12:05:51.156280+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:05:51.156280+0000","last_undegraded":"2023-06-09T12:05:51.156280+0000","last_fullsized":"2023-06-09T12:05:51.156280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":953,"log_dups_size":0,"ondisk_log_size":953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3139,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326950+0000","last_change":"2023-06-09T12:05:44.454243+0000","last_active":"2023-06-09T12:05:02.326950+0000","last_peered":"2023-06-09T12:05:02.326950+0000","last_clean":"2023-06-09T12:05:02.326950+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:05:02.326950+0000","last_undegraded":"2023-06-09T12:05:02.326950+0000","last_fullsized":"2023-06-09T12:05:02.326950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3311,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.082714+0000","last_change":"2023-06-09T12:05:02.520050+0000","last_active":"2023-06-09T12:03:58.082714+0000","last_peered":"2023-06-09T12:03:58.082714+0000","last_clean":"2023-06-09T12:03:58.082714+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:58.082714+0000","last_undegraded":"2023-06-09T12:03:58.082714+0000","last_fullsized":"2023-06-09T12:03:58.082714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":364,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.283986+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.283986+0000","last_peered":"2023-06-09T12:06:10.283986+0000","last_clean":"2023-06-09T12:06:10.283986+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.283986+0000","last_undegraded":"2023-06-09T12:06:10.283986+0000","last_fullsized":"2023-06-09T12:06:10.283986+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2965,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.312358+0000","last_change":"2023-06-09T12:02:12.312358+0000","last_active":"2023-06-09T12:02:12.312358+0000","last_peered":"2023-06-09T12:02:12.312358+0000","last_clean":"2023-06-09T12:02:12.312358+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:02:12.312358+0000","last_undegraded":"2023-06-09T12:02:12.312358+0000","last_fullsized":"2023-06-09T12:02:12.312358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1077","reported_seq":3235,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:09.983281+0000","last_change":"2023-06-09T12:05:55.824750+0000","last_active":"2023-06-09T12:06:09.983281+0000","last_peered":"2023-06-09T12:06:09.983281+0000","last_clean":"2023-06-09T12:06:09.983281+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:09.983281+0000","last_undegraded":"2023-06-09T12:06:09.983281+0000","last_fullsized":"2023-06-09T12:06:09.983281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":120,"log_size":1077,"log_dups_size":0,"ondisk_log_size":1077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3561,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313831+0000","last_change":"2023-06-09T12:02:22.313831+0000","last_active":"2023-06-09T12:02:22.313831+0000","last_peered":"2023-06-09T12:02:22.313831+0000","last_clean":"2023-06-09T12:02:22.313831+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:02:22.313831+0000","last_undegraded":"2023-06-09T12:02:22.313831+0000","last_fullsized":"2023-06-09T12:02:22.313831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3592,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:49.090152+0000","last_change":"2023-06-09T12:05:49.090152+0000","last_active":"2023-06-09T12:05:49.090152+0000","last_peered":"2023-06-09T12:05:49.090152+0000","last_clean":"2023-06-09T12:05:49.090152+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:05:49.090152+0000","last_undegraded":"2023-06-09T12:05:49.090152+0000","last_fullsized":"2023-06-09T12:05:49.090152+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'952","reported_seq":3027,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:02.319111+0000","last_change":"2023-06-09T12:05:50.361843+0000","last_active":"2023-06-09T12:04:02.319111+0000","last_peered":"2023-06-09T12:04:02.319111+0000","last_clean":"2023-06-09T12:04:02.319111+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:04:02.319111+0000","last_undegraded":"2023-06-09T12:04:02.319111+0000","last_fullsized":"2023-06-09T12:04:02.319111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"73'550","reported_seq":735,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:10.981310+0000","last_change":"2023-06-09T12:06:10.608887+0000","last_active":"2023-06-09T12:06:10.981310+0000","last_peered":"2023-06-09T12:06:10.981310+0000","last_clean":"2023-06-09T12:06:10.981310+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:10.981310+0000","last_undegraded":"2023-06-09T12:06:10.981310+0000","last_fullsized":"2023-06-09T12:06:10.981310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":550,"log_dups_size":0,"ondisk_log_size":550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428433+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:02:52.428433+0000","last_peered":"2023-06-09T12:02:52.428433+0000","last_clean":"2023-06-09T12:02:52.428433+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:02:52.428433+0000","last_undegraded":"2023-06-09T12:02:52.428433+0000","last_fullsized":"2023-06-09T12:02:52.428433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3545,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:12.521142+0000","last_change":"2023-06-09T12:05:54.809296+0000","last_active":"2023-06-09T12:05:12.521142+0000","last_peered":"2023-06-09T12:05:12.521142+0000","last_clean":"2023-06-09T12:05:12.521142+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:05:12.521142+0000","last_undegraded":"2023-06-09T12:05:12.521142+0000","last_fullsized":"2023-06-09T12:05:12.521142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":696,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.349590+0000","last_change":"2023-06-09T12:06:01.131264+0000","last_active":"2023-06-09T12:04:52.349590+0000","last_peered":"2023-06-09T12:04:52.349590+0000","last_clean":"2023-06-09T12:04:52.349590+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:04:52.349590+0000","last_undegraded":"2023-06-09T12:04:52.349590+0000","last_fullsized":"2023-06-09T12:04:52.349590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4071,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.325633+0000","last_change":"2023-06-09T12:05:57.344054+0000","last_active":"2023-06-09T12:04:52.325633+0000","last_peered":"2023-06-09T12:04:52.325633+0000","last_clean":"2023-06-09T12:04:52.325633+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:04:52.325633+0000","last_undegraded":"2023-06-09T12:04:52.325633+0000","last_fullsized":"2023-06-09T12:04:52.325633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:02.499568+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:02:02.499568+0000","last_peered":"2023-06-09T12:02:02.499568+0000","last_clean":"2023-06-09T12:02:02.499568+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:02:02.499568+0000","last_undegraded":"2023-06-09T12:02:02.499568+0000","last_fullsized":"2023-06-09T12:02:02.499568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'843","reported_seq":2929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:37.426543+0000","last_change":"2023-06-09T12:00:22.407587+0000","last_active":"2023-06-09T12:02:37.426543+0000","last_peered":"2023-06-09T12:02:37.426543+0000","last_clean":"2023-06-09T12:02:37.426543+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:02:37.426543+0000","last_undegraded":"2023-06-09T12:02:37.426543+0000","last_fullsized":"2023-06-09T12:02:37.426543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":757,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334635+0000","last_change":"2023-06-09T12:05:30.983184+0000","last_active":"2023-06-09T12:05:02.334635+0000","last_peered":"2023-06-09T12:05:02.334635+0000","last_clean":"2023-06-09T12:05:02.334635+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:02.334635+0000","last_undegraded":"2023-06-09T12:05:02.334635+0000","last_fullsized":"2023-06-09T12:05:02.334635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:17.778068+0000","last_change":"2023-06-09T12:05:02.322556+0000","last_active":"2023-06-09T12:03:17.778068+0000","last_peered":"2023-06-09T12:03:17.778068+0000","last_clean":"2023-06-09T12:03:17.778068+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:03:17.778068+0000","last_undegraded":"2023-06-09T12:03:17.778068+0000","last_fullsized":"2023-06-09T12:03:17.778068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400263+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400263+0000","last_peered":"2023-06-09T12:05:55.400263+0000","last_clean":"2023-06-09T12:05:55.400263+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400263+0000","last_undegraded":"2023-06-09T12:05:55.400263+0000","last_fullsized":"2023-06-09T12:05:55.400263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1349,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:07.311696+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:02:07.311696+0000","last_peered":"2023-06-09T12:02:07.311696+0000","last_clean":"2023-06-09T12:02:07.311696+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:02:07.311696+0000","last_undegraded":"2023-06-09T12:02:07.311696+0000","last_fullsized":"2023-06-09T12:02:07.311696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3589,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746760+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746760+0000","last_peered":"2023-06-09T12:05:54.746760+0000","last_clean":"2023-06-09T12:05:54.746760+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746760+0000","last_undegraded":"2023-06-09T12:05:54.746760+0000","last_fullsized":"2023-06-09T12:05:54.746760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.426475+0000","last_change":"2023-06-09T12:05:53.666015+0000","last_active":"2023-06-09T12:04:52.426475+0000","last_peered":"2023-06-09T12:04:52.426475+0000","last_clean":"2023-06-09T12:04:52.426475+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:04:52.426475+0000","last_undegraded":"2023-06-09T12:04:52.426475+0000","last_fullsized":"2023-06-09T12:04:52.426475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2990,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:01.358848+0000","last_change":"2023-06-09T12:05:02.326837+0000","last_active":"2023-06-09T12:04:01.358848+0000","last_peered":"2023-06-09T12:04:01.358848+0000","last_clean":"2023-06-09T12:04:01.358848+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:04:01.358848+0000","last_undegraded":"2023-06-09T12:04:01.358848+0000","last_fullsized":"2023-06-09T12:04:01.358848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:54.261006+0000","last_change":"2023-06-09T12:05:02.326991+0000","last_active":"2023-06-09T12:04:54.261006+0000","last_peered":"2023-06-09T12:04:54.261006+0000","last_clean":"2023-06-09T12:04:54.261006+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:04:54.261006+0000","last_undegraded":"2023-06-09T12:04:54.261006+0000","last_fullsized":"2023-06-09T12:04:54.261006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4116,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326908+0000","last_change":"2023-06-09T12:05:51.360005+0000","last_active":"2023-06-09T12:05:02.326908+0000","last_peered":"2023-06-09T12:05:02.326908+0000","last_clean":"2023-06-09T12:05:02.326908+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:05:02.326908+0000","last_undegraded":"2023-06-09T12:05:02.326908+0000","last_fullsized":"2023-06-09T12:05:02.326908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":163,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:57.297935+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:01:57.297935+0000","last_peered":"2023-06-09T12:01:57.297935+0000","last_clean":"2023-06-09T12:01:57.297935+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:01:57.297935+0000","last_undegraded":"2023-06-09T12:01:57.297935+0000","last_fullsized":"2023-06-09T12:01:57.297935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3371,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.318985+0000","last_change":"2023-06-09T12:01:02.302532+0000","last_active":"2023-06-09T12:03:02.318985+0000","last_peered":"2023-06-09T12:03:02.318985+0000","last_clean":"2023-06-09T12:03:02.318985+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:03:02.318985+0000","last_undegraded":"2023-06-09T12:03:02.318985+0000","last_fullsized":"2023-06-09T12:03:02.318985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.112083+0000","last_change":"2023-06-09T12:04:57.519309+0000","last_active":"2023-06-09T12:03:57.112083+0000","last_peered":"2023-06-09T12:03:57.112083+0000","last_clean":"2023-06-09T12:03:57.112083+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:03:57.112083+0000","last_undegraded":"2023-06-09T12:03:57.112083+0000","last_fullsized":"2023-06-09T12:03:57.112083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:28.374029+0000","last_change":"2023-06-09T12:05:02.326978+0000","last_active":"2023-06-09T12:04:28.374029+0000","last_peered":"2023-06-09T12:04:28.374029+0000","last_clean":"2023-06-09T12:04:28.374029+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:04:28.374029+0000","last_undegraded":"2023-06-09T12:04:28.374029+0000","last_fullsized":"2023-06-09T12:04:28.374029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":885,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.346421+0000","last_change":"2023-06-09T12:05:52.332905+0000","last_active":"2023-06-09T12:04:52.346421+0000","last_peered":"2023-06-09T12:04:52.346421+0000","last_clean":"2023-06-09T12:04:52.346421+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:04:52.346421+0000","last_undegraded":"2023-06-09T12:04:52.346421+0000","last_fullsized":"2023-06-09T12:04:52.346421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":773,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.357458+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:05:52.357458+0000","last_peered":"2023-06-09T12:05:52.357458+0000","last_clean":"2023-06-09T12:05:52.357458+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:05:52.357458+0000","last_undegraded":"2023-06-09T12:05:52.357458+0000","last_fullsized":"2023-06-09T12:05:52.357458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3827,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332309+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332309+0000","last_peered":"2023-06-09T12:03:58.332309+0000","last_clean":"2023-06-09T12:03:58.332309+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332309+0000","last_undegraded":"2023-06-09T12:03:58.332309+0000","last_fullsized":"2023-06-09T12:03:58.332309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1063,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.313878+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:02:22.313878+0000","last_peered":"2023-06-09T12:02:22.313878+0000","last_clean":"2023-06-09T12:02:22.313878+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:02:22.313878+0000","last_undegraded":"2023-06-09T12:02:22.313878+0000","last_fullsized":"2023-06-09T12:02:22.313878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1624,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305572+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:02:12.305572+0000","last_peered":"2023-06-09T12:02:12.305572+0000","last_clean":"2023-06-09T12:02:12.305572+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:02:12.305572+0000","last_undegraded":"2023-06-09T12:02:12.305572+0000","last_fullsized":"2023-06-09T12:02:12.305572+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:56.996951+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:02:56.996951+0000","last_peered":"2023-06-09T12:02:56.996951+0000","last_clean":"2023-06-09T12:02:56.996951+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:02:56.996951+0000","last_undegraded":"2023-06-09T12:02:56.996951+0000","last_fullsized":"2023-06-09T12:02:56.996951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.124241+0000","last_change":"2023-06-09T12:05:02.519982+0000","last_active":"2023-06-09T12:03:59.124241+0000","last_peered":"2023-06-09T12:03:59.124241+0000","last_clean":"2023-06-09T12:03:59.124241+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:59.124241+0000","last_undegraded":"2023-06-09T12:03:59.124241+0000","last_fullsized":"2023-06-09T12:03:59.124241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":64099,"ondisk_log_size":64099,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1260437,"num_omap_keys":2391,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30799,"ondisk_log_size":30799,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575517,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":259984,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80128,"kb_avail":94111856,"statfs":{"total":96636764160,"available":96370540544,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82051072},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.63},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71699999999999997,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.0549999999999999,"5min":1.256,"15min":1.3320000000000001},"last":0.64700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72699999999999998,"15min":0.68999999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.28199999999999997},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":1.081},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.05},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.85499999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.72899999999999998},"min":{"1min":0.53500000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.50700000000000001,"5min":0.441,"15min":0.379},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72399999999999998,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.629,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.32400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":1.1899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575517,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208664,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52544,"kb_avail":94163176,"statfs":{"total":96636764160,"available":96423092224,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53805056},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.90100000000000002,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71699999999999997,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.72999999999999998},"min":{"1min":0.621,"5min":0.60499999999999998,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72499999999999998,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.98499999999999999,"15min":1.024},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.64900000000000002,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.871,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77200000000000002,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.92200000000000004,"5min":1.03,"15min":1.0369999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80600000000000005,"15min":0.753},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.94999999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.748}]},{"osd":4,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.57799999999999996},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.35199999999999998},"max":{"1min":0.86499999999999999,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.58299999999999996,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.052},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65600000000000003,"15min":0.64700000000000002},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.071},"last":0.58799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575518,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406256,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99840,"kb_avail":93965584,"statfs":{"total":96636764160,"available":96220758016,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102236160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.498}]},{"osd":2,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.47299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28000000000000003},"max":{"1min":0.81299999999999994,"5min":0.995,"15min":0.995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.83799999999999997,"5min":0.86099999999999999,"15min":0.871},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.79800000000000004,"15min":0.90200000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.38700000000000001},"max":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":1.091},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77700000000000002,"15min":0.70499999999999996},"min":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.0569999999999999,"15min":1.0649999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.877,"15min":0.82599999999999996},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.75}]},{"osd":5,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.85899999999999999,"15min":0.79800000000000004},"min":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90200000000000002,"15min":0.84099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.82799999999999996},"min":{"1min":0.71199999999999997,"5min":0.59999999999999998,"15min":0.504},"max":{"1min":1.155,"5min":1.27,"15min":1.4039999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82499999999999996,"15min":0.76500000000000001},"min":{"1min":0.67100000000000004,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.92000000000000004,"15min":0.878},"min":{"1min":0.72499999999999998,"5min":0.68999999999999995,"15min":0.56299999999999994},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82099999999999995,"15min":0.77600000000000002},"min":{"1min":0.64300000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.2150000000000001,"5min":1.355,"15min":1.355},"last":0.872}]}]},{"osd":0,"up_from":13,"seq":55834575517,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332968,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":105152,"kb_avail":94038872,"statfs":{"total":96636764160,"available":96295804928,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107675648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45300000000000001,"15min":0.443},"min":{"1min":0.34300000000000003,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.443,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.70999999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56000000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58999999999999997,"15min":0.54900000000000004},"min":{"1min":0.38300000000000001,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.72299999999999998,"5min":1.032,"15min":1.032},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.502},"min":{"1min":0.36199999999999999,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.73399999999999999,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.46500000000000002,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77400000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.59099999999999997,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.84099999999999997,"5min":0.98899999999999999,"15min":1.024},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80700000000000005,"15min":0.72099999999999997},"min":{"1min":0.68300000000000005,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":0.99299999999999999,"5min":1.175,"15min":1.175},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.86199999999999999,"15min":0.79900000000000004},"min":{"1min":0.77200000000000002,"5min":0.58799999999999997,"15min":0.42899999999999999},"max":{"1min":1.0489999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.98799999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.77500000000000002,"15min":0.753},"min":{"1min":0.72999999999999998,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.012,"5min":1.034,"15min":1.236},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.85899999999999999,"15min":0.78000000000000003},"min":{"1min":0.68000000000000005,"5min":0.63200000000000001,"15min":0.46600000000000003},"max":{"1min":1.103,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.89200000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.083,"5min":1.1850000000000001,"15min":9.0790000000000006},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.84299999999999997,"15min":0.83199999999999996},"min":{"1min":0.72799999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0589999999999999,"5min":1.1100000000000001,"15min":1.3149999999999999},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:05:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.85899999999999999,"15min":0.79300000000000004},"min":{"1min":0.69799999999999995,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.1100000000000001,"5min":1.298,"15min":1.298},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.94099999999999995,"15min":0.85699999999999998},"min":{"1min":0.84599999999999997,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1359999999999999,"5min":1.351,"15min":1.351},"last":1.077}]}]},{"osd":2,"up_from":13,"seq":55834575518,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242528,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62720,"kb_avail":94129312,"statfs":{"total":96636764160,"available":96388415488,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64225280},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.45800000000000002},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":1.367},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.85299999999999998,"15min":1.546},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.014,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78500000000000003,"15min":0.76400000000000001},"min":{"1min":0.60799999999999998,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":0.629}]},{"osd":5,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83199999999999996,"15min":0.79900000000000004},"min":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.96299999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.872,"15min":0.83999999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.77500000000000002},"min":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.438},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.3029999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77800000000000002,"15min":0.76300000000000001},"min":{"1min":0.629,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.056,"15min":1.3540000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84099999999999997,"15min":0.79800000000000004},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.497},"max":{"1min":1.069,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":0.753},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83999999999999997,"15min":0.82499999999999996},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.3080000000000001},"last":0.81699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575517,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422524,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":99328,"kb_avail":93949316,"statfs":{"total":96636764160,"available":96204099584,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101711872},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.58099999999999996,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.26900000000000002},"max":{"1min":0.81999999999999995,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.51100000000000001},"min":{"1min":0.39800000000000002,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77400000000000002,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65500000000000003,"15min":0.64000000000000001},"min":{"1min":0.52300000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":0.97099999999999997,"15min":1.355},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56699999999999995,"15min":0.54700000000000004},"min":{"1min":0.39000000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.85899999999999999,"5min":0.91200000000000003,"15min":1.198},"last":0.54800000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79600000000000004,"15min":0.73499999999999999},"min":{"1min":0.67100000000000004,"5min":0.51500000000000001,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.073,"15min":1.635},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.748},"min":{"1min":0.61199999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":1.0369999999999999,"5min":1.0980000000000001,"15min":3.081},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83099999999999996,"15min":0.77400000000000002},"min":{"1min":0.66300000000000003,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.038,"5min":1.165,"15min":1.2110000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88400000000000001,"15min":0.81399999999999995},"min":{"1min":0.74199999999999999,"5min":0.66600000000000004,"15min":0.47899999999999998},"max":{"1min":1.095,"5min":1.1970000000000001,"15min":1.367},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.78300000000000003},"min":{"1min":0.627,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":0.94799999999999995,"5min":1.2649999999999999,"15min":1.286},"last":0.997},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83099999999999996,"15min":0.77900000000000003},"min":{"1min":0.65800000000000003,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0860000000000001,"5min":1.103,"15min":3.3490000000000002},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:05:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83399999999999996,"15min":0.81000000000000005},"min":{"1min":0.628,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.306,"15min":1.413},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.85599999999999998,"15min":0.85899999999999999},"min":{"1min":0.70299999999999996,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0740000000000001,"5min":1.1659999999999999,"15min":3.1160000000000001},"last":0.94299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575516,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423380,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130624,"kb_avail":93948460,"statfs":{"total":96636764160,"available":96203223040,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133758976},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72399999999999998,"15min":0.67500000000000004},"min":{"1min":0.49099999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":0.77600000000000002,"5min":1.0640000000000001,"15min":1.137},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78700000000000003,"15min":0.749},"min":{"1min":0.58799999999999997,"5min":0.54000000000000004,"15min":0.43099999999999999},"max":{"1min":0.85199999999999998,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.75,"15min":0.72099999999999997},"min":{"1min":0.63800000000000001,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.95199999999999996,"5min":1.002,"15min":1.0640000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":1.03,"15min":1.1279999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.73299999999999998,"15min":0.66300000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83299999999999996,"5min":1.117,"15min":1.609},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.68899999999999995},"min":{"1min":0.63100000000000001,"5min":0.59499999999999997,"15min":0.38100000000000001},"max":{"1min":1.0129999999999999,"5min":1.147,"15min":1.159},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":1.014,"15min":1.407},"last":0.622},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72999999999999998,"15min":0.73199999999999998},"min":{"1min":0.53600000000000003,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.57499999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.53400000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.63600000000000001,"15min":0.61799999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.1719999999999999},"last":0.59299999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58999999999999997,"15min":0.56999999999999995},"min":{"1min":0.41599999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.76400000000000001,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.70299999999999996},"min":{"1min":0.55900000000000005,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":1.131,"5min":1.131,"15min":1.3100000000000001},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66300000000000003,"15min":0.60999999999999999},"min":{"1min":0.48499999999999999,"5min":0.441,"15min":0.27400000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.496},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.361},"last":0.46500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575516,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293464,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76096,"kb_avail":94078376,"statfs":{"total":96636764160,"available":96336257024,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77922304},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56599999999999995,"5min":0.44,"15min":0.318},"max":{"1min":0.90700000000000003,"5min":1.177,"15min":5.6269999999999998},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":1.083,"15min":19.715},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.59299999999999997,"5min":0.45700000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.71999999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.54500000000000004,"5min":0.53800000000000003,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":1.145,"15min":14.439},"last":0.78400000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.75700000000000001},"min":{"1min":0.5,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.81399999999999995,"5min":1.006,"15min":12.130000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.624,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59499999999999997,"15min":0.83199999999999996},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.56000000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63800000000000001,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":12.163},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.76900000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.81999999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":1.135,"15min":19.834},"last":0.81699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:06:16.380 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.381 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.381 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.382 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.382 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.382 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.383 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.383 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.383 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.384 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.384 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.384 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.385 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.385 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.386 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.386 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.386 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.387 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.387 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.387 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.388 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.388 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.388 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.389 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.389 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.389 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.390 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.390 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.390 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.391 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.391 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.391 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.392 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.392 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.393 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.393 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.393 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.394 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.394 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.394 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.395 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.395 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.395 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.396 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.396 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.396 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.397 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.397 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.397 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.398 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.398 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.399 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.399 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.399 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.400 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.400 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:16.400 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:06:27.640 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:27.645 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:36.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:06:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.610+0000 7fdffafff700 1 Processor -- start 2023-06-09T12:06:36.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.610+0000 7fdffafff700 1 -- start start 2023-06-09T12:06:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdffafff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdff40c1bc0 0x7fdff41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdffafff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff40c0bf0 con 0x7fdff40c0140 2023-06-09T12:06:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdffafff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdff40702f0 con 0x7fdff40c1f90 2023-06-09T12:06:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdffafff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdff4070110 con 0x7fdff40c1bc0 2023-06-09T12:06:36.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdff8d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdff40c1bc0 0x7fdff41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdff8d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdff40c1bc0 0x7fdff41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36448/0 (socket says 172.21.15.7:36448) 2023-06-09T12:06:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.611+0000 7fdff8d9b700 1 -- 172.21.15.7:0/2392907375 learned_addr learned my addr 172.21.15.7:0/2392907375 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:36.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff8d9b700 1 -- 172.21.15.7:0/2392907375 >> v1:172.21.15.117:6789/0 conn(0x7fdff40c1f90 legacy=0x7fdff4160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff8d9b700 1 -- 172.21.15.7:0/2392907375 >> v1:172.21.15.7:6789/0 conn(0x7fdff40c0140 legacy=0x7fdff41501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff8d9b700 1 -- 172.21.15.7:0/2392907375 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40c0e90 con 0x7fdff40c1bc0 2023-06-09T12:06:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff8d9b700 1 --2- 172.21.15.7:0/2392907375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c1bc0 0x7fdff41599b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098f0 tx=0x7fdfe8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e66282ff01a8b7c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff37fe700 1 -- 172.21.15.7:0/2392907375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800f8a0 con 0x7fdff40c1bc0 2023-06-09T12:06:36.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff37fe700 1 -- 172.21.15.7:0/2392907375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe800fa00 con 0x7fdff40c1bc0 2023-06-09T12:06:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.612+0000 7fdff37fe700 1 -- 172.21.15.7:0/2392907375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe800fd90 con 0x7fdff40c1bc0 2023-06-09T12:06:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 -- 172.21.15.7:0/2392907375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c1bc0 msgr2=0x7fdff41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 --2- 172.21.15.7:0/2392907375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c1bc0 0x7fdff41599b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80098f0 tx=0x7fdfe8009c00 comp rx=0 tx=0).stop 2023-06-09T12:06:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 -- 172.21.15.7:0/2392907375 shutdown_connections 2023-06-09T12:06:36.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 --2- 172.21.15.7:0/2392907375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c1bc0 0x7fdff41599b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 -- 172.21.15.7:0/2392907375 >> 172.21.15.7:0/2392907375 conn(0x7fdff406a4d0 msgr2=0x7fdff406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 -- 172.21.15.7:0/2392907375 shutdown_connections 2023-06-09T12:06:36.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.613+0000 7fdffafff700 1 -- 172.21.15.7:0/2392907375 wait complete. 2023-06-09T12:06:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.615+0000 7fdffafff700 1 Processor -- start 2023-06-09T12:06:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.615+0000 7fdffafff700 1 -- start start 2023-06-09T12:06:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:36.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdff40c1bc0 0x7fdff41f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdff40c1f90 0x7fdff41511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40b05f0 con 0x7fdff40c0140 2023-06-09T12:06:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdff41f0590 con 0x7fdff40c1bc0 2023-06-09T12:06:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdffafff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdff40c3870 con 0x7fdff40c1f90 2023-06-09T12:06:36.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36464/0 (socket says 172.21.15.7:36464) 2023-06-09T12:06:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 -- 172.21.15.7:0/1054997916 learned_addr learned my addr 172.21.15.7:0/1054997916 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff3fff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdff40c1bc0 0x7fdff41f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff959c700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdff40c1f90 0x7fdff41511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:36.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 -- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdff40c1f90 msgr2=0x7fdff41511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdff40c1f90 0x7fdff41511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 -- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdff40c1bc0 msgr2=0x7fdff41f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdff40c1bc0 0x7fdff41f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.616+0000 7fdff8d9b700 1 -- 172.21.15.7:0/1054997916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8009550 con 0x7fdff40c0140 2023-06-09T12:06:36.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdff8d9b700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdff406faf0 tx=0x7fdfe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8003ab0 con 0x7fdff40c0140 2023-06-09T12:06:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe8003c10 con 0x7fdff40c0140 2023-06-09T12:06:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdff40c1450 con 0x7fdff40c0140 2023-06-09T12:06:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8010470 con 0x7fdff40c0140 2023-06-09T12:06:36.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.617+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdff40c1170 con 0x7fdff40c0140 2023-06-09T12:06:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.618+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff40705d0 con 0x7fdff40c0140 2023-06-09T12:06:36.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.619+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdfe8024450 con 0x7fdff40c0140 2023-06-09T12:06:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.620+0000 7fdff17fa700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.620+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fdfe808b4e0 con 0x7fdff40c0140 2023-06-09T12:06:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.620+0000 7fdff3fff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:36.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.621+0000 7fdff3fff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 0x7fdfd806d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdff406eca0 tx=0x7fdfe0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:36.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.625+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdfe8055c10 con 0x7fdff40c0140 2023-06-09T12:06:36.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.842+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdff4156470 con 0x7fdfd806ace0 2023-06-09T12:06:36.854 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.851+0000 7fdff17fa700 1 -- 172.21.15.7:0/1054997916 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316494 (secure 0 0 0) 0x7fdff4156470 con 0x7fdfd806ace0 2023-06-09T12:06:36.857 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:06:36.859 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 msgr2=0x7fdfd806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 0x7fdfd806d190 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fdff406eca0 tx=0x7fdfe0009040 comp rx=0 tx=0).stop 2023-06-09T12:06:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 msgr2=0x7fdff41f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdff406faf0 tx=0x7fdfe8004750 comp rx=0 tx=0).stop 2023-06-09T12:06:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 shutdown_connections 2023-06-09T12:06:36.860 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdfd806ace0 0x7fdfd806d190 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdff40c1f90 0x7fdff41511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdff40c0140 0x7fdff41f0050 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 --2- 172.21.15.7:0/1054997916 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdff40c1bc0 0x7fdff41f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 >> 172.21.15.7:0/1054997916 conn(0x7fdff406a4d0 msgr2=0x7fdff40c5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:36.861 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 shutdown_connections 2023-06-09T12:06:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:36.856+0000 7fdffafff700 1 -- 172.21.15.7:0/1054997916 wait complete. 2023-06-09T12:06:36.862 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:06:36.877 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1744,"stamp":"2023-06-09T12:06:35.041530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95361,"ondisk_log_size":95361,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2590408,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":707072,"kb_avail":752384312,"statfs":{"total":773094113280,"available":770441535488,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":724041728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215545+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215545+0000","last_peered":"2023-06-09T12:06:15.215545+0000","last_clean":"2023-06-09T12:06:15.215545+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215545+0000","last_undegraded":"2023-06-09T12:06:15.215545+0000","last_fullsized":"2023-06-09T12:06:15.215545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3225,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.126507+0000","last_change":"2023-06-09T12:05:52.126507+0000","last_active":"2023-06-09T12:05:52.126507+0000","last_peered":"2023-06-09T12:05:52.126507+0000","last_clean":"2023-06-09T12:05:52.126507+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:05:52.126507+0000","last_undegraded":"2023-06-09T12:05:52.126507+0000","last_fullsized":"2023-06-09T12:05:52.126507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332933+0000","last_change":"2023-06-09T12:05:55.319769+0000","last_active":"2023-06-09T12:05:52.332933+0000","last_peered":"2023-06-09T12:05:52.332933+0000","last_clean":"2023-06-09T12:05:52.332933+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:05:52.332933+0000","last_undegraded":"2023-06-09T12:05:52.332933+0000","last_fullsized":"2023-06-09T12:05:52.332933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3492,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:01:55.529709+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:01:55.529709+0000","last_peered":"2023-06-09T12:01:55.529709+0000","last_clean":"2023-06-09T12:01:55.529709+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:01:55.529709+0000","last_undegraded":"2023-06-09T12:01:55.529709+0000","last_fullsized":"2023-06-09T12:01:55.529709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4202,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:12.320379+0000","last_change":"2023-06-09T12:06:04.285410+0000","last_active":"2023-06-09T12:04:12.320379+0000","last_peered":"2023-06-09T12:04:12.320379+0000","last_clean":"2023-06-09T12:04:12.320379+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:04:12.320379+0000","last_undegraded":"2023-06-09T12:04:12.320379+0000","last_fullsized":"2023-06-09T12:04:12.320379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3413,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.656755+0000","last_change":"2023-06-09T12:06:27.412175+0000","last_active":"2023-06-09T12:06:18.656755+0000","last_peered":"2023-06-09T12:06:18.656755+0000","last_clean":"2023-06-09T12:06:18.656755+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:06:18.656755+0000","last_undegraded":"2023-06-09T12:06:18.656755+0000","last_fullsized":"2023-06-09T12:06:18.656755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3738,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.323061+0000","last_change":"2023-06-09T12:05:59.293378+0000","last_active":"2023-06-09T12:04:32.323061+0000","last_peered":"2023-06-09T12:04:32.323061+0000","last_clean":"2023-06-09T12:04:32.323061+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:04:32.323061+0000","last_undegraded":"2023-06-09T12:04:32.323061+0000","last_fullsized":"2023-06-09T12:04:32.323061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3493,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.441395+0000","last_change":"2023-06-09T12:05:55.201855+0000","last_active":"2023-06-09T12:04:32.441395+0000","last_peered":"2023-06-09T12:04:32.441395+0000","last_clean":"2023-06-09T12:04:32.441395+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:04:32.441395+0000","last_undegraded":"2023-06-09T12:04:32.441395+0000","last_fullsized":"2023-06-09T12:04:32.441395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3386,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030491+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030491+0000","last_peered":"2023-06-09T12:04:42.030491+0000","last_clean":"2023-06-09T12:04:42.030491+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030491+0000","last_undegraded":"2023-06-09T12:04:42.030491+0000","last_fullsized":"2023-06-09T12:04:42.030491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3486,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:41.406462+0000","last_change":"2023-06-09T12:05:41.406462+0000","last_active":"2023-06-09T12:05:41.406462+0000","last_peered":"2023-06-09T12:05:41.406462+0000","last_clean":"2023-06-09T12:05:41.406462+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:05:41.406462+0000","last_undegraded":"2023-06-09T12:05:41.406462+0000","last_fullsized":"2023-06-09T12:05:41.406462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.924294+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:02:38.924294+0000","last_peered":"2023-06-09T12:02:38.924294+0000","last_clean":"2023-06-09T12:02:38.924294+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:02:38.924294+0000","last_undegraded":"2023-06-09T12:02:38.924294+0000","last_fullsized":"2023-06-09T12:02:38.924294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1026","reported_seq":3691,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:30.813309+0000","last_change":"2023-06-09T12:05:51.156280+0000","last_active":"2023-06-09T12:06:30.813309+0000","last_peered":"2023-06-09T12:06:30.813309+0000","last_clean":"2023-06-09T12:06:30.813309+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:06:30.813309+0000","last_undegraded":"2023-06-09T12:06:30.813309+0000","last_fullsized":"2023-06-09T12:06:30.813309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":1026,"log_dups_size":0,"ondisk_log_size":1026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3140,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:44.454243+0000","last_change":"2023-06-09T12:05:44.454243+0000","last_active":"2023-06-09T12:05:44.454243+0000","last_peered":"2023-06-09T12:05:44.454243+0000","last_clean":"2023-06-09T12:05:44.454243+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:05:44.454243+0000","last_undegraded":"2023-06-09T12:05:44.454243+0000","last_fullsized":"2023-06-09T12:05:44.454243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3311,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.082714+0000","last_change":"2023-06-09T12:05:02.520050+0000","last_active":"2023-06-09T12:03:58.082714+0000","last_peered":"2023-06-09T12:03:58.082714+0000","last_clean":"2023-06-09T12:03:58.082714+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:03:58.082714+0000","last_undegraded":"2023-06-09T12:03:58.082714+0000","last_fullsized":"2023-06-09T12:03:58.082714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2966,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:32.330969+0000","last_change":"2023-06-09T12:06:20.832681+0000","last_active":"2023-06-09T12:04:32.330969+0000","last_peered":"2023-06-09T12:04:32.330969+0000","last_clean":"2023-06-09T12:04:32.330969+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:04:32.330969+0000","last_undegraded":"2023-06-09T12:04:32.330969+0000","last_fullsized":"2023-06-09T12:04:32.330969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3279,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:29.148543+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:02:29.148543+0000","last_peered":"2023-06-09T12:02:29.148543+0000","last_clean":"2023-06-09T12:02:29.148543+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:02:29.148543+0000","last_undegraded":"2023-06-09T12:02:29.148543+0000","last_fullsized":"2023-06-09T12:02:29.148543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1174","reported_seq":3337,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:27.848862+0000","last_change":"2023-06-09T12:05:55.824750+0000","last_active":"2023-06-09T12:06:27.848862+0000","last_peered":"2023-06-09T12:06:27.848862+0000","last_clean":"2023-06-09T12:06:27.848862+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:27.848862+0000","last_undegraded":"2023-06-09T12:06:27.848862+0000","last_fullsized":"2023-06-09T12:06:27.848862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":217,"log_size":1174,"log_dups_size":0,"ondisk_log_size":1174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 32s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3099,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.033567+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:03:02.033567+0000","last_peered":"2023-06-09T12:03:02.033567+0000","last_clean":"2023-06-09T12:03:02.033567+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:03:02.033567+0000","last_undegraded":"2023-06-09T12:03:02.033567+0000","last_fullsized":"2023-06-09T12:03:02.033567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3562,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:42.332315+0000","last_change":"2023-06-09T12:06:24.871532+0000","last_active":"2023-06-09T12:04:42.332315+0000","last_peered":"2023-06-09T12:04:42.332315+0000","last_clean":"2023-06-09T12:04:42.332315+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:04:42.332315+0000","last_undegraded":"2023-06-09T12:04:42.332315+0000","last_fullsized":"2023-06-09T12:04:42.332315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:12.305422+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:02:12.305422+0000","last_peered":"2023-06-09T12:02:12.305422+0000","last_clean":"2023-06-09T12:02:12.305422+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:02:12.305422+0000","last_undegraded":"2023-06-09T12:02:12.305422+0000","last_fullsized":"2023-06-09T12:02:12.305422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.965058+0000","last_change":"2023-06-09T12:06:27.362178+0000","last_active":"2023-06-09T12:06:18.965058+0000","last_peered":"2023-06-09T12:06:18.965058+0000","last_clean":"2023-06-09T12:06:18.965058+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:06:18.965058+0000","last_undegraded":"2023-06-09T12:06:18.965058+0000","last_fullsized":"2023-06-09T12:06:18.965058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1000","reported_seq":3081,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:30.203656+0000","last_change":"2023-06-09T12:05:50.361843+0000","last_active":"2023-06-09T12:06:30.203656+0000","last_peered":"2023-06-09T12:06:30.203656+0000","last_clean":"2023-06-09T12:06:30.203656+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:30.203656+0000","last_undegraded":"2023-06-09T12:06:30.203656+0000","last_fullsized":"2023-06-09T12:06:30.203656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":48,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428433+0000","last_change":"2023-06-09T12:00:42.410140+0000","last_active":"2023-06-09T12:02:52.428433+0000","last_peered":"2023-06-09T12:02:52.428433+0000","last_clean":"2023-06-09T12:02:52.428433+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:02:52.428433+0000","last_undegraded":"2023-06-09T12:02:52.428433+0000","last_fullsized":"2023-06-09T12:02:52.428433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3545,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:12.521142+0000","last_change":"2023-06-09T12:05:54.809296+0000","last_active":"2023-06-09T12:05:12.521142+0000","last_peered":"2023-06-09T12:05:12.521142+0000","last_clean":"2023-06-09T12:05:12.521142+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:05:12.521142+0000","last_undegraded":"2023-06-09T12:05:12.521142+0000","last_fullsized":"2023-06-09T12:05:12.521142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":697,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:01.131264+0000","last_change":"2023-06-09T12:06:01.131264+0000","last_active":"2023-06-09T12:06:01.131264+0000","last_peered":"2023-06-09T12:06:01.131264+0000","last_clean":"2023-06-09T12:06:01.131264+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:06:01.131264+0000","last_undegraded":"2023-06-09T12:06:01.131264+0000","last_fullsized":"2023-06-09T12:06:01.131264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4071,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.325633+0000","last_change":"2023-06-09T12:05:57.344054+0000","last_active":"2023-06-09T12:04:52.325633+0000","last_peered":"2023-06-09T12:04:52.325633+0000","last_clean":"2023-06-09T12:04:52.325633+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:04:52.325633+0000","last_undegraded":"2023-06-09T12:04:52.325633+0000","last_fullsized":"2023-06-09T12:04:52.325633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1379,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590464+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590464+0000","last_peered":"2023-06-09T12:04:27.590464+0000","last_clean":"2023-06-09T12:04:27.590464+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590464+0000","last_undegraded":"2023-06-09T12:04:27.590464+0000","last_fullsized":"2023-06-09T12:04:27.590464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'941","reported_seq":3035,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:31.092585+0000","last_change":"2023-06-09T12:06:20.273474+0000","last_active":"2023-06-09T12:06:31.092585+0000","last_peered":"2023-06-09T12:06:31.092585+0000","last_clean":"2023-06-09T12:06:31.092585+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:31.092585+0000","last_undegraded":"2023-06-09T12:06:31.092585+0000","last_fullsized":"2023-06-09T12:06:31.092585+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":98,"log_size":941,"log_dups_size":0,"ondisk_log_size":941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":757,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334635+0000","last_change":"2023-06-09T12:05:30.983184+0000","last_active":"2023-06-09T12:05:02.334635+0000","last_peered":"2023-06-09T12:05:02.334635+0000","last_clean":"2023-06-09T12:05:02.334635+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:02.334635+0000","last_undegraded":"2023-06-09T12:05:02.334635+0000","last_fullsized":"2023-06-09T12:05:02.334635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3600,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:17.778068+0000","last_change":"2023-06-09T12:05:02.322556+0000","last_active":"2023-06-09T12:03:17.778068+0000","last_peered":"2023-06-09T12:03:17.778068+0000","last_clean":"2023-06-09T12:03:17.778068+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:03:17.778068+0000","last_undegraded":"2023-06-09T12:03:17.778068+0000","last_fullsized":"2023-06-09T12:03:17.778068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400263+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400263+0000","last_peered":"2023-06-09T12:05:55.400263+0000","last_clean":"2023-06-09T12:05:55.400263+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400263+0000","last_undegraded":"2023-06-09T12:05:55.400263+0000","last_fullsized":"2023-06-09T12:05:55.400263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4136,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619009+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619009+0000","last_peered":"2023-06-09T12:04:38.619009+0000","last_clean":"2023-06-09T12:04:38.619009+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619009+0000","last_undegraded":"2023-06-09T12:04:38.619009+0000","last_fullsized":"2023-06-09T12:04:38.619009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:38.268961+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:02:38.268961+0000","last_peered":"2023-06-09T12:02:38.268961+0000","last_clean":"2023-06-09T12:02:38.268961+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:02:38.268961+0000","last_undegraded":"2023-06-09T12:02:38.268961+0000","last_fullsized":"2023-06-09T12:02:38.268961+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3589,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746760+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746760+0000","last_peered":"2023-06-09T12:05:54.746760+0000","last_clean":"2023-06-09T12:05:54.746760+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746760+0000","last_undegraded":"2023-06-09T12:05:54.746760+0000","last_fullsized":"2023-06-09T12:05:54.746760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.426475+0000","last_change":"2023-06-09T12:05:53.666015+0000","last_active":"2023-06-09T12:04:52.426475+0000","last_peered":"2023-06-09T12:04:52.426475+0000","last_clean":"2023-06-09T12:04:52.426475+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:04:52.426475+0000","last_undegraded":"2023-06-09T12:04:52.426475+0000","last_fullsized":"2023-06-09T12:04:52.426475+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2991,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326837+0000","last_change":"2023-06-09T12:06:12.336756+0000","last_active":"2023-06-09T12:05:02.326837+0000","last_peered":"2023-06-09T12:05:02.326837+0000","last_clean":"2023-06-09T12:05:02.326837+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:05:02.326837+0000","last_undegraded":"2023-06-09T12:05:02.326837+0000","last_fullsized":"2023-06-09T12:05:02.326837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326991+0000","last_change":"2023-06-09T12:06:17.370378+0000","last_active":"2023-06-09T12:05:02.326991+0000","last_peered":"2023-06-09T12:05:02.326991+0000","last_clean":"2023-06-09T12:05:02.326991+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:05:02.326991+0000","last_undegraded":"2023-06-09T12:05:02.326991+0000","last_fullsized":"2023-06-09T12:05:02.326991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4116,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326908+0000","last_change":"2023-06-09T12:05:51.360005+0000","last_active":"2023-06-09T12:05:02.326908+0000","last_peered":"2023-06-09T12:05:02.326908+0000","last_clean":"2023-06-09T12:05:02.326908+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:05:02.326908+0000","last_undegraded":"2023-06-09T12:05:02.326908+0000","last_fullsized":"2023-06-09T12:05:02.326908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3372,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334604+0000","last_change":"2023-06-09T12:06:21.882864+0000","last_active":"2023-06-09T12:05:02.334604+0000","last_peered":"2023-06-09T12:05:02.334604+0000","last_clean":"2023-06-09T12:05:02.334604+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:05:02.334604+0000","last_undegraded":"2023-06-09T12:05:02.334604+0000","last_fullsized":"2023-06-09T12:05:02.334604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":160,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.112083+0000","last_change":"2023-06-09T12:04:57.519309+0000","last_active":"2023-06-09T12:03:57.112083+0000","last_peered":"2023-06-09T12:03:57.112083+0000","last_clean":"2023-06-09T12:03:57.112083+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:03:57.112083+0000","last_undegraded":"2023-06-09T12:03:57.112083+0000","last_fullsized":"2023-06-09T12:03:57.112083+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":832,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326978+0000","last_change":"2023-06-09T12:06:14.354842+0000","last_active":"2023-06-09T12:05:02.326978+0000","last_peered":"2023-06-09T12:05:02.326978+0000","last_clean":"2023-06-09T12:05:02.326978+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:05:02.326978+0000","last_undegraded":"2023-06-09T12:05:02.326978+0000","last_fullsized":"2023-06-09T12:05:02.326978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":774,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:03.186309+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:03.186309+0000","last_peered":"2023-06-09T12:06:03.186309+0000","last_clean":"2023-06-09T12:06:03.186309+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:03.186309+0000","last_undegraded":"2023-06-09T12:06:03.186309+0000","last_fullsized":"2023-06-09T12:06:03.186309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3151,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:08.983351+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:03:08.983351+0000","last_peered":"2023-06-09T12:03:08.983351+0000","last_clean":"2023-06-09T12:03:08.983351+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:03:08.983351+0000","last_undegraded":"2023-06-09T12:03:08.983351+0000","last_fullsized":"2023-06-09T12:03:08.983351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3432,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.124241+0000","last_change":"2023-06-09T12:05:02.519982+0000","last_active":"2023-06-09T12:03:59.124241+0000","last_peered":"2023-06-09T12:03:59.124241+0000","last_clean":"2023-06-09T12:03:59.124241+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:03:59.124241+0000","last_undegraded":"2023-06-09T12:03:59.124241+0000","last_fullsized":"2023-06-09T12:03:59.124241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":64429,"ondisk_log_size":64429,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30813,"ondisk_log_size":30813,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575521,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260112,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80256,"kb_avail":94111728,"statfs":{"total":96636764160,"available":96370409472,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82182144},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.70599999999999996,"15min":0.67200000000000004},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71699999999999997,"15min":0.68999999999999995},"min":{"1min":0.53100000000000003,"5min":0.433,"15min":0.35299999999999998},"max":{"1min":1.0549999999999999,"5min":1.256,"15min":1.3320000000000001},"last":0.82299999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72699999999999998,"15min":0.68999999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.28199999999999997},"max":{"1min":0.89900000000000002,"5min":1.0229999999999999,"15min":1.081},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.48599999999999999,"15min":0.32900000000000001},"max":{"1min":0.97599999999999998,"5min":0.99099999999999999,"15min":1.05},"last":0.83499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52400000000000002,"5min":0.44800000000000001,"15min":0.38400000000000001},"max":{"1min":0.85499999999999998,"5min":0.97599999999999998,"15min":1.099},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.753,"15min":0.72899999999999998},"min":{"1min":0.53500000000000003,"5min":0.47099999999999997,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.70299999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.69099999999999995,"15min":0.67100000000000004},"min":{"1min":0.50700000000000001,"5min":0.441,"15min":0.379},"max":{"1min":0.85199999999999998,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.70799999999999996},"min":{"1min":0.52500000000000002,"5min":0.505,"15min":0.41499999999999998},"max":{"1min":0.84399999999999997,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54500000000000004,"15min":0.53400000000000003},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.501},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72399999999999998,"15min":0.68799999999999994},"min":{"1min":0.497,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.64500000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.60299999999999998},"min":{"1min":0.40500000000000003,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.89300000000000002,"5min":0.90700000000000003,"15min":1.3959999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.629,"15min":0.623},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.32400000000000001},"max":{"1min":0.83699999999999997,"5min":0.94999999999999996,"15min":0.97999999999999998},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:05:36 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.372},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.72799999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":1.1899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.78200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575521,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208728,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52608,"kb_avail":94163112,"statfs":{"total":96636764160,"available":96423026688,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":53870592},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66700000000000004,"15min":0.64000000000000001},"min":{"1min":0.441,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.90100000000000002,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.65000000000000002},"min":{"1min":0.52200000000000002,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.84399999999999997,"5min":0.96799999999999997,"15min":1.069},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71699999999999997,"15min":0.67500000000000004},"min":{"1min":0.498,"5min":0.42399999999999999,"15min":0.29999999999999999},"max":{"1min":0.873,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.76500000000000001,"15min":0.72999999999999998},"min":{"1min":0.621,"5min":0.60499999999999998,"15min":0.40799999999999997},"max":{"1min":0.873,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.66800000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72499999999999998,"15min":0.68600000000000005},"min":{"1min":0.503,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.84799999999999998,"5min":0.98499999999999999,"15min":1.024},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.73399999999999999},"min":{"1min":0.64900000000000002,"5min":0.54700000000000004,"15min":0.44800000000000001},"max":{"1min":0.871,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.621}]},{"osd":3,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77200000000000002,"15min":0.71299999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.92200000000000004,"5min":1.03,"15min":1.0369999999999999},"last":0.499},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80600000000000005,"15min":0.753},"min":{"1min":0.63500000000000001,"5min":0.55900000000000005,"15min":0.39500000000000002},"max":{"1min":0.94999999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60699999999999998,"15min":0.57799999999999996},"min":{"1min":0.438,"5min":0.41099999999999998,"15min":0.27400000000000002},"max":{"1min":0.84199999999999997,"5min":0.93300000000000005,"15min":0.95299999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.45400000000000001,"5min":0.371,"15min":0.31900000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.747}]},{"osd":5,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66700000000000004,"15min":0.67700000000000005},"min":{"1min":0.46999999999999997,"5min":0.45200000000000001,"15min":0.35199999999999998},"max":{"1min":0.86499999999999999,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.68899999999999995,"15min":0.65100000000000002},"min":{"1min":0.58299999999999996,"5min":0.40600000000000003,"15min":0.33800000000000002},"max":{"1min":0.91700000000000004,"5min":1.0129999999999999,"15min":1.052},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.70199999999999996,"15min":0.69299999999999995},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.65600000000000003,"15min":0.64700000000000002},"min":{"1min":0.503,"5min":0.42399999999999999,"15min":0.318},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.071},"last":0.79400000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575522,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406384,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99968,"kb_avail":93965456,"statfs":{"total":96636764160,"available":96220626944,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102367232},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.45100000000000001,"15min":0.41299999999999998},"min":{"1min":0.32200000000000001,"5min":0.318,"15min":0.248},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.56000000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.36899999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51300000000000001,"15min":0.47299999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.28000000000000003},"max":{"1min":0.81299999999999994,"5min":0.995,"15min":0.995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.83799999999999997,"5min":0.86099999999999999,"15min":0.871},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58199999999999996,"15min":0.56100000000000005},"min":{"1min":0.498,"5min":0.39600000000000002,"15min":0.34699999999999998},"max":{"1min":0.77400000000000002,"5min":0.79800000000000004,"15min":0.90200000000000002},"last":0.745},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67400000000000004,"15min":0.65200000000000002},"min":{"1min":0.55700000000000005,"5min":0.52100000000000002,"15min":0.38700000000000001},"max":{"1min":0.91000000000000003,"5min":0.93300000000000005,"15min":1.091},"last":0.80500000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.77700000000000002,"15min":0.70499999999999996},"min":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.014,"5min":1.0569999999999999,"15min":1.0649999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.877,"15min":0.82599999999999996},"min":{"1min":0.68100000000000005,"5min":0.64900000000000002,"15min":0.53800000000000003},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.004}]},{"osd":5,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.85899999999999999,"15min":0.79800000000000004},"min":{"1min":0.63100000000000001,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.083},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.90200000000000002,"15min":0.84099999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.1060000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.83499999999999996,"15min":0.82799999999999996},"min":{"1min":0.71199999999999997,"5min":0.59999999999999998,"15min":0.504},"max":{"1min":1.155,"5min":1.27,"15min":1.4039999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82499999999999996,"15min":0.76500000000000001},"min":{"1min":0.67100000000000004,"5min":0.53400000000000003,"15min":0.42299999999999999},"max":{"1min":0.999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.95499999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:05:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":0.92000000000000004,"15min":0.878},"min":{"1min":0.72499999999999998,"5min":0.68999999999999995,"15min":0.56299999999999994},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.82099999999999995,"15min":0.77600000000000002},"min":{"1min":0.64300000000000002,"5min":0.51400000000000001,"15min":0.47299999999999998},"max":{"1min":1.2150000000000001,"5min":1.355,"15min":1.355},"last":0.82699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575521,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":332968,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":105152,"kb_avail":94038872,"statfs":{"total":96636764160,"available":96295804928,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":107675648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.35099999999999998,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.82699999999999996},"last":0.53400000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.81899999999999995,"15min":1.032},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":0.96599999999999997,"15min":1.024},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.84199999999999997,"15min":0.74199999999999999},"min":{"1min":0.71199999999999997,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88800000000000001,"15min":0.81599999999999995},"min":{"1min":0.82799999999999996,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1879999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.73899999999999999,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.236},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.89400000000000002,"15min":0.79900000000000004},"min":{"1min":0.80100000000000005,"5min":0.63900000000000001,"15min":0.46600000000000003},"max":{"1min":1.1339999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.99299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.72299999999999998,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.874},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84699999999999998,"15min":0.83799999999999997},"min":{"1min":0.69099999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0700000000000001,"15min":1.1599999999999999},"last":1.147}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.80800000000000005},"min":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.298,"15min":1.298},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.873},"min":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.2589999999999999,"5min":1.351,"15min":1.351},"last":1.1990000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575522,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242656,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62848,"kb_avail":94129184,"statfs":{"total":96636764160,"available":96388284416,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64356352},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.42999999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.55300000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.68000000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48099999999999998,"15min":0.45800000000000002},"min":{"1min":0.34000000000000002,"5min":0.33900000000000002,"15min":0.26200000000000001},"max":{"1min":0.65500000000000003,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.64300000000000002,"15min":0.60699999999999998},"min":{"1min":0.46000000000000002,"5min":0.42199999999999999,"15min":0.36099999999999999},"max":{"1min":0.89000000000000001,"5min":0.89200000000000002,"15min":1.367},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60799999999999998,"15min":0.58699999999999997},"min":{"1min":0.49299999999999999,"5min":0.379,"15min":0.34399999999999997},"max":{"1min":0.72399999999999998,"5min":0.85299999999999998,"15min":1.546},"last":0.878},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.68700000000000006,"15min":0.64400000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.42799999999999999},"max":{"1min":0.86399999999999999,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.745},"min":{"1min":0.496,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.014,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.78500000000000003,"15min":0.76400000000000001},"min":{"1min":0.60799999999999998,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":1.05}]},{"osd":5,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83199999999999996,"15min":0.79900000000000004},"min":{"1min":0.61799999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.96299999999999997,"5min":1.3580000000000001,"15min":1.601},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.872,"15min":0.83999999999999997},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":0.99399999999999999,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.89100000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.77500000000000002},"min":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.438},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.3029999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.77800000000000002,"15min":0.76300000000000001},"min":{"1min":0.629,"5min":0.54300000000000004,"15min":0.46100000000000002},"max":{"1min":1.0329999999999999,"5min":1.056,"15min":1.3540000000000001},"last":0.96099999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.84099999999999997,"15min":0.79800000000000004},"min":{"1min":0.63900000000000001,"5min":0.58899999999999997,"15min":0.497},"max":{"1min":1.069,"5min":1.1659999999999999,"15min":1.2270000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.83999999999999997,"15min":0.82499999999999996},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.3080000000000001},"last":1.0029999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422588,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":99392,"kb_avail":93949252,"statfs":{"total":96636764160,"available":96204034048,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":101777408},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47899999999999998,"15min":0.46300000000000002},"min":{"1min":0.33100000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.58099999999999996,"5min":0.68799999999999994,"15min":1.7969999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57099999999999995,"15min":0.54600000000000004},"min":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.33600000000000002},"max":{"1min":0.68300000000000005,"5min":0.90900000000000003,"15min":1.5669999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52700000000000002,"15min":0.503},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52600000000000002,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77900000000000003,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.75800000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.51800000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":1.355},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.76300000000000001,"5min":0.91200000000000003,"15min":1.198},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":1.01,"15min":1.635},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.753},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":0.89700000000000002,"5min":1.0980000000000001,"15min":3.081},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82999999999999996,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.2110000000000001},"last":1.093},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88100000000000001,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.133,"15min":1.367},"last":1.1299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80500000000000005,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.496},"max":{"1min":1.0229999999999999,"5min":1.0309999999999999,"15min":1.286},"last":0.879},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.78600000000000003},"min":{"1min":0.71199999999999997,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":0.99399999999999999,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":1.0209999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.81299999999999994},"min":{"1min":0.69999999999999996,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.413},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.66300000000000003,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0289999999999999,"5min":1.137,"15min":3.1160000000000001},"last":1.056}]}]},{"osd":4,"up_from":13,"seq":55834575520,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423444,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130688,"kb_avail":93948396,"statfs":{"total":96636764160,"available":96203157504,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133824512},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.314},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":1.0289999999999999,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.72799999999999998},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.0640000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.995,"5min":1.03,"15min":1.1279999999999999},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73699999999999999,"15min":0.67700000000000005},"min":{"1min":0.64500000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.117,"15min":1.609},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.70399999999999996},"min":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.38100000000000001},"max":{"1min":1.107,"5min":1.147,"15min":1.159},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.501,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.014,"15min":1.407},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.55000000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.622},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":0.80200000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.61199999999999999,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.997,"5min":1.131,"15min":1.3100000000000001},"last":0.69599999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":1.091,"15min":1.361},"last":0.53700000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575520,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293528,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76160,"kb_avail":94078312,"statfs":{"total":96636764160,"available":96336191488,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":77987840},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.88500000000000001,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.745},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.56599999999999995,"5min":0.44,"15min":0.318},"max":{"1min":0.90700000000000003,"5min":1.177,"15min":5.6269999999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72199999999999998,"15min":0.74199999999999999},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.29899999999999999},"max":{"1min":0.87,"5min":1.083,"15min":19.715},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.751,"15min":0.72899999999999998},"min":{"1min":0.59299999999999997,"5min":0.45700000000000002,"15min":0.33400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.55600000000000005,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.88300000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.76400000000000001},"min":{"1min":0.54500000000000004,"5min":0.53800000000000003,"15min":0.39200000000000002},"max":{"1min":0.90300000000000002,"5min":1.145,"15min":14.439},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73399999999999999,"15min":0.75700000000000001},"min":{"1min":0.5,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.81399999999999995,"5min":1.006,"15min":12.130000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.79300000000000004,"15min":0.81299999999999994},"min":{"1min":0.624,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":0.98299999999999998,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.86499999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.81799999999999995},"min":{"1min":0.39400000000000002,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.78900000000000003,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.624,"5min":0.59499999999999997,"15min":0.83199999999999996},"min":{"1min":0.46300000000000002,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63800000000000001,"15min":0.68300000000000005},"min":{"1min":0.51000000000000001,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.93400000000000005,"5min":1.097,"15min":12.163},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63600000000000001,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.41599999999999998,"15min":0.35899999999999999},"max":{"1min":0.82399999999999995,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:05:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64000000000000001,"15min":0.67500000000000004},"min":{"1min":0.438,"5min":0.438,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76000000000000001,"15min":0.81999999999999995},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.41999999999999998},"max":{"1min":0.99299999999999999,"5min":1.135,"15min":19.834},"last":0.77100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:06:36.885 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.885 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.886 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.886 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.886 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.887 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.887 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.887 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.888 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.888 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.888 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.889 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.889 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.890 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.890 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.890 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.891 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.891 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.891 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.892 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.892 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.892 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.893 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.893 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.893 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.894 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.894 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.894 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.895 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.895 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.895 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.896 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.896 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.897 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.897 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.897 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.898 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.898 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.898 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.899 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.899 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.899 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.900 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.900 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.901 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.901 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.901 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.902 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.902 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.903 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.903 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.903 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.904 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.904 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:36.904 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:06:56.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.118+0000 7fa1e1fff700 1 Processor -- start 2023-06-09T12:06:57.121 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.118+0000 7fa1e1fff700 1 -- start start 2023-06-09T12:06:57.122 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.119+0000 7fa1e1fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.119+0000 7fa1e1fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0c0bf0 con 0x7fa1dc0c0140 2023-06-09T12:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1e1fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0702f0 con 0x7fa1dc0c1bc0 2023-06-09T12:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1e1fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc070110 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35160/0 (socket says 172.21.15.7:35160) 2023-06-09T12:06:57.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 -- 172.21.15.7:0/3697247332 learned_addr learned my addr 172.21.15.7:0/3697247332 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 -- 172.21.15.7:0/3697247332 >> v1:172.21.15.117:6789/0 conn(0x7fa1dc0c1bc0 legacy=0x7fa1dc1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 -- 172.21.15.7:0/3697247332 >> v1:172.21.15.7:6789/0 conn(0x7fa1dc0c0140 legacy=0x7fa1dc160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 -- 172.21.15.7:0/3697247332 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0c0e90 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.120+0000 7fa1daffd700 1 --2- 172.21.15.7:0/3697247332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00098f0 tx=0x7fa1d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7fe8af470d2a38 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1da7fc700 1 -- 172.21.15.7:0/3697247332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000b8a0 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1da7fc700 1 -- 172.21.15.7:0/3697247332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d000ba00 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1da7fc700 1 -- 172.21.15.7:0/3697247332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000bd90 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3697247332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 msgr2=0x7fa1dc1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3697247332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa1d00098f0 tx=0x7fa1d0009c00 comp rx=0 tx=0).stop 2023-06-09T12:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3697247332 shutdown_connections 2023-06-09T12:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3697247332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc1501b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3697247332 >> 172.21.15.7:0/3697247332 conn(0x7fa1dc06a4d0 msgr2=0x7fa1dc06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3697247332 shutdown_connections 2023-06-09T12:06:57.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.121+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3697247332 wait complete. 2023-06-09T12:06:57.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.123+0000 7fa1e1fff700 1 Processor -- start 2023-06-09T12:06:57.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.123+0000 7fa1e1fff700 1 -- start start 2023-06-09T12:06:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 0x7fa1dc14b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 0x7fa1dc14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc159500 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc159de0 con 0x7fa1dc0c1bc0 2023-06-09T12:06:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1e1fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc1562c0 con 0x7fa1dc0c0140 2023-06-09T12:06:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1dbfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35172/0 (socket says 172.21.15.7:35172) 2023-06-09T12:06:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.124+0000 7fa1dbfff700 1 -- 172.21.15.7:0/3094313120 learned_addr learned my addr 172.21.15.7:0/3094313120 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:06:57.131 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1daffd700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 0x7fa1dc14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1db7fe700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 0x7fa1dc14b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 -- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 msgr2=0x7fa1dc14b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 0x7fa1dc14b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 -- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 msgr2=0x7fa1dc14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 0x7fa1dc14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 -- 172.21.15.7:0/3094313120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0009550 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1daffd700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 0x7fa1dc14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:06:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1db7fe700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 0x7fa1dc14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:06:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1dbfff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400cc50 tx=0x7fa1d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:57.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d400ad80 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d400ce50 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0c1450 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003c40 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.125+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc0c1170 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.127+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc0705d0 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.128+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa1d40050d0 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.128+0000 7fa1c3fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 0x7fa1c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:06:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.128+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fa1d4089e80 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.129+0000 7fa1db7fe700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 0x7fa1c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:06:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.129+0000 7fa1db7fe700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 0x7fa1c806d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc14d2b0 tx=0x7fa1cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:06:57.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.133+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa1d4054640 con 0x7fa1dc15a0c0 2023-06-09T12:06:57.342 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.339+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1dc157d60 con 0x7fa1c806ace0 2023-06-09T12:06:57.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.348+0000 7fa1c3fff700 1 -- 172.21.15.7:0/3094313120 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316686 (secure 0 0 0) 0x7fa1dc157d60 con 0x7fa1c806ace0 2023-06-09T12:06:57.355 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:06:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 msgr2=0x7fa1c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 0x7fa1c806d190 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc14d2b0 tx=0x7fa1cc009040 comp rx=0 tx=0).stop 2023-06-09T12:06:57.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 msgr2=0x7fa1dc158080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:06:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400cc50 tx=0x7fa1d400a340 comp rx=0 tx=0).stop 2023-06-09T12:06:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 shutdown_connections 2023-06-09T12:06:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1dc15a0c0 0x7fa1dc158080 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa1c806ace0 0x7fa1c806d190 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa1dc0c0140 0x7fa1dc14b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 --2- 172.21.15.7:0/3094313120 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa1dc0c1bc0 0x7fa1dc14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:06:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 >> 172.21.15.7:0/3094313120 conn(0x7fa1dc06a4d0 msgr2=0x7fa1dc072580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:06:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 shutdown_connections 2023-06-09T12:06:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:06:57.353+0000 7fa1e1fff700 1 -- 172.21.15.7:0/3094313120 wait complete. 2023-06-09T12:06:57.360 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:06:57.375 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1755,"stamp":"2023-06-09T12:06:57.048098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95846,"ondisk_log_size":95846,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2599688,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":716352,"kb_avail":752375032,"statfs":{"total":773094113280,"available":770432032768,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":733544448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003283"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215545+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215545+0000","last_peered":"2023-06-09T12:06:15.215545+0000","last_clean":"2023-06-09T12:06:15.215545+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215545+0000","last_undegraded":"2023-06-09T12:06:15.215545+0000","last_fullsized":"2023-06-09T12:06:15.215545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3225,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.126507+0000","last_change":"2023-06-09T12:05:52.126507+0000","last_active":"2023-06-09T12:05:52.126507+0000","last_peered":"2023-06-09T12:05:52.126507+0000","last_clean":"2023-06-09T12:05:52.126507+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:05:52.126507+0000","last_undegraded":"2023-06-09T12:05:52.126507+0000","last_fullsized":"2023-06-09T12:05:52.126507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.029687+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:03:07.029687+0000","last_peered":"2023-06-09T12:03:07.029687+0000","last_clean":"2023-06-09T12:03:07.029687+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:03:07.029687+0000","last_undegraded":"2023-06-09T12:03:07.029687+0000","last_fullsized":"2023-06-09T12:03:07.029687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'906","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332933+0000","last_change":"2023-06-09T12:05:55.319769+0000","last_active":"2023-06-09T12:05:52.332933+0000","last_peered":"2023-06-09T12:05:52.332933+0000","last_clean":"2023-06-09T12:05:52.332933+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:05:52.332933+0000","last_undegraded":"2023-06-09T12:05:52.332933+0000","last_fullsized":"2023-06-09T12:05:52.332933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":906,"log_dups_size":0,"ondisk_log_size":906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:04.285410+0000","last_change":"2023-06-09T12:06:37.338973+0000","last_active":"2023-06-09T12:06:04.285410+0000","last_peered":"2023-06-09T12:06:04.285410+0000","last_clean":"2023-06-09T12:06:04.285410+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:04.285410+0000","last_undegraded":"2023-06-09T12:06:04.285410+0000","last_fullsized":"2023-06-09T12:06:04.285410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'952","reported_seq":3413,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.656755+0000","last_change":"2023-06-09T12:06:27.412175+0000","last_active":"2023-06-09T12:06:18.656755+0000","last_peered":"2023-06-09T12:06:18.656755+0000","last_clean":"2023-06-09T12:06:18.656755+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:06:18.656755+0000","last_undegraded":"2023-06-09T12:06:18.656755+0000","last_fullsized":"2023-06-09T12:06:18.656755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":952,"log_dups_size":0,"ondisk_log_size":952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:34.000810+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:02:34.000810+0000","last_peered":"2023-06-09T12:02:34.000810+0000","last_clean":"2023-06-09T12:02:34.000810+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:02:34.000810+0000","last_undegraded":"2023-06-09T12:02:34.000810+0000","last_fullsized":"2023-06-09T12:02:34.000810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3739,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:59.293378+0000","last_change":"2023-06-09T12:06:52.340791+0000","last_active":"2023-06-09T12:05:59.293378+0000","last_peered":"2023-06-09T12:05:59.293378+0000","last_clean":"2023-06-09T12:05:59.293378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:05:59.293378+0000","last_undegraded":"2023-06-09T12:05:59.293378+0000","last_fullsized":"2023-06-09T12:05:59.293378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3486,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:41.406462+0000","last_change":"2023-06-09T12:05:41.406462+0000","last_active":"2023-06-09T12:05:41.406462+0000","last_peered":"2023-06-09T12:05:41.406462+0000","last_clean":"2023-06-09T12:05:41.406462+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:05:41.406462+0000","last_undegraded":"2023-06-09T12:05:41.406462+0000","last_fullsized":"2023-06-09T12:05:41.406462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1149","reported_seq":3821,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:51.271767+0000","last_change":"2023-06-09T12:05:51.156280+0000","last_active":"2023-06-09T12:06:51.271767+0000","last_peered":"2023-06-09T12:06:51.271767+0000","last_clean":"2023-06-09T12:06:51.271767+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:06:51.271767+0000","last_undegraded":"2023-06-09T12:06:51.271767+0000","last_fullsized":"2023-06-09T12:06:51.271767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":196,"log_size":1149,"log_dups_size":0,"ondisk_log_size":1149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 32s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3140,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:44.454243+0000","last_change":"2023-06-09T12:05:44.454243+0000","last_active":"2023-06-09T12:05:44.454243+0000","last_peered":"2023-06-09T12:05:44.454243+0000","last_clean":"2023-06-09T12:05:44.454243+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:05:44.454243+0000","last_undegraded":"2023-06-09T12:05:44.454243+0000","last_fullsized":"2023-06-09T12:05:44.454243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2025,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730607+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730607+0000","last_peered":"2023-06-09T12:04:49.730607+0000","last_clean":"2023-06-09T12:04:49.730607+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730607+0000","last_undegraded":"2023-06-09T12:04:49.730607+0000","last_fullsized":"2023-06-09T12:04:49.730607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3312,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.520050+0000","last_change":"2023-06-09T12:06:41.985930+0000","last_active":"2023-06-09T12:05:02.520050+0000","last_peered":"2023-06-09T12:05:02.520050+0000","last_clean":"2023-06-09T12:05:02.520050+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:05:02.520050+0000","last_undegraded":"2023-06-09T12:05:02.520050+0000","last_fullsized":"2023-06-09T12:05:02.520050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2967,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:20.832681+0000","last_change":"2023-06-09T12:06:52.348318+0000","last_active":"2023-06-09T12:06:20.832681+0000","last_peered":"2023-06-09T12:06:20.832681+0000","last_clean":"2023-06-09T12:06:20.832681+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:20.832681+0000","last_undegraded":"2023-06-09T12:06:20.832681+0000","last_fullsized":"2023-06-09T12:06:20.832681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.331019+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:04:32.331019+0000","last_peered":"2023-06-09T12:04:32.331019+0000","last_clean":"2023-06-09T12:04:32.331019+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:04:32.331019+0000","last_undegraded":"2023-06-09T12:04:32.331019+0000","last_fullsized":"2023-06-09T12:04:32.331019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1037,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:22.424240+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:02:22.424240+0000","last_peered":"2023-06-09T12:02:22.424240+0000","last_clean":"2023-06-09T12:02:22.424240+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:02:22.424240+0000","last_undegraded":"2023-06-09T12:02:22.424240+0000","last_fullsized":"2023-06-09T12:02:22.424240+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3562,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:42.332315+0000","last_change":"2023-06-09T12:06:24.871532+0000","last_active":"2023-06-09T12:04:42.332315+0000","last_peered":"2023-06-09T12:04:42.332315+0000","last_clean":"2023-06-09T12:04:42.332315+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:04:42.332315+0000","last_undegraded":"2023-06-09T12:04:42.332315+0000","last_fullsized":"2023-06-09T12:04:42.332315+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.965058+0000","last_change":"2023-06-09T12:06:27.362178+0000","last_active":"2023-06-09T12:06:18.965058+0000","last_peered":"2023-06-09T12:06:18.965058+0000","last_clean":"2023-06-09T12:06:18.965058+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:06:18.965058+0000","last_undegraded":"2023-06-09T12:06:18.965058+0000","last_fullsized":"2023-06-09T12:06:18.965058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:27.409136+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:02:27.409136+0000","last_peered":"2023-06-09T12:02:27.409136+0000","last_clean":"2023-06-09T12:02:27.409136+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:02:27.409136+0000","last_undegraded":"2023-06-09T12:02:27.409136+0000","last_fullsized":"2023-06-09T12:02:27.409136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1171","reported_seq":3260,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:51.047069+0000","last_change":"2023-06-09T12:05:50.361843+0000","last_active":"2023-06-09T12:06:51.047069+0000","last_peered":"2023-06-09T12:06:51.047069+0000","last_clean":"2023-06-09T12:06:51.047069+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:51.047069+0000","last_undegraded":"2023-06-09T12:06:51.047069+0000","last_fullsized":"2023-06-09T12:06:51.047069+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":219,"log_size":1171,"log_dups_size":0,"ondisk_log_size":1171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4046,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:28.629528+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:02:28.629528+0000","last_peered":"2023-06-09T12:02:28.629528+0000","last_clean":"2023-06-09T12:02:28.629528+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:02:28.629528+0000","last_undegraded":"2023-06-09T12:02:28.629528+0000","last_fullsized":"2023-06-09T12:02:28.629528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":694,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:51.154631+0000","last_change":"2023-06-09T12:04:52.426561+0000","last_active":"2023-06-09T12:02:51.154631+0000","last_peered":"2023-06-09T12:02:51.154631+0000","last_clean":"2023-06-09T12:02:51.154631+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:02:51.154631+0000","last_undegraded":"2023-06-09T12:02:51.154631+0000","last_fullsized":"2023-06-09T12:02:51.154631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3654,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.445621+0000","last_change":"2023-06-09T12:06:50.410816+0000","last_active":"2023-06-09T12:05:02.445621+0000","last_peered":"2023-06-09T12:05:02.445621+0000","last_clean":"2023-06-09T12:05:02.445621+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:05:02.445621+0000","last_undegraded":"2023-06-09T12:05:02.445621+0000","last_fullsized":"2023-06-09T12:05:02.445621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.187861+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:03:02.187861+0000","last_peered":"2023-06-09T12:03:02.187861+0000","last_clean":"2023-06-09T12:03:02.187861+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:03:02.187861+0000","last_undegraded":"2023-06-09T12:03:02.187861+0000","last_fullsized":"2023-06-09T12:03:02.187861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'907","reported_seq":3549,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:48.450630+0000","last_change":"2023-06-09T12:05:54.809296+0000","last_active":"2023-06-09T12:06:48.450630+0000","last_peered":"2023-06-09T12:06:48.450630+0000","last_clean":"2023-06-09T12:06:48.450630+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:06:48.450630+0000","last_undegraded":"2023-06-09T12:06:48.450630+0000","last_fullsized":"2023-06-09T12:06:48.450630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":697,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:01.131264+0000","last_change":"2023-06-09T12:06:01.131264+0000","last_active":"2023-06-09T12:06:01.131264+0000","last_peered":"2023-06-09T12:06:01.131264+0000","last_clean":"2023-06-09T12:06:01.131264+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:06:01.131264+0000","last_undegraded":"2023-06-09T12:06:01.131264+0000","last_fullsized":"2023-06-09T12:06:01.131264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4071,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:52.325633+0000","last_change":"2023-06-09T12:05:57.344054+0000","last_active":"2023-06-09T12:04:52.325633+0000","last_peered":"2023-06-09T12:04:52.325633+0000","last_clean":"2023-06-09T12:04:52.325633+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:04:52.325633+0000","last_undegraded":"2023-06-09T12:04:52.325633+0000","last_fullsized":"2023-06-09T12:04:52.325633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3842,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:03.011006+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:03:03.011006+0000","last_peered":"2023-06-09T12:03:03.011006+0000","last_clean":"2023-06-09T12:03:03.011006+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:03:03.011006+0000","last_undegraded":"2023-06-09T12:03:03.011006+0000","last_fullsized":"2023-06-09T12:03:03.011006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.428455+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:02:52.428455+0000","last_peered":"2023-06-09T12:02:52.428455+0000","last_clean":"2023-06-09T12:02:52.428455+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:02:52.428455+0000","last_undegraded":"2023-06-09T12:02:52.428455+0000","last_fullsized":"2023-06-09T12:02:52.428455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":757,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334635+0000","last_change":"2023-06-09T12:05:30.983184+0000","last_active":"2023-06-09T12:05:02.334635+0000","last_peered":"2023-06-09T12:05:02.334635+0000","last_clean":"2023-06-09T12:05:02.334635+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:02.334635+0000","last_undegraded":"2023-06-09T12:05:02.334635+0000","last_fullsized":"2023-06-09T12:05:02.334635+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.305145+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:02:52.305145+0000","last_peered":"2023-06-09T12:02:52.305145+0000","last_clean":"2023-06-09T12:02:52.305145+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:02:52.305145+0000","last_undegraded":"2023-06-09T12:02:52.305145+0000","last_fullsized":"2023-06-09T12:02:52.305145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:06.064836+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:03:06.064836+0000","last_peered":"2023-06-09T12:03:06.064836+0000","last_clean":"2023-06-09T12:03:06.064836+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:03:06.064836+0000","last_undegraded":"2023-06-09T12:03:06.064836+0000","last_fullsized":"2023-06-09T12:03:06.064836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3601,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.322556+0000","last_change":"2023-06-09T12:06:44.672790+0000","last_active":"2023-06-09T12:05:02.322556+0000","last_peered":"2023-06-09T12:05:02.322556+0000","last_clean":"2023-06-09T12:05:02.322556+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:05:02.322556+0000","last_undegraded":"2023-06-09T12:05:02.322556+0000","last_fullsized":"2023-06-09T12:05:02.322556+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400263+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400263+0000","last_peered":"2023-06-09T12:05:55.400263+0000","last_clean":"2023-06-09T12:05:55.400263+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400263+0000","last_undegraded":"2023-06-09T12:05:55.400263+0000","last_fullsized":"2023-06-09T12:05:55.400263+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3893,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.981824+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:03:46.981824+0000","last_peered":"2023-06-09T12:03:46.981824+0000","last_clean":"2023-06-09T12:03:46.981824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:03:46.981824+0000","last_undegraded":"2023-06-09T12:03:46.981824+0000","last_fullsized":"2023-06-09T12:03:46.981824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.389481+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:02:52.389481+0000","last_peered":"2023-06-09T12:02:52.389481+0000","last_clean":"2023-06-09T12:02:52.389481+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:02:52.389481+0000","last_undegraded":"2023-06-09T12:02:52.389481+0000","last_fullsized":"2023-06-09T12:02:52.389481+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3589,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746760+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746760+0000","last_peered":"2023-06-09T12:05:54.746760+0000","last_clean":"2023-06-09T12:05:54.746760+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746760+0000","last_undegraded":"2023-06-09T12:05:54.746760+0000","last_fullsized":"2023-06-09T12:05:54.746760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:53.666015+0000","last_change":"2023-06-09T12:06:52.442159+0000","last_active":"2023-06-09T12:05:53.666015+0000","last_peered":"2023-06-09T12:05:53.666015+0000","last_clean":"2023-06-09T12:05:53.666015+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:05:53.666015+0000","last_undegraded":"2023-06-09T12:05:53.666015+0000","last_fullsized":"2023-06-09T12:05:53.666015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2991,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326837+0000","last_change":"2023-06-09T12:06:12.336756+0000","last_active":"2023-06-09T12:05:02.326837+0000","last_peered":"2023-06-09T12:05:02.326837+0000","last_clean":"2023-06-09T12:05:02.326837+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:05:02.326837+0000","last_undegraded":"2023-06-09T12:05:02.326837+0000","last_fullsized":"2023-06-09T12:05:02.326837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326991+0000","last_change":"2023-06-09T12:06:17.370378+0000","last_active":"2023-06-09T12:05:02.326991+0000","last_peered":"2023-06-09T12:05:02.326991+0000","last_clean":"2023-06-09T12:05:02.326991+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:05:02.326991+0000","last_undegraded":"2023-06-09T12:05:02.326991+0000","last_fullsized":"2023-06-09T12:05:02.326991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.412911+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:02:57.412911+0000","last_peered":"2023-06-09T12:02:57.412911+0000","last_clean":"2023-06-09T12:02:57.412911+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:02:57.412911+0000","last_undegraded":"2023-06-09T12:02:57.412911+0000","last_fullsized":"2023-06-09T12:02:57.412911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4116,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326908+0000","last_change":"2023-06-09T12:05:51.360005+0000","last_active":"2023-06-09T12:05:02.326908+0000","last_peered":"2023-06-09T12:05:02.326908+0000","last_clean":"2023-06-09T12:05:02.326908+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:05:02.326908+0000","last_undegraded":"2023-06-09T12:05:02.326908+0000","last_fullsized":"2023-06-09T12:05:02.326908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.117422+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:04:02.117422+0000","last_peered":"2023-06-09T12:04:02.117422+0000","last_clean":"2023-06-09T12:04:02.117422+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:04:02.117422+0000","last_undegraded":"2023-06-09T12:04:02.117422+0000","last_fullsized":"2023-06-09T12:04:02.117422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3372,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.334604+0000","last_change":"2023-06-09T12:06:21.882864+0000","last_active":"2023-06-09T12:05:02.334604+0000","last_peered":"2023-06-09T12:05:02.334604+0000","last_clean":"2023-06-09T12:05:02.334604+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:05:02.334604+0000","last_undegraded":"2023-06-09T12:05:02.334604+0000","last_fullsized":"2023-06-09T12:05:02.334604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:04:57.519309+0000","last_change":"2023-06-09T12:06:41.018780+0000","last_active":"2023-06-09T12:04:57.519309+0000","last_peered":"2023-06-09T12:04:57.519309+0000","last_clean":"2023-06-09T12:04:57.519309+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:04:57.519309+0000","last_undegraded":"2023-06-09T12:04:57.519309+0000","last_fullsized":"2023-06-09T12:04:57.519309+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":832,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.326978+0000","last_change":"2023-06-09T12:06:14.354842+0000","last_active":"2023-06-09T12:05:02.326978+0000","last_peered":"2023-06-09T12:05:02.326978+0000","last_clean":"2023-06-09T12:05:02.326978+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:05:02.326978+0000","last_undegraded":"2023-06-09T12:05:02.326978+0000","last_fullsized":"2023-06-09T12:05:02.326978+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:58.042721+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:02:58.042721+0000","last_peered":"2023-06-09T12:02:58.042721+0000","last_clean":"2023-06-09T12:02:58.042721+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:02:58.042721+0000","last_undegraded":"2023-06-09T12:02:58.042721+0000","last_fullsized":"2023-06-09T12:02:58.042721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":774,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:03.186309+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:03.186309+0000","last_peered":"2023-06-09T12:06:03.186309+0000","last_clean":"2023-06-09T12:06:03.186309+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:03.186309+0000","last_undegraded":"2023-06-09T12:06:03.186309+0000","last_fullsized":"2023-06-09T12:06:03.186309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328449+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:04:12.328449+0000","last_peered":"2023-06-09T12:04:12.328449+0000","last_clean":"2023-06-09T12:04:12.328449+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:04:12.328449+0000","last_undegraded":"2023-06-09T12:04:12.328449+0000","last_fullsized":"2023-06-09T12:04:12.328449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3433,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:02.519982+0000","last_change":"2023-06-09T12:06:43.002745+0000","last_active":"2023-06-09T12:05:02.519982+0000","last_peered":"2023-06-09T12:05:02.519982+0000","last_clean":"2023-06-09T12:05:02.519982+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:05:02.519982+0000","last_undegraded":"2023-06-09T12:05:02.519982+0000","last_fullsized":"2023-06-09T12:05:02.519982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3940,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:09.981419+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:03:09.981419+0000","last_peered":"2023-06-09T12:03:09.981419+0000","last_clean":"2023-06-09T12:03:09.981419+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:03:09.981419+0000","last_undegraded":"2023-06-09T12:03:09.981419+0000","last_fullsized":"2023-06-09T12:03:09.981419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:57.429089+0000","last_change":"2023-06-09T12:02:57.429089+0000","last_active":"2023-06-09T12:02:57.429089+0000","last_peered":"2023-06-09T12:02:57.429089+0000","last_clean":"2023-06-09T12:02:57.429089+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:02:57.429089+0000","last_undegraded":"2023-06-09T12:02:57.429089+0000","last_fullsized":"2023-06-09T12:02:57.429089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":64914,"ondisk_log_size":64914,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30813,"ondisk_log_size":30813,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575525,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260240,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80384,"kb_avail":94111600,"statfs":{"total":96636764160,"available":96370278400,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82313216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.51700000000000002,"5min":0.44,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":1.0549999999999999,"15min":1.3320000000000001},"last":0.71499999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71999999999999997,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":1.0229999999999999,"15min":1.081},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.752,"15min":0.71099999999999997},"min":{"1min":0.59099999999999997,"5min":0.504,"15min":0.32900000000000001},"max":{"1min":0.92200000000000004,"5min":0.99099999999999999,"15min":1.05},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47599999999999998,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.39500000000000002},"max":{"1min":0.85499999999999998,"5min":1.1100000000000001,"15min":1.3089999999999999},"last":0.84899999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.64100000000000001,"5min":0.505,"15min":0.441},"max":{"1min":0.88,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.753}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":1.006,"15min":1.006},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72599999999999998,"15min":0.68899999999999995},"min":{"1min":0.56899999999999995,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":0.86199999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.80700000000000005,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.84999999999999998,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.76900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575525,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208856,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52736,"kb_avail":94162984,"statfs":{"total":96636764160,"available":96422895616,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54001664},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.64800000000000002},"min":{"1min":0.46300000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.96799999999999997,"15min":1.069},"last":0.60399999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.439,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.73399999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.75}]},{"osd":2,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.69299999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":0.98499999999999999,"15min":1.024},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.71699999999999997},"min":{"1min":0.499,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":1.03,"15min":1.0369999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.76300000000000001},"min":{"1min":0.64300000000000002,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.67500000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92900000000000005,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.66100000000000003},"min":{"1min":0.52300000000000002,"5min":0.41899999999999998,"15min":0.33800000000000002},"max":{"1min":0.91400000000000003,"5min":1.0129999999999999,"15min":1.052},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.69899999999999995},"min":{"1min":0.47599999999999998,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42399999999999999,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":0.64500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575526,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406448,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":100032,"kb_avail":93965392,"statfs":{"total":96636764160,"available":96220561408,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102432768},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.69999999999999996,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78700000000000003,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.72699999999999998,"5min":0.995,"15min":0.995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.871},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78500000000000003,"15min":0.71699999999999997},"min":{"1min":0.56699999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.879,"15min":0.83299999999999996},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.53800000000000003},"max":{"1min":1.1000000000000001,"5min":1.597,"15min":1.597},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.80600000000000005},"min":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.083,"5min":1.544,"15min":1.544},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90200000000000002,"15min":0.84899999999999998},"min":{"1min":0.75700000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.1060000000000001,"5min":1.704,"15min":1.704},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.056,"5min":1.155,"15min":1.4039999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84099999999999997,"15min":0.77400000000000002},"min":{"1min":0.73599999999999999,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.04,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.91600000000000004,"15min":0.88100000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1990000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82999999999999996,"15min":0.78300000000000003},"min":{"1min":0.63400000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.113,"5min":1.355,"15min":1.355},"last":0.85999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337256,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":109440,"kb_avail":94034584,"statfs":{"total":96636764160,"available":96291414016,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":112066560},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.35099999999999998,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.82699999999999996},"last":0.47099999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.81899999999999995,"15min":1.032},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.498}]},{"osd":3,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.439},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":0.96599999999999997,"15min":1.024},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.84199999999999997,"15min":0.74199999999999999},"min":{"1min":0.71199999999999997,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88800000000000001,"15min":0.81599999999999995},"min":{"1min":0.82799999999999996,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1879999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.85299999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.73899999999999999,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.236},"last":0.746},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.89400000000000002,"15min":0.79900000000000004},"min":{"1min":0.80100000000000005,"5min":0.63900000000000001,"15min":0.46600000000000003},"max":{"1min":1.1339999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.72299999999999998,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84699999999999998,"15min":0.83799999999999997},"min":{"1min":0.69099999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0700000000000001,"15min":1.1599999999999999},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.80800000000000005},"min":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.298,"15min":1.298},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.873},"min":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.2589999999999999,"5min":1.351,"15min":1.351},"last":0.88}]}]},{"osd":2,"up_from":13,"seq":55834575526,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242784,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":62976,"kb_avail":94129056,"statfs":{"total":96636764160,"available":96388153344,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64487424},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.436},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.52700000000000002},"min":{"1min":0.46899999999999997,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.68000000000000005,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48499999999999999,"15min":0.46100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.26200000000000001},"max":{"1min":0.628,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.61299999999999999},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.36099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89200000000000002,"15min":1.367},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.878,"15min":1.546},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.79300000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.622,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96299999999999997,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":1.028},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.1140000000000001,"5min":1.214,"15min":1.7370000000000001},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.74399999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":1.3580000000000001,"15min":1.601},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0960000000000001,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":1.0960000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":1.115},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.056,"15min":1.3540000000000001},"last":1.075}]},{"osd":7,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1060000000000001,"5min":1.1659999999999999,"15min":1.208},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85299999999999998,"15min":0.83399999999999996},"min":{"1min":0.71299999999999997,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.3080000000000001},"last":1.1879999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575525,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426876,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":93944964,"statfs":{"total":96636764160,"available":96199643136,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49199999999999999,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.5669999999999999},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52700000000000002,"15min":0.503},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52600000000000002,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77900000000000003,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.52800000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.51800000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":1.355},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.76300000000000001,"5min":0.91200000000000003,"15min":1.198},"last":0.56599999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":1.01,"15min":1.635},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.753},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":0.89700000000000002,"5min":1.0980000000000001,"15min":3.081},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82999999999999996,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.2110000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88100000000000001,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.133,"15min":1.367},"last":0.92700000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80500000000000005,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.496},"max":{"1min":1.0229999999999999,"5min":1.0309999999999999,"15min":1.286},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.78600000000000003},"min":{"1min":0.71199999999999997,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":0.99399999999999999,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":0.872}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.81299999999999994},"min":{"1min":0.69999999999999996,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.413},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.66300000000000003,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0289999999999999,"5min":1.137,"15min":3.1160000000000001},"last":0.83499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575524,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423572,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":130816,"kb_avail":93948268,"statfs":{"total":96636764160,"available":96203026432,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":133955584},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.314},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":1.0289999999999999,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.72799999999999998},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.0640000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.995,"5min":1.03,"15min":1.1279999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73699999999999999,"15min":0.67700000000000005},"min":{"1min":0.64500000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.117,"15min":1.609},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.70399999999999996},"min":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.38100000000000001},"max":{"1min":1.107,"5min":1.147,"15min":1.159},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.501,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.014,"15min":1.407},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.504},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":0.80200000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.61199999999999999,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.997,"5min":1.131,"15min":1.3100000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":1.091,"15min":1.361},"last":0.70499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293656,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76288,"kb_avail":94078184,"statfs":{"total":96636764160,"available":96336060416,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78118912},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.57299999999999995,"5min":0.44,"15min":0.377},"max":{"1min":0.88700000000000001,"5min":0.90700000000000003,"15min":5.6269999999999998},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.754},"min":{"1min":0.54900000000000004,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.93500000000000005,"5min":1.083,"15min":19.715},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.74099999999999999},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.97099999999999997,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.70699999999999996},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.94299999999999995,"5min":1.016,"15min":5.5620000000000003},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74399999999999999,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.998,"5min":1.145,"15min":14.439},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":12.130000000000001},"last":0.623},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79900000000000004,"15min":0.81999999999999995},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":1.0169999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.81899999999999995},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":1.097,"15min":12.163},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.86799999999999999,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.135,"15min":19.834},"last":0.75600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:06:57.383 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.383 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.384 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.384 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.384 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.385 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.385 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.385 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.386 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.386 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.387 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.387 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.387 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.388 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.388 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.388 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.389 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.389 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.389 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.390 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.390 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.391 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.391 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.391 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.392 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.392 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.392 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.393 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.393 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.393 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.394 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.394 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.394 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.395 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.395 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.396 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.396 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.396 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.397 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.397 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.397 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.398 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.398 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.398 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.399 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.399 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.399 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.400 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.400 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.400 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.401 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.401 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:06:57.401 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:06:57.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:06:57.824 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:17.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:07:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.610+0000 7f2efbdff700 1 Processor -- start 2023-06-09T12:07:17.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.610+0000 7f2efbdff700 1 -- start start 2023-06-09T12:07:17.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 0x7f2ef40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ef406a4d0 0x7f2ef40737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2ef406e460 0x7f2ef4073d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef40c8870 con 0x7f2ef406a4d0 2023-06-09T12:07:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef415cac0 con 0x7f2ef406e460 2023-06-09T12:07:17.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2efbdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef40c37d0 con 0x7f2ef40c6280 2023-06-09T12:07:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2ef939a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ef406a4d0 0x7f2ef40737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2ef939a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f2ef406a4d0 0x7f2ef40737f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38190/0 (socket says 172.21.15.7:38190) 2023-06-09T12:07:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.611+0000 7f2ef939a700 1 -- 172.21.15.7:0/3123933157 learned_addr learned my addr 172.21.15.7:0/3123933157 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:07:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef9b9b700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 0x7f2ef40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2efa39c700 1 --2- 172.21.15.7:0/3123933157 >> v2:172.21.15.117:3300/0 conn(0x7f2ef406e460 0x7f2ef4073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 -- 172.21.15.7:0/3123933157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 msgr2=0x7f2ef40c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:07:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 0x7f2ef40c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 -- 172.21.15.7:0/3123933157 >> v2:172.21.15.117:3300/0 conn(0x7f2ef406e460 msgr2=0x7f2ef4073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:07:17.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 --2- 172.21.15.7:0/3123933157 >> v2:172.21.15.117:3300/0 conn(0x7f2ef406e460 0x7f2ef4073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 -- 172.21.15.7:0/3123933157 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef415cd30 con 0x7f2ef406a4d0 2023-06-09T12:07:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef9b9b700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 0x7f2ef40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:07:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef939a700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef406a4d0 0x7f2ef40737f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2ef415a580 tx=0x7f2ee4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddbb27aadd7ffd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef8b99700 1 -- 172.21.15.7:0/3123933157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400b870 con 0x7f2ef406a4d0 2023-06-09T12:07:17.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef8b99700 1 -- 172.21.15.7:0/3123933157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee400b9d0 con 0x7f2ef406a4d0 2023-06-09T12:07:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.612+0000 7f2ef8b99700 1 -- 172.21.15.7:0/3123933157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee400bd60 con 0x7f2ef406a4d0 2023-06-09T12:07:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 -- 172.21.15.7:0/3123933157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef406a4d0 msgr2=0x7f2ef40737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef406a4d0 0x7f2ef40737f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f2ef415a580 tx=0x7f2ee4009bd0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 -- 172.21.15.7:0/3123933157 shutdown_connections 2023-06-09T12:07:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3123933157 >> v2:172.21.15.117:3300/0 conn(0x7f2ef406e460 0x7f2ef4073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef40c6280 0x7f2ef40c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3123933157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef406a4d0 0x7f2ef40737f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 -- 172.21.15.7:0/3123933157 >> 172.21.15.7:0/3123933157 conn(0x7f2ef406f980 msgr2=0x7f2ef40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 -- 172.21.15.7:0/3123933157 shutdown_connections 2023-06-09T12:07:17.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.613+0000 7f2efbdff700 1 -- 172.21.15.7:0/3123933157 wait complete. 2023-06-09T12:07:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.615+0000 7f2efbdff700 1 Processor -- start 2023-06-09T12:07:17.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.615+0000 7f2efbdff700 1 -- start start 2023-06-09T12:07:17.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 0x7f2ef40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef406e460 0x7f2ef414d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef414e890 con 0x7f2ef40c6280 2023-06-09T12:07:17.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41f2930 con 0x7f2ef406e460 2023-06-09T12:07:17.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efbdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef41f2c10 con 0x7f2ef406a4d0 2023-06-09T12:07:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2ef9b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 0x7f2ef40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38198/0 (socket says 172.21.15.7:38198) 2023-06-09T12:07:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2ef9b9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 0x7f2ef40cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:47344/0 (socket says 172.21.15.7:47344) 2023-06-09T12:07:17.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 -- 172.21.15.7:0/3266423915 learned_addr learned my addr 172.21.15.7:0/3266423915 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:07:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2ef939a700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef406e460 0x7f2ef414d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 -- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 msgr2=0x7f2ef40cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 0x7f2ef40cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 -- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef406e460 msgr2=0x7f2ef414d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:17.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.616+0000 7f2efa39c700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef406e460 0x7f2ef414d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2efa39c700 1 -- 172.21.15.7:0/3266423915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee4009550 con 0x7f2ef40c6280 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2efa39c700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2eec007e60 tx=0x7f2eec00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec00ed30 con 0x7f2ef40c6280 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eec014420 con 0x7f2ef40c6280 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef415d2f0 con 0x7f2ef40c6280 2023-06-09T12:07:17.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eec0147d0 con 0x7f2ef40c6280 2023-06-09T12:07:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.617+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef415d010 con 0x7f2ef40c6280 2023-06-09T12:07:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.618+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef415d5b0 con 0x7f2ef40c6280 2023-06-09T12:07:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.619+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2eec014930 con 0x7f2ef40c6280 2023-06-09T12:07:17.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.621+0000 7f2eea7fc700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 0x7f2ed806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.621+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f2eec08a510 con 0x7f2ef40c6280 2023-06-09T12:07:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.621+0000 7f2ef9b9b700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 0x7f2ed806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:17.628 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.621+0000 7f2ef9b9b700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 0x7f2ed806d190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c940 tx=0x7f2ef0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:17.629 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.625+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2eec054cf0 con 0x7f2ef40c6280 2023-06-09T12:07:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.830+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ef4074cf0 con 0x7f2ed806ace0 2023-06-09T12:07:17.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.839+0000 7f2eea7fc700 1 -- 172.21.15.7:0/3266423915 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316595 (secure 0 0 0) 0x7f2ef4074cf0 con 0x7f2ed806ace0 2023-06-09T12:07:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:07:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 msgr2=0x7f2ed806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 0x7f2ed806d190 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000c940 tx=0x7f2ef0008040 comp rx=0 tx=0).stop 2023-06-09T12:07:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 msgr2=0x7f2ef414ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f2eec007e60 tx=0x7f2eec00e3c0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 shutdown_connections 2023-06-09T12:07:17.847 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ef40c6280 0x7f2ef414ddd0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2ed806ace0 0x7f2ed806d190 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ef406a4d0 0x7f2ef40cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 --2- 172.21.15.7:0/3266423915 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ef406e460 0x7f2ef414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 >> 172.21.15.7:0/3266423915 conn(0x7f2ef406f980 msgr2=0x7f2ef415b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 shutdown_connections 2023-06-09T12:07:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:17.844+0000 7f2efbdff700 1 -- 172.21.15.7:0/3266423915 wait complete. 2023-06-09T12:07:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:07:17.864 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1765,"stamp":"2023-06-09T12:07:17.053591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96277,"ondisk_log_size":96277,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2600584,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":717248,"kb_avail":752374136,"statfs":{"total":773094113280,"available":770431115264,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":734461952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215545+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215545+0000","last_peered":"2023-06-09T12:06:15.215545+0000","last_clean":"2023-06-09T12:06:15.215545+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215545+0000","last_undegraded":"2023-06-09T12:06:15.215545+0000","last_fullsized":"2023-06-09T12:06:15.215545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3227,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367642+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:07:07.367642+0000","last_peered":"2023-06-09T12:07:07.367642+0000","last_clean":"2023-06-09T12:07:07.367642+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:07:07.367642+0000","last_undegraded":"2023-06-09T12:07:07.367642+0000","last_fullsized":"2023-06-09T12:07:07.367642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3700,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:46.088571+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:03:46.088571+0000","last_peered":"2023-06-09T12:03:46.088571+0000","last_clean":"2023-06-09T12:03:46.088571+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:03:46.088571+0000","last_undegraded":"2023-06-09T12:03:46.088571+0000","last_fullsized":"2023-06-09T12:03:46.088571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'954","reported_seq":3662,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:06.960406+0000","last_change":"2023-06-09T12:05:55.319769+0000","last_active":"2023-06-09T12:07:06.960406+0000","last_peered":"2023-06-09T12:07:06.960406+0000","last_clean":"2023-06-09T12:07:06.960406+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:06.960406+0000","last_undegraded":"2023-06-09T12:07:06.960406+0000","last_fullsized":"2023-06-09T12:07:06.960406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":48,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:04.285410+0000","last_change":"2023-06-09T12:06:37.338973+0000","last_active":"2023-06-09T12:06:04.285410+0000","last_peered":"2023-06-09T12:06:04.285410+0000","last_clean":"2023-06-09T12:06:04.285410+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:04.285410+0000","last_undegraded":"2023-06-09T12:06:04.285410+0000","last_fullsized":"2023-06-09T12:06:04.285410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1074","reported_seq":3545,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:09.484507+0000","last_change":"2023-06-09T12:06:55.688413+0000","last_active":"2023-06-09T12:07:09.484507+0000","last_peered":"2023-06-09T12:07:09.484507+0000","last_clean":"2023-06-09T12:07:09.484507+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:09.484507+0000","last_undegraded":"2023-06-09T12:07:09.484507+0000","last_fullsized":"2023-06-09T12:07:09.484507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064478+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064478+0000","last_peered":"2023-06-09T12:05:17.064478+0000","last_clean":"2023-06-09T12:05:17.064478+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064478+0000","last_undegraded":"2023-06-09T12:05:17.064478+0000","last_fullsized":"2023-06-09T12:05:17.064478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:52.490865+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:02:52.490865+0000","last_peered":"2023-06-09T12:02:52.490865+0000","last_clean":"2023-06-09T12:02:52.490865+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:02:52.490865+0000","last_undegraded":"2023-06-09T12:02:52.490865+0000","last_fullsized":"2023-06-09T12:02:52.490865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3739,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:59.293378+0000","last_change":"2023-06-09T12:06:52.340791+0000","last_active":"2023-06-09T12:05:59.293378+0000","last_peered":"2023-06-09T12:05:59.293378+0000","last_clean":"2023-06-09T12:05:59.293378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:05:59.293378+0000","last_undegraded":"2023-06-09T12:05:59.293378+0000","last_fullsized":"2023-06-09T12:05:59.293378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3360,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:38.024447+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:03:38.024447+0000","last_peered":"2023-06-09T12:03:38.024447+0000","last_clean":"2023-06-09T12:03:38.024447+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:03:38.024447+0000","last_undegraded":"2023-06-09T12:03:38.024447+0000","last_fullsized":"2023-06-09T12:03:38.024447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3281,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.780906+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:03:12.780906+0000","last_peered":"2023-06-09T12:03:12.780906+0000","last_clean":"2023-06-09T12:03:12.780906+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:03:12.780906+0000","last_undegraded":"2023-06-09T12:03:12.780906+0000","last_fullsized":"2023-06-09T12:03:12.780906+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3313,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:41.985930+0000","last_change":"2023-06-09T12:07:07.534035+0000","last_active":"2023-06-09T12:06:41.985930+0000","last_peered":"2023-06-09T12:06:41.985930+0000","last_clean":"2023-06-09T12:06:41.985930+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:06:41.985930+0000","last_undegraded":"2023-06-09T12:06:41.985930+0000","last_fullsized":"2023-06-09T12:06:41.985930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2967,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:20.832681+0000","last_change":"2023-06-09T12:06:52.348318+0000","last_active":"2023-06-09T12:06:20.832681+0000","last_peered":"2023-06-09T12:06:20.832681+0000","last_clean":"2023-06-09T12:06:20.832681+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:20.832681+0000","last_undegraded":"2023-06-09T12:06:20.832681+0000","last_fullsized":"2023-06-09T12:06:20.832681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.331019+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:04:32.331019+0000","last_peered":"2023-06-09T12:04:32.331019+0000","last_clean":"2023-06-09T12:04:32.331019+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:04:32.331019+0000","last_undegraded":"2023-06-09T12:04:32.331019+0000","last_fullsized":"2023-06-09T12:04:32.331019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:02:49.066073+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:02:49.066073+0000","last_peered":"2023-06-09T12:02:49.066073+0000","last_clean":"2023-06-09T12:02:49.066073+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:02:49.066073+0000","last_undegraded":"2023-06-09T12:02:49.066073+0000","last_fullsized":"2023-06-09T12:02:49.066073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1038,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.442799+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:04:42.442799+0000","last_peered":"2023-06-09T12:04:42.442799+0000","last_clean":"2023-06-09T12:04:42.442799+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:04:42.442799+0000","last_undegraded":"2023-06-09T12:04:42.442799+0000","last_fullsized":"2023-06-09T12:04:42.442799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.965058+0000","last_change":"2023-06-09T12:06:27.362178+0000","last_active":"2023-06-09T12:06:18.965058+0000","last_peered":"2023-06-09T12:06:18.965058+0000","last_clean":"2023-06-09T12:06:18.965058+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:06:18.965058+0000","last_undegraded":"2023-06-09T12:06:18.965058+0000","last_fullsized":"2023-06-09T12:06:18.965058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":696,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:06.327823+0000","last_change":"2023-06-09T12:07:07.443860+0000","last_active":"2023-06-09T12:07:06.327823+0000","last_peered":"2023-06-09T12:07:06.327823+0000","last_clean":"2023-06-09T12:07:06.327823+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:07:06.327823+0000","last_undegraded":"2023-06-09T12:07:06.327823+0000","last_fullsized":"2023-06-09T12:07:06.327823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.426548+0000","last_change":"2023-06-09T12:04:52.426548+0000","last_active":"2023-06-09T12:04:52.426548+0000","last_peered":"2023-06-09T12:04:52.426548+0000","last_clean":"2023-06-09T12:04:52.426548+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:04:52.426548+0000","last_undegraded":"2023-06-09T12:04:52.426548+0000","last_fullsized":"2023-06-09T12:04:52.426548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1053","reported_seq":3702,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:10.431960+0000","last_change":"2023-06-09T12:05:54.809296+0000","last_active":"2023-06-09T12:07:10.431960+0000","last_peered":"2023-06-09T12:07:10.431960+0000","last_clean":"2023-06-09T12:07:10.431960+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:10.431960+0000","last_undegraded":"2023-06-09T12:07:10.431960+0000","last_fullsized":"2023-06-09T12:07:10.431960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":146,"log_size":1053,"log_dups_size":0,"ondisk_log_size":1053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":699,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367541+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:07.367541+0000","last_peered":"2023-06-09T12:07:07.367541+0000","last_clean":"2023-06-09T12:07:07.367541+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:07.367541+0000","last_undegraded":"2023-06-09T12:07:07.367541+0000","last_fullsized":"2023-06-09T12:07:07.367541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4072,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.344054+0000","last_change":"2023-06-09T12:07:07.342663+0000","last_active":"2023-06-09T12:05:57.344054+0000","last_peered":"2023-06-09T12:05:57.344054+0000","last_clean":"2023-06-09T12:05:57.344054+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:05:57.344054+0000","last_undegraded":"2023-06-09T12:05:57.344054+0000","last_fullsized":"2023-06-09T12:05:57.344054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1639,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445644+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:05:02.445644+0000","last_peered":"2023-06-09T12:05:02.445644+0000","last_clean":"2023-06-09T12:05:02.445644+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:05:02.445644+0000","last_undegraded":"2023-06-09T12:05:02.445644+0000","last_fullsized":"2023-06-09T12:05:02.445644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3455,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228513+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228513+0000","last_peered":"2023-06-09T12:05:15.228513+0000","last_clean":"2023-06-09T12:05:15.228513+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228513+0000","last_undegraded":"2023-06-09T12:05:15.228513+0000","last_fullsized":"2023-06-09T12:05:15.228513+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:30.983184+0000","last_change":"2023-06-09T12:07:12.350677+0000","last_active":"2023-06-09T12:05:30.983184+0000","last_peered":"2023-06-09T12:05:30.983184+0000","last_clean":"2023-06-09T12:05:30.983184+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:30.983184+0000","last_undegraded":"2023-06-09T12:05:30.983184+0000","last_fullsized":"2023-06-09T12:05:30.983184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3211,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:35.770715+0000","last_change":"2023-06-09T12:05:02.322645+0000","last_active":"2023-06-09T12:03:35.770715+0000","last_peered":"2023-06-09T12:03:35.770715+0000","last_clean":"2023-06-09T12:03:35.770715+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:03:35.770715+0000","last_undegraded":"2023-06-09T12:03:35.770715+0000","last_fullsized":"2023-06-09T12:03:35.770715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3602,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:44.672790+0000","last_change":"2023-06-09T12:07:07.340470+0000","last_active":"2023-06-09T12:06:44.672790+0000","last_peered":"2023-06-09T12:06:44.672790+0000","last_clean":"2023-06-09T12:06:44.672790+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:06:44.672790+0000","last_undegraded":"2023-06-09T12:06:44.672790+0000","last_fullsized":"2023-06-09T12:06:44.672790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3894,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.322599+0000","last_change":"2023-06-09T12:05:02.322599+0000","last_active":"2023-06-09T12:05:02.322599+0000","last_peered":"2023-06-09T12:05:02.322599+0000","last_clean":"2023-06-09T12:05:02.322599+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:05:02.322599+0000","last_undegraded":"2023-06-09T12:05:02.322599+0000","last_fullsized":"2023-06-09T12:05:02.322599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3589,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746760+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746760+0000","last_peered":"2023-06-09T12:05:54.746760+0000","last_clean":"2023-06-09T12:05:54.746760+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746760+0000","last_undegraded":"2023-06-09T12:05:54.746760+0000","last_fullsized":"2023-06-09T12:05:54.746760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:53.666015+0000","last_change":"2023-06-09T12:06:52.442159+0000","last_active":"2023-06-09T12:05:53.666015+0000","last_peered":"2023-06-09T12:05:53.666015+0000","last_clean":"2023-06-09T12:05:53.666015+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:05:53.666015+0000","last_undegraded":"2023-06-09T12:05:53.666015+0000","last_fullsized":"2023-06-09T12:05:53.666015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:12.336756+0000","last_change":"2023-06-09T12:07:07.342586+0000","last_active":"2023-06-09T12:06:12.336756+0000","last_peered":"2023-06-09T12:06:12.336756+0000","last_clean":"2023-06-09T12:06:12.336756+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:06:12.336756+0000","last_undegraded":"2023-06-09T12:06:12.336756+0000","last_fullsized":"2023-06-09T12:06:12.336756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:31.696276+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:03:31.696276+0000","last_peered":"2023-06-09T12:03:31.696276+0000","last_clean":"2023-06-09T12:03:31.696276+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:03:31.696276+0000","last_undegraded":"2023-06-09T12:03:31.696276+0000","last_fullsized":"2023-06-09T12:03:31.696276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.334821+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:05:02.334821+0000","last_peered":"2023-06-09T12:05:02.334821+0000","last_clean":"2023-06-09T12:05:02.334821+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:05:02.334821+0000","last_undegraded":"2023-06-09T12:05:02.334821+0000","last_fullsized":"2023-06-09T12:05:02.334821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3373,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:21.882864+0000","last_change":"2023-06-09T12:07:02.349457+0000","last_active":"2023-06-09T12:06:21.882864+0000","last_peered":"2023-06-09T12:06:21.882864+0000","last_clean":"2023-06-09T12:06:21.882864+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:06:21.882864+0000","last_undegraded":"2023-06-09T12:06:21.882864+0000","last_fullsized":"2023-06-09T12:06:21.882864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:41.018780+0000","last_change":"2023-06-09T12:07:07.534022+0000","last_active":"2023-06-09T12:06:41.018780+0000","last_peered":"2023-06-09T12:06:41.018780+0000","last_clean":"2023-06-09T12:06:41.018780+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:06:41.018780+0000","last_undegraded":"2023-06-09T12:06:41.018780+0000","last_fullsized":"2023-06-09T12:06:41.018780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":833,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:14.354842+0000","last_change":"2023-06-09T12:07:12.343238+0000","last_active":"2023-06-09T12:06:14.354842+0000","last_peered":"2023-06-09T12:06:14.354842+0000","last_clean":"2023-06-09T12:06:14.354842+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:06:14.354842+0000","last_undegraded":"2023-06-09T12:06:14.354842+0000","last_fullsized":"2023-06-09T12:06:14.354842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3719,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445633+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:05:02.445633+0000","last_peered":"2023-06-09T12:05:02.445633+0000","last_clean":"2023-06-09T12:05:02.445633+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:05:02.445633+0000","last_undegraded":"2023-06-09T12:05:02.445633+0000","last_fullsized":"2023-06-09T12:05:02.445633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":774,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:03.186309+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:03.186309+0000","last_peered":"2023-06-09T12:06:03.186309+0000","last_clean":"2023-06-09T12:06:03.186309+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:03.186309+0000","last_undegraded":"2023-06-09T12:06:03.186309+0000","last_fullsized":"2023-06-09T12:06:03.186309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328449+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:04:12.328449+0000","last_peered":"2023-06-09T12:04:12.328449+0000","last_clean":"2023-06-09T12:04:12.328449+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:04:12.328449+0000","last_undegraded":"2023-06-09T12:04:12.328449+0000","last_fullsized":"2023-06-09T12:04:12.328449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3434,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:43.002745+0000","last_change":"2023-06-09T12:07:07.533977+0000","last_active":"2023-06-09T12:06:43.002745+0000","last_peered":"2023-06-09T12:06:43.002745+0000","last_clean":"2023-06-09T12:06:43.002745+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:06:43.002745+0000","last_undegraded":"2023-06-09T12:06:43.002745+0000","last_fullsized":"2023-06-09T12:06:43.002745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3941,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.334685+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:05:02.334685+0000","last_peered":"2023-06-09T12:05:02.334685+0000","last_clean":"2023-06-09T12:05:02.334685+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:05:02.334685+0000","last_undegraded":"2023-06-09T12:05:02.334685+0000","last_fullsized":"2023-06-09T12:05:02.334685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2982,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:59.102505+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:03:59.102505+0000","last_peered":"2023-06-09T12:03:59.102505+0000","last_clean":"2023-06-09T12:03:59.102505+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:03:59.102505+0000","last_undegraded":"2023-06-09T12:03:59.102505+0000","last_fullsized":"2023-06-09T12:03:59.102505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":65345,"ondisk_log_size":65345,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30813,"ondisk_log_size":30813,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575529,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260304,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80448,"kb_avail":94111536,"statfs":{"total":96636764160,"available":96370212864,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82378752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.51700000000000002,"5min":0.44,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":1.0549999999999999,"15min":1.3320000000000001},"last":0.83599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71999999999999997,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":1.0229999999999999,"15min":1.081},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.752,"15min":0.71099999999999997},"min":{"1min":0.59099999999999997,"5min":0.504,"15min":0.32900000000000001},"max":{"1min":0.92200000000000004,"5min":0.99099999999999999,"15min":1.05},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47599999999999998,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.39500000000000002},"max":{"1min":0.85499999999999998,"5min":1.1100000000000001,"15min":1.3089999999999999},"last":0.79800000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.64100000000000001,"5min":0.505,"15min":0.441},"max":{"1min":0.88,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":1.006,"15min":1.006},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72599999999999998,"15min":0.68899999999999995},"min":{"1min":0.56899999999999995,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":0.86199999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.59099999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.80700000000000005,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.84999999999999998,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.72399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575529,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":208984,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52864,"kb_avail":94162856,"statfs":{"total":96636764160,"available":96422764544,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54132736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.64800000000000002},"min":{"1min":0.46300000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.96799999999999997,"15min":1.069},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.439,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.73399999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.69299999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":0.98499999999999999,"15min":1.024},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.79200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.71699999999999997},"min":{"1min":0.499,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":1.03,"15min":1.0369999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.76300000000000001},"min":{"1min":0.64300000000000002,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.73899999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.67500000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92900000000000005,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.66100000000000003},"min":{"1min":0.52300000000000002,"5min":0.41899999999999998,"15min":0.33800000000000002},"max":{"1min":0.91400000000000003,"5min":1.0129999999999999,"15min":1.052},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.69899999999999995},"min":{"1min":0.47599999999999998,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42399999999999999,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":0.80900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575530,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406576,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":100160,"kb_avail":93965264,"statfs":{"total":96636764160,"available":96220430336,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102563840},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.69999999999999996,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78700000000000003,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.72699999999999998,"5min":0.995,"15min":0.995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.871},"last":0.46200000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78500000000000003,"15min":0.71699999999999997},"min":{"1min":0.56699999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.879,"15min":0.83299999999999996},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.53800000000000003},"max":{"1min":1.1000000000000001,"5min":1.597,"15min":1.597},"last":0.97399999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.80600000000000005},"min":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.083,"5min":1.544,"15min":1.544},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90200000000000002,"15min":0.84899999999999998},"min":{"1min":0.75700000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.1060000000000001,"5min":1.704,"15min":1.704},"last":0.88}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.056,"5min":1.155,"15min":1.4039999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84099999999999997,"15min":0.77400000000000002},"min":{"1min":0.73599999999999999,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.04,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.91600000000000004,"15min":0.88100000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1990000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82999999999999996,"15min":0.78300000000000003},"min":{"1min":0.63400000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.113,"5min":1.355,"15min":1.355},"last":0.85699999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575529,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337384,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":109568,"kb_avail":94034456,"statfs":{"total":96636764160,"available":96291282944,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":112197632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46100000000000002,"15min":0.44900000000000001},"min":{"1min":0.35099999999999998,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.317},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.54400000000000004},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.82699999999999996},"last":0.442}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.36799999999999999,"15min":0.28999999999999998},"max":{"1min":0.69199999999999995,"5min":0.81899999999999995,"15min":1.032},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55200000000000005,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.57999999999999996,"15min":0.56799999999999995},"min":{"1min":0.41799999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.69799999999999995,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.69899999999999995},"min":{"1min":0.53400000000000003,"5min":0.51600000000000001,"15min":0.46400000000000002},"max":{"1min":0.83399999999999996,"5min":0.96599999999999997,"15min":1.024},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.84199999999999997,"15min":0.74199999999999999},"min":{"1min":0.71199999999999997,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.88800000000000001,"15min":0.81599999999999995},"min":{"1min":0.82799999999999996,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1879999999999999,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80000000000000004,"15min":0.77000000000000002},"min":{"1min":0.73899999999999999,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":1.121,"5min":1.121,"15min":1.236},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.89400000000000002,"15min":0.79900000000000004},"min":{"1min":0.80100000000000005,"5min":0.63900000000000001,"15min":0.46600000000000003},"max":{"1min":1.1339999999999999,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.86599999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.81599999999999995,"15min":0.81299999999999994},"min":{"1min":0.72299999999999998,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.84699999999999998,"15min":0.83799999999999997},"min":{"1min":0.69099999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.0369999999999999,"5min":1.0700000000000001,"15min":1.1599999999999999},"last":0.751}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.88,"15min":0.80800000000000005},"min":{"1min":0.76600000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.298,"15min":1.298},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.996,"5min":0.96199999999999997,"15min":0.873},"min":{"1min":0.73599999999999999,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.2589999999999999,"5min":1.351,"15min":1.351},"last":0.89900000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575530,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":242912,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":63104,"kb_avail":94128928,"statfs":{"total":96636764160,"available":96388022272,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64618496},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.436},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.52700000000000002},"min":{"1min":0.46899999999999997,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.68000000000000005,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.57499999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48499999999999999,"15min":0.46100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.26200000000000001},"max":{"1min":0.628,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.61299999999999999},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.36099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89200000000000002,"15min":1.367},"last":0.623}]},{"osd":3,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.878,"15min":1.546},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.622,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96299999999999997,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":0.745},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.1140000000000001,"5min":1.214,"15min":1.7370000000000001},"last":0.63700000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.74399999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":1.3580000000000001,"15min":1.601},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0960000000000001,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":0.73099999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.056,"15min":1.3540000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1060000000000001,"5min":1.1659999999999999,"15min":1.208},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85299999999999998,"15min":0.83399999999999996},"min":{"1min":0.71299999999999997,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.3080000000000001},"last":0.78000000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575529,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426940,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103744,"kb_avail":93944900,"statfs":{"total":96636764160,"available":96199577600,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106233856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49199999999999999,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.5669999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.52700000000000002,"15min":0.503},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.26900000000000002},"max":{"1min":0.67700000000000005,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52600000000000002,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.77900000000000003,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.623}]},{"osd":2,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.51800000000000002,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.77300000000000002,"5min":0.97099999999999997,"15min":1.355},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56899999999999995,"15min":0.55200000000000005},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.76300000000000001,"5min":0.91200000000000003,"15min":1.198},"last":0.64100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.79300000000000004,"15min":0.74399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":0.97999999999999998,"5min":1.01,"15min":1.635},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78600000000000003,"15min":0.753},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.41899999999999998},"max":{"1min":0.89700000000000002,"5min":1.0980000000000001,"15min":3.081},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82999999999999996,"15min":0.78200000000000003},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.39200000000000002},"max":{"1min":1.0349999999999999,"5min":1.0920000000000001,"15min":1.2110000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88100000000000001,"15min":0.82499999999999996},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.052,"5min":1.133,"15min":1.367},"last":0.876}]},{"osd":6,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.80500000000000005,"15min":0.79000000000000004},"min":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.496},"max":{"1min":1.0229999999999999,"5min":1.0309999999999999,"15min":1.286},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.81699999999999995,"15min":0.78600000000000003},"min":{"1min":0.71199999999999997,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":0.99399999999999999,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82099999999999995,"15min":0.81299999999999994},"min":{"1min":0.69999999999999996,"5min":0.57299999999999995,"15min":0.51100000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.413},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84599999999999997,"15min":0.85999999999999999},"min":{"1min":0.66300000000000003,"5min":0.629,"15min":0.54700000000000004},"max":{"1min":1.0289999999999999,"5min":1.137,"15min":3.1160000000000001},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575528,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423764,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131008,"kb_avail":93948076,"statfs":{"total":96636764160,"available":96202829824,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134152192},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.314},"max":{"1min":0.94899999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78200000000000003,"15min":0.75700000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45000000000000001},"max":{"1min":1.0289999999999999,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.755,"15min":0.72799999999999998},"min":{"1min":0.60399999999999998,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":1.002,"15min":1.0640000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.76800000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.995,"5min":1.03,"15min":1.1279999999999999},"last":0.86499999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.73699999999999999,"15min":0.67700000000000005},"min":{"1min":0.64500000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":1.0149999999999999,"5min":1.117,"15min":1.609},"last":0.875},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.70399999999999996},"min":{"1min":0.67800000000000005,"5min":0.60099999999999998,"15min":0.38100000000000001},"max":{"1min":1.107,"5min":1.147,"15min":1.159},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.70199999999999996,"15min":0.68799999999999994},"min":{"1min":0.501,"5min":0.501,"15min":0.42399999999999999},"max":{"1min":0.95899999999999996,"5min":1.014,"15min":1.407},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73899999999999999,"15min":0.73799999999999999},"min":{"1min":0.57499999999999996,"5min":0.48299999999999998,"15min":0.45500000000000002},"max":{"1min":1.0469999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.55600000000000005,"15min":0.53800000000000003},"min":{"1min":0.44500000000000001,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.70399999999999996,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.499,"5min":0.44400000000000001,"15min":0.34100000000000003},"max":{"1min":0.80200000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.63}]},{"osd":6,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59899999999999998,"15min":0.57299999999999995},"min":{"1min":0.46200000000000002,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.73399999999999999,"15min":0.70999999999999996},"min":{"1min":0.61199999999999999,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.997,"5min":1.131,"15min":1.3100000000000001},"last":0.84599999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:06:17 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68000000000000005,"15min":0.61599999999999999},"min":{"1min":0.496,"5min":0.46200000000000002,"15min":0.27400000000000002},"max":{"1min":0.90200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.95699999999999996,"5min":1.091,"15min":1.361},"last":0.65800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575528,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293720,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76352,"kb_avail":94078120,"statfs":{"total":96636764160,"available":96335994880,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78184448},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.57299999999999995,"5min":0.44,"15min":0.377},"max":{"1min":0.88700000000000001,"5min":0.90700000000000003,"15min":5.6269999999999998},"last":0.73499999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.754},"min":{"1min":0.54900000000000004,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.93500000000000005,"5min":1.083,"15min":19.715},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.74099999999999999},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.97099999999999997,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.70699999999999996},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.94299999999999995,"5min":1.016,"15min":5.5620000000000003},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74399999999999999,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.998,"5min":1.145,"15min":14.439},"last":0.78000000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":12.130000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79900000000000004,"15min":0.81999999999999995},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":1.0169999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.81899999999999995},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":1.097,"15min":12.163},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.86799999999999999,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.752}]},{"osd":7,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.135,"15min":19.834},"last":0.64700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:07:17.872 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.872 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.872 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.873 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.873 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.874 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.874 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.874 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.875 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.875 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.875 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.876 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.876 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.876 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.877 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.877 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.878 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.878 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.878 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.879 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.879 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.879 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.880 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.880 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.880 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.881 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.881 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.881 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.882 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.882 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.882 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.883 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.883 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.884 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.884 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.884 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.885 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.885 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.885 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.886 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.886 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.886 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.887 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.887 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.887 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.888 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.888 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.888 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.889 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.889 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:17.889 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:07:27.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:27.944 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:37.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:07:38.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.097+0000 7f5d7edff700 1 Processor -- start 2023-06-09T12:07:38.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.097+0000 7f5d7edff700 1 -- start start 2023-06-09T12:07:38.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 0x7f5d780c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7edff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d780c5a80 con 0x7f5d78153880 2023-06-09T12:07:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7edff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7815a500 con 0x7f5d7815b4a0 2023-06-09T12:07:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c5f10 con 0x7f5d780c9f40 2023-06-09T12:07:38.102 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7cb9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5d78153880 0x7f5d7815ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54802/0 (socket says 172.21.15.7:54802) 2023-06-09T12:07:38.103 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7cb9b700 1 -- 172.21.15.7:0/3474125252 learned_addr learned my addr 172.21.15.7:0/3474125252 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:07:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d7d39c700 1 --2- 172.21.15.7:0/3474125252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 0x7f5d780c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:38.104 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578760146 0 0) 0x7f5d780c5a80 con 0x7f5d78153880 2023-06-09T12:07:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d680035b0 con 0x7f5d78153880 2023-06-09T12:07:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095530405 0 0) 0x7f5d64001110 con 0x7f5d7815b4a0 2023-06-09T12:07:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.099+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d780c5a80 con 0x7f5d7815b4a0 2023-06-09T12:07:38.105 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d7d39c700 1 -- 172.21.15.7:0/3474125252 >> v1:172.21.15.117:6789/0 conn(0x7f5d7815b4a0 legacy=0x7f5d780c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d7d39c700 1 -- 172.21.15.7:0/3474125252 >> v1:172.21.15.7:6789/0 conn(0x7f5d78153880 legacy=0x7f5d7815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d7d39c700 1 -- 172.21.15.7:0/3474125252 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d781596e0 con 0x7f5d780c9f40 2023-06-09T12:07:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d7d39c700 1 --2- 172.21.15.7:0/3474125252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 0x7f5d780c5540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009dc0 tx=0x7f5d7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff78c4a6a58e2576 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000e070 con 0x7f5d780c9f40 2023-06-09T12:07:38.106 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7000bb20 con 0x7f5d780c9f40 2023-06-09T12:07:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.100+0000 7f5d777fe700 1 -- 172.21.15.7:0/3474125252 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70012430 con 0x7f5d780c9f40 2023-06-09T12:07:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 -- 172.21.15.7:0/3474125252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 msgr2=0x7f5d780c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 --2- 172.21.15.7:0/3474125252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 0x7f5d780c5540 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5d70009dc0 tx=0x7f5d7000b330 comp rx=0 tx=0).stop 2023-06-09T12:07:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 -- 172.21.15.7:0/3474125252 shutdown_connections 2023-06-09T12:07:38.107 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 --2- 172.21.15.7:0/3474125252 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d780c9f40 0x7f5d780c5540 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 -- 172.21.15.7:0/3474125252 >> 172.21.15.7:0/3474125252 conn(0x7f5d780737f0 msgr2=0x7f5d780720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 -- 172.21.15.7:0/3474125252 shutdown_connections 2023-06-09T12:07:38.108 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.101+0000 7f5d7edff700 1 -- 172.21.15.7:0/3474125252 wait complete. 2023-06-09T12:07:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.103+0000 7f5d7edff700 1 Processor -- start 2023-06-09T12:07:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.103+0000 7f5d7edff700 1 -- start start 2023-06-09T12:07:38.110 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780c9f40 0x7f5d781e4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:38.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:38.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d7815b4a0 0x7f5d7814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:38.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78159440 con 0x7f5d78153880 2023-06-09T12:07:38.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d780c5920 con 0x7f5d780c9f40 2023-06-09T12:07:38.111 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7edff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d78151ff0 con 0x7f5d7815b4a0 2023-06-09T12:07:38.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:38.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41138/0 (socket says 172.21.15.7:41138) 2023-06-09T12:07:38.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 -- 172.21.15.7:0/1776629941 learned_addr learned my addr 172.21.15.7:0/1776629941 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:07:38.112 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7cb9b700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780c9f40 0x7f5d781e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d7d39c700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d7815b4a0 0x7f5d7814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 -- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d7815b4a0 msgr2=0x7f5d7814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d7815b4a0 0x7f5d7814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 -- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780c9f40 msgr2=0x7f5d781e4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.113 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780c9f40 0x7f5d781e4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.104+0000 7f5d77fff700 1 -- 172.21.15.7:0/1776629941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d70009a70 con 0x7f5d78153880 2023-06-09T12:07:38.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d77fff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c760 tx=0x7f5d6400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:38.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d640052d0 con 0x7f5d78153880 2023-06-09T12:07:38.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d64005430 con 0x7f5d78153880 2023-06-09T12:07:38.114 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d78159c60 con 0x7f5d78153880 2023-06-09T12:07:38.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d640038c0 con 0x7f5d78153880 2023-06-09T12:07:38.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.105+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d78159980 con 0x7f5d78153880 2023-06-09T12:07:38.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.106+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d78159f40 con 0x7f5d78153880 2023-06-09T12:07:38.115 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.107+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d64003a20 con 0x7f5d78153880 2023-06-09T12:07:38.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.108+0000 7f5d757fa700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 0x7f5d5c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:38.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.109+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f5d640890b0 con 0x7f5d78153880 2023-06-09T12:07:38.116 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.109+0000 7f5d7cb9b700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 0x7f5d5c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:38.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.109+0000 7f5d7cb9b700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 0x7f5d5c06d1c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c007fd0 tx=0x7f5d6c007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:38.117 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.113+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5d64053750 con 0x7f5d78153880 2023-06-09T12:07:38.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.316+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5d7815a500 con 0x7f5d5c06ad10 2023-06-09T12:07:38.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.327+0000 7f5d757fa700 1 -- 172.21.15.7:0/1776629941 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316564 (secure 0 0 0) 0x7f5d7815a500 con 0x7f5d5c06ad10 2023-06-09T12:07:38.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 msgr2=0x7f5d5c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 0x7f5d5c06d1c0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c007fd0 tx=0x7f5d6c007de0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 msgr2=0x7f5d781f28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5d6400c760 tx=0x7f5d6400ca70 comp rx=0 tx=0).stop 2023-06-09T12:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 shutdown_connections 2023-06-09T12:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5d5c06ad10 0x7f5d5c06d1c0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5d7815b4a0 0x7f5d7814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5d780c9f40 0x7f5d781e4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 --2- 172.21.15.7:0/1776629941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d78153880 0x7f5d781f28a0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 >> 172.21.15.7:0/1776629941 conn(0x7f5d780737f0 msgr2=0x7f5d7814d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.331+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 shutdown_connections 2023-06-09T12:07:38.337 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:38.332+0000 7f5d7edff700 1 -- 172.21.15.7:0/1776629941 wait complete. 2023-06-09T12:07:38.338 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:07:38.353 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1775,"stamp":"2023-06-09T12:07:37.059445+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96596,"ondisk_log_size":96596,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2584904,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":701568,"kb_avail":752389816,"statfs":{"total":773094113280,"available":770447171584,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":718405632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215545+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215545+0000","last_peered":"2023-06-09T12:06:15.215545+0000","last_clean":"2023-06-09T12:06:15.215545+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215545+0000","last_undegraded":"2023-06-09T12:06:15.215545+0000","last_fullsized":"2023-06-09T12:06:15.215545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3227,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367642+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:07:07.367642+0000","last_peered":"2023-06-09T12:07:07.367642+0000","last_clean":"2023-06-09T12:07:07.367642+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:07:07.367642+0000","last_undegraded":"2023-06-09T12:07:07.367642+0000","last_fullsized":"2023-06-09T12:07:07.367642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1075","reported_seq":3789,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:30.779038+0000","last_change":"2023-06-09T12:05:55.319769+0000","last_active":"2023-06-09T12:07:30.779038+0000","last_peered":"2023-06-09T12:07:30.779038+0000","last_clean":"2023-06-09T12:07:30.779038+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:30.779038+0000","last_undegraded":"2023-06-09T12:07:30.779038+0000","last_fullsized":"2023-06-09T12:07:30.779038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":169,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 28s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:04.285410+0000","last_change":"2023-06-09T12:06:37.338973+0000","last_active":"2023-06-09T12:06:04.285410+0000","last_peered":"2023-06-09T12:06:04.285410+0000","last_clean":"2023-06-09T12:06:04.285410+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:04.285410+0000","last_undegraded":"2023-06-09T12:06:04.285410+0000","last_fullsized":"2023-06-09T12:06:04.285410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1197","reported_seq":3674,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:24.380016+0000","last_change":"2023-06-09T12:06:55.688413+0000","last_active":"2023-06-09T12:07:24.380016+0000","last_peered":"2023-06-09T12:07:24.380016+0000","last_clean":"2023-06-09T12:07:24.380016+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:24.380016+0000","last_undegraded":"2023-06-09T12:07:24.380016+0000","last_fullsized":"2023-06-09T12:07:24.380016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":245,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3522,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365218+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365218+0000","last_peered":"2023-06-09T12:05:29.365218+0000","last_clean":"2023-06-09T12:05:29.365218+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365218+0000","last_undegraded":"2023-06-09T12:05:29.365218+0000","last_fullsized":"2023-06-09T12:05:29.365218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3739,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:59.293378+0000","last_change":"2023-06-09T12:06:52.340791+0000","last_active":"2023-06-09T12:05:59.293378+0000","last_peered":"2023-06-09T12:05:59.293378+0000","last_clean":"2023-06-09T12:05:59.293378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:05:59.293378+0000","last_undegraded":"2023-06-09T12:05:59.293378+0000","last_fullsized":"2023-06-09T12:05:59.293378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3361,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:07.428337+0000","last_change":"2023-06-09T12:05:07.428337+0000","last_active":"2023-06-09T12:05:07.428337+0000","last_peered":"2023-06-09T12:05:07.428337+0000","last_clean":"2023-06-09T12:05:07.428337+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:05:07.428337+0000","last_undegraded":"2023-06-09T12:05:07.428337+0000","last_fullsized":"2023-06-09T12:05:07.428337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.321790+0000","last_change":"2023-06-09T12:03:22.321790+0000","last_active":"2023-06-09T12:03:22.321790+0000","last_peered":"2023-06-09T12:03:22.321790+0000","last_clean":"2023-06-09T12:03:22.321790+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:03:22.321790+0000","last_undegraded":"2023-06-09T12:03:22.321790+0000","last_fullsized":"2023-06-09T12:03:22.321790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3314,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.534035+0000","last_change":"2023-06-09T12:07:27.036778+0000","last_active":"2023-06-09T12:07:07.534035+0000","last_peered":"2023-06-09T12:07:07.534035+0000","last_clean":"2023-06-09T12:07:07.534035+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:07:07.534035+0000","last_undegraded":"2023-06-09T12:07:07.534035+0000","last_fullsized":"2023-06-09T12:07:07.534035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2967,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:20.832681+0000","last_change":"2023-06-09T12:06:52.348318+0000","last_active":"2023-06-09T12:06:20.832681+0000","last_peered":"2023-06-09T12:06:20.832681+0000","last_clean":"2023-06-09T12:06:20.832681+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:20.832681+0000","last_undegraded":"2023-06-09T12:06:20.832681+0000","last_fullsized":"2023-06-09T12:06:20.832681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1565,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.308038+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:03:02.308038+0000","last_peered":"2023-06-09T12:03:02.308038+0000","last_clean":"2023-06-09T12:03:02.308038+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:03:02.308038+0000","last_undegraded":"2023-06-09T12:03:02.308038+0000","last_fullsized":"2023-06-09T12:03:02.308038+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3280,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.331019+0000","last_change":"2023-06-09T12:04:32.331019+0000","last_active":"2023-06-09T12:04:32.331019+0000","last_peered":"2023-06-09T12:04:32.331019+0000","last_clean":"2023-06-09T12:04:32.331019+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:04:32.331019+0000","last_undegraded":"2023-06-09T12:04:32.331019+0000","last_fullsized":"2023-06-09T12:04:32.331019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1038,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.442799+0000","last_change":"2023-06-09T12:02:22.424240+0000","last_active":"2023-06-09T12:04:42.442799+0000","last_peered":"2023-06-09T12:04:42.442799+0000","last_clean":"2023-06-09T12:04:42.442799+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:04:42.442799+0000","last_undegraded":"2023-06-09T12:04:42.442799+0000","last_fullsized":"2023-06-09T12:04:42.442799+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'905","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:18.965058+0000","last_change":"2023-06-09T12:06:27.362178+0000","last_active":"2023-06-09T12:06:18.965058+0000","last_peered":"2023-06-09T12:06:18.965058+0000","last_clean":"2023-06-09T12:06:18.965058+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:06:18.965058+0000","last_undegraded":"2023-06-09T12:06:18.965058+0000","last_fullsized":"2023-06-09T12:06:18.965058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":905,"log_dups_size":0,"ondisk_log_size":905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1338,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:02.206482+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:03:02.206482+0000","last_peered":"2023-06-09T12:03:02.206482+0000","last_clean":"2023-06-09T12:03:02.206482+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:03:02.206482+0000","last_undegraded":"2023-06-09T12:03:02.206482+0000","last_fullsized":"2023-06-09T12:03:02.206482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:36.404106+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:03:36.404106+0000","last_peered":"2023-06-09T12:03:36.404106+0000","last_clean":"2023-06-09T12:03:36.404106+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:03:36.404106+0000","last_undegraded":"2023-06-09T12:03:36.404106+0000","last_fullsized":"2023-06-09T12:03:36.404106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":696,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:06.327823+0000","last_change":"2023-06-09T12:07:07.443860+0000","last_active":"2023-06-09T12:07:06.327823+0000","last_peered":"2023-06-09T12:07:06.327823+0000","last_clean":"2023-06-09T12:07:06.327823+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:07:06.327823+0000","last_undegraded":"2023-06-09T12:07:06.327823+0000","last_fullsized":"2023-06-09T12:07:06.327823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'433","reported_seq":611,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.522157+0000","last_change":"2023-06-09T12:07:17.149635+0000","last_active":"2023-06-09T12:07:17.522157+0000","last_peered":"2023-06-09T12:07:17.522157+0000","last_clean":"2023-06-09T12:07:17.522157+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:17.522157+0000","last_undegraded":"2023-06-09T12:07:17.522157+0000","last_fullsized":"2023-06-09T12:07:17.522157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":699,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367541+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:07.367541+0000","last_peered":"2023-06-09T12:07:07.367541+0000","last_clean":"2023-06-09T12:07:07.367541+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:07.367541+0000","last_undegraded":"2023-06-09T12:07:07.367541+0000","last_fullsized":"2023-06-09T12:07:07.367541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'871","reported_seq":4072,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.344054+0000","last_change":"2023-06-09T12:07:07.342663+0000","last_active":"2023-06-09T12:05:57.344054+0000","last_peered":"2023-06-09T12:05:57.344054+0000","last_clean":"2023-06-09T12:05:57.344054+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:05:57.344054+0000","last_undegraded":"2023-06-09T12:05:57.344054+0000","last_fullsized":"2023-06-09T12:05:57.344054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":871,"log_dups_size":0,"ondisk_log_size":871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1639,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445644+0000","last_change":"2023-06-09T12:02:52.428455+0000","last_active":"2023-06-09T12:05:02.445644+0000","last_peered":"2023-06-09T12:05:02.445644+0000","last_clean":"2023-06-09T12:05:02.445644+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:05:02.445644+0000","last_undegraded":"2023-06-09T12:05:02.445644+0000","last_fullsized":"2023-06-09T12:05:02.445644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:30.983184+0000","last_change":"2023-06-09T12:07:12.350677+0000","last_active":"2023-06-09T12:05:30.983184+0000","last_peered":"2023-06-09T12:05:30.983184+0000","last_clean":"2023-06-09T12:05:30.983184+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:30.983184+0000","last_undegraded":"2023-06-09T12:05:30.983184+0000","last_fullsized":"2023-06-09T12:05:30.983184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3604,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:26.474113+0000","last_change":"2023-06-09T12:07:26.474113+0000","last_active":"2023-06-09T12:07:26.474113+0000","last_peered":"2023-06-09T12:07:26.474113+0000","last_clean":"2023-06-09T12:07:26.474113+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:26.474113+0000","last_undegraded":"2023-06-09T12:07:26.474113+0000","last_fullsized":"2023-06-09T12:07:26.474113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3896,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:25.510423+0000","last_change":"2023-06-09T12:07:25.510423+0000","last_active":"2023-06-09T12:07:25.510423+0000","last_peered":"2023-06-09T12:07:25.510423+0000","last_clean":"2023-06-09T12:07:25.510423+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:25.510423+0000","last_undegraded":"2023-06-09T12:07:25.510423+0000","last_fullsized":"2023-06-09T12:07:25.510423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1288,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.312924+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:03:52.312924+0000","last_peered":"2023-06-09T12:03:52.312924+0000","last_clean":"2023-06-09T12:03:52.312924+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:03:52.312924+0000","last_undegraded":"2023-06-09T12:03:52.312924+0000","last_fullsized":"2023-06-09T12:03:52.312924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3589,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746760+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746760+0000","last_peered":"2023-06-09T12:05:54.746760+0000","last_clean":"2023-06-09T12:05:54.746760+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746760+0000","last_undegraded":"2023-06-09T12:05:54.746760+0000","last_fullsized":"2023-06-09T12:05:54.746760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:53.666015+0000","last_change":"2023-06-09T12:06:52.442159+0000","last_active":"2023-06-09T12:05:53.666015+0000","last_peered":"2023-06-09T12:05:53.666015+0000","last_clean":"2023-06-09T12:05:53.666015+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:05:53.666015+0000","last_undegraded":"2023-06-09T12:05:53.666015+0000","last_fullsized":"2023-06-09T12:05:53.666015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:12.336756+0000","last_change":"2023-06-09T12:07:07.342586+0000","last_active":"2023-06-09T12:06:12.336756+0000","last_peered":"2023-06-09T12:06:12.336756+0000","last_clean":"2023-06-09T12:06:12.336756+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:06:12.336756+0000","last_undegraded":"2023-06-09T12:06:12.336756+0000","last_fullsized":"2023-06-09T12:06:12.336756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.334821+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:05:02.334821+0000","last_peered":"2023-06-09T12:05:02.334821+0000","last_clean":"2023-06-09T12:05:02.334821+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:05:02.334821+0000","last_undegraded":"2023-06-09T12:05:02.334821+0000","last_fullsized":"2023-06-09T12:05:02.334821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3373,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:21.882864+0000","last_change":"2023-06-09T12:07:02.349457+0000","last_active":"2023-06-09T12:06:21.882864+0000","last_peered":"2023-06-09T12:06:21.882864+0000","last_clean":"2023-06-09T12:06:21.882864+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:06:21.882864+0000","last_undegraded":"2023-06-09T12:06:21.882864+0000","last_fullsized":"2023-06-09T12:06:21.882864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:25.019244+0000","last_change":"2023-06-09T12:07:25.019244+0000","last_active":"2023-06-09T12:07:25.019244+0000","last_peered":"2023-06-09T12:07:25.019244+0000","last_clean":"2023-06-09T12:07:25.019244+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:07:25.019244+0000","last_undegraded":"2023-06-09T12:07:25.019244+0000","last_fullsized":"2023-06-09T12:07:25.019244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":833,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:14.354842+0000","last_change":"2023-06-09T12:07:12.343238+0000","last_active":"2023-06-09T12:06:14.354842+0000","last_peered":"2023-06-09T12:06:14.354842+0000","last_clean":"2023-06-09T12:06:14.354842+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:06:14.354842+0000","last_undegraded":"2023-06-09T12:06:14.354842+0000","last_fullsized":"2023-06-09T12:06:14.354842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3719,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445633+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:05:02.445633+0000","last_peered":"2023-06-09T12:05:02.445633+0000","last_clean":"2023-06-09T12:05:02.445633+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:05:02.445633+0000","last_undegraded":"2023-06-09T12:05:02.445633+0000","last_fullsized":"2023-06-09T12:05:02.445633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":774,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:03.186309+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:03.186309+0000","last_peered":"2023-06-09T12:06:03.186309+0000","last_clean":"2023-06-09T12:06:03.186309+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:03.186309+0000","last_undegraded":"2023-06-09T12:06:03.186309+0000","last_fullsized":"2023-06-09T12:06:03.186309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3152,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328449+0000","last_change":"2023-06-09T12:04:12.328449+0000","last_active":"2023-06-09T12:04:12.328449+0000","last_peered":"2023-06-09T12:04:12.328449+0000","last_clean":"2023-06-09T12:04:12.328449+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:04:12.328449+0000","last_undegraded":"2023-06-09T12:04:12.328449+0000","last_fullsized":"2023-06-09T12:04:12.328449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3435,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.533977+0000","last_change":"2023-06-09T12:07:31.039611+0000","last_active":"2023-06-09T12:07:07.533977+0000","last_peered":"2023-06-09T12:07:07.533977+0000","last_clean":"2023-06-09T12:07:07.533977+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:07:07.533977+0000","last_undegraded":"2023-06-09T12:07:07.533977+0000","last_fullsized":"2023-06-09T12:07:07.533977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3941,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.334685+0000","last_change":"2023-06-09T12:05:02.334685+0000","last_active":"2023-06-09T12:05:02.334685+0000","last_peered":"2023-06-09T12:05:02.334685+0000","last_clean":"2023-06-09T12:05:02.334685+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:05:02.334685+0000","last_undegraded":"2023-06-09T12:05:02.334685+0000","last_fullsized":"2023-06-09T12:05:02.334685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2983,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:12.336006+0000","last_change":"2023-06-09T12:05:12.336006+0000","last_active":"2023-06-09T12:05:12.336006+0000","last_peered":"2023-06-09T12:05:12.336006+0000","last_clean":"2023-06-09T12:05:12.336006+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:05:12.336006+0000","last_undegraded":"2023-06-09T12:05:12.336006+0000","last_fullsized":"2023-06-09T12:05:12.336006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":65664,"ondisk_log_size":65664,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1319116,"num_omap_keys":2491,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30813,"ondisk_log_size":30813,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575533,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260304,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80448,"kb_avail":94111536,"statfs":{"total":96636764160,"available":96370212864,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82378752},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.28699999999999998},"max":{"1min":0.874,"5min":0.96799999999999997,"15min":1.0640000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.71399999999999997,"15min":0.69199999999999995},"min":{"1min":0.51700000000000002,"5min":0.44,"15min":0.35299999999999998},"max":{"1min":0.84299999999999997,"5min":1.0549999999999999,"15min":1.3320000000000001},"last":0.72699999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:06:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.71999999999999997,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.88400000000000001,"5min":1.0229999999999999,"15min":1.081},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.752,"15min":0.71099999999999997},"min":{"1min":0.59099999999999997,"5min":0.504,"15min":0.32900000000000001},"max":{"1min":0.92200000000000004,"5min":0.99099999999999999,"15min":1.05},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72899999999999998},"min":{"1min":0.48099999999999998,"5min":0.47599999999999998,"15min":0.44800000000000001},"max":{"1min":0.96299999999999997,"5min":0.97599999999999998,"15min":1.099},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74199999999999999,"15min":0.72999999999999998},"min":{"1min":0.51700000000000002,"5min":0.47099999999999997,"15min":0.39500000000000002},"max":{"1min":0.85499999999999998,"5min":1.1100000000000001,"15min":1.3089999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.64100000000000001,"5min":0.505,"15min":0.441},"max":{"1min":0.88,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54000000000000004,"15min":0.53400000000000003},"min":{"1min":0.43099999999999999,"5min":0.33500000000000002,"15min":0.28199999999999997},"max":{"1min":0.72599999999999998,"5min":1.006,"15min":1.006},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72599999999999998,"15min":0.68899999999999995},"min":{"1min":0.56899999999999995,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":0.86199999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.61799999999999999,"15min":0.60899999999999999},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.62,"15min":0.624},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.79600000000000004,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.52700000000000002,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.80700000000000005,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73299999999999998},"min":{"1min":0.52900000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.84999999999999998,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.66000000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575533,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209048,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52928,"kb_avail":94162792,"statfs":{"total":96636764160,"available":96422699008,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54198272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67800000000000005,"15min":0.64800000000000002},"min":{"1min":0.46300000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.85699999999999998,"5min":0.95899999999999996,"15min":1.0780000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70099999999999996,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.89500000000000002,"5min":0.96799999999999997,"15min":1.069},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71799999999999997,"15min":0.68200000000000005},"min":{"1min":0.54600000000000004,"5min":0.439,"15min":0.29999999999999999},"max":{"1min":0.83399999999999996,"5min":0.94999999999999996,"15min":1.0529999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76300000000000001,"15min":0.73399999999999999},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.40799999999999997},"max":{"1min":0.88900000000000001,"5min":1.0289999999999999,"15min":1.1319999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72699999999999998,"15min":0.69299999999999995},"min":{"1min":0.56899999999999995,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.85199999999999998,"5min":0.98499999999999999,"15min":1.024},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.90700000000000003,"5min":1.0149999999999999,"15min":1.0329999999999999},"last":0.99099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.75600000000000001,"15min":0.71699999999999997},"min":{"1min":0.499,"5min":0.46500000000000002,"15min":0.42899999999999999},"max":{"1min":0.89900000000000002,"5min":1.03,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.76300000000000001},"min":{"1min":0.64300000000000002,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.93400000000000005,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.873}]},{"osd":4,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.57999999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.73299999999999998,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.371,"15min":0.34000000000000002},"max":{"1min":0.77800000000000002,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64500000000000002,"15min":0.67500000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.92900000000000005,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70399999999999996,"15min":0.66100000000000003},"min":{"1min":0.52300000000000002,"5min":0.41899999999999998,"15min":0.33800000000000002},"max":{"1min":0.91400000000000003,"5min":1.0129999999999999,"15min":1.052},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:06:43 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69799999999999995,"15min":0.69899999999999995},"min":{"1min":0.47599999999999998,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42399999999999999,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575534,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406704,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":100288,"kb_avail":93965136,"statfs":{"total":96636764160,"available":96220299264,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":102694912},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44800000000000001,"15min":0.41799999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.69999999999999996,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.59599999999999997,"15min":0.56399999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78700000000000003,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.51300000000000001,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.72699999999999998,"5min":0.995,"15min":0.995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.32400000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.871},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67800000000000005,"15min":0.65900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.73699999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78500000000000003,"15min":0.71699999999999997},"min":{"1min":0.56699999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.879,"15min":0.83299999999999996},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.53800000000000003},"max":{"1min":1.1000000000000001,"5min":1.597,"15min":1.597},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86099999999999999,"15min":0.80600000000000005},"min":{"1min":0.64400000000000002,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.083,"5min":1.544,"15min":1.544},"last":0.873},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.90200000000000002,"15min":0.84899999999999998},"min":{"1min":0.75700000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.1060000000000001,"5min":1.704,"15min":1.704},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.056,"5min":1.155,"15min":1.4039999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.84099999999999997,"15min":0.77400000000000002},"min":{"1min":0.73599999999999999,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.04,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":1.071}]},{"osd":7,"last update":"Fri Jun 9 12:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.91600000000000004,"15min":0.88100000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1990000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.82999999999999996,"15min":0.78300000000000003},"min":{"1min":0.63400000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.113,"5min":1.355,"15min":1.355},"last":0.89200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575533,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321128,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93312,"kb_avail":94050712,"statfs":{"total":96636764160,"available":96307929088,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95551488},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.45000000000000001},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.81899999999999995,"15min":1.032},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":0.80600000000000005,"5min":0.96599999999999997,"15min":1.024},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83999999999999997,"15min":0.754},"min":{"1min":0.627,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.1399999999999999,"5min":1.204,"15min":1.204},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89500000000000002,"15min":0.82499999999999996},"min":{"1min":0.71699999999999997,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":0.82999999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.95499999999999996,"5min":1.121,"15min":1.236},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89500000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.0860000000000001,"5min":1.1339999999999999,"15min":1.2210000000000001},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.59199999999999997,"5min":0.51300000000000001,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.81599999999999995},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.298,"15min":1.298},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.88800000000000001},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1990000000000001,"5min":1.2589999999999999,"15min":1.351},"last":0.92900000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575534,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243040,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":63232,"kb_avail":94128800,"statfs":{"total":96636764160,"available":96387891200,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64749568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46800000000000003,"15min":0.436},"min":{"1min":0.314,"5min":0.28799999999999998,"15min":0.26800000000000002},"max":{"1min":0.60299999999999998,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.52700000000000002},"min":{"1min":0.46899999999999997,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.68000000000000005,"5min":0.76900000000000002,"15min":1.0940000000000001},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.48499999999999999,"15min":0.46100000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.26200000000000001},"max":{"1min":0.628,"5min":0.67100000000000004,"15min":1.3109999999999999},"last":0.443},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64800000000000002,"15min":0.61299999999999999},"min":{"1min":0.47499999999999998,"5min":0.46000000000000002,"15min":0.36099999999999999},"max":{"1min":0.76800000000000002,"5min":0.89200000000000002,"15min":1.367},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60999999999999999,"15min":0.59099999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":0.878,"15min":1.546},"last":0.505},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68899999999999995,"15min":0.64900000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81399999999999995,"5min":0.94799999999999995,"15min":1.4850000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77000000000000002,"15min":0.754},"min":{"1min":0.622,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":0.96299999999999997,"5min":1.0509999999999999,"15min":1.8220000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.61399999999999999,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.1140000000000001,"5min":1.214,"15min":1.7370000000000001},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84099999999999997,"15min":0.80700000000000005},"min":{"1min":0.74399999999999999,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.97499999999999998,"5min":1.3580000000000001,"15min":1.601},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.874,"5min":0.88100000000000001,"15min":0.84999999999999998},"min":{"1min":0.73699999999999999,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0960000000000001,"5min":1.4019999999999999,"15min":1.6479999999999999},"last":1.002}]},{"osd":6,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81799999999999995,"15min":0.78400000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79000000000000004,"15min":0.77000000000000002},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.46899999999999997},"max":{"1min":0.99199999999999999,"5min":1.056,"15min":1.3540000000000001},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.80500000000000005},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1060000000000001,"5min":1.1659999999999999,"15min":1.208},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.85299999999999998,"15min":0.83399999999999996},"min":{"1min":0.71299999999999997,"5min":0.57699999999999996,"15min":0.52800000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.3080000000000001},"last":0.88900000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575533,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427068,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103872,"kb_avail":93944772,"statfs":{"total":96636764160,"available":96199446528,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106364928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49199999999999999,"15min":0.46899999999999997},"min":{"1min":0.34000000000000002,"5min":0.31900000000000001,"15min":0.223},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":1.5669999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.5}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.877,"5min":0.91200000000000003,"15min":1.198},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.004,"5min":1.006,"15min":1.635},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.76500000000000001},"min":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.42999999999999999},"max":{"1min":1.105,"5min":1.105,"15min":3.081},"last":0.80800000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.89200000000000002,"15min":0.83399999999999996},"min":{"1min":0.72699999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.367},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.111,"5min":1.111,"15min":1.286},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.82899999999999996,"15min":0.79500000000000004},"min":{"1min":0.68500000000000005,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0800000000000001,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":1.046}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.042,"5min":1.073,"15min":1.413},"last":0.871},{"interface":"front","average":{"1min":0.87,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":3.1160000000000001},"last":0.91200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575532,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423892,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131136,"kb_avail":93947948,"statfs":{"total":96636764160,"available":96202698752,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134283264},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.315},"max":{"1min":0.94499999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.87,"5min":1.002,"15min":1.0640000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.58799999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.03,"15min":1.1279999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":1.117,"15min":1.609},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80400000000000005,"15min":0.71699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.028,"5min":1.147,"15min":1.159},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53300000000000003,"5min":0.501,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":1.014,"15min":1.407},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.439},{"interface":"front","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.49299999999999999,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.73899999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.86699999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.625},"min":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.27400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.73199999999999998,"5min":1.091,"15min":1.361},"last":0.497}]}]},{"osd":5,"up_from":13,"seq":55834575532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293720,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76352,"kb_avail":94078120,"statfs":{"total":96636764160,"available":96335994880,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78184448},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.56399999999999995,"5min":0.432,"15min":0.32600000000000001},"max":{"1min":0.85499999999999998,"5min":0.96299999999999997,"15min":5.4969999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.72099999999999997,"15min":0.72699999999999998},"min":{"1min":0.57299999999999995,"5min":0.44,"15min":0.377},"max":{"1min":0.88700000000000001,"5min":0.90700000000000003,"15min":5.6269999999999998},"last":0.84599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72199999999999998,"15min":0.754},"min":{"1min":0.54900000000000004,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.93500000000000005,"5min":1.083,"15min":19.715},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.74099999999999999},"min":{"1min":0.58899999999999997,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.97099999999999997,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.621}]},{"osd":2,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71199999999999997,"15min":0.70699999999999996},"min":{"1min":0.58099999999999996,"5min":0.47799999999999998,"15min":0.38},"max":{"1min":0.94299999999999995,"5min":1.016,"15min":5.5620000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74399999999999999,"15min":0.77200000000000002},"min":{"1min":0.628,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.998,"5min":1.145,"15min":14.439},"last":0.73599999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.71299999999999997,"15min":0.76000000000000001},"min":{"1min":0.55100000000000005,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":12.130000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.79900000000000004,"15min":0.81999999999999995},"min":{"1min":0.73999999999999999,"5min":0.61299999999999999,"15min":0.52300000000000002},"max":{"1min":1.0169999999999999,"5min":1.2330000000000001,"15min":12.218999999999999},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56200000000000006,"15min":0.81899999999999995},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.32900000000000001},"max":{"1min":0.79900000000000004,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58699999999999997,"15min":0.83199999999999996},"min":{"1min":0.41099999999999998,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63300000000000001,"15min":0.68400000000000005},"min":{"1min":0.46700000000000003,"5min":0.434,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":1.097,"15min":12.163},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.42299999999999999,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.86799999999999999,"5min":0.93400000000000005,"15min":7.6040000000000001},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63500000000000001,"15min":0.67500000000000004},"min":{"1min":0.45800000000000002,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76100000000000001,"15min":0.82399999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.0660000000000001,"5min":1.135,"15min":19.834},"last":0.86099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:07:38.360 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.360 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.361 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.361 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.362 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.362 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.362 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.363 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.363 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.364 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.364 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.364 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.365 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.365 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.365 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.366 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.366 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.367 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.367 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.367 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.368 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.368 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.369 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.369 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.369 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.370 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.370 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.370 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.371 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.371 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.372 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.372 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.372 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.373 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.373 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.374 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.374 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.374 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.375 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.375 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.375 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.376 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.376 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.376 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.377 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.377 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.378 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.378 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.378 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:38.379 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:07:58.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:58.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:07:58.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:07:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.587+0000 7f35731ff700 1 Processor -- start 2023-06-09T12:07:58.590 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.587+0000 7f35731ff700 1 -- start start 2023-06-09T12:07:58.591 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f35731ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f356c0c2ae0 0x7f356c0c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f35731ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c0c9bd0 con 0x7f356c0c88e0 2023-06-09T12:07:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f35731ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356c159570 con 0x7f356c15b060 2023-06-09T12:07:58.592 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f35731ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356c159360 con 0x7f356c0c2ae0 2023-06-09T12:07:58.593 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f3570f9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f356c0c2ae0 0x7f356c0c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f356bfff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f356c0c88e0 0x7f356c15a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42884/0 (socket says 172.21.15.7:42884) 2023-06-09T12:07:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f356bfff700 1 -- 172.21.15.7:0/1593700511 learned_addr learned my addr 172.21.15.7:0/1593700511 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:07:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f3570f9b700 1 -- 172.21.15.7:0/1593700511 >> v1:172.21.15.117:6789/0 conn(0x7f356c15b060 legacy=0x7f356c163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f3570f9b700 1 -- 172.21.15.7:0/1593700511 >> v1:172.21.15.7:6789/0 conn(0x7f356c0c88e0 legacy=0x7f356c15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.594 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.589+0000 7f3570f9b700 1 -- 172.21.15.7:0/1593700511 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c159810 con 0x7f356c0c2ae0 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f3570f9b700 1 --2- 172.21.15.7:0/1593700511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c2ae0 0x7f356c0c2eb0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3560009840 tx=0x7f3560009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a2cd0067f8e493 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f356b7fe700 1 -- 172.21.15.7:0/1593700511 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560011830 con 0x7f356c0c2ae0 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f356b7fe700 1 -- 172.21.15.7:0/1593700511 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3560011990 con 0x7f356c0c2ae0 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f356b7fe700 1 -- 172.21.15.7:0/1593700511 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3560011d20 con 0x7f356c0c2ae0 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f35731ff700 1 -- 172.21.15.7:0/1593700511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c2ae0 msgr2=0x7f356c0c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.595 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f35731ff700 1 --2- 172.21.15.7:0/1593700511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c2ae0 0x7f356c0c2eb0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3560009840 tx=0x7f3560009b50 comp rx=0 tx=0).stop 2023-06-09T12:07:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f35731ff700 1 -- 172.21.15.7:0/1593700511 shutdown_connections 2023-06-09T12:07:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f35731ff700 1 --2- 172.21.15.7:0/1593700511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c2ae0 0x7f356c0c2eb0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.590+0000 7f35731ff700 1 -- 172.21.15.7:0/1593700511 >> 172.21.15.7:0/1593700511 conn(0x7f356c0cd330 msgr2=0x7f356c0cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.591+0000 7f35731ff700 1 -- 172.21.15.7:0/1593700511 shutdown_connections 2023-06-09T12:07:58.596 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.591+0000 7f35731ff700 1 -- 172.21.15.7:0/1593700511 wait complete. 2023-06-09T12:07:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.592+0000 7f35731ff700 1 Processor -- start 2023-06-09T12:07:58.598 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.592+0000 7f35731ff700 1 -- start start 2023-06-09T12:07:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f356c0c2ae0 0x7f356c14a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 0x7f356c142a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:58.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0c65e0 con 0x7f356c0c88e0 2023-06-09T12:07:58.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356c14bf90 con 0x7f356c0c2ae0 2023-06-09T12:07:58.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f35731ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f356c0ccc60 con 0x7f356c15b060 2023-06-09T12:07:58.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.593+0000 7f356bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:58.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43522/0 (socket says 172.21.15.7:43522) 2023-06-09T12:07:58.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 -- 172.21.15.7:0/1208844666 learned_addr learned my addr 172.21.15.7:0/1208844666 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f3570f9b700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f356c0c2ae0 0x7f356c14a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f357179c700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 0x7f356c142a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 -- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 msgr2=0x7f356c142a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 0x7f356c142a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 -- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f356c0c2ae0 msgr2=0x7f356c14a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f356c0c2ae0 0x7f356c14a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 -- 172.21.15.7:0/1208844666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35600094a0 con 0x7f356c0c88e0 2023-06-09T12:07:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f357179c700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 0x7f356c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:07:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f356bfff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f355800c520 tx=0x7f355800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558005d70 con 0x7f356c0c88e0 2023-06-09T12:07:58.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35580070d0 con 0x7f356c0c88e0 2023-06-09T12:07:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.594+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3558007440 con 0x7f356c0c88e0 2023-06-09T12:07:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.595+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f356c159dd0 con 0x7f356c0c88e0 2023-06-09T12:07:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.595+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f356c159af0 con 0x7f356c0c88e0 2023-06-09T12:07:58.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.596+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356c15a0b0 con 0x7f356c0c88e0 2023-06-09T12:07:58.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.597+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f35580075a0 con 0x7f356c0c88e0 2023-06-09T12:07:58.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.599+0000 7f35697fa700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 0x7f355006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:07:58.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.599+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f35580890e0 con 0x7f356c0c88e0 2023-06-09T12:07:58.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.599+0000 7f3570f9b700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 0x7f355006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:07:58.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.599+0000 7f3570f9b700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 0x7f355006d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f35600090b0 tx=0x7f3560008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:07:58.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.603+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f35580539b0 con 0x7f356c0c88e0 2023-06-09T12:07:58.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.805+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f356c074cf0 con 0x7f355006ace0 2023-06-09T12:07:58.817 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.814+0000 7f35697fa700 1 -- 172.21.15.7:0/1208844666 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+316702 (secure 0 0 0) 0x7f356c074cf0 con 0x7f355006ace0 2023-06-09T12:07:58.820 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:07:58.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 msgr2=0x7f355006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 0x7f355006d190 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f35600090b0 tx=0x7f3560008030 comp rx=0 tx=0).stop 2023-06-09T12:07:58.822 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 msgr2=0x7f356c14c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:07:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f355800c520 tx=0x7f355800c830 comp rx=0 tx=0).stop 2023-06-09T12:07:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 shutdown_connections 2023-06-09T12:07:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f355006ace0 0x7f355006d190 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f356c15b060 0x7f356c142a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f356c0c2ae0 0x7f356c14a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 --2- 172.21.15.7:0/1208844666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f356c0c88e0 0x7f356c14c540 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:07:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 >> 172.21.15.7:0/1208844666 conn(0x7f356c0cd330 msgr2=0x7f356c145100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:07:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.818+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 shutdown_connections 2023-06-09T12:07:58.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:07:58.819+0000 7f35731ff700 1 -- 172.21.15.7:0/1208844666 wait complete. 2023-06-09T12:07:58.825 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:07:58.840 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1785,"stamp":"2023-06-09T12:07:57.065062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1365040,"num_omap_keys":2582,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":96902,"ondisk_log_size":96902,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589576,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":706240,"kb_avail":752385144,"statfs":{"total":773094113280,"available":770442387456,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":723189760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003284"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215545+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215545+0000","last_peered":"2023-06-09T12:06:15.215545+0000","last_clean":"2023-06-09T12:06:15.215545+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215545+0000","last_undegraded":"2023-06-09T12:06:15.215545+0000","last_fullsized":"2023-06-09T12:06:15.215545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3227,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367642+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:07:07.367642+0000","last_peered":"2023-06-09T12:07:07.367642+0000","last_clean":"2023-06-09T12:07:07.367642+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:07:07.367642+0000","last_undegraded":"2023-06-09T12:07:07.367642+0000","last_fullsized":"2023-06-09T12:07:07.367642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.430363+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:03:07.430363+0000","last_peered":"2023-06-09T12:03:07.430363+0000","last_clean":"2023-06-09T12:03:07.430363+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:03:07.430363+0000","last_undegraded":"2023-06-09T12:03:07.430363+0000","last_fullsized":"2023-06-09T12:03:07.430363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:04.285410+0000","last_change":"2023-06-09T12:06:37.338973+0000","last_active":"2023-06-09T12:06:04.285410+0000","last_peered":"2023-06-09T12:06:04.285410+0000","last_clean":"2023-06-09T12:06:04.285410+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:04.285410+0000","last_undegraded":"2023-06-09T12:06:04.285410+0000","last_fullsized":"2023-06-09T12:06:04.285410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384479+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384479+0000","last_peered":"2023-06-09T12:05:41.384479+0000","last_clean":"2023-06-09T12:05:41.384479+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384479+0000","last_undegraded":"2023-06-09T12:05:41.384479+0000","last_fullsized":"2023-06-09T12:05:41.384479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3739,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:59.293378+0000","last_change":"2023-06-09T12:06:52.340791+0000","last_active":"2023-06-09T12:05:59.293378+0000","last_peered":"2023-06-09T12:05:59.293378+0000","last_clean":"2023-06-09T12:05:59.293378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:05:59.293378+0000","last_undegraded":"2023-06-09T12:05:59.293378+0000","last_fullsized":"2023-06-09T12:05:59.293378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.394183+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:03:37.394183+0000","last_peered":"2023-06-09T12:03:37.394183+0000","last_clean":"2023-06-09T12:03:37.394183+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:03:37.394183+0000","last_undegraded":"2023-06-09T12:03:37.394183+0000","last_fullsized":"2023-06-09T12:03:37.394183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3808,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:42.324299+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:03:42.324299+0000","last_peered":"2023-06-09T12:03:42.324299+0000","last_clean":"2023-06-09T12:03:42.324299+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:03:42.324299+0000","last_undegraded":"2023-06-09T12:03:42.324299+0000","last_fullsized":"2023-06-09T12:03:42.324299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3362,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.445157+0000","last_change":"2023-06-09T12:07:36.313766+0000","last_active":"2023-06-09T12:07:17.445157+0000","last_peered":"2023-06-09T12:07:17.445157+0000","last_clean":"2023-06-09T12:07:17.445157+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:07:17.445157+0000","last_undegraded":"2023-06-09T12:07:17.445157+0000","last_fullsized":"2023-06-09T12:07:17.445157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4086,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432140+0000","last_change":"2023-06-09T12:03:22.432140+0000","last_active":"2023-06-09T12:03:22.432140+0000","last_peered":"2023-06-09T12:03:22.432140+0000","last_clean":"2023-06-09T12:03:22.432140+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:03:22.432140+0000","last_undegraded":"2023-06-09T12:03:22.432140+0000","last_fullsized":"2023-06-09T12:03:22.432140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3321,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.340855+0000","last_change":"2023-06-09T12:07:45.034121+0000","last_active":"2023-06-09T12:05:52.340855+0000","last_peered":"2023-06-09T12:05:52.340855+0000","last_clean":"2023-06-09T12:05:52.340855+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:05:52.340855+0000","last_undegraded":"2023-06-09T12:05:52.340855+0000","last_fullsized":"2023-06-09T12:05:52.340855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2266,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:12.307579+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:03:12.307579+0000","last_peered":"2023-06-09T12:03:12.307579+0000","last_clean":"2023-06-09T12:03:12.307579+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:03:12.307579+0000","last_undegraded":"2023-06-09T12:03:12.307579+0000","last_fullsized":"2023-06-09T12:03:12.307579+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3316,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:40.134499+0000","last_change":"2023-06-09T12:07:27.036778+0000","last_active":"2023-06-09T12:07:40.134499+0000","last_peered":"2023-06-09T12:07:40.134499+0000","last_clean":"2023-06-09T12:07:40.134499+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:07:40.134499+0000","last_undegraded":"2023-06-09T12:07:40.134499+0000","last_fullsized":"2023-06-09T12:07:40.134499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2967,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:20.832681+0000","last_change":"2023-06-09T12:06:52.348318+0000","last_active":"2023-06-09T12:06:20.832681+0000","last_peered":"2023-06-09T12:06:20.832681+0000","last_clean":"2023-06-09T12:06:20.832681+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:20.832681+0000","last_undegraded":"2023-06-09T12:06:20.832681+0000","last_fullsized":"2023-06-09T12:06:20.832681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3281,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348357+0000","last_change":"2023-06-09T12:07:47.094943+0000","last_active":"2023-06-09T12:06:52.348357+0000","last_peered":"2023-06-09T12:06:52.348357+0000","last_clean":"2023-06-09T12:06:52.348357+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:06:52.348357+0000","last_undegraded":"2023-06-09T12:06:52.348357+0000","last_fullsized":"2023-06-09T12:06:52.348357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1039,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:02.461482+0000","last_change":"2023-06-09T12:07:44.447705+0000","last_active":"2023-06-09T12:07:02.461482+0000","last_peered":"2023-06-09T12:07:02.461482+0000","last_clean":"2023-06-09T12:07:02.461482+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:02.461482+0000","last_undegraded":"2023-06-09T12:07:02.461482+0000","last_fullsized":"2023-06-09T12:07:02.461482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1115,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469728+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469728+0000","last_peered":"2023-06-09T12:06:10.469728+0000","last_clean":"2023-06-09T12:06:10.469728+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469728+0000","last_undegraded":"2023-06-09T12:06:10.469728+0000","last_fullsized":"2023-06-09T12:06:10.469728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1000","reported_seq":3697,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:48.158679+0000","last_change":"2023-06-09T12:07:37.345759+0000","last_active":"2023-06-09T12:07:48.158679+0000","last_peered":"2023-06-09T12:07:48.158679+0000","last_clean":"2023-06-09T12:07:48.158679+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:07:48.158679+0000","last_undegraded":"2023-06-09T12:07:48.158679+0000","last_fullsized":"2023-06-09T12:07:48.158679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":95,"log_size":1000,"log_dups_size":0,"ondisk_log_size":1000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":696,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:06.327823+0000","last_change":"2023-06-09T12:07:07.443860+0000","last_active":"2023-06-09T12:07:06.327823+0000","last_peered":"2023-06-09T12:07:06.327823+0000","last_clean":"2023-06-09T12:07:06.327823+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:07:06.327823+0000","last_undegraded":"2023-06-09T12:07:06.327823+0000","last_fullsized":"2023-06-09T12:07:06.327823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010592+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010592+0000","last_peered":"2023-06-09T12:07:35.010592+0000","last_clean":"2023-06-09T12:07:35.010592+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010592+0000","last_undegraded":"2023-06-09T12:07:35.010592+0000","last_fullsized":"2023-06-09T12:07:35.010592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":699,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.367541+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:07.367541+0000","last_peered":"2023-06-09T12:07:07.367541+0000","last_clean":"2023-06-09T12:07:07.367541+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:07.367541+0000","last_undegraded":"2023-06-09T12:07:07.367541+0000","last_fullsized":"2023-06-09T12:07:07.367541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'944","reported_seq":4128,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:52.344445+0000","last_change":"2023-06-09T12:07:41.478667+0000","last_active":"2023-06-09T12:07:52.344445+0000","last_peered":"2023-06-09T12:07:52.344445+0000","last_clean":"2023-06-09T12:07:52.344445+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:07:52.344445+0000","last_undegraded":"2023-06-09T12:07:52.344445+0000","last_fullsized":"2023-06-09T12:07:52.344445+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1456","reported_seq":1643,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:45.792479+0000","last_change":"2023-06-09T12:07:45.419993+0000","last_active":"2023-06-09T12:07:45.792479+0000","last_peered":"2023-06-09T12:07:45.792479+0000","last_clean":"2023-06-09T12:07:45.792479+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:45.792479+0000","last_undegraded":"2023-06-09T12:07:45.792479+0000","last_fullsized":"2023-06-09T12:07:45.792479+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:30.983184+0000","last_change":"2023-06-09T12:07:12.350677+0000","last_active":"2023-06-09T12:05:30.983184+0000","last_peered":"2023-06-09T12:05:30.983184+0000","last_clean":"2023-06-09T12:05:30.983184+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:05:30.983184+0000","last_undegraded":"2023-06-09T12:05:30.983184+0000","last_fullsized":"2023-06-09T12:05:30.983184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:30.580107+0000","last_change":"2023-06-09T12:07:26.474113+0000","last_active":"2023-06-09T12:07:30.580107+0000","last_peered":"2023-06-09T12:07:30.580107+0000","last_clean":"2023-06-09T12:07:30.580107+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:30.580107+0000","last_undegraded":"2023-06-09T12:07:30.580107+0000","last_fullsized":"2023-06-09T12:07:30.580107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":925,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437090+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:04:02.437090+0000","last_peered":"2023-06-09T12:04:02.437090+0000","last_clean":"2023-06-09T12:04:02.437090+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:04:02.437090+0000","last_undegraded":"2023-06-09T12:04:02.437090+0000","last_fullsized":"2023-06-09T12:04:02.437090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3897,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:29.552824+0000","last_change":"2023-06-09T12:07:25.510423+0000","last_active":"2023-06-09T12:07:29.552824+0000","last_peered":"2023-06-09T12:07:29.552824+0000","last_clean":"2023-06-09T12:07:29.552824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:29.552824+0000","last_undegraded":"2023-06-09T12:07:29.552824+0000","last_fullsized":"2023-06-09T12:07:29.552824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":318,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:57.419901+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:03:57.419901+0000","last_peered":"2023-06-09T12:03:57.419901+0000","last_clean":"2023-06-09T12:03:57.419901+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:03:57.419901+0000","last_undegraded":"2023-06-09T12:03:57.419901+0000","last_fullsized":"2023-06-09T12:03:57.419901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":161,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:53.666015+0000","last_change":"2023-06-09T12:06:52.442159+0000","last_active":"2023-06-09T12:05:53.666015+0000","last_peered":"2023-06-09T12:05:53.666015+0000","last_clean":"2023-06-09T12:05:53.666015+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:05:53.666015+0000","last_undegraded":"2023-06-09T12:05:53.666015+0000","last_fullsized":"2023-06-09T12:05:53.666015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:12.336756+0000","last_change":"2023-06-09T12:07:07.342586+0000","last_active":"2023-06-09T12:06:12.336756+0000","last_peered":"2023-06-09T12:06:12.336756+0000","last_clean":"2023-06-09T12:06:12.336756+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:06:12.336756+0000","last_undegraded":"2023-06-09T12:06:12.336756+0000","last_fullsized":"2023-06-09T12:06:12.336756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.334821+0000","last_change":"2023-06-09T12:05:02.334821+0000","last_active":"2023-06-09T12:05:02.334821+0000","last_peered":"2023-06-09T12:05:02.334821+0000","last_clean":"2023-06-09T12:05:02.334821+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:05:02.334821+0000","last_undegraded":"2023-06-09T12:05:02.334821+0000","last_fullsized":"2023-06-09T12:05:02.334821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3373,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:21.882864+0000","last_change":"2023-06-09T12:07:02.349457+0000","last_active":"2023-06-09T12:06:21.882864+0000","last_peered":"2023-06-09T12:06:21.882864+0000","last_clean":"2023-06-09T12:06:21.882864+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:06:21.882864+0000","last_undegraded":"2023-06-09T12:06:21.882864+0000","last_fullsized":"2023-06-09T12:06:21.882864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:32.062217+0000","last_change":"2023-06-09T12:07:25.019244+0000","last_active":"2023-06-09T12:07:32.062217+0000","last_peered":"2023-06-09T12:07:32.062217+0000","last_clean":"2023-06-09T12:07:32.062217+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:07:32.062217+0000","last_undegraded":"2023-06-09T12:07:32.062217+0000","last_fullsized":"2023-06-09T12:07:32.062217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":833,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:14.354842+0000","last_change":"2023-06-09T12:07:12.343238+0000","last_active":"2023-06-09T12:06:14.354842+0000","last_peered":"2023-06-09T12:06:14.354842+0000","last_clean":"2023-06-09T12:06:14.354842+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:06:14.354842+0000","last_undegraded":"2023-06-09T12:06:14.354842+0000","last_fullsized":"2023-06-09T12:06:14.354842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3719,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445633+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:05:02.445633+0000","last_peered":"2023-06-09T12:05:02.445633+0000","last_clean":"2023-06-09T12:05:02.445633+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:05:02.445633+0000","last_undegraded":"2023-06-09T12:05:02.445633+0000","last_fullsized":"2023-06-09T12:05:02.445633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:07.390990+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:03:07.390990+0000","last_peered":"2023-06-09T12:03:07.390990+0000","last_clean":"2023-06-09T12:03:07.390990+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:03:07.390990+0000","last_undegraded":"2023-06-09T12:03:07.390990+0000","last_fullsized":"2023-06-09T12:03:07.390990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":775,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:17.970099+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:17.970099+0000","last_peered":"2023-06-09T12:06:17.970099+0000","last_clean":"2023-06-09T12:06:17.970099+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:17.970099+0000","last_undegraded":"2023-06-09T12:06:17.970099+0000","last_fullsized":"2023-06-09T12:06:17.970099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'901","reported_seq":3678,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:22.432164+0000","last_change":"2023-06-09T12:03:22.432164+0000","last_active":"2023-06-09T12:03:22.432164+0000","last_peered":"2023-06-09T12:03:22.432164+0000","last_clean":"2023-06-09T12:03:22.432164+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:03:22.432164+0000","last_undegraded":"2023-06-09T12:03:22.432164+0000","last_fullsized":"2023-06-09T12:03:22.432164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":901,"log_dups_size":0,"ondisk_log_size":901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.346516+0000","last_change":"2023-06-09T12:07:48.112027+0000","last_active":"2023-06-09T12:06:37.346516+0000","last_peered":"2023-06-09T12:06:37.346516+0000","last_clean":"2023-06-09T12:06:37.346516+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:06:37.346516+0000","last_undegraded":"2023-06-09T12:06:37.346516+0000","last_fullsized":"2023-06-09T12:06:37.346516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'933","reported_seq":3436,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:31.039611+0000","last_change":"2023-06-09T12:07:31.039611+0000","last_active":"2023-06-09T12:07:31.039611+0000","last_peered":"2023-06-09T12:07:31.039611+0000","last_clean":"2023-06-09T12:07:31.039611+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:07:31.039611+0000","last_undegraded":"2023-06-09T12:07:31.039611+0000","last_fullsized":"2023-06-09T12:07:31.039611+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":933,"log_dups_size":0,"ondisk_log_size":933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3942,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.350050+0000","last_change":"2023-06-09T12:07:49.110762+0000","last_active":"2023-06-09T12:07:07.350050+0000","last_peered":"2023-06-09T12:07:07.350050+0000","last_clean":"2023-06-09T12:07:07.350050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:07.350050+0000","last_undegraded":"2023-06-09T12:07:07.350050+0000","last_fullsized":"2023-06-09T12:07:07.350050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2984,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.351371+0000","last_change":"2023-06-09T12:07:51.130097+0000","last_active":"2023-06-09T12:07:17.351371+0000","last_peered":"2023-06-09T12:07:17.351371+0000","last_clean":"2023-06-09T12:07:17.351371+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:17.351371+0000","last_undegraded":"2023-06-09T12:07:17.351371+0000","last_fullsized":"2023-06-09T12:07:17.351371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958625+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958625+0000","last_peered":"2023-06-09T12:05:52.958625+0000","last_clean":"2023-06-09T12:05:52.958625+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958625+0000","last_undegraded":"2023-06-09T12:05:52.958625+0000","last_fullsized":"2023-06-09T12:05:52.958625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":65953,"ondisk_log_size":65953,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1365040,"num_omap_keys":2582,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30830,"ondisk_log_size":30830,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575537,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260432,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80576,"kb_avail":94111408,"statfs":{"total":96636764160,"available":96370081792,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82509824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.3320000000000001},"last":0.622}]},{"osd":1,"last update":"Fri Jun 9 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.081},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.71699999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.39500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.56699999999999995,"5min":0.50700000000000001,"15min":0.379},"max":{"1min":0.86399999999999999,"5min":0.92300000000000004,"15min":1.4450000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.64100000000000001,"5min":0.505,"15min":0.441},"max":{"1min":0.88,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.72899999999999998,"5min":1.006,"15min":1.006},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.60799999999999998,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.76900000000000002,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.96899999999999997,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.92500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575537,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209048,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52928,"kb_avail":94162792,"statfs":{"total":96636764160,"available":96422699008,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54198272},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.91400000000000003,"5min":0.96799999999999997,"15min":1.069},"last":0.71299999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.498,"15min":0.312},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74299999999999999},"min":{"1min":0.60799999999999998,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.0289999999999999},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73499999999999999,"15min":0.70499999999999996},"min":{"1min":0.52900000000000003,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.746,"15min":0.72499999999999998},"min":{"1min":0.51400000000000001,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":1.022,"5min":1.03,"15min":1.0369999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80500000000000005,"15min":0.77500000000000002},"min":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.82699999999999996,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59699999999999998,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.34000000000000002},"max":{"1min":0.84499999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73599999999999999,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.437,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":1.0129999999999999,"15min":1.052},"last":0.625}]},{"osd":7,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.36299999999999999},"max":{"1min":0.91800000000000004,"5min":0.98699999999999999,"15min":1.071},"last":0.52800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575538,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410864,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":104448,"kb_avail":93960976,"statfs":{"total":96636764160,"available":96216039424,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":106954752},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.42399999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.48099999999999998},"min":{"1min":0.371,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.64000000000000001,"5min":0.995,"15min":0.995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.69299999999999995,"5min":1.141,"15min":1.141},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.70099999999999996,"5min":0.66900000000000004,"15min":0.54100000000000004},"max":{"1min":1.093,"5min":1.597,"15min":1.597},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.1539999999999999,"5min":1.544,"15min":1.544},"last":0.753},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.03,"5min":1.704,"15min":1.704},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1299999999999999,"5min":1.155,"15min":1.4039999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.78200000000000003},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.49399999999999999},"max":{"1min":1.111,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91400000000000003,"15min":0.88400000000000001},"min":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1930000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.355,"15min":1.355},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575537,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321192,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":94050648,"statfs":{"total":96636764160,"available":96307863552,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.45000000000000001},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.81899999999999995,"15min":1.032},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.61099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":0.80600000000000005,"5min":0.96599999999999997,"15min":1.024},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83999999999999997,"15min":0.754},"min":{"1min":0.627,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.1399999999999999,"5min":1.204,"15min":1.204},"last":1.0149999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89500000000000002,"15min":0.82499999999999996},"min":{"1min":0.71699999999999997,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.95499999999999996,"5min":1.121,"15min":1.236},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89500000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.0860000000000001,"5min":1.1339999999999999,"15min":1.2210000000000001},"last":1.0720000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.59199999999999997,"5min":0.51300000000000001,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.89700000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.81599999999999995},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.298,"15min":1.298},"last":1.0980000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.88800000000000001},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1990000000000001,"5min":1.2589999999999999,"15min":1.351},"last":0.98299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575538,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":243104,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":63296,"kb_avail":94128736,"statfs":{"total":96636764160,"available":96387825664,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":64815104},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.442},"min":{"1min":0.34999999999999998,"5min":0.28799999999999998,"15min":0.27100000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.3109999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65900000000000003,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.367},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.44,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.546},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.88400000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.76100000000000001},"min":{"1min":0.63600000000000001,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.202,"5min":1.202,"15min":1.8220000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":0.96899999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.81699999999999995},"min":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.86199999999999999},"min":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82899999999999996,"15min":0.79300000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79200000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.121},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85599999999999998,"15min":0.81499999999999995},"min":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1559999999999999,"5min":1.1659999999999999,"15min":1.208},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":1.0189999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575537,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427132,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103936,"kb_avail":93944708,"statfs":{"total":96636764160,"available":96199380992,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106430464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.47499999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.223},"max":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":1.5669999999999999},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.877,"5min":0.91200000000000003,"15min":1.198},"last":0.52800000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.004,"5min":1.006,"15min":1.635},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.76500000000000001},"min":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.42999999999999999},"max":{"1min":1.105,"5min":1.105,"15min":3.081},"last":1.171}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.89200000000000002,"15min":0.83399999999999996},"min":{"1min":0.72699999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.367},"last":1.073}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.111,"5min":1.111,"15min":1.286},"last":1.198},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.82899999999999996,"15min":0.79500000000000004},"min":{"1min":0.68500000000000005,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0800000000000001,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":1.117}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.042,"5min":1.073,"15min":1.413},"last":1.286},{"interface":"front","average":{"1min":0.87,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":3.1160000000000001},"last":1.234}]}]},{"osd":4,"up_from":13,"seq":55834575536,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423956,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131200,"kb_avail":93947884,"statfs":{"total":96636764160,"available":96202633216,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134348800},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.315},"max":{"1min":0.94499999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.87,"5min":1.002,"15min":1.0640000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.58799999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.03,"15min":1.1279999999999999},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":1.117,"15min":1.609},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80400000000000005,"15min":0.71699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.028,"5min":1.147,"15min":1.159},"last":0.94599999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53300000000000003,"5min":0.501,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":1.014,"15min":1.407},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.49299999999999999,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.73899999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.86699999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.751}]},{"osd":7,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.625},"min":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.27400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.73199999999999998,"5min":1.091,"15min":1.361},"last":0.623}]}]},{"osd":5,"up_from":13,"seq":55834575536,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":293848,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76480,"kb_avail":94077992,"statfs":{"total":96636764160,"available":96335863808,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78315520},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.501},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":5.6269999999999998},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72999999999999998,"15min":0.76100000000000001},"min":{"1min":0.48799999999999999,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":1.083,"15min":19.715},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.749},"min":{"1min":0.49099999999999999,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.94899999999999995,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.91200000000000003,"5min":1.016,"15min":5.5620000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.145,"15min":14.439},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":12.130000000000001},"last":0.622},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.274,"5min":1.274,"15min":12.218999999999999},"last":0.749}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.81499999999999995},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.75600000000000001,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.83499999999999996},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":12.163},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":1.095,"5min":1.095,"15min":7.6040000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.67300000000000004},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.83199999999999996,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.82699999999999996},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":19.834},"last":0.71999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:07:58.847 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.847 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.848 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.848 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.848 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.849 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.849 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.850 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.850 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.850 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.851 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.851 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.851 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.852 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.852 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.852 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.853 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.853 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.854 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.854 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.854 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.855 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.855 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.855 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.856 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.856 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.856 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.857 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.857 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.858 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.858 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.858 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.859 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.859 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.859 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.860 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.860 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.860 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.861 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.861 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.861 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.862 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.862 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.862 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.863 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.863 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:07:58.863 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:08:18.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:08:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.074+0000 7f1f6759e700 1 Processor -- start 2023-06-09T12:08:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.075+0000 7f1f6759e700 1 -- start start 2023-06-09T12:08:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f6759e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f680b0be0 0x7f1f68157e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:19.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f6759e700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f680c8210 con 0x7f1f68151340 2023-06-09T12:08:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f6759e700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f680c8a20 con 0x7f1f6806a4d0 2023-06-09T12:08:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f6759e700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f680c8460 con 0x7f1f680b0be0 2023-06-09T12:08:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f5ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f680b0be0 0x7f1f68157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f5ffff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1f680b0be0 0x7f1f68157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47544/0 (socket says 172.21.15.7:47544) 2023-06-09T12:08:19.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.076+0000 7f1f5ffff700 1 -- 172.21.15.7:0/2986904890 learned_addr learned my addr 172.21.15.7:0/2986904890 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:08:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f5ffff700 1 -- 172.21.15.7:0/2986904890 >> v1:172.21.15.117:6789/0 conn(0x7f1f6806a4d0 legacy=0x7f1f68072070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f5ffff700 1 -- 172.21.15.7:0/2986904890 >> v1:172.21.15.7:6789/0 conn(0x7f1f68151340 legacy=0x7f1f680c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f5ffff700 1 -- 172.21.15.7:0/2986904890 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680c8740 con 0x7f1f680b0be0 2023-06-09T12:08:19.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f5ffff700 1 --2- 172.21.15.7:0/2986904890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f680b0be0 0x7f1f68157e20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098f0 tx=0x7f1f50009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41bfcb2a269cd28 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f65d9b700 1 -- 172.21.15.7:0/2986904890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000b8a0 con 0x7f1f680b0be0 2023-06-09T12:08:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f65d9b700 1 -- 172.21.15.7:0/2986904890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5000ba00 con 0x7f1f680b0be0 2023-06-09T12:08:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.077+0000 7f1f65d9b700 1 -- 172.21.15.7:0/2986904890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000bd90 con 0x7f1f680b0be0 2023-06-09T12:08:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 -- 172.21.15.7:0/2986904890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f680b0be0 msgr2=0x7f1f68157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 --2- 172.21.15.7:0/2986904890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f680b0be0 0x7f1f68157e20 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f1f500098f0 tx=0x7f1f50009c00 comp rx=0 tx=0).stop 2023-06-09T12:08:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 -- 172.21.15.7:0/2986904890 shutdown_connections 2023-06-09T12:08:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 --2- 172.21.15.7:0/2986904890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f680b0be0 0x7f1f68157e20 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 -- 172.21.15.7:0/2986904890 >> 172.21.15.7:0/2986904890 conn(0x7f1f680cbcb0 msgr2=0x7f1f680c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:08:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 -- 172.21.15.7:0/2986904890 shutdown_connections 2023-06-09T12:08:19.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.078+0000 7f1f6759e700 1 -- 172.21.15.7:0/2986904890 wait complete. 2023-06-09T12:08:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.080+0000 7f1f6759e700 1 Processor -- start 2023-06-09T12:08:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.080+0000 7f1f6759e700 1 -- start start 2023-06-09T12:08:19.085 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f6806a4d0 0x7f1f68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f680b0be0 0x7f1f680c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6815ac30 con 0x7f1f68151340 2023-06-09T12:08:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68070ec0 con 0x7f1f680b0be0 2023-06-09T12:08:19.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6759e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f680701a0 con 0x7f1f6806a4d0 2023-06-09T12:08:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f66d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f66d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47558/0 (socket says 172.21.15.7:47558) 2023-06-09T12:08:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f66d9d700 1 -- 172.21.15.7:0/1614798164 learned_addr learned my addr 172.21.15.7:0/1614798164 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:08:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f6659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f6806a4d0 0x7f1f68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:19.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f5ffff700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f680b0be0 0x7f1f680c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f66d9d700 1 -- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f6806a4d0 msgr2=0x7f1f68071910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.081+0000 7f1f66d9d700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f6806a4d0 0x7f1f68071910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f66d9d700 1 -- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f680b0be0 msgr2=0x7f1f680c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f66d9d700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f680b0be0 0x7f1f680c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f66d9d700 1 -- 172.21.15.7:0/1614798164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f50009550 con 0x7f1f68151340 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f66d9d700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000cc80 tx=0x7f1f6000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f60003c00 con 0x7f1f68151340 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f60003d60 con 0x7f1f68151340 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6000aaa0 con 0x7f1f68151340 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f68157620 con 0x7f1f68151340 2023-06-09T12:08:19.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.082+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f68157340 con 0x7f1f68151340 2023-06-09T12:08:19.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.083+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68157900 con 0x7f1f68151340 2023-06-09T12:08:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.084+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1f6000ac00 con 0x7f1f68151340 2023-06-09T12:08:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.085+0000 7f1f5effd700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 0x7f1f4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.085+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f1f60089d50 con 0x7f1f68151340 2023-06-09T12:08:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.085+0000 7f1f6659c700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 0x7f1f4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:19.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.086+0000 7f1f6659c700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 0x7f1f4806d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007fd0 tx=0x7f1f58009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:19.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.090+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1f6008d030 con 0x7f1f68151340 2023-06-09T12:08:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.297+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f680c0340 con 0x7f1f4806ace0 2023-06-09T12:08:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.306+0000 7f1f5effd700 1 -- 172.21.15.7:0/1614798164 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317049 (secure 0 0 0) 0x7f1f680c0340 con 0x7f1f4806ace0 2023-06-09T12:08:19.311 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:08:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 msgr2=0x7f1f4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 0x7f1f4806d190 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1f58007fd0 tx=0x7f1f58009040 comp rx=0 tx=0).stop 2023-06-09T12:08:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 msgr2=0x7f1f680c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000cc80 tx=0x7f1f6000cf90 comp rx=0 tx=0).stop 2023-06-09T12:08:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 shutdown_connections 2023-06-09T12:08:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f68151340 0x7f1f680c6fc0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1f4806ace0 0x7f1f4806d190 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1f6806a4d0 0x7f1f68071910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 --2- 172.21.15.7:0/1614798164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1f680b0be0 0x7f1f680c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:19.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.310+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 >> 172.21.15.7:0/1614798164 conn(0x7f1f680cbcb0 msgr2=0x7f1f6814cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:08:19.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.311+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 shutdown_connections 2023-06-09T12:08:19.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:19.311+0000 7f1f6759e700 1 -- 172.21.15.7:0/1614798164 wait complete. 2023-06-09T12:08:19.318 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:08:19.330 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1796,"stamp":"2023-06-09T12:08:19.070873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1449985,"num_omap_keys":2724,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97390,"ondisk_log_size":97390,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594760,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711424,"kb_avail":752379960,"statfs":{"total":773094113280,"available":770437079040,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728498176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003245"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'882","reported_seq":3228,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:08.104741+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:07:08.104741+0000","last_peered":"2023-06-09T12:07:08.104741+0000","last_clean":"2023-06-09T12:07:08.104741+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:07:08.104741+0000","last_undegraded":"2023-06-09T12:07:08.104741+0000","last_fullsized":"2023-06-09T12:07:08.104741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4203,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:04.285410+0000","last_change":"2023-06-09T12:06:37.338973+0000","last_active":"2023-06-09T12:06:04.285410+0000","last_peered":"2023-06-09T12:06:04.285410+0000","last_clean":"2023-06-09T12:06:04.285410+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:04.285410+0000","last_undegraded":"2023-06-09T12:06:04.285410+0000","last_fullsized":"2023-06-09T12:06:04.285410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3739,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:59.293378+0000","last_change":"2023-06-09T12:06:52.340791+0000","last_active":"2023-06-09T12:05:59.293378+0000","last_peered":"2023-06-09T12:05:59.293378+0000","last_clean":"2023-06-09T12:05:59.293378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:05:59.293378+0000","last_undegraded":"2023-06-09T12:05:59.293378+0000","last_fullsized":"2023-06-09T12:05:59.293378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3362,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.445157+0000","last_change":"2023-06-09T12:07:36.313766+0000","last_active":"2023-06-09T12:07:17.445157+0000","last_peered":"2023-06-09T12:07:17.445157+0000","last_clean":"2023-06-09T12:07:17.445157+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:07:17.445157+0000","last_undegraded":"2023-06-09T12:07:17.445157+0000","last_fullsized":"2023-06-09T12:07:17.445157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4087,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.452166+0000","last_change":"2023-06-09T12:07:57.489680+0000","last_active":"2023-06-09T12:05:52.452166+0000","last_peered":"2023-06-09T12:05:52.452166+0000","last_clean":"2023-06-09T12:05:52.452166+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:05:52.452166+0000","last_undegraded":"2023-06-09T12:05:52.452166+0000","last_fullsized":"2023-06-09T12:05:52.452166+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3321,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.340855+0000","last_change":"2023-06-09T12:07:45.034121+0000","last_active":"2023-06-09T12:05:52.340855+0000","last_peered":"2023-06-09T12:05:52.340855+0000","last_clean":"2023-06-09T12:05:52.340855+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:05:52.340855+0000","last_undegraded":"2023-06-09T12:05:52.340855+0000","last_fullsized":"2023-06-09T12:05:52.340855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3317,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:48.175151+0000","last_change":"2023-06-09T12:07:27.036778+0000","last_active":"2023-06-09T12:07:48.175151+0000","last_peered":"2023-06-09T12:07:48.175151+0000","last_clean":"2023-06-09T12:07:48.175151+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:07:48.175151+0000","last_undegraded":"2023-06-09T12:07:48.175151+0000","last_fullsized":"2023-06-09T12:07:48.175151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2968,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348318+0000","last_change":"2023-06-09T12:07:55.142523+0000","last_active":"2023-06-09T12:06:52.348318+0000","last_peered":"2023-06-09T12:06:52.348318+0000","last_clean":"2023-06-09T12:06:52.348318+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:52.348318+0000","last_undegraded":"2023-06-09T12:06:52.348318+0000","last_fullsized":"2023-06-09T12:06:52.348318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3281,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348357+0000","last_change":"2023-06-09T12:07:47.094943+0000","last_active":"2023-06-09T12:06:52.348357+0000","last_peered":"2023-06-09T12:06:52.348357+0000","last_clean":"2023-06-09T12:06:52.348357+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:06:52.348357+0000","last_undegraded":"2023-06-09T12:06:52.348357+0000","last_fullsized":"2023-06-09T12:06:52.348357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1039,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:02.461482+0000","last_change":"2023-06-09T12:07:44.447705+0000","last_active":"2023-06-09T12:07:02.461482+0000","last_peered":"2023-06-09T12:07:02.461482+0000","last_clean":"2023-06-09T12:07:02.461482+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:02.461482+0000","last_undegraded":"2023-06-09T12:07:02.461482+0000","last_fullsized":"2023-06-09T12:07:02.461482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539608+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539608+0000","last_peered":"2023-06-09T12:06:19.539608+0000","last_clean":"2023-06-09T12:06:19.539608+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539608+0000","last_undegraded":"2023-06-09T12:06:19.539608+0000","last_fullsized":"2023-06-09T12:06:19.539608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":696,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:06.327823+0000","last_change":"2023-06-09T12:07:07.443860+0000","last_active":"2023-06-09T12:07:06.327823+0000","last_peered":"2023-06-09T12:07:06.327823+0000","last_clean":"2023-06-09T12:07:06.327823+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:07:06.327823+0000","last_undegraded":"2023-06-09T12:07:06.327823+0000","last_fullsized":"2023-06-09T12:07:06.327823+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010592+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010592+0000","last_peered":"2023-06-09T12:07:35.010592+0000","last_clean":"2023-06-09T12:07:35.010592+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010592+0000","last_undegraded":"2023-06-09T12:07:35.010592+0000","last_fullsized":"2023-06-09T12:07:35.010592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":700,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.238428+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:12.238428+0000","last_peered":"2023-06-09T12:07:12.238428+0000","last_clean":"2023-06-09T12:07:12.238428+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:12.238428+0000","last_undegraded":"2023-06-09T12:07:12.238428+0000","last_fullsized":"2023-06-09T12:07:12.238428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1067","reported_seq":4282,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:13.207204+0000","last_change":"2023-06-09T12:07:41.478667+0000","last_active":"2023-06-09T12:08:13.207204+0000","last_peered":"2023-06-09T12:08:13.207204+0000","last_clean":"2023-06-09T12:08:13.207204+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:13.207204+0000","last_undegraded":"2023-06-09T12:08:13.207204+0000","last_fullsized":"2023-06-09T12:08:13.207204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":196,"log_size":1067,"log_dups_size":0,"ondisk_log_size":1067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.514471+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:04:12.514471+0000","last_peered":"2023-06-09T12:04:12.514471+0000","last_clean":"2023-06-09T12:04:12.514471+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:04:12.514471+0000","last_undegraded":"2023-06-09T12:04:12.514471+0000","last_fullsized":"2023-06-09T12:04:12.514471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329373+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329373+0000","last_peered":"2023-06-09T12:07:57.329373+0000","last_clean":"2023-06-09T12:07:57.329373+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329373+0000","last_undegraded":"2023-06-09T12:07:57.329373+0000","last_fullsized":"2023-06-09T12:07:57.329373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":759,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350677+0000","last_change":"2023-06-09T12:07:54.093801+0000","last_active":"2023-06-09T12:07:12.350677+0000","last_peered":"2023-06-09T12:07:12.350677+0000","last_clean":"2023-06-09T12:07:12.350677+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:12.350677+0000","last_undegraded":"2023-06-09T12:07:12.350677+0000","last_fullsized":"2023-06-09T12:07:12.350677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:30.580107+0000","last_change":"2023-06-09T12:07:26.474113+0000","last_active":"2023-06-09T12:07:30.580107+0000","last_peered":"2023-06-09T12:07:30.580107+0000","last_clean":"2023-06-09T12:07:30.580107+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:30.580107+0000","last_undegraded":"2023-06-09T12:07:30.580107+0000","last_fullsized":"2023-06-09T12:07:30.580107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3897,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:29.552824+0000","last_change":"2023-06-09T12:07:25.510423+0000","last_active":"2023-06-09T12:07:29.552824+0000","last_peered":"2023-06-09T12:07:29.552824+0000","last_clean":"2023-06-09T12:07:29.552824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:29.552824+0000","last_undegraded":"2023-06-09T12:07:29.552824+0000","last_fullsized":"2023-06-09T12:07:29.552824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.434156+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:05:57.434156+0000","last_peered":"2023-06-09T12:05:57.434156+0000","last_clean":"2023-06-09T12:05:57.434156+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:05:57.434156+0000","last_undegraded":"2023-06-09T12:05:57.434156+0000","last_fullsized":"2023-06-09T12:05:57.434156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1350,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:12.328478+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:04:12.328478+0000","last_peered":"2023-06-09T12:04:12.328478+0000","last_clean":"2023-06-09T12:04:12.328478+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:04:12.328478+0000","last_undegraded":"2023-06-09T12:04:12.328478+0000","last_fullsized":"2023-06-09T12:04:12.328478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":162,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.442159+0000","last_change":"2023-06-09T12:08:17.405114+0000","last_active":"2023-06-09T12:06:52.442159+0000","last_peered":"2023-06-09T12:06:52.442159+0000","last_clean":"2023-06-09T12:06:52.442159+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:06:52.442159+0000","last_undegraded":"2023-06-09T12:06:52.442159+0000","last_fullsized":"2023-06-09T12:06:52.442159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2992,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:12.336756+0000","last_change":"2023-06-09T12:07:07.342586+0000","last_active":"2023-06-09T12:06:12.336756+0000","last_peered":"2023-06-09T12:06:12.336756+0000","last_clean":"2023-06-09T12:06:12.336756+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:06:12.336756+0000","last_undegraded":"2023-06-09T12:06:12.336756+0000","last_fullsized":"2023-06-09T12:06:12.336756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":526,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350787+0000","last_change":"2023-06-09T12:07:53.051874+0000","last_active":"2023-06-09T12:07:12.350787+0000","last_peered":"2023-06-09T12:07:12.350787+0000","last_clean":"2023-06-09T12:07:12.350787+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:12.350787+0000","last_undegraded":"2023-06-09T12:07:12.350787+0000","last_fullsized":"2023-06-09T12:07:12.350787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:07.315002+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:04:07.315002+0000","last_peered":"2023-06-09T12:04:07.315002+0000","last_clean":"2023-06-09T12:04:07.315002+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:04:07.315002+0000","last_undegraded":"2023-06-09T12:04:07.315002+0000","last_fullsized":"2023-06-09T12:04:07.315002+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3373,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:21.882864+0000","last_change":"2023-06-09T12:07:02.349457+0000","last_active":"2023-06-09T12:06:21.882864+0000","last_peered":"2023-06-09T12:06:21.882864+0000","last_clean":"2023-06-09T12:06:21.882864+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:06:21.882864+0000","last_undegraded":"2023-06-09T12:06:21.882864+0000","last_fullsized":"2023-06-09T12:06:21.882864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:42.107995+0000","last_change":"2023-06-09T12:07:25.019244+0000","last_active":"2023-06-09T12:07:42.107995+0000","last_peered":"2023-06-09T12:07:42.107995+0000","last_clean":"2023-06-09T12:07:42.107995+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:07:42.107995+0000","last_undegraded":"2023-06-09T12:07:42.107995+0000","last_fullsized":"2023-06-09T12:07:42.107995+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":833,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:14.354842+0000","last_change":"2023-06-09T12:07:12.343238+0000","last_active":"2023-06-09T12:06:14.354842+0000","last_peered":"2023-06-09T12:06:14.354842+0000","last_clean":"2023-06-09T12:06:14.354842+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:06:14.354842+0000","last_undegraded":"2023-06-09T12:06:14.354842+0000","last_fullsized":"2023-06-09T12:06:14.354842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3719,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445633+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:05:02.445633+0000","last_peered":"2023-06-09T12:05:02.445633+0000","last_clean":"2023-06-09T12:05:02.445633+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:05:02.445633+0000","last_undegraded":"2023-06-09T12:05:02.445633+0000","last_fullsized":"2023-06-09T12:05:02.445633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":886,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:05:52.332905+0000","last_change":"2023-06-09T12:06:16.409548+0000","last_active":"2023-06-09T12:05:52.332905+0000","last_peered":"2023-06-09T12:05:52.332905+0000","last_clean":"2023-06-09T12:05:52.332905+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:05:52.332905+0000","last_undegraded":"2023-06-09T12:05:52.332905+0000","last_fullsized":"2023-06-09T12:05:52.332905+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":775,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:17.970099+0000","last_change":"2023-06-09T12:06:03.186309+0000","last_active":"2023-06-09T12:06:17.970099+0000","last_peered":"2023-06-09T12:06:17.970099+0000","last_clean":"2023-06-09T12:06:17.970099+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:06:17.970099+0000","last_undegraded":"2023-06-09T12:06:17.970099+0000","last_fullsized":"2023-06-09T12:06:17.970099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:37.340494+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:03:37.340494+0000","last_peered":"2023-06-09T12:03:37.340494+0000","last_clean":"2023-06-09T12:03:37.340494+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:03:37.340494+0000","last_undegraded":"2023-06-09T12:03:37.340494+0000","last_fullsized":"2023-06-09T12:03:37.340494+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1021","reported_seq":3808,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:14.870636+0000","last_change":"2023-06-09T12:07:58.475670+0000","last_active":"2023-06-09T12:08:14.870636+0000","last_peered":"2023-06-09T12:08:14.870636+0000","last_clean":"2023-06-09T12:08:14.870636+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:14.870636+0000","last_undegraded":"2023-06-09T12:08:14.870636+0000","last_fullsized":"2023-06-09T12:08:14.870636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":120,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1064,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.329874+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:04:22.329874+0000","last_peered":"2023-06-09T12:04:22.329874+0000","last_clean":"2023-06-09T12:04:22.329874+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:04:22.329874+0000","last_undegraded":"2023-06-09T12:04:22.329874+0000","last_fullsized":"2023-06-09T12:04:22.329874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.346516+0000","last_change":"2023-06-09T12:07:48.112027+0000","last_active":"2023-06-09T12:06:37.346516+0000","last_peered":"2023-06-09T12:06:37.346516+0000","last_clean":"2023-06-09T12:06:37.346516+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:06:37.346516+0000","last_undegraded":"2023-06-09T12:06:37.346516+0000","last_fullsized":"2023-06-09T12:06:37.346516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1625,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:17.321076+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:04:17.321076+0000","last_peered":"2023-06-09T12:04:17.321076+0000","last_clean":"2023-06-09T12:04:17.321076+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:04:17.321076+0000","last_undegraded":"2023-06-09T12:04:17.321076+0000","last_fullsized":"2023-06-09T12:04:17.321076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437020+0000","last_change":"2023-06-09T12:04:02.437020+0000","last_active":"2023-06-09T12:04:02.437020+0000","last_peered":"2023-06-09T12:04:02.437020+0000","last_clean":"2023-06-09T12:04:02.437020+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:04:02.437020+0000","last_undegraded":"2023-06-09T12:04:02.437020+0000","last_fullsized":"2023-06-09T12:04:02.437020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1006","reported_seq":3517,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:14.909331+0000","last_change":"2023-06-09T12:07:31.039611+0000","last_active":"2023-06-09T12:08:14.909331+0000","last_peered":"2023-06-09T12:08:14.909331+0000","last_clean":"2023-06-09T12:08:14.909331+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:14.909331+0000","last_undegraded":"2023-06-09T12:08:14.909331+0000","last_fullsized":"2023-06-09T12:08:14.909331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":73,"log_size":1006,"log_dups_size":0,"ondisk_log_size":1006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3942,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.350050+0000","last_change":"2023-06-09T12:07:49.110762+0000","last_active":"2023-06-09T12:07:07.350050+0000","last_peered":"2023-06-09T12:07:07.350050+0000","last_clean":"2023-06-09T12:07:07.350050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:07.350050+0000","last_undegraded":"2023-06-09T12:07:07.350050+0000","last_fullsized":"2023-06-09T12:07:07.350050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2984,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.351371+0000","last_change":"2023-06-09T12:07:51.130097+0000","last_active":"2023-06-09T12:07:17.351371+0000","last_peered":"2023-06-09T12:07:17.351371+0000","last_clean":"2023-06-09T12:07:17.351371+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:17.351371+0000","last_undegraded":"2023-06-09T12:07:17.351371+0000","last_fullsized":"2023-06-09T12:07:17.351371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":66427,"ondisk_log_size":66427,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1449985,"num_omap_keys":2724,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30844,"ondisk_log_size":30844,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575542,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260560,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80704,"kb_avail":94111280,"statfs":{"total":96636764160,"available":96369950720,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82640896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.3320000000000001},"last":0.62}]},{"osd":1,"last update":"Fri Jun 9 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.081},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.39500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.68300000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.505,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.76000000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.72899999999999998,"5min":1.006,"15min":1.006},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.60799999999999998,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.47799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.76900000000000002,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.96899999999999997,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.71699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575542,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209112,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":52992,"kb_avail":94162728,"statfs":{"total":96636764160,"available":96422633472,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54263808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.91400000000000003,"5min":0.96799999999999997,"15min":1.069},"last":0.70699999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.498,"15min":0.312},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74299999999999999},"min":{"1min":0.60799999999999998,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.0289999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73499999999999999,"15min":0.70499999999999996},"min":{"1min":0.52900000000000003,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.67700000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.746,"15min":0.72499999999999998},"min":{"1min":0.51400000000000001,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":1.022,"5min":1.03,"15min":1.0369999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80500000000000005,"15min":0.77500000000000002},"min":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.82699999999999996,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59699999999999998,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.34000000000000002},"max":{"1min":0.84499999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73599999999999999,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.437,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":1.0129999999999999,"15min":1.052},"last":0.73799999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.36299999999999999},"max":{"1min":0.91800000000000004,"5min":0.98699999999999999,"15min":1.071},"last":0.65100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575543,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410992,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":104576,"kb_avail":93960848,"statfs":{"total":96636764160,"available":96215908352,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":107085824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.42399999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.48099999999999998},"min":{"1min":0.371,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.64000000000000001,"5min":0.995,"15min":0.995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.69299999999999995,"5min":1.141,"15min":1.141},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.70099999999999996,"5min":0.66900000000000004,"15min":0.54100000000000004},"max":{"1min":1.093,"5min":1.597,"15min":1.597},"last":0.745}]},{"osd":5,"last update":"Fri Jun 9 12:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.1539999999999999,"5min":1.544,"15min":1.544},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.03,"5min":1.704,"15min":1.704},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1299999999999999,"5min":1.155,"15min":1.4039999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.78200000000000003},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.49399999999999999},"max":{"1min":1.111,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91400000000000003,"15min":0.88400000000000001},"min":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1930000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.355,"15min":1.355},"last":0.63800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575542,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321384,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93568,"kb_avail":94050456,"statfs":{"total":96636764160,"available":96307666944,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95813632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46800000000000003,"15min":0.45000000000000001},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.247},"max":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41199999999999998,"15min":0.34000000000000002},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57199999999999995,"15min":0.56100000000000005},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.73199999999999998,"5min":0.81899999999999995,"15min":1.032},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55700000000000005,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.34499999999999997,"15min":0.27400000000000002},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.80000000000000004},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58299999999999996,"15min":0.56899999999999995},"min":{"1min":0.439,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.77700000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.69599999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.46400000000000002},"max":{"1min":0.80600000000000005,"5min":0.96599999999999997,"15min":1.024},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83999999999999997,"15min":0.754},"min":{"1min":0.627,"5min":0.44600000000000001,"15min":0.36899999999999999},"max":{"1min":1.1399999999999999,"5min":1.204,"15min":1.204},"last":0.748},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89500000000000002,"15min":0.82499999999999996},"min":{"1min":0.71699999999999997,"5min":0.61699999999999999,"15min":0.42899999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.53500000000000003,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.95499999999999996,"5min":1.121,"15min":1.236},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.89500000000000002,"15min":0.80900000000000005},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.0860000000000001,"5min":1.1339999999999999,"15min":1.2210000000000001},"last":0.88600000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.59199999999999997,"5min":0.51300000000000001,"15min":0.48099999999999998},"max":{"1min":1.0669999999999999,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.69499999999999995,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.97899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.88100000000000001,"15min":0.81599999999999995},"min":{"1min":0.63900000000000001,"5min":0.55400000000000005,"15min":0.47299999999999998},"max":{"1min":1.141,"5min":1.298,"15min":1.298},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.96699999999999997,"15min":0.88800000000000001},"min":{"1min":0.76900000000000002,"5min":0.66300000000000003,"15min":0.52400000000000002},"max":{"1min":1.1990000000000001,"5min":1.2589999999999999,"15min":1.351},"last":1.0109999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575543,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247328,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":67520,"kb_avail":94124512,"statfs":{"total":96636764160,"available":96383500288,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":69140480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.442},"min":{"1min":0.34999999999999998,"5min":0.28799999999999998,"15min":0.27100000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.78600000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.3109999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65900000000000003,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.367},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.44,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.546},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.54700000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.76100000000000001},"min":{"1min":0.63600000000000001,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.202,"5min":1.202,"15min":1.8220000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.81699999999999995},"min":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.86199999999999999},"min":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82899999999999996,"15min":0.79300000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79200000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.121},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85599999999999998,"15min":0.81499999999999995},"min":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1559999999999999,"5min":1.1659999999999999,"15min":1.208},"last":0.63},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.82199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575542,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427260,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104064,"kb_avail":93944580,"statfs":{"total":96636764160,"available":96199249920,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106561536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.47499999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.223},"max":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":1.5669999999999999},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52900000000000003,"15min":0.50800000000000001},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.79000000000000004,"5min":0.88700000000000001,"15min":0.95499999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.51700000000000002},"min":{"1min":0.379,"5min":0.36499999999999999,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":0.85099999999999998,"15min":1.4970000000000001},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.46000000000000002,"15min":0.42299999999999999},"max":{"1min":0.85999999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.29799999999999999},"max":{"1min":0.877,"5min":0.91200000000000003,"15min":1.198},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.80200000000000005,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.004,"5min":1.006,"15min":1.635},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.79700000000000004,"15min":0.76500000000000001},"min":{"1min":0.56499999999999995,"5min":0.53300000000000003,"15min":0.42999999999999999},"max":{"1min":1.105,"5min":1.105,"15min":3.081},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.83899999999999997,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.57099999999999995,"15min":0.39200000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.89200000000000002,"15min":0.83399999999999996},"min":{"1min":0.72699999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.165,"5min":1.165,"15min":1.367},"last":0.84499999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.81100000000000005,"15min":0.79700000000000004},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.111,"5min":1.111,"15min":1.286},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.82899999999999996,"15min":0.79500000000000004},"min":{"1min":0.68500000000000005,"5min":0.54200000000000004,"15min":0.434},"max":{"1min":1.0800000000000001,"5min":1.0860000000000001,"15min":3.3490000000000002},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.82099999999999995,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.60599999999999998,"15min":0.51100000000000001},"max":{"1min":1.042,"5min":1.073,"15min":1.413},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.84799999999999998,"15min":0.86299999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":3.1160000000000001},"last":0.82499999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575541,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424084,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131328,"kb_avail":93947756,"statfs":{"total":96636764160,"available":96202502144,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134479872},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73099999999999998,"15min":0.69399999999999995},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.315},"max":{"1min":0.94499999999999995,"5min":1.0640000000000001,"15min":1.137},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78400000000000003,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.89100000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76400000000000001,"15min":0.73399999999999999},"min":{"1min":0.56000000000000005,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.87,"5min":1.002,"15min":1.0640000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79200000000000004,"15min":0.77300000000000002},"min":{"1min":0.58799999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92100000000000004,"5min":1.03,"15min":1.1279999999999999},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.68799999999999994},"min":{"1min":0.49199999999999999,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":1.117,"15min":1.609},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.80400000000000005,"15min":0.71699999999999997},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.028,"5min":1.147,"15min":1.159},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.68999999999999995},"min":{"1min":0.53300000000000003,"5min":0.501,"15min":0.44400000000000001},"max":{"1min":0.871,"5min":1.014,"15min":1.407},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.73999999999999999},"min":{"1min":0.55000000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.95299999999999996,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.436,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.95499999999999996,"15min":1.0349999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.63700000000000001,"15min":0.627},"min":{"1min":0.49299999999999999,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":1.05,"15min":1.1719999999999999},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59799999999999998,"15min":0.57799999999999996},"min":{"1min":0.437,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.73899999999999999,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.435},{"interface":"front","average":{"1min":0.747,"5min":0.73799999999999999,"15min":0.71899999999999997},"min":{"1min":0.59899999999999998,"5min":0.47899999999999998,"15min":0.44},"max":{"1min":0.86699999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:07:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.625},"min":{"1min":0.49299999999999999,"5min":0.48499999999999999,"15min":0.27400000000000002},"max":{"1min":0.90500000000000003,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.497},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.63600000000000001,"15min":0.65500000000000003},"min":{"1min":0.48199999999999998,"5min":0.46500000000000002,"15min":0.39800000000000002},"max":{"1min":0.73199999999999998,"5min":1.091,"15min":1.361},"last":0.45900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294040,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76672,"kb_avail":94077800,"statfs":{"total":96636764160,"available":96335667200,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78512128},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":5.6269999999999998},"last":0.72999999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72999999999999998,"15min":0.76100000000000001},"min":{"1min":0.48799999999999999,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":1.083,"15min":19.715},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.749},"min":{"1min":0.49099999999999999,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.94899999999999995,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.70399999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.91200000000000003,"5min":1.016,"15min":5.5620000000000003},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.145,"15min":14.439},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":12.130000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.274,"5min":1.274,"15min":12.218999999999999},"last":0.77100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.81499999999999995},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.75600000000000001,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.83499999999999996},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.49299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":12.163},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":1.095,"5min":1.095,"15min":7.6040000000000001},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.67300000000000004},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.83199999999999996,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.82699999999999996},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":19.834},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:08:19.338 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.339 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.339 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.339 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.340 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.340 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.341 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.341 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.341 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.342 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.342 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.342 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.343 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.343 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.343 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.344 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.344 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.345 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.345 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.345 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.346 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.346 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.346 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.347 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.347 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.348 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.348 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.348 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.349 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.349 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.349 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.350 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.350 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.350 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.351 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.351 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.351 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.352 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.352 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.352 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.353 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.353 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.353 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.354 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:19.354 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:08:28.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:28.169 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:39.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:08:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.562+0000 7fabb21ff700 1 Processor -- start 2023-06-09T12:08:39.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.562+0000 7fabb21ff700 1 -- start start 2023-06-09T12:08:39.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.563+0000 7fabb21ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabac0c1bc0 0x7fabac1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.563+0000 7fabb21ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fabac0c0140 0x7fabac159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.563+0000 7fabb21ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0c0bf0 con 0x7fabac0c1f90 2023-06-09T12:08:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.563+0000 7fabb21ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac070110 con 0x7fabac0c1bc0 2023-06-09T12:08:39.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.563+0000 7fabb21ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac071300 con 0x7fabac0c0140 2023-06-09T12:08:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabac0c1bc0 0x7fabac1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fabac0c1bc0 0x7fabac1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45820/0 (socket says 172.21.15.7:45820) 2023-06-09T12:08:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 -- 172.21.15.7:0/2028409641 learned_addr learned my addr 172.21.15.7:0/2028409641 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:08:39.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabaaffd700 1 --2- 172.21.15.7:0/2028409641 >> v2:172.21.15.117:3300/0 conn(0x7fabac0c0140 0x7fabac159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 -- 172.21.15.7:0/2028409641 >> v2:172.21.15.117:3300/0 conn(0x7fabac0c0140 msgr2=0x7fabac159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:08:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 --2- 172.21.15.7:0/2028409641 >> v2:172.21.15.117:3300/0 conn(0x7fabac0c0140 0x7fabac159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 -- 172.21.15.7:0/2028409641 >> v1:172.21.15.7:6789/0 conn(0x7fabac0c1f90 legacy=0x7fabac1501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 -- 172.21.15.7:0/2028409641 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac0702f0 con 0x7fabac0c1bc0 2023-06-09T12:08:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.564+0000 7fabab7fe700 1 --2- 172.21.15.7:0/2028409641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1bc0 0x7fabac1599b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fabac06d950 tx=0x7fab9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c03e0bb15204daf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:39.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabaa7fc700 1 -- 172.21.15.7:0/2028409641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011830 con 0x7fabac0c1bc0 2023-06-09T12:08:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabaa7fc700 1 -- 172.21.15.7:0/2028409641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9c011990 con 0x7fabac0c1bc0 2023-06-09T12:08:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabaa7fc700 1 -- 172.21.15.7:0/2028409641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9c011d20 con 0x7fabac0c1bc0 2023-06-09T12:08:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 -- 172.21.15.7:0/2028409641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1bc0 msgr2=0x7fabac1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 --2- 172.21.15.7:0/2028409641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1bc0 0x7fabac1599b0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fabac06d950 tx=0x7fab9c009bd0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 -- 172.21.15.7:0/2028409641 shutdown_connections 2023-06-09T12:08:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 --2- 172.21.15.7:0/2028409641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1bc0 0x7fabac1599b0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 --2- 172.21.15.7:0/2028409641 >> v2:172.21.15.117:3300/0 conn(0x7fabac0c0140 0x7fabac159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 -- 172.21.15.7:0/2028409641 >> 172.21.15.7:0/2028409641 conn(0x7fabac06a4d0 msgr2=0x7fabac06a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:08:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 -- 172.21.15.7:0/2028409641 shutdown_connections 2023-06-09T12:08:39.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.565+0000 7fabb21ff700 1 -- 172.21.15.7:0/2028409641 wait complete. 2023-06-09T12:08:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.567+0000 7fabb21ff700 1 Processor -- start 2023-06-09T12:08:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.567+0000 7fabb21ff700 1 -- start start 2023-06-09T12:08:39.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabac0c0140 0x7fabac14c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabac0c1bc0 0x7fabac1eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1ee900 con 0x7fabac0c1f90 2023-06-09T12:08:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac14e990 con 0x7fabac0c0140 2023-06-09T12:08:39.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fabb21ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1f2c10 con 0x7fabac0c1bc0 2023-06-09T12:08:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fababfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.568+0000 7fababfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45822/0 (socket says 172.21.15.7:45822) 2023-06-09T12:08:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 -- 172.21.15.7:0/138514229 learned_addr learned my addr 172.21.15.7:0/138514229 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:08:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fabab7fe700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabac0c0140 0x7fabac14c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fabaaffd700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabac0c1bc0 0x7fabac1eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 -- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabac0c1bc0 msgr2=0x7fabac1eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabac0c1bc0 0x7fabac1eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 -- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabac0c0140 msgr2=0x7fabac14c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabac0c0140 0x7fabac14c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 -- 172.21.15.7:0/138514229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c009550 con 0x7fabac0c1f90 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fababfff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7faba400cb50 tx=0x7faba400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba4003c00 con 0x7fabac0c1f90 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba4003d60 con 0x7fabac0c1f90 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac0c1170 con 0x7fabac0c1f90 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.569+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba400aa20 con 0x7fabac0c1f90 2023-06-09T12:08:39.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.570+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac0c0e90 con 0x7fabac0c1f90 2023-06-09T12:08:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.571+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac0c1450 con 0x7fabac0c1f90 2023-06-09T12:08:39.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.572+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faba400ab80 con 0x7fabac0c1f90 2023-06-09T12:08:39.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.573+0000 7fab93fff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 0x7fab9806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:08:39.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.573+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7faba4089e70 con 0x7fabac0c1f90 2023-06-09T12:08:39.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.573+0000 7fabab7fe700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 0x7fab9806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:08:39.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.573+0000 7fabab7fe700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 0x7fab9806d1c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009c10 tx=0x7fab9c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:08:39.581 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.577+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faba4054510 con 0x7fabac0c1f90 2023-06-09T12:08:39.782 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.779+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabac1eea40 con 0x7fab9806ad10 2023-06-09T12:08:39.791 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.788+0000 7fab93fff700 1 -- 172.21.15.7:0/138514229 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317331 (secure 0 0 0) 0x7fabac1eea40 con 0x7fab9806ad10 2023-06-09T12:08:39.795 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:08:39.796 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 msgr2=0x7fab9806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 0x7fab9806d1c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009c10 tx=0x7fab9c008030 comp rx=0 tx=0).stop 2023-06-09T12:08:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 msgr2=0x7fabac1ef290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:08:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7faba400cb50 tx=0x7faba400ce60 comp rx=0 tx=0).stop 2023-06-09T12:08:39.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 shutdown_connections 2023-06-09T12:08:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fabac0c1f90 0x7fabac1ef290 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fabac0c0140 0x7fabac14c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fab9806ad10 0x7fab9806d1c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 --2- 172.21.15.7:0/138514229 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fabac0c1bc0 0x7fabac1eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:08:39.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 >> 172.21.15.7:0/138514229 conn(0x7fabac06a4d0 msgr2=0x7fabac156a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:08:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 shutdown_connections 2023-06-09T12:08:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:08:39.793+0000 7fabb21ff700 1 -- 172.21.15.7:0/138514229 wait complete. 2023-06-09T12:08:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:08:39.814 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1806,"stamp":"2023-06-09T12:08:39.076118+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1449985,"num_omap_keys":2724,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97786,"ondisk_log_size":97786,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2579208,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":695872,"kb_avail":752395512,"statfs":{"total":773094113280,"available":770453004288,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":712572928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003017"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1002","reported_seq":3358,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:34.230396+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:08:34.230396+0000","last_peered":"2023-06-09T12:08:34.230396+0000","last_clean":"2023-06-09T12:08:34.230396+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:34.230396+0000","last_undegraded":"2023-06-09T12:08:34.230396+0000","last_fullsized":"2023-06-09T12:08:34.230396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":120,"log_size":1002,"log_dups_size":0,"ondisk_log_size":1002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:52.419323+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:03:52.419323+0000","last_peered":"2023-06-09T12:03:52.419323+0000","last_clean":"2023-06-09T12:03:52.419323+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:03:52.419323+0000","last_undegraded":"2023-06-09T12:03:52.419323+0000","last_fullsized":"2023-06-09T12:03:52.419323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4204,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.338973+0000","last_change":"2023-06-09T12:08:29.205789+0000","last_active":"2023-06-09T12:06:37.338973+0000","last_peered":"2023-06-09T12:06:37.338973+0000","last_clean":"2023-06-09T12:06:37.338973+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:37.338973+0000","last_undegraded":"2023-06-09T12:06:37.338973+0000","last_fullsized":"2023-06-09T12:06:37.338973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:02.437140+0000","last_change":"2023-06-09T12:04:02.437140+0000","last_active":"2023-06-09T12:04:02.437140+0000","last_peered":"2023-06-09T12:04:02.437140+0000","last_clean":"2023-06-09T12:04:02.437140+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:04:02.437140+0000","last_undegraded":"2023-06-09T12:04:02.437140+0000","last_fullsized":"2023-06-09T12:04:02.437140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3740,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.340791+0000","last_change":"2023-06-09T12:08:25.145466+0000","last_active":"2023-06-09T12:06:52.340791+0000","last_peered":"2023-06-09T12:06:52.340791+0000","last_clean":"2023-06-09T12:06:52.340791+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:06:52.340791+0000","last_undegraded":"2023-06-09T12:06:52.340791+0000","last_fullsized":"2023-06-09T12:06:52.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:19.929377+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:04:19.929377+0000","last_peered":"2023-06-09T12:04:19.929377+0000","last_clean":"2023-06-09T12:04:19.929377+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:04:19.929377+0000","last_undegraded":"2023-06-09T12:04:19.929377+0000","last_fullsized":"2023-06-09T12:04:19.929377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3364,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:20.424864+0000","last_change":"2023-06-09T12:07:36.313766+0000","last_active":"2023-06-09T12:08:20.424864+0000","last_peered":"2023-06-09T12:08:20.424864+0000","last_clean":"2023-06-09T12:08:20.424864+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:08:20.424864+0000","last_undegraded":"2023-06-09T12:08:20.424864+0000","last_fullsized":"2023-06-09T12:08:20.424864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:37.318955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:04:37.318955+0000","last_peered":"2023-06-09T12:04:37.318955+0000","last_clean":"2023-06-09T12:04:37.318955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:04:37.318955+0000","last_undegraded":"2023-06-09T12:04:37.318955+0000","last_fullsized":"2023-06-09T12:04:37.318955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3317,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:48.175151+0000","last_change":"2023-06-09T12:07:27.036778+0000","last_active":"2023-06-09T12:07:48.175151+0000","last_peered":"2023-06-09T12:07:48.175151+0000","last_clean":"2023-06-09T12:07:48.175151+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:07:48.175151+0000","last_undegraded":"2023-06-09T12:07:48.175151+0000","last_fullsized":"2023-06-09T12:07:48.175151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":365,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.284077+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:10.284077+0000","last_peered":"2023-06-09T12:06:10.284077+0000","last_clean":"2023-06-09T12:06:10.284077+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:10.284077+0000","last_undegraded":"2023-06-09T12:06:10.284077+0000","last_fullsized":"2023-06-09T12:06:10.284077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2968,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348318+0000","last_change":"2023-06-09T12:07:55.142523+0000","last_active":"2023-06-09T12:06:52.348318+0000","last_peered":"2023-06-09T12:06:52.348318+0000","last_clean":"2023-06-09T12:06:52.348318+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:52.348318+0000","last_undegraded":"2023-06-09T12:06:52.348318+0000","last_fullsized":"2023-06-09T12:06:52.348318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3281,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348357+0000","last_change":"2023-06-09T12:07:47.094943+0000","last_active":"2023-06-09T12:06:52.348357+0000","last_peered":"2023-06-09T12:06:52.348357+0000","last_clean":"2023-06-09T12:06:52.348357+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:06:52.348357+0000","last_undegraded":"2023-06-09T12:06:52.348357+0000","last_fullsized":"2023-06-09T12:06:52.348357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3414,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854411+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854411+0000","last_peered":"2023-06-09T12:06:40.854411+0000","last_clean":"2023-06-09T12:06:40.854411+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854411+0000","last_undegraded":"2023-06-09T12:06:40.854411+0000","last_fullsized":"2023-06-09T12:06:40.854411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1039,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:02.461482+0000","last_change":"2023-06-09T12:07:44.447705+0000","last_active":"2023-06-09T12:07:02.461482+0000","last_peered":"2023-06-09T12:07:02.461482+0000","last_clean":"2023-06-09T12:07:02.461482+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:02.461482+0000","last_undegraded":"2023-06-09T12:07:02.461482+0000","last_fullsized":"2023-06-09T12:07:02.461482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":698,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:18.428660+0000","last_change":"2023-06-09T12:08:18.428660+0000","last_active":"2023-06-09T12:08:18.428660+0000","last_peered":"2023-06-09T12:08:18.428660+0000","last_clean":"2023-06-09T12:08:18.428660+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:08:18.428660+0000","last_undegraded":"2023-06-09T12:08:18.428660+0000","last_fullsized":"2023-06-09T12:08:18.428660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010592+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010592+0000","last_peered":"2023-06-09T12:07:35.010592+0000","last_clean":"2023-06-09T12:07:35.010592+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010592+0000","last_undegraded":"2023-06-09T12:07:35.010592+0000","last_fullsized":"2023-06-09T12:07:35.010592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":700,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.238428+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:12.238428+0000","last_peered":"2023-06-09T12:07:12.238428+0000","last_clean":"2023-06-09T12:07:12.238428+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:12.238428+0000","last_undegraded":"2023-06-09T12:07:12.238428+0000","last_fullsized":"2023-06-09T12:07:12.238428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1380,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:27.590556+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:04:27.590556+0000","last_peered":"2023-06-09T12:04:27.590556+0000","last_clean":"2023-06-09T12:04:27.590556+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:04:27.590556+0000","last_undegraded":"2023-06-09T12:04:27.590556+0000","last_fullsized":"2023-06-09T12:04:27.590556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045859+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045859+0000","last_peered":"2023-06-09T12:06:50.045859+0000","last_clean":"2023-06-09T12:06:50.045859+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045859+0000","last_undegraded":"2023-06-09T12:06:50.045859+0000","last_fullsized":"2023-06-09T12:06:50.045859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329373+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329373+0000","last_peered":"2023-06-09T12:07:57.329373+0000","last_clean":"2023-06-09T12:07:57.329373+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329373+0000","last_undegraded":"2023-06-09T12:07:57.329373+0000","last_fullsized":"2023-06-09T12:07:57.329373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":759,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350677+0000","last_change":"2023-06-09T12:07:54.093801+0000","last_active":"2023-06-09T12:07:12.350677+0000","last_peered":"2023-06-09T12:07:12.350677+0000","last_clean":"2023-06-09T12:07:12.350677+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:12.350677+0000","last_undegraded":"2023-06-09T12:07:12.350677+0000","last_fullsized":"2023-06-09T12:07:12.350677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:30.580107+0000","last_change":"2023-06-09T12:07:26.474113+0000","last_active":"2023-06-09T12:07:30.580107+0000","last_peered":"2023-06-09T12:07:30.580107+0000","last_clean":"2023-06-09T12:07:30.580107+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:30.580107+0000","last_undegraded":"2023-06-09T12:07:30.580107+0000","last_fullsized":"2023-06-09T12:07:30.580107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3897,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:29.552824+0000","last_change":"2023-06-09T12:07:25.510423+0000","last_active":"2023-06-09T12:07:29.552824+0000","last_peered":"2023-06-09T12:07:29.552824+0000","last_clean":"2023-06-09T12:07:29.552824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:29.552824+0000","last_undegraded":"2023-06-09T12:07:29.552824+0000","last_fullsized":"2023-06-09T12:07:29.552824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4137,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:38.619113+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:04:38.619113+0000","last_peered":"2023-06-09T12:04:38.619113+0000","last_clean":"2023-06-09T12:04:38.619113+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:04:38.619113+0000","last_undegraded":"2023-06-09T12:04:38.619113+0000","last_fullsized":"2023-06-09T12:04:38.619113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.348407+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:04:42.348407+0000","last_peered":"2023-06-09T12:04:42.348407+0000","last_clean":"2023-06-09T12:04:42.348407+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:04:42.348407+0000","last_undegraded":"2023-06-09T12:04:42.348407+0000","last_fullsized":"2023-06-09T12:04:42.348407+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.434156+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:05:57.434156+0000","last_peered":"2023-06-09T12:05:57.434156+0000","last_clean":"2023-06-09T12:05:57.434156+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:05:57.434156+0000","last_undegraded":"2023-06-09T12:05:57.434156+0000","last_fullsized":"2023-06-09T12:05:57.434156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:22.362173+0000","last_change":"2023-06-09T12:08:17.405114+0000","last_active":"2023-06-09T12:08:22.362173+0000","last_peered":"2023-06-09T12:08:22.362173+0000","last_clean":"2023-06-09T12:08:22.362173+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:08:22.362173+0000","last_undegraded":"2023-06-09T12:08:22.362173+0000","last_fullsized":"2023-06-09T12:08:22.362173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2993,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.342586+0000","last_change":"2023-06-09T12:08:31.205177+0000","last_active":"2023-06-09T12:07:07.342586+0000","last_peered":"2023-06-09T12:07:07.342586+0000","last_clean":"2023-06-09T12:07:07.342586+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:07:07.342586+0000","last_undegraded":"2023-06-09T12:07:07.342586+0000","last_fullsized":"2023-06-09T12:07:07.342586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":526,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350787+0000","last_change":"2023-06-09T12:07:53.051874+0000","last_active":"2023-06-09T12:07:12.350787+0000","last_peered":"2023-06-09T12:07:12.350787+0000","last_clean":"2023-06-09T12:07:12.350787+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:12.350787+0000","last_undegraded":"2023-06-09T12:07:12.350787+0000","last_fullsized":"2023-06-09T12:07:12.350787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'874","reported_seq":3377,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:37.329215+0000","last_change":"2023-06-09T12:08:36.954546+0000","last_active":"2023-06-09T12:08:37.329215+0000","last_peered":"2023-06-09T12:08:37.329215+0000","last_clean":"2023-06-09T12:08:37.329215+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:08:37.329215+0000","last_undegraded":"2023-06-09T12:08:37.329215+0000","last_fullsized":"2023-06-09T12:08:37.329215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:42.107995+0000","last_change":"2023-06-09T12:07:25.019244+0000","last_active":"2023-06-09T12:07:42.107995+0000","last_peered":"2023-06-09T12:07:42.107995+0000","last_clean":"2023-06-09T12:07:42.107995+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:07:42.107995+0000","last_undegraded":"2023-06-09T12:07:42.107995+0000","last_fullsized":"2023-06-09T12:07:42.107995+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":834,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.343238+0000","last_change":"2023-06-09T12:08:34.171989+0000","last_active":"2023-06-09T12:07:12.343238+0000","last_peered":"2023-06-09T12:07:12.343238+0000","last_clean":"2023-06-09T12:07:12.343238+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:07:12.343238+0000","last_undegraded":"2023-06-09T12:07:12.343238+0000","last_fullsized":"2023-06-09T12:07:12.343238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3719,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445633+0000","last_change":"2023-06-09T12:05:02.445633+0000","last_active":"2023-06-09T12:05:02.445633+0000","last_peered":"2023-06-09T12:05:02.445633+0000","last_clean":"2023-06-09T12:05:02.445633+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:05:02.445633+0000","last_undegraded":"2023-06-09T12:05:02.445633+0000","last_fullsized":"2023-06-09T12:05:02.445633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'692","reported_seq":887,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:16.409548+0000","last_change":"2023-06-09T12:08:22.366980+0000","last_active":"2023-06-09T12:06:16.409548+0000","last_peered":"2023-06-09T12:06:16.409548+0000","last_clean":"2023-06-09T12:06:16.409548+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:06:16.409548+0000","last_undegraded":"2023-06-09T12:06:16.409548+0000","last_fullsized":"2023-06-09T12:06:16.409548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":777,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.420320+0000","last_change":"2023-06-09T12:08:22.376989+0000","last_active":"2023-06-09T12:08:17.420320+0000","last_peered":"2023-06-09T12:08:17.420320+0000","last_clean":"2023-06-09T12:08:17.420320+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:08:17.420320+0000","last_undegraded":"2023-06-09T12:08:17.420320+0000","last_fullsized":"2023-06-09T12:08:17.420320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3828,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:03:58.332393+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:03:58.332393+0000","last_peered":"2023-06-09T12:03:58.332393+0000","last_clean":"2023-06-09T12:03:58.332393+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:03:58.332393+0000","last_undegraded":"2023-06-09T12:03:58.332393+0000","last_fullsized":"2023-06-09T12:03:58.332393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.346516+0000","last_change":"2023-06-09T12:07:48.112027+0000","last_active":"2023-06-09T12:06:37.346516+0000","last_peered":"2023-06-09T12:06:37.346516+0000","last_clean":"2023-06-09T12:06:37.346516+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:06:37.346516+0000","last_undegraded":"2023-06-09T12:06:37.346516+0000","last_fullsized":"2023-06-09T12:06:37.346516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3486,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:27.456861+0000","last_change":"2023-06-09T12:08:37.363233+0000","last_active":"2023-06-09T12:06:27.456861+0000","last_peered":"2023-06-09T12:06:27.456861+0000","last_clean":"2023-06-09T12:06:27.456861+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:06:27.456861+0000","last_undegraded":"2023-06-09T12:06:27.456861+0000","last_fullsized":"2023-06-09T12:06:27.456861+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1154","reported_seq":3673,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:35.742196+0000","last_change":"2023-06-09T12:07:31.039611+0000","last_active":"2023-06-09T12:08:35.742196+0000","last_peered":"2023-06-09T12:08:35.742196+0000","last_clean":"2023-06-09T12:08:35.742196+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:35.742196+0000","last_undegraded":"2023-06-09T12:08:35.742196+0000","last_fullsized":"2023-06-09T12:08:35.742196+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":221,"log_size":1154,"log_dups_size":0,"ondisk_log_size":1154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 33s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3942,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.350050+0000","last_change":"2023-06-09T12:07:49.110762+0000","last_active":"2023-06-09T12:07:07.350050+0000","last_peered":"2023-06-09T12:07:07.350050+0000","last_clean":"2023-06-09T12:07:07.350050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:07.350050+0000","last_undegraded":"2023-06-09T12:07:07.350050+0000","last_fullsized":"2023-06-09T12:07:07.350050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2984,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.351371+0000","last_change":"2023-06-09T12:07:51.130097+0000","last_active":"2023-06-09T12:07:17.351371+0000","last_peered":"2023-06-09T12:07:17.351371+0000","last_clean":"2023-06-09T12:07:17.351371+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:17.351371+0000","last_undegraded":"2023-06-09T12:07:17.351371+0000","last_fullsized":"2023-06-09T12:07:17.351371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":66823,"ondisk_log_size":66823,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1449985,"num_omap_keys":2724,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30844,"ondisk_log_size":30844,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575546,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260560,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80704,"kb_avail":94111280,"statfs":{"total":96636764160,"available":96369950720,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82640896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68999999999999995,"15min":0.68200000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.378,"5min":0.378,"15min":0.35299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.3320000000000001},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72499999999999998,"15min":0.70099999999999996},"min":{"1min":0.47399999999999998,"5min":0.42999999999999999,"15min":0.36899999999999999},"max":{"1min":0.91100000000000003,"5min":1.0229999999999999,"15min":1.081},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76600000000000001,"15min":0.71899999999999997},"min":{"1min":0.55600000000000005,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94499999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.746}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.748,"15min":0.73599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.39500000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:07:54 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.68300000000000005},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71299999999999997,"15min":0.71699999999999997},"min":{"1min":0.51700000000000002,"5min":0.505,"15min":0.441},"max":{"1min":0.80100000000000005,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.72899999999999998,"5min":1.006,"15min":1.006},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.73499999999999999,"15min":0.69799999999999995},"min":{"1min":0.60799999999999998,"5min":0.497,"15min":0.32700000000000001},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.61399999999999999},"min":{"1min":0.41399999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.76900000000000002,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.627},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":0.92500000000000004,"15min":0.97999999999999998},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64700000000000002,"15min":0.65700000000000003},"min":{"1min":0.48799999999999999,"5min":0.44400000000000001,"15min":0.38100000000000001},"max":{"1min":0.84799999999999998,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.96899999999999997,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.78200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575546,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209304,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94162536,"statfs":{"total":96636764160,"available":96422436864,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.67800000000000005,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69599999999999995,"15min":0.66800000000000004},"min":{"1min":0.45100000000000001,"5min":0.41099999999999998,"15min":0.30599999999999999},"max":{"1min":0.91400000000000003,"5min":0.96799999999999997,"15min":1.069},"last":0.628}]},{"osd":1,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71899999999999997,"15min":0.69099999999999995},"min":{"1min":0.54700000000000004,"5min":0.498,"15min":0.312},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76600000000000001,"15min":0.74299999999999999},"min":{"1min":0.60799999999999998,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":1.0029999999999999,"5min":1.004,"15min":1.0289999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73499999999999999,"15min":0.70499999999999996},"min":{"1min":0.52900000000000003,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76800000000000002,"15min":0.747},"min":{"1min":0.54700000000000004,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.72799999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.746,"15min":0.72499999999999998},"min":{"1min":0.51400000000000001,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":1.022,"5min":1.03,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80500000000000005,"15min":0.77500000000000002},"min":{"1min":0.60299999999999998,"5min":0.55900000000000005,"15min":0.439},"max":{"1min":0.97699999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79900000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.82699999999999996,"5min":0.91100000000000003,"15min":0.95299999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59699999999999998,"15min":0.57899999999999996},"min":{"1min":0.44500000000000001,"5min":0.38300000000000001,"15min":0.34000000000000002},"max":{"1min":0.84499999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.46800000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.628,"15min":0.67100000000000004},"min":{"1min":0.45000000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83899999999999997,"5min":0.96999999999999997,"15min":1.0700000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73599999999999999,"15min":0.67200000000000004},"min":{"1min":0.56200000000000006,"5min":0.437,"15min":0.33800000000000002},"max":{"1min":0.89500000000000002,"5min":1.0129999999999999,"15min":1.052},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.70399999999999996},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.95199999999999996,"5min":1.0029999999999999,"15min":1.1100000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.65700000000000003},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.36299999999999999},"max":{"1min":0.91800000000000004,"5min":0.98699999999999999,"15min":1.071},"last":0.55300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575547,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394736,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88320,"kb_avail":93977104,"statfs":{"total":96636764160,"available":96232554496,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":90439680},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.45200000000000001,"15min":0.42399999999999999},"min":{"1min":0.317,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.76400000000000001,"5min":0.79500000000000004,"15min":0.79900000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.82699999999999996,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50900000000000001,"15min":0.48099999999999998},"min":{"1min":0.371,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.64000000000000001,"5min":0.995,"15min":0.995},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.32700000000000001},"max":{"1min":0.77800000000000002,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58099999999999996,"15min":0.56799999999999995},"min":{"1min":0.47999999999999998,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.69299999999999995,"5min":1.141,"15min":1.141},"last":0.504},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66900000000000004,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.78100000000000003,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.72499999999999998},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.0209999999999999,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.88300000000000001,"15min":0.83899999999999997},"min":{"1min":0.70099999999999996,"5min":0.66900000000000004,"15min":0.54100000000000004},"max":{"1min":1.093,"5min":1.597,"15min":1.597},"last":0.77400000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:07:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.86099999999999999,"15min":0.81299999999999994},"min":{"1min":0.61099999999999999,"5min":0.60799999999999998,"15min":0.54500000000000004},"max":{"1min":1.1539999999999999,"5min":1.544,"15min":1.544},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.88200000000000001,"15min":0.84999999999999998},"min":{"1min":0.63400000000000001,"5min":0.59899999999999998,"15min":0.55600000000000005},"max":{"1min":1.03,"5min":1.704,"15min":1.704},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83299999999999996,"15min":0.83099999999999996},"min":{"1min":0.622,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1299999999999999,"5min":1.155,"15min":1.4039999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.83599999999999997,"15min":0.78200000000000003},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.49399999999999999},"max":{"1min":1.111,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.745}]},{"osd":7,"last update":"Fri Jun 9 12:07:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.91400000000000003,"15min":0.88400000000000001},"min":{"1min":0.72999999999999998,"5min":0.71899999999999997,"15min":0.56299999999999994},"max":{"1min":1.1930000000000001,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.59099999999999997,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.177,"5min":1.355,"15min":1.355},"last":0.84599999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575546,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321384,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93568,"kb_avail":94050456,"statfs":{"total":96636764160,"available":96307666944,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95813632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.44700000000000001},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.65700000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34000000000000002},"max":{"1min":0.64900000000000002,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.48999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.81899999999999995,"15min":1.032},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.35699999999999998,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.91700000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":0.79900000000000004,"5min":0.85799999999999998,"15min":1.0109999999999999},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84399999999999997,"15min":0.75900000000000001},"min":{"1min":0.68600000000000005,"5min":0.627,"15min":0.36899999999999999},"max":{"1min":1.0149999999999999,"5min":1.204,"15min":1.204},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":0.82599999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.42899999999999999},"max":{"1min":1.0629999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":1.121,"15min":1.121},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.91400000000000003,"15min":0.82099999999999995},"min":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":0.97999999999999998,"5min":1.232,"15min":1.232},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86599999999999999,"15min":0.81899999999999995},"min":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.47299999999999998},"max":{"1min":1.101,"5min":1.298,"15min":1.298},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95899999999999996,"15min":0.89400000000000002},"min":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.52400000000000002},"max":{"1min":1.143,"5min":1.2589999999999999,"15min":1.351},"last":0.75600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575547,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247456,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":67648,"kb_avail":94124384,"statfs":{"total":96636764160,"available":96383369216,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":69271552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47199999999999998,"15min":0.442},"min":{"1min":0.34999999999999998,"5min":0.28799999999999998,"15min":0.27100000000000002},"max":{"1min":0.69599999999999995,"5min":0.70299999999999996,"15min":0.94399999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.53300000000000003},"min":{"1min":0.439,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.34899999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.3109999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65900000000000003,"15min":0.623},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.367},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.44,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.546},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.68899999999999995,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77200000000000002,"15min":0.76100000000000001},"min":{"1min":0.63600000000000001,"5min":0.48999999999999999,"15min":0.373},"max":{"1min":1.202,"5min":1.202,"15min":1.8220000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80100000000000005,"15min":0.77700000000000002},"min":{"1min":0.63700000000000001,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":1.0409999999999999,"5min":1.214,"15min":1.7370000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.84799999999999998,"15min":0.81699999999999995},"min":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3580000000000001,"15min":1.601},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.89400000000000002,"15min":0.86199999999999999},"min":{"1min":0.73099999999999998,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.72699999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.82899999999999996,"15min":0.79300000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":1.115,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.79200000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.121},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:07:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.85599999999999998,"15min":0.81499999999999995},"min":{"1min":0.68200000000000005,"5min":0.59799999999999998,"15min":0.497},"max":{"1min":1.1559999999999999,"5min":1.1659999999999999,"15min":1.208},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.86199999999999999,"15min":0.83999999999999997},"min":{"1min":0.68799999999999994,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.78700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575546,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427388,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104192,"kb_avail":93944452,"statfs":{"total":96636764160,"available":96199118848,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106692608},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48999999999999999,"15min":0.47499999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.223},"max":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":1.7969999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57699999999999996,"15min":0.55500000000000005},"min":{"1min":0.45100000000000001,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":1.5669999999999999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.40100000000000002,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.377,"15min":0.29799999999999999},"max":{"1min":0.84399999999999997,"5min":0.877,"15min":1.198},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.76200000000000001},"min":{"1min":0.58699999999999997,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.635},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80200000000000005,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.171,"5min":1.171,"15min":3.081},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.80200000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0449999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.90300000000000002,"15min":0.84499999999999997},"min":{"1min":0.73399999999999999,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.1000000000000001,"5min":1.165,"15min":1.367},"last":0.92800000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.80000000000000004},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.198,"5min":1.198,"15min":1.286},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84099999999999997,"15min":0.80300000000000005},"min":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.434},"max":{"1min":1.117,"5min":1.117,"15min":3.3490000000000002},"last":0.96499999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.51600000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.413},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.86199999999999999,"15min":0.86699999999999999},"min":{"1min":0.70299999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.94199999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575545,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424212,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131456,"kb_avail":93947628,"statfs":{"total":96636764160,"available":96202371072,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134610944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.54100000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0529999999999999,"5min":1.0600000000000001,"15min":1.137},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.55600000000000005,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.66100000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.73399999999999999},"min":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.002,"15min":1.0640000000000001},"last":0.752},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.69699999999999995},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.609},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.72499999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":0.96699999999999997,"5min":1.107,"15min":1.159},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.86299999999999999,"5min":0.999,"15min":1.407},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.48299999999999998},"max":{"1min":0.86099999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.71599999999999997,"5min":0.90400000000000003,"15min":1.0349999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.622},"min":{"1min":0.46600000000000003,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83699999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.85699999999999998,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":1.131,"15min":1.3100000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92300000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.091,"15min":1.361},"last":0.501}]}]},{"osd":5,"up_from":13,"seq":55834575545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294168,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76800,"kb_avail":94077672,"statfs":{"total":96636764160,"available":96335536128,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78643200},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.70199999999999996,"15min":0.70899999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.40999999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":5.6269999999999998},"last":1.0469999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72999999999999998,"15min":0.76100000000000001},"min":{"1min":0.48799999999999999,"5min":0.41199999999999998,"15min":0.35299999999999998},"max":{"1min":0.97999999999999998,"5min":1.083,"15min":19.715},"last":1.167},{"interface":"front","average":{"1min":0.752,"5min":0.751,"15min":0.749},"min":{"1min":0.49099999999999999,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.94899999999999995,"5min":1.1870000000000001,"15min":7.4089999999999998},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.91200000000000003,"5min":1.016,"15min":5.5620000000000003},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74299999999999999,"15min":0.77600000000000002},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.45400000000000001},"max":{"1min":0.95899999999999996,"5min":1.145,"15min":14.439},"last":1.083}]},{"osd":3,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72599999999999998,"15min":0.76700000000000002},"min":{"1min":0.53000000000000003,"5min":0.49299999999999999,"15min":0.41699999999999998},"max":{"1min":1.254,"5min":1.254,"15min":12.130000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.82399999999999995},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.274,"5min":1.274,"15min":12.218999999999999},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.55300000000000005,"15min":0.81499999999999995},"min":{"1min":0.40999999999999998,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.75600000000000001,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58799999999999997,"15min":0.83499999999999996},"min":{"1min":0.45200000000000001,"5min":0.40899999999999997,"15min":0.35199999999999998},"max":{"1min":0.71699999999999997,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.62,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.83799999999999997,"5min":1.097,"15min":12.163},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.66500000000000004},"min":{"1min":0.56699999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":1.095,"5min":1.095,"15min":7.6040000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:07:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63200000000000001,"15min":0.67300000000000004},"min":{"1min":0.49199999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.83199999999999996,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76100000000000001,"15min":0.82699999999999996},"min":{"1min":0.54500000000000004,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":1.238,"5min":1.238,"15min":19.834},"last":0.59899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:08:39.821 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.821 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.822 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.822 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.823 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.823 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.823 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.824 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.824 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.825 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.825 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.825 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.826 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.826 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.826 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.827 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.827 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.827 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.828 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.828 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.829 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.829 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.829 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.830 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.830 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.830 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.831 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.831 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.831 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.832 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.832 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.833 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.833 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.833 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.834 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.834 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.834 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.835 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.835 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.835 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.836 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.836 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:08:39.836 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:08:58.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:58.289 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:08:59.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:09:00.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.047+0000 7fadcf7ff700 1 Processor -- start 2023-06-09T12:09:00.050 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.047+0000 7fadcf7ff700 1 -- start start 2023-06-09T12:09:00.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.048+0000 7fadcf7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fadc80ca1b0 0x7fadc815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.048+0000 7fadcf7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadc80b5360 0x7fadc80c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadcf7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc815dcf0 con 0x7fadc80ca580 2023-06-09T12:09:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadcf7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc80c5e00 con 0x7fadc80b5360 2023-06-09T12:09:00.052 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadcf7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc815d450 con 0x7fadc80ca1b0 2023-06-09T12:09:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadc80b5360 0x7fadc80c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fadc80b5360 0x7fadc80c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37248/0 (socket says 172.21.15.7:37248) 2023-06-09T12:09:00.053 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 -- 172.21.15.7:0/2254673124 learned_addr learned my addr 172.21.15.7:0/2254673124 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:09:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadcd59b700 1 --2- 172.21.15.7:0/2254673124 >> v2:172.21.15.117:3300/0 conn(0x7fadc80ca1b0 0x7fadc815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.054 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 -- 172.21.15.7:0/2254673124 >> v2:172.21.15.117:3300/0 conn(0x7fadc80ca1b0 msgr2=0x7fadc815cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:09:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 --2- 172.21.15.7:0/2254673124 >> v2:172.21.15.117:3300/0 conn(0x7fadc80ca1b0 0x7fadc815cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 -- 172.21.15.7:0/2254673124 >> v1:172.21.15.7:6789/0 conn(0x7fadc80ca580 legacy=0x7fadc81536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.049+0000 7fadccd9a700 1 -- 172.21.15.7:0/2254673124 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc815d730 con 0x7fadc80b5360 2023-06-09T12:09:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadccd9a700 1 --2- 172.21.15.7:0/2254673124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80b5360 0x7fadc80c58c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fadc806bb80 tx=0x7fadb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2908e3236c6e95 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:00.055 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadbffff700 1 -- 172.21.15.7:0/2254673124 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb800b870 con 0x7fadc80b5360 2023-06-09T12:09:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadbffff700 1 -- 172.21.15.7:0/2254673124 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb800b9d0 con 0x7fadc80b5360 2023-06-09T12:09:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadbffff700 1 -- 172.21.15.7:0/2254673124 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb800bd60 con 0x7fadc80b5360 2023-06-09T12:09:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 -- 172.21.15.7:0/2254673124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80b5360 msgr2=0x7fadc80c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/2254673124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80b5360 0x7fadc80c58c0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fadc806bb80 tx=0x7fadb8009bd0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.056 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 -- 172.21.15.7:0/2254673124 shutdown_connections 2023-06-09T12:09:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/2254673124 >> v2:172.21.15.117:3300/0 conn(0x7fadc80ca1b0 0x7fadc815cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/2254673124 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80b5360 0x7fadc80c58c0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 -- 172.21.15.7:0/2254673124 >> 172.21.15.7:0/2254673124 conn(0x7fadc8072090 msgr2=0x7fadc8072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.050+0000 7fadcf7ff700 1 -- 172.21.15.7:0/2254673124 shutdown_connections 2023-06-09T12:09:00.057 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.051+0000 7fadcf7ff700 1 -- 172.21.15.7:0/2254673124 wait complete. 2023-06-09T12:09:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.052+0000 7fadcf7ff700 1 Processor -- start 2023-06-09T12:09:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.052+0000 7fadcf7ff700 1 -- start start 2023-06-09T12:09:00.059 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 0x7fadc8157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc80ca580 0x7fadc81f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc814f000 con 0x7fadc80ca1b0 2023-06-09T12:09:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8159470 con 0x7fadc80ca580 2023-06-09T12:09:00.060 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadcf7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc80b46a0 con 0x7fadc80b5360 2023-06-09T12:09:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadccd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadccd9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37258/0 (socket says 172.21.15.7:37258) 2023-06-09T12:09:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.053+0000 7fadccd9a700 1 -- 172.21.15.7:0/3754765385 learned_addr learned my addr 172.21.15.7:0/3754765385 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:09:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadcd59b700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 0x7fadc8157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.061 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadcdd9c700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc80ca580 0x7fadc81f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 -- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 msgr2=0x7fadc8157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 0x7fadc8157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 -- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc80ca580 msgr2=0x7fadc81f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc80ca580 0x7fadc81f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.062 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 -- 172.21.15.7:0/3754765385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009550 con 0x7fadc80ca1b0 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadcd59b700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 0x7fadc8157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadccd9a700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8004540 con 0x7fadc80ca1b0 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb80036a0 con 0x7fadc80ca1b0 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8003a30 con 0x7fadc80ca1b0 2023-06-09T12:09:00.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.054+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc806a570 con 0x7fadc80ca1b0 2023-06-09T12:09:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.055+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc815da10 con 0x7fadc80ca1b0 2023-06-09T12:09:00.064 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.056+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc806a850 con 0x7fadc80ca1b0 2023-06-09T12:09:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.056+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fadb8021070 con 0x7fadc80ca1b0 2023-06-09T12:09:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.057+0000 7fadbdffb700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 0x7fadac06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.058+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fadb808b230 con 0x7fadc80ca1b0 2023-06-09T12:09:00.065 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.058+0000 7fadcd59b700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 0x7fadac06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.058+0000 7fadcd59b700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 0x7fadac06d1c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fadc40099f0 tx=0x7fadc4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:00.066 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.062+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fadb8055960 con 0x7fadc80ca1b0 2023-06-09T12:09:00.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.265+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fadc8074cf0 con 0x7fadac06ad10 2023-06-09T12:09:00.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.274+0000 7fadbdffb700 1 -- 172.21.15.7:0/3754765385 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317261 (secure 0 0 0) 0x7fadc8074cf0 con 0x7fadac06ad10 2023-06-09T12:09:00.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:09:00.282 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 msgr2=0x7fadac06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 0x7fadac06d1c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fadc40099f0 tx=0x7fadc4008040 comp rx=0 tx=0).stop 2023-06-09T12:09:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 msgr2=0x7fadc8157b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000c00 tx=0x7fadb8004750 comp rx=0 tx=0).stop 2023-06-09T12:09:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 shutdown_connections 2023-06-09T12:09:00.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fadc80ca580 0x7fadc81f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fadac06ad10 0x7fadac06d1c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fadc80b5360 0x7fadc8157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 --2- 172.21.15.7:0/3754765385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fadc80ca1b0 0x7fadc8157b90 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 >> 172.21.15.7:0/3754765385 conn(0x7fadc8072090 msgr2=0x7fadc8151cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:00.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 shutdown_connections 2023-06-09T12:09:00.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:00.279+0000 7fadcf7ff700 1 -- 172.21.15.7:0/3754765385 wait complete. 2023-06-09T12:09:00.285 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:09:00.300 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1816,"stamp":"2023-06-09T12:08:59.081768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1512399,"num_omap_keys":2843,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98070,"ondisk_log_size":98070,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2579720,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":696384,"kb_avail":752395000,"statfs":{"total":773094113280,"available":770452480000,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":713097216},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003282"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3480,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:56.944589+0000","last_change":"2023-06-09T12:07:08.104741+0000","last_active":"2023-06-09T12:08:56.944589+0000","last_peered":"2023-06-09T12:08:56.944589+0000","last_clean":"2023-06-09T12:08:56.944589+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:56.944589+0000","last_undegraded":"2023-06-09T12:08:56.944589+0000","last_fullsized":"2023-06-09T12:08:56.944589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 36s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445731+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:05:02.445731+0000","last_peered":"2023-06-09T12:05:02.445731+0000","last_clean":"2023-06-09T12:05:02.445731+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:05:02.445731+0000","last_undegraded":"2023-06-09T12:05:02.445731+0000","last_fullsized":"2023-06-09T12:05:02.445731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4204,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.338973+0000","last_change":"2023-06-09T12:08:29.205789+0000","last_active":"2023-06-09T12:06:37.338973+0000","last_peered":"2023-06-09T12:06:37.338973+0000","last_clean":"2023-06-09T12:06:37.338973+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:06:37.338973+0000","last_undegraded":"2023-06-09T12:06:37.338973+0000","last_fullsized":"2023-06-09T12:06:37.338973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3740,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.340791+0000","last_change":"2023-06-09T12:08:25.145466+0000","last_active":"2023-06-09T12:06:52.340791+0000","last_peered":"2023-06-09T12:06:52.340791+0000","last_clean":"2023-06-09T12:06:52.340791+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:06:52.340791+0000","last_undegraded":"2023-06-09T12:06:52.340791+0000","last_fullsized":"2023-06-09T12:06:52.340791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.201855+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:05:55.201855+0000","last_peered":"2023-06-09T12:05:55.201855+0000","last_clean":"2023-06-09T12:05:55.201855+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:05:55.201855+0000","last_undegraded":"2023-06-09T12:05:55.201855+0000","last_fullsized":"2023-06-09T12:05:55.201855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3387,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:42.030595+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:04:42.030595+0000","last_peered":"2023-06-09T12:04:42.030595+0000","last_clean":"2023-06-09T12:04:42.030595+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:04:42.030595+0000","last_undegraded":"2023-06-09T12:04:42.030595+0000","last_fullsized":"2023-06-09T12:04:42.030595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3364,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:20.424864+0000","last_change":"2023-06-09T12:07:36.313766+0000","last_active":"2023-06-09T12:08:20.424864+0000","last_peered":"2023-06-09T12:08:20.424864+0000","last_clean":"2023-06-09T12:08:20.424864+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:08:20.424864+0000","last_undegraded":"2023-06-09T12:08:20.424864+0000","last_fullsized":"2023-06-09T12:08:20.424864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3909,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274080+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274080+0000","last_peered":"2023-06-09T12:07:04.274080+0000","last_clean":"2023-06-09T12:07:04.274080+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274080+0000","last_undegraded":"2023-06-09T12:07:04.274080+0000","last_fullsized":"2023-06-09T12:07:04.274080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.316048+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:06:22.316048+0000","last_peered":"2023-06-09T12:06:22.316048+0000","last_clean":"2023-06-09T12:06:22.316048+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:06:22.316048+0000","last_undegraded":"2023-06-09T12:06:22.316048+0000","last_fullsized":"2023-06-09T12:06:22.316048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3320,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:53.143889+0000","last_change":"2023-06-09T12:07:27.036778+0000","last_active":"2023-06-09T12:08:53.143889+0000","last_peered":"2023-06-09T12:08:53.143889+0000","last_clean":"2023-06-09T12:08:53.143889+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:08:53.143889+0000","last_undegraded":"2023-06-09T12:08:53.143889+0000","last_fullsized":"2023-06-09T12:08:53.143889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2968,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348318+0000","last_change":"2023-06-09T12:07:55.142523+0000","last_active":"2023-06-09T12:06:52.348318+0000","last_peered":"2023-06-09T12:06:52.348318+0000","last_clean":"2023-06-09T12:06:52.348318+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:06:52.348318+0000","last_undegraded":"2023-06-09T12:06:52.348318+0000","last_fullsized":"2023-06-09T12:06:52.348318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3281,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:52.348357+0000","last_change":"2023-06-09T12:07:47.094943+0000","last_active":"2023-06-09T12:06:52.348357+0000","last_peered":"2023-06-09T12:06:52.348357+0000","last_clean":"2023-06-09T12:06:52.348357+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:06:52.348357+0000","last_undegraded":"2023-06-09T12:06:52.348357+0000","last_fullsized":"2023-06-09T12:06:52.348357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1039,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:02.461482+0000","last_change":"2023-06-09T12:07:44.447705+0000","last_active":"2023-06-09T12:07:02.461482+0000","last_peered":"2023-06-09T12:07:02.461482+0000","last_clean":"2023-06-09T12:07:02.461482+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:02.461482+0000","last_undegraded":"2023-06-09T12:07:02.461482+0000","last_fullsized":"2023-06-09T12:07:02.461482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'939","reported_seq":3100,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.441310+0000","last_change":"2023-06-09T12:04:32.441310+0000","last_active":"2023-06-09T12:04:32.441310+0000","last_peered":"2023-06-09T12:04:32.441310+0000","last_clean":"2023-06-09T12:04:32.441310+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:04:32.441310+0000","last_undegraded":"2023-06-09T12:04:32.441310+0000","last_fullsized":"2023-06-09T12:04:32.441310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":939,"log_dups_size":0,"ondisk_log_size":939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:32.322967+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:04:32.322967+0000","last_peered":"2023-06-09T12:04:32.322967+0000","last_clean":"2023-06-09T12:04:32.322967+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:04:32.322967+0000","last_undegraded":"2023-06-09T12:04:32.322967+0000","last_fullsized":"2023-06-09T12:04:32.322967+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":698,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:18.428660+0000","last_change":"2023-06-09T12:08:18.428660+0000","last_active":"2023-06-09T12:08:18.428660+0000","last_peered":"2023-06-09T12:08:18.428660+0000","last_clean":"2023-06-09T12:08:18.428660+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:08:18.428660+0000","last_undegraded":"2023-06-09T12:08:18.428660+0000","last_fullsized":"2023-06-09T12:08:18.428660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010592+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010592+0000","last_peered":"2023-06-09T12:07:35.010592+0000","last_clean":"2023-06-09T12:07:35.010592+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010592+0000","last_undegraded":"2023-06-09T12:07:35.010592+0000","last_fullsized":"2023-06-09T12:07:35.010592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":700,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.238428+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:07:12.238428+0000","last_peered":"2023-06-09T12:07:12.238428+0000","last_clean":"2023-06-09T12:07:12.238428+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:07:12.238428+0000","last_undegraded":"2023-06-09T12:07:12.238428+0000","last_fullsized":"2023-06-09T12:07:12.238428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329373+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329373+0000","last_peered":"2023-06-09T12:07:57.329373+0000","last_clean":"2023-06-09T12:07:57.329373+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329373+0000","last_undegraded":"2023-06-09T12:07:57.329373+0000","last_fullsized":"2023-06-09T12:07:57.329373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":759,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350677+0000","last_change":"2023-06-09T12:07:54.093801+0000","last_active":"2023-06-09T12:07:12.350677+0000","last_peered":"2023-06-09T12:07:12.350677+0000","last_clean":"2023-06-09T12:07:12.350677+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:12.350677+0000","last_undegraded":"2023-06-09T12:07:12.350677+0000","last_fullsized":"2023-06-09T12:07:12.350677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.322021+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:04:57.322021+0000","last_peered":"2023-06-09T12:04:57.322021+0000","last_clean":"2023-06-09T12:04:57.322021+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:04:57.322021+0000","last_undegraded":"2023-06-09T12:04:57.322021+0000","last_fullsized":"2023-06-09T12:04:57.322021+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.350328+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:04:57.350328+0000","last_peered":"2023-06-09T12:04:57.350328+0000","last_clean":"2023-06-09T12:04:57.350328+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:04:57.350328+0000","last_undegraded":"2023-06-09T12:04:57.350328+0000","last_fullsized":"2023-06-09T12:04:57.350328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3605,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:30.580107+0000","last_change":"2023-06-09T12:07:26.474113+0000","last_active":"2023-06-09T12:07:30.580107+0000","last_peered":"2023-06-09T12:07:30.580107+0000","last_clean":"2023-06-09T12:07:30.580107+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:30.580107+0000","last_undegraded":"2023-06-09T12:07:30.580107+0000","last_fullsized":"2023-06-09T12:07:30.580107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:55.400314+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:05:55.400314+0000","last_peered":"2023-06-09T12:05:55.400314+0000","last_clean":"2023-06-09T12:05:55.400314+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:05:55.400314+0000","last_undegraded":"2023-06-09T12:05:55.400314+0000","last_fullsized":"2023-06-09T12:05:55.400314+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3897,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:29.552824+0000","last_change":"2023-06-09T12:07:25.510423+0000","last_active":"2023-06-09T12:07:29.552824+0000","last_peered":"2023-06-09T12:07:29.552824+0000","last_clean":"2023-06-09T12:07:29.552824+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:29.552824+0000","last_undegraded":"2023-06-09T12:07:29.552824+0000","last_fullsized":"2023-06-09T12:07:29.552824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:57.402555+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:04:57.402555+0000","last_peered":"2023-06-09T12:04:57.402555+0000","last_clean":"2023-06-09T12:04:57.402555+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:04:57.402555+0000","last_undegraded":"2023-06-09T12:04:57.402555+0000","last_fullsized":"2023-06-09T12:04:57.402555+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.434156+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:05:57.434156+0000","last_peered":"2023-06-09T12:05:57.434156+0000","last_clean":"2023-06-09T12:05:57.434156+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:05:57.434156+0000","last_undegraded":"2023-06-09T12:05:57.434156+0000","last_fullsized":"2023-06-09T12:05:57.434156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.553028+0000","last_change":"2023-06-09T12:08:52.455933+0000","last_active":"2023-06-09T12:08:34.553028+0000","last_peered":"2023-06-09T12:08:34.553028+0000","last_clean":"2023-06-09T12:08:34.553028+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:08:34.553028+0000","last_undegraded":"2023-06-09T12:08:34.553028+0000","last_fullsized":"2023-06-09T12:08:34.553028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2993,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.342586+0000","last_change":"2023-06-09T12:08:31.205177+0000","last_active":"2023-06-09T12:07:07.342586+0000","last_peered":"2023-06-09T12:07:07.342586+0000","last_clean":"2023-06-09T12:07:07.342586+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:07:07.342586+0000","last_undegraded":"2023-06-09T12:07:07.342586+0000","last_fullsized":"2023-06-09T12:07:07.342586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'883","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.370378+0000","last_change":"2023-06-09T12:07:12.343255+0000","last_active":"2023-06-09T12:06:17.370378+0000","last_peered":"2023-06-09T12:06:17.370378+0000","last_clean":"2023-06-09T12:06:17.370378+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:06:17.370378+0000","last_undegraded":"2023-06-09T12:06:17.370378+0000","last_fullsized":"2023-06-09T12:06:17.370378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":883,"log_dups_size":0,"ondisk_log_size":883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427674+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:05:02.427674+0000","last_peered":"2023-06-09T12:05:02.427674+0000","last_clean":"2023-06-09T12:05:02.427674+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:05:02.427674+0000","last_undegraded":"2023-06-09T12:05:02.427674+0000","last_fullsized":"2023-06-09T12:05:02.427674+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:59.176571+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:06:59.176571+0000","last_peered":"2023-06-09T12:06:59.176571+0000","last_clean":"2023-06-09T12:06:59.176571+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:06:59.176571+0000","last_undegraded":"2023-06-09T12:06:59.176571+0000","last_fullsized":"2023-06-09T12:06:59.176571+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":526,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350787+0000","last_change":"2023-06-09T12:07:53.051874+0000","last_active":"2023-06-09T12:07:12.350787+0000","last_peered":"2023-06-09T12:07:12.350787+0000","last_clean":"2023-06-09T12:07:12.350787+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:12.350787+0000","last_undegraded":"2023-06-09T12:07:12.350787+0000","last_fullsized":"2023-06-09T12:07:12.350787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'996","reported_seq":3505,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:56.441144+0000","last_change":"2023-06-09T12:08:36.954546+0000","last_active":"2023-06-09T12:08:56.441144+0000","last_peered":"2023-06-09T12:08:56.441144+0000","last_clean":"2023-06-09T12:08:56.441144+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:08:56.441144+0000","last_undegraded":"2023-06-09T12:08:56.441144+0000","last_fullsized":"2023-06-09T12:08:56.441144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":996,"log_dups_size":0,"ondisk_log_size":996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:47.134410+0000","last_change":"2023-06-09T12:07:25.019244+0000","last_active":"2023-06-09T12:08:47.134410+0000","last_peered":"2023-06-09T12:08:47.134410+0000","last_clean":"2023-06-09T12:08:47.134410+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:08:47.134410+0000","last_undegraded":"2023-06-09T12:08:47.134410+0000","last_fullsized":"2023-06-09T12:08:47.134410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":834,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.343238+0000","last_change":"2023-06-09T12:08:34.171989+0000","last_active":"2023-06-09T12:07:12.343238+0000","last_peered":"2023-06-09T12:07:12.343238+0000","last_clean":"2023-06-09T12:07:12.343238+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:07:12.343238+0000","last_undegraded":"2023-06-09T12:07:12.343238+0000","last_fullsized":"2023-06-09T12:07:12.343238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3720,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.462029+0000","last_change":"2023-06-09T12:08:38.366273+0000","last_active":"2023-06-09T12:07:07.462029+0000","last_peered":"2023-06-09T12:07:07.462029+0000","last_clean":"2023-06-09T12:07:07.462029+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:07:07.462029+0000","last_undegraded":"2023-06-09T12:07:07.462029+0000","last_fullsized":"2023-06-09T12:07:07.462029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":777,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.420320+0000","last_change":"2023-06-09T12:08:22.376989+0000","last_active":"2023-06-09T12:08:17.420320+0000","last_peered":"2023-06-09T12:08:17.420320+0000","last_clean":"2023-06-09T12:08:17.420320+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:08:17.420320+0000","last_undegraded":"2023-06-09T12:08:17.420320+0000","last_fullsized":"2023-06-09T12:08:17.420320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3153,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:06:37.346516+0000","last_change":"2023-06-09T12:07:48.112027+0000","last_active":"2023-06-09T12:06:37.346516+0000","last_peered":"2023-06-09T12:06:37.346516+0000","last_clean":"2023-06-09T12:06:37.346516+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:06:37.346516+0000","last_undegraded":"2023-06-09T12:06:37.346516+0000","last_fullsized":"2023-06-09T12:06:37.346516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.363233+0000","last_change":"2023-06-09T12:08:52.476110+0000","last_active":"2023-06-09T12:08:37.363233+0000","last_peered":"2023-06-09T12:08:37.363233+0000","last_clean":"2023-06-09T12:08:37.363233+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:37.363233+0000","last_undegraded":"2023-06-09T12:08:37.363233+0000","last_fullsized":"2023-06-09T12:08:37.363233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3942,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:07.350050+0000","last_change":"2023-06-09T12:07:49.110762+0000","last_active":"2023-06-09T12:07:07.350050+0000","last_peered":"2023-06-09T12:07:07.350050+0000","last_clean":"2023-06-09T12:07:07.350050+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:07.350050+0000","last_undegraded":"2023-06-09T12:07:07.350050+0000","last_fullsized":"2023-06-09T12:07:07.350050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3393,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:53.439204+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:06:53.439204+0000","last_peered":"2023-06-09T12:06:53.439204+0000","last_clean":"2023-06-09T12:06:53.439204+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:06:53.439204+0000","last_undegraded":"2023-06-09T12:06:53.439204+0000","last_fullsized":"2023-06-09T12:06:53.439204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2984,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.351371+0000","last_change":"2023-06-09T12:07:51.130097+0000","last_active":"2023-06-09T12:07:17.351371+0000","last_peered":"2023-06-09T12:07:17.351371+0000","last_clean":"2023-06-09T12:07:17.351371+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:17.351371+0000","last_undegraded":"2023-06-09T12:07:17.351371+0000","last_fullsized":"2023-06-09T12:07:17.351371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":67088,"ondisk_log_size":67088,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1512399,"num_omap_keys":2843,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30863,"ondisk_log_size":30863,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575550,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260688,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80832,"kb_avail":94111152,"statfs":{"total":96636764160,"available":96369819648,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82771968},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.505,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.85099999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.92000000000000004,"5min":1.202,"15min":1.3320000000000001},"last":0.68200000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72699999999999998,"15min":0.70199999999999996},"min":{"1min":0.46700000000000003,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.871,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.72099999999999997},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.90100000000000002,"5min":1.0649999999999999,"15min":1.099},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.88400000000000001,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.72599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.92200000000000004,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":1.006,"15min":1.006},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.96099999999999997,"5min":1.226,"15min":1.226},"last":0.753}]},{"osd":5,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.51800000000000002,"5min":0.45800000000000002,"15min":0.38100000000000001},"max":{"1min":0.78600000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.63400000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575550,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":209304,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94162536,"statfs":{"total":96636764160,"available":96422436864,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.51900000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.876,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.81999999999999995,"5min":0.91400000000000003,"15min":1.069},"last":0.67700000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.56000000000000005,"5min":0.498,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.91900000000000004,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.70499999999999996},"min":{"1min":0.57499999999999996,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.91900000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.93899999999999995,"5min":1.022,"15min":1.0369999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.77700000000000002},"min":{"1min":0.73599999999999999,"5min":0.55900000000000005,"15min":0.52900000000000003},"max":{"1min":0.96899999999999997,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":1.0189999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.437},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.748,"15min":0.67800000000000005},"min":{"1min":0.51900000000000002,"5min":0.498,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.052},"last":0.96599999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":1.1100000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.66100000000000003},"min":{"1min":0.52700000000000002,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":1.0720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575551,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394864,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88448,"kb_avail":93976976,"statfs":{"total":96636764160,"available":96232423424,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":90570752},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.33800000000000002,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.442},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.40300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.32700000000000001},"max":{"1min":0.71599999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.38800000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.03,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.0640000000000001,"5min":1.597,"15min":1.597},"last":0.80900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.54500000000000004},"max":{"1min":1.254,"5min":1.544,"15min":1.544},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.57599999999999996},"max":{"1min":1.1919999999999999,"5min":1.704,"15min":1.704},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84299999999999997,"15min":0.83599999999999997},"min":{"1min":0.63400000000000001,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.1439999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":1.03}]}]},{"osd":0,"up_from":13,"seq":55834575550,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321384,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93568,"kb_avail":94050456,"statfs":{"total":96636764160,"available":96307666944,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95813632},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.44700000000000001},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.65700000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34000000000000002},"max":{"1min":0.64900000000000002,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.61199999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.81899999999999995,"15min":1.032},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.35699999999999998,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.91700000000000004},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":0.79900000000000004,"5min":0.85799999999999998,"15min":1.0109999999999999},"last":1.0229999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84399999999999997,"15min":0.75900000000000001},"min":{"1min":0.68600000000000005,"5min":0.627,"15min":0.36899999999999999},"max":{"1min":1.0149999999999999,"5min":1.204,"15min":1.204},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":0.82599999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.42899999999999999},"max":{"1min":1.0629999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":0.91300000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":1.121,"15min":1.121},"last":1.004},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.91400000000000003,"15min":0.82099999999999995},"min":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.079}]},{"osd":6,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":1.139},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":0.97999999999999998,"5min":1.232,"15min":1.232},"last":0.877}]},{"osd":7,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86599999999999999,"15min":0.81899999999999995},"min":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.47299999999999998},"max":{"1min":1.101,"5min":1.298,"15min":1.298},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95899999999999996,"15min":0.89400000000000002},"min":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.52400000000000002},"max":{"1min":1.143,"5min":1.2589999999999999,"15min":1.351},"last":1.111}]}]},{"osd":2,"up_from":13,"seq":55834575551,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247520,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":67712,"kb_avail":94124320,"statfs":{"total":96636764160,"available":96383303680,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":69337088},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.44800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65800000000000003,"5min":1.236,"15min":1.3109999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65600000000000003,"15min":0.629},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.2969999999999999,"15min":1.367},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.77100000000000002,"5min":1.4490000000000001,"15min":1.546},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.435,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88400000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.496,"15min":0.373},"max":{"1min":1.026,"5min":1.202,"15min":1.8220000000000001},"last":1.079},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.40999999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83199999999999996,"15min":0.82099999999999995},"min":{"1min":0.627,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.999,"5min":1.3540000000000001,"15min":1.601},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.86699999999999999},"min":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0529999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":1.45}]},{"osd":6,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":0.98899999999999999,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":1.034},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.121},"last":0.94399999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.84399999999999997},"min":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.877}]}]},{"osd":3,"up_from":13,"seq":55834575550,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427516,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104320,"kb_avail":93944324,"statfs":{"total":96636764160,"available":96198987776,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106823680},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.223},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.40100000000000002,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.442},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.623}]},{"osd":2,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.377,"15min":0.29799999999999999},"max":{"1min":0.84399999999999997,"5min":0.877,"15min":1.198},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.76200000000000001},"min":{"1min":0.58699999999999997,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.635},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80200000000000005,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.171,"5min":1.171,"15min":3.081},"last":0.98999999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.80200000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0449999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":1.101},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.90300000000000002,"15min":0.84499999999999997},"min":{"1min":0.73399999999999999,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.1000000000000001,"5min":1.165,"15min":1.367},"last":1.143}]},{"osd":6,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.80000000000000004},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.198,"5min":1.198,"15min":1.286},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84099999999999997,"15min":0.80300000000000005},"min":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.434},"max":{"1min":1.117,"5min":1.117,"15min":3.3490000000000002},"last":0.97999999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.51600000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.413},"last":0.872},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.86199999999999999,"15min":0.86699999999999999},"min":{"1min":0.70299999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":1.0449999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575549,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424212,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131456,"kb_avail":93947628,"statfs":{"total":96636764160,"available":96202371072,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134610944},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.54100000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0529999999999999,"5min":1.0600000000000001,"15min":1.137},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.55600000000000005,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.73399999999999999},"min":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.002,"15min":1.0640000000000001},"last":0.377},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.62}]},{"osd":2,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.69699999999999995},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.609},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.72499999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":0.96699999999999997,"5min":1.107,"15min":1.159},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.86299999999999999,"5min":0.999,"15min":1.407},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.48299999999999998},"max":{"1min":0.86099999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.71599999999999997,"5min":0.90400000000000003,"15min":1.0349999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.622},"min":{"1min":0.46600000000000003,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83699999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.53200000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.85699999999999998,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":1.131,"15min":1.3100000000000001},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92300000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.091,"15min":1.361},"last":0.51000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575549,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294232,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76864,"kb_avail":94077608,"statfs":{"total":96636764160,"available":96335470592,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78708736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":0.86299999999999999,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.71299999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.35599999999999998},"max":{"1min":1.167,"5min":1.167,"15min":19.715},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57599999999999996,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.96799999999999997,"5min":1.127,"15min":7.4089999999999998},"last":0.628}]},{"osd":2,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.60999999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.88900000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.46100000000000002},"max":{"1min":1.083,"5min":1.145,"15min":14.439},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73199999999999998,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.5,"15min":0.41899999999999998},"max":{"1min":0.91200000000000003,"5min":1.254,"15min":12.130000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.2310000000000001,"5min":1.274,"15min":12.218999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54700000000000004,"15min":0.80600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.82799999999999996},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.67700000000000005},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":12.163},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.56499999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":1.095,"15min":7.6040000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.749,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.91500000000000004,"5min":1.238,"15min":19.834},"last":0.69699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:09:00.307 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.308 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.308 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.309 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.309 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.309 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.310 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.310 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.310 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.311 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.311 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.311 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.312 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.312 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.313 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.313 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.313 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.314 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.314 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.314 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.315 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.315 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.315 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.316 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.316 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.317 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.317 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.317 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.318 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.318 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.318 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.319 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.319 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.319 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.320 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.320 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.321 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.321 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.321 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.322 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:00.322 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:09:20.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:09:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.532+0000 7fb21cfff700 1 Processor -- start 2023-06-09T12:09:20.534 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.532+0000 7fb21cfff700 1 -- start start 2023-06-09T12:09:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb2181599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2180c0140 0x7fb218159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb2180c1f90 0x7fb2181501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.536 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb218070110 con 0x7fb2180c0140 2023-06-09T12:09:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb218071300 con 0x7fb2180c1f90 2023-06-09T12:09:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.533+0000 7fb21cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180cbd20 con 0x7fb2180c1bc0 2023-06-09T12:09:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2180c0140 0x7fb218159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb2180c0140 0x7fb218159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46602/0 (socket says 172.21.15.7:46602) 2023-06-09T12:09:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 -- 172.21.15.7:0/2458619835 learned_addr learned my addr 172.21.15.7:0/2458619835 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:09:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb216d9d700 1 --2- 172.21.15.7:0/2458619835 >> v2:172.21.15.117:3300/0 conn(0x7fb2180c1f90 0x7fb2181501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb21659c700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb2181599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 -- 172.21.15.7:0/2458619835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 msgr2=0x7fb2181599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:09:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb2181599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 -- 172.21.15.7:0/2458619835 >> v2:172.21.15.117:3300/0 conn(0x7fb2180c1f90 msgr2=0x7fb2181501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:09:20.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 --2- 172.21.15.7:0/2458619835 >> v2:172.21.15.117:3300/0 conn(0x7fb2180c1f90 0x7fb2181501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.539 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 -- 172.21.15.7:0/2458619835 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180c0bf0 con 0x7fb2180c0140 2023-06-09T12:09:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.534+0000 7fb215d9b700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c0140 0x7fb218159ef0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb2180cc250 tx=0x7fb20c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e967c8de93cdad5e server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21559a700 1 -- 172.21.15.7:0/2458619835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c00dd10 con 0x7fb2180c0140 2023-06-09T12:09:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21559a700 1 -- 172.21.15.7:0/2458619835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c00de70 con 0x7fb2180c0140 2023-06-09T12:09:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21559a700 1 -- 172.21.15.7:0/2458619835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c010670 con 0x7fb2180c0140 2023-06-09T12:09:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 -- 172.21.15.7:0/2458619835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c0140 msgr2=0x7fb218159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c0140 0x7fb218159ef0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb2180cc250 tx=0x7fb20c00bea0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 -- 172.21.15.7:0/2458619835 shutdown_connections 2023-06-09T12:09:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 --2- 172.21.15.7:0/2458619835 >> v2:172.21.15.117:3300/0 conn(0x7fb2180c1f90 0x7fb2181501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb2181599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 --2- 172.21.15.7:0/2458619835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c0140 0x7fb218159ef0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 -- 172.21.15.7:0/2458619835 >> 172.21.15.7:0/2458619835 conn(0x7fb21806a4d0 msgr2=0x7fb21806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 -- 172.21.15.7:0/2458619835 shutdown_connections 2023-06-09T12:09:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.535+0000 7fb21cfff700 1 -- 172.21.15.7:0/2458619835 wait complete. 2023-06-09T12:09:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.537+0000 7fb21cfff700 1 Processor -- start 2023-06-09T12:09:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.537+0000 7fb21cfff700 1 -- start start 2023-06-09T12:09:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2180c0140 0x7fb2180af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb218158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180c48c0 con 0x7fb2180c1f90 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2180c5410 con 0x7fb2180c0140 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb21cfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181f1a50 con 0x7fb2180c1bc0 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.538+0000 7fb216d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46604/0 (socket says 172.21.15.7:46604) 2023-06-09T12:09:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 -- 172.21.15.7:0/528698884 learned_addr learned my addr 172.21.15.7:0/528698884 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:09:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb21659c700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2180c0140 0x7fb2180af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb215d9b700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb218158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 -- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 msgr2=0x7fb218158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:20.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb218158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 -- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2180c0140 msgr2=0x7fb2180af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2180c0140 0x7fb2180af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 -- 172.21.15.7:0/528698884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c00b7f0 con 0x7fb2180c1f90 2023-06-09T12:09:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb216d9d700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb210007e90 tx=0x7fb21000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:20.547 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000ed30 con 0x7fb2180c1f90 2023-06-09T12:09:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21000ee90 con 0x7fb2180c1f90 2023-06-09T12:09:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb210014660 con 0x7fb2180c1f90 2023-06-09T12:09:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.539+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2180c0e90 con 0x7fb2180c1f90 2023-06-09T12:09:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.540+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2180702f0 con 0x7fb2180c1f90 2023-06-09T12:09:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.541+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2180c1170 con 0x7fb2180c1f90 2023-06-09T12:09:20.549 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.542+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb2100147c0 con 0x7fb2180c1f90 2023-06-09T12:09:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.542+0000 7fb206ffd700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 0x7fb1f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.543+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb21008a480 con 0x7fb2180c1f90 2023-06-09T12:09:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.543+0000 7fb21659c700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 0x7fb1f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.543+0000 7fb21659c700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 0x7fb1f806d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb208009950 tx=0x7fb208008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:20.550 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.547+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb210054c30 con 0x7fb2180c1f90 2023-06-09T12:09:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.755+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2181f1d10 con 0x7fb1f806ace0 2023-06-09T12:09:20.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.764+0000 7fb206ffd700 1 -- 172.21.15.7:0/528698884 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317327 (secure 0 0 0) 0x7fb2181f1d10 con 0x7fb1f806ace0 2023-06-09T12:09:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:09:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.768+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 msgr2=0x7fb1f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.768+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 0x7fb1f806d190 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fb208009950 tx=0x7fb208008040 comp rx=0 tx=0).stop 2023-06-09T12:09:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.768+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 msgr2=0x7fb2180c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:20.772 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.768+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fb210007e90 tx=0x7fb21000e3c0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 shutdown_connections 2023-06-09T12:09:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb2180c1f90 0x7fb2180c2760 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb2180c0140 0x7fb2180af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb1f806ace0 0x7fb1f806d190 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.773 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 --2- 172.21.15.7:0/528698884 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb2180c1bc0 0x7fb218158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 >> 172.21.15.7:0/528698884 conn(0x7fb21806a4d0 msgr2=0x7fb2180c7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 shutdown_connections 2023-06-09T12:09:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:20.769+0000 7fb21cfff700 1 -- 172.21.15.7:0/528698884 wait complete. 2023-06-09T12:09:20.774 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:09:20.789 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1826,"stamp":"2023-06-09T12:09:19.087179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1512399,"num_omap_keys":2843,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98413,"ondisk_log_size":98413,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2584648,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":701312,"kb_avail":752390072,"statfs":{"total":773094113280,"available":770447433728,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":718143488},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003136"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045709+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045709+0000","last_peered":"2023-06-09T12:08:58.045709+0000","last_clean":"2023-06-09T12:08:58.045709+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045709+0000","last_undegraded":"2023-06-09T12:08:58.045709+0000","last_fullsized":"2023-06-09T12:08:58.045709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462188+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:07:07.462188+0000","last_peered":"2023-06-09T12:07:07.462188+0000","last_clean":"2023-06-09T12:07:07.462188+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:07:07.462188+0000","last_undegraded":"2023-06-09T12:07:07.462188+0000","last_fullsized":"2023-06-09T12:07:07.462188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3701,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:32.523325+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:05:32.523325+0000","last_peered":"2023-06-09T12:05:32.523325+0000","last_clean":"2023-06-09T12:05:32.523325+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:05:32.523325+0000","last_undegraded":"2023-06-09T12:05:32.523325+0000","last_fullsized":"2023-06-09T12:05:32.523325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3493,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:22.345868+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:04:22.345868+0000","last_peered":"2023-06-09T12:04:22.345868+0000","last_clean":"2023-06-09T12:04:22.345868+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:04:22.345868+0000","last_undegraded":"2023-06-09T12:04:22.345868+0000","last_fullsized":"2023-06-09T12:04:22.345868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3532,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.349543+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:04:52.349543+0000","last_peered":"2023-06-09T12:04:52.349543+0000","last_clean":"2023-06-09T12:04:52.349543+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:04:52.349543+0000","last_undegraded":"2023-06-09T12:04:52.349543+0000","last_fullsized":"2023-06-09T12:04:52.349543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:25.145466+0000","last_change":"2023-06-09T12:09:12.373897+0000","last_active":"2023-06-09T12:08:25.145466+0000","last_peered":"2023-06-09T12:08:25.145466+0000","last_clean":"2023-06-09T12:08:25.145466+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:08:25.145466+0000","last_undegraded":"2023-06-09T12:08:25.145466+0000","last_fullsized":"2023-06-09T12:08:25.145466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.460322+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:06:52.460322+0000","last_peered":"2023-06-09T12:06:52.460322+0000","last_clean":"2023-06-09T12:06:52.460322+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:06:52.460322+0000","last_undegraded":"2023-06-09T12:06:52.460322+0000","last_fullsized":"2023-06-09T12:06:52.460322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'908","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.330730+0000","last_change":"2023-06-09T12:07:07.342635+0000","last_active":"2023-06-09T12:06:19.330730+0000","last_peered":"2023-06-09T12:06:19.330730+0000","last_clean":"2023-06-09T12:06:19.330730+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:06:19.330730+0000","last_undegraded":"2023-06-09T12:06:19.330730+0000","last_fullsized":"2023-06-09T12:06:19.330730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3364,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:20.424864+0000","last_change":"2023-06-09T12:07:36.313766+0000","last_active":"2023-06-09T12:08:20.424864+0000","last_peered":"2023-06-09T12:08:20.424864+0000","last_clean":"2023-06-09T12:08:20.424864+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:08:20.424864+0000","last_undegraded":"2023-06-09T12:08:20.424864+0000","last_fullsized":"2023-06-09T12:08:20.424864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.341930+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:07:02.341930+0000","last_peered":"2023-06-09T12:07:02.341930+0000","last_clean":"2023-06-09T12:07:02.341930+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:07:02.341930+0000","last_undegraded":"2023-06-09T12:07:02.341930+0000","last_fullsized":"2023-06-09T12:07:02.341930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2026,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:49.730741+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:04:49.730741+0000","last_peered":"2023-06-09T12:04:49.730741+0000","last_clean":"2023-06-09T12:04:49.730741+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:04:49.730741+0000","last_undegraded":"2023-06-09T12:04:49.730741+0000","last_fullsized":"2023-06-09T12:04:49.730741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.102771+0000","last_change":"2023-06-09T12:09:12.548172+0000","last_active":"2023-06-09T12:08:57.102771+0000","last_peered":"2023-06-09T12:08:57.102771+0000","last_clean":"2023-06-09T12:08:57.102771+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:08:57.102771+0000","last_undegraded":"2023-06-09T12:08:57.102771+0000","last_fullsized":"2023-06-09T12:08:57.102771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1039,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:02.461482+0000","last_change":"2023-06-09T12:07:44.447705+0000","last_active":"2023-06-09T12:07:02.461482+0000","last_peered":"2023-06-09T12:07:02.461482+0000","last_clean":"2023-06-09T12:07:02.461482+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:02.461482+0000","last_undegraded":"2023-06-09T12:07:02.461482+0000","last_fullsized":"2023-06-09T12:07:02.461482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1061","reported_seq":3232,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:15.439078+0000","last_change":"2023-06-09T12:08:58.671029+0000","last_active":"2023-06-09T12:09:15.439078+0000","last_peered":"2023-06-09T12:09:15.439078+0000","last_clean":"2023-06-09T12:09:15.439078+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:15.439078+0000","last_undegraded":"2023-06-09T12:09:15.439078+0000","last_fullsized":"2023-06-09T12:09:15.439078+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'908","reported_seq":3563,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:24.871532+0000","last_change":"2023-06-09T12:07:02.349585+0000","last_active":"2023-06-09T12:06:24.871532+0000","last_peered":"2023-06-09T12:06:24.871532+0000","last_clean":"2023-06-09T12:06:24.871532+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:06:24.871532+0000","last_undegraded":"2023-06-09T12:06:24.871532+0000","last_fullsized":"2023-06-09T12:06:24.871532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.425923+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:04:47.425923+0000","last_peered":"2023-06-09T12:04:47.425923+0000","last_clean":"2023-06-09T12:04:47.425923+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:04:47.425923+0000","last_undegraded":"2023-06-09T12:04:47.425923+0000","last_fullsized":"2023-06-09T12:04:47.425923+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095747+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095747+0000","last_peered":"2023-06-09T12:06:58.095747+0000","last_clean":"2023-06-09T12:06:58.095747+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095747+0000","last_undegraded":"2023-06-09T12:06:58.095747+0000","last_fullsized":"2023-06-09T12:06:58.095747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4047,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:47.333006+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:04:47.333006+0000","last_peered":"2023-06-09T12:04:47.333006+0000","last_clean":"2023-06-09T12:04:47.333006+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:04:47.333006+0000","last_undegraded":"2023-06-09T12:04:47.333006+0000","last_fullsized":"2023-06-09T12:04:47.333006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":698,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:18.428660+0000","last_change":"2023-06-09T12:08:18.428660+0000","last_active":"2023-06-09T12:08:18.428660+0000","last_peered":"2023-06-09T12:08:18.428660+0000","last_clean":"2023-06-09T12:08:18.428660+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:08:18.428660+0000","last_undegraded":"2023-06-09T12:08:18.428660+0000","last_fullsized":"2023-06-09T12:08:18.428660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3655,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.410816+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:06:50.410816+0000","last_peered":"2023-06-09T12:06:50.410816+0000","last_clean":"2023-06-09T12:06:50.410816+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:06:50.410816+0000","last_undegraded":"2023-06-09T12:06:50.410816+0000","last_fullsized":"2023-06-09T12:06:50.410816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":633,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010592+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010592+0000","last_peered":"2023-06-09T12:07:35.010592+0000","last_clean":"2023-06-09T12:07:35.010592+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010592+0000","last_undegraded":"2023-06-09T12:07:35.010592+0000","last_fullsized":"2023-06-09T12:07:35.010592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3785,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428062+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428062+0000","last_peered":"2023-06-09T12:07:23.428062+0000","last_clean":"2023-06-09T12:07:23.428062+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428062+0000","last_undegraded":"2023-06-09T12:07:23.428062+0000","last_fullsized":"2023-06-09T12:07:23.428062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'518","reported_seq":703,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:05.602894+0000","last_change":"2023-06-09T12:07:12.238428+0000","last_active":"2023-06-09T12:09:05.602894+0000","last_peered":"2023-06-09T12:09:05.602894+0000","last_clean":"2023-06-09T12:09:05.602894+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:05.602894+0000","last_undegraded":"2023-06-09T12:09:05.602894+0000","last_fullsized":"2023-06-09T12:09:05.602894+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3843,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:04:52.444292+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:04:52.444292+0000","last_peered":"2023-06-09T12:04:52.444292+0000","last_clean":"2023-06-09T12:04:52.444292+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:04:52.444292+0000","last_undegraded":"2023-06-09T12:04:52.444292+0000","last_fullsized":"2023-06-09T12:04:52.444292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329373+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329373+0000","last_peered":"2023-06-09T12:07:57.329373+0000","last_clean":"2023-06-09T12:07:57.329373+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329373+0000","last_undegraded":"2023-06-09T12:07:57.329373+0000","last_fullsized":"2023-06-09T12:07:57.329373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3456,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:15.228654+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:05:15.228654+0000","last_peered":"2023-06-09T12:05:15.228654+0000","last_clean":"2023-06-09T12:05:15.228654+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:05:15.228654+0000","last_undegraded":"2023-06-09T12:05:15.228654+0000","last_fullsized":"2023-06-09T12:05:15.228654+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":759,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350677+0000","last_change":"2023-06-09T12:07:54.093801+0000","last_active":"2023-06-09T12:07:12.350677+0000","last_peered":"2023-06-09T12:07:12.350677+0000","last_clean":"2023-06-09T12:07:12.350677+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:12.350677+0000","last_undegraded":"2023-06-09T12:07:12.350677+0000","last_fullsized":"2023-06-09T12:07:12.350677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3215,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:27.473750+0000","last_change":"2023-06-09T12:07:24.516964+0000","last_active":"2023-06-09T12:07:27.473750+0000","last_peered":"2023-06-09T12:07:27.473750+0000","last_clean":"2023-06-09T12:07:27.473750+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:27.473750+0000","last_undegraded":"2023-06-09T12:07:27.473750+0000","last_fullsized":"2023-06-09T12:07:27.473750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:38.526282+0000","last_change":"2023-06-09T12:09:12.355679+0000","last_active":"2023-06-09T12:07:38.526282+0000","last_peered":"2023-06-09T12:07:38.526282+0000","last_clean":"2023-06-09T12:07:38.526282+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:38.526282+0000","last_undegraded":"2023-06-09T12:07:38.526282+0000","last_fullsized":"2023-06-09T12:07:38.526282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3898,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:34.557829+0000","last_change":"2023-06-09T12:09:12.355730+0000","last_active":"2023-06-09T12:07:34.557829+0000","last_peered":"2023-06-09T12:07:34.557829+0000","last_clean":"2023-06-09T12:07:34.557829+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:34.557829+0000","last_undegraded":"2023-06-09T12:07:34.557829+0000","last_fullsized":"2023-06-09T12:07:34.557829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.434156+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:05:57.434156+0000","last_peered":"2023-06-09T12:05:57.434156+0000","last_clean":"2023-06-09T12:05:57.434156+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:05:57.434156+0000","last_undegraded":"2023-06-09T12:05:57.434156+0000","last_fullsized":"2023-06-09T12:05:57.434156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.553028+0000","last_change":"2023-06-09T12:08:52.455933+0000","last_active":"2023-06-09T12:08:34.553028+0000","last_peered":"2023-06-09T12:08:34.553028+0000","last_clean":"2023-06-09T12:08:34.553028+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:08:34.553028+0000","last_undegraded":"2023-06-09T12:08:34.553028+0000","last_fullsized":"2023-06-09T12:08:34.553028+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1006","reported_seq":3304,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:15.838700+0000","last_change":"2023-06-09T12:08:59.070369+0000","last_active":"2023-06-09T12:09:15.838700+0000","last_peered":"2023-06-09T12:09:15.838700+0000","last_clean":"2023-06-09T12:09:15.838700+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:15.838700+0000","last_undegraded":"2023-06-09T12:09:15.838700+0000","last_fullsized":"2023-06-09T12:09:15.838700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":123,"log_size":1006,"log_dups_size":0,"ondisk_log_size":1006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":850,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.325690+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:05:27.325690+0000","last_peered":"2023-06-09T12:05:27.325690+0000","last_clean":"2023-06-09T12:05:27.325690+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:05:27.325690+0000","last_undegraded":"2023-06-09T12:05:27.325690+0000","last_fullsized":"2023-06-09T12:05:27.325690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.342702+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:07:07.342702+0000","last_peered":"2023-06-09T12:07:07.342702+0000","last_clean":"2023-06-09T12:07:07.342702+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:07:07.342702+0000","last_undegraded":"2023-06-09T12:07:07.342702+0000","last_fullsized":"2023-06-09T12:07:07.342702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":526,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.350787+0000","last_change":"2023-06-09T12:07:53.051874+0000","last_active":"2023-06-09T12:07:12.350787+0000","last_peered":"2023-06-09T12:07:12.350787+0000","last_clean":"2023-06-09T12:07:12.350787+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:12.350787+0000","last_undegraded":"2023-06-09T12:07:12.350787+0000","last_fullsized":"2023-06-09T12:07:12.350787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1094","reported_seq":3608,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:14.416644+0000","last_change":"2023-06-09T12:08:36.954546+0000","last_active":"2023-06-09T12:09:14.416644+0000","last_peered":"2023-06-09T12:09:14.416644+0000","last_clean":"2023-06-09T12:09:14.416644+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:14.416644+0000","last_undegraded":"2023-06-09T12:09:14.416644+0000","last_fullsized":"2023-06-09T12:09:14.416644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":220,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.163476+0000","last_change":"2023-06-09T12:09:17.548842+0000","last_active":"2023-06-09T12:08:55.163476+0000","last_peered":"2023-06-09T12:08:55.163476+0000","last_clean":"2023-06-09T12:08:55.163476+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:08:55.163476+0000","last_undegraded":"2023-06-09T12:08:55.163476+0000","last_fullsized":"2023-06-09T12:08:55.163476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":834,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:12.343238+0000","last_change":"2023-06-09T12:08:34.171989+0000","last_active":"2023-06-09T12:07:12.343238+0000","last_peered":"2023-06-09T12:07:12.343238+0000","last_clean":"2023-06-09T12:07:12.343238+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:07:12.343238+0000","last_undegraded":"2023-06-09T12:07:12.343238+0000","last_fullsized":"2023-06-09T12:07:12.343238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3721,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:38.366273+0000","last_change":"2023-06-09T12:09:12.478817+0000","last_active":"2023-06-09T12:08:38.366273+0000","last_peered":"2023-06-09T12:08:38.366273+0000","last_clean":"2023-06-09T12:08:38.366273+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:08:38.366273+0000","last_undegraded":"2023-06-09T12:08:38.366273+0000","last_fullsized":"2023-06-09T12:08:38.366273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":780,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:07.572553+0000","last_change":"2023-06-09T12:09:02.540941+0000","last_active":"2023-06-09T12:09:07.572553+0000","last_peered":"2023-06-09T12:09:07.572553+0000","last_clean":"2023-06-09T12:09:07.572553+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:09:07.572553+0000","last_undegraded":"2023-06-09T12:09:07.572553+0000","last_fullsized":"2023-06-09T12:09:07.572553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.363233+0000","last_change":"2023-06-09T12:08:52.476110+0000","last_active":"2023-06-09T12:08:37.363233+0000","last_peered":"2023-06-09T12:08:37.363233+0000","last_clean":"2023-06-09T12:08:37.363233+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:37.363233+0000","last_undegraded":"2023-06-09T12:08:37.363233+0000","last_fullsized":"2023-06-09T12:08:37.363233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3394,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462072+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:07:07.462072+0000","last_peered":"2023-06-09T12:07:07.462072+0000","last_clean":"2023-06-09T12:07:07.462072+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:07:07.462072+0000","last_undegraded":"2023-06-09T12:07:07.462072+0000","last_fullsized":"2023-06-09T12:07:07.462072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2984,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:07:17.351371+0000","last_change":"2023-06-09T12:07:51.130097+0000","last_active":"2023-06-09T12:07:17.351371+0000","last_peered":"2023-06-09T12:07:17.351371+0000","last_clean":"2023-06-09T12:07:17.351371+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:17.351371+0000","last_undegraded":"2023-06-09T12:07:17.351371+0000","last_fullsized":"2023-06-09T12:07:17.351371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":67431,"ondisk_log_size":67431,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1512399,"num_omap_keys":2843,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30863,"ondisk_log_size":30863,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575554,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260816,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":80960,"kb_avail":94111024,"statfs":{"total":96636764160,"available":96369688576,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82903040},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.505,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.85099999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.92000000000000004,"5min":1.202,"15min":1.3320000000000001},"last":0.69399999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72699999999999998,"15min":0.70199999999999996},"min":{"1min":0.46700000000000003,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.871,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.72099999999999997},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.66100000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.90100000000000002,"5min":1.0649999999999999,"15min":1.099},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.88400000000000001,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.92200000000000004,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":1.006,"15min":1.006},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.96099999999999997,"5min":1.226,"15min":1.226},"last":0.77200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.44},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.51800000000000002,"5min":0.45800000000000002,"15min":0.38100000000000001},"max":{"1min":0.78600000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.747}]}]},{"osd":6,"up_from":13,"seq":55834575554,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213528,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57408,"kb_avail":94158312,"statfs":{"total":96636764160,"available":96418111488,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":58785792},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.51900000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.876,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.81999999999999995,"5min":0.91400000000000003,"15min":1.069},"last":0.83699999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.56000000000000005,"5min":0.498,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.91900000000000004,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.76100000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.70499999999999996},"min":{"1min":0.57499999999999996,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.91900000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.93899999999999995,"5min":1.022,"15min":1.0369999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.77700000000000002},"min":{"1min":0.73599999999999999,"5min":0.55900000000000005,"15min":0.52900000000000003},"max":{"1min":0.96899999999999997,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.94299999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.98599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.748,"15min":0.67800000000000005},"min":{"1min":0.51900000000000002,"5min":0.498,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.052},"last":1.024}]},{"osd":7,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":1.1100000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.66100000000000003},"min":{"1min":0.52700000000000002,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":1.0609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575555,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394992,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88576,"kb_avail":93976848,"statfs":{"total":96636764160,"available":96232292352,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":90701824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.33800000000000002,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.40300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.32700000000000001},"max":{"1min":0.71599999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.95299999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.03,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.0640000000000001,"5min":1.597,"15min":1.597},"last":0.84199999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.54500000000000004},"max":{"1min":1.254,"5min":1.544,"15min":1.544},"last":0.753},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.57599999999999996},"max":{"1min":1.1919999999999999,"5min":1.704,"15min":1.704},"last":0.98399999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84299999999999997,"15min":0.83599999999999997},"min":{"1min":0.63400000000000001,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.86399999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.1439999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.82699999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575554,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321512,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93696,"kb_avail":94050328,"statfs":{"total":96636764160,"available":96307535872,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":95944704},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.44700000000000001},"min":{"1min":0.33600000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.65700000000000003,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.34000000000000002},"max":{"1min":0.64900000000000002,"5min":0.72899999999999998,"15min":0.82699999999999996},"last":0.749}]},{"osd":2,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56399999999999995},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.70899999999999996,"5min":0.81899999999999995,"15min":1.032},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55200000000000005,"15min":0.51800000000000002},"min":{"1min":0.35699999999999998,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.68100000000000005,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41399999999999998,"5min":0.38400000000000001,"15min":0.33000000000000002},"max":{"1min":0.71499999999999997,"5min":0.78200000000000003,"15min":0.91700000000000004},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68899999999999995,"15min":0.69399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":0.79900000000000004,"5min":0.85799999999999998,"15min":1.0109999999999999},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.84399999999999997,"15min":0.75900000000000001},"min":{"1min":0.68600000000000005,"5min":0.627,"15min":0.36899999999999999},"max":{"1min":1.0149999999999999,"5min":1.204,"15min":1.204},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.88900000000000001,"15min":0.82599999999999996},"min":{"1min":0.625,"5min":0.625,"15min":0.42899999999999999},"max":{"1min":1.0629999999999999,"5min":1.1970000000000001,"15min":1.2190000000000001},"last":1.0129999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.76800000000000002},"min":{"1min":0.52500000000000002,"5min":0.495,"15min":0.39900000000000002},"max":{"1min":0.93500000000000005,"5min":1.121,"15min":1.121},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.97699999999999998,"5min":0.91400000000000003,"15min":0.82099999999999995},"min":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.81299999999999994,"15min":0.81699999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.48099999999999998},"max":{"1min":0.94799999999999995,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.84999999999999998,"15min":0.83799999999999997},"min":{"1min":0.60299999999999998,"5min":0.55700000000000005,"15min":0.52300000000000002},"max":{"1min":0.97999999999999998,"5min":1.232,"15min":1.232},"last":0.83199999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:08:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.86599999999999999,"15min":0.81899999999999995},"min":{"1min":0.65200000000000002,"5min":0.59299999999999997,"15min":0.47299999999999998},"max":{"1min":1.101,"5min":1.298,"15min":1.298},"last":0.873},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.95899999999999996,"15min":0.89400000000000002},"min":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.52400000000000002},"max":{"1min":1.143,"5min":1.2589999999999999,"15min":1.351},"last":1.048}]}]},{"osd":2,"up_from":13,"seq":55834575555,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247520,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":67712,"kb_avail":94124320,"statfs":{"total":96636764160,"available":96383303680,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":69337088},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.44800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65800000000000003,"5min":1.236,"15min":1.3109999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65600000000000003,"15min":0.629},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.2969999999999999,"15min":1.367},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.77100000000000002,"5min":1.4490000000000001,"15min":1.546},"last":0.626},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.435,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88400000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.66700000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.496,"15min":0.373},"max":{"1min":1.026,"5min":1.202,"15min":1.8220000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.40999999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.78100000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83199999999999996,"15min":0.82099999999999995},"min":{"1min":0.627,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.999,"5min":1.3540000000000001,"15min":1.601},"last":0.749},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.86699999999999999},"min":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0529999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":0.98899999999999999,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.121},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.84399999999999997},"min":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.83699999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575554,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427644,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104448,"kb_avail":93944196,"statfs":{"total":96636764160,"available":96198856704,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106954752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.223},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.40100000000000002,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.67500000000000004,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.498},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52100000000000002},"min":{"1min":0.442,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.86099999999999999,"5min":0.97099999999999997,"15min":1.355},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.57899999999999996,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.377,"15min":0.29799999999999999},"max":{"1min":0.84399999999999997,"5min":0.877,"15min":1.198},"last":0.52400000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80900000000000005,"15min":0.76200000000000001},"min":{"1min":0.58699999999999997,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.635},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80200000000000005,"15min":0.77100000000000002},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.171,"5min":1.171,"15min":3.081},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.80200000000000005},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0449999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.90300000000000002,"15min":0.84499999999999997},"min":{"1min":0.73399999999999999,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.1000000000000001,"5min":1.165,"15min":1.367},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.82699999999999996,"15min":0.80000000000000004},"min":{"1min":0.60699999999999998,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.198,"5min":1.198,"15min":1.286},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.84099999999999997,"15min":0.80300000000000005},"min":{"1min":0.64500000000000002,"5min":0.61399999999999999,"15min":0.434},"max":{"1min":1.117,"5min":1.117,"15min":3.3490000000000002},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:08:24 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.83999999999999997,"15min":0.82399999999999995},"min":{"1min":0.68200000000000005,"5min":0.625,"15min":0.51600000000000001},"max":{"1min":1.286,"5min":1.286,"15min":1.413},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.86199999999999999,"15min":0.86699999999999999},"min":{"1min":0.70299999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.84599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575553,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424340,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131584,"kb_avail":93947500,"statfs":{"total":96636764160,"available":96202240000,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134742016},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.54100000000000004,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0529999999999999,"5min":1.0600000000000001,"15min":1.137},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.76200000000000001},"min":{"1min":0.55600000000000005,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.0980000000000001,"5min":1.1080000000000001,"15min":1.1899999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.75600000000000001,"15min":0.73399999999999999},"min":{"1min":0.45300000000000001,"5min":0.45000000000000001,"15min":0.41199999999999998},"max":{"1min":0.88700000000000001,"5min":1.002,"15min":1.0640000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.59199999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.69699999999999995},"min":{"1min":0.52200000000000002,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.93600000000000005,"5min":1.0149999999999999,"15min":1.609},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.79600000000000004,"15min":0.72499999999999998},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":0.96699999999999997,"5min":1.107,"15min":1.159},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.86299999999999999,"5min":0.999,"15min":1.407},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.629,"5min":0.53600000000000003,"15min":0.48299999999999998},"max":{"1min":0.86099999999999999,"5min":1.0780000000000001,"15min":1.4430000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.54000000000000004,"15min":0.53900000000000003},"min":{"1min":0.40699999999999997,"5min":0.377,"15min":0.30299999999999999},"max":{"1min":0.71599999999999997,"5min":0.90400000000000003,"15min":1.0349999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.622},"min":{"1min":0.46600000000000003,"5min":0.44400000000000001,"15min":0.38400000000000001},"max":{"1min":0.83699999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.629}]},{"osd":6,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.40799999999999997,"15min":0.33700000000000002},"max":{"1min":0.85699999999999998,"5min":1.0509999999999999,"15min":1.2490000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73999999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.95299999999999996,"5min":1.131,"15min":1.3100000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:08:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.67400000000000004,"15min":0.625},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92300000000000004,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.61599999999999999,"15min":0.64500000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97399999999999998,"5min":1.091,"15min":1.361},"last":0.52600000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575553,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":294296,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":76928,"kb_avail":94077544,"statfs":{"total":96636764160,"available":96335405056,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":78774272},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":0.86299999999999999,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.35599999999999998},"max":{"1min":1.167,"5min":1.167,"15min":19.715},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57599999999999996,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.96799999999999997,"5min":1.127,"15min":7.4089999999999998},"last":0.64200000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.60999999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.88900000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.46100000000000002},"max":{"1min":1.083,"5min":1.145,"15min":14.439},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73199999999999998,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.5,"15min":0.41899999999999998},"max":{"1min":0.91200000000000003,"5min":1.254,"15min":12.130000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.2310000000000001,"5min":1.274,"15min":12.218999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54700000000000004,"15min":0.80600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.82799999999999996},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.46100000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.67700000000000005},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":12.163},"last":0.503},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.56499999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":1.095,"15min":7.6040000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.749,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.91500000000000004,"5min":1.238,"15min":19.834},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:09:20.797 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.797 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.797 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.798 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.798 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.799 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.799 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.799 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.800 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.800 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.800 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.801 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.801 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.802 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.802 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.802 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.803 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.803 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.803 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.804 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.804 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.804 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.805 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.805 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.806 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.806 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.806 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.807 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.807 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.808 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.808 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.808 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.809 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.809 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.809 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.810 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.810 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.811 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.811 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:20.811 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:09:28.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:28.412 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:40.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:09:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.021+0000 7fb3b83ff700 1 Processor -- start 2023-06-09T12:09:41.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.021+0000 7fb3b83ff700 1 -- start start 2023-06-09T12:09:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.022+0000 7fb3b83ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fb3b015a0c0 0x7fb3b01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:41.025 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.022+0000 7fb3b83ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00c0bf0 con 0x7fb3b00c0140 2023-06-09T12:09:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.022+0000 7fb3b83ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b00702f0 con 0x7fb3b00c1bc0 2023-06-09T12:09:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.022+0000 7fb3b83ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b0070110 con 0x7fb3b015a0c0 2023-06-09T12:09:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b699c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb3b00c0140 0x7fb3b0160ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41382/0 (socket says 172.21.15.7:41382) 2023-06-09T12:09:41.026 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b699c700 1 -- 172.21.15.7:0/1947021720 learned_addr learned my addr 172.21.15.7:0/1947021720 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:09:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b599a700 1 --2- 172.21.15.7:0/1947021720 >> v2:172.21.15.117:3300/0 conn(0x7fb3b015a0c0 0x7fb3b01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443692247 0 0) 0x7fb3b00c0bf0 con 0x7fb3b00c0140 2023-06-09T12:09:41.028 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb39c0035b0 con 0x7fb3b00c0140 2023-06-09T12:09:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615854030 0 0) 0x7fb3b00702f0 con 0x7fb3b00c1bc0 2023-06-09T12:09:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3b00c0bf0 con 0x7fb3b00c1bc0 2023-06-09T12:09:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b599a700 1 -- 172.21.15.7:0/1947021720 >> v1:172.21.15.117:6789/0 conn(0x7fb3b00c1bc0 legacy=0x7fb3b01599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b599a700 1 -- 172.21.15.7:0/1947021720 >> v1:172.21.15.7:6789/0 conn(0x7fb3b00c0140 legacy=0x7fb3b0160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.023+0000 7fb3b599a700 1 -- 172.21.15.7:0/1947021720 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00c0e90 con 0x7fb3b015a0c0 2023-06-09T12:09:41.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b599a700 1 --2- 172.21.15.7:0/1947021720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01501b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009840 tx=0x7fb3a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b31f07ceab29f2 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:41.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0011830 con 0x7fb3b015a0c0 2023-06-09T12:09:41.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a0011990 con 0x7fb3b015a0c0 2023-06-09T12:09:41.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b5199700 1 -- 172.21.15.7:0/1947021720 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0011d20 con 0x7fb3b015a0c0 2023-06-09T12:09:41.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1947021720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b015a0c0 msgr2=0x7fb3b01501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.024+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1947021720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01501b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0009840 tx=0x7fb3a0009b50 comp rx=0 tx=0).stop 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.025+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1947021720 shutdown_connections 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.025+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1947021720 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01501b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.025+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1947021720 >> 172.21.15.7:0/1947021720 conn(0x7fb3b006a4d0 msgr2=0x7fb3b006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.025+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1947021720 shutdown_connections 2023-06-09T12:09:41.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.025+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1947021720 wait complete. 2023-06-09T12:09:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.026+0000 7fb3b83ff700 1 Processor -- start 2023-06-09T12:09:41.033 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.027+0000 7fb3b83ff700 1 -- start start 2023-06-09T12:09:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.027+0000 7fb3b83ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b00c0140 0x7fb3b014ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b83ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3b00c1bc0 0x7fb3b00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b83ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:41.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b83ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01576f0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b83ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b014f820 con 0x7fb3b00c0140 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b83ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b014f260 con 0x7fb3b00c1bc0 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49282/0 (socket says 172.21.15.7:49282) 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 -- 172.21.15.7:0/1133218192 learned_addr learned my addr 172.21.15.7:0/1133218192 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:09:41.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b619b700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b00c0140 0x7fb3b014ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b599a700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3b00c1bc0 0x7fb3b00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 -- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3b00c1bc0 msgr2=0x7fb3b00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3b00c1bc0 0x7fb3b00cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 -- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b00c0140 msgr2=0x7fb3b014ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b00c0140 0x7fb3b014ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.028+0000 7fb3b699c700 1 -- 172.21.15.7:0/1133218192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00094a0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3b699c700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800cb40 tx=0x7fb3a800ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800a9e0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a800ab40 con 0x7fb3b015a0c0 2023-06-09T12:09:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a8012400 con 0x7fb3b015a0c0 2023-06-09T12:09:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b00c1450 con 0x7fb3b015a0c0 2023-06-09T12:09:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.029+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b00c1170 con 0x7fb3b015a0c0 2023-06-09T12:09:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.030+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b00705d0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.031+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3a801b3b0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.032+0000 7fb3a6ffd700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:09:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.033+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb3a8089ae0 con 0x7fb3b015a0c0 2023-06-09T12:09:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.033+0000 7fb3b619b700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:09:41.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.033+0000 7fb3b619b700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 0x7fb39406d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0011e0 tx=0x7fb3ac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:09:41.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.037+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb3a8054210 con 0x7fb3b015a0c0 2023-06-09T12:09:41.246 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.244+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3b00c7450 con 0x7fb39406ace0 2023-06-09T12:09:41.256 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.253+0000 7fb3a6ffd700 1 -- 172.21.15.7:0/1133218192 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317387 (secure 0 0 0) 0x7fb3b00c7450 con 0x7fb39406ace0 2023-06-09T12:09:41.259 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:09:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 msgr2=0x7fb39406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.261 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 0x7fb39406d190 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac0011e0 tx=0x7fb3ac009040 comp rx=0 tx=0).stop 2023-06-09T12:09:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 msgr2=0x7fb3b01f2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:09:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fb3a800cb40 tx=0x7fb3a800ce50 comp rx=0 tx=0).stop 2023-06-09T12:09:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 shutdown_connections 2023-06-09T12:09:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3b015a0c0 0x7fb3b01f2320 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.262 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb3b00c0140 0x7fb3b014ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb39406ace0 0x7fb39406d190 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 --2- 172.21.15.7:0/1133218192 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb3b00c1bc0 0x7fb3b00cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:09:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 >> 172.21.15.7:0/1133218192 conn(0x7fb3b006a4d0 msgr2=0x7fb3b014d110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:09:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 shutdown_connections 2023-06-09T12:09:41.263 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:09:41.257+0000 7fb3b83ff700 1 -- 172.21.15.7:0/1133218192 wait complete. 2023-06-09T12:09:41.264 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:09:41.278 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1837,"stamp":"2023-06-09T12:09:41.093542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98813,"ondisk_log_size":98813,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589640,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":706304,"kb_avail":752385080,"statfs":{"total":773094113280,"available":770442321920,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":723255296},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045709+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045709+0000","last_peered":"2023-06-09T12:08:58.045709+0000","last_clean":"2023-06-09T12:08:58.045709+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045709+0000","last_undegraded":"2023-06-09T12:08:58.045709+0000","last_fullsized":"2023-06-09T12:08:58.045709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462188+0000","last_change":"2023-06-09T12:05:02.445731+0000","last_active":"2023-06-09T12:07:07.462188+0000","last_peered":"2023-06-09T12:07:07.462188+0000","last_clean":"2023-06-09T12:07:07.462188+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:07:07.462188+0000","last_undegraded":"2023-06-09T12:07:07.462188+0000","last_fullsized":"2023-06-09T12:07:07.462188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.064577+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:05:17.064577+0000","last_peered":"2023-06-09T12:05:17.064577+0000","last_clean":"2023-06-09T12:05:17.064577+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:05:17.064577+0000","last_undegraded":"2023-06-09T12:05:17.064577+0000","last_fullsized":"2023-06-09T12:05:17.064577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.445591+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:05:02.445591+0000","last_peered":"2023-06-09T12:05:02.445591+0000","last_clean":"2023-06-09T12:05:02.445591+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:05:02.445591+0000","last_undegraded":"2023-06-09T12:05:02.445591+0000","last_fullsized":"2023-06-09T12:05:02.445591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:25.145466+0000","last_change":"2023-06-09T12:09:12.373897+0000","last_active":"2023-06-09T12:08:25.145466+0000","last_peered":"2023-06-09T12:08:25.145466+0000","last_clean":"2023-06-09T12:08:25.145466+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:08:25.145466+0000","last_undegraded":"2023-06-09T12:08:25.145466+0000","last_fullsized":"2023-06-09T12:08:25.145466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'821","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.460322+0000","last_change":"2023-06-09T12:06:52.460322+0000","last_active":"2023-06-09T12:06:52.460322+0000","last_peered":"2023-06-09T12:06:52.460322+0000","last_clean":"2023-06-09T12:06:52.460322+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:06:52.460322+0000","last_undegraded":"2023-06-09T12:06:52.460322+0000","last_fullsized":"2023-06-09T12:06:52.460322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":821,"log_dups_size":0,"ondisk_log_size":821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'957","reported_seq":3543,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:36.915945+0000","last_change":"2023-06-09T12:09:32.057017+0000","last_active":"2023-06-09T12:09:36.915945+0000","last_peered":"2023-06-09T12:09:36.915945+0000","last_clean":"2023-06-09T12:09:36.915945+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:09:36.915945+0000","last_undegraded":"2023-06-09T12:09:36.915945+0000","last_fullsized":"2023-06-09T12:09:36.915945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":49,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3367,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:27.459762+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:09:27.459762+0000","last_peered":"2023-06-09T12:09:27.459762+0000","last_clean":"2023-06-09T12:09:27.459762+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:09:27.459762+0000","last_undegraded":"2023-06-09T12:09:27.459762+0000","last_fullsized":"2023-06-09T12:09:27.459762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:22.325091+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:05:22.325091+0000","last_peered":"2023-06-09T12:05:22.325091+0000","last_clean":"2023-06-09T12:05:22.325091+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:05:22.325091+0000","last_undegraded":"2023-06-09T12:05:22.325091+0000","last_fullsized":"2023-06-09T12:05:22.325091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.341930+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:07:02.341930+0000","last_peered":"2023-06-09T12:07:02.341930+0000","last_clean":"2023-06-09T12:07:02.341930+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:07:02.341930+0000","last_undegraded":"2023-06-09T12:07:02.341930+0000","last_fullsized":"2023-06-09T12:07:02.341930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.102771+0000","last_change":"2023-06-09T12:09:12.548172+0000","last_active":"2023-06-09T12:08:57.102771+0000","last_peered":"2023-06-09T12:08:57.102771+0000","last_clean":"2023-06-09T12:08:57.102771+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:08:57.102771+0000","last_undegraded":"2023-06-09T12:08:57.102771+0000","last_fullsized":"2023-06-09T12:08:57.102771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:02.427662+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:05:02.427662+0000","last_peered":"2023-06-09T12:05:02.427662+0000","last_clean":"2023-06-09T12:05:02.427662+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:05:02.427662+0000","last_undegraded":"2023-06-09T12:05:02.427662+0000","last_fullsized":"2023-06-09T12:05:02.427662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1040,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:44.447705+0000","last_change":"2023-06-09T12:09:22.480454+0000","last_active":"2023-06-09T12:07:44.447705+0000","last_peered":"2023-06-09T12:07:44.447705+0000","last_clean":"2023-06-09T12:07:44.447705+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:44.447705+0000","last_undegraded":"2023-06-09T12:07:44.447705+0000","last_fullsized":"2023-06-09T12:07:44.447705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1209","reported_seq":3387,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:36.284877+0000","last_change":"2023-06-09T12:08:58.671029+0000","last_active":"2023-06-09T12:09:36.284877+0000","last_peered":"2023-06-09T12:09:36.284877+0000","last_clean":"2023-06-09T12:09:36.284877+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:36.284877+0000","last_undegraded":"2023-06-09T12:09:36.284877+0000","last_fullsized":"2023-06-09T12:09:36.284877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":270,"log_size":1209,"log_dups_size":0,"ondisk_log_size":1209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'956","reported_seq":3618,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:34.653199+0000","last_change":"2023-06-09T12:09:29.763495+0000","last_active":"2023-06-09T12:09:34.653199+0000","last_peered":"2023-06-09T12:09:34.653199+0000","last_clean":"2023-06-09T12:09:34.653199+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:09:34.653199+0000","last_undegraded":"2023-06-09T12:09:34.653199+0000","last_fullsized":"2023-06-09T12:09:34.653199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":48,"log_size":956,"log_dups_size":0,"ondisk_log_size":956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1339,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:17.521777+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:05:17.521777+0000","last_peered":"2023-06-09T12:05:17.521777+0000","last_clean":"2023-06-09T12:05:17.521777+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:05:17.521777+0000","last_undegraded":"2023-06-09T12:05:17.521777+0000","last_fullsized":"2023-06-09T12:05:17.521777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.355388+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:05:37.355388+0000","last_peered":"2023-06-09T12:05:37.355388+0000","last_clean":"2023-06-09T12:05:37.355388+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:05:37.355388+0000","last_undegraded":"2023-06-09T12:05:37.355388+0000","last_fullsized":"2023-06-09T12:05:37.355388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":701,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:23.466959+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:09:23.466959+0000","last_peered":"2023-06-09T12:09:23.466959+0000","last_clean":"2023-06-09T12:09:23.466959+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:09:23.466959+0000","last_undegraded":"2023-06-09T12:09:23.466959+0000","last_fullsized":"2023-06-09T12:09:23.466959+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3656,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.462853+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:07:12.462853+0000","last_peered":"2023-06-09T12:07:12.462853+0000","last_clean":"2023-06-09T12:07:12.462853+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:07:12.462853+0000","last_undegraded":"2023-06-09T12:07:12.462853+0000","last_fullsized":"2023-06-09T12:07:12.462853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3844,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462158+0000","last_change":"2023-06-09T12:04:52.444292+0000","last_active":"2023-06-09T12:07:07.462158+0000","last_peered":"2023-06-09T12:07:07.462158+0000","last_clean":"2023-06-09T12:07:07.462158+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:07:07.462158+0000","last_undegraded":"2023-06-09T12:07:07.462158+0000","last_fullsized":"2023-06-09T12:07:07.462158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329373+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329373+0000","last_peered":"2023-06-09T12:07:57.329373+0000","last_clean":"2023-06-09T12:07:57.329373+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329373+0000","last_undegraded":"2023-06-09T12:07:57.329373+0000","last_fullsized":"2023-06-09T12:07:57.329373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3216,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:31.609236+0000","last_change":"2023-06-09T12:09:32.359589+0000","last_active":"2023-06-09T12:07:31.609236+0000","last_peered":"2023-06-09T12:07:31.609236+0000","last_clean":"2023-06-09T12:07:31.609236+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:31.609236+0000","last_undegraded":"2023-06-09T12:07:31.609236+0000","last_fullsized":"2023-06-09T12:07:31.609236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:38.526282+0000","last_change":"2023-06-09T12:09:12.355679+0000","last_active":"2023-06-09T12:07:38.526282+0000","last_peered":"2023-06-09T12:07:38.526282+0000","last_clean":"2023-06-09T12:07:38.526282+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:38.526282+0000","last_undegraded":"2023-06-09T12:07:38.526282+0000","last_fullsized":"2023-06-09T12:07:38.526282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3898,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:34.557829+0000","last_change":"2023-06-09T12:09:12.355730+0000","last_active":"2023-06-09T12:07:34.557829+0000","last_peered":"2023-06-09T12:07:34.557829+0000","last_clean":"2023-06-09T12:07:34.557829+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:34.557829+0000","last_undegraded":"2023-06-09T12:07:34.557829+0000","last_fullsized":"2023-06-09T12:07:34.557829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1289,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.331118+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:05:52.331118+0000","last_peered":"2023-06-09T12:05:52.331118+0000","last_clean":"2023-06-09T12:05:52.331118+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:05:52.331118+0000","last_undegraded":"2023-06-09T12:05:52.331118+0000","last_fullsized":"2023-06-09T12:05:52.331118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":319,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.434156+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:05:57.434156+0000","last_peered":"2023-06-09T12:05:57.434156+0000","last_clean":"2023-06-09T12:05:57.434156+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:05:57.434156+0000","last_undegraded":"2023-06-09T12:05:57.434156+0000","last_fullsized":"2023-06-09T12:05:57.434156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3590,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:54.746857+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:05:54.746857+0000","last_peered":"2023-06-09T12:05:54.746857+0000","last_clean":"2023-06-09T12:05:54.746857+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:05:54.746857+0000","last_undegraded":"2023-06-09T12:05:54.746857+0000","last_fullsized":"2023-06-09T12:05:54.746857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.455933+0000","last_change":"2023-06-09T12:09:27.528163+0000","last_active":"2023-06-09T12:08:52.455933+0000","last_peered":"2023-06-09T12:08:52.455933+0000","last_clean":"2023-06-09T12:08:52.455933+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:08:52.455933+0000","last_undegraded":"2023-06-09T12:08:52.455933+0000","last_fullsized":"2023-06-09T12:08:52.455933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.342702+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:07:07.342702+0000","last_peered":"2023-06-09T12:07:07.342702+0000","last_clean":"2023-06-09T12:07:07.342702+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:07:07.342702+0000","last_undegraded":"2023-06-09T12:07:07.342702+0000","last_fullsized":"2023-06-09T12:07:07.342702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870217+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870217+0000","last_peered":"2023-06-09T12:09:21.870217+0000","last_clean":"2023-06-09T12:09:21.870217+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870217+0000","last_undegraded":"2023-06-09T12:09:21.870217+0000","last_fullsized":"2023-06-09T12:09:21.870217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.163476+0000","last_change":"2023-06-09T12:09:17.548842+0000","last_active":"2023-06-09T12:08:55.163476+0000","last_peered":"2023-06-09T12:08:55.163476+0000","last_clean":"2023-06-09T12:08:55.163476+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:08:55.163476+0000","last_undegraded":"2023-06-09T12:08:55.163476+0000","last_fullsized":"2023-06-09T12:08:55.163476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":835,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.171989+0000","last_change":"2023-06-09T12:09:22.375312+0000","last_active":"2023-06-09T12:08:34.171989+0000","last_peered":"2023-06-09T12:08:34.171989+0000","last_clean":"2023-06-09T12:08:34.171989+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:08:34.171989+0000","last_undegraded":"2023-06-09T12:08:34.171989+0000","last_fullsized":"2023-06-09T12:08:34.171989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3721,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:38.366273+0000","last_change":"2023-06-09T12:09:12.478817+0000","last_active":"2023-06-09T12:08:38.366273+0000","last_peered":"2023-06-09T12:08:38.366273+0000","last_clean":"2023-06-09T12:08:38.366273+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:08:38.366273+0000","last_undegraded":"2023-06-09T12:08:38.366273+0000","last_fullsized":"2023-06-09T12:08:38.366273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":782,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:18.500382+0000","last_change":"2023-06-09T12:09:02.540941+0000","last_active":"2023-06-09T12:09:18.500382+0000","last_peered":"2023-06-09T12:09:18.500382+0000","last_clean":"2023-06-09T12:09:18.500382+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:09:18.500382+0000","last_undegraded":"2023-06-09T12:09:18.500382+0000","last_fullsized":"2023-06-09T12:09:18.500382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.363233+0000","last_change":"2023-06-09T12:08:52.476110+0000","last_active":"2023-06-09T12:08:37.363233+0000","last_peered":"2023-06-09T12:08:37.363233+0000","last_clean":"2023-06-09T12:08:37.363233+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:37.363233+0000","last_undegraded":"2023-06-09T12:08:37.363233+0000","last_fullsized":"2023-06-09T12:08:37.363233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3394,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462072+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:07:07.462072+0000","last_peered":"2023-06-09T12:07:07.462072+0000","last_clean":"2023-06-09T12:07:07.462072+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:07:07.462072+0000","last_undegraded":"2023-06-09T12:07:07.462072+0000","last_fullsized":"2023-06-09T12:07:07.462072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":67817,"ondisk_log_size":67817,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30877,"ondisk_log_size":30877,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575558,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260880,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":81024,"kb_avail":94110960,"statfs":{"total":96636764160,"available":96369623040,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82968576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68000000000000005,"15min":0.67900000000000005},"min":{"1min":0.505,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.85099999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70299999999999996},"min":{"1min":0.47999999999999998,"5min":0.378,"15min":0.36199999999999999},"max":{"1min":0.92000000000000004,"5min":1.202,"15min":1.3320000000000001},"last":0.70799999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72699999999999998,"15min":0.70199999999999996},"min":{"1min":0.46700000000000003,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.871,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.875},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.72099999999999997},"min":{"1min":0.61399999999999999,"5min":0.53700000000000003,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.99099999999999999,"15min":1.05},"last":0.81299999999999994}]},{"osd":2,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.72999999999999998},"min":{"1min":0.53500000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.90100000000000002,"5min":1.0649999999999999,"15min":1.099},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73999999999999999,"15min":0.73499999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.88400000000000001,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.751}]},{"osd":3,"last update":"Fri Jun 9 12:08:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.51500000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.92200000000000004,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.57199999999999995,"5min":0.505,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":0.97099999999999997,"15min":1.4870000000000001},"last":0.78100000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.54200000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.89200000000000002,"5min":1.006,"15min":1.006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72999999999999998,"15min":0.70199999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.96099999999999997,"5min":1.226,"15min":1.226},"last":0.67000000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.41199999999999998,"5min":0.378,"15min":0.34999999999999998},"max":{"1min":0.80500000000000005,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.625},"min":{"1min":0.44400000000000001,"5min":0.42699999999999999,"15min":0.34399999999999997},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Jun 9 12:08:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.65600000000000003},"min":{"1min":0.51800000000000002,"5min":0.45800000000000002,"15min":0.38100000000000001},"max":{"1min":0.78600000000000003,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.501,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.626}]}]},{"osd":6,"up_from":13,"seq":55834575558,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213720,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57600,"kb_avail":94158120,"statfs":{"total":96636764160,"available":96417914880,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":58982400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.51900000000000002,"5min":0.33100000000000002,"15min":0.28299999999999997},"max":{"1min":0.876,"5min":0.99199999999999999,"15min":1.0780000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68400000000000005,"15min":0.66800000000000004},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.30599999999999999},"max":{"1min":0.81999999999999995,"5min":0.91400000000000003,"15min":1.069},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.71499999999999997,"15min":0.69099999999999995},"min":{"1min":0.56000000000000005,"5min":0.498,"15min":0.312},"max":{"1min":0.88600000000000001,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.74399999999999999},"min":{"1min":0.61199999999999999,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.91900000000000004,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73399999999999999,"15min":0.70499999999999996},"min":{"1min":0.57499999999999996,"5min":0.42699999999999999,"15min":0.36799999999999999},"max":{"1min":0.91900000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.625},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.61399999999999999,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.95799999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74399999999999999,"15min":0.72499999999999998},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.93899999999999995,"5min":1.022,"15min":1.0369999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.79900000000000004,"15min":0.77700000000000002},"min":{"1min":0.73599999999999999,"5min":0.55900000000000005,"15min":0.52900000000000003},"max":{"1min":0.96899999999999997,"5min":0.97699999999999998,"15min":1.0580000000000001},"last":0.81399999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.45000000000000001,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.40100000000000002,"15min":0.34000000000000002},"max":{"1min":0.83799999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.90000000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.60899999999999999,"15min":0.66600000000000004},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84699999999999998,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.748,"15min":0.67800000000000005},"min":{"1min":0.51900000000000002,"5min":0.498,"15min":0.33800000000000002},"max":{"1min":0.997,"5min":0.997,"15min":1.052},"last":0.84899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:08:50 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.69899999999999995,"15min":0.69999999999999996},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.42099999999999999},"max":{"1min":0.83399999999999996,"5min":0.99299999999999999,"15min":1.1100000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68100000000000005,"15min":0.66100000000000003},"min":{"1min":0.52700000000000002,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":0.89800000000000002,"5min":0.98699999999999999,"15min":1.071},"last":0.873}]}]},{"osd":1,"up_from":13,"seq":55834575559,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395120,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88704,"kb_avail":93976720,"statfs":{"total":96636764160,"available":96232161280,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":90832896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.46899999999999997,"15min":0.43099999999999999},"min":{"1min":0.33800000000000002,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.40300000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.50900000000000001,"15min":0.48199999999999998},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.32700000000000001},"max":{"1min":0.71599999999999997,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.57999999999999996,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.70299999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.03,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.878,"15min":0.84299999999999997},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.0640000000000001,"5min":1.597,"15min":1.597},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:08:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.54500000000000004},"max":{"1min":1.254,"5min":1.544,"15min":1.544},"last":0.873},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.872,"15min":0.85499999999999998},"min":{"1min":0.623,"5min":0.59899999999999998,"15min":0.57599999999999996},"max":{"1min":1.1919999999999999,"5min":1.704,"15min":1.704},"last":0.84199999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84299999999999997,"15min":0.83599999999999997},"min":{"1min":0.63400000000000001,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.83499999999999996,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:08:47 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.88800000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.1439999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.84799999999999998,"15min":0.80000000000000004},"min":{"1min":0.63800000000000001,"5min":0.52100000000000002,"15min":0.47299999999999998},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.85399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575558,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321704,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":93888,"kb_avail":94050136,"statfs":{"total":96636764160,"available":96307339264,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":96141312},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.33100000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.82699999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":1.032},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.62}]},{"osd":4,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84599999999999997,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1539999999999999,"5min":1.204,"15min":1.204},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89000000000000001,"15min":0.83399999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.2190000000000001},"last":0.88600000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79800000000000004,"15min":0.77000000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.004,"5min":1.121,"15min":1.121},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.82999999999999996},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.145,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81999999999999995,"15min":0.81699999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.139,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.1739999999999999,"5min":1.232,"15min":1.232},"last":0.85799999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.175,"5min":1.177,"15min":1.298},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.95499999999999996,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.2589999999999999,"15min":1.351},"last":0.77400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575559,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":247520,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":67712,"kb_avail":94124320,"statfs":{"total":96636764160,"available":96383303680,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":69337088},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.47499999999999998,"15min":0.44800000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.27100000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56599999999999995,"15min":0.53800000000000003},"min":{"1min":0.438,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.78600000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.58399999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48099999999999998,"15min":0.46999999999999997},"min":{"1min":0.34699999999999998,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65800000000000003,"5min":1.236,"15min":1.3109999999999999},"last":0.497},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65600000000000003,"15min":0.629},"min":{"1min":0.46700000000000003,"5min":0.45400000000000001,"15min":0.38800000000000001},"max":{"1min":0.80400000000000005,"5min":1.2969999999999999,"15min":1.367},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.77100000000000002,"5min":1.4490000000000001,"15min":1.546},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.68200000000000005,"15min":0.66300000000000003},"min":{"1min":0.435,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88400000000000001,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78300000000000003,"15min":0.76800000000000002},"min":{"1min":0.61699999999999999,"5min":0.496,"15min":0.373},"max":{"1min":1.026,"5min":1.202,"15min":1.8220000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80400000000000005,"15min":0.78400000000000003},"min":{"1min":0.59299999999999997,"5min":0.56000000000000005,"15min":0.40999999999999998},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.73399999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.83199999999999996,"15min":0.82099999999999995},"min":{"1min":0.627,"5min":0.61299999999999999,"15min":0.51000000000000001},"max":{"1min":0.999,"5min":1.3540000000000001,"15min":1.601},"last":0.753},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.88400000000000001,"15min":0.86699999999999999},"min":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.53400000000000003},"max":{"1min":1.0529999999999999,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.82699999999999996,"15min":0.79900000000000004},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.438},"max":{"1min":0.98899999999999999,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79300000000000004,"15min":0.77800000000000002},"min":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.121},"last":0.86199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:08:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.84399999999999997,"15min":0.81799999999999995},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.86399999999999999,"15min":0.84399999999999997},"min":{"1min":0.65700000000000003,"5min":0.64500000000000002,"15min":0.52800000000000002},"max":{"1min":1.0600000000000001,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.82799999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575558,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427708,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104512,"kb_avail":93944132,"statfs":{"total":96636764160,"available":96198791168,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":107020288},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.223},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.437,"5min":0.38100000000000001,"15min":0.33600000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.68300000000000005,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.61399999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.37,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.81200000000000006,"5min":0.877,"15min":1.198},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.635},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.77900000000000003},"min":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.151,"5min":1.171,"15min":3.081},"last":0.96399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.80900000000000005},"min":{"1min":0.61499999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.101,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.89600000000000002,"15min":0.85299999999999998},"min":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.143,"5min":1.165,"15min":1.367},"last":0.92900000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0820000000000001,"5min":1.198,"15min":1.286},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83599999999999997,"15min":0.81000000000000005},"min":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.434},"max":{"1min":0.99199999999999999,"5min":1.117,"15min":3.3490000000000002},"last":0.73099999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.117,"5min":1.286,"15min":1.413},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.875,"5min":0.86199999999999999,"15min":0.86899999999999999},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1060000000000001,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.90700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575557,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424532,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":131776,"kb_avail":93947308,"statfs":{"total":96636764160,"available":96202043392,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":134938624},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.70699999999999996},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0529999999999999,"15min":1.137},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.64500000000000002,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.97899999999999998,"5min":1.0980000000000001,"15min":1.1899999999999999},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88300000000000001,"5min":0.95199999999999996,"15min":1.0640000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.57999999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83699999999999997,"5min":1.0149999999999999,"15min":1.609},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.73499999999999999},"min":{"1min":0.60299999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.88500000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66700000000000004,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.78900000000000003,"5min":0.95899999999999996,"15min":1.407},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.4430000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.61599999999999999,"5min":0.90400000000000003,"15min":1.0189999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.83899999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.72499999999999998},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0489999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.63200000000000001},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.89900000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.64500000000000002},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76100000000000001,"5min":0.97399999999999998,"15min":1.361},"last":0.46500000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575557,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298456,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81088,"kb_avail":94073384,"statfs":{"total":96636764160,"available":96331145216,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83034112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71299999999999997},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.33800000000000002},"max":{"1min":0.86299999999999999,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.74399999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.74299999999999999,"15min":0.76900000000000002},"min":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.35599999999999998},"max":{"1min":1.167,"5min":1.167,"15min":19.715},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.748,"15min":0.749},"min":{"1min":0.57599999999999996,"5min":0.45700000000000002,"15min":0.42599999999999999},"max":{"1min":0.96799999999999997,"5min":1.127,"15min":7.4089999999999998},"last":0.82299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72099999999999997,"15min":0.71399999999999997},"min":{"1min":0.60999999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.88900000000000001,"5min":1.016,"15min":5.5620000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.77900000000000003},"min":{"1min":0.60199999999999998,"5min":0.54500000000000004,"15min":0.46100000000000002},"max":{"1min":1.083,"5min":1.145,"15min":14.439},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73199999999999998,"15min":0.77100000000000002},"min":{"1min":0.55000000000000004,"5min":0.5,"15min":0.41899999999999998},"max":{"1min":0.91200000000000003,"5min":1.254,"15min":12.130000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.621,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":1.2310000000000001,"5min":1.274,"15min":12.218999999999999},"last":0.85799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.54700000000000004,"15min":0.80600000000000005},"min":{"1min":0.39800000000000002,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83899999999999997,"15min":67.590000000000003},"last":0.63},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.57999999999999996,"15min":0.82799999999999996},"min":{"1min":0.442,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.73099999999999998,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.621,"15min":0.67700000000000005},"min":{"1min":0.44500000000000001,"5min":0.42499999999999999,"15min":0.40400000000000003},"max":{"1min":0.92800000000000005,"5min":1.097,"15min":12.163},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.56499999999999995,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.92600000000000005,"5min":1.095,"15min":7.6040000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:08:41 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.66800000000000004},"min":{"1min":0.45000000000000001,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.90700000000000003,"5min":0.95599999999999996,"15min":3.8039999999999998},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.749,"15min":0.82399999999999995},"min":{"1min":0.52300000000000002,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.91500000000000004,"5min":1.238,"15min":19.834},"last":0.88300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:09:41.285 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.286 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.286 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.287 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.287 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.287 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.288 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.288 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.289 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.289 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.289 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.290 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.290 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.290 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.291 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.291 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.292 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.292 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.292 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.293 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.293 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.293 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.294 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.294 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.295 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.295 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.295 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.296 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.296 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.296 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.297 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.297 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.297 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.298 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.298 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.298 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:09:41.299 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:09:58.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:09:58.529 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:01.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:10:01.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.512+0000 7f56553ff700 1 Processor -- start 2023-06-09T12:10:01.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.512+0000 7f56553ff700 1 -- start start 2023-06-09T12:10:01.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.513+0000 7f56553ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 0x7f565015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.513+0000 7f56553ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 0x7f565015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.513+0000 7f56553ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f565015a500 con 0x7f565015b2d0 2023-06-09T12:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.513+0000 7f56553ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56500c5f10 con 0x7f56500c9f40 2023-06-09T12:10:01.517 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.513+0000 7f56553ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f565015da00 con 0x7f5650153880 2023-06-09T12:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 0x7f565015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564f7fe700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7f565015b2d0 0x7f56500c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:46444/0 (socket says 172.21.15.7:46444) 2023-06-09T12:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564f7fe700 1 -- 172.21.15.7:0/948138065 learned_addr learned my addr 172.21.15.7:0/948138065 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:10:01.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564e7fc700 1 --2- 172.21.15.7:0/948138065 >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 0x7f565015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564dffb700 1 -- 172.21.15.7:0/948138065 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614109460 0 0) 0x7f5648001490 con 0x7f565015b2d0 2023-06-09T12:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564effd700 1 -- 172.21.15.7:0/948138065 >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 msgr2=0x7f565015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564effd700 1 --2- 172.21.15.7:0/948138065 >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 0x7f565015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.514+0000 7f564effd700 1 -- 172.21.15.7:0/948138065 >> v1:172.21.15.117:6789/0 conn(0x7f565015b2d0 legacy=0x7f56500c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564effd700 1 -- 172.21.15.7:0/948138065 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f565015b750 con 0x7f5650153880 2023-06-09T12:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564e7fc700 1 --2- 172.21.15.7:0/948138065 >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 0x7f565015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564effd700 1 --2- 172.21.15.7:0/948138065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 0x7f565015d4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5640009840 tx=0x7f5640009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e360a9fe30f52e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564dffb700 1 -- 172.21.15.7:0/948138065 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011830 con 0x7f5650153880 2023-06-09T12:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564dffb700 1 -- 172.21.15.7:0/948138065 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640011990 con 0x7f5650153880 2023-06-09T12:10:01.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.515+0000 7f564dffb700 1 -- 172.21.15.7:0/948138065 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640011d20 con 0x7f5650153880 2023-06-09T12:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 -- 172.21.15.7:0/948138065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 msgr2=0x7f565015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 --2- 172.21.15.7:0/948138065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 0x7f565015d4c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5640009840 tx=0x7f5640009b50 comp rx=0 tx=0).stop 2023-06-09T12:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 -- 172.21.15.7:0/948138065 shutdown_connections 2023-06-09T12:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 --2- 172.21.15.7:0/948138065 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5650153880 0x7f565015d4c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 --2- 172.21.15.7:0/948138065 >> v2:172.21.15.117:3300/0 conn(0x7f56500c9f40 0x7f565015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 -- 172.21.15.7:0/948138065 >> 172.21.15.7:0/948138065 conn(0x7f56500737f0 msgr2=0x7f56500720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 -- 172.21.15.7:0/948138065 shutdown_connections 2023-06-09T12:10:01.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.516+0000 7f56553ff700 1 -- 172.21.15.7:0/948138065 wait complete. 2023-06-09T12:10:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.518+0000 7f56553ff700 1 Processor -- start 2023-06-09T12:10:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.518+0000 7f56553ff700 1 -- start start 2023-06-09T12:10:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.518+0000 7f56553ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f56553ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5650153880 0x7f5650155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f56553ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015b2d0 0x7f56500b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f56553ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5650154030 con 0x7f56500c9f40 2023-06-09T12:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f56553ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56501ef340 con 0x7f5650153880 2023-06-09T12:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f56553ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56501ef620 con 0x7f565015b2d0 2023-06-09T12:10:01.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:56542/0 (socket says 172.21.15.7:56542) 2023-06-09T12:10:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 -- 172.21.15.7:0/1097849155 learned_addr learned my addr 172.21.15.7:0/1097849155 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:10:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564f7fe700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015b2d0 0x7f56500b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564e7fc700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5650153880 0x7f5650155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.527 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 -- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015b2d0 msgr2=0x7f56500b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015b2d0 0x7f56500b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 -- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5650153880 msgr2=0x7f5650155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5650153880 0x7f5650155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.528 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.519+0000 7f564effd700 1 -- 172.21.15.7:0/1097849155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56440094a0 con 0x7f56500c9f40 2023-06-09T12:10:01.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f564effd700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5640009fd0 tx=0x7f5640004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:01.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400036a0 con 0x7f56500c9f40 2023-06-09T12:10:01.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f564001f070 con 0x7f56500c9f40 2023-06-09T12:10:01.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640003a20 con 0x7f56500c9f40 2023-06-09T12:10:01.529 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56400094a0 con 0x7f56500c9f40 2023-06-09T12:10:01.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.520+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5650159980 con 0x7f56500c9f40 2023-06-09T12:10:01.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.521+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56501596e0 con 0x7f56500c9f40 2023-06-09T12:10:01.530 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.522+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5640003b80 con 0x7f56500c9f40 2023-06-09T12:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.523+0000 7f56377fe700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 0x7f563c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.523+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f5640021070 con 0x7f56500c9f40 2023-06-09T12:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.523+0000 7f564e7fc700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 0x7f563c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.524+0000 7f564e7fc700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 0x7f563c06d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f56500cf8b0 tx=0x7f5644008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:01.531 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.528+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5640055ed0 con 0x7f56500c9f40 2023-06-09T12:10:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.731+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56501ef4b0 con 0x7f563c06ace0 2023-06-09T12:10:01.743 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.740+0000 7f56377fe700 1 -- 172.21.15.7:0/1097849155 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317597 (secure 0 0 0) 0x7f56501ef4b0 con 0x7f563c06ace0 2023-06-09T12:10:01.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:10:01.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 msgr2=0x7f563c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 0x7f563c06d190 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f56500cf8b0 tx=0x7f5644008040 comp rx=0 tx=0).stop 2023-06-09T12:10:01.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 msgr2=0x7f565014d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:01.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f5640009fd0 tx=0x7f5640004750 comp rx=0 tx=0).stop 2023-06-09T12:10:01.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 shutdown_connections 2023-06-09T12:10:01.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f563c06ace0 0x7f563c06d190 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f565015b2d0 0x7f56500b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56500c9f40 0x7f565014d2b0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 --2- 172.21.15.7:0/1097849155 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5650153880 0x7f5650155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 >> 172.21.15.7:0/1097849155 conn(0x7f56500737f0 msgr2=0x7f56500cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 shutdown_connections 2023-06-09T12:10:01.750 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:01.745+0000 7f56553ff700 1 -- 172.21.15.7:0/1097849155 wait complete. 2023-06-09T12:10:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:10:01.766 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1847,"stamp":"2023-06-09T12:10:01.099064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99112,"ondisk_log_size":99112,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2577992,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":694656,"kb_avail":752396728,"statfs":{"total":773094113280,"available":770454249472,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":711327744},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003390"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:15.215648+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:06:15.215648+0000","last_peered":"2023-06-09T12:06:15.215648+0000","last_clean":"2023-06-09T12:06:15.215648+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:06:15.215648+0000","last_undegraded":"2023-06-09T12:06:15.215648+0000","last_fullsized":"2023-06-09T12:06:15.215648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045709+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045709+0000","last_peered":"2023-06-09T12:08:58.045709+0000","last_clean":"2023-06-09T12:08:58.045709+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045709+0000","last_undegraded":"2023-06-09T12:08:58.045709+0000","last_fullsized":"2023-06-09T12:08:58.045709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3262,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478915+0000","last_change":"2023-06-09T12:09:44.639071+0000","last_active":"2023-06-09T12:09:12.478915+0000","last_peered":"2023-06-09T12:09:12.478915+0000","last_clean":"2023-06-09T12:09:12.478915+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:09:12.478915+0000","last_undegraded":"2023-06-09T12:09:12.478915+0000","last_fullsized":"2023-06-09T12:09:12.478915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807577+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807577+0000","last_peered":"2023-06-09T12:07:40.807577+0000","last_clean":"2023-06-09T12:07:40.807577+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807577+0000","last_undegraded":"2023-06-09T12:07:40.807577+0000","last_fullsized":"2023-06-09T12:07:40.807577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327350+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327350+0000","last_peered":"2023-06-09T12:07:43.327350+0000","last_clean":"2023-06-09T12:07:43.327350+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327350+0000","last_undegraded":"2023-06-09T12:07:43.327350+0000","last_fullsized":"2023-06-09T12:07:43.327350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3523,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:29.365345+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:05:29.365345+0000","last_peered":"2023-06-09T12:05:29.365345+0000","last_clean":"2023-06-09T12:05:29.365345+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:05:29.365345+0000","last_undegraded":"2023-06-09T12:05:29.365345+0000","last_fullsized":"2023-06-09T12:05:29.365345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:25.145466+0000","last_change":"2023-06-09T12:09:12.373897+0000","last_active":"2023-06-09T12:08:25.145466+0000","last_peered":"2023-06-09T12:08:25.145466+0000","last_clean":"2023-06-09T12:08:25.145466+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:08:25.145466+0000","last_undegraded":"2023-06-09T12:08:25.145466+0000","last_fullsized":"2023-06-09T12:08:25.145466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'893","reported_seq":3574,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:53.434016+0000","last_change":"2023-06-09T12:09:45.596505+0000","last_active":"2023-06-09T12:09:53.434016+0000","last_peered":"2023-06-09T12:09:53.434016+0000","last_clean":"2023-06-09T12:09:53.434016+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:09:53.434016+0000","last_undegraded":"2023-06-09T12:09:53.434016+0000","last_fullsized":"2023-06-09T12:09:53.434016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":72,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1057","reported_seq":3648,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:54.781963+0000","last_change":"2023-06-09T12:09:32.057017+0000","last_active":"2023-06-09T12:09:54.781963+0000","last_peered":"2023-06-09T12:09:54.781963+0000","last_clean":"2023-06-09T12:09:54.781963+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:09:54.781963+0000","last_undegraded":"2023-06-09T12:09:54.781963+0000","last_fullsized":"2023-06-09T12:09:54.781963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":149,"log_size":1057,"log_dups_size":0,"ondisk_log_size":1057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3370,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:48.541964+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:09:48.541964+0000","last_peered":"2023-06-09T12:09:48.541964+0000","last_clean":"2023-06-09T12:09:48.541964+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:09:48.541964+0000","last_undegraded":"2023-06-09T12:09:48.541964+0000","last_fullsized":"2023-06-09T12:09:48.541964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'927","reported_seq":3142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.341930+0000","last_change":"2023-06-09T12:07:02.341930+0000","last_active":"2023-06-09T12:07:02.341930+0000","last_peered":"2023-06-09T12:07:02.341930+0000","last_clean":"2023-06-09T12:07:02.341930+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:07:02.341930+0000","last_undegraded":"2023-06-09T12:07:02.341930+0000","last_fullsized":"2023-06-09T12:07:02.341930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":927,"log_dups_size":0,"ondisk_log_size":927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3325,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:47.064796+0000","last_change":"2023-06-09T12:09:44.139026+0000","last_active":"2023-06-09T12:09:47.064796+0000","last_peered":"2023-06-09T12:09:47.064796+0000","last_clean":"2023-06-09T12:09:47.064796+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:09:47.064796+0000","last_undegraded":"2023-06-09T12:09:47.064796+0000","last_fullsized":"2023-06-09T12:09:47.064796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1566,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:27.354007+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:05:27.354007+0000","last_peered":"2023-06-09T12:05:27.354007+0000","last_clean":"2023-06-09T12:05:27.354007+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:05:27.354007+0000","last_undegraded":"2023-06-09T12:05:27.354007+0000","last_fullsized":"2023-06-09T12:05:27.354007+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1040,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:44.447705+0000","last_change":"2023-06-09T12:09:22.480454+0000","last_active":"2023-06-09T12:07:44.447705+0000","last_peered":"2023-06-09T12:07:44.447705+0000","last_clean":"2023-06-09T12:07:44.447705+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:44.447705+0000","last_undegraded":"2023-06-09T12:07:44.447705+0000","last_fullsized":"2023-06-09T12:07:44.447705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1080","reported_seq":3748,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:55.611422+0000","last_change":"2023-06-09T12:09:29.763495+0000","last_active":"2023-06-09T12:09:55.611422+0000","last_peered":"2023-06-09T12:09:55.611422+0000","last_clean":"2023-06-09T12:09:55.611422+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:09:55.611422+0000","last_undegraded":"2023-06-09T12:09:55.611422+0000","last_fullsized":"2023-06-09T12:09:55.611422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":172,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":704,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:46.555103+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:09:46.555103+0000","last_peered":"2023-06-09T12:09:46.555103+0000","last_clean":"2023-06-09T12:09:46.555103+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:09:46.555103+0000","last_undegraded":"2023-06-09T12:09:46.555103+0000","last_fullsized":"2023-06-09T12:09:46.555103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3656,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.462853+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:07:12.462853+0000","last_peered":"2023-06-09T12:07:12.462853+0000","last_clean":"2023-06-09T12:07:12.462853+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:07:12.462853+0000","last_undegraded":"2023-06-09T12:07:12.462853+0000","last_fullsized":"2023-06-09T12:07:12.462853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3845,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480430+0000","last_change":"2023-06-09T12:09:42.640933+0000","last_active":"2023-06-09T12:09:22.480430+0000","last_peered":"2023-06-09T12:09:22.480430+0000","last_clean":"2023-06-09T12:09:22.480430+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:09:22.480430+0000","last_undegraded":"2023-06-09T12:09:22.480430+0000","last_fullsized":"2023-06-09T12:09:22.480430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3216,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:31.609236+0000","last_change":"2023-06-09T12:09:32.359589+0000","last_active":"2023-06-09T12:07:31.609236+0000","last_peered":"2023-06-09T12:07:31.609236+0000","last_clean":"2023-06-09T12:07:31.609236+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:07:31.609236+0000","last_undegraded":"2023-06-09T12:07:31.609236+0000","last_fullsized":"2023-06-09T12:07:31.609236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3606,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:38.526282+0000","last_change":"2023-06-09T12:09:12.355679+0000","last_active":"2023-06-09T12:07:38.526282+0000","last_peered":"2023-06-09T12:07:38.526282+0000","last_clean":"2023-06-09T12:07:38.526282+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:07:38.526282+0000","last_undegraded":"2023-06-09T12:07:38.526282+0000","last_fullsized":"2023-06-09T12:07:38.526282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":926,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.454258+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:06:07.454258+0000","last_peered":"2023-06-09T12:06:07.454258+0000","last_clean":"2023-06-09T12:06:07.454258+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:06:07.454258+0000","last_undegraded":"2023-06-09T12:06:07.454258+0000","last_fullsized":"2023-06-09T12:06:07.454258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3898,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:34.557829+0000","last_change":"2023-06-09T12:09:12.355730+0000","last_active":"2023-06-09T12:07:34.557829+0000","last_peered":"2023-06-09T12:07:34.557829+0000","last_clean":"2023-06-09T12:07:34.557829+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:07:34.557829+0000","last_undegraded":"2023-06-09T12:07:34.557829+0000","last_fullsized":"2023-06-09T12:07:34.557829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:49.493713+0000","last_change":"2023-06-09T12:09:27.528163+0000","last_active":"2023-06-09T12:09:49.493713+0000","last_peered":"2023-06-09T12:09:49.493713+0000","last_clean":"2023-06-09T12:09:49.493713+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:09:49.493713+0000","last_undegraded":"2023-06-09T12:09:49.493713+0000","last_fullsized":"2023-06-09T12:09:49.493713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.342702+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:07:07.342702+0000","last_peered":"2023-06-09T12:07:07.342702+0000","last_clean":"2023-06-09T12:07:07.342702+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:07:07.342702+0000","last_undegraded":"2023-06-09T12:07:07.342702+0000","last_fullsized":"2023-06-09T12:07:07.342702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870217+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870217+0000","last_peered":"2023-06-09T12:09:21.870217+0000","last_clean":"2023-06-09T12:09:21.870217+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870217+0000","last_undegraded":"2023-06-09T12:09:21.870217+0000","last_fullsized":"2023-06-09T12:09:21.870217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":835,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.171989+0000","last_change":"2023-06-09T12:09:22.375312+0000","last_active":"2023-06-09T12:08:34.171989+0000","last_peered":"2023-06-09T12:08:34.171989+0000","last_clean":"2023-06-09T12:08:34.171989+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:08:34.171989+0000","last_undegraded":"2023-06-09T12:08:34.171989+0000","last_fullsized":"2023-06-09T12:08:34.171989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3721,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:38.366273+0000","last_change":"2023-06-09T12:09:12.478817+0000","last_active":"2023-06-09T12:08:38.366273+0000","last_peered":"2023-06-09T12:08:38.366273+0000","last_clean":"2023-06-09T12:08:38.366273+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:08:38.366273+0000","last_undegraded":"2023-06-09T12:08:38.366273+0000","last_fullsized":"2023-06-09T12:08:38.366273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":782,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:18.500382+0000","last_change":"2023-06-09T12:09:02.540941+0000","last_active":"2023-06-09T12:09:18.500382+0000","last_peered":"2023-06-09T12:09:18.500382+0000","last_clean":"2023-06-09T12:09:18.500382+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:09:18.500382+0000","last_undegraded":"2023-06-09T12:09:18.500382+0000","last_fullsized":"2023-06-09T12:09:18.500382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.363233+0000","last_change":"2023-06-09T12:08:52.476110+0000","last_active":"2023-06-09T12:08:37.363233+0000","last_peered":"2023-06-09T12:08:37.363233+0000","last_clean":"2023-06-09T12:08:37.363233+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:37.363233+0000","last_undegraded":"2023-06-09T12:08:37.363233+0000","last_fullsized":"2023-06-09T12:08:37.363233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3394,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462072+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:07:07.462072+0000","last_peered":"2023-06-09T12:07:07.462072+0000","last_clean":"2023-06-09T12:07:07.462072+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:07:07.462072+0000","last_undegraded":"2023-06-09T12:07:07.462072+0000","last_fullsized":"2023-06-09T12:07:07.462072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.958702+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:05:52.958702+0000","last_peered":"2023-06-09T12:05:52.958702+0000","last_clean":"2023-06-09T12:05:52.958702+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:05:52.958702+0000","last_undegraded":"2023-06-09T12:05:52.958702+0000","last_fullsized":"2023-06-09T12:05:52.958702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":68116,"ondisk_log_size":68116,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30877,"ondisk_log_size":30877,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575562,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260880,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":81024,"kb_avail":94110960,"statfs":{"total":96636764160,"available":96369623040,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82968576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.54000000000000004,"5min":0.33700000000000002,"15min":0.29699999999999999},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.79200000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.56699999999999995,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.875,"5min":0.91100000000000003,"15min":1.0229999999999999},"last":0.745},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75700000000000001,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.89000000000000001,"5min":0.97599999999999998,"15min":1.05},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.96599999999999997,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.88400000000000001,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.754}]},{"osd":4,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.83199999999999996,"5min":1.006,"15min":1.006},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":1.226,"15min":1.226},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.54700000000000004,"5min":0.501,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.73099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575562,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213720,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57600,"kb_avail":94158120,"statfs":{"total":96636764160,"available":96417914880,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":58982400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":1.069},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.62,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.70899999999999996},"min":{"1min":0.625,"5min":0.503,"15min":0.36799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.91100000000000003,"5min":1.022,"15min":1.0369999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.78000000000000003},"min":{"1min":0.624,"5min":0.60299999999999998,"15min":0.53100000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0580000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55600000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.65900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.052},"last":0.71199999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:09:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.86499999999999999,"5min":0.95199999999999996,"15min":1.1100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.752}]}]},{"osd":1,"up_from":13,"seq":55834575563,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395248,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88832,"kb_avail":93976592,"statfs":{"total":96636764160,"available":96232030208,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":90963968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46700000000000003,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.51700000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.502,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.56299999999999994,"5min":0.94599999999999995,"15min":0.995},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46400000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.79600000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69999999999999996,"15min":0.66400000000000003},"min":{"1min":0.52800000000000002,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.84899999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":0.95299999999999996,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.876,"15min":0.83899999999999997},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.2829999999999999,"5min":1.597,"15min":1.597},"last":0.81200000000000006}]},{"osd":5,"last update":"Fri Jun 9 12:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.81599999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":1.544,"15min":1.544},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":0.98399999999999999,"5min":1.704,"15min":1.704},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85499999999999998,"15min":0.83299999999999996},"min":{"1min":0.624,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1719999999999999,"5min":1.548,"15min":1.548},"last":0.752},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.79300000000000004},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.0149999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90600000000000003,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.226,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86599999999999999,"15min":0.80600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.498,"15min":1.498},"last":0.69599999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575562,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":321832,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":94016,"kb_avail":94050008,"statfs":{"total":96636764160,"available":96307208192,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":96272384},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.33100000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.82699999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":1.032},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.91100000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.83999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84599999999999997,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1539999999999999,"5min":1.204,"15min":1.204},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89000000000000001,"15min":0.83399999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.2190000000000001},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79800000000000004,"15min":0.77000000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.004,"5min":1.121,"15min":1.121},"last":1.004},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.82999999999999996},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.145,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81999999999999995,"15min":0.81699999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.139,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":1.137},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.1739999999999999,"5min":1.232,"15min":1.232},"last":1.2050000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.175,"5min":1.177,"15min":1.298},"last":1.177},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.95499999999999996,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.2589999999999999,"15min":1.351},"last":1.2370000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575563,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231200,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51392,"kb_avail":94140640,"statfs":{"total":96636764160,"available":96400015360,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52625408},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.27300000000000002},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.432,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.66000000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.505}]},{"osd":1,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.63800000000000001,"5min":1.236,"15min":1.3109999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.38800000000000001},"max":{"1min":0.75700000000000001,"5min":1.2969999999999999,"15min":1.367},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.546},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80200000000000005,"15min":0.77300000000000002},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.373},"max":{"1min":1.079,"5min":1.202,"15min":1.8220000000000001},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.85499999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.601},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.89100000000000001,"15min":0.86899999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4670000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.034,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.78200000000000003},"min":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.67800000000000005,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.87,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.89400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575562,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427836,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":104640,"kb_avail":93944004,"statfs":{"total":96636764160,"available":96198660096,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":107151360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.31,"15min":0.223},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.504}]},{"osd":1,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.68300000000000005,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.56499999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.37,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.81200000000000006,"5min":0.877,"15min":1.198},"last":0.45200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.635},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.77900000000000003},"min":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.151,"5min":1.171,"15min":3.081},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.80900000000000005},"min":{"1min":0.61499999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.101,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.879},{"interface":"front","average":{"1min":0.872,"5min":0.89600000000000002,"15min":0.85299999999999998},"min":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.143,"5min":1.165,"15min":1.367},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0820000000000001,"5min":1.198,"15min":1.286},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83599999999999997,"15min":0.81000000000000005},"min":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.434},"max":{"1min":0.99199999999999999,"5min":1.117,"15min":3.3490000000000002},"last":0.79700000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.117,"5min":1.286,"15min":1.413},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.86199999999999999,"15min":0.86899999999999999},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1060000000000001,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.86399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575561,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428692,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":135936,"kb_avail":93943148,"statfs":{"total":96636764160,"available":96197783552,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":139198464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.70699999999999996},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0529999999999999,"15min":1.137},"last":1.419},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.64500000000000002,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.97899999999999998,"5min":1.0980000000000001,"15min":1.1899999999999999},"last":1.383}]},{"osd":1,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88300000000000001,"5min":0.95199999999999996,"15min":1.0640000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.57999999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83699999999999997,"5min":1.0149999999999999,"15min":1.609},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.73499999999999999},"min":{"1min":0.60299999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66700000000000004,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.78900000000000003,"5min":0.95899999999999996,"15min":1.407},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.4430000000000001},"last":0.754}]},{"osd":5,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.61599999999999999,"5min":0.90400000000000003,"15min":1.0189999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.83899999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.46800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.435},{"interface":"front","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.72499999999999998},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0489999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.72099999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.63200000000000001},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.89900000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.64500000000000002},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76100000000000001,"5min":0.97399999999999998,"15min":1.361},"last":0.55500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575561,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298584,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81216,"kb_avail":94073256,"statfs":{"total":96636764160,"available":96331014144,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83165184},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.45700000000000002,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.89100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73699999999999999,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":1.0129999999999999,"5min":1.167,"15min":19.715},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.49099999999999999,"15min":0.42599999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.66600000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":5.5620000000000003},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":1.083,"15min":14.439},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":12.130000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":0.98699999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.80200000000000005},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.82499999999999996},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.621}]},{"osd":6,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60299999999999998,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.93400000000000005,"15min":12.163},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.61099999999999999,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.86099999999999999,"5min":1.095,"15min":7.6040000000000001},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.85299999999999998,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.88600000000000001,"5min":1.238,"15min":19.834},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:10:01.774 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.774 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.775 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.775 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.775 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.776 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.776 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.777 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.777 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.777 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.778 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.778 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.778 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.779 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.779 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.780 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.780 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.780 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.781 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.781 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.781 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.782 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.782 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.783 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.783 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.783 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.784 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.784 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.784 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.785 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.785 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.786 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.786 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.786 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:01.787 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:10:21.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:10:22.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:21.999+0000 7fac257ff700 1 Processor -- start 2023-06-09T12:10:22.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:21.999+0000 7fac257ff700 1 -- start start 2023-06-09T12:10:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac257ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac257ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 0x7fac20159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac257ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac200c0bf0 con 0x7fac200c0140 2023-06-09T12:10:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac257ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac20070110 con 0x7fac200c1f90 2023-06-09T12:10:22.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac257ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac20071300 con 0x7fac200c1bc0 2023-06-09T12:10:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac1effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac1effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:39486/0 (socket says 172.21.15.7:39486) 2023-06-09T12:10:22.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac1effd700 1 -- 172.21.15.7:0/2604321159 learned_addr learned my addr 172.21.15.7:0/2604321159 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:10:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac17fff700 1 --1- 172.21.15.7:0/2604321159 >> v1:172.21.15.117:6789/0 conn(0x7fac200c0140 0x7fac201501b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:49240/0 (socket says 172.21.15.7:49240) 2023-06-09T12:10:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.001+0000 7fac1f7fe700 1 --2- 172.21.15.7:0/2604321159 >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 0x7fac20159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1e7fc700 1 -- 172.21.15.7:0/2604321159 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869760491 0 0) 0x7fac200c0bf0 con 0x7fac200c0140 2023-06-09T12:10:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1effd700 1 -- 172.21.15.7:0/2604321159 >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 msgr2=0x7fac20159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:10:22.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1effd700 1 --2- 172.21.15.7:0/2604321159 >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 0x7fac20159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1effd700 1 -- 172.21.15.7:0/2604321159 >> v1:172.21.15.117:6789/0 conn(0x7fac200c0140 legacy=0x7fac201501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1effd700 1 -- 172.21.15.7:0/2604321159 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200702f0 con 0x7fac200c1bc0 2023-06-09T12:10:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1f7fe700 1 --2- 172.21.15.7:0/2604321159 >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 0x7fac20159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:10:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1effd700 1 --2- 172.21.15.7:0/2604321159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac1000bb90 tx=0x7fac1000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ad17842d140e43 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:22.007 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1e7fc700 1 -- 172.21.15.7:0/2604321159 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000dd30 con 0x7fac200c1bc0 2023-06-09T12:10:22.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1e7fc700 1 -- 172.21.15.7:0/2604321159 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1000de90 con 0x7fac200c1bc0 2023-06-09T12:10:22.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.002+0000 7fac1e7fc700 1 -- 172.21.15.7:0/2604321159 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10010690 con 0x7fac200c1bc0 2023-06-09T12:10:22.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 -- 172.21.15.7:0/2604321159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 msgr2=0x7fac201599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 --2- 172.21.15.7:0/2604321159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac1000bb90 tx=0x7fac1000bea0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.008 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 -- 172.21.15.7:0/2604321159 shutdown_connections 2023-06-09T12:10:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 --2- 172.21.15.7:0/2604321159 >> v2:172.21.15.117:3300/0 conn(0x7fac200c1f90 0x7fac20159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 --2- 172.21.15.7:0/2604321159 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c1bc0 0x7fac201599b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 -- 172.21.15.7:0/2604321159 >> 172.21.15.7:0/2604321159 conn(0x7fac2006a4d0 msgr2=0x7fac2006a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 -- 172.21.15.7:0/2604321159 shutdown_connections 2023-06-09T12:10:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.003+0000 7fac257ff700 1 -- 172.21.15.7:0/2604321159 wait complete. 2023-06-09T12:10:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.005+0000 7fac257ff700 1 Processor -- start 2023-06-09T12:10:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.005+0000 7fac257ff700 1 -- start start 2023-06-09T12:10:22.011 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c0140 0x7fac201ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 0x7fac201ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200c2520 con 0x7fac200c1f90 2023-06-09T12:10:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200c2750 con 0x7fac200c1bc0 2023-06-09T12:10:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac257ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200c2a30 con 0x7fac200c0140 2023-06-09T12:10:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49646/0 (socket says 172.21.15.7:49646) 2023-06-09T12:10:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac1f7fe700 1 -- 172.21.15.7:0/335651921 learned_addr learned my addr 172.21.15.7:0/335651921 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:10:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac17fff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 0x7fac201ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.006+0000 7fac1effd700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c0140 0x7fac201ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 -- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c0140 msgr2=0x7fac201ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c0140 0x7fac201ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 -- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 msgr2=0x7fac201ea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 0x7fac201ea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 -- 172.21.15.7:0/335651921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009a70 con 0x7fac200c1f90 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac17fff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 0x7fac201ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac1f7fe700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fac1800efc0 tx=0x7fac1800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18009cd0 con 0x7fac200c1f90 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1800f040 con 0x7fac200c1f90 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1800cd60 con 0x7fac200c1f90 2023-06-09T12:10:22.015 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1000b7f0 con 0x7fac200c1f90 2023-06-09T12:10:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.007+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac200c1170 con 0x7fac200c1f90 2023-06-09T12:10:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.008+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac200c0e90 con 0x7fac200c1f90 2023-06-09T12:10:22.016 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.009+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fac18014480 con 0x7fac200c1f90 2023-06-09T12:10:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.010+0000 7fac177fe700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.010+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fac1808a5f0 con 0x7fac200c1f90 2023-06-09T12:10:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.011+0000 7fac1effd700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:22.017 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.011+0000 7fac1effd700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 0x7fac0006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fac1000bee0 tx=0x7fac10013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:22.018 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.015+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fac18054c90 con 0x7fac200c1f90 2023-06-09T12:10:22.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.224+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac200c1450 con 0x7fac0006ace0 2023-06-09T12:10:22.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.233+0000 7fac177fe700 1 -- 172.21.15.7:0/335651921 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317862 (secure 0 0 0) 0x7fac200c1450 con 0x7fac0006ace0 2023-06-09T12:10:22.238 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:10:22.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 msgr2=0x7fac0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 0x7fac0006d190 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fac1000bee0 tx=0x7fac10013000 comp rx=0 tx=0).stop 2023-06-09T12:10:22.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 msgr2=0x7fac200c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:22.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fac1800efc0 tx=0x7fac1800c3a0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 shutdown_connections 2023-06-09T12:10:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac200c1f90 0x7fac200c3ad0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fac0006ace0 0x7fac0006d190 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fac200c0140 0x7fac201ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.241 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 --2- 172.21.15.7:0/335651921 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fac200c1bc0 0x7fac201ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:22.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 >> 172.21.15.7:0/335651921 conn(0x7fac2006a4d0 msgr2=0x7fac200c6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:22.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 shutdown_connections 2023-06-09T12:10:22.242 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:22.237+0000 7fac257ff700 1 -- 172.21.15.7:0/335651921 wait complete. 2023-06-09T12:10:22.242 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:10:22.257 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1857,"stamp":"2023-06-09T12:10:21.104417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99449,"ondisk_log_size":99449,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2586952,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":703616,"kb_avail":752387768,"statfs":{"total":773094113280,"available":770445074432,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":720502784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003386"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045709+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045709+0000","last_peered":"2023-06-09T12:08:58.045709+0000","last_clean":"2023-06-09T12:08:58.045709+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045709+0000","last_undegraded":"2023-06-09T12:08:58.045709+0000","last_fullsized":"2023-06-09T12:08:58.045709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3262,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478915+0000","last_change":"2023-06-09T12:09:44.639071+0000","last_active":"2023-06-09T12:09:12.478915+0000","last_peered":"2023-06-09T12:09:12.478915+0000","last_clean":"2023-06-09T12:09:12.478915+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:09:12.478915+0000","last_undegraded":"2023-06-09T12:09:12.478915+0000","last_fullsized":"2023-06-09T12:09:12.478915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3607,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.450099+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:05:37.450099+0000","last_peered":"2023-06-09T12:05:37.450099+0000","last_clean":"2023-06-09T12:05:37.450099+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:05:37.450099+0000","last_undegraded":"2023-06-09T12:05:37.450099+0000","last_fullsized":"2023-06-09T12:05:37.450099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:41.384554+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:05:41.384554+0000","last_peered":"2023-06-09T12:05:41.384554+0000","last_clean":"2023-06-09T12:05:41.384554+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:05:41.384554+0000","last_undegraded":"2023-06-09T12:05:41.384554+0000","last_fullsized":"2023-06-09T12:05:41.384554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:25.145466+0000","last_change":"2023-06-09T12:09:12.373897+0000","last_active":"2023-06-09T12:08:25.145466+0000","last_peered":"2023-06-09T12:08:25.145466+0000","last_clean":"2023-06-09T12:08:25.145466+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:08:25.145466+0000","last_undegraded":"2023-06-09T12:08:25.145466+0000","last_fullsized":"2023-06-09T12:08:25.145466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1017","reported_seq":3705,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:14.268466+0000","last_change":"2023-06-09T12:09:45.596505+0000","last_active":"2023-06-09T12:10:14.268466+0000","last_peered":"2023-06-09T12:10:14.268466+0000","last_clean":"2023-06-09T12:10:14.268466+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:14.268466+0000","last_undegraded":"2023-06-09T12:10:14.268466+0000","last_fullsized":"2023-06-09T12:10:14.268466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":196,"log_size":1017,"log_dups_size":0,"ondisk_log_size":1017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:52.408446+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:05:52.408446+0000","last_peered":"2023-06-09T12:05:52.408446+0000","last_clean":"2023-06-09T12:05:52.408446+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:05:52.408446+0000","last_undegraded":"2023-06-09T12:05:52.408446+0000","last_fullsized":"2023-06-09T12:05:52.408446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3809,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:57.341583+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:05:57.341583+0000","last_peered":"2023-06-09T12:05:57.341583+0000","last_clean":"2023-06-09T12:05:57.341583+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:05:57.341583+0000","last_undegraded":"2023-06-09T12:05:57.341583+0000","last_fullsized":"2023-06-09T12:05:57.341583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3371,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:55.411595+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:09:55.411595+0000","last_peered":"2023-06-09T12:09:55.411595+0000","last_clean":"2023-06-09T12:09:55.411595+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:09:55.411595+0000","last_undegraded":"2023-06-09T12:09:55.411595+0000","last_fullsized":"2023-06-09T12:09:55.411595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4088,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.489680+0000","last_change":"2023-06-09T12:08:22.472324+0000","last_active":"2023-06-09T12:07:57.489680+0000","last_peered":"2023-06-09T12:07:57.489680+0000","last_clean":"2023-06-09T12:07:57.489680+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:07:57.489680+0000","last_undegraded":"2023-06-09T12:07:57.489680+0000","last_fullsized":"2023-06-09T12:07:57.489680+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'893","reported_seq":3322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:45.034121+0000","last_change":"2023-06-09T12:08:22.359777+0000","last_active":"2023-06-09T12:07:45.034121+0000","last_peered":"2023-06-09T12:07:45.034121+0000","last_clean":"2023-06-09T12:07:45.034121+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:07:45.034121+0000","last_undegraded":"2023-06-09T12:07:45.034121+0000","last_fullsized":"2023-06-09T12:07:45.034121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2267,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.327223+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:05:37.327223+0000","last_peered":"2023-06-09T12:05:37.327223+0000","last_clean":"2023-06-09T12:05:37.327223+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:05:37.327223+0000","last_undegraded":"2023-06-09T12:05:37.327223+0000","last_fullsized":"2023-06-09T12:05:37.327223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'950","reported_seq":3170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:16.875983+0000","last_change":"2023-06-09T12:10:14.991383+0000","last_active":"2023-06-09T12:10:16.875983+0000","last_peered":"2023-06-09T12:10:16.875983+0000","last_clean":"2023-06-09T12:10:16.875983+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:16.875983+0000","last_undegraded":"2023-06-09T12:10:16.875983+0000","last_fullsized":"2023-06-09T12:10:16.875983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":23,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'862","reported_seq":3335,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:17.363185+0000","last_change":"2023-06-09T12:09:44.139026+0000","last_active":"2023-06-09T12:10:17.363185+0000","last_peered":"2023-06-09T12:10:17.363185+0000","last_clean":"2023-06-09T12:10:17.363185+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:17.363185+0000","last_undegraded":"2023-06-09T12:10:17.363185+0000","last_fullsized":"2023-06-09T12:10:17.363185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1040,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:44.447705+0000","last_change":"2023-06-09T12:09:22.480454+0000","last_active":"2023-06-09T12:07:44.447705+0000","last_peered":"2023-06-09T12:07:44.447705+0000","last_clean":"2023-06-09T12:07:44.447705+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:07:44.447705+0000","last_undegraded":"2023-06-09T12:07:44.447705+0000","last_fullsized":"2023-06-09T12:07:44.447705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1116,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:10.469903+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:06:10.469903+0000","last_peered":"2023-06-09T12:06:10.469903+0000","last_clean":"2023-06-09T12:06:10.469903+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:06:10.469903+0000","last_undegraded":"2023-06-09T12:06:10.469903+0000","last_fullsized":"2023-06-09T12:06:10.469903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3868,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301798+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301798+0000","last_peered":"2023-06-09T12:08:15.301798+0000","last_clean":"2023-06-09T12:08:15.301798+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301798+0000","last_undegraded":"2023-06-09T12:08:15.301798+0000","last_fullsized":"2023-06-09T12:08:15.301798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":705,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:51.464012+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:09:51.464012+0000","last_peered":"2023-06-09T12:09:51.464012+0000","last_clean":"2023-06-09T12:09:51.464012+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:09:51.464012+0000","last_undegraded":"2023-06-09T12:09:51.464012+0000","last_fullsized":"2023-06-09T12:09:51.464012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3656,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.462853+0000","last_change":"2023-06-09T12:07:12.462853+0000","last_active":"2023-06-09T12:07:12.462853+0000","last_peered":"2023-06-09T12:07:12.462853+0000","last_clean":"2023-06-09T12:07:12.462853+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:07:12.462853+0000","last_undegraded":"2023-06-09T12:07:12.462853+0000","last_fullsized":"2023-06-09T12:07:12.462853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4300,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256603+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256603+0000","last_peered":"2023-06-09T12:08:20.256603+0000","last_clean":"2023-06-09T12:08:20.256603+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256603+0000","last_undegraded":"2023-06-09T12:08:20.256603+0000","last_fullsized":"2023-06-09T12:08:20.256603+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3845,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480430+0000","last_change":"2023-06-09T12:09:42.640933+0000","last_active":"2023-06-09T12:09:22.480430+0000","last_peered":"2023-06-09T12:09:22.480430+0000","last_clean":"2023-06-09T12:09:22.480430+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:09:22.480430+0000","last_undegraded":"2023-06-09T12:09:22.480430+0000","last_fullsized":"2023-06-09T12:09:22.480430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3217,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:32.359589+0000","last_change":"2023-06-09T12:10:12.514350+0000","last_active":"2023-06-09T12:09:32.359589+0000","last_peered":"2023-06-09T12:09:32.359589+0000","last_clean":"2023-06-09T12:09:32.359589+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:09:32.359589+0000","last_undegraded":"2023-06-09T12:09:32.359589+0000","last_fullsized":"2023-06-09T12:09:32.359589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3607,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355679+0000","last_change":"2023-06-09T12:10:14.478668+0000","last_active":"2023-06-09T12:09:12.355679+0000","last_peered":"2023-06-09T12:09:12.355679+0000","last_clean":"2023-06-09T12:09:12.355679+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:09:12.355679+0000","last_undegraded":"2023-06-09T12:09:12.355679+0000","last_fullsized":"2023-06-09T12:09:12.355679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3899,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355730+0000","last_change":"2023-06-09T12:10:13.485039+0000","last_active":"2023-06-09T12:09:12.355730+0000","last_peered":"2023-06-09T12:09:12.355730+0000","last_clean":"2023-06-09T12:09:12.355730+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:09:12.355730+0000","last_undegraded":"2023-06-09T12:09:12.355730+0000","last_fullsized":"2023-06-09T12:09:12.355730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1351,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.344216+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:06:17.344216+0000","last_peered":"2023-06-09T12:06:17.344216+0000","last_clean":"2023-06-09T12:06:17.344216+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:06:17.344216+0000","last_undegraded":"2023-06-09T12:06:17.344216+0000","last_fullsized":"2023-06-09T12:06:17.344216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:49.493713+0000","last_change":"2023-06-09T12:09:27.528163+0000","last_active":"2023-06-09T12:09:49.493713+0000","last_peered":"2023-06-09T12:09:49.493713+0000","last_clean":"2023-06-09T12:09:49.493713+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:09:49.493713+0000","last_undegraded":"2023-06-09T12:09:49.493713+0000","last_fullsized":"2023-06-09T12:09:49.493713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.342702+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:07:07.342702+0000","last_peered":"2023-06-09T12:07:07.342702+0000","last_clean":"2023-06-09T12:07:07.342702+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:07:07.342702+0000","last_undegraded":"2023-06-09T12:07:07.342702+0000","last_fullsized":"2023-06-09T12:07:07.342702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":165,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:17.334061+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:06:17.334061+0000","last_peered":"2023-06-09T12:06:17.334061+0000","last_clean":"2023-06-09T12:06:17.334061+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:06:17.334061+0000","last_undegraded":"2023-06-09T12:06:17.334061+0000","last_fullsized":"2023-06-09T12:06:17.334061+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870217+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870217+0000","last_peered":"2023-06-09T12:09:21.870217+0000","last_clean":"2023-06-09T12:09:21.870217+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870217+0000","last_undegraded":"2023-06-09T12:09:21.870217+0000","last_fullsized":"2023-06-09T12:09:21.870217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":835,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.171989+0000","last_change":"2023-06-09T12:09:22.375312+0000","last_active":"2023-06-09T12:08:34.171989+0000","last_peered":"2023-06-09T12:08:34.171989+0000","last_clean":"2023-06-09T12:08:34.171989+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:08:34.171989+0000","last_undegraded":"2023-06-09T12:08:34.171989+0000","last_fullsized":"2023-06-09T12:08:34.171989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3721,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:38.366273+0000","last_change":"2023-06-09T12:09:12.478817+0000","last_active":"2023-06-09T12:08:38.366273+0000","last_peered":"2023-06-09T12:08:38.366273+0000","last_clean":"2023-06-09T12:08:38.366273+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:08:38.366273+0000","last_undegraded":"2023-06-09T12:08:38.366273+0000","last_fullsized":"2023-06-09T12:08:38.366273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:05:37.406642+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:05:37.406642+0000","last_peered":"2023-06-09T12:05:37.406642+0000","last_clean":"2023-06-09T12:05:37.406642+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:05:37.406642+0000","last_undegraded":"2023-06-09T12:05:37.406642+0000","last_fullsized":"2023-06-09T12:05:37.406642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":783,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:20.548654+0000","last_change":"2023-06-09T12:09:02.540941+0000","last_active":"2023-06-09T12:09:20.548654+0000","last_peered":"2023-06-09T12:09:20.548654+0000","last_clean":"2023-06-09T12:09:20.548654+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:09:20.548654+0000","last_undegraded":"2023-06-09T12:09:20.548654+0000","last_fullsized":"2023-06-09T12:09:20.548654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1065,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.344957+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:06:22.344957+0000","last_peered":"2023-06-09T12:06:22.344957+0000","last_clean":"2023-06-09T12:06:22.344957+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:06:22.344957+0000","last_undegraded":"2023-06-09T12:06:22.344957+0000","last_fullsized":"2023-06-09T12:06:22.344957+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1626,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.336909+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:06:22.336909+0000","last_peered":"2023-06-09T12:06:22.336909+0000","last_clean":"2023-06-09T12:06:22.336909+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:06:22.336909+0000","last_undegraded":"2023-06-09T12:06:22.336909+0000","last_fullsized":"2023-06-09T12:06:22.336909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.363233+0000","last_change":"2023-06-09T12:08:52.476110+0000","last_active":"2023-06-09T12:08:37.363233+0000","last_peered":"2023-06-09T12:08:37.363233+0000","last_clean":"2023-06-09T12:08:37.363233+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:37.363233+0000","last_undegraded":"2023-06-09T12:08:37.363233+0000","last_fullsized":"2023-06-09T12:08:37.363233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3394,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.462072+0000","last_change":"2023-06-09T12:07:07.462072+0000","last_active":"2023-06-09T12:07:07.462072+0000","last_peered":"2023-06-09T12:07:07.462072+0000","last_clean":"2023-06-09T12:07:07.462072+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:07:07.462072+0000","last_undegraded":"2023-06-09T12:07:07.462072+0000","last_fullsized":"2023-06-09T12:07:07.462072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":68453,"ondisk_log_size":68453,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30877,"ondisk_log_size":30877,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575566,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260880,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":81024,"kb_avail":94110960,"statfs":{"total":96636764160,"available":96369623040,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":82968576},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.54000000000000004,"5min":0.33700000000000002,"15min":0.29699999999999999},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.78300000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.56699999999999995,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.875,"5min":0.91100000000000003,"15min":1.0229999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75700000000000001,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.89000000000000001,"5min":0.97599999999999998,"15min":1.05},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.96599999999999997,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.88400000000000001,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.83199999999999996,"5min":1.006,"15min":1.006},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":1.226,"15min":1.226},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.54700000000000004,"5min":0.501,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.71299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575566,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213720,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57600,"kb_avail":94158120,"statfs":{"total":96636764160,"available":96417914880,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":58982400},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":1.069},"last":0.82099999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.62,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.70899999999999996},"min":{"1min":0.625,"5min":0.503,"15min":0.36799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.91100000000000003,"5min":1.022,"15min":1.0369999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.78000000000000003},"min":{"1min":0.624,"5min":0.60299999999999998,"15min":0.53100000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0580000000000001},"last":0.78900000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.65900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.052},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.86499999999999999,"5min":0.95199999999999996,"15min":1.1100000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.876}]}]},{"osd":1,"up_from":13,"seq":55834575567,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395376,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":88960,"kb_avail":93976464,"statfs":{"total":96636764160,"available":96231899136,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":91095040},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46700000000000003,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.51700000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.502,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.56299999999999994,"5min":0.94599999999999995,"15min":0.995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.79600000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69999999999999996,"15min":0.66400000000000003},"min":{"1min":0.52800000000000002,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.84899999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":0.95299999999999996,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.876,"15min":0.83899999999999997},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.2829999999999999,"5min":1.597,"15min":1.597},"last":0.92500000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.81599999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":1.544,"15min":1.544},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":0.98399999999999999,"5min":1.704,"15min":1.704},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85499999999999998,"15min":0.83299999999999996},"min":{"1min":0.624,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1719999999999999,"5min":1.548,"15min":1.548},"last":0.94699999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.79300000000000004},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.0149999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90600000000000003,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.226,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86599999999999999,"15min":0.80600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.498,"15min":1.498},"last":1.006}]}]},{"osd":0,"up_from":13,"seq":55834575566,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326056,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98240,"kb_avail":94045784,"statfs":{"total":96636764160,"available":96302882816,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":100597760},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47899999999999998,"15min":0.45000000000000001},"min":{"1min":0.33100000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.74299999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.54500000000000004},"min":{"1min":0.42799999999999999,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.749,"5min":0.749,"15min":0.82699999999999996},"last":0.435}]},{"osd":2,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.41199999999999998,"5min":0.38300000000000001,"15min":0.28999999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":1.032},"last":0.502},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54800000000000004,"15min":0.52100000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.71999999999999997,"5min":0.75900000000000001,"15min":0.80000000000000004},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57699999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69399999999999995,"15min":0.69599999999999995},"min":{"1min":0.60899999999999999,"5min":0.52500000000000002,"15min":0.46400000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.84599999999999997,"15min":0.77100000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1539999999999999,"5min":1.204,"15min":1.204},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.89000000000000001,"15min":0.83399999999999996},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.2190000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79800000000000004,"15min":0.77000000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.004,"5min":1.121,"15min":1.121},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.91700000000000004,"15min":0.82999999999999996},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.46600000000000003},"max":{"1min":1.145,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81999999999999995,"15min":0.81699999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.139,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.63,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.1739999999999999,"5min":1.232,"15min":1.232},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:30 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.874,"15min":0.82799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.175,"5min":1.177,"15min":1.298},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.95499999999999996,"15min":0.90400000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.52400000000000002},"max":{"1min":1.2030000000000001,"5min":1.2589999999999999,"15min":1.351},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575567,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231328,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51520,"kb_avail":94140512,"statfs":{"total":96636764160,"available":96399884288,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52756480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.27300000000000002},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.432,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.66000000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.63800000000000001,"5min":1.236,"15min":1.3109999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.38800000000000001},"max":{"1min":0.75700000000000001,"5min":1.2969999999999999,"15min":1.367},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.546},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.63600000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80200000000000005,"15min":0.77300000000000002},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.373},"max":{"1min":1.079,"5min":1.202,"15min":1.8220000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.85699999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.601},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.89100000000000001,"15min":0.86899999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4670000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.877}]},{"osd":6,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.034,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.78200000000000003},"min":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.89300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.67800000000000005,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.87,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575566,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432060,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":108864,"kb_avail":93939780,"statfs":{"total":96636764160,"available":96194334720,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":111476736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.31,"15min":0.223},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51800000000000002,"15min":0.51100000000000001},"min":{"1min":0.36099999999999999,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.68300000000000005,"5min":0.81999999999999995,"15min":0.95499999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55900000000000005,"15min":0.52900000000000003},"min":{"1min":0.45900000000000002,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.75800000000000001,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.64600000000000002,"15min":0.64500000000000002},"min":{"1min":0.51200000000000001,"5min":0.50600000000000001,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.37,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.81200000000000006,"5min":0.877,"15min":1.198},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.80600000000000005,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.48599999999999999,"15min":0.38},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.635},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.80000000000000004,"15min":0.77900000000000003},"min":{"1min":0.63300000000000001,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.151,"5min":1.171,"15min":3.081},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.85299999999999998,"15min":0.80900000000000005},"min":{"1min":0.61499999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.101,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.89600000000000002,"15min":0.85299999999999998},"min":{"1min":0.72199999999999998,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.143,"5min":1.165,"15min":1.367},"last":0.87}]},{"osd":6,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81999999999999995,"15min":0.80100000000000005},"min":{"1min":0.65000000000000002,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":1.0820000000000001,"5min":1.198,"15min":1.286},"last":0.747},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.83599999999999997,"15min":0.81000000000000005},"min":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.434},"max":{"1min":0.99199999999999999,"5min":1.117,"15min":3.3490000000000002},"last":0.80000000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:09:26 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.82499999999999996},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.117,"5min":1.286,"15min":1.413},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.86199999999999999,"15min":0.86899999999999999},"min":{"1min":0.58399999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1060000000000001,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.81899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575565,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428820,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":136064,"kb_avail":93943020,"statfs":{"total":96636764160,"available":96197652480,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":139329536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.70699999999999996},"min":{"1min":0.59299999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0229999999999999,"5min":1.0529999999999999,"15min":1.137},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76500000000000001,"15min":0.76700000000000002},"min":{"1min":0.64500000000000002,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":0.97899999999999998,"5min":1.0980000000000001,"15min":1.1899999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72999999999999998,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88300000000000001,"5min":0.95199999999999996,"15min":1.0640000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.77400000000000002,"15min":0.77500000000000002},"min":{"1min":0.57999999999999996,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83699999999999997,"5min":1.0149999999999999,"15min":1.609},"last":0.621},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":0.73499999999999999},"min":{"1min":0.60299999999999998,"5min":0.59799999999999998,"15min":0.38100000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.82999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.66700000000000004,"15min":0.68600000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.44400000000000001},"max":{"1min":0.78900000000000003,"5min":0.95899999999999996,"15min":1.407},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":1.111,"5min":1.111,"15min":1.4430000000000001},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52700000000000002,"15min":0.53900000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.61599999999999999,"5min":0.90400000000000003,"15min":1.0189999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.60699999999999998,"15min":0.623},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.83899999999999997,"5min":1.05,"15min":1.1719999999999999},"last":0.49099999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.42899999999999999,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.70899999999999996,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.73999999999999999,"15min":0.72499999999999998},"min":{"1min":0.63800000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0489999999999999,"5min":1.131,"15min":1.3100000000000001},"last":0.56200000000000006}]},{"osd":7,"last update":"Fri Jun 9 12:09:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66800000000000004,"15min":0.63200000000000001},"min":{"1min":0.52200000000000002,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.89900000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59499999999999997,"15min":0.64500000000000002},"min":{"1min":0.38900000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.76100000000000001,"5min":0.97399999999999998,"15min":1.361},"last":0.51100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575565,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298712,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81344,"kb_avail":94073128,"statfs":{"total":96636764160,"available":96330883072,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83296256},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.45700000000000002,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.89100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73699999999999999,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":1.0129999999999999,"5min":1.167,"15min":19.715},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.49099999999999999,"15min":0.42599999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":5.5620000000000003},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":1.083,"15min":14.439},"last":0.90300000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":12.130000000000001},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":0.98699999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.94299999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.80200000000000005},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.82499999999999996},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.627}]},{"osd":6,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60299999999999998,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.93400000000000005,"15min":12.163},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.61099999999999999,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.86099999999999999,"5min":1.095,"15min":7.6040000000000001},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.85299999999999998,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.88600000000000001,"5min":1.238,"15min":19.834},"last":0.86299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:10:22.265 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.265 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.266 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.266 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.266 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.267 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.267 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.268 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.268 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.268 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.269 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.269 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.269 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.270 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.270 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.271 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.271 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.271 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.272 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.272 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.272 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.273 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.273 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.274 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.274 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.274 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.275 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.275 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.275 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.276 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.276 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.276 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:22.277 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:10:28.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:28.658 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:42.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:10:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.490+0000 7f3f5d5ff700 1 Processor -- start 2023-06-09T12:10:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.490+0000 7f3f5d5ff700 1 -- start start 2023-06-09T12:10:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.491+0000 7f3f5d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.491+0000 7f3f5d5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f3f5815b380 0x7f3f58070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.491+0000 7f3f5d5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f581580a0 con 0x7f3f58071060 2023-06-09T12:10:42.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.491+0000 7f3f5d5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f580714e0 con 0x7f3f5815b380 2023-06-09T12:10:42.495 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.491+0000 7f3f5d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f581518e0 con 0x7f3f58151510 2023-06-09T12:10:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:50984/0 (socket says 172.21.15.7:50984) 2023-06-09T12:10:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f56ffd700 1 -- 172.21.15.7:0/158746943 learned_addr learned my addr 172.21.15.7:0/158746943 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:10:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f567fc700 1 --2- 172.21.15.7:0/158746943 >> v2:172.21.15.117:3300/0 conn(0x7f3f5815b380 0x7f3f58070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f567fc700 1 -- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 msgr2=0x7f3f580c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:10:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f567fc700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f567fc700 1 -- 172.21.15.7:0/158746943 >> v1:172.21.15.117:6789/0 conn(0x7f3f58071060 legacy=0x7f3f58158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f567fc700 1 -- 172.21.15.7:0/158746943 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580c6180 con 0x7f3f5815b380 2023-06-09T12:10:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.492+0000 7f3f56ffd700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:10:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.493+0000 7f3f567fc700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f58070b20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009840 tx=0x7f3f4c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26beaeab4680a978 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.493+0000 7f3f55ffb700 1 -- 172.21.15.7:0/158746943 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c011830 con 0x7f3f5815b380 2023-06-09T12:10:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.493+0000 7f3f55ffb700 1 -- 172.21.15.7:0/158746943 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c011990 con 0x7f3f5815b380 2023-06-09T12:10:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.493+0000 7f3f55ffb700 1 -- 172.21.15.7:0/158746943 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c011d20 con 0x7f3f5815b380 2023-06-09T12:10:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.493+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 msgr2=0x7f3f58070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f58070b20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009840 tx=0x7f3f4c009b50 comp rx=0 tx=0).stop 2023-06-09T12:10:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/158746943 shutdown_connections 2023-06-09T12:10:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f580c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/158746943 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f58070b20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/158746943 >> 172.21.15.7:0/158746943 conn(0x7f3f5806a4d0 msgr2=0x7f3f5806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/158746943 shutdown_connections 2023-06-09T12:10:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.494+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/158746943 wait complete. 2023-06-09T12:10:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.495+0000 7f3f5d5ff700 1 Processor -- start 2023-06-09T12:10:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.496+0000 7f3f5d5ff700 1 -- start start 2023-06-09T12:10:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f58158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f580c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5815a6c0 con 0x7f3f58071060 2023-06-09T12:10:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580c0b30 con 0x7f3f5815b380 2023-06-09T12:10:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f5d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580b0ec0 con 0x7f3f58151510 2023-06-09T12:10:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41390/0 (socket says 172.21.15.7:41390) 2023-06-09T12:10:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 -- 172.21.15.7:0/504583418 learned_addr learned my addr 172.21.15.7:0/504583418 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:10:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f567fc700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f58158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f577fe700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f580c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 -- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 msgr2=0x7f3f58158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.505 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f58158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 -- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 msgr2=0x7f3f580c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f580c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f56ffd700 1 -- 172.21.15.7:0/504583418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f48009550 con 0x7f3f58071060 2023-06-09T12:10:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.497+0000 7f3f567fc700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f58158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:10:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f56ffd700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800ec10 tx=0x7f3f4800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800c850 con 0x7f3f58071060 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4800c9b0 con 0x7f3f58071060 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800cd40 con 0x7f3f58071060 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c0094a0 con 0x7f3f58071060 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.498+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f58157770 con 0x7f3f58071060 2023-06-09T12:10:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.499+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f58157490 con 0x7f3f58071060 2023-06-09T12:10:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.500+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f48013470 con 0x7f3f58071060 2023-06-09T12:10:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.501+0000 7f3f3f7fe700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:10:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.501+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f3f48089f70 con 0x7f3f58071060 2023-06-09T12:10:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.501+0000 7f3f567fc700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:10:42.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.502+0000 7f3f567fc700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 0x7f3f4406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000c00 tx=0x7f3f4c01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:10:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.506+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f3f48054610 con 0x7f3f58071060 2023-06-09T12:10:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.709+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f58157a50 con 0x7f3f4406ace0 2023-06-09T12:10:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.718+0000 7f3f3f7fe700 1 -- 172.21.15.7:0/504583418 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+317924 (secure 0 0 0) 0x7f3f58157a50 con 0x7f3f4406ace0 2023-06-09T12:10:42.724 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:10:42.726 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 msgr2=0x7f3f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 0x7f3f4406d190 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000c00 tx=0x7f3f4c01c040 comp rx=0 tx=0).stop 2023-06-09T12:10:42.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 msgr2=0x7f3f5814b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:10:42.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800ec10 tx=0x7f3f4800ef20 comp rx=0 tx=0).stop 2023-06-09T12:10:42.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 shutdown_connections 2023-06-09T12:10:42.727 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3f5815b380 0x7f3f580c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f58071060 0x7f3f5814b840 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 --2- 172.21.15.7:0/504583418 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3f58151510 0x7f3f58158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:10:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 >> 172.21.15.7:0/504583418 conn(0x7f3f5806a4d0 msgr2=0x7f3f581e0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:10:42.728 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 shutdown_connections 2023-06-09T12:10:42.729 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:10:42.723+0000 7f3f5d5ff700 1 -- 172.21.15.7:0/504583418 wait complete. 2023-06-09T12:10:42.729 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:10:42.744 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1867,"stamp":"2023-06-09T12:10:41.109975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99894,"ondisk_log_size":99894,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2591752,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":708416,"kb_avail":752382968,"statfs":{"total":773094113280,"available":770440159232,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":725417984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003434"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3484,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045709+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045709+0000","last_peered":"2023-06-09T12:08:58.045709+0000","last_clean":"2023-06-09T12:08:58.045709+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045709+0000","last_undegraded":"2023-06-09T12:08:58.045709+0000","last_fullsized":"2023-06-09T12:08:58.045709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3262,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478915+0000","last_change":"2023-06-09T12:09:44.639071+0000","last_active":"2023-06-09T12:09:12.478915+0000","last_peered":"2023-06-09T12:09:12.478915+0000","last_clean":"2023-06-09T12:09:12.478915+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:09:12.478915+0000","last_undegraded":"2023-06-09T12:09:12.478915+0000","last_fullsized":"2023-06-09T12:09:12.478915+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3463,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476279+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:52.476279+0000","last_peered":"2023-06-09T12:08:52.476279+0000","last_clean":"2023-06-09T12:08:52.476279+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:52.476279+0000","last_undegraded":"2023-06-09T12:08:52.476279+0000","last_fullsized":"2023-06-09T12:08:52.476279+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3741,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:25.145466+0000","last_change":"2023-06-09T12:09:12.373897+0000","last_active":"2023-06-09T12:08:25.145466+0000","last_peered":"2023-06-09T12:08:25.145466+0000","last_clean":"2023-06-09T12:08:25.145466+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:08:25.145466+0000","last_undegraded":"2023-06-09T12:08:25.145466+0000","last_fullsized":"2023-06-09T12:08:25.145466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.439029+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:06:27.439029+0000","last_peered":"2023-06-09T12:06:27.439029+0000","last_clean":"2023-06-09T12:06:27.439029+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:06:27.439029+0000","last_undegraded":"2023-06-09T12:06:27.439029+0000","last_fullsized":"2023-06-09T12:06:27.439029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3371,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:55.411595+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:09:55.411595+0000","last_peered":"2023-06-09T12:09:55.411595+0000","last_clean":"2023-06-09T12:09:55.411595+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:09:55.411595+0000","last_undegraded":"2023-06-09T12:09:55.411595+0000","last_fullsized":"2023-06-09T12:09:55.411595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4089,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:22.472324+0000","last_change":"2023-06-09T12:10:29.777049+0000","last_active":"2023-06-09T12:08:22.472324+0000","last_peered":"2023-06-09T12:08:22.472324+0000","last_clean":"2023-06-09T12:08:22.472324+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:08:22.472324+0000","last_undegraded":"2023-06-09T12:08:22.472324+0000","last_fullsized":"2023-06-09T12:08:22.472324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1039","reported_seq":3479,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:37.296988+0000","last_change":"2023-06-09T12:10:17.458560+0000","last_active":"2023-06-09T12:10:37.296988+0000","last_peered":"2023-06-09T12:10:37.296988+0000","last_clean":"2023-06-09T12:10:37.296988+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:37.296988+0000","last_undegraded":"2023-06-09T12:10:37.296988+0000","last_fullsized":"2023-06-09T12:10:37.296988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":146,"log_size":1039,"log_dups_size":0,"ondisk_log_size":1039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1071","reported_seq":3297,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:31.758089+0000","last_change":"2023-06-09T12:10:14.991383+0000","last_active":"2023-06-09T12:10:31.758089+0000","last_peered":"2023-06-09T12:10:31.758089+0000","last_clean":"2023-06-09T12:10:31.758089+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:31.758089+0000","last_undegraded":"2023-06-09T12:10:31.758089+0000","last_fullsized":"2023-06-09T12:10:31.758089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":144,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 17s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1008","reported_seq":3488,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:36.739209+0000","last_change":"2023-06-09T12:09:44.139026+0000","last_active":"2023-06-09T12:10:36.739209+0000","last_peered":"2023-06-09T12:10:36.739209+0000","last_clean":"2023-06-09T12:10:36.739209+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:36.739209+0000","last_undegraded":"2023-06-09T12:10:36.739209+0000","last_fullsized":"2023-06-09T12:10:36.739209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":146,"log_size":1008,"log_dups_size":0,"ondisk_log_size":1008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1041,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480454+0000","last_change":"2023-06-09T12:10:27.777932+0000","last_active":"2023-06-09T12:09:22.480454+0000","last_peered":"2023-06-09T12:09:22.480454+0000","last_clean":"2023-06-09T12:09:22.480454+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:09:22.480454+0000","last_undegraded":"2023-06-09T12:09:22.480454+0000","last_fullsized":"2023-06-09T12:09:22.480454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:19.539716+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:06:19.539716+0000","last_peered":"2023-06-09T12:06:19.539716+0000","last_clean":"2023-06-09T12:06:19.539716+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:06:19.539716+0000","last_undegraded":"2023-06-09T12:06:19.539716+0000","last_fullsized":"2023-06-09T12:06:19.539716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":705,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:51.464012+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:09:51.464012+0000","last_peered":"2023-06-09T12:09:51.464012+0000","last_clean":"2023-06-09T12:09:51.464012+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:09:51.464012+0000","last_undegraded":"2023-06-09T12:09:51.464012+0000","last_fullsized":"2023-06-09T12:09:51.464012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3657,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480291+0000","last_change":"2023-06-09T12:10:25.770236+0000","last_active":"2023-06-09T12:09:22.480291+0000","last_peered":"2023-06-09T12:09:22.480291+0000","last_clean":"2023-06-09T12:09:22.480291+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:09:22.480291+0000","last_undegraded":"2023-06-09T12:09:22.480291+0000","last_fullsized":"2023-06-09T12:09:22.480291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3845,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480430+0000","last_change":"2023-06-09T12:09:42.640933+0000","last_active":"2023-06-09T12:09:22.480430+0000","last_peered":"2023-06-09T12:09:22.480430+0000","last_clean":"2023-06-09T12:09:22.480430+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:09:22.480430+0000","last_undegraded":"2023-06-09T12:09:22.480430+0000","last_fullsized":"2023-06-09T12:09:22.480430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.528805+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:06:22.528805+0000","last_peered":"2023-06-09T12:06:22.528805+0000","last_clean":"2023-06-09T12:06:22.528805+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:06:22.528805+0000","last_undegraded":"2023-06-09T12:06:22.528805+0000","last_fullsized":"2023-06-09T12:06:22.528805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'957","reported_seq":3217,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:32.359589+0000","last_change":"2023-06-09T12:10:12.514350+0000","last_active":"2023-06-09T12:09:32.359589+0000","last_peered":"2023-06-09T12:09:32.359589+0000","last_clean":"2023-06-09T12:09:32.359589+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:09:32.359589+0000","last_undegraded":"2023-06-09T12:09:32.359589+0000","last_fullsized":"2023-06-09T12:09:32.359589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3607,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355679+0000","last_change":"2023-06-09T12:10:14.478668+0000","last_active":"2023-06-09T12:09:12.355679+0000","last_peered":"2023-06-09T12:09:12.355679+0000","last_clean":"2023-06-09T12:09:12.355679+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:09:12.355679+0000","last_undegraded":"2023-06-09T12:09:12.355679+0000","last_fullsized":"2023-06-09T12:09:12.355679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3899,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355730+0000","last_change":"2023-06-09T12:10:13.485039+0000","last_active":"2023-06-09T12:09:12.355730+0000","last_peered":"2023-06-09T12:09:12.355730+0000","last_clean":"2023-06-09T12:09:12.355730+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:09:12.355730+0000","last_undegraded":"2023-06-09T12:09:12.355730+0000","last_fullsized":"2023-06-09T12:09:12.355730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.364902+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:06:47.364902+0000","last_peered":"2023-06-09T12:06:47.364902+0000","last_clean":"2023-06-09T12:06:47.364902+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:06:47.364902+0000","last_undegraded":"2023-06-09T12:06:47.364902+0000","last_fullsized":"2023-06-09T12:06:47.364902+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:49.493713+0000","last_change":"2023-06-09T12:09:27.528163+0000","last_active":"2023-06-09T12:09:49.493713+0000","last_peered":"2023-06-09T12:09:49.493713+0000","last_clean":"2023-06-09T12:09:49.493713+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:09:49.493713+0000","last_undegraded":"2023-06-09T12:09:49.493713+0000","last_fullsized":"2023-06-09T12:09:49.493713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.342702+0000","last_change":"2023-06-09T12:07:07.342702+0000","last_active":"2023-06-09T12:07:07.342702+0000","last_peered":"2023-06-09T12:07:07.342702+0000","last_clean":"2023-06-09T12:07:07.342702+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:07:07.342702+0000","last_undegraded":"2023-06-09T12:07:07.342702+0000","last_fullsized":"2023-06-09T12:07:07.342702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870217+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870217+0000","last_peered":"2023-06-09T12:09:21.870217+0000","last_clean":"2023-06-09T12:09:21.870217+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870217+0000","last_undegraded":"2023-06-09T12:09:21.870217+0000","last_fullsized":"2023-06-09T12:09:21.870217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":835,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.171989+0000","last_change":"2023-06-09T12:09:22.375312+0000","last_active":"2023-06-09T12:08:34.171989+0000","last_peered":"2023-06-09T12:08:34.171989+0000","last_clean":"2023-06-09T12:08:34.171989+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:08:34.171989+0000","last_undegraded":"2023-06-09T12:08:34.171989+0000","last_fullsized":"2023-06-09T12:08:34.171989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3721,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:38.366273+0000","last_change":"2023-06-09T12:09:12.478817+0000","last_active":"2023-06-09T12:08:38.366273+0000","last_peered":"2023-06-09T12:08:38.366273+0000","last_clean":"2023-06-09T12:08:38.366273+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:08:38.366273+0000","last_undegraded":"2023-06-09T12:08:38.366273+0000","last_fullsized":"2023-06-09T12:08:38.366273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'593","reported_seq":783,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:20.548654+0000","last_change":"2023-06-09T12:09:02.540941+0000","last_active":"2023-06-09T12:09:20.548654+0000","last_peered":"2023-06-09T12:09:20.548654+0000","last_clean":"2023-06-09T12:09:20.548654+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:09:20.548654+0000","last_undegraded":"2023-06-09T12:09:20.548654+0000","last_fullsized":"2023-06-09T12:09:20.548654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":593,"log_dups_size":0,"ondisk_log_size":593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3933,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435184+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435184+0000","last_peered":"2023-06-09T12:08:36.435184+0000","last_clean":"2023-06-09T12:08:36.435184+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435184+0000","last_undegraded":"2023-06-09T12:08:36.435184+0000","last_fullsized":"2023-06-09T12:08:36.435184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359803+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:08:22.359803+0000","last_peered":"2023-06-09T12:08:22.359803+0000","last_clean":"2023-06-09T12:08:22.359803+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:08:22.359803+0000","last_undegraded":"2023-06-09T12:08:22.359803+0000","last_fullsized":"2023-06-09T12:08:22.359803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3488,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476110+0000","last_change":"2023-06-09T12:10:35.734483+0000","last_active":"2023-06-09T12:08:52.476110+0000","last_peered":"2023-06-09T12:08:52.476110+0000","last_clean":"2023-06-09T12:08:52.476110+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:52.476110+0000","last_undegraded":"2023-06-09T12:08:52.476110+0000","last_fullsized":"2023-06-09T12:08:52.476110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423211+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423211+0000","last_peered":"2023-06-09T12:08:42.423211+0000","last_clean":"2023-06-09T12:08:42.423211+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423211+0000","last_undegraded":"2023-06-09T12:08:42.423211+0000","last_fullsized":"2023-06-09T12:08:42.423211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3395,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478850+0000","last_change":"2023-06-09T12:10:26.753614+0000","last_active":"2023-06-09T12:09:12.478850+0000","last_peered":"2023-06-09T12:09:12.478850+0000","last_clean":"2023-06-09T12:09:12.478850+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:09:12.478850+0000","last_undegraded":"2023-06-09T12:09:12.478850+0000","last_fullsized":"2023-06-09T12:09:12.478850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":68898,"ondisk_log_size":68898,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1590184,"num_omap_keys":2984,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30877,"ondisk_log_size":30877,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575570,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":261008,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":81152,"kb_avail":94110832,"statfs":{"total":96636764160,"available":96369491968,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":83099648},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.68200000000000005},"min":{"1min":0.54000000000000004,"5min":0.33700000000000002,"15min":0.29699999999999999},"max":{"1min":0.93000000000000005,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.72299999999999998,"15min":0.70399999999999996},"min":{"1min":0.54000000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.69599999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.56699999999999995,"5min":0.42999999999999999,"15min":0.371},"max":{"1min":0.875,"5min":0.91100000000000003,"15min":1.0229999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.75700000000000001,"15min":0.72399999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.89000000000000001,"5min":0.97599999999999998,"15min":1.05},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.72399999999999998,"15min":0.72799999999999998},"min":{"1min":0.54200000000000004,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.86199999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.75},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.54200000000000004,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.96599999999999997,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:09:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.68999999999999995,"15min":0.68600000000000005},"min":{"1min":0.48299999999999998,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.88400000000000001,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.71699999999999997},"min":{"1min":0.52800000000000002,"5min":0.51700000000000002,"15min":0.441},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.28199999999999997},"max":{"1min":0.83199999999999996,"5min":1.006,"15min":1.006},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.70799999999999996},"min":{"1min":0.505,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":1.0009999999999999,"5min":1.226,"15min":1.226},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59199999999999997,"15min":0.61299999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.624},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.34399999999999997},"max":{"1min":0.84399999999999997,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.46000000000000002,"5min":0.45800000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.73899999999999999},"min":{"1min":0.54700000000000004,"5min":0.501,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":1.1899999999999999,"15min":1.2869999999999999},"last":0.66600000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575570,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213848,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57728,"kb_avail":94157992,"statfs":{"total":96636764160,"available":96417783808,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59113472},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67300000000000004,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69599999999999995,"15min":0.67600000000000005},"min":{"1min":0.56599999999999995,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":1.069},"last":0.55900000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71099999999999997,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.498,"15min":0.312},"max":{"1min":0.85599999999999998,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75800000000000001,"15min":0.745},"min":{"1min":0.62,"5min":0.56899999999999995,"15min":0.43099999999999999},"max":{"1min":0.86299999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.70899999999999996},"min":{"1min":0.625,"5min":0.503,"15min":0.36799999999999999},"max":{"1min":0.91500000000000004,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.752},"min":{"1min":0.64800000000000002,"5min":0.53400000000000003,"15min":0.44800000000000001},"max":{"1min":0.94399999999999995,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.72899999999999998},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.436},"max":{"1min":0.91100000000000003,"5min":1.022,"15min":1.0369999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80100000000000005,"15min":0.78000000000000003},"min":{"1min":0.624,"5min":0.60299999999999998,"15min":0.53100000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0580000000000001},"last":0.85999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.78600000000000003,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.89800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.60499999999999998,"15min":0.65900000000000003},"min":{"1min":0.38800000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.86699999999999999,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.754,"15min":0.68799999999999994},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.052},"last":0.92900000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:09:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.46800000000000003,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.86499999999999999,"5min":0.95199999999999996,"15min":1.1100000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.95499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575571,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399600,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93184,"kb_avail":93972240,"statfs":{"total":96636764160,"available":96227573760,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":95420416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46700000000000003,"15min":0.432},"min":{"1min":0.32200000000000001,"5min":0.25800000000000001,"15min":0.248},"max":{"1min":0.51700000000000002,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.378},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.83799999999999997,"5min":0.85099999999999998,"15min":0.89400000000000002},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.502,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.31,"15min":0.28000000000000003},"max":{"1min":0.56299999999999994,"5min":0.94599999999999995,"15min":0.995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60199999999999998,"15min":0.57399999999999995},"min":{"1min":0.42199999999999999,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.79600000000000004,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69999999999999996,"15min":0.66400000000000003},"min":{"1min":0.52800000000000002,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":0.84899999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.79000000000000004,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":0.95299999999999996,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.876,"15min":0.83899999999999997},"min":{"1min":0.64600000000000002,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.2829999999999999,"5min":1.597,"15min":1.597},"last":1.1040000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:09:54 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.84399999999999997,"15min":0.81599999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.92300000000000004,"5min":1.544,"15min":1.544},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.85999999999999999,"15min":0.84999999999999998},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":0.98399999999999999,"5min":1.704,"15min":1.704},"last":1.0609999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.85499999999999998,"15min":0.83299999999999996},"min":{"1min":0.624,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.1719999999999999,"5min":1.548,"15min":1.548},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.83299999999999996,"15min":0.79300000000000004},"min":{"1min":0.57499999999999996,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.0149999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:09:49 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.90600000000000003,"15min":0.88300000000000001},"min":{"1min":0.67200000000000004,"5min":0.66600000000000004,"15min":0.56299999999999994},"max":{"1min":1.226,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.022},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.86599999999999999,"15min":0.80600000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.125,"5min":1.498,"15min":1.498},"last":1.0880000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575570,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326120,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98304,"kb_avail":94045720,"statfs":{"total":96636764160,"available":96302817280,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":100663296},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47699999999999998,"15min":0.45400000000000001},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.754,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.69799999999999995,"5min":0.749,"15min":0.82699999999999996},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.34999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.503}]},{"osd":3,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.34599999999999997},"max":{"1min":0.871,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.59299999999999997,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60399999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.78400000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1779999999999999,"5min":1.204,"15min":1.204},"last":1.0620000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.016}]},{"osd":5,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.65400000000000003,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.073,"5min":1.121,"15min":1.121},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.90900000000000003,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.085,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.22,"5min":1.232,"15min":1.232},"last":0.90900000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88300000000000001,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.298},"last":1.103},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95599999999999996,"15min":0.91300000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.321,"5min":1.321,"15min":1.351},"last":0.98399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575571,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231328,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51520,"kb_avail":94140512,"statfs":{"total":96636764160,"available":96399884288,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52756480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.32600000000000001,"5min":0.28000000000000003,"15min":0.27300000000000002},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.432,"5min":0.38600000000000001,"15min":0.33000000000000002},"max":{"1min":0.66000000000000003,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.51200000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47799999999999998,"15min":0.46700000000000003},"min":{"1min":0.34300000000000003,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.63800000000000001,"5min":1.236,"15min":1.3109999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.38800000000000001},"max":{"1min":0.75700000000000001,"5min":1.2969999999999999,"15min":1.367},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.60999999999999999,"15min":0.59899999999999998},"min":{"1min":0.40899999999999997,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":0.81100000000000005,"5min":1.4490000000000001,"15min":1.546},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67600000000000005,"15min":0.66200000000000003},"min":{"1min":0.46000000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.85699999999999998,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.63900000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.80200000000000005,"15min":0.77300000000000002},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.373},"max":{"1min":1.079,"5min":1.202,"15min":1.8220000000000001},"last":0.873},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.79700000000000004,"15min":0.78200000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.089,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":1.095}]},{"osd":5,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.83899999999999997,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.601},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.89100000000000001,"15min":0.86899999999999999},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.4670000000000001,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.99299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.82999999999999996,"15min":0.80100000000000005},"min":{"1min":0.60899999999999999,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.034,"5min":1.1160000000000001,"15min":1.3029999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80500000000000005,"15min":0.78200000000000003},"min":{"1min":0.64200000000000002,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.97299999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.67800000000000005,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.87,"15min":0.84599999999999997},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1419999999999999,"5min":1.1879999999999999,"15min":1.3080000000000001},"last":1.034}]}]},{"osd":3,"up_from":13,"seq":55834575570,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432188,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":108992,"kb_avail":93939652,"statfs":{"total":96636764160,"available":96194203648,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":111607808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:47 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.497,"15min":0.48099999999999998},"min":{"1min":0.315,"5min":0.31,"15min":0.223},"max":{"1min":0.81499999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56200000000000006,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.67900000000000005,"5min":0.78600000000000003,"15min":1.5669999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.69799999999999995,"5min":0.79000000000000004,"15min":0.95499999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.877,"15min":1.198},"last":0.496}]},{"osd":4,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.63700000000000001,"5min":0.48599999999999999,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":1.0649999999999999,"15min":1.635},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.78600000000000003},"min":{"1min":0.64300000000000002,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.0289999999999999,"5min":1.171,"15min":3.081},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.81599999999999995},"min":{"1min":0.57399999999999995,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.85999999999999999},"min":{"1min":0.754,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.103,"5min":1.165,"15min":1.367},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.0840000000000001,"5min":1.198,"15min":1.286},"last":0.747},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.81000000000000005},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.026,"5min":1.117,"15min":3.3490000000000002},"last":0.628}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1659999999999999,"5min":1.286,"15min":1.413},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1299999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.83299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575569,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428820,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":136064,"kb_avail":93943020,"statfs":{"total":96636764160,"available":96197652480,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":139329536},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.74299999999999999,"15min":0.71399999999999997},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.75},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.72899999999999998},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":0.92600000000000005,"15min":1.0640000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.34300000000000003},"max":{"1min":0.84699999999999998,"5min":1.0149999999999999,"15min":1.609},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.73799999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.84199999999999997,"5min":0.95899999999999996,"15min":1.407},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.111,"15min":1.4430000000000001},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":1.1719999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64000000000000001,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.81399999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.88400000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.72499999999999998,"5min":0.92300000000000004,"15min":1.1990000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68000000000000005,"5min":0.97399999999999998,"15min":1.361},"last":0.51700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575569,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298840,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81472,"kb_avail":94073000,"statfs":{"total":96636764160,"available":96330752000,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83427328},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70799999999999996,"15min":0.71599999999999997},"min":{"1min":0.45700000000000002,"5min":0.41499999999999998,"15min":0.36599999999999999},"max":{"1min":0.89100000000000001,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.5,"15min":0.40999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73699999999999999,"15min":0.77400000000000002},"min":{"1min":0.49299999999999999,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":1.0129999999999999,"5min":1.167,"15min":19.715},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.73999999999999999,"15min":0.755},"min":{"1min":0.52500000000000002,"5min":0.49099999999999999,"15min":0.42599999999999999},"max":{"1min":0.89900000000000002,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.623}]},{"osd":2,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.48899999999999999,"5min":0.42499999999999999,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":5.5620000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.77900000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.91800000000000004,"5min":1.083,"15min":14.439},"last":0.56000000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73199999999999998,"15min":0.77500000000000002},"min":{"1min":0.56499999999999995,"5min":0.5,"15min":0.433},"max":{"1min":0.90300000000000002,"5min":1.254,"15min":12.130000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81599999999999995,"15min":0.82999999999999996},"min":{"1min":0.69599999999999995,"5min":0.60199999999999998,"15min":0.52300000000000002},"max":{"1min":0.98699999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.80200000000000005},"min":{"1min":0.42299999999999999,"5min":0.39400000000000002,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.56899999999999995,"15min":0.82499999999999996},"min":{"1min":0.46100000000000002,"5min":0.41099999999999998,"15min":0.35199999999999998},"max":{"1min":0.68999999999999995,"5min":0.86599999999999999,"15min":67.521000000000001},"last":0.40300000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.60299999999999998,"15min":0.67100000000000004},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.878,"5min":0.93400000000000005,"15min":12.163},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.61099999999999999,"5min":0.41999999999999998,"15min":0.377},"max":{"1min":0.86099999999999999,"5min":1.095,"15min":7.6040000000000001},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:09:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63100000000000001,"15min":0.66600000000000004},"min":{"1min":0.48599999999999999,"5min":0.438,"15min":0.42399999999999999},"max":{"1min":0.85299999999999998,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.82499999999999996},"min":{"1min":0.57999999999999996,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.88600000000000001,"5min":1.238,"15min":19.834},"last":0.53100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:10:42.752 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.752 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.753 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.753 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.753 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.754 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.754 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.755 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.755 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.755 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.756 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.756 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.757 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.757 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.757 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.758 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.758 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.758 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.759 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.759 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.759 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.760 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.760 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.761 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.761 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.761 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.762 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.762 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.762 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.763 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.763 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:10:42.763 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:10:58.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:10:58.781 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:02.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:11:02.975 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.972+0000 7f6c353ff700 1 Processor -- start 2023-06-09T12:11:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.972+0000 7f6c353ff700 1 -- start start 2023-06-09T12:11:02.976 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.973+0000 7f6c353ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f6c300cb5d0 0x7f6c300b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.973+0000 7f6c353ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c30159b20 con 0x7f6c300cb9a0 2023-06-09T12:11:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.973+0000 7f6c353ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300cac80 con 0x7f6c3015cd40 2023-06-09T12:11:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.973+0000 7f6c353ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c301597d0 con 0x7f6c300cb5d0 2023-06-09T12:11:02.977 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2e7fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6c300cb9a0 0x7f6c30159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53034/0 (socket says 172.21.15.7:53034) 2023-06-09T12:11:02.978 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2e7fc700 1 -- 172.21.15.7:0/4172915989 learned_addr learned my addr 172.21.15.7:0/4172915989 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:11:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814250124 0 0) 0x7f6c30159b20 con 0x7f6c300cb9a0 2023-06-09T12:11:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c180035b0 con 0x7f6c300cb9a0 2023-06-09T12:11:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2effd700 1 --2- 172.21.15.7:0/4172915989 >> v2:172.21.15.117:3300/0 conn(0x7f6c300cb5d0 0x7f6c300b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.980 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 831860744 0 0) 0x7f6c180035b0 con 0x7f6c300cb9a0 2023-06-09T12:11:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 >> v2:172.21.15.117:3300/0 conn(0x7f6c300cb5d0 msgr2=0x7f6c300b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:11:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 --2- 172.21.15.7:0/4172915989 >> v2:172.21.15.117:3300/0 conn(0x7f6c300cb5d0 0x7f6c300b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 >> v1:172.21.15.117:6789/0 conn(0x7f6c3015cd40 legacy=0x7f6c300cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30071ab0 con 0x7f6c300cb9a0 2023-06-09T12:11:02.981 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f6c24001670 con 0x7f6c300cb9a0 2023-06-09T12:11:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 >> v1:172.21.15.7:6789/0 conn(0x7f6c300cb9a0 legacy=0x7f6c30159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.974+0000 7f6c2dffb700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c18003a50 0x7f6c18003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2dffb700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c18004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2dffb700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c18005200 0x7f6c1800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.982 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301597d0 con 0x7f6c18004850 2023-06-09T12:11:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1800c0c0 con 0x7f6c18005200 2023-06-09T12:11:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1800c280 con 0x7f6c18003a50 2023-06-09T12:11:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c18004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2effd700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c18003a50 0x7f6c18003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.983 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2f7fe700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c18005200 0x7f6c1800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 -- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c18003a50 msgr2=0x7f6c18003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:11:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c18003a50 0x7f6c18003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 -- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c18005200 msgr2=0x7f6c1800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:11:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c18005200 0x7f6c1800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.984 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 -- 172.21.15.7:0/4172915989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c180035b0 con 0x7f6c18004850 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2f7fe700 1 -- 172.21.15.7:0/4172915989 reap_dead start 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.975+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c18004cc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c160 tx=0x7f6c18005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ead83098849ccf server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24007b30 con 0x7f6c18004850 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2400cce0 con 0x7f6c18004850 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c2dffb700 1 -- 172.21.15.7:0/4172915989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24005ba0 con 0x7f6c18004850 2023-06-09T12:11:02.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 -- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 msgr2=0x7f6c18004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c18004cc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c160 tx=0x7f6c18005880 comp rx=0 tx=0).stop 2023-06-09T12:11:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 -- 172.21.15.7:0/4172915989 shutdown_connections 2023-06-09T12:11:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 --2- 172.21.15.7:0/4172915989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c18004cc0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 -- 172.21.15.7:0/4172915989 >> 172.21.15.7:0/4172915989 conn(0x7f6c30070af0 msgr2=0x7f6c30070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:02.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.976+0000 7f6c353ff700 1 -- 172.21.15.7:0/4172915989 shutdown_connections 2023-06-09T12:11:02.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.977+0000 7f6c353ff700 1 -- 172.21.15.7:0/4172915989 wait complete. 2023-06-09T12:11:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.978+0000 7f6c353ff700 1 Processor -- start 2023-06-09T12:11:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.978+0000 7f6c353ff700 1 -- start start 2023-06-09T12:11:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 0x7f6c300c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.989 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c3015c430 0x7f6c30153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300ca1a0 con 0x7f6c18004850 2023-06-09T12:11:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c3a30 con 0x7f6c300b1c10 2023-06-09T12:11:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c353ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c3190 con 0x7f6c3015c430 2023-06-09T12:11:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c2effd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40570/0 (socket says 172.21.15.7:40570) 2023-06-09T12:11:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.979+0000 7f6c2effd700 1 -- 172.21.15.7:0/2205617966 learned_addr learned my addr 172.21.15.7:0/2205617966 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:11:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 0x7f6c300c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 -- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c3015c430 msgr2=0x7f6c30153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c3015c430 0x7f6c30153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 -- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 msgr2=0x7f6c300c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 0x7f6c300c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 -- 172.21.15.7:0/2205617966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3015d4f0 con 0x7f6c18004850 2023-06-09T12:11:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 0x7f6c300c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:11:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c2effd700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6c200119d0 tx=0x7f6c20011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:02.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20009630 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20004240 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3015d210 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000f730 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.980+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3015d7d0 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.981+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c30074cf0 con 0x7f6c18004850 2023-06-09T12:11:02.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.982+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6c2000f890 con 0x7f6c18004850 2023-06-09T12:11:02.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.983+0000 7f6c177fe700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 0x7f6c1c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:02.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.984+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f6c2008a3d0 con 0x7f6c18004850 2023-06-09T12:11:02.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.984+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 0x7f6c1c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:02.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.984+0000 7f6c2e7fc700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 0x7f6c1c06d1c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b5380 tx=0x7f6c24006040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:02.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:02.988+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f6c20054b00 con 0x7f6c18004850 2023-06-09T12:11:03.195 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.192+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6c300caa60 con 0x7f6c1c06ad10 2023-06-09T12:11:03.204 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.201+0000 7f6c177fe700 1 -- 172.21.15.7:0/2205617966 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318157 (secure 0 0 0) 0x7f6c300caa60 con 0x7f6c1c06ad10 2023-06-09T12:11:03.208 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:11:03.209 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 msgr2=0x7f6c1c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 0x7f6c1c06d1c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6c300b5380 tx=0x7f6c24006040 comp rx=0 tx=0).stop 2023-06-09T12:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 msgr2=0x7f6c301ef070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6c200119d0 tx=0x7f6c20011d90 comp rx=0 tx=0).stop 2023-06-09T12:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 shutdown_connections 2023-06-09T12:11:03.210 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f6c1c06ad10 0x7f6c1c06d1c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6c3015c430 0x7f6c30153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6c18004850 0x7f6c301ef070 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 --2- 172.21.15.7:0/2205617966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6c300b1c10 0x7f6c300c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 >> 172.21.15.7:0/2205617966 conn(0x7f6c30070af0 msgr2=0x7f6c300b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 shutdown_connections 2023-06-09T12:11:03.211 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:03.206+0000 7f6c353ff700 1 -- 172.21.15.7:0/2205617966 wait complete. 2023-06-09T12:11:03.212 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:11:03.227 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1878,"stamp":"2023-06-09T12:11:03.115589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100341,"ondisk_log_size":100341,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2596936,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":713600,"kb_avail":752377784,"statfs":{"total":773094113280,"available":770434850816,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":730726400},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003018"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3263,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:44.639071+0000","last_change":"2023-06-09T12:09:44.639071+0000","last_active":"2023-06-09T12:09:44.639071+0000","last_peered":"2023-06-09T12:09:44.639071+0000","last_clean":"2023-06-09T12:09:44.639071+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:09:44.639071+0000","last_undegraded":"2023-06-09T12:09:44.639071+0000","last_fullsized":"2023-06-09T12:09:44.639071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:22.437301+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:06:22.437301+0000","last_peered":"2023-06-09T12:06:22.437301+0000","last_clean":"2023-06-09T12:06:22.437301+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:06:22.437301+0000","last_undegraded":"2023-06-09T12:06:22.437301+0000","last_fullsized":"2023-06-09T12:06:22.437301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4205,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:29.205789+0000","last_change":"2023-06-09T12:09:02.372701+0000","last_active":"2023-06-09T12:08:29.205789+0000","last_peered":"2023-06-09T12:08:29.205789+0000","last_clean":"2023-06-09T12:08:29.205789+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:08:29.205789+0000","last_undegraded":"2023-06-09T12:08:29.205789+0000","last_fullsized":"2023-06-09T12:08:29.205789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3464,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:55.633016+0000","last_change":"2023-06-09T12:08:55.633016+0000","last_active":"2023-06-09T12:08:55.633016+0000","last_peered":"2023-06-09T12:08:55.633016+0000","last_clean":"2023-06-09T12:08:55.633016+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:08:55.633016+0000","last_undegraded":"2023-06-09T12:08:55.633016+0000","last_fullsized":"2023-06-09T12:08:55.633016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3742,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.373897+0000","last_change":"2023-06-09T12:10:59.510409+0000","last_active":"2023-06-09T12:09:12.373897+0000","last_peered":"2023-06-09T12:09:12.373897+0000","last_clean":"2023-06-09T12:09:12.373897+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:09:12.373897+0000","last_undegraded":"2023-06-09T12:09:12.373897+0000","last_fullsized":"2023-06-09T12:09:12.373897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3388,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.459638+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:06:47.459638+0000","last_peered":"2023-06-09T12:06:47.459638+0000","last_clean":"2023-06-09T12:06:47.459638+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:06:47.459638+0000","last_undegraded":"2023-06-09T12:06:47.459638+0000","last_fullsized":"2023-06-09T12:06:47.459638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3371,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:55.411595+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:09:55.411595+0000","last_peered":"2023-06-09T12:09:55.411595+0000","last_clean":"2023-06-09T12:09:55.411595+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:09:55.411595+0000","last_undegraded":"2023-06-09T12:09:55.411595+0000","last_fullsized":"2023-06-09T12:09:55.411595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338088+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:06:47.338088+0000","last_peered":"2023-06-09T12:06:47.338088+0000","last_clean":"2023-06-09T12:06:47.338088+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:06:47.338088+0000","last_undegraded":"2023-06-09T12:06:47.338088+0000","last_fullsized":"2023-06-09T12:06:47.338088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:04.274195+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:07:04.274195+0000","last_peered":"2023-06-09T12:07:04.274195+0000","last_clean":"2023-06-09T12:07:04.274195+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:07:04.274195+0000","last_undegraded":"2023-06-09T12:07:04.274195+0000","last_fullsized":"2023-06-09T12:07:04.274195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4090,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:29.777049+0000","last_change":"2023-06-09T12:10:52.493483+0000","last_active":"2023-06-09T12:10:29.777049+0000","last_peered":"2023-06-09T12:10:29.777049+0000","last_clean":"2023-06-09T12:10:29.777049+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:29.777049+0000","last_undegraded":"2023-06-09T12:10:29.777049+0000","last_fullsized":"2023-06-09T12:10:29.777049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":366,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.337752+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:06:27.337752+0000","last_peered":"2023-06-09T12:06:27.337752+0000","last_clean":"2023-06-09T12:06:27.337752+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:06:27.337752+0000","last_undegraded":"2023-06-09T12:06:27.337752+0000","last_fullsized":"2023-06-09T12:06:27.337752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3415,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:40.854512+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:06:40.854512+0000","last_peered":"2023-06-09T12:06:40.854512+0000","last_clean":"2023-06-09T12:06:40.854512+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:06:40.854512+0000","last_undegraded":"2023-06-09T12:06:40.854512+0000","last_fullsized":"2023-06-09T12:06:40.854512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1041,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480454+0000","last_change":"2023-06-09T12:10:27.777932+0000","last_active":"2023-06-09T12:09:22.480454+0000","last_peered":"2023-06-09T12:09:22.480454+0000","last_clean":"2023-06-09T12:09:22.480454+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:09:22.480454+0000","last_undegraded":"2023-06-09T12:09:22.480454+0000","last_fullsized":"2023-06-09T12:09:22.480454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":706,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:02.361562+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:10:02.361562+0000","last_peered":"2023-06-09T12:10:02.361562+0000","last_clean":"2023-06-09T12:10:02.361562+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:10:02.361562+0000","last_undegraded":"2023-06-09T12:10:02.361562+0000","last_fullsized":"2023-06-09T12:10:02.361562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3658,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:25.770236+0000","last_change":"2023-06-09T12:10:25.770236+0000","last_active":"2023-06-09T12:10:25.770236+0000","last_peered":"2023-06-09T12:10:25.770236+0000","last_clean":"2023-06-09T12:10:25.770236+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:10:25.770236+0000","last_undegraded":"2023-06-09T12:10:25.770236+0000","last_fullsized":"2023-06-09T12:10:25.770236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1381,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:42.347283+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:06:42.347283+0000","last_peered":"2023-06-09T12:06:42.347283+0000","last_clean":"2023-06-09T12:06:42.347283+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:06:42.347283+0000","last_undegraded":"2023-06-09T12:06:42.347283+0000","last_fullsized":"2023-06-09T12:06:42.347283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3846,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:42.640933+0000","last_change":"2023-06-09T12:09:42.640933+0000","last_active":"2023-06-09T12:09:42.640933+0000","last_peered":"2023-06-09T12:09:42.640933+0000","last_clean":"2023-06-09T12:09:42.640933+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:09:42.640933+0000","last_undegraded":"2023-06-09T12:09:42.640933+0000","last_fullsized":"2023-06-09T12:09:42.640933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:50.045989+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:06:50.045989+0000","last_peered":"2023-06-09T12:06:50.045989+0000","last_clean":"2023-06-09T12:06:50.045989+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:06:50.045989+0000","last_undegraded":"2023-06-09T12:06:50.045989+0000","last_fullsized":"2023-06-09T12:06:50.045989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1054","reported_seq":3322,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:01.392399+0000","last_change":"2023-06-09T12:10:12.514350+0000","last_active":"2023-06-09T12:11:01.392399+0000","last_peered":"2023-06-09T12:11:01.392399+0000","last_clean":"2023-06-09T12:11:01.392399+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:01.392399+0000","last_undegraded":"2023-06-09T12:11:01.392399+0000","last_fullsized":"2023-06-09T12:11:01.392399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":97,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.339834+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:07:02.339834+0000","last_peered":"2023-06-09T12:07:02.339834+0000","last_clean":"2023-06-09T12:07:02.339834+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:07:02.339834+0000","last_undegraded":"2023-06-09T12:07:02.339834+0000","last_fullsized":"2023-06-09T12:07:02.339834+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1080,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366865+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:07:02.366865+0000","last_peered":"2023-06-09T12:07:02.366865+0000","last_clean":"2023-06-09T12:07:02.366865+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:07:02.366865+0000","last_undegraded":"2023-06-09T12:07:02.366865+0000","last_fullsized":"2023-06-09T12:07:02.366865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3607,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355679+0000","last_change":"2023-06-09T12:10:14.478668+0000","last_active":"2023-06-09T12:09:12.355679+0000","last_peered":"2023-06-09T12:09:12.355679+0000","last_clean":"2023-06-09T12:09:12.355679+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:09:12.355679+0000","last_undegraded":"2023-06-09T12:09:12.355679+0000","last_fullsized":"2023-06-09T12:09:12.355679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.366815+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:07:02.366815+0000","last_peered":"2023-06-09T12:07:02.366815+0000","last_clean":"2023-06-09T12:07:02.366815+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:07:02.366815+0000","last_undegraded":"2023-06-09T12:07:02.366815+0000","last_fullsized":"2023-06-09T12:07:02.366815+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3899,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.355730+0000","last_change":"2023-06-09T12:10:13.485039+0000","last_active":"2023-06-09T12:09:12.355730+0000","last_peered":"2023-06-09T12:09:12.355730+0000","last_clean":"2023-06-09T12:09:12.355730+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:09:12.355730+0000","last_undegraded":"2023-06-09T12:09:12.355730+0000","last_fullsized":"2023-06-09T12:09:12.355730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:02.416114+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:07:02.416114+0000","last_peered":"2023-06-09T12:07:02.416114+0000","last_clean":"2023-06-09T12:07:02.416114+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:07:02.416114+0000","last_undegraded":"2023-06-09T12:07:02.416114+0000","last_fullsized":"2023-06-09T12:07:02.416114+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4138,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:47.338115+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:06:47.338115+0000","last_peered":"2023-06-09T12:06:47.338115+0000","last_clean":"2023-06-09T12:06:47.338115+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:06:47.338115+0000","last_undegraded":"2023-06-09T12:06:47.338115+0000","last_fullsized":"2023-06-09T12:06:47.338115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:52.470014+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:10:52.470014+0000","last_peered":"2023-06-09T12:10:52.470014+0000","last_clean":"2023-06-09T12:10:52.470014+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:10:52.470014+0000","last_undegraded":"2023-06-09T12:10:52.470014+0000","last_fullsized":"2023-06-09T12:10:52.470014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:31.205177+0000","last_change":"2023-06-09T12:09:12.373740+0000","last_active":"2023-06-09T12:08:31.205177+0000","last_peered":"2023-06-09T12:08:31.205177+0000","last_clean":"2023-06-09T12:08:31.205177+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:08:31.205177+0000","last_undegraded":"2023-06-09T12:08:31.205177+0000","last_fullsized":"2023-06-09T12:08:31.205177+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443836+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:07:07.443836+0000","last_peered":"2023-06-09T12:07:07.443836+0000","last_clean":"2023-06-09T12:07:07.443836+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:07:07.443836+0000","last_undegraded":"2023-06-09T12:07:07.443836+0000","last_fullsized":"2023-06-09T12:07:07.443836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4120,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.373846+0000","last_change":"2023-06-09T12:10:58.548220+0000","last_active":"2023-06-09T12:09:12.373846+0000","last_peered":"2023-06-09T12:09:12.373846+0000","last_clean":"2023-06-09T12:09:12.373846+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:09:12.373846+0000","last_undegraded":"2023-06-09T12:09:12.373846+0000","last_fullsized":"2023-06-09T12:09:12.373846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3648,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870217+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870217+0000","last_peered":"2023-06-09T12:09:21.870217+0000","last_clean":"2023-06-09T12:09:21.870217+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870217+0000","last_undegraded":"2023-06-09T12:09:21.870217+0000","last_fullsized":"2023-06-09T12:09:21.870217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":835,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:34.171989+0000","last_change":"2023-06-09T12:09:22.375312+0000","last_active":"2023-06-09T12:08:34.171989+0000","last_peered":"2023-06-09T12:08:34.171989+0000","last_clean":"2023-06-09T12:08:34.171989+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:08:34.171989+0000","last_undegraded":"2023-06-09T12:08:34.171989+0000","last_fullsized":"2023-06-09T12:08:34.171989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3722,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478817+0000","last_change":"2023-06-09T12:10:42.796732+0000","last_active":"2023-06-09T12:09:12.478817+0000","last_peered":"2023-06-09T12:09:12.478817+0000","last_clean":"2023-06-09T12:09:12.478817+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:09:12.478817+0000","last_undegraded":"2023-06-09T12:09:12.478817+0000","last_fullsized":"2023-06-09T12:09:12.478817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":915,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.134944+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.134944+0000","last_peered":"2023-06-09T12:08:55.134944+0000","last_clean":"2023-06-09T12:08:55.134944+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.134944+0000","last_undegraded":"2023-06-09T12:08:55.134944+0000","last_fullsized":"2023-06-09T12:08:55.134944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:07.359525+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:06:07.359525+0000","last_peered":"2023-06-09T12:06:07.359525+0000","last_clean":"2023-06-09T12:06:07.359525+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:06:07.359525+0000","last_undegraded":"2023-06-09T12:06:07.359525+0000","last_fullsized":"2023-06-09T12:06:07.359525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359803+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:08:22.359803+0000","last_peered":"2023-06-09T12:08:22.359803+0000","last_clean":"2023-06-09T12:08:22.359803+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:08:22.359803+0000","last_undegraded":"2023-06-09T12:08:22.359803+0000","last_fullsized":"2023-06-09T12:08:22.359803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3488,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:08:52.476110+0000","last_change":"2023-06-09T12:10:35.734483+0000","last_active":"2023-06-09T12:08:52.476110+0000","last_peered":"2023-06-09T12:08:52.476110+0000","last_clean":"2023-06-09T12:08:52.476110+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:08:52.476110+0000","last_undegraded":"2023-06-09T12:08:52.476110+0000","last_fullsized":"2023-06-09T12:08:52.476110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3943,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:49.110762+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:07:49.110762+0000","last_peered":"2023-06-09T12:07:49.110762+0000","last_clean":"2023-06-09T12:07:49.110762+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:07:49.110762+0000","last_undegraded":"2023-06-09T12:07:49.110762+0000","last_fullsized":"2023-06-09T12:07:49.110762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'936","reported_seq":3395,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.478850+0000","last_change":"2023-06-09T12:10:26.753614+0000","last_active":"2023-06-09T12:09:12.478850+0000","last_peered":"2023-06-09T12:09:12.478850+0000","last_clean":"2023-06-09T12:09:12.478850+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:09:12.478850+0000","last_undegraded":"2023-06-09T12:09:12.478850+0000","last_fullsized":"2023-06-09T12:09:12.478850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":69337,"ondisk_log_size":69337,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575575,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265232,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85376,"kb_avail":94106608,"statfs":{"total":96636764160,"available":96365166592,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87425024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.33700000000000002,"15min":0.32200000000000001},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.44400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":1.202,"15min":1.3320000000000001},"last":0.67000000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.57899999999999996,"5min":0.441,"15min":0.371},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.72899999999999998},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.94699999999999995,"15min":1.05},"last":0.745}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.627},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.90000000000000002,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.58999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.89200000000000002,"15min":1.006},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.55300000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.226,"15min":1.226},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.746,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.496,"5min":0.46000000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":0.875,"15min":1.2230000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.85499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575575,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":213976,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":57856,"kb_avail":94157864,"statfs":{"total":96636764160,"available":96417652736,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59244544},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.28299999999999997},"max":{"1min":0.86899999999999999,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.069},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.69499999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":1.032}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73999999999999999,"15min":0.71199999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.752},"min":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.46400000000000002},"max":{"1min":0.95699999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":1.0549999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80200000000000005,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.048}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.748},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49099999999999999,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77100000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.749,"15min":0.69599999999999995},"min":{"1min":0.624,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.006,"5min":1.024,"15min":1.052},"last":1.002}]},{"osd":7,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.71099999999999997,"15min":0.67500000000000004},"min":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.88900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575576,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399792,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93376,"kb_avail":93972048,"statfs":{"total":96636764160,"available":96227377152,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":95617024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46600000000000003,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72699999999999998,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.34499999999999997,"5min":0.31,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.94599999999999995,"15min":0.995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.81899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.62}]},{"osd":4,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.1040000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.91800000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.93300000000000005,"5min":1.254,"15min":1.544},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.70399999999999996,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1919999999999999,"15min":1.704},"last":0.76900000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.83499999999999996},"min":{"1min":0.67900000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.0620000000000001,"5min":1.548,"15min":1.548},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.79900000000000004},"min":{"1min":0.69199999999999995,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":0.97199999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.88500000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.73199999999999998,"5min":0.66600000000000004,"15min":0.58399999999999996},"max":{"1min":1.0780000000000001,"5min":1.226,"15min":1.6519999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86799999999999999,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.119,"5min":1.498,"15min":1.498},"last":0.71199999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575575,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326312,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98496,"kb_avail":94045528,"statfs":{"total":96636764160,"available":96302620672,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":100859904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47699999999999998,"15min":0.45400000000000001},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.754,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.69799999999999995,"5min":0.749,"15min":0.82699999999999996},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.34999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.442},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.34599999999999997},"max":{"1min":0.871,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.59299999999999997,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.64700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.78400000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1779999999999999,"5min":1.204,"15min":1.204},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":1.0649999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.65400000000000003,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.073,"5min":1.121,"15min":1.121},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.90900000000000003,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.085,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.97899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.22,"5min":1.232,"15min":1.232},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88300000000000001,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.298},"last":1.012},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95599999999999996,"15min":0.91300000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.321,"5min":1.321,"15min":1.351},"last":1.0369999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575576,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231328,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51520,"kb_avail":94140512,"statfs":{"total":96636764160,"available":96399884288,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52756480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.67900000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.623,"5min":1.236,"15min":1.3109999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.2969999999999999,"15min":1.367},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.2709999999999999,"5min":1.4490000000000001,"15min":1.546},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.66800000000000004},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.319,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81200000000000006,"15min":0.78000000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.202,"15min":1.8220000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.095,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.80600000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3740000000000001,"5min":1.4359999999999999,"15min":1.601},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.417,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.84699999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.81799999999999995,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.75800000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85099999999999998,"15min":0.82399999999999995},"min":{"1min":0.745,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1319999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86899999999999999,"15min":0.84799999999999998},"min":{"1min":0.73399999999999999,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.0429999999999999,"5min":1.1879999999999999,"15min":1.262},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575575,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432444,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":109248,"kb_avail":93939396,"statfs":{"total":96636764160,"available":96193941504,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":111869952},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.31,"15min":0.223},"max":{"1min":0.57299999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.87,"5min":0.87,"15min":1.5669999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.69799999999999995,"5min":0.79000000000000004,"15min":0.95499999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.51600000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.877,"15min":1.198},"last":0.503}]},{"osd":4,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.63700000000000001,"5min":0.48599999999999999,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":1.0649999999999999,"15min":1.635},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.78600000000000003},"min":{"1min":0.64300000000000002,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.0289999999999999,"5min":1.171,"15min":3.081},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.81599999999999995},"min":{"1min":0.57399999999999995,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.85999999999999999},"min":{"1min":0.754,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.103,"5min":1.165,"15min":1.367},"last":0.94299999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.0840000000000001,"5min":1.198,"15min":1.286},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.81000000000000005},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.026,"5min":1.117,"15min":3.3490000000000002},"last":0.82899999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1659999999999999,"5min":1.286,"15min":1.413},"last":0.878},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1299999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575574,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":428884,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":136128,"kb_avail":93942956,"statfs":{"total":96636764160,"available":96197586944,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":139395072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.74299999999999999,"15min":0.71399999999999997},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.72899999999999998},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":0.92600000000000005,"15min":1.0640000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.753}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.34300000000000003},"max":{"1min":0.84699999999999998,"5min":1.0149999999999999,"15min":1.609},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.73799999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.84199999999999997,"5min":0.95899999999999996,"15min":1.407},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.111,"15min":1.4430000000000001},"last":0.76100000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":1.1719999999999999},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64000000000000001,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.81399999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.71299999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.72499999999999998,"5min":0.92300000000000004,"15min":1.1990000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68000000000000005,"5min":0.97399999999999998,"15min":1.361},"last":0.58299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575574,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298968,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81600,"kb_avail":94072872,"statfs":{"total":96636764160,"available":96330620928,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83558400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.48099999999999998,"5min":0.41499999999999998,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.84999999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73199999999999998,"15min":0.78100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":1.167,"15min":19.715},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.92600000000000005,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.90300000000000002,"5min":1.083,"15min":14.439},"last":0.748}]},{"osd":3,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.433},"max":{"1min":0.94999999999999996,"5min":1.254,"15min":12.130000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.79700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.82299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":67.521000000000001},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":7.6040000000000001},"last":0.86499999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.876},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.82199999999999995},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.238,"15min":19.834},"last":0.90200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:11:03.234 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.235 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.235 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.236 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.236 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.237 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.237 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.237 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.238 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.238 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.239 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.239 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.239 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.240 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.240 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.240 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.241 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.241 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.242 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.242 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.242 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.243 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.243 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.243 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.244 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.244 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.245 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:03.245 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:11:23.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:11:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.450+0000 7fa3bcfff700 1 Processor -- start 2023-06-09T12:11:23.453 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.450+0000 7fa3bcfff700 1 -- start start 2023-06-09T12:11:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3bcfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3bcfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa3b806a4d0 0x7fa3b80737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3bcfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b815cac0 con 0x7fa3b806e460 2023-06-09T12:11:23.454 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3bcfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b80c8870 con 0x7fa3b806a4d0 2023-06-09T12:11:23.455 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3bcfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80c37d0 con 0x7fa3b80c6280 2023-06-09T12:11:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3b659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.456 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3affff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fa3b806a4d0 0x7fa3b80737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3b659c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:43620/0 (socket says 172.21.15.7:43620) 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3b6d9d700 1 --1- >> v1:172.21.15.117:6789/0 conn(0x7fa3b806e460 0x7fa3b806cf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:43026/0 (socket says 172.21.15.7:43026) 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.452+0000 7fa3b659c700 1 -- 172.21.15.7:0/307967706 learned_addr learned my addr 172.21.15.7:0/307967706 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3affff700 1 -- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 msgr2=0x7fa3b80c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3affff700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3affff700 1 -- 172.21.15.7:0/307967706 >> v1:172.21.15.117:6789/0 conn(0x7fa3b806e460 legacy=0x7fa3b806cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3affff700 1 -- 172.21.15.7:0/307967706 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b815cd30 con 0x7fa3b806a4d0 2023-06-09T12:11:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3b659c700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:11:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3affff700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b806a4d0 0x7fa3b80737f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009840 tx=0x7fa3a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949b7a6d04998e7c server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:23.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3b5d9b700 1 -- 172.21.15.7:0/307967706 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011830 con 0x7fa3b806a4d0 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3b5d9b700 1 -- 172.21.15.7:0/307967706 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a0011990 con 0x7fa3b806a4d0 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.453+0000 7fa3b5d9b700 1 -- 172.21.15.7:0/307967706 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0011d20 con 0x7fa3b806a4d0 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 -- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b806a4d0 msgr2=0x7fa3b80737f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b806a4d0 0x7fa3b80737f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009840 tx=0x7fa3a0009b50 comp rx=0 tx=0).stop 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 -- 172.21.15.7:0/307967706 shutdown_connections 2023-06-09T12:11:23.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b80c6280 0x7fa3b80c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/307967706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b806a4d0 0x7fa3b80737f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 -- 172.21.15.7:0/307967706 >> 172.21.15.7:0/307967706 conn(0x7fa3b806f980 msgr2=0x7fa3b80cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 -- 172.21.15.7:0/307967706 shutdown_connections 2023-06-09T12:11:23.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.454+0000 7fa3bcfff700 1 -- 172.21.15.7:0/307967706 wait complete. 2023-06-09T12:11:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.456+0000 7fa3bcfff700 1 Processor -- start 2023-06-09T12:11:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.456+0000 7fa3bcfff700 1 -- start start 2023-06-09T12:11:23.462 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b806a4d0 0x7fa3b81ef740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b80c6280 0x7fa3b81598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81592b0 con 0x7fa3b806e460 2023-06-09T12:11:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b8157240 con 0x7fa3b80c6280 2023-06-09T12:11:23.463 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3bcfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81e2fd0 con 0x7fa3b806a4d0 2023-06-09T12:11:23.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3affff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42238/0 (socket says 172.21.15.7:42238) 2023-06-09T12:11:23.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3affff700 1 -- 172.21.15.7:0/2591120758 learned_addr learned my addr 172.21.15.7:0/2591120758 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:11:23.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3b659c700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b806a4d0 0x7fa3b81ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.464 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.457+0000 7fa3b6d9d700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b80c6280 0x7fa3b81598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 -- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b806a4d0 msgr2=0x7fa3b81ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b806a4d0 0x7fa3b81ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 -- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b80c6280 msgr2=0x7fa3b81598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b80c6280 0x7fa3b81598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.465 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 -- 172.21.15.7:0/2591120758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8009550 con 0x7fa3b806e460 2023-06-09T12:11:23.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3affff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009fd0 tx=0x7fa3a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:23.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00036a0 con 0x7fa3b806e460 2023-06-09T12:11:23.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a001f070 con 0x7fa3b806e460 2023-06-09T12:11:23.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0003a60 con 0x7fa3b806e460 2023-06-09T12:11:23.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a00094a0 con 0x7fa3b806e460 2023-06-09T12:11:23.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.458+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b815d2f0 con 0x7fa3b806e460 2023-06-09T12:11:23.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.459+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b815d010 con 0x7fa3b806e460 2023-06-09T12:11:23.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.460+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3a0003bc0 con 0x7fa3b806e460 2023-06-09T12:11:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.462+0000 7fa3aeffd700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 0x7fa39806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.462+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fa3a0021070 con 0x7fa3b806e460 2023-06-09T12:11:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.462+0000 7fa3b659c700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 0x7fa39806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:23.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.462+0000 7fa3b659c700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 0x7fa39806d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8006d30 tx=0x7fa3a80092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:23.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.466+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fa3a00559a0 con 0x7fa3b806e460 2023-06-09T12:11:23.673 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.671+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3b815d5d0 con 0x7fa39806ace0 2023-06-09T12:11:23.683 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.681+0000 7fa3aeffd700 1 -- 172.21.15.7:0/2591120758 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318113 (secure 0 0 0) 0x7fa3b815d5d0 con 0x7fa39806ace0 2023-06-09T12:11:23.686 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:11:23.688 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 msgr2=0x7fa39806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 0x7fa39806d190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8006d30 tx=0x7fa3a80092c0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 msgr2=0x7fa3b8158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009fd0 tx=0x7fa3a0004750 comp rx=0 tx=0).stop 2023-06-09T12:11:23.689 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 shutdown_connections 2023-06-09T12:11:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa3b80c6280 0x7fa3b81598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fa39806ace0 0x7fa39806d190 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa3b806a4d0 0x7fa3b81ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 --2- 172.21.15.7:0/2591120758 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3b806e460 0x7fa3b8158d70 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:23.690 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.685+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 >> 172.21.15.7:0/2591120758 conn(0x7fa3b806f980 msgr2=0x7fa3b806aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.686+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 shutdown_connections 2023-06-09T12:11:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:23.686+0000 7fa3bcfff700 1 -- 172.21.15.7:0/2591120758 wait complete. 2023-06-09T12:11:23.691 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:11:23.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1888,"stamp":"2023-06-09T12:11:23.121093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100636,"ondisk_log_size":100636,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2581192,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":697856,"kb_avail":752393528,"statfs":{"total":773094113280,"available":770450972672,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":714604544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:53.792614+0000","last_change":"2023-06-09T12:11:17.496889+0000","last_active":"2023-06-09T12:10:53.792614+0000","last_peered":"2023-06-09T12:10:53.792614+0000","last_clean":"2023-06-09T12:10:53.792614+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:10:53.792614+0000","last_undegraded":"2023-06-09T12:10:53.792614+0000","last_fullsized":"2023-06-09T12:10:53.792614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3702,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.536766+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:07:32.536766+0000","last_peered":"2023-06-09T12:07:32.536766+0000","last_clean":"2023-06-09T12:07:32.536766+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:07:32.536766+0000","last_undegraded":"2023-06-09T12:07:32.536766+0000","last_fullsized":"2023-06-09T12:07:32.536766+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4206,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:02.372701+0000","last_change":"2023-06-09T12:11:06.505322+0000","last_active":"2023-06-09T12:09:02.372701+0000","last_peered":"2023-06-09T12:09:02.372701+0000","last_clean":"2023-06-09T12:09:02.372701+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:09:02.372701+0000","last_undegraded":"2023-06-09T12:09:02.372701+0000","last_fullsized":"2023-06-09T12:09:02.372701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3465,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.795842+0000","last_change":"2023-06-09T12:11:17.496908+0000","last_active":"2023-06-09T12:10:50.795842+0000","last_peered":"2023-06-09T12:10:50.795842+0000","last_clean":"2023-06-09T12:10:50.795842+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:10:50.795842+0000","last_undegraded":"2023-06-09T12:10:50.795842+0000","last_fullsized":"2023-06-09T12:10:50.795842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3742,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:12.373897+0000","last_change":"2023-06-09T12:10:59.510409+0000","last_active":"2023-06-09T12:09:12.373897+0000","last_peered":"2023-06-09T12:09:12.373897+0000","last_clean":"2023-06-09T12:09:12.373897+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:09:12.373897+0000","last_undegraded":"2023-06-09T12:09:12.373897+0000","last_fullsized":"2023-06-09T12:09:12.373897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'931","reported_seq":3373,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:04.464907+0000","last_change":"2023-06-09T12:09:32.502539+0000","last_active":"2023-06-09T12:11:04.464907+0000","last_peered":"2023-06-09T12:11:04.464907+0000","last_clean":"2023-06-09T12:11:04.464907+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:11:04.464907+0000","last_undegraded":"2023-06-09T12:11:04.464907+0000","last_fullsized":"2023-06-09T12:11:04.464907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":931,"log_dups_size":0,"ondisk_log_size":931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4090,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:29.777049+0000","last_change":"2023-06-09T12:10:52.493483+0000","last_active":"2023-06-09T12:10:29.777049+0000","last_peered":"2023-06-09T12:10:29.777049+0000","last_clean":"2023-06-09T12:10:29.777049+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:29.777049+0000","last_undegraded":"2023-06-09T12:10:29.777049+0000","last_fullsized":"2023-06-09T12:10:29.777049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2969,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:55.142523+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:07:55.142523+0000","last_peered":"2023-06-09T12:07:55.142523+0000","last_clean":"2023-06-09T12:07:55.142523+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:07:55.142523+0000","last_undegraded":"2023-06-09T12:07:55.142523+0000","last_fullsized":"2023-06-09T12:07:55.142523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3282,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.094943+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:07:47.094943+0000","last_peered":"2023-06-09T12:07:47.094943+0000","last_clean":"2023-06-09T12:07:47.094943+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:07:47.094943+0000","last_undegraded":"2023-06-09T12:07:47.094943+0000","last_fullsized":"2023-06-09T12:07:47.094943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1041,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.480454+0000","last_change":"2023-06-09T12:10:27.777932+0000","last_active":"2023-06-09T12:09:22.480454+0000","last_peered":"2023-06-09T12:09:22.480454+0000","last_clean":"2023-06-09T12:09:22.480454+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:09:22.480454+0000","last_undegraded":"2023-06-09T12:09:22.480454+0000","last_fullsized":"2023-06-09T12:09:22.480454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.340669+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:06:52.340669+0000","last_peered":"2023-06-09T12:06:52.340669+0000","last_clean":"2023-06-09T12:06:52.340669+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:06:52.340669+0000","last_undegraded":"2023-06-09T12:06:52.340669+0000","last_fullsized":"2023-06-09T12:06:52.340669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":708,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:18.425571+0000","last_change":"2023-06-09T12:09:23.466959+0000","last_active":"2023-06-09T12:11:18.425571+0000","last_peered":"2023-06-09T12:11:18.425571+0000","last_clean":"2023-06-09T12:11:18.425571+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:18.425571+0000","last_undegraded":"2023-06-09T12:11:18.425571+0000","last_fullsized":"2023-06-09T12:11:18.425571+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3658,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:25.770236+0000","last_change":"2023-06-09T12:10:25.770236+0000","last_active":"2023-06-09T12:10:25.770236+0000","last_peered":"2023-06-09T12:10:25.770236+0000","last_clean":"2023-06-09T12:10:25.770236+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:10:25.770236+0000","last_undegraded":"2023-06-09T12:10:25.770236+0000","last_fullsized":"2023-06-09T12:10:25.770236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3786,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:23.428242+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:07:23.428242+0000","last_peered":"2023-06-09T12:07:23.428242+0000","last_clean":"2023-06-09T12:07:23.428242+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:07:23.428242+0000","last_undegraded":"2023-06-09T12:07:23.428242+0000","last_fullsized":"2023-06-09T12:07:23.428242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":728,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.566937+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.566937+0000","last_peered":"2023-06-09T12:09:28.566937+0000","last_clean":"2023-06-09T12:09:28.566937+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.566937+0000","last_undegraded":"2023-06-09T12:09:28.566937+0000","last_fullsized":"2023-06-09T12:09:28.566937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3846,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:42.640933+0000","last_change":"2023-06-09T12:09:42.640933+0000","last_active":"2023-06-09T12:09:42.640933+0000","last_peered":"2023-06-09T12:09:42.640933+0000","last_clean":"2023-06-09T12:09:42.640933+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:09:42.640933+0000","last_undegraded":"2023-06-09T12:09:42.640933+0000","last_fullsized":"2023-06-09T12:09:42.640933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":760,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:54.093801+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:07:54.093801+0000","last_peered":"2023-06-09T12:07:54.093801+0000","last_clean":"2023-06-09T12:07:54.093801+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:07:54.093801+0000","last_undegraded":"2023-06-09T12:07:54.093801+0000","last_fullsized":"2023-06-09T12:07:54.093801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1201","reported_seq":3477,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:22.234311+0000","last_change":"2023-06-09T12:10:12.514350+0000","last_active":"2023-06-09T12:11:22.234311+0000","last_peered":"2023-06-09T12:11:22.234311+0000","last_clean":"2023-06-09T12:11:22.234311+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:22.234311+0000","last_undegraded":"2023-06-09T12:11:22.234311+0000","last_fullsized":"2023-06-09T12:11:22.234311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":244,"log_size":1201,"log_dups_size":0,"ondisk_log_size":1201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 31s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.478668+0000","last_change":"2023-06-09T12:11:17.372631+0000","last_active":"2023-06-09T12:10:14.478668+0000","last_peered":"2023-06-09T12:10:14.478668+0000","last_clean":"2023-06-09T12:10:14.478668+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:10:14.478668+0000","last_undegraded":"2023-06-09T12:10:14.478668+0000","last_fullsized":"2023-06-09T12:10:14.478668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'926","reported_seq":3900,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:13.485039+0000","last_change":"2023-06-09T12:11:17.372667+0000","last_active":"2023-06-09T12:10:13.485039+0000","last_peered":"2023-06-09T12:10:13.485039+0000","last_clean":"2023-06-09T12:10:13.485039+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:10:13.485039+0000","last_undegraded":"2023-06-09T12:10:13.485039+0000","last_fullsized":"2023-06-09T12:10:13.485039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:16.513265+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:11:16.513265+0000","last_peered":"2023-06-09T12:11:16.513265+0000","last_clean":"2023-06-09T12:11:16.513265+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:11:16.513265+0000","last_undegraded":"2023-06-09T12:11:16.513265+0000","last_fullsized":"2023-06-09T12:11:16.513265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:11.540546+0000","last_change":"2023-06-09T12:11:17.389907+0000","last_active":"2023-06-09T12:11:11.540546+0000","last_peered":"2023-06-09T12:11:11.540546+0000","last_clean":"2023-06-09T12:11:11.540546+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:11:11.540546+0000","last_undegraded":"2023-06-09T12:11:11.540546+0000","last_fullsized":"2023-06-09T12:11:11.540546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3420,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820848+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820848+0000","last_peered":"2023-06-09T12:09:31.820848+0000","last_clean":"2023-06-09T12:09:31.820848+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820848+0000","last_undegraded":"2023-06-09T12:09:31.820848+0000","last_fullsized":"2023-06-09T12:09:31.820848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343607+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:07:32.343607+0000","last_peered":"2023-06-09T12:07:32.343607+0000","last_clean":"2023-06-09T12:07:32.343607+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:07:32.343607+0000","last_undegraded":"2023-06-09T12:07:32.343607+0000","last_fullsized":"2023-06-09T12:07:32.343607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'961","reported_seq":4124,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:18.740250+0000","last_change":"2023-06-09T12:11:18.367860+0000","last_active":"2023-06-09T12:11:18.740250+0000","last_peered":"2023-06-09T12:11:18.740250+0000","last_clean":"2023-06-09T12:11:18.740250+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:11:18.740250+0000","last_undegraded":"2023-06-09T12:11:18.740250+0000","last_fullsized":"2023-06-09T12:11:18.740250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":961,"log_dups_size":0,"ondisk_log_size":961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:53.051874+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:07:53.051874+0000","last_peered":"2023-06-09T12:07:53.051874+0000","last_clean":"2023-06-09T12:07:53.051874+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:07:53.051874+0000","last_undegraded":"2023-06-09T12:07:53.051874+0000","last_fullsized":"2023-06-09T12:07:53.051874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":836,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:09:22.375312+0000","last_change":"2023-06-09T12:11:17.403692+0000","last_active":"2023-06-09T12:09:22.375312+0000","last_peered":"2023-06-09T12:09:22.375312+0000","last_clean":"2023-06-09T12:09:22.375312+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:09:22.375312+0000","last_undegraded":"2023-06-09T12:09:22.375312+0000","last_fullsized":"2023-06-09T12:09:22.375312+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3723,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.796732+0000","last_change":"2023-06-09T12:11:17.496741+0000","last_active":"2023-06-09T12:10:42.796732+0000","last_peered":"2023-06-09T12:10:42.796732+0000","last_clean":"2023-06-09T12:10:42.796732+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:10:42.796732+0000","last_undegraded":"2023-06-09T12:10:42.796732+0000","last_fullsized":"2023-06-09T12:10:42.796732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3829,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:27.456975+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:06:27.456975+0000","last_peered":"2023-06-09T12:06:27.456975+0000","last_clean":"2023-06-09T12:06:27.456975+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:06:27.456975+0000","last_undegraded":"2023-06-09T12:06:27.456975+0000","last_fullsized":"2023-06-09T12:06:27.456975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359803+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:08:22.359803+0000","last_peered":"2023-06-09T12:08:22.359803+0000","last_clean":"2023-06-09T12:08:22.359803+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:08:22.359803+0000","last_undegraded":"2023-06-09T12:08:22.359803+0000","last_fullsized":"2023-06-09T12:08:22.359803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3154,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:48.112027+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:07:48.112027+0000","last_peered":"2023-06-09T12:07:48.112027+0000","last_clean":"2023-06-09T12:07:48.112027+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:07:48.112027+0000","last_undegraded":"2023-06-09T12:07:48.112027+0000","last_fullsized":"2023-06-09T12:07:48.112027+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3489,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:35.734483+0000","last_change":"2023-06-09T12:11:17.496694+0000","last_active":"2023-06-09T12:10:35.734483+0000","last_peered":"2023-06-09T12:10:35.734483+0000","last_clean":"2023-06-09T12:10:35.734483+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:10:35.734483+0000","last_undegraded":"2023-06-09T12:10:35.734483+0000","last_fullsized":"2023-06-09T12:10:35.734483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1084","reported_seq":3554,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:20.455949+0000","last_change":"2023-06-09T12:10:26.753614+0000","last_active":"2023-06-09T12:11:20.455949+0000","last_peered":"2023-06-09T12:11:20.455949+0000","last_clean":"2023-06-09T12:11:20.455949+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:20.455949+0000","last_undegraded":"2023-06-09T12:11:20.455949+0000","last_fullsized":"2023-06-09T12:11:20.455949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":148,"log_size":1084,"log_dups_size":0,"ondisk_log_size":1084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2985,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:51.130097+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:07:51.130097+0000","last_peered":"2023-06-09T12:07:51.130097+0000","last_clean":"2023-06-09T12:07:51.130097+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:07:51.130097+0000","last_undegraded":"2023-06-09T12:07:51.130097+0000","last_fullsized":"2023-06-09T12:07:51.130097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":69632,"ondisk_log_size":69632,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575579,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265232,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85376,"kb_avail":94106608,"statfs":{"total":96636764160,"available":96365166592,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87425024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.33700000000000002,"15min":0.32200000000000001},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.44400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":1.202,"15min":1.3320000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.57899999999999996,"5min":0.441,"15min":0.371},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.72899999999999998},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.94699999999999995,"15min":1.05},"last":0.65300000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.90000000000000002,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.89200000000000002,"15min":1.006},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.55300000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.226,"15min":1.226},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.746,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.53800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.496,"5min":0.46000000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":0.875,"15min":1.2230000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":1.075}]}]},{"osd":6,"up_from":13,"seq":55834575579,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214168,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58048,"kb_avail":94157672,"statfs":{"total":96636764160,"available":96417456128,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59441152},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.28299999999999997},"max":{"1min":0.86899999999999999,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.629},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.069},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.69499999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73999999999999999,"15min":0.71199999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.752},"min":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.46400000000000002},"max":{"1min":0.95699999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80200000000000005,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69599999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49099999999999999,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.749,"15min":0.69599999999999995},"min":{"1min":0.624,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.006,"5min":1.024,"15min":1.052},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.71099999999999997,"15min":0.67500000000000004},"min":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.73199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575580,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399984,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93568,"kb_avail":93971856,"statfs":{"total":96636764160,"available":96227180544,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":95813632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46600000000000003,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72699999999999998,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.379},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.34499999999999997,"5min":0.31,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.94599999999999995,"15min":0.995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.81899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63}]},{"osd":3,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.747}]},{"osd":4,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.1040000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.93300000000000005,"5min":1.254,"15min":1.544},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.70399999999999996,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1919999999999999,"15min":1.704},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.83499999999999996},"min":{"1min":0.67900000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.0620000000000001,"5min":1.548,"15min":1.548},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.79900000000000004},"min":{"1min":0.69199999999999995,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":0.97199999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.73199999999999998,"5min":0.66600000000000004,"15min":0.58399999999999996},"max":{"1min":1.0780000000000001,"5min":1.226,"15min":1.6519999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.876,"5min":0.86799999999999999,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.119,"5min":1.498,"15min":1.498},"last":0.72599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575579,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326312,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98496,"kb_avail":94045528,"statfs":{"total":96636764160,"available":96302620672,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":100859904},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.47699999999999998,"15min":0.45400000000000001},"min":{"1min":0.33500000000000002,"5min":0.317,"15min":0.247},"max":{"1min":0.754,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.69799999999999995,"5min":0.749,"15min":0.82699999999999996},"last":0.435}]},{"osd":2,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.41099999999999998,"5min":0.40100000000000002,"15min":0.34999999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.40799999999999997,"5min":0.34999999999999998,"15min":0.27400000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58099999999999996,"15min":0.57099999999999995},"min":{"1min":0.47299999999999998,"5min":0.41399999999999998,"15min":0.34599999999999997},"max":{"1min":0.871,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.499},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.59299999999999997,"5min":0.52500000000000002,"15min":0.48099999999999998},"max":{"1min":0.99199999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.626}]},{"osd":4,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.85199999999999998,"15min":0.78400000000000003},"min":{"1min":0.53700000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.1779999999999999,"5min":1.204,"15min":1.204},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.89900000000000002,"15min":0.84499999999999997},"min":{"1min":0.66200000000000003,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.79600000000000004,"15min":0.77500000000000002},"min":{"1min":0.65400000000000003,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.073,"5min":1.121,"15min":1.121},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.90900000000000003,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.085,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.80200000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81499999999999995,"15min":0.81899999999999995},"min":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.137,"5min":1.2330000000000001,"15min":9.0790000000000006},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85699999999999998,"15min":0.83999999999999997},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.22,"5min":1.232,"15min":1.232},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:10:31 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.88300000000000001,"15min":0.83799999999999997},"min":{"1min":0.64000000000000001,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.298},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95599999999999996,"15min":0.91300000000000003},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.321,"5min":1.321,"15min":1.351},"last":0.89500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575580,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231328,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51520,"kb_avail":94140512,"statfs":{"total":96636764160,"available":96399884288,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52756480},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.67900000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.623,"5min":1.236,"15min":1.3109999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.2969999999999999,"15min":1.367},"last":0.69799999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.2709999999999999,"5min":1.4490000000000001,"15min":1.546},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.66800000000000004},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.319,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81200000000000006,"15min":0.78000000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.202,"15min":1.8220000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.095,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3740000000000001,"5min":1.4359999999999999,"15min":1.601},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.417,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":1.3899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.81799999999999995,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.871}]},{"osd":7,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85099999999999998,"15min":0.82399999999999995},"min":{"1min":0.745,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1319999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86899999999999999,"15min":0.84799999999999998},"min":{"1min":0.73399999999999999,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.0429999999999999,"5min":1.1879999999999999,"15min":1.262},"last":0.99399999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575579,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432572,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":109376,"kb_avail":93939268,"statfs":{"total":96636764160,"available":96193810432,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112001024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.31,"15min":0.223},"max":{"1min":0.57299999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.496},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.87,"5min":0.87,"15min":1.5669999999999999},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52100000000000002,"15min":0.51400000000000001},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.27100000000000002},"max":{"1min":0.69799999999999995,"5min":0.79000000000000004,"15min":0.95499999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.57699999999999996,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.379,"15min":0.28599999999999998},"max":{"1min":0.80300000000000005,"5min":1.0600000000000001,"15min":1.4970000000000001},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64900000000000002,"15min":0.64500000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.81599999999999995,"5min":0.86099999999999999,"15min":1.355},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.56299999999999994},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.29799999999999999},"max":{"1min":0.82599999999999996,"5min":0.877,"15min":1.198},"last":0.38900000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.79800000000000004,"15min":0.77200000000000002},"min":{"1min":0.63700000000000001,"5min":0.48599999999999999,"15min":0.39400000000000002},"max":{"1min":0.93000000000000005,"5min":1.0649999999999999,"15min":1.635},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.78600000000000003},"min":{"1min":0.64300000000000002,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.0289999999999999,"5min":1.171,"15min":3.081},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84599999999999997,"15min":0.81599999999999995},"min":{"1min":0.57399999999999995,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0149999999999999,"5min":1.1160000000000001,"15min":1.2110000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90100000000000002,"15min":0.85999999999999999},"min":{"1min":0.754,"5min":0.64500000000000002,"15min":0.47899999999999998},"max":{"1min":1.103,"5min":1.165,"15min":1.367},"last":0.82499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.80200000000000005},"min":{"1min":0.55400000000000005,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.0840000000000001,"5min":1.198,"15min":1.286},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.81000000000000005},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.026,"5min":1.117,"15min":3.3490000000000002},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.83899999999999997,"15min":0.82599999999999996},"min":{"1min":0.58799999999999997,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1659999999999999,"5min":1.286,"15min":1.413},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.61599999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1299999999999999,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.70299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575578,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412628,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":119872,"kb_avail":93959212,"statfs":{"total":96636764160,"available":96214233088,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":122748928},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.74299999999999999,"15min":0.71399999999999997},"min":{"1min":0.38600000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.419,"5min":1.419,"15min":1.419},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46200000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.879}]},{"osd":1,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71099999999999997,"15min":0.72899999999999998},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.85599999999999998,"5min":0.92600000000000005,"15min":1.0640000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77300000000000002,"15min":0.77200000000000002},"min":{"1min":0.54900000000000004,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.94199999999999995,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.96999999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71999999999999997,"15min":0.69999999999999996},"min":{"1min":0.55400000000000005,"5min":0.44900000000000001,"15min":0.34300000000000003},"max":{"1min":0.84699999999999998,"5min":1.0149999999999999,"15min":1.609},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78500000000000003,"15min":0.73799999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.96599999999999997,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.67300000000000004,"15min":0.68100000000000005},"min":{"1min":0.51500000000000001,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.84199999999999997,"5min":0.95899999999999996,"15min":1.407},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.499,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.89700000000000002,"5min":1.111,"15min":1.4430000000000001},"last":0.71099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.38900000000000001,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.623},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.443,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":0.79800000000000004,"5min":0.83899999999999997,"15min":1.1719999999999999},"last":1.0069999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.55700000000000005,"15min":0.57299999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.64000000000000001,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.505},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.73099999999999998,"15min":0.71999999999999997},"min":{"1min":0.55100000000000005,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.81399999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:10:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.64700000000000002,"15min":0.63100000000000001},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.72499999999999998,"5min":0.92300000000000004,"15min":1.1990000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.63700000000000001},"min":{"1min":0.46500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68000000000000005,"5min":0.97399999999999998,"15min":1.361},"last":0.54400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575578,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298968,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81600,"kb_avail":94072872,"statfs":{"total":96636764160,"available":96330620928,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83558400},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.48099999999999998,"5min":0.41499999999999998,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73199999999999998,"15min":0.78100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":1.167,"15min":19.715},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.92600000000000005,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.90300000000000002,"5min":1.083,"15min":14.439},"last":0.61599999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.433},"max":{"1min":0.94999999999999996,"5min":1.254,"15min":12.130000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.79800000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.79700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.82299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":67.521000000000001},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":7.6040000000000001},"last":0.82299999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.82199999999999995},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.238,"15min":19.834},"last":0.86399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:11:23.714 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.715 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.715 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.716 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.716 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.717 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.717 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.717 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.718 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.718 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.719 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.719 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.719 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.720 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.720 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.721 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.721 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.721 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.722 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.722 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.722 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.723 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.723 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.724 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.724 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.724 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.725 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:23.725 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:11:28.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:28.889 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:43.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:11:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.935+0000 7f775ffff700 1 Processor -- start 2023-06-09T12:11:43.938 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.935+0000 7f775ffff700 1 -- start start 2023-06-09T12:11:43.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.936+0000 7f775ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.939 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.936+0000 7f775ffff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f7758071060 0x7f7758158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.936+0000 7f775ffff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77581580a0 con 0x7f7758151510 2023-06-09T12:11:43.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.936+0000 7f775ffff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580714e0 con 0x7f7758071060 2023-06-09T12:11:43.940 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.936+0000 7f775ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77581518e0 con 0x7f77580c5b40 2023-06-09T12:11:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:36248/0 (socket says 172.21.15.7:36248) 2023-06-09T12:11:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 -- 172.21.15.7:0/3733568639 learned_addr learned my addr 172.21.15.7:0/3733568639 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:11:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775dd9b700 1 --1- 172.21.15.7:0/3733568639 >> v1:172.21.15.117:6789/0 conn(0x7f7758151510 0x7f775815b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.117:6789/0 says I am v1:172.21.15.7:42908/0 (socket says 172.21.15.7:42908) 2023-06-09T12:11:43.942 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775e59c700 1 --2- 172.21.15.7:0/3733568639 >> v2:172.21.15.117:3300/0 conn(0x7f7758071060 0x7f7758158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775cd99700 1 -- 172.21.15.7:0/3733568639 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158933088 0 0) 0x7f77581580a0 con 0x7f7758151510 2023-06-09T12:11:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775cd99700 1 -- 172.21.15.7:0/3733568639 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77440035b0 con 0x7f7758151510 2023-06-09T12:11:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 -- 172.21.15.7:0/3733568639 >> v2:172.21.15.117:3300/0 conn(0x7f7758071060 msgr2=0x7f7758158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:11:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 --2- 172.21.15.7:0/3733568639 >> v2:172.21.15.117:3300/0 conn(0x7f7758071060 0x7f7758158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:43.943 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 -- 172.21.15.7:0/3733568639 >> v1:172.21.15.117:6789/0 conn(0x7f7758151510 legacy=0x7f775815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.937+0000 7f775d59a700 1 -- 172.21.15.7:0/3733568639 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580c6180 con 0x7f77580c5b40 2023-06-09T12:11:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775d59a700 1 --2- 172.21.15.7:0/3733568639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77480098f0 tx=0x7f7748009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db222f71d62d019 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775cd99700 1 -- 172.21.15.7:0/3733568639 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011830 con 0x7f77580c5b40 2023-06-09T12:11:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775cd99700 1 -- 172.21.15.7:0/3733568639 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7748011990 con 0x7f77580c5b40 2023-06-09T12:11:43.944 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775cd99700 1 -- 172.21.15.7:0/3733568639 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748011d20 con 0x7f77580c5b40 2023-06-09T12:11:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775ffff700 1 -- 172.21.15.7:0/3733568639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 msgr2=0x7f7758070b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.938+0000 7f775ffff700 1 --2- 172.21.15.7:0/3733568639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77480098f0 tx=0x7f7748009c00 comp rx=0 tx=0).stop 2023-06-09T12:11:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 -- 172.21.15.7:0/3733568639 shutdown_connections 2023-06-09T12:11:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 --2- 172.21.15.7:0/3733568639 >> v2:172.21.15.117:3300/0 conn(0x7f7758071060 0x7f7758158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:43.945 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 --2- 172.21.15.7:0/3733568639 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f77580c5b40 0x7f7758070b20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 -- 172.21.15.7:0/3733568639 >> 172.21.15.7:0/3733568639 conn(0x7f775806a4d0 msgr2=0x7f775806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 -- 172.21.15.7:0/3733568639 shutdown_connections 2023-06-09T12:11:43.946 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.939+0000 7f775ffff700 1 -- 172.21.15.7:0/3733568639 wait complete. 2023-06-09T12:11:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.940+0000 7f775ffff700 1 Processor -- start 2023-06-09T12:11:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.940+0000 7f775ffff700 1 -- start start 2023-06-09T12:11:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7758071060 0x7f7758158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.948 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7758151510 0x7f77580c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7758159380 con 0x7f77580c5b40 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77580c2120 con 0x7f7758151510 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.941+0000 7f775ffff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7758159580 con 0x7f7758071060 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.949 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57866/0 (socket says 172.21.15.7:57866) 2023-06-09T12:11:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 -- 172.21.15.7:0/2881070145 learned_addr learned my addr 172.21.15.7:0/2881070145 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:11:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775dd9b700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7758071060 0x7f7758158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775e59c700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7758151510 0x7f77580c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 -- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7758071060 msgr2=0x7f7758158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:43.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7758071060 0x7f7758158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 -- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7758151510 msgr2=0x7f77580c18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7758151510 0x7f77580c18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 -- 172.21.15.7:0/2881070145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7748009550 con 0x7f77580c5b40 2023-06-09T12:11:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.942+0000 7f775d59a700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7748011de0 tx=0x7f7748004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:43.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.943+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7748004480 con 0x7f77580c5b40 2023-06-09T12:11:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.943+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774801f070 con 0x7f77580c5b40 2023-06-09T12:11:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.943+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77480038e0 con 0x7f77580c5b40 2023-06-09T12:11:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.943+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758157770 con 0x7f77580c5b40 2023-06-09T12:11:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.943+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7758157490 con 0x7f77580c5b40 2023-06-09T12:11:43.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.944+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7758157a50 con 0x7f77580c5b40 2023-06-09T12:11:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.945+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7748021030 con 0x7f77580c5b40 2023-06-09T12:11:43.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.946+0000 7f774e7fc700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 0x7f773c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:11:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.946+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f774808b290 con 0x7f77580c5b40 2023-06-09T12:11:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.946+0000 7f775dd9b700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 0x7f773c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:11:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.947+0000 7f775dd9b700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 0x7f773c06d1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f77540011e0 tx=0x7f7754007c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:11:43.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:43.950+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7748055a50 con 0x7f77580c5b40 2023-06-09T12:11:44.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.157+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7758074cf0 con 0x7f773c06ad10 2023-06-09T12:11:44.169 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.167+0000 7f774e7fc700 1 -- 172.21.15.7:0/2881070145 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318125 (secure 0 0 0) 0x7f7758074cf0 con 0x7f773c06ad10 2023-06-09T12:11:44.173 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:11:44.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 msgr2=0x7f773c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:44.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 0x7f773c06d1c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f77540011e0 tx=0x7f7754007c00 comp rx=0 tx=0).stop 2023-06-09T12:11:44.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 msgr2=0x7f77580c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:11:44.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f7748011de0 tx=0x7f7748004750 comp rx=0 tx=0).stop 2023-06-09T12:11:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 shutdown_connections 2023-06-09T12:11:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7758151510 0x7f77580c18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f773c06ad10 0x7f773c06d1c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:44.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7758071060 0x7f7758158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 --2- 172.21.15.7:0/2881070145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77580c5b40 0x7f77580c1380 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:11:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 >> 172.21.15.7:0/2881070145 conn(0x7f775806a4d0 msgr2=0x7f775814adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:11:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 shutdown_connections 2023-06-09T12:11:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:11:44.171+0000 7f775ffff700 1 -- 172.21.15.7:0/2881070145 wait complete. 2023-06-09T12:11:44.177 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:11:44.193 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1898,"stamp":"2023-06-09T12:11:43.126870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100956,"ondisk_log_size":100956,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2581896,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":698560,"kb_avail":752392824,"statfs":{"total":773094113280,"available":770450251776,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":715325440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003221"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:53.792614+0000","last_change":"2023-06-09T12:11:17.496889+0000","last_active":"2023-06-09T12:10:53.792614+0000","last_peered":"2023-06-09T12:10:53.792614+0000","last_clean":"2023-06-09T12:10:53.792614+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:10:53.792614+0000","last_undegraded":"2023-06-09T12:10:53.792614+0000","last_fullsized":"2023-06-09T12:10:53.792614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3494,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:52.365591+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:06:52.365591+0000","last_peered":"2023-06-09T12:06:52.365591+0000","last_clean":"2023-06-09T12:06:52.365591+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:06:52.365591+0000","last_undegraded":"2023-06-09T12:06:52.365591+0000","last_fullsized":"2023-06-09T12:06:52.365591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:06.505322+0000","last_change":"2023-06-09T12:11:27.391425+0000","last_active":"2023-06-09T12:11:06.505322+0000","last_peered":"2023-06-09T12:11:06.505322+0000","last_clean":"2023-06-09T12:11:06.505322+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:11:06.505322+0000","last_undegraded":"2023-06-09T12:11:06.505322+0000","last_fullsized":"2023-06-09T12:11:06.505322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3465,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.795842+0000","last_change":"2023-06-09T12:11:17.496908+0000","last_active":"2023-06-09T12:10:50.795842+0000","last_peered":"2023-06-09T12:10:50.795842+0000","last_clean":"2023-06-09T12:10:50.795842+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:10:50.795842+0000","last_undegraded":"2023-06-09T12:10:50.795842+0000","last_fullsized":"2023-06-09T12:10:50.795842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3533,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.368124+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:07:12.368124+0000","last_peered":"2023-06-09T12:07:12.368124+0000","last_clean":"2023-06-09T12:07:12.368124+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:07:12.368124+0000","last_undegraded":"2023-06-09T12:07:12.368124+0000","last_fullsized":"2023-06-09T12:07:12.368124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.445816+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:07:22.445816+0000","last_peered":"2023-06-09T12:07:22.445816+0000","last_clean":"2023-06-09T12:07:22.445816+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:07:22.445816+0000","last_undegraded":"2023-06-09T12:07:22.445816+0000","last_fullsized":"2023-06-09T12:07:22.445816+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:22.464494+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:07:22.464494+0000","last_peered":"2023-06-09T12:07:22.464494+0000","last_clean":"2023-06-09T12:07:22.464494+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:07:22.464494+0000","last_undegraded":"2023-06-09T12:07:22.464494+0000","last_fullsized":"2023-06-09T12:07:22.464494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:59.510409+0000","last_change":"2023-06-09T12:11:32.392186+0000","last_active":"2023-06-09T12:10:59.510409+0000","last_peered":"2023-06-09T12:10:59.510409+0000","last_clean":"2023-06-09T12:10:59.510409+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:10:59.510409+0000","last_undegraded":"2023-06-09T12:10:59.510409+0000","last_fullsized":"2023-06-09T12:10:59.510409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'955","reported_seq":3404,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:40.495187+0000","last_change":"2023-06-09T12:11:38.262317+0000","last_active":"2023-06-09T12:11:40.495187+0000","last_peered":"2023-06-09T12:11:40.495187+0000","last_clean":"2023-06-09T12:11:40.495187+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:11:40.495187+0000","last_undegraded":"2023-06-09T12:11:40.495187+0000","last_fullsized":"2023-06-09T12:11:40.495187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":955,"log_dups_size":0,"ondisk_log_size":955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:32.343645+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:07:32.343645+0000","last_peered":"2023-06-09T12:07:32.343645+0000","last_clean":"2023-06-09T12:07:32.343645+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:07:32.343645+0000","last_undegraded":"2023-06-09T12:07:32.343645+0000","last_fullsized":"2023-06-09T12:07:32.343645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4090,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:29.777049+0000","last_change":"2023-06-09T12:10:52.493483+0000","last_active":"2023-06-09T12:10:29.777049+0000","last_peered":"2023-06-09T12:10:29.777049+0000","last_clean":"2023-06-09T12:10:29.777049+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:29.777049+0000","last_undegraded":"2023-06-09T12:10:29.777049+0000","last_fullsized":"2023-06-09T12:10:29.777049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2027,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350703+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:07:12.350703+0000","last_peered":"2023-06-09T12:07:12.350703+0000","last_clean":"2023-06-09T12:07:12.350703+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:07:12.350703+0000","last_undegraded":"2023-06-09T12:07:12.350703+0000","last_fullsized":"2023-06-09T12:07:12.350703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365982+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:09:12.365982+0000","last_peered":"2023-06-09T12:09:12.365982+0000","last_clean":"2023-06-09T12:09:12.365982+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:09:12.365982+0000","last_undegraded":"2023-06-09T12:09:12.365982+0000","last_fullsized":"2023-06-09T12:09:12.365982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1042,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.777932+0000","last_change":"2023-06-09T12:11:42.500508+0000","last_active":"2023-06-09T12:10:27.777932+0000","last_peered":"2023-06-09T12:10:27.777932+0000","last_clean":"2023-06-09T12:10:27.777932+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:10:27.777932+0000","last_undegraded":"2023-06-09T12:10:27.777932+0000","last_fullsized":"2023-06-09T12:10:27.777932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3395,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352121+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352121+0000","last_peered":"2023-06-09T12:09:40.352121+0000","last_clean":"2023-06-09T12:09:40.352121+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352121+0000","last_undegraded":"2023-06-09T12:09:40.352121+0000","last_fullsized":"2023-06-09T12:09:40.352121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:07.443784+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:07:07.443784+0000","last_peered":"2023-06-09T12:07:07.443784+0000","last_clean":"2023-06-09T12:07:07.443784+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:07:07.443784+0000","last_undegraded":"2023-06-09T12:07:07.443784+0000","last_fullsized":"2023-06-09T12:07:07.443784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:06:58.095827+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:06:58.095827+0000","last_peered":"2023-06-09T12:06:58.095827+0000","last_clean":"2023-06-09T12:06:58.095827+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:06:58.095827+0000","last_undegraded":"2023-06-09T12:06:58.095827+0000","last_fullsized":"2023-06-09T12:06:58.095827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4048,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:12.350800+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:07:12.350800+0000","last_peered":"2023-06-09T12:07:12.350800+0000","last_clean":"2023-06-09T12:07:12.350800+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:07:12.350800+0000","last_undegraded":"2023-06-09T12:07:12.350800+0000","last_fullsized":"2023-06-09T12:07:12.350800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":711,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.328419+0000","last_change":"2023-06-09T12:11:37.475265+0000","last_active":"2023-06-09T12:11:35.328419+0000","last_peered":"2023-06-09T12:11:35.328419+0000","last_clean":"2023-06-09T12:11:35.328419+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:35.328419+0000","last_undegraded":"2023-06-09T12:11:35.328419+0000","last_fullsized":"2023-06-09T12:11:35.328419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3659,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:01.674317+0000","last_change":"2023-06-09T12:11:32.498998+0000","last_active":"2023-06-09T12:11:01.674317+0000","last_peered":"2023-06-09T12:11:01.674317+0000","last_clean":"2023-06-09T12:11:01.674317+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:11:01.674317+0000","last_undegraded":"2023-06-09T12:11:01.674317+0000","last_fullsized":"2023-06-09T12:11:01.674317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":634,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:35.010734+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:07:35.010734+0000","last_peered":"2023-06-09T12:07:35.010734+0000","last_clean":"2023-06-09T12:07:35.010734+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:07:35.010734+0000","last_undegraded":"2023-06-09T12:07:35.010734+0000","last_fullsized":"2023-06-09T12:07:35.010734+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:51.791731+0000","last_change":"2023-06-09T12:11:37.499717+0000","last_active":"2023-06-09T12:10:51.791731+0000","last_peered":"2023-06-09T12:10:51.791731+0000","last_clean":"2023-06-09T12:10:51.791731+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:10:51.791731+0000","last_undegraded":"2023-06-09T12:10:51.791731+0000","last_fullsized":"2023-06-09T12:10:51.791731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3457,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:17.463873+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:07:17.463873+0000","last_peered":"2023-06-09T12:07:17.463873+0000","last_clean":"2023-06-09T12:07:17.463873+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:07:17.463873+0000","last_undegraded":"2023-06-09T12:07:17.463873+0000","last_fullsized":"2023-06-09T12:07:17.463873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.478668+0000","last_change":"2023-06-09T12:11:17.372631+0000","last_active":"2023-06-09T12:10:14.478668+0000","last_peered":"2023-06-09T12:10:14.478668+0000","last_clean":"2023-06-09T12:10:14.478668+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:10:14.478668+0000","last_undegraded":"2023-06-09T12:10:14.478668+0000","last_fullsized":"2023-06-09T12:10:14.478668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'950","reported_seq":3929,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:38.601943+0000","last_change":"2023-06-09T12:11:36.721617+0000","last_active":"2023-06-09T12:11:38.601943+0000","last_peered":"2023-06-09T12:11:38.601943+0000","last_clean":"2023-06-09T12:11:38.601943+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:11:38.601943+0000","last_undegraded":"2023-06-09T12:11:38.601943+0000","last_fullsized":"2023-06-09T12:11:38.601943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":950,"log_dups_size":0,"ondisk_log_size":950,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1290,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.346003+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:07:52.346003+0000","last_peered":"2023-06-09T12:07:52.346003+0000","last_clean":"2023-06-09T12:07:52.346003+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:07:52.346003+0000","last_undegraded":"2023-06-09T12:07:52.346003+0000","last_fullsized":"2023-06-09T12:07:52.346003+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":320,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.449691+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:07:57.449691+0000","last_peered":"2023-06-09T12:07:57.449691+0000","last_clean":"2023-06-09T12:07:57.449691+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:07:57.449691+0000","last_undegraded":"2023-06-09T12:07:57.449691+0000","last_fullsized":"2023-06-09T12:07:57.449691+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3591,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.538486+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:07:47.538486+0000","last_peered":"2023-06-09T12:07:47.538486+0000","last_clean":"2023-06-09T12:07:47.538486+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:07:47.538486+0000","last_undegraded":"2023-06-09T12:07:47.538486+0000","last_fullsized":"2023-06-09T12:07:47.538486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:27.347596+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:11:27.347596+0000","last_peered":"2023-06-09T12:11:27.347596+0000","last_clean":"2023-06-09T12:11:27.347596+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:11:27.347596+0000","last_undegraded":"2023-06-09T12:11:27.347596+0000","last_fullsized":"2023-06-09T12:11:27.347596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:11.540546+0000","last_change":"2023-06-09T12:11:17.389907+0000","last_active":"2023-06-09T12:11:11.540546+0000","last_peered":"2023-06-09T12:11:11.540546+0000","last_clean":"2023-06-09T12:11:11.540546+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:11:11.540546+0000","last_undegraded":"2023-06-09T12:11:11.540546+0000","last_fullsized":"2023-06-09T12:11:11.540546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1083","reported_seq":4252,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:38.109458+0000","last_change":"2023-06-09T12:11:18.367860+0000","last_active":"2023-06-09T12:11:38.109458+0000","last_peered":"2023-06-09T12:11:38.109458+0000","last_clean":"2023-06-09T12:11:38.109458+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:11:38.109458+0000","last_undegraded":"2023-06-09T12:11:38.109458+0000","last_fullsized":"2023-06-09T12:11:38.109458+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":1083,"log_dups_size":0,"ondisk_log_size":1083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":179,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473491+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473491+0000","last_peered":"2023-06-09T12:09:43.473491+0000","last_clean":"2023-06-09T12:09:43.473491+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473491+0000","last_undegraded":"2023-06-09T12:09:43.473491+0000","last_fullsized":"2023-06-09T12:09:43.473491+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.403692+0000","last_change":"2023-06-09T12:11:32.392160+0000","last_active":"2023-06-09T12:11:17.403692+0000","last_peered":"2023-06-09T12:11:17.403692+0000","last_clean":"2023-06-09T12:11:17.403692+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:11:17.403692+0000","last_undegraded":"2023-06-09T12:11:17.403692+0000","last_fullsized":"2023-06-09T12:11:17.403692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3723,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.796732+0000","last_change":"2023-06-09T12:11:17.496741+0000","last_active":"2023-06-09T12:10:42.796732+0000","last_peered":"2023-06-09T12:10:42.796732+0000","last_clean":"2023-06-09T12:10:42.796732+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:10:42.796732+0000","last_undegraded":"2023-06-09T12:10:42.796732+0000","last_fullsized":"2023-06-09T12:10:42.796732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359803+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:08:22.359803+0000","last_peered":"2023-06-09T12:08:22.359803+0000","last_clean":"2023-06-09T12:08:22.359803+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:08:22.359803+0000","last_undegraded":"2023-06-09T12:08:22.359803+0000","last_fullsized":"2023-06-09T12:08:22.359803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.364676+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:09:02.364676+0000","last_peered":"2023-06-09T12:09:02.364676+0000","last_clean":"2023-06-09T12:09:02.364676+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:09:02.364676+0000","last_undegraded":"2023-06-09T12:09:02.364676+0000","last_fullsized":"2023-06-09T12:09:02.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1028","reported_seq":3489,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:35.734483+0000","last_change":"2023-06-09T12:11:17.496694+0000","last_active":"2023-06-09T12:10:35.734483+0000","last_peered":"2023-06-09T12:10:35.734483+0000","last_clean":"2023-06-09T12:10:35.734483+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:10:35.734483+0000","last_undegraded":"2023-06-09T12:10:35.734483+0000","last_fullsized":"2023-06-09T12:10:35.734483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":69952,"ondisk_log_size":69952,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575583,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265296,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85440,"kb_avail":94106544,"statfs":{"total":96636764160,"available":96365101056,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87490560},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.68799999999999994},"min":{"1min":0.51500000000000001,"5min":0.33700000000000002,"15min":0.32200000000000001},"max":{"1min":0.84899999999999998,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71399999999999997,"15min":0.70799999999999996},"min":{"1min":0.44400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.95699999999999996,"5min":1.202,"15min":1.3320000000000001},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.57899999999999996,"5min":0.441,"15min":0.371},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.746,"15min":0.72899999999999998},"min":{"1min":0.55400000000000005,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.91500000000000004,"5min":0.94699999999999995,"15min":1.05},"last":0.81899999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.71699999999999997,"15min":0.72599999999999998},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":1.0089999999999999,"5min":1.0649999999999999,"15min":1.099},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72899999999999998,"15min":0.73599999999999999},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.39500000000000002},"max":{"1min":0.90000000000000002,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69699999999999995,"15min":0.68999999999999995},"min":{"1min":0.50900000000000001,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.93100000000000005,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.55500000000000005,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94799999999999995,"15min":1.4870000000000001},"last":0.61799999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.73199999999999998,"5min":0.89200000000000002,"15min":1.006},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73199999999999998,"15min":0.70799999999999996},"min":{"1min":0.55300000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.85799999999999998,"5min":1.226,"15min":1.226},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58899999999999997,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.746,"5min":0.93200000000000005,"15min":1.3959999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.622},"min":{"1min":0.44,"5min":0.41899999999999998,"15min":0.38200000000000001},"max":{"1min":0.876,"5min":0.89400000000000002,"15min":0.97999999999999998},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.496,"5min":0.46000000000000002,"15min":0.40300000000000002},"max":{"1min":0.78800000000000003,"5min":0.875,"15min":1.2230000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72599999999999998,"15min":0.73799999999999999},"min":{"1min":0.56699999999999995,"5min":0.52900000000000003,"15min":0.44},"max":{"1min":0.91000000000000003,"5min":0.96899999999999997,"15min":1.2869999999999999},"last":0.85999999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575583,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214232,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58112,"kb_avail":94157608,"statfs":{"total":96636764160,"available":96417390592,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59506688},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.67000000000000004,"15min":0.65700000000000003},"min":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.28299999999999997},"max":{"1min":0.86899999999999999,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70099999999999996,"15min":0.67900000000000005},"min":{"1min":0.47299999999999998,"5min":0.45100000000000001,"15min":0.30599999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.069},"last":0.70099999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.70999999999999996,"15min":0.69499999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.84299999999999997,"5min":0.96299999999999997,"15min":0.99299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.75800000000000001,"15min":0.747},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":0.98699999999999999,"5min":1.0029999999999999,"15min":1.0289999999999999},"last":0.63}]},{"osd":2,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.73999999999999999,"15min":0.71199999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.36799999999999999},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.76500000000000001,"15min":0.752},"min":{"1min":0.56499999999999995,"5min":0.53400000000000003,"15min":0.46400000000000002},"max":{"1min":0.95699999999999996,"5min":0.99099999999999999,"15min":1.0329999999999999},"last":0.64500000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80200000000000005,"15min":0.78500000000000003},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.58599999999999997},"min":{"1min":0.42999999999999999,"5min":0.40100000000000002,"15min":0.36899999999999999},"max":{"1min":0.81100000000000005,"5min":0.85199999999999998,"15min":0.95299999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.49099999999999999,"5min":0.40100000000000002,"15min":0.35599999999999998},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59999999999999998,"15min":0.65500000000000003},"min":{"1min":0.46300000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.92900000000000005,"15min":1.0700000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.749,"15min":0.69599999999999995},"min":{"1min":0.624,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.006,"5min":1.024,"15min":1.052},"last":0.73399999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68500000000000005,"15min":0.70099999999999996},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.71099999999999997,"15min":0.67500000000000004},"min":{"1min":0.56499999999999995,"5min":0.48199999999999998,"15min":0.36299999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.52100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575584,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400048,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93632,"kb_avail":93971792,"statfs":{"total":96636764160,"available":96227115008,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":95879168},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46600000000000003,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.72699999999999998,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.40600000000000003,"5min":0.311,"15min":0.311},"max":{"1min":0.78800000000000003,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.34499999999999997,"5min":0.31,"15min":0.28699999999999998},"max":{"1min":0.55300000000000005,"5min":0.94599999999999995,"15min":0.995},"last":0.439},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.54000000000000004,"15min":0.54400000000000004},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.32700000000000001},"max":{"1min":0.81899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.47499999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.61099999999999999,"15min":0.58199999999999996},"min":{"1min":0.503,"5min":0.41999999999999998,"15min":0.34699999999999998},"max":{"1min":0.97899999999999998,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70999999999999996,"15min":0.67200000000000004},"min":{"1min":0.56299999999999994,"5min":0.503,"15min":0.38700000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.75}]},{"osd":4,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.55300000000000005,"5min":0.48599999999999999,"15min":0.36099999999999999},"max":{"1min":1.026,"5min":1.0620000000000001,"15min":1.0649999999999999},"last":0.87},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.85299999999999998,"15min":0.84299999999999997},"min":{"1min":0.59499999999999997,"5min":0.56599999999999995,"15min":0.54100000000000004},"max":{"1min":1.1040000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.88}]},{"osd":5,"last update":"Fri Jun 9 12:10:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81899999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.54500000000000004},"max":{"1min":0.93300000000000005,"5min":1.254,"15min":1.544},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.83899999999999997,"15min":0.85599999999999998},"min":{"1min":0.70399999999999996,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.0609999999999999,"5min":1.1919999999999999,"15min":1.704},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84699999999999998,"15min":0.83499999999999996},"min":{"1min":0.67900000000000005,"5min":0.55300000000000005,"15min":0.504},"max":{"1min":1.0620000000000001,"5min":1.548,"15min":1.548},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.79900000000000004},"min":{"1min":0.69199999999999995,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":0.97199999999999998,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:10:51 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88900000000000001,"15min":0.88700000000000001},"min":{"1min":0.73199999999999998,"5min":0.66600000000000004,"15min":0.58399999999999996},"max":{"1min":1.0780000000000001,"5min":1.226,"15min":1.6519999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.86799999999999999,"15min":0.81499999999999995},"min":{"1min":0.62,"5min":0.57899999999999996,"15min":0.47299999999999998},"max":{"1min":1.119,"5min":1.498,"15min":1.498},"last":0.90400000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575583,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326440,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98624,"kb_avail":94045400,"statfs":{"total":96636764160,"available":96302489600,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":100990976},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.46200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.247},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82199999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83799999999999997,"15min":0.79700000000000004},"min":{"1min":0.65500000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.0980000000000001,"5min":1.1779999999999999,"15min":1.204},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.89200000000000002,"15min":0.85299999999999998},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.155,"5min":1.268,"15min":1.268},"last":0.83999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.0720000000000001,"5min":1.073,"15min":1.121},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.88700000000000001,"15min":0.84899999999999998},"min":{"1min":0.68600000000000005,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.0620000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.76300000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.81999999999999995},"min":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.1359999999999999,"5min":1.139,"15min":9.0790000000000006},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.21,"5min":1.232,"15min":1.232},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.177,"15min":1.298},"last":0.876},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94699999999999995,"15min":0.92300000000000004},"min":{"1min":0.76200000000000001,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.163,"5min":1.321,"15min":1.351},"last":0.90400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575584,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231392,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51584,"kb_avail":94140448,"statfs":{"total":96636764160,"available":96399818752,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52822016},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.47199999999999998,"15min":0.45300000000000001},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.67900000000000005,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.33000000000000002},"max":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":1.0940000000000001},"last":0.66800000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47499999999999998,"15min":0.47099999999999997},"min":{"1min":0.32600000000000001,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.623,"5min":1.236,"15min":1.3109999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.626},"min":{"1min":0.47399999999999998,"5min":0.42099999999999999,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.2969999999999999,"15min":1.367},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.61399999999999999,"15min":0.60199999999999998},"min":{"1min":0.436,"5min":0.36599999999999999,"15min":0.34399999999999997},"max":{"1min":1.2709999999999999,"5min":1.4490000000000001,"15min":1.546},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67500000000000004,"15min":0.66800000000000004},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.319,"5min":1.4650000000000001,"15min":1.4850000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81200000000000006,"15min":0.78000000000000003},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.42299999999999999},"max":{"1min":1.0900000000000001,"5min":1.202,"15min":1.8220000000000001},"last":0.745},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79200000000000004,"15min":0.78300000000000003},"min":{"1min":0.56799999999999995,"5min":0.53500000000000003,"15min":0.41799999999999998},"max":{"1min":1.095,"5min":1.2669999999999999,"15min":1.7370000000000001},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.624,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3740000000000001,"5min":1.4359999999999999,"15min":1.601},"last":0.877},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89200000000000002,"15min":0.871},"min":{"1min":0.73199999999999998,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.417,"5min":1.5189999999999999,"15min":1.6479999999999999},"last":0.92400000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82399999999999995,"15min":0.80200000000000005},"min":{"1min":0.65200000000000002,"5min":0.56599999999999995,"15min":0.496},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.81799999999999995,"15min":0.78900000000000003},"min":{"1min":0.71799999999999997,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0840000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85099999999999998,"15min":0.82399999999999995},"min":{"1min":0.745,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1319999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.86899999999999999,"15min":0.84799999999999998},"min":{"1min":0.73399999999999999,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.0429999999999999,"5min":1.1879999999999999,"15min":1.262},"last":0.96899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575583,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432636,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":109440,"kb_avail":93939204,"statfs":{"total":96636764160,"available":96193744896,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112066560},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:48 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49199999999999999,"15min":0.48099999999999998},"min":{"1min":0.34999999999999998,"5min":0.31,"15min":0.223},"max":{"1min":0.57299999999999995,"5min":0.81699999999999995,"15min":1.7969999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56599999999999995,"15min":0.55900000000000005},"min":{"1min":0.42599999999999999,"5min":0.37,"15min":0.33600000000000002},"max":{"1min":0.87,"5min":0.87,"15min":1.5669999999999999},"last":0.73099999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.79000000000000004,"15min":0.95299999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.81499999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":1.355},"last":0.62},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.37,"15min":0.312},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82099999999999995,"15min":0.78800000000000003},"min":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.083,"5min":1.171,"15min":3.081},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0309999999999999,"5min":1.1160000000000001,"15min":1.198},"last":0.874},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.89600000000000002,"15min":0.86299999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.165,"15min":1.234},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.115,"5min":1.198,"15min":1.286},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.80500000000000005},"min":{"1min":0.628,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1279999999999999,"5min":1.286,"15min":1.306},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.014,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.91300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575582,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412756,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120000,"kb_avail":93959084,"statfs":{"total":96636764160,"available":96214102016,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":122880000},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.71699999999999997},"min":{"1min":0.57499999999999996,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":1.419,"15min":1.419},"last":0.754},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56599999999999995,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":0.94099999999999995,"5min":1.383,"15min":1.383},"last":0.80400000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.0640000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.58499999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.68100000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.609},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.64900000000000002,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.996,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.84099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68700000000000006},"min":{"1min":0.57299999999999995,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.407},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72799999999999998,"15min":0.73899999999999999},"min":{"1min":0.65200000000000002,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":1.111,"15min":1.4430000000000001},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1719999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.64000000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.62}]},{"osd":7,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.54900000000000004,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.63700000000000001},"min":{"1min":0.499,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97399999999999998,"15min":1.361},"last":0.52900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575582,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299096,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81728,"kb_avail":94072744,"statfs":{"total":96636764160,"available":96330489856,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83689472},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.48099999999999998,"5min":0.41499999999999998,"15min":0.39600000000000002},"max":{"1min":0.82899999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.40999999999999998},"max":{"1min":1.0009999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73199999999999998,"15min":0.78100000000000003},"min":{"1min":0.52600000000000002,"5min":0.47499999999999998,"15min":0.35599999999999998},"max":{"1min":0.91700000000000004,"5min":1.167,"15min":19.715},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.54000000000000004,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.92600000000000005,"5min":0.97099999999999997,"15min":7.4089999999999998},"last":0.64000000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.54400000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.54400000000000004,"15min":0.46100000000000002},"max":{"1min":0.90300000000000002,"5min":1.083,"15min":14.439},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74299999999999999,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.53000000000000003,"15min":0.433},"max":{"1min":0.94999999999999996,"5min":1.254,"15min":12.130000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.83299999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.52300000000000002},"max":{"1min":1.0209999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.79700000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.82999999999999996,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.82299999999999995},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.35199999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":67.521000000000001},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.67100000000000004},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.497},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71899999999999997,"15min":0.68799999999999994},"min":{"1min":0.505,"5min":0.42299999999999999,"15min":0.377},"max":{"1min":0.91600000000000004,"5min":1.095,"15min":7.6040000000000001},"last":0.748}]},{"osd":7,"last update":"Fri Jun 9 12:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.88500000000000001,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73199999999999998,"15min":0.82199999999999995},"min":{"1min":0.53100000000000003,"5min":0.51700000000000002,"15min":0.45600000000000002},"max":{"1min":0.97799999999999998,"5min":1.238,"15min":19.834},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:11:44.201 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.201 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.202 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.202 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.202 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.203 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.203 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.204 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.204 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.204 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.205 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.205 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.206 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.206 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.206 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.207 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.207 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.208 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.208 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.208 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.209 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.209 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.209 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.210 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.210 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:11:44.210 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:11:58.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:11:59.001 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:04.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:12:04.422 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.419+0000 7f69f4dff700 1 Processor -- start 2023-06-09T12:12:04.423 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.419+0000 7f69f4dff700 1 -- start start 2023-06-09T12:12:04.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.420+0000 7f69f4dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69f00cb5d0 0x7f69f00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.420+0000 7f69f4dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f69f00cb9a0 0x7f69f0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69f4dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f0159b20 con 0x7f69f015cd40 2023-06-09T12:12:04.424 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69f4dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f00cac80 con 0x7f69f00cb5d0 2023-06-09T12:12:04.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69f4dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f01597d0 con 0x7f69f00cb9a0 2023-06-09T12:12:04.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69f00cb5d0 0x7f69f00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:04.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69f00cb5d0 0x7f69f00b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47410/0 (socket says 172.21.15.7:47410) 2023-06-09T12:12:04.425 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 -- 172.21.15.7:0/2515990390 learned_addr learned my addr 172.21.15.7:0/2515990390 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:12:04.426 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 -- 172.21.15.7:0/2515990390 >> v2:172.21.15.117:3300/0 conn(0x7f69f00cb9a0 msgr2=0x7f69f0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:12:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 --2- 172.21.15.7:0/2515990390 >> v2:172.21.15.117:3300/0 conn(0x7f69f00cb9a0 0x7f69f0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 -- 172.21.15.7:0/2515990390 >> v1:172.21.15.117:6789/0 conn(0x7f69f015cd40 legacy=0x7f69f00cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:12:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.421+0000 7f69ee59c700 1 -- 172.21.15.7:0/2515990390 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0071ab0 con 0x7f69f00cb5d0 2023-06-09T12:12:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69ee59c700 1 --2- 172.21.15.7:0/2515990390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f00b52b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f69f00c9c60 tx=0x7f69e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834134cccee03aca server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:04.427 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69edd9b700 1 -- 172.21.15.7:0/2515990390 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0004230 con 0x7f69f00cb5d0 2023-06-09T12:12:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69edd9b700 1 -- 172.21.15.7:0/2515990390 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0004390 con 0x7f69f00cb5d0 2023-06-09T12:12:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69edd9b700 1 -- 172.21.15.7:0/2515990390 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e00036a0 con 0x7f69f00cb5d0 2023-06-09T12:12:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 -- 172.21.15.7:0/2515990390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 msgr2=0x7f69f00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 --2- 172.21.15.7:0/2515990390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f00b52b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f69f00c9c60 tx=0x7f69e0009bd0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 -- 172.21.15.7:0/2515990390 shutdown_connections 2023-06-09T12:12:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 --2- 172.21.15.7:0/2515990390 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f00b52b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 --2- 172.21.15.7:0/2515990390 >> v2:172.21.15.117:3300/0 conn(0x7f69f00cb9a0 0x7f69f0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 -- 172.21.15.7:0/2515990390 >> 172.21.15.7:0/2515990390 conn(0x7f69f0070af0 msgr2=0x7f69f0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.422+0000 7f69f4dff700 1 -- 172.21.15.7:0/2515990390 shutdown_connections 2023-06-09T12:12:04.429 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.423+0000 7f69f4dff700 1 -- 172.21.15.7:0/2515990390 wait complete. 2023-06-09T12:12:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.425+0000 7f69f4dff700 1 Processor -- start 2023-06-09T12:12:04.431 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.425+0000 7f69f4dff700 1 -- start start 2023-06-09T12:12:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 0x7f69f01f0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69f015cd40 0x7f69f01f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00ce280 con 0x7f69f00cb5d0 2023-06-09T12:12:04.432 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00c2fa0 con 0x7f69f00cb9a0 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.426+0000 7f69f4dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d0440 con 0x7f69f015cd40 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47424/0 (socket says 172.21.15.7:47424) 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 -- 172.21.15.7:0/1496973912 learned_addr learned my addr 172.21.15.7:0/1496973912 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69e7fff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 0x7f69f01f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:04.433 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69eed9d700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69f015cd40 0x7f69f01f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 -- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69f015cd40 msgr2=0x7f69f01f05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69f015cd40 0x7f69f01f05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 -- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 msgr2=0x7f69f01f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 0x7f69f01f0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.434 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 -- 172.21.15.7:0/1496973912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0009550 con 0x7f69f00cb5d0 2023-06-09T12:12:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69e7fff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 0x7f69f01f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:12:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69ee59c700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f69f0067180 tx=0x7f69e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.427+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0003dc0 con 0x7f69f00cb5d0 2023-06-09T12:12:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.428+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e001f070 con 0x7f69f00cb5d0 2023-06-09T12:12:04.435 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.428+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f015d4f0 con 0x7f69f00cb5d0 2023-06-09T12:12:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.428+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0011870 con 0x7f69f00cb5d0 2023-06-09T12:12:04.436 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.428+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f015d210 con 0x7f69f00cb5d0 2023-06-09T12:12:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.429+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f015d7d0 con 0x7f69f00cb5d0 2023-06-09T12:12:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.430+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69e0021030 con 0x7f69f00cb5d0 2023-06-09T12:12:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.431+0000 7f69e6ffd700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 0x7f69d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.431+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f69e008b520 con 0x7f69f00cb5d0 2023-06-09T12:12:04.437 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.431+0000 7f69e7fff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 0x7f69d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.432+0000 7f69e7fff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 0x7f69d006d190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b4090 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:04.438 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.435+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f69e0055c50 con 0x7f69f00cb5d0 2023-06-09T12:12:04.640 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.638+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69f00ce4f0 con 0x7f69d006ace0 2023-06-09T12:12:04.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.647+0000 7f69e6ffd700 1 -- 172.21.15.7:0/1496973912 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318118 (secure 0 0 0) 0x7f69f00ce4f0 con 0x7f69d006ace0 2023-06-09T12:12:04.654 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:12:04.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.651+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 msgr2=0x7f69d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.651+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 0x7f69d006d190 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b4090 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2023-06-09T12:12:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.651+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 msgr2=0x7f69f0071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.651+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f69f0067180 tx=0x7f69e0004750 comp rx=0 tx=0).stop 2023-06-09T12:12:04.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 shutdown_connections 2023-06-09T12:12:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69d006ace0 0x7f69d006d190 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69f015cd40 0x7f69f01f05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69f00cb5d0 0x7f69f0071130 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 --2- 172.21.15.7:0/1496973912 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69f00cb9a0 0x7f69f01f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:04.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 >> 172.21.15.7:0/1496973912 conn(0x7f69f0070af0 msgr2=0x7f69f015b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 shutdown_connections 2023-06-09T12:12:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:04.652+0000 7f69f4dff700 1 -- 172.21.15.7:0/1496973912 wait complete. 2023-06-09T12:12:04.658 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:12:04.673 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1908,"stamp":"2023-06-09T12:12:03.132573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101516,"ondisk_log_size":101516,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2583048,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":699712,"kb_avail":752391672,"statfs":{"total":773094113280,"available":770449072128,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":716505088},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003315"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.423349+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:08:12.423349+0000","last_peered":"2023-06-09T12:08:12.423349+0000","last_clean":"2023-06-09T12:08:12.423349+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:08:12.423349+0000","last_undegraded":"2023-06-09T12:08:12.423349+0000","last_fullsized":"2023-06-09T12:08:12.423349+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:53.792614+0000","last_change":"2023-06-09T12:11:17.496889+0000","last_active":"2023-06-09T12:10:53.792614+0000","last_peered":"2023-06-09T12:10:53.792614+0000","last_clean":"2023-06-09T12:10:53.792614+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:10:53.792614+0000","last_undegraded":"2023-06-09T12:10:53.792614+0000","last_fullsized":"2023-06-09T12:10:53.792614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4207,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:06.505322+0000","last_change":"2023-06-09T12:11:27.391425+0000","last_active":"2023-06-09T12:11:06.505322+0000","last_peered":"2023-06-09T12:11:06.505322+0000","last_clean":"2023-06-09T12:11:06.505322+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:11:06.505322+0000","last_undegraded":"2023-06-09T12:11:06.505322+0000","last_fullsized":"2023-06-09T12:11:06.505322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3465,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.795842+0000","last_change":"2023-06-09T12:11:17.496908+0000","last_active":"2023-06-09T12:10:50.795842+0000","last_peered":"2023-06-09T12:10:50.795842+0000","last_clean":"2023-06-09T12:10:50.795842+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:10:50.795842+0000","last_undegraded":"2023-06-09T12:10:50.795842+0000","last_fullsized":"2023-06-09T12:10:50.795842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:59.510409+0000","last_change":"2023-06-09T12:11:32.392186+0000","last_active":"2023-06-09T12:10:59.510409+0000","last_peered":"2023-06-09T12:10:59.510409+0000","last_clean":"2023-06-09T12:10:59.510409+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:10:59.510409+0000","last_undegraded":"2023-06-09T12:10:59.510409+0000","last_fullsized":"2023-06-09T12:10:59.510409+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3747,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762164+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762164+0000","last_peered":"2023-06-09T12:10:10.762164+0000","last_clean":"2023-06-09T12:10:10.762164+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762164+0000","last_undegraded":"2023-06-09T12:10:10.762164+0000","last_fullsized":"2023-06-09T12:10:10.762164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1102","reported_seq":3558,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:00.231781+0000","last_change":"2023-06-09T12:11:38.262317+0000","last_active":"2023-06-09T12:12:00.231781+0000","last_peered":"2023-06-09T12:12:00.231781+0000","last_clean":"2023-06-09T12:12:00.231781+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:00.231781+0000","last_undegraded":"2023-06-09T12:12:00.231781+0000","last_fullsized":"2023-06-09T12:12:00.231781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":171,"log_size":1102,"log_dups_size":0,"ondisk_log_size":1102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 21s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4091,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:52.493483+0000","last_change":"2023-06-09T12:11:44.623056+0000","last_active":"2023-06-09T12:10:52.493483+0000","last_peered":"2023-06-09T12:10:52.493483+0000","last_clean":"2023-06-09T12:10:52.493483+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:52.493483+0000","last_undegraded":"2023-06-09T12:10:52.493483+0000","last_fullsized":"2023-06-09T12:10:52.493483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'944","reported_seq":3283,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365982+0000","last_change":"2023-06-09T12:09:12.365982+0000","last_active":"2023-06-09T12:09:12.365982+0000","last_peered":"2023-06-09T12:09:12.365982+0000","last_clean":"2023-06-09T12:09:12.365982+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:09:12.365982+0000","last_undegraded":"2023-06-09T12:09:12.365982+0000","last_fullsized":"2023-06-09T12:09:12.365982+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":944,"log_dups_size":0,"ondisk_log_size":944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:27.446436+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:07:27.446436+0000","last_peered":"2023-06-09T12:07:27.446436+0000","last_clean":"2023-06-09T12:07:27.446436+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:07:27.446436+0000","last_undegraded":"2023-06-09T12:07:27.446436+0000","last_fullsized":"2023-06-09T12:07:27.446436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1340,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:37.537229+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:07:37.537229+0000","last_peered":"2023-06-09T12:07:37.537229+0000","last_clean":"2023-06-09T12:07:37.537229+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:07:37.537229+0000","last_undegraded":"2023-06-09T12:07:37.537229+0000","last_fullsized":"2023-06-09T12:07:37.537229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.372549+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:07:47.372549+0000","last_peered":"2023-06-09T12:07:47.372549+0000","last_clean":"2023-06-09T12:07:47.372549+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:07:47.372549+0000","last_undegraded":"2023-06-09T12:07:47.372549+0000","last_fullsized":"2023-06-09T12:07:47.372549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":711,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.328419+0000","last_change":"2023-06-09T12:11:37.475265+0000","last_active":"2023-06-09T12:11:35.328419+0000","last_peered":"2023-06-09T12:11:35.328419+0000","last_clean":"2023-06-09T12:11:35.328419+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:35.328419+0000","last_undegraded":"2023-06-09T12:11:35.328419+0000","last_fullsized":"2023-06-09T12:11:35.328419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3659,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:01.674317+0000","last_change":"2023-06-09T12:11:32.498998+0000","last_active":"2023-06-09T12:11:01.674317+0000","last_peered":"2023-06-09T12:11:01.674317+0000","last_clean":"2023-06-09T12:11:01.674317+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:11:01.674317+0000","last_undegraded":"2023-06-09T12:11:01.674317+0000","last_fullsized":"2023-06-09T12:11:01.674317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:51.791731+0000","last_change":"2023-06-09T12:11:37.499717+0000","last_active":"2023-06-09T12:10:51.791731+0000","last_peered":"2023-06-09T12:10:51.791731+0000","last_clean":"2023-06-09T12:10:51.791731+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:10:51.791731+0000","last_undegraded":"2023-06-09T12:10:51.791731+0000","last_fullsized":"2023-06-09T12:10:51.791731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.329503+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:07:57.329503+0000","last_peered":"2023-06-09T12:07:57.329503+0000","last_clean":"2023-06-09T12:07:57.329503+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:07:57.329503+0000","last_undegraded":"2023-06-09T12:07:57.329503+0000","last_fullsized":"2023-06-09T12:07:57.329503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.478668+0000","last_change":"2023-06-09T12:11:17.372631+0000","last_active":"2023-06-09T12:10:14.478668+0000","last_peered":"2023-06-09T12:10:14.478668+0000","last_clean":"2023-06-09T12:10:14.478668+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:10:14.478668+0000","last_undegraded":"2023-06-09T12:10:14.478668+0000","last_fullsized":"2023-06-09T12:10:14.478668+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":927,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.471087+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:08:12.471087+0000","last_peered":"2023-06-09T12:08:12.471087+0000","last_clean":"2023-06-09T12:08:12.471087+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:08:12.471087+0000","last_undegraded":"2023-06-09T12:08:12.471087+0000","last_fullsized":"2023-06-09T12:08:12.471087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1098","reported_seq":4084,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:59.445003+0000","last_change":"2023-06-09T12:11:36.721617+0000","last_active":"2023-06-09T12:11:59.445003+0000","last_peered":"2023-06-09T12:11:59.445003+0000","last_clean":"2023-06-09T12:11:59.445003+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:11:59.445003+0000","last_undegraded":"2023-06-09T12:11:59.445003+0000","last_fullsized":"2023-06-09T12:11:59.445003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":172,"log_size":1098,"log_dups_size":0,"ondisk_log_size":1098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:27.347596+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:11:27.347596+0000","last_peered":"2023-06-09T12:11:27.347596+0000","last_clean":"2023-06-09T12:11:27.347596+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:11:27.347596+0000","last_undegraded":"2023-06-09T12:11:27.347596+0000","last_fullsized":"2023-06-09T12:11:27.347596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2996,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:11.540546+0000","last_change":"2023-06-09T12:11:17.389907+0000","last_active":"2023-06-09T12:11:11.540546+0000","last_peered":"2023-06-09T12:11:11.540546+0000","last_clean":"2023-06-09T12:11:11.540546+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:11:11.540546+0000","last_undegraded":"2023-06-09T12:11:11.540546+0000","last_fullsized":"2023-06-09T12:11:11.540546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4402,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:01.925379+0000","last_change":"2023-06-09T12:11:18.367860+0000","last_active":"2023-06-09T12:12:01.925379+0000","last_peered":"2023-06-09T12:12:01.925379+0000","last_clean":"2023-06-09T12:12:01.925379+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:01.925379+0000","last_undegraded":"2023-06-09T12:12:01.925379+0000","last_fullsized":"2023-06-09T12:12:01.925379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 44s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":837,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.403692+0000","last_change":"2023-06-09T12:11:32.392160+0000","last_active":"2023-06-09T12:11:17.403692+0000","last_peered":"2023-06-09T12:11:17.403692+0000","last_clean":"2023-06-09T12:11:17.403692+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:11:17.403692+0000","last_undegraded":"2023-06-09T12:11:17.403692+0000","last_fullsized":"2023-06-09T12:11:17.403692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3723,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.796732+0000","last_change":"2023-06-09T12:11:17.496741+0000","last_active":"2023-06-09T12:10:42.796732+0000","last_peered":"2023-06-09T12:10:42.796732+0000","last_clean":"2023-06-09T12:10:42.796732+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:10:42.796732+0000","last_undegraded":"2023-06-09T12:10:42.796732+0000","last_fullsized":"2023-06-09T12:10:42.796732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1066,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359803+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:08:22.359803+0000","last_peered":"2023-06-09T12:08:22.359803+0000","last_clean":"2023-06-09T12:08:22.359803+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:08:22.359803+0000","last_undegraded":"2023-06-09T12:08:22.359803+0000","last_fullsized":"2023-06-09T12:08:22.359803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.364676+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:09:02.364676+0000","last_peered":"2023-06-09T12:09:02.364676+0000","last_clean":"2023-06-09T12:09:02.364676+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:09:02.364676+0000","last_undegraded":"2023-06-09T12:09:02.364676+0000","last_fullsized":"2023-06-09T12:09:02.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1151","reported_seq":3622,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:59.383758+0000","last_change":"2023-06-09T12:11:45.588065+0000","last_active":"2023-06-09T12:11:59.383758+0000","last_peered":"2023-06-09T12:11:59.383758+0000","last_clean":"2023-06-09T12:11:59.383758+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:11:59.383758+0000","last_undegraded":"2023-06-09T12:11:59.383758+0000","last_fullsized":"2023-06-09T12:11:59.383758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":123,"log_size":1151,"log_dups_size":0,"ondisk_log_size":1151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347368+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:08:02.347368+0000","last_peered":"2023-06-09T12:08:02.347368+0000","last_clean":"2023-06-09T12:08:02.347368+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:08:02.347368+0000","last_undegraded":"2023-06-09T12:08:02.347368+0000","last_fullsized":"2023-06-09T12:08:02.347368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":70512,"ondisk_log_size":70512,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575587,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265424,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85568,"kb_avail":94106416,"statfs":{"total":96636764160,"available":96364969984,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87621632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.51300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.97099999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.65200000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.46700000000000003,"15min":0.38200000000000001},"max":{"1min":0.91300000000000003,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.73599999999999999},"min":{"1min":0.65300000000000002,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.72599999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.55800000000000005,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0649999999999999,"15min":1.099},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.4870000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.89200000000000002,"15min":1.006},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":1.226,"15min":1.226},"last":0.754}]},{"osd":5,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":1.3959999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.97999999999999998},"last":0.85799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64200000000000002,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":1.075,"5min":1.075,"15min":1.2869999999999999},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575587,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214360,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58240,"kb_avail":94157480,"statfs":{"total":96636764160,"available":96417259520,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59637760},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.33100000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69699999999999995,"15min":0.68700000000000006},"min":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.91000000000000003,"5min":0.94099999999999995,"15min":1.069},"last":0.88300000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.99299999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.38},"max":{"1min":1.0209999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.626,"5min":0.54700000000000004,"15min":0.46400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.872}]},{"osd":3,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63700000000000001,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80500000000000005,"15min":0.79200000000000004},"min":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.435,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76000000000000001,"15min":0.70899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76600000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.68400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.36299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.65800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575588,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400240,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93824,"kb_avail":93971600,"statfs":{"total":96636764160,"available":96226918400,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96075776},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.375,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.70999999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.49299999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.61099999999999999,"5min":0.94599999999999995,"15min":0.995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.747,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.40500000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.58799999999999997},"min":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77700000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.288,"15min":1.288},"last":0.504}]},{"osd":4,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.75900000000000001},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0780000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.81799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":0.99399999999999999,"5min":1.254,"15min":1.544},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.1020000000000001,"5min":1.1919999999999999,"15min":1.704},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.83699999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.548,"15min":1.548},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83199999999999996,"15min":0.80600000000000005},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.1319999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.89400000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.028,"5min":1.226,"15min":1.6519999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.63800000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.115,"5min":1.498,"15min":1.498},"last":0.79100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575587,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326568,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98752,"kb_avail":94045272,"statfs":{"total":96636764160,"available":96302358528,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101122048},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.46200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.247},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.55200000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82199999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83799999999999997,"15min":0.79700000000000004},"min":{"1min":0.65500000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.0980000000000001,"5min":1.1779999999999999,"15min":1.204},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.89200000000000002,"15min":0.85299999999999998},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.155,"5min":1.268,"15min":1.268},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.0720000000000001,"5min":1.073,"15min":1.121},"last":0.751},{"interface":"front","average":{"1min":0.874,"5min":0.88700000000000001,"15min":0.84899999999999998},"min":{"1min":0.68600000000000005,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.0620000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.81999999999999995},"min":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.1359999999999999,"5min":1.139,"15min":9.0790000000000006},"last":1.04},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.21,"5min":1.232,"15min":1.232},"last":1.0149999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.177,"15min":1.298},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94699999999999995,"15min":0.92300000000000004},"min":{"1min":0.76200000000000001,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.163,"5min":1.321,"15min":1.351},"last":1.071}]}]},{"osd":2,"up_from":13,"seq":55834575588,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231520,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51712,"kb_avail":94140320,"statfs":{"total":96636764160,"available":96399687680,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":52953088},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.45700000000000002},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":0.79900000000000004,"15min":0.999},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.65200000000000002,"5min":1.236,"15min":1.236},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72399999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.443},"max":{"1min":1.056,"5min":1.202,"15min":1.202},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.94399999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.876,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3899999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.79500000000000004},"min":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.871}]},{"osd":7,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.85499999999999998,"15min":0.83199999999999996},"min":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.0980000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.878,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1240000000000001,"5min":1.1879999999999999,"15min":1.262},"last":0.97199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575587,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432828,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":109632,"kb_avail":93939012,"statfs":{"total":96636764160,"available":96193548288,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112263168},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.223},"max":{"1min":0.57199999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.33800000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.2130000000000001},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.79000000000000004,"15min":0.95299999999999996},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":1.355},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.37,"15min":0.312},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82099999999999995,"15min":0.78800000000000003},"min":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.083,"5min":1.171,"15min":3.081},"last":0.78000000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0309999999999999,"5min":1.1160000000000001,"15min":1.198},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.89600000000000002,"15min":0.86299999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.165,"15min":1.234},"last":0.85199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.115,"5min":1.198,"15min":1.286},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.80500000000000005},"min":{"1min":0.628,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.68899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1279999999999999,"5min":1.286,"15min":1.306},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.014,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.71899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575586,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412884,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120128,"kb_avail":93958956,"statfs":{"total":96636764160,"available":96213970944,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123011072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.71699999999999997},"min":{"1min":0.57499999999999996,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":1.419,"15min":1.419},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56599999999999995,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":0.94099999999999995,"5min":1.383,"15min":1.383},"last":0.68300000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.0640000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.58499999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.609},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.64900000000000002,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.996,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68700000000000006},"min":{"1min":0.57299999999999995,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.407},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.72799999999999998,"15min":0.73899999999999999},"min":{"1min":0.65200000000000002,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":1.111,"15min":1.4430000000000001},"last":0.54600000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1719999999999999},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.439},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.64000000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.54900000000000004,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.63700000000000001},"min":{"1min":0.499,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97399999999999998,"15min":1.361},"last":0.57999999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575586,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299224,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81856,"kb_avail":94072616,"statfs":{"total":96636764160,"available":96330358784,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83820544},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.65700000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.78400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.35599999999999998},"max":{"1min":0.97799999999999998,"5min":1.167,"15min":19.715},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.76200000000000001},"min":{"1min":0.498,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.91900000000000004,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.61099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92400000000000004,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":1.083,"15min":14.439},"last":0.71999999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.254,"15min":12.130000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0049999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.79500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.76100000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.56799999999999995,"15min":0.82799999999999996},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":67.521000000000001},"last":0.61299999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59999999999999998,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":12.163},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.751,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.505,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":1.095,"15min":7.6040000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.45000000000000001,"15min":0.438},"max":{"1min":0.876,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.75},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":0.90200000000000002,"5min":1.238,"15min":19.834},"last":0.79700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:12:04.681 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.681 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.682 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.682 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.682 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.683 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.683 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.684 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.684 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.684 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.685 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.685 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.686 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.686 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.686 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.687 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.687 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.687 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.688 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.688 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.689 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.689 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.689 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.690 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.690 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:04.690 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:12:24.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:12:24.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.900+0000 7f8f14fff700 1 Processor -- start 2023-06-09T12:12:24.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.900+0000 7f8f14fff700 1 -- start start 2023-06-09T12:12:24.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f8f100c2ae0 0x7f8f100c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f1015a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f100c7cb0 0x7f8f1015ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f10159360 con 0x7f8f100c7cb0 2023-06-09T12:12:24.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f100c8e20 con 0x7f8f100c2ae0 2023-06-09T12:12:24.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f14fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100c33f0 con 0x7f8f100c88e0 2023-06-09T12:12:24.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f100c7cb0 0x7f8f1015ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.905 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f8f100c7cb0 0x7f8f1015ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55598/0 (socket says 172.21.15.7:55598) 2023-06-09T12:12:24.906 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 -- 172.21.15.7:0/3573112795 learned_addr learned my addr 172.21.15.7:0/3573112795 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:12:24.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f07fff700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f1015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0e59c700 1 --2- 172.21.15.7:0/3573112795 >> v2:172.21.15.117:3300/0 conn(0x7f8f100c2ae0 0x7f8f100c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.907 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 -- 172.21.15.7:0/3573112795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 msgr2=0x7f8f1015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:12:24.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f1015a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 -- 172.21.15.7:0/3573112795 >> v2:172.21.15.117:3300/0 conn(0x7f8f100c2ae0 msgr2=0x7f8f100c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:12:24.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 --2- 172.21.15.7:0/3573112795 >> v2:172.21.15.117:3300/0 conn(0x7f8f100c2ae0 0x7f8f100c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.902+0000 7f8f0ed9d700 1 -- 172.21.15.7:0/3573112795 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f100c9bd0 con 0x7f8f100c7cb0 2023-06-09T12:12:24.908 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f0ed9d700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f1015ae90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8f10071720 tx=0x7f8f0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e57ece503a57309 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:24.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f0dd9b700 1 -- 172.21.15.7:0/3573112795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800e070 con 0x7f8f100c7cb0 2023-06-09T12:12:24.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f0dd9b700 1 -- 172.21.15.7:0/3573112795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0800b980 con 0x7f8f100c7cb0 2023-06-09T12:12:24.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f0dd9b700 1 -- 172.21.15.7:0/3573112795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0800bd30 con 0x7f8f100c7cb0 2023-06-09T12:12:24.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f14fff700 1 -- 172.21.15.7:0/3573112795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 msgr2=0x7f8f1015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:24.909 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.903+0000 7f8f14fff700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f1015ae90 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f8f10071720 tx=0x7f8f0800b330 comp rx=0 tx=0).stop 2023-06-09T12:12:24.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 -- 172.21.15.7:0/3573112795 shutdown_connections 2023-06-09T12:12:24.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 --2- 172.21.15.7:0/3573112795 >> v2:172.21.15.117:3300/0 conn(0x7f8f100c2ae0 0x7f8f100c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f1015ae90 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 --2- 172.21.15.7:0/3573112795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f1015a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.910 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 -- 172.21.15.7:0/3573112795 >> 172.21.15.7:0/3573112795 conn(0x7f8f100cd330 msgr2=0x7f8f100cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:24.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 -- 172.21.15.7:0/3573112795 shutdown_connections 2023-06-09T12:12:24.911 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.904+0000 7f8f14fff700 1 -- 172.21.15.7:0/3573112795 wait complete. 2023-06-09T12:12:24.912 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.906+0000 7f8f14fff700 1 Processor -- start 2023-06-09T12:12:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.906+0000 7f8f14fff700 1 -- start start 2023-06-09T12:12:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 0x7f8f101565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f101e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.913 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100c63b0 con 0x7f8f100c7cb0 2023-06-09T12:12:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f100c6020 con 0x7f8f100c2ae0 2023-06-09T12:12:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.907+0000 7f8f14fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f10144d20 con 0x7f8f100c88e0 2023-06-09T12:12:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55612/0 (socket says 172.21.15.7:55612) 2023-06-09T12:12:24.914 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 -- 172.21.15.7:0/1369894528 learned_addr learned my addr 172.21.15.7:0/1369894528 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:12:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f0e59c700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 0x7f8f101565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f0ed9d700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f101e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 -- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 msgr2=0x7f8f101e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f101e21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.915 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 -- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 msgr2=0x7f8f101565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 0x7f8f101565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 -- 172.21.15.7:0/1369894528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f08009b30 con 0x7f8f100c7cb0 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f0e59c700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 0x7f8f101565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.908+0000 7f8f07fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d800 tx=0x7f8ef800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.909+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80125f0 con 0x7f8f100c7cb0 2023-06-09T12:12:24.916 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.909+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef8012750 con 0x7f8f100c7cb0 2023-06-09T12:12:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.909+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef8012b00 con 0x7f8f100c7cb0 2023-06-09T12:12:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.909+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f10159810 con 0x7f8f100c7cb0 2023-06-09T12:12:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.909+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f10159570 con 0x7f8f100c7cb0 2023-06-09T12:12:24.917 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.910+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f10159af0 con 0x7f8f100c7cb0 2023-06-09T12:12:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.911+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8ef8012c60 con 0x7f8f100c7cb0 2023-06-09T12:12:24.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.912+0000 7f8f06ffd700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 0x7f8ef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.912+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8ef808a0e0 con 0x7f8f100c7cb0 2023-06-09T12:12:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.912+0000 7f8f0e59c700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 0x7f8ef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.912+0000 7f8f0e59c700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 0x7f8ef006d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009970 tx=0x7f8f00008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:24.919 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:24.916+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8ef8054870 con 0x7f8f100c7cb0 2023-06-09T12:12:25.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.121+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f101e34e0 con 0x7f8ef006ace0 2023-06-09T12:12:25.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.130+0000 7f8f06ffd700 1 -- 172.21.15.7:0/1369894528 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318310 (secure 0 0 0) 0x7f8f101e34e0 con 0x7f8ef006ace0 2023-06-09T12:12:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:12:25.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.134+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 msgr2=0x7f8ef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.134+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 0x7f8ef006d190 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8f00009970 tx=0x7f8f00008040 comp rx=0 tx=0).stop 2023-06-09T12:12:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.134+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 msgr2=0x7f8f101e1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.134+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800d800 tx=0x7f8ef800dbc0 comp rx=0 tx=0).stop 2023-06-09T12:12:25.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 shutdown_connections 2023-06-09T12:12:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f8ef006ace0 0x7f8ef006d190 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8f100c88e0 0x7f8f101e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8f100c2ae0 0x7f8f101565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 --2- 172.21.15.7:0/1369894528 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8f100c7cb0 0x7f8f101e1c80 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 >> 172.21.15.7:0/1369894528 conn(0x7f8f100cd330 msgr2=0x7f8f100c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 shutdown_connections 2023-06-09T12:12:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:25.135+0000 7f8f14fff700 1 -- 172.21.15.7:0/1369894528 wait complete. 2023-06-09T12:12:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:12:25.155 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1918,"stamp":"2023-06-09T12:12:23.137963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101825,"ondisk_log_size":101825,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2583688,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":700352,"kb_avail":752391032,"statfs":{"total":773094113280,"available":770448416768,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":717160448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003037"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'846","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:53.792614+0000","last_change":"2023-06-09T12:11:17.496889+0000","last_active":"2023-06-09T12:10:53.792614+0000","last_peered":"2023-06-09T12:10:53.792614+0000","last_clean":"2023-06-09T12:10:53.792614+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:10:53.792614+0000","last_undegraded":"2023-06-09T12:10:53.792614+0000","last_fullsized":"2023-06-09T12:10:53.792614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":846,"log_dups_size":0,"ondisk_log_size":846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:40.807676+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:07:40.807676+0000","last_peered":"2023-06-09T12:07:40.807676+0000","last_clean":"2023-06-09T12:07:40.807676+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:07:40.807676+0000","last_undegraded":"2023-06-09T12:07:40.807676+0000","last_fullsized":"2023-06-09T12:07:40.807676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4209,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:06.494913+0000","last_change":"2023-06-09T12:12:06.494913+0000","last_active":"2023-06-09T12:12:06.494913+0000","last_peered":"2023-06-09T12:12:06.494913+0000","last_clean":"2023-06-09T12:12:06.494913+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:12:06.494913+0000","last_undegraded":"2023-06-09T12:12:06.494913+0000","last_fullsized":"2023-06-09T12:12:06.494913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3465,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.795842+0000","last_change":"2023-06-09T12:11:17.496908+0000","last_active":"2023-06-09T12:10:50.795842+0000","last_peered":"2023-06-09T12:10:50.795842+0000","last_clean":"2023-06-09T12:10:50.795842+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:10:50.795842+0000","last_undegraded":"2023-06-09T12:10:50.795842+0000","last_fullsized":"2023-06-09T12:10:50.795842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3742,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:43.327453+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:07:43.327453+0000","last_peered":"2023-06-09T12:07:43.327453+0000","last_clean":"2023-06-09T12:07:43.327453+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:07:43.327453+0000","last_undegraded":"2023-06-09T12:07:43.327453+0000","last_fullsized":"2023-06-09T12:07:43.327453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3524,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:47.420771+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:07:47.420771+0000","last_peered":"2023-06-09T12:07:47.420771+0000","last_clean":"2023-06-09T12:07:47.420771+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:07:47.420771+0000","last_undegraded":"2023-06-09T12:07:47.420771+0000","last_fullsized":"2023-06-09T12:07:47.420771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3745,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:04.518091+0000","last_change":"2023-06-09T12:12:04.518091+0000","last_active":"2023-06-09T12:12:04.518091+0000","last_peered":"2023-06-09T12:12:04.518091+0000","last_clean":"2023-06-09T12:12:04.518091+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:12:04.518091+0000","last_undegraded":"2023-06-09T12:12:04.518091+0000","last_fullsized":"2023-06-09T12:12:04.518091+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300402+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300402+0000","last_peered":"2023-06-09T12:10:24.300402+0000","last_clean":"2023-06-09T12:10:24.300402+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300402+0000","last_undegraded":"2023-06-09T12:10:24.300402+0000","last_fullsized":"2023-06-09T12:10:24.300402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422830+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:08:07.422830+0000","last_peered":"2023-06-09T12:08:07.422830+0000","last_clean":"2023-06-09T12:08:07.422830+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:08:07.422830+0000","last_undegraded":"2023-06-09T12:08:07.422830+0000","last_fullsized":"2023-06-09T12:08:07.422830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3810,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:12.358430+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:08:12.358430+0000","last_peered":"2023-06-09T12:08:12.358430+0000","last_clean":"2023-06-09T12:08:12.358430+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:08:12.358430+0000","last_undegraded":"2023-06-09T12:08:12.358430+0000","last_fullsized":"2023-06-09T12:08:12.358430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4091,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:52.493483+0000","last_change":"2023-06-09T12:11:44.623056+0000","last_active":"2023-06-09T12:10:52.493483+0000","last_peered":"2023-06-09T12:10:52.493483+0000","last_clean":"2023-06-09T12:10:52.493483+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:52.493483+0000","last_undegraded":"2023-06-09T12:10:52.493483+0000","last_fullsized":"2023-06-09T12:10:52.493483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1567,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:52.373465+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:07:52.373465+0000","last_peered":"2023-06-09T12:07:52.373465+0000","last_clean":"2023-06-09T12:07:52.373465+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:07:52.373465+0000","last_undegraded":"2023-06-09T12:07:52.373465+0000","last_fullsized":"2023-06-09T12:07:52.373465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'967","reported_seq":3311,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:22.289284+0000","last_change":"2023-06-09T12:12:20.401920+0000","last_active":"2023-06-09T12:12:22.289284+0000","last_peered":"2023-06-09T12:12:22.289284+0000","last_clean":"2023-06-09T12:12:22.289284+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:22.289284+0000","last_undegraded":"2023-06-09T12:12:22.289284+0000","last_fullsized":"2023-06-09T12:12:22.289284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":23,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3858,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.678946+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.678946+0000","last_peered":"2023-06-09T12:10:14.678946+0000","last_clean":"2023-06-09T12:10:14.678946+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.678946+0000","last_undegraded":"2023-06-09T12:10:14.678946+0000","last_fullsized":"2023-06-09T12:10:14.678946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":711,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.328419+0000","last_change":"2023-06-09T12:11:37.475265+0000","last_active":"2023-06-09T12:11:35.328419+0000","last_peered":"2023-06-09T12:11:35.328419+0000","last_clean":"2023-06-09T12:11:35.328419+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:35.328419+0000","last_undegraded":"2023-06-09T12:11:35.328419+0000","last_fullsized":"2023-06-09T12:11:35.328419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3659,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:01.674317+0000","last_change":"2023-06-09T12:11:32.498998+0000","last_active":"2023-06-09T12:11:01.674317+0000","last_peered":"2023-06-09T12:11:01.674317+0000","last_clean":"2023-06-09T12:11:01.674317+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:11:01.674317+0000","last_undegraded":"2023-06-09T12:11:01.674317+0000","last_fullsized":"2023-06-09T12:11:01.674317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4301,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:20.256714+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:08:20.256714+0000","last_peered":"2023-06-09T12:08:20.256714+0000","last_clean":"2023-06-09T12:08:20.256714+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:08:20.256714+0000","last_undegraded":"2023-06-09T12:08:20.256714+0000","last_fullsized":"2023-06-09T12:08:20.256714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3847,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:51.791731+0000","last_change":"2023-06-09T12:11:37.499717+0000","last_active":"2023-06-09T12:10:51.791731+0000","last_peered":"2023-06-09T12:10:51.791731+0000","last_clean":"2023-06-09T12:10:51.791731+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:10:51.791731+0000","last_undegraded":"2023-06-09T12:10:51.791731+0000","last_fullsized":"2023-06-09T12:10:51.791731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'935","reported_seq":3612,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:21.812638+0000","last_change":"2023-06-09T12:12:21.440139+0000","last_active":"2023-06-09T12:12:21.812638+0000","last_peered":"2023-06-09T12:12:21.812638+0000","last_clean":"2023-06-09T12:12:21.812638+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:12:21.812638+0000","last_undegraded":"2023-06-09T12:12:21.812638+0000","last_fullsized":"2023-06-09T12:12:21.812638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1352,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:22.359715+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:08:22.359715+0000","last_peered":"2023-06-09T12:08:22.359715+0000","last_clean":"2023-06-09T12:08:22.359715+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:08:22.359715+0000","last_undegraded":"2023-06-09T12:08:22.359715+0000","last_fullsized":"2023-06-09T12:08:22.359715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:27.347596+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:11:27.347596+0000","last_peered":"2023-06-09T12:11:27.347596+0000","last_clean":"2023-06-09T12:11:27.347596+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:11:27.347596+0000","last_undegraded":"2023-06-09T12:11:27.347596+0000","last_fullsized":"2023-06-09T12:11:27.347596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2998,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:10.529212+0000","last_change":"2023-06-09T12:12:10.529212+0000","last_active":"2023-06-09T12:12:10.529212+0000","last_peered":"2023-06-09T12:12:10.529212+0000","last_clean":"2023-06-09T12:12:10.529212+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:12:10.529212+0000","last_undegraded":"2023-06-09T12:12:10.529212+0000","last_fullsized":"2023-06-09T12:12:10.529212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":838,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:32.392160+0000","last_change":"2023-06-09T12:12:13.519341+0000","last_active":"2023-06-09T12:11:32.392160+0000","last_peered":"2023-06-09T12:11:32.392160+0000","last_clean":"2023-06-09T12:11:32.392160+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:11:32.392160+0000","last_undegraded":"2023-06-09T12:11:32.392160+0000","last_fullsized":"2023-06-09T12:11:32.392160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3723,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.796732+0000","last_change":"2023-06-09T12:11:17.496741+0000","last_active":"2023-06-09T12:10:42.796732+0000","last_peered":"2023-06-09T12:10:42.796732+0000","last_clean":"2023-06-09T12:10:42.796732+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:10:42.796732+0000","last_undegraded":"2023-06-09T12:10:42.796732+0000","last_fullsized":"2023-06-09T12:10:42.796732+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.364676+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:09:02.364676+0000","last_peered":"2023-06-09T12:09:02.364676+0000","last_clean":"2023-06-09T12:09:02.364676+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:09:02.364676+0000","last_undegraded":"2023-06-09T12:09:02.364676+0000","last_fullsized":"2023-06-09T12:09:02.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1627,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.367771+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:08:27.367771+0000","last_peered":"2023-06-09T12:08:27.367771+0000","last_clean":"2023-06-09T12:08:27.367771+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:08:27.367771+0000","last_undegraded":"2023-06-09T12:08:27.367771+0000","last_fullsized":"2023-06-09T12:08:27.367771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1273","reported_seq":3750,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:20.229171+0000","last_change":"2023-06-09T12:11:45.588065+0000","last_active":"2023-06-09T12:12:20.229171+0000","last_peered":"2023-06-09T12:12:20.229171+0000","last_clean":"2023-06-09T12:12:20.229171+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:20.229171+0000","last_undegraded":"2023-06-09T12:12:20.229171+0000","last_fullsized":"2023-06-09T12:12:20.229171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":245,"log_size":1273,"log_dups_size":0,"ondisk_log_size":1273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":70821,"ondisk_log_size":70821,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575591,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265552,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85696,"kb_avail":94106288,"statfs":{"total":96636764160,"available":96364838912,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87752704},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.51300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.97099999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.83299999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.46700000000000003,"15min":0.38200000000000001},"max":{"1min":0.91300000000000003,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.73599999999999999},"min":{"1min":0.65300000000000002,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.55800000000000005,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0649999999999999,"15min":1.099},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.85899999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.4870000000000001},"last":0.67200000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.89200000000000002,"15min":1.006},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":1.226,"15min":1.226},"last":0.73599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":1.3959999999999999},"last":0.624},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.97999999999999998},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64200000000000002,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":1.075,"5min":1.075,"15min":1.2869999999999999},"last":0.79300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575591,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214488,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58368,"kb_avail":94157352,"statfs":{"total":96636764160,"available":96417128448,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59768832},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.33100000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69699999999999995,"15min":0.68700000000000006},"min":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.91000000000000003,"5min":0.94099999999999995,"15min":1.069},"last":0.876}]},{"osd":1,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.99299999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.81599999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.38},"max":{"1min":1.0209999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.626,"5min":0.54700000000000004,"15min":0.46400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63700000000000001,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80500000000000005,"15min":0.79200000000000004},"min":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.435,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.56499999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76000000000000001,"15min":0.70899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.624},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.68400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.36299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575592,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400304,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":93888,"kb_avail":93971536,"statfs":{"total":96636764160,"available":96226852864,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96141312},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.375,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.70999999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.61099999999999999,"5min":0.94599999999999995,"15min":0.995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.747,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.58799999999999997},"min":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77700000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.75},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.288,"15min":1.288},"last":0.83799999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.75900000000000001},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0780000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.90300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":0.99399999999999999,"5min":1.254,"15min":1.544},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.1020000000000001,"5min":1.1919999999999999,"15min":1.704},"last":0.91500000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.83699999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.548,"15min":1.548},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83199999999999996,"15min":0.80600000000000005},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.1319999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.85999999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.028,"5min":1.226,"15min":1.6519999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.63800000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.115,"5min":1.498,"15min":1.498},"last":0.878}]}]},{"osd":0,"up_from":13,"seq":55834575591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326568,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98752,"kb_avail":94045272,"statfs":{"total":96636764160,"available":96302358528,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101122048},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.48299999999999998,"15min":0.46200000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.247},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.435,"5min":0.41099999999999998,"15min":0.34100000000000003},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.73099999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55600000000000005,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34999999999999998},"max":{"1min":0.69799999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.435,"5min":0.35699999999999998,"15min":0.27400000000000002},"max":{"1min":0.65600000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.64000000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.69299999999999995},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.82199999999999995,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83799999999999997,"15min":0.79700000000000004},"min":{"1min":0.65500000000000003,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":1.0980000000000001,"5min":1.1779999999999999,"15min":1.204},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.89200000000000002,"15min":0.85299999999999998},"min":{"1min":0.70999999999999996,"5min":0.60499999999999998,"15min":0.42899999999999999},"max":{"1min":1.155,"5min":1.268,"15min":1.268},"last":1.1419999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.78100000000000003,"15min":0.78000000000000003},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.39900000000000002},"max":{"1min":1.0720000000000001,"5min":1.073,"15min":1.121},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.88700000000000001,"15min":0.84899999999999998},"min":{"1min":0.68600000000000005,"5min":0.60299999999999998,"15min":0.501},"max":{"1min":1.0620000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":1.0529999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79000000000000004,"15min":0.81999999999999995},"min":{"1min":0.55400000000000005,"5min":0.52600000000000002,"15min":0.48099999999999998},"max":{"1min":1.1359999999999999,"5min":1.139,"15min":9.0790000000000006},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83999999999999997,"15min":0.83699999999999997},"min":{"1min":0.62,"5min":0.60299999999999998,"15min":0.52300000000000002},"max":{"1min":1.21,"5min":1.232,"15min":1.232},"last":1.081}]},{"osd":7,"last update":"Fri Jun 9 12:11:31 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.871,"15min":0.84599999999999997},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.113,"5min":1.177,"15min":1.298},"last":1.109},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.94699999999999995,"15min":0.92300000000000004},"min":{"1min":0.76200000000000001,"5min":0.66400000000000003,"15min":0.58199999999999996},"max":{"1min":1.163,"5min":1.321,"15min":1.351},"last":1.0209999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575592,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231648,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51840,"kb_avail":94140192,"statfs":{"total":96636764160,"available":96399556608,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53084160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.45700000000000002},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":0.79900000000000004,"15min":0.999},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.65200000000000002,"5min":1.236,"15min":1.236},"last":0.434},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72399999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.443},"max":{"1min":1.056,"5min":1.202,"15min":1.202},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.70199999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.876,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3899999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.79500000000000004},"min":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.85499999999999998,"15min":0.83199999999999996},"min":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.0980000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.878,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1240000000000001,"5min":1.1879999999999999,"15min":1.262},"last":0.73499999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575591,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432892,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":109696,"kb_avail":93938948,"statfs":{"total":96636764160,"available":96193482752,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":112328704},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.223},"max":{"1min":0.57199999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.33800000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.2130000000000001},"last":0.443}]},{"osd":1,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.51900000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.27100000000000002},"max":{"1min":0.70299999999999996,"5min":0.79000000000000004,"15min":0.95299999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58899999999999997,"15min":0.53700000000000003},"min":{"1min":0.45700000000000002,"5min":0.379,"15min":0.29699999999999999},"max":{"1min":0.73999999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.48499999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.56100000000000005,"5min":0.48899999999999999,"15min":0.42299999999999999},"max":{"1min":0.872,"5min":0.872,"15min":1.355},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58599999999999997,"15min":0.56499999999999995},"min":{"1min":0.38900000000000001,"5min":0.37,"15min":0.312},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.55200000000000005,"5min":0.54100000000000004,"15min":0.39400000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.82099999999999995,"15min":0.78800000000000003},"min":{"1min":0.56599999999999995,"5min":0.56100000000000005,"15min":0.42999999999999999},"max":{"1min":1.083,"5min":1.171,"15min":3.081},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.84299999999999997,"15min":0.81899999999999995},"min":{"1min":0.61199999999999999,"5min":0.52800000000000002,"15min":0.39200000000000002},"max":{"1min":1.0309999999999999,"5min":1.1160000000000001,"15min":1.198},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.89600000000000002,"15min":0.86299999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.47899999999999998},"max":{"1min":1.0600000000000001,"5min":1.165,"15min":1.234},"last":0.96099999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.54900000000000004,"5min":0.54400000000000004,"15min":0.503},"max":{"1min":1.115,"5min":1.198,"15min":1.286},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.83399999999999996,"15min":0.80500000000000005},"min":{"1min":0.628,"5min":0.60999999999999999,"15min":0.434},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:11:28 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.83699999999999997,"15min":0.82299999999999995},"min":{"1min":0.63900000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1279999999999999,"5min":1.286,"15min":1.306},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.86299999999999999},"min":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.014,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.94099999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575590,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413012,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120256,"kb_avail":93958828,"statfs":{"total":96636764160,"available":96213839872,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123142144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73999999999999999,"15min":0.71699999999999997},"min":{"1min":0.57499999999999996,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":1.419,"15min":1.419},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56599999999999995,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":0.94099999999999995,"5min":1.383,"15min":1.383},"last":0.749}]},{"osd":1,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70599999999999996,"15min":0.73199999999999998},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":1.0640000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.58499999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":0.97299999999999998,"5min":1.0009999999999999,"15min":1.1279999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69799999999999995,"15min":0.70299999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.609},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78000000000000003,"15min":0.751},"min":{"1min":0.64900000000000002,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":0.996,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.72199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.66800000000000004,"15min":0.68700000000000006},"min":{"1min":0.57299999999999995,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.407},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.72799999999999998,"15min":0.73899999999999999},"min":{"1min":0.65200000000000002,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.95699999999999996,"5min":1.111,"15min":1.4430000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.52700000000000002,"15min":0.54200000000000004},"min":{"1min":0.437,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60199999999999998,"15min":0.625},"min":{"1min":0.502,"5min":0.41999999999999998,"15min":0.38400000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.1719999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55900000000000005,"15min":0.57799999999999996},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.79400000000000004,"5min":0.85699999999999998,"15min":1.2490000000000001},"last":0.505},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.73299999999999998,"15min":0.72899999999999998},"min":{"1min":0.64000000000000001,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":0.94199999999999995,"5min":1.0489999999999999,"15min":1.3100000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:11:32 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.54900000000000004,"5min":0.46700000000000003,"15min":0.27400000000000002},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.58399999999999996,"15min":0.63700000000000001},"min":{"1min":0.499,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97399999999999998,"15min":1.361},"last":0.54700000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575590,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299224,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81856,"kb_avail":94072616,"statfs":{"total":96636764160,"available":96330358784,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83820544},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.78400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.35599999999999998},"max":{"1min":0.97799999999999998,"5min":1.167,"15min":19.715},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.76200000000000001},"min":{"1min":0.498,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.91900000000000004,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.61099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92400000000000004,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.622},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":1.083,"15min":14.439},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.254,"15min":12.130000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0049999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.79500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.76100000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.56799999999999995,"15min":0.82799999999999996},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":67.521000000000001},"last":0.438}]},{"osd":6,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59999999999999998,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":12.163},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.505,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":1.095,"15min":7.6040000000000001},"last":0.73499999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.45000000000000001,"15min":0.438},"max":{"1min":0.876,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":0.90200000000000002,"5min":1.238,"15min":19.834},"last":0.53200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:12:25.163 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.163 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.164 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.164 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.165 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.165 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.165 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.166 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.166 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.167 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.167 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.167 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.168 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.168 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.169 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.169 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.169 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.170 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.170 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.170 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.171 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.171 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:25.171 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:12:29.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:29.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:45.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:12:45.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.380+0000 7f868e1ff700 1 Processor -- start 2023-06-09T12:12:45.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.381+0000 7f868e1ff700 1 -- start start 2023-06-09T12:12:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f868e1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 0x7f868815d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f868e1ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868815dc90 con 0x7f86880b5370 2023-06-09T12:12:45.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f868e1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86880cfaf0 con 0x7f86880c8e90 2023-06-09T12:12:45.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f868e1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86881597d0 con 0x7f86880c9260 2023-06-09T12:12:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f8687fff700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f86880b5370 0x7f86880c9fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39752/0 (socket says 172.21.15.7:39752) 2023-06-09T12:12:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f8687fff700 1 -- 172.21.15.7:0/1674160663 learned_addr learned my addr 172.21.15.7:0/1674160663 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:12:45.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f8686ffd700 1 --2- 172.21.15.7:0/1674160663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 0x7f868815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.382+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952060505 0 0) 0x7f8680001490 con 0x7f86880b5370 2023-06-09T12:12:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86700035b0 con 0x7f86880b5370 2023-06-09T12:12:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111901068 0 0) 0x7f86880cfaf0 con 0x7f86880c8e90 2023-06-09T12:12:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 --> v1:172.21.15.117:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8680001490 con 0x7f86880c8e90 2023-06-09T12:12:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f8686ffd700 1 -- 172.21.15.7:0/1674160663 >> v1:172.21.15.117:6789/0 conn(0x7f86880c8e90 legacy=0x7f86880cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f8686ffd700 1 -- 172.21.15.7:0/1674160663 >> v1:172.21.15.7:6789/0 conn(0x7f86880b5370 legacy=0x7f86880c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f8686ffd700 1 -- 172.21.15.7:0/1674160663 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880cfdd0 con 0x7f86880c9260 2023-06-09T12:12:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f8686ffd700 1 --2- 172.21.15.7:0/1674160663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 0x7f868815d750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f867c0098a0 tx=0x7f867c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d95729dc6dabe24 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c011830 con 0x7f86880c9260 2023-06-09T12:12:45.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c011990 con 0x7f86880c9260 2023-06-09T12:12:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.383+0000 7f86867fc700 1 -- 172.21.15.7:0/1674160663 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c011d20 con 0x7f86880c9260 2023-06-09T12:12:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 -- 172.21.15.7:0/1674160663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 msgr2=0x7f868815d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 --2- 172.21.15.7:0/1674160663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 0x7f868815d750 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f867c0098a0 tx=0x7f867c009bb0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 -- 172.21.15.7:0/1674160663 shutdown_connections 2023-06-09T12:12:45.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 --2- 172.21.15.7:0/1674160663 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880c9260 0x7f868815d750 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 -- 172.21.15.7:0/1674160663 >> 172.21.15.7:0/1674160663 conn(0x7f8688070af0 msgr2=0x7f8688070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 -- 172.21.15.7:0/1674160663 shutdown_connections 2023-06-09T12:12:45.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.384+0000 7f868e1ff700 1 -- 172.21.15.7:0/1674160663 wait complete. 2023-06-09T12:12:45.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.386+0000 7f868e1ff700 1 Processor -- start 2023-06-09T12:12:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.386+0000 7f868e1ff700 1 -- start start 2023-06-09T12:12:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880b5370 0x7f86881f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 0x7f86880c3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:45.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86880c5eb0 con 0x7f86880c9260 2023-06-09T12:12:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8688071280 con 0x7f86880c8e90 2023-06-09T12:12:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f868e1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86880cf0f0 con 0x7f86880b5370 2023-06-09T12:12:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f8687fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:45.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f8687fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46226/0 (socket says 172.21.15.7:46226) 2023-06-09T12:12:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f8687fff700 1 -- 172.21.15.7:0/3947688290 learned_addr learned my addr 172.21.15.7:0/3947688290 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:12:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f86877fe700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880b5370 0x7f86881f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.387+0000 7f8686ffd700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 0x7f86880c3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 -- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880b5370 msgr2=0x7f86881f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.394 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880b5370 0x7f86881f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 -- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 msgr2=0x7f86880c3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 0x7f86880c3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 -- 172.21.15.7:0/3947688290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c0094a0 con 0x7f86880c9260 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8686ffd700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 0x7f86880c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f8687fff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f868000cfb0 tx=0x7f868000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:45.395 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000abf0 con 0x7f86880c9260 2023-06-09T12:12:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868000ad50 con 0x7f86880c9260 2023-06-09T12:12:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8680012600 con 0x7f86880c9260 2023-06-09T12:12:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8688159ca0 con 0x7f86880c9260 2023-06-09T12:12:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.388+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86881599c0 con 0x7f86880c9260 2023-06-09T12:12:45.396 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.389+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8688159f80 con 0x7f86880c9260 2023-06-09T12:12:45.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.390+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f868001b3b0 con 0x7f86880c9260 2023-06-09T12:12:45.397 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.391+0000 7f866ffff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 0x7f867406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:12:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.391+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f8680089ad0 con 0x7f86880c9260 2023-06-09T12:12:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.391+0000 7f86877fe700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 0x7f867406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:12:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.392+0000 7f86877fe700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 0x7f867406d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f86780011e0 tx=0x7f8678009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:12:45.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.396+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8680054220 con 0x7f86880c9260 2023-06-09T12:12:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.603+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f868815dc90 con 0x7f867406ace0 2023-06-09T12:12:45.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.613+0000 7f866ffff700 1 -- 172.21.15.7:0/3947688290 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318482 (secure 0 0 0) 0x7f868815dc90 con 0x7f867406ace0 2023-06-09T12:12:45.618 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:12:45.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 msgr2=0x7f867406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 0x7f867406d190 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f86780011e0 tx=0x7f8678009040 comp rx=0 tx=0).stop 2023-06-09T12:12:45.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 msgr2=0x7f86880c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:12:45.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f868000cfb0 tx=0x7f868000a060 comp rx=0 tx=0).stop 2023-06-09T12:12:45.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 shutdown_connections 2023-06-09T12:12:45.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86880c9260 0x7f86880c36d0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f867406ace0 0x7f867406d190 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f86880b5370 0x7f86881f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 --2- 172.21.15.7:0/3947688290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f86880c8e90 0x7f86880c3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:12:45.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 >> 172.21.15.7:0/3947688290 conn(0x7f8688070af0 msgr2=0x7f86881576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:12:45.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 shutdown_connections 2023-06-09T12:12:45.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:12:45.617+0000 7f868e1ff700 1 -- 172.21.15.7:0/3947688290 wait complete. 2023-06-09T12:12:45.623 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:12:45.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1929,"stamp":"2023-06-09T12:12:45.144208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102131,"ondisk_log_size":102131,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2588552,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":705216,"kb_avail":752386168,"statfs":{"total":773094113280,"available":770443436032,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":722141184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003692"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3485,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:58.045836+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:08:58.045836+0000","last_peered":"2023-06-09T12:08:58.045836+0000","last_clean":"2023-06-09T12:08:58.045836+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:08:58.045836+0000","last_undegraded":"2023-06-09T12:08:58.045836+0000","last_fullsized":"2023-06-09T12:08:58.045836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'895","reported_seq":3319,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:40.519000+0000","last_change":"2023-06-09T12:12:35.664853+0000","last_active":"2023-06-09T12:12:40.519000+0000","last_peered":"2023-06-09T12:12:40.519000+0000","last_clean":"2023-06-09T12:12:40.519000+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:12:40.519000+0000","last_undegraded":"2023-06-09T12:12:40.519000+0000","last_fullsized":"2023-06-09T12:12:40.519000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":49,"log_size":895,"log_dups_size":0,"ondisk_log_size":895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:16.502221+0000","last_change":"2023-06-09T12:12:06.494913+0000","last_active":"2023-06-09T12:12:16.502221+0000","last_peered":"2023-06-09T12:12:16.502221+0000","last_clean":"2023-06-09T12:12:16.502221+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:12:16.502221+0000","last_undegraded":"2023-06-09T12:12:16.502221+0000","last_fullsized":"2023-06-09T12:12:16.502221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3972,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:07:57.374073+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:07:57.374073+0000","last_peered":"2023-06-09T12:07:57.374073+0000","last_clean":"2023-06-09T12:07:57.374073+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:07:57.374073+0000","last_undegraded":"2023-06-09T12:07:57.374073+0000","last_fullsized":"2023-06-09T12:07:57.374073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3466,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:17.496908+0000","last_change":"2023-06-09T12:12:32.666639+0000","last_active":"2023-06-09T12:11:17.496908+0000","last_peered":"2023-06-09T12:11:17.496908+0000","last_clean":"2023-06-09T12:11:17.496908+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:11:17.496908+0000","last_undegraded":"2023-06-09T12:11:17.496908+0000","last_fullsized":"2023-06-09T12:11:17.496908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3747,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:25.482600+0000","last_change":"2023-06-09T12:12:04.518091+0000","last_active":"2023-06-09T12:12:25.482600+0000","last_peered":"2023-06-09T12:12:25.482600+0000","last_clean":"2023-06-09T12:12:25.482600+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:12:25.482600+0000","last_undegraded":"2023-06-09T12:12:25.482600+0000","last_fullsized":"2023-06-09T12:12:25.482600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4091,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:52.493483+0000","last_change":"2023-06-09T12:11:44.623056+0000","last_active":"2023-06-09T12:10:52.493483+0000","last_peered":"2023-06-09T12:10:52.493483+0000","last_clean":"2023-06-09T12:10:52.493483+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:52.493483+0000","last_undegraded":"2023-06-09T12:10:52.493483+0000","last_fullsized":"2023-06-09T12:10:52.493483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2268,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:02.347341+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:08:02.347341+0000","last_peered":"2023-06-09T12:08:02.347341+0000","last_clean":"2023-06-09T12:08:02.347341+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:08:02.347341+0000","last_undegraded":"2023-06-09T12:08:02.347341+0000","last_fullsized":"2023-06-09T12:08:02.347341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3445,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673812+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673812+0000","last_peered":"2023-06-09T12:10:56.673812+0000","last_clean":"2023-06-09T12:10:56.673812+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673812+0000","last_undegraded":"2023-06-09T12:10:56.673812+0000","last_fullsized":"2023-06-09T12:10:56.673812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712655+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712655+0000","last_peered":"2023-06-09T12:10:52.712655+0000","last_clean":"2023-06-09T12:10:52.712655+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712655+0000","last_undegraded":"2023-06-09T12:10:52.712655+0000","last_fullsized":"2023-06-09T12:10:52.712655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1091","reported_seq":3441,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:40.262381+0000","last_change":"2023-06-09T12:12:20.401920+0000","last_active":"2023-06-09T12:12:40.262381+0000","last_peered":"2023-06-09T12:12:40.262381+0000","last_clean":"2023-06-09T12:12:40.262381+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:40.262381+0000","last_undegraded":"2023-06-09T12:12:40.262381+0000","last_fullsized":"2023-06-09T12:12:40.262381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":147,"log_size":1091,"log_dups_size":0,"ondisk_log_size":1091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1117,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:17.451979+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:08:17.451979+0000","last_peered":"2023-06-09T12:08:17.451979+0000","last_clean":"2023-06-09T12:08:17.451979+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:08:17.451979+0000","last_undegraded":"2023-06-09T12:08:17.451979+0000","last_fullsized":"2023-06-09T12:08:17.451979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3869,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:15.301877+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:08:15.301877+0000","last_peered":"2023-06-09T12:08:15.301877+0000","last_clean":"2023-06-09T12:08:15.301877+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:08:15.301877+0000","last_undegraded":"2023-06-09T12:08:15.301877+0000","last_fullsized":"2023-06-09T12:08:15.301877+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.453267+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:08:27.453267+0000","last_peered":"2023-06-09T12:08:27.453267+0000","last_clean":"2023-06-09T12:08:27.453267+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:08:27.453267+0000","last_undegraded":"2023-06-09T12:08:27.453267+0000","last_fullsized":"2023-06-09T12:08:27.453267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":711,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.328419+0000","last_change":"2023-06-09T12:11:37.475265+0000","last_active":"2023-06-09T12:11:35.328419+0000","last_peered":"2023-06-09T12:11:35.328419+0000","last_clean":"2023-06-09T12:11:35.328419+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:35.328419+0000","last_undegraded":"2023-06-09T12:11:35.328419+0000","last_fullsized":"2023-06-09T12:11:35.328419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3659,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:01.674317+0000","last_change":"2023-06-09T12:11:32.498998+0000","last_active":"2023-06-09T12:11:01.674317+0000","last_peered":"2023-06-09T12:11:01.674317+0000","last_clean":"2023-06-09T12:11:01.674317+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:11:01.674317+0000","last_undegraded":"2023-06-09T12:11:01.674317+0000","last_fullsized":"2023-06-09T12:11:01.674317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3848,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:37.499717+0000","last_change":"2023-06-09T12:12:34.653035+0000","last_active":"2023-06-09T12:11:37.499717+0000","last_peered":"2023-06-09T12:11:37.499717+0000","last_clean":"2023-06-09T12:11:37.499717+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:11:37.499717+0000","last_undegraded":"2023-06-09T12:11:37.499717+0000","last_fullsized":"2023-06-09T12:11:37.499717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:32.543463+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:08:32.543463+0000","last_peered":"2023-06-09T12:08:32.543463+0000","last_clean":"2023-06-09T12:08:32.543463+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:08:32.543463+0000","last_undegraded":"2023-06-09T12:08:32.543463+0000","last_fullsized":"2023-06-09T12:08:32.543463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1033","reported_seq":3715,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:41.181588+0000","last_change":"2023-06-09T12:12:21.440139+0000","last_active":"2023-06-09T12:12:41.181588+0000","last_peered":"2023-06-09T12:12:41.181588+0000","last_clean":"2023-06-09T12:12:41.181588+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:12:41.181588+0000","last_undegraded":"2023-06-09T12:12:41.181588+0000","last_fullsized":"2023-06-09T12:12:41.181588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":98,"log_size":1033,"log_dups_size":0,"ondisk_log_size":1033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.380669+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:08:52.380669+0000","last_peered":"2023-06-09T12:08:52.380669+0000","last_clean":"2023-06-09T12:08:52.380669+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:08:52.380669+0000","last_undegraded":"2023-06-09T12:08:52.380669+0000","last_fullsized":"2023-06-09T12:08:52.380669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":176,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:27.347596+0000","last_change":"2023-06-09T12:10:58.600174+0000","last_active":"2023-06-09T12:11:27.347596+0000","last_peered":"2023-06-09T12:11:27.347596+0000","last_clean":"2023-06-09T12:11:27.347596+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:11:27.347596+0000","last_undegraded":"2023-06-09T12:11:27.347596+0000","last_fullsized":"2023-06-09T12:11:27.347596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:20.520984+0000","last_change":"2023-06-09T12:12:10.529212+0000","last_active":"2023-06-09T12:12:20.520984+0000","last_peered":"2023-06-09T12:12:20.520984+0000","last_clean":"2023-06-09T12:12:20.520984+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:12:20.520984+0000","last_undegraded":"2023-06-09T12:12:20.520984+0000","last_fullsized":"2023-06-09T12:12:20.520984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":166,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:27.350265+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:08:27.350265+0000","last_peered":"2023-06-09T12:08:27.350265+0000","last_clean":"2023-06-09T12:08:27.350265+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:08:27.350265+0000","last_undegraded":"2023-06-09T12:08:27.350265+0000","last_fullsized":"2023-06-09T12:08:27.350265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":840,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:23.459008+0000","last_change":"2023-06-09T12:12:13.519341+0000","last_active":"2023-06-09T12:12:23.459008+0000","last_peered":"2023-06-09T12:12:23.459008+0000","last_clean":"2023-06-09T12:12:23.459008+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:12:23.459008+0000","last_undegraded":"2023-06-09T12:12:23.459008+0000","last_fullsized":"2023-06-09T12:12:23.459008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3724,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:17.496741+0000","last_change":"2023-06-09T12:12:31.701489+0000","last_active":"2023-06-09T12:11:17.496741+0000","last_peered":"2023-06-09T12:11:17.496741+0000","last_clean":"2023-06-09T12:11:17.496741+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:11:17.496741+0000","last_undegraded":"2023-06-09T12:11:17.496741+0000","last_fullsized":"2023-06-09T12:11:17.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.364676+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:09:02.364676+0000","last_peered":"2023-06-09T12:09:02.364676+0000","last_clean":"2023-06-09T12:09:02.364676+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:09:02.364676+0000","last_undegraded":"2023-06-09T12:09:02.364676+0000","last_fullsized":"2023-06-09T12:09:02.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.423321+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:08:42.423321+0000","last_peered":"2023-06-09T12:08:42.423321+0000","last_clean":"2023-06-09T12:08:42.423321+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:08:42.423321+0000","last_undegraded":"2023-06-09T12:08:42.423321+0000","last_fullsized":"2023-06-09T12:08:42.423321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":71127,"ondisk_log_size":71127,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575595,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265680,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85824,"kb_avail":94106160,"statfs":{"total":96636764160,"available":96364707840,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87883776},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.68999999999999995,"15min":0.69099999999999995},"min":{"1min":0.51300000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.95799999999999996,"5min":1.0229999999999999,"15min":1.0640000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.57899999999999996,"5min":0.378,"15min":0.378},"max":{"1min":0.97099999999999997,"5min":1.202,"15min":1.3320000000000001},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73199999999999998,"15min":0.71299999999999997},"min":{"1min":0.57799999999999996,"5min":0.46700000000000003,"15min":0.38200000000000001},"max":{"1min":0.91300000000000003,"5min":0.92400000000000004,"15min":1.0229999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.747,"15min":0.73599999999999999},"min":{"1min":0.65300000000000002,"5min":0.48399999999999999,"15min":0.434},"max":{"1min":0.94599999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.71999999999999997,"15min":0.72599999999999998},"min":{"1min":0.55800000000000005,"5min":0.46100000000000002,"15min":0.44800000000000001},"max":{"1min":0.98199999999999998,"5min":1.0649999999999999,"15min":1.099},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74299999999999999,"15min":0.73899999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.39500000000000002},"max":{"1min":0.94299999999999995,"5min":1.1679999999999999,"15min":1.3089999999999999},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.4870000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.54200000000000004},"min":{"1min":0.39200000000000002,"5min":0.39000000000000001,"15min":0.28199999999999997},"max":{"1min":0.67800000000000005,"5min":0.89200000000000002,"15min":1.006},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73799999999999999,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.47799999999999998,"15min":0.32700000000000001},"max":{"1min":0.92800000000000005,"5min":1.226,"15min":1.226},"last":0.70399999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":1.3959999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60799999999999998,"15min":0.626},"min":{"1min":0.499,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.97999999999999998},"last":0.499}]},{"osd":6,"last update":"Fri Jun 9 12:11:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.64200000000000002,"15min":0.65900000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.626},{"interface":"front","average":{"1min":0.752,"5min":0.72699999999999998,"15min":0.74099999999999999},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.496},"max":{"1min":1.075,"5min":1.075,"15min":1.2869999999999999},"last":0.755}]}]},{"osd":6,"up_from":13,"seq":55834575595,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67100000000000004,"15min":0.66200000000000003},"min":{"1min":0.52100000000000002,"5min":0.45600000000000002,"15min":0.33100000000000002},"max":{"1min":0.875,"5min":0.99199999999999999,"15min":1.0129999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69699999999999995,"15min":0.68700000000000006},"min":{"1min":0.47599999999999998,"5min":0.45100000000000001,"15min":0.33500000000000002},"max":{"1min":0.91000000000000003,"5min":0.94099999999999995,"15min":1.069},"last":0.47899999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.71399999999999997,"15min":0.70099999999999996},"min":{"1min":0.501,"5min":0.41899999999999998,"15min":0.312},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.99299999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75900000000000001,"15min":0.75},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.43099999999999999},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.74099999999999999,"15min":0.71999999999999997},"min":{"1min":0.59399999999999997,"5min":0.51800000000000002,"15min":0.38},"max":{"1min":1.0209999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77100000000000002,"15min":0.75900000000000001},"min":{"1min":0.626,"5min":0.54700000000000004,"15min":0.46400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66600000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.74399999999999999},"min":{"1min":0.63700000000000001,"5min":0.47999999999999998,"15min":0.436},"max":{"1min":0.91900000000000004,"5min":1.046,"15min":1.046},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80500000000000005,"15min":0.79200000000000004},"min":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.53100000000000003},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.746}]},{"osd":4,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.58699999999999997},"min":{"1min":0.435,"5min":0.42199999999999999,"15min":0.36899999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60499999999999998,"15min":0.58999999999999997},"min":{"1min":0.42899999999999999,"5min":0.40200000000000002,"15min":0.35599999999999998},"max":{"1min":0.77100000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.59799999999999998,"15min":0.64900000000000002},"min":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76000000000000001,"15min":0.70899999999999996},"min":{"1min":0.60599999999999998,"5min":0.51900000000000002,"15min":0.33800000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:11:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.92800000000000005,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72799999999999998,"15min":0.68400000000000005},"min":{"1min":0.505,"5min":0.505,"15min":0.36299999999999999},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575596,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400432,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94016,"kb_avail":93971408,"statfs":{"total":96636764160,"available":96226721792,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96272384},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.375,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55900000000000005,"15min":0.56799999999999995},"min":{"1min":0.41899999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.70999999999999996,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.28699999999999998},"max":{"1min":0.61099999999999999,"5min":0.94599999999999995,"15min":0.995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54500000000000004,"15min":0.54700000000000004},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.34000000000000002},"max":{"1min":0.747,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61699999999999999,"15min":0.58799999999999997},"min":{"1min":0.45600000000000002,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77700000000000002,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.67800000000000005},"min":{"1min":0.5,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.92500000000000004,"5min":1.288,"15min":1.288},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.80900000000000005,"15min":0.75900000000000001},"min":{"1min":0.63600000000000001,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84899999999999998,"15min":0.84599999999999997},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0780000000000001,"5min":1.2829999999999999,"15min":1.597},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":0.99399999999999999,"5min":1.254,"15min":1.544},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.873,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.1020000000000001,"5min":1.1919999999999999,"15min":1.704},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.85299999999999998,"15min":0.83699999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1599999999999999,"5min":1.548,"15min":1.548},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83199999999999996,"15min":0.80600000000000005},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.1319999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.028,"5min":1.226,"15min":1.6519999999999999},"last":0.754},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.86899999999999999,"15min":0.82099999999999995},"min":{"1min":0.63800000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.115,"5min":1.498,"15min":1.498},"last":0.63400000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575595,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326632,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":94045208,"statfs":{"total":96636764160,"available":96302292992,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.46800000000000003},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.253},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.433},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.35799999999999998},"max":{"1min":0.73099999999999998,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.29899999999999999},"max":{"1min":0.73899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84099999999999997,"15min":0.80900000000000005},"min":{"1min":0.61099999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.86499999999999999},"min":{"1min":0.71099999999999997,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.2490000000000001,"5min":1.268,"15min":1.268},"last":0.97999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.121},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.89100000000000001,"15min":0.85899999999999999},"min":{"1min":0.68400000000000005,"5min":0.61799999999999999,"15min":0.53900000000000003},"max":{"1min":1.371,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.95099999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.82599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":9.0790000000000006},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82699999999999996,"15min":0.83799999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.1060000000000001,"5min":1.22,"15min":1.232},"last":1.0409999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.109,"5min":1.177,"15min":1.298},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.59499999999999997},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":1.0089999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575596,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231648,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51840,"kb_avail":94140192,"statfs":{"total":96636764160,"available":96399556608,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53084160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47899999999999998,"15min":0.45700000000000002},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.27800000000000002},"max":{"1min":0.65500000000000003,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.33000000000000002},"max":{"1min":0.67200000000000004,"5min":0.79900000000000004,"15min":0.999},"last":0.53100000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48499999999999999,"15min":0.47299999999999998},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.65200000000000002,"5min":1.236,"15min":1.236},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.622,"15min":0.622},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.72399999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.62,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.878,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.66700000000000004},"min":{"1min":0.46500000000000002,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.85399999999999998,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.80400000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.80700000000000005,"15min":0.77800000000000002},"min":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.443},"max":{"1min":1.056,"5min":1.202,"15min":1.202},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.81999999999999995},"min":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.3520000000000001,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.754},{"interface":"front","average":{"1min":0.876,"5min":0.89300000000000002,"15min":0.87},"min":{"1min":0.67200000000000004,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3899999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81100000000000005,"15min":0.80200000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":0.90200000000000002,"5min":1.1200000000000001,"15min":1.3029999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.82299999999999995,"15min":0.79500000000000004},"min":{"1min":0.60999999999999999,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":1.0189999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.85499999999999998,"15min":0.83199999999999996},"min":{"1min":0.66200000000000003,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.0980000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.878,"5min":0.86799999999999999,"15min":0.85199999999999998},"min":{"1min":0.57899999999999996,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.1240000000000001,"5min":1.1879999999999999,"15min":1.262},"last":0.85699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575595,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437180,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":113984,"kb_avail":93934660,"statfs":{"total":96636764160,"available":96189091840,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":116719616},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.223},"max":{"1min":0.57199999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.42199999999999999,"5min":0.37,"15min":0.33800000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.2130000000000001},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.95299999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60499999999999998,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.44600000000000001},"max":{"1min":0.77300000000000002,"5min":0.872,"15min":1.355},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.37,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.78200000000000003},"min":{"1min":0.70499999999999996,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.05,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81499999999999995,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.56100000000000005,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.171,"15min":3.081},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83599999999999997,"15min":0.82999999999999996},"min":{"1min":0.69899999999999995,"5min":0.52800000000000002,"15min":0.46800000000000003},"max":{"1min":1.0089999999999999,"5min":1.101,"15min":1.198},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.877},"min":{"1min":0.78400000000000003,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.1020000000000001,"5min":1.143,"15min":1.234},"last":0.91200000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.286},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.81399999999999995},"min":{"1min":0.65300000000000002,"5min":0.60999999999999999,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":1.159,"15min":1.159},"last":0.876}]},{"osd":7,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.0029999999999999,"5min":1.286,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.002,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.80300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575594,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413076,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120320,"kb_avail":93958764,"statfs":{"total":96636764160,"available":96213774336,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123207680},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.753,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.419,"15min":1.419},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.68300000000000005,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":1.294,"5min":1.383,"15min":1.383},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71099999999999997,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.747},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.044},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.64900000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.97599999999999998,"5min":0.99299999999999999,"15min":1.117},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.63900000000000001,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":1.1890000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.59499999999999997,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.014},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.84299999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":1.0509999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0029999999999999,"5min":1.0489999999999999,"15min":1.131},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.27400000000000002},"max":{"1min":0.84599999999999997,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.52900000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.091},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575594,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299352,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":81984,"kb_avail":94072488,"statfs":{"total":96636764160,"available":96330227712,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":83951616},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69499999999999995,"15min":0.71899999999999997},"min":{"1min":0.47699999999999998,"5min":0.41499999999999998,"15min":0.40100000000000002},"max":{"1min":0.83499999999999996,"5min":1.1100000000000001,"15min":5.4969999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.86199999999999999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.70899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.71599999999999997,"15min":0.78400000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.35599999999999998},"max":{"1min":0.97799999999999998,"5min":1.167,"15min":19.715},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.73199999999999998,"15min":0.76200000000000001},"min":{"1min":0.498,"5min":0.49099999999999999,"15min":0.439},"max":{"1min":0.91900000000000004,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72099999999999997,"15min":0.72499999999999998},"min":{"1min":0.61099999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92400000000000004,"5min":0.95899999999999996,"15min":5.5620000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":1.083,"15min":14.439},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.745,"15min":0.78100000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.254,"15min":12.130000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.80600000000000005,"15min":0.83399999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0049999999999999,"5min":1.274,"15min":12.218999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.53600000000000003,"15min":0.79500000000000004},"min":{"1min":0.42599999999999999,"5min":0.35999999999999999,"15min":0.35499999999999998},"max":{"1min":0.76100000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.56799999999999995,"15min":0.82799999999999996},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":67.521000000000001},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.59999999999999998,"15min":0.67200000000000004},"min":{"1min":0.46200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":12.163},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.72999999999999998,"15min":0.69499999999999995},"min":{"1min":0.55100000000000005,"5min":0.505,"15min":0.40799999999999997},"max":{"1min":0.94399999999999995,"5min":1.095,"15min":7.6040000000000001},"last":0.53700000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:11:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.66000000000000003},"min":{"1min":0.45100000000000001,"5min":0.45000000000000001,"15min":0.438},"max":{"1min":0.876,"5min":0.90700000000000003,"15min":3.8039999999999998},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73699999999999999,"15min":0.82299999999999995},"min":{"1min":0.63600000000000001,"5min":0.52300000000000002,"15min":0.46000000000000002},"max":{"1min":0.90200000000000002,"5min":1.238,"15min":19.834},"last":0.56399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:12:45.645 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.645 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.646 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.646 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.646 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.647 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.647 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.648 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.648 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.648 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.649 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.649 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.650 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.650 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.651 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.651 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.651 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.652 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.652 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.653 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.653 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:12:45.653 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:12:59.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:12:59.354 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:05.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:13:05.866 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.863+0000 7f85a91ff700 1 Processor -- start 2023-06-09T12:13:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.863+0000 7f85a91ff700 1 -- start start 2023-06-09T12:13:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f85a41535e0 0x7f85a4159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a41539b0 0x7f85a40c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40c4ce0 0x7f85a40c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a40c5a00 con 0x7f85a41539b0 2023-06-09T12:13:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a415a420 con 0x7f85a41535e0 2023-06-09T12:13:05.868 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a406b460 con 0x7f85a40c4ce0 2023-06-09T12:13:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a41539b0 0x7f85a40c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a259c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f85a41539b0 0x7f85a40c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60108/0 (socket says 172.21.15.7:60108) 2023-06-09T12:13:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.864+0000 7f85a259c700 1 -- 172.21.15.7:0/1539487927 learned_addr learned my addr 172.21.15.7:0/1539487927 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:13:05.870 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a2d9d700 1 --2- 172.21.15.7:0/1539487927 >> v2:172.21.15.117:3300/0 conn(0x7f85a41535e0 0x7f85a4159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a359e700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40c4ce0 0x7f85a40c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 -- 172.21.15.7:0/1539487927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40c4ce0 msgr2=0x7f85a40c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:13:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40c4ce0 0x7f85a40c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 -- 172.21.15.7:0/1539487927 >> v2:172.21.15.117:3300/0 conn(0x7f85a41535e0 msgr2=0x7f85a4159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:13:05.871 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 --2- 172.21.15.7:0/1539487927 >> v2:172.21.15.117:3300/0 conn(0x7f85a41535e0 0x7f85a4159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 -- 172.21.15.7:0/1539487927 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a406a570 con 0x7f85a41539b0 2023-06-09T12:13:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a259c700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41539b0 0x7f85a40c9f60 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f85a4073d90 tx=0x7f8598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ad0ab3247faecc server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.865+0000 7f85a1d9b700 1 -- 172.21.15.7:0/1539487927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598011830 con 0x7f85a41539b0 2023-06-09T12:13:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a1d9b700 1 -- 172.21.15.7:0/1539487927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598011990 con 0x7f85a41539b0 2023-06-09T12:13:05.872 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a1d9b700 1 -- 172.21.15.7:0/1539487927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598011d20 con 0x7f85a41539b0 2023-06-09T12:13:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 -- 172.21.15.7:0/1539487927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41539b0 msgr2=0x7f85a40c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41539b0 0x7f85a40c9f60 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f85a4073d90 tx=0x7f8598009bd0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 -- 172.21.15.7:0/1539487927 shutdown_connections 2023-06-09T12:13:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1539487927 >> v2:172.21.15.117:3300/0 conn(0x7f85a41535e0 0x7f85a4159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.873 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a40c4ce0 0x7f85a40c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1539487927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41539b0 0x7f85a40c9f60 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 -- 172.21.15.7:0/1539487927 >> 172.21.15.7:0/1539487927 conn(0x7f85a40737f0 msgr2=0x7f85a4072060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 -- 172.21.15.7:0/1539487927 shutdown_connections 2023-06-09T12:13:05.874 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.866+0000 7f85a91ff700 1 -- 172.21.15.7:0/1539487927 wait complete. 2023-06-09T12:13:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.868+0000 7f85a91ff700 1 Processor -- start 2023-06-09T12:13:05.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.868+0000 7f85a91ff700 1 -- start start 2023-06-09T12:13:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 0x7f85a41f0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a41539b0 0x7f85a41f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41ee5a0 con 0x7f85a41535e0 2023-06-09T12:13:05.877 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41f0840 con 0x7f85a40c4ce0 2023-06-09T12:13:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a91ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4156ea0 con 0x7f85a41539b0 2023-06-09T12:13:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a259c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60112/0 (socket says 172.21.15.7:60112) 2023-06-09T12:13:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a259c700 1 -- 172.21.15.7:0/1256252485 learned_addr learned my addr 172.21.15.7:0/1256252485 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:13:05.878 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a2d9d700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 0x7f85a41f0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a359e700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a41539b0 0x7f85a41f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.869+0000 7f85a259c700 1 -- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a41539b0 msgr2=0x7f85a41f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a259c700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a41539b0 0x7f85a41f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a259c700 1 -- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 msgr2=0x7f85a41f0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:05.879 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a259c700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 0x7f85a41f0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a259c700 1 -- 172.21.15.7:0/1256252485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8598009550 con 0x7f85a41535e0 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a2d9d700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 0x7f85a41f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a259c700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8598009c10 tx=0x7f8598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598004510 con 0x7f85a41535e0 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85980036a0 con 0x7f85a41535e0 2023-06-09T12:13:05.880 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a406ab30 con 0x7f85a41535e0 2023-06-09T12:13:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598003a10 con 0x7f85a41535e0 2023-06-09T12:13:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.870+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a406a850 con 0x7f85a41535e0 2023-06-09T12:13:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.871+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a406ae10 con 0x7f85a41535e0 2023-06-09T12:13:05.881 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.872+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8598003b70 con 0x7f85a41535e0 2023-06-09T12:13:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.873+0000 7f858b7fe700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 0x7f859006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.874+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f859808aeb0 con 0x7f85a41535e0 2023-06-09T12:13:05.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.874+0000 7f85a2d9d700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 0x7f859006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:05.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.874+0000 7f85a2d9d700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 0x7f859006d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8594009c30 tx=0x7f8594008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:05.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:05.878+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f8598055670 con 0x7f85a41535e0 2023-06-09T12:13:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.081+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85a41f31d0 con 0x7f859006ace0 2023-06-09T12:13:06.093 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.090+0000 7f858b7fe700 1 -- 172.21.15.7:0/1256252485 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318605 (secure 0 0 0) 0x7f85a41f31d0 con 0x7f859006ace0 2023-06-09T12:13:06.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:13:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 msgr2=0x7f859006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 0x7f859006d190 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8594009c30 tx=0x7f8594008040 comp rx=0 tx=0).stop 2023-06-09T12:13:06.099 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 msgr2=0x7f85a41f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8598009c10 tx=0x7f8598004750 comp rx=0 tx=0).stop 2023-06-09T12:13:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 shutdown_connections 2023-06-09T12:13:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f859006ace0 0x7f859006d190 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f85a41539b0 0x7f85a41f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:06.100 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f85a40c4ce0 0x7f85a41f0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 --2- 172.21.15.7:0/1256252485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85a41535e0 0x7f85a41f2670 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 >> 172.21.15.7:0/1256252485 conn(0x7f85a40737f0 msgr2=0x7f85a40caa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 shutdown_connections 2023-06-09T12:13:06.101 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:06.095+0000 7f85a91ff700 1 -- 172.21.15.7:0/1256252485 wait complete. 2023-06-09T12:13:06.102 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:13:06.116 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1939,"stamp":"2023-06-09T12:13:05.149784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102520,"ondisk_log_size":102520,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589192,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":705856,"kb_avail":752385528,"statfs":{"total":773094113280,"available":770442780672,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":722796544},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003360"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1041","reported_seq":3472,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:58.383846+0000","last_change":"2023-06-09T12:12:35.664853+0000","last_active":"2023-06-09T12:12:58.383846+0000","last_peered":"2023-06-09T12:12:58.383846+0000","last_clean":"2023-06-09T12:12:58.383846+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:12:58.383846+0000","last_undegraded":"2023-06-09T12:12:58.383846+0000","last_fullsized":"2023-06-09T12:12:58.383846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":195,"log_size":1041,"log_dups_size":0,"ondisk_log_size":1041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 22s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3608,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.470277+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:08:07.470277+0000","last_peered":"2023-06-09T12:08:07.470277+0000","last_clean":"2023-06-09T12:08:07.470277+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:08:07.470277+0000","last_undegraded":"2023-06-09T12:08:07.470277+0000","last_fullsized":"2023-06-09T12:08:07.470277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4210,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:16.502221+0000","last_change":"2023-06-09T12:12:06.494913+0000","last_active":"2023-06-09T12:12:16.502221+0000","last_peered":"2023-06-09T12:12:16.502221+0000","last_clean":"2023-06-09T12:12:16.502221+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:12:16.502221+0000","last_undegraded":"2023-06-09T12:12:16.502221+0000","last_fullsized":"2023-06-09T12:12:16.502221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3466,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:17.496908+0000","last_change":"2023-06-09T12:12:32.666639+0000","last_active":"2023-06-09T12:11:17.496908+0000","last_peered":"2023-06-09T12:11:17.496908+0000","last_clean":"2023-06-09T12:11:17.496908+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:11:17.496908+0000","last_undegraded":"2023-06-09T12:11:17.496908+0000","last_fullsized":"2023-06-09T12:11:17.496908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3747,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:25.482600+0000","last_change":"2023-06-09T12:12:04.518091+0000","last_active":"2023-06-09T12:12:25.482600+0000","last_peered":"2023-06-09T12:12:25.482600+0000","last_clean":"2023-06-09T12:12:25.482600+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:12:25.482600+0000","last_undegraded":"2023-06-09T12:12:25.482600+0000","last_fullsized":"2023-06-09T12:12:25.482600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:42.454962+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:08:42.454962+0000","last_peered":"2023-06-09T12:08:42.454962+0000","last_clean":"2023-06-09T12:08:42.454962+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:08:42.454962+0000","last_undegraded":"2023-06-09T12:08:42.454962+0000","last_fullsized":"2023-06-09T12:08:42.454962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3261,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353770+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:08:57.353770+0000","last_peered":"2023-06-09T12:08:57.353770+0000","last_clean":"2023-06-09T12:08:57.353770+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:08:57.353770+0000","last_undegraded":"2023-06-09T12:08:57.353770+0000","last_fullsized":"2023-06-09T12:08:57.353770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.382029+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:09:02.382029+0000","last_peered":"2023-06-09T12:09:02.382029+0000","last_clean":"2023-06-09T12:09:02.382029+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:09:02.382029+0000","last_undegraded":"2023-06-09T12:09:02.382029+0000","last_fullsized":"2023-06-09T12:09:02.382029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4091,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:10:52.493483+0000","last_change":"2023-06-09T12:11:44.623056+0000","last_active":"2023-06-09T12:10:52.493483+0000","last_peered":"2023-06-09T12:10:52.493483+0000","last_clean":"2023-06-09T12:10:52.493483+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:10:52.493483+0000","last_undegraded":"2023-06-09T12:10:52.493483+0000","last_fullsized":"2023-06-09T12:10:52.493483+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3582,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396814+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396814+0000","last_peered":"2023-06-09T12:10:50.396814+0000","last_clean":"2023-06-09T12:10:50.396814+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396814+0000","last_undegraded":"2023-06-09T12:10:50.396814+0000","last_fullsized":"2023-06-09T12:10:50.396814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":712,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:37.475265+0000","last_change":"2023-06-09T12:13:02.241593+0000","last_active":"2023-06-09T12:11:37.475265+0000","last_peered":"2023-06-09T12:11:37.475265+0000","last_clean":"2023-06-09T12:11:37.475265+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:11:37.475265+0000","last_undegraded":"2023-06-09T12:11:37.475265+0000","last_fullsized":"2023-06-09T12:11:37.475265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3659,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:01.674317+0000","last_change":"2023-06-09T12:11:32.498998+0000","last_active":"2023-06-09T12:11:01.674317+0000","last_peered":"2023-06-09T12:11:01.674317+0000","last_clean":"2023-06-09T12:11:01.674317+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:11:01.674317+0000","last_undegraded":"2023-06-09T12:11:01.674317+0000","last_fullsized":"2023-06-09T12:11:01.674317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3848,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:37.499717+0000","last_change":"2023-06-09T12:12:34.653035+0000","last_active":"2023-06-09T12:11:37.499717+0000","last_peered":"2023-06-09T12:11:37.499717+0000","last_clean":"2023-06-09T12:11:37.499717+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:11:37.499717+0000","last_undegraded":"2023-06-09T12:11:37.499717+0000","last_fullsized":"2023-06-09T12:11:37.499717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.476300+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:08:52.476300+0000","last_peered":"2023-06-09T12:08:52.476300+0000","last_clean":"2023-06-09T12:08:52.476300+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:08:52.476300+0000","last_undegraded":"2023-06-09T12:08:52.476300+0000","last_fullsized":"2023-06-09T12:08:52.476300+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.354989+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:09:07.354989+0000","last_peered":"2023-06-09T12:09:07.354989+0000","last_clean":"2023-06-09T12:09:07.354989+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:09:07.354989+0000","last_undegraded":"2023-06-09T12:09:07.354989+0000","last_fullsized":"2023-06-09T12:09:07.354989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1081,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382657+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:09:07.382657+0000","last_peered":"2023-06-09T12:09:07.382657+0000","last_clean":"2023-06-09T12:09:07.382657+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:09:07.382657+0000","last_undegraded":"2023-06-09T12:09:07.382657+0000","last_fullsized":"2023-06-09T12:09:07.382657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1179","reported_seq":3868,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:59.044437+0000","last_change":"2023-06-09T12:12:21.440139+0000","last_active":"2023-06-09T12:12:59.044437+0000","last_peered":"2023-06-09T12:12:59.044437+0000","last_clean":"2023-06-09T12:12:59.044437+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:12:59.044437+0000","last_undegraded":"2023-06-09T12:12:59.044437+0000","last_fullsized":"2023-06-09T12:12:59.044437+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":244,"log_size":1179,"log_dups_size":0,"ondisk_log_size":1179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.382601+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:09:07.382601+0000","last_peered":"2023-06-09T12:09:07.382601+0000","last_clean":"2023-06-09T12:09:07.382601+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:09:07.382601+0000","last_undegraded":"2023-06-09T12:09:07.382601+0000","last_fullsized":"2023-06-09T12:09:07.382601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:07.429132+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:09:07.429132+0000","last_peered":"2023-06-09T12:09:07.429132+0000","last_clean":"2023-06-09T12:09:07.429132+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:09:07.429132+0000","last_undegraded":"2023-06-09T12:09:07.429132+0000","last_fullsized":"2023-06-09T12:09:07.429132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4139,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.353786+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:08:57.353786+0000","last_peered":"2023-06-09T12:08:57.353786+0000","last_clean":"2023-06-09T12:08:57.353786+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:08:57.353786+0000","last_undegraded":"2023-06-09T12:08:57.353786+0000","last_fullsized":"2023-06-09T12:08:57.353786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":178,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:52.484000+0000","last_change":"2023-06-09T12:12:59.281132+0000","last_active":"2023-06-09T12:12:52.484000+0000","last_peered":"2023-06-09T12:12:52.484000+0000","last_clean":"2023-06-09T12:12:52.484000+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:12:52.484000+0000","last_undegraded":"2023-06-09T12:12:52.484000+0000","last_fullsized":"2023-06-09T12:12:52.484000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":2999,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:20.520984+0000","last_change":"2023-06-09T12:12:10.529212+0000","last_active":"2023-06-09T12:12:20.520984+0000","last_peered":"2023-06-09T12:12:20.520984+0000","last_clean":"2023-06-09T12:12:20.520984+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:12:20.520984+0000","last_undegraded":"2023-06-09T12:12:20.520984+0000","last_fullsized":"2023-06-09T12:12:20.520984+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.458323+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:09:12.458323+0000","last_peered":"2023-06-09T12:09:12.458323+0000","last_clean":"2023-06-09T12:09:12.458323+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:09:12.458323+0000","last_undegraded":"2023-06-09T12:09:12.458323+0000","last_fullsized":"2023-06-09T12:09:12.458323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:21.870328+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:09:21.870328+0000","last_peered":"2023-06-09T12:09:21.870328+0000","last_clean":"2023-06-09T12:09:21.870328+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:09:21.870328+0000","last_undegraded":"2023-06-09T12:09:21.870328+0000","last_fullsized":"2023-06-09T12:09:21.870328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":840,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:23.459008+0000","last_change":"2023-06-09T12:12:13.519341+0000","last_active":"2023-06-09T12:12:23.459008+0000","last_peered":"2023-06-09T12:12:23.459008+0000","last_clean":"2023-06-09T12:12:23.459008+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:12:23.459008+0000","last_undegraded":"2023-06-09T12:12:23.459008+0000","last_fullsized":"2023-06-09T12:12:23.459008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3724,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:17.496741+0000","last_change":"2023-06-09T12:12:31.701489+0000","last_active":"2023-06-09T12:11:17.496741+0000","last_peered":"2023-06-09T12:11:17.496741+0000","last_clean":"2023-06-09T12:11:17.496741+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:11:17.496741+0000","last_undegraded":"2023-06-09T12:11:17.496741+0000","last_fullsized":"2023-06-09T12:11:17.496741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:07.422802+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:08:07.422802+0000","last_peered":"2023-06-09T12:08:07.422802+0000","last_clean":"2023-06-09T12:08:07.422802+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:08:07.422802+0000","last_undegraded":"2023-06-09T12:08:07.422802+0000","last_fullsized":"2023-06-09T12:08:07.422802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":800,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616201+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616201+0000","last_peered":"2023-06-09T12:11:00.616201+0000","last_clean":"2023-06-09T12:11:00.616201+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616201+0000","last_undegraded":"2023-06-09T12:11:00.616201+0000","last_fullsized":"2023-06-09T12:11:00.616201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'866","reported_seq":3155,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.364676+0000","last_change":"2023-06-09T12:09:02.364676+0000","last_active":"2023-06-09T12:09:02.364676+0000","last_peered":"2023-06-09T12:09:02.364676+0000","last_clean":"2023-06-09T12:09:02.364676+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:09:02.364676+0000","last_undegraded":"2023-06-09T12:09:02.364676+0000","last_fullsized":"2023-06-09T12:09:02.364676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3944,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365964+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:09:12.365964+0000","last_peered":"2023-06-09T12:09:12.365964+0000","last_clean":"2023-06-09T12:09:12.365964+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:09:12.365964+0000","last_undegraded":"2023-06-09T12:09:12.365964+0000","last_fullsized":"2023-06-09T12:09:12.365964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":71516,"ondisk_log_size":71516,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575599,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265744,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85888,"kb_avail":94106096,"statfs":{"total":96636764160,"available":96364642304,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87949312},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.505,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0229999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.3320000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.0229999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.499,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.44800000000000001},"max":{"1min":0.85799999999999998,"5min":1.0089999999999999,"15min":1.0649999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.47099999999999997},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.3089999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:12:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.379},"max":{"1min":0.89700000000000002,"5min":0.93999999999999995,"15min":1.4450000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.58999999999999997,"5min":0.51700000000000002,"15min":0.45100000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":1.4870000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.89200000000000002,"15min":1.006},"last":0.503},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.56899999999999995,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.84299999999999997,"5min":1.0009999999999999,"15min":1.226},"last":0.72099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.93200000000000005},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.70699999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.82099999999999995,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.2869999999999999},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575599,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.94099999999999995,"15min":1.0129999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.70799999999999996,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.069},"last":0.84299999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.746},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.626,"5min":0.55400000000000005,"15min":0.50800000000000001},"max":{"1min":0.97499999999999998,"5min":1.032,"15min":1.032},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.877,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76800000000000002,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":1.046},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.64400000000000002,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":0.97499999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.876}]},{"osd":4,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.69899999999999995,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.70399999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.69099999999999995},"min":{"1min":0.54100000000000004,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.95799999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575600,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400560,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94144,"kb_avail":93971280,"statfs":{"total":96636764160,"available":96226590720,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96403456},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.44600000000000001},"min":{"1min":0.33600000000000002,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.377},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.745,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.94599999999999995,"15min":0.995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.504}]},{"osd":3,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.94599999999999995,"5min":1.288,"15min":1.288},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82199999999999995,"15min":0.77300000000000002},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.129,"5min":1.143,"15min":1.143},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83299999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.597},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:00 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":0.99399999999999999,"5min":1.254,"15min":1.544},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.83499999999999996,"15min":0.86299999999999999},"min":{"1min":0.68400000000000005,"5min":0.59499999999999997,"15min":0.57599999999999996},"max":{"1min":1.1020000000000001,"5min":1.1919999999999999,"15min":1.704},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85099999999999998,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1399999999999999,"5min":1.548,"15min":1.548},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.61099999999999999,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.077,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.58699999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.165,"5min":1.226,"15min":1.6519999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.873,"15min":0.83099999999999996},"min":{"1min":0.63400000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.0649999999999999,"5min":1.498,"15min":1.498},"last":0.63200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575599,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326760,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":98944,"kb_avail":94045080,"statfs":{"total":96636764160,"available":96302161920,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101318656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.46800000000000003},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.253},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.35799999999999998},"max":{"1min":0.73099999999999998,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.50600000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.29899999999999999},"max":{"1min":0.73899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64800000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84099999999999997,"15min":0.80900000000000005},"min":{"1min":0.61099999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.86499999999999999},"min":{"1min":0.71099999999999997,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.2490000000000001,"5min":1.268,"15min":1.268},"last":0.628}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.121},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.89100000000000001,"15min":0.85899999999999999},"min":{"1min":0.68400000000000005,"5min":0.61799999999999999,"15min":0.53900000000000003},"max":{"1min":1.371,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.82599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":9.0790000000000006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82699999999999996,"15min":0.83799999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.1060000000000001,"5min":1.22,"15min":1.232},"last":0.80400000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.109,"5min":1.177,"15min":1.298},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.59499999999999997},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575600,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231648,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51840,"kb_avail":94140192,"statfs":{"total":96636764160,"available":96399556608,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53084160},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.999},"last":0.44600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.34300000000000003,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.501},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.62}]},{"osd":3,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.53100000000000003,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.86399999999999999,"5min":1.319,"15min":1.4650000000000001},"last":0.78300000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.202},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.71599999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.63100000000000001,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.105,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.873},"min":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0409999999999999,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.99099999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.63800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":1.143,"5min":1.143,"15min":1.3029999999999999},"last":0.879},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.79800000000000004},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.51400000000000001},"max":{"1min":1.006,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1359999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.67600000000000005,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.024,"5min":1.1419999999999999,"15min":1.262},"last":0.93999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575599,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437244,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114048,"kb_avail":93934596,"statfs":{"total":96636764160,"available":96189026304,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":116785152},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.37,"15min":0.37},"max":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.95299999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60499999999999998,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.625}]},{"osd":2,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.44600000000000001},"max":{"1min":0.77300000000000002,"5min":0.872,"15min":1.355},"last":0.747},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.37,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.78200000000000003},"min":{"1min":0.70499999999999996,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.05,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81499999999999995,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.56100000000000005,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.171,"15min":3.081},"last":0.76400000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83599999999999997,"15min":0.82999999999999996},"min":{"1min":0.69899999999999995,"5min":0.52800000000000002,"15min":0.46800000000000003},"max":{"1min":1.0089999999999999,"5min":1.101,"15min":1.198},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.877},"min":{"1min":0.78400000000000003,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.1020000000000001,"5min":1.143,"15min":1.234},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.286},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.81399999999999995},"min":{"1min":0.65300000000000002,"5min":0.60999999999999999,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":1.159,"15min":1.159},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.0029999999999999,"5min":1.286,"15min":1.306},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.002,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.72999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575598,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413204,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120448,"kb_avail":93958636,"statfs":{"total":96636764160,"available":96213643264,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123338752},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.753,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.419,"15min":1.419},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.68300000000000005,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":1.294,"5min":1.383,"15min":1.383},"last":0.90400000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71099999999999997,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.044},"last":0.92700000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.64900000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.97599999999999998,"5min":0.99299999999999999,"15min":1.117},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.63900000000000001,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":1.1890000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.872}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.59499999999999997,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.014},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.84299999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.501}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":1.0509999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0029999999999999,"5min":1.0489999999999999,"15min":1.131},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.27400000000000002},"max":{"1min":0.84599999999999997,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.52900000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.091},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834575598,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299480,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82112,"kb_avail":94072360,"statfs":{"total":96636764160,"available":96330096640,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84082688},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41499999999999998},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":5.4969999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.80100000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.44,"15min":0.377},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":19.715},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.439},"max":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.42499999999999999},"max":{"1min":1.119,"5min":1.119,"15min":5.5620000000000003},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.92800000000000005,"5min":1.083,"15min":14.439},"last":0.92900000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74399999999999999,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":12.130000000000001},"last":0.88},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.67900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.79400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.82899999999999996},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":67.521000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.53700000000000003,"5min":0.505,"15min":0.41599999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":3.8039999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.51700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":19.834},"last":0.71199999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:13:06.124 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.124 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.125 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.125 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.125 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.126 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.126 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.127 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.127 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.127 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.128 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.128 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.129 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.129 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.130 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.130 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.130 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.131 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.131 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.132 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:06.132 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:13:26.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:13:26.347 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.345+0000 7f14a4fff700 1 Processor -- start 2023-06-09T12:13:26.348 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.345+0000 7f14a4fff700 1 -- start start 2023-06-09T12:13:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.346+0000 7f14a4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a015b4a0 0x7f14a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.346+0000 7f14a4fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f14a00c9f40 0x7f14a00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.346+0000 7f14a4fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a015a500 con 0x7f14a0153880 2023-06-09T12:13:26.349 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f14a4fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a00c5f10 con 0x7f14a00c9f40 2023-06-09T12:13:26.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f14a4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a015da00 con 0x7f14a015b4a0 2023-06-09T12:13:26.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149e59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f14a0153880 0x7f14a015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:54434/0 (socket says 172.21.15.7:54434) 2023-06-09T12:13:26.351 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149e59c700 1 -- 172.21.15.7:0/265374396 learned_addr learned my addr 172.21.15.7:0/265374396 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:13:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149ed9d700 1 --2- 172.21.15.7:0/265374396 >> v2:172.21.15.7:3300/0 conn(0x7f14a00c9f40 0x7f14a00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f1497fff700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a015b4a0 0x7f14a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149dd9b700 1 -- 172.21.15.7:0/265374396 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420402485 0 0) 0x7f1490001110 con 0x7f14a0153880 2023-06-09T12:13:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149dd9b700 1 -- 172.21.15.7:0/265374396 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f148c0035b0 con 0x7f14a0153880 2023-06-09T12:13:26.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149ed9d700 1 -- 172.21.15.7:0/265374396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a015b4a0 msgr2=0x7f14a015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:13:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149ed9d700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a015b4a0 0x7f14a015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149ed9d700 1 -- 172.21.15.7:0/265374396 >> v1:172.21.15.7:6789/0 conn(0x7f14a0153880 legacy=0x7f14a015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.347+0000 7f149ed9d700 1 -- 172.21.15.7:0/265374396 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a01596e0 con 0x7f14a00c9f40 2023-06-09T12:13:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f149ed9d700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a00c9f40 0x7f14a00c4e30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f1498009d90 tx=0x7f149800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41975133647ea21 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:26.353 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f149dd9b700 1 -- 172.21.15.7:0/265374396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800e070 con 0x7f14a00c9f40 2023-06-09T12:13:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f149dd9b700 1 -- 172.21.15.7:0/265374396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149800bb20 con 0x7f14a00c9f40 2023-06-09T12:13:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f149dd9b700 1 -- 172.21.15.7:0/265374396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800be90 con 0x7f14a00c9f40 2023-06-09T12:13:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f14a4fff700 1 -- 172.21.15.7:0/265374396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a00c9f40 msgr2=0x7f14a00c4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.348+0000 7f14a4fff700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a00c9f40 0x7f14a00c4e30 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f1498009d90 tx=0x7f149800b330 comp rx=0 tx=0).stop 2023-06-09T12:13:26.354 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 -- 172.21.15.7:0/265374396 shutdown_connections 2023-06-09T12:13:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a00c9f40 0x7f14a00c4e30 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 --2- 172.21.15.7:0/265374396 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a015b4a0 0x7f14a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 -- 172.21.15.7:0/265374396 >> 172.21.15.7:0/265374396 conn(0x7f14a00737f0 msgr2=0x7f14a00720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 -- 172.21.15.7:0/265374396 shutdown_connections 2023-06-09T12:13:26.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.349+0000 7f14a4fff700 1 -- 172.21.15.7:0/265374396 wait complete. 2023-06-09T12:13:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.351+0000 7f14a4fff700 1 Processor -- start 2023-06-09T12:13:26.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.351+0000 7f14a4fff700 1 -- start start 2023-06-09T12:13:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 0x7f14a01f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 0x7f14a0148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a01f2050 con 0x7f14a0153880 2023-06-09T12:13:26.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a015ca50 con 0x7f14a015b4a0 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.352+0000 7f14a4fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a00b3d80 con 0x7f14a00c9f40 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55956/0 (socket says 172.21.15.7:55956) 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 -- 172.21.15.7:0/2558607456 learned_addr learned my addr 172.21.15.7:0/2558607456 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f149e59c700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 0x7f14a01f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f149ed9d700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 0x7f14a0148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 -- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 msgr2=0x7f14a01f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 0x7f14a01f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 -- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 msgr2=0x7f14a0148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 0x7f14a0148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 -- 172.21.15.7:0/2558607456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498009a70 con 0x7f14a0153880 2023-06-09T12:13:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f149e59c700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 0x7f14a01f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:13:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f149ed9d700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 0x7f14a0148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:13:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.353+0000 7f1497fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f148800e870 tx=0x7f148800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.354+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488009590 con 0x7f14a0153880 2023-06-09T12:13:26.361 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.354+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148800ee70 con 0x7f14a0153880 2023-06-09T12:13:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.354+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148800ca50 con 0x7f14a0153880 2023-06-09T12:13:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.354+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a0159c60 con 0x7f14a0153880 2023-06-09T12:13:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.354+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a0159980 con 0x7f14a0153880 2023-06-09T12:13:26.362 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.355+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a0159f40 con 0x7f14a0153880 2023-06-09T12:13:26.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.356+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f148800cbb0 con 0x7f14a0153880 2023-06-09T12:13:26.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.357+0000 7f1496ffd700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 0x7f148006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:26.363 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.357+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f1488089d60 con 0x7f14a0153880 2023-06-09T12:13:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.357+0000 7f149e59c700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 0x7f148006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.358+0000 7f149e59c700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 0x7f148006d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1490005910 tx=0x7f1490009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:26.364 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.362+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1488057f20 con 0x7f14a0153880 2023-06-09T12:13:26.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.567+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14a01f1db0 con 0x7f148006ace0 2023-06-09T12:13:26.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.576+0000 7f1496ffd700 1 -- 172.21.15.7:0/2558607456 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318746 (secure 0 0 0) 0x7f14a01f1db0 con 0x7f148006ace0 2023-06-09T12:13:26.582 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:13:26.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.580+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 msgr2=0x7f148006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.580+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 0x7f148006d190 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1490005910 tx=0x7f1490009040 comp rx=0 tx=0).stop 2023-06-09T12:13:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.580+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 msgr2=0x7f14a01f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.580+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f148800e870 tx=0x7f148800ec30 comp rx=0 tx=0).stop 2023-06-09T12:13:26.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 shutdown_connections 2023-06-09T12:13:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f14a015b4a0 0x7f14a0148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f148006ace0 0x7f148006d190 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f14a00c9f40 0x7f14a01f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 --2- 172.21.15.7:0/2558607456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14a0153880 0x7f14a01f29b0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:26.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 >> 172.21.15.7:0/2558607456 conn(0x7f14a00737f0 msgr2=0x7f14a015b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 shutdown_connections 2023-06-09T12:13:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:26.581+0000 7f14a4fff700 1 -- 172.21.15.7:0/2558607456 wait complete. 2023-06-09T12:13:26.586 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:13:26.601 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1949,"stamp":"2023-06-09T12:13:25.155365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102717,"ondisk_log_size":102717,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2589704,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":706368,"kb_avail":752385016,"statfs":{"total":773094113280,"available":770442256384,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":723320832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003526"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3703,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550524+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:09:32.550524+0000","last_peered":"2023-06-09T12:09:32.550524+0000","last_clean":"2023-06-09T12:09:32.550524+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:09:32.550524+0000","last_undegraded":"2023-06-09T12:09:32.550524+0000","last_fullsized":"2023-06-09T12:09:32.550524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4212,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:12.537164+0000","last_change":"2023-06-09T12:12:06.494913+0000","last_active":"2023-06-09T12:13:12.537164+0000","last_peered":"2023-06-09T12:13:12.537164+0000","last_clean":"2023-06-09T12:13:12.537164+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:12.537164+0000","last_undegraded":"2023-06-09T12:13:12.537164+0000","last_fullsized":"2023-06-09T12:13:12.537164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3466,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:17.496908+0000","last_change":"2023-06-09T12:12:32.666639+0000","last_active":"2023-06-09T12:11:17.496908+0000","last_peered":"2023-06-09T12:11:17.496908+0000","last_clean":"2023-06-09T12:11:17.496908+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:11:17.496908+0000","last_undegraded":"2023-06-09T12:11:17.496908+0000","last_fullsized":"2023-06-09T12:11:17.496908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3748,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:12:30.534378+0000","last_change":"2023-06-09T12:12:04.518091+0000","last_active":"2023-06-09T12:12:30.534378+0000","last_peered":"2023-06-09T12:12:30.534378+0000","last_clean":"2023-06-09T12:12:30.534378+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:12:30.534378+0000","last_undegraded":"2023-06-09T12:12:30.534378+0000","last_fullsized":"2023-06-09T12:12:30.534378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3389,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:02.477649+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:09:02.477649+0000","last_peered":"2023-06-09T12:09:02.477649+0000","last_clean":"2023-06-09T12:09:02.477649+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:09:02.477649+0000","last_undegraded":"2023-06-09T12:09:02.477649+0000","last_fullsized":"2023-06-09T12:09:02.477649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4092,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:44.623056+0000","last_change":"2023-06-09T12:13:22.513656+0000","last_active":"2023-06-09T12:11:44.623056+0000","last_peered":"2023-06-09T12:11:44.623056+0000","last_clean":"2023-06-09T12:11:44.623056+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:11:44.623056+0000","last_undegraded":"2023-06-09T12:11:44.623056+0000","last_fullsized":"2023-06-09T12:11:44.623056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":367,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.371210+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:08:52.371210+0000","last_peered":"2023-06-09T12:08:52.371210+0000","last_clean":"2023-06-09T12:08:52.371210+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:08:52.371210+0000","last_undegraded":"2023-06-09T12:08:52.371210+0000","last_fullsized":"2023-06-09T12:08:52.371210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3416,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:47.352340+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:08:47.352340+0000","last_peered":"2023-06-09T12:08:47.352340+0000","last_clean":"2023-06-09T12:08:47.352340+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:08:47.352340+0000","last_undegraded":"2023-06-09T12:08:47.352340+0000","last_fullsized":"2023-06-09T12:08:47.352340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":715,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:13.317274+0000","last_change":"2023-06-09T12:13:02.241593+0000","last_active":"2023-06-09T12:13:13.317274+0000","last_peered":"2023-06-09T12:13:13.317274+0000","last_clean":"2023-06-09T12:13:13.317274+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:13:13.317274+0000","last_undegraded":"2023-06-09T12:13:13.317274+0000","last_fullsized":"2023-06-09T12:13:13.317274+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'891","reported_seq":3663,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:12.811675+0000","last_change":"2023-06-09T12:13:12.439206+0000","last_active":"2023-06-09T12:13:12.811675+0000","last_peered":"2023-06-09T12:13:12.811675+0000","last_clean":"2023-06-09T12:13:12.811675+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:12.811675+0000","last_undegraded":"2023-06-09T12:13:12.811675+0000","last_fullsized":"2023-06-09T12:13:12.811675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":891,"log_dups_size":0,"ondisk_log_size":891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1382,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.364142+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:08:57.364142+0000","last_peered":"2023-06-09T12:08:57.364142+0000","last_clean":"2023-06-09T12:08:57.364142+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:08:57.364142+0000","last_undegraded":"2023-06-09T12:08:57.364142+0000","last_fullsized":"2023-06-09T12:08:57.364142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3848,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:37.499717+0000","last_change":"2023-06-09T12:12:34.653035+0000","last_active":"2023-06-09T12:11:37.499717+0000","last_peered":"2023-06-09T12:11:37.499717+0000","last_clean":"2023-06-09T12:11:37.499717+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:11:37.499717+0000","last_undegraded":"2023-06-09T12:11:37.499717+0000","last_fullsized":"2023-06-09T12:11:37.499717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":761,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367237+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:09:22.367237+0000","last_peered":"2023-06-09T12:09:22.367237+0000","last_clean":"2023-06-09T12:09:22.367237+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:09:22.367237+0000","last_undegraded":"2023-06-09T12:09:22.367237+0000","last_fullsized":"2023-06-09T12:09:22.367237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237629+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237629+0000","last_peered":"2023-06-09T12:11:35.237629+0000","last_clean":"2023-06-09T12:11:35.237629+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237629+0000","last_undegraded":"2023-06-09T12:11:35.237629+0000","last_fullsized":"2023-06-09T12:11:35.237629+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:10.287973+0000","last_change":"2023-06-09T12:12:59.281132+0000","last_active":"2023-06-09T12:13:10.287973+0000","last_peered":"2023-06-09T12:13:10.287973+0000","last_clean":"2023-06-09T12:13:10.287973+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:13:10.287973+0000","last_undegraded":"2023-06-09T12:13:10.287973+0000","last_fullsized":"2023-06-09T12:13:10.287973+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3002,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:18.553729+0000","last_change":"2023-06-09T12:13:22.407018+0000","last_active":"2023-06-09T12:13:18.553729+0000","last_peered":"2023-06-09T12:13:18.553729+0000","last_clean":"2023-06-09T12:13:18.553729+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:18.553729+0000","last_undegraded":"2023-06-09T12:13:18.553729+0000","last_fullsized":"2023-06-09T12:13:18.553729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3421,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:31.820935+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:09:31.820935+0000","last_peered":"2023-06-09T12:09:31.820935+0000","last_clean":"2023-06-09T12:09:31.820935+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:09:31.820935+0000","last_undegraded":"2023-06-09T12:09:31.820935+0000","last_fullsized":"2023-06-09T12:09:31.820935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":528,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367445+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:09:22.367445+0000","last_peered":"2023-06-09T12:09:22.367445+0000","last_clean":"2023-06-09T12:09:22.367445+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:09:22.367445+0000","last_undegraded":"2023-06-09T12:09:22.367445+0000","last_fullsized":"2023-06-09T12:09:22.367445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":842,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:14.504651+0000","last_change":"2023-06-09T12:12:13.519341+0000","last_active":"2023-06-09T12:13:14.504651+0000","last_peered":"2023-06-09T12:13:14.504651+0000","last_clean":"2023-06-09T12:13:14.504651+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:13:14.504651+0000","last_undegraded":"2023-06-09T12:13:14.504651+0000","last_fullsized":"2023-06-09T12:13:14.504651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:31.701489+0000","last_change":"2023-06-09T12:13:22.513625+0000","last_active":"2023-06-09T12:12:31.701489+0000","last_peered":"2023-06-09T12:12:31.701489+0000","last_clean":"2023-06-09T12:12:31.701489+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:12:31.701489+0000","last_undegraded":"2023-06-09T12:12:31.701489+0000","last_fullsized":"2023-06-09T12:12:31.701489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":916,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:55.135076+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:08:55.135076+0000","last_peered":"2023-06-09T12:08:55.135076+0000","last_clean":"2023-06-09T12:08:55.135076+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:08:55.135076+0000","last_undegraded":"2023-06-09T12:08:55.135076+0000","last_fullsized":"2023-06-09T12:08:55.135076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:37.378757+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:08:37.378757+0000","last_peered":"2023-06-09T12:08:37.378757+0000","last_clean":"2023-06-09T12:08:37.378757+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:08:37.378757+0000","last_undegraded":"2023-06-09T12:08:37.378757+0000","last_fullsized":"2023-06-09T12:08:37.378757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3934,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:36.435334+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:08:36.435334+0000","last_peered":"2023-06-09T12:08:36.435334+0000","last_clean":"2023-06-09T12:08:36.435334+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:08:36.435334+0000","last_undegraded":"2023-06-09T12:08:36.435334+0000","last_fullsized":"2023-06-09T12:08:36.435334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'988","reported_seq":3287,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:21.110014+0000","last_change":"2023-06-09T12:13:04.265692+0000","last_active":"2023-06-09T12:13:21.110014+0000","last_peered":"2023-06-09T12:13:21.110014+0000","last_clean":"2023-06-09T12:13:21.110014+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:21.110014+0000","last_undegraded":"2023-06-09T12:13:21.110014+0000","last_fullsized":"2023-06-09T12:13:21.110014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":122,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3945,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.382450+0000","last_change":"2023-06-09T12:09:12.365964+0000","last_active":"2023-06-09T12:11:17.382450+0000","last_peered":"2023-06-09T12:11:17.382450+0000","last_clean":"2023-06-09T12:11:17.382450+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:11:17.382450+0000","last_undegraded":"2023-06-09T12:11:17.382450+0000","last_fullsized":"2023-06-09T12:11:17.382450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3660,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403275+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403275+0000","last_peered":"2023-06-09T12:11:39.403275+0000","last_clean":"2023-06-09T12:11:39.403275+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403275+0000","last_undegraded":"2023-06-09T12:11:39.403275+0000","last_fullsized":"2023-06-09T12:11:39.403275+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'900","reported_seq":2986,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.367340+0000","last_change":"2023-06-09T12:09:22.367340+0000","last_active":"2023-06-09T12:09:22.367340+0000","last_peered":"2023-06-09T12:09:22.367340+0000","last_clean":"2023-06-09T12:09:22.367340+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:09:22.367340+0000","last_undegraded":"2023-06-09T12:09:22.367340+0000","last_fullsized":"2023-06-09T12:09:22.367340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":71713,"ondisk_log_size":71713,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575603,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265744,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85888,"kb_avail":94106096,"statfs":{"total":96636764160,"available":96364642304,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":87949312},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.505,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0229999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.3320000000000001},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.0229999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.499,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.86699999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.44800000000000001},"max":{"1min":0.85799999999999998,"5min":1.0089999999999999,"15min":1.0649999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.47099999999999997},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.3089999999999999},"last":0.84999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48299999999999998,"15min":0.441},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.624,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.92700000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.89200000000000002,"15min":1.006},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.56899999999999995,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.84299999999999997,"5min":1.0009999999999999,"15min":1.226},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.93200000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.46700000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.82099999999999995,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.2869999999999999},"last":0.60699999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575603,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.94099999999999995,"15min":1.0129999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.70799999999999996,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.069},"last":0.61699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.626,"5min":0.55400000000000005,"15min":0.50800000000000001},"max":{"1min":0.97499999999999998,"5min":1.032,"15min":1.032},"last":0.69499999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.877,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76800000000000002,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":1.046},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.64400000000000002,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":0.97499999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.69899999999999995,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.70399999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.69099999999999995},"min":{"1min":0.54100000000000004,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.95799999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.71199999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575604,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":400624,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":94208,"kb_avail":93971216,"statfs":{"total":96636764160,"available":96226525184,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":96468992},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.44600000000000001},"min":{"1min":0.33600000000000002,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.436},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.745,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.94599999999999995,"15min":0.995},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.94599999999999995,"5min":1.288,"15min":1.288},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82199999999999995,"15min":0.77300000000000002},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.129,"5min":1.143,"15min":1.143},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83299999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.597},"last":0.77300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.82699999999999996},"min":{"1min":0.628,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.196,"5min":1.254,"15min":1.544},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.69799999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.048,"5min":1.1919999999999999,"15min":1.704},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85099999999999998,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1399999999999999,"5min":1.548,"15min":1.548},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.61099999999999999,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.077,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.165,"5min":1.226,"15min":1.6519999999999999},"last":0.751},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.873,"15min":0.83099999999999996},"min":{"1min":0.63400000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.0649999999999999,"5min":1.498,"15min":1.498},"last":0.82299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326824,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94045016,"statfs":{"total":96636764160,"available":96302096384,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.46800000000000003},"min":{"1min":0.33300000000000002,"5min":0.30599999999999999,"15min":0.253},"max":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.55500000000000005,"15min":0.54900000000000004},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.35799999999999998},"max":{"1min":0.73099999999999998,"5min":0.78900000000000003,"15min":0.82699999999999996},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54800000000000004,"15min":0.56999999999999995},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69299999999999995,"5min":0.81499999999999995,"15min":1.032},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52800000000000002,"15min":0.53300000000000003},"min":{"1min":0.42399999999999999,"5min":0.35699999999999998,"15min":0.29899999999999999},"max":{"1min":0.73899999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.435,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.81599999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.69499999999999995},"min":{"1min":0.54400000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.84099999999999997,"15min":0.80900000000000005},"min":{"1min":0.61099999999999999,"5min":0.42199999999999999,"15min":0.40100000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.89200000000000002,"15min":0.86499999999999999},"min":{"1min":0.71099999999999997,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.2490000000000001,"5min":1.268,"15min":1.268},"last":0.88100000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77400000000000002,"15min":0.78100000000000003},"min":{"1min":0.53900000000000003,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":0.95599999999999996,"5min":1.073,"15min":1.121},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.89100000000000001,"15min":0.85899999999999999},"min":{"1min":0.68400000000000005,"5min":0.61799999999999999,"15min":0.53900000000000003},"max":{"1min":1.371,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.84399999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.79200000000000004,"15min":0.82599999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":9.0790000000000006},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.82699999999999996,"15min":0.83799999999999997},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.1060000000000001,"5min":1.22,"15min":1.232},"last":0.93999999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.84999999999999998},"min":{"1min":0.53400000000000003,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.109,"5min":1.177,"15min":1.298},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.94199999999999995,"15min":0.93100000000000005},"min":{"1min":0.73799999999999999,"5min":0.66400000000000003,"15min":0.59499999999999997},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.96999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575604,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231776,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":51968,"kb_avail":94140064,"statfs":{"total":96636764160,"available":96399425536,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53215232},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.999},"last":0.55600000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.34300000000000003,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.53100000000000003,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.86399999999999999,"5min":1.319,"15min":1.4650000000000001},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.202},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.63100000000000001,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.105,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.873},"min":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0409999999999999,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.90100000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.63800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":1.143,"5min":1.143,"15min":1.3029999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.79800000000000004},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.51400000000000001},"max":{"1min":1.006,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.92400000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1359999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.67600000000000005,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.024,"5min":1.1419999999999999,"15min":1.262},"last":1.0089999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575603,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437372,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114176,"kb_avail":93934468,"statfs":{"total":96636764160,"available":96188895232,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":116916224},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.37,"15min":0.37},"max":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.53900000000000003,"15min":0.52400000000000002},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.69099999999999995,"5min":0.70299999999999996,"15min":0.95299999999999996},"last":0.439},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60499999999999998,"15min":0.54600000000000004},"min":{"1min":0.45000000000000001,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.81499999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64800000000000002,"15min":0.64500000000000002},"min":{"1min":0.52700000000000002,"5min":0.48899999999999999,"15min":0.44600000000000001},"max":{"1min":0.77300000000000002,"5min":0.872,"15min":1.355},"last":0.623},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.38,"5min":0.37,"15min":0.312},"max":{"1min":0.78400000000000003,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.78200000000000003},"min":{"1min":0.70499999999999996,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.05,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.81499999999999995,"15min":0.79800000000000004},"min":{"1min":0.65600000000000003,"5min":0.56100000000000005,"15min":0.47399999999999998},"max":{"1min":1.01,"5min":1.171,"15min":3.081},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.83599999999999997,"15min":0.82999999999999996},"min":{"1min":0.69899999999999995,"5min":0.52800000000000002,"15min":0.46800000000000003},"max":{"1min":1.0089999999999999,"5min":1.101,"15min":1.198},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.91900000000000004,"5min":0.89700000000000002,"15min":0.877},"min":{"1min":0.78400000000000003,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.1020000000000001,"5min":1.143,"15min":1.234},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.66600000000000004,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0269999999999999,"5min":1.198,"15min":1.286},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.82099999999999995,"15min":0.81399999999999995},"min":{"1min":0.65300000000000002,"5min":0.60999999999999999,"15min":0.52500000000000002},"max":{"1min":0.92600000000000005,"5min":1.159,"15min":1.159},"last":0.69299999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:12:31 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.83699999999999997,"15min":0.82799999999999996},"min":{"1min":0.64000000000000001,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.0029999999999999,"5min":1.286,"15min":1.306},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85899999999999999,"15min":0.86599999999999999},"min":{"1min":0.71899999999999997,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.002,"5min":1.4099999999999999,"15min":3.1160000000000001},"last":0.77600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575602,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413268,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120512,"kb_avail":93958572,"statfs":{"total":96636764160,"available":96213577728,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123404288},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.753,"15min":0.72399999999999998},"min":{"1min":0.52800000000000002,"5min":0.38600000000000001,"15min":0.35699999999999998},"max":{"1min":1.266,"5min":1.419,"15min":1.419},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78600000000000003,"15min":0.77500000000000002},"min":{"1min":0.68300000000000005,"5min":0.51200000000000001,"15min":0.46200000000000002},"max":{"1min":1.294,"5min":1.383,"15min":1.383},"last":0.71599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71099999999999997,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.78000000000000003},"min":{"1min":0.59799999999999998,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.044},"last":0.747}]},{"osd":2,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.64900000000000002,"5min":0.39700000000000002,"15min":0.34300000000000003},"max":{"1min":0.97599999999999998,"5min":0.99299999999999999,"15min":1.117},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78600000000000003,"15min":0.76100000000000001},"min":{"1min":0.63900000000000001,"5min":0.441,"15min":0.38100000000000001},"max":{"1min":1.1890000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.80100000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.59499999999999997,"5min":0.46899999999999997,"15min":0.45700000000000002},"max":{"1min":1.008,"5min":1.008,"15min":1.014},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.54600000000000004,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.96299999999999997,"5min":1.111,"15min":1.111},"last":0.70499999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.40300000000000002,"5min":0.36199999999999999,"15min":0.30299999999999999},"max":{"1min":0.84299999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.5},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38400000000000001},"max":{"1min":0.86099999999999999,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.438,"5min":0.41399999999999998,"15min":0.33700000000000002},"max":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":1.0509999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.58299999999999996,"5min":0.54700000000000004,"15min":0.44},"max":{"1min":1.0029999999999999,"5min":1.0489999999999999,"15min":1.131},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:12:33 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.64200000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.27400000000000002},"max":{"1min":0.84599999999999997,"5min":0.92900000000000005,"15min":1.1990000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.52900000000000003,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.091},"last":0.73799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575602,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299544,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82176,"kb_avail":94072296,"statfs":{"total":96636764160,"available":96330031104,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84148224},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41499999999999998},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":5.4969999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.745}]},{"osd":1,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.44,"15min":0.377},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":19.715},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.439},"max":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.42499999999999999},"max":{"1min":1.119,"5min":1.119,"15min":5.5620000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.92800000000000005,"5min":1.083,"15min":14.439},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74399999999999999,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":12.130000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.67900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.70499999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.79400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.82899999999999996},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":67.521000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.53700000000000003,"5min":0.505,"15min":0.41599999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":3.8039999999999998},"last":0.623},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.51700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":19.834},"last":0.81100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:13:26.609 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.609 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.609 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.610 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.610 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.611 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.611 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.612 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.612 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.613 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.613 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.613 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.614 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.614 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.615 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.615 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.616 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.616 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:26.616 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:13:29.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:29.468 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:46.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:13:46.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.826+0000 7f305e5ff700 1 Processor -- start 2023-06-09T12:13:46.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.826+0000 7f305e5ff700 1 -- start start 2023-06-09T12:13:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.827+0000 7f305e5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3058153880 0x7f305815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.827+0000 7f305e5ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f30580c9f40 0x7f305815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.827+0000 7f305e5ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305815a500 con 0x7f305815b2d0 2023-06-09T12:13:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.827+0000 7f305e5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30580c5f10 con 0x7f3058153880 2023-06-09T12:13:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.827+0000 7f305e5ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305815da00 con 0x7f30580c9f40 2023-06-09T12:13:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3057fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f3058153880 0x7f305815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f305cb9c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f305815b2d0 0x7f30580c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:38284/0 (socket says 172.21.15.7:38284) 2023-06-09T12:13:46.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f305cb9c700 1 -- 172.21.15.7:0/3874687680 learned_addr learned my addr 172.21.15.7:0/3874687680 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:13:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f30577fe700 1 --2- 172.21.15.7:0/3874687680 >> v2:172.21.15.117:3300/0 conn(0x7f30580c9f40 0x7f305815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3056ffd700 1 -- 172.21.15.7:0/3874687680 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409086973 0 0) 0x7f3050001490 con 0x7f305815b2d0 2023-06-09T12:13:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3056ffd700 1 -- 172.21.15.7:0/3874687680 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30400035b0 con 0x7f305815b2d0 2023-06-09T12:13:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3057fff700 1 -- 172.21.15.7:0/3874687680 >> v2:172.21.15.117:3300/0 conn(0x7f30580c9f40 msgr2=0x7f305815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:13:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3057fff700 1 --2- 172.21.15.7:0/3874687680 >> v2:172.21.15.117:3300/0 conn(0x7f30580c9f40 0x7f305815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3057fff700 1 -- 172.21.15.7:0/3874687680 >> v1:172.21.15.7:6789/0 conn(0x7f305815b2d0 legacy=0x7f30580c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.828+0000 7f3057fff700 1 -- 172.21.15.7:0/3874687680 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30581596e0 con 0x7f3058153880 2023-06-09T12:13:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f3057fff700 1 --2- 172.21.15.7:0/3874687680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f305815d4c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f30580cce30 tx=0x7f3048009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72d95448692a082 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f3056ffd700 1 -- 172.21.15.7:0/3874687680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011830 con 0x7f3058153880 2023-06-09T12:13:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f3056ffd700 1 -- 172.21.15.7:0/3874687680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048011990 con 0x7f3058153880 2023-06-09T12:13:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f3056ffd700 1 -- 172.21.15.7:0/3874687680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048011d20 con 0x7f3058153880 2023-06-09T12:13:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 -- 172.21.15.7:0/3874687680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 msgr2=0x7f305815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 --2- 172.21.15.7:0/3874687680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f305815d4c0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f30580cce30 tx=0x7f3048009b20 comp rx=0 tx=0).stop 2023-06-09T12:13:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 -- 172.21.15.7:0/3874687680 shutdown_connections 2023-06-09T12:13:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 --2- 172.21.15.7:0/3874687680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f305815d4c0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 --2- 172.21.15.7:0/3874687680 >> v2:172.21.15.117:3300/0 conn(0x7f30580c9f40 0x7f305815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.829+0000 7f305e5ff700 1 -- 172.21.15.7:0/3874687680 >> 172.21.15.7:0/3874687680 conn(0x7f30580737f0 msgr2=0x7f30580720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.830+0000 7f305e5ff700 1 -- 172.21.15.7:0/3874687680 shutdown_connections 2023-06-09T12:13:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.830+0000 7f305e5ff700 1 -- 172.21.15.7:0/3874687680 wait complete. 2023-06-09T12:13:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.831+0000 7f305e5ff700 1 Processor -- start 2023-06-09T12:13:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.832+0000 7f305e5ff700 1 -- start start 2023-06-09T12:13:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.832+0000 7f305e5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30580c9f40 0x7f30580b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.832+0000 7f305e5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.832+0000 7f305e5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 0x7f30580cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f305e5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30580b4cc0 con 0x7f3058153880 2023-06-09T12:13:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f305e5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30580b3dd0 con 0x7f305815b2d0 2023-06-09T12:13:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f305e5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30581f3b20 con 0x7f30580c9f40 2023-06-09T12:13:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52744/0 (socket says 172.21.15.7:52744) 2023-06-09T12:13:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 -- 172.21.15.7:0/4246645079 learned_addr learned my addr 172.21.15.7:0/4246645079 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:13:46.841 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f3057fff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30580c9f40 0x7f30580b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f305cb9c700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 0x7f30580cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 -- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30580c9f40 msgr2=0x7f30580b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30580c9f40 0x7f30580b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 -- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 msgr2=0x7f30580cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 0x7f30580cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:46.842 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f30577fe700 1 -- 172.21.15.7:0/4246645079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480094a0 con 0x7f3058153880 2023-06-09T12:13:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.833+0000 7f305cb9c700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 0x7f30580cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:13:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f30577fe700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e9e0 tx=0x7f304c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c009710 con 0x7f3058153880 2023-06-09T12:13:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304c00c850 con 0x7f3058153880 2023-06-09T12:13:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304c00cbc0 con 0x7f3058153880 2023-06-09T12:13:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3058159c60 con 0x7f3058153880 2023-06-09T12:13:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.834+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3058159980 con 0x7f3058153880 2023-06-09T12:13:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.835+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3058159f40 con 0x7f3058153880 2023-06-09T12:13:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.836+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f304c00cd20 con 0x7f3058153880 2023-06-09T12:13:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.837+0000 7f3054ff9700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 0x7f304406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:13:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.837+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f304c089f10 con 0x7f3058153880 2023-06-09T12:13:46.845 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.837+0000 7f3057fff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 0x7f304406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:13:46.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.838+0000 7f3057fff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 0x7f304406d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f30480090b0 tx=0x7f3048008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:13:46.846 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:46.842+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f304c0545b0 con 0x7f3058153880 2023-06-09T12:13:47.051 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.049+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f305804e8c0 con 0x7f304406ace0 2023-06-09T12:13:47.063 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.061+0000 7f3054ff9700 1 -- 172.21.15.7:0/4246645079 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318719 (secure 0 0 0) 0x7f305804e8c0 con 0x7f304406ace0 2023-06-09T12:13:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:13:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 msgr2=0x7f304406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 0x7f304406d190 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f30480090b0 tx=0x7f3048008030 comp rx=0 tx=0).stop 2023-06-09T12:13:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 msgr2=0x7f30580cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:13:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f304c00e9e0 tx=0x7f304c00ecf0 comp rx=0 tx=0).stop 2023-06-09T12:13:47.069 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 shutdown_connections 2023-06-09T12:13:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f305815b2d0 0x7f30580cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f304406ace0 0x7f304406d190 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f30580c9f40 0x7f30580b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 --2- 172.21.15.7:0/4246645079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3058153880 0x7f30580cbd40 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:13:47.070 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 >> 172.21.15.7:0/4246645079 conn(0x7f30580737f0 msgr2=0x7f30581e28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:13:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 shutdown_connections 2023-06-09T12:13:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:13:47.066+0000 7f305e5ff700 1 -- 172.21.15.7:0/4246645079 wait complete. 2023-06-09T12:13:47.071 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:13:47.088 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1959,"stamp":"2023-06-09T12:13:45.161043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102999,"ondisk_log_size":102999,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594376,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711040,"kb_avail":752380344,"statfs":{"total":773094113280,"available":770437472256,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728104960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003350"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:52.455950+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:08:52.455950+0000","last_peered":"2023-06-09T12:08:52.455950+0000","last_clean":"2023-06-09T12:08:52.455950+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:08:52.455950+0000","last_undegraded":"2023-06-09T12:08:52.455950+0000","last_fullsized":"2023-06-09T12:08:52.455950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4214,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:31.627845+0000","last_change":"2023-06-09T12:12:06.494913+0000","last_active":"2023-06-09T12:13:31.627845+0000","last_peered":"2023-06-09T12:13:31.627845+0000","last_clean":"2023-06-09T12:13:31.627845+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:31.627845+0000","last_undegraded":"2023-06-09T12:13:31.627845+0000","last_fullsized":"2023-06-09T12:13:31.627845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3750,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:24.528858+0000","last_change":"2023-06-09T12:12:04.518091+0000","last_active":"2023-06-09T12:13:24.528858+0000","last_peered":"2023-06-09T12:13:24.528858+0000","last_clean":"2023-06-09T12:13:24.528858+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:13:24.528858+0000","last_undegraded":"2023-06-09T12:13:24.528858+0000","last_fullsized":"2023-06-09T12:13:24.528858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3284,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.360899+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:09:42.360899+0000","last_peered":"2023-06-09T12:09:42.360899+0000","last_clean":"2023-06-09T12:09:42.360899+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:09:42.360899+0000","last_undegraded":"2023-06-09T12:09:42.360899+0000","last_fullsized":"2023-06-09T12:09:42.360899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4092,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:44.623056+0000","last_change":"2023-06-09T12:13:22.513656+0000","last_active":"2023-06-09T12:11:44.623056+0000","last_peered":"2023-06-09T12:11:44.623056+0000","last_clean":"2023-06-09T12:11:44.623056+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:11:44.623056+0000","last_undegraded":"2023-06-09T12:11:44.623056+0000","last_fullsized":"2023-06-09T12:11:44.623056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2970,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.365941+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:09:12.365941+0000","last_peered":"2023-06-09T12:09:12.365941+0000","last_clean":"2023-06-09T12:09:12.365941+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:09:12.365941+0000","last_undegraded":"2023-06-09T12:09:12.365941+0000","last_fullsized":"2023-06-09T12:09:12.365941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1043,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:42.500508+0000","last_change":"2023-06-09T12:11:42.617002+0000","last_active":"2023-06-09T12:11:42.500508+0000","last_peered":"2023-06-09T12:11:42.500508+0000","last_clean":"2023-06-09T12:11:42.500508+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:11:42.500508+0000","last_undegraded":"2023-06-09T12:11:42.500508+0000","last_fullsized":"2023-06-09T12:11:42.500508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":596,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:12.373772+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:09:12.373772+0000","last_peered":"2023-06-09T12:09:12.373772+0000","last_clean":"2023-06-09T12:09:12.373772+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:09:12.373772+0000","last_undegraded":"2023-06-09T12:09:12.373772+0000","last_fullsized":"2023-06-09T12:09:12.373772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":717,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:24.372452+0000","last_change":"2023-06-09T12:13:02.241593+0000","last_active":"2023-06-09T12:13:24.372452+0000","last_peered":"2023-06-09T12:13:24.372452+0000","last_clean":"2023-06-09T12:13:24.372452+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:13:24.372452+0000","last_undegraded":"2023-06-09T12:13:24.372452+0000","last_fullsized":"2023-06-09T12:13:24.372452+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1038","reported_seq":3817,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:41.113110+0000","last_change":"2023-06-09T12:13:12.439206+0000","last_active":"2023-06-09T12:13:41.113110+0000","last_peered":"2023-06-09T12:13:41.113110+0000","last_clean":"2023-06-09T12:13:41.113110+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:41.113110+0000","last_undegraded":"2023-06-09T12:13:41.113110+0000","last_fullsized":"2023-06-09T12:13:41.113110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":147,"log_size":1038,"log_dups_size":0,"ondisk_log_size":1038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 28s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3787,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.550500+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:09:32.550500+0000","last_peered":"2023-06-09T12:09:32.550500+0000","last_clean":"2023-06-09T12:09:32.550500+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:09:32.550500+0000","last_undegraded":"2023-06-09T12:09:32.550500+0000","last_fullsized":"2023-06-09T12:09:32.550500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":729,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:28.567064+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:09:28.567064+0000","last_peered":"2023-06-09T12:09:28.567064+0000","last_clean":"2023-06-09T12:09:28.567064+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:09:28.567064+0000","last_undegraded":"2023-06-09T12:09:28.567064+0000","last_fullsized":"2023-06-09T12:09:28.567064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3848,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:11:37.499717+0000","last_change":"2023-06-09T12:12:34.653035+0000","last_active":"2023-06-09T12:11:37.499717+0000","last_peered":"2023-06-09T12:11:37.499717+0000","last_clean":"2023-06-09T12:11:37.499717+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:11:37.499717+0000","last_undegraded":"2023-06-09T12:11:37.499717+0000","last_fullsized":"2023-06-09T12:11:37.499717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3458,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.481994+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:09:32.481994+0000","last_peered":"2023-06-09T12:09:32.481994+0000","last_clean":"2023-06-09T12:09:32.481994+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:09:32.481994+0000","last_undegraded":"2023-06-09T12:09:32.481994+0000","last_fullsized":"2023-06-09T12:09:32.481994+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384752+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:11:32.384752+0000","last_peered":"2023-06-09T12:11:32.384752+0000","last_clean":"2023-06-09T12:11:32.384752+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:11:32.384752+0000","last_undegraded":"2023-06-09T12:11:32.384752+0000","last_fullsized":"2023-06-09T12:11:32.384752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1291,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.362141+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:09:52.362141+0000","last_peered":"2023-06-09T12:09:52.362141+0000","last_clean":"2023-06-09T12:09:52.362141+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:09:52.362141+0000","last_undegraded":"2023-06-09T12:09:52.362141+0000","last_fullsized":"2023-06-09T12:09:52.362141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":321,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.463604+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:09:57.463604+0000","last_peered":"2023-06-09T12:09:57.463604+0000","last_clean":"2023-06-09T12:09:57.463604+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:09:57.463604+0000","last_undegraded":"2023-06-09T12:09:57.463604+0000","last_fullsized":"2023-06-09T12:09:57.463604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3592,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.552574+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:09:52.552574+0000","last_peered":"2023-06-09T12:09:52.552574+0000","last_clean":"2023-06-09T12:09:52.552574+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:09:52.552574+0000","last_undegraded":"2023-06-09T12:09:52.552574+0000","last_fullsized":"2023-06-09T12:09:52.552574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":183,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:23.337854+0000","last_change":"2023-06-09T12:12:59.281132+0000","last_active":"2023-06-09T12:13:23.337854+0000","last_peered":"2023-06-09T12:13:23.337854+0000","last_clean":"2023-06-09T12:13:23.337854+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:13:23.337854+0000","last_undegraded":"2023-06-09T12:13:23.337854+0000","last_fullsized":"2023-06-09T12:13:23.337854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3003,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.407018+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:13:22.407018+0000","last_peered":"2023-06-09T12:13:22.407018+0000","last_clean":"2023-06-09T12:13:22.407018+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:22.407018+0000","last_undegraded":"2023-06-09T12:13:22.407018+0000","last_fullsized":"2023-06-09T12:13:22.407018+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":852,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.360227+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:09:37.360227+0000","last_peered":"2023-06-09T12:09:37.360227+0000","last_clean":"2023-06-09T12:09:37.360227+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:09:37.360227+0000","last_undegraded":"2023-06-09T12:09:37.360227+0000","last_fullsized":"2023-06-09T12:09:37.360227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384938+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:11:32.384938+0000","last_peered":"2023-06-09T12:11:32.384938+0000","last_clean":"2023-06-09T12:11:32.384938+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:11:32.384938+0000","last_undegraded":"2023-06-09T12:11:32.384938+0000","last_fullsized":"2023-06-09T12:11:32.384938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":180,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:43.473569+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:09:43.473569+0000","last_peered":"2023-06-09T12:09:43.473569+0000","last_clean":"2023-06-09T12:09:43.473569+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:09:43.473569+0000","last_undegraded":"2023-06-09T12:09:43.473569+0000","last_fullsized":"2023-06-09T12:09:43.473569+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":844,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:38.708690+0000","last_change":"2023-06-09T12:13:42.409902+0000","last_active":"2023-06-09T12:13:38.708690+0000","last_peered":"2023-06-09T12:13:38.708690+0000","last_clean":"2023-06-09T12:13:38.708690+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:13:38.708690+0000","last_undegraded":"2023-06-09T12:13:38.708690+0000","last_fullsized":"2023-06-09T12:13:38.708690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:31.701489+0000","last_change":"2023-06-09T12:13:22.513625+0000","last_active":"2023-06-09T12:12:31.701489+0000","last_peered":"2023-06-09T12:12:31.701489+0000","last_clean":"2023-06-09T12:12:31.701489+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:12:31.701489+0000","last_undegraded":"2023-06-09T12:12:31.701489+0000","last_fullsized":"2023-06-09T12:12:31.701489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3830,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:08:57.476755+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:08:57.476755+0000","last_peered":"2023-06-09T12:08:57.476755+0000","last_clean":"2023-06-09T12:08:57.476755+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:08:57.476755+0000","last_undegraded":"2023-06-09T12:08:57.476755+0000","last_fullsized":"2023-06-09T12:08:57.476755+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3946,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.398742+0000","last_change":"2023-06-09T12:13:37.318635+0000","last_active":"2023-06-09T12:13:22.398742+0000","last_peered":"2023-06-09T12:13:22.398742+0000","last_clean":"2023-06-09T12:13:22.398742+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:22.398742+0000","last_undegraded":"2023-06-09T12:13:22.398742+0000","last_fullsized":"2023-06-09T12:13:22.398742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'924","reported_seq":3015,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:40.241405+0000","last_change":"2023-06-09T12:13:38.348067+0000","last_active":"2023-06-09T12:13:40.241405+0000","last_peered":"2023-06-09T12:13:40.241405+0000","last_clean":"2023-06-09T12:13:40.241405+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:13:40.241405+0000","last_undegraded":"2023-06-09T12:13:40.241405+0000","last_fullsized":"2023-06-09T12:13:40.241405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":24,"log_size":924,"log_dups_size":0,"ondisk_log_size":924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":71995,"ondisk_log_size":71995,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575607,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265808,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":85952,"kb_avail":94106032,"statfs":{"total":96636764160,"available":96364576768,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88014848},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68300000000000005,"15min":0.69299999999999995},"min":{"1min":0.52900000000000003,"5min":0.505,"15min":0.33700000000000002},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0229999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.72299999999999998,"15min":0.71599999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.378},"max":{"1min":1.014,"5min":1.014,"15min":1.3320000000000001},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.55500000000000005,"5min":0.46700000000000003,"15min":0.41999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":1.0229999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.74399999999999999,"15min":0.73999999999999999},"min":{"1min":0.499,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.94499999999999995,"5min":0.94699999999999995,"15min":1.05},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71399999999999997,"15min":0.72599999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.44800000000000001},"max":{"1min":0.85799999999999998,"5min":1.0089999999999999,"15min":1.0649999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.74399999999999999,"15min":0.74299999999999999},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.47099999999999997},"max":{"1min":0.88300000000000001,"5min":0.96599999999999997,"15min":1.3089999999999999},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48299999999999998,"15min":0.441},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.624,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.92700000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.72899999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.77500000000000002,"5min":0.89200000000000002,"15min":1.006},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72799999999999998,"15min":0.71999999999999997},"min":{"1min":0.56899999999999995,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":0.84299999999999997,"5min":1.0009999999999999,"15min":1.226},"last":0.84699999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.93200000000000005},"last":0.745},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.47099999999999997,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.85799999999999998,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.64400000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64200000000000002,"15min":0.65700000000000003},"min":{"1min":0.498,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.82099999999999995,"5min":1.0409999999999999,"15min":1.2230000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72999999999999998,"15min":0.74199999999999999},"min":{"1min":0.60699999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":1.075,"15min":1.2869999999999999},"last":0.82299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575607,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":0.82799999999999996,"5min":0.94099999999999995,"15min":1.0129999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.70799999999999996,"15min":0.69899999999999995},"min":{"1min":0.47899999999999998,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.069},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71199999999999997,"15min":0.70799999999999996},"min":{"1min":0.50900000000000001,"5min":0.41899999999999998,"15min":0.39800000000000002},"max":{"1min":0.93899999999999995,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.626,"5min":0.55400000000000005,"15min":0.50800000000000001},"max":{"1min":0.97499999999999998,"5min":1.032,"15min":1.032},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.42699999999999999},"max":{"1min":0.877,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.622},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.77600000000000002,"15min":0.76600000000000001},"min":{"1min":0.60299999999999998,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":0.93200000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.76800000000000002,"15min":0.754},"min":{"1min":0.52700000000000002,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0209999999999999,"5min":1.046,"15min":1.046},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80000000000000004,"15min":0.79800000000000004},"min":{"1min":0.64400000000000002,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":0.97499999999999998,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72399999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57099999999999995,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.69899999999999995,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.436},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59999999999999998,"15min":0.59199999999999997},"min":{"1min":0.45500000000000002,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.70399999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:12:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59899999999999998,"15min":0.64500000000000002},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.77400000000000002,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76400000000000001,"15min":0.71899999999999997},"min":{"1min":0.60499999999999998,"5min":0.51900000000000002,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:12:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66900000000000004,"15min":0.70199999999999996},"min":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.435},"max":{"1min":0.88200000000000001,"5min":1.0289999999999999,"15min":1.1100000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.73499999999999999,"15min":0.69099999999999995},"min":{"1min":0.54100000000000004,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.95799999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.55300000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575608,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404848,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98432,"kb_avail":93966992,"statfs":{"total":96636764160,"available":96222199808,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":100794368},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46300000000000002,"15min":0.44600000000000001},"min":{"1min":0.33600000000000002,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.58599999999999997,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56000000000000005,"15min":0.57299999999999995},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.311},"max":{"1min":0.745,"5min":0.84499999999999997,"15min":0.85099999999999998},"last":0.624}]},{"osd":2,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.308},"max":{"1min":0.67200000000000004,"5min":0.94599999999999995,"15min":0.995},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54400000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.69899999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.622,"15min":0.59199999999999997},"min":{"1min":0.45000000000000001,"5min":0.42199999999999999,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72099999999999997,"15min":0.68300000000000005},"min":{"1min":0.504,"5min":0.5,"15min":0.38700000000000001},"max":{"1min":0.94599999999999995,"5min":1.288,"15min":1.288},"last":0.82699999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82199999999999995,"15min":0.77300000000000002},"min":{"1min":0.53800000000000003,"5min":0.48599999999999999,"15min":0.45700000000000002},"max":{"1min":1.129,"5min":1.143,"15min":1.143},"last":1.2669999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.83299999999999996,"15min":0.84599999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.032,"5min":1.2829999999999999,"15min":1.597},"last":1.175}]},{"osd":5,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.82699999999999996},"min":{"1min":0.628,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.196,"5min":1.254,"15min":1.544},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.69799999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.048,"5min":1.1919999999999999,"15min":1.704},"last":1.1539999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.85099999999999998,"15min":0.83799999999999997},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.504},"max":{"1min":1.1399999999999999,"5min":1.548,"15min":1.548},"last":1.1910000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82899999999999996,"15min":0.81499999999999995},"min":{"1min":0.61099999999999999,"5min":0.52400000000000002,"15min":0.49399999999999999},"max":{"1min":1.077,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.052}]},{"osd":7,"last update":"Fri Jun 9 12:12:54 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.67700000000000005,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.165,"5min":1.226,"15min":1.6519999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.873,"15min":0.83099999999999996},"min":{"1min":0.63400000000000001,"5min":0.57899999999999996,"15min":0.48799999999999999},"max":{"1min":1.0649999999999999,"5min":1.498,"15min":1.498},"last":1.24}]}]},{"osd":0,"up_from":13,"seq":55834575607,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326824,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94045016,"statfs":{"total":96636764160,"available":96302096384,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51000000000000001,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72699999999999998,"5min":0.81499999999999995,"15min":1.032},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51800000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.71899999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.61499999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0680000000000001,"5min":1.421,"15min":1.421},"last":0.875},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.87},"min":{"1min":0.628,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.1830000000000001,"5min":1.268,"15min":1.268},"last":0.90300000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.121},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.874,"15min":0.87},"min":{"1min":0.748,"5min":0.63,"15min":0.53900000000000003},"max":{"1min":1.153,"5min":1.371,"15min":3.5840000000000001},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.53100000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0900000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.113,"5min":1.22,"15min":1.232},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.125,"5min":1.177,"15min":1.298},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":0.93899999999999995},"min":{"1min":0.76000000000000001,"5min":0.66400000000000003,"15min":0.60999999999999999},"max":{"1min":1.214,"5min":1.371,"15min":1.371},"last":0.92800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575608,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.47699999999999998,"15min":0.46100000000000002},"min":{"1min":0.32400000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.621,"5min":0.80600000000000005,"15min":0.94399999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.434,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.78600000000000003,"15min":0.999},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47999999999999998,"15min":0.47499999999999998},"min":{"1min":0.34300000000000003,"5min":0.32600000000000001,"15min":0.27800000000000002},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.40999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.75,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.39800000000000002,"15min":0.34399999999999997},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.53100000000000003,"5min":0.435,"15min":0.40500000000000003},"max":{"1min":0.86399999999999999,"5min":1.319,"15min":1.4650000000000001},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.51800000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.202},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.78400000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.443,"15min":0.41799999999999998},"max":{"1min":1.173,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.755}]},{"osd":5,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.63100000000000001,"5min":0.56200000000000006,"15min":0.51000000000000001},"max":{"1min":1.105,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.875,"15min":0.873},"min":{"1min":0.70899999999999996,"5min":0.66400000000000003,"15min":0.53400000000000003},"max":{"1min":1.0409999999999999,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.80500000000000005,"15min":0.80800000000000005},"min":{"1min":0.63800000000000001,"5min":0.50800000000000001,"15min":0.496},"max":{"1min":1.143,"5min":1.143,"15min":1.3029999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83099999999999996,"15min":0.79800000000000004},"min":{"1min":0.624,"5min":0.57199999999999995,"15min":0.51400000000000001},"max":{"1min":1.006,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.83099999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83599999999999997},"min":{"1min":0.56399999999999995,"5min":0.54000000000000004,"15min":0.497},"max":{"1min":1.1359999999999999,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.85999999999999999,"15min":0.85499999999999998},"min":{"1min":0.67600000000000005,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.024,"5min":1.1419999999999999,"15min":1.262},"last":0.89200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575607,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437500,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114304,"kb_avail":93934340,"statfs":{"total":96636764160,"available":96188764160,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117047296},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:57 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.25800000000000001},"max":{"1min":0.77800000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.443,"5min":0.37,"15min":0.37},"max":{"1min":0.75700000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57599999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.83999999999999997,"5min":0.872,"15min":0.97099999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.34100000000000003},"max":{"1min":0.81999999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.65400000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.65500000000000003,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.67600000000000005,"5min":0.56599999999999995,"15min":0.52100000000000002},"max":{"1min":1.0309999999999999,"5min":1.151,"15min":3.081},"last":0.94199999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.89700000000000002,"15min":0.88600000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.93100000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.627,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.68400000000000005,"5min":0.60999999999999999,"15min":0.54200000000000004},"max":{"1min":1.0309999999999999,"5min":1.159,"15min":1.159},"last":0.85299999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1659999999999999,"15min":1.306},"last":0.998},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":1.046}]}]},{"osd":4,"up_from":13,"seq":55834575606,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413460,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120704,"kb_avail":93958380,"statfs":{"total":96636764160,"available":96213381120,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123600896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.95699999999999996,"5min":1.419,"15min":1.419},"last":0.748},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.171,"5min":1.383,"15min":1.383},"last":0.67100000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71599999999999997,"15min":0.73499999999999999},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":1.226,"15min":1.226},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91200000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.117},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0680000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.57199999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.008,"15min":1.014},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.60599999999999998,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.71199999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.753,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":1.0509999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.83699999999999997,"5min":1.0489999999999999,"15min":1.131},"last":0.89600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.49199999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.624,"15min":0.64200000000000002},"min":{"1min":0.54700000000000004,"5min":0.38900000000000001,"15min":0.38300000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.72399999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575606,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299544,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82176,"kb_avail":94072296,"statfs":{"total":96636764160,"available":96330031104,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84148224},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68600000000000005,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41499999999999998},"max":{"1min":0.79600000000000004,"5min":0.89100000000000001,"15min":5.4969999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.40999999999999998},"max":{"1min":0.999,"5min":1.0469999999999999,"15min":5.6269999999999998},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70899999999999996,"15min":0.78900000000000003},"min":{"1min":0.54100000000000004,"5min":0.44,"15min":0.377},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":19.715},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72999999999999998,"15min":0.76800000000000002},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.439},"max":{"1min":0.95799999999999996,"5min":0.96799999999999997,"15min":7.4089999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.51400000000000001,"5min":0.48899999999999999,"15min":0.42499999999999999},"max":{"1min":1.119,"5min":1.119,"15min":5.5620000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73099999999999998,"15min":0.78000000000000003},"min":{"1min":0.58299999999999996,"5min":0.437,"15min":0.437},"max":{"1min":0.92800000000000005,"5min":1.083,"15min":14.439},"last":0.63500000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.74399999999999999,"15min":0.78500000000000003},"min":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":12.130000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81100000000000005,"15min":0.83899999999999997},"min":{"1min":0.67900000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54000000000000004,"15min":0.79400000000000004},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.84999999999999998,"15min":67.590000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.82899999999999996},"min":{"1min":0.438,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":67.521000000000001},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.61799999999999999,"15min":0.67600000000000005},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":0.92800000000000005,"15min":12.163},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73699999999999999,"15min":0.70699999999999996},"min":{"1min":0.53700000000000003,"5min":0.505,"15min":0.41599999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.51400000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:12:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":3.8039999999999998},"last":0.622},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73299999999999998,"15min":0.82599999999999996},"min":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.51700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":19.834},"last":0.77400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:13:47.096 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.097 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.097 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.097 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.098 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.098 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.099 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.099 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.099 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.100 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.100 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.101 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.101 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.102 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.102 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.102 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.103 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:13:47.103 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:13:59.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:13:59.599 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:07.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:14:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.313+0000 7f9bebfff700 1 Processor -- start 2023-06-09T12:14:07.317 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.313+0000 7f9bebfff700 1 -- start start 2023-06-09T12:14:07.318 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9bebfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9be40c5b40 0x7f9be40c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9bebfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be415dab0 con 0x7f9be40c6980 2023-06-09T12:14:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9bebfff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be4159aa0 con 0x7f9be4142850 2023-06-09T12:14:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9bebfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be415d210 con 0x7f9be40c5b40 2023-06-09T12:14:07.319 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9be9d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9be40c5b40 0x7f9be40c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:07.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9be9d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f9be40c5b40 0x7f9be40c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46246/0 (socket says 172.21.15.7:46246) 2023-06-09T12:14:07.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9be9d9b700 1 -- 172.21.15.7:0/1662386822 learned_addr learned my addr 172.21.15.7:0/1662386822 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:14:07.320 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.315+0000 7f9be959a700 1 --1- 172.21.15.7:0/1662386822 >> v1:172.21.15.7:6789/0 conn(0x7f9be40c6980 0x7f9be4159f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:36730/0 (socket says 172.21.15.7:36730) 2023-06-09T12:14:07.321 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be9d9b700 1 -- 172.21.15.7:0/1662386822 >> v1:172.21.15.117:6789/0 conn(0x7f9be4142850 legacy=0x7f9be40b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:14:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be9d9b700 1 -- 172.21.15.7:0/1662386822 >> v1:172.21.15.7:6789/0 conn(0x7f9be40c6980 legacy=0x7f9be4159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be8d99700 1 -- 172.21.15.7:0/1662386822 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001669553 0 0) 0x7f9be415dab0 con 0x7f9be40c6980 2023-06-09T12:14:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be9d9b700 1 -- 172.21.15.7:0/1662386822 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be415d4f0 con 0x7f9be40c5b40 2023-06-09T12:14:07.322 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be9d9b700 1 --2- 172.21.15.7:0/1662386822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be40c5b40 0x7f9be40c0b50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9be00098f0 tx=0x7f9be0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9990abcd6c2659 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be8d99700 1 -- 172.21.15.7:0/1662386822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be0004230 con 0x7f9be40c5b40 2023-06-09T12:14:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be8d99700 1 -- 172.21.15.7:0/1662386822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be0004390 con 0x7f9be40c5b40 2023-06-09T12:14:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.316+0000 7f9be8d99700 1 -- 172.21.15.7:0/1662386822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be00036a0 con 0x7f9be40c5b40 2023-06-09T12:14:07.323 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 -- 172.21.15.7:0/1662386822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be40c5b40 msgr2=0x7f9be40c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1662386822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be40c5b40 0x7f9be40c0b50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f9be00098f0 tx=0x7f9be0009c00 comp rx=0 tx=0).stop 2023-06-09T12:14:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 -- 172.21.15.7:0/1662386822 shutdown_connections 2023-06-09T12:14:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1662386822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be40c5b40 0x7f9be40c0b50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.324 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 -- 172.21.15.7:0/1662386822 >> 172.21.15.7:0/1662386822 conn(0x7f9be406a4d0 msgr2=0x7f9be406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 -- 172.21.15.7:0/1662386822 shutdown_connections 2023-06-09T12:14:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.317+0000 7f9bebfff700 1 -- 172.21.15.7:0/1662386822 wait complete. 2023-06-09T12:14:07.325 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.318+0000 7f9bebfff700 1 Processor -- start 2023-06-09T12:14:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.319+0000 7f9bebfff700 1 -- start start 2023-06-09T12:14:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40c5b40 0x7f9be413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 0x7f9be406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:07.327 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40ca5d0 con 0x7f9be4142850 2023-06-09T12:14:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be413f6b0 con 0x7f9be40c5b40 2023-06-09T12:14:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bebfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40cb820 con 0x7f9be40c6980 2023-06-09T12:14:07.328 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:46252/0 (socket says 172.21.15.7:46252) 2023-06-09T12:14:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 -- 172.21.15.7:0/1586725654 learned_addr learned my addr 172.21.15.7:0/1586725654 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:14:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9be9d9b700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40c5b40 0x7f9be413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9be959a700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 0x7f9be406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:07.329 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 -- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 msgr2=0x7f9be406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 0x7f9be406ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 -- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40c5b40 msgr2=0x7f9be413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40c5b40 0x7f9be413efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9bea59c700 1 -- 172.21.15.7:0/1586725654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0009550 con 0x7f9be4142850 2023-06-09T12:14:07.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.320+0000 7f9be959a700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 0x7f9be406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:14:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bea59c700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00b9a0 tx=0x7f9bdc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00c930 con 0x7f9be4142850 2023-06-09T12:14:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00ca90 con 0x7f9be4142850 2023-06-09T12:14:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc00ce40 con 0x7f9be4142850 2023-06-09T12:14:07.331 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be4159730 con 0x7f9be4142850 2023-06-09T12:14:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.321+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be415d7d0 con 0x7f9be4142850 2023-06-09T12:14:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.322+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be40b0fd0 con 0x7f9be4142850 2023-06-09T12:14:07.332 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.323+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9bdc01d3b0 con 0x7f9be4142850 2023-06-09T12:14:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.325+0000 7f9bda7fc700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 0x7f9bc806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.325+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f9bdc0898f0 con 0x7f9be4142850 2023-06-09T12:14:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.325+0000 7f9be9d9b700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 0x7f9bc806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:07.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.325+0000 7f9be9d9b700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 0x7f9bc806d1c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009130 tx=0x7f9be0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:07.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.330+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f9bdc0540b0 con 0x7f9be4142850 2023-06-09T12:14:07.537 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.534+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9be4074cf0 con 0x7f9bc806ad10 2023-06-09T12:14:07.546 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.543+0000 7f9bda7fc700 1 -- 172.21.15.7:0/1586725654 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318407 (secure 0 0 0) 0x7f9be4074cf0 con 0x7f9bc806ad10 2023-06-09T12:14:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:14:07.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 msgr2=0x7f9bc806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 0x7f9bc806d1c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009130 tx=0x7f9be0008030 comp rx=0 tx=0).stop 2023-06-09T12:14:07.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 msgr2=0x7f9be40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:07.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00b9a0 tx=0x7f9bdc00bcb0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.553 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 shutdown_connections 2023-06-09T12:14:07.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9be4142850 0x7f9be40ca9f0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f9be40c5b40 0x7f9be413efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f9bc806ad10 0x7f9bc806d1c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.547+0000 7f9bebfff700 1 --2- 172.21.15.7:0/1586725654 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f9be40c6980 0x7f9be406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:07.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.548+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 >> 172.21.15.7:0/1586725654 conn(0x7f9be406a4d0 msgr2=0x7f9be4070210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.548+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 shutdown_connections 2023-06-09T12:14:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:07.548+0000 7f9bebfff700 1 -- 172.21.15.7:0/1586725654 wait complete. 2023-06-09T12:14:07.555 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:14:07.569 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1970,"stamp":"2023-06-09T12:14:07.167226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103247,"ondisk_log_size":103247,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594952,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711616,"kb_avail":752379768,"statfs":{"total":773094113280,"available":770436882432,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728694784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003404"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3495,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.384549+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:09:22.384549+0000","last_peered":"2023-06-09T12:09:22.384549+0000","last_clean":"2023-06-09T12:09:22.384549+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:09:22.384549+0000","last_undegraded":"2023-06-09T12:09:22.384549+0000","last_fullsized":"2023-06-09T12:09:22.384549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4216,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:51.764433+0000","last_change":"2023-06-09T12:13:52.411188+0000","last_active":"2023-06-09T12:13:51.764433+0000","last_peered":"2023-06-09T12:13:51.764433+0000","last_clean":"2023-06-09T12:13:51.764433+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:51.764433+0000","last_undegraded":"2023-06-09T12:13:51.764433+0000","last_fullsized":"2023-06-09T12:13:51.764433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3534,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:32.385827+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:09:32.385827+0000","last_peered":"2023-06-09T12:09:32.385827+0000","last_clean":"2023-06-09T12:09:32.385827+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:09:32.385827+0000","last_undegraded":"2023-06-09T12:09:32.385827+0000","last_fullsized":"2023-06-09T12:09:32.385827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.461777+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:09:42.461777+0000","last_peered":"2023-06-09T12:09:42.461777+0000","last_clean":"2023-06-09T12:09:42.461777+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:09:42.461777+0000","last_undegraded":"2023-06-09T12:09:42.461777+0000","last_fullsized":"2023-06-09T12:09:42.461777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:42.483107+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:09:42.483107+0000","last_peered":"2023-06-09T12:09:42.483107+0000","last_clean":"2023-06-09T12:09:42.483107+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:09:42.483107+0000","last_undegraded":"2023-06-09T12:09:42.483107+0000","last_fullsized":"2023-06-09T12:09:42.483107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'922","reported_seq":3752,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:48.734660+0000","last_change":"2023-06-09T12:13:52.411201+0000","last_active":"2023-06-09T12:13:48.734660+0000","last_peered":"2023-06-09T12:13:48.734660+0000","last_clean":"2023-06-09T12:13:48.734660+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:13:48.734660+0000","last_undegraded":"2023-06-09T12:13:48.734660+0000","last_fullsized":"2023-06-09T12:13:48.734660+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":922,"log_dups_size":0,"ondisk_log_size":922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3649,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820650+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820650+0000","last_peered":"2023-06-09T12:12:18.820650+0000","last_clean":"2023-06-09T12:12:18.820650+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820650+0000","last_undegraded":"2023-06-09T12:12:18.820650+0000","last_fullsized":"2023-06-09T12:12:18.820650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384781+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:11:32.384781+0000","last_peered":"2023-06-09T12:11:32.384781+0000","last_clean":"2023-06-09T12:11:32.384781+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:11:32.384781+0000","last_undegraded":"2023-06-09T12:11:32.384781+0000","last_fullsized":"2023-06-09T12:11:32.384781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.387434+0000","last_change":"2023-06-09T12:14:02.519430+0000","last_active":"2023-06-09T12:13:57.387434+0000","last_peered":"2023-06-09T12:13:57.387434+0000","last_clean":"2023-06-09T12:13:57.387434+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:13:57.387434+0000","last_undegraded":"2023-06-09T12:13:57.387434+0000","last_fullsized":"2023-06-09T12:13:57.387434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3396,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:40.352210+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:09:40.352210+0000","last_peered":"2023-06-09T12:09:40.352210+0000","last_clean":"2023-06-09T12:09:40.352210+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:09:40.352210+0000","last_undegraded":"2023-06-09T12:09:40.352210+0000","last_fullsized":"2023-06-09T12:09:40.352210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:27.459795+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:09:27.459795+0000","last_peered":"2023-06-09T12:09:27.459795+0000","last_clean":"2023-06-09T12:09:27.459795+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:09:27.459795+0000","last_undegraded":"2023-06-09T12:09:27.459795+0000","last_fullsized":"2023-06-09T12:09:27.459795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:22.375293+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:09:22.375293+0000","last_peered":"2023-06-09T12:09:22.375293+0000","last_clean":"2023-06-09T12:09:22.375293+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:09:22.375293+0000","last_undegraded":"2023-06-09T12:09:22.375293+0000","last_fullsized":"2023-06-09T12:09:22.375293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3257,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.388763+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:09:57.388763+0000","last_peered":"2023-06-09T12:09:57.388763+0000","last_clean":"2023-06-09T12:09:57.388763+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:09:57.388763+0000","last_undegraded":"2023-06-09T12:09:57.388763+0000","last_fullsized":"2023-06-09T12:09:57.388763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":718,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:34.380216+0000","last_change":"2023-06-09T12:13:52.491270+0000","last_active":"2023-06-09T12:13:34.380216+0000","last_peered":"2023-06-09T12:13:34.380216+0000","last_clean":"2023-06-09T12:13:34.380216+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:13:34.380216+0000","last_undegraded":"2023-06-09T12:13:34.380216+0000","last_fullsized":"2023-06-09T12:13:34.380216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":635,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.461071+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:09:37.461071+0000","last_peered":"2023-06-09T12:09:37.461071+0000","last_clean":"2023-06-09T12:09:37.461071+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:09:37.461071+0000","last_undegraded":"2023-06-09T12:09:37.461071+0000","last_fullsized":"2023-06-09T12:09:37.461071+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:34.653035+0000","last_change":"2023-06-09T12:13:52.518166+0000","last_active":"2023-06-09T12:12:34.653035+0000","last_peered":"2023-06-09T12:12:34.653035+0000","last_clean":"2023-06-09T12:12:34.653035+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:12:34.653035+0000","last_undegraded":"2023-06-09T12:12:34.653035+0000","last_fullsized":"2023-06-09T12:12:34.653035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1664,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.485124+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:09:57.485124+0000","last_peered":"2023-06-09T12:09:57.485124+0000","last_clean":"2023-06-09T12:09:57.485124+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:09:57.485124+0000","last_undegraded":"2023-06-09T12:09:57.485124+0000","last_fullsized":"2023-06-09T12:09:57.485124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384752+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:11:32.384752+0000","last_peered":"2023-06-09T12:11:32.384752+0000","last_clean":"2023-06-09T12:11:32.384752+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:11:32.384752+0000","last_undegraded":"2023-06-09T12:11:32.384752+0000","last_fullsized":"2023-06-09T12:11:32.384752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379224+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379224+0000","last_peered":"2023-06-09T12:12:21.379224+0000","last_clean":"2023-06-09T12:12:21.379224+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379224+0000","last_undegraded":"2023-06-09T12:12:21.379224+0000","last_fullsized":"2023-06-09T12:12:21.379224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":183,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:23.337854+0000","last_change":"2023-06-09T12:12:59.281132+0000","last_active":"2023-06-09T12:13:23.337854+0000","last_peered":"2023-06-09T12:13:23.337854+0000","last_clean":"2023-06-09T12:13:23.337854+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:13:23.337854+0000","last_undegraded":"2023-06-09T12:13:23.337854+0000","last_fullsized":"2023-06-09T12:13:23.337854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:42.544795+0000","last_clean_scrub_stamp":"2023-06-09T11:10:42.544795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3005,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:43.756144+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:13:43.756144+0000","last_peered":"2023-06-09T12:13:43.756144+0000","last_clean":"2023-06-09T12:13:43.756144+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:43.756144+0000","last_undegraded":"2023-06-09T12:13:43.756144+0000","last_fullsized":"2023-06-09T12:13:43.756144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4406,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027553+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027553+0000","last_peered":"2023-06-09T12:12:03.027553+0000","last_clean":"2023-06-09T12:12:03.027553+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027553+0000","last_undegraded":"2023-06-09T12:12:03.027553+0000","last_fullsized":"2023-06-09T12:12:03.027553+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384938+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:11:32.384938+0000","last_peered":"2023-06-09T12:11:32.384938+0000","last_clean":"2023-06-09T12:11:32.384938+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:11:32.384938+0000","last_undegraded":"2023-06-09T12:11:32.384938+0000","last_fullsized":"2023-06-09T12:11:32.384938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'646","reported_seq":849,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:58.131423+0000","last_change":"2023-06-09T12:13:44.776820+0000","last_active":"2023-06-09T12:13:58.131423+0000","last_peered":"2023-06-09T12:13:58.131423+0000","last_clean":"2023-06-09T12:13:58.131423+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:13:58.131423+0000","last_undegraded":"2023-06-09T12:13:58.131423+0000","last_fullsized":"2023-06-09T12:13:58.131423+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'894","reported_seq":3725,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:31.701489+0000","last_change":"2023-06-09T12:13:22.513625+0000","last_active":"2023-06-09T12:12:31.701489+0000","last_peered":"2023-06-09T12:12:31.701489+0000","last_clean":"2023-06-09T12:12:31.701489+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:12:31.701489+0000","last_undegraded":"2023-06-09T12:12:31.701489+0000","last_fullsized":"2023-06-09T12:12:31.701489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1067,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.375317+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:10:22.375317+0000","last_peered":"2023-06-09T12:10:22.375317+0000","last_clean":"2023-06-09T12:10:22.375317+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:10:22.375317+0000","last_undegraded":"2023-06-09T12:10:22.375317+0000","last_fullsized":"2023-06-09T12:10:22.375317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3946,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.398742+0000","last_change":"2023-06-09T12:13:37.318635+0000","last_active":"2023-06-09T12:13:22.398742+0000","last_peered":"2023-06-09T12:13:22.398742+0000","last_clean":"2023-06-09T12:13:22.398742+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:22.398742+0000","last_undegraded":"2023-06-09T12:13:22.398742+0000","last_fullsized":"2023-06-09T12:13:22.398742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1095","reported_seq":3195,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:01.205574+0000","last_change":"2023-06-09T12:13:38.348067+0000","last_active":"2023-06-09T12:14:01.205574+0000","last_peered":"2023-06-09T12:14:01.205574+0000","last_clean":"2023-06-09T12:14:01.205574+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:01.205574+0000","last_undegraded":"2023-06-09T12:14:01.205574+0000","last_fullsized":"2023-06-09T12:14:01.205574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":195,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 24s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":72243,"ondisk_log_size":72243,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1647831,"num_omap_keys":3076,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30885,"ondisk_log_size":30885,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575611,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266000,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86144,"kb_avail":94105840,"statfs":{"total":96636764160,"available":96364380160,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88211456},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69599999999999995},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0229999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.143,"5min":1.143,"15min":1.3320000000000001},"last":0.49399999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":1.05},"last":0.66000000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0649999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.745},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3089999999999999},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.69899999999999995},"min":{"1min":0.54000000000000004,"5min":0.48299999999999998,"15min":0.441},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.72399999999999998},"min":{"1min":0.624,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.92700000000000005,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69899999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.505,"15min":0.45900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.226},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.752}]},{"osd":6,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.68600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575611,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.59599999999999997,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.59999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.93600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73099999999999998,"15min":0.72399999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42699999999999999},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.57599999999999996,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0189999999999999,"5min":1.046,"15min":1.046},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80800000000000005,"15min":0.80300000000000005},"min":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":1.028}]},{"osd":4,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.98799999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.79000000000000004,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77200000000000002,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":1.0860000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.69599999999999995},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.0609999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":1.0609999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575612,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404976,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98560,"kb_avail":93966864,"statfs":{"total":96636764160,"available":96222068736,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":100925440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.63600000000000001,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.83799999999999997,"15min":0.85099999999999998},"last":0.55500000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.38,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.995},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.69099999999999995},"min":{"1min":0.60899999999999999,"5min":0.5,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":1.288,"15min":1.288},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.83899999999999997,"15min":0.78700000000000003},"min":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.45700000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.84699999999999998},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.175,"5min":1.2829999999999999,"15min":1.597},"last":1.012}]},{"osd":5,"last update":"Fri Jun 9 12:13:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80200000000000005,"15min":0.82699999999999996},"min":{"1min":0.628,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.196,"5min":1.254,"15min":1.544},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.84199999999999997,"15min":0.86599999999999999},"min":{"1min":0.69799999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.048,"5min":1.1919999999999999,"15min":1.704},"last":0.89600000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.548},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.1319999999999999,"15min":1.2869999999999999},"last":1.0509999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.751,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.2190000000000001,"5min":1.226,"15min":1.6519999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.51400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.498},"last":0.95399999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575611,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326824,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99008,"kb_avail":94045016,"statfs":{"total":96636764160,"available":96302096384,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101384192},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51000000000000001,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72699999999999998,"5min":0.81499999999999995,"15min":1.032},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.67800000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.71899999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.88500000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0680000000000001,"5min":1.421,"15min":1.421},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.87},"min":{"1min":0.628,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.1830000000000001,"5min":1.268,"15min":1.268},"last":0.93000000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.121},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.874,"15min":0.87},"min":{"1min":0.748,"5min":0.63,"15min":0.53900000000000003},"max":{"1min":1.153,"5min":1.371,"15min":3.5840000000000001},"last":0.86899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.53100000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0900000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.113,"5min":1.22,"15min":1.232},"last":0.79000000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.125,"5min":1.177,"15min":1.298},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":0.93899999999999995},"min":{"1min":0.76000000000000001,"5min":0.66400000000000003,"15min":0.60999999999999999},"max":{"1min":1.214,"5min":1.371,"15min":1.371},"last":0.98599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575612,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.53800000000000003,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.999},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.625,"5min":0.66900000000000004,"15min":1.236},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.624},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.86699999999999999,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":1.319,"15min":1.4650000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58199999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.0980000000000001,"15min":1.202},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.624,"5min":0.443,"15min":0.443},"max":{"1min":0.99099999999999999,"5min":1.173,"15min":1.2669999999999999},"last":0.746}]},{"osd":5,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.52800000000000002},"max":{"1min":0.95799999999999996,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.008,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.872}]},{"osd":6,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.62,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98999999999999999,"5min":1.143,"15min":1.3029999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83599999999999997,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.79200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.67600000000000005,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.87,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.70799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.1419999999999999,"15min":1.262},"last":0.94999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575611,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437692,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114496,"kb_avail":93934148,"statfs":{"total":96636764160,"available":96188567552,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117243904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.48099999999999998},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":0.81499999999999995,"15min":0.81699999999999995},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.64400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.48699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.55700000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.83999999999999997,"5min":0.872,"15min":0.97099999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.34100000000000003},"max":{"1min":0.81999999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.68000000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.65500000000000003,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.873,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.67600000000000005,"5min":0.56599999999999995,"15min":0.52100000000000002},"max":{"1min":1.0309999999999999,"5min":1.151,"15min":3.081},"last":0.94799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.876},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.89700000000000002,"15min":0.88600000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.92800000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.627,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.68400000000000005,"5min":0.60999999999999999,"15min":0.54200000000000004},"max":{"1min":1.0309999999999999,"5min":1.159,"15min":1.159},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1659999999999999,"15min":1.306},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":0.73499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575610,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413524,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120768,"kb_avail":93958316,"statfs":{"total":96636764160,"available":96213315584,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123666432},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.95699999999999996,"5min":1.419,"15min":1.419},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.171,"5min":1.383,"15min":1.383},"last":0.878}]},{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71599999999999997,"15min":0.73499999999999999},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":1.226,"15min":1.226},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.117},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0680000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.012}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.57199999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.008,"15min":1.014},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.60599999999999998,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":0.91000000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.71199999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.753,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":1.0509999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.83699999999999997,"5min":1.0489999999999999,"15min":1.131},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.49199999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.623},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.624,"15min":0.64200000000000002},"min":{"1min":0.54700000000000004,"5min":0.38900000000000001,"15min":0.38300000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.60699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575610,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299544,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82176,"kb_avail":94072296,"statfs":{"total":96636764160,"available":96330031104,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84148224},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68000000000000005,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":5.4969999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71399999999999997,"15min":0.73299999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40999999999999998},"max":{"1min":1.579,"5min":1.579,"15min":5.6269999999999998},"last":0.66000000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.78700000000000003},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.377},"max":{"1min":0.87,"5min":1.2689999999999999,"15min":19.715},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":0.95799999999999996,"15min":7.4089999999999998},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.42499999999999999},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":5.5620000000000003},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.73199999999999998,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.391,"5min":1.391,"15min":14.439},"last":0.749}]},{"osd":3,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":12.130000000000001},"last":0.629},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.84399999999999997},"min":{"1min":0.53400000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.296,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.77700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.79300000000000004},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.522,"5min":1.522,"15min":67.590000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.82699999999999996},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.28,"5min":1.28,"15min":67.521000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":12.163},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.41599999999999998},"max":{"1min":1.121,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":3.8039999999999998},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.82799999999999996},"min":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.226,"5min":1.226,"15min":19.834},"last":0.68000000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:14:07.576 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.577 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.577 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.578 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.578 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.579 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.579 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.579 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.580 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.580 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-06-09T11:10:42.544795+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=42, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.581 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.581 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.581 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.582 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.582 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.583 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:07.583 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:14:27.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:14:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.793+0000 7faf097ff700 1 Processor -- start 2023-06-09T12:14:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.793+0000 7faf097ff700 1 -- start start 2023-06-09T12:14:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf097ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf040ca1b0 0x7faf0415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf097ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 0x7faf040c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf097ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0415dcf0 con 0x7faf040b5360 2023-06-09T12:14:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf097ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf040c5e00 con 0x7faf040ca1b0 2023-06-09T12:14:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf097ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0415d450 con 0x7faf040ca580 2023-06-09T12:14:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf040ca1b0 0x7faf0415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf02ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faf040ca1b0 0x7faf0415cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50122/0 (socket says 172.21.15.7:50122) 2023-06-09T12:14:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.794+0000 7faf02ffd700 1 -- 172.21.15.7:0/1638767949 learned_addr learned my addr 172.21.15.7:0/1638767949 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:14:27.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf02ffd700 1 -- 172.21.15.7:0/1638767949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 msgr2=0x7faf040c58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:14:27.799 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf037fe700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 0x7faf040c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf02ffd700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 0x7faf040c58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf02ffd700 1 -- 172.21.15.7:0/1638767949 >> v1:172.21.15.117:6789/0 conn(0x7faf040b5360 legacy=0x7faf041536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:14:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf02ffd700 1 -- 172.21.15.7:0/1638767949 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0415d730 con 0x7faf040ca1b0 2023-06-09T12:14:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf037fe700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 0x7faf040c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:14:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf02ffd700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040ca1b0 0x7faf0415cdc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7faef40098f0 tx=0x7faef4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923be9e7ee05b69b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf01ffb700 1 -- 172.21.15.7:0/1638767949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4004230 con 0x7faf040ca1b0 2023-06-09T12:14:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.795+0000 7faf01ffb700 1 -- 172.21.15.7:0/1638767949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4004390 con 0x7faf040ca1b0 2023-06-09T12:14:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf01ffb700 1 -- 172.21.15.7:0/1638767949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40036a0 con 0x7faf040ca1b0 2023-06-09T12:14:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 -- 172.21.15.7:0/1638767949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040ca1b0 msgr2=0x7faf0415cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040ca1b0 0x7faf0415cdc0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7faef40098f0 tx=0x7faef4009c00 comp rx=0 tx=0).stop 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 -- 172.21.15.7:0/1638767949 shutdown_connections 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca580 0x7faf040c58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 --2- 172.21.15.7:0/1638767949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040ca1b0 0x7faf0415cdc0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 -- 172.21.15.7:0/1638767949 >> 172.21.15.7:0/1638767949 conn(0x7faf04072090 msgr2=0x7faf04072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 -- 172.21.15.7:0/1638767949 shutdown_connections 2023-06-09T12:14:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.796+0000 7faf097ff700 1 -- 172.21.15.7:0/1638767949 wait complete. 2023-06-09T12:14:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.798+0000 7faf097ff700 1 Processor -- start 2023-06-09T12:14:27.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.799+0000 7faf097ff700 1 -- start start 2023-06-09T12:14:27.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca1b0 0x7faf0415ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf040ca580 0x7faf04151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.805 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf040cfff0 con 0x7faf040b5360 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf040b4b20 con 0x7faf040ca580 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf097ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0415cda0 con 0x7faf040ca1b0 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf02ffd700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50124/0 (socket says 172.21.15.7:50124) 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf02ffd700 1 -- 172.21.15.7:0/3903699058 learned_addr learned my addr 172.21.15.7:0/3903699058 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:14:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf027fc700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca1b0 0x7faf0415ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.800+0000 7faf037fe700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf040ca580 0x7faf04151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 -- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca1b0 msgr2=0x7faf0415ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca1b0 0x7faf0415ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 -- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf040ca580 msgr2=0x7faf04151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf040ca580 0x7faf04151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 -- 172.21.15.7:0/3903699058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009550 con 0x7faf040b5360 2023-06-09T12:14:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf02ffd700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7faf040d0710 tx=0x7faef4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4003cd0 con 0x7faf040b5360 2023-06-09T12:14:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef401f070 con 0x7faf040b5360 2023-06-09T12:14:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0406a570 con 0x7faf040b5360 2023-06-09T12:14:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.801+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400b8a0 con 0x7faf040b5360 2023-06-09T12:14:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.802+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0415da10 con 0x7faf040b5360 2023-06-09T12:14:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.803+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0406a850 con 0x7faf040b5360 2023-06-09T12:14:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.803+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faef4021030 con 0x7faf040b5360 2023-06-09T12:14:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.804+0000 7faeeb7fe700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 0x7faef006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.804+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7faef408a890 con 0x7faf040b5360 2023-06-09T12:14:27.810 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.804+0000 7faf027fc700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 0x7faef006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:27.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.805+0000 7faf027fc700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 0x7faef006d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7faf0415a940 tx=0x7faef8007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:27.811 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:27.809+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faef4054f30 con 0x7faf040b5360 2023-06-09T12:14:28.014 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.012+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf0415cee0 con 0x7faef006ace0 2023-06-09T12:14:28.024 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.021+0000 7faeeb7fe700 1 -- 172.21.15.7:0/3903699058 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318490 (secure 0 0 0) 0x7faef8003a90 con 0x7faef006ace0 2023-06-09T12:14:28.027 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:14:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 msgr2=0x7faef006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:28.029 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 0x7faef006d190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7faf0415a940 tx=0x7faef8007ec0 comp rx=0 tx=0).stop 2023-06-09T12:14:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 msgr2=0x7faf0414bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7faf040d0710 tx=0x7faef4004750 comp rx=0 tx=0).stop 2023-06-09T12:14:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 shutdown_connections 2023-06-09T12:14:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faf040ca580 0x7faf04151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:28.030 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf040b5360 0x7faf0414bab0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faef006ace0 0x7faef006d190 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 --2- 172.21.15.7:0/3903699058 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faf040ca1b0 0x7faf0415ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 >> 172.21.15.7:0/3903699058 conn(0x7faf04072090 msgr2=0x7faf0414c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 shutdown_connections 2023-06-09T12:14:28.031 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:28.026+0000 7faf097ff700 1 -- 172.21.15.7:0/3903699058 wait complete. 2023-06-09T12:14:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:14:28.047 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1980,"stamp":"2023-06-09T12:14:27.172534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103478,"ondisk_log_size":103478,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2595400,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":712064,"kb_avail":752379320,"statfs":{"total":773094113280,"available":770436423680,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":729153536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003247"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3753,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.436941+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:10:17.436941+0000","last_peered":"2023-06-09T12:10:17.436941+0000","last_clean":"2023-06-09T12:10:17.436941+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:10:17.436941+0000","last_undegraded":"2023-06-09T12:10:17.436941+0000","last_fullsized":"2023-06-09T12:10:17.436941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4217,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:52.411188+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:13:52.411188+0000","last_peered":"2023-06-09T12:13:52.411188+0000","last_clean":"2023-06-09T12:13:52.411188+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:52.411188+0000","last_undegraded":"2023-06-09T12:13:52.411188+0000","last_fullsized":"2023-06-09T12:13:52.411188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'994","reported_seq":3832,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:20.561761+0000","last_change":"2023-06-09T12:14:04.668556+0000","last_active":"2023-06-09T12:14:20.561761+0000","last_peered":"2023-06-09T12:14:20.561761+0000","last_clean":"2023-06-09T12:14:20.561761+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:20.561761+0000","last_undegraded":"2023-06-09T12:14:20.561761+0000","last_fullsized":"2023-06-09T12:14:20.561761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":72,"log_size":994,"log_dups_size":0,"ondisk_log_size":994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3748,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:10.762290+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:10:10.762290+0000","last_peered":"2023-06-09T12:10:10.762290+0000","last_clean":"2023-06-09T12:10:10.762290+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:10:10.762290+0000","last_undegraded":"2023-06-09T12:10:10.762290+0000","last_fullsized":"2023-06-09T12:10:10.762290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2028,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369453+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:09:37.369453+0000","last_peered":"2023-06-09T12:09:37.369453+0000","last_clean":"2023-06-09T12:09:37.369453+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:09:37.369453+0000","last_undegraded":"2023-06-09T12:09:37.369453+0000","last_fullsized":"2023-06-09T12:09:37.369453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384781+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:11:32.384781+0000","last_peered":"2023-06-09T12:11:32.384781+0000","last_clean":"2023-06-09T12:11:32.384781+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:11:32.384781+0000","last_undegraded":"2023-06-09T12:11:32.384781+0000","last_fullsized":"2023-06-09T12:11:32.384781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:52.463025+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:09:52.463025+0000","last_peered":"2023-06-09T12:09:52.463025+0000","last_clean":"2023-06-09T12:09:52.463025+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:09:52.463025+0000","last_undegraded":"2023-06-09T12:09:52.463025+0000","last_fullsized":"2023-06-09T12:09:52.463025+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.387434+0000","last_change":"2023-06-09T12:14:02.519430+0000","last_active":"2023-06-09T12:13:57.387434+0000","last_peered":"2023-06-09T12:13:57.387434+0000","last_clean":"2023-06-09T12:13:57.387434+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:13:57.387434+0000","last_undegraded":"2023-06-09T12:13:57.387434+0000","last_fullsized":"2023-06-09T12:13:57.387434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1341,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:57.553130+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:09:57.553130+0000","last_peered":"2023-06-09T12:09:57.553130+0000","last_clean":"2023-06-09T12:09:57.553130+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:09:57.553130+0000","last_undegraded":"2023-06-09T12:09:57.553130+0000","last_fullsized":"2023-06-09T12:09:57.553130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4049,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:09:37.369566+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:09:37.369566+0000","last_peered":"2023-06-09T12:09:37.369566+0000","last_clean":"2023-06-09T12:09:37.369566+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:09:37.369566+0000","last_undegraded":"2023-06-09T12:09:37.369566+0000","last_fullsized":"2023-06-09T12:09:37.369566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":723,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:19.192612+0000","last_change":"2023-06-09T12:14:15.227448+0000","last_active":"2023-06-09T12:14:19.192612+0000","last_peered":"2023-06-09T12:14:19.192612+0000","last_clean":"2023-06-09T12:14:19.192612+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:14:19.192612+0000","last_undegraded":"2023-06-09T12:14:19.192612+0000","last_fullsized":"2023-06-09T12:14:19.192612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:34.653035+0000","last_change":"2023-06-09T12:13:52.518166+0000","last_active":"2023-06-09T12:12:34.653035+0000","last_peered":"2023-06-09T12:12:34.653035+0000","last_clean":"2023-06-09T12:12:34.653035+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:12:34.653035+0000","last_undegraded":"2023-06-09T12:12:34.653035+0000","last_fullsized":"2023-06-09T12:12:34.653035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384752+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:11:32.384752+0000","last_peered":"2023-06-09T12:11:32.384752+0000","last_clean":"2023-06-09T12:11:32.384752+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:11:32.384752+0000","last_undegraded":"2023-06-09T12:11:32.384752+0000","last_fullsized":"2023-06-09T12:11:32.384752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":928,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.488180+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:10:17.488180+0000","last_peered":"2023-06-09T12:10:17.488180+0000","last_clean":"2023-06-09T12:10:17.488180+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:10:17.488180+0000","last_undegraded":"2023-06-09T12:10:17.488180+0000","last_fullsized":"2023-06-09T12:10:17.488180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1353,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375893+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:10:27.375893+0000","last_peered":"2023-06-09T12:10:27.375893+0000","last_clean":"2023-06-09T12:10:27.375893+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:10:27.375893+0000","last_undegraded":"2023-06-09T12:10:27.375893+0000","last_fullsized":"2023-06-09T12:10:27.375893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":191,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458337+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458337+0000","last_peered":"2023-06-09T12:14:14.458337+0000","last_clean":"2023-06-09T12:14:14.458337+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458337+0000","last_undegraded":"2023-06-09T12:14:14.458337+0000","last_fullsized":"2023-06-09T12:14:14.458337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3006,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:54.797717+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:13:54.797717+0000","last_peered":"2023-06-09T12:13:54.797717+0000","last_clean":"2023-06-09T12:13:54.797717+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:54.797717+0000","last_undegraded":"2023-06-09T12:13:54.797717+0000","last_fullsized":"2023-06-09T12:13:54.797717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384938+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:11:32.384938+0000","last_peered":"2023-06-09T12:11:32.384938+0000","last_clean":"2023-06-09T12:11:32.384938+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:11:32.384938+0000","last_undegraded":"2023-06-09T12:11:32.384938+0000","last_fullsized":"2023-06-09T12:11:32.384938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'990","reported_seq":3830,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:14.187369+0000","last_change":"2023-06-09T12:14:03.360737+0000","last_active":"2023-06-09T12:14:14.187369+0000","last_peered":"2023-06-09T12:14:14.187369+0000","last_clean":"2023-06-09T12:14:14.187369+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:14.187369+0000","last_undegraded":"2023-06-09T12:14:14.187369+0000","last_fullsized":"2023-06-09T12:14:14.187369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":96,"log_size":990,"log_dups_size":0,"ondisk_log_size":990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1628,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:32.383960+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:10:32.383960+0000","last_peered":"2023-06-09T12:10:32.383960+0000","last_clean":"2023-06-09T12:10:32.383960+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:10:32.383960+0000","last_undegraded":"2023-06-09T12:10:32.383960+0000","last_fullsized":"2023-06-09T12:10:32.383960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246265+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246265+0000","last_peered":"2023-06-09T12:12:30.246265+0000","last_clean":"2023-06-09T12:12:30.246265+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246265+0000","last_undegraded":"2023-06-09T12:12:30.246265+0000","last_fullsized":"2023-06-09T12:12:30.246265+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3946,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.398742+0000","last_change":"2023-06-09T12:13:37.318635+0000","last_active":"2023-06-09T12:13:22.398742+0000","last_peered":"2023-06-09T12:13:22.398742+0000","last_clean":"2023-06-09T12:13:22.398742+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:22.398742+0000","last_undegraded":"2023-06-09T12:13:22.398742+0000","last_fullsized":"2023-06-09T12:13:22.398742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286291+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286291+0000","last_peered":"2023-06-09T12:14:11.286291+0000","last_clean":"2023-06-09T12:14:11.286291+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286291+0000","last_undegraded":"2023-06-09T12:14:11.286291+0000","last_fullsized":"2023-06-09T12:14:11.286291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.364917+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:10:12.364917+0000","last_peered":"2023-06-09T12:10:12.364917+0000","last_clean":"2023-06-09T12:10:12.364917+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:10:12.364917+0000","last_undegraded":"2023-06-09T12:10:12.364917+0000","last_fullsized":"2023-06-09T12:10:12.364917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":72463,"ondisk_log_size":72463,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30896,"ondisk_log_size":30896,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575615,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69599999999999995},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0229999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.143,"5min":1.143,"15min":1.3320000000000001},"last":0.94499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":1.05},"last":0.98999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0649999999999999},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.745},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3089999999999999},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71199999999999997,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.86899999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":1.079}]},{"osd":4,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.871},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.505,"15min":0.45900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.226},"last":0.91500000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.76500000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.83099999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575615,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.59599999999999997,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.85199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.59999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.85999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73099999999999998,"15min":0.72399999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42699999999999999},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.57599999999999996,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0189999999999999,"5min":1.046,"15min":1.046},"last":0.624},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80800000000000005,"15min":0.80300000000000005},"min":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.879}]},{"osd":4,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.79000000000000004,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77200000000000002,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.69599999999999995},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.0609999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.69799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575616,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404976,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98560,"kb_avail":93966864,"statfs":{"total":96636764160,"available":96222068736,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":100925440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.63600000000000001,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.83799999999999997,"15min":0.85099999999999998},"last":0.47199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.38,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.5}]},{"osd":3,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.69099999999999995},"min":{"1min":0.60899999999999999,"5min":0.5,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":1.288,"15min":1.288},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.83899999999999997,"15min":0.78700000000000003},"min":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.45700000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.84699999999999998},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.175,"5min":1.2829999999999999,"15min":1.597},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.82799999999999996},"min":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0720000000000001,"5min":1.196,"15min":1.544},"last":0.87},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.86799999999999999},"min":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.751}]},{"osd":6,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.548},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.1319999999999999,"15min":1.2869999999999999},"last":0.84699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.751,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.2190000000000001,"5min":1.226,"15min":1.6519999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.51400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.498},"last":0.82599999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575615,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326888,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99072,"kb_avail":94044952,"statfs":{"total":96636764160,"available":96302030848,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101449728},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51000000000000001,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.42799999999999999,"15min":0.35799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.56899999999999995},"min":{"1min":0.42999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72699999999999998,"5min":0.81499999999999995,"15min":1.032},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.52400000000000002,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.30599999999999999},"max":{"1min":0.621,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.442}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60999999999999999,"15min":0.57999999999999996},"min":{"1min":0.47699999999999998,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.71899999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.621},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86799999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.85099999999999998,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0680000000000001,"5min":1.421,"15min":1.421},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.89700000000000002,"15min":0.87},"min":{"1min":0.628,"5min":0.60499999999999998,"15min":0.47699999999999998},"max":{"1min":1.1830000000000001,"5min":1.268,"15min":1.268},"last":0.86699999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.46100000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.121},"last":0.747},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.874,"15min":0.87},"min":{"1min":0.748,"5min":0.63,"15min":0.53900000000000003},"max":{"1min":1.153,"5min":1.371,"15min":3.5840000000000001},"last":0.92700000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79800000000000004,"15min":0.79900000000000004},"min":{"1min":0.53100000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0900000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83399999999999996,"15min":0.83899999999999997},"min":{"1min":0.63200000000000001,"5min":0.58299999999999996,"15min":0.52300000000000002},"max":{"1min":1.113,"5min":1.22,"15min":1.232},"last":1.038}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.58499999999999996,"5min":0.48299999999999998,"15min":0.47699999999999998},"max":{"1min":1.125,"5min":1.177,"15min":1.298},"last":1.105},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.95099999999999996,"15min":0.93899999999999995},"min":{"1min":0.76000000000000001,"5min":0.66400000000000003,"15min":0.60999999999999999},"max":{"1min":1.214,"5min":1.371,"15min":1.371},"last":1.147}]}]},{"osd":2,"up_from":13,"seq":55834575616,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.53800000000000003,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.442},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.999},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.625,"5min":0.66900000000000004,"15min":1.236},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.624},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.752}]},{"osd":3,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.86699999999999999,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":1.319,"15min":1.4650000000000001},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58199999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.0980000000000001,"15min":1.202},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.624,"5min":0.443,"15min":0.443},"max":{"1min":0.99099999999999999,"5min":1.173,"15min":1.2669999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.52800000000000002},"max":{"1min":0.95799999999999996,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.008,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.84299999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.62,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98999999999999999,"5min":1.143,"15min":1.3029999999999999},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83599999999999997,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.63}]},{"osd":7,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.67600000000000005,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.87,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.70799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.1419999999999999,"15min":1.262},"last":0.73299999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575615,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437820,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114624,"kb_avail":93934020,"statfs":{"total":96636764160,"available":96188436480,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117374976},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.48099999999999998},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":0.81499999999999995,"15min":0.81699999999999995},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.64400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63400000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54100000000000004,"15min":0.52900000000000003},"min":{"1min":0.39000000000000001,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.61099999999999999,"15min":0.55600000000000005},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.33800000000000002},"max":{"1min":0.753,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.51300000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.64700000000000002},"min":{"1min":0.5,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.83999999999999997,"5min":0.872,"15min":0.97099999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.39000000000000001,"5min":0.37,"15min":0.34100000000000003},"max":{"1min":0.81999999999999995,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.58199999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.65500000000000003,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.754},{"interface":"front","average":{"1min":0.873,"5min":0.82899999999999996,"15min":0.80900000000000005},"min":{"1min":0.67600000000000005,"5min":0.56599999999999995,"15min":0.52100000000000002},"max":{"1min":1.0309999999999999,"5min":1.151,"15min":3.081},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83299999999999996,"15min":0.83599999999999997},"min":{"1min":0.63300000000000001,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.89700000000000002,"15min":0.88600000000000001},"min":{"1min":0.76100000000000001,"5min":0.70399999999999996,"15min":0.52500000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.79200000000000004,"15min":0.80500000000000005},"min":{"1min":0.627,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.68400000000000005,"5min":0.60999999999999999,"15min":0.54200000000000004},"max":{"1min":1.0309999999999999,"5min":1.159,"15min":1.159},"last":0.66800000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:13:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.82699999999999996,"15min":0.82899999999999996},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.51600000000000001},"max":{"1min":1.1459999999999999,"5min":1.1659999999999999,"15min":1.306},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.65200000000000002,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":0.77800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575614,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413588,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120832,"kb_avail":93958252,"statfs":{"total":96636764160,"available":96213250048,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123731968},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.753,"15min":0.72999999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.95699999999999996,"5min":1.419,"15min":1.419},"last":1.0529999999999999},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58699999999999997,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.171,"5min":1.383,"15min":1.383},"last":1.1160000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.71599999999999997,"15min":0.73499999999999999},"min":{"1min":0.40500000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.93500000000000005,"5min":1.226,"15min":1.226},"last":1.242},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.58999999999999997,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.21}]},{"osd":2,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.52400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.117},"last":1.5209999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79100000000000004,"15min":0.77200000000000002},"min":{"1min":0.64700000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.0680000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":1.167}]},{"osd":3,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.57199999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90800000000000003,"5min":1.008,"15min":1.014},"last":1.6200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.60599999999999998,"5min":0.49199999999999999,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":1.111,"15min":1.111},"last":1.5740000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.42599999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.71199999999999997,"5min":0.91500000000000004,"15min":1.0189999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.626},"min":{"1min":0.47299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.753,"5min":1.0069999999999999,"15min":1.0620000000000001},"last":0.91800000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56399999999999995,"15min":0.57899999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.72499999999999998,"5min":0.80100000000000005,"15min":1.0509999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73799999999999999,"15min":0.73599999999999999},"min":{"1min":0.59099999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.83699999999999997,"5min":1.0489999999999999,"15min":1.131},"last":0.90300000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:13:34 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.63300000000000001,"15min":0.64900000000000002},"min":{"1min":0.49199999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.624,"15min":0.64200000000000002},"min":{"1min":0.54700000000000004,"5min":0.38900000000000001,"15min":0.38300000000000001},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":1.0229999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575614,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299672,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82304,"kb_avail":94072168,"statfs":{"total":96636764160,"available":96329900032,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84279296},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68000000000000005,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":5.4969999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71399999999999997,"15min":0.73299999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40999999999999998},"max":{"1min":1.579,"5min":1.579,"15min":5.6269999999999998},"last":0.77300000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.78700000000000003},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.377},"max":{"1min":0.87,"5min":1.2689999999999999,"15min":19.715},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":0.95799999999999996,"15min":7.4089999999999998},"last":0.88700000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.42499999999999999},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":5.5620000000000003},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.73199999999999998,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.391,"5min":1.391,"15min":14.439},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":12.130000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.84399999999999997},"min":{"1min":0.53400000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.296,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.86099999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.79300000000000004},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.522,"5min":1.522,"15min":67.590000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.82699999999999996},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.28,"5min":1.28,"15min":67.521000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":12.163},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.41599999999999998},"max":{"1min":1.121,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.628}]},{"osd":7,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":3.8039999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.82799999999999996},"min":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.226,"5min":1.226,"15min":19.834},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:14:28.054 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.055 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.055 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.056 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.056 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.057 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.057 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.057 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.058 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.058 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.059 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.059 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.060 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:28.060 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:14:29.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:29.776 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:48.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:14:48.274 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.272+0000 7f212d3ff700 1 Processor -- start 2023-06-09T12:14:48.275 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.272+0000 7f212d3ff700 1 -- start start 2023-06-09T12:14:48.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.273+0000 7f212d3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21280c88e0 0x7f21280c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:48.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.273+0000 7f212d3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280c9bd0 con 0x7f212815b060 2023-06-09T12:14:48.276 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f212d3ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2128159570 con 0x7f21280c2ae0 2023-06-09T12:14:48.277 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f212d3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2128159360 con 0x7f21280c88e0 2023-06-09T12:14:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21277fe700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f21280c88e0 0x7f21280c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21267fc700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f212815b060 0x7f2128161cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34794/0 (socket says 172.21.15.7:34794) 2023-06-09T12:14:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21267fc700 1 -- 172.21.15.7:0/3047802350 learned_addr learned my addr 172.21.15.7:0/3047802350 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:14:48.278 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21277fe700 1 -- 172.21.15.7:0/3047802350 >> v1:172.21.15.117:6789/0 conn(0x7f21280c2ae0 legacy=0x7f212815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21277fe700 1 -- 172.21.15.7:0/3047802350 >> v1:172.21.15.7:6789/0 conn(0x7f212815b060 legacy=0x7f2128161cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.274+0000 7f21277fe700 1 -- 172.21.15.7:0/3047802350 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128159810 con 0x7f21280c88e0 2023-06-09T12:14:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f21277fe700 1 --2- 172.21.15.7:0/3047802350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21280c88e0 0x7f21280c2eb0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2120009d90 tx=0x7f212000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84be09b39de7fc17 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f2125ffb700 1 -- 172.21.15.7:0/3047802350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212000e070 con 0x7f21280c88e0 2023-06-09T12:14:48.279 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f2125ffb700 1 -- 172.21.15.7:0/3047802350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000bbd0 con 0x7f21280c88e0 2023-06-09T12:14:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f2125ffb700 1 -- 172.21.15.7:0/3047802350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120012430 con 0x7f21280c88e0 2023-06-09T12:14:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 -- 172.21.15.7:0/3047802350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21280c88e0 msgr2=0x7f21280c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 --2- 172.21.15.7:0/3047802350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21280c88e0 0x7f21280c2eb0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f2120009d90 tx=0x7f212000b330 comp rx=0 tx=0).stop 2023-06-09T12:14:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 -- 172.21.15.7:0/3047802350 shutdown_connections 2023-06-09T12:14:48.280 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 --2- 172.21.15.7:0/3047802350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21280c88e0 0x7f21280c2eb0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 -- 172.21.15.7:0/3047802350 >> 172.21.15.7:0/3047802350 conn(0x7f21280cd330 msgr2=0x7f21280cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 -- 172.21.15.7:0/3047802350 shutdown_connections 2023-06-09T12:14:48.281 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.275+0000 7f212d3ff700 1 -- 172.21.15.7:0/3047802350 wait complete. 2023-06-09T12:14:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.277+0000 7f212d3ff700 1 Processor -- start 2023-06-09T12:14:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.277+0000 7f212d3ff700 1 -- start start 2023-06-09T12:14:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21280c2ae0 0x7f21281f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:48.283 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21280c88e0 0x7f21280c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280c51c0 con 0x7f212815b060 2023-06-09T12:14:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21281e5730 con 0x7f21280c2ae0 2023-06-09T12:14:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f212d3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21281f1cb0 con 0x7f21280c88e0 2023-06-09T12:14:48.284 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f21277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f21277fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34906/0 (socket says 172.21.15.7:34906) 2023-06-09T12:14:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.278+0000 7f21277fe700 1 -- 172.21.15.7:0/696750214 learned_addr learned my addr 172.21.15.7:0/696750214 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:14:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21267fc700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21280c88e0 0x7f21280c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f2126ffd700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21280c2ae0 0x7f21281f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:48.285 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 -- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21280c88e0 msgr2=0x7f21280c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21280c88e0 0x7f21280c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 -- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21280c2ae0 msgr2=0x7f21281f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21280c2ae0 0x7f21281f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 -- 172.21.15.7:0/696750214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2120009a70 con 0x7f212815b060 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f21277fe700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2120014040 tx=0x7f2120004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:48.286 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21200098f0 con 0x7f212815b060 2023-06-09T12:14:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2120020070 con 0x7f212815b060 2023-06-09T12:14:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120004280 con 0x7f212815b060 2023-06-09T12:14:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.279+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2128159dd0 con 0x7f212815b060 2023-06-09T12:14:48.287 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.280+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2128159af0 con 0x7f212815b060 2023-06-09T12:14:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.281+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212815a0b0 con 0x7f212815b060 2023-06-09T12:14:48.288 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.282+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f21200043e0 con 0x7f212815b060 2023-06-09T12:14:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.283+0000 7f210f7fe700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 0x7f211406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:14:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.284+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f2120022070 con 0x7f212815b060 2023-06-09T12:14:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.284+0000 7f2126ffd700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 0x7f211406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:14:48.289 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.284+0000 7f2126ffd700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 0x7f211406d1c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2118000f60 tx=0x7f2118009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:14:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.288+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2120056490 con 0x7f212815b060 2023-06-09T12:14:48.492 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.490+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2128074cf0 con 0x7f211406ad10 2023-06-09T12:14:48.501 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.499+0000 7f210f7fe700 1 -- 172.21.15.7:0/696750214 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318604 (secure 0 0 0) 0x7f2128074cf0 con 0x7f211406ad10 2023-06-09T12:14:48.505 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:14:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 msgr2=0x7f211406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 0x7f211406d1c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2118000f60 tx=0x7f2118009040 comp rx=0 tx=0).stop 2023-06-09T12:14:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 msgr2=0x7f21280cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:14:48.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f2120014040 tx=0x7f2120004ad0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 shutdown_connections 2023-06-09T12:14:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f212815b060 0x7f21280cc990 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f21280c2ae0 0x7f21281f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f211406ad10 0x7f211406d1c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 --2- 172.21.15.7:0/696750214 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f21280c88e0 0x7f21280c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:14:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 >> 172.21.15.7:0/696750214 conn(0x7f21280cd330 msgr2=0x7f212815bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:14:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.503+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 shutdown_connections 2023-06-09T12:14:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:14:48.504+0000 7f212d3ff700 1 -- 172.21.15.7:0/696750214 wait complete. 2023-06-09T12:14:48.509 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:14:48.524 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":1990,"stamp":"2023-06-09T12:14:47.178427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103674,"ondisk_log_size":103674,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2595912,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":712576,"kb_avail":752378808,"statfs":{"total":773094113280,"available":770435899392,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":729677824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003250"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4217,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:52.411188+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:13:52.411188+0000","last_peered":"2023-06-09T12:13:52.411188+0000","last_clean":"2023-06-09T12:13:52.411188+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:52.411188+0000","last_undegraded":"2023-06-09T12:13:52.411188+0000","last_fullsized":"2023-06-09T12:13:52.411188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3743,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.435591+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:10:07.435591+0000","last_peered":"2023-06-09T12:10:07.435591+0000","last_clean":"2023-06-09T12:10:07.435591+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:10:07.435591+0000","last_undegraded":"2023-06-09T12:10:07.435591+0000","last_fullsized":"2023-06-09T12:10:07.435591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3525,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:12.436260+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:10:12.436260+0000","last_peered":"2023-06-09T12:10:12.436260+0000","last_clean":"2023-06-09T12:10:12.436260+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:10:12.436260+0000","last_undegraded":"2023-06-09T12:10:12.436260+0000","last_fullsized":"2023-06-09T12:10:12.436260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1067","reported_seq":3909,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:38.425944+0000","last_change":"2023-06-09T12:14:04.668556+0000","last_active":"2023-06-09T12:14:38.425944+0000","last_peered":"2023-06-09T12:14:38.425944+0000","last_clean":"2023-06-09T12:14:38.425944+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:38.425944+0000","last_undegraded":"2023-06-09T12:14:38.425944+0000","last_fullsized":"2023-06-09T12:14:38.425944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":145,"log_size":1067,"log_dups_size":0,"ondisk_log_size":1067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:24.300499+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:10:24.300499+0000","last_peered":"2023-06-09T12:10:24.300499+0000","last_clean":"2023-06-09T12:10:24.300499+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:10:24.300499+0000","last_undegraded":"2023-06-09T12:10:24.300499+0000","last_fullsized":"2023-06-09T12:10:24.300499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.437304+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:10:22.437304+0000","last_peered":"2023-06-09T12:10:22.437304+0000","last_clean":"2023-06-09T12:10:22.437304+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:10:22.437304+0000","last_undegraded":"2023-06-09T12:10:22.437304+0000","last_fullsized":"2023-06-09T12:10:22.437304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3811,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.375878+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:10:27.375878+0000","last_peered":"2023-06-09T12:10:27.375878+0000","last_clean":"2023-06-09T12:10:27.375878+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:10:27.375878+0000","last_undegraded":"2023-06-09T12:10:27.375878+0000","last_fullsized":"2023-06-09T12:10:27.375878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3446,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:56.673924+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:10:56.673924+0000","last_peered":"2023-06-09T12:10:56.673924+0000","last_clean":"2023-06-09T12:10:56.673924+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:10:56.673924+0000","last_undegraded":"2023-06-09T12:10:56.673924+0000","last_fullsized":"2023-06-09T12:10:56.673924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:52.712736+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:10:52.712736+0000","last_peered":"2023-06-09T12:10:52.712736+0000","last_clean":"2023-06-09T12:10:52.712736+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:10:52.712736+0000","last_undegraded":"2023-06-09T12:10:52.712736+0000","last_fullsized":"2023-06-09T12:10:52.712736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384781+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:11:32.384781+0000","last_peered":"2023-06-09T12:11:32.384781+0000","last_clean":"2023-06-09T12:11:32.384781+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:11:32.384781+0000","last_undegraded":"2023-06-09T12:11:32.384781+0000","last_fullsized":"2023-06-09T12:11:32.384781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.387434+0000","last_change":"2023-06-09T12:14:02.519430+0000","last_active":"2023-06-09T12:13:57.387434+0000","last_peered":"2023-06-09T12:13:57.387434+0000","last_clean":"2023-06-09T12:13:57.387434+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:13:57.387434+0000","last_undegraded":"2023-06-09T12:13:57.387434+0000","last_fullsized":"2023-06-09T12:13:57.387434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3859,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:14.679024+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:10:14.679024+0000","last_peered":"2023-06-09T12:10:14.679024+0000","last_clean":"2023-06-09T12:10:14.679024+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:10:14.679024+0000","last_undegraded":"2023-06-09T12:10:14.679024+0000","last_fullsized":"2023-06-09T12:10:14.679024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":727,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:35.038378+0000","last_change":"2023-06-09T12:14:15.227448+0000","last_active":"2023-06-09T12:14:35.038378+0000","last_peered":"2023-06-09T12:14:35.038378+0000","last_clean":"2023-06-09T12:14:35.038378+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:14:35.038378+0000","last_undegraded":"2023-06-09T12:14:35.038378+0000","last_fullsized":"2023-06-09T12:14:35.038378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4302,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.383050+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:10:27.383050+0000","last_peered":"2023-06-09T12:10:27.383050+0000","last_clean":"2023-06-09T12:10:27.383050+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:10:27.383050+0000","last_undegraded":"2023-06-09T12:10:27.383050+0000","last_fullsized":"2023-06-09T12:10:27.383050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:34.653035+0000","last_change":"2023-06-09T12:13:52.518166+0000","last_active":"2023-06-09T12:12:34.653035+0000","last_peered":"2023-06-09T12:12:34.653035+0000","last_clean":"2023-06-09T12:12:34.653035+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:12:34.653035+0000","last_undegraded":"2023-06-09T12:12:34.653035+0000","last_fullsized":"2023-06-09T12:12:34.653035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558106+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:10:42.558106+0000","last_peered":"2023-06-09T12:10:42.558106+0000","last_clean":"2023-06-09T12:10:42.558106+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:10:42.558106+0000","last_undegraded":"2023-06-09T12:10:42.558106+0000","last_fullsized":"2023-06-09T12:10:42.558106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":762,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384752+0000","last_change":"2023-06-09T12:09:22.367237+0000","last_active":"2023-06-09T12:11:32.384752+0000","last_peered":"2023-06-09T12:11:32.384752+0000","last_clean":"2023-06-09T12:11:32.384752+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:11:32.384752+0000","last_undegraded":"2023-06-09T12:11:32.384752+0000","last_fullsized":"2023-06-09T12:11:32.384752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":191,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458337+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458337+0000","last_peered":"2023-06-09T12:14:14.458337+0000","last_clean":"2023-06-09T12:14:14.458337+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458337+0000","last_undegraded":"2023-06-09T12:14:14.458337+0000","last_fullsized":"2023-06-09T12:14:14.458337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3006,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:54.797717+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:13:54.797717+0000","last_peered":"2023-06-09T12:13:54.797717+0000","last_clean":"2023-06-09T12:13:54.797717+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:54.797717+0000","last_undegraded":"2023-06-09T12:13:54.797717+0000","last_fullsized":"2023-06-09T12:13:54.797717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":529,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384938+0000","last_change":"2023-06-09T12:09:22.367445+0000","last_active":"2023-06-09T12:11:32.384938+0000","last_peered":"2023-06-09T12:11:32.384938+0000","last_clean":"2023-06-09T12:11:32.384938+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:11:32.384938+0000","last_undegraded":"2023-06-09T12:11:32.384938+0000","last_fullsized":"2023-06-09T12:11:32.384938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":167,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.367738+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:10:37.367738+0000","last_peered":"2023-06-09T12:10:37.367738+0000","last_clean":"2023-06-09T12:10:37.367738+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:10:37.367738+0000","last_undegraded":"2023-06-09T12:10:37.367738+0000","last_fullsized":"2023-06-09T12:10:37.367738+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1113","reported_seq":3959,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:40.968646+0000","last_change":"2023-06-09T12:14:03.360737+0000","last_active":"2023-06-09T12:14:40.968646+0000","last_peered":"2023-06-09T12:14:40.968646+0000","last_clean":"2023-06-09T12:14:40.968646+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:40.968646+0000","last_undegraded":"2023-06-09T12:14:40.968646+0000","last_fullsized":"2023-06-09T12:14:40.968646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":219,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.558082+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:10:42.558082+0000","last_peered":"2023-06-09T12:10:42.558082+0000","last_clean":"2023-06-09T12:10:42.558082+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:10:42.558082+0000","last_undegraded":"2023-06-09T12:10:42.558082+0000","last_fullsized":"2023-06-09T12:10:42.558082+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3946,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.398742+0000","last_change":"2023-06-09T12:13:37.318635+0000","last_active":"2023-06-09T12:13:22.398742+0000","last_peered":"2023-06-09T12:13:22.398742+0000","last_clean":"2023-06-09T12:13:22.398742+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:22.398742+0000","last_undegraded":"2023-06-09T12:13:22.398742+0000","last_fullsized":"2023-06-09T12:13:22.398742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286291+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286291+0000","last_peered":"2023-06-09T12:14:11.286291+0000","last_clean":"2023-06-09T12:14:11.286291+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286291+0000","last_undegraded":"2023-06-09T12:14:11.286291+0000","last_fullsized":"2023-06-09T12:14:11.286291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":72659,"ondisk_log_size":72659,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30896,"ondisk_log_size":30896,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575619,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69599999999999995},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0229999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.143,"5min":1.143,"15min":1.3320000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":1.05},"last":0.92300000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0649999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.745},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3089999999999999},"last":0.89700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71199999999999997,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.86899999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.505,"15min":0.45900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.226},"last":0.69399999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.60199999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.73299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575619,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69299999999999995,"15min":0.67600000000000005},"min":{"1min":0.52800000000000002,"5min":0.45900000000000002,"15min":0.33100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.59599999999999997,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.97199999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.63}]},{"osd":1,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.72499999999999998,"15min":0.71399999999999997},"min":{"1min":0.59999999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76900000000000002,"15min":0.76200000000000001},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.73999999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73099999999999998,"15min":0.72399999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.42699999999999999},"max":{"1min":0.98799999999999999,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78300000000000003,"15min":0.76900000000000002},"min":{"1min":0.63600000000000001,"5min":0.56499999999999995,"15min":0.53400000000000003},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.57599999999999996,"5min":0.47999999999999998,"15min":0.45100000000000001},"max":{"1min":1.0189999999999999,"5min":1.046,"15min":1.046},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80800000000000005,"15min":0.80300000000000005},"min":{"1min":0.67700000000000005,"5min":0.59199999999999997,"15min":0.53500000000000003},"max":{"1min":1.056,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57399999999999995,"15min":0.58999999999999997},"min":{"1min":0.436,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.89300000000000002,"15min":0.95299999999999996},"last":0.443},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.82199999999999995,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.46500000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:13:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.59799999999999998,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.38800000000000001,"15min":0.38200000000000001},"max":{"1min":0.79000000000000004,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77200000000000002,"15min":0.72699999999999998},"min":{"1min":0.56599999999999995,"5min":0.54400000000000004,"15min":0.40600000000000003},"max":{"1min":1.006,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":0.504},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.69599999999999995},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.0609999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.56200000000000006}]}]},{"osd":1,"up_from":13,"seq":55834575620,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404976,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98560,"kb_avail":93966864,"statfs":{"total":96636764160,"available":96222068736,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":100925440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32000000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.63600000000000001,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56000000000000005,"15min":0.57399999999999995},"min":{"1min":0.44,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.69499999999999995,"5min":0.83799999999999997,"15min":0.85099999999999998},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.48599999999999999,"15min":0.496},"min":{"1min":0.38,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54600000000000004,"15min":0.55100000000000005},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.35699999999999998},"max":{"1min":0.73799999999999999,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.505}]},{"osd":3,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.61799999999999999,"15min":0.59699999999999998},"min":{"1min":0.5,"5min":0.42199999999999999,"15min":0.374},"max":{"1min":0.71899999999999997,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71999999999999997,"15min":0.69099999999999995},"min":{"1min":0.60899999999999999,"5min":0.5,"15min":0.44},"max":{"1min":0.88300000000000001,"5min":1.288,"15min":1.288},"last":0.75700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.83899999999999997,"15min":0.78700000000000003},"min":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.45700000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.016},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81699999999999995,"15min":0.84699999999999998},"min":{"1min":0.55000000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.175,"5min":1.2829999999999999,"15min":1.597},"last":0.96699999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.82799999999999996},"min":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0720000000000001,"5min":1.196,"15min":1.544},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.86799999999999999},"min":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.879}]},{"osd":6,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.58199999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.548},"last":0.996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.58699999999999997,"5min":0.53000000000000003,"15min":0.49399999999999999},"max":{"1min":1.052,"5min":1.1319999999999999,"15min":1.2869999999999999},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:13:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.89800000000000002},"min":{"1min":0.751,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.2190000000000001,"5min":1.226,"15min":1.6519999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":0.83899999999999997},"min":{"1min":0.61199999999999999,"5min":0.57899999999999996,"15min":0.51400000000000001},"max":{"1min":1.24,"5min":1.24,"15min":1.498},"last":1.0389999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575619,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327016,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":99200,"kb_avail":94044824,"statfs":{"total":96636764160,"available":96301899776,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":101580800},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.48099999999999998},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.68600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78000000000000003,"5min":0.81499999999999995,"15min":1.032},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":0.871,"15min":0.96199999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88500000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.86499999999999999,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":1.1040000000000001,"5min":1.421,"15min":1.421},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90300000000000002,"15min":0.878},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.58799999999999997},"max":{"1min":1.024,"5min":1.268,"15min":1.268},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.036,"5min":1.077,"15min":1.121},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.879,"15min":0.878},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.111,"5min":1.371,"15min":3.5840000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.60299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0740000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83599999999999997,"15min":0.83999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0409999999999999,"5min":1.22,"15min":1.232},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.873,"15min":0.86299999999999999},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.47699999999999998},"max":{"1min":1.1299999999999999,"5min":1.177,"15min":1.298},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96099999999999997,"15min":0.94599999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60999999999999999},"max":{"1min":1.147,"5min":1.371,"15min":1.371},"last":0.85399999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575620,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.376,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.53800000000000003,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55200000000000005},"min":{"1min":0.44600000000000001,"5min":0.39300000000000002,"15min":0.38400000000000001},"max":{"1min":0.70099999999999996,"5min":0.747,"15min":0.999},"last":0.52600000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48299999999999998,"15min":0.47999999999999998},"min":{"1min":0.33400000000000002,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.625,"5min":0.66900000000000004,"15min":1.236},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58899999999999997,"15min":0.624},"min":{"1min":0.47299999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81799999999999995,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.55500000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63300000000000001,"15min":0.61399999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.86699999999999999,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.78300000000000003,"5min":1.319,"15min":1.4650000000000001},"last":0.625}]},{"osd":4,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.58199999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.0980000000000001,"15min":1.202},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77500000000000002,"15min":0.78900000000000003},"min":{"1min":0.624,"5min":0.443,"15min":0.443},"max":{"1min":0.99099999999999999,"5min":1.173,"15min":1.2669999999999999},"last":0.755}]},{"osd":5,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"min":{"1min":0.63600000000000001,"5min":0.56200000000000006,"15min":0.52800000000000002},"max":{"1min":0.95799999999999996,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.877,"15min":0.876},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.008,"5min":1.4670000000000001,"15min":1.5189999999999999},"last":0.745}]},{"osd":6,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80600000000000005,"15min":0.81100000000000005},"min":{"1min":0.62,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.98999999999999999,"5min":1.143,"15min":1.3029999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83599999999999997,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.0329999999999999,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.72399999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:13:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.84099999999999997},"min":{"1min":0.67600000000000005,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":0.93899999999999995,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.87,"5min":0.86699999999999999,"15min":0.85899999999999999},"min":{"1min":0.70799999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0089999999999999,"5min":1.1419999999999999,"15min":1.262},"last":0.78400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575619,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437948,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114752,"kb_avail":93933892,"statfs":{"total":96636764160,"available":96188305408,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117506048},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.48099999999999998},"min":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68700000000000006,"5min":0.81499999999999995,"15min":0.81699999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.55400000000000005,"15min":0.56100000000000005},"min":{"1min":0.42999999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.64400000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.747,"5min":0.872,"15min":0.97099999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82099999999999995,"15min":0.80600000000000005},"min":{"1min":0.64100000000000001,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.0169999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":3.081},"last":0.95199999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84599999999999997,"15min":0.84299999999999997},"min":{"1min":0.66000000000000003,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.52500000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.96599999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.64600000000000002,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57299999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.63100000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.046,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":0.91500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575618,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":413716,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":120960,"kb_avail":93958124,"statfs":{"total":96636764160,"available":96213118976,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":123863040},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77600000000000002,"15min":0.74199999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":1.0549999999999999,"5min":1.419,"15min":1.419},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.78600000000000003},"min":{"1min":0.67100000000000004,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.1160000000000001,"5min":1.383,"15min":1.383},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84699999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.1890000000000001,"15min":1.2410000000000001},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72999999999999998,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48299999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.91500000000000004,"15min":0.95999999999999996},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.625},"min":{"1min":0.51400000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0509999999999999},"last":0.624},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.96799999999999997,"5min":1.0029999999999999,"15min":1.131},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.38300000000000001},"max":{"1min":1.0229999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.68000000000000005,"15min":0.71799999999999997},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":5.4969999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.71399999999999997,"15min":0.73299999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.40999999999999998},"max":{"1min":1.579,"5min":1.579,"15min":5.6269999999999998},"last":0.74099999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70099999999999996,"15min":0.78700000000000003},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.377},"max":{"1min":0.87,"5min":1.2689999999999999,"15min":19.715},"last":0.753},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72299999999999998,"15min":0.76700000000000002},"min":{"1min":0.55900000000000005,"5min":0.498,"15min":0.439},"max":{"1min":0.92700000000000005,"5min":0.95799999999999996,"15min":7.4089999999999998},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72599999999999998,"15min":0.73699999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.42499999999999999},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":5.5620000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.73199999999999998,"15min":0.78300000000000003},"min":{"1min":0.54900000000000004,"5min":0.437,"15min":0.437},"max":{"1min":1.391,"5min":1.391,"15min":14.439},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76000000000000001,"15min":0.78700000000000003},"min":{"1min":0.54000000000000004,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":12.130000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.81499999999999995,"15min":0.84399999999999997},"min":{"1min":0.53400000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.296,"5min":1.3120000000000001,"15min":12.218999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.79300000000000004},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.522,"5min":1.522,"15min":67.590000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.59499999999999997,"15min":0.82699999999999996},"min":{"1min":0.44600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.28,"5min":1.28,"15min":67.521000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.65600000000000003,"15min":0.68100000000000005},"min":{"1min":0.50900000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":12.163},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73599999999999999,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.41599999999999998},"max":{"1min":1.121,"5min":1.1779999999999999,"15min":7.6040000000000001},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:13:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64100000000000001,"15min":0.65800000000000003},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":3.8039999999999998},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.75600000000000001,"15min":0.82799999999999996},"min":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.226,"5min":1.226,"15min":19.834},"last":0.78800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:14:48.533 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.533 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.534 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.534 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.534 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.535 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.535 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.536 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.536 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.537 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.537 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.538 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.538 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:14:48.538 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:14:59.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:14:59.898 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:08.539 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:15:08.751 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.748+0000 7f1bd33ff700 1 Processor -- start 2023-06-09T12:15:08.752 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.748+0000 7f1bd33ff700 1 -- start start 2023-06-09T12:15:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd33ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc15cd40 0x7f1bcc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd33ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bcc159b20 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.754 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd33ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cac80 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc1597d0 con 0x7f1bcc15cd40 2023-06-09T12:15:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd099a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd099a700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37174/0 (socket says 172.21.15.7:37174) 2023-06-09T12:15:08.755 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd119b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1bcc0cb5d0 0x7f1bcc159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39898/0 (socket says 172.21.15.7:39898) 2023-06-09T12:15:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.749+0000 7f1bd099a700 1 -- 172.21.15.7:0/2886214286 learned_addr learned my addr 172.21.15.7:0/2886214286 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd199c700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc15cd40 0x7f1bcc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd099a700 1 -- 172.21.15.7:0/2886214286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc15cd40 msgr2=0x7f1bcc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:15:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd099a700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc15cd40 0x7f1bcc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.756 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd099a700 1 -- 172.21.15.7:0/2886214286 >> v1:172.21.15.7:6789/0 conn(0x7f1bcc0cb5d0 legacy=0x7f1bcc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd099a700 1 -- 172.21.15.7:0/2886214286 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc071ab0 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bc3fff700 1 -- 172.21.15.7:0/2886214286 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944945252 0 0) 0x7f1bcc159b20 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bd099a700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0098f0 tx=0x7f1bbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84afc8b54e86f6f7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bc3fff700 1 -- 172.21.15.7:0/2886214286 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00b8a0 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.757 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bc3fff700 1 -- 172.21.15.7:0/2886214286 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bbc00ba00 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.758 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.750+0000 7f1bc3fff700 1 -- 172.21.15.7:0/2886214286 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00bd90 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2886214286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb9a0 msgr2=0x7f1bcc0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0098f0 tx=0x7f1bbc009c00 comp rx=0 tx=0).stop 2023-06-09T12:15:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2886214286 shutdown_connections 2023-06-09T12:15:08.760 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc15cd40 0x7f1bcc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2886214286 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0b52b0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2886214286 >> 172.21.15.7:0/2886214286 conn(0x7f1bcc070af0 msgr2=0x7f1bcc070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2886214286 shutdown_connections 2023-06-09T12:15:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.751+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2886214286 wait complete. 2023-06-09T12:15:08.761 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.753+0000 7f1bd33ff700 1 Processor -- start 2023-06-09T12:15:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.753+0000 7f1bd33ff700 1 -- start start 2023-06-09T12:15:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bcc15cd40 0x7f1bcc1f0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.762 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc156730 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cfaf0 con 0x7f1bcc15cd40 2023-06-09T12:15:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd33ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bcc0cf620 con 0x7f1bcc0cb9a0 2023-06-09T12:15:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.763 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37190/0 (socket says 172.21.15.7:37190) 2023-06-09T12:15:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 -- 172.21.15.7:0/2799347732 learned_addr learned my addr 172.21.15.7:0/2799347732 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd099a700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd199c700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bcc15cd40 0x7f1bcc1f0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 -- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc0cb9a0 msgr2=0x7f1bcc0c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.764 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 -- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bcc15cd40 msgr2=0x7f1bcc1f0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.754+0000 7f1bd119b700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bcc15cd40 0x7f1bcc1f0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bd119b700 1 -- 172.21.15.7:0/2799347732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bbc009550 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bd119b700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8001f80 tx=0x7f1bc800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:08.765 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc800acd0 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc800ae50 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bcc15d4f0 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bc8007310 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.755+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bcc15d210 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.766 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.756+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bcc15d7d0 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.757+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1bc8007470 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.767 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.758+0000 7f1bc1ffb700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 0x7f1bb006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.758+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f1bc8089c60 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.759+0000 7f1bd099a700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 0x7f1bb006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.759+0000 7f1bd099a700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 0x7f1bb006d1c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cbd70 tx=0x7f1bbc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:08.768 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.763+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f1bc8054390 con 0x7f1bcc0cb5d0 2023-06-09T12:15:08.970 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.966+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1bcc074cf0 con 0x7f1bb006ad10 2023-06-09T12:15:08.979 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.975+0000 7f1bc1ffb700 1 -- 172.21.15.7:0/2799347732 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318776 (secure 0 0 0) 0x7f1bcc074cf0 con 0x7f1bb006ad10 2023-06-09T12:15:08.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:15:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 msgr2=0x7f1bb006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 0x7f1bb006d1c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0cbd70 tx=0x7f1bbc008030 comp rx=0 tx=0).stop 2023-06-09T12:15:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 msgr2=0x7f1bcc0cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:08.985 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f1bc8001f80 tx=0x7f1bc800a340 comp rx=0 tx=0).stop 2023-06-09T12:15:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 shutdown_connections 2023-06-09T12:15:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1bcc15cd40 0x7f1bcc1f0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1bcc0cb5d0 0x7f1bcc0cc010 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.979+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f1bb006ad10 0x7f1bb006d1c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.980+0000 7f1bd33ff700 1 --2- 172.21.15.7:0/2799347732 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1bcc0cb9a0 0x7f1bcc0c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.980+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 >> 172.21.15.7:0/2799347732 conn(0x7f1bcc070af0 msgr2=0x7f1bcc158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.980+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 shutdown_connections 2023-06-09T12:15:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:08.980+0000 7f1bd33ff700 1 -- 172.21.15.7:0/2799347732 wait complete. 2023-06-09T12:15:08.987 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:15:09.001 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2000,"stamp":"2023-06-09T12:15:07.184322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103790,"ondisk_log_size":103790,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2604296,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":720960,"kb_avail":752370424,"statfs":{"total":773094113280,"available":770427314176,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":738263040},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003271"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3486,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396085+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:10:57.396085+0000","last_peered":"2023-06-09T12:10:57.396085+0000","last_clean":"2023-06-09T12:10:57.396085+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:10:57.396085+0000","last_undegraded":"2023-06-09T12:10:57.396085+0000","last_fullsized":"2023-06-09T12:10:57.396085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3519,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391455+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391455+0000","last_peered":"2023-06-09T12:13:11.391455+0000","last_clean":"2023-06-09T12:13:11.391455+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391455+0000","last_undegraded":"2023-06-09T12:13:11.391455+0000","last_fullsized":"2023-06-09T12:13:11.391455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:07.380944+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:10:07.380944+0000","last_peered":"2023-06-09T12:10:07.380944+0000","last_clean":"2023-06-09T12:10:07.380944+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:10:07.380944+0000","last_undegraded":"2023-06-09T12:10:07.380944+0000","last_fullsized":"2023-06-09T12:10:07.380944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4217,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:52.411188+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:13:52.411188+0000","last_peered":"2023-06-09T12:13:52.411188+0000","last_clean":"2023-06-09T12:13:52.411188+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:13:52.411188+0000","last_undegraded":"2023-06-09T12:13:52.411188+0000","last_fullsized":"2023-06-09T12:13:52.411188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3973,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:22.391770+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:10:22.391770+0000","last_peered":"2023-06-09T12:10:22.391770+0000","last_clean":"2023-06-09T12:10:22.391770+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:10:22.391770+0000","last_undegraded":"2023-06-09T12:10:22.391770+0000","last_fullsized":"2023-06-09T12:10:22.391770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.470804+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:10:57.470804+0000","last_peered":"2023-06-09T12:10:57.470804+0000","last_clean":"2023-06-09T12:10:57.470804+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:10:57.470804+0000","last_undegraded":"2023-06-09T12:10:57.470804+0000","last_fullsized":"2023-06-09T12:10:57.470804+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3262,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371107+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:11:07.371107+0000","last_peered":"2023-06-09T12:11:07.371107+0000","last_clean":"2023-06-09T12:11:07.371107+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:11:07.371107+0000","last_undegraded":"2023-06-09T12:11:07.371107+0000","last_fullsized":"2023-06-09T12:11:07.371107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2269,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:27.366381+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:10:27.366381+0000","last_peered":"2023-06-09T12:10:27.366381+0000","last_clean":"2023-06-09T12:10:27.366381+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:10:27.366381+0000","last_undegraded":"2023-06-09T12:10:27.366381+0000","last_fullsized":"2023-06-09T12:10:27.366381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3447,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.402619+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:12:47.402619+0000","last_peered":"2023-06-09T12:12:47.402619+0000","last_clean":"2023-06-09T12:12:47.402619+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:12:47.402619+0000","last_undegraded":"2023-06-09T12:12:47.402619+0000","last_fullsized":"2023-06-09T12:12:47.402619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'893","reported_seq":2971,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.384781+0000","last_change":"2023-06-09T12:09:12.365941+0000","last_active":"2023-06-09T12:11:32.384781+0000","last_peered":"2023-06-09T12:11:32.384781+0000","last_clean":"2023-06-09T12:11:32.384781+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:11:32.384781+0000","last_undegraded":"2023-06-09T12:11:32.384781+0000","last_fullsized":"2023-06-09T12:11:32.384781+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1568,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:17.391146+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:10:17.391146+0000","last_peered":"2023-06-09T12:10:17.391146+0000","last_clean":"2023-06-09T12:10:17.391146+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:10:17.391146+0000","last_undegraded":"2023-06-09T12:10:17.391146+0000","last_fullsized":"2023-06-09T12:10:17.391146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3547,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334498+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334498+0000","last_peered":"2023-06-09T12:12:56.334498+0000","last_clean":"2023-06-09T12:12:56.334498+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334498+0000","last_undegraded":"2023-06-09T12:12:56.334498+0000","last_fullsized":"2023-06-09T12:12:56.334498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.387434+0000","last_change":"2023-06-09T12:14:02.519430+0000","last_active":"2023-06-09T12:13:57.387434+0000","last_peered":"2023-06-09T12:13:57.387434+0000","last_clean":"2023-06-09T12:13:57.387434+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:13:57.387434+0000","last_undegraded":"2023-06-09T12:13:57.387434+0000","last_fullsized":"2023-06-09T12:13:57.387434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1118,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.468201+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:10:37.468201+0000","last_peered":"2023-06-09T12:10:37.468201+0000","last_clean":"2023-06-09T12:10:37.468201+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:10:37.468201+0000","last_undegraded":"2023-06-09T12:10:37.468201+0000","last_fullsized":"2023-06-09T12:10:37.468201+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3870,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.393338+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:10:37.393338+0000","last_peered":"2023-06-09T12:10:37.393338+0000","last_clean":"2023-06-09T12:10:37.393338+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:10:37.393338+0000","last_undegraded":"2023-06-09T12:10:37.393338+0000","last_fullsized":"2023-06-09T12:10:37.393338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":757,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:42.468786+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:10:42.468786+0000","last_peered":"2023-06-09T12:10:42.468786+0000","last_clean":"2023-06-09T12:10:42.468786+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:10:42.468786+0000","last_undegraded":"2023-06-09T12:10:42.468786+0000","last_fullsized":"2023-06-09T12:10:42.468786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'511","reported_seq":732,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:55.464918+0000","last_change":"2023-06-09T12:14:15.227448+0000","last_active":"2023-06-09T12:14:55.464918+0000","last_peered":"2023-06-09T12:14:55.464918+0000","last_clean":"2023-06-09T12:14:55.464918+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:14:55.464918+0000","last_undegraded":"2023-06-09T12:14:55.464918+0000","last_fullsized":"2023-06-09T12:14:55.464918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:34.653035+0000","last_change":"2023-06-09T12:13:52.518166+0000","last_active":"2023-06-09T12:12:34.653035+0000","last_peered":"2023-06-09T12:12:34.653035+0000","last_clean":"2023-06-09T12:12:34.653035+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:12:34.653035+0000","last_undegraded":"2023-06-09T12:12:34.653035+0000","last_fullsized":"2023-06-09T12:12:34.653035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"73'575","reported_seq":766,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:01.857229+0000","last_change":"2023-06-09T12:15:01.482540+0000","last_active":"2023-06-09T12:15:01.857229+0000","last_peered":"2023-06-09T12:15:01.857229+0000","last_clean":"2023-06-09T12:15:01.857229+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:01.857229+0000","last_undegraded":"2023-06-09T12:15:01.857229+0000","last_fullsized":"2023-06-09T12:15:01.857229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.371860+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:11:12.371860+0000","last_peered":"2023-06-09T12:11:12.371860+0000","last_clean":"2023-06-09T12:11:12.371860+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:11:12.371860+0000","last_undegraded":"2023-06-09T12:11:12.371860+0000","last_fullsized":"2023-06-09T12:11:12.371860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1082,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397901+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:11:12.397901+0000","last_peered":"2023-06-09T12:11:12.397901+0000","last_clean":"2023-06-09T12:11:12.397901+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:11:12.397901+0000","last_undegraded":"2023-06-09T12:11:12.397901+0000","last_fullsized":"2023-06-09T12:11:12.397901+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3910,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.051997+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.051997+0000","last_peered":"2023-06-09T12:13:12.051997+0000","last_clean":"2023-06-09T12:13:12.051997+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.051997+0000","last_undegraded":"2023-06-09T12:13:12.051997+0000","last_fullsized":"2023-06-09T12:13:12.051997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397841+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:11:12.397841+0000","last_peered":"2023-06-09T12:11:12.397841+0000","last_clean":"2023-06-09T12:11:12.397841+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:11:12.397841+0000","last_undegraded":"2023-06-09T12:11:12.397841+0000","last_fullsized":"2023-06-09T12:11:12.397841+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.442440+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:11:12.442440+0000","last_peered":"2023-06-09T12:11:12.442440+0000","last_clean":"2023-06-09T12:11:12.442440+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:11:12.442440+0000","last_undegraded":"2023-06-09T12:11:12.442440+0000","last_fullsized":"2023-06-09T12:11:12.442440+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4140,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371134+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:11:07.371134+0000","last_peered":"2023-06-09T12:11:07.371134+0000","last_clean":"2023-06-09T12:11:07.371134+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:11:07.371134+0000","last_undegraded":"2023-06-09T12:11:07.371134+0000","last_fullsized":"2023-06-09T12:11:07.371134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:57.396024+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:10:57.396024+0000","last_peered":"2023-06-09T12:10:57.396024+0000","last_clean":"2023-06-09T12:10:57.396024+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:10:57.396024+0000","last_undegraded":"2023-06-09T12:10:57.396024+0000","last_fullsized":"2023-06-09T12:10:57.396024+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3006,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:54.797717+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:13:54.797717+0000","last_peered":"2023-06-09T12:13:54.797717+0000","last_clean":"2023-06-09T12:13:54.797717+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:13:54.797717+0000","last_undegraded":"2023-06-09T12:13:54.797717+0000","last_fullsized":"2023-06-09T12:13:54.797717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.472933+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:11:17.472933+0000","last_peered":"2023-06-09T12:11:17.472933+0000","last_clean":"2023-06-09T12:11:17.472933+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:11:17.472933+0000","last_undegraded":"2023-06-09T12:11:17.472933+0000","last_fullsized":"2023-06-09T12:11:17.472933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":530,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:42.401621+0000","last_change":"2023-06-09T12:14:59.442204+0000","last_active":"2023-06-09T12:13:42.401621+0000","last_peered":"2023-06-09T12:13:42.401621+0000","last_clean":"2023-06-09T12:13:42.401621+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:13:42.401621+0000","last_undegraded":"2023-06-09T12:13:42.401621+0000","last_fullsized":"2023-06-09T12:13:42.401621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.381230+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:11:07.381230+0000","last_peered":"2023-06-09T12:11:07.381230+0000","last_clean":"2023-06-09T12:11:07.381230+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:11:07.381230+0000","last_undegraded":"2023-06-09T12:11:07.381230+0000","last_fullsized":"2023-06-09T12:11:07.381230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.623947+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.623947+0000","last_peered":"2023-06-09T12:14:50.623947+0000","last_clean":"2023-06-09T12:14:50.623947+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.623947+0000","last_undegraded":"2023-06-09T12:14:50.623947+0000","last_fullsized":"2023-06-09T12:14:50.623947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3946,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.398742+0000","last_change":"2023-06-09T12:13:37.318635+0000","last_active":"2023-06-09T12:13:22.398742+0000","last_peered":"2023-06-09T12:13:22.398742+0000","last_clean":"2023-06-09T12:13:22.398742+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:22.398742+0000","last_undegraded":"2023-06-09T12:13:22.398742+0000","last_fullsized":"2023-06-09T12:13:22.398742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286291+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286291+0000","last_peered":"2023-06-09T12:14:11.286291+0000","last_clean":"2023-06-09T12:14:11.286291+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286291+0000","last_undegraded":"2023-06-09T12:14:11.286291+0000","last_fullsized":"2023-06-09T12:14:11.286291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":72775,"ondisk_log_size":72775,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1734253,"num_omap_keys":3219,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30896,"ondisk_log_size":30896,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575623,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69599999999999995},"min":{"1min":0.55200000000000005,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.0229999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.143,"5min":1.143,"15min":1.3320000000000001},"last":0.94599999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73699999999999999,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.75,"15min":0.745},"min":{"1min":0.61299999999999999,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":0.93899999999999995,"5min":0.94599999999999995,"15min":1.05},"last":0.84399999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0649999999999999},"last":0.745},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.745},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.3089999999999999},"last":0.78800000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:14:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71199999999999997,"15min":0.69899999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.86899999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.72599999999999998,"15min":0.72199999999999998},"min":{"1min":0.56799999999999995,"5min":0.52800000000000002,"15min":0.505},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.41799999999999998,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.73799999999999999,"15min":0.72799999999999998},"min":{"1min":0.624,"5min":0.505,"15min":0.45900000000000002},"max":{"1min":1.115,"5min":1.115,"15min":1.226},"last":0.85799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.623,"15min":0.625},"min":{"1min":0.46400000000000002,"5min":0.41899999999999998,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90900000000000003,"15min":0.94999999999999996},"last":0.63300000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.66200000000000003,"15min":0.65900000000000003},"min":{"1min":0.52500000000000002,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.90600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575623,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.044},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.54600000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0960000000000001,"5min":1.266,"15min":1.266},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.59199999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.139,"5min":1.298,"15min":1.298},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":1.091,"15min":1.091},"last":0.68200000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.046,"15min":1.046},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.623,"5min":0.59199999999999997,"15min":0.55900000000000005},"max":{"1min":1.077,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.47699999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.73899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.435},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":0.501},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.74299999999999999,"15min":0.69599999999999995},"min":{"1min":0.55300000000000005,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.0609999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.53300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575624,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404976,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98560,"kb_avail":93966864,"statfs":{"total":96636764160,"available":96222068736,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":100925440},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.55600000000000005,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.78800000000000003,"15min":0.85099999999999998},"last":0.77000000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.432,"5min":0.376,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.81899999999999995,"15min":0.95299999999999996},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":0.77700000000000002,"5min":1.288,"15min":1.288},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.873,"15min":0.80400000000000005},"min":{"1min":0.74199999999999999,"5min":0.53800000000000003,"15min":0.45700000000000002},"max":{"1min":1.0840000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.012,"5min":1.175,"15min":1.597},"last":0.754}]},{"osd":5,"last update":"Fri Jun 9 12:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.79600000000000004,"15min":0.82799999999999996},"min":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0720000000000001,"5min":1.196,"15min":1.544},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.84499999999999997,"15min":0.86799999999999999},"min":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.874}]},{"osd":6,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84099999999999997},"min":{"1min":0.70099999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0409999999999999,"5min":1.1910000000000001,"15min":1.548},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.83999999999999997,"15min":0.82899999999999996},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.96799999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89700000000000002,"15min":0.90000000000000002},"min":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.131,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.86799999999999999,"15min":0.84499999999999997},"min":{"1min":0.73599999999999999,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.095,"5min":1.24,"15min":1.498},"last":1.0049999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575623,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331176,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103360,"kb_avail":94040664,"statfs":{"total":96636764160,"available":96297639936,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":105840640},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.48099999999999998},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.371},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.68600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57699999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78000000000000003,"5min":0.81499999999999995,"15min":1.032},"last":0.435},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":0.871,"15min":0.96199999999999997},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88500000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.86499999999999999,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":1.1040000000000001,"5min":1.421,"15min":1.421},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90300000000000002,"15min":0.878},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.58799999999999997},"max":{"1min":1.024,"5min":1.268,"15min":1.268},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.036,"5min":1.077,"15min":1.121},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.879,"15min":0.878},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.111,"5min":1.371,"15min":3.5840000000000001},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.60299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0740000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83599999999999997,"15min":0.83999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0409999999999999,"5min":1.22,"15min":1.232},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.873,"15min":0.86299999999999999},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.47699999999999998},"max":{"1min":1.1299999999999999,"5min":1.177,"15min":1.298},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96099999999999997,"15min":0.94599999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60999999999999999},"max":{"1min":1.147,"5min":1.371,"15min":1.371},"last":0.86599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575624,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56799999999999995,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.999},"last":0.49299999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":1.236},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.624},"min":{"1min":0.52300000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.47499999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":1.298,"5min":1.319,"15min":1.4650000000000001},"last":0.56299999999999994}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.82399999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.70199999999999996,"5min":0.56899999999999995,"15min":0.52800000000000002},"max":{"1min":1.337,"5min":1.3740000000000001,"15min":1.4359999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.3640000000000001,"5min":1.417,"15min":1.5189999999999999},"last":0.89600000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.3029999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85799999999999998,"15min":0.81299999999999994},"min":{"1min":0.63,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.92200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86699999999999999,"15min":0.84599999999999997},"min":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.878},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.88500000000000001,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.97399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575623,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438012,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114816,"kb_avail":93933828,"statfs":{"total":96636764160,"available":96188239872,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117571584},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.498}]},{"osd":1,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.747,"5min":0.872,"15min":0.97099999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82099999999999995,"15min":0.80600000000000005},"min":{"1min":0.64100000000000001,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":3.081},"last":0.72299999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84599999999999997,"15min":0.84299999999999997},"min":{"1min":0.66000000000000003,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.52500000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.64600000000000002,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57299999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.63100000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.046,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":0.69499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575622,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417876,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125120,"kb_avail":93953964,"statfs":{"total":96636764160,"available":96208859136,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128122880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77600000000000002,"15min":0.74199999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":1.0549999999999999,"5min":1.419,"15min":1.419},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.78600000000000003},"min":{"1min":0.67100000000000004,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.1160000000000001,"5min":1.383,"15min":1.383},"last":0.70499999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.432},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.629}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.1890000000000001,"15min":1.2410000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72999999999999998,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48299999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.68200000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.91500000000000004,"15min":0.95999999999999996},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.625},"min":{"1min":0.51400000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0509999999999999},"last":0.749},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.96799999999999997,"5min":1.0029999999999999,"15min":1.131},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.38300000000000001},"max":{"1min":1.0229999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.71799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575622,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0329999999999999,"15min":3.468},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":0.90000000000000002,"5min":1.579,"15min":1.579},"last":0.73699999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.377},"max":{"1min":0.78800000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.498,"15min":0.439},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.60499999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.98299999999999998,"5min":1.391,"15min":1.391},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.169,"15min":1.254},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.68500000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":1.522,"15min":1.522},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70699999999999996,"5min":1.28,"15min":1.28},"last":0.60399999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.40200000000000002},"max":{"1min":0.999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.64700000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78100000000000003,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.433},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92600000000000005,"5min":1.226,"15min":1.238},"last":0.52300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:15:09.009 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.010 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.010 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.011 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.011 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.012 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.012 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.013 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.013 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.014 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.014 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:09.015 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:15:29.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:15:29.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 Processor -- start 2023-06-09T12:15:29.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 -- start start 2023-06-09T12:15:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 0x7f90b00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 0x7f90b0159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90b015cd40 0x7f90b00cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b0159b20 con 0x7f90b015cd40 2023-06-09T12:15:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00cac80 con 0x7f90b00cb9a0 2023-06-09T12:15:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.221+0000 7f90b49ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01597d0 con 0x7f90b00cb5d0 2023-06-09T12:15:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90b015cd40 0x7f90b00cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.226 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f90b015cd40 0x7f90b00cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45676/0 (socket says 172.21.15.7:45676) 2023-06-09T12:15:29.227 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 -- 172.21.15.7:0/307931611 learned_addr learned my addr 172.21.15.7:0/307931611 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90ae59c700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 0x7f90b00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90add9b700 1 --2- 172.21.15.7:0/307931611 >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 0x7f90b0159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.228 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 -- 172.21.15.7:0/307931611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 msgr2=0x7f90b00b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:15:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 0x7f90b00b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 -- 172.21.15.7:0/307931611 >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 msgr2=0x7f90b0159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:15:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 --2- 172.21.15.7:0/307931611 >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 0x7f90b0159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 -- 172.21.15.7:0/307931611 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b0071ab0 con 0x7f90b015cd40 2023-06-09T12:15:29.229 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90ae59c700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 0x7f90b00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:15:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90add9b700 1 --2- 172.21.15.7:0/307931611 >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 0x7f90b0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:15:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.222+0000 7f90aed9d700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b015cd40 0x7f90b00cbea0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f90b0067150 tx=0x7f90a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4de0070e269c0f9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90ad59a700 1 -- 172.21.15.7:0/307931611 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a800e070 con 0x7f90b015cd40 2023-06-09T12:15:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90ad59a700 1 -- 172.21.15.7:0/307931611 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a800b9c0 con 0x7f90b015cd40 2023-06-09T12:15:29.230 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90ad59a700 1 -- 172.21.15.7:0/307931611 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a800bd70 con 0x7f90b015cd40 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 -- 172.21.15.7:0/307931611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b015cd40 msgr2=0x7f90b00cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b015cd40 0x7f90b00cbea0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f90b0067150 tx=0x7f90a800b330 comp rx=0 tx=0).stop 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 -- 172.21.15.7:0/307931611 shutdown_connections 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b015cd40 0x7f90b00cbea0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 --2- 172.21.15.7:0/307931611 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb5d0 0x7f90b00b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.231 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 --2- 172.21.15.7:0/307931611 >> v2:172.21.15.117:3300/0 conn(0x7f90b00cb9a0 0x7f90b0159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 -- 172.21.15.7:0/307931611 >> 172.21.15.7:0/307931611 conn(0x7f90b0070af0 msgr2=0x7f90b0070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 -- 172.21.15.7:0/307931611 shutdown_connections 2023-06-09T12:15:29.232 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.223+0000 7f90b49ff700 1 -- 172.21.15.7:0/307931611 wait complete. 2023-06-09T12:15:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.226+0000 7f90b49ff700 1 Processor -- start 2023-06-09T12:15:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.226+0000 7f90b49ff700 1 -- start start 2023-06-09T12:15:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.234 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb9a0 0x7f90b00b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b015cd40 0x7f90b0154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01ef590 con 0x7f90b00cb5d0 2023-06-09T12:15:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00d02f0 con 0x7f90b015cd40 2023-06-09T12:15:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90b49ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b00ceac0 con 0x7f90b00cb9a0 2023-06-09T12:15:29.235 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.227+0000 7f90ae59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45688/0 (socket says 172.21.15.7:45688) 2023-06-09T12:15:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 -- 172.21.15.7:0/833406880 learned_addr learned my addr 172.21.15.7:0/833406880 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90add9b700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb9a0 0x7f90b00b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90aed9d700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b015cd40 0x7f90b0154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.236 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 -- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb9a0 msgr2=0x7f90b00b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb9a0 0x7f90b00b29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 -- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b015cd40 msgr2=0x7f90b0154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b015cd40 0x7f90b0154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 -- 172.21.15.7:0/833406880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a8009b30 con 0x7f90b00cb5d0 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90ae59c700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f90a000d800 tx=0x7f90a000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:29.237 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004030 con 0x7f90b00cb5d0 2023-06-09T12:15:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0004190 con 0x7f90b00cb5d0 2023-06-09T12:15:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a0004520 con 0x7f90b00cb5d0 2023-06-09T12:15:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.228+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b015d4f0 con 0x7f90b00cb5d0 2023-06-09T12:15:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.229+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b015d210 con 0x7f90b00cb5d0 2023-06-09T12:15:29.238 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.230+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b015d7d0 con 0x7f90b00cb5d0 2023-06-09T12:15:29.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.231+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f90a00036a0 con 0x7f90b00cb5d0 2023-06-09T12:15:29.239 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.231+0000 7f909effd700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.232+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f90a0089720 con 0x7f90b00cb5d0 2023-06-09T12:15:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.232+0000 7f90add9b700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.232+0000 7f90add9b700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f90b01589c0 tx=0x7f90a4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:29.240 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.237+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f90a0057990 con 0x7f90b00cb5d0 2023-06-09T12:15:29.443 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.439+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90b00ceda0 con 0x7f909006ace0 2023-06-09T12:15:29.452 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.449+0000 7f909effd700 1 -- 172.21.15.7:0/833406880 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+318936 (secure 0 0 0) 0x7f90b00ceda0 con 0x7f909006ace0 2023-06-09T12:15:29.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:15:29.457 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 msgr2=0x7f909006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f90b01589c0 tx=0x7f90a4008040 comp rx=0 tx=0).stop 2023-06-09T12:15:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 msgr2=0x7f90b00cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f90a000d800 tx=0x7f90a000dbc0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 shutdown_connections 2023-06-09T12:15:29.458 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90b015cd40 0x7f90b0154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90b00cb5d0 0x7f90b00cef50 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f909006ace0 0x7f909006d190 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 --2- 172.21.15.7:0/833406880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90b00cb9a0 0x7f90b00b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:29.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 >> 172.21.15.7:0/833406880 conn(0x7f90b0070af0 msgr2=0x7f90b00c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:29.459 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 shutdown_connections 2023-06-09T12:15:29.460 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:29.453+0000 7f90b49ff700 1 -- 172.21.15.7:0/833406880 wait complete. 2023-06-09T12:15:29.460 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:15:29.475 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2011,"stamp":"2023-06-09T12:15:29.190330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1829068,"num_omap_keys":3407,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103919,"ondisk_log_size":103919,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2604552,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":721216,"kb_avail":752370168,"statfs":{"total":773094113280,"available":770427052032,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":738525184},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003278"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3704,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.563921+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:11:32.563921+0000","last_peered":"2023-06-09T12:11:32.563921+0000","last_clean":"2023-06-09T12:11:32.563921+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:11:32.563921+0000","last_undegraded":"2023-06-09T12:11:32.563921+0000","last_fullsized":"2023-06-09T12:11:32.563921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3609,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.491093+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:10:37.491093+0000","last_peered":"2023-06-09T12:10:37.491093+0000","last_clean":"2023-06-09T12:10:37.491093+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:10:37.491093+0000","last_undegraded":"2023-06-09T12:10:37.491093+0000","last_fullsized":"2023-06-09T12:10:37.491093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4218,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:14:07.706040+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:14:07.706040+0000","last_peered":"2023-06-09T12:14:07.706040+0000","last_clean":"2023-06-09T12:14:07.706040+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:14:07.706040+0000","last_undegraded":"2023-06-09T12:14:07.706040+0000","last_fullsized":"2023-06-09T12:14:07.706040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3467,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.666639+0000","last_change":"2023-06-09T12:13:42.516760+0000","last_active":"2023-06-09T12:12:32.666639+0000","last_peered":"2023-06-09T12:12:32.666639+0000","last_clean":"2023-06-09T12:12:32.666639+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:12:32.666639+0000","last_undegraded":"2023-06-09T12:12:32.666639+0000","last_fullsized":"2023-06-09T12:12:32.666639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3390,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.496849+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:11:17.496849+0000","last_peered":"2023-06-09T12:11:17.496849+0000","last_clean":"2023-06-09T12:11:17.496849+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:11:17.496849+0000","last_undegraded":"2023-06-09T12:11:17.496849+0000","last_fullsized":"2023-06-09T12:11:17.496849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.397879+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:11:12.397879+0000","last_peered":"2023-06-09T12:11:12.397879+0000","last_clean":"2023-06-09T12:11:12.397879+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:11:12.397879+0000","last_undegraded":"2023-06-09T12:11:12.397879+0000","last_fullsized":"2023-06-09T12:11:12.397879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3583,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:50.396949+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:10:50.396949+0000","last_peered":"2023-06-09T12:10:50.396949+0000","last_clean":"2023-06-09T12:10:50.396949+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:10:50.396949+0000","last_undegraded":"2023-06-09T12:10:50.396949+0000","last_fullsized":"2023-06-09T12:10:50.396949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3447,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.402619+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:12:47.402619+0000","last_peered":"2023-06-09T12:12:47.402619+0000","last_clean":"2023-06-09T12:12:47.402619+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:12:47.402619+0000","last_undegraded":"2023-06-09T12:12:47.402619+0000","last_fullsized":"2023-06-09T12:12:47.402619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'991","reported_seq":3077,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:24.709014+0000","last_change":"2023-06-09T12:15:13.473468+0000","last_active":"2023-06-09T12:15:24.709014+0000","last_peered":"2023-06-09T12:15:24.709014+0000","last_clean":"2023-06-09T12:15:24.709014+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:24.709014+0000","last_undegraded":"2023-06-09T12:15:24.709014+0000","last_fullsized":"2023-06-09T12:15:24.709014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":98,"log_size":991,"log_dups_size":0,"ondisk_log_size":991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 10s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3417,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.371011+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:11:07.371011+0000","last_peered":"2023-06-09T12:11:07.371011+0000","last_clean":"2023-06-09T12:11:07.371011+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:11:07.371011+0000","last_undegraded":"2023-06-09T12:11:07.371011+0000","last_fullsized":"2023-06-09T12:11:07.371011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1045,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.387434+0000","last_change":"2023-06-09T12:14:02.519430+0000","last_active":"2023-06-09T12:13:57.387434+0000","last_peered":"2023-06-09T12:13:57.387434+0000","last_clean":"2023-06-09T12:13:57.387434+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:13:57.387434+0000","last_undegraded":"2023-06-09T12:13:57.387434+0000","last_fullsized":"2023-06-09T12:13:57.387434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1383,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.381918+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:11:12.381918+0000","last_peered":"2023-06-09T12:11:12.381918+0000","last_clean":"2023-06-09T12:11:12.381918+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:11:12.381918+0000","last_undegraded":"2023-06-09T12:11:12.381918+0000","last_fullsized":"2023-06-09T12:11:12.381918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3849,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:34.653035+0000","last_change":"2023-06-09T12:13:52.518166+0000","last_active":"2023-06-09T12:12:34.653035+0000","last_peered":"2023-06-09T12:12:34.653035+0000","last_clean":"2023-06-09T12:12:34.653035+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:12:34.653035+0000","last_undegraded":"2023-06-09T12:12:34.653035+0000","last_fullsized":"2023-06-09T12:12:34.653035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.495555+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:11:07.495555+0000","last_peered":"2023-06-09T12:11:07.495555+0000","last_clean":"2023-06-09T12:11:07.495555+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:11:07.495555+0000","last_undegraded":"2023-06-09T12:11:07.495555+0000","last_fullsized":"2023-06-09T12:11:07.495555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1930","reported_seq":3011,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:12.421414+0000","last_change":"2023-06-09T12:13:37.678741+0000","last_active":"2023-06-09T12:15:12.421414+0000","last_peered":"2023-06-09T12:15:12.421414+0000","last_clean":"2023-06-09T12:15:12.421414+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:12.421414+0000","last_undegraded":"2023-06-09T12:15:12.421414+0000","last_fullsized":"2023-06-09T12:15:12.421414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":1930,"log_dups_size":0,"ondisk_log_size":1930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3422,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.391379+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:11:27.391379+0000","last_peered":"2023-06-09T12:11:27.391379+0000","last_clean":"2023-06-09T12:11:27.391379+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:11:27.391379+0000","last_undegraded":"2023-06-09T12:11:27.391379+0000","last_fullsized":"2023-06-09T12:11:27.391379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":530,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:42.401621+0000","last_change":"2023-06-09T12:14:59.442204+0000","last_active":"2023-06-09T12:13:42.401621+0000","last_peered":"2023-06-09T12:13:42.401621+0000","last_clean":"2023-06-09T12:13:42.401621+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:13:42.401621+0000","last_undegraded":"2023-06-09T12:13:42.401621+0000","last_fullsized":"2023-06-09T12:13:42.401621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.396890+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:13:07.396890+0000","last_peered":"2023-06-09T12:13:07.396890+0000","last_clean":"2023-06-09T12:13:07.396890+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:13:07.396890+0000","last_undegraded":"2023-06-09T12:13:07.396890+0000","last_fullsized":"2023-06-09T12:13:07.396890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.623947+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.623947+0000","last_peered":"2023-06-09T12:14:50.623947+0000","last_clean":"2023-06-09T12:14:50.623947+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.623947+0000","last_undegraded":"2023-06-09T12:14:50.623947+0000","last_fullsized":"2023-06-09T12:14:50.623947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4817,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:10:37.438806+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:10:37.438806+0000","last_peered":"2023-06-09T12:10:37.438806+0000","last_clean":"2023-06-09T12:10:37.438806+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:10:37.438806+0000","last_undegraded":"2023-06-09T12:10:37.438806+0000","last_fullsized":"2023-06-09T12:10:37.438806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203772+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203772+0000","last_peered":"2023-06-09T12:13:37.203772+0000","last_clean":"2023-06-09T12:13:37.203772+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203772+0000","last_undegraded":"2023-06-09T12:13:37.203772+0000","last_fullsized":"2023-06-09T12:13:37.203772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3947,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.318635+0000","last_change":"2023-06-09T12:15:27.414934+0000","last_active":"2023-06-09T12:13:37.318635+0000","last_peered":"2023-06-09T12:13:37.318635+0000","last_clean":"2023-06-09T12:13:37.318635+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:37.318635+0000","last_undegraded":"2023-06-09T12:13:37.318635+0000","last_fullsized":"2023-06-09T12:13:37.318635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3661,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:39.403399+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:11:39.403399+0000","last_peered":"2023-06-09T12:11:39.403399+0000","last_clean":"2023-06-09T12:11:39.403399+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:11:39.403399+0000","last_undegraded":"2023-06-09T12:11:39.403399+0000","last_fullsized":"2023-06-09T12:11:39.403399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286291+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286291+0000","last_peered":"2023-06-09T12:14:11.286291+0000","last_clean":"2023-06-09T12:14:11.286291+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286291+0000","last_undegraded":"2023-06-09T12:14:11.286291+0000","last_fullsized":"2023-06-09T12:14:11.286291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":72873,"ondisk_log_size":72873,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1829068,"num_omap_keys":3407,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30927,"ondisk_log_size":30927,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575628,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73399999999999999,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":1.143,"15min":1.256},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.748},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.65100000000000002,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.78600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3089999999999999},"last":0.624}]},{"osd":3,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72399999999999998,"15min":0.70299999999999996},"min":{"1min":0.61599999999999999,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.505},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.63600000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":1.115,"15min":1.226},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.65700000000000003},"min":{"1min":0.51300000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.97099999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.47599999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575628,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.044},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.54600000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0960000000000001,"5min":1.266,"15min":1.266},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.59199999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.139,"5min":1.298,"15min":1.298},"last":0.72999999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.495},{"interface":"front","average":{"1min":0.753,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":1.091,"15min":1.091},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.046,"15min":1.046},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.623,"5min":0.59199999999999997,"15min":0.55900000000000005},"max":{"1min":1.077,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.47699999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.77800000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.69599999999999995},"min":{"1min":0.53300000000000003,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.75}]}]},{"osd":1,"up_from":13,"seq":55834575629,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405104,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98688,"kb_avail":93966736,"statfs":{"total":96636764160,"available":96221937664,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101056512},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.55600000000000005,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.78800000000000003,"15min":0.85099999999999998},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.432,"5min":0.376,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.81899999999999995,"15min":0.95299999999999996},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":0.77700000000000002,"5min":1.288,"15min":1.288},"last":0.76300000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.873,"15min":0.80400000000000005},"min":{"1min":0.74199999999999999,"5min":0.53800000000000003,"15min":0.45700000000000002},"max":{"1min":1.0840000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.012,"5min":1.175,"15min":1.597},"last":0.79800000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80700000000000005,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.091,"5min":1.196,"15min":1.544},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.59499999999999997},"max":{"1min":1.044,"5min":1.1539999999999999,"15min":1.704},"last":0.86599999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84099999999999997},"min":{"1min":0.70099999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0409999999999999,"5min":1.1910000000000001,"15min":1.548},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.83999999999999997,"15min":0.82899999999999996},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.83899999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89700000000000002,"15min":0.90000000000000002},"min":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.131,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.86799999999999999,"15min":0.84499999999999997},"min":{"1min":0.73599999999999999,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.095,"5min":1.24,"15min":1.498},"last":0.80900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575628,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331176,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103360,"kb_avail":94040664,"statfs":{"total":96636764160,"available":96297639936,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":105840640},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.52200000000000002,"15min":0.48099999999999998},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.5},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46200000000000002,"5min":0.435,"15min":0.35799999999999998},"max":{"1min":0.68600000000000005,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55000000000000004,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.78000000000000003,"5min":0.81499999999999995,"15min":1.032},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.53100000000000003,"15min":0.53500000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.30599999999999999},"max":{"1min":0.70999999999999996,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.61699999999999999,"15min":0.58499999999999996},"min":{"1min":0.432,"5min":0.40799999999999997,"15min":0.34599999999999997},"max":{"1min":0.80700000000000005,"5min":0.871,"15min":0.96199999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.88500000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.752}]},{"osd":4,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.86499999999999999,"15min":0.83499999999999996},"min":{"1min":0.65000000000000002,"5min":0.53700000000000003,"15min":0.42199999999999999},"max":{"1min":1.1040000000000001,"5min":1.421,"15min":1.421},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.90300000000000002,"15min":0.878},"min":{"1min":0.63100000000000001,"5min":0.628,"15min":0.58799999999999997},"max":{"1min":1.024,"5min":1.268,"15min":1.268},"last":1.0680000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.036,"5min":1.077,"15min":1.121},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.876,"5min":0.879,"15min":0.878},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.111,"5min":1.371,"15min":3.5840000000000001},"last":1.1259999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.81299999999999994,"15min":0.80400000000000005},"min":{"1min":0.60299999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0740000000000001,"5min":1.1499999999999999,"15min":1.2330000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83599999999999997,"15min":0.83999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0409999999999999,"5min":1.22,"15min":1.232},"last":1.03}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.873,"15min":0.86299999999999999},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.47699999999999998},"max":{"1min":1.1299999999999999,"5min":1.177,"15min":1.298},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96099999999999997,"15min":0.94599999999999995},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.60999999999999999},"max":{"1min":1.147,"5min":1.371,"15min":1.371},"last":1.179}]}]},{"osd":2,"up_from":13,"seq":55834575629,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56799999999999995,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.999},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":1.236},"last":0.44},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.624},"min":{"1min":0.52300000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.60399999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.5},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.47499999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":1.298,"5min":1.319,"15min":1.4650000000000001},"last":0.53500000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.70199999999999996,"5min":0.56899999999999995,"15min":0.52800000000000002},"max":{"1min":1.337,"5min":1.3740000000000001,"15min":1.4359999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.3640000000000001,"5min":1.417,"15min":1.5189999999999999},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.3029999999999999},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85799999999999998,"15min":0.81299999999999994},"min":{"1min":0.63,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.98199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86699999999999999,"15min":0.84599999999999997},"min":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.88500000000000001,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575628,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438140,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":114944,"kb_avail":93933700,"statfs":{"total":96636764160,"available":96188108800,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":117702656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56699999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.54400000000000004,"15min":0.53200000000000003},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.69199999999999995,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60699999999999998,"15min":0.56200000000000006},"min":{"1min":0.443,"5min":0.42299999999999999,"15min":0.35899999999999999},"max":{"1min":0.71099999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.747,"5min":0.872,"15min":0.97099999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.34100000000000003},"max":{"1min":0.97699999999999998,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82099999999999995,"15min":0.80600000000000005},"min":{"1min":0.64100000000000001,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.83099999999999996,"15min":0.81799999999999995},"min":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":3.081},"last":0.72199999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.84599999999999997,"15min":0.84299999999999997},"min":{"1min":0.66000000000000003,"5min":0.57399999999999995,"15min":0.52800000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90300000000000002,"15min":0.89300000000000002},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.52500000000000002},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.80300000000000005,"15min":0.81200000000000006},"min":{"1min":0.64600000000000002,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":0.98999999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.81499999999999995,"15min":0.82299999999999995},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:14:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57299999999999995},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.83799999999999997,"15min":0.86299999999999999},"min":{"1min":0.63100000000000001,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":1.046,"5min":1.1910000000000001,"15min":3.1160000000000001},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575627,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417876,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125120,"kb_avail":93953964,"statfs":{"total":96636764160,"available":96208859136,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128122880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.77600000000000002,"15min":0.74199999999999999},"min":{"1min":0.66400000000000003,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":1.0549999999999999,"5min":1.419,"15min":1.419},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.80100000000000005,"15min":0.78600000000000003},"min":{"1min":0.67100000000000004,"5min":0.51200000000000001,"15min":0.46500000000000002},"max":{"1min":1.1160000000000001,"5min":1.383,"15min":1.383},"last":0.90000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.73599999999999999,"15min":0.73699999999999999},"min":{"1min":0.48199999999999998,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.79900000000000004,"15min":0.78800000000000003},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.41599999999999998},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.50600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.80500000000000005,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.441,"15min":0.441},"max":{"1min":1.167,"5min":1.1890000000000001,"15min":1.2410000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.72999999999999998,"15min":0.70099999999999996},"min":{"1min":0.54500000000000004,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.55200000000000005,"5min":0.499,"15min":0.48299999999999998},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.94699999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.36199999999999999},"max":{"1min":0.83599999999999997,"5min":0.91500000000000004,"15min":0.95999999999999996},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.623,"15min":0.625},"min":{"1min":0.51400000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57899999999999996,"15min":0.58099999999999996},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0509999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.746,"15min":0.73899999999999999},"min":{"1min":0.59599999999999997,"5min":0.55100000000000005,"15min":0.47899999999999998},"max":{"1min":0.96799999999999997,"5min":1.0029999999999999,"15min":1.131},"last":0.88300000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:14:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.497,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81200000000000006,"5min":0.98099999999999998,"15min":1.1990000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.745,"5min":0.65800000000000003,"15min":0.64400000000000002},"min":{"1min":0.56100000000000005,"5min":0.46500000000000002,"15min":0.38300000000000001},"max":{"1min":1.0229999999999999,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.91900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0329999999999999,"15min":3.468},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":0.90000000000000002,"5min":1.579,"15min":1.579},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.377},"max":{"1min":0.78800000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.498,"15min":0.439},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.60499999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.98299999999999998,"5min":1.391,"15min":1.391},"last":0.68100000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.169,"15min":1.254},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.68500000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":1.522,"15min":1.522},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70699999999999996,"5min":1.28,"15min":1.28},"last":0.38400000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.40200000000000002},"max":{"1min":0.999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.45600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78100000000000003,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.434},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92600000000000005,"5min":1.226,"15min":1.238},"last":0.50700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:15:29.482 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.482 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.483 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.483 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.484 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.484 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.485 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.485 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.486 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:29.486 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:15:30.029 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:30.034 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:15:49.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:15:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.695+0000 7fb44d5ff700 1 Processor -- start 2023-06-09T12:15:49.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.695+0000 7fb44d5ff700 1 -- start start 2023-06-09T12:15:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.696+0000 7fb44d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb5d0 0x7fb4480b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.696+0000 7fb44d5ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4480cb9a0 0x7fb448159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.696+0000 7fb44d5ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb448159b20 con 0x7fb44815cd40 2023-06-09T12:15:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.696+0000 7fb44d5ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4480cac80 con 0x7fb4480cb9a0 2023-06-09T12:15:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.696+0000 7fb44d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4481597d0 con 0x7fb4480cb5d0 2023-06-09T12:15:49.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4480cb9a0 0x7fb448159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb4480cb9a0 0x7fb448159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34826/0 (socket says 172.21.15.7:34826) 2023-06-09T12:15:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 -- 172.21.15.7:0/479217943 learned_addr learned my addr 172.21.15.7:0/479217943 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:49.701 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb446ffd700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb5d0 0x7fb4480b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 -- 172.21.15.7:0/479217943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb5d0 msgr2=0x7fb4480b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:15:49.702 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb5d0 0x7fb4480b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 -- 172.21.15.7:0/479217943 >> v1:172.21.15.117:6789/0 conn(0x7fb44815cd40 legacy=0x7fb4480cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 -- 172.21.15.7:0/479217943 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb448071ab0 con 0x7fb4480cb9a0 2023-06-09T12:15:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.697+0000 7fb4467fc700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4480cb9a0 0x7fb448159fb0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098f0 tx=0x7fb43c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521f1c31a86cd5a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb445ffb700 1 -- 172.21.15.7:0/479217943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c011830 con 0x7fb4480cb9a0 2023-06-09T12:15:49.703 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb445ffb700 1 -- 172.21.15.7:0/479217943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43c011990 con 0x7fb4480cb9a0 2023-06-09T12:15:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb445ffb700 1 -- 172.21.15.7:0/479217943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43c011d20 con 0x7fb4480cb9a0 2023-06-09T12:15:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 -- 172.21.15.7:0/479217943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4480cb9a0 msgr2=0x7fb448159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4480cb9a0 0x7fb448159fb0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb43c0098f0 tx=0x7fb43c009c00 comp rx=0 tx=0).stop 2023-06-09T12:15:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 -- 172.21.15.7:0/479217943 shutdown_connections 2023-06-09T12:15:49.704 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb5d0 0x7fb4480b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/479217943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4480cb9a0 0x7fb448159fb0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 -- 172.21.15.7:0/479217943 >> 172.21.15.7:0/479217943 conn(0x7fb448070af0 msgr2=0x7fb448070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.698+0000 7fb44d5ff700 1 -- 172.21.15.7:0/479217943 shutdown_connections 2023-06-09T12:15:49.705 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.699+0000 7fb44d5ff700 1 -- 172.21.15.7:0/479217943 wait complete. 2023-06-09T12:15:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.700+0000 7fb44d5ff700 1 Processor -- start 2023-06-09T12:15:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.700+0000 7fb44d5ff700 1 -- start start 2023-06-09T12:15:49.707 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 0x7fb44815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 0x7fb4480cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4480ad700 con 0x7fb44815cd40 2023-06-09T12:15:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4481efbb0 con 0x7fb4480cb5d0 2023-06-09T12:15:49.708 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.701+0000 7fb44d5ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4480ce2a0 con 0x7fb4480cb9a0 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34832/0 (socket says 172.21.15.7:34832) 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 -- 172.21.15.7:0/1170202290 learned_addr learned my addr 172.21.15.7:0/1170202290 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb446ffd700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 0x7fb44815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4467fc700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 0x7fb4480cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 -- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 msgr2=0x7fb4480cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 0x7fb4480cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 -- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 msgr2=0x7fb44815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 0x7fb44815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 -- 172.21.15.7:0/1170202290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c009550 con 0x7fb44815cd40 2023-06-09T12:15:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4467fc700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 0x7fb4480cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:15:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb446ffd700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 0x7fb44815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:15:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb4477fe700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb44000cb40 tx=0x7fb44000cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb440003c00 con 0x7fb44815cd40 2023-06-09T12:15:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb440003d60 con 0x7fb44815cd40 2023-06-09T12:15:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44815d4f0 con 0x7fb44815cd40 2023-06-09T12:15:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.702+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44000ab50 con 0x7fb44815cd40 2023-06-09T12:15:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.703+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44815d210 con 0x7fb44815cd40 2023-06-09T12:15:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.704+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44815d7d0 con 0x7fb44815cd40 2023-06-09T12:15:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.705+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb44000acb0 con 0x7fb44815cd40 2023-06-09T12:15:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.706+0000 7fb42f7fe700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 0x7fb43406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:15:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.706+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb440089de0 con 0x7fb44815cd40 2023-06-09T12:15:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.706+0000 7fb446ffd700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 0x7fb43406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:15:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.707+0000 7fb446ffd700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 0x7fb43406d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb438009b60 tx=0x7fb438008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:15:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.711+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb440054570 con 0x7fb44815cd40 2023-06-09T12:15:49.918 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.915+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb448157530 con 0x7fb43406ace0 2023-06-09T12:15:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.925+0000 7fb42f7fe700 1 -- 172.21.15.7:0/1170202290 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319368 (secure 0 0 0) 0x7fb448157530 con 0x7fb43406ace0 2023-06-09T12:15:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:15:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.929+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 msgr2=0x7fb43406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.933 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 0x7fb43406d190 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb438009b60 tx=0x7fb438008040 comp rx=0 tx=0).stop 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 msgr2=0x7fb4480cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb44000cb40 tx=0x7fb44000cf00 comp rx=0 tx=0).stop 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 shutdown_connections 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb44815cd40 0x7fb4480cf120 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb4480cb5d0 0x7fb44815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.934 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb43406ace0 0x7fb43406d190 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 --2- 172.21.15.7:0/1170202290 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb4480cb9a0 0x7fb4480cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:15:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 >> 172.21.15.7:0/1170202290 conn(0x7fb448070af0 msgr2=0x7fb44815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:15:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 shutdown_connections 2023-06-09T12:15:49.935 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:15:49.930+0000 7fb44d5ff700 1 -- 172.21.15.7:0/1170202290 wait complete. 2023-06-09T12:15:49.936 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:15:49.950 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2021,"stamp":"2023-06-09T12:15:49.195882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1933227,"num_omap_keys":3595,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104080,"ondisk_log_size":104080,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2608904,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":725568,"kb_avail":752365816,"statfs":{"total":773094113280,"available":770422595584,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":742981632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003377"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'969","reported_seq":4228,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:46.007552+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:15:46.007552+0000","last_peered":"2023-06-09T12:15:46.007552+0000","last_clean":"2023-06-09T12:15:46.007552+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:15:46.007552+0000","last_undegraded":"2023-06-09T12:15:46.007552+0000","last_fullsized":"2023-06-09T12:15:46.007552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3468,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:42.516760+0000","last_change":"2023-06-09T12:15:30.383425+0000","last_active":"2023-06-09T12:13:42.516760+0000","last_peered":"2023-06-09T12:13:42.516760+0000","last_clean":"2023-06-09T12:13:42.516760+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:13:42.516760+0000","last_undegraded":"2023-06-09T12:13:42.516760+0000","last_fullsized":"2023-06-09T12:13:42.516760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4093,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:22.513656+0000","last_change":"2023-06-09T12:14:01.380126+0000","last_active":"2023-06-09T12:13:22.513656+0000","last_peered":"2023-06-09T12:13:22.513656+0000","last_clean":"2023-06-09T12:13:22.513656+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:13:22.513656+0000","last_undegraded":"2023-06-09T12:13:22.513656+0000","last_fullsized":"2023-06-09T12:13:22.513656+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3447,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.402619+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:12:47.402619+0000","last_peered":"2023-06-09T12:12:47.402619+0000","last_clean":"2023-06-09T12:12:47.402619+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:12:47.402619+0000","last_undegraded":"2023-06-09T12:12:47.402619+0000","last_fullsized":"2023-06-09T12:12:47.402619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":368,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:17.390057+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:11:17.390057+0000","last_peered":"2023-06-09T12:11:17.390057+0000","last_clean":"2023-06-09T12:11:17.390057+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:11:17.390057+0000","last_undegraded":"2023-06-09T12:11:17.390057+0000","last_fullsized":"2023-06-09T12:11:17.390057+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'824","reported_seq":1049,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:37.802480+0000","last_change":"2023-06-09T12:15:37.429869+0000","last_active":"2023-06-09T12:15:37.802480+0000","last_peered":"2023-06-09T12:15:37.802480+0000","last_clean":"2023-06-09T12:15:37.802480+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:37.802480+0000","last_undegraded":"2023-06-09T12:15:37.802480+0000","last_fullsized":"2023-06-09T12:15:37.802480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":824,"log_dups_size":0,"ondisk_log_size":824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3903,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097424+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097424+0000","last_peered":"2023-06-09T12:13:57.097424+0000","last_clean":"2023-06-09T12:13:57.097424+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097424+0000","last_undegraded":"2023-06-09T12:13:57.097424+0000","last_fullsized":"2023-06-09T12:13:57.097424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3788,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.565036+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:11:42.565036+0000","last_peered":"2023-06-09T12:11:42.565036+0000","last_clean":"2023-06-09T12:11:42.565036+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:11:42.565036+0000","last_undegraded":"2023-06-09T12:11:42.565036+0000","last_fullsized":"2023-06-09T12:11:42.565036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":730,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:37.401016+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:11:37.401016+0000","last_peered":"2023-06-09T12:11:37.401016+0000","last_clean":"2023-06-09T12:11:37.401016+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:11:37.401016+0000","last_undegraded":"2023-06-09T12:11:37.401016+0000","last_fullsized":"2023-06-09T12:11:37.401016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3850,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:52.518166+0000","last_change":"2023-06-09T12:15:34.446550+0000","last_active":"2023-06-09T12:13:52.518166+0000","last_peered":"2023-06-09T12:13:52.518166+0000","last_clean":"2023-06-09T12:13:52.518166+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:13:52.518166+0000","last_undegraded":"2023-06-09T12:13:52.518166+0000","last_fullsized":"2023-06-09T12:13:52.518166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3538,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:35.237703+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:11:35.237703+0000","last_peered":"2023-06-09T12:11:35.237703+0000","last_clean":"2023-06-09T12:11:35.237703+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:11:35.237703+0000","last_undegraded":"2023-06-09T12:11:35.237703+0000","last_fullsized":"2023-06-09T12:11:35.237703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1292,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377026+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:11:52.377026+0000","last_peered":"2023-06-09T12:11:52.377026+0000","last_clean":"2023-06-09T12:11:52.377026+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:11:52.377026+0000","last_undegraded":"2023-06-09T12:11:52.377026+0000","last_fullsized":"2023-06-09T12:11:52.377026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":322,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.477420+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:11:57.477420+0000","last_peered":"2023-06-09T12:11:57.477420+0000","last_clean":"2023-06-09T12:11:57.477420+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:11:57.477420+0000","last_undegraded":"2023-06-09T12:11:57.477420+0000","last_fullsized":"2023-06-09T12:11:57.477420+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3593,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:57.566627+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:11:57.566627+0000","last_peered":"2023-06-09T12:11:57.566627+0000","last_clean":"2023-06-09T12:11:57.566627+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:11:57.566627+0000","last_undegraded":"2023-06-09T12:11:57.566627+0000","last_fullsized":"2023-06-09T12:11:57.566627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":853,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:42.375819+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:11:42.375819+0000","last_peered":"2023-06-09T12:11:42.375819+0000","last_clean":"2023-06-09T12:11:42.375819+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:11:42.375819+0000","last_undegraded":"2023-06-09T12:11:42.375819+0000","last_fullsized":"2023-06-09T12:11:42.375819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":530,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:13:42.401621+0000","last_change":"2023-06-09T12:14:59.442204+0000","last_active":"2023-06-09T12:13:42.401621+0000","last_peered":"2023-06-09T12:13:42.401621+0000","last_clean":"2023-06-09T12:13:42.401621+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:13:42.401621+0000","last_undegraded":"2023-06-09T12:13:42.401621+0000","last_fullsized":"2023-06-09T12:13:42.401621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.396890+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:13:07.396890+0000","last_peered":"2023-06-09T12:13:07.396890+0000","last_clean":"2023-06-09T12:13:07.396890+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:13:07.396890+0000","last_undegraded":"2023-06-09T12:13:07.396890+0000","last_fullsized":"2023-06-09T12:13:07.396890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.623947+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.623947+0000","last_peered":"2023-06-09T12:14:50.623947+0000","last_clean":"2023-06-09T12:14:50.623947+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.623947+0000","last_undegraded":"2023-06-09T12:14:50.623947+0000","last_fullsized":"2023-06-09T12:14:50.623947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":801,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:00.616288+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:11:00.616288+0000","last_peered":"2023-06-09T12:11:00.616288+0000","last_clean":"2023-06-09T12:11:00.616288+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:11:00.616288+0000","last_undegraded":"2023-06-09T12:11:00.616288+0000","last_fullsized":"2023-06-09T12:11:00.616288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:07.397228+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:11:07.397228+0000","last_peered":"2023-06-09T12:11:07.397228+0000","last_clean":"2023-06-09T12:11:07.397228+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:11:07.397228+0000","last_undegraded":"2023-06-09T12:11:07.397228+0000","last_fullsized":"2023-06-09T12:11:07.397228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3935,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:02.494855+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:11:02.494855+0000","last_peered":"2023-06-09T12:11:02.494855+0000","last_clean":"2023-06-09T12:11:02.494855+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:11:02.494855+0000","last_undegraded":"2023-06-09T12:11:02.494855+0000","last_fullsized":"2023-06-09T12:11:02.494855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3947,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.318635+0000","last_change":"2023-06-09T12:15:27.414934+0000","last_active":"2023-06-09T12:13:37.318635+0000","last_peered":"2023-06-09T12:13:37.318635+0000","last_clean":"2023-06-09T12:13:37.318635+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:37.318635+0000","last_undegraded":"2023-06-09T12:13:37.318635+0000","last_fullsized":"2023-06-09T12:13:37.318635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3254,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286291+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286291+0000","last_peered":"2023-06-09T12:14:11.286291+0000","last_clean":"2023-06-09T12:14:11.286291+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286291+0000","last_undegraded":"2023-06-09T12:14:11.286291+0000","last_fullsized":"2023-06-09T12:14:11.286291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73015,"ondisk_log_size":73015,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1933227,"num_omap_keys":3595,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30946,"ondisk_log_size":30946,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575632,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73399999999999999,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":1.143,"15min":1.256},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.65100000000000002,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3089999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72399999999999998,"15min":0.70299999999999996},"min":{"1min":0.61599999999999999,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.505},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":1.115,"15min":1.226},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.76100000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.65700000000000003},"min":{"1min":0.51300000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.97099999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.86199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575632,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.68799999999999994,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.96599999999999997,"5min":1.044,"15min":1.044},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72099999999999997,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.47299999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.54600000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0960000000000001,"5min":1.266,"15min":1.266},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77200000000000002,"15min":0.76400000000000001},"min":{"1min":0.59199999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.139,"5min":1.298,"15min":1.298},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.01,"5min":1.091,"15min":1.091},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76900000000000002,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45100000000000001},"max":{"1min":0.99299999999999999,"5min":1.046,"15min":1.046},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.623,"5min":0.59199999999999997,"15min":0.55900000000000005},"max":{"1min":1.077,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58099999999999996,"15min":0.58899999999999997},"min":{"1min":0.40699999999999997,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.60699999999999998,"15min":0.59599999999999997},"min":{"1min":0.46500000000000002,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.443}]},{"osd":5,"last update":"Fri Jun 9 12:15:00 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.63100000000000001},"min":{"1min":0.47699999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.873,"5min":0.98399999999999999,"15min":1.0700000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77600000000000002,"15min":0.73299999999999998},"min":{"1min":0.59599999999999997,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.68999999999999995},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.435},"max":{"1min":0.89000000000000001,"5min":1.0429999999999999,"15min":1.1100000000000001},"last":0.749},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.69599999999999995},"min":{"1min":0.53300000000000003,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575633,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405232,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":93966608,"statfs":{"total":96636764160,"available":96221806592,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44900000000000001,"15min":0.45100000000000001},"min":{"1min":0.32300000000000001,"5min":0.28999999999999998,"15min":0.25800000000000001},"max":{"1min":0.55600000000000005,"5min":0.72699999999999998,"15min":1.4470000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55000000000000004,"15min":0.57199999999999995},"min":{"1min":0.40899999999999997,"5min":0.40600000000000003,"15min":0.311},"max":{"1min":0.65200000000000002,"5min":0.78800000000000003,"15min":0.85099999999999998},"last":0.58099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48699999999999999,"15min":0.497},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.432},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.55000000000000004,"15min":0.55100000000000005},"min":{"1min":0.432,"5min":0.376,"15min":0.35699999999999998},"max":{"1min":0.69699999999999995,"5min":0.81899999999999995,"15min":0.95299999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.61299999999999999,"15min":0.59799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45000000000000001,"15min":0.39600000000000002},"max":{"1min":0.72299999999999998,"5min":0.97899999999999998,"15min":1.2450000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.54900000000000004,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":0.77700000000000002,"5min":1.288,"15min":1.288},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.873,"15min":0.80400000000000005},"min":{"1min":0.74199999999999999,"5min":0.53800000000000003,"15min":0.45700000000000002},"max":{"1min":1.0840000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81299999999999994,"15min":0.84499999999999997},"min":{"1min":0.58299999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.012,"5min":1.175,"15min":1.597},"last":0.79100000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80700000000000005,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.091,"5min":1.196,"15min":1.544},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.59499999999999997},"max":{"1min":1.044,"5min":1.1539999999999999,"15min":1.704},"last":0.68999999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.84399999999999997,"15min":0.84099999999999997},"min":{"1min":0.70099999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0409999999999999,"5min":1.1910000000000001,"15min":1.548},"last":0.748},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.83999999999999997,"15min":0.82899999999999996},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.627}]},{"osd":7,"last update":"Fri Jun 9 12:14:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.89700000000000002,"15min":0.90000000000000002},"min":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.58399999999999996},"max":{"1min":1.131,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.86799999999999999,"15min":0.84499999999999997},"min":{"1min":0.73599999999999999,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.095,"5min":1.24,"15min":1.498},"last":0.82899999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331176,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103360,"kb_avail":94040664,"statfs":{"total":96636764160,"available":96297639936,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":105840640},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.745},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41099999999999998},"max":{"1min":0.73399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54900000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.032},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.82099999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.119,"5min":1.421,"15min":1.421},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"max":{"1min":1.075,"5min":1.2490000000000001,"15min":1.268},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.94099999999999995,"5min":1.077,"15min":1.121},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1259999999999999,"5min":1.371,"15min":3.5840000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.83999999999999997},"min":{"1min":0.53900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.21,"15min":1.232},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68400000000000005,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1299999999999999,"15min":1.298},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.95799999999999996,"15min":0.95199999999999996},"min":{"1min":0.78100000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.179,"5min":1.371,"15min":1.371},"last":0.89100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575633,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46999999999999997,"15min":0.46500000000000002},"min":{"1min":0.33600000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56799999999999995,"5min":0.67900000000000005,"15min":0.94399999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56699999999999995,"15min":0.55800000000000005},"min":{"1min":0.5,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.999},"last":0.503}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48599999999999999,"15min":0.47999999999999998},"min":{"1min":0.36199999999999999,"5min":0.32600000000000001,"15min":0.318},"max":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":1.236},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.59099999999999997,"15min":0.624},"min":{"1min":0.52300000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.752,"5min":0.84899999999999998,"15min":1.2969999999999999},"last":0.58099999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63,"15min":0.61499999999999999},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80500000000000005,"5min":1.2709999999999999,"15min":1.4490000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.47499999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":1.298,"5min":1.319,"15min":1.4650000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.80800000000000005,"15min":0.79200000000000004},"min":{"1min":0.64300000000000002,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.79900000000000004,"15min":0.79300000000000004},"min":{"1min":0.67900000000000005,"5min":0.443,"15min":0.443},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.751}]},{"osd":5,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.83399999999999996,"15min":0.83099999999999996},"min":{"1min":0.70199999999999996,"5min":0.56899999999999995,"15min":0.52800000000000002},"max":{"1min":1.337,"5min":1.3740000000000001,"15min":1.4359999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88300000000000001,"15min":0.88100000000000001},"min":{"1min":0.745,"5min":0.65600000000000003,"15min":0.60899999999999999},"max":{"1min":1.3640000000000001,"5min":1.417,"15min":1.5189999999999999},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.58099999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.3029999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.85799999999999998,"15min":0.81299999999999994},"min":{"1min":0.63,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.86699999999999999,"15min":0.84599999999999997},"min":{"1min":0.61099999999999999,"5min":0.56399999999999995,"15min":0.53600000000000003},"max":{"1min":1.286,"5min":1.286,"15min":1.286},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.88500000000000001,"15min":0.86499999999999999},"min":{"1min":0.73299999999999998,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.83899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575632,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442364,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":119168,"kb_avail":93929476,"statfs":{"total":96636764160,"available":96183783424,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":122028032},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.48399999999999999},"min":{"1min":0.32800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.72599999999999998,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.65000000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.79900000000000004,"5min":0.872,"15min":0.97099999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68100000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.55600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.0169999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":0.95199999999999996,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84099999999999997},"min":{"1min":0.71199999999999997,"5min":0.61199999999999999,"15min":0.52800000000000002},"max":{"1min":1.008,"5min":1.409,"15min":1.409},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.89000000000000001},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.96599999999999997,"5min":1.452,"15min":1.452},"last":0.93200000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0289999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.748}]},{"osd":7,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.371,"15min":1.371},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.95099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417876,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125120,"kb_avail":93953964,"statfs":{"total":96636764160,"available":96208859136,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128122880},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.748},"min":{"1min":0.48199999999999998,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.93999999999999995,"5min":1.266,"15min":1.419},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.96299999999999997,"5min":1.294,"15min":1.383},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.876,"5min":1.242,"15min":1.242},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81299999999999994,"15min":0.79000000000000004},"min":{"1min":0.629,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.754}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97799999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.749},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.441},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.77600000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73799999999999999,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":0.90400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.622}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.497},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76600000000000001,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.131},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70199999999999996,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.499,"15min":0.38300000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.68000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575631,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.70199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.95899999999999996,"5min":1.0329999999999999,"15min":3.468},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.71499999999999997,"15min":0.71699999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.40999999999999998},"max":{"1min":0.90000000000000002,"5min":1.579,"15min":1.579},"last":0.81799999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.71399999999999997},"min":{"1min":0.45900000000000002,"5min":0.44,"15min":0.377},"max":{"1min":0.78800000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72599999999999998,"15min":0.73999999999999999},"min":{"1min":0.52000000000000002,"5min":0.498,"15min":0.439},"max":{"1min":0.93799999999999994,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73399999999999999,"15min":0.71999999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.94899999999999995,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.60499999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.98299999999999998,"5min":1.391,"15min":1.391},"last":0.85799999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.76900000000000002,"15min":0.748},"min":{"1min":0.55300000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0589999999999999,"5min":1.169,"15min":1.254},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.81499999999999995,"15min":0.80500000000000005},"min":{"1min":0.68500000000000005,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.55200000000000005,"15min":0.55100000000000005},"min":{"1min":0.42199999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72199999999999998,"5min":1.522,"15min":1.522},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.439,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70699999999999996,"5min":1.28,"15min":1.28},"last":0.59899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67800000000000005,"15min":0.63800000000000001},"min":{"1min":0.51600000000000001,"5min":0.43099999999999999,"15min":0.40200000000000002},"max":{"1min":0.999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.71699999999999997,"15min":0.68300000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.41599999999999998},"max":{"1min":0.86599999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:14:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.78100000000000003,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.75,"15min":0.75},"min":{"1min":0.53000000000000003,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92600000000000005,"5min":1.226,"15min":1.238},"last":0.747}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:15:49.958 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.959 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.959 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.960 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.961 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.961 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.962 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:15:49.962 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:16:00.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:00.145 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:09.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:16:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.170+0000 7f58051ff700 1 Processor -- start 2023-06-09T12:16:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.170+0000 7f58051ff700 1 -- start start 2023-06-09T12:16:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f58051ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5800153a70 0x7f58000c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f58051ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000c2d90 0x7f580015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f58051ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58000c88d0 con 0x7f58000ca7e0 2023-06-09T12:16:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f58051ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000c2bf0 con 0x7f5800153a70 2023-06-09T12:16:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f58051ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5800153e40 con 0x7f58000c2d90 2023-06-09T12:16:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f57fed9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5800153a70 0x7f58000c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f57fe59c700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000c2d90 0x7f580015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f57fed9d700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f5800153a70 0x7f58000c26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.7:50680/0 (socket says 172.21.15.7:50680) 2023-06-09T12:16:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.172+0000 7f57fed9d700 1 -- 172.21.15.7:0/4039966647 learned_addr learned my addr 172.21.15.7:0/4039966647 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:16:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fed9d700 1 -- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000c2d90 msgr2=0x7f580015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:16:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fed9d700 1 --2- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000c2d90 0x7f580015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fed9d700 1 -- 172.21.15.7:0/4039966647 >> v1:172.21.15.117:6789/0 conn(0x7f58000ca7e0 legacy=0x7f58000cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fed9d700 1 -- 172.21.15.7:0/4039966647 --> v2:172.21.15.117:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580015a630 con 0x7f5800153a70 2023-06-09T12:16:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fed9d700 1 --2- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800153a70 0x7f58000c26b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f57f00098f0 tx=0x7f57f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a945ab5b94ece88 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fdd9b700 1 -- 172.21.15.7:0/4039966647 <== mon.1 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0011830 con 0x7f5800153a70 2023-06-09T12:16:10.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fdd9b700 1 -- 172.21.15.7:0/4039966647 <== mon.1 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0011990 con 0x7f5800153a70 2023-06-09T12:16:10.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.173+0000 7f57fdd9b700 1 -- 172.21.15.7:0/4039966647 <== mon.1 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0011d20 con 0x7f5800153a70 2023-06-09T12:16:10.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 -- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800153a70 msgr2=0x7f58000c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 --2- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800153a70 0x7f58000c26b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f57f00098f0 tx=0x7f57f0009c00 comp rx=0 tx=0).stop 2023-06-09T12:16:10.180 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 -- 172.21.15.7:0/4039966647 shutdown_connections 2023-06-09T12:16:10.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 --2- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5800153a70 0x7f58000c26b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 --2- 172.21.15.7:0/4039966647 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f58000c2d90 0x7f580015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 -- 172.21.15.7:0/4039966647 >> 172.21.15.7:0/4039966647 conn(0x7f580006e410 msgr2=0x7f580006e810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:10.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 -- 172.21.15.7:0/4039966647 shutdown_connections 2023-06-09T12:16:10.181 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.174+0000 7f58051ff700 1 -- 172.21.15.7:0/4039966647 wait complete. 2023-06-09T12:16:10.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.176+0000 7f58051ff700 1 Processor -- start 2023-06-09T12:16:10.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.176+0000 7f58051ff700 1 -- start start 2023-06-09T12:16:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58000ca7e0 0x7f58000c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153a70 0x7f58001eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580014fc30 con 0x7f58000c2d90 2023-06-09T12:16:10.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580006a640 con 0x7f58000ca7e0 2023-06-09T12:16:10.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f58051ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58000cd980 con 0x7f5800153a70 2023-06-09T12:16:10.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40438/0 (socket says 172.21.15.7:40438) 2023-06-09T12:16:10.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 -- 172.21.15.7:0/1400083164 learned_addr learned my addr 172.21.15.7:0/1400083164 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:16:10.185 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fe59c700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58000ca7e0 0x7f58000c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57ff59e700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153a70 0x7f58001eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 -- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153a70 msgr2=0x7f58001eeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153a70 0x7f58001eeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 -- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58000ca7e0 msgr2=0x7f58000c73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58000ca7e0 0x7f58000c73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.186 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.177+0000 7f57fed9d700 1 -- 172.21.15.7:0/1400083164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009550 con 0x7f58000c2d90 2023-06-09T12:16:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f57fed9d700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c40 tx=0x7f57f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0004590 con 0x7f58000c2d90 2023-06-09T12:16:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f001e070 con 0x7f58000c2d90 2023-06-09T12:16:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58000c8350 con 0x7f58000c2d90 2023-06-09T12:16:10.187 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00038e0 con 0x7f58000c2d90 2023-06-09T12:16:10.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.178+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58000ca350 con 0x7f58000c2d90 2023-06-09T12:16:10.188 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.179+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58000c85f0 con 0x7f58000c2d90 2023-06-09T12:16:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.180+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57f0003a40 con 0x7f58000c2d90 2023-06-09T12:16:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.182+0000 7f57e77fe700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 0x7f57ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.182+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f57f008aaa0 con 0x7f58000c2d90 2023-06-09T12:16:10.189 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.182+0000 7f57fe59c700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 0x7f57ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.182+0000 7f57fe59c700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 0x7f57ec06d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f58000c9cf0 tx=0x7f57f4009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:10.190 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.186+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f57f00551d0 con 0x7f58000c2d90 2023-06-09T12:16:10.398 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.395+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5800159d70 con 0x7f57ec06ace0 2023-06-09T12:16:10.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.404+0000 7f57e77fe700 1 -- 172.21.15.7:0/1400083164 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319426 (secure 0 0 0) 0x7f5800159d70 con 0x7f57ec06ace0 2023-06-09T12:16:10.411 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:16:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 msgr2=0x7f57ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 0x7f57ec06d190 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f58000c9cf0 tx=0x7f57f4009210 comp rx=0 tx=0).stop 2023-06-09T12:16:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 msgr2=0x7f58001e3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009c40 tx=0x7f57f0004750 comp rx=0 tx=0).stop 2023-06-09T12:16:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 shutdown_connections 2023-06-09T12:16:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f57ec06ace0 0x7f57ec06d190 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5800153a70 0x7f58001eeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58000c2d90 0x7f58001e3020 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 --2- 172.21.15.7:0/1400083164 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f58000ca7e0 0x7f58000c73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.408+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 >> 172.21.15.7:0/1400083164 conn(0x7f580006e410 msgr2=0x7f58000c5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.409+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 shutdown_connections 2023-06-09T12:16:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:10.409+0000 7f58051ff700 1 -- 172.21.15.7:0/1400083164 wait complete. 2023-06-09T12:16:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:16:10.430 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2031,"stamp":"2023-06-09T12:16:09.201176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2032363,"num_omap_keys":3760,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104199,"ondisk_log_size":104199,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2609224,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":725888,"kb_avail":752365496,"statfs":{"total":773094113280,"available":770422267904,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":743309312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003113"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:22.473404+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:11:22.473404+0000","last_peered":"2023-06-09T12:11:22.473404+0000","last_clean":"2023-06-09T12:11:22.473404+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:11:22.473404+0000","last_undegraded":"2023-06-09T12:11:22.473404+0000","last_fullsized":"2023-06-09T12:11:22.473404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1065","reported_seq":4329,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:16:05.381525+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:16:05.381525+0000","last_peered":"2023-06-09T12:16:05.381525+0000","last_clean":"2023-06-09T12:16:05.381525+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:05.381525+0000","last_undegraded":"2023-06-09T12:16:05.381525+0000","last_fullsized":"2023-06-09T12:16:05.381525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":96,"log_size":1065,"log_dups_size":0,"ondisk_log_size":1065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 19s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3469,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:30.383425+0000","last_change":"2023-06-09T12:16:07.536779+0000","last_active":"2023-06-09T12:15:30.383425+0000","last_peered":"2023-06-09T12:15:30.383425+0000","last_clean":"2023-06-09T12:15:30.383425+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:15:30.383425+0000","last_undegraded":"2023-06-09T12:15:30.383425+0000","last_fullsized":"2023-06-09T12:15:30.383425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3650,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:18.820760+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:12:18.820760+0000","last_peered":"2023-06-09T12:12:18.820760+0000","last_clean":"2023-06-09T12:12:18.820760+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:12:18.820760+0000","last_undegraded":"2023-06-09T12:12:18.820760+0000","last_fullsized":"2023-06-09T12:12:18.820760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3285,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.377160+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:11:52.377160+0000","last_peered":"2023-06-09T12:11:52.377160+0000","last_clean":"2023-06-09T12:11:52.377160+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:11:52.377160+0000","last_undegraded":"2023-06-09T12:11:52.377160+0000","last_fullsized":"2023-06-09T12:11:52.377160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4098,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:16:01.984871+0000","last_change":"2023-06-09T12:16:01.612368+0000","last_active":"2023-06-09T12:16:01.984871+0000","last_peered":"2023-06-09T12:16:01.984871+0000","last_clean":"2023-06-09T12:16:01.984871+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:01.984871+0000","last_undegraded":"2023-06-09T12:16:01.984871+0000","last_fullsized":"2023-06-09T12:16:01.984871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 5s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3447,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.402619+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:12:47.402619+0000","last_peered":"2023-06-09T12:12:47.402619+0000","last_clean":"2023-06-09T12:12:47.402619+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:12:47.402619+0000","last_undegraded":"2023-06-09T12:12:47.402619+0000","last_fullsized":"2023-06-09T12:12:47.402619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":597,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:32.392082+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:11:32.392082+0000","last_peered":"2023-06-09T12:11:32.392082+0000","last_clean":"2023-06-09T12:11:32.392082+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:11:32.392082+0000","last_undegraded":"2023-06-09T12:11:32.392082+0000","last_fullsized":"2023-06-09T12:11:32.392082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1174,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.476237+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:11:47.476237+0000","last_peered":"2023-06-09T12:11:47.476237+0000","last_clean":"2023-06-09T12:11:47.476237+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:11:47.476237+0000","last_undegraded":"2023-06-09T12:11:47.476237+0000","last_fullsized":"2023-06-09T12:11:47.476237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3258,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.404463+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:12:07.404463+0000","last_peered":"2023-06-09T12:12:07.404463+0000","last_clean":"2023-06-09T12:12:07.404463+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:12:07.404463+0000","last_undegraded":"2023-06-09T12:12:07.404463+0000","last_fullsized":"2023-06-09T12:12:07.404463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":636,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.476927+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:11:52.476927+0000","last_peered":"2023-06-09T12:11:52.476927+0000","last_clean":"2023-06-09T12:11:52.476927+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:11:52.476927+0000","last_undegraded":"2023-06-09T12:11:52.476927+0000","last_fullsized":"2023-06-09T12:11:52.476927+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:34.446550+0000","last_change":"2023-06-09T12:16:07.536754+0000","last_active":"2023-06-09T12:15:34.446550+0000","last_peered":"2023-06-09T12:15:34.446550+0000","last_clean":"2023-06-09T12:15:34.446550+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:15:34.446550+0000","last_undegraded":"2023-06-09T12:15:34.446550+0000","last_fullsized":"2023-06-09T12:15:34.446550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1665,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:07.503793+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:12:07.503793+0000","last_peered":"2023-06-09T12:12:07.503793+0000","last_clean":"2023-06-09T12:12:07.503793+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:12:07.503793+0000","last_undegraded":"2023-06-09T12:12:07.503793+0000","last_fullsized":"2023-06-09T12:12:07.503793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3459,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.501236+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:11:47.501236+0000","last_peered":"2023-06-09T12:11:47.501236+0000","last_clean":"2023-06-09T12:11:47.501236+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:11:47.501236+0000","last_undegraded":"2023-06-09T12:11:47.501236+0000","last_fullsized":"2023-06-09T12:11:47.501236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:21.379303+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:12:21.379303+0000","last_peered":"2023-06-09T12:12:21.379303+0000","last_clean":"2023-06-09T12:12:21.379303+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:12:21.379303+0000","last_undegraded":"2023-06-09T12:12:21.379303+0000","last_fullsized":"2023-06-09T12:12:21.379303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4407,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:03.027691+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:12:03.027691+0000","last_peered":"2023-06-09T12:12:03.027691+0000","last_clean":"2023-06-09T12:12:03.027691+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:12:03.027691+0000","last_undegraded":"2023-06-09T12:12:03.027691+0000","last_fullsized":"2023-06-09T12:12:03.027691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:59.442204+0000","last_change":"2023-06-09T12:15:52.418697+0000","last_active":"2023-06-09T12:14:59.442204+0000","last_peered":"2023-06-09T12:14:59.442204+0000","last_clean":"2023-06-09T12:14:59.442204+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:14:59.442204+0000","last_undegraded":"2023-06-09T12:14:59.442204+0000","last_fullsized":"2023-06-09T12:14:59.442204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.396890+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:13:07.396890+0000","last_peered":"2023-06-09T12:13:07.396890+0000","last_clean":"2023-06-09T12:13:07.396890+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:13:07.396890+0000","last_undegraded":"2023-06-09T12:13:07.396890+0000","last_fullsized":"2023-06-09T12:13:07.396890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":181,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.565955+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:11:52.565955+0000","last_peered":"2023-06-09T12:11:52.565955+0000","last_clean":"2023-06-09T12:11:52.565955+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:11:52.565955+0000","last_undegraded":"2023-06-09T12:11:52.565955+0000","last_fullsized":"2023-06-09T12:11:52.565955+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":865,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714000+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714000+0000","last_peered":"2023-06-09T12:14:03.714000+0000","last_clean":"2023-06-09T12:14:03.714000+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714000+0000","last_undegraded":"2023-06-09T12:14:03.714000+0000","last_fullsized":"2023-06-09T12:14:03.714000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.623947+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.623947+0000","last_peered":"2023-06-09T12:14:50.623947+0000","last_clean":"2023-06-09T12:14:50.623947+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.623947+0000","last_undegraded":"2023-06-09T12:14:50.623947+0000","last_fullsized":"2023-06-09T12:14:50.623947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":917,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:12.389313+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:11:12.389313+0000","last_peered":"2023-06-09T12:11:12.389313+0000","last_clean":"2023-06-09T12:11:12.389313+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:11:12.389313+0000","last_undegraded":"2023-06-09T12:11:12.389313+0000","last_fullsized":"2023-06-09T12:11:12.389313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3831,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:27.498187+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:11:27.498187+0000","last_peered":"2023-06-09T12:11:27.498187+0000","last_clean":"2023-06-09T12:11:27.498187+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:11:27.498187+0000","last_undegraded":"2023-06-09T12:11:27.498187+0000","last_fullsized":"2023-06-09T12:11:27.498187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1068,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.391150+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:12:22.391150+0000","last_peered":"2023-06-09T12:12:22.391150+0000","last_clean":"2023-06-09T12:12:22.391150+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:12:22.391150+0000","last_undegraded":"2023-06-09T12:12:22.391150+0000","last_fullsized":"2023-06-09T12:12:22.391150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3947,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.318635+0000","last_change":"2023-06-09T12:15:27.414934+0000","last_active":"2023-06-09T12:13:37.318635+0000","last_peered":"2023-06-09T12:13:37.318635+0000","last_clean":"2023-06-09T12:13:37.318635+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:37.318635+0000","last_undegraded":"2023-06-09T12:13:37.318635+0000","last_fullsized":"2023-06-09T12:13:37.318635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73111,"ondisk_log_size":73111,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2032363,"num_omap_keys":3760,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30969,"ondisk_log_size":30969,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575636,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.51800000000000002,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.73399999999999999,"15min":0.71899999999999997},"min":{"1min":0.56699999999999995,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.94599999999999995,"5min":1.143,"15min":1.256},"last":0.72499999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.47499999999999998,"5min":0.441,"15min":0.41999999999999998},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.65100000000000002,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.72599999999999998,"15min":0.72399999999999998},"min":{"1min":0.56799999999999995,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76600000000000001,"15min":0.749},"min":{"1min":0.66700000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.3089999999999999},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.72399999999999998,"15min":0.70299999999999996},"min":{"1min":0.61599999999999999,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.505},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.754}]},{"osd":4,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55100000000000005,"15min":0.54600000000000004},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.32900000000000001},"max":{"1min":0.871,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.55900000000000005,"5min":0.55300000000000005,"15min":0.47799999999999998},"max":{"1min":0.91500000000000004,"5min":1.115,"15min":1.226},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.61099999999999999,"15min":0.60899999999999999},"min":{"1min":0.45400000000000001,"5min":0.40100000000000002,"15min":0.378},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63500000000000001,"15min":0.624},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.52000000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:15:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67200000000000004,"15min":0.65700000000000003},"min":{"1min":0.51300000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.97099999999999997,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.622},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75900000000000001,"15min":0.745},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.501},"max":{"1min":1.0109999999999999,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575636,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68200000000000005,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":1.044,"15min":1.044},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66300000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":1.266,"15min":1.266},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":1.298,"15min":1.298},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.626,"5min":0.501,"15min":0.501},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.091},"last":0.79900000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.747,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.83199999999999996,"5min":1.0209999999999999,"15min":1.046},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81000000000000005,"15min":0.80600000000000005},"min":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.55900000000000005},"max":{"1min":0.88900000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.82999999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.66400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.86099999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.161,"15min":1.161},"last":0.628}]},{"osd":7,"last update":"Fri Jun 9 12:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.435},"max":{"1min":0.878,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.59599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575637,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405232,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":93966608,"statfs":{"total":96636764160,"available":96221806592,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.68600000000000005,"5min":0.747,"15min":0.95299999999999996},"last":0.498}]},{"osd":3,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.39600000000000002},"max":{"1min":0.73299999999999998,"5min":0.77900000000000003,"15min":1.2450000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81499999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.81699999999999995},"min":{"1min":0.76400000000000001,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.625,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0529999999999999,"5min":1.175,"15min":1.597},"last":0.79700000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.80700000000000005,"15min":0.82799999999999996},"min":{"1min":0.64700000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.091,"5min":1.196,"15min":1.544},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.85199999999999998,"15min":0.86399999999999999},"min":{"1min":0.72299999999999998,"5min":0.68400000000000005,"15min":0.59499999999999997},"max":{"1min":1.044,"5min":1.1539999999999999,"15min":1.704},"last":0.77500000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.1910000000000001,"15min":1.548},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83599999999999997,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":0.97699999999999998,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.73599999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.159,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.139,"5min":1.24,"15min":1.498},"last":0.56399999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575636,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331304,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103488,"kb_avail":94040536,"statfs":{"total":96636764160,"available":96297508864,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":105971712},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.434},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41099999999999998},"max":{"1min":0.73399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.032},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.50800000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.72799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.119,"5min":1.421,"15min":1.421},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"max":{"1min":1.075,"5min":1.2490000000000001,"15min":1.268},"last":0.86099999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.94099999999999995,"5min":1.077,"15min":1.121},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1259999999999999,"5min":1.371,"15min":3.5840000000000001},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.83999999999999997},"min":{"1min":0.53900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.21,"15min":1.232},"last":0.93700000000000006}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68400000000000005,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1299999999999999,"15min":1.298},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.95799999999999996,"15min":0.95199999999999996},"min":{"1min":0.78100000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.179,"5min":1.371,"15min":1.371},"last":1.026}]}]},{"osd":2,"up_from":13,"seq":55834575637,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45300000000000001,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.51000000000000001,"5min":0.65500000000000003,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.59599999999999997,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.52000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.54200000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73899999999999999,"5min":0.81799999999999995,"15min":1.2969999999999999},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.4490000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.67600000000000005},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.75800000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":1.125,"5min":1.218,"15min":1.218},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.048,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69299999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"max":{"1min":1.0920000000000001,"5min":1.3520000000000001,"15min":1.4359999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.88},"min":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.3899999999999999,"15min":1.5189999999999999},"last":0.86099999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.64100000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.747,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2350000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":1.286,"15min":1.286},"last":0.751},{"interface":"front","average":{"1min":0.877,"5min":0.88200000000000001,"15min":0.86399999999999999},"min":{"1min":0.69099999999999995,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.1870000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.82899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575636,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442492,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":119296,"kb_avail":93929348,"statfs":{"total":96636764160,"available":96183652352,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":122159104},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.64000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57999999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.79900000000000004,"5min":0.872,"15min":0.97099999999999997},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68100000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.625}]},{"osd":4,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.0169999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":0.95199999999999996,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.91600000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84099999999999997},"min":{"1min":0.71199999999999997,"5min":0.61199999999999999,"15min":0.52800000000000002},"max":{"1min":1.008,"5min":1.409,"15min":1.409},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.89000000000000001},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.96599999999999997,"5min":1.452,"15min":1.452},"last":1.056}]},{"osd":6,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0289999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":1.034},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.371,"15min":1.371},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":1.0149999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575635,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417940,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125184,"kb_avail":93953900,"statfs":{"total":96636764160,"available":96208793600,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128188416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.748},"min":{"1min":0.48199999999999998,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.93999999999999995,"5min":1.266,"15min":1.419},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.96299999999999997,"5min":1.294,"15min":1.383},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.876,"5min":1.242,"15min":1.242},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81299999999999994,"15min":0.79000000000000004},"min":{"1min":0.629,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.95999999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97799999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.441},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73799999999999999,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":0.90400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.85399999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.503},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76600000000000001,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.131},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70199999999999996,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.499,"15min":0.38300000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.56200000000000006}]}]},{"osd":5,"up_from":13,"seq":55834575635,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92100000000000004,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.579,"15min":1.579},"last":0.753}]},{"osd":1,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.41199999999999998},"max":{"1min":0.97199999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.72799999999999998,"15min":0.73999999999999999},"min":{"1min":0.57899999999999996,"5min":0.498,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.77100000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.97799999999999998,"5min":1.391,"15min":1.391},"last":0.82199999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.749},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93400000000000005,"5min":1.169,"15min":1.254},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96299999999999997,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79400000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":1.522,"15min":1.522},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65800000000000003,"5min":1.28,"15min":1.28},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68000000000000005,"15min":0.63700000000000001},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80000000000000004,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.85899999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.877},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88800000000000001,"5min":1.226,"15min":1.238},"last":0.89700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:16:10.438 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.438 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.439 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.439 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.440 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.441 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:10.441 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:16:30.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:30.263 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:16:30.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:16:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.646+0000 7fdb9bfff700 1 Processor -- start 2023-06-09T12:16:30.649 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.646+0000 7fdb9bfff700 1 -- start start 2023-06-09T12:16:30.650 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb940cb5d0 0x7fdb940b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9bfff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 0x7fdb940cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9bfff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb94159b20 con 0x7fdb940cb9a0 2023-06-09T12:16:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9bfff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb940cac80 con 0x7fdb940cb5d0 2023-06-09T12:16:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9bfff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb941597d0 con 0x7fdb9415cd40 2023-06-09T12:16:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb940cb5d0 0x7fdb940b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fdb940cb5d0 0x7fdb940b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54858/0 (socket says 172.21.15.7:54858) 2023-06-09T12:16:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 -- 172.21.15.7:0/3128386091 learned_addr learned my addr 172.21.15.7:0/3128386091 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:16:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9959a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fdb940cb9a0 0x7fdb94159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:46706/0 (socket says 172.21.15.7:46706) 2023-06-09T12:16:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 -- 172.21.15.7:0/3128386091 >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 msgr2=0x7fdb940cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:16:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9a59c700 1 --2- 172.21.15.7:0/3128386091 >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 0x7fdb940cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 --2- 172.21.15.7:0/3128386091 >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 0x7fdb940cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb98d99700 1 -- 172.21.15.7:0/3128386091 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174902984 0 0) 0x7fdb94159b20 con 0x7fdb940cb9a0 2023-06-09T12:16:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 -- 172.21.15.7:0/3128386091 >> v1:172.21.15.7:6789/0 conn(0x7fdb940cb9a0 legacy=0x7fdb94159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb99d9b700 1 -- 172.21.15.7:0/3128386091 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb94071ab0 con 0x7fdb940cb5d0 2023-06-09T12:16:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.647+0000 7fdb9a59c700 1 --2- 172.21.15.7:0/3128386091 >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 0x7fdb940cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:16:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb99d9b700 1 --2- 172.21.15.7:0/3128386091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb940cb5d0 0x7fdb940b52b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fdb940c9b80 tx=0x7fdb90009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79444db101b5583 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb98d99700 1 -- 172.21.15.7:0/3128386091 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000b870 con 0x7fdb940cb5d0 2023-06-09T12:16:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb98d99700 1 -- 172.21.15.7:0/3128386091 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000b9d0 con 0x7fdb940cb5d0 2023-06-09T12:16:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb98d99700 1 -- 172.21.15.7:0/3128386091 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000bd60 con 0x7fdb940cb5d0 2023-06-09T12:16:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3128386091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb940cb5d0 msgr2=0x7fdb940b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.648+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3128386091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb940cb5d0 0x7fdb940b52b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fdb940c9b80 tx=0x7fdb90009bd0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3128386091 shutdown_connections 2023-06-09T12:16:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3128386091 >> v2:172.21.15.117:3300/0 conn(0x7fdb9415cd40 0x7fdb940cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3128386091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb940cb5d0 0x7fdb940b52b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3128386091 >> 172.21.15.7:0/3128386091 conn(0x7fdb94070af0 msgr2=0x7fdb94070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3128386091 shutdown_connections 2023-06-09T12:16:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.649+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3128386091 wait complete. 2023-06-09T12:16:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.650+0000 7fdb9bfff700 1 Processor -- start 2023-06-09T12:16:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.651+0000 7fdb9bfff700 1 -- start start 2023-06-09T12:16:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.651+0000 7fdb9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb940cb5d0 0x7fdb940c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.659 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9bfff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb940cb9a0 0x7fdb941ee670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9bfff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9bfff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941eeed0 con 0x7fdb9415cd40 2023-06-09T12:16:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9bfff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c4f60 con 0x7fdb940cb5d0 2023-06-09T12:16:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9bfff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940c3750 con 0x7fdb940cb9a0 2023-06-09T12:16:30.660 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54874/0 (socket says 172.21.15.7:54874) 2023-06-09T12:16:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 -- 172.21.15.7:0/3665504630 learned_addr learned my addr 172.21.15.7:0/3665504630 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:16:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb99d9b700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb940cb5d0 0x7fdb940c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9959a700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb940cb9a0 0x7fdb941ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.661 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 -- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb940cb9a0 msgr2=0x7fdb941ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb940cb9a0 0x7fdb941ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 -- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb940cb5d0 msgr2=0x7fdb940c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb940cb5d0 0x7fdb940c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.652+0000 7fdb9a59c700 1 -- 172.21.15.7:0/3665504630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90009550 con 0x7fdb9415cd40 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb9a59c700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cd80 tx=0x7fdb8c00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:30.662 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00f480 con 0x7fdb9415cd40 2023-06-09T12:16:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00f600 con 0x7fdb9415cd40 2023-06-09T12:16:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00f9b0 con 0x7fdb9415cd40 2023-06-09T12:16:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9415d4f0 con 0x7fdb9415cd40 2023-06-09T12:16:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.653+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9415d210 con 0x7fdb9415cd40 2023-06-09T12:16:30.663 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.654+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9415d7d0 con 0x7fdb9415cd40 2023-06-09T12:16:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.656+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb8c00b040 con 0x7fdb9415cd40 2023-06-09T12:16:30.664 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.656+0000 7fdb8a7fc700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.656+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fdb8c08a560 con 0x7fdb9415cd40 2023-06-09T12:16:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.656+0000 7fdb99d9b700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.657+0000 7fdb99d9b700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 0x7fdb7806d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb90008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.662+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fdb8c054c00 con 0x7fdb9415cd40 2023-06-09T12:16:30.867 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.863+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb94074cf0 con 0x7fdb7806ace0 2023-06-09T12:16:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.873+0000 7fdb8a7fc700 1 -- 172.21.15.7:0/3665504630 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319333 (secure 0 0 0) 0x7fdb94074cf0 con 0x7fdb7806ace0 2023-06-09T12:16:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:16:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 msgr2=0x7fdb7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 0x7fdb7806d190 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb90008030 comp rx=0 tx=0).stop 2023-06-09T12:16:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 msgr2=0x7fdb941ef800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00cd80 tx=0x7fdb8c00a560 comp rx=0 tx=0).stop 2023-06-09T12:16:30.882 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 shutdown_connections 2023-06-09T12:16:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdb9415cd40 0x7fdb941ef800 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdb940cb5d0 0x7fdb940c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fdb7806ace0 0x7fdb7806d190 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 --2- 172.21.15.7:0/3665504630 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdb940cb9a0 0x7fdb941ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:30.883 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.877+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 >> 172.21.15.7:0/3665504630 conn(0x7fdb94070af0 msgr2=0x7fdb94158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:30.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.878+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 shutdown_connections 2023-06-09T12:16:30.884 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:30.878+0000 7fdb9bfff700 1 -- 172.21.15.7:0/3665504630 wait complete. 2023-06-09T12:16:30.884 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:16:30.899 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2041,"stamp":"2023-06-09T12:16:29.207032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2032363,"num_omap_keys":3760,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104297,"ondisk_log_size":104297,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2593096,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":709760,"kb_avail":752381624,"statfs":{"total":773094113280,"available":770438782976,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":726794240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003562"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3754,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.450221+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:12:22.450221+0000","last_peered":"2023-06-09T12:12:22.450221+0000","last_clean":"2023-06-09T12:12:22.450221+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:12:22.450221+0000","last_undegraded":"2023-06-09T12:12:22.450221+0000","last_fullsized":"2023-06-09T12:12:22.450221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1163","reported_seq":4432,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:16:26.221919+0000","last_change":"2023-06-09T12:14:07.706040+0000","last_active":"2023-06-09T12:16:26.221919+0000","last_peered":"2023-06-09T12:16:26.221919+0000","last_clean":"2023-06-09T12:16:26.221919+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:26.221919+0000","last_undegraded":"2023-06-09T12:16:26.221919+0000","last_fullsized":"2023-06-09T12:16:26.221919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":194,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 40s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'865","reported_seq":3469,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:30.383425+0000","last_change":"2023-06-09T12:16:07.536779+0000","last_active":"2023-06-09T12:15:30.383425+0000","last_peered":"2023-06-09T12:15:30.383425+0000","last_clean":"2023-06-09T12:15:30.383425+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:15:30.383425+0000","last_undegraded":"2023-06-09T12:15:30.383425+0000","last_fullsized":"2023-06-09T12:15:30.383425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":865,"log_dups_size":0,"ondisk_log_size":865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3535,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402942+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:11:52.402942+0000","last_peered":"2023-06-09T12:11:52.402942+0000","last_clean":"2023-06-09T12:11:52.402942+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:11:52.402942+0000","last_undegraded":"2023-06-09T12:11:52.402942+0000","last_fullsized":"2023-06-09T12:11:52.402942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.477990+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:12:02.477990+0000","last_peered":"2023-06-09T12:12:02.477990+0000","last_clean":"2023-06-09T12:12:02.477990+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:12:02.477990+0000","last_undegraded":"2023-06-09T12:12:02.477990+0000","last_fullsized":"2023-06-09T12:12:02.477990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3707,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.503075+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:12:02.503075+0000","last_peered":"2023-06-09T12:12:02.503075+0000","last_clean":"2023-06-09T12:12:02.503075+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:12:02.503075+0000","last_undegraded":"2023-06-09T12:12:02.503075+0000","last_fullsized":"2023-06-09T12:12:02.503075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'2979","reported_seq":4099,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:16:07.536702+0000","last_change":"2023-06-09T12:16:01.612368+0000","last_active":"2023-06-09T12:16:07.536702+0000","last_peered":"2023-06-09T12:16:07.536702+0000","last_clean":"2023-06-09T12:16:07.536702+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:07.536702+0000","last_undegraded":"2023-06-09T12:16:07.536702+0000","last_fullsized":"2023-06-09T12:16:07.536702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":2979,"log_dups_size":0,"ondisk_log_size":2979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 20s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3447,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.402619+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:12:47.402619+0000","last_peered":"2023-06-09T12:12:47.402619+0000","last_clean":"2023-06-09T12:12:47.402619+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:12:47.402619+0000","last_undegraded":"2023-06-09T12:12:47.402619+0000","last_fullsized":"2023-06-09T12:12:47.402619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3397,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.501927+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:11:52.501927+0000","last_peered":"2023-06-09T12:11:52.501927+0000","last_clean":"2023-06-09T12:11:52.501927+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:11:52.501927+0000","last_undegraded":"2023-06-09T12:11:52.501927+0000","last_fullsized":"2023-06-09T12:11:52.501927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3305,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:47.394377+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:11:47.394377+0000","last_peered":"2023-06-09T12:11:47.394377+0000","last_clean":"2023-06-09T12:11:47.394377+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:11:47.394377+0000","last_undegraded":"2023-06-09T12:11:47.394377+0000","last_fullsized":"2023-06-09T12:11:47.394377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:34.446550+0000","last_change":"2023-06-09T12:16:07.536754+0000","last_active":"2023-06-09T12:15:34.446550+0000","last_peered":"2023-06-09T12:15:34.446550+0000","last_clean":"2023-06-09T12:15:34.446550+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:15:34.446550+0000","last_undegraded":"2023-06-09T12:15:34.446550+0000","last_fullsized":"2023-06-09T12:15:34.446550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":929,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.505835+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:12:22.505835+0000","last_peered":"2023-06-09T12:12:22.505835+0000","last_clean":"2023-06-09T12:12:22.505835+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:12:22.505835+0000","last_undegraded":"2023-06-09T12:12:22.505835+0000","last_fullsized":"2023-06-09T12:12:22.505835+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1354,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.392300+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:12:32.392300+0000","last_peered":"2023-06-09T12:12:32.392300+0000","last_clean":"2023-06-09T12:12:32.392300+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:12:32.392300+0000","last_undegraded":"2023-06-09T12:12:32.392300+0000","last_fullsized":"2023-06-09T12:12:32.392300+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:59.442204+0000","last_change":"2023-06-09T12:15:52.418697+0000","last_active":"2023-06-09T12:14:59.442204+0000","last_peered":"2023-06-09T12:14:59.442204+0000","last_clean":"2023-06-09T12:14:59.442204+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:14:59.442204+0000","last_undegraded":"2023-06-09T12:14:59.442204+0000","last_fullsized":"2023-06-09T12:14:59.442204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.396890+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:13:07.396890+0000","last_peered":"2023-06-09T12:13:07.396890+0000","last_clean":"2023-06-09T12:13:07.396890+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:13:07.396890+0000","last_undegraded":"2023-06-09T12:13:07.396890+0000","last_fullsized":"2023-06-09T12:13:07.396890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3993,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.623947+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.623947+0000","last_peered":"2023-06-09T12:14:50.623947+0000","last_clean":"2023-06-09T12:14:50.623947+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.623947+0000","last_undegraded":"2023-06-09T12:14:50.623947+0000","last_fullsized":"2023-06-09T12:14:50.623947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1629,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401099+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:12:37.401099+0000","last_peered":"2023-06-09T12:12:37.401099+0000","last_clean":"2023-06-09T12:12:37.401099+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:12:37.401099+0000","last_undegraded":"2023-06-09T12:12:37.401099+0000","last_fullsized":"2023-06-09T12:12:37.401099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3947,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.318635+0000","last_change":"2023-06-09T12:15:27.414934+0000","last_active":"2023-06-09T12:13:37.318635+0000","last_peered":"2023-06-09T12:13:37.318635+0000","last_clean":"2023-06-09T12:13:37.318635+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:13:37.318635+0000","last_undegraded":"2023-06-09T12:13:37.318635+0000","last_fullsized":"2023-06-09T12:13:37.318635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.380832+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:12:22.380832+0000","last_peered":"2023-06-09T12:12:22.380832+0000","last_clean":"2023-06-09T12:12:22.380832+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:12:22.380832+0000","last_undegraded":"2023-06-09T12:12:22.380832+0000","last_fullsized":"2023-06-09T12:12:22.380832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73209,"ondisk_log_size":73209,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2032363,"num_omap_keys":3760,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30969,"ondisk_log_size":30969,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575640,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70499999999999996,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.84699999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.256},"last":0.68500000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.751,"15min":0.73199999999999998},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.95299999999999996,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.90300000000000002,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.56000000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.754}]},{"osd":3,"last update":"Fri Jun 9 12:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.622},{"interface":"front","average":{"1min":0.752,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.71999999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.33500000000000002},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.47799999999999998},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.41499999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.94499999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77500000000000002,"15min":0.749},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.53600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575640,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68200000000000005,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":1.044,"15min":1.044},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.79300000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":1.266,"15min":1.266},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":1.298,"15min":1.298},"last":0.75900000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.626,"5min":0.501,"15min":0.501},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.091},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.747,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.83199999999999996,"5min":1.0209999999999999,"15min":1.046},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81000000000000005,"15min":0.80600000000000005},"min":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.55900000000000005},"max":{"1min":0.88900000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.878}]},{"osd":4,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.66400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.86099999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.161,"15min":1.161},"last":0.90100000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.435},"max":{"1min":0.878,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.627},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.82599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575641,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405232,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98816,"kb_avail":93966608,"statfs":{"total":96636764160,"available":96221806592,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101187584},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.68600000000000005,"5min":0.747,"15min":0.95299999999999996},"last":0.45400000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.39600000000000002},"max":{"1min":0.73299999999999998,"5min":0.77900000000000003,"15min":1.2450000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81499999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.81699999999999995},"min":{"1min":0.76400000000000001,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.625,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0529999999999999,"5min":1.175,"15min":1.597},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.65600000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0069999999999999,"5min":1.196,"15min":1.544},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":0.97799999999999998,"5min":1.1539999999999999,"15min":1.704},"last":0.83799999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.1910000000000001,"15min":1.548},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83599999999999997,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":0.97699999999999998,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.81200000000000006}]},{"osd":7,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.159,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.874},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.139,"5min":1.24,"15min":1.498},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575640,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331432,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103616,"kb_avail":94040408,"statfs":{"total":96636764160,"available":96297377792,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106102784},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.51600000000000001,"15min":0.48199999999999998},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.27600000000000002},"max":{"1min":0.76000000000000001,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.55700000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41099999999999998},"max":{"1min":0.73399999999999999,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58599999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54300000000000004,"15min":0.56499999999999995},"min":{"1min":0.39200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":1.032},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52700000000000002,"15min":0.53700000000000003},"min":{"1min":0.35099999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.61299999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61399999999999999,"15min":0.59399999999999997},"min":{"1min":0.46100000000000002,"5min":0.40799999999999997,"15min":0.38400000000000001},"max":{"1min":0.80500000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.67600000000000005,"15min":0.69299999999999995},"min":{"1min":0.54900000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.90900000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.81799999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.84099999999999997},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.119,"5min":1.421,"15min":1.421},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.88300000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.58799999999999997},"max":{"1min":1.075,"5min":1.2490000000000001,"15min":1.268},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77800000000000002,"15min":0.78300000000000003},"min":{"1min":0.55500000000000005,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.94099999999999995,"5min":1.077,"15min":1.121},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88600000000000001,"15min":0.88500000000000001},"min":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.1259999999999999,"5min":1.371,"15min":3.5840000000000001},"last":0.79500000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.66200000000000003,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.81899999999999995,"15min":0.83999999999999997},"min":{"1min":0.53900000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.03,"5min":1.21,"15min":1.232},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.86799999999999999,"15min":0.87},"min":{"1min":0.68400000000000005,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.1000000000000001,"5min":1.1299999999999999,"15min":1.298},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.95799999999999996,"15min":0.95199999999999996},"min":{"1min":0.78100000000000003,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.179,"5min":1.371,"15min":1.371},"last":0.93600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575641,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45300000000000001,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.51000000000000001,"5min":0.65500000000000003,"15min":0.80600000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.59599999999999997,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.54200000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73899999999999999,"5min":0.81799999999999995,"15min":1.2969999999999999},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.4490000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.67600000000000005},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.75800000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.73999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":1.125,"5min":1.218,"15min":1.218},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.048,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.91100000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"max":{"1min":1.0920000000000001,"5min":1.3520000000000001,"15min":1.4359999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.88},"min":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.3899999999999999,"15min":1.5189999999999999},"last":1.073}]},{"osd":6,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.64100000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.872},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.747,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2350000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.99199999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":1.286,"15min":1.286},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.88200000000000001,"15min":0.86399999999999999},"min":{"1min":0.69099999999999995,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.1870000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0940000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575640,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426236,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103040,"kb_avail":93945604,"statfs":{"total":96636764160,"available":96200298496,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105512960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.64000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.745}]},{"osd":1,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.39400000000000002,"5min":0.36099999999999999,"15min":0.34300000000000003},"max":{"1min":0.66000000000000003,"5min":0.80300000000000005,"15min":0.88700000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.59399999999999997,"15min":0.56599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.78800000000000003,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.63700000000000001,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.46000000000000002},"max":{"1min":0.79900000000000004,"5min":0.872,"15min":0.97099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.505,"5min":0.55700000000000005,"15min":0.56999999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68100000000000005,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.82799999999999996,"15min":0.80700000000000005},"min":{"1min":0.57499999999999996,"5min":0.55200000000000005,"15min":0.48599999999999999},"max":{"1min":1.0169999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.82299999999999995,"15min":0.81000000000000005},"min":{"1min":0.63,"5min":0.56599999999999995,"15min":0.52500000000000002},"max":{"1min":0.95199999999999996,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84099999999999997},"min":{"1min":0.71199999999999997,"5min":0.61199999999999999,"15min":0.52800000000000002},"max":{"1min":1.008,"5min":1.409,"15min":1.409},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.89000000000000001},"min":{"1min":0.65600000000000003,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":0.96599999999999997,"5min":1.452,"15min":1.452},"last":0.76000000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79800000000000004,"15min":0.80900000000000005},"min":{"1min":0.56000000000000005,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.0289999999999999,"5min":1.1180000000000001,"15min":1.2649999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.81599999999999995,"15min":0.82599999999999996},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.1559999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.78700000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:15:36 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.81399999999999995,"15min":0.82899999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":0.97499999999999998,"5min":1.371,"15min":1.371},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.82599999999999996,"15min":0.84999999999999998},"min":{"1min":0.68000000000000005,"5min":0.55700000000000005,"15min":0.53700000000000003},"max":{"1min":0.97799999999999998,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.91400000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575639,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417940,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125184,"kb_avail":93953900,"statfs":{"total":96636764160,"available":96208793600,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128188416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.748},"min":{"1min":0.48199999999999998,"5min":0.38200000000000001,"15min":0.35699999999999998},"max":{"1min":0.93999999999999995,"5min":1.266,"15min":1.419},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80500000000000005,"15min":0.78900000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.96299999999999997,"5min":1.294,"15min":1.383},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.432,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.876,"5min":1.242,"15min":1.242},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.81299999999999994,"15min":0.79000000000000004},"min":{"1min":0.629,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79600000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.754,"15min":0.73499999999999999},"min":{"1min":0.53300000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.97799999999999998,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81299999999999994,"15min":0.79300000000000004},"min":{"1min":0.64100000000000001,"5min":0.63900000000000001,"15min":0.441},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.753}]},{"osd":3,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73799999999999999,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":0.90400000000000003,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.56200000000000006,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.56599999999999995,"15min":0.54800000000000004},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.36199999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.48299999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92500000000000004,"5min":1.0069999999999999,"15min":1.05},"last":0.54900000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.59799999999999998,"15min":0.58099999999999996},"min":{"1min":0.45200000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.76600000000000001,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.131},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:15:37 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.65100000000000002,"15min":0.65400000000000003},"min":{"1min":0.51700000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.501},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70199999999999996,"15min":0.65000000000000002},"min":{"1min":0.501,"5min":0.499,"15min":0.38300000000000001},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575639,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92100000000000004,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.579,"15min":1.579},"last":0.64600000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.41199999999999998},"max":{"1min":0.97199999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.749},{"interface":"front","average":{"1min":0.752,"5min":0.72799999999999998,"15min":0.73999999999999999},"min":{"1min":0.57899999999999996,"5min":0.498,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.97799999999999998,"5min":1.391,"15min":1.391},"last":0.79700000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.749},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93400000000000005,"5min":1.169,"15min":1.254},"last":0.629},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96299999999999997,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":1.522,"15min":1.522},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65800000000000003,"5min":1.28,"15min":1.28},"last":0.46600000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68000000000000005,"15min":0.63700000000000001},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80000000000000004,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88800000000000001,"5min":1.226,"15min":1.238},"last":0.78600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:16:30.907 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.907 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.908 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.908 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.909 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.910 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:30.910 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:16:50.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:16:51.123 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.120+0000 7f7bb2fff700 1 Processor -- start 2023-06-09T12:16:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.120+0000 7f7bb2fff700 1 -- start start 2023-06-09T12:16:51.124 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb2fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 0x7f7bac0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:51.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb2fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac159b20 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb2fff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac0cac80 con 0x7f7bac15cd40 2023-06-09T12:16:51.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb2fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1597d0 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb0d9b700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7bac0cb5d0 0x7f7bac159fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41228/0 (socket says 172.21.15.7:41228) 2023-06-09T12:16:51.125 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bb0d9b700 1 -- 172.21.15.7:0/2342434190 learned_addr learned my addr 172.21.15.7:0/2342434190 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:16:51.126 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7babfff700 1 --2- 172.21.15.7:0/2342434190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 0x7f7bac0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.122+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846273047 0 0) 0x7f7bac159b20 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b9c0035b0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.127 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7babfff700 1 -- 172.21.15.7:0/2342434190 >> v1:172.21.15.117:6789/0 conn(0x7f7bac15cd40 legacy=0x7f7bac0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772543264 0 0) 0x7f7bac0cac80 con 0x7f7bac15cd40 2023-06-09T12:16:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7babfff700 1 -- 172.21.15.7:0/2342434190 >> v1:172.21.15.7:6789/0 conn(0x7f7bac0cb5d0 legacy=0x7f7bac159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7babfff700 1 -- 172.21.15.7:0/2342434190 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac071ab0 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.128 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7babfff700 1 --2- 172.21.15.7:0/2342434190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 0x7f7bac0b52b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009870 tx=0x7f7b98009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6cf8ecdae29080 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98011830 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98011990 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.123+0000 7f7bab7fe700 1 -- 172.21.15.7:0/2342434190 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98011d20 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 -- 172.21.15.7:0/2342434190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 msgr2=0x7f7bac0b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.129 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/2342434190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 0x7f7bac0b52b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7b98009870 tx=0x7f7b98009b80 comp rx=0 tx=0).stop 2023-06-09T12:16:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 -- 172.21.15.7:0/2342434190 shutdown_connections 2023-06-09T12:16:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/2342434190 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac0cb9a0 0x7f7bac0b52b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 -- 172.21.15.7:0/2342434190 >> 172.21.15.7:0/2342434190 conn(0x7f7bac070af0 msgr2=0x7f7bac070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 -- 172.21.15.7:0/2342434190 shutdown_connections 2023-06-09T12:16:51.130 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.124+0000 7f7bb2fff700 1 -- 172.21.15.7:0/2342434190 wait complete. 2023-06-09T12:16:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.126+0000 7f7bb2fff700 1 Processor -- start 2023-06-09T12:16:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.126+0000 7f7bb2fff700 1 -- start start 2023-06-09T12:16:51.132 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bac0cb9a0 0x7f7bac155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac15cd40 0x7f7bac1f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac151b00 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac150f30 con 0x7f7bac0cb9a0 2023-06-09T12:16:51.133 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb2fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac15bb10 con 0x7f7bac15cd40 2023-06-09T12:16:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59072/0 (socket says 172.21.15.7:59072) 2023-06-09T12:16:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 -- 172.21.15.7:0/3802273205 learned_addr learned my addr 172.21.15.7:0/3802273205 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:16:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7babfff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bac0cb9a0 0x7f7bac155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:51.134 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb159c700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac15cd40 0x7f7bac1f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 -- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac15cd40 msgr2=0x7f7bac1f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac15cd40 0x7f7bac1f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 -- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bac0cb9a0 msgr2=0x7f7bac155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bac0cb9a0 0x7f7bac155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.127+0000 7f7bb0d9b700 1 -- 172.21.15.7:0/3802273205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980094a0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.135 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7bb0d9b700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c4d0 tx=0x7f7ba000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0005d70 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac15d4f0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba00070d0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.136 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0007440 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.128+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac15d210 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.129+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac15d7d0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.137 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.130+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7ba00075a0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.131+0000 7f7ba97fa700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 0x7f7b9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:16:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.131+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f7ba0089250 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.131+0000 7f7babfff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 0x7f7b9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:16:51.138 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.132+0000 7f7babfff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 0x7f7b9006d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cbd70 tx=0x7f7b9801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:16:51.139 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.136+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7ba00539b0 con 0x7f7bac0cb5d0 2023-06-09T12:16:51.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.340+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bac0b4570 con 0x7f7b9006ace0 2023-06-09T12:16:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.350+0000 7f7ba97fa700 1 -- 172.21.15.7:0/3802273205 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319253 (secure 0 0 0) 0x7f7bac0b4570 con 0x7f7b9006ace0 2023-06-09T12:16:51.356 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:16:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 msgr2=0x7f7b9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 0x7f7b9006d190 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cbd70 tx=0x7f7b9801c040 comp rx=0 tx=0).stop 2023-06-09T12:16:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 msgr2=0x7f7bac150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:16:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f7ba000c4d0 tx=0x7f7ba000c7e0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 shutdown_connections 2023-06-09T12:16:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f7b9006ace0 0x7f7b9006d190 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7bac15cd40 0x7f7bac1f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bac0cb5d0 0x7f7bac150550 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 --2- 172.21.15.7:0/3802273205 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7bac0cb9a0 0x7f7bac155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:16:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 >> 172.21.15.7:0/3802273205 conn(0x7f7bac070af0 msgr2=0x7f7bac158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:16:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 shutdown_connections 2023-06-09T12:16:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:16:51.354+0000 7f7bb2fff700 1 -- 172.21.15.7:0/3802273205 wait complete. 2023-06-09T12:16:51.360 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:16:51.375 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2052,"stamp":"2023-06-09T12:16:51.212841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2118499,"num_omap_keys":3925,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104544,"ondisk_log_size":104544,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2593480,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":710144,"kb_avail":752381240,"statfs":{"total":773094113280,"available":770438389760,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":727187456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003380"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3496,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:11:52.402891+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:11:52.402891+0000","last_peered":"2023-06-09T12:11:52.402891+0000","last_clean":"2023-06-09T12:11:52.402891+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:11:52.402891+0000","last_undegraded":"2023-06-09T12:11:52.402891+0000","last_fullsized":"2023-06-09T12:11:52.402891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'988","reported_seq":3601,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:16:46.774932+0000","last_change":"2023-06-09T12:16:32.963788+0000","last_active":"2023-06-09T12:16:46.774932+0000","last_peered":"2023-06-09T12:16:46.774932+0000","last_clean":"2023-06-09T12:16:46.774932+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:16:46.774932+0000","last_undegraded":"2023-06-09T12:16:46.774932+0000","last_fullsized":"2023-06-09T12:16:46.774932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":123,"log_size":988,"log_dups_size":0,"ondisk_log_size":988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 14s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4006,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385354+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385354+0000","last_peered":"2023-06-09T12:14:57.385354+0000","last_clean":"2023-06-09T12:14:57.385354+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385354+0000","last_undegraded":"2023-06-09T12:14:57.385354+0000","last_fullsized":"2023-06-09T12:14:57.385354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3815,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452149+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:12:37.452149+0000","last_peered":"2023-06-09T12:12:37.452149+0000","last_clean":"2023-06-09T12:12:37.452149+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:12:37.452149+0000","last_undegraded":"2023-06-09T12:12:37.452149+0000","last_fullsized":"2023-06-09T12:12:37.452149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:22.398790+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:12:22.398790+0000","last_peered":"2023-06-09T12:12:22.398790+0000","last_clean":"2023-06-09T12:12:22.398790+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:12:22.398790+0000","last_undegraded":"2023-06-09T12:12:22.398790+0000","last_fullsized":"2023-06-09T12:12:22.398790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2029,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388485+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:12:02.388485+0000","last_peered":"2023-06-09T12:12:02.388485+0000","last_clean":"2023-06-09T12:12:02.388485+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:12:02.388485+0000","last_undegraded":"2023-06-09T12:12:02.388485+0000","last_fullsized":"2023-06-09T12:12:02.388485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572416+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:12:52.572416+0000","last_peered":"2023-06-09T12:12:52.572416+0000","last_clean":"2023-06-09T12:12:52.572416+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:12:52.572416+0000","last_undegraded":"2023-06-09T12:12:52.572416+0000","last_fullsized":"2023-06-09T12:12:52.572416+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.479833+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:12:17.479833+0000","last_peered":"2023-06-09T12:12:17.479833+0000","last_clean":"2023-06-09T12:12:17.479833+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:12:17.479833+0000","last_undegraded":"2023-06-09T12:12:17.479833+0000","last_fullsized":"2023-06-09T12:12:17.479833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3860,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:27.391742+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:12:27.391742+0000","last_peered":"2023-06-09T12:12:27.391742+0000","last_clean":"2023-06-09T12:12:27.391742+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:12:27.391742+0000","last_undegraded":"2023-06-09T12:12:27.391742+0000","last_fullsized":"2023-06-09T12:12:27.391742+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1342,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:17.568711+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:12:17.568711+0000","last_peered":"2023-06-09T12:12:17.568711+0000","last_clean":"2023-06-09T12:12:17.568711+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:12:17.568711+0000","last_undegraded":"2023-06-09T12:12:17.568711+0000","last_fullsized":"2023-06-09T12:12:17.568711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4050,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:02.388595+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:12:02.388595+0000","last_peered":"2023-06-09T12:12:02.388595+0000","last_clean":"2023-06-09T12:12:02.388595+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:12:02.388595+0000","last_undegraded":"2023-06-09T12:12:02.388595+0000","last_fullsized":"2023-06-09T12:12:02.388595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:34.446550+0000","last_change":"2023-06-09T12:16:07.536754+0000","last_active":"2023-06-09T12:15:34.446550+0000","last_peered":"2023-06-09T12:15:34.446550+0000","last_clean":"2023-06-09T12:15:34.446550+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:15:34.446550+0000","last_undegraded":"2023-06-09T12:15:34.446550+0000","last_fullsized":"2023-06-09T12:15:34.446550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":192,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:14.458419+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:14.458419+0000","last_peered":"2023-06-09T12:14:14.458419+0000","last_clean":"2023-06-09T12:14:14.458419+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:14.458419+0000","last_undegraded":"2023-06-09T12:14:14.458419+0000","last_fullsized":"2023-06-09T12:14:14.458419+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'342","reported_seq":531,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:59.442204+0000","last_change":"2023-06-09T12:15:52.418697+0000","last_active":"2023-06-09T12:14:59.442204+0000","last_peered":"2023-06-09T12:14:59.442204+0000","last_clean":"2023-06-09T12:14:59.442204+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:14:59.442204+0000","last_undegraded":"2023-06-09T12:14:59.442204+0000","last_fullsized":"2023-06-09T12:14:59.442204+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:48.640112+0000","last_clean_scrub_stamp":"2023-06-09T11:10:48.640112+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":168,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.384090+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:12:47.384090+0000","last_peered":"2023-06-09T12:12:47.384090+0000","last_clean":"2023-06-09T12:12:47.384090+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:12:47.384090+0000","last_undegraded":"2023-06-09T12:12:47.384090+0000","last_fullsized":"2023-06-09T12:12:47.384090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.396890+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:13:07.396890+0000","last_peered":"2023-06-09T12:13:07.396890+0000","last_clean":"2023-06-09T12:13:07.396890+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:13:07.396890+0000","last_undegraded":"2023-06-09T12:13:07.396890+0000","last_fullsized":"2023-06-09T12:13:07.396890+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:30.246382+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:12:30.246382+0000","last_peered":"2023-06-09T12:12:30.246382+0000","last_clean":"2023-06-09T12:12:30.246382+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:12:30.246382+0000","last_undegraded":"2023-06-09T12:12:30.246382+0000","last_fullsized":"2023-06-09T12:12:30.246382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.571832+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:12:47.571832+0000","last_peered":"2023-06-09T12:12:47.571832+0000","last_clean":"2023-06-09T12:12:47.571832+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:12:47.571832+0000","last_undegraded":"2023-06-09T12:12:47.571832+0000","last_fullsized":"2023-06-09T12:12:47.571832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3948,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:15:27.414934+0000","last_change":"2023-06-09T12:16:42.577379+0000","last_active":"2023-06-09T12:15:27.414934+0000","last_peered":"2023-06-09T12:15:27.414934+0000","last_clean":"2023-06-09T12:15:27.414934+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:15:27.414934+0000","last_undegraded":"2023-06-09T12:15:27.414934+0000","last_fullsized":"2023-06-09T12:15:27.414934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73432,"ondisk_log_size":73432,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2118499,"num_omap_keys":3925,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":30993,"ondisk_log_size":30993,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575644,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70499999999999996,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.84699999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.256},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.751,"15min":0.73199999999999998},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.95299999999999996,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.92900000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.90300000000000002,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.56000000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.33500000000000002},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.5},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.47799999999999998},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.94499999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77500000000000002,"15min":0.749},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.69299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575644,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.68200000000000005,"15min":0.67300000000000004},"min":{"1min":0.46000000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.80100000000000005,"5min":1.044,"15min":1.044},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.70899999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72699999999999998,"15min":0.71799999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.86599999999999999,"5min":1.266,"15min":1.266},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.745,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63200000000000001,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.91200000000000003,"5min":1.298,"15min":1.298},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.84599999999999997,"5min":1.0209999999999999,"15min":1.0369999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.626,"5min":0.501,"15min":0.501},"max":{"1min":0.88600000000000001,"5min":1.091,"15min":1.091},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.747,"15min":0.755},"min":{"1min":0.49199999999999999,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.83199999999999996,"5min":1.0209999999999999,"15min":1.046},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.81000000000000005,"15min":0.80600000000000005},"min":{"1min":0.69399999999999995,"5min":0.61499999999999999,"15min":0.55900000000000005},"max":{"1min":0.88900000000000001,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57799999999999996,"15min":0.58599999999999997},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.66400000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.44},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.59699999999999998},"min":{"1min":0.443,"5min":0.42899999999999999,"15min":0.371},"max":{"1min":0.86099999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60499999999999998,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.40400000000000003,"15min":0.38200000000000001},"max":{"1min":0.79900000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.77300000000000002,"15min":0.73699999999999999},"min":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.161,"15min":1.161},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:16:05 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67100000000000004,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.46100000000000002,"15min":0.435},"max":{"1min":0.878,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.70099999999999996},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.42399999999999999},"max":{"1min":0.90100000000000002,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575645,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405360,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":98944,"kb_avail":93966480,"statfs":{"total":96636764160,"available":96221675520,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101318656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.45400000000000001},"min":{"1min":0.33400000000000002,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.378},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.45900000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.5},"min":{"1min":0.35299999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.68899999999999995,"5min":0.71699999999999997,"15min":0.995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.68600000000000005,"5min":0.747,"15min":0.95299999999999996},"last":0.39700000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.60199999999999998,"15min":0.59799999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.39600000000000002},"max":{"1min":0.73299999999999998,"5min":0.77900000000000003,"15min":1.2450000000000001},"last":0.439},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69899999999999995,"15min":0.69399999999999995},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81499999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.625}]},{"osd":4,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89300000000000002,"15min":0.81699999999999995},"min":{"1min":0.76400000000000001,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.0940000000000001,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.79700000000000004,"15min":0.84199999999999997},"min":{"1min":0.625,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0529999999999999,"5min":1.175,"15min":1.597},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.65600000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0069999999999999,"5min":1.196,"15min":1.544},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":0.97799999999999998,"5min":1.1539999999999999,"15min":1.704},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.61499999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.071,"5min":1.1910000000000001,"15min":1.548},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.83599999999999997,"15min":0.83199999999999996},"min":{"1min":0.627,"5min":0.53000000000000003,"15min":0.52400000000000002},"max":{"1min":0.97699999999999998,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:16:01 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89900000000000002,"15min":0.90300000000000002},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.159,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.86099999999999999,"15min":0.84999999999999998},"min":{"1min":0.65300000000000002,"5min":0.61199999999999999,"15min":0.51400000000000001},"max":{"1min":1.139,"5min":1.24,"15min":1.498},"last":0.96299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575644,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.48699999999999999},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70399999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64600000000000002,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0149999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.84899999999999998},"min":{"1min":0.64100000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.131,"5min":1.421,"15min":1.421},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.878,"15min":0.88600000000000001},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.0760000000000001,"5min":1.2490000000000001,"15min":1.268},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.121},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.093,"5min":1.371,"15min":3.5840000000000001},"last":1.0049999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.61299999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.069,"5min":1.113,"15min":1.232},"last":0.92300000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.879,"15min":0.876},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.103,"5min":1.1299999999999999,"15min":1.298},"last":1.036},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.96899999999999997,"15min":0.95899999999999996},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1539999999999999,"5min":1.371,"15min":1.371},"last":1.0820000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575645,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.45300000000000001,"15min":0.46200000000000002},"min":{"1min":0.33000000000000002,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.51000000000000001,"5min":0.65500000000000003,"15min":0.80600000000000005},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.38600000000000001},"max":{"1min":0.59599999999999997,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.58599999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.36199999999999999,"5min":0.33400000000000002,"15min":0.318},"max":{"1min":0.54200000000000004,"5min":0.66900000000000004,"15min":1.236},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58799999999999997,"15min":0.622},"min":{"1min":0.44500000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73899999999999999,"5min":0.81799999999999995,"15min":1.2969999999999999},"last":0.56699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.60599999999999998,"15min":0.60899999999999999},"min":{"1min":0.436,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.68999999999999995,"5min":0.878,"15min":1.4490000000000001},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66600000000000004,"15min":0.67600000000000005},"min":{"1min":0.52700000000000002,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.75800000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.79300000000000004,"15min":0.79100000000000004},"min":{"1min":0.61299999999999999,"5min":0.49099999999999999,"15min":0.48099999999999998},"max":{"1min":1.125,"5min":1.218,"15min":1.218},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79500000000000004,"15min":0.79100000000000004},"min":{"1min":0.59899999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.048,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.86599999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.82999999999999996,"15min":0.83199999999999996},"min":{"1min":0.61899999999999999,"5min":0.56899999999999995,"15min":0.56200000000000006},"max":{"1min":1.0920000000000001,"5min":1.3520000000000001,"15min":1.4359999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.875,"15min":0.88},"min":{"1min":0.72099999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.147,"5min":1.3899999999999999,"15min":1.5189999999999999},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82599999999999996,"15min":0.82099999999999995},"min":{"1min":0.64100000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1020000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.85899999999999999,"15min":0.81899999999999995},"min":{"1min":0.747,"5min":0.60999999999999999,"15min":0.54300000000000004},"max":{"1min":1.2350000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.92600000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:15:54 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85799999999999998,"15min":0.84999999999999998},"min":{"1min":0.65800000000000003,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.075,"5min":1.286,"15min":1.286},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.877,"5min":0.88200000000000001,"15min":0.86399999999999999},"min":{"1min":0.69099999999999995,"5min":0.57899999999999996,"15min":0.57399999999999995},"max":{"1min":1.1870000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575644,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426300,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103104,"kb_avail":93945540,"statfs":{"total":96636764160,"available":96200232960,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105578496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:04 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.48199999999999998,"15min":0.48399999999999999},"min":{"1min":0.33300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.67000000000000004,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54700000000000004,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.64000000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.56299999999999994}]},{"osd":1,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.624}]},{"osd":2,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85499999999999998,"15min":0.81299999999999994},"min":{"1min":0.69899999999999995,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.2130000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.83899999999999997,"15min":0.81499999999999995},"min":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85299999999999998,"15min":0.84199999999999997},"min":{"1min":0.63800000000000001,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.409,"15min":1.409},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89600000000000002,"15min":0.89100000000000001},"min":{"1min":0.72599999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2250000000000001,"5min":1.452,"15min":1.452},"last":0.90700000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.80700000000000005},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.099,"5min":1.1180000000000001,"15min":1.198},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.70499999999999996,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.113,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":1.038,"5min":1.371,"15min":1.371},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.163,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.79100000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575643,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418068,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125312,"kb_avail":93953772,"statfs":{"total":96636764160,"available":96208662528,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128319488},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":1.266,"15min":1.419},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.78900000000000003},"min":{"1min":0.55800000000000005,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":1.294,"15min":1.383},"last":0.61499999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.621,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.242,"15min":1.242},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81699999999999995,"15min":0.79400000000000004},"min":{"1min":0.68999999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":0.97899999999999998,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.77200000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.39700000000000002},"max":{"1min":1.0409999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.441},"max":{"1min":0.96299999999999997,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.81699999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.70599999999999996},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":0.96099999999999997,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.437},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":0.92500000000000004,"15min":1.05},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.505},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.131},"last":0.74399999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.624,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71099999999999997,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.64300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575643,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68400000000000005,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.92100000000000004,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71099999999999997,"15min":0.71999999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.579,"15min":1.579},"last":0.73199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.47599999999999998,"5min":0.44,"15min":0.41199999999999998},"max":{"1min":0.97199999999999998,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.72799999999999998,"15min":0.73999999999999999},"min":{"1min":0.57899999999999996,"5min":0.498,"15min":0.45700000000000002},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.49199999999999999,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.90600000000000003,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.53500000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.97799999999999998,"5min":1.391,"15min":1.391},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77000000000000002,"15min":0.749},"min":{"1min":0.628,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.93400000000000005,"5min":1.169,"15min":1.254},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.81699999999999995,"15min":0.80900000000000005},"min":{"1min":0.66700000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96299999999999997,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.629}]},{"osd":4,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.78300000000000003,"5min":1.522,"15min":1.522},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65800000000000003,"5min":1.28,"15min":1.28},"last":0.439}]},{"osd":6,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68000000000000005,"15min":0.63700000000000001},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.40200000000000002},"max":{"1min":0.80000000000000004,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.83399999999999996,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:15:51 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.433,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.84999999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.501},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.746,"15min":0.746},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.88800000000000001,"5min":1.226,"15min":1.238},"last":0.75}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:16:51.386 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:51.387 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:51.387 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-06-09T11:10:48.640112+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=48, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:51.388 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:16:51.388 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:17:00.472 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:00.477 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:11.389 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:17:11.599 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.596+0000 7f2c5f1ff700 1 Processor -- start 2023-06-09T12:17:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.596+0000 7f2c5f1ff700 1 -- start start 2023-06-09T12:17:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.597+0000 7f2c5f1ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f2c580cb5d0 0x7f2c580b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.597+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.597+0000 7f2c5f1ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c58159b20 con 0x7f2c5815cd40 2023-06-09T12:17:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.597+0000 7f2c5f1ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c580cac80 con 0x7f2c580cb5d0 2023-06-09T12:17:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.597+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581597d0 con 0x7f2c580cb9a0 2023-06-09T12:17:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:60934/0 (socket says 172.21.15.7:60934) 2023-06-09T12:17:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 -- 172.21.15.7:0/1544461153 learned_addr learned my addr 172.21.15.7:0/1544461153 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:17:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c5cf9b700 1 --2- 172.21.15.7:0/1544461153 >> v2:172.21.15.117:3300/0 conn(0x7f2c580cb5d0 0x7f2c580b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 -- 172.21.15.7:0/1544461153 >> v2:172.21.15.117:3300/0 conn(0x7f2c580cb5d0 msgr2=0x7f2c580b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:17:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 --2- 172.21.15.7:0/1544461153 >> v2:172.21.15.117:3300/0 conn(0x7f2c580cb5d0 0x7f2c580b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 -- 172.21.15.7:0/1544461153 >> v1:172.21.15.117:6789/0 conn(0x7f2c5815cd40 legacy=0x7f2c580cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.598+0000 7f2c57fff700 1 -- 172.21.15.7:0/1544461153 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58071ab0 con 0x7f2c580cb9a0 2023-06-09T12:17:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c57fff700 1 --2- 172.21.15.7:0/1544461153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c440098f0 tx=0x7f2c44009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1be7cb6aae18ec35 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c577fe700 1 -- 172.21.15.7:0/1544461153 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011830 con 0x7f2c580cb9a0 2023-06-09T12:17:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c577fe700 1 -- 172.21.15.7:0/1544461153 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c44011990 con 0x7f2c580cb9a0 2023-06-09T12:17:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c577fe700 1 -- 172.21.15.7:0/1544461153 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44011d20 con 0x7f2c580cb9a0 2023-06-09T12:17:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/1544461153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 msgr2=0x7f2c58159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.599+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/1544461153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2c440098f0 tx=0x7f2c44009c00 comp rx=0 tx=0).stop 2023-06-09T12:17:11.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/1544461153 shutdown_connections 2023-06-09T12:17:11.607 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/1544461153 >> v2:172.21.15.117:3300/0 conn(0x7f2c580cb5d0 0x7f2c580b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/1544461153 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c580cb9a0 0x7f2c58159fb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/1544461153 >> 172.21.15.7:0/1544461153 conn(0x7f2c58070af0 msgr2=0x7f2c58070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:11.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/1544461153 shutdown_connections 2023-06-09T12:17:11.608 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.600+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/1544461153 wait complete. 2023-06-09T12:17:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.601+0000 7f2c5f1ff700 1 Processor -- start 2023-06-09T12:17:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.601+0000 7f2c5f1ff700 1 -- start start 2023-06-09T12:17:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c580cb5d0 0x7f2c581ef880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.609 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c5815cd40 0x7f2c581f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580cd560 con 0x7f2c580cb9a0 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c581516c0 con 0x7f2c580cb5d0 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.602+0000 7f2c5f1ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580cd7b0 con 0x7f2c5815cd40 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:50334/0 (socket says 172.21.15.7:50334) 2023-06-09T12:17:11.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 -- 172.21.15.7:0/3828351423 learned_addr learned my addr 172.21.15.7:0/3828351423 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:17:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c5d79c700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c5815cd40 0x7f2c581f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c5cf9b700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c580cb5d0 0x7f2c581ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 -- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c5815cd40 msgr2=0x7f2c581f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c5815cd40 0x7f2c581f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 -- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c580cb5d0 msgr2=0x7f2c581ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c580cb5d0 0x7f2c581ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 -- 172.21.15.7:0/3828351423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44009550 con 0x7f2c580cb9a0 2023-06-09T12:17:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.603+0000 7f2c57fff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2c44000c00 tx=0x7f2c44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.604+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44004480 con 0x7f2c580cb9a0 2023-06-09T12:17:11.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.604+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4401f070 con 0x7f2c580cb9a0 2023-06-09T12:17:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.604+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5815d4f0 con 0x7f2c580cb9a0 2023-06-09T12:17:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.604+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440038c0 con 0x7f2c580cb9a0 2023-06-09T12:17:11.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.604+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5815d210 con 0x7f2c580cb9a0 2023-06-09T12:17:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.605+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5815d7d0 con 0x7f2c580cb9a0 2023-06-09T12:17:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.606+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c44021030 con 0x7f2c580cb9a0 2023-06-09T12:17:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.607+0000 7f2c557fa700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 0x7f2c3c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:11.614 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.607+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f2c4408b160 con 0x7f2c580cb9a0 2023-06-09T12:17:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.607+0000 7f2c5cf9b700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 0x7f2c3c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.608+0000 7f2c5cf9b700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 0x7f2c3c06d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009be0 tx=0x7f2c4c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:11.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.612+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f2c44055890 con 0x7f2c580cb9a0 2023-06-09T12:17:11.821 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.818+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c5815ae90 con 0x7f2c3c06ace0 2023-06-09T12:17:11.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.827+0000 7f2c557fa700 1 -- 172.21.15.7:0/3828351423 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319424 (secure 0 0 0) 0x7f2c5815ae90 con 0x7f2c3c06ace0 2023-06-09T12:17:11.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:17:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 msgr2=0x7f2c3c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 0x7f2c3c06d190 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009be0 tx=0x7f2c4c009210 comp rx=0 tx=0).stop 2023-06-09T12:17:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 msgr2=0x7f2c580ce850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f2c44000c00 tx=0x7f2c44004750 comp rx=0 tx=0).stop 2023-06-09T12:17:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 shutdown_connections 2023-06-09T12:17:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f2c3c06ace0 0x7f2c3c06d190 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2c5815cd40 0x7f2c581f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2c580cb5d0 0x7f2c581ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 --2- 172.21.15.7:0/3828351423 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c580cb9a0 0x7f2c580ce850 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 >> 172.21.15.7:0/3828351423 conn(0x7f2c58070af0 msgr2=0x7f2c580cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 shutdown_connections 2023-06-09T12:17:11.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:11.832+0000 7f2c5f1ff700 1 -- 172.21.15.7:0/3828351423 wait complete. 2023-06-09T12:17:11.839 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:17:11.854 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2062,"stamp":"2023-06-09T12:17:11.218251+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104679,"ondisk_log_size":104679,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2593800,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":710464,"kb_avail":752380920,"statfs":{"total":773094113280,"available":770438062080,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":727515136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003437"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3487,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411828+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:13:02.411828+0000","last_peered":"2023-06-09T12:13:02.411828+0000","last_clean":"2023-06-09T12:13:02.411828+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:13:02.411828+0000","last_undegraded":"2023-06-09T12:13:02.411828+0000","last_fullsized":"2023-06-09T12:13:02.411828+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3520,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:11.391565+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:13:11.391565+0000","last_peered":"2023-06-09T12:13:11.391565+0000","last_clean":"2023-06-09T12:13:11.391565+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:13:11.391565+0000","last_undegraded":"2023-06-09T12:13:11.391565+0000","last_fullsized":"2023-06-09T12:13:11.391565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1111","reported_seq":3731,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:17:07.219298+0000","last_change":"2023-06-09T12:16:32.963788+0000","last_active":"2023-06-09T12:17:07.219298+0000","last_peered":"2023-06-09T12:17:07.219298+0000","last_clean":"2023-06-09T12:17:07.219298+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:07.219298+0000","last_undegraded":"2023-06-09T12:17:07.219298+0000","last_fullsized":"2023-06-09T12:17:07.219298+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":246,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3744,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:32.451533+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:12:32.451533+0000","last_peered":"2023-06-09T12:12:32.451533+0000","last_clean":"2023-06-09T12:12:32.451533+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:12:32.451533+0000","last_undegraded":"2023-06-09T12:12:32.451533+0000","last_fullsized":"2023-06-09T12:12:32.451533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3526,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.452121+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:12:37.452121+0000","last_peered":"2023-06-09T12:12:37.452121+0000","last_clean":"2023-06-09T12:12:37.452121+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:12:37.452121+0000","last_undegraded":"2023-06-09T12:12:37.452121+0000","last_fullsized":"2023-06-09T12:12:37.452121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.507954+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:12:37.507954+0000","last_peered":"2023-06-09T12:12:37.507954+0000","last_clean":"2023-06-09T12:12:37.507954+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:12:37.507954+0000","last_undegraded":"2023-06-09T12:12:37.507954+0000","last_fullsized":"2023-06-09T12:12:37.507954+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3812,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.393680+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:12:42.393680+0000","last_peered":"2023-06-09T12:12:42.393680+0000","last_clean":"2023-06-09T12:12:42.393680+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:12:42.393680+0000","last_undegraded":"2023-06-09T12:12:42.393680+0000","last_fullsized":"2023-06-09T12:12:42.393680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":758,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484846+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:12:57.484846+0000","last_peered":"2023-06-09T12:12:57.484846+0000","last_clean":"2023-06-09T12:12:57.484846+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:12:57.484846+0000","last_undegraded":"2023-06-09T12:12:57.484846+0000","last_fullsized":"2023-06-09T12:12:57.484846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":749,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976825+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976825+0000","last_peered":"2023-06-09T12:15:09.976825+0000","last_clean":"2023-06-09T12:15:09.976825+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976825+0000","last_undegraded":"2023-06-09T12:15:09.976825+0000","last_fullsized":"2023-06-09T12:15:09.976825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4303,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.401722+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:12:42.401722+0000","last_peered":"2023-06-09T12:12:42.401722+0000","last_clean":"2023-06-09T12:12:42.401722+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:12:42.401722+0000","last_undegraded":"2023-06-09T12:12:42.401722+0000","last_fullsized":"2023-06-09T12:12:42.401722+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'975","reported_seq":3851,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:34.446550+0000","last_change":"2023-06-09T12:16:07.536754+0000","last_active":"2023-06-09T12:15:34.446550+0000","last_peered":"2023-06-09T12:15:34.446550+0000","last_clean":"2023-06-09T12:15:34.446550+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:15:34.446550+0000","last_undegraded":"2023-06-09T12:15:34.446550+0000","last_fullsized":"2023-06-09T12:15:34.446550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":975,"log_dups_size":0,"ondisk_log_size":975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1272,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.572367+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:12:52.572367+0000","last_peered":"2023-06-09T12:12:52.572367+0000","last_clean":"2023-06-09T12:12:52.572367+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:12:52.572367+0000","last_undegraded":"2023-06-09T12:12:52.572367+0000","last_fullsized":"2023-06-09T12:12:52.572367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086123+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086123+0000","last_peered":"2023-06-09T12:15:13.086123+0000","last_clean":"2023-06-09T12:15:13.086123+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086123+0000","last_undegraded":"2023-06-09T12:15:13.086123+0000","last_fullsized":"2023-06-09T12:15:13.086123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387905+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:13:17.387905+0000","last_peered":"2023-06-09T12:13:17.387905+0000","last_clean":"2023-06-09T12:13:17.387905+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:13:17.387905+0000","last_undegraded":"2023-06-09T12:13:17.387905+0000","last_fullsized":"2023-06-09T12:13:17.387905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1083,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413824+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:13:17.413824+0000","last_peered":"2023-06-09T12:13:17.413824+0000","last_clean":"2023-06-09T12:13:17.413824+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:13:17.413824+0000","last_undegraded":"2023-06-09T12:13:17.413824+0000","last_fullsized":"2023-06-09T12:13:17.413824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3911,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.052087+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:13:12.052087+0000","last_peered":"2023-06-09T12:13:12.052087+0000","last_clean":"2023-06-09T12:13:12.052087+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:13:12.052087+0000","last_undegraded":"2023-06-09T12:13:12.052087+0000","last_fullsized":"2023-06-09T12:13:12.052087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":172,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.413775+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:13:17.413775+0000","last_peered":"2023-06-09T12:13:17.413775+0000","last_clean":"2023-06-09T12:13:17.413775+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:13:17.413775+0000","last_undegraded":"2023-06-09T12:13:17.413775+0000","last_fullsized":"2023-06-09T12:13:17.413775+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.456586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:13:17.456586+0000","last_peered":"2023-06-09T12:13:17.456586+0000","last_clean":"2023-06-09T12:13:17.456586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:13:17.456586+0000","last_undegraded":"2023-06-09T12:13:17.456586+0000","last_fullsized":"2023-06-09T12:13:17.456586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411752+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:13:02.411752+0000","last_peered":"2023-06-09T12:13:02.411752+0000","last_clean":"2023-06-09T12:13:02.411752+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:13:02.411752+0000","last_undegraded":"2023-06-09T12:13:02.411752+0000","last_fullsized":"2023-06-09T12:13:02.411752+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.498887+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:52.498887+0000","last_peered":"2023-06-09T12:14:52.498887+0000","last_clean":"2023-06-09T12:14:52.498887+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:52.498887+0000","last_undegraded":"2023-06-09T12:14:52.498887+0000","last_fullsized":"2023-06-09T12:14:52.498887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3952,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:17:05.723887+0000","last_change":"2023-06-09T12:16:42.577379+0000","last_active":"2023-06-09T12:17:05.723887+0000","last_peered":"2023-06-09T12:17:05.723887+0000","last_clean":"2023-06-09T12:17:05.723887+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:17:05.723887+0000","last_undegraded":"2023-06-09T12:17:05.723887+0000","last_fullsized":"2023-06-09T12:17:05.723887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73555,"ondisk_log_size":73555,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575648,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70499999999999996,"15min":0.69899999999999995},"min":{"1min":0.51600000000000001,"5min":0.51300000000000001,"15min":0.33700000000000002},"max":{"1min":0.84699999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74099999999999999,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":0.93899999999999995,"5min":1.143,"15min":1.256},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.751,"15min":0.73199999999999998},"min":{"1min":0.56299999999999994,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.91100000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.753},"min":{"1min":0.54500000000000004,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":0.95299999999999996,"5min":0.98999999999999999,"15min":0.99099999999999999},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73299999999999998,"15min":0.72499999999999998},"min":{"1min":0.53100000000000003,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.90300000000000002,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.753},"min":{"1min":0.56000000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:16:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.56100000000000005,"5min":0.48199999999999998,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56899999999999995,"5min":0.56799999999999995,"15min":0.505},"max":{"1min":0.92300000000000004,"5min":1.079,"15min":1.079},"last":0.68600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.33500000000000002},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.755,"15min":0.73699999999999999},"min":{"1min":0.56999999999999995,"5min":0.55900000000000005,"15min":0.47799999999999998},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.623,"15min":0.60999999999999999},"min":{"1min":0.44800000000000001,"5min":0.42199999999999999,"15min":0.378},"max":{"1min":0.93200000000000005,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":0.90400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.627}]},{"osd":6,"last update":"Fri Jun 9 12:16:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.68200000000000005,"15min":0.65600000000000003},"min":{"1min":0.48999999999999999,"5min":0.44600000000000001,"15min":0.44400000000000001},"max":{"1min":0.94499999999999995,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.77500000000000002,"15min":0.749},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.82299999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575648,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":1.044,"15min":1.044},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.94099999999999995,"5min":1.266,"15min":1.266},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":1.298,"15min":1.298},"last":0.77100000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.76700000000000002},"min":{"1min":0.57599999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.81599999999999995,"5min":1.091,"15min":1.091},"last":0.72099999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.86499999999999999,"5min":1.0209999999999999,"15min":1.046},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.878,"5min":1.077,"15min":1.1259999999999999},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.71899999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.443,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.873,"15min":0.98399999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.745},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.41899999999999998},"max":{"1min":0.91100000000000003,"5min":1.161,"15min":1.161},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42399999999999999},"max":{"1min":0.86799999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.75800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575649,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405552,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99136,"kb_avail":93966288,"statfs":{"total":96636764160,"available":96221478912,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101515264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.439,"15min":0.45300000000000001},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.44800000000000001,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.71799999999999997,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.499},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65400000000000003,"5min":0.71699999999999997,"15min":0.995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.41999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":1.2450000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.57799999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89300000000000002,"15min":0.82899999999999996},"min":{"1min":0.747,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.1419999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78700000000000003,"15min":0.83799999999999997},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0229999999999999,"5min":1.175,"15min":1.597},"last":0.59599999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:16:13 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.65600000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0069999999999999,"5min":1.196,"15min":1.544},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.84499999999999997,"15min":0.86199999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":0.97799999999999998,"5min":1.1539999999999999,"15min":1.704},"last":0.69999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":0.91800000000000004,"5min":1.1910000000000001,"15min":1.548},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.70999999999999996,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.72199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1779999999999999,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.044,"5min":1.24,"15min":1.498},"last":0.74399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575648,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.48699999999999999},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70399999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64600000000000002,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.496}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0149999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.84899999999999998},"min":{"1min":0.64100000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.131,"5min":1.421,"15min":1.421},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.878,"15min":0.88600000000000001},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.0760000000000001,"5min":1.2490000000000001,"15min":1.268},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.121},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.093,"5min":1.371,"15min":3.5840000000000001},"last":0.877}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.61299999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.069,"5min":1.113,"15min":1.232},"last":0.77600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.879,"15min":0.876},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.103,"5min":1.1299999999999999,"15min":1.298},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.96899999999999997,"15min":0.95899999999999996},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1539999999999999,"5min":1.371,"15min":1.371},"last":0.96399999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575649,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.621,"15min":0.80600000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":1.236},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.624},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.2969999999999999},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.86599999999999999,"5min":1.298,"15min":1.4650000000000001},"last":0.69099999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":1.337,"15min":1.4359999999999999},"last":1.032},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.86699999999999999,"15min":0.88},"min":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.073,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":1.0680000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85699999999999998,"15min":0.82299999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54300000000000004},"max":{"1min":0.998,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85299999999999998,"15min":0.85199999999999998},"min":{"1min":0.63600000000000001,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.0600000000000001,"5min":1.286,"15min":1.286},"last":1.014},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.86599999999999999},"min":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.57399999999999995},"max":{"1min":1.0940000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.94699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575648,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426300,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103104,"kb_avail":93945540,"statfs":{"total":96636764160,"available":96200232960,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105578496},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.434},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.69399999999999995}]},{"osd":4,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85499999999999998,"15min":0.81299999999999994},"min":{"1min":0.69899999999999995,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.2130000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.83899999999999997,"15min":0.81499999999999995},"min":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85299999999999998,"15min":0.84199999999999997},"min":{"1min":0.63800000000000001,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.409,"15min":1.409},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89600000000000002,"15min":0.89100000000000001},"min":{"1min":0.72599999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2250000000000001,"5min":1.452,"15min":1.452},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.80700000000000005},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.099,"5min":1.1180000000000001,"15min":1.198},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.70499999999999996,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.113,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":1.038,"5min":1.371,"15min":1.371},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.163,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.78100000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575647,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418196,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125440,"kb_avail":93953644,"statfs":{"total":96636764160,"available":96208531456,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128450560},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":1.266,"15min":1.419},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.78900000000000003},"min":{"1min":0.55800000000000005,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":1.294,"15min":1.383},"last":0.92900000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.621,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.242,"15min":1.242},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81699999999999995,"15min":0.79400000000000004},"min":{"1min":0.68999999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":0.97899999999999998,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.97899999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.39700000000000002},"max":{"1min":1.0409999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.441},"max":{"1min":0.96299999999999997,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.70599999999999996},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":0.96099999999999997,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.85999999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":0.92500000000000004,"15min":1.05},"last":0.75}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.131},"last":0.91100000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.624,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.873},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71099999999999997,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.83299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575647,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85099999999999998,"5min":1.579,"15min":1.579},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.61499999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.92400000000000004,"5min":1.391,"15min":1.391},"last":0.76300000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.74399999999999999},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.169,"15min":1.254},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81599999999999995,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.94799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":1.522,"15min":1.522},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":1.28,"15min":1.28},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68500000000000005,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.504},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.86199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.877,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.74399999999999999,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91200000000000003,"5min":1.226,"15min":1.238},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:17:11.861 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:11.862 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:11.863 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:11.863 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:17:30.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:30.583 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:17:31.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:17:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.071+0000 7fde791ff700 1 Processor -- start 2023-06-09T12:17:32.075 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.071+0000 7fde791ff700 1 -- start start 2023-06-09T12:17:32.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 0x7fde740c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 0x7fde74070b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde74071060 0x7fde74158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde740714e0 con 0x7fde74071060 2023-06-09T12:17:32.077 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde741518e0 con 0x7fde7415b380 2023-06-09T12:17:32.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde791ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde741580a0 con 0x7fde74151510 2023-06-09T12:17:32.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde74071060 0x7fde74158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fde74071060 0x7fde74158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42926/0 (socket says 172.21.15.7:42926) 2023-06-09T12:17:32.078 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 -- 172.21.15.7:0/1673742921 learned_addr learned my addr 172.21.15.7:0/1673742921 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:17:32.079 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde72d9d700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 0x7fde740c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7259c700 1 --2- 172.21.15.7:0/1673742921 >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 0x7fde74070b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 -- 172.21.15.7:0/1673742921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 msgr2=0x7fde740c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:17:32.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 0x7fde740c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.080 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 -- 172.21.15.7:0/1673742921 >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 msgr2=0x7fde74070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:17:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 --2- 172.21.15.7:0/1673742921 >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 0x7fde74070b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7359e700 1 -- 172.21.15.7:0/1673742921 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740c6180 con 0x7fde74071060 2023-06-09T12:17:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde72d9d700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 0x7fde740c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:17:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.073+0000 7fde7259c700 1 --2- 172.21.15.7:0/1673742921 >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 0x7fde74070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:17:32.081 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde7359e700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74158870 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fde740cc290 tx=0x7fde6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e701b9c2a8afb9 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:32.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde71d9b700 1 -- 172.21.15.7:0/1673742921 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00e070 con 0x7fde74071060 2023-06-09T12:17:32.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde71d9b700 1 -- 172.21.15.7:0/1673742921 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6c00b9c0 con 0x7fde74071060 2023-06-09T12:17:32.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde71d9b700 1 -- 172.21.15.7:0/1673742921 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00bd70 con 0x7fde74071060 2023-06-09T12:17:32.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde791ff700 1 -- 172.21.15.7:0/1673742921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 msgr2=0x7fde74158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:32.082 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.074+0000 7fde791ff700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74158870 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fde740cc290 tx=0x7fde6c00b330 comp rx=0 tx=0).stop 2023-06-09T12:17:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 -- 172.21.15.7:0/1673742921 shutdown_connections 2023-06-09T12:17:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74158870 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 --2- 172.21.15.7:0/1673742921 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde74151510 0x7fde740c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 --2- 172.21.15.7:0/1673742921 >> v2:172.21.15.117:3300/0 conn(0x7fde7415b380 0x7fde74070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.083 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 -- 172.21.15.7:0/1673742921 >> 172.21.15.7:0/1673742921 conn(0x7fde7406a4d0 msgr2=0x7fde7406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 -- 172.21.15.7:0/1673742921 shutdown_connections 2023-06-09T12:17:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.075+0000 7fde791ff700 1 -- 172.21.15.7:0/1673742921 wait complete. 2023-06-09T12:17:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.076+0000 7fde791ff700 1 Processor -- start 2023-06-09T12:17:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.077+0000 7fde791ff700 1 -- start start 2023-06-09T12:17:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.077+0000 7fde791ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde791ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde74151510 0x7fde740c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde791ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde7415b380 0x7fde740c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde791ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7414e3a0 con 0x7fde74071060 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde791ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde740cbab0 con 0x7fde74151510 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde791ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde740cb7b0 con 0x7fde7415b380 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.087 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42934/0 (socket says 172.21.15.7:42934) 2023-06-09T12:17:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 -- 172.21.15.7:0/641893563 learned_addr learned my addr 172.21.15.7:0/641893563 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:17:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde7259c700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde74151510 0x7fde740c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde7359e700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde7415b380 0x7fde740c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 -- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde7415b380 msgr2=0x7fde740c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:32.088 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde7415b380 0x7fde740c94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 -- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde74151510 msgr2=0x7fde740c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde74151510 0x7fde740c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 -- 172.21.15.7:0/641893563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6c009b30 con 0x7fde74071060 2023-06-09T12:17:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.078+0000 7fde72d9d700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fde6400c740 tx=0x7fde6400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:32.089 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.079+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64011800 con 0x7fde74071060 2023-06-09T12:17:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.079+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde64011960 con 0x7fde74071060 2023-06-09T12:17:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.079+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde64011cd0 con 0x7fde74071060 2023-06-09T12:17:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.079+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde74157770 con 0x7fde74071060 2023-06-09T12:17:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.079+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde74157490 con 0x7fde74071060 2023-06-09T12:17:32.090 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.080+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde74157a50 con 0x7fde74071060 2023-06-09T12:17:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.081+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fde64021350 con 0x7fde74071060 2023-06-09T12:17:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.082+0000 7fde5b7fe700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.082+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fde64089d70 con 0x7fde74071060 2023-06-09T12:17:32.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.082+0000 7fde7259c700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.083+0000 7fde7259c700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 0x7fde6006d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde68006fd0 tx=0x7fde68008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:32.092 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.087+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fde640544b0 con 0x7fde74071060 2023-06-09T12:17:32.297 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.294+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde740cb920 con 0x7fde6006ace0 2023-06-09T12:17:32.306 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.303+0000 7fde5b7fe700 1 -- 172.21.15.7:0/641893563 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319371 (secure 0 0 0) 0x7fde740cb920 con 0x7fde6006ace0 2023-06-09T12:17:32.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:17:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.307+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 msgr2=0x7fde6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:32.312 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.307+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 0x7fde6006d190 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fde68006fd0 tx=0x7fde68008040 comp rx=0 tx=0).stop 2023-06-09T12:17:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.307+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 msgr2=0x7fde74155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.307+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fde6400c740 tx=0x7fde6400cb00 comp rx=0 tx=0).stop 2023-06-09T12:17:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 shutdown_connections 2023-06-09T12:17:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fde6006ace0 0x7fde6006d190 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.313 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fde7415b380 0x7fde740c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fde74071060 0x7fde74155dd0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 --2- 172.21.15.7:0/641893563 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fde74151510 0x7fde740c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 >> 172.21.15.7:0/641893563 conn(0x7fde7406a4d0 msgr2=0x7fde7414d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 shutdown_connections 2023-06-09T12:17:32.314 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:32.308+0000 7fde791ff700 1 -- 172.21.15.7:0/641893563 wait complete. 2023-06-09T12:17:32.315 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:17:32.329 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2072,"stamp":"2023-06-09T12:17:31.224010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104784,"ondisk_log_size":104784,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594056,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":710720,"kb_avail":752380664,"statfs":{"total":773094113280,"available":770437799936,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":727777280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003338"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3705,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.577293+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:13:32.577293+0000","last_peered":"2023-06-09T12:13:32.577293+0000","last_clean":"2023-06-09T12:13:32.577293+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:13:32.577293+0000","last_undegraded":"2023-06-09T12:13:32.577293+0000","last_fullsized":"2023-06-09T12:13:32.577293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3862,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:37.401030+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:12:37.401030+0000","last_peered":"2023-06-09T12:12:37.401030+0000","last_clean":"2023-06-09T12:12:37.401030+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:12:37.401030+0000","last_undegraded":"2023-06-09T12:12:37.401030+0000","last_fullsized":"2023-06-09T12:12:37.401030+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3974,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:47.409868+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:12:47.409868+0000","last_peered":"2023-06-09T12:12:47.409868+0000","last_clean":"2023-06-09T12:12:47.409868+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:12:47.409868+0000","last_undegraded":"2023-06-09T12:12:47.409868+0000","last_fullsized":"2023-06-09T12:12:47.409868+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:12.486464+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:13:12.486464+0000","last_peered":"2023-06-09T12:13:12.486464+0000","last_clean":"2023-06-09T12:13:12.486464+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:13:12.486464+0000","last_undegraded":"2023-06-09T12:13:12.486464+0000","last_fullsized":"2023-06-09T12:13:12.486464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3263,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387955+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:13:17.387955+0000","last_peered":"2023-06-09T12:13:17.387955+0000","last_clean":"2023-06-09T12:13:17.387955+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:13:17.387955+0000","last_undegraded":"2023-06-09T12:13:17.387955+0000","last_fullsized":"2023-06-09T12:13:17.387955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3913,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414514+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:13:22.414514+0000","last_peered":"2023-06-09T12:13:22.414514+0000","last_clean":"2023-06-09T12:13:22.414514+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:13:22.414514+0000","last_undegraded":"2023-06-09T12:13:22.414514+0000","last_fullsized":"2023-06-09T12:13:22.414514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2270,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:52.384703+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:12:52.384703+0000","last_peered":"2023-06-09T12:12:52.384703+0000","last_clean":"2023-06-09T12:12:52.384703+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:12:52.384703+0000","last_undegraded":"2023-06-09T12:12:52.384703+0000","last_fullsized":"2023-06-09T12:12:52.384703+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1569,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:42.409179+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:12:42.409179+0000","last_peered":"2023-06-09T12:12:42.409179+0000","last_clean":"2023-06-09T12:12:42.409179+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:12:42.409179+0000","last_undegraded":"2023-06-09T12:12:42.409179+0000","last_fullsized":"2023-06-09T12:12:42.409179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3548,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:56.334647+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:12:56.334647+0000","last_peered":"2023-06-09T12:12:56.334647+0000","last_clean":"2023-06-09T12:12:56.334647+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:12:56.334647+0000","last_undegraded":"2023-06-09T12:12:56.334647+0000","last_fullsized":"2023-06-09T12:12:56.334647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1119,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:12:57.484792+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:12:57.484792+0000","last_peered":"2023-06-09T12:12:57.484792+0000","last_clean":"2023-06-09T12:12:57.484792+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:12:57.484792+0000","last_undegraded":"2023-06-09T12:12:57.484792+0000","last_fullsized":"2023-06-09T12:12:57.484792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1071","reported_seq":3955,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:17:26.779869+0000","last_change":"2023-06-09T12:17:12.637912+0000","last_active":"2023-06-09T12:17:26.779869+0000","last_peered":"2023-06-09T12:17:26.779869+0000","last_clean":"2023-06-09T12:17:26.779869+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:26.779869+0000","last_undegraded":"2023-06-09T12:17:26.779869+0000","last_fullsized":"2023-06-09T12:17:26.779869+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":96,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 14s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4141,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:17.387968+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:13:17.387968+0000","last_peered":"2023-06-09T12:13:17.387968+0000","last_clean":"2023-06-09T12:13:17.387968+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:13:17.387968+0000","last_undegraded":"2023-06-09T12:13:17.387968+0000","last_fullsized":"2023-06-09T12:13:17.387968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.498887+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:52.498887+0000","last_peered":"2023-06-09T12:14:52.498887+0000","last_clean":"2023-06-09T12:14:52.498887+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:52.498887+0000","last_undegraded":"2023-06-09T12:14:52.498887+0000","last_fullsized":"2023-06-09T12:14:52.498887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3035,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.733972+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.733972+0000","last_peered":"2023-06-09T12:15:28.733972+0000","last_clean":"2023-06-09T12:15:28.733972+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.733972+0000","last_undegraded":"2023-06-09T12:15:28.733972+0000","last_fullsized":"2023-06-09T12:15:28.733972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.487601+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:13:22.487601+0000","last_peered":"2023-06-09T12:13:22.487601+0000","last_clean":"2023-06-09T12:13:22.487601+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:13:22.487601+0000","last_undegraded":"2023-06-09T12:13:22.487601+0000","last_fullsized":"2023-06-09T12:13:22.487601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3954,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:17:07.774691+0000","last_change":"2023-06-09T12:16:42.577379+0000","last_active":"2023-06-09T12:17:07.774691+0000","last_peered":"2023-06-09T12:17:07.774691+0000","last_clean":"2023-06-09T12:17:07.774691+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:17:07.774691+0000","last_undegraded":"2023-06-09T12:17:07.774691+0000","last_fullsized":"2023-06-09T12:17:07.774691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73660,"ondisk_log_size":73660,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575652,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.749,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.55700000000000005,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.751},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.754},"min":{"1min":0.497,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.28,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.628},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.006,"5min":1.079,"15min":1.079},"last":0.76400000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.33500000000000002},"max":{"1min":0.80400000000000005,"5min":1.591,"15min":1.591},"last":0.436},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.56100000000000005,"5min":0.55900000000000005,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.619,"15min":1.619},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.41899999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.48999999999999999,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.747},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.085,"5min":1.492,"15min":1.492},"last":0.60099999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575652,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":1.044,"15min":1.044},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.94099999999999995,"5min":1.266,"15min":1.266},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":1.298,"15min":1.298},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.76700000000000002},"min":{"1min":0.57599999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.81599999999999995,"5min":1.091,"15min":1.091},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.86499999999999999,"5min":1.0209999999999999,"15min":1.046},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.878,"5min":1.077,"15min":1.1259999999999999},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.71899999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.443,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.76700000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.873,"15min":0.98399999999999999},"last":0.495},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.745},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.41899999999999998},"max":{"1min":0.91100000000000003,"5min":1.161,"15min":1.161},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42399999999999999},"max":{"1min":0.86799999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.79300000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575653,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405552,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99136,"kb_avail":93966288,"statfs":{"total":96636764160,"available":96221478912,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101515264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.439,"15min":0.45300000000000001},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.441},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.44800000000000001,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.71799999999999997,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.499},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65400000000000003,"5min":0.71699999999999997,"15min":0.995},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.41999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":1.2450000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.57799999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.753}]},{"osd":4,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89300000000000002,"15min":0.82899999999999996},"min":{"1min":0.747,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.1419999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78700000000000003,"15min":0.83799999999999997},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0229999999999999,"5min":1.175,"15min":1.597},"last":0.92900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.55500000000000005},"max":{"1min":1.097,"5min":1.196,"15min":1.544},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.81699999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":0.91800000000000004,"5min":1.1910000000000001,"15min":1.548},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.70999999999999996,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1779999999999999,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.044,"5min":1.24,"15min":1.498},"last":0.86499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575652,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.51700000000000002,"15min":0.48699999999999999},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.745,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.496},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56899999999999995,"15min":0.55500000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.70399999999999996,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.433,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64600000000000002,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.52200000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.62,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81599999999999995,"15min":0.96199999999999997},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0149999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.70599999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.86599999999999999,"15min":0.84899999999999998},"min":{"1min":0.64100000000000001,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.131,"5min":1.421,"15min":1.421},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.878,"15min":0.88600000000000001},"min":{"1min":0.69699999999999995,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.0760000000000001,"5min":1.2490000000000001,"15min":1.268},"last":0.88500000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.64600000000000002,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":0.97499999999999998,"5min":1.077,"15min":1.121},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89300000000000002,"15min":0.89100000000000001},"min":{"1min":0.69999999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.093,"5min":1.371,"15min":3.5840000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83699999999999997,"15min":0.81399999999999995},"min":{"1min":0.61299999999999999,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1180000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82299999999999995,"15min":0.83699999999999997},"min":{"1min":0.60599999999999998,"5min":0.495,"15min":0.495},"max":{"1min":1.069,"5min":1.113,"15min":1.232},"last":0.85099999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.879,"15min":0.876},"min":{"1min":0.621,"5min":0.53400000000000003,"15min":0.48299999999999998},"max":{"1min":1.103,"5min":1.1299999999999999,"15min":1.298},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.0009999999999999,"5min":0.96899999999999997,"15min":0.95899999999999996},"min":{"1min":0.85999999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1539999999999999,"5min":1.371,"15min":1.371},"last":0.93899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575653,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.621,"15min":0.80600000000000005},"last":0.439},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":1.236},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.624},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.2969999999999999},"last":0.92200000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.86599999999999999,"5min":1.298,"15min":1.4650000000000001},"last":0.70999999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.90200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":1.337,"15min":1.4359999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.86699999999999999,"15min":0.88},"min":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.073,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":1.016}]},{"osd":6,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85699999999999998,"15min":0.82299999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54300000000000004},"max":{"1min":0.998,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.874}]},{"osd":7,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85299999999999998,"15min":0.85199999999999998},"min":{"1min":0.63600000000000001,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.0600000000000001,"5min":1.286,"15min":1.286},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.86599999999999999},"min":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.57399999999999995},"max":{"1min":1.0940000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.96599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575652,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426428,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103232,"kb_avail":93945412,"statfs":{"total":96636764160,"available":96200101888,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105709568},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.66500000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53500000000000003,"15min":0.53200000000000003},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.71599999999999997,"5min":0.81499999999999995,"15min":1.0600000000000001},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.629,"15min":0.64500000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.77200000000000002,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.85499999999999998,"15min":0.81299999999999994},"min":{"1min":0.69899999999999995,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.2130000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.83899999999999997,"15min":0.81499999999999995},"min":{"1min":0.64600000000000002,"5min":0.59999999999999998,"15min":0.52500000000000002},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.85299999999999998,"15min":0.84199999999999997},"min":{"1min":0.63800000000000001,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0680000000000001,"5min":1.409,"15min":1.409},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.89600000000000002,"15min":0.89100000000000001},"min":{"1min":0.72599999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2250000000000001,"5min":1.452,"15min":1.452},"last":0.89500000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.81000000000000005,"15min":0.80700000000000005},"min":{"1min":0.61099999999999999,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.099,"5min":1.1180000000000001,"15min":1.198},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.82199999999999995,"15min":0.82399999999999995},"min":{"1min":0.70499999999999996,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.113,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.77700000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:36 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.60599999999999998,"5min":0.58899999999999997,"15min":0.57299999999999995},"max":{"1min":1.038,"5min":1.371,"15min":1.371},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.84999999999999998,"15min":0.85199999999999998},"min":{"1min":0.71799999999999997,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.163,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.79500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575651,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418324,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125568,"kb_avail":93953516,"statfs":{"total":96636764160,"available":96208400384,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128581632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.78400000000000003,"15min":0.748},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90600000000000003,"5min":1.266,"15min":1.419},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81399999999999995,"15min":0.78900000000000003},"min":{"1min":0.55800000000000005,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":0.98099999999999998,"5min":1.294,"15min":1.383},"last":0.70599999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.76200000000000001,"15min":0.74099999999999999},"min":{"1min":0.621,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":0.90700000000000003,"5min":1.242,"15min":1.242},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.81699999999999995,"15min":0.79400000000000004},"min":{"1min":0.68999999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":0.97899999999999998,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.77100000000000002,"15min":0.73499999999999999},"min":{"1min":0.53500000000000003,"5min":0.50600000000000001,"15min":0.39700000000000002},"max":{"1min":1.0409999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.441},"max":{"1min":0.96299999999999997,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.749,"15min":0.70599999999999996},"min":{"1min":0.61399999999999999,"5min":0.54500000000000004,"15min":0.46899999999999997},"max":{"1min":1.0640000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.77500000000000002,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.54600000000000004,"15min":0.48299999999999998},"max":{"1min":0.96099999999999997,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.72999999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.55900000000000005,"15min":0.54700000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.745,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.378},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.622,"15min":0.621},"min":{"1min":0.41999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":0.92500000000000004,"15min":1.05},"last":0.41499999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.59399999999999997,"15min":0.58399999999999996},"min":{"1min":0.442,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85799999999999998,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.434},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.56599999999999995,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.996,"5min":1.0229999999999999,"15min":1.131},"last":0.627}]},{"osd":7,"last update":"Fri Jun 9 12:16:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.624,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.496},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71099999999999997,"15min":0.64900000000000002},"min":{"1min":0.495,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":0.90500000000000003,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.51800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575651,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85099999999999998,"5min":1.579,"15min":1.579},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.625}]},{"osd":2,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.92400000000000004,"5min":1.391,"15min":1.391},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.74399999999999999},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.169,"15min":1.254},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81599999999999995,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.94799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.80600000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":1.522,"15min":1.522},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":1.28,"15min":1.28},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68500000000000005,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.86199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.85599999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.877,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.74399999999999999,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91200000000000003,"5min":1.226,"15min":1.238},"last":0.82099999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:17:32.337 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:32.338 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:32.338 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:17:52.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:17:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.548+0000 7fb06fdff700 1 Processor -- start 2023-06-09T12:17:52.551 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.548+0000 7fb06fdff700 1 -- start start 2023-06-09T12:17:52.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06fdff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0680cc5b0 0x7fb0680b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:52.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06fdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb068159b20 con 0x7fb0680cb9a0 2023-06-09T12:17:52.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06fdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680cac80 con 0x7fb0680cb5d0 2023-06-09T12:17:52.552 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06fdff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0681597d0 con 0x7fb0680cc5b0 2023-06-09T12:17:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06e39c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fb0680cc5b0 0x7fb0680b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.549+0000 7fb06d39a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb0680cb9a0 0x7fb0680cbea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35526/0 (socket says 172.21.15.7:35526) 2023-06-09T12:17:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06d39a700 1 -- 172.21.15.7:0/3820527414 learned_addr learned my addr 172.21.15.7:0/3820527414 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:17:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06e39c700 1 -- 172.21.15.7:0/3820527414 >> v1:172.21.15.117:6789/0 conn(0x7fb0680cb5d0 legacy=0x7fb068159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:17:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06e39c700 1 -- 172.21.15.7:0/3820527414 >> v1:172.21.15.7:6789/0 conn(0x7fb0680cb9a0 legacy=0x7fb0680cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06e39c700 1 -- 172.21.15.7:0/3820527414 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb068071ab0 con 0x7fb0680cc5b0 2023-06-09T12:17:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06cb99700 1 -- 172.21.15.7:0/3820527414 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211518201 0 0) 0x7fb068159b20 con 0x7fb0680cb9a0 2023-06-09T12:17:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06e39c700 1 --2- 172.21.15.7:0/3820527414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cc5b0 0x7fb0680b52b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb060009d90 tx=0x7fb06000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b394cdef0e67997b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:52.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.550+0000 7fb06cb99700 1 -- 172.21.15.7:0/3820527414 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000e070 con 0x7fb0680cc5b0 2023-06-09T12:17:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06cb99700 1 -- 172.21.15.7:0/3820527414 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06000bb90 con 0x7fb0680cc5b0 2023-06-09T12:17:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06cb99700 1 -- 172.21.15.7:0/3820527414 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060003a20 con 0x7fb0680cc5b0 2023-06-09T12:17:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 -- 172.21.15.7:0/3820527414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cc5b0 msgr2=0x7fb0680b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3820527414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cc5b0 0x7fb0680b52b0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fb060009d90 tx=0x7fb06000b330 comp rx=0 tx=0).stop 2023-06-09T12:17:52.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 -- 172.21.15.7:0/3820527414 shutdown_connections 2023-06-09T12:17:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3820527414 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cc5b0 0x7fb0680b52b0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 -- 172.21.15.7:0/3820527414 >> 172.21.15.7:0/3820527414 conn(0x7fb068070af0 msgr2=0x7fb068070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 -- 172.21.15.7:0/3820527414 shutdown_connections 2023-06-09T12:17:52.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.551+0000 7fb06fdff700 1 -- 172.21.15.7:0/3820527414 wait complete. 2023-06-09T12:17:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.553+0000 7fb06fdff700 1 Processor -- start 2023-06-09T12:17:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.553+0000 7fb06fdff700 1 -- start start 2023-06-09T12:17:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0680cb5d0 0x7fb0680cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:52.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0680cc5b0 0x7fb0681f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c8f80 con 0x7fb0680cb9a0 2023-06-09T12:17:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680ce550 con 0x7fb0680cb5d0 2023-06-09T12:17:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06fdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680c8740 con 0x7fb0680cc5b0 2023-06-09T12:17:52.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06d39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34828/0 (socket says 172.21.15.7:34828) 2023-06-09T12:17:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06d39a700 1 -- 172.21.15.7:0/3313917815 learned_addr learned my addr 172.21.15.7:0/3313917815 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:17:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06db9b700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0680cb5d0 0x7fb0680cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.554+0000 7fb06e39c700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0680cc5b0 0x7fb0681f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:52.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 -- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0680cc5b0 msgr2=0x7fb0681f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0680cc5b0 0x7fb0681f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 -- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0680cb5d0 msgr2=0x7fb0680cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0680cb5d0 0x7fb0680cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 -- 172.21.15.7:0/3313917815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060009a70 con 0x7fb0680cb9a0 2023-06-09T12:17:52.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06d39a700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c570 tx=0x7fb05800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058004c80 con 0x7fb0680cb9a0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb058004e00 con 0x7fb0680cb9a0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058007310 con 0x7fb0680cb9a0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06815d4f0 con 0x7fb0680cb9a0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.555+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06815d210 con 0x7fb0680cb9a0 2023-06-09T12:17:52.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.556+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb06815d7d0 con 0x7fb0680cb9a0 2023-06-09T12:17:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.557+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb058007470 con 0x7fb0680cb9a0 2023-06-09T12:17:52.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.558+0000 7fb05e7fc700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 0x7fb04c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:17:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.558+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fb058089c00 con 0x7fb0680cb9a0 2023-06-09T12:17:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.559+0000 7fb06db9b700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 0x7fb04c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:17:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.559+0000 7fb06db9b700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 0x7fb04c06d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb064004f30 tx=0x7fb064009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:17:52.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.563+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fb0580542a0 con 0x7fb0680cb9a0 2023-06-09T12:17:52.771 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.769+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb0680cf040 con 0x7fb04c06ace0 2023-06-09T12:17:52.780 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.778+0000 7fb05e7fc700 1 -- 172.21.15.7:0/3313917815 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319426 (secure 0 0 0) 0x7fb0680cf040 con 0x7fb04c06ace0 2023-06-09T12:17:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:17:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 msgr2=0x7fb04c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 0x7fb04c06d190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fb064004f30 tx=0x7fb064009040 comp rx=0 tx=0).stop 2023-06-09T12:17:52.786 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 msgr2=0x7fb0681f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fb05800c570 tx=0x7fb05800c930 comp rx=0 tx=0).stop 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 shutdown_connections 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fb04c06ace0 0x7fb04c06d190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb0680cc5b0 0x7fb0681f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb0680cb5d0 0x7fb0680cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.787 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 --2- 172.21.15.7:0/3313917815 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb0680cb9a0 0x7fb0681f1bd0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:17:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 >> 172.21.15.7:0/3313917815 conn(0x7fb068070af0 msgr2=0x7fb068158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:17:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 shutdown_connections 2023-06-09T12:17:52.788 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:17:52.782+0000 7fb06fdff700 1 -- 172.21.15.7:0/3313917815 wait complete. 2023-06-09T12:17:52.789 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:17:52.804 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2082,"stamp":"2023-06-09T12:17:51.229370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104908,"ondisk_log_size":104908,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594312,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":710976,"kb_avail":752380408,"statfs":{"total":773094113280,"available":770437537792,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728039424},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003316"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.591494+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:15:32.591494+0000","last_peered":"2023-06-09T12:15:32.591494+0000","last_clean":"2023-06-09T12:15:32.591494+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:15:32.591494+0000","last_undegraded":"2023-06-09T12:15:32.591494+0000","last_fullsized":"2023-06-09T12:15:32.591494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3863,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.420832+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:15:07.420832+0000","last_peered":"2023-06-09T12:15:07.420832+0000","last_clean":"2023-06-09T12:15:07.420832+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:15:07.420832+0000","last_undegraded":"2023-06-09T12:15:07.420832+0000","last_fullsized":"2023-06-09T12:15:07.420832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3610,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.511758+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:13:07.511758+0000","last_peered":"2023-06-09T12:13:07.511758+0000","last_clean":"2023-06-09T12:13:07.511758+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:13:07.511758+0000","last_undegraded":"2023-06-09T12:13:07.511758+0000","last_fullsized":"2023-06-09T12:13:07.511758+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3975,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.428702+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:15:12.428702+0000","last_peered":"2023-06-09T12:15:12.428702+0000","last_clean":"2023-06-09T12:15:12.428702+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:15:12.428702+0000","last_undegraded":"2023-06-09T12:15:12.428702+0000","last_fullsized":"2023-06-09T12:15:12.428702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3391,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515099+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:13:32.515099+0000","last_peered":"2023-06-09T12:13:32.515099+0000","last_clean":"2023-06-09T12:13:32.515099+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:13:32.515099+0000","last_undegraded":"2023-06-09T12:13:32.515099+0000","last_fullsized":"2023-06-09T12:13:32.515099+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503208+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:15:27.503208+0000","last_peered":"2023-06-09T12:15:27.503208+0000","last_clean":"2023-06-09T12:15:27.503208+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:15:27.503208+0000","last_undegraded":"2023-06-09T12:15:27.503208+0000","last_fullsized":"2023-06-09T12:15:27.503208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404456+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:15:27.404456+0000","last_peered":"2023-06-09T12:15:27.404456+0000","last_clean":"2023-06-09T12:15:27.404456+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:15:27.404456+0000","last_undegraded":"2023-06-09T12:15:27.404456+0000","last_fullsized":"2023-06-09T12:15:27.404456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3914,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.431434+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:15:32.431434+0000","last_peered":"2023-06-09T12:15:32.431434+0000","last_clean":"2023-06-09T12:15:32.431434+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:15:32.431434+0000","last_undegraded":"2023-06-09T12:15:32.431434+0000","last_fullsized":"2023-06-09T12:15:32.431434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3584,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.397003+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:13:07.397003+0000","last_peered":"2023-06-09T12:13:07.397003+0000","last_clean":"2023-06-09T12:13:07.397003+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:13:07.397003+0000","last_undegraded":"2023-06-09T12:13:07.397003+0000","last_fullsized":"2023-06-09T12:13:07.397003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.403174+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:15:17.403174+0000","last_peered":"2023-06-09T12:15:17.403174+0000","last_clean":"2023-06-09T12:15:17.403174+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:15:17.403174+0000","last_undegraded":"2023-06-09T12:15:17.403174+0000","last_fullsized":"2023-06-09T12:15:17.403174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3230,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.777978+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.777978+0000","last_peered":"2023-06-09T12:15:43.777978+0000","last_clean":"2023-06-09T12:15:43.777978+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.777978+0000","last_undegraded":"2023-06-09T12:15:43.777978+0000","last_fullsized":"2023-06-09T12:15:43.777978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1570,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427884+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:15:07.427884+0000","last_peered":"2023-06-09T12:15:07.427884+0000","last_clean":"2023-06-09T12:15:07.427884+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:15:07.427884+0000","last_undegraded":"2023-06-09T12:15:07.427884+0000","last_fullsized":"2023-06-09T12:15:07.427884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3549,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.412993+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:15:12.412993+0000","last_peered":"2023-06-09T12:15:12.412993+0000","last_clean":"2023-06-09T12:15:12.412993+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:15:12.412993+0000","last_undegraded":"2023-06-09T12:15:12.412993+0000","last_fullsized":"2023-06-09T12:15:12.412993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3418,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.389236+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:13:27.389236+0000","last_peered":"2023-06-09T12:13:27.389236+0000","last_clean":"2023-06-09T12:13:27.389236+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:13:27.389236+0000","last_undegraded":"2023-06-09T12:13:27.389236+0000","last_fullsized":"2023-06-09T12:13:27.389236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1120,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.501981+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:15:17.501981+0000","last_peered":"2023-06-09T12:15:17.501981+0000","last_clean":"2023-06-09T12:15:17.501981+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:15:17.501981+0000","last_undegraded":"2023-06-09T12:15:17.501981+0000","last_fullsized":"2023-06-09T12:15:17.501981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3871,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:02.411803+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:13:02.411803+0000","last_peered":"2023-06-09T12:13:02.411803+0000","last_clean":"2023-06-09T12:13:02.411803+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:13:02.411803+0000","last_undegraded":"2023-06-09T12:13:02.411803+0000","last_fullsized":"2023-06-09T12:13:02.411803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3904,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.097526+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:13:57.097526+0000","last_peered":"2023-06-09T12:13:57.097526+0000","last_clean":"2023-06-09T12:13:57.097526+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:13:57.097526+0000","last_undegraded":"2023-06-09T12:13:57.097526+0000","last_fullsized":"2023-06-09T12:13:57.097526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3789,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.579743+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:13:52.579743+0000","last_peered":"2023-06-09T12:13:52.579743+0000","last_clean":"2023-06-09T12:13:52.579743+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:13:52.579743+0000","last_undegraded":"2023-06-09T12:13:52.579743+0000","last_fullsized":"2023-06-09T12:13:52.579743+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1384,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:27.399512+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:13:27.399512+0000","last_peered":"2023-06-09T12:13:27.399512+0000","last_clean":"2023-06-09T12:13:27.399512+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:13:27.399512+0000","last_undegraded":"2023-06-09T12:13:27.399512+0000","last_fullsized":"2023-06-09T12:13:27.399512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1195","reported_seq":4086,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:17:43.929451+0000","last_change":"2023-06-09T12:17:12.637912+0000","last_active":"2023-06-09T12:17:43.929451+0000","last_peered":"2023-06-09T12:17:43.929451+0000","last_clean":"2023-06-09T12:17:43.929451+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:43.929451+0000","last_undegraded":"2023-06-09T12:17:43.929451+0000","last_fullsized":"2023-06-09T12:17:43.929451+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":220,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 34s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.513721+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:13:22.513721+0000","last_peered":"2023-06-09T12:13:22.513721+0000","last_clean":"2023-06-09T12:13:22.513721+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:13:22.513721+0000","last_undegraded":"2023-06-09T12:13:22.513721+0000","last_fullsized":"2023-06-09T12:13:22.513721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3539,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.390492+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:13:37.390492+0000","last_peered":"2023-06-09T12:13:37.390492+0000","last_clean":"2023-06-09T12:13:37.390492+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:13:37.390492+0000","last_undegraded":"2023-06-09T12:13:37.390492+0000","last_fullsized":"2023-06-09T12:13:37.390492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1293,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.391971+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:13:52.391971+0000","last_peered":"2023-06-09T12:13:52.391971+0000","last_clean":"2023-06-09T12:13:52.391971+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:13:52.391971+0000","last_undegraded":"2023-06-09T12:13:52.391971+0000","last_fullsized":"2023-06-09T12:13:52.391971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404473+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:15:27.404473+0000","last_peered":"2023-06-09T12:15:27.404473+0000","last_clean":"2023-06-09T12:15:27.404473+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:15:27.404473+0000","last_undegraded":"2023-06-09T12:15:27.404473+0000","last_fullsized":"2023-06-09T12:15:27.404473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":323,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.491684+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:13:57.491684+0000","last_peered":"2023-06-09T12:13:57.491684+0000","last_clean":"2023-06-09T12:13:57.491684+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:13:57.491684+0000","last_undegraded":"2023-06-09T12:13:57.491684+0000","last_fullsized":"2023-06-09T12:13:57.491684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3594,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581084+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:14:02.581084+0000","last_peered":"2023-06-09T12:14:02.581084+0000","last_clean":"2023-06-09T12:14:02.581084+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:14:02.581084+0000","last_undegraded":"2023-06-09T12:14:02.581084+0000","last_fullsized":"2023-06-09T12:14:02.581084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.498887+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:52.498887+0000","last_peered":"2023-06-09T12:14:52.498887+0000","last_clean":"2023-06-09T12:14:52.498887+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:52.498887+0000","last_undegraded":"2023-06-09T12:14:52.498887+0000","last_fullsized":"2023-06-09T12:14:52.498887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.734088+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.734088+0000","last_peered":"2023-06-09T12:15:28.734088+0000","last_clean":"2023-06-09T12:15:28.734088+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.734088+0000","last_undegraded":"2023-06-09T12:15:28.734088+0000","last_fullsized":"2023-06-09T12:15:28.734088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3423,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.409110+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:13:37.409110+0000","last_peered":"2023-06-09T12:13:37.409110+0000","last_clean":"2023-06-09T12:13:37.409110+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:13:37.409110+0000","last_undegraded":"2023-06-09T12:13:37.409110+0000","last_fullsized":"2023-06-09T12:13:37.409110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503243+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:15:27.503243+0000","last_peered":"2023-06-09T12:15:27.503243+0000","last_clean":"2023-06-09T12:15:27.503243+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:15:27.503243+0000","last_undegraded":"2023-06-09T12:15:27.503243+0000","last_fullsized":"2023-06-09T12:15:27.503243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":854,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:47.391536+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:13:47.391536+0000","last_peered":"2023-06-09T12:13:47.391536+0000","last_clean":"2023-06-09T12:13:47.391536+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:13:47.391536+0000","last_undegraded":"2023-06-09T12:13:47.391536+0000","last_fullsized":"2023-06-09T12:13:47.391536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4818,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:07.455374+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:13:07.455374+0000","last_peered":"2023-06-09T12:13:07.455374+0000","last_clean":"2023-06-09T12:13:07.455374+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:13:07.455374+0000","last_undegraded":"2023-06-09T12:13:07.455374+0000","last_fullsized":"2023-06-09T12:13:07.455374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.203901+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:13:37.203901+0000","last_peered":"2023-06-09T12:13:37.203901+0000","last_clean":"2023-06-09T12:13:37.203901+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:13:37.203901+0000","last_undegraded":"2023-06-09T12:13:37.203901+0000","last_fullsized":"2023-06-09T12:13:37.203901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'982","reported_seq":3955,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:10.765072+0000","last_change":"2023-06-09T12:17:32.431626+0000","last_active":"2023-06-09T12:17:10.765072+0000","last_peered":"2023-06-09T12:17:10.765072+0000","last_clean":"2023-06-09T12:17:10.765072+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:17:10.765072+0000","last_undegraded":"2023-06-09T12:17:10.765072+0000","last_fullsized":"2023-06-09T12:17:10.765072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":0,"log_size":982,"log_dups_size":0,"ondisk_log_size":982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3662,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.516677+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:13:42.516677+0000","last_peered":"2023-06-09T12:13:42.516677+0000","last_clean":"2023-06-09T12:13:42.516677+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:13:42.516677+0000","last_undegraded":"2023-06-09T12:13:42.516677+0000","last_fullsized":"2023-06-09T12:13:42.516677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3255,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:11.286353+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:14:11.286353+0000","last_peered":"2023-06-09T12:14:11.286353+0000","last_clean":"2023-06-09T12:14:11.286353+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:14:11.286353+0000","last_undegraded":"2023-06-09T12:14:11.286353+0000","last_fullsized":"2023-06-09T12:14:11.286353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73784,"ondisk_log_size":73784,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575656,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.33700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.749,"15min":0.72899999999999998},"min":{"1min":0.57099999999999995,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.55700000000000005,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.98599999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.75700000000000001},"min":{"1min":0.55000000000000004,"5min":0.499,"15min":0.48399999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.81699999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.93000000000000005,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.626},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.754},"min":{"1min":0.497,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.28,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.82799999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.006,"5min":1.079,"15min":1.079},"last":0.78700000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57599999999999996,"15min":0.55100000000000005},"min":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.33500000000000002},"max":{"1min":0.80400000000000005,"5min":1.591,"15min":1.591},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.75600000000000001,"15min":0.73999999999999999},"min":{"1min":0.56100000000000005,"5min":0.55900000000000005,"15min":0.47799999999999998},"max":{"1min":1.0580000000000001,"5min":1.619,"15min":1.619},"last":0.80500000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.622,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.753,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64600000000000002,"15min":0.625},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94799999999999995,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.70999999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:17:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68500000000000005,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.48999999999999999,"15min":0.44400000000000001},"max":{"1min":1.026,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.77700000000000002,"15min":0.747},"min":{"1min":0.53600000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.085,"5min":1.492,"15min":1.492},"last":0.77300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575656,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.47599999999999998,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.84599999999999997,"5min":1.044,"15min":1.044},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.48199999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.83199999999999996,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58099999999999996}]},{"osd":1,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.56299999999999994,"5min":0.47799999999999998,"15min":0.41899999999999998},"max":{"1min":0.94099999999999995,"5min":1.266,"15min":1.266},"last":0.496},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.59899999999999998,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89600000000000002,"5min":1.298,"15min":1.298},"last":0.56999999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.71999999999999997},"min":{"1min":0.45300000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76500000000000001,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76400000000000001,"15min":0.76700000000000002},"min":{"1min":0.57599999999999996,"5min":0.501,"15min":0.501},"max":{"1min":0.81599999999999995,"5min":1.091,"15min":1.091},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.54300000000000004,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.86499999999999999,"5min":1.0209999999999999,"15min":1.046},"last":0.753},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79400000000000004,"15min":0.80200000000000005},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.878,"5min":1.077,"15min":1.1259999999999999},"last":0.71499999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.40400000000000003,"15min":0.40100000000000002},"max":{"1min":0.71899999999999997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60599999999999998,"15min":0.59899999999999998},"min":{"1min":0.45300000000000001,"5min":0.443,"15min":0.371},"max":{"1min":0.78100000000000003,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60499999999999998,"15min":0.61599999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.87,"5min":0.873,"15min":0.98399999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.77100000000000002,"15min":0.745},"min":{"1min":0.626,"5min":0.56599999999999995,"15min":0.41899999999999998},"max":{"1min":0.91100000000000003,"5min":1.161,"15min":1.161},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42399999999999999},"max":{"1min":0.86799999999999999,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.63}]}]},{"osd":1,"up_from":13,"seq":55834575657,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405552,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99136,"kb_avail":93966288,"statfs":{"total":96636764160,"available":96221478912,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101515264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.439,"15min":0.45300000000000001},"min":{"1min":0.34899999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.63700000000000001,"15min":1.4470000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.44800000000000001,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.71799999999999997,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.495,"15min":0.499},"min":{"1min":0.39000000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.65400000000000003,"5min":0.71699999999999997,"15min":0.995},"last":0.441},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53800000000000003,"15min":0.54400000000000004},"min":{"1min":0.39700000000000002,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.628}]},{"osd":3,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.41999999999999998},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":1.2450000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70299999999999996,"15min":0.69799999999999995},"min":{"1min":0.57799999999999996,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.94199999999999995,"5min":0.94599999999999995,"15min":1.288},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.89300000000000002,"15min":0.82899999999999996},"min":{"1min":0.747,"5min":0.53800000000000003,"15min":0.48599999999999999},"max":{"1min":1.1419999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78700000000000003,"15min":0.83799999999999997},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0229999999999999,"5min":1.175,"15min":1.597},"last":0.80900000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.55500000000000005},"max":{"1min":1.097,"5min":1.196,"15min":1.544},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.749}]},{"osd":6,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":0.91800000000000004,"5min":1.1910000000000001,"15min":1.548},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.70999999999999996,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.1559999999999999,"15min":1.2869999999999999},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:17:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.80000000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1779999999999999,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.044,"5min":1.24,"15min":1.498},"last":0.97299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575656,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.50900000000000001,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.65700000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.50700000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68899999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96199999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82799999999999996,"5min":1.0149999999999999,"15min":1.0740000000000001},"last":0.629}]},{"osd":4,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86299999999999999,"15min":0.84799999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.421},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.77700000000000002,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.087,"5min":1.1830000000000001,"15min":1.268},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.012,"5min":1.077,"15min":1.121},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.622,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.198,"5min":1.198,"15min":3.5840000000000001},"last":1.0580000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.81499999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.47699999999999998},"max":{"1min":1.1140000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.0720000000000001,"5min":1.113,"15min":1.232},"last":1.028}]},{"osd":7,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.877},"min":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.48299999999999998},"max":{"1min":1.107,"5min":1.1299999999999999,"15min":1.298},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.96099999999999997},"min":{"1min":0.84499999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1659999999999999,"5min":1.214,"15min":1.371},"last":0.999}]}]},{"osd":2,"up_from":13,"seq":55834575657,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.438,"15min":0.46200000000000002},"min":{"1min":0.32500000000000001,"5min":0.32400000000000001,"15min":0.28000000000000003},"max":{"1min":0.56499999999999995,"5min":0.621,"15min":0.80600000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.68000000000000005,"5min":0.78000000000000003,"15min":0.79900000000000004},"last":0.68600000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46100000000000002,"15min":0.47699999999999998},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.55700000000000005,"5min":0.66900000000000004,"15min":1.236},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60099999999999998,"15min":0.624},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40200000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":1.2969999999999999},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.80900000000000005,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.53300000000000003,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.86599999999999999,"5min":1.298,"15min":1.4650000000000001},"last":0.88100000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.79100000000000004,"15min":0.78900000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.95499999999999996,"5min":1.218,"15min":1.218},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.79000000000000004,"15min":0.79100000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.82199999999999995,"15min":0.82999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":0.97499999999999998,"5min":1.337,"15min":1.4359999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.86699999999999999,"15min":0.88},"min":{"1min":0.68400000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.073,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":1.0029999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.83299999999999996,"15min":0.82099999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50800000000000001},"max":{"1min":0.97999999999999998,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.85699999999999998,"15min":0.82299999999999995},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.54300000000000004},"max":{"1min":0.998,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.75600000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:16:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85299999999999998,"15min":0.85199999999999998},"min":{"1min":0.63600000000000001,"5min":0.56399999999999995,"15min":0.54000000000000004},"max":{"1min":1.0600000000000001,"5min":1.286,"15min":1.286},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.879,"15min":0.86599999999999999},"min":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.57399999999999995},"max":{"1min":1.0940000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.97899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575656,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426556,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103360,"kb_avail":93945284,"statfs":{"total":96636764160,"available":96199970816,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105840640},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48099999999999998,"15min":0.48599999999999999},"min":{"1min":0.36099999999999999,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.61799999999999999,"5min":0.77800000000000002,"15min":0.81699999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54100000000000004,"15min":0.55800000000000005},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.745,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78800000000000003,"15min":1.0600000000000001},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.70699999999999996,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86199999999999999,"15min":0.81699999999999995},"min":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.099,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.022},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.81799999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53300000000000003},"max":{"1min":0.97599999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.96999999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.85599999999999998,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0269999999999999,"5min":1.409,"15min":1.409},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0609999999999999,"5min":1.452,"15min":1.452},"last":0.97999999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.64500000000000002,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.0680000000000001,"5min":1.1180000000000001,"15min":1.198},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.65100000000000002,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.0469999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.371,"15min":1.371},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.85099999999999998},"min":{"1min":0.69799999999999995,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.0569999999999999,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":1.0049999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575655,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418452,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125696,"kb_avail":93953388,"statfs":{"total":96636764160,"available":96208269312,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128712704},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.53900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.002,"5min":1.0549999999999999,"15min":1.419},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80400000000000005,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.171,"15min":1.383},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76600000000000001,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.1319999999999999,"5min":1.242,"15min":1.242},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83399999999999996,"15min":0.79900000000000004},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.171,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.84999999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.73899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.39700000000000002},"max":{"1min":1.2789999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.441},"max":{"1min":1.0880000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.72399999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.71299999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":0.93700000000000006,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.62},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.05},"last":0.622}]},{"osd":6,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81899999999999995,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.91400000000000003,"5min":1.0229999999999999,"15min":1.131},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.624,"15min":0.65200000000000002},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71699999999999997,"15min":0.65100000000000002},"min":{"1min":0.50700000000000001,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":1.2270000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.59899999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575655,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67600000000000005,"15min":0.68999999999999995},"min":{"1min":0.44700000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86799999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.54200000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85099999999999998,"5min":1.579,"15min":1.579},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.91300000000000003,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73599999999999999,"15min":0.73899999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.45700000000000002},"max":{"1min":0.93600000000000005,"5min":0.95799999999999996,"15min":1.1870000000000001},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72099999999999997,"15min":0.71799999999999997},"min":{"1min":0.52000000000000002,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.89200000000000002,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.60199999999999998,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.92400000000000004,"5min":1.391,"15min":1.391},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.77500000000000002,"15min":0.74399999999999999},"min":{"1min":0.57199999999999995,"5min":0.53700000000000003,"15min":0.47399999999999998},"max":{"1min":0.88100000000000001,"5min":1.169,"15min":1.254},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.81599999999999995,"15min":0.80700000000000005},"min":{"1min":0.60399999999999998,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.94799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.70899999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.44700000000000001,"5min":0.36399999999999999,"15min":0.35999999999999999},"max":{"1min":0.77200000000000002,"5min":1.522,"15min":1.522},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57599999999999996,"15min":0.57699999999999996},"min":{"1min":0.40100000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.70499999999999996,"5min":1.28,"15min":1.28},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.68500000000000005,"15min":0.63900000000000001},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.62},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48999999999999999,"5min":0.45600000000000002,"15min":0.41599999999999998},"max":{"1min":0.86199999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Jun 9 12:16:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.629},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.877,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.74399999999999999,"15min":0.747},"min":{"1min":0.56999999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91200000000000003,"5min":1.226,"15min":1.238},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:17:52.812 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:52.813 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:17:52.813 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:18:00.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:00.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:12.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:18:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.028+0000 7fd7c85ff700 1 Processor -- start 2023-06-09T12:18:13.032 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.028+0000 7fd7c85ff700 1 -- start start 2023-06-09T12:18:13.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.030+0000 7fd7c85ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7c00c2ae0 0x7fd7c00c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.030+0000 7fd7c85ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 0x7fd7c0163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.034 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.030+0000 7fd7c85ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00c9bd0 con 0x7fd7c00c88e0 2023-06-09T12:18:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.030+0000 7fd7c85ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c0159360 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.030+0000 7fd7c85ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00c8e20 con 0x7fd7c015b060 2023-06-09T12:18:13.036 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd7c00c2ae0 0x7fd7c00c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c5b9a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd7c00c88e0 0x7fd7c015a950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:53580/0 (socket says 172.21.15.7:53580) 2023-06-09T12:18:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c5b9a700 1 -- 172.21.15.7:0/3935103714 learned_addr learned my addr 172.21.15.7:0/3935103714 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:18:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c6b9c700 1 --2- 172.21.15.7:0/3935103714 >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 0x7fd7c0163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 -- 172.21.15.7:0/3935103714 >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 msgr2=0x7fd7c0163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:18:13.037 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 --2- 172.21.15.7:0/3935103714 >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 0x7fd7c0163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 -- 172.21.15.7:0/3935103714 >> v1:172.21.15.7:6789/0 conn(0x7fd7c00c88e0 legacy=0x7fd7c015a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 -- 172.21.15.7:0/3935103714 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0159570 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c5399700 1 -- 172.21.15.7:0/3935103714 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728470388 0 0) 0x7fd7c00c9bd0 con 0x7fd7c00c88e0 2023-06-09T12:18:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c6b9c700 1 --2- 172.21.15.7:0/3935103714 >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 0x7fd7c0163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:18:13.038 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.031+0000 7fd7c639b700 1 --2- 172.21.15.7:0/3935103714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c2ae0 0x7fd7c00c2eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006f7e0 tx=0x7fd7bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212a04ba16913115 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c5399700 1 -- 172.21.15.7:0/3935103714 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc011830 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c5399700 1 -- 172.21.15.7:0/3935103714 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc011990 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c5399700 1 -- 172.21.15.7:0/3935103714 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc011d20 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 -- 172.21.15.7:0/3935103714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c2ae0 msgr2=0x7fd7c00c2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.039 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/3935103714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c2ae0 0x7fd7c00c2eb0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006f7e0 tx=0x7fd7bc009b20 comp rx=0 tx=0).stop 2023-06-09T12:18:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 -- 172.21.15.7:0/3935103714 shutdown_connections 2023-06-09T12:18:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/3935103714 >> v2:172.21.15.117:3300/0 conn(0x7fd7c015b060 0x7fd7c0163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/3935103714 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c2ae0 0x7fd7c00c2eb0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 -- 172.21.15.7:0/3935103714 >> 172.21.15.7:0/3935103714 conn(0x7fd7c00cd330 msgr2=0x7fd7c00cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:13.040 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 -- 172.21.15.7:0/3935103714 shutdown_connections 2023-06-09T12:18:13.041 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.032+0000 7fd7c85ff700 1 -- 172.21.15.7:0/3935103714 wait complete. 2023-06-09T12:18:13.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.034+0000 7fd7c85ff700 1 Processor -- start 2023-06-09T12:18:13.042 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.034+0000 7fd7c85ff700 1 -- start start 2023-06-09T12:18:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c00c2ae0 0x7fd7c00c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c015b060 0x7fd7c00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cc120 con 0x7fd7c00c88e0 2023-06-09T12:18:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01f2ae0 con 0x7fd7c015b060 2023-06-09T12:18:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c85ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00c68d0 con 0x7fd7c00c2ae0 2023-06-09T12:18:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c5b9a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55522/0 (socket says 172.21.15.7:55522) 2023-06-09T12:18:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c5b9a700 1 -- 172.21.15.7:0/2526796355 learned_addr learned my addr 172.21.15.7:0/2526796355 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:18:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.035+0000 7fd7c639b700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c00c2ae0 0x7fd7c00c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c6b9c700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c015b060 0x7fd7c00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 -- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c00c2ae0 msgr2=0x7fd7c00c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c00c2ae0 0x7fd7c00c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 -- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c015b060 msgr2=0x7fd7c00ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.045 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c015b060 0x7fd7c00ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 -- 172.21.15.7:0/2526796355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0094a0 con 0x7fd7c00c88e0 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c5b9a700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00011e0 tx=0x7fd7b000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0004c80 con 0x7fd7c00c88e0 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0004e00 con 0x7fd7c00c88e0 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c0159af0 con 0x7fd7c00c88e0 2023-06-09T12:18:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.036+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0007310 con 0x7fd7c00c88e0 2023-06-09T12:18:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.037+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c0159810 con 0x7fd7c00c88e0 2023-06-09T12:18:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.038+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0159dd0 con 0x7fd7c00c88e0 2023-06-09T12:18:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.039+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd7b0007470 con 0x7fd7c00c88e0 2023-06-09T12:18:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.040+0000 7fd7b6ffd700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 0x7fd7a406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.040+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fd7b0089a80 con 0x7fd7c00c88e0 2023-06-09T12:18:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.040+0000 7fd7c639b700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 0x7fd7a406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:13.048 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.040+0000 7fd7c639b700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 0x7fd7a406d1c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0090b0 tx=0x7fd7bc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:13.049 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.044+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd7b0054120 con 0x7fd7c00c88e0 2023-06-09T12:18:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.247+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7c00cb570 con 0x7fd7a406ad10 2023-06-09T12:18:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.256+0000 7fd7b6ffd700 1 -- 172.21.15.7:0/2526796355 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319451 (secure 0 0 0) 0x7fd7c00cb570 con 0x7fd7a406ad10 2023-06-09T12:18:13.263 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:18:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 msgr2=0x7fd7a406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 0x7fd7a406d1c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc0090b0 tx=0x7fd7bc008030 comp rx=0 tx=0).stop 2023-06-09T12:18:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 msgr2=0x7fd7c00c9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fd7b00011e0 tx=0x7fd7b000ca30 comp rx=0 tx=0).stop 2023-06-09T12:18:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 shutdown_connections 2023-06-09T12:18:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd7c015b060 0x7fd7c00ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd7a406ad10 0x7fd7a406d1c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd7c00c2ae0 0x7fd7c00c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 --2- 172.21.15.7:0/2526796355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd7c00c88e0 0x7fd7c00c9fb0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 >> 172.21.15.7:0/2526796355 conn(0x7fd7c00cd330 msgr2=0x7fd7c0158850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 shutdown_connections 2023-06-09T12:18:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:13.260+0000 7fd7c85ff700 1 -- 172.21.15.7:0/2526796355 wait complete. 2023-06-09T12:18:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:18:13.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2093,"stamp":"2023-06-09T12:18:13.235172+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104995,"ondisk_log_size":104995,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594440,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711104,"kb_avail":752380280,"statfs":{"total":773094113280,"available":770437406720,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728170496},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.591494+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:15:32.591494+0000","last_peered":"2023-06-09T12:15:32.591494+0000","last_clean":"2023-06-09T12:15:32.591494+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:15:32.591494+0000","last_undegraded":"2023-06-09T12:15:32.591494+0000","last_fullsized":"2023-06-09T12:15:32.591494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3863,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.420832+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:15:07.420832+0000","last_peered":"2023-06-09T12:15:07.420832+0000","last_clean":"2023-06-09T12:15:07.420832+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:15:07.420832+0000","last_undegraded":"2023-06-09T12:15:07.420832+0000","last_fullsized":"2023-06-09T12:15:07.420832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532404+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:15:37.532404+0000","last_peered":"2023-06-09T12:15:37.532404+0000","last_clean":"2023-06-09T12:15:37.532404+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:15:37.532404+0000","last_undegraded":"2023-06-09T12:15:37.532404+0000","last_fullsized":"2023-06-09T12:15:37.532404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3975,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.428702+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:15:12.428702+0000","last_peered":"2023-06-09T12:15:12.428702+0000","last_clean":"2023-06-09T12:15:12.428702+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:15:12.428702+0000","last_undegraded":"2023-06-09T12:15:12.428702+0000","last_fullsized":"2023-06-09T12:15:12.428702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3392,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533865+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:15:47.533865+0000","last_peered":"2023-06-09T12:15:47.533865+0000","last_clean":"2023-06-09T12:15:47.533865+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:15:47.533865+0000","last_undegraded":"2023-06-09T12:15:47.533865+0000","last_fullsized":"2023-06-09T12:15:47.533865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503208+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:15:27.503208+0000","last_peered":"2023-06-09T12:15:27.503208+0000","last_clean":"2023-06-09T12:15:27.503208+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:15:27.503208+0000","last_undegraded":"2023-06-09T12:15:27.503208+0000","last_fullsized":"2023-06-09T12:15:27.503208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404456+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:15:27.404456+0000","last_peered":"2023-06-09T12:15:27.404456+0000","last_clean":"2023-06-09T12:15:27.404456+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:15:27.404456+0000","last_undegraded":"2023-06-09T12:15:27.404456+0000","last_fullsized":"2023-06-09T12:15:27.404456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3286,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.393342+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:14:02.393342+0000","last_peered":"2023-06-09T12:14:02.393342+0000","last_clean":"2023-06-09T12:14:02.393342+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:14:02.393342+0000","last_undegraded":"2023-06-09T12:14:02.393342+0000","last_fullsized":"2023-06-09T12:14:02.393342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3914,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.431434+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:15:32.431434+0000","last_peered":"2023-06-09T12:15:32.431434+0000","last_clean":"2023-06-09T12:15:32.431434+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:15:32.431434+0000","last_undegraded":"2023-06-09T12:15:32.431434+0000","last_fullsized":"2023-06-09T12:15:32.431434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.416523+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:15:37.416523+0000","last_peered":"2023-06-09T12:15:37.416523+0000","last_clean":"2023-06-09T12:15:37.416523+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:15:37.416523+0000","last_undegraded":"2023-06-09T12:15:37.416523+0000","last_fullsized":"2023-06-09T12:15:37.416523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.403174+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:15:17.403174+0000","last_peered":"2023-06-09T12:15:17.403174+0000","last_clean":"2023-06-09T12:15:17.403174+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:15:17.403174+0000","last_undegraded":"2023-06-09T12:15:17.403174+0000","last_fullsized":"2023-06-09T12:15:17.403174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":369,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409938+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:13:42.409938+0000","last_peered":"2023-06-09T12:13:42.409938+0000","last_clean":"2023-06-09T12:13:42.409938+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:13:42.409938+0000","last_undegraded":"2023-06-09T12:13:42.409938+0000","last_fullsized":"2023-06-09T12:13:42.409938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3231,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.778093+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.778093+0000","last_peered":"2023-06-09T12:15:43.778093+0000","last_clean":"2023-06-09T12:15:43.778093+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.778093+0000","last_undegraded":"2023-06-09T12:15:43.778093+0000","last_fullsized":"2023-06-09T12:15:43.778093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1570,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427884+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:15:07.427884+0000","last_peered":"2023-06-09T12:15:07.427884+0000","last_clean":"2023-06-09T12:15:07.427884+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:15:07.427884+0000","last_undegraded":"2023-06-09T12:15:07.427884+0000","last_fullsized":"2023-06-09T12:15:07.427884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3549,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.412993+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:15:12.412993+0000","last_peered":"2023-06-09T12:15:12.412993+0000","last_clean":"2023-06-09T12:15:12.412993+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:15:12.412993+0000","last_undegraded":"2023-06-09T12:15:12.412993+0000","last_fullsized":"2023-06-09T12:15:12.412993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3419,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.406822+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:15:47.406822+0000","last_peered":"2023-06-09T12:15:47.406822+0000","last_clean":"2023-06-09T12:15:47.406822+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:15:47.406822+0000","last_undegraded":"2023-06-09T12:15:47.406822+0000","last_fullsized":"2023-06-09T12:15:47.406822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1078,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898412+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898412+0000","last_peered":"2023-06-09T12:15:57.898412+0000","last_clean":"2023-06-09T12:15:57.898412+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898412+0000","last_undegraded":"2023-06-09T12:15:57.898412+0000","last_fullsized":"2023-06-09T12:15:57.898412+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1120,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.501981+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:15:17.501981+0000","last_peered":"2023-06-09T12:15:17.501981+0000","last_clean":"2023-06-09T12:15:17.501981+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:15:17.501981+0000","last_undegraded":"2023-06-09T12:15:17.501981+0000","last_fullsized":"2023-06-09T12:15:17.501981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":598,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.411090+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:13:52.411090+0000","last_peered":"2023-06-09T12:13:52.411090+0000","last_clean":"2023-06-09T12:13:52.411090+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:13:52.411090+0000","last_undegraded":"2023-06-09T12:13:52.411090+0000","last_fullsized":"2023-06-09T12:13:52.411090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3872,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.430700+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:15:27.430700+0000","last_peered":"2023-06-09T12:15:27.430700+0000","last_clean":"2023-06-09T12:15:27.430700+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:15:27.430700+0000","last_undegraded":"2023-06-09T12:15:27.430700+0000","last_fullsized":"2023-06-09T12:15:27.430700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3905,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.535402+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:15:57.535402+0000","last_peered":"2023-06-09T12:15:57.535402+0000","last_clean":"2023-06-09T12:15:57.535402+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:15:57.535402+0000","last_undegraded":"2023-06-09T12:15:57.535402+0000","last_fullsized":"2023-06-09T12:15:57.535402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.594789+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:16:02.594789+0000","last_peered":"2023-06-09T12:16:02.594789+0000","last_clean":"2023-06-09T12:16:02.594789+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:16:02.594789+0000","last_undegraded":"2023-06-09T12:16:02.594789+0000","last_fullsized":"2023-06-09T12:16:02.594789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":731,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.417993+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:13:52.417993+0000","last_peered":"2023-06-09T12:13:52.417993+0000","last_clean":"2023-06-09T12:13:52.417993+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:13:52.417993+0000","last_undegraded":"2023-06-09T12:13:52.417993+0000","last_fullsized":"2023-06-09T12:13:52.417993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1385,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:42.417181+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:15:42.417181+0000","last_peered":"2023-06-09T12:15:42.417181+0000","last_clean":"2023-06-09T12:15:42.417181+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:15:42.417181+0000","last_undegraded":"2023-06-09T12:15:42.417181+0000","last_fullsized":"2023-06-09T12:15:42.417181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1234","reported_seq":4131,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:56.924071+0000","last_change":"2023-06-09T12:17:56.924235+0000","last_active":"2023-06-09T12:17:56.924071+0000","last_peered":"2023-06-09T12:17:56.924071+0000","last_clean":"2023-06-09T12:17:56.924071+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:56.924071+0000","last_undegraded":"2023-06-09T12:17:56.924071+0000","last_fullsized":"2023-06-09T12:17:56.924071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1234","last_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_deep_scrub":"75'1234","last_deep_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_clean_scrub_stamp":"2023-06-09T12:17:56.923881+0000","objects_scrubbed":259,"log_size":1234,"log_dups_size":0,"ondisk_log_size":1234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:39:41.044787+0000","scrub_duration":44.285760504999999,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532549+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:15:37.532549+0000","last_peered":"2023-06-09T12:15:37.532549+0000","last_clean":"2023-06-09T12:15:37.532549+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:15:37.532549+0000","last_undegraded":"2023-06-09T12:15:37.532549+0000","last_fullsized":"2023-06-09T12:15:37.532549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3460,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.519478+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:14:02.519478+0000","last_peered":"2023-06-09T12:14:02.519478+0000","last_clean":"2023-06-09T12:14:02.519478+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:14:02.519478+0000","last_undegraded":"2023-06-09T12:14:02.519478+0000","last_fullsized":"2023-06-09T12:14:02.519478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3540,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407602+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:15:52.407602+0000","last_peered":"2023-06-09T12:15:52.407602+0000","last_clean":"2023-06-09T12:15:52.407602+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:15:52.407602+0000","last_undegraded":"2023-06-09T12:15:52.407602+0000","last_fullsized":"2023-06-09T12:15:52.407602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4177,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.394517+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:14:12.394517+0000","last_peered":"2023-06-09T12:14:12.394517+0000","last_clean":"2023-06-09T12:14:12.394517+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:14:12.394517+0000","last_undegraded":"2023-06-09T12:14:12.394517+0000","last_fullsized":"2023-06-09T12:14:12.394517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407473+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:15:52.407473+0000","last_peered":"2023-06-09T12:15:52.407473+0000","last_clean":"2023-06-09T12:15:52.407473+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:15:52.407473+0000","last_undegraded":"2023-06-09T12:15:52.407473+0000","last_fullsized":"2023-06-09T12:15:52.407473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404473+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:15:27.404473+0000","last_peered":"2023-06-09T12:15:27.404473+0000","last_clean":"2023-06-09T12:15:27.404473+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:15:27.404473+0000","last_undegraded":"2023-06-09T12:15:27.404473+0000","last_fullsized":"2023-06-09T12:15:27.404473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":324,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.506645+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:15:57.506645+0000","last_peered":"2023-06-09T12:15:57.506645+0000","last_clean":"2023-06-09T12:15:57.506645+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:15:57.506645+0000","last_undegraded":"2023-06-09T12:15:57.506645+0000","last_fullsized":"2023-06-09T12:15:57.506645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.595312+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:16:07.595312+0000","last_peered":"2023-06-09T12:16:07.595312+0000","last_clean":"2023-06-09T12:16:07.595312+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:16:07.595312+0000","last_undegraded":"2023-06-09T12:16:07.595312+0000","last_fullsized":"2023-06-09T12:16:07.595312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.498887+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:52.498887+0000","last_peered":"2023-06-09T12:14:52.498887+0000","last_clean":"2023-06-09T12:14:52.498887+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:52.498887+0000","last_undegraded":"2023-06-09T12:14:52.498887+0000","last_fullsized":"2023-06-09T12:14:52.498887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.734088+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.734088+0000","last_peered":"2023-06-09T12:15:28.734088+0000","last_clean":"2023-06-09T12:15:28.734088+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.734088+0000","last_undegraded":"2023-06-09T12:15:28.734088+0000","last_fullsized":"2023-06-09T12:15:28.734088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.425813+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:15:47.425813+0000","last_peered":"2023-06-09T12:15:47.425813+0000","last_clean":"2023-06-09T12:15:47.425813+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:15:47.425813+0000","last_undegraded":"2023-06-09T12:15:47.425813+0000","last_fullsized":"2023-06-09T12:15:47.425813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503243+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:15:27.503243+0000","last_peered":"2023-06-09T12:15:27.503243+0000","last_clean":"2023-06-09T12:15:27.503243+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:15:27.503243+0000","last_undegraded":"2023-06-09T12:15:27.503243+0000","last_fullsized":"2023-06-09T12:15:27.503243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":855,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407539+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:15:52.407539+0000","last_peered":"2023-06-09T12:15:52.407539+0000","last_clean":"2023-06-09T12:15:52.407539+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:15:52.407539+0000","last_undegraded":"2023-06-09T12:15:52.407539+0000","last_fullsized":"2023-06-09T12:15:52.407539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4408,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.413201+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:14:07.413201+0000","last_peered":"2023-06-09T12:14:07.413201+0000","last_clean":"2023-06-09T12:14:07.413201+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:14:07.413201+0000","last_undegraded":"2023-06-09T12:14:07.413201+0000","last_fullsized":"2023-06-09T12:14:07.413201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":182,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:02.581107+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:14:02.581107+0000","last_peered":"2023-06-09T12:14:02.581107+0000","last_clean":"2023-06-09T12:14:02.581107+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:14:02.581107+0000","last_undegraded":"2023-06-09T12:14:02.581107+0000","last_fullsized":"2023-06-09T12:14:02.581107+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":866,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:03.714142+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:14:03.714142+0000","last_peered":"2023-06-09T12:14:03.714142+0000","last_clean":"2023-06-09T12:14:03.714142+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:14:03.714142+0000","last_undegraded":"2023-06-09T12:14:03.714142+0000","last_fullsized":"2023-06-09T12:14:03.714142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.471224+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:15:37.471224+0000","last_peered":"2023-06-09T12:15:37.471224+0000","last_clean":"2023-06-09T12:15:37.471224+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:15:37.471224+0000","last_undegraded":"2023-06-09T12:15:37.471224+0000","last_fullsized":"2023-06-09T12:15:37.471224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":802,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:22.414489+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:13:22.414489+0000","last_peered":"2023-06-09T12:13:22.414489+0000","last_clean":"2023-06-09T12:13:22.414489+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:13:22.414489+0000","last_undegraded":"2023-06-09T12:13:22.414489+0000","last_fullsized":"2023-06-09T12:13:22.414489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3936,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:32.515049+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:13:32.515049+0000","last_peered":"2023-06-09T12:13:32.515049+0000","last_clean":"2023-06-09T12:13:32.515049+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:13:32.515049+0000","last_undegraded":"2023-06-09T12:13:32.515049+0000","last_fullsized":"2023-06-09T12:13:32.515049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1069,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.406565+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:14:22.406565+0000","last_peered":"2023-06-09T12:14:22.406565+0000","last_clean":"2023-06-09T12:14:22.406565+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:14:22.406565+0000","last_undegraded":"2023-06-09T12:14:22.406565+0000","last_fullsized":"2023-06-09T12:14:22.406565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.417826+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:15:47.417826+0000","last_peered":"2023-06-09T12:15:47.417826+0000","last_clean":"2023-06-09T12:15:47.417826+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:15:47.417826+0000","last_undegraded":"2023-06-09T12:15:47.417826+0000","last_fullsized":"2023-06-09T12:15:47.417826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'1030","reported_seq":4009,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:18:09.933971+0000","last_change":"2023-06-09T12:18:02.426298+0000","last_active":"2023-06-09T12:18:09.933971+0000","last_peered":"2023-06-09T12:18:09.933971+0000","last_clean":"2023-06-09T12:18:09.933971+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:18:09.933971+0000","last_undegraded":"2023-06-09T12:18:09.933971+0000","last_fullsized":"2023-06-09T12:18:09.933971+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":48,"log_size":1030,"log_dups_size":0,"ondisk_log_size":1030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 7s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533815+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:15:47.533815+0000","last_peered":"2023-06-09T12:15:47.533815+0000","last_clean":"2023-06-09T12:15:47.533815+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:15:47.533815+0000","last_undegraded":"2023-06-09T12:15:47.533815+0000","last_fullsized":"2023-06-09T12:15:47.533815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.420522+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:16:07.420522+0000","last_peered":"2023-06-09T12:16:07.420522+0000","last_clean":"2023-06-09T12:16:07.420522+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:16:07.420522+0000","last_undegraded":"2023-06-09T12:16:07.420522+0000","last_fullsized":"2023-06-09T12:16:07.420522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73871,"ondisk_log_size":73871,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575661,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97899999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.0189999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.54900000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.749,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.92300000000000004,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.745},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":1.071,"15min":1.071},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.86699999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.77400000000000002,"15min":0.755},"min":{"1min":0.51100000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:17:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.70999999999999996,"15min":0.70199999999999996},"min":{"1min":0.51800000000000002,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.97899999999999998,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.747,"15min":0.72799999999999998},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.006,"5min":1.079,"15min":1.079},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.57399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34599999999999997},"max":{"1min":0.74099999999999999,"5min":1.591,"15min":1.591},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.47799999999999998},"max":{"1min":0.86299999999999999,"5min":1.619,"15min":1.619},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.501}]},{"osd":6,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69299999999999995,"15min":0.66000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.77800000000000002,"15min":0.747},"min":{"1min":0.56299999999999994,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95199999999999996,"5min":1.492,"15min":1.492},"last":0.76700000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575661,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.51700000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":1.044,"15min":1.044},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.875,"5min":0.97199999999999998,"15min":0.98999999999999999},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.41899999999999998},"max":{"1min":0.93000000000000005,"5min":1.266,"15min":1.266},"last":0.432},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97199999999999998,"5min":1.298,"15min":1.298},"last":0.65400000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.71799999999999997},"min":{"1min":0.55500000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.091,"15min":1.091},"last":0.75}]},{"osd":3,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0189999999999999,"15min":1.046},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.84099999999999997,"5min":1.077,"15min":1.1259999999999999},"last":0.76200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68700000000000006,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38300000000000001},"max":{"1min":0.76700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.625}]},{"osd":5,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.873,"15min":0.98399999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.437},"max":{"1min":0.89800000000000002,"5min":1.161,"15min":1.161},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65200000000000002,"15min":0.67900000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.80500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575662,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405552,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99136,"kb_avail":93966288,"statfs":{"total":96636764160,"available":96221478912,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101515264},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":1.4470000000000001},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.58199999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58799999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.44}]},{"osd":3,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.2450000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.288},"last":0.72199999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.48599999999999999},"max":{"1min":1.113,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.53200000000000003,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.175,"15min":1.597},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:17:17 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.82099999999999995,"15min":0.83099999999999996},"min":{"1min":0.68999999999999995,"5min":0.628,"15min":0.55500000000000005},"max":{"1min":1.097,"5min":1.196,"15min":1.544},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84099999999999997,"15min":0.85899999999999999},"min":{"1min":0.69999999999999996,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0409999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.77700000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82799999999999996,"15min":0.83699999999999997},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.29,"5min":1.29,"15min":1.548},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.2869999999999999},"last":0.753}]},{"osd":7,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.75600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.161,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.498},"last":0.85199999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575661,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.50900000000000001,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.65700000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.626}]},{"osd":2,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68899999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.68799999999999994}]},{"osd":3,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96199999999999997},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82799999999999996,"5min":1.0149999999999999,"15min":1.0740000000000001},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86299999999999999,"15min":0.84799999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.421},"last":1.1299999999999999},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.77700000000000002,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.087,"5min":1.1830000000000001,"15min":1.268},"last":1.073}]},{"osd":5,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.012,"5min":1.077,"15min":1.121},"last":1.016},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.622,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.198,"5min":1.198,"15min":3.5840000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.81499999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.47699999999999998},"max":{"1min":1.1140000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.0720000000000001,"5min":1.113,"15min":1.232},"last":1.034}]},{"osd":7,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.877},"min":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.48299999999999998},"max":{"1min":1.107,"5min":1.1299999999999999,"15min":1.298},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.96099999999999997},"min":{"1min":0.84499999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1659999999999999,"5min":1.214,"15min":1.371},"last":1.101}]}]},{"osd":2,"up_from":13,"seq":55834575662,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.435,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.28000000000000003},"max":{"1min":0.55500000000000005,"5min":0.56799999999999995,"15min":0.80600000000000005},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.48099999999999998,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79900000000000004},"last":0.50700000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":1.236},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.2969999999999999},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80300000000000005,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.218,"15min":1.218},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79800000000000004,"15min":0.79400000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":1.032,"5min":1.337,"15min":1.4359999999999999},"last":0.752},{"interface":"front","average":{"1min":0.871,"5min":0.86899999999999999,"15min":0.879},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0680000000000001,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83399999999999996,"15min":0.82299999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86199999999999999,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.54500000000000004},"max":{"1min":1.0900000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"max":{"1min":1.0660000000000001,"5min":1.286,"15min":1.286},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.873},"min":{"1min":0.629,"5min":0.629,"15min":0.57399999999999995},"max":{"1min":1.139,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.79100000000000004}]}]},{"osd":3,"up_from":13,"seq":55834575661,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426620,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103424,"kb_avail":93945220,"statfs":{"total":96636764160,"available":96199905280,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":105906176},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":0.72599999999999998,"15min":0.81699999999999995},"last":0.436},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.502}]},{"osd":1,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78800000000000003,"15min":1.0600000000000001},"last":0.60199999999999998}]},{"osd":2,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.70699999999999996,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86199999999999999,"15min":0.81699999999999995},"min":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.099,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.81799999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53300000000000003},"max":{"1min":0.97599999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.97399999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.85599999999999998,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0269999999999999,"5min":1.409,"15min":1.409},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0609999999999999,"5min":1.452,"15min":1.452},"last":0.93500000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.64500000000000002,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.0680000000000001,"5min":1.1180000000000001,"15min":1.198},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.65100000000000002,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.0469999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.83499999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.371,"15min":1.371},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.85099999999999998},"min":{"1min":0.69799999999999995,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.0569999999999999,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.88800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418516,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125760,"kb_avail":93953324,"statfs":{"total":96636764160,"available":96208203776,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128778240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.53900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.002,"5min":1.0549999999999999,"15min":1.419},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80400000000000005,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.171,"15min":1.383},"last":0.94599999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76600000000000001,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.1319999999999999,"5min":1.242,"15min":1.242},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83399999999999996,"15min":0.79900000000000004},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.171,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.069}]},{"osd":2,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.73899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.39700000000000002},"max":{"1min":1.2789999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.873},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.441},"max":{"1min":1.0880000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":1.0309999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.71299999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":0.93700000000000006,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80700000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.62},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.05},"last":0.60899999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81899999999999995,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.91400000000000003,"5min":1.0229999999999999,"15min":1.131},"last":0.63200000000000001}]},{"osd":7,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.624,"15min":0.65200000000000002},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71699999999999997,"15min":0.65100000000000002},"min":{"1min":0.50700000000000001,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":1.2270000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.55800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575660,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.54300000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":1.579,"15min":1.579},"last":0.79700000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":0.97199999999999998,"15min":1.2689999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1870000000000001},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.84999999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.97099999999999997,"5min":1.391,"15min":1.391},"last":0.95499999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.47399999999999998},"max":{"1min":1.016,"5min":1.169,"15min":1.254},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.296,"15min":1.3120000000000001},"last":0.96499999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.522,"15min":1.522},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96199999999999997,"5min":1.28,"15min":1.28},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.64100000000000001},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.40200000000000002},"max":{"1min":0.92800000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.62},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68100000000000005,"15min":0.69599999999999995},"min":{"1min":0.53600000000000003,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.91000000000000003,"5min":1.121,"15min":1.1779999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.63},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.749},"min":{"1min":0.59799999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99399999999999999,"5min":1.226,"15min":1.238},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:18:13.291 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:18:13.291 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:18:30.848 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:30.853 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:18:33.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:18:33.507 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.503+0000 7ff812bff700 1 Processor -- start 2023-06-09T12:18:33.508 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.504+0000 7ff812bff700 1 -- start start 2023-06-09T12:18:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.505+0000 7ff812bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff812bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 0x7ff80c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff812bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c0c95d0 con 0x7ff80c0c9150 2023-06-09T12:18:33.510 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff812bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c0c28b0 con 0x7ff80c1514f0 2023-06-09T12:18:33.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff812bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c0c7e40 con 0x7ff80c0c8250 2023-06-09T12:18:33.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff81099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff81099b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43064/0 (socket says 172.21.15.7:43064) 2023-06-09T12:18:33.511 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff81099b700 1 -- 172.21.15.7:0/1290593225 learned_addr learned my addr 172.21.15.7:0/1290593225 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:18:33.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.506+0000 7ff80bfff700 1 --2- 172.21.15.7:0/1290593225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 0x7ff80c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.512 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80bfff700 1 -- 172.21.15.7:0/1290593225 >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 msgr2=0x7ff80c06d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:18:33.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80b7fe700 1 -- 172.21.15.7:0/1290593225 <== mon.1 v1:172.21.15.117:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988636846 0 0) 0x7ff80c0c95d0 con 0x7ff80c0c9150 2023-06-09T12:18:33.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80bfff700 1 --2- 172.21.15.7:0/1290593225 >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80bfff700 1 -- 172.21.15.7:0/1290593225 >> v1:172.21.15.117:6789/0 conn(0x7ff80c0c9150 legacy=0x7ff80c0c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80bfff700 1 -- 172.21.15.7:0/1290593225 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c158010 con 0x7ff80c0c8250 2023-06-09T12:18:33.513 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff81099b700 1 --2- 172.21.15.7:0/1290593225 >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:18:33.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.507+0000 7ff80bfff700 1 --2- 172.21.15.7:0/1290593225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 0x7ff80c0c8c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009840 tx=0x7ff7f8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1072ece69072c0a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:33.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.508+0000 7ff80b7fe700 1 -- 172.21.15.7:0/1290593225 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8011830 con 0x7ff80c0c8250 2023-06-09T12:18:33.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.508+0000 7ff80b7fe700 1 -- 172.21.15.7:0/1290593225 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8011990 con 0x7ff80c0c8250 2023-06-09T12:18:33.514 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.508+0000 7ff80b7fe700 1 -- 172.21.15.7:0/1290593225 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8011d20 con 0x7ff80c0c8250 2023-06-09T12:18:33.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 -- 172.21.15.7:0/1290593225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 msgr2=0x7ff80c0c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 --2- 172.21.15.7:0/1290593225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 0x7ff80c0c8c10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009840 tx=0x7ff7f8009b50 comp rx=0 tx=0).stop 2023-06-09T12:18:33.515 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 -- 172.21.15.7:0/1290593225 shutdown_connections 2023-06-09T12:18:33.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 --2- 172.21.15.7:0/1290593225 >> v2:172.21.15.7:3300/0 conn(0x7ff80c1514f0 0x7ff80c06d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 --2- 172.21.15.7:0/1290593225 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c0c8250 0x7ff80c0c8c10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 -- 172.21.15.7:0/1290593225 >> 172.21.15.7:0/1290593225 conn(0x7ff80c0cd330 msgr2=0x7ff80c0cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:33.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 -- 172.21.15.7:0/1290593225 shutdown_connections 2023-06-09T12:18:33.516 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.509+0000 7ff812bff700 1 -- 172.21.15.7:0/1290593225 wait complete. 2023-06-09T12:18:33.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.512+0000 7ff812bff700 1 Processor -- start 2023-06-09T12:18:33.518 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.512+0000 7ff812bff700 1 -- start start 2023-06-09T12:18:33.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 0x7ff80c1ef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c1514f0 0x7ff80c1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c1f3320 con 0x7ff80c0c8250 2023-06-09T12:18:33.519 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c141f70 con 0x7ff80c0c9150 2023-06-09T12:18:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff812bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c142250 con 0x7ff80c1514f0 2023-06-09T12:18:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43076/0 (socket says 172.21.15.7:43076) 2023-06-09T12:18:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 -- 172.21.15.7:0/43760364 learned_addr learned my addr 172.21.15.7:0/43760364 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:18:33.520 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff80bfff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 0x7ff80c1ef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81119c700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c1514f0 0x7ff80c1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 -- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c1514f0 msgr2=0x7ff80c1f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c1514f0 0x7ff80c1f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 -- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 msgr2=0x7ff80c1ef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.521 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 0x7ff80c1ef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff81099b700 1 -- 172.21.15.7:0/43760364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800009b30 con 0x7ff80c0c8250 2023-06-09T12:18:33.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.513+0000 7ff80bfff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 0x7ff80c1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:18:33.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff81099b700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff80000efc0 tx=0x7ff80000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:33.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000cc70 con 0x7ff80c0c8250 2023-06-09T12:18:33.522 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000f040 con 0x7ff80c0c8250 2023-06-09T12:18:33.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800014480 con 0x7ff80c0c8250 2023-06-09T12:18:33.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f80094a0 con 0x7ff80c0c8250 2023-06-09T12:18:33.523 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.514+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c157490 con 0x7ff80c0c8250 2023-06-09T12:18:33.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.515+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80c1571b0 con 0x7ff80c0c8250 2023-06-09T12:18:33.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.516+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff8000145e0 con 0x7ff80c0c8250 2023-06-09T12:18:33.524 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.518+0000 7ff8097fa700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 0x7ff7f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:33.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.518+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7ff80008a520 con 0x7ff80c0c8250 2023-06-09T12:18:33.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.518+0000 7ff80bfff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 0x7ff7f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:33.525 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.519+0000 7ff80bfff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 0x7ff7f006d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80090b0 tx=0x7ff7f801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:33.526 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.523+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7ff800054c50 con 0x7ff80c0c8250 2023-06-09T12:18:33.731 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.728+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff80c157770 con 0x7ff7f006ace0 2023-06-09T12:18:33.741 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.737+0000 7ff8097fa700 1 -- 172.21.15.7:0/43760364 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319498 (secure 0 0 0) 0x7ff80c157770 con 0x7ff7f006ace0 2023-06-09T12:18:33.744 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:18:33.746 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 msgr2=0x7ff7f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 0x7ff7f006d190 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff7f80090b0 tx=0x7ff7f801c040 comp rx=0 tx=0).stop 2023-06-09T12:18:33.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 msgr2=0x7ff80c0cb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:33.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7ff80000efc0 tx=0x7ff80000c3a0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 shutdown_connections 2023-06-09T12:18:33.747 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7ff7f006ace0 0x7ff7f006d190 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff80c1514f0 0x7ff80c1f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff80c0c8250 0x7ff80c0cb110 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 --2- 172.21.15.7:0/43760364 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff80c0c9150 0x7ff80c1ef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:33.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 >> 172.21.15.7:0/43760364 conn(0x7ff80c0cd330 msgr2=0x7ff80c0ca640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:33.748 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 shutdown_connections 2023-06-09T12:18:33.749 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:33.742+0000 7ff812bff700 1 -- 172.21.15.7:0/43760364 wait complete. 2023-06-09T12:18:33.749 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:18:33.764 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2103,"stamp":"2023-06-09T12:18:33.240417+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105093,"ondisk_log_size":105093,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594696,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711360,"kb_avail":752380024,"statfs":{"total":773094113280,"available":770437144576,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728432640},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003037"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3755,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.463798+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:14:27.463798+0000","last_peered":"2023-06-09T12:14:27.463798+0000","last_clean":"2023-06-09T12:14:27.463798+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:14:27.463798+0000","last_undegraded":"2023-06-09T12:14:27.463798+0000","last_fullsized":"2023-06-09T12:14:27.463798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.591494+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:15:32.591494+0000","last_peered":"2023-06-09T12:15:32.591494+0000","last_clean":"2023-06-09T12:15:32.591494+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:15:32.591494+0000","last_undegraded":"2023-06-09T12:15:32.591494+0000","last_fullsized":"2023-06-09T12:15:32.591494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3863,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.420832+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:15:07.420832+0000","last_peered":"2023-06-09T12:15:07.420832+0000","last_clean":"2023-06-09T12:15:07.420832+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:15:07.420832+0000","last_undegraded":"2023-06-09T12:15:07.420832+0000","last_fullsized":"2023-06-09T12:15:07.420832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532404+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:15:37.532404+0000","last_peered":"2023-06-09T12:15:37.532404+0000","last_clean":"2023-06-09T12:15:37.532404+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:15:37.532404+0000","last_undegraded":"2023-06-09T12:15:37.532404+0000","last_fullsized":"2023-06-09T12:15:37.532404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:52.491201+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:13:52.491201+0000","last_peered":"2023-06-09T12:13:52.491201+0000","last_clean":"2023-06-09T12:13:52.491201+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:13:52.491201+0000","last_undegraded":"2023-06-09T12:13:52.491201+0000","last_fullsized":"2023-06-09T12:13:52.491201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3975,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.428702+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:15:12.428702+0000","last_peered":"2023-06-09T12:15:12.428702+0000","last_clean":"2023-06-09T12:15:12.428702+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:15:12.428702+0000","last_undegraded":"2023-06-09T12:15:12.428702+0000","last_fullsized":"2023-06-09T12:15:12.428702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3536,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.420671+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:14:12.420671+0000","last_peered":"2023-06-09T12:14:12.420671+0000","last_clean":"2023-06-09T12:14:12.420671+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:14:12.420671+0000","last_undegraded":"2023-06-09T12:14:12.420671+0000","last_fullsized":"2023-06-09T12:14:12.420671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3392,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533865+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:15:47.533865+0000","last_peered":"2023-06-09T12:15:47.533865+0000","last_clean":"2023-06-09T12:15:47.533865+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:15:47.533865+0000","last_undegraded":"2023-06-09T12:15:47.533865+0000","last_fullsized":"2023-06-09T12:15:47.533865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503208+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:15:27.503208+0000","last_peered":"2023-06-09T12:15:27.503208+0000","last_clean":"2023-06-09T12:15:27.503208+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:15:27.503208+0000","last_undegraded":"2023-06-09T12:15:27.503208+0000","last_fullsized":"2023-06-09T12:15:27.503208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3651,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.494425+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:14:17.494425+0000","last_peered":"2023-06-09T12:14:17.494425+0000","last_clean":"2023-06-09T12:14:17.494425+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:14:17.494425+0000","last_undegraded":"2023-06-09T12:14:17.494425+0000","last_fullsized":"2023-06-09T12:14:17.494425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404456+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:15:27.404456+0000","last_peered":"2023-06-09T12:15:27.404456+0000","last_clean":"2023-06-09T12:15:27.404456+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:15:27.404456+0000","last_undegraded":"2023-06-09T12:15:27.404456+0000","last_fullsized":"2023-06-09T12:15:27.404456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.409964+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:16:12.409964+0000","last_peered":"2023-06-09T12:16:12.409964+0000","last_clean":"2023-06-09T12:16:12.409964+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:16:12.409964+0000","last_undegraded":"2023-06-09T12:16:12.409964+0000","last_fullsized":"2023-06-09T12:16:12.409964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3914,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.431434+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:15:32.431434+0000","last_peered":"2023-06-09T12:15:32.431434+0000","last_clean":"2023-06-09T12:15:32.431434+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:15:32.431434+0000","last_undegraded":"2023-06-09T12:15:32.431434+0000","last_fullsized":"2023-06-09T12:15:32.431434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4128,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129024+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129024+0000","last_peered":"2023-06-09T12:16:32.129024+0000","last_clean":"2023-06-09T12:16:32.129024+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129024+0000","last_undegraded":"2023-06-09T12:16:32.129024+0000","last_fullsized":"2023-06-09T12:16:32.129024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.416523+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:15:37.416523+0000","last_peered":"2023-06-09T12:15:37.416523+0000","last_clean":"2023-06-09T12:15:37.416523+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:15:37.416523+0000","last_undegraded":"2023-06-09T12:15:37.416523+0000","last_fullsized":"2023-06-09T12:15:37.416523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.403174+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:15:17.403174+0000","last_peered":"2023-06-09T12:15:17.403174+0000","last_clean":"2023-06-09T12:15:17.403174+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:15:17.403174+0000","last_undegraded":"2023-06-09T12:15:17.403174+0000","last_fullsized":"2023-06-09T12:15:17.403174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3448,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.418309+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:14:47.418309+0000","last_peered":"2023-06-09T12:14:47.418309+0000","last_clean":"2023-06-09T12:14:47.418309+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:14:47.418309+0000","last_undegraded":"2023-06-09T12:14:47.418309+0000","last_fullsized":"2023-06-09T12:14:47.418309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.428358+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:16:07.428358+0000","last_peered":"2023-06-09T12:16:07.428358+0000","last_clean":"2023-06-09T12:16:07.428358+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:16:07.428358+0000","last_undegraded":"2023-06-09T12:16:07.428358+0000","last_fullsized":"2023-06-09T12:16:07.428358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3231,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.778093+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.778093+0000","last_peered":"2023-06-09T12:15:43.778093+0000","last_clean":"2023-06-09T12:15:43.778093+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.778093+0000","last_undegraded":"2023-06-09T12:15:43.778093+0000","last_fullsized":"2023-06-09T12:15:43.778093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1570,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427884+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:15:07.427884+0000","last_peered":"2023-06-09T12:15:07.427884+0000","last_clean":"2023-06-09T12:15:07.427884+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:15:07.427884+0000","last_undegraded":"2023-06-09T12:15:07.427884+0000","last_fullsized":"2023-06-09T12:15:07.427884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3549,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.412993+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:15:12.412993+0000","last_peered":"2023-06-09T12:15:12.412993+0000","last_clean":"2023-06-09T12:15:12.412993+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:15:12.412993+0000","last_undegraded":"2023-06-09T12:15:12.412993+0000","last_fullsized":"2023-06-09T12:15:12.412993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3419,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.406822+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:15:47.406822+0000","last_peered":"2023-06-09T12:15:47.406822+0000","last_clean":"2023-06-09T12:15:47.406822+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:15:47.406822+0000","last_undegraded":"2023-06-09T12:15:47.406822+0000","last_fullsized":"2023-06-09T12:15:47.406822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898525+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898525+0000","last_peered":"2023-06-09T12:15:57.898525+0000","last_clean":"2023-06-09T12:15:57.898525+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898525+0000","last_undegraded":"2023-06-09T12:15:57.898525+0000","last_fullsized":"2023-06-09T12:15:57.898525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3398,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.520911+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:14:12.520911+0000","last_peered":"2023-06-09T12:14:12.520911+0000","last_clean":"2023-06-09T12:14:12.520911+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:14:12.520911+0000","last_undegraded":"2023-06-09T12:14:12.520911+0000","last_fullsized":"2023-06-09T12:14:12.520911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1120,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.501981+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:15:17.501981+0000","last_peered":"2023-06-09T12:15:17.501981+0000","last_clean":"2023-06-09T12:15:17.501981+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:15:17.501981+0000","last_undegraded":"2023-06-09T12:15:17.501981+0000","last_fullsized":"2023-06-09T12:15:17.501981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":599,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428881+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:16:12.428881+0000","last_peered":"2023-06-09T12:16:12.428881+0000","last_clean":"2023-06-09T12:16:12.428881+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:16:12.428881+0000","last_undegraded":"2023-06-09T12:16:12.428881+0000","last_fullsized":"2023-06-09T12:16:12.428881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3872,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.430700+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:15:27.430700+0000","last_peered":"2023-06-09T12:15:27.430700+0000","last_clean":"2023-06-09T12:15:27.430700+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:15:27.430700+0000","last_undegraded":"2023-06-09T12:15:27.430700+0000","last_fullsized":"2023-06-09T12:15:27.430700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1175,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493144+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:14:07.493144+0000","last_peered":"2023-06-09T12:14:07.493144+0000","last_clean":"2023-06-09T12:14:07.493144+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:14:07.493144+0000","last_undegraded":"2023-06-09T12:14:07.493144+0000","last_fullsized":"2023-06-09T12:14:07.493144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3259,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.421383+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:14:17.421383+0000","last_peered":"2023-06-09T12:14:17.421383+0000","last_clean":"2023-06-09T12:14:17.421383+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:14:17.421383+0000","last_undegraded":"2023-06-09T12:14:17.421383+0000","last_fullsized":"2023-06-09T12:14:17.421383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3905,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.535402+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:15:57.535402+0000","last_peered":"2023-06-09T12:15:57.535402+0000","last_clean":"2023-06-09T12:15:57.535402+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:15:57.535402+0000","last_undegraded":"2023-06-09T12:15:57.535402+0000","last_fullsized":"2023-06-09T12:15:57.535402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":637,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:07.493196+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:14:07.493196+0000","last_peered":"2023-06-09T12:14:07.493196+0000","last_clean":"2023-06-09T12:14:07.493196+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:14:07.493196+0000","last_undegraded":"2023-06-09T12:14:07.493196+0000","last_fullsized":"2023-06-09T12:14:07.493196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.594789+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:16:02.594789+0000","last_peered":"2023-06-09T12:16:02.594789+0000","last_clean":"2023-06-09T12:16:02.594789+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:16:02.594789+0000","last_undegraded":"2023-06-09T12:16:02.594789+0000","last_fullsized":"2023-06-09T12:16:02.594789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.435910+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:16:07.435910+0000","last_peered":"2023-06-09T12:16:07.435910+0000","last_clean":"2023-06-09T12:16:07.435910+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:16:07.435910+0000","last_undegraded":"2023-06-09T12:16:07.435910+0000","last_fullsized":"2023-06-09T12:16:07.435910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1385,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:42.417181+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:15:42.417181+0000","last_peered":"2023-06-09T12:15:42.417181+0000","last_clean":"2023-06-09T12:15:42.417181+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:15:42.417181+0000","last_undegraded":"2023-06-09T12:15:42.417181+0000","last_fullsized":"2023-06-09T12:15:42.417181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1234","reported_seq":4131,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:56.924071+0000","last_change":"2023-06-09T12:17:56.924235+0000","last_active":"2023-06-09T12:17:56.924071+0000","last_peered":"2023-06-09T12:17:56.924071+0000","last_clean":"2023-06-09T12:17:56.924071+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:56.924071+0000","last_undegraded":"2023-06-09T12:17:56.924071+0000","last_fullsized":"2023-06-09T12:17:56.924071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1234","last_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_deep_scrub":"75'1234","last_deep_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_clean_scrub_stamp":"2023-06-09T12:17:56.923881+0000","objects_scrubbed":259,"log_size":1234,"log_dups_size":0,"ondisk_log_size":1234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:39:41.044787+0000","scrub_duration":44.285760504999999,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532549+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:15:37.532549+0000","last_peered":"2023-06-09T12:15:37.532549+0000","last_clean":"2023-06-09T12:15:37.532549+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:15:37.532549+0000","last_undegraded":"2023-06-09T12:15:37.532549+0000","last_fullsized":"2023-06-09T12:15:37.532549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1666,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:17.521509+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:14:17.521509+0000","last_peered":"2023-06-09T12:14:17.521509+0000","last_clean":"2023-06-09T12:14:17.521509+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:14:17.521509+0000","last_undegraded":"2023-06-09T12:14:17.521509+0000","last_fullsized":"2023-06-09T12:14:17.521509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.538160+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:16:17.538160+0000","last_peered":"2023-06-09T12:16:17.538160+0000","last_clean":"2023-06-09T12:16:17.538160+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:16:17.538160+0000","last_undegraded":"2023-06-09T12:16:17.538160+0000","last_fullsized":"2023-06-09T12:16:17.538160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3540,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407602+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:15:52.407602+0000","last_peered":"2023-06-09T12:15:52.407602+0000","last_clean":"2023-06-09T12:15:52.407602+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:15:52.407602+0000","last_undegraded":"2023-06-09T12:15:52.407602+0000","last_fullsized":"2023-06-09T12:15:52.407602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":930,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.522772+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:14:27.522772+0000","last_peered":"2023-06-09T12:14:27.522772+0000","last_clean":"2023-06-09T12:14:27.522772+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:14:27.522772+0000","last_undegraded":"2023-06-09T12:14:27.522772+0000","last_fullsized":"2023-06-09T12:14:27.522772+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4178,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.410673+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:16:17.410673+0000","last_peered":"2023-06-09T12:16:17.410673+0000","last_clean":"2023-06-09T12:16:17.410673+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:16:17.410673+0000","last_undegraded":"2023-06-09T12:16:17.410673+0000","last_fullsized":"2023-06-09T12:16:17.410673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407473+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:15:52.407473+0000","last_peered":"2023-06-09T12:15:52.407473+0000","last_clean":"2023-06-09T12:15:52.407473+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:15:52.407473+0000","last_undegraded":"2023-06-09T12:15:52.407473+0000","last_fullsized":"2023-06-09T12:15:52.407473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404473+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:15:27.404473+0000","last_peered":"2023-06-09T12:15:27.404473+0000","last_clean":"2023-06-09T12:15:27.404473+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:15:27.404473+0000","last_undegraded":"2023-06-09T12:15:27.404473+0000","last_fullsized":"2023-06-09T12:15:27.404473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":324,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.506645+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:15:57.506645+0000","last_peered":"2023-06-09T12:15:57.506645+0000","last_clean":"2023-06-09T12:15:57.506645+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:15:57.506645+0000","last_undegraded":"2023-06-09T12:15:57.506645+0000","last_fullsized":"2023-06-09T12:15:57.506645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1355,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.408325+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:14:37.408325+0000","last_peered":"2023-06-09T12:14:37.408325+0000","last_clean":"2023-06-09T12:14:37.408325+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:14:37.408325+0000","last_undegraded":"2023-06-09T12:14:37.408325+0000","last_fullsized":"2023-06-09T12:14:37.408325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.595312+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:16:07.595312+0000","last_peered":"2023-06-09T12:16:07.595312+0000","last_clean":"2023-06-09T12:16:07.595312+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:16:07.595312+0000","last_undegraded":"2023-06-09T12:16:07.595312+0000","last_fullsized":"2023-06-09T12:16:07.595312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":193,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.498887+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:14:52.498887+0000","last_peered":"2023-06-09T12:14:52.498887+0000","last_clean":"2023-06-09T12:14:52.498887+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:14:52.498887+0000","last_undegraded":"2023-06-09T12:14:52.498887+0000","last_fullsized":"2023-06-09T12:14:52.498887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.734088+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.734088+0000","last_peered":"2023-06-09T12:15:28.734088+0000","last_clean":"2023-06-09T12:15:28.734088+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.734088+0000","last_undegraded":"2023-06-09T12:15:28.734088+0000","last_fullsized":"2023-06-09T12:15:28.734088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.425813+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:15:47.425813+0000","last_peered":"2023-06-09T12:15:47.425813+0000","last_clean":"2023-06-09T12:15:47.425813+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:15:47.425813+0000","last_undegraded":"2023-06-09T12:15:47.425813+0000","last_fullsized":"2023-06-09T12:15:47.425813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503243+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:15:27.503243+0000","last_peered":"2023-06-09T12:15:27.503243+0000","last_clean":"2023-06-09T12:15:27.503243+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:15:27.503243+0000","last_undegraded":"2023-06-09T12:15:27.503243+0000","last_fullsized":"2023-06-09T12:15:27.503243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":855,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407539+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:15:52.407539+0000","last_peered":"2023-06-09T12:15:52.407539+0000","last_clean":"2023-06-09T12:15:52.407539+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:15:52.407539+0000","last_undegraded":"2023-06-09T12:15:52.407539+0000","last_fullsized":"2023-06-09T12:15:52.407539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428936+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:16:12.428936+0000","last_peered":"2023-06-09T12:16:12.428936+0000","last_clean":"2023-06-09T12:16:12.428936+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:16:12.428936+0000","last_undegraded":"2023-06-09T12:16:12.428936+0000","last_fullsized":"2023-06-09T12:16:12.428936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":183,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.596001+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:16:12.596001+0000","last_peered":"2023-06-09T12:16:12.596001+0000","last_clean":"2023-06-09T12:16:12.596001+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:16:12.596001+0000","last_undegraded":"2023-06-09T12:16:12.596001+0000","last_fullsized":"2023-06-09T12:16:12.596001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":867,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428963+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:16:12.428963+0000","last_peered":"2023-06-09T12:16:12.428963+0000","last_clean":"2023-06-09T12:16:12.428963+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:16:12.428963+0000","last_undegraded":"2023-06-09T12:16:12.428963+0000","last_fullsized":"2023-06-09T12:16:12.428963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3994,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:50.624016+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:14:50.624016+0000","last_peered":"2023-06-09T12:14:50.624016+0000","last_clean":"2023-06-09T12:14:50.624016+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:14:50.624016+0000","last_undegraded":"2023-06-09T12:14:50.624016+0000","last_fullsized":"2023-06-09T12:14:50.624016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":918,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:42.409846+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:13:42.409846+0000","last_peered":"2023-06-09T12:13:42.409846+0000","last_clean":"2023-06-09T12:13:42.409846+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:13:42.409846+0000","last_undegraded":"2023-06-09T12:13:42.409846+0000","last_fullsized":"2023-06-09T12:13:42.409846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.471224+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:15:37.471224+0000","last_peered":"2023-06-09T12:15:37.471224+0000","last_clean":"2023-06-09T12:15:37.471224+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:15:37.471224+0000","last_undegraded":"2023-06-09T12:15:37.471224+0000","last_fullsized":"2023-06-09T12:15:37.471224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":803,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.433955+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:15:52.433955+0000","last_peered":"2023-06-09T12:15:52.433955+0000","last_clean":"2023-06-09T12:15:52.433955+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:15:52.433955+0000","last_undegraded":"2023-06-09T12:15:52.433955+0000","last_fullsized":"2023-06-09T12:15:52.433955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:37.416106+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:13:37.416106+0000","last_peered":"2023-06-09T12:13:37.416106+0000","last_clean":"2023-06-09T12:13:37.416106+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:13:37.416106+0000","last_undegraded":"2023-06-09T12:13:37.416106+0000","last_fullsized":"2023-06-09T12:13:37.416106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.536093+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:16:02.536093+0000","last_peered":"2023-06-09T12:16:02.536093+0000","last_clean":"2023-06-09T12:16:02.536093+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:16:02.536093+0000","last_undegraded":"2023-06-09T12:16:02.536093+0000","last_fullsized":"2023-06-09T12:16:02.536093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.422783+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:16:22.422783+0000","last_peered":"2023-06-09T12:16:22.422783+0000","last_clean":"2023-06-09T12:16:22.422783+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:16:22.422783+0000","last_undegraded":"2023-06-09T12:16:22.422783+0000","last_fullsized":"2023-06-09T12:16:22.422783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.417826+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:15:47.417826+0000","last_peered":"2023-06-09T12:15:47.417826+0000","last_clean":"2023-06-09T12:15:47.417826+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:15:47.417826+0000","last_undegraded":"2023-06-09T12:15:47.417826+0000","last_fullsized":"2023-06-09T12:15:47.417826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.417759+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:14:42.417759+0000","last_peered":"2023-06-09T12:14:42.417759+0000","last_clean":"2023-06-09T12:14:42.417759+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:14:42.417759+0000","last_undegraded":"2023-06-09T12:14:42.417759+0000","last_fullsized":"2023-06-09T12:14:42.417759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'1128","reported_seq":4112,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:18:27.901858+0000","last_change":"2023-06-09T12:18:02.426298+0000","last_active":"2023-06-09T12:18:27.901858+0000","last_peered":"2023-06-09T12:18:27.901858+0000","last_clean":"2023-06-09T12:18:27.901858+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:18:27.901858+0000","last_undegraded":"2023-06-09T12:18:27.901858+0000","last_fullsized":"2023-06-09T12:18:27.901858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":146,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 25s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533815+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:15:47.533815+0000","last_peered":"2023-06-09T12:15:47.533815+0000","last_clean":"2023-06-09T12:15:47.533815+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:15:47.533815+0000","last_undegraded":"2023-06-09T12:15:47.533815+0000","last_fullsized":"2023-06-09T12:15:47.533815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.420522+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:16:07.420522+0000","last_peered":"2023-06-09T12:16:07.420522+0000","last_clean":"2023-06-09T12:16:07.420522+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:16:07.420522+0000","last_undegraded":"2023-06-09T12:16:07.420522+0000","last_fullsized":"2023-06-09T12:16:07.420522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:32.397232+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:14:32.397232+0000","last_peered":"2023-06-09T12:14:32.397232+0000","last_clean":"2023-06-09T12:14:32.397232+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:14:32.397232+0000","last_undegraded":"2023-06-09T12:14:32.397232+0000","last_fullsized":"2023-06-09T12:14:32.397232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":73969,"ondisk_log_size":73969,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575665,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97899999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.0189999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.749,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.92300000000000004,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":1.071,"15min":1.071},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.86699999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.77400000000000002,"15min":0.755},"min":{"1min":0.51100000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.80000000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.079,"15min":1.079},"last":0.74199999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.57399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34599999999999997},"max":{"1min":0.74099999999999999,"5min":1.591,"15min":1.591},"last":0.503},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.47799999999999998},"max":{"1min":0.86299999999999999,"5min":1.619,"15min":1.619},"last":0.73099999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.55000000000000004}]},{"osd":6,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69299999999999995,"15min":0.66000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.77800000000000002,"15min":0.747},"min":{"1min":0.56299999999999994,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95199999999999996,"5min":1.492,"15min":1.492},"last":0.77200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575665,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.51700000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":1.044,"15min":1.044},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.875,"5min":0.97199999999999998,"15min":0.98999999999999999},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.41899999999999998},"max":{"1min":0.93000000000000005,"5min":1.266,"15min":1.266},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97199999999999998,"5min":1.298,"15min":1.298},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.71799999999999997},"min":{"1min":0.55500000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.091,"15min":1.091},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0189999999999999,"15min":1.046},"last":0.624},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.84099999999999997,"5min":1.077,"15min":1.1259999999999999},"last":0.746}]},{"osd":4,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68700000000000006,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38300000000000001},"max":{"1min":0.76700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.58799999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.873,"15min":0.98399999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.437},"max":{"1min":0.89800000000000002,"5min":1.161,"15min":1.161},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65200000000000002,"15min":0.67900000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575666,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405680,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99264,"kb_avail":93966160,"statfs":{"total":96636764160,"available":96221347840,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101646336},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":1.4470000000000001},"last":0.495},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58799999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.46600000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.2450000000000001},"last":0.628},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.288},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.48599999999999999},"max":{"1min":1.113,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.53200000000000003,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.175,"15min":1.597},"last":0.88800000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":1.097,"15min":1.544},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0369999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82799999999999996,"15min":0.83699999999999997},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.29,"5min":1.29,"15min":1.548},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.2869999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.75600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.161,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.498},"last":0.97499999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575665,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.50900000000000001,"15min":0.48999999999999999},"min":{"1min":0.34399999999999997,"5min":0.31900000000000001,"15min":0.27600000000000002},"max":{"1min":0.65700000000000003,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56899999999999995,"15min":0.55700000000000005},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.71599999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45200000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.80800000000000005,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.53100000000000003,"15min":0.53900000000000003},"min":{"1min":0.41899999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.68899999999999995,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.39400000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.62,"15min":0.59999999999999998},"min":{"1min":0.45400000000000001,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.96199999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.67800000000000005,"15min":0.68899999999999995},"min":{"1min":0.502,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.82799999999999996,"5min":1.0149999999999999,"15min":1.0740000000000001},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86299999999999999,"15min":0.84799999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.421},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.879,"15min":0.88900000000000001},"min":{"1min":0.77700000000000002,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.087,"5min":1.1830000000000001,"15min":1.268},"last":0.98099999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.63200000000000001,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.012,"5min":1.077,"15min":1.121},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.622,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.198,"5min":1.198,"15min":3.5840000000000001},"last":0.85399999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.83199999999999996,"15min":0.81499999999999995},"min":{"1min":0.55900000000000005,"5min":0.53100000000000003,"15min":0.47699999999999998},"max":{"1min":1.1140000000000001,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.64000000000000001,"5min":0.495,"15min":0.495},"max":{"1min":1.0720000000000001,"5min":1.113,"15min":1.232},"last":0.95299999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:17:38 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88800000000000001,"15min":0.877},"min":{"1min":0.66000000000000003,"5min":0.58499999999999996,"15min":0.48299999999999998},"max":{"1min":1.107,"5min":1.1299999999999999,"15min":1.298},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97299999999999998,"15min":0.96099999999999997},"min":{"1min":0.84499999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.1659999999999999,"5min":1.214,"15min":1.371},"last":1.01}]}]},{"osd":2,"up_from":13,"seq":55834575666,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.435,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.28000000000000003},"max":{"1min":0.55500000000000005,"5min":0.56799999999999995,"15min":0.80600000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.48099999999999998,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79900000000000004},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":1.236},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.2969999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.623}]},{"osd":4,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80300000000000005,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.218,"15min":1.218},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79800000000000004,"15min":0.79400000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.96099999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":1.032,"5min":1.337,"15min":1.4359999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.871,"5min":0.86899999999999999,"15min":0.879},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0680000000000001,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":0.92600000000000005}]},{"osd":6,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83399999999999996,"15min":0.82299999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86199999999999999,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.54500000000000004},"max":{"1min":1.0900000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.90500000000000003}]},{"osd":7,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"max":{"1min":1.0660000000000001,"5min":1.286,"15min":1.286},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.873},"min":{"1min":0.629,"5min":0.629,"15min":0.57399999999999995},"max":{"1min":1.139,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.93600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575665,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426748,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103552,"kb_avail":93945092,"statfs":{"total":96636764160,"available":96199774208,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106037248},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":0.72599999999999998,"15min":0.81699999999999995},"last":0.435},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.53300000000000003},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.34300000000000003},"max":{"1min":0.73899999999999999,"5min":0.80600000000000005,"15min":0.88700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59499999999999997,"15min":0.57599999999999996},"min":{"1min":0.45400000000000001,"5min":0.41899999999999998,"15min":0.36499999999999999},"max":{"1min":0.73899999999999999,"5min":0.78800000000000003,"15min":1.0600000000000001},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.623,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.46500000000000002,"15min":0.46000000000000002},"max":{"1min":0.70699999999999996,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.62},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55300000000000005,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.35899999999999999},"max":{"1min":0.69399999999999995,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86199999999999999,"15min":0.81699999999999995},"min":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.48599999999999999},"max":{"1min":1.099,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84299999999999997,"15min":0.81799999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.53300000000000003},"max":{"1min":0.97599999999999998,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.79900000000000004}]},{"osd":5,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.85599999999999998,"15min":0.84399999999999997},"min":{"1min":0.68899999999999995,"5min":0.63300000000000001,"15min":0.52800000000000002},"max":{"1min":1.0269999999999999,"5min":1.409,"15min":1.409},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.89700000000000002,"15min":0.89600000000000002},"min":{"1min":0.72399999999999998,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.0609999999999999,"5min":1.452,"15min":1.452},"last":0.98799999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.81000000000000005,"15min":0.80500000000000005},"min":{"1min":0.64500000000000002,"5min":0.56000000000000005,"15min":0.54400000000000004},"max":{"1min":1.0680000000000001,"5min":1.1180000000000001,"15min":1.198},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83299999999999996,"15min":0.82799999999999996},"min":{"1min":0.65100000000000002,"5min":0.57099999999999995,"15min":0.54200000000000004},"max":{"1min":1.0469999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:17:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.82299999999999995,"15min":0.82699999999999996},"min":{"1min":0.63100000000000001,"5min":0.58899999999999997,"15min":0.58699999999999997},"max":{"1min":1.02,"5min":1.371,"15min":1.371},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.85099999999999998},"min":{"1min":0.69799999999999995,"5min":0.63100000000000001,"15min":0.53700000000000003},"max":{"1min":1.0569999999999999,"5min":1.1910000000000001,"15min":1.4099999999999999},"last":0.76600000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575664,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418516,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125760,"kb_avail":93953324,"statfs":{"total":96636764160,"available":96208203776,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128778240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76600000000000001,"15min":0.75},"min":{"1min":0.53900000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.002,"5min":1.0549999999999999,"15min":1.419},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.80400000000000005,"15min":0.79100000000000004},"min":{"1min":0.61499999999999999,"5min":0.51600000000000001,"15min":0.46500000000000002},"max":{"1min":1.0309999999999999,"5min":1.171,"15min":1.383},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.76600000000000001,"15min":0.747},"min":{"1min":0.49399999999999999,"5min":0.40500000000000003,"15min":0.377},"max":{"1min":1.1319999999999999,"5min":1.242,"15min":1.242},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.83399999999999996,"15min":0.79900000000000004},"min":{"1min":0.69099999999999995,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.171,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.76300000000000001,"15min":0.73899999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.39700000000000002},"max":{"1min":1.2789999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81200000000000006,"15min":0.80000000000000004},"min":{"1min":0.629,"5min":0.629,"15min":0.441},"max":{"1min":1.0880000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.70199999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.748,"15min":0.71299999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":1.3280000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.753},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.749},"min":{"1min":0.58899999999999997,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":0.93700000000000006,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.378,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.79200000000000004,"5min":0.85099999999999998,"15min":0.95499999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.624,"15min":0.62},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":1.05},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59099999999999997,"15min":0.58399999999999996},"min":{"1min":0.434,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.81899999999999995,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.47899999999999998},"max":{"1min":0.91400000000000003,"5min":1.0229999999999999,"15min":1.131},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.624,"15min":0.65200000000000002},"min":{"1min":0.436,"5min":0.41999999999999998,"15min":0.39500000000000002},"max":{"1min":0.95999999999999996,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71699999999999997,"15min":0.65100000000000002},"min":{"1min":0.50700000000000001,"5min":0.495,"15min":0.38300000000000001},"max":{"1min":1.2270000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.49399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575664,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.54300000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":1.579,"15min":1.579},"last":0.442}]},{"osd":1,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":0.97199999999999998,"15min":1.2689999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1870000000000001},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.84999999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.97099999999999997,"5min":1.391,"15min":1.391},"last":0.48799999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.47399999999999998},"max":{"1min":1.016,"5min":1.169,"15min":1.254},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.296,"15min":1.3120000000000001},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.522,"15min":1.522},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96199999999999997,"5min":1.28,"15min":1.28},"last":0.58499999999999996}]},{"osd":6,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.64100000000000001},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.40200000000000002},"max":{"1min":0.92800000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68100000000000005,"15min":0.69599999999999995},"min":{"1min":0.53600000000000003,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.91000000000000003,"5min":1.121,"15min":1.1779999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.63},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.749},"min":{"1min":0.59799999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99399999999999999,"5min":1.226,"15min":1.238},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:18:33.773 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:18:33.773 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:18:53.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:18:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.983+0000 7f33bfdff700 1 Processor -- start 2023-06-09T12:18:53.986 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.983+0000 7f33bfdff700 1 -- start start 2023-06-09T12:18:53.987 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bfdff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f33b8151510 0x7f33b80c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bfdff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b81580a0 con 0x7f33b815b380 2023-06-09T12:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bfdff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b80c6180 con 0x7f33b8071230 2023-06-09T12:18:53.988 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bfdff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b81518e0 con 0x7f33b8151510 2023-06-09T12:18:53.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bd39a700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f33b815b380 0x7f33b8070b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:37690/0 (socket says 172.21.15.7:37690) 2023-06-09T12:18:53.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bd39a700 1 -- 172.21.15.7:0/2642154026 learned_addr learned my addr 172.21.15.7:0/2642154026 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:18:53.990 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512304610 0 0) 0x7f33b81580a0 con 0x7f33b815b380 2023-06-09T12:18:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.985+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a40035b0 con 0x7f33b815b380 2023-06-09T12:18:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2642154026 >> v2:172.21.15.117:3300/0 conn(0x7f33b8151510 0x7f33b80c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448121537 0 0) 0x7f33a40035b0 con 0x7f33b815b380 2023-06-09T12:18:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 >> v2:172.21.15.117:3300/0 conn(0x7f33b8151510 msgr2=0x7f33b80c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:18:53.991 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 --2- 172.21.15.7:0/2642154026 >> v2:172.21.15.117:3300/0 conn(0x7f33b8151510 0x7f33b80c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 >> v1:172.21.15.117:6789/0 conn(0x7f33b8071230 legacy=0x7f33b8158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8157490 con 0x7f33b815b380 2023-06-09T12:18:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2841377223 0 0) 0x7f33a8001670 con 0x7f33b815b380 2023-06-09T12:18:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 >> v1:172.21.15.7:6789/0 conn(0x7f33b815b380 legacy=0x7f33b8070b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:53.992 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 0x7f33a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4004850 0x7f33a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33a4005200 0x7f33a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b81518e0 con 0x7f33a4003a00 2023-06-09T12:18:53.993 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a400c080 con 0x7f33a4005200 2023-06-09T12:18:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a400c240 con 0x7f33a4004850 2023-06-09T12:18:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 0x7f33a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33be39c700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33a4005200 0x7f33a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:53.994 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.986+0000 7f33bd39a700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4004850 0x7f33a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 -- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4004850 msgr2=0x7f33a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:18:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4004850 0x7f33a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 -- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33a4005200 msgr2=0x7f33a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:18:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33a4005200 0x7f33a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:53.995 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 -- 172.21.15.7:0/2642154026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a40035b0 con 0x7f33a4003a00 2023-06-09T12:18:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33be39c700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33a4005200 0x7f33a400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:18:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33be39c700 1 -- 172.21.15.7:0/2642154026 reap_dead start 2023-06-09T12:18:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 0x7f33a4003e10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009b60 tx=0x7f33a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e491f32ec1ae0ee server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400e6c0 con 0x7f33a4003a00 2023-06-09T12:18:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b400e820 con 0x7f33a4003a00 2023-06-09T12:18:53.997 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.987+0000 7f33bcb99700 1 -- 172.21.15.7:0/2642154026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b400eb90 con 0x7f33a4003a00 2023-06-09T12:18:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 -- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 msgr2=0x7f33a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 0x7f33a4003e10 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009b60 tx=0x7f33a4005880 comp rx=0 tx=0).stop 2023-06-09T12:18:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 -- 172.21.15.7:0/2642154026 shutdown_connections 2023-06-09T12:18:53.999 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2642154026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33a4003a00 0x7f33a4003e10 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 -- 172.21.15.7:0/2642154026 >> 172.21.15.7:0/2642154026 conn(0x7f33b806a4d0 msgr2=0x7f33b806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 -- 172.21.15.7:0/2642154026 shutdown_connections 2023-06-09T12:18:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.988+0000 7f33bfdff700 1 -- 172.21.15.7:0/2642154026 wait complete. 2023-06-09T12:18:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.989+0000 7f33bfdff700 1 Processor -- start 2023-06-09T12:18:54.000 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.990+0000 7f33bfdff700 1 -- start start 2023-06-09T12:18:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.990+0000 7f33bfdff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4003a00 0x7f33b8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bfdff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bfdff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33b80c0a80 0x7f33b80c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bfdff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80c8570 con 0x7f33b814b4c0 2023-06-09T12:18:54.001 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bfdff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80c1af0 con 0x7f33b80c0a80 2023-06-09T12:18:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bfdff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b80c0f00 con 0x7f33a4003a00 2023-06-09T12:18:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36582/0 (socket says 172.21.15.7:36582) 2023-06-09T12:18:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 -- 172.21.15.7:0/2982900138 learned_addr learned my addr 172.21.15.7:0/2982900138 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:18:54.002 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33be39c700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33b80c0a80 0x7f33b80c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:54.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4003a00 0x7f33b8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:54.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 -- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4003a00 msgr2=0x7f33b8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:54.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4003a00 0x7f33b8155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 -- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33b80c0a80 msgr2=0x7f33b80c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:54.003 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33b80c0a80 0x7f33b80c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.991+0000 7f33bd39a700 1 -- 172.21.15.7:0/2982900138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8157a50 con 0x7f33b814b4c0 2023-06-09T12:18:54.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33bd39a700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f33a800ea70 tx=0x7f33a800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:54.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8005e50 con 0x7f33b814b4c0 2023-06-09T12:18:54.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a800c7e0 con 0x7f33b814b4c0 2023-06-09T12:18:54.004 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b8157770 con 0x7f33b814b4c0 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a800cb90 con 0x7f33b814b4c0 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.992+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b8157d30 con 0x7f33b814b4c0 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.993+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b8074cf0 con 0x7f33b814b4c0 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.994+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33a800ccf0 con 0x7f33b814b4c0 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.995+0000 7f33ae7fc700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 0x7f339c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:18:54.005 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.995+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f33a8089c00 con 0x7f33b814b4c0 2023-06-09T12:18:54.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.995+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 0x7f339c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:18:54.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:53.996+0000 7f33bdb9b700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 0x7f339c06d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f33b40090b0 tx=0x7f33b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:18:54.006 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.000+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f33a80543c0 con 0x7f33b814b4c0 2023-06-09T12:18:54.208 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.205+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33b80b0c40 con 0x7f339c06ace0 2023-06-09T12:18:54.217 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.214+0000 7f33ae7fc700 1 -- 172.21.15.7:0/2982900138 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319468 (secure 0 0 0) 0x7f33b80b0c40 con 0x7f339c06ace0 2023-06-09T12:18:54.221 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 msgr2=0x7f339c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 0x7f339c06d190 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f33b40090b0 tx=0x7f33b4008030 comp rx=0 tx=0).stop 2023-06-09T12:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 msgr2=0x7f33b80c0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f33a800ea70 tx=0x7f33a800ee30 comp rx=0 tx=0).stop 2023-06-09T12:18:54.223 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 shutdown_connections 2023-06-09T12:18:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f33b80c0a80 0x7f33b80c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f339c06ace0 0x7f339c06d190 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f33a4003a00 0x7f33b8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 --2- 172.21.15.7:0/2982900138 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33b814b4c0 0x7f33b80c0540 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:18:54.224 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 >> 172.21.15.7:0/2982900138 conn(0x7f33b806a4d0 msgr2=0x7f33b814bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:18:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 shutdown_connections 2023-06-09T12:18:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:18:54.219+0000 7f33bfdff700 1 -- 172.21.15.7:0/2982900138 wait complete. 2023-06-09T12:18:54.225 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:18:54.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2113,"stamp":"2023-06-09T12:18:53.246220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105202,"ondisk_log_size":105202,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594952,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711616,"kb_avail":752379768,"statfs":{"total":773094113280,"available":770436882432,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728694784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003453"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.477422+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:16:32.477422+0000","last_peered":"2023-06-09T12:16:32.477422+0000","last_clean":"2023-06-09T12:16:32.477422+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:16:32.477422+0000","last_undegraded":"2023-06-09T12:16:32.477422+0000","last_fullsized":"2023-06-09T12:16:32.477422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.591494+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:15:32.591494+0000","last_peered":"2023-06-09T12:15:32.591494+0000","last_clean":"2023-06-09T12:15:32.591494+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:15:32.591494+0000","last_undegraded":"2023-06-09T12:15:32.591494+0000","last_fullsized":"2023-06-09T12:15:32.591494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3863,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.420832+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:15:07.420832+0000","last_peered":"2023-06-09T12:15:07.420832+0000","last_clean":"2023-06-09T12:15:07.420832+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:15:07.420832+0000","last_undegraded":"2023-06-09T12:15:07.420832+0000","last_fullsized":"2023-06-09T12:15:07.420832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532404+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:15:37.532404+0000","last_peered":"2023-06-09T12:15:37.532404+0000","last_clean":"2023-06-09T12:15:37.532404+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:15:37.532404+0000","last_undegraded":"2023-06-09T12:15:37.532404+0000","last_fullsized":"2023-06-09T12:15:37.532404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.509778+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:16:22.509778+0000","last_peered":"2023-06-09T12:16:22.509778+0000","last_clean":"2023-06-09T12:16:22.509778+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:16:22.509778+0000","last_undegraded":"2023-06-09T12:16:22.509778+0000","last_fullsized":"2023-06-09T12:16:22.509778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4542,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.203991+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.203991+0000","last_peered":"2023-06-09T12:16:42.203991+0000","last_clean":"2023-06-09T12:16:42.203991+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.203991+0000","last_undegraded":"2023-06-09T12:16:42.203991+0000","last_fullsized":"2023-06-09T12:16:42.203991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3975,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.428702+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:15:12.428702+0000","last_peered":"2023-06-09T12:15:12.428702+0000","last_clean":"2023-06-09T12:15:12.428702+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:15:12.428702+0000","last_undegraded":"2023-06-09T12:15:12.428702+0000","last_fullsized":"2023-06-09T12:15:12.428702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.439149+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:16:32.439149+0000","last_peered":"2023-06-09T12:16:32.439149+0000","last_clean":"2023-06-09T12:16:32.439149+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:16:32.439149+0000","last_undegraded":"2023-06-09T12:16:32.439149+0000","last_fullsized":"2023-06-09T12:16:32.439149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3630,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.495052+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:14:22.495052+0000","last_peered":"2023-06-09T12:14:22.495052+0000","last_clean":"2023-06-09T12:14:22.495052+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:14:22.495052+0000","last_undegraded":"2023-06-09T12:14:22.495052+0000","last_fullsized":"2023-06-09T12:14:22.495052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3708,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.522049+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:14:22.522049+0000","last_peered":"2023-06-09T12:14:22.522049+0000","last_clean":"2023-06-09T12:14:22.522049+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:14:22.522049+0000","last_undegraded":"2023-06-09T12:14:22.522049+0000","last_fullsized":"2023-06-09T12:14:22.522049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3392,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533865+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:15:47.533865+0000","last_peered":"2023-06-09T12:15:47.533865+0000","last_clean":"2023-06-09T12:15:47.533865+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:15:47.533865+0000","last_undegraded":"2023-06-09T12:15:47.533865+0000","last_fullsized":"2023-06-09T12:15:47.533865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503208+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:15:27.503208+0000","last_peered":"2023-06-09T12:15:27.503208+0000","last_clean":"2023-06-09T12:15:27.503208+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:15:27.503208+0000","last_undegraded":"2023-06-09T12:15:27.503208+0000","last_fullsized":"2023-06-09T12:15:27.503208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.417087+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:14:37.417087+0000","last_peered":"2023-06-09T12:14:37.417087+0000","last_clean":"2023-06-09T12:14:37.417087+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:14:37.417087+0000","last_undegraded":"2023-06-09T12:14:37.417087+0000","last_fullsized":"2023-06-09T12:14:37.417087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.510374+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:16:27.510374+0000","last_peered":"2023-06-09T12:16:27.510374+0000","last_clean":"2023-06-09T12:16:27.510374+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:16:27.510374+0000","last_undegraded":"2023-06-09T12:16:27.510374+0000","last_fullsized":"2023-06-09T12:16:27.510374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404456+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:15:27.404456+0000","last_peered":"2023-06-09T12:15:27.404456+0000","last_clean":"2023-06-09T12:15:27.404456+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:15:27.404456+0000","last_undegraded":"2023-06-09T12:15:27.404456+0000","last_fullsized":"2023-06-09T12:15:27.404456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.409964+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:16:12.409964+0000","last_peered":"2023-06-09T12:16:12.409964+0000","last_clean":"2023-06-09T12:16:12.409964+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:16:12.409964+0000","last_undegraded":"2023-06-09T12:16:12.409964+0000","last_fullsized":"2023-06-09T12:16:12.409964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3914,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.431434+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:15:32.431434+0000","last_peered":"2023-06-09T12:15:32.431434+0000","last_clean":"2023-06-09T12:15:32.431434+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:15:32.431434+0000","last_undegraded":"2023-06-09T12:15:32.431434+0000","last_fullsized":"2023-06-09T12:15:32.431434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4129,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129157+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129157+0000","last_peered":"2023-06-09T12:16:32.129157+0000","last_clean":"2023-06-09T12:16:32.129157+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129157+0000","last_undegraded":"2023-06-09T12:16:32.129157+0000","last_fullsized":"2023-06-09T12:16:32.129157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.416523+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:15:37.416523+0000","last_peered":"2023-06-09T12:15:37.416523+0000","last_clean":"2023-06-09T12:15:37.416523+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:15:37.416523+0000","last_undegraded":"2023-06-09T12:15:37.416523+0000","last_fullsized":"2023-06-09T12:15:37.416523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.403174+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:15:17.403174+0000","last_peered":"2023-06-09T12:15:17.403174+0000","last_clean":"2023-06-09T12:15:17.403174+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:15:17.403174+0000","last_undegraded":"2023-06-09T12:15:17.403174+0000","last_fullsized":"2023-06-09T12:15:17.403174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:47.433411+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:16:47.433411+0000","last_peered":"2023-06-09T12:16:47.433411+0000","last_clean":"2023-06-09T12:16:47.433411+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:16:47.433411+0000","last_undegraded":"2023-06-09T12:16:47.433411+0000","last_fullsized":"2023-06-09T12:16:47.433411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.587481+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:14:57.587481+0000","last_peered":"2023-06-09T12:14:57.587481+0000","last_clean":"2023-06-09T12:14:57.587481+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:14:57.587481+0000","last_undegraded":"2023-06-09T12:14:57.587481+0000","last_fullsized":"2023-06-09T12:14:57.587481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.428358+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:16:07.428358+0000","last_peered":"2023-06-09T12:16:07.428358+0000","last_clean":"2023-06-09T12:16:07.428358+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:16:07.428358+0000","last_undegraded":"2023-06-09T12:16:07.428358+0000","last_fullsized":"2023-06-09T12:16:07.428358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3231,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.778093+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.778093+0000","last_peered":"2023-06-09T12:15:43.778093+0000","last_clean":"2023-06-09T12:15:43.778093+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.778093+0000","last_undegraded":"2023-06-09T12:15:43.778093+0000","last_fullsized":"2023-06-09T12:15:43.778093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1570,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427884+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:15:07.427884+0000","last_peered":"2023-06-09T12:15:07.427884+0000","last_clean":"2023-06-09T12:15:07.427884+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:15:07.427884+0000","last_undegraded":"2023-06-09T12:15:07.427884+0000","last_fullsized":"2023-06-09T12:15:07.427884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3549,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.412993+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:15:12.412993+0000","last_peered":"2023-06-09T12:15:12.412993+0000","last_clean":"2023-06-09T12:15:12.412993+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:15:12.412993+0000","last_undegraded":"2023-06-09T12:15:12.412993+0000","last_fullsized":"2023-06-09T12:15:12.412993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3419,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.406822+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:15:47.406822+0000","last_peered":"2023-06-09T12:15:47.406822+0000","last_clean":"2023-06-09T12:15:47.406822+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:15:47.406822+0000","last_undegraded":"2023-06-09T12:15:47.406822+0000","last_fullsized":"2023-06-09T12:15:47.406822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898525+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898525+0000","last_peered":"2023-06-09T12:15:57.898525+0000","last_clean":"2023-06-09T12:15:57.898525+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898525+0000","last_undegraded":"2023-06-09T12:15:57.898525+0000","last_fullsized":"2023-06-09T12:15:57.898525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.540101+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:16:32.540101+0000","last_peered":"2023-06-09T12:16:32.540101+0000","last_clean":"2023-06-09T12:16:32.540101+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:16:32.540101+0000","last_undegraded":"2023-06-09T12:16:32.540101+0000","last_fullsized":"2023-06-09T12:16:32.540101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1120,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.501981+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:15:17.501981+0000","last_peered":"2023-06-09T12:15:17.501981+0000","last_clean":"2023-06-09T12:15:17.501981+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:15:17.501981+0000","last_undegraded":"2023-06-09T12:15:17.501981+0000","last_fullsized":"2023-06-09T12:15:17.501981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":599,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428881+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:16:12.428881+0000","last_peered":"2023-06-09T12:16:12.428881+0000","last_clean":"2023-06-09T12:16:12.428881+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:16:12.428881+0000","last_undegraded":"2023-06-09T12:16:12.428881+0000","last_fullsized":"2023-06-09T12:16:12.428881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3872,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.430700+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:15:27.430700+0000","last_peered":"2023-06-09T12:15:27.430700+0000","last_clean":"2023-06-09T12:15:27.430700+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:15:27.430700+0000","last_undegraded":"2023-06-09T12:15:27.430700+0000","last_fullsized":"2023-06-09T12:15:27.430700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.510413+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:16:27.510413+0000","last_peered":"2023-06-09T12:16:27.510413+0000","last_clean":"2023-06-09T12:16:27.510413+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:16:27.510413+0000","last_undegraded":"2023-06-09T12:16:27.510413+0000","last_fullsized":"2023-06-09T12:16:27.510413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3306,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:12.413942+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:14:12.413942+0000","last_peered":"2023-06-09T12:14:12.413942+0000","last_clean":"2023-06-09T12:14:12.413942+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:14:12.413942+0000","last_undegraded":"2023-06-09T12:14:12.413942+0000","last_fullsized":"2023-06-09T12:14:12.413942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.438375+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:16:27.438375+0000","last_peered":"2023-06-09T12:16:27.438375+0000","last_clean":"2023-06-09T12:16:27.438375+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:16:27.438375+0000","last_undegraded":"2023-06-09T12:16:27.438375+0000","last_fullsized":"2023-06-09T12:16:27.438375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3905,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.535402+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:15:57.535402+0000","last_peered":"2023-06-09T12:15:57.535402+0000","last_clean":"2023-06-09T12:15:57.535402+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:15:57.535402+0000","last_undegraded":"2023-06-09T12:15:57.535402+0000","last_fullsized":"2023-06-09T12:15:57.535402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.509845+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:16:22.509845+0000","last_peered":"2023-06-09T12:16:22.509845+0000","last_clean":"2023-06-09T12:16:22.509845+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:16:22.509845+0000","last_undegraded":"2023-06-09T12:16:22.509845+0000","last_fullsized":"2023-06-09T12:16:22.509845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.594789+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:16:02.594789+0000","last_peered":"2023-06-09T12:16:02.594789+0000","last_clean":"2023-06-09T12:16:02.594789+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:16:02.594789+0000","last_undegraded":"2023-06-09T12:16:02.594789+0000","last_fullsized":"2023-06-09T12:16:02.594789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.435910+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:16:07.435910+0000","last_peered":"2023-06-09T12:16:07.435910+0000","last_clean":"2023-06-09T12:16:07.435910+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:16:07.435910+0000","last_undegraded":"2023-06-09T12:16:07.435910+0000","last_fullsized":"2023-06-09T12:16:07.435910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1385,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:42.417181+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:15:42.417181+0000","last_peered":"2023-06-09T12:15:42.417181+0000","last_clean":"2023-06-09T12:15:42.417181+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:15:42.417181+0000","last_undegraded":"2023-06-09T12:15:42.417181+0000","last_fullsized":"2023-06-09T12:15:42.417181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1234","reported_seq":4131,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:56.924071+0000","last_change":"2023-06-09T12:17:56.924235+0000","last_active":"2023-06-09T12:17:56.924071+0000","last_peered":"2023-06-09T12:17:56.924071+0000","last_clean":"2023-06-09T12:17:56.924071+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:56.924071+0000","last_undegraded":"2023-06-09T12:17:56.924071+0000","last_fullsized":"2023-06-09T12:17:56.924071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1234","last_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_deep_scrub":"75'1234","last_deep_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_clean_scrub_stamp":"2023-06-09T12:17:56.923881+0000","objects_scrubbed":259,"log_size":1234,"log_dups_size":0,"ondisk_log_size":1234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:39:41.044787+0000","scrub_duration":44.285760504999999,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532549+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:15:37.532549+0000","last_peered":"2023-06-09T12:15:37.532549+0000","last_clean":"2023-06-09T12:15:37.532549+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:15:37.532549+0000","last_undegraded":"2023-06-09T12:15:37.532549+0000","last_fullsized":"2023-06-09T12:15:37.532549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1667,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.539444+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:16:27.539444+0000","last_peered":"2023-06-09T12:16:27.539444+0000","last_clean":"2023-06-09T12:16:27.539444+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:16:27.539444+0000","last_undegraded":"2023-06-09T12:16:27.539444+0000","last_fullsized":"2023-06-09T12:16:27.539444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.538160+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:16:17.538160+0000","last_peered":"2023-06-09T12:16:17.538160+0000","last_clean":"2023-06-09T12:16:17.538160+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:16:17.538160+0000","last_undegraded":"2023-06-09T12:16:17.538160+0000","last_fullsized":"2023-06-09T12:16:17.538160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3540,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407602+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:15:52.407602+0000","last_peered":"2023-06-09T12:15:52.407602+0000","last_clean":"2023-06-09T12:15:52.407602+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:15:52.407602+0000","last_undegraded":"2023-06-09T12:15:52.407602+0000","last_fullsized":"2023-06-09T12:15:52.407602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":931,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.540086+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:16:32.540086+0000","last_peered":"2023-06-09T12:16:32.540086+0000","last_clean":"2023-06-09T12:16:32.540086+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:16:32.540086+0000","last_undegraded":"2023-06-09T12:16:32.540086+0000","last_fullsized":"2023-06-09T12:16:32.540086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4178,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.410673+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:16:17.410673+0000","last_peered":"2023-06-09T12:16:17.410673+0000","last_clean":"2023-06-09T12:16:17.410673+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:16:17.410673+0000","last_undegraded":"2023-06-09T12:16:17.410673+0000","last_fullsized":"2023-06-09T12:16:17.410673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407473+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:15:52.407473+0000","last_peered":"2023-06-09T12:15:52.407473+0000","last_clean":"2023-06-09T12:15:52.407473+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:15:52.407473+0000","last_undegraded":"2023-06-09T12:15:52.407473+0000","last_fullsized":"2023-06-09T12:15:52.407473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404473+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:15:27.404473+0000","last_peered":"2023-06-09T12:15:27.404473+0000","last_clean":"2023-06-09T12:15:27.404473+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:15:27.404473+0000","last_undegraded":"2023-06-09T12:15:27.404473+0000","last_fullsized":"2023-06-09T12:15:27.404473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":324,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.506645+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:15:57.506645+0000","last_peered":"2023-06-09T12:15:57.506645+0000","last_clean":"2023-06-09T12:15:57.506645+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:15:57.506645+0000","last_undegraded":"2023-06-09T12:15:57.506645+0000","last_fullsized":"2023-06-09T12:15:57.506645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.425601+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:16:42.425601+0000","last_peered":"2023-06-09T12:16:42.425601+0000","last_clean":"2023-06-09T12:16:42.425601+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:16:42.425601+0000","last_undegraded":"2023-06-09T12:16:42.425601+0000","last_fullsized":"2023-06-09T12:16:42.425601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.595312+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:16:07.595312+0000","last_peered":"2023-06-09T12:16:07.595312+0000","last_clean":"2023-06-09T12:16:07.595312+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:16:07.595312+0000","last_undegraded":"2023-06-09T12:16:07.595312+0000","last_fullsized":"2023-06-09T12:16:07.595312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:52.513564+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:16:52.513564+0000","last_peered":"2023-06-09T12:16:52.513564+0000","last_clean":"2023-06-09T12:16:52.513564+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:16:52.513564+0000","last_undegraded":"2023-06-09T12:16:52.513564+0000","last_fullsized":"2023-06-09T12:16:52.513564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.734088+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.734088+0000","last_peered":"2023-06-09T12:15:28.734088+0000","last_clean":"2023-06-09T12:15:28.734088+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.734088+0000","last_undegraded":"2023-06-09T12:15:28.734088+0000","last_fullsized":"2023-06-09T12:15:28.734088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.425813+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:15:47.425813+0000","last_peered":"2023-06-09T12:15:47.425813+0000","last_clean":"2023-06-09T12:15:47.425813+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:15:47.425813+0000","last_undegraded":"2023-06-09T12:15:47.425813+0000","last_fullsized":"2023-06-09T12:15:47.425813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503243+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:15:27.503243+0000","last_peered":"2023-06-09T12:15:27.503243+0000","last_clean":"2023-06-09T12:15:27.503243+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:15:27.503243+0000","last_undegraded":"2023-06-09T12:15:27.503243+0000","last_fullsized":"2023-06-09T12:15:27.503243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":855,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407539+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:15:52.407539+0000","last_peered":"2023-06-09T12:15:52.407539+0000","last_clean":"2023-06-09T12:15:52.407539+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:15:52.407539+0000","last_undegraded":"2023-06-09T12:15:52.407539+0000","last_fullsized":"2023-06-09T12:15:52.407539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428936+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:16:12.428936+0000","last_peered":"2023-06-09T12:16:12.428936+0000","last_clean":"2023-06-09T12:16:12.428936+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:16:12.428936+0000","last_undegraded":"2023-06-09T12:16:12.428936+0000","last_fullsized":"2023-06-09T12:16:12.428936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":552,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233457+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233457+0000","last_peered":"2023-06-09T12:16:58.233457+0000","last_clean":"2023-06-09T12:16:58.233457+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233457+0000","last_undegraded":"2023-06-09T12:16:58.233457+0000","last_fullsized":"2023-06-09T12:16:58.233457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.412273+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:15:07.412273+0000","last_peered":"2023-06-09T12:15:07.412273+0000","last_clean":"2023-06-09T12:15:07.412273+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:15:07.412273+0000","last_undegraded":"2023-06-09T12:15:07.412273+0000","last_fullsized":"2023-06-09T12:15:07.412273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":183,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.596001+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:16:12.596001+0000","last_peered":"2023-06-09T12:16:12.596001+0000","last_clean":"2023-06-09T12:16:12.596001+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:16:12.596001+0000","last_undegraded":"2023-06-09T12:16:12.596001+0000","last_fullsized":"2023-06-09T12:16:12.596001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":867,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428963+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:16:12.428963+0000","last_peered":"2023-06-09T12:16:12.428963+0000","last_clean":"2023-06-09T12:16:12.428963+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:16:12.428963+0000","last_undegraded":"2023-06-09T12:16:12.428963+0000","last_fullsized":"2023-06-09T12:16:12.428963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.541548+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:16:42.541548+0000","last_peered":"2023-06-09T12:16:42.541548+0000","last_clean":"2023-06-09T12:16:42.541548+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:16:42.541548+0000","last_undegraded":"2023-06-09T12:16:42.541548+0000","last_fullsized":"2023-06-09T12:16:42.541548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":919,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428917+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:16:12.428917+0000","last_peered":"2023-06-09T12:16:12.428917+0000","last_clean":"2023-06-09T12:16:12.428917+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:16:12.428917+0000","last_undegraded":"2023-06-09T12:16:12.428917+0000","last_fullsized":"2023-06-09T12:16:12.428917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.471224+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:15:37.471224+0000","last_peered":"2023-06-09T12:15:37.471224+0000","last_clean":"2023-06-09T12:15:37.471224+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:15:37.471224+0000","last_undegraded":"2023-06-09T12:15:37.471224+0000","last_fullsized":"2023-06-09T12:15:37.471224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":803,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.433955+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:15:52.433955+0000","last_peered":"2023-06-09T12:15:52.433955+0000","last_clean":"2023-06-09T12:15:52.433955+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:15:52.433955+0000","last_undegraded":"2023-06-09T12:15:52.433955+0000","last_fullsized":"2023-06-09T12:15:52.433955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3832,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:13:57.518789+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:13:57.518789+0000","last_peered":"2023-06-09T12:13:57.518789+0000","last_clean":"2023-06-09T12:13:57.518789+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:13:57.518789+0000","last_undegraded":"2023-06-09T12:13:57.518789+0000","last_fullsized":"2023-06-09T12:13:57.518789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.435952+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:16:07.435952+0000","last_peered":"2023-06-09T12:16:07.435952+0000","last_clean":"2023-06-09T12:16:07.435952+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:16:07.435952+0000","last_undegraded":"2023-06-09T12:16:07.435952+0000","last_fullsized":"2023-06-09T12:16:07.435952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.536093+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:16:02.536093+0000","last_peered":"2023-06-09T12:16:02.536093+0000","last_clean":"2023-06-09T12:16:02.536093+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:16:02.536093+0000","last_undegraded":"2023-06-09T12:16:02.536093+0000","last_fullsized":"2023-06-09T12:16:02.536093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.422783+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:16:22.422783+0000","last_peered":"2023-06-09T12:16:22.422783+0000","last_clean":"2023-06-09T12:16:22.422783+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:16:22.422783+0000","last_undegraded":"2023-06-09T12:16:22.422783+0000","last_fullsized":"2023-06-09T12:16:22.422783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.417826+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:15:47.417826+0000","last_peered":"2023-06-09T12:15:47.417826+0000","last_clean":"2023-06-09T12:15:47.417826+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:15:47.417826+0000","last_undegraded":"2023-06-09T12:15:47.417826+0000","last_fullsized":"2023-06-09T12:15:47.417826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1631,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:47.433464+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:16:47.433464+0000","last_peered":"2023-06-09T12:16:47.433464+0000","last_clean":"2023-06-09T12:16:47.433464+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:16:47.433464+0000","last_undegraded":"2023-06-09T12:16:47.433464+0000","last_fullsized":"2023-06-09T12:16:47.433464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3710,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.586760+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:14:52.586760+0000","last_peered":"2023-06-09T12:14:52.586760+0000","last_clean":"2023-06-09T12:14:52.586760+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:14:52.586760+0000","last_undegraded":"2023-06-09T12:14:52.586760+0000","last_fullsized":"2023-06-09T12:14:52.586760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'1237","reported_seq":4228,"reported_epoch":75,"state":"active+clean+scrubbing+deep","last_fresh":"2023-06-09T12:18:51.843633+0000","last_change":"2023-06-09T12:18:02.426298+0000","last_active":"2023-06-09T12:18:51.843633+0000","last_peered":"2023-06-09T12:18:51.843633+0000","last_clean":"2023-06-09T12:18:51.843633+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:18:51.843633+0000","last_undegraded":"2023-06-09T12:18:51.843633+0000","last_fullsized":"2023-06-09T12:18:51.843633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:10:49.647129+0000","last_clean_scrub_stamp":"2023-06-09T11:10:49.647129+0000","objects_scrubbed":255,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 50s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533815+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:15:47.533815+0000","last_peered":"2023-06-09T12:15:47.533815+0000","last_clean":"2023-06-09T12:15:47.533815+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:15:47.533815+0000","last_undegraded":"2023-06-09T12:15:47.533815+0000","last_fullsized":"2023-06-09T12:15:47.533815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.420522+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:16:07.420522+0000","last_peered":"2023-06-09T12:16:07.420522+0000","last_clean":"2023-06-09T12:16:07.420522+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:16:07.420522+0000","last_undegraded":"2023-06-09T12:16:07.420522+0000","last_fullsized":"2023-06-09T12:16:07.420522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.413594+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:16:42.413594+0000","last_peered":"2023-06-09T12:16:42.413594+0000","last_clean":"2023-06-09T12:16:42.413594+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:16:42.413594+0000","last_undegraded":"2023-06-09T12:16:42.413594+0000","last_fullsized":"2023-06-09T12:16:42.413594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":74078,"ondisk_log_size":74078,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575669,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.69899999999999995,"15min":0.69099999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.33700000000000002},"max":{"1min":0.97899999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.54900000000000004,"5min":0.41799999999999998,"15min":0.378},"max":{"1min":1.0189999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.749}]},{"osd":1,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.749,"15min":0.73699999999999999},"min":{"1min":0.57099999999999995,"5min":0.441,"15min":0.42999999999999999},"max":{"1min":0.92300000000000004,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77600000000000002,"15min":0.76200000000000001},"min":{"1min":0.57899999999999996,"5min":0.54500000000000004,"15min":0.48399999999999999},"max":{"1min":0.94999999999999996,"5min":1.071,"15min":1.071},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.52600000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.86699999999999999,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.77400000000000002,"15min":0.755},"min":{"1min":0.51100000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.81299999999999994}]},{"osd":3,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.079,"15min":1.079},"last":0.77600000000000002}]},{"osd":4,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.57399999999999995,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34599999999999997},"max":{"1min":0.74099999999999999,"5min":1.591,"15min":1.591},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.47799999999999998},"max":{"1min":0.86299999999999999,"5min":1.619,"15min":1.619},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.61699999999999999,"15min":0.60599999999999998},"min":{"1min":0.42799999999999999,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.77300000000000002,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.64100000000000001,"15min":0.623},"min":{"1min":0.41899999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78400000000000003,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.875}]},{"osd":6,"last update":"Fri Jun 9 12:18:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69299999999999995,"15min":0.66000000000000003},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.89500000000000002,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.77800000000000002,"15min":0.747},"min":{"1min":0.56299999999999994,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.95199999999999996,"5min":1.492,"15min":1.492},"last":0.92300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575669,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.66500000000000004,"15min":0.67200000000000004},"min":{"1min":0.51700000000000002,"5min":0.45300000000000001,"15min":0.33100000000000002},"max":{"1min":0.83499999999999996,"5min":1.044,"15min":1.044},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69099999999999995,"15min":0.69799999999999995},"min":{"1min":0.48899999999999999,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.875,"5min":0.97199999999999998,"15min":0.98999999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.432,"5min":0.432,"15min":0.41899999999999998},"max":{"1min":0.93000000000000005,"5min":1.266,"15min":1.266},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.76000000000000001,"15min":0.76200000000000001},"min":{"1min":0.56999999999999995,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.97199999999999998,"5min":1.298,"15min":1.298},"last":0.78900000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.68400000000000005,"15min":0.71799999999999997},"min":{"1min":0.55500000000000005,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.85399999999999998,"5min":0.98799999999999999,"15min":1.0369999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.755,"15min":0.76600000000000001},"min":{"1min":0.59799999999999998,"5min":0.501,"15min":0.501},"max":{"1min":0.91500000000000004,"5min":1.091,"15min":1.091},"last":0.82099999999999995}]},{"osd":3,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.73599999999999999,"15min":0.75},"min":{"1min":0.60799999999999998,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.94699999999999995,"5min":1.0189999999999999,"15min":1.046},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.78300000000000003,"15min":0.79600000000000004},"min":{"1min":0.57699999999999996,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":0.84099999999999997,"5min":1.077,"15min":1.1259999999999999},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.68700000000000006,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.60299999999999998,"15min":0.59999999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38300000000000001},"max":{"1min":0.76700000000000002,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51800000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.41799999999999998,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.80000000000000004,"5min":0.873,"15min":0.98399999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.437},"max":{"1min":0.89800000000000002,"5min":1.161,"15min":1.161},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:18:09 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65200000000000002,"15min":0.67900000000000005},"min":{"1min":0.44,"5min":0.44,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71999999999999997,"15min":0.70999999999999996},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46000000000000002},"max":{"1min":0.88500000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.58499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575670,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405808,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99392,"kb_avail":93966032,"statfs":{"total":96636764160,"available":96221216768,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101777408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.44600000000000001,"15min":0.45300000000000001},"min":{"1min":0.35399999999999998,"5min":0.32000000000000001,"15min":0.25800000000000001},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":1.4470000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.55300000000000005,"15min":0.56599999999999995},"min":{"1min":0.47399999999999998,"5min":0.40000000000000002,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.77000000000000002,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":2,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58799999999999997,"5min":0.71699999999999997,"15min":0.995},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53900000000000003,"15min":0.54300000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36699999999999999},"max":{"1min":0.79700000000000004,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":1.2450000000000001},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.69799999999999995},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.288},"last":0.59199999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.89200000000000002,"15min":0.83599999999999997},"min":{"1min":0.67200000000000004,"5min":0.65900000000000003,"15min":0.48599999999999999},"max":{"1min":1.113,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78900000000000003,"15min":0.83499999999999996},"min":{"1min":0.58599999999999997,"5min":0.53200000000000003,"15min":0.48499999999999999},"max":{"1min":1.141,"5min":1.175,"15min":1.597},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":1.097,"15min":1.544},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0369999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.625}]},{"osd":6,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.82799999999999996,"15min":0.83699999999999997},"min":{"1min":0.67800000000000005,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.29,"5min":1.29,"15min":1.548},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.83099999999999996,"15min":0.83199999999999996},"min":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.52400000000000002},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.2869999999999999},"last":0.68200000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:18:04 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.75600000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.161,"5min":1.2190000000000001,"15min":1.6519999999999999},"last":0.746},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.61199999999999999,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.498},"last":0.70499999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575669,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30599999999999999},"max":{"1min":0.64900000000000002,"5min":0.76000000000000001,"15min":0.81299999999999994},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72499999999999998,"5min":0.73399999999999999,"15min":0.91600000000000004},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.72399999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.96199999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.56699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83999999999999997,"5min":1.0149999999999999,"15min":1.0740000000000001},"last":0.68500000000000005}]},{"osd":4,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.879,"15min":0.85799999999999998},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.421},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.89700000000000002,"15min":0.89500000000000002},"min":{"1min":0.72099999999999997,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.268},"last":0.88900000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.79400000000000004},"min":{"1min":0.71899999999999997,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.877},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90600000000000003,"15min":0.89800000000000002},"min":{"1min":0.67400000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.234,"5min":1.234,"15min":3.5840000000000001},"last":0.98299999999999998}]},{"osd":6,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.83399999999999996,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.47699999999999998},"max":{"1min":1.008,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.84999999999999998,"15min":0.84399999999999997},"min":{"1min":0.70999999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.232},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89800000000000002,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.48299999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.98899999999999999,"15min":0.96599999999999997},"min":{"1min":0.83999999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.0129999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575670,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.435,"15min":0.46200000000000002},"min":{"1min":0.34999999999999998,"5min":0.32500000000000001,"15min":0.28000000000000003},"max":{"1min":0.55500000000000005,"5min":0.56799999999999995,"15min":0.80600000000000005},"last":0.442},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.48099999999999998,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.79900000000000004},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46400000000000002,"15min":0.47699999999999998},"min":{"1min":0.33700000000000002,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":1.236},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.621,"15min":0.626},"min":{"1min":0.47099999999999997,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.2969999999999999},"last":0.65200000000000002}]},{"osd":3,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57999999999999996,"15min":0.60299999999999998},"min":{"1min":0.38500000000000001,"5min":0.38200000000000001,"15min":0.36599999999999999},"max":{"1min":0.75900000000000001,"5min":0.86699999999999999,"15min":1.4490000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65700000000000003,"15min":0.67400000000000004},"min":{"1min":0.42799999999999999,"5min":0.41999999999999998,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":1.298,"15min":1.4650000000000001},"last":0.79400000000000004}]},{"osd":4,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80300000000000005,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":1.0580000000000001,"5min":1.218,"15min":1.218},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79800000000000004,"15min":0.79400000000000004},"min":{"1min":0.64500000000000002,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":1.0469999999999999,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.89700000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82899999999999996,"15min":0.82999999999999996},"min":{"1min":0.59199999999999997,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":1.032,"5min":1.337,"15min":1.4359999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.86899999999999999,"15min":0.879},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.0680000000000001,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":0.90800000000000003}]},{"osd":6,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83399999999999996,"15min":0.82299999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86199999999999999,"15min":0.82899999999999996},"min":{"1min":0.60399999999999998,"5min":0.59099999999999997,"15min":0.54500000000000004},"max":{"1min":1.0900000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:17:58 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.87,"15min":0.85599999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.54000000000000004},"max":{"1min":1.0660000000000001,"5min":1.286,"15min":1.286},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.89700000000000002,"15min":0.873},"min":{"1min":0.629,"5min":0.629,"15min":0.57399999999999995},"max":{"1min":1.139,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.87}]}]},{"osd":3,"up_from":13,"seq":55834575669,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426876,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":93944964,"statfs":{"total":96636764160,"available":96199643136,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":0.72599999999999998,"15min":0.81699999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.624}]},{"osd":1,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63,"5min":0.80600000000000005,"15min":0.81999999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.379},"max":{"1min":0.70099999999999996,"5min":0.78800000000000003,"15min":1.0600000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79700000000000004,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.82199999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0649999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.65000000000000002,"5min":0.56200000000000006,"15min":0.56100000000000005},"max":{"1min":1.1120000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.90100000000000002}]},{"osd":5,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.52800000000000002},"max":{"1min":0.995,"5min":1.409,"15min":1.409},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1930000000000001,"5min":1.452,"15min":1.452},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97899999999999998,"5min":1.099,"15min":1.198},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.94899999999999995,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.876}]},{"osd":7,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80900000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1659999999999999,"5min":1.371,"15min":1.371},"last":0.755},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83099999999999996,"15min":0.84799999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.163,"15min":1.4099999999999999},"last":0.85699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575668,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418516,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125760,"kb_avail":93953324,"statfs":{"total":96636764160,"available":96208203776,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128778240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.747,"15min":0.74399999999999999},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":1.0549999999999999,"15min":1.419},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.78700000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.46500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1160000000000001,"15min":1.383},"last":0.96199999999999997}]},{"osd":1,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.377},"max":{"1min":0.99099999999999999,"5min":1.242,"15min":1.242},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.80100000000000005},"min":{"1min":0.71299999999999997,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.069,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.88300000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.755,"15min":0.73499999999999999},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.39700000000000002},"max":{"1min":0.873,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.80000000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.441},"max":{"1min":1.0309999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.91000000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.747,"15min":0.71099999999999997},"min":{"1min":0.52700000000000002,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":0.91600000000000004,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.755},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77800000000000002,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":0.88600000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.436,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.91500000000000004},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.92500000000000004,"15min":1.05},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.80300000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.46899999999999997,"5min":0.41999999999999998,"15min":0.39500000000000002},"max":{"1min":0.74099999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38300000000000001},"max":{"1min":1.101,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.83799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575668,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.67200000000000004,"15min":0.68700000000000006},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86899999999999999,"5min":1.0329999999999999,"15min":1.1100000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.54300000000000004,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90700000000000003,"5min":1.579,"15min":1.579},"last":0.55800000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70499999999999996,"15min":0.71399999999999997},"min":{"1min":0.45700000000000002,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.89000000000000001,"5min":0.97199999999999998,"15min":1.2689999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.72699999999999998,"15min":0.73599999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1870000000000001},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":0.84999999999999998,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.53600000000000003,"5min":0.53500000000000003,"15min":0.437},"max":{"1min":0.97099999999999997,"5min":1.391,"15min":1.391},"last":0.67900000000000005}]},{"osd":3,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78000000000000003,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.54000000000000004,"15min":0.47399999999999998},"max":{"1min":1.016,"5min":1.169,"15min":1.254},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80500000000000005,"15min":0.80900000000000005},"min":{"1min":0.58299999999999996,"5min":0.53400000000000003,"15min":0.48199999999999998},"max":{"1min":0.95399999999999996,"5min":1.296,"15min":1.3120000000000001},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56699999999999995,"15min":0.55300000000000005},"min":{"1min":0.42099999999999999,"5min":0.38700000000000001,"15min":0.35999999999999999},"max":{"1min":0.89000000000000001,"5min":1.522,"15min":1.522},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96199999999999997,"5min":1.28,"15min":1.28},"last":0.44700000000000001}]},{"osd":6,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.68700000000000006,"15min":0.64100000000000001},"min":{"1min":0.504,"5min":0.47799999999999998,"15min":0.40200000000000002},"max":{"1min":0.92800000000000005,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68100000000000005,"15min":0.69599999999999995},"min":{"1min":0.53600000000000003,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":0.91000000000000003,"5min":1.121,"15min":1.1779999999999999},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Jun 9 12:17:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.624,"15min":0.63},"min":{"1min":0.434,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98099999999999998,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.752,"15min":0.749},"min":{"1min":0.59799999999999998,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.99399999999999999,"5min":1.226,"15min":1.238},"last":0.71599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:18:54.248 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2023-06-09T11:10:49.647129+0000 time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=10, tm_sec=49, tm_wday=4, tm_yday=160, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=6, tm_mday=9, tm_hour=11, tm_min=59, tm_sec=39, tm_wday=4, tm_yday=160, tm_isdst=0) 2023-06-09T12:18:54.248 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-06-09T12:19:00.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:00.957 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:14.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-09T12:19:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.462+0000 7faaaddff700 1 Processor -- start 2023-06-09T12:19:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.462+0000 7faaaddff700 1 -- start start 2023-06-09T12:19:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 0x7faaa81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7faaa80c0140 0x7faaa8159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faaa80c1f90 0x7faaa81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa8070110 con 0x7faaa80c1f90 2023-06-09T12:19:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa8071300 con 0x7faaa80c0140 2023-06-09T12:19:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.463+0000 7faaaddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80cbd20 con 0x7faaa80c1bc0 2023-06-09T12:19:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faaa80c1f90 0x7faaa81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7faaa80c1f90 0x7faaa81501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58142/0 (socket says 172.21.15.7:58142) 2023-06-09T12:19:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 -- 172.21.15.7:0/2188152580 learned_addr learned my addr 172.21.15.7:0/2188152580 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa6ffd700 1 --2- 172.21.15.7:0/2188152580 >> v2:172.21.15.117:3300/0 conn(0x7faaa80c0140 0x7faaa8159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa77fe700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 0x7faaa81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 -- 172.21.15.7:0/2188152580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 msgr2=0x7faaa81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 0x7faaa81599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 -- 172.21.15.7:0/2188152580 >> v2:172.21.15.117:3300/0 conn(0x7faaa80c0140 msgr2=0x7faaa8159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 --2- 172.21.15.7:0/2188152580 >> v2:172.21.15.117:3300/0 conn(0x7faaa80c0140 0x7faaa8159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 -- 172.21.15.7:0/2188152580 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80c0bf0 con 0x7faaa80c1f90 2023-06-09T12:19:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa77fe700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 0x7faaa81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:19:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.464+0000 7faaa7fff700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c1f90 0x7faaa81501b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7faaa806f740 tx=0x7faaa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e743371dbea5c661 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaa67fc700 1 -- 172.21.15.7:0/2188152580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000e070 con 0x7faaa80c1f90 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaa67fc700 1 -- 172.21.15.7:0/2188152580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa000b8d0 con 0x7faaa80c1f90 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaa67fc700 1 -- 172.21.15.7:0/2188152580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000bc80 con 0x7faaa80c1f90 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 -- 172.21.15.7:0/2188152580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c1f90 msgr2=0x7faaa81501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c1f90 0x7faaa81501b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7faaa806f740 tx=0x7faaa000b330 comp rx=0 tx=0).stop 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 -- 172.21.15.7:0/2188152580 shutdown_connections 2023-06-09T12:19:14.473 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c1f90 0x7faaa81501b0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 --2- 172.21.15.7:0/2188152580 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1bc0 0x7faaa81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 --2- 172.21.15.7:0/2188152580 >> v2:172.21.15.117:3300/0 conn(0x7faaa80c0140 0x7faaa8159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 -- 172.21.15.7:0/2188152580 >> 172.21.15.7:0/2188152580 conn(0x7faaa806a4d0 msgr2=0x7faaa806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 -- 172.21.15.7:0/2188152580 shutdown_connections 2023-06-09T12:19:14.474 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.465+0000 7faaaddff700 1 -- 172.21.15.7:0/2188152580 wait complete. 2023-06-09T12:19:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.467+0000 7faaaddff700 1 Processor -- start 2023-06-09T12:19:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.467+0000 7faaaddff700 1 -- start start 2023-06-09T12:19:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaa80c1bc0 0x7faaa81f0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1f90 0x7faaa814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81553f0 con 0x7faaa80c0140 2023-06-09T12:19:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8155d50 con 0x7faaa80c1bc0 2023-06-09T12:19:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaaddff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80cad50 con 0x7faaa80c1f90 2023-06-09T12:19:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58150/0 (socket says 172.21.15.7:58150) 2023-06-09T12:19:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.468+0000 7faaa77fe700 1 -- 172.21.15.7:0/4200429180 learned_addr learned my addr 172.21.15.7:0/4200429180 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 -- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1f90 msgr2=0x7faaa814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1f90 0x7faaa814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 -- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaa80c1bc0 msgr2=0x7faaa81f0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaa80c1bc0 0x7faaa81f0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 -- 172.21.15.7:0/4200429180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa0009b30 con 0x7faaa80c0140 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaa77fe700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7faa9800e810 tx=0x7faa9800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98004230 con 0x7faaa80c0140 2023-06-09T12:19:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98004390 con 0x7faaa80c0140 2023-06-09T12:19:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa980036a0 con 0x7faaa80c0140 2023-06-09T12:19:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa80c0e90 con 0x7faaa80c0140 2023-06-09T12:19:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.469+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa80702f0 con 0x7faaa80c0140 2023-06-09T12:19:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.471+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa80c1170 con 0x7faaa80c0140 2023-06-09T12:19:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.471+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faa98003800 con 0x7faaa80c0140 2023-06-09T12:19:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.472+0000 7faa8ffff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 0x7faa9406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.472+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7faa980899f0 con 0x7faaa80c0140 2023-06-09T12:19:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.473+0000 7faaa6ffd700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 0x7faa9406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.473+0000 7faaa6ffd700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 0x7faa9406d1c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faaa80c77a0 tx=0x7faa9c0095a0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.482 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.477+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7faa98054090 con 0x7faaa80c0140 2023-06-09T12:19:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.680+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 --> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faaa80c8200 con 0x7faa9406ad10 2023-06-09T12:19:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.689+0000 7faa8ffff700 1 -- 172.21.15.7:0/4200429180 <== mgr.4098 v2:172.21.15.7:6832/2153300544 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+319538 (secure 0 0 0) 0x7faaa80c8200 con 0x7faa9406ad10 2023-06-09T12:19:14.696 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:19:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 msgr2=0x7faa9406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 0x7faa9406d1c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7faaa80c77a0 tx=0x7faa9c0095a0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 msgr2=0x7faaa8155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7faa9800e810 tx=0x7faa9800ebd0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 shutdown_connections 2023-06-09T12:19:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7faa9406ad10 0x7faa9406d1c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faaa80c1f90 0x7faaa814ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faaa80c0140 0x7faaa8155810 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 --2- 172.21.15.7:0/4200429180 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faaa80c1bc0 0x7faaa81f0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 >> 172.21.15.7:0/4200429180 conn(0x7faaa806a4d0 msgr2=0x7faaa814ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 shutdown_connections 2023-06-09T12:19:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.694+0000 7faaaddff700 1 -- 172.21.15.7:0/4200429180 wait complete. 2023-06-09T12:19:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2023-06-09T12:19:14.717 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":2123,"stamp":"2023-06-09T12:19:13.252120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1374707313,"num_objects":16484,"num_object_clones":7273,"num_object_copies":32968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16484,"num_whiteouts":7203,"num_read":35203,"num_read_kb":2614691,"num_write":76426,"num_write_kb":1830803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105202,"ondisk_log_size":105202,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2594952,"kb_used_data":1883272,"kb_used_omap":0,"kb_used_meta":711616,"kb_avail":752379768,"statfs":{"total":773094113280,"available":770436882432,"internally_reserved":0,"allocated":1928470528,"data_stored":2753765126,"data_compressed":269403948,"data_compressed_allocated":325812224,"data_compressed_original":1193787392,"omap_allocated":0,"internal_metadata":728694784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003211"},"pg_stats":[{"pgid":"4.3f","version":"75'1184","reported_seq":3756,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.477422+0000","last_change":"2023-06-09T12:06:15.215648+0000","last_active":"2023-06-09T12:16:32.477422+0000","last_peered":"2023-06-09T12:16:32.477422+0000","last_clean":"2023-06-09T12:16:32.477422+0000","last_became_active":"2023-06-09T11:10:50.213842+0000","last_became_peered":"2023-06-09T11:10:50.213842+0000","last_unstale":"2023-06-09T12:16:32.477422+0000","last_undegraded":"2023-06-09T12:16:32.477422+0000","last_fullsized":"2023-06-09T12:16:32.477422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:06:15.215347+0000","last_clean_scrub_stamp":"2023-06-09T12:06:15.215347+0000","objects_scrubbed":257,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:12:14.826169+0000","scrub_duration":41.683848488999999,"objects_trimmed":0,"snaptrim_duration":0.010510139999999999,"stat_sum":{"num_bytes":14417819,"num_objects":257,"num_object_clones":115,"num_object_copies":514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":257,"num_whiteouts":115,"num_read":482,"num_read_kb":9798,"num_write":742,"num_write_kb":20732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"75'1118","reported_seq":3488,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427870+0000","last_change":"2023-06-09T12:08:58.045836+0000","last_active":"2023-06-09T12:15:07.427870+0000","last_peered":"2023-06-09T12:15:07.427870+0000","last_clean":"2023-06-09T12:15:07.427870+0000","last_became_active":"2023-06-09T11:10:50.203521+0000","last_became_peered":"2023-06-09T11:10:50.203521+0000","last_unstale":"2023-06-09T12:15:07.427870+0000","last_undegraded":"2023-06-09T12:15:07.427870+0000","last_fullsized":"2023-06-09T12:15:07.427870+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:08:58.045584+0000","last_clean_scrub_stamp":"2023-06-09T12:08:58.045584+0000","objects_scrubbed":236,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:31:01.023425+0000","scrub_duration":37.583542534000003,"objects_trimmed":0,"snaptrim_duration":0.019096709999999999,"stat_sum":{"num_bytes":14787829,"num_objects":236,"num_object_clones":106,"num_object_copies":472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":236,"num_whiteouts":106,"num_read":274,"num_read_kb":8371,"num_write":705,"num_write_kb":17830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"75'1081","reported_seq":3521,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.528705+0000","last_change":"2023-06-09T12:13:11.391565+0000","last_active":"2023-06-09T12:15:12.528705+0000","last_peered":"2023-06-09T12:15:12.528705+0000","last_clean":"2023-06-09T12:15:12.528705+0000","last_became_active":"2023-06-09T11:10:50.197441+0000","last_became_peered":"2023-06-09T11:10:50.197441+0000","last_unstale":"2023-06-09T12:15:12.528705+0000","last_undegraded":"2023-06-09T12:15:12.528705+0000","last_fullsized":"2023-06-09T12:15:12.528705+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1081","last_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_deep_scrub":"75'1081","last_deep_scrub_stamp":"2023-06-09T12:13:11.391302+0000","last_clean_scrub_stamp":"2023-06-09T12:13:11.391302+0000","objects_scrubbed":235,"log_size":1081,"log_dups_size":0,"ondisk_log_size":1081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:34:43.898975+0000","scrub_duration":35.726274396000001,"objects_trimmed":0,"snaptrim_duration":0.037273504999999998,"stat_sum":{"num_bytes":10595216,"num_objects":235,"num_object_clones":105,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":105,"num_read":346,"num_read_kb":14146,"num_write":673,"num_write_kb":19913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"75'1110","reported_seq":3706,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.591494+0000","last_change":"2023-06-09T12:03:46.088571+0000","last_active":"2023-06-09T12:15:32.591494+0000","last_peered":"2023-06-09T12:15:32.591494+0000","last_clean":"2023-06-09T12:15:32.591494+0000","last_became_active":"2023-06-09T11:10:50.202023+0000","last_became_peered":"2023-06-09T11:10:50.202023+0000","last_unstale":"2023-06-09T12:15:32.591494+0000","last_undegraded":"2023-06-09T12:15:32.591494+0000","last_fullsized":"2023-06-09T12:15:32.591494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1110","last_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_deep_scrub":"75'1110","last_deep_scrub_stamp":"2023-06-09T12:03:46.088309+0000","last_clean_scrub_stamp":"2023-06-09T12:03:46.088309+0000","objects_scrubbed":226,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:49:30.465820+0000","scrub_duration":40.931250177999999,"objects_trimmed":0,"snaptrim_duration":0.0098423690000000001,"stat_sum":{"num_bytes":18896273,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":500,"num_read_kb":13712,"num_write":701,"num_write_kb":21541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"75'1137","reported_seq":3863,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.420832+0000","last_change":"2023-06-09T12:07:40.807676+0000","last_active":"2023-06-09T12:15:07.420832+0000","last_peered":"2023-06-09T12:15:07.420832+0000","last_clean":"2023-06-09T12:15:07.420832+0000","last_became_active":"2023-06-09T11:10:50.212391+0000","last_became_peered":"2023-06-09T11:10:50.212391+0000","last_unstale":"2023-06-09T12:15:07.420832+0000","last_undegraded":"2023-06-09T12:15:07.420832+0000","last_fullsized":"2023-06-09T12:15:07.420832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1137","last_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_deep_scrub":"75'1137","last_deep_scrub_stamp":"2023-06-09T12:07:40.807458+0000","last_clean_scrub_stamp":"2023-06-09T12:07:40.807458+0000","objects_scrubbed":231,"log_size":1137,"log_dups_size":0,"ondisk_log_size":1137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:26:17.887740+0000","scrub_duration":38.704764068999999,"objects_trimmed":0,"snaptrim_duration":0.046179427000000002,"stat_sum":{"num_bytes":13836389,"num_objects":231,"num_object_clones":105,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":105,"num_read":638,"num_read_kb":16386,"num_write":732,"num_write_kb":16066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"75'1107","reported_seq":3611,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532404+0000","last_change":"2023-06-09T12:00:51.403386+0000","last_active":"2023-06-09T12:15:37.532404+0000","last_peered":"2023-06-09T12:15:37.532404+0000","last_clean":"2023-06-09T12:15:37.532404+0000","last_became_active":"2023-06-09T11:10:50.196743+0000","last_became_peered":"2023-06-09T11:10:50.196743+0000","last_unstale":"2023-06-09T12:15:37.532404+0000","last_undegraded":"2023-06-09T12:15:37.532404+0000","last_fullsized":"2023-06-09T12:15:37.532404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1107","last_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_deep_scrub":"75'1107","last_deep_scrub_stamp":"2023-06-09T12:00:51.403083+0000","last_clean_scrub_stamp":"2023-06-09T12:00:51.403083+0000","objects_scrubbed":243,"log_size":1107,"log_dups_size":0,"ondisk_log_size":1107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:28:40.423465+0000","scrub_duration":32.749174429,"objects_trimmed":0,"snaptrim_duration":0.028147716,"stat_sum":{"num_bytes":5582221,"num_objects":243,"num_object_clones":108,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":108,"num_read":422,"num_read_kb":5561,"num_write":693,"num_write_kb":7660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"75'1113","reported_seq":3356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.509778+0000","last_change":"2023-06-09T12:01:30.587711+0000","last_active":"2023-06-09T12:16:22.509778+0000","last_peered":"2023-06-09T12:16:22.509778+0000","last_clean":"2023-06-09T12:16:22.509778+0000","last_became_active":"2023-06-09T11:10:50.210022+0000","last_became_peered":"2023-06-09T11:10:50.210022+0000","last_unstale":"2023-06-09T12:16:22.509778+0000","last_undegraded":"2023-06-09T12:16:22.509778+0000","last_fullsized":"2023-06-09T12:16:22.509778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:01:30.587475+0000","last_clean_scrub_stamp":"2023-06-09T12:01:30.587475+0000","objects_scrubbed":240,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:03:43.001290+0000","scrub_duration":34.604680133000002,"objects_trimmed":0,"snaptrim_duration":0.018730389,"stat_sum":{"num_bytes":10082034,"num_objects":240,"num_object_clones":110,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":110,"num_read":162,"num_read_kb":3835,"num_write":686,"num_write_kb":14925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"75'1094","reported_seq":3497,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:22.421827+0000","last_change":"2023-06-09T12:01:55.529709+0000","last_active":"2023-06-09T12:14:22.421827+0000","last_peered":"2023-06-09T12:14:22.421827+0000","last_clean":"2023-06-09T12:14:22.421827+0000","last_became_active":"2023-06-09T11:10:50.199994+0000","last_became_peered":"2023-06-09T11:10:50.199994+0000","last_unstale":"2023-06-09T12:14:22.421827+0000","last_undegraded":"2023-06-09T12:14:22.421827+0000","last_fullsized":"2023-06-09T12:14:22.421827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1094","last_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_deep_scrub":"75'1094","last_deep_scrub_stamp":"2023-06-09T12:01:55.529432+0000","last_clean_scrub_stamp":"2023-06-09T12:01:55.529432+0000","objects_scrubbed":226,"log_size":1094,"log_dups_size":0,"ondisk_log_size":1094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:19:01.047550+0000","scrub_duration":35.350869512000003,"objects_trimmed":0,"snaptrim_duration":0.018982470000000001,"stat_sum":{"num_bytes":9841285,"num_objects":226,"num_object_clones":104,"num_object_copies":452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":226,"num_whiteouts":104,"num_read":316,"num_read_kb":7798,"num_write":688,"num_write_kb":10725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"75'1263","reported_seq":4543,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.204064+0000","last_change":"2023-06-09T12:16:42.204064+0000","last_active":"2023-06-09T12:16:42.204064+0000","last_peered":"2023-06-09T12:16:42.204064+0000","last_clean":"2023-06-09T12:16:42.204064+0000","last_became_active":"2023-06-09T11:10:50.213640+0000","last_became_peered":"2023-06-09T11:10:50.213640+0000","last_unstale":"2023-06-09T12:16:42.204064+0000","last_undegraded":"2023-06-09T12:16:42.204064+0000","last_fullsized":"2023-06-09T12:16:42.204064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1263","last_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_deep_scrub":"75'1263","last_deep_scrub_stamp":"2023-06-09T12:16:42.203814+0000","last_clean_scrub_stamp":"2023-06-09T12:16:42.203814+0000","objects_scrubbed":294,"log_size":1263,"log_dups_size":0,"ondisk_log_size":1263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":57,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:54:08.280794+0000","scrub_duration":56.568470230000003,"objects_trimmed":0,"snaptrim_duration":0.037435484999999998,"stat_sum":{"num_bytes":27700896,"num_objects":294,"num_object_clones":131,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":131,"num_read":1195,"num_read_kb":27445,"num_write":764,"num_write_kb":33492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"75'1116","reported_seq":3975,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.428702+0000","last_change":"2023-06-09T12:05:41.384554+0000","last_active":"2023-06-09T12:15:12.428702+0000","last_peered":"2023-06-09T12:15:12.428702+0000","last_clean":"2023-06-09T12:15:12.428702+0000","last_became_active":"2023-06-09T11:10:50.202325+0000","last_became_peered":"2023-06-09T11:10:50.202325+0000","last_unstale":"2023-06-09T12:15:12.428702+0000","last_undegraded":"2023-06-09T12:15:12.428702+0000","last_fullsized":"2023-06-09T12:15:12.428702+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1116","last_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_deep_scrub":"75'1116","last_deep_scrub_stamp":"2023-06-09T12:05:41.384373+0000","last_clean_scrub_stamp":"2023-06-09T12:05:41.384373+0000","objects_scrubbed":254,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:56:00.057863+0000","scrub_duration":44.281539770999998,"objects_trimmed":0,"snaptrim_duration":0.027921923000000001,"stat_sum":{"num_bytes":19182897,"num_objects":254,"num_object_clones":112,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":112,"num_read":776,"num_read_kb":20764,"num_write":676,"num_write_kb":20191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"75'1120","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:11.295517+0000","last_change":"2023-06-09T12:17:11.295604+0000","last_active":"2023-06-09T12:17:11.295517+0000","last_peered":"2023-06-09T12:17:11.295517+0000","last_clean":"2023-06-09T12:17:11.295517+0000","last_became_active":"2023-06-09T11:10:50.207914+0000","last_became_peered":"2023-06-09T11:10:50.207914+0000","last_unstale":"2023-06-09T12:17:11.295517+0000","last_undegraded":"2023-06-09T12:17:11.295517+0000","last_fullsized":"2023-06-09T12:17:11.295517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1120","last_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_deep_scrub":"75'1120","last_deep_scrub_stamp":"2023-06-09T12:17:11.295402+0000","last_clean_scrub_stamp":"2023-06-09T12:17:11.295402+0000","objects_scrubbed":255,"log_size":1120,"log_dups_size":0,"ondisk_log_size":1120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:40:59.715584+0000","scrub_duration":38.331414459000001,"objects_trimmed":0,"snaptrim_duration":0.046316651,"stat_sum":{"num_bytes":10377032,"num_objects":255,"num_object_clones":117,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":117,"num_read":544,"num_read_kb":7888,"num_write":675,"num_write_kb":12109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"75'1256","reported_seq":3745,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.467192+0000","last_change":"2023-06-09T12:07:43.327453+0000","last_active":"2023-06-09T12:14:57.467192+0000","last_peered":"2023-06-09T12:14:57.467192+0000","last_clean":"2023-06-09T12:14:57.467192+0000","last_became_active":"2023-06-09T11:10:50.199090+0000","last_became_peered":"2023-06-09T11:10:50.199090+0000","last_unstale":"2023-06-09T12:14:57.467192+0000","last_undegraded":"2023-06-09T12:14:57.467192+0000","last_fullsized":"2023-06-09T12:14:57.467192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1256","last_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_deep_scrub":"75'1256","last_deep_scrub_stamp":"2023-06-09T12:07:43.327229+0000","last_clean_scrub_stamp":"2023-06-09T12:07:43.327229+0000","objects_scrubbed":304,"log_size":1256,"log_dups_size":0,"ondisk_log_size":1256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:55:44.933504+0000","scrub_duration":47.638681759000001,"objects_trimmed":0,"snaptrim_duration":0.028041825999999999,"stat_sum":{"num_bytes":15038820,"num_objects":304,"num_object_clones":157,"num_object_copies":608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":122,"num_read":440,"num_read_kb":10110,"num_write":733,"num_write_kb":21183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"75'1124","reported_seq":3527,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.467639+0000","last_change":"2023-06-09T12:05:29.365345+0000","last_active":"2023-06-09T12:15:02.467639+0000","last_peered":"2023-06-09T12:15:02.467639+0000","last_clean":"2023-06-09T12:15:02.467639+0000","last_became_active":"2023-06-09T11:10:50.207668+0000","last_became_peered":"2023-06-09T11:10:50.207668+0000","last_unstale":"2023-06-09T12:15:02.467639+0000","last_undegraded":"2023-06-09T12:15:02.467639+0000","last_fullsized":"2023-06-09T12:15:02.467639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1124","last_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_deep_scrub":"75'1124","last_deep_scrub_stamp":"2023-06-09T12:05:29.365069+0000","last_clean_scrub_stamp":"2023-06-09T12:05:29.365069+0000","objects_scrubbed":262,"log_size":1124,"log_dups_size":0,"ondisk_log_size":1124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:16:57.982968+0000","scrub_duration":38.706507154999997,"objects_trimmed":0,"snaptrim_duration":0.01027873,"stat_sum":{"num_bytes":10639129,"num_objects":262,"num_object_clones":121,"num_object_copies":524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":262,"num_whiteouts":121,"num_read":324,"num_read_kb":12934,"num_write":662,"num_write_kb":20376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"75'1062","reported_seq":3537,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.439149+0000","last_change":"2023-06-09T12:02:34.000810+0000","last_active":"2023-06-09T12:16:32.439149+0000","last_peered":"2023-06-09T12:16:32.439149+0000","last_clean":"2023-06-09T12:16:32.439149+0000","last_became_active":"2023-06-09T11:10:50.196270+0000","last_became_peered":"2023-06-09T11:10:50.196270+0000","last_unstale":"2023-06-09T12:16:32.439149+0000","last_undegraded":"2023-06-09T12:16:32.439149+0000","last_fullsized":"2023-06-09T12:16:32.439149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:02:34.000589+0000","last_clean_scrub_stamp":"2023-06-09T12:02:34.000589+0000","objects_scrubbed":203,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:52:03.512874+0000","scrub_duration":34.978714775999997,"objects_trimmed":0,"snaptrim_duration":0.010790174,"stat_sum":{"num_bytes":14219573,"num_objects":203,"num_object_clones":90,"num_object_copies":406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":203,"num_whiteouts":90,"num_read":373,"num_read_kb":12499,"num_write":689,"num_write_kb":15910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"75'1195","reported_seq":3631,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.512396+0000","last_change":"2023-06-09T12:05:17.064577+0000","last_active":"2023-06-09T12:16:42.512396+0000","last_peered":"2023-06-09T12:16:42.512396+0000","last_clean":"2023-06-09T12:16:42.512396+0000","last_became_active":"2023-06-09T11:10:50.209837+0000","last_became_peered":"2023-06-09T11:10:50.209837+0000","last_unstale":"2023-06-09T12:16:42.512396+0000","last_undegraded":"2023-06-09T12:16:42.512396+0000","last_fullsized":"2023-06-09T12:16:42.512396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1195","last_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_deep_scrub":"75'1195","last_deep_scrub_stamp":"2023-06-09T12:05:17.064334+0000","last_clean_scrub_stamp":"2023-06-09T12:05:17.064334+0000","objects_scrubbed":260,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:51:17.433531+0000","scrub_duration":38.325622912,"objects_trimmed":0,"snaptrim_duration":0.027604704000000001,"stat_sum":{"num_bytes":10141393,"num_objects":260,"num_object_clones":119,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":119,"num_read":360,"num_read_kb":6648,"num_write":750,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"75'1207","reported_seq":3709,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.541526+0000","last_change":"2023-06-09T12:02:52.490865+0000","last_active":"2023-06-09T12:16:42.541526+0000","last_peered":"2023-06-09T12:16:42.541526+0000","last_clean":"2023-06-09T12:16:42.541526+0000","last_became_active":"2023-06-09T11:10:50.197240+0000","last_became_peered":"2023-06-09T11:10:50.197240+0000","last_unstale":"2023-06-09T12:16:42.541526+0000","last_undegraded":"2023-06-09T12:16:42.541526+0000","last_fullsized":"2023-06-09T12:16:42.541526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1207","last_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_deep_scrub":"75'1207","last_deep_scrub_stamp":"2023-06-09T12:02:52.490531+0000","last_clean_scrub_stamp":"2023-06-09T12:02:52.490531+0000","objects_scrubbed":254,"log_size":1207,"log_dups_size":0,"ondisk_log_size":1207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:44:27.570653+0000","scrub_duration":38.703532963000001,"objects_trimmed":0,"snaptrim_duration":0.04607704,"stat_sum":{"num_bytes":10061013,"num_objects":254,"num_object_clones":116,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":116,"num_read":424,"num_read_kb":9054,"num_write":758,"num_write_kb":15588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"75'1155","reported_seq":4007,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.385503+0000","last_change":"2023-06-09T12:14:57.385503+0000","last_active":"2023-06-09T12:14:57.385503+0000","last_peered":"2023-06-09T12:14:57.385503+0000","last_clean":"2023-06-09T12:14:57.385503+0000","last_became_active":"2023-06-09T11:10:50.212139+0000","last_became_peered":"2023-06-09T11:10:50.212139+0000","last_unstale":"2023-06-09T12:14:57.385503+0000","last_undegraded":"2023-06-09T12:14:57.385503+0000","last_fullsized":"2023-06-09T12:14:57.385503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1155","last_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_deep_scrub":"75'1155","last_deep_scrub_stamp":"2023-06-09T12:14:57.385221+0000","last_clean_scrub_stamp":"2023-06-09T12:14:57.385221+0000","objects_scrubbed":233,"log_size":1155,"log_dups_size":0,"ondisk_log_size":1155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:07:46.947987+0000","scrub_duration":44.659341412000003,"objects_trimmed":0,"snaptrim_duration":0.046450808000000003,"stat_sum":{"num_bytes":23004865,"num_objects":233,"num_object_clones":102,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":102,"num_read":748,"num_read_kb":13187,"num_write":737,"num_write_kb":29258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"75'1049","reported_seq":3746,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.526689+0000","last_change":"2023-06-09T12:10:24.300499+0000","last_active":"2023-06-09T12:14:57.526689+0000","last_peered":"2023-06-09T12:14:57.526689+0000","last_clean":"2023-06-09T12:14:57.526689+0000","last_became_active":"2023-06-09T11:10:50.198656+0000","last_became_peered":"2023-06-09T11:10:50.198656+0000","last_unstale":"2023-06-09T12:14:57.526689+0000","last_undegraded":"2023-06-09T12:14:57.526689+0000","last_fullsized":"2023-06-09T12:14:57.526689+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1049","last_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_deep_scrub":"75'1049","last_deep_scrub_stamp":"2023-06-09T12:10:24.300253+0000","last_clean_scrub_stamp":"2023-06-09T12:10:24.300253+0000","objects_scrubbed":228,"log_size":1049,"log_dups_size":0,"ondisk_log_size":1049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:15:02.548511+0000","scrub_duration":38.703563711999998,"objects_trimmed":0,"snaptrim_duration":0.046198698000000003,"stat_sum":{"num_bytes":14554771,"num_objects":228,"num_object_clones":99,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":228,"num_whiteouts":99,"num_read":602,"num_read_kb":14720,"num_write":647,"num_write_kb":19973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"75'1114","reported_seq":3816,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:52.466545+0000","last_change":"2023-06-09T12:01:29.655122+0000","last_active":"2023-06-09T12:14:52.466545+0000","last_peered":"2023-06-09T12:14:52.466545+0000","last_clean":"2023-06-09T12:14:52.466545+0000","last_became_active":"2023-06-09T11:10:50.202928+0000","last_became_peered":"2023-06-09T11:10:50.202928+0000","last_unstale":"2023-06-09T12:14:52.466545+0000","last_undegraded":"2023-06-09T12:14:52.466545+0000","last_fullsized":"2023-06-09T12:14:52.466545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1114","last_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_deep_scrub":"75'1114","last_deep_scrub_stamp":"2023-06-09T12:01:29.654929+0000","last_clean_scrub_stamp":"2023-06-09T12:01:29.654929+0000","objects_scrubbed":234,"log_size":1114,"log_dups_size":0,"ondisk_log_size":1114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:49:53.482545+0000","scrub_duration":41.683815174999999,"objects_trimmed":0,"snaptrim_duration":0.010075739,"stat_sum":{"num_bytes":18250362,"num_objects":234,"num_object_clones":105,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":105,"num_read":613,"num_read_kb":17338,"num_write":690,"num_write_kb":23972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"75'1187","reported_seq":3392,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533865+0000","last_change":"2023-06-09T12:04:42.030595+0000","last_active":"2023-06-09T12:15:47.533865+0000","last_peered":"2023-06-09T12:15:47.533865+0000","last_clean":"2023-06-09T12:15:47.533865+0000","last_became_active":"2023-06-09T11:10:50.205910+0000","last_became_peered":"2023-06-09T11:10:50.205910+0000","last_unstale":"2023-06-09T12:15:47.533865+0000","last_undegraded":"2023-06-09T12:15:47.533865+0000","last_fullsized":"2023-06-09T12:15:47.533865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1187","last_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_deep_scrub":"75'1187","last_deep_scrub_stamp":"2023-06-09T12:04:42.030372+0000","last_clean_scrub_stamp":"2023-06-09T12:04:42.030372+0000","objects_scrubbed":263,"log_size":1187,"log_dups_size":0,"ondisk_log_size":1187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:59:38.308620+0000","scrub_duration":41.680736346000003,"objects_trimmed":0,"snaptrim_duration":0.036938741999999997,"stat_sum":{"num_bytes":14670699,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":129,"num_read_kb":10157,"num_write":730,"num_write_kb":25995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"75'1139","reported_seq":3792,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503208+0000","last_change":"2023-06-09T12:04:19.929377+0000","last_active":"2023-06-09T12:15:27.503208+0000","last_peered":"2023-06-09T12:15:27.503208+0000","last_clean":"2023-06-09T12:15:27.503208+0000","last_became_active":"2023-06-09T11:11:38.191685+0000","last_became_peered":"2023-06-09T11:11:38.191685+0000","last_unstale":"2023-06-09T12:15:27.503208+0000","last_undegraded":"2023-06-09T12:15:27.503208+0000","last_fullsized":"2023-06-09T12:15:27.503208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1139","last_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_deep_scrub":"75'1139","last_deep_scrub_stamp":"2023-06-09T12:04:19.929151+0000","last_clean_scrub_stamp":"2023-06-09T12:04:19.929151+0000","objects_scrubbed":240,"log_size":1139,"log_dups_size":0,"ondisk_log_size":1139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:31:51.058062+0000","scrub_duration":37.953514339000002,"objects_trimmed":0,"snaptrim_duration":0.0095549889999999998,"stat_sum":{"num_bytes":14650127,"num_objects":240,"num_object_clones":108,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":108,"num_read":561,"num_read_kb":9869,"num_write":732,"num_write_kb":16314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"75'1147","reported_seq":3751,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:52.434169+0000","last_change":"2023-06-09T12:10:10.762290+0000","last_active":"2023-06-09T12:16:52.434169+0000","last_peered":"2023-06-09T12:16:52.434169+0000","last_clean":"2023-06-09T12:16:52.434169+0000","last_became_active":"2023-06-09T11:10:50.209973+0000","last_became_peered":"2023-06-09T11:10:50.209973+0000","last_unstale":"2023-06-09T12:16:52.434169+0000","last_undegraded":"2023-06-09T12:16:52.434169+0000","last_fullsized":"2023-06-09T12:16:52.434169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:10:10.762056+0000","last_clean_scrub_stamp":"2023-06-09T12:10:10.762056+0000","objects_scrubbed":239,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:07:39.969348+0000","scrub_duration":38.704844121000001,"objects_trimmed":0,"snaptrim_duration":0.0098203979999999993,"stat_sum":{"num_bytes":14341899,"num_objects":239,"num_object_clones":106,"num_object_copies":478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":239,"num_whiteouts":106,"num_read":512,"num_read_kb":7203,"num_write":721,"num_write_kb":20218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"75'1163","reported_seq":3813,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.410913+0000","last_change":"2023-06-09T12:01:33.603990+0000","last_active":"2023-06-09T12:14:57.410913+0000","last_peered":"2023-06-09T12:14:57.410913+0000","last_clean":"2023-06-09T12:14:57.410913+0000","last_became_active":"2023-06-09T11:10:50.203690+0000","last_became_peered":"2023-06-09T11:10:50.203690+0000","last_unstale":"2023-06-09T12:14:57.410913+0000","last_undegraded":"2023-06-09T12:14:57.410913+0000","last_fullsized":"2023-06-09T12:14:57.410913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1163","last_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_deep_scrub":"75'1163","last_deep_scrub_stamp":"2023-06-09T12:01:33.603702+0000","last_clean_scrub_stamp":"2023-06-09T12:01:33.603702+0000","objects_scrubbed":254,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:11:51.353276+0000","scrub_duration":38.926514513999997,"objects_trimmed":0,"snaptrim_duration":0.068541098999999994,"stat_sum":{"num_bytes":10171364,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":569,"num_read_kb":13313,"num_write":718,"num_write_kb":16254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"75'1184","reported_seq":3652,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.510374+0000","last_change":"2023-06-09T12:12:18.820760+0000","last_active":"2023-06-09T12:16:27.510374+0000","last_peered":"2023-06-09T12:16:27.510374+0000","last_clean":"2023-06-09T12:16:27.510374+0000","last_became_active":"2023-06-09T11:10:50.210993+0000","last_became_peered":"2023-06-09T11:10:50.210993+0000","last_unstale":"2023-06-09T12:16:27.510374+0000","last_undegraded":"2023-06-09T12:16:27.510374+0000","last_fullsized":"2023-06-09T12:16:27.510374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1184","last_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_deep_scrub":"75'1184","last_deep_scrub_stamp":"2023-06-09T12:12:18.820517+0000","last_clean_scrub_stamp":"2023-06-09T12:12:18.820517+0000","objects_scrubbed":253,"log_size":1184,"log_dups_size":0,"ondisk_log_size":1184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":41,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:48:36.915332+0000","scrub_duration":40.558004861999997,"objects_trimmed":0,"snaptrim_duration":0.0096362300000000008,"stat_sum":{"num_bytes":15007983,"num_objects":253,"num_object_clones":116,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":116,"num_read":373,"num_read_kb":12337,"num_write":739,"num_write_kb":17529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"75'1075","reported_seq":3264,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404456+0000","last_change":"2023-06-09T12:02:38.924294+0000","last_active":"2023-06-09T12:15:27.404456+0000","last_peered":"2023-06-09T12:15:27.404456+0000","last_clean":"2023-06-09T12:15:27.404456+0000","last_became_active":"2023-06-09T11:10:50.204429+0000","last_became_peered":"2023-06-09T11:10:50.204429+0000","last_unstale":"2023-06-09T12:15:27.404456+0000","last_undegraded":"2023-06-09T12:15:27.404456+0000","last_fullsized":"2023-06-09T12:15:27.404456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1075","last_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_deep_scrub":"75'1075","last_deep_scrub_stamp":"2023-06-09T12:02:38.924052+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.924052+0000","objects_scrubbed":221,"log_size":1075,"log_dups_size":0,"ondisk_log_size":1075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:20:52.387782+0000","scrub_duration":29.771506761000001,"objects_trimmed":0,"snaptrim_duration":0.028092003000000001,"stat_sum":{"num_bytes":6152813,"num_objects":221,"num_object_clones":100,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":100,"num_read":96,"num_read_kb":4116,"num_write":695,"num_write_kb":7307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"75'1183","reported_seq":3287,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.409964+0000","last_change":"2023-06-09T12:03:12.780906+0000","last_active":"2023-06-09T12:16:12.409964+0000","last_peered":"2023-06-09T12:16:12.409964+0000","last_clean":"2023-06-09T12:16:12.409964+0000","last_became_active":"2023-06-09T11:10:50.202822+0000","last_became_peered":"2023-06-09T11:10:50.202822+0000","last_unstale":"2023-06-09T12:16:12.409964+0000","last_undegraded":"2023-06-09T12:16:12.409964+0000","last_fullsized":"2023-06-09T12:16:12.409964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1183","last_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_deep_scrub":"75'1183","last_deep_scrub_stamp":"2023-06-09T12:03:12.780620+0000","last_clean_scrub_stamp":"2023-06-09T12:03:12.780620+0000","objects_scrubbed":251,"log_size":1183,"log_dups_size":0,"ondisk_log_size":1183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:52.863588+0000","scrub_duration":32.748620819999999,"objects_trimmed":0,"snaptrim_duration":0.037338903999999999,"stat_sum":{"num_bytes":1656046,"num_objects":251,"num_object_clones":113,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":113,"num_read":21,"num_read_kb":26,"num_write":733,"num_write_kb":8563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"75'1229","reported_seq":3914,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:32.431434+0000","last_change":"2023-06-09T12:07:04.274195+0000","last_active":"2023-06-09T12:15:32.431434+0000","last_peered":"2023-06-09T12:15:32.431434+0000","last_clean":"2023-06-09T12:15:32.431434+0000","last_became_active":"2023-06-09T11:10:50.199349+0000","last_became_peered":"2023-06-09T11:10:50.199349+0000","last_unstale":"2023-06-09T12:15:32.431434+0000","last_undegraded":"2023-06-09T12:15:32.431434+0000","last_fullsized":"2023-06-09T12:15:32.431434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1229","last_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_deep_scrub":"75'1229","last_deep_scrub_stamp":"2023-06-09T12:07:04.273939+0000","last_clean_scrub_stamp":"2023-06-09T12:07:04.273939+0000","objects_scrubbed":276,"log_size":1229,"log_dups_size":0,"ondisk_log_size":1229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:54:53.829668+0000","scrub_duration":44.281562334,"objects_trimmed":0,"snaptrim_duration":0.019365734999999999,"stat_sum":{"num_bytes":14195707,"num_objects":276,"num_object_clones":124,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":124,"num_read":608,"num_read_kb":9807,"num_write":766,"num_write_kb":15531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"75'3003","reported_seq":4129,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.129157+0000","last_change":"2023-06-09T12:16:32.129157+0000","last_active":"2023-06-09T12:16:32.129157+0000","last_peered":"2023-06-09T12:16:32.129157+0000","last_clean":"2023-06-09T12:16:32.129157+0000","last_became_active":"2023-06-09T11:10:49.683788+0000","last_became_peered":"2023-06-09T11:10:49.683788+0000","last_unstale":"2023-06-09T12:16:32.129157+0000","last_undegraded":"2023-06-09T12:16:32.129157+0000","last_fullsized":"2023-06-09T12:16:32.129157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'3003","last_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_deep_scrub":"75'3003","last_deep_scrub_stamp":"2023-06-09T12:16:32.128844+0000","last_clean_scrub_stamp":"2023-06-09T12:16:32.128844+0000","objects_scrubbed":24,"log_size":3003,"log_dups_size":0,"ondisk_log_size":3003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":31,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:57:24.168692+0000","scrub_duration":30.516283182999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38800005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":989,"num_read_kb":130413,"num_write":2910,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"75'1134","reported_seq":3585,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.416523+0000","last_change":"2023-06-09T12:10:50.396949+0000","last_active":"2023-06-09T12:15:37.416523+0000","last_peered":"2023-06-09T12:15:37.416523+0000","last_clean":"2023-06-09T12:15:37.416523+0000","last_became_active":"2023-06-09T11:10:50.218056+0000","last_became_peered":"2023-06-09T11:10:50.218056+0000","last_unstale":"2023-06-09T12:15:37.416523+0000","last_undegraded":"2023-06-09T12:15:37.416523+0000","last_fullsized":"2023-06-09T12:15:37.416523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1134","last_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_deep_scrub":"75'1134","last_deep_scrub_stamp":"2023-06-09T12:10:50.396611+0000","last_clean_scrub_stamp":"2023-06-09T12:10:50.396611+0000","objects_scrubbed":241,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:36:02.200274+0000","scrub_duration":32.937853038999997,"objects_trimmed":0,"snaptrim_duration":0.057440660999999997,"stat_sum":{"num_bytes":6373211,"num_objects":241,"num_object_clones":108,"num_object_copies":482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":241,"num_whiteouts":108,"num_read":367,"num_read_kb":5803,"num_write":708,"num_write_kb":12593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"75'1252","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.403174+0000","last_change":"2023-06-09T12:00:54.225246+0000","last_active":"2023-06-09T12:15:17.403174+0000","last_peered":"2023-06-09T12:15:17.403174+0000","last_clean":"2023-06-09T12:15:17.403174+0000","last_became_active":"2023-06-09T11:10:49.678689+0000","last_became_peered":"2023-06-09T11:10:49.678689+0000","last_unstale":"2023-06-09T12:15:17.403174+0000","last_undegraded":"2023-06-09T12:15:17.403174+0000","last_fullsized":"2023-06-09T12:15:17.403174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1252","last_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_deep_scrub":"75'1252","last_deep_scrub_stamp":"2023-06-09T12:00:54.224999+0000","last_clean_scrub_stamp":"2023-06-09T12:00:54.224999+0000","objects_scrubbed":14,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":10,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:03:12.251129+0000","scrub_duration":9.3036959590000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9319694,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":883,"num_read_kb":22083,"num_write":1172,"num_write_kb":14087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":76023,"num_omap_keys":143,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"75'1208","reported_seq":3449,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:47.433411+0000","last_change":"2023-06-09T12:10:56.673924+0000","last_active":"2023-06-09T12:16:47.433411+0000","last_peered":"2023-06-09T12:16:47.433411+0000","last_clean":"2023-06-09T12:16:47.433411+0000","last_became_active":"2023-06-09T11:10:50.200789+0000","last_became_peered":"2023-06-09T11:10:50.200789+0000","last_unstale":"2023-06-09T12:16:47.433411+0000","last_undegraded":"2023-06-09T12:16:47.433411+0000","last_fullsized":"2023-06-09T12:16:47.433411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1208","last_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_deep_scrub":"75'1208","last_deep_scrub_stamp":"2023-06-09T12:10:56.673650+0000","last_clean_scrub_stamp":"2023-06-09T12:10:56.673650+0000","objects_scrubbed":281,"log_size":1208,"log_dups_size":0,"ondisk_log_size":1208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:20:49.444604+0000","scrub_duration":41.682058517999998,"objects_trimmed":0,"snaptrim_duration":0.037195203000000003,"stat_sum":{"num_bytes":9905016,"num_objects":281,"num_object_clones":127,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":127,"num_read":166,"num_read_kb":3569,"num_write":736,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"75'1834","reported_seq":2030,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.406983+0000","last_change":"2023-06-09T12:04:49.730741+0000","last_active":"2023-06-09T12:14:27.406983+0000","last_peered":"2023-06-09T12:14:27.406983+0000","last_clean":"2023-06-09T12:14:27.406983+0000","last_became_active":"2023-06-09T11:10:49.677081+0000","last_became_peered":"2023-06-09T11:10:49.677081+0000","last_unstale":"2023-06-09T12:14:27.406983+0000","last_undegraded":"2023-06-09T12:14:27.406983+0000","last_fullsized":"2023-06-09T12:14:27.406983+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1834","last_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_deep_scrub":"75'1834","last_deep_scrub_stamp":"2023-06-09T12:04:49.730476+0000","last_clean_scrub_stamp":"2023-06-09T12:04:49.730476+0000","objects_scrubbed":16,"log_size":1834,"log_dups_size":0,"ondisk_log_size":1834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:35:16.858655+0000","scrub_duration":20.586260668000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":89548,"num_write":1758,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":92728,"num_omap_keys":185,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"75'1118","reported_seq":3612,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:02.601744+0000","last_change":"2023-06-09T12:10:52.712736+0000","last_active":"2023-06-09T12:17:02.601744+0000","last_peered":"2023-06-09T12:17:02.601744+0000","last_clean":"2023-06-09T12:17:02.601744+0000","last_became_active":"2023-06-09T11:10:50.202759+0000","last_became_peered":"2023-06-09T11:10:50.202759+0000","last_unstale":"2023-06-09T12:17:02.601744+0000","last_undegraded":"2023-06-09T12:17:02.601744+0000","last_fullsized":"2023-06-09T12:17:02.601744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1118","last_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_deep_scrub":"75'1118","last_deep_scrub_stamp":"2023-06-09T12:10:52.712527+0000","last_clean_scrub_stamp":"2023-06-09T12:10:52.712527+0000","objects_scrubbed":256,"log_size":1118,"log_dups_size":0,"ondisk_log_size":1118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:34:00.304462+0000","scrub_duration":35.721786051000002,"objects_trimmed":0,"snaptrim_duration":0.0097759040000000002,"stat_sum":{"num_bytes":5798941,"num_objects":256,"num_object_clones":118,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":118,"num_read":391,"num_read_kb":5654,"num_write":696,"num_write_kb":7895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"75'188","reported_seq":370,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.428358+0000","last_change":"2023-06-09T12:06:10.284077+0000","last_active":"2023-06-09T12:16:07.428358+0000","last_peered":"2023-06-09T12:16:07.428358+0000","last_clean":"2023-06-09T12:16:07.428358+0000","last_became_active":"2023-06-09T11:10:49.677193+0000","last_became_peered":"2023-06-09T11:10:49.677193+0000","last_unstale":"2023-06-09T12:16:07.428358+0000","last_undegraded":"2023-06-09T12:16:07.428358+0000","last_fullsized":"2023-06-09T12:16:07.428358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'188","last_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_deep_scrub":"75'188","last_deep_scrub_stamp":"2023-06-09T12:06:10.283839+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.283839+0000","objects_scrubbed":6,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:40:08.359217+0000","scrub_duration":2.9774605809999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":84,"num_write":126,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":31076,"num_omap_keys":62,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"75'1133","reported_seq":3231,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:43.778093+0000","last_change":"2023-06-09T12:15:43.778093+0000","last_active":"2023-06-09T12:15:43.778093+0000","last_peered":"2023-06-09T12:15:43.778093+0000","last_clean":"2023-06-09T12:15:43.778093+0000","last_became_active":"2023-06-09T11:10:50.204619+0000","last_became_peered":"2023-06-09T11:10:50.204619+0000","last_unstale":"2023-06-09T12:15:43.778093+0000","last_undegraded":"2023-06-09T12:15:43.778093+0000","last_fullsized":"2023-06-09T12:15:43.778093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1133","last_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_deep_scrub":"75'1133","last_deep_scrub_stamp":"2023-06-09T12:15:43.777855+0000","last_clean_scrub_stamp":"2023-06-09T12:15:43.777855+0000","objects_scrubbed":240,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:14:22.445205+0000","scrub_duration":29.943340903999999,"objects_trimmed":0,"snaptrim_duration":0.035451466000000001,"stat_sum":{"num_bytes":1584071,"num_objects":240,"num_object_clones":111,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":111,"num_read":15,"num_read_kb":14,"num_write":710,"num_write_kb":3190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"75'1381","reported_seq":1570,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427884+0000","last_change":"2023-06-09T12:00:44.087225+0000","last_active":"2023-06-09T12:15:07.427884+0000","last_peered":"2023-06-09T12:15:07.427884+0000","last_clean":"2023-06-09T12:15:07.427884+0000","last_became_active":"2023-06-09T11:10:49.676491+0000","last_became_peered":"2023-06-09T11:10:49.676491+0000","last_unstale":"2023-06-09T12:15:07.427884+0000","last_undegraded":"2023-06-09T12:15:07.427884+0000","last_fullsized":"2023-06-09T12:15:07.427884+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1381","last_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_deep_scrub":"75'1381","last_deep_scrub_stamp":"2023-06-09T12:00:44.086934+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.086934+0000","objects_scrubbed":16,"log_size":1381,"log_dups_size":0,"ondisk_log_size":1381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":24,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:39:58.641154+0000","scrub_duration":23.815321589,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29360128,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":45,"num_read_kb":62103,"num_write":1314,"num_write_kb":37606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57983,"num_omap_keys":111,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"75'1188","reported_seq":3549,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.412993+0000","last_change":"2023-06-09T12:12:56.334647+0000","last_active":"2023-06-09T12:15:12.412993+0000","last_peered":"2023-06-09T12:15:12.412993+0000","last_clean":"2023-06-09T12:15:12.412993+0000","last_became_active":"2023-06-09T11:10:50.210781+0000","last_became_peered":"2023-06-09T11:10:50.210781+0000","last_unstale":"2023-06-09T12:15:12.412993+0000","last_undegraded":"2023-06-09T12:15:12.412993+0000","last_fullsized":"2023-06-09T12:15:12.412993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1188","last_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_deep_scrub":"75'1188","last_deep_scrub_stamp":"2023-06-09T12:12:56.334355+0000","last_clean_scrub_stamp":"2023-06-09T12:12:56.334355+0000","objects_scrubbed":244,"log_size":1188,"log_dups_size":0,"ondisk_log_size":1188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:51:37.507901+0000","scrub_duration":35.932222592999999,"objects_trimmed":0,"snaptrim_duration":0.079779599000000007,"stat_sum":{"num_bytes":10275535,"num_objects":244,"num_object_clones":108,"num_object_copies":488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":244,"num_whiteouts":108,"num_read":274,"num_read_kb":14257,"num_write":766,"num_write_kb":20687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"75'1087","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.497674+0000","last_change":"2023-06-09T12:02:49.066073+0000","last_active":"2023-06-09T12:14:42.497674+0000","last_peered":"2023-06-09T12:14:42.497674+0000","last_clean":"2023-06-09T12:14:42.497674+0000","last_became_active":"2023-06-09T11:10:49.685551+0000","last_became_peered":"2023-06-09T11:10:49.685551+0000","last_unstale":"2023-06-09T12:14:42.497674+0000","last_undegraded":"2023-06-09T12:14:42.497674+0000","last_fullsized":"2023-06-09T12:14:42.497674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1087","last_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_deep_scrub":"75'1087","last_deep_scrub_stamp":"2023-06-09T12:02:49.065840+0000","last_clean_scrub_stamp":"2023-06-09T12:02:49.065840+0000","objects_scrubbed":14,"log_size":1087,"log_dups_size":0,"ondisk_log_size":1087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":16,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:53:32.027000+0000","scrub_duration":16.000097102000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18491981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":24474,"num_write":1032,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":52418,"num_omap_keys":104,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"75'1244","reported_seq":3419,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.406822+0000","last_change":"2023-06-09T12:06:40.854512+0000","last_active":"2023-06-09T12:15:47.406822+0000","last_peered":"2023-06-09T12:15:47.406822+0000","last_clean":"2023-06-09T12:15:47.406822+0000","last_became_active":"2023-06-09T11:10:50.209177+0000","last_became_peered":"2023-06-09T11:10:50.209177+0000","last_unstale":"2023-06-09T12:15:47.406822+0000","last_undegraded":"2023-06-09T12:15:47.406822+0000","last_fullsized":"2023-06-09T12:15:47.406822+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1244","last_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_deep_scrub":"75'1244","last_deep_scrub_stamp":"2023-06-09T12:06:40.854284+0000","last_clean_scrub_stamp":"2023-06-09T12:06:40.854284+0000","objects_scrubbed":287,"log_size":1244,"log_dups_size":0,"ondisk_log_size":1244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":46,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:43:49.548002+0000","scrub_duration":45.029385699000002,"objects_trimmed":0,"snaptrim_duration":0.028323963000000001,"stat_sum":{"num_bytes":14923133,"num_objects":287,"num_object_clones":129,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":129,"num_read":108,"num_read_kb":6422,"num_write":747,"num_write_kb":24690,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"75'847","reported_seq":1079,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.898525+0000","last_change":"2023-06-09T12:15:57.898525+0000","last_active":"2023-06-09T12:15:57.898525+0000","last_peered":"2023-06-09T12:15:57.898525+0000","last_clean":"2023-06-09T12:15:57.898525+0000","last_became_active":"2023-06-09T11:10:49.677274+0000","last_became_peered":"2023-06-09T11:10:49.677274+0000","last_unstale":"2023-06-09T12:15:57.898525+0000","last_undegraded":"2023-06-09T12:15:57.898525+0000","last_fullsized":"2023-06-09T12:15:57.898525+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'847","last_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_deep_scrub":"75'847","last_deep_scrub_stamp":"2023-06-09T12:15:57.898282+0000","last_clean_scrub_stamp":"2023-06-09T12:15:57.898282+0000","objects_scrubbed":23,"log_size":847,"log_dups_size":0,"ondisk_log_size":847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:48:25.775201+0000","scrub_duration":20.468254956999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25166496,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":108,"num_read_kb":82075,"num_write":769,"num_write_kb":25249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":99136,"num_omap_keys":165,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"75'1212","reported_seq":3399,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.540101+0000","last_change":"2023-06-09T12:09:40.352210+0000","last_active":"2023-06-09T12:16:32.540101+0000","last_peered":"2023-06-09T12:16:32.540101+0000","last_clean":"2023-06-09T12:16:32.540101+0000","last_became_active":"2023-06-09T11:10:50.201388+0000","last_became_peered":"2023-06-09T11:10:50.201388+0000","last_unstale":"2023-06-09T12:16:32.540101+0000","last_undegraded":"2023-06-09T12:16:32.540101+0000","last_fullsized":"2023-06-09T12:16:32.540101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1212","last_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_deep_scrub":"75'1212","last_deep_scrub_stamp":"2023-06-09T12:09:40.351999+0000","last_clean_scrub_stamp":"2023-06-09T12:09:40.351999+0000","objects_scrubbed":273,"log_size":1212,"log_dups_size":0,"ondisk_log_size":1212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":42,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:39:07.560588+0000","scrub_duration":41.680750533000001,"objects_trimmed":0,"snaptrim_duration":0.046478222,"stat_sum":{"num_bytes":11056282,"num_objects":273,"num_object_clones":120,"num_object_copies":546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":273,"num_whiteouts":120,"num_read":111,"num_read_kb":5485,"num_write":749,"num_write_kb":12793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"75'886","reported_seq":1120,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:17.501981+0000","last_change":"2023-06-09T12:06:10.469903+0000","last_active":"2023-06-09T12:15:17.501981+0000","last_peered":"2023-06-09T12:15:17.501981+0000","last_clean":"2023-06-09T12:15:17.501981+0000","last_became_active":"2023-06-09T11:10:49.679177+0000","last_became_peered":"2023-06-09T11:10:49.679177+0000","last_unstale":"2023-06-09T12:15:17.501981+0000","last_undegraded":"2023-06-09T12:15:17.501981+0000","last_fullsized":"2023-06-09T12:15:17.501981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'886","last_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_deep_scrub":"75'886","last_deep_scrub_stamp":"2023-06-09T12:06:10.469540+0000","last_clean_scrub_stamp":"2023-06-09T12:06:10.469540+0000","objects_scrubbed":21,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":14,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:44:38.172159+0000","scrub_duration":13.767529996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15650038,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":143,"num_read_kb":73393,"num_write":807,"num_write_kb":15875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":100203,"num_omap_keys":187,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"75'1180","reported_seq":3861,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:47.409537+0000","last_change":"2023-06-09T12:10:14.679024+0000","last_active":"2023-06-09T12:14:47.409537+0000","last_peered":"2023-06-09T12:14:47.409537+0000","last_clean":"2023-06-09T12:14:47.409537+0000","last_became_active":"2023-06-09T11:11:38.562330+0000","last_became_peered":"2023-06-09T11:11:38.562330+0000","last_unstale":"2023-06-09T12:14:47.409537+0000","last_undegraded":"2023-06-09T12:14:47.409537+0000","last_fullsized":"2023-06-09T12:14:47.409537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:10:14.678842+0000","last_clean_scrub_stamp":"2023-06-09T12:10:14.678842+0000","objects_scrubbed":272,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:26:47.399203+0000","scrub_duration":44.915187011,"objects_trimmed":0,"snaptrim_duration":0.057505518999999998,"stat_sum":{"num_bytes":14875521,"num_objects":272,"num_object_clones":125,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":125,"num_read":604,"num_read_kb":5534,"num_write":708,"num_write_kb":20262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"75'419","reported_seq":599,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428881+0000","last_change":"2023-06-09T11:59:54.393683+0000","last_active":"2023-06-09T12:16:12.428881+0000","last_peered":"2023-06-09T12:16:12.428881+0000","last_clean":"2023-06-09T12:16:12.428881+0000","last_became_active":"2023-06-09T11:10:49.674480+0000","last_became_peered":"2023-06-09T11:10:49.674480+0000","last_unstale":"2023-06-09T12:16:12.428881+0000","last_undegraded":"2023-06-09T12:16:12.428881+0000","last_fullsized":"2023-06-09T12:16:12.428881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'419","last_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_deep_scrub":"75'419","last_deep_scrub_stamp":"2023-06-09T11:59:54.393479+0000","last_clean_scrub_stamp":"2023-06-09T11:59:54.393479+0000","objects_scrubbed":7,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:31:57.733757+0000","scrub_duration":5.9546084170000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":36,"num_read_kb":40990,"num_write":357,"num_write_kb":8486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":40576,"num_omap_keys":81,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"75'1158","reported_seq":3872,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.430700+0000","last_change":"2023-06-09T12:08:15.301877+0000","last_active":"2023-06-09T12:15:27.430700+0000","last_peered":"2023-06-09T12:15:27.430700+0000","last_clean":"2023-06-09T12:15:27.430700+0000","last_became_active":"2023-06-09T11:10:50.199253+0000","last_became_peered":"2023-06-09T11:10:50.199253+0000","last_unstale":"2023-06-09T12:15:27.430700+0000","last_undegraded":"2023-06-09T12:15:27.430700+0000","last_fullsized":"2023-06-09T12:15:27.430700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1158","last_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_deep_scrub":"75'1158","last_deep_scrub_stamp":"2023-06-09T12:08:15.301599+0000","last_clean_scrub_stamp":"2023-06-09T12:08:15.301599+0000","objects_scrubbed":253,"log_size":1158,"log_dups_size":0,"ondisk_log_size":1158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:46:25.308578+0000","scrub_duration":37.955615172999998,"objects_trimmed":0,"snaptrim_duration":0.028463352000000001,"stat_sum":{"num_bytes":10558808,"num_objects":253,"num_object_clones":119,"num_object_copies":506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":253,"num_whiteouts":119,"num_read":635,"num_read_kb":9803,"num_write":718,"num_write_kb":12970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"75'974","reported_seq":1176,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.510413+0000","last_change":"2023-06-09T12:00:17.302647+0000","last_active":"2023-06-09T12:16:27.510413+0000","last_peered":"2023-06-09T12:16:27.510413+0000","last_clean":"2023-06-09T12:16:27.510413+0000","last_became_active":"2023-06-09T11:10:49.678997+0000","last_became_peered":"2023-06-09T11:10:49.678997+0000","last_unstale":"2023-06-09T12:16:27.510413+0000","last_undegraded":"2023-06-09T12:16:27.510413+0000","last_fullsized":"2023-06-09T12:16:27.510413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'974","last_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_deep_scrub":"75'974","last_deep_scrub_stamp":"2023-06-09T12:00:17.302425+0000","last_clean_scrub_stamp":"2023-06-09T12:00:17.302425+0000","objects_scrubbed":13,"log_size":974,"log_dups_size":0,"ondisk_log_size":974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:35:35.058020+0000","scrub_duration":11.907051654,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":55,"num_read_kb":79939,"num_write":891,"num_write_kb":21099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":59538,"num_omap_keys":119,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"75'1206","reported_seq":3307,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:37.432203+0000","last_change":"2023-06-09T12:06:58.095827+0000","last_active":"2023-06-09T12:16:37.432203+0000","last_peered":"2023-06-09T12:16:37.432203+0000","last_clean":"2023-06-09T12:16:37.432203+0000","last_became_active":"2023-06-09T11:10:50.216079+0000","last_became_peered":"2023-06-09T11:10:50.216079+0000","last_unstale":"2023-06-09T12:16:37.432203+0000","last_undegraded":"2023-06-09T12:16:37.432203+0000","last_fullsized":"2023-06-09T12:16:37.432203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1206","last_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_deep_scrub":"75'1206","last_deep_scrub_stamp":"2023-06-09T12:06:58.095640+0000","last_clean_scrub_stamp":"2023-06-09T12:06:58.095640+0000","objects_scrubbed":254,"log_size":1206,"log_dups_size":0,"ondisk_log_size":1206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:24:01.211266+0000","scrub_duration":32.750171831000003,"objects_trimmed":0,"snaptrim_duration":0.028002429999999998,"stat_sum":{"num_bytes":1509659,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":20,"num_read_kb":20,"num_write":760,"num_write_kb":3421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"75'1074","reported_seq":1343,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:37.584861+0000","last_change":"2023-06-09T12:03:02.206482+0000","last_active":"2023-06-09T12:14:37.584861+0000","last_peered":"2023-06-09T12:14:37.584861+0000","last_clean":"2023-06-09T12:14:37.584861+0000","last_became_active":"2023-06-09T11:10:49.676854+0000","last_became_peered":"2023-06-09T11:10:49.676854+0000","last_unstale":"2023-06-09T12:14:37.584861+0000","last_undegraded":"2023-06-09T12:14:37.584861+0000","last_fullsized":"2023-06-09T12:14:37.584861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1074","last_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_deep_scrub":"75'1074","last_deep_scrub_stamp":"2023-06-09T12:03:02.206200+0000","last_clean_scrub_stamp":"2023-06-09T12:03:02.206200+0000","objects_scrubbed":19,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:31:43.162681+0000","scrub_duration":8.9305465280000007,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388720,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":252,"num_read_kb":45363,"num_write":1014,"num_write_kb":13158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":54663,"num_omap_keys":108,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"75'1147","reported_seq":4051,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:27.407076+0000","last_change":"2023-06-09T12:02:28.629528+0000","last_active":"2023-06-09T12:14:27.407076+0000","last_peered":"2023-06-09T12:14:27.407076+0000","last_clean":"2023-06-09T12:14:27.407076+0000","last_became_active":"2023-06-09T11:10:50.213070+0000","last_became_peered":"2023-06-09T11:10:50.213070+0000","last_unstale":"2023-06-09T12:14:27.407076+0000","last_undegraded":"2023-06-09T12:14:27.407076+0000","last_fullsized":"2023-06-09T12:14:27.407076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:02:28.629296+0000","last_clean_scrub_stamp":"2023-06-09T12:02:28.629296+0000","objects_scrubbed":260,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:14:37.138868+0000","scrub_duration":50.903932963999999,"objects_trimmed":0,"snaptrim_duration":0.068078019000000004,"stat_sum":{"num_bytes":27773284,"num_objects":260,"num_object_clones":118,"num_object_copies":520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":260,"num_whiteouts":118,"num_read":829,"num_read_kb":23287,"num_write":688,"num_write_kb":29027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"75'564","reported_seq":759,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.501493+0000","last_change":"2023-06-09T12:06:19.539716+0000","last_active":"2023-06-09T12:15:12.501493+0000","last_peered":"2023-06-09T12:15:12.501493+0000","last_clean":"2023-06-09T12:15:12.501493+0000","last_became_active":"2023-06-09T11:10:49.679844+0000","last_became_peered":"2023-06-09T11:10:49.679844+0000","last_unstale":"2023-06-09T12:15:12.501493+0000","last_undegraded":"2023-06-09T12:15:12.501493+0000","last_fullsized":"2023-06-09T12:15:12.501493+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'564","last_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_deep_scrub":"75'564","last_deep_scrub_stamp":"2023-06-09T12:06:19.539473+0000","last_clean_scrub_stamp":"2023-06-09T12:06:19.539473+0000","objects_scrubbed":14,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:57:58.379392+0000","scrub_duration":8.9304010510000005,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388630,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":46,"num_read_kb":24688,"num_write":874,"num_write_kb":12891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":58679,"num_omap_keys":100,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"75'1153","reported_seq":3260,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.438375+0000","last_change":"2023-06-09T12:03:36.404106+0000","last_active":"2023-06-09T12:16:27.438375+0000","last_peered":"2023-06-09T12:16:27.438375+0000","last_clean":"2023-06-09T12:16:27.438375+0000","last_became_active":"2023-06-09T11:10:50.197063+0000","last_became_peered":"2023-06-09T11:10:50.197063+0000","last_unstale":"2023-06-09T12:16:27.438375+0000","last_undegraded":"2023-06-09T12:16:27.438375+0000","last_fullsized":"2023-06-09T12:16:27.438375+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:03:36.403919+0000","last_clean_scrub_stamp":"2023-06-09T12:03:36.403919+0000","objects_scrubbed":234,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:41:04.280426+0000","scrub_duration":29.396987043999999,"objects_trimmed":0,"snaptrim_duration":0.027945232,"stat_sum":{"num_bytes":2005775,"num_objects":234,"num_object_clones":106,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":234,"num_whiteouts":106,"num_read":17,"num_read_kb":18,"num_write":734,"num_write_kb":8400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"75'523","reported_seq":750,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:09.976955+0000","last_change":"2023-06-09T12:15:09.976955+0000","last_active":"2023-06-09T12:15:09.976955+0000","last_peered":"2023-06-09T12:15:09.976955+0000","last_clean":"2023-06-09T12:15:09.976955+0000","last_became_active":"2023-06-09T11:10:49.681757+0000","last_became_peered":"2023-06-09T11:10:49.681757+0000","last_unstale":"2023-06-09T12:15:09.976955+0000","last_undegraded":"2023-06-09T12:15:09.976955+0000","last_fullsized":"2023-06-09T12:15:09.976955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'523","last_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_deep_scrub":"75'523","last_deep_scrub_stamp":"2023-06-09T12:15:09.976613+0000","last_clean_scrub_stamp":"2023-06-09T12:15:09.976613+0000","objects_scrubbed":12,"log_size":523,"log_dups_size":0,"ondisk_log_size":523,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:14:22.569148+0000","scrub_duration":14.883885531000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":69,"num_read_kb":53319,"num_write":454,"num_write_kb":20836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":34728,"num_omap_keys":69,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"75'1115","reported_seq":3905,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.535402+0000","last_change":"2023-06-09T12:13:57.097526+0000","last_active":"2023-06-09T12:15:57.535402+0000","last_peered":"2023-06-09T12:15:57.535402+0000","last_clean":"2023-06-09T12:15:57.535402+0000","last_became_active":"2023-06-09T11:10:50.207320+0000","last_became_peered":"2023-06-09T11:10:50.207320+0000","last_unstale":"2023-06-09T12:15:57.535402+0000","last_undegraded":"2023-06-09T12:15:57.535402+0000","last_fullsized":"2023-06-09T12:15:57.535402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1115","last_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_deep_scrub":"75'1115","last_deep_scrub_stamp":"2023-06-09T12:13:57.097287+0000","last_clean_scrub_stamp":"2023-06-09T12:13:57.097287+0000","objects_scrubbed":224,"log_size":1115,"log_dups_size":0,"ondisk_log_size":1115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:25:24.496085+0000","scrub_duration":44.657919253999999,"objects_trimmed":0,"snaptrim_duration":0.028987071999999999,"stat_sum":{"num_bytes":22673652,"num_objects":224,"num_object_clones":98,"num_object_copies":448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":98,"num_read":689,"num_read_kb":16591,"num_write":720,"num_write_kb":24975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"75'450","reported_seq":638,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.509845+0000","last_change":"2023-06-09T12:07:35.010734+0000","last_active":"2023-06-09T12:16:22.509845+0000","last_peered":"2023-06-09T12:16:22.509845+0000","last_clean":"2023-06-09T12:16:22.509845+0000","last_became_active":"2023-06-09T11:10:49.681322+0000","last_became_peered":"2023-06-09T11:10:49.681322+0000","last_unstale":"2023-06-09T12:16:22.509845+0000","last_undegraded":"2023-06-09T12:16:22.509845+0000","last_fullsized":"2023-06-09T12:16:22.509845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'450","last_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_deep_scrub":"75'450","last_deep_scrub_stamp":"2023-06-09T12:07:35.010396+0000","last_clean_scrub_stamp":"2023-06-09T12:07:35.010396+0000","objects_scrubbed":17,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:05:42.695272+0000","scrub_duration":17.860522096,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971590,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":27,"num_read_kb":36918,"num_write":371,"num_write_kb":20729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":45924,"num_omap_keys":91,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"75'1128","reported_seq":3790,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.594789+0000","last_change":"2023-06-09T12:07:23.428242+0000","last_active":"2023-06-09T12:16:02.594789+0000","last_peered":"2023-06-09T12:16:02.594789+0000","last_clean":"2023-06-09T12:16:02.594789+0000","last_became_active":"2023-06-09T11:10:50.212627+0000","last_became_peered":"2023-06-09T11:10:50.212627+0000","last_unstale":"2023-06-09T12:16:02.594789+0000","last_undegraded":"2023-06-09T12:16:02.594789+0000","last_fullsized":"2023-06-09T12:16:02.594789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:07:23.427889+0000","last_clean_scrub_stamp":"2023-06-09T12:07:23.427889+0000","objects_scrubbed":221,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:29:01.281626+0000","scrub_duration":35.349685563000001,"objects_trimmed":0,"snaptrim_duration":0.0099389390000000008,"stat_sum":{"num_bytes":14209575,"num_objects":221,"num_object_clones":103,"num_object_copies":442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":221,"num_whiteouts":103,"num_read":566,"num_read_kb":11994,"num_write":733,"num_write_kb":15896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"75'532","reported_seq":732,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.435910+0000","last_change":"2023-06-09T12:09:28.567064+0000","last_active":"2023-06-09T12:16:07.435910+0000","last_peered":"2023-06-09T12:16:07.435910+0000","last_clean":"2023-06-09T12:16:07.435910+0000","last_became_active":"2023-06-09T11:10:49.672633+0000","last_became_peered":"2023-06-09T11:10:49.672633+0000","last_unstale":"2023-06-09T12:16:07.435910+0000","last_undegraded":"2023-06-09T12:16:07.435910+0000","last_fullsized":"2023-06-09T12:16:07.435910+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'532","last_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_deep_scrub":"75'532","last_deep_scrub_stamp":"2023-06-09T12:09:28.566729+0000","last_clean_scrub_stamp":"2023-06-09T12:09:28.566729+0000","objects_scrubbed":14,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:28:10.269879+0000","scrub_duration":5.953879047,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":54,"num_read_kb":4192,"num_write":472,"num_write_kb":12647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":77785,"num_omap_keys":141,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"75'1080","reported_seq":4304,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.419527+0000","last_change":"2023-06-09T12:08:20.256714+0000","last_active":"2023-06-09T12:14:57.419527+0000","last_peered":"2023-06-09T12:14:57.419527+0000","last_clean":"2023-06-09T12:14:57.419527+0000","last_became_active":"2023-06-09T11:10:50.217337+0000","last_became_peered":"2023-06-09T11:10:50.217337+0000","last_unstale":"2023-06-09T12:14:57.419527+0000","last_undegraded":"2023-06-09T12:14:57.419527+0000","last_fullsized":"2023-06-09T12:14:57.419527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:08:20.256465+0000","last_clean_scrub_stamp":"2023-06-09T12:08:20.256465+0000","objects_scrubbed":209,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-11T00:05:25.898831+0000","scrub_duration":38.704703279999997,"objects_trimmed":0,"snaptrim_duration":0.064204864,"stat_sum":{"num_bytes":18268718,"num_objects":209,"num_object_clones":94,"num_object_copies":418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":94,"num_read":1123,"num_read_kb":17258,"num_write":721,"num_write_kb":19872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"75'1182","reported_seq":1385,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:42.417181+0000","last_change":"2023-06-09T12:04:27.590556+0000","last_active":"2023-06-09T12:15:42.417181+0000","last_peered":"2023-06-09T12:15:42.417181+0000","last_clean":"2023-06-09T12:15:42.417181+0000","last_became_active":"2023-06-09T11:10:49.674904+0000","last_became_peered":"2023-06-09T11:10:49.674904+0000","last_unstale":"2023-06-09T12:15:42.417181+0000","last_undegraded":"2023-06-09T12:15:42.417181+0000","last_fullsized":"2023-06-09T12:15:42.417181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1182","last_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_deep_scrub":"75'1182","last_deep_scrub_stamp":"2023-06-09T12:04:27.590269+0000","last_clean_scrub_stamp":"2023-06-09T12:04:27.590269+0000","objects_scrubbed":14,"log_size":1182,"log_dups_size":0,"ondisk_log_size":1182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:26:21.506775+0000","scrub_duration":20.586238249000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":65,"num_read_kb":86060,"num_write":1095,"num_write_kb":33313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":30868,"num_omap_keys":37,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"75'1234","reported_seq":4131,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:56.924071+0000","last_change":"2023-06-09T12:17:56.924235+0000","last_active":"2023-06-09T12:17:56.924071+0000","last_peered":"2023-06-09T12:17:56.924071+0000","last_clean":"2023-06-09T12:17:56.924071+0000","last_became_active":"2023-06-09T11:10:50.211035+0000","last_became_peered":"2023-06-09T11:10:50.211035+0000","last_unstale":"2023-06-09T12:17:56.924071+0000","last_undegraded":"2023-06-09T12:17:56.924071+0000","last_fullsized":"2023-06-09T12:17:56.924071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1234","last_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_deep_scrub":"75'1234","last_deep_scrub_stamp":"2023-06-09T12:17:56.923881+0000","last_clean_scrub_stamp":"2023-06-09T12:17:56.923881+0000","objects_scrubbed":259,"log_size":1234,"log_dups_size":0,"ondisk_log_size":1234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:39:41.044787+0000","scrub_duration":44.285760504999999,"objects_trimmed":0,"snaptrim_duration":0.046108875000000001,"stat_sum":{"num_bytes":18931345,"num_objects":259,"num_object_clones":119,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":119,"num_read":816,"num_read_kb":17298,"num_write":788,"num_write_kb":24619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'1080","reported_seq":1273,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:02.588093+0000","last_change":"2023-06-09T12:00:04.445847+0000","last_active":"2023-06-09T12:15:02.588093+0000","last_peered":"2023-06-09T12:15:02.588093+0000","last_clean":"2023-06-09T12:15:02.588093+0000","last_became_active":"2023-06-09T11:10:49.682428+0000","last_became_peered":"2023-06-09T11:10:49.682428+0000","last_unstale":"2023-06-09T12:15:02.588093+0000","last_undegraded":"2023-06-09T12:15:02.588093+0000","last_fullsized":"2023-06-09T12:15:02.588093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1080","last_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_deep_scrub":"75'1080","last_deep_scrub_stamp":"2023-06-09T12:00:04.445598+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.445598+0000","objects_scrubbed":17,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:12:33.086596+0000","scrub_duration":14.884146700000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777216,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":45,"num_read_kb":57470,"num_write":1406,"num_write_kb":25558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":93331,"num_omap_keys":182,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"75'1062","reported_seq":3171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.532549+0000","last_change":"2023-06-09T12:06:50.045989+0000","last_active":"2023-06-09T12:15:37.532549+0000","last_peered":"2023-06-09T12:15:37.532549+0000","last_clean":"2023-06-09T12:15:37.532549+0000","last_became_active":"2023-06-09T11:10:50.199603+0000","last_became_peered":"2023-06-09T11:10:50.199603+0000","last_unstale":"2023-06-09T12:15:37.532549+0000","last_undegraded":"2023-06-09T12:15:37.532549+0000","last_fullsized":"2023-06-09T12:15:37.532549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1062","last_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_deep_scrub":"75'1062","last_deep_scrub_stamp":"2023-06-09T12:06:50.045654+0000","last_clean_scrub_stamp":"2023-06-09T12:06:50.045654+0000","objects_scrubbed":219,"log_size":1062,"log_dups_size":0,"ondisk_log_size":1062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":30,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:59:17.446851+0000","scrub_duration":29.771967292999999,"objects_trimmed":0,"snaptrim_duration":0.055186668000000001,"stat_sum":{"num_bytes":5841353,"num_objects":219,"num_object_clones":103,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":219,"num_whiteouts":103,"num_read":19,"num_read_kb":4113,"num_write":667,"num_write_kb":13284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"75'1470","reported_seq":1667,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.539444+0000","last_change":"2023-06-09T12:07:57.329503+0000","last_active":"2023-06-09T12:16:27.539444+0000","last_peered":"2023-06-09T12:16:27.539444+0000","last_clean":"2023-06-09T12:16:27.539444+0000","last_became_active":"2023-06-09T11:10:49.685424+0000","last_became_peered":"2023-06-09T11:10:49.685424+0000","last_unstale":"2023-06-09T12:16:27.539444+0000","last_undegraded":"2023-06-09T12:16:27.539444+0000","last_fullsized":"2023-06-09T12:16:27.539444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1470","last_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_deep_scrub":"75'1470","last_deep_scrub_stamp":"2023-06-09T12:07:57.329166+0000","last_clean_scrub_stamp":"2023-06-09T12:07:57.329166+0000","objects_scrubbed":14,"log_size":1470,"log_dups_size":0,"ondisk_log_size":1470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:58:01.642784+0000","scrub_duration":11.908883135,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12583002,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":56,"num_read_kb":29484,"num_write":1415,"num_write_kb":13547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":84945,"num_omap_keys":142,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"75'1096","reported_seq":3461,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.538160+0000","last_change":"2023-06-09T12:05:15.228654+0000","last_active":"2023-06-09T12:16:17.538160+0000","last_peered":"2023-06-09T12:16:17.538160+0000","last_clean":"2023-06-09T12:16:17.538160+0000","last_became_active":"2023-06-09T11:10:50.205776+0000","last_became_peered":"2023-06-09T11:10:50.205776+0000","last_unstale":"2023-06-09T12:16:17.538160+0000","last_undegraded":"2023-06-09T12:16:17.538160+0000","last_fullsized":"2023-06-09T12:16:17.538160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1096","last_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_deep_scrub":"75'1096","last_deep_scrub_stamp":"2023-06-09T12:05:15.228339+0000","last_clean_scrub_stamp":"2023-06-09T12:05:15.228339+0000","objects_scrubbed":231,"log_size":1096,"log_dups_size":0,"ondisk_log_size":1096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:06:44.545436+0000","scrub_duration":32.749196716999997,"objects_trimmed":0,"snaptrim_duration":0.055344869999999997,"stat_sum":{"num_bytes":5575265,"num_objects":231,"num_object_clones":104,"num_object_copies":462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":104,"num_read":276,"num_read_kb":8554,"num_write":707,"num_write_kb":13452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"75'594","reported_seq":791,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:13.086236+0000","last_change":"2023-06-09T12:15:13.086236+0000","last_active":"2023-06-09T12:15:13.086236+0000","last_peered":"2023-06-09T12:15:13.086236+0000","last_clean":"2023-06-09T12:15:13.086236+0000","last_became_active":"2023-06-09T11:10:49.683143+0000","last_became_peered":"2023-06-09T11:10:49.683143+0000","last_unstale":"2023-06-09T12:15:13.086236+0000","last_undegraded":"2023-06-09T12:15:13.086236+0000","last_fullsized":"2023-06-09T12:15:13.086236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'594","last_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_deep_scrub":"75'594","last_deep_scrub_stamp":"2023-06-09T12:15:13.085940+0000","last_clean_scrub_stamp":"2023-06-09T12:15:13.085940+0000","objects_scrubbed":19,"log_size":594,"log_dups_size":0,"ondisk_log_size":594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:03:28.604662+0000","scrub_duration":11.603203580000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":59,"num_read_kb":59068,"num_write":522,"num_write_kb":12678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":60087,"num_omap_keys":119,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"75'1254","reported_seq":3540,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407602+0000","last_change":"2023-06-09T12:11:35.237703+0000","last_active":"2023-06-09T12:15:52.407602+0000","last_peered":"2023-06-09T12:15:52.407602+0000","last_clean":"2023-06-09T12:15:52.407602+0000","last_became_active":"2023-06-09T11:10:50.211254+0000","last_became_peered":"2023-06-09T11:10:50.211254+0000","last_unstale":"2023-06-09T12:15:52.407602+0000","last_undegraded":"2023-06-09T12:15:52.407602+0000","last_fullsized":"2023-06-09T12:15:52.407602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1254","last_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_deep_scrub":"75'1254","last_deep_scrub_stamp":"2023-06-09T12:11:35.237490+0000","last_clean_scrub_stamp":"2023-06-09T12:11:35.237490+0000","objects_scrubbed":297,"log_size":1254,"log_dups_size":0,"ondisk_log_size":1254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:16:14.106105+0000","scrub_duration":44.657268713999997,"objects_trimmed":0,"snaptrim_duration":0.046190788000000003,"stat_sum":{"num_bytes":10571086,"num_objects":297,"num_object_clones":136,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":136,"num_read":219,"num_read_kb":8211,"num_write":755,"num_write_kb":21435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.403839+0000","last_change":"2023-06-09T11:59:44.525083+0000","last_active":"2023-06-09T12:15:22.403839+0000","last_peered":"2023-06-09T12:15:22.403839+0000","last_clean":"2023-06-09T12:15:22.403839+0000","last_became_active":"2023-06-09T11:10:43.861847+0000","last_became_peered":"2023-06-09T11:10:43.861847+0000","last_unstale":"2023-06-09T12:15:22.403839+0000","last_undegraded":"2023-06-09T12:15:22.403839+0000","last_fullsized":"2023-06-09T12:15:22.403839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:44.524898+0000","last_clean_scrub_stamp":"2023-06-09T11:59:44.524898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:04:22.468076+0000","scrub_duration":2.232892476,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"75'894","reported_seq":1084,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.430026+0000","last_change":"2023-06-09T12:03:06.064836+0000","last_active":"2023-06-09T12:15:22.430026+0000","last_peered":"2023-06-09T12:15:22.430026+0000","last_clean":"2023-06-09T12:15:22.430026+0000","last_became_active":"2023-06-09T11:10:49.673507+0000","last_became_peered":"2023-06-09T11:10:49.673507+0000","last_unstale":"2023-06-09T12:15:22.430026+0000","last_undegraded":"2023-06-09T12:15:22.430026+0000","last_fullsized":"2023-06-09T12:15:22.430026+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'894","last_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_deep_scrub":"75'894","last_deep_scrub_stamp":"2023-06-09T12:03:06.064586+0000","last_clean_scrub_stamp":"2023-06-09T12:03:06.064586+0000","objects_scrubbed":16,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:19:05.368429+0000","scrub_duration":14.884628745000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777238,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":57389,"num_write":834,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":82856,"num_omap_keys":161,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"75'1214","reported_seq":3912,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:12.402540+0000","last_change":"2023-06-09T12:13:12.052087+0000","last_active":"2023-06-09T12:15:12.402540+0000","last_peered":"2023-06-09T12:15:12.402540+0000","last_clean":"2023-06-09T12:15:12.402540+0000","last_became_active":"2023-06-09T11:10:50.208823+0000","last_became_peered":"2023-06-09T11:10:50.208823+0000","last_unstale":"2023-06-09T12:15:12.402540+0000","last_undegraded":"2023-06-09T12:15:12.402540+0000","last_fullsized":"2023-06-09T12:15:12.402540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1214","last_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_deep_scrub":"75'1214","last_deep_scrub_stamp":"2023-06-09T12:13:12.051890+0000","last_clean_scrub_stamp":"2023-06-09T12:13:12.051890+0000","objects_scrubbed":279,"log_size":1214,"log_dups_size":0,"ondisk_log_size":1214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:45:25.214692+0000","scrub_duration":50.611564710000003,"objects_trimmed":0,"snaptrim_duration":0.037412973000000002,"stat_sum":{"num_bytes":23043156,"num_objects":279,"num_object_clones":145,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":110,"num_read":638,"num_read_kb":13239,"num_write":732,"num_write_kb":29010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.429976+0000","last_change":"2023-06-09T12:05:55.400314+0000","last_active":"2023-06-09T12:15:22.429976+0000","last_peered":"2023-06-09T12:15:22.429976+0000","last_clean":"2023-06-09T12:15:22.429976+0000","last_became_active":"2023-06-09T11:10:43.559465+0000","last_became_peered":"2023-06-09T11:10:43.559465+0000","last_unstale":"2023-06-09T12:15:22.429976+0000","last_undegraded":"2023-06-09T12:15:22.429976+0000","last_fullsized":"2023-06-09T12:15:22.429976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:05:55.400133+0000","last_clean_scrub_stamp":"2023-06-09T12:05:55.400133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T13:54:29.283513+0000","scrub_duration":2.232734379,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"75'731","reported_seq":931,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:32.540086+0000","last_change":"2023-06-09T12:02:10.590528+0000","last_active":"2023-06-09T12:16:32.540086+0000","last_peered":"2023-06-09T12:16:32.540086+0000","last_clean":"2023-06-09T12:16:32.540086+0000","last_became_active":"2023-06-09T11:10:49.687583+0000","last_became_peered":"2023-06-09T11:10:49.687583+0000","last_unstale":"2023-06-09T12:16:32.540086+0000","last_undegraded":"2023-06-09T12:16:32.540086+0000","last_fullsized":"2023-06-09T12:16:32.540086+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'731","last_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_deep_scrub":"75'731","last_deep_scrub_stamp":"2023-06-09T12:02:10.590196+0000","last_clean_scrub_stamp":"2023-06-09T12:02:10.590196+0000","objects_scrubbed":17,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:32:19.064588+0000","scrub_duration":14.886037007000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777346,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":62,"num_read_kb":24695,"num_write":669,"num_write_kb":16797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":65629,"num_omap_keys":120,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"75'1180","reported_seq":4178,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:17.410673+0000","last_change":"2023-06-09T12:12:21.379303+0000","last_active":"2023-06-09T12:16:17.410673+0000","last_peered":"2023-06-09T12:16:17.410673+0000","last_clean":"2023-06-09T12:16:17.410673+0000","last_became_active":"2023-06-09T11:10:50.203128+0000","last_became_peered":"2023-06-09T11:10:50.203128+0000","last_unstale":"2023-06-09T12:16:17.410673+0000","last_undegraded":"2023-06-09T12:16:17.410673+0000","last_fullsized":"2023-06-09T12:16:17.410673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1180","last_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_deep_scrub":"75'1180","last_deep_scrub_stamp":"2023-06-09T12:12:21.379053+0000","last_clean_scrub_stamp":"2023-06-09T12:12:21.379053+0000","objects_scrubbed":254,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:04:35.887453+0000","scrub_duration":44.657235192000002,"objects_trimmed":0,"snaptrim_duration":0.037012461000000003,"stat_sum":{"num_bytes":18668565,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":908,"num_read_kb":13974,"num_write":749,"num_write_kb":20593,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:22.469586+0000","last_change":"2023-06-09T11:59:45.937245+0000","last_active":"2023-06-09T12:15:22.469586+0000","last_peered":"2023-06-09T12:15:22.469586+0000","last_clean":"2023-06-09T12:15:22.469586+0000","last_became_active":"2023-06-09T11:10:43.558043+0000","last_became_peered":"2023-06-09T11:10:43.558043+0000","last_unstale":"2023-06-09T12:15:22.469586+0000","last_undegraded":"2023-06-09T12:15:22.469586+0000","last_fullsized":"2023-06-09T12:15:22.469586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:45.937033+0000","last_clean_scrub_stamp":"2023-06-09T11:59:45.937033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:13:02.477836+0000","scrub_duration":2.2331132220000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"75'1070","reported_seq":1294,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407473+0000","last_change":"2023-06-09T12:01:53.763053+0000","last_active":"2023-06-09T12:15:52.407473+0000","last_peered":"2023-06-09T12:15:52.407473+0000","last_clean":"2023-06-09T12:15:52.407473+0000","last_became_active":"2023-06-09T11:10:49.676344+0000","last_became_peered":"2023-06-09T11:10:49.676344+0000","last_unstale":"2023-06-09T12:15:52.407473+0000","last_undegraded":"2023-06-09T12:15:52.407473+0000","last_fullsized":"2023-06-09T12:15:52.407473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1070","last_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_deep_scrub":"75'1070","last_deep_scrub_stamp":"2023-06-09T12:01:53.762764+0000","last_clean_scrub_stamp":"2023-06-09T12:01:53.762764+0000","objects_scrubbed":17,"log_size":1070,"log_dups_size":0,"ondisk_log_size":1070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:49:40.686052+0000","scrub_duration":20.840151357,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":88,"num_read_kb":132264,"num_write":984,"num_write_kb":25201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32984,"num_omap_keys":64,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"75'1146","reported_seq":4142,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.404473+0000","last_change":"2023-06-09T12:04:38.619113+0000","last_active":"2023-06-09T12:15:27.404473+0000","last_peered":"2023-06-09T12:15:27.404473+0000","last_clean":"2023-06-09T12:15:27.404473+0000","last_became_active":"2023-06-09T11:10:50.202482+0000","last_became_peered":"2023-06-09T11:10:50.202482+0000","last_unstale":"2023-06-09T12:15:27.404473+0000","last_undegraded":"2023-06-09T12:15:27.404473+0000","last_fullsized":"2023-06-09T12:15:27.404473+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1146","last_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_deep_scrub":"75'1146","last_deep_scrub_stamp":"2023-06-09T12:04:38.618848+0000","last_clean_scrub_stamp":"2023-06-09T12:04:38.618848+0000","objects_scrubbed":263,"log_size":1146,"log_dups_size":0,"ondisk_log_size":1146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:53:36.346578+0000","scrub_duration":50.611579055999997,"objects_trimmed":0,"snaptrim_duration":0.037254715000000001,"stat_sum":{"num_bytes":27012036,"num_objects":263,"num_object_clones":119,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":119,"num_read":917,"num_read_kb":17547,"num_write":704,"num_write_kb":32093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:07.427813+0000","last_change":"2023-06-09T12:02:38.268961+0000","last_active":"2023-06-09T12:15:07.427813+0000","last_peered":"2023-06-09T12:15:07.427813+0000","last_clean":"2023-06-09T12:15:07.427813+0000","last_became_active":"2023-06-09T11:10:43.558215+0000","last_became_peered":"2023-06-09T11:10:43.558215+0000","last_unstale":"2023-06-09T12:15:07.427813+0000","last_undegraded":"2023-06-09T12:15:07.427813+0000","last_fullsized":"2023-06-09T12:15:07.427813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:02:38.268698+0000","last_clean_scrub_stamp":"2023-06-09T12:02:38.268698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:51:39.668555+0000","scrub_duration":2.232811678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'119","reported_seq":324,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:57.506645+0000","last_change":"2023-06-09T11:59:58.218793+0000","last_active":"2023-06-09T12:15:57.506645+0000","last_peered":"2023-06-09T12:15:57.506645+0000","last_clean":"2023-06-09T12:15:57.506645+0000","last_became_active":"2023-06-09T11:10:43.861984+0000","last_became_peered":"2023-06-09T11:10:43.861984+0000","last_unstale":"2023-06-09T12:15:57.506645+0000","last_undegraded":"2023-06-09T12:15:57.506645+0000","last_fullsized":"2023-06-09T12:15:57.506645+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'119","last_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_deep_scrub":"75'119","last_deep_scrub_stamp":"2023-06-09T11:59:58.218476+0000","last_clean_scrub_stamp":"2023-06-09T11:59:58.218476+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:07.972525+0000","scrub_duration":2.9769141599999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"75'1153","reported_seq":1356,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.425601+0000","last_change":"2023-06-09T12:00:03.602373+0000","last_active":"2023-06-09T12:16:42.425601+0000","last_peered":"2023-06-09T12:16:42.425601+0000","last_clean":"2023-06-09T12:16:42.425601+0000","last_became_active":"2023-06-09T11:10:49.673417+0000","last_became_peered":"2023-06-09T11:10:49.673417+0000","last_unstale":"2023-06-09T12:16:42.425601+0000","last_undegraded":"2023-06-09T12:16:42.425601+0000","last_fullsized":"2023-06-09T12:16:42.425601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1153","last_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_deep_scrub":"75'1153","last_deep_scrub_stamp":"2023-06-09T12:00:03.602144+0000","last_clean_scrub_stamp":"2023-06-09T12:00:03.602144+0000","objects_scrubbed":14,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":18,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:08:29.200191+0000","scrub_duration":17.591884690000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":45,"num_read_kb":106905,"num_write":1082,"num_write_kb":25256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":90188,"num_omap_keys":146,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"75'1227","reported_seq":3595,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.595312+0000","last_change":"2023-06-09T12:05:54.746857+0000","last_active":"2023-06-09T12:16:07.595312+0000","last_peered":"2023-06-09T12:16:07.595312+0000","last_clean":"2023-06-09T12:16:07.595312+0000","last_became_active":"2023-06-09T11:10:50.202185+0000","last_became_peered":"2023-06-09T11:10:50.202185+0000","last_unstale":"2023-06-09T12:16:07.595312+0000","last_undegraded":"2023-06-09T12:16:07.595312+0000","last_fullsized":"2023-06-09T12:16:07.595312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1227","last_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_deep_scrub":"75'1227","last_deep_scrub_stamp":"2023-06-09T12:05:54.746646+0000","last_clean_scrub_stamp":"2023-06-09T12:05:54.746646+0000","objects_scrubbed":266,"log_size":1227,"log_dups_size":0,"ondisk_log_size":1227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T21:38:07.342175+0000","scrub_duration":38.698651097000003,"objects_trimmed":0,"snaptrim_duration":0.009672801,"stat_sum":{"num_bytes":10493737,"num_objects":266,"num_object_clones":124,"num_object_copies":532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":266,"num_whiteouts":124,"num_read":291,"num_read_kb":3713,"num_write":766,"num_write_kb":16649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:52.513564+0000","last_change":"2023-06-09T12:14:14.458419+0000","last_active":"2023-06-09T12:16:52.513564+0000","last_peered":"2023-06-09T12:16:52.513564+0000","last_clean":"2023-06-09T12:16:52.513564+0000","last_became_active":"2023-06-09T11:10:43.556556+0000","last_became_peered":"2023-06-09T11:10:43.556556+0000","last_unstale":"2023-06-09T12:16:52.513564+0000","last_undegraded":"2023-06-09T12:16:52.513564+0000","last_fullsized":"2023-06-09T12:16:52.513564+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:14:14.458218+0000","last_clean_scrub_stamp":"2023-06-09T12:14:14.458218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:56:31.652891+0000","scrub_duration":2.2326679230000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"75'1949","reported_seq":3036,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:28.734088+0000","last_change":"2023-06-09T12:15:28.734088+0000","last_active":"2023-06-09T12:15:28.734088+0000","last_peered":"2023-06-09T12:15:28.734088+0000","last_clean":"2023-06-09T12:15:28.734088+0000","last_became_active":"2023-06-09T11:10:49.684444+0000","last_became_peered":"2023-06-09T11:10:49.684444+0000","last_unstale":"2023-06-09T12:15:28.734088+0000","last_undegraded":"2023-06-09T12:15:28.734088+0000","last_fullsized":"2023-06-09T12:15:28.734088+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1949","last_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_deep_scrub":"75'1949","last_deep_scrub_stamp":"2023-06-09T12:15:28.733845+0000","last_clean_scrub_stamp":"2023-06-09T12:15:28.733845+0000","objects_scrubbed":19,"log_size":1949,"log_dups_size":0,"ondisk_log_size":1949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":19,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:34:01.350006+0000","scrub_duration":18.236003920999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848182,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":934,"num_read_kb":79442,"num_write":1860,"num_write_kb":30790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":104159,"num_omap_keys":188,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"75'1113","reported_seq":3424,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.425813+0000","last_change":"2023-06-09T12:09:31.820935+0000","last_active":"2023-06-09T12:15:47.425813+0000","last_peered":"2023-06-09T12:15:47.425813+0000","last_clean":"2023-06-09T12:15:47.425813+0000","last_became_active":"2023-06-09T11:10:50.211508+0000","last_became_peered":"2023-06-09T11:10:50.211508+0000","last_unstale":"2023-06-09T12:15:47.425813+0000","last_undegraded":"2023-06-09T12:15:47.425813+0000","last_fullsized":"2023-06-09T12:15:47.425813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1113","last_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_deep_scrub":"75'1113","last_deep_scrub_stamp":"2023-06-09T12:09:31.820730+0000","last_clean_scrub_stamp":"2023-06-09T12:09:31.820730+0000","objects_scrubbed":230,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:32:12.414397+0000","scrub_duration":32.750189312000003,"objects_trimmed":0,"snaptrim_duration":0.046173609999999997,"stat_sum":{"num_bytes":6102428,"num_objects":230,"num_object_clones":102,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":102,"num_read":216,"num_read_kb":4129,"num_write":723,"num_write_kb":8214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:27.503243+0000","last_change":"2023-06-09T12:00:04.637668+0000","last_active":"2023-06-09T12:15:27.503243+0000","last_peered":"2023-06-09T12:15:27.503243+0000","last_clean":"2023-06-09T12:15:27.503243+0000","last_became_active":"2023-06-09T11:10:43.861929+0000","last_became_peered":"2023-06-09T11:10:43.861929+0000","last_unstale":"2023-06-09T12:15:27.503243+0000","last_undegraded":"2023-06-09T12:15:27.503243+0000","last_fullsized":"2023-06-09T12:15:27.503243+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:00:04.637519+0000","last_clean_scrub_stamp":"2023-06-09T12:00:04.637519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:33:32.657299+0000","scrub_duration":2.2326605150000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"75'670","reported_seq":855,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.407539+0000","last_change":"2023-06-09T12:03:31.696276+0000","last_active":"2023-06-09T12:15:52.407539+0000","last_peered":"2023-06-09T12:15:52.407539+0000","last_clean":"2023-06-09T12:15:52.407539+0000","last_became_active":"2023-06-09T11:10:49.682162+0000","last_became_peered":"2023-06-09T11:10:49.682162+0000","last_unstale":"2023-06-09T12:15:52.407539+0000","last_undegraded":"2023-06-09T12:15:52.407539+0000","last_fullsized":"2023-06-09T12:15:52.407539+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'670","last_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_deep_scrub":"75'670","last_deep_scrub_stamp":"2023-06-09T12:03:31.695942+0000","last_clean_scrub_stamp":"2023-06-09T12:03:31.695942+0000","objects_scrubbed":16,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":12,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:36:48.126184+0000","scrub_duration":11.908724279999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":29,"num_read_kb":29474,"num_write":613,"num_write_kb":16865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":85744,"num_omap_keys":168,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"75'1225","reported_seq":4409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428936+0000","last_change":"2023-06-09T12:12:03.027691+0000","last_active":"2023-06-09T12:16:12.428936+0000","last_peered":"2023-06-09T12:16:12.428936+0000","last_clean":"2023-06-09T12:16:12.428936+0000","last_became_active":"2023-06-09T11:10:50.201063+0000","last_became_peered":"2023-06-09T11:10:50.201063+0000","last_unstale":"2023-06-09T12:16:12.428936+0000","last_undegraded":"2023-06-09T12:16:12.428936+0000","last_fullsized":"2023-06-09T12:16:12.428936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:12:03.027416+0000","last_clean_scrub_stamp":"2023-06-09T12:12:03.027416+0000","objects_scrubbed":264,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:17:06.433707+0000","scrub_duration":44.659364171999997,"objects_trimmed":0,"snaptrim_duration":0.055219696999999998,"stat_sum":{"num_bytes":18505912,"num_objects":264,"num_object_clones":118,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":118,"num_read":1100,"num_read_kb":17184,"num_write":776,"num_write_kb":19703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"75'354","reported_seq":553,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:58.233546+0000","last_change":"2023-06-09T12:16:58.233546+0000","last_active":"2023-06-09T12:16:58.233546+0000","last_peered":"2023-06-09T12:16:58.233546+0000","last_clean":"2023-06-09T12:16:58.233546+0000","last_became_active":"2023-06-09T11:10:49.681909+0000","last_became_peered":"2023-06-09T11:10:49.681909+0000","last_unstale":"2023-06-09T12:16:58.233546+0000","last_undegraded":"2023-06-09T12:16:58.233546+0000","last_fullsized":"2023-06-09T12:16:58.233546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'354","last_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_deep_scrub":"75'354","last_deep_scrub_stamp":"2023-06-09T12:16:58.233320+0000","last_clean_scrub_stamp":"2023-06-09T12:16:58.233320+0000","objects_scrubbed":12,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:27:56.802904+0000","scrub_duration":8.6088365610000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":83,"num_read_kb":12393,"num_write":297,"num_write_kb":8453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":72736,"num_omap_keys":145,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":169,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:57.400558+0000","last_change":"2023-06-09T11:59:48.465622+0000","last_active":"2023-06-09T12:14:57.400558+0000","last_peered":"2023-06-09T12:14:57.400558+0000","last_clean":"2023-06-09T12:14:57.400558+0000","last_became_active":"2023-06-09T11:10:43.561035+0000","last_became_peered":"2023-06-09T11:10:43.561035+0000","last_unstale":"2023-06-09T12:14:57.400558+0000","last_undegraded":"2023-06-09T12:14:57.400558+0000","last_fullsized":"2023-06-09T12:14:57.400558+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T11:59:48.465302+0000","last_clean_scrub_stamp":"2023-06-09T11:59:48.465302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:29:24.255336+0000","scrub_duration":2.2330028620000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"75'1128","reported_seq":3653,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:17:07.428876+0000","last_change":"2023-06-09T12:09:21.870328+0000","last_active":"2023-06-09T12:17:07.428876+0000","last_peered":"2023-06-09T12:17:07.428876+0000","last_clean":"2023-06-09T12:17:07.428876+0000","last_became_active":"2023-06-09T11:10:50.204256+0000","last_became_peered":"2023-06-09T11:10:50.204256+0000","last_unstale":"2023-06-09T12:17:07.428876+0000","last_undegraded":"2023-06-09T12:17:07.428876+0000","last_fullsized":"2023-06-09T12:17:07.428876+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1128","last_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_deep_scrub":"75'1128","last_deep_scrub_stamp":"2023-06-09T12:09:21.870088+0000","last_clean_scrub_stamp":"2023-06-09T12:09:21.870088+0000","objects_scrubbed":254,"log_size":1128,"log_dups_size":0,"ondisk_log_size":1128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T20:56:10.205074+0000","scrub_duration":44.915296265999999,"objects_trimmed":0,"snaptrim_duration":0.024338133000000001,"stat_sum":{"num_bytes":18440299,"num_objects":254,"num_object_clones":114,"num_object_copies":508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":254,"num_whiteouts":114,"num_read":438,"num_read_kb":17269,"num_write":696,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":183,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.596001+0000","last_change":"2023-06-09T12:09:43.473569+0000","last_active":"2023-06-09T12:16:12.596001+0000","last_peered":"2023-06-09T12:16:12.596001+0000","last_clean":"2023-06-09T12:16:12.596001+0000","last_became_active":"2023-06-09T11:10:43.560814+0000","last_became_peered":"2023-06-09T11:10:43.560814+0000","last_unstale":"2023-06-09T12:16:12.596001+0000","last_undegraded":"2023-06-09T12:16:12.596001+0000","last_fullsized":"2023-06-09T12:16:12.596001+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-09T12:09:43.473383+0000","last_clean_scrub_stamp":"2023-06-09T12:09:43.473383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T18:50:19.078470+0000","scrub_duration":2.2327503110000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"75'657","reported_seq":867,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428963+0000","last_change":"2023-06-09T12:14:03.714142+0000","last_active":"2023-06-09T12:16:12.428963+0000","last_peered":"2023-06-09T12:16:12.428963+0000","last_clean":"2023-06-09T12:16:12.428963+0000","last_became_active":"2023-06-09T11:10:49.681096+0000","last_became_peered":"2023-06-09T11:10:49.681096+0000","last_unstale":"2023-06-09T12:16:12.428963+0000","last_undegraded":"2023-06-09T12:16:12.428963+0000","last_fullsized":"2023-06-09T12:16:12.428963+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'657","last_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_deep_scrub":"75'657","last_deep_scrub_stamp":"2023-06-09T12:14:03.713798+0000","last_clean_scrub_stamp":"2023-06-09T12:14:03.713798+0000","objects_scrubbed":11,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:23:48.665405+0000","scrub_duration":5.9546337319999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":73,"num_read_kb":16699,"num_write":1043,"num_write_kb":9884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":86422,"num_omap_keys":143,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"75'1141","reported_seq":3995,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.541548+0000","last_change":"2023-06-09T12:14:50.624016+0000","last_active":"2023-06-09T12:16:42.541548+0000","last_peered":"2023-06-09T12:16:42.541548+0000","last_clean":"2023-06-09T12:16:42.541548+0000","last_became_active":"2023-06-09T11:10:50.198399+0000","last_became_peered":"2023-06-09T11:10:50.198399+0000","last_unstale":"2023-06-09T12:16:42.541548+0000","last_undegraded":"2023-06-09T12:16:42.541548+0000","last_fullsized":"2023-06-09T12:16:42.541548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1141","last_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_deep_scrub":"75'1141","last_deep_scrub_stamp":"2023-06-09T12:14:50.623812+0000","last_clean_scrub_stamp":"2023-06-09T12:14:50.623812+0000","objects_scrubbed":247,"log_size":1141,"log_dups_size":0,"ondisk_log_size":1141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:04:32.209869+0000","scrub_duration":47.26292505,"objects_trimmed":0,"snaptrim_duration":0.038200913000000003,"stat_sum":{"num_bytes":22358874,"num_objects":247,"num_object_clones":111,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":111,"num_read":763,"num_read_kb":15273,"num_write":717,"num_write_kb":24441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'711","reported_seq":919,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:12.428917+0000","last_change":"2023-06-09T12:08:55.135076+0000","last_active":"2023-06-09T12:16:12.428917+0000","last_peered":"2023-06-09T12:16:12.428917+0000","last_clean":"2023-06-09T12:16:12.428917+0000","last_became_active":"2023-06-09T11:10:49.683822+0000","last_became_peered":"2023-06-09T11:10:49.683822+0000","last_unstale":"2023-06-09T12:16:12.428917+0000","last_undegraded":"2023-06-09T12:16:12.428917+0000","last_fullsized":"2023-06-09T12:16:12.428917+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'711","last_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_deep_scrub":"75'711","last_deep_scrub_stamp":"2023-06-09T12:08:55.134718+0000","last_clean_scrub_stamp":"2023-06-09T12:08:55.134718+0000","objects_scrubbed":19,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":15,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:07:03.171538+0000","scrub_duration":14.886574833999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16777250,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":52,"num_read_kb":115732,"num_write":651,"num_write_kb":25253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":62414,"num_omap_keys":119,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"75'1225","reported_seq":4819,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:37.471224+0000","last_change":"2023-06-09T12:00:44.355731+0000","last_active":"2023-06-09T12:15:37.471224+0000","last_peered":"2023-06-09T12:15:37.471224+0000","last_clean":"2023-06-09T12:15:37.471224+0000","last_became_active":"2023-06-09T11:10:50.205630+0000","last_became_peered":"2023-06-09T11:10:50.205630+0000","last_unstale":"2023-06-09T12:15:37.471224+0000","last_undegraded":"2023-06-09T12:15:37.471224+0000","last_fullsized":"2023-06-09T12:15:37.471224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1225","last_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_deep_scrub":"75'1225","last_deep_scrub_stamp":"2023-06-09T12:00:44.355535+0000","last_clean_scrub_stamp":"2023-06-09T12:00:44.355535+0000","objects_scrubbed":270,"log_size":1225,"log_dups_size":0,"ondisk_log_size":1225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":54,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:01:47.556049+0000","scrub_duration":53.593411773,"objects_trimmed":0,"snaptrim_duration":0.019097020999999999,"stat_sum":{"num_bytes":31532756,"num_objects":270,"num_object_clones":119,"num_object_copies":540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":270,"num_whiteouts":119,"num_read":1520,"num_read_kb":25270,"num_write":770,"num_write_kb":38181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"75'601","reported_seq":803,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:52.433955+0000","last_change":"2023-06-09T12:11:00.616288+0000","last_active":"2023-06-09T12:15:52.433955+0000","last_peered":"2023-06-09T12:15:52.433955+0000","last_clean":"2023-06-09T12:15:52.433955+0000","last_became_active":"2023-06-09T11:10:49.681739+0000","last_became_peered":"2023-06-09T11:10:49.681739+0000","last_unstale":"2023-06-09T12:15:52.433955+0000","last_undegraded":"2023-06-09T12:15:52.433955+0000","last_fullsized":"2023-06-09T12:15:52.433955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'601","last_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_deep_scrub":"75'601","last_deep_scrub_stamp":"2023-06-09T12:11:00.616072+0000","last_clean_scrub_stamp":"2023-06-09T12:11:00.616072+0000","objects_scrubbed":8,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:21:02.358390+0000","scrub_duration":5.953914814,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":66,"num_read_kb":41061,"num_write":544,"num_write_kb":8620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":57647,"num_omap_keys":92,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"75'1111","reported_seq":3833,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:27.539490+0000","last_change":"2023-06-09T12:03:58.332393+0000","last_active":"2023-06-09T12:16:27.539490+0000","last_peered":"2023-06-09T12:16:27.539490+0000","last_clean":"2023-06-09T12:16:27.539490+0000","last_became_active":"2023-06-09T11:10:50.212491+0000","last_became_peered":"2023-06-09T11:10:50.212491+0000","last_unstale":"2023-06-09T12:16:27.539490+0000","last_undegraded":"2023-06-09T12:16:27.539490+0000","last_fullsized":"2023-06-09T12:16:27.539490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1111","last_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_deep_scrub":"75'1111","last_deep_scrub_stamp":"2023-06-09T12:03:58.332155+0000","last_clean_scrub_stamp":"2023-06-09T12:03:58.332155+0000","objects_scrubbed":245,"log_size":1111,"log_dups_size":0,"ondisk_log_size":1111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":48,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:26:35.559199+0000","scrub_duration":47.262931569999999,"objects_trimmed":0,"snaptrim_duration":0.037150844000000002,"stat_sum":{"num_bytes":23035485,"num_objects":245,"num_object_clones":110,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":110,"num_read":639,"num_read_kb":10579,"num_write":695,"num_write_kb":34048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"75'603","reported_seq":814,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.435952+0000","last_change":"2023-06-09T12:01:13.303360+0000","last_active":"2023-06-09T12:16:07.435952+0000","last_peered":"2023-06-09T12:16:07.435952+0000","last_clean":"2023-06-09T12:16:07.435952+0000","last_became_active":"2023-06-09T11:10:49.683705+0000","last_became_peered":"2023-06-09T11:10:49.683705+0000","last_unstale":"2023-06-09T12:16:07.435952+0000","last_undegraded":"2023-06-09T12:16:07.435952+0000","last_fullsized":"2023-06-09T12:16:07.435952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'603","last_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_deep_scrub":"75'603","last_deep_scrub_stamp":"2023-06-09T12:01:13.302995+0000","last_clean_scrub_stamp":"2023-06-09T12:01:13.302995+0000","objects_scrubbed":15,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:13:37.083063+0000","scrub_duration":20.094280498,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24376472,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":90,"num_read_kb":127085,"num_write":514,"num_write_kb":28253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":63146,"num_omap_keys":125,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"75'1136","reported_seq":3937,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:02.536093+0000","last_change":"2023-06-09T12:08:36.435334+0000","last_active":"2023-06-09T12:16:02.536093+0000","last_peered":"2023-06-09T12:16:02.536093+0000","last_clean":"2023-06-09T12:16:02.536093+0000","last_became_active":"2023-06-09T11:10:50.210409+0000","last_became_peered":"2023-06-09T11:10:50.210409+0000","last_unstale":"2023-06-09T12:16:02.536093+0000","last_undegraded":"2023-06-09T12:16:02.536093+0000","last_fullsized":"2023-06-09T12:16:02.536093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1136","last_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_deep_scrub":"75'1136","last_deep_scrub_stamp":"2023-06-09T12:08:36.435052+0000","last_clean_scrub_stamp":"2023-06-09T12:08:36.435052+0000","objects_scrubbed":235,"log_size":1136,"log_dups_size":0,"ondisk_log_size":1136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":38,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T23:18:08.051778+0000","scrub_duration":37.959205871000002,"objects_trimmed":0,"snaptrim_duration":0.046027080999999997,"stat_sum":{"num_bytes":14503007,"num_objects":235,"num_object_clones":107,"num_object_copies":470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":235,"num_whiteouts":107,"num_read":716,"num_read_kb":13349,"num_write":717,"num_write_kb":19851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"75'877","reported_seq":1070,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:22.422783+0000","last_change":"2023-06-09T12:00:25.446355+0000","last_active":"2023-06-09T12:16:22.422783+0000","last_peered":"2023-06-09T12:16:22.422783+0000","last_clean":"2023-06-09T12:16:22.422783+0000","last_became_active":"2023-06-09T11:10:49.677246+0000","last_became_peered":"2023-06-09T11:10:49.677246+0000","last_unstale":"2023-06-09T12:16:22.422783+0000","last_undegraded":"2023-06-09T12:16:22.422783+0000","last_fullsized":"2023-06-09T12:16:22.422783+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'877","last_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_deep_scrub":"75'877","last_deep_scrub_stamp":"2023-06-09T12:00:25.446148+0000","last_clean_scrub_stamp":"2023-06-09T12:00:25.446148+0000","objects_scrubbed":19,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":21,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T16:58:31.740570+0000","scrub_duration":20.586133717999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":27,"num_read_kb":98079,"num_write":793,"num_write_kb":37527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":117618,"num_omap_keys":232,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"75'1099","reported_seq":3409,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.417826+0000","last_change":"2023-06-09T12:13:37.203901+0000","last_active":"2023-06-09T12:15:47.417826+0000","last_peered":"2023-06-09T12:15:47.417826+0000","last_clean":"2023-06-09T12:15:47.417826+0000","last_became_active":"2023-06-09T11:10:50.215411+0000","last_became_peered":"2023-06-09T11:10:50.215411+0000","last_unstale":"2023-06-09T12:15:47.417826+0000","last_undegraded":"2023-06-09T12:15:47.417826+0000","last_fullsized":"2023-06-09T12:15:47.417826+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1099","last_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_deep_scrub":"75'1099","last_deep_scrub_stamp":"2023-06-09T12:13:37.203636+0000","last_clean_scrub_stamp":"2023-06-09T12:13:37.203636+0000","objects_scrubbed":233,"log_size":1099,"log_dups_size":0,"ondisk_log_size":1099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T12:47:56.850079+0000","scrub_duration":32.937791355999998,"objects_trimmed":0,"snaptrim_duration":0.057463380000000001,"stat_sum":{"num_bytes":5828397,"num_objects":233,"num_object_clones":108,"num_object_copies":466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":233,"num_whiteouts":108,"num_read":225,"num_read_kb":8422,"num_write":687,"num_write_kb":11396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"75'1395","reported_seq":1631,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:47.433464+0000","last_change":"2023-06-09T12:00:15.282168+0000","last_active":"2023-06-09T12:16:47.433464+0000","last_peered":"2023-06-09T12:16:47.433464+0000","last_clean":"2023-06-09T12:16:47.433464+0000","last_became_active":"2023-06-09T11:10:49.673958+0000","last_became_peered":"2023-06-09T11:10:49.673958+0000","last_unstale":"2023-06-09T12:16:47.433464+0000","last_undegraded":"2023-06-09T12:16:47.433464+0000","last_fullsized":"2023-06-09T12:16:47.433464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1395","last_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_deep_scrub":"75'1395","last_deep_scrub_stamp":"2023-06-09T12:00:15.281952+0000","last_clean_scrub_stamp":"2023-06-09T12:00:15.281952+0000","objects_scrubbed":12,"log_size":1395,"log_dups_size":0,"ondisk_log_size":1395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":9,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T17:52:53.832114+0000","scrub_duration":8.9318835169999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389190,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":137,"num_read_kb":95291,"num_write":1332,"num_write_kb":25289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":32865,"num_omap_keys":56,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"75'1308","reported_seq":3793,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:14:42.524596+0000","last_change":"2023-06-09T12:12:30.246382+0000","last_active":"2023-06-09T12:14:42.524596+0000","last_peered":"2023-06-09T12:14:42.524596+0000","last_clean":"2023-06-09T12:14:42.524596+0000","last_became_active":"2023-06-09T11:10:50.207525+0000","last_became_peered":"2023-06-09T11:10:50.207525+0000","last_unstale":"2023-06-09T12:14:42.524596+0000","last_undegraded":"2023-06-09T12:14:42.524596+0000","last_fullsized":"2023-06-09T12:14:42.524596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1308","last_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_deep_scrub":"75'1308","last_deep_scrub_stamp":"2023-06-09T12:12:30.246145+0000","last_clean_scrub_stamp":"2023-06-09T12:12:30.246145+0000","objects_scrubbed":280,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":45,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:31:02.832150+0000","scrub_duration":44.657888772,"objects_trimmed":0,"snaptrim_duration":0.055166592,"stat_sum":{"num_bytes":14362719,"num_objects":280,"num_object_clones":129,"num_object_copies":560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":280,"num_whiteouts":129,"num_read":417,"num_read_kb":14222,"num_write":824,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"75'1181","reported_seq":3711,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:57.601004+0000","last_change":"2023-06-09T12:08:42.423321+0000","last_active":"2023-06-09T12:16:57.601004+0000","last_peered":"2023-06-09T12:16:57.601004+0000","last_clean":"2023-06-09T12:16:57.601004+0000","last_became_active":"2023-06-09T11:10:50.201299+0000","last_became_peered":"2023-06-09T11:10:50.201299+0000","last_unstale":"2023-06-09T12:16:57.601004+0000","last_undegraded":"2023-06-09T12:16:57.601004+0000","last_fullsized":"2023-06-09T12:16:57.601004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:08:42.423116+0000","last_clean_scrub_stamp":"2023-06-09T12:08:42.423116+0000","objects_scrubbed":248,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":39,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:36:06.928599+0000","scrub_duration":38.326458660999997,"objects_trimmed":0,"snaptrim_duration":0.0095822119999999997,"stat_sum":{"num_bytes":10152700,"num_objects":248,"num_object_clones":114,"num_object_copies":496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":248,"num_whiteouts":114,"num_read":437,"num_read_kb":8788,"num_write":748,"num_write_kb":11535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"75'1237","reported_seq":4232,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:18:52.956159+0000","last_change":"2023-06-09T12:18:52.956249+0000","last_active":"2023-06-09T12:18:52.956159+0000","last_peered":"2023-06-09T12:18:52.956159+0000","last_clean":"2023-06-09T12:18:52.956159+0000","last_became_active":"2023-06-09T11:10:50.218740+0000","last_became_peered":"2023-06-09T11:10:50.218740+0000","last_unstale":"2023-06-09T12:18:52.956159+0000","last_undegraded":"2023-06-09T12:18:52.956159+0000","last_fullsized":"2023-06-09T12:18:52.956159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1237","last_scrub_stamp":"2023-06-09T12:18:52.956036+0000","last_deep_scrub":"75'1237","last_deep_scrub_stamp":"2023-06-09T12:18:52.956036+0000","last_clean_scrub_stamp":"2023-06-09T12:18:52.956036+0000","objects_scrubbed":255,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":51,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T22:46:58.623099+0000","scrub_duration":50.529573540999998,"objects_trimmed":0,"snaptrim_duration":0.046171759999999999,"stat_sum":{"num_bytes":27780457,"num_objects":255,"num_object_clones":113,"num_object_copies":510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":255,"num_whiteouts":113,"num_read":899,"num_read_kb":15982,"num_write":783,"num_write_kb":34444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"75'1181","reported_seq":3663,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:15:47.533815+0000","last_change":"2023-06-09T12:11:39.403399+0000","last_active":"2023-06-09T12:15:47.533815+0000","last_peered":"2023-06-09T12:15:47.533815+0000","last_clean":"2023-06-09T12:15:47.533815+0000","last_became_active":"2023-06-09T11:10:50.207223+0000","last_became_peered":"2023-06-09T11:10:50.207223+0000","last_unstale":"2023-06-09T12:15:47.533815+0000","last_undegraded":"2023-06-09T12:15:47.533815+0000","last_fullsized":"2023-06-09T12:15:47.533815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1181","last_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_deep_scrub":"75'1181","last_deep_scrub_stamp":"2023-06-09T12:11:39.403160+0000","last_clean_scrub_stamp":"2023-06-09T12:11:39.403160+0000","objects_scrubbed":245,"log_size":1181,"log_dups_size":0,"ondisk_log_size":1181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":36,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T15:59:22.020309+0000","scrub_duration":35.726312849000003,"objects_trimmed":0,"snaptrim_duration":0.037034356999999997,"stat_sum":{"num_bytes":10498236,"num_objects":245,"num_object_clones":115,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":115,"num_read":398,"num_read_kb":9260,"num_write":751,"num_write_kb":12119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"75'1147","reported_seq":3256,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:07.420522+0000","last_change":"2023-06-09T12:14:11.286353+0000","last_active":"2023-06-09T12:16:07.420522+0000","last_peered":"2023-06-09T12:16:07.420522+0000","last_clean":"2023-06-09T12:16:07.420522+0000","last_became_active":"2023-06-09T11:10:50.209943+0000","last_became_peered":"2023-06-09T11:10:50.209943+0000","last_unstale":"2023-06-09T12:16:07.420522+0000","last_undegraded":"2023-06-09T12:16:07.420522+0000","last_fullsized":"2023-06-09T12:16:07.420522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1147","last_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_deep_scrub":"75'1147","last_deep_scrub_stamp":"2023-06-09T12:14:11.286125+0000","last_clean_scrub_stamp":"2023-06-09T12:14:11.286125+0000","objects_scrubbed":247,"log_size":1147,"log_dups_size":0,"ondisk_log_size":1147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":33,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T14:29:43.295450+0000","scrub_duration":32.937867779999998,"objects_trimmed":0,"snaptrim_duration":0.035558355999999999,"stat_sum":{"num_bytes":2182521,"num_objects":247,"num_object_clones":109,"num_object_copies":494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":247,"num_whiteouts":109,"num_read":19,"num_read_kb":20,"num_write":723,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"75'1117","reported_seq":3274,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-06-09T12:16:42.413594+0000","last_change":"2023-06-09T12:05:52.958702+0000","last_active":"2023-06-09T12:16:42.413594+0000","last_peered":"2023-06-09T12:16:42.413594+0000","last_clean":"2023-06-09T12:16:42.413594+0000","last_became_active":"2023-06-09T11:10:50.203541+0000","last_became_peered":"2023-06-09T11:10:50.203541+0000","last_unstale":"2023-06-09T12:16:42.413594+0000","last_undegraded":"2023-06-09T12:16:42.413594+0000","last_fullsized":"2023-06-09T12:16:42.413594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'1117","last_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_deep_scrub":"75'1117","last_deep_scrub_stamp":"2023-06-09T12:05:52.958515+0000","last_clean_scrub_stamp":"2023-06-09T12:05:52.958515+0000","objects_scrubbed":251,"log_size":1117,"log_dups_size":0,"ondisk_log_size":1117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":35,"scrub_schedule":"periodic scrub scheduled @ 2023-06-10T19:33:14.168059+0000","scrub_duration":34.237172248,"objects_trimmed":0,"snaptrim_duration":0.028278115999999999,"stat_sum":{"num_bytes":4678533,"num_objects":251,"num_object_clones":116,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":116,"num_read":75,"num_read_kb":5291,"num_write":685,"num_write_kb":9411,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":863946503,"num_objects":15993,"num_object_clones":7273,"num_object_copies":31986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15993,"num_whiteouts":7203,"num_read":30269,"num_read_kb":676418,"num_write":46292,"num_write_kb":1156700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1630887936,"data_stored":1728344090,"data_compressed":141998654,"data_compressed_allocated":157450240,"data_compressed_original":294207488,"omap_allocated":0,"internal_metadata":0},"log_size":74078,"ondisk_log_size":74078,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":510170442,"num_objects":489,"num_object_clones":0,"num_object_copies":978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":489,"num_whiteouts":0,"num_read":4818,"num_read_kb":1938173,"num_write":29975,"num_write_kb":671687,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":325,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2191235,"num_omap_keys":4070,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":290406400,"data_stored":1020340884,"data_compressed":127294428,"data_compressed_allocated":168206336,"data_compressed_original":898301952,"omap_allocated":0,"internal_metadata":0},"log_size":31005,"ondisk_log_size":31005,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":64778,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575673,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266064,"kb_used_data":179848,"kb_used_omap":0,"kb_used_meta":86208,"kb_avail":94105776,"statfs":{"total":96636764160,"available":96364314624,"internally_reserved":0,"allocated":184164352,"data_stored":310946713,"data_compressed":32007065,"data_compressed_allocated":39620608,"data_compressed_original":171003904,"omap_allocated":0,"internal_metadata":88276992},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.69399999999999995,"15min":0.68799999999999994},"min":{"1min":0.53500000000000003,"5min":0.39700000000000002,"15min":0.33700000000000002},"max":{"1min":0.93500000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.747,"15min":0.72999999999999998},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.378},"max":{"1min":0.91800000000000004,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.90200000000000002}]},{"osd":1,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76200000000000001,"15min":0.74199999999999999},"min":{"1min":0.59199999999999997,"5min":0.47499999999999998,"15min":0.42999999999999999},"max":{"1min":0.98899999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.78400000000000003,"15min":0.76500000000000001},"min":{"1min":0.64000000000000001,"5min":0.54500000000000004,"15min":0.48399999999999999},"max":{"1min":1.0009999999999999,"5min":1.071,"15min":1.071},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71799999999999997},"min":{"1min":0.52200000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":0.90800000000000003,"5min":1.0629999999999999,"15min":1.0649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.77600000000000002,"15min":0.75700000000000001},"min":{"1min":0.56599999999999995,"5min":0.497,"15min":0.45200000000000001},"max":{"1min":1.018,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":0.83499999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.497,"5min":0.48199999999999998,"15min":0.47299999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.745,"15min":0.72999999999999998},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.079,"15min":1.079},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.441,"5min":0.40100000000000002,"15min":0.34599999999999997},"max":{"1min":0.87,"5min":1.591,"15min":1.591},"last":0.497},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.753,"15min":0.73999999999999999},"min":{"1min":0.60199999999999998,"5min":0.55400000000000005,"15min":0.47799999999999998},"max":{"1min":0.98099999999999998,"5min":1.619,"15min":1.619},"last":0.874}]},{"osd":5,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.625,"15min":0.60599999999999998},"min":{"1min":0.51100000000000001,"5min":0.39200000000000002,"15min":0.378},"max":{"1min":0.94099999999999995,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.65300000000000002,"15min":0.627},"min":{"1min":0.51200000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:19:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.66500000000000004},"min":{"1min":0.501,"5min":0.48699999999999999,"15min":0.44600000000000001},"max":{"1min":0.96699999999999997,"5min":1.026,"15min":1.3080000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77200000000000002,"15min":0.751},"min":{"1min":0.64400000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.99099999999999999,"5min":1.492,"15min":1.492},"last":0.747}]}]},{"osd":6,"up_from":13,"seq":55834575673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":214552,"kb_used_data":156112,"kb_used_omap":0,"kb_used_meta":58432,"kb_avail":94157288,"statfs":{"total":96636764160,"available":96417062912,"internally_reserved":0,"allocated":159858688,"data_stored":205859922,"data_compressed":18077834,"data_compressed_allocated":21663744,"data_compressed_original":71991296,"omap_allocated":0,"internal_metadata":59834368},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64800000000000002,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.33100000000000002},"max":{"1min":0.89600000000000002,"5min":0.96599999999999997,"15min":1.044},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67700000000000005,"15min":0.69599999999999995},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.42799999999999999},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.98999999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69599999999999995,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.432,"15min":0.41899999999999998},"max":{"1min":0.95799999999999996,"5min":1.0960000000000001,"15min":1.266},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.76100000000000001},"min":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"max":{"1min":1.0049999999999999,"5min":1.139,"15min":1.298},"last":0.755}]},{"osd":2,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.46500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0369999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.751,"15min":0.76500000000000001},"min":{"1min":0.51100000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.96799999999999997,"5min":1.01,"15min":1.091},"last":0.74299999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.73799999999999999,"15min":0.753},"min":{"1min":0.624,"5min":0.47899999999999998,"15min":0.46500000000000002},"max":{"1min":0.96099999999999997,"5min":0.99299999999999999,"15min":1.046},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.69699999999999995,"5min":0.57599999999999996,"15min":0.55900000000000005},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.1259999999999999},"last":0.71899999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.45700000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.78100000000000003,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60499999999999998,"15min":0.60199999999999998},"min":{"1min":0.44700000000000001,"5min":0.41799999999999998,"15min":0.40100000000000002},"max":{"1min":0.85499999999999998,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.98399999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.58299999999999996,"5min":0.53100000000000003,"15min":0.498},"max":{"1min":1.036,"5min":1.161,"15min":1.161},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Jun 9 12:19:10 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.65300000000000002,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.44,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.0429999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72599999999999998,"15min":0.71699999999999997},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95399999999999996,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.84399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575674,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405808,"kb_used_data":306408,"kb_used_omap":0,"kb_used_meta":99392,"kb_avail":93966032,"statfs":{"total":96636764160,"available":96221216768,"internally_reserved":0,"allocated":313761792,"data_stored":398206272,"data_compressed":35305586,"data_compressed_allocated":41877504,"data_compressed_original":133144576,"omap_allocated":0,"internal_metadata":101777408},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.45600000000000002},"min":{"1min":0.33500000000000002,"5min":0.32300000000000001,"15min":0.25800000000000001},"max":{"1min":0.55200000000000005,"5min":0.67700000000000005,"15min":1.4470000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55900000000000005,"15min":0.56699999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.311},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85099999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.48499999999999999,"15min":0.49299999999999999},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.56100000000000005,"5min":0.71699999999999997,"15min":0.94599999999999995},"last":0.497},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53400000000000003,"15min":0.54200000000000004},"min":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.36699999999999999},"max":{"1min":0.64800000000000002,"5min":0.82699999999999996,"15min":0.95299999999999996},"last":0.58199999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.59399999999999997,"15min":0.60299999999999998},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":1.2450000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.56699999999999995,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.99399999999999999,"5min":1.0649999999999999,"15min":1.288},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.88500000000000001,"15min":0.83899999999999997},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.48599999999999999},"max":{"1min":1.0329999999999999,"5min":1.1419999999999999,"15min":1.2669999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.80300000000000005,"15min":0.83299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.48499999999999999},"max":{"1min":0.98299999999999998,"5min":1.141,"15min":1.597},"last":0.89000000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:18:18 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.55500000000000005},"max":{"1min":0.99399999999999999,"5min":1.097,"15min":1.544},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83999999999999997,"15min":0.85499999999999998},"min":{"1min":0.68300000000000005,"5min":0.66700000000000004,"15min":0.59499999999999997},"max":{"1min":1.0369999999999999,"5min":1.1539999999999999,"15min":1.704},"last":0.86199999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.82899999999999996,"15min":0.83799999999999997},"min":{"1min":0.626,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.0289999999999999,"5min":1.29,"15min":1.548},"last":0.753},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.82799999999999996,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.52400000000000002},"max":{"1min":1.0369999999999999,"5min":1.1850000000000001,"15min":1.2869999999999999},"last":0.80100000000000005}]},{"osd":7,"last update":"Fri Jun 9 12:19:04 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.90500000000000003,"15min":0.90000000000000002},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.02,"5min":1.1779999999999999,"15min":1.6519999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.86099999999999999,"15min":0.85699999999999998},"min":{"1min":0.63900000000000001,"5min":0.56399999999999995,"15min":0.52100000000000002},"max":{"1min":1.0509999999999999,"5min":1.3320000000000001,"15min":1.498},"last":0.82299999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575673,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":331496,"kb_used_data":227808,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":94040344,"statfs":{"total":96636764160,"available":96297312256,"internally_reserved":0,"allocated":233275392,"data_stored":339643595,"data_compressed":35944445,"data_compressed_allocated":43143168,"data_compressed_original":154546176,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.32600000000000001,"5min":0.31900000000000001,"15min":0.30599999999999999},"max":{"1min":0.64900000000000002,"5min":0.76000000000000001,"15min":0.81299999999999994},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.56000000000000005},"min":{"1min":0.45200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.72499999999999998,"5min":0.73399999999999999,"15min":0.91600000000000004},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.35999999999999999},"max":{"1min":0.72399999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.75,"5min":0.82299999999999995,"15min":0.91100000000000003},"last":0.46700000000000003}]},{"osd":3,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.63,"15min":0.60499999999999998},"min":{"1min":0.48699999999999999,"5min":0.432,"15min":0.38400000000000001},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.96199999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.56699999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83999999999999997,"5min":1.0149999999999999,"15min":1.0740000000000001},"last":0.71699999999999997}]},{"osd":4,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.879,"15min":0.85799999999999998},"min":{"1min":0.73899999999999999,"5min":0.51200000000000001,"15min":0.42199999999999999},"max":{"1min":1.286,"5min":1.286,"15min":1.421},"last":1.2470000000000001},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.89700000000000002,"15min":0.89500000000000002},"min":{"1min":0.72099999999999997,"5min":0.61399999999999999,"15min":0.60499999999999998},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.268},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.80300000000000005,"15min":0.79400000000000004},"min":{"1min":0.71899999999999997,"5min":0.50700000000000001,"15min":0.495},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.90600000000000003,"15min":0.89800000000000002},"min":{"1min":0.67400000000000004,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.234,"5min":1.234,"15min":3.5840000000000001},"last":1.141}]},{"osd":6,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.83399999999999996,"15min":0.81499999999999995},"min":{"1min":0.61699999999999999,"5min":0.55900000000000005,"15min":0.47699999999999998},"max":{"1min":1.008,"5min":1.1519999999999999,"15min":1.2330000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.84999999999999998,"15min":0.84399999999999997},"min":{"1min":0.70999999999999996,"5min":0.495,"15min":0.495},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.232},"last":1.298}]},{"osd":7,"last update":"Fri Jun 9 12:18:44 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89800000000000002,"15min":0.878},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.48299999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.349},{"interface":"front","average":{"1min":1.0309999999999999,"5min":0.98899999999999999,"15min":0.96599999999999997},"min":{"1min":0.83999999999999997,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":1.3979999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575674,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":231840,"kb_used_data":179800,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":94140000,"statfs":{"total":96636764160,"available":96399360000,"internally_reserved":0,"allocated":184115200,"data_stored":226382354,"data_compressed":19600304,"data_compressed_allocated":23134208,"data_compressed_original":69718016,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42899999999999999,"15min":0.45800000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.28000000000000003},"max":{"1min":0.629,"5min":0.629,"15min":0.80600000000000005},"last":0.313},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57499999999999996,"15min":0.56399999999999995},"min":{"1min":0.46600000000000003,"5min":0.432,"15min":0.38600000000000001},"max":{"1min":0.755,"5min":0.78800000000000003,"15min":0.79900000000000004},"last":0.5}]},{"osd":1,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45800000000000002,"15min":0.47399999999999998},"min":{"1min":0.32900000000000001,"5min":0.31900000000000001,"15min":0.318},"max":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":1.236},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.625,"15min":0.623},"min":{"1min":0.46899999999999997,"5min":0.44500000000000001,"15min":0.40200000000000002},"max":{"1min":0.80000000000000004,"5min":0.92200000000000004,"15min":1.2969999999999999},"last":0.58699999999999997}]},{"osd":3,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56399999999999995,"15min":0.59999999999999998},"min":{"1min":0.39100000000000001,"5min":0.38500000000000001,"15min":0.36599999999999999},"max":{"1min":0.79000000000000004,"5min":0.80900000000000005,"15min":1.4490000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.67000000000000004},"min":{"1min":0.435,"5min":0.42799999999999999,"15min":0.40500000000000003},"max":{"1min":0.872,"5min":1.298,"15min":1.4650000000000001},"last":0.53300000000000003}]},{"osd":4,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.52400000000000002,"5min":0.48199999999999998,"15min":0.48099999999999998},"max":{"1min":0.98799999999999999,"5min":1.218,"15min":1.218},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79600000000000004,"15min":0.79200000000000004},"min":{"1min":0.55800000000000005,"5min":0.55100000000000005,"15min":0.443},"max":{"1min":1.012,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.97999999999999998,"5min":1.337,"15min":1.4359999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.86399999999999999,"15min":0.875},"min":{"1min":0.63900000000000001,"5min":0.621,"15min":0.621},"max":{"1min":1.034,"5min":1.3640000000000001,"15min":1.5189999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.54700000000000004,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0900000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.86499999999999999,"15min":0.83099999999999996},"min":{"1min":0.69299999999999995,"5min":0.59099999999999997,"15min":0.55200000000000005},"max":{"1min":1.117,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.82699999999999996}]},{"osd":7,"last update":"Fri Jun 9 12:19:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.85899999999999999,"15min":0.84999999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54000000000000004},"max":{"1min":0.98499999999999999,"5min":1.286,"15min":1.286},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.89500000000000002,"15min":0.875},"min":{"1min":0.65000000000000002,"5min":0.629,"15min":0.57399999999999995},"max":{"1min":1.1399999999999999,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.66300000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575673,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426876,"kb_used_data":323188,"kb_used_omap":0,"kb_used_meta":103680,"kb_avail":93944964,"statfs":{"total":96636764160,"available":96199643136,"internally_reserved":0,"allocated":330944512,"data_stored":465154061,"data_compressed":44578155,"data_compressed_allocated":54284288,"data_compressed_original":195469312,"omap_allocated":0,"internal_metadata":106168320},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33900000000000002,"5min":0.32800000000000001,"15min":0.30499999999999999},"max":{"1min":0.64500000000000002,"5min":0.72599999999999998,"15min":0.81699999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.46800000000000003,"5min":0.40100000000000002,"15min":0.37},"max":{"1min":0.73399999999999999,"5min":0.77700000000000002,"15min":0.92300000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52200000000000002,"15min":0.53000000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63,"5min":0.80600000000000005,"15min":0.81999999999999995},"last":0.502},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.379},"max":{"1min":0.70099999999999996,"5min":0.78800000000000003,"15min":1.0600000000000001},"last":0.64500000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61799999999999999,"15min":0.63800000000000001},"min":{"1min":0.47499999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79700000000000004,"5min":0.93700000000000006,"15min":0.97099999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.56599999999999995},"min":{"1min":0.41599999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.70699999999999996,"5min":0.97699999999999998,"15min":1.1950000000000001},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.85399999999999998,"15min":0.82199999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0649999999999999,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83399999999999996,"15min":0.82199999999999995},"min":{"1min":0.65000000000000002,"5min":0.56200000000000006,"15min":0.56100000000000005},"max":{"1min":1.1120000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.84799999999999998,"15min":0.84599999999999997},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.52800000000000002},"max":{"1min":0.995,"5min":1.409,"15min":1.409},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.88600000000000001,"15min":0.89500000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.1930000000000001,"5min":1.452,"15min":1.452},"last":1.016}]},{"osd":6,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80400000000000005,"15min":0.80800000000000005},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97899999999999998,"5min":1.099,"15min":1.198},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.82399999999999995,"15min":0.82599999999999996},"min":{"1min":0.59999999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.94899999999999995,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.85199999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:18:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80900000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1659999999999999,"5min":1.371,"15min":1.371},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.83099999999999996,"15min":0.84799999999999998},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0649999999999999,"5min":1.163,"15min":1.4099999999999999},"last":0.91800000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575672,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418516,"kb_used_data":292748,"kb_used_omap":0,"kb_used_meta":125760,"kb_avail":93953324,"statfs":{"total":96636764160,"available":96208203776,"internally_reserved":0,"allocated":299773952,"data_stored":481350279,"data_compressed":51603129,"data_compressed_allocated":62935040,"data_compressed_original":250671104,"omap_allocated":0,"internal_metadata":128778240},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.747,"15min":0.74399999999999999},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.90000000000000002,"5min":1.0549999999999999,"15min":1.419},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.75,"5min":0.79800000000000004,"15min":0.78700000000000003},"min":{"1min":0.502,"5min":0.502,"15min":0.46500000000000002},"max":{"1min":0.94599999999999995,"5min":1.1160000000000001,"15min":1.383},"last":0.81899999999999995}]},{"osd":1,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.377},"max":{"1min":0.99099999999999999,"5min":1.242,"15min":1.242},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.83899999999999997,"15min":0.80100000000000005},"min":{"1min":0.71299999999999997,"5min":0.58299999999999996,"15min":0.41599999999999998},"max":{"1min":1.069,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.77600000000000002}]},{"osd":2,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.755,"15min":0.73499999999999999},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.39700000000000002},"max":{"1min":0.873,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.81299999999999994,"15min":0.80000000000000004},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.441},"max":{"1min":1.0309999999999999,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.86099999999999999}]},{"osd":3,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.747,"15min":0.71099999999999997},"min":{"1min":0.52700000000000002,"5min":0.51500000000000001,"15min":0.46899999999999997},"max":{"1min":0.91600000000000004,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77800000000000002,"15min":0.75},"min":{"1min":0.59399999999999997,"5min":0.55200000000000005,"15min":0.49199999999999999},"max":{"1min":0.88600000000000001,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.80200000000000005}]},{"osd":5,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.436,"5min":0.36299999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":0.85099999999999998,"15min":0.91500000000000004},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.44800000000000001,"5min":0.41499999999999998,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.92500000000000004,"15min":1.05},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.58199999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78600000000000003,"5min":0.97799999999999998,"15min":1.0509999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76000000000000001,"15min":0.746},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Jun 9 12:18:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.628,"15min":0.64500000000000002},"min":{"1min":0.46899999999999997,"5min":0.41999999999999998,"15min":0.39500000000000002},"max":{"1min":0.74099999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71899999999999997,"15min":0.65300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38300000000000001},"max":{"1min":1.101,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575672,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299800,"kb_used_data":217360,"kb_used_omap":0,"kb_used_meta":82432,"kb_avail":94072040,"statfs":{"total":96636764160,"available":96329768960,"internally_reserved":0,"allocated":222576640,"data_stored":326221930,"data_compressed":32287430,"data_compressed_allocated":39153664,"data_compressed_original":147243008,"omap_allocated":0,"internal_metadata":84410368},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.68100000000000005,"15min":0.68999999999999995},"min":{"1min":0.51700000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.89600000000000002,"5min":0.95899999999999996,"15min":1.1100000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71399999999999997,"15min":0.72099999999999997},"min":{"1min":0.442,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.96199999999999997,"5min":0.98399999999999999,"15min":1.579},"last":0.92400000000000004}]},{"osd":1,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71099999999999997,"15min":0.71799999999999997},"min":{"1min":0.497,"5min":0.439,"15min":0.41199999999999998},"max":{"1min":0.92600000000000005,"5min":0.97199999999999998,"15min":1.2689999999999999},"last":1.012},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.57099999999999995,"5min":0.47399999999999998,"15min":0.45700000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.127},"last":0.90800000000000003}]},{"osd":2,"last update":"Fri Jun 9 12:18:58 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.72499999999999998},"min":{"1min":0.51000000000000001,"5min":0.47799999999999998,"15min":0.42499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.4339999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.751,"15min":0.74399999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.437},"max":{"1min":0.95499999999999996,"5min":0.98299999999999998,"15min":1.391},"last":0.94999999999999996}]},{"osd":3,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.78100000000000003,"15min":0.75800000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47399999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.254},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.80400000000000005,"15min":0.81100000000000005},"min":{"1min":0.65000000000000002,"5min":0.58299999999999996,"15min":0.48199999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.3120000000000001},"last":1.042}]},{"osd":4,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56499999999999995,"15min":0.55500000000000005},"min":{"1min":0.43099999999999999,"5min":0.40699999999999997,"15min":0.35999999999999999},"max":{"1min":0.69199999999999995,"5min":0.89000000000000001,"15min":1.522},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56799999999999995,"15min":0.58099999999999996},"min":{"1min":0.44700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.88900000000000001,"5min":0.96199999999999997,"15min":1.28},"last":0.88}]},{"osd":6,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.65900000000000003,"15min":0.64500000000000002},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.40200000000000002},"max":{"1min":0.748,"5min":0.999,"15min":1.6619999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.69299999999999995,"15min":0.70399999999999996},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.41999999999999998},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.97699999999999998}]},{"osd":7,"last update":"Fri Jun 9 12:18:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.75600000000000001,"5min":0.98099999999999998,"15min":1.2529999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73999999999999999,"15min":0.748},"min":{"1min":0.66200000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.91300000000000003,"5min":0.99399999999999999,"15min":1.238},"last":1.095}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":32389,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":36626432,"data_stored":130901034,"data_compressed":16198988,"data_compressed_allocated":21467136,"data_compressed_original":115773440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":33550336,"data_stored":102645651,"data_compressed":11967129,"data_compressed_allocated":15982592,"data_compressed_original":85082112,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16809984,"data_stored":50331738,"data_compressed":5846951,"data_compressed_allocated":7913472,"data_compressed_original":41439232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48644096,"data_stored":166224311,"data_compressed":20747038,"data_compressed_allocated":27779072,"data_compressed_original":145379328,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":64053248,"data_stored":230699069,"data_compressed":29157108,"data_compressed_allocated":38182912,"data_compressed_original":204869632,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":35729408,"data_stored":129235418,"data_compressed":16265872,"data_compressed_allocated":21504000,"data_compressed_original":115032064,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12550144,"data_stored":50331760,"data_compressed":6404815,"data_compressed_allocated":8544256,"data_compressed_original":46333952,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":42442752,"data_stored":159971903,"data_compressed":20706527,"data_compressed_allocated":26832896,"data_compressed_original":144392192,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195702784,"data_stored":207664766,"data_compressed":19707307,"data_compressed_allocated":21622784,"data_compressed_original":38182912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":279330816,"data_stored":295073194,"data_compressed":23332696,"data_compressed_allocated":25886720,"data_compressed_original":48046080,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166424576,"data_stored":175563189,"data_compressed":13747592,"data_compressed_allocated":15212544,"data_compressed_original":28262400,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":281419776,"data_stored":298442323,"data_compressed":23825356,"data_compressed_allocated":26497024,"data_compressed_original":50073600,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":234840064,"data_stored":250163783,"data_compressed":22440260,"data_compressed_allocated":24743936,"data_compressed_original":45785088,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":185966592,"data_stored":196499085,"data_compressed":16015797,"data_compressed_allocated":17641472,"data_compressed_original":32194560,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":146427904,"data_stored":155040735,"data_compressed":11667258,"data_compressed_allocated":13111296,"data_compressed_original":25640960,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":140775424,"data_stored":149897015,"data_compressed":11262388,"data_compressed_allocated":12734464,"data_compressed_original":26021888,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-09T12:19:14.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-06-09T12:19:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.946+0000 7feb4a7ff700 1 Processor -- start 2023-06-09T12:19:14.950 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.947+0000 7feb4a7ff700 1 -- start start 2023-06-09T12:19:14.951 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb4a7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb44153370 0x7feb440b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb4a7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44071ca0 con 0x7feb44153740 2023-06-09T12:19:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb4a7ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44071120 con 0x7feb440c9fa0 2023-06-09T12:19:14.952 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb4a7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb44159770 con 0x7feb44153370 2023-06-09T12:19:14.953 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb44153370 0x7feb440b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb43fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7feb44153370 0x7feb440b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58158/0 (socket says 172.21.15.7:58158) 2023-06-09T12:19:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb43fff700 1 -- 172.21.15.7:0/894667350 learned_addr learned my addr 172.21.15.7:0/894667350 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb48d9c700 1 --1- 172.21.15.7:0/894667350 >> v1:172.21.15.7:6789/0 conn(0x7feb44153740 0x7feb44159b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43420/0 (socket says 172.21.15.7:43420) 2023-06-09T12:19:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb43fff700 1 -- 172.21.15.7:0/894667350 >> v1:172.21.15.117:6789/0 conn(0x7feb440c9fa0 legacy=0x7feb440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:14.954 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.948+0000 7feb43fff700 1 -- 172.21.15.7:0/894667350 >> v1:172.21.15.7:6789/0 conn(0x7feb44153740 legacy=0x7feb44159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb43fff700 1 -- 172.21.15.7:0/894667350 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44071400 con 0x7feb44153370 2023-06-09T12:19:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb43fff700 1 --2- 172.21.15.7:0/894667350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153370 0x7feb440b4450 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7feb340098f0 tx=0x7feb34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43507bfe24b81d11 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb42ffd700 1 -- 172.21.15.7:0/894667350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34004230 con 0x7feb44153370 2023-06-09T12:19:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb42ffd700 1 -- 172.21.15.7:0/894667350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34004390 con 0x7feb44153370 2023-06-09T12:19:14.955 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb42ffd700 1 -- 172.21.15.7:0/894667350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340036a0 con 0x7feb44153370 2023-06-09T12:19:14.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb4a7ff700 1 -- 172.21.15.7:0/894667350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153370 msgr2=0x7feb440b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.949+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/894667350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153370 0x7feb440b4450 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7feb340098f0 tx=0x7feb34009c00 comp rx=0 tx=0).stop 2023-06-09T12:19:14.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.950+0000 7feb4a7ff700 1 -- 172.21.15.7:0/894667350 shutdown_connections 2023-06-09T12:19:14.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.950+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/894667350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153370 0x7feb440b4450 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.956 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.950+0000 7feb4a7ff700 1 -- 172.21.15.7:0/894667350 >> 172.21.15.7:0/894667350 conn(0x7feb440737f0 msgr2=0x7feb44073bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.950+0000 7feb4a7ff700 1 -- 172.21.15.7:0/894667350 shutdown_connections 2023-06-09T12:19:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.950+0000 7feb4a7ff700 1 -- 172.21.15.7:0/894667350 wait complete. 2023-06-09T12:19:14.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.951+0000 7feb4a7ff700 1 Processor -- start 2023-06-09T12:19:14.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 -- start start 2023-06-09T12:19:14.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb440c9fa0 0x7feb441e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb44153370 0x7feb441eef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.959 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441dfa20 con 0x7feb44153740 2023-06-09T12:19:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb440b21f0 con 0x7feb44153370 2023-06-09T12:19:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.952+0000 7feb4a7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440c7270 con 0x7feb440c9fa0 2023-06-09T12:19:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58166/0 (socket says 172.21.15.7:58166) 2023-06-09T12:19:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 -- 172.21.15.7:0/1499101090 learned_addr learned my addr 172.21.15.7:0/1499101090 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb43fff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb440c9fa0 0x7feb441e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb437fe700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb44153370 0x7feb441eef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 -- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb440c9fa0 msgr2=0x7feb441e5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb440c9fa0 0x7feb441e5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 -- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb44153370 msgr2=0x7feb441eef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb44153370 0x7feb441eef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 -- 172.21.15.7:0/1499101090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34009550 con 0x7feb44153740 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.953+0000 7feb48d9c700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00cbc0 tx=0x7feb3c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.954+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00a9e0 con 0x7feb44153740 2023-06-09T12:19:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.954+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c00ab40 con 0x7feb44153740 2023-06-09T12:19:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.954+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c007070 con 0x7feb44153740 2023-06-09T12:19:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.954+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb440719c0 con 0x7feb44153740 2023-06-09T12:19:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.954+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb440716e0 con 0x7feb44153740 2023-06-09T12:19:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.955+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440c5040 con 0x7feb44153740 2023-06-09T12:19:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.956+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb3c0071d0 con 0x7feb44153740 2023-06-09T12:19:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.957+0000 7feb40ff9700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 0x7feb3006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.957+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7feb3c089a30 con 0x7feb44153740 2023-06-09T12:19:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.957+0000 7feb43fff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 0x7feb3006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.958+0000 7feb43fff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 0x7feb3006d1c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feb34009130 tx=0x7feb34008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:14.965 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:14.961+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7feb3c054180 con 0x7feb44153740 2023-06-09T12:19:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.171+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7feb440c5c60 con 0x7feb44153740 2023-06-09T12:19:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.175+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7feb3c001c10 con 0x7feb44153740 2023-06-09T12:19:15.179 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.175+0000 7feb40ff9700 1 -- 172.21.15.7:0/1499101090 <== mon.0 v2:172.21.15.7:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7feb3c017020 con 0x7feb44153740 2023-06-09T12:19:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 msgr2=0x7feb3006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.182 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 0x7feb3006d1c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7feb34009130 tx=0x7feb34008030 comp rx=0 tx=0).stop 2023-06-09T12:19:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 msgr2=0x7feb4414d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00cbc0 tx=0x7feb3c00ced0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 shutdown_connections 2023-06-09T12:19:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb44153740 0x7feb4414d4e0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.183 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7feb3006ad10 0x7feb3006d1c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb440c9fa0 0x7feb441e5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 --2- 172.21.15.7:0/1499101090 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb44153370 0x7feb441eef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 >> 172.21.15.7:0/1499101090 conn(0x7feb440737f0 msgr2=0x7feb440b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 shutdown_connections 2023-06-09T12:19:15.184 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.178+0000 7feb4a7ff700 1 -- 172.21.15.7:0/1499101090 wait complete. 2023-06-09T12:19:15.201 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-06-09T12:19:15.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T12:19:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.398+0000 7fc0b1fff700 1 Processor -- start 2023-06-09T12:19:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.398+0000 7fc0b1fff700 1 -- start start 2023-06-09T12:19:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.399+0000 7fc0b1fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0ac15b4a0 0x7fc0ac15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.399+0000 7fc0b1fff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fc0ac0c9f40 0x7fc0ac0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.399+0000 7fc0b1fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ac15a500 con 0x7fc0ac153880 2023-06-09T12:19:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.399+0000 7fc0b1fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac0c5f10 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.399+0000 7fc0b1fff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ac15da00 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fc0ac15b4a0 0x7fc0ac15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0ab7fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fc0ac153880 0x7fc0ac15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43430/0 (socket says 172.21.15.7:43430) 2023-06-09T12:19:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/1914685535 learned_addr learned my addr 172.21.15.7:0/1914685535 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0abfff700 1 --2- 172.21.15.7:0/1914685535 >> v2:172.21.15.117:3300/0 conn(0x7fc0ac0c9f40 0x7fc0ac0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 -- 172.21.15.7:0/1914685535 >> v2:172.21.15.117:3300/0 conn(0x7fc0ac0c9f40 msgr2=0x7fc0ac0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 --2- 172.21.15.7:0/1914685535 >> v2:172.21.15.117:3300/0 conn(0x7fc0ac0c9f40 0x7fc0ac0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 -- 172.21.15.7:0/1914685535 >> v1:172.21.15.7:6789/0 conn(0x7fc0ac153880 legacy=0x7fc0ac15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 -- 172.21.15.7:0/1914685535 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac1596e0 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.400+0000 7fc0aaffd700 1 --2- 172.21.15.7:0/1914685535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac15b4a0 0x7fc0ac15d4c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0749f0 tx=0x7fc0a0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65691158147bf88 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0aa7fc700 1 -- 172.21.15.7:0/1914685535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0011830 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0aa7fc700 1 -- 172.21.15.7:0/1914685535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc0a0011990 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0aa7fc700 1 -- 172.21.15.7:0/1914685535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0011d40 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 -- 172.21.15.7:0/1914685535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac15b4a0 msgr2=0x7fc0ac15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/1914685535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac15b4a0 0x7fc0ac15d4c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0749f0 tx=0x7fc0a0009b20 comp rx=0 tx=0).stop 2023-06-09T12:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 -- 172.21.15.7:0/1914685535 shutdown_connections 2023-06-09T12:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/1914685535 >> v2:172.21.15.117:3300/0 conn(0x7fc0ac0c9f40 0x7fc0ac0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.408 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/1914685535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac15b4a0 0x7fc0ac15d4c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 -- 172.21.15.7:0/1914685535 >> 172.21.15.7:0/1914685535 conn(0x7fc0ac0737f0 msgr2=0x7fc0ac0720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 -- 172.21.15.7:0/1914685535 shutdown_connections 2023-06-09T12:19:15.409 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.401+0000 7fc0b1fff700 1 -- 172.21.15.7:0/1914685535 wait complete. 2023-06-09T12:19:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.403+0000 7fc0b1fff700 1 Processor -- start 2023-06-09T12:19:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.403+0000 7fc0b1fff700 1 -- start start 2023-06-09T12:19:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ac153880 0x7fc0ac0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.411 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ac15b4a0 0x7fc0ac1ef230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cb880 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac14dbc0 con 0x7fc0ac153880 2023-06-09T12:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0b1fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac0cac90 con 0x7fc0ac15b4a0 2023-06-09T12:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.412 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0ab7fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58188/0 (socket says 172.21.15.7:58188) 2023-06-09T12:19:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.404+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/2141835893 learned_addr learned my addr 172.21.15.7:0/2141835893 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ac15b4a0 msgr2=0x7fc0ac1ef230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ac15b4a0 0x7fc0ac1ef230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ac153880 msgr2=0x7fc0ac0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ac153880 0x7fc0ac0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 -- 172.21.15.7:0/2141835893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a00094a0 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0ab7fe700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c550 tx=0x7fc09c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c005d70 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fc09c0070d0 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.414 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c007460 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac159c60 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.405+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac159980 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.415 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.406+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac159f40 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.407+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc09c0075c0 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.409+0000 7fc093fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 0x7fc09806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.409+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7fc09c0891e0 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.416 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.409+0000 7fc0aaffd700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 0x7fc09806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.409+0000 7fc0aaffd700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 0x7fc09806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac15c6f0 tx=0x7fc0a0010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.417 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.413+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fc09c053880 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.663+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0ac15a500 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.667 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.664+0000 7fc093fff700 1 -- 172.21.15.7:0/2141835893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 228 v228) v1 ==== 95+0+6208 (secure 0 0 0) 0x7fc0ac15a500 con 0x7fc0ac0c9f40 2023-06-09T12:19:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:19:15.668 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":228,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8886,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1757884923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1757884923},{"type":"v1","addr":"172.21.15.7:6839","nonce":1757884923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T12:02:40.028342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T12:19:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 msgr2=0x7fc09806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 0x7fc09806d190 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac15c6f0 tx=0x7fc0a0010000 comp rx=0 tx=0).stop 2023-06-09T12:19:15.670 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 msgr2=0x7fc0ac0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c550 tx=0x7fc09c00c910 comp rx=0 tx=0).stop 2023-06-09T12:19:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 shutdown_connections 2023-06-09T12:19:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fc09806ace0 0x7fc09806d190 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc0ac15b4a0 0x7fc0ac1ef230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.671 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc0ac0c9f40 0x7fc0ac0cf010 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 --2- 172.21.15.7:0/2141835893 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc0ac153880 0x7fc0ac0cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 >> 172.21.15.7:0/2141835893 conn(0x7fc0ac0737f0 msgr2=0x7fc0ac06ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 shutdown_connections 2023-06-09T12:19:15.672 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.666+0000 7fc0b1fff700 1 -- 172.21.15.7:0/2141835893 wait complete. 2023-06-09T12:19:15.673 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 228 2023-06-09T12:19:15.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-09T12:19:15.888 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.885+0000 7f5677dff700 1 Processor -- start 2023-06-09T12:19:15.889 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.885+0000 7f5677dff700 1 -- start start 2023-06-09T12:19:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5677dff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5670153880 0x7f567015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5677dff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 0x7f56700c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.890 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5677dff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567015a500 con 0x7f56700c9f40 2023-06-09T12:19:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5677dff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56700c5f10 con 0x7f5670153880 2023-06-09T12:19:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5677dff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567015da00 con 0x7f567015b4a0 2023-06-09T12:19:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5675b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5670153880 0x7f567015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5675b9b700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5670153880 0x7f567015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58204/0 (socket says 172.21.15.7:58204) 2023-06-09T12:19:15.891 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.886+0000 7f5675b9b700 1 -- 172.21.15.7:0/271968830 learned_addr learned my addr 172.21.15.7:0/271968830 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5675b9b700 1 -- 172.21.15.7:0/271968830 >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 msgr2=0x7f56700c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f567639c700 1 --2- 172.21.15.7:0/271968830 >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 0x7f56700c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5675b9b700 1 --2- 172.21.15.7:0/271968830 >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 0x7f56700c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.893 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5675b9b700 1 -- 172.21.15.7:0/271968830 >> v1:172.21.15.117:6789/0 conn(0x7f56700c9f40 legacy=0x7f567015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5675b9b700 1 -- 172.21.15.7:0/271968830 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56701596e0 con 0x7f5670153880 2023-06-09T12:19:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f567639c700 1 --2- 172.21.15.7:0/271968830 >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 0x7f56700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:19:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5675b9b700 1 --2- 172.21.15.7:0/271968830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670153880 0x7f567015d4c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f56700c4640 tx=0x7f566c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400b889dea3f5e51 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5674b99700 1 -- 172.21.15.7:0/271968830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c004230 con 0x7f5670153880 2023-06-09T12:19:15.894 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5674b99700 1 -- 172.21.15.7:0/271968830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f566c004390 con 0x7f5670153880 2023-06-09T12:19:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.887+0000 7f5674b99700 1 -- 172.21.15.7:0/271968830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0036a0 con 0x7f5670153880 2023-06-09T12:19:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 -- 172.21.15.7:0/271968830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670153880 msgr2=0x7f567015d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 --2- 172.21.15.7:0/271968830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670153880 0x7f567015d4c0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f56700c4640 tx=0x7f566c009bd0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 -- 172.21.15.7:0/271968830 shutdown_connections 2023-06-09T12:19:15.895 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 --2- 172.21.15.7:0/271968830 >> v2:172.21.15.117:3300/0 conn(0x7f567015b4a0 0x7f56700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 --2- 172.21.15.7:0/271968830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670153880 0x7f567015d4c0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 -- 172.21.15.7:0/271968830 >> 172.21.15.7:0/271968830 conn(0x7f56700737f0 msgr2=0x7f56700720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 -- 172.21.15.7:0/271968830 shutdown_connections 2023-06-09T12:19:15.896 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.888+0000 7f5677dff700 1 -- 172.21.15.7:0/271968830 wait complete. 2023-06-09T12:19:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.890+0000 7f5677dff700 1 Processor -- start 2023-06-09T12:19:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.890+0000 7f5677dff700 1 -- start start 2023-06-09T12:19:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5670153880 0x7f567015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f567015b4a0 0x7f56701eef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56701f22d0 con 0x7f56700c9f40 2023-06-09T12:19:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700b2070 con 0x7f5670153880 2023-06-09T12:19:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5677dff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56701f24e0 con 0x7f567015b4a0 2023-06-09T12:19:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5675b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5675b9b700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58210/0 (socket says 172.21.15.7:58210) 2023-06-09T12:19:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f5675b9b700 1 -- 172.21.15.7:0/3952833871 learned_addr learned my addr 172.21.15.7:0/3952833871 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f567539a700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5670153880 0x7f567015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.891+0000 7f567639c700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f567015b4a0 0x7f56701eef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.900 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 -- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f567015b4a0 msgr2=0x7f56701eef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f567015b4a0 0x7f56701eef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 -- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5670153880 msgr2=0x7f567015bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5670153880 0x7f567015bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 -- 172.21.15.7:0/3952833871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c009550 con 0x7f56700c9f40 2023-06-09T12:19:15.901 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5675b9b700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f567006b580 tx=0x7f566c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c003c50 con 0x7f56700c9f40 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f566c01f070 con 0x7f56700c9f40 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c011630 con 0x7f56700c9f40 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5670159c60 con 0x7f56700c9f40 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.892+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5670159980 con 0x7f56700c9f40 2023-06-09T12:19:15.902 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.893+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670159f40 con 0x7f56700c9f40 2023-06-09T12:19:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.894+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f566c021030 con 0x7f56700c9f40 2023-06-09T12:19:15.903 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.895+0000 7f56667fc700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 0x7f565406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.895+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f566c08b140 con 0x7f56700c9f40 2023-06-09T12:19:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.896+0000 7f567539a700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 0x7f565406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.896+0000 7f567539a700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 0x7f565406d190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f56700b1dd0 tx=0x7f5660007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:15.899+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f566c055900 con 0x7f56700c9f40 2023-06-09T12:19:16.155 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.152+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56701f30f0 con 0x7f56700c9f40 2023-06-09T12:19:16.156 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.152+0000 7f56667fc700 1 -- 172.21.15.7:0/3952833871 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 228 v228) v1 ==== 95+0+6208 (secure 0 0 0) 0x7f566c029090 con 0x7f56700c9f40 2023-06-09T12:19:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:19:16.156 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":228,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6836/1511608042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6835","nonce":1511608042},{"type":"v1","addr":"172.21.15.7:6836","nonce":1511608042}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6837/2096062193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":2096062193},{"type":"v1","addr":"172.21.15.7:6837","nonce":2096062193}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":7930,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.117:6835/3975520914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6834","nonce":3975520914},{"type":"v1","addr":"172.21.15.117:6835","nonce":3975520914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":209},{"gid":8886,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6839/1757884923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6838","nonce":1757884923},{"type":"v1","addr":"172.21.15.7:6839","nonce":1757884923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":226}],"filesystems":[{"mdsmap":{"epoch":228,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-06-09T11:10:50.179197+0000","modified":"2023-06-09T12:02:40.028342+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":74,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":8056,"mds_1":7807},"failed":[],"damaged":[],"stopped":[2],"info":{"gid_7807":{"gid":7807,"name":"f","rank":1,"incarnation":204,"state":"up:active","state_seq":37,"addr":"172.21.15.117:6837/3903572761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6836","nonce":3903572761},{"type":"v1","addr":"172.21.15.117:6837","nonce":3903572761}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_8056":{"gid":8056,"name":"b","rank":0,"incarnation":219,"state":"up:active","state_seq":26,"addr":"172.21.15.117:6833/2893966006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":2893966006},{"type":"v1","addr":"172.21.15.117:6833","nonce":2893966006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-09T12:19:16.158 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 msgr2=0x7f565406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 0x7f565406d190 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f56700b1dd0 tx=0x7f5660007ec0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 msgr2=0x7f56700b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f567006b580 tx=0x7f566c004750 comp rx=0 tx=0).stop 2023-06-09T12:19:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 shutdown_connections 2023-06-09T12:19:16.159 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f565406ace0 0x7f565406d190 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f567015b4a0 0x7f56701eef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56700c9f40 0x7f56700b3910 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 --2- 172.21.15.7:0/3952833871 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5670153880 0x7f567015bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 >> 172.21.15.7:0/3952833871 conn(0x7f56700737f0 msgr2=0x7f567014b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:16.160 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 shutdown_connections 2023-06-09T12:19:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.155+0000 7f5677dff700 1 -- 172.21.15.7:0/3952833871 wait complete. 2023-06-09T12:19:16.161 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 228 2023-06-09T12:19:16.176 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-09T12:19:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.373+0000 7f6a00fff700 1 Processor -- start 2023-06-09T12:19:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.373+0000 7f6a00fff700 1 -- start start 2023-06-09T12:19:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.374+0000 7f6a00fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 0x7f69fc15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.374+0000 7f6a00fff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f69fc0ca580 0x7f69fc0c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.374+0000 7f6a00fff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69fc15dcf0 con 0x7f69fc0ca1b0 2023-06-09T12:19:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.374+0000 7f6a00fff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69fc0c5e00 con 0x7f69fc0ca580 2023-06-09T12:19:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.374+0000 7f6a00fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc15d450 con 0x7f69fc0b5360 2023-06-09T12:19:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fa59c700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f69fc0ca1b0 0x7f69fc1536f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43434/0 (socket says 172.21.15.7:43434) 2023-06-09T12:19:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fa59c700 1 -- 172.21.15.7:0/2027171049 learned_addr learned my addr 172.21.15.7:0/2027171049 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:19:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 --2- 172.21.15.7:0/2027171049 >> v2:172.21.15.7:3300/0 conn(0x7f69fc0ca580 0x7f69fc0c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69f9d9b700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 0x7f69fc15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69f959a700 1 -- 172.21.15.7:0/2027171049 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654008931 0 0) 0x7f69fc15dcf0 con 0x7f69fc0ca1b0 2023-06-09T12:19:16.381 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69f959a700 1 -- 172.21.15.7:0/2027171049 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69e40035b0 con 0x7f69fc0ca1b0 2023-06-09T12:19:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 -- 172.21.15.7:0/2027171049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 msgr2=0x7f69fc15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 0x7f69fc15cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 -- 172.21.15.7:0/2027171049 >> v1:172.21.15.7:6789/0 conn(0x7f69fc0ca1b0 legacy=0x7f69fc1536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 -- 172.21.15.7:0/2027171049 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69fc15d730 con 0x7f69fc0ca580 2023-06-09T12:19:16.382 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69f9d9b700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 0x7f69fc15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:19:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.375+0000 7f69fad9d700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0ca580 0x7f69fc0c58c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009d90 tx=0x7f69f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd7e2808f5351c8 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f69f959a700 1 -- 172.21.15.7:0/2027171049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f400e070 con 0x7f69fc0ca580 2023-06-09T12:19:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f69f959a700 1 -- 172.21.15.7:0/2027171049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f69f400bb20 con 0x7f69fc0ca580 2023-06-09T12:19:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f69f959a700 1 -- 172.21.15.7:0/2027171049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69f4012430 con 0x7f69fc0ca580 2023-06-09T12:19:16.383 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 -- 172.21.15.7:0/2027171049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0ca580 msgr2=0x7f69fc0c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0ca580 0x7f69fc0c58c0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009d90 tx=0x7f69f400b330 comp rx=0 tx=0).stop 2023-06-09T12:19:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 -- 172.21.15.7:0/2027171049 shutdown_connections 2023-06-09T12:19:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0ca580 0x7f69fc0c58c0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2027171049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0b5360 0x7f69fc15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.384 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.376+0000 7f6a00fff700 1 -- 172.21.15.7:0/2027171049 >> 172.21.15.7:0/2027171049 conn(0x7f69fc072090 msgr2=0x7f69fc072490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.377+0000 7f6a00fff700 1 -- 172.21.15.7:0/2027171049 shutdown_connections 2023-06-09T12:19:16.385 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.377+0000 7f6a00fff700 1 -- 172.21.15.7:0/2027171049 wait complete. 2023-06-09T12:19:16.386 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.378+0000 7f6a00fff700 1 Processor -- start 2023-06-09T12:19:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.378+0000 7f6a00fff700 1 -- start start 2023-06-09T12:19:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69fc0ca1b0 0x7f69fc1f1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0ca580 0x7f69fc149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc148b80 con 0x7f69fc0b5360 2023-06-09T12:19:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc1f00a0 con 0x7f69fc0ca1b0 2023-06-09T12:19:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f6a00fff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69fc14b810 con 0x7f69fc0ca580 2023-06-09T12:19:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f69fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.379+0000 7f69fa59c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58228/0 (socket says 172.21.15.7:58228) 2023-06-09T12:19:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 -- 172.21.15.7:0/2245438931 learned_addr learned my addr 172.21.15.7:0/2245438931 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69f9d9b700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69fc0ca1b0 0x7f69fc1f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fad9d700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0ca580 0x7f69fc149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 -- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0ca580 msgr2=0x7f69fc149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0ca580 0x7f69fc149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 -- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69fc0ca1b0 msgr2=0x7f69fc1f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69fc0ca1b0 0x7f69fc1f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 -- 172.21.15.7:0/2245438931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4009a70 con 0x7f69fc0b5360 2023-06-09T12:19:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69fa59c700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c700 tx=0x7f69ec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.380+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec005d70 con 0x7f69fc0b5360 2023-06-09T12:19:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.381+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f69ec0070d0 con 0x7f69fc0b5360 2023-06-09T12:19:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.381+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69fc06a570 con 0x7f69fc0b5360 2023-06-09T12:19:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.381+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ec007480 con 0x7f69fc0b5360 2023-06-09T12:19:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.381+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69fc15da10 con 0x7f69fc0b5360 2023-06-09T12:19:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.382+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69fc06a850 con 0x7f69fc0b5360 2023-06-09T12:19:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.383+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f69ec0075e0 con 0x7f69fc0b5360 2023-06-09T12:19:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.384+0000 7f69eaffd700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 0x7f69dc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.384+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f69ec089310 con 0x7f69fc0b5360 2023-06-09T12:19:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.384+0000 7f69f9d9b700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 0x7f69dc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.384+0000 7f69f9d9b700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 0x7f69dc06d1c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f69fc14ed20 tx=0x7f69f0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.388+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f69ec0539b0 con 0x7f69fc0b5360 2023-06-09T12:19:16.600 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.597+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69fc1f0990 con 0x7f69fc0b5360 2023-06-09T12:19:16.602 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.598+0000 7f69eaffd700 1 -- 172.21.15.7:0/2245438931 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+21641 (secure 0 0 0) 0x7f69ec017070 con 0x7f69fc0b5360 2023-06-09T12:19:16.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2023-06-09T12:19:16.603 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"fsid":"cb7aac03-4bc4-45fa-8177-804a746951e5","created":"2023-06-09T11:10:29.135622+0000","modified":"2023-06-09T11:59:17.800788+0000","last_up_change":"2023-06-09T11:10:41.542612+0000","last_in_change":"2023-06-09T11:10:33.070734+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-09T11:10:42.244017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-06-09T11:10:42.285603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-06-09T11:10:47.777397+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-06-09T11:10:49.153345+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":505,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8beb7b04-04e2-4c76-843c-6347fd4b7423","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6809","nonce":1473672210}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6811","nonce":1473672210}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6815","nonce":1473672210}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":1473672210},{"type":"v1","addr":"172.21.15.7:6813","nonce":1473672210}]},"public_addr":"172.21.15.7:6809/1473672210","cluster_addr":"172.21.15.7:6811/1473672210","heartbeat_back_addr":"172.21.15.7:6815/1473672210","heartbeat_front_addr":"172.21.15.7:6813/1473672210","state":["exists","up"]},{"osd":1,"uuid":"c86f3c75-2178-4498-9490-309afe7973e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6801","nonce":48537347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6803","nonce":48537347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6807","nonce":48537347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":48537347},{"type":"v1","addr":"172.21.15.7:6805","nonce":48537347}]},"public_addr":"172.21.15.7:6801/48537347","cluster_addr":"172.21.15.7:6803/48537347","heartbeat_back_addr":"172.21.15.7:6807/48537347","heartbeat_front_addr":"172.21.15.7:6805/48537347","state":["exists","up"]},{"osd":2,"uuid":"10f09bd4-5cd4-4ae1-84c1-9b7a62f876bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6817","nonce":843671507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6819","nonce":843671507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6823","nonce":843671507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":843671507},{"type":"v1","addr":"172.21.15.7:6821","nonce":843671507}]},"public_addr":"172.21.15.7:6817/843671507","cluster_addr":"172.21.15.7:6819/843671507","heartbeat_back_addr":"172.21.15.7:6823/843671507","heartbeat_front_addr":"172.21.15.7:6821/843671507","state":["exists","up"]},{"osd":3,"uuid":"caab7ecc-cafe-4ff5-8472-dffc52ac78d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6825","nonce":353233282}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6827","nonce":353233282}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6831","nonce":353233282}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":353233282},{"type":"v1","addr":"172.21.15.7:6829","nonce":353233282}]},"public_addr":"172.21.15.7:6825/353233282","cluster_addr":"172.21.15.7:6827/353233282","heartbeat_back_addr":"172.21.15.7:6831/353233282","heartbeat_front_addr":"172.21.15.7:6829/353233282","state":["exists","up"]},{"osd":4,"uuid":"267f3c53-d8f4-4f37-90bd-8b39be8322f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6801","nonce":3721658998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6803","nonce":3721658998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6807","nonce":3721658998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":3721658998},{"type":"v1","addr":"172.21.15.117:6805","nonce":3721658998}]},"public_addr":"172.21.15.117:6801/3721658998","cluster_addr":"172.21.15.117:6803/3721658998","heartbeat_back_addr":"172.21.15.117:6807/3721658998","heartbeat_front_addr":"172.21.15.117:6805/3721658998","state":["exists","up"]},{"osd":5,"uuid":"2a285f66-b951-48de-9909-2de669a508bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6825","nonce":644345713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6827","nonce":644345713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6831","nonce":644345713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":644345713},{"type":"v1","addr":"172.21.15.117:6829","nonce":644345713}]},"public_addr":"172.21.15.117:6825/644345713","cluster_addr":"172.21.15.117:6827/644345713","heartbeat_back_addr":"172.21.15.117:6831/644345713","heartbeat_front_addr":"172.21.15.117:6829/644345713","state":["exists","up"]},{"osd":6,"uuid":"ed9a82da-beba-41bd-a493-90c966e52b1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6809","nonce":2098282767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6811","nonce":2098282767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6815","nonce":2098282767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":2098282767},{"type":"v1","addr":"172.21.15.117:6813","nonce":2098282767}]},"public_addr":"172.21.15.117:6809/2098282767","cluster_addr":"172.21.15.117:6811/2098282767","heartbeat_back_addr":"172.21.15.117:6815/2098282767","heartbeat_front_addr":"172.21.15.117:6813/2098282767","state":["exists","up"]},{"osd":7,"uuid":"97300c66-1e58-4029-91ed-e65bc4497fe3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6817","nonce":1383862628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6819","nonce":1383862628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6823","nonce":1383862628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1383862628},{"type":"v1","addr":"172.21.15.117:6821","nonce":1383862628}]},"public_addr":"172.21.15.117:6817/1383862628","cluster_addr":"172.21.15.117:6819/1383862628","heartbeat_back_addr":"172.21.15.117:6823/1383862628","heartbeat_front_addr":"172.21.15.117:6821/1383862628","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.429996+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.361601+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.459580+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.443035+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.327807+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.512114+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.382670+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-09T11:10:38.396358+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:6837/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6833/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6835/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.117:6834/477143596":"2023-06-10T11:46:44.607439+0000","172.21.15.7:6838/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6836/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6833/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6832/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6834/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.117:6834/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/3302752268":"2023-06-10T11:49:49.651187+0000","172.21.15.117:6833/1018690909":"2023-06-10T11:33:04.415919+0000","172.21.15.117:6833/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6832/1239341654":"2023-06-10T11:22:09.265354+0000","172.21.15.117:6835/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.117:6833/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.117:6837/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6832/1113843600":"2023-06-10T11:12:14.125539+0000","172.21.15.117:6837/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6836/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.117:6834/3642322665":"2023-06-10T11:55:04.729418+0000","172.21.15.7:6839/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6835/1617501683":"2023-06-10T11:20:01.246951+0000","172.21.15.7:6839/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.7:6838/967742498":"2023-06-10T11:13:19.140088+0000","172.21.15.117:6837/3174501100":"2023-06-10T11:24:19.294959+0000","172.21.15.117:6834/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6836/3727664816":"2023-06-10T11:47:54.623961+0000","172.21.15.117:6836/1520141131":"2023-06-10T11:44:19.573452+0000","172.21.15.117:6833/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/4051500902":"2023-06-10T11:15:19.165964+0000","172.21.15.117:6832/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6835/2166536228":"2023-06-10T11:27:44.342982+0000","172.21.15.117:6833/2112619043":"2023-06-10T11:29:34.367463+0000","172.21.15.117:6834/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6835/3983837537":"2023-06-10T11:30:34.382664+0000","172.21.15.117:6834/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.117:6832/1900293448":"2023-06-10T11:45:24.588596+0000","172.21.15.117:6835/3162263716":"2023-06-10T11:34:49.438691+0000","172.21.15.7:6838/2631794280":"2023-06-10T11:25:04.306342+0000","172.21.15.117:6836/3311310614":"2023-06-10T11:36:19.460437+0000","172.21.15.117:6832/557223025":"2023-06-10T11:38:14.486645+0000","172.21.15.7:6839/1569034808":"2023-06-10T11:58:09.772592+0000","172.21.15.117:6834/1343666954":"2023-06-10T11:40:29.517640+0000","172.21.15.117:6835/1343666954":"2023-06-10T11:40:29.517640+0000","192.168.0.1:0/1078574456":"2023-06-09T12:59:16.804971+0000","172.21.15.7:6838/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6837/2733808306":"2023-06-10T11:17:14.192970+0000","172.21.15.7:6839/2829758995":"2023-06-10T11:42:19.540762+0000","172.21.15.117:6836/3744293936":"2023-06-10T11:52:39.695648+0000","172.21.15.117:6837/1520141131":"2023-06-10T11:44:19.573452+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-09T12:19:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 msgr2=0x7f69dc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.604 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 0x7f69dc06d1c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f69fc14ed20 tx=0x7f69f0008040 comp rx=0 tx=0).stop 2023-06-09T12:19:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 msgr2=0x7f69fc1f0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f69ec00c700 tx=0x7f69ec00ca10 comp rx=0 tx=0).stop 2023-06-09T12:19:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 shutdown_connections 2023-06-09T12:19:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f69dc06ad10 0x7f69dc06d1c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f69fc0ca580 0x7f69fc149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69fc0b5360 0x7f69fc1f0b40 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 --2- 172.21.15.7:0/2245438931 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f69fc0ca1b0 0x7f69fc1f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 >> 172.21.15.7:0/2245438931 conn(0x7f69fc072090 msgr2=0x7f69fc151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 shutdown_connections 2023-06-09T12:19:16.606 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.600+0000 7f6a00fff700 1 -- 172.21.15.7:0/2245438931 wait complete. 2023-06-09T12:19:16.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-06-09T12:19:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.820+0000 7f5fbd3ff700 1 Processor -- start 2023-06-09T12:19:16.824 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.820+0000 7f5fbd3ff700 1 -- start start 2023-06-09T12:19:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.821+0000 7f5fbd3ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fb8153880 0x7f5fb815d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.821+0000 7f5fbd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb80c9f40 0x7f5fb815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.825 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.821+0000 7f5fbd3ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb815a500 con 0x7f5fb815b2d0 2023-06-09T12:19:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.821+0000 7f5fbd3ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb80c5f10 con 0x7f5fb8153880 2023-06-09T12:19:16.826 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.821+0000 7f5fbd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb815da00 con 0x7f5fb80c9f40 2023-06-09T12:19:16.827 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f5fb8153880 0x7f5fb815d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb77fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5fb815b2d0 0x7f5fb80c4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43440/0 (socket says 172.21.15.7:43440) 2023-06-09T12:19:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb77fe700 1 -- 172.21.15.7:0/1080246485 learned_addr learned my addr 172.21.15.7:0/1080246485 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:19:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb67fc700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb80c9f40 0x7f5fb815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 -- 172.21.15.7:0/1080246485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb80c9f40 msgr2=0x7f5fb815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb80c9f40 0x7f5fb815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 -- 172.21.15.7:0/1080246485 >> v1:172.21.15.7:6789/0 conn(0x7f5fb815b2d0 legacy=0x7f5fb80c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 -- 172.21.15.7:0/1080246485 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb815b750 con 0x7f5fb8153880 2023-06-09T12:19:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb5ffb700 1 -- 172.21.15.7:0/1080246485 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609690136 0 0) 0x7f5fb0001490 con 0x7f5fb815b2d0 2023-06-09T12:19:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.822+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb8153880 0x7f5fb815d4c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009840 tx=0x7f5fa8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb89145b5cf51 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.829 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fb5ffb700 1 -- 172.21.15.7:0/1080246485 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8011830 con 0x7f5fb8153880 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fb5ffb700 1 -- 172.21.15.7:0/1080246485 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5fa8011990 con 0x7f5fb8153880 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fb5ffb700 1 -- 172.21.15.7:0/1080246485 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8011d40 con 0x7f5fb8153880 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/1080246485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb8153880 msgr2=0x7f5fb815d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb8153880 0x7f5fb815d4c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009840 tx=0x7f5fa8009b50 comp rx=0 tx=0).stop 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/1080246485 shutdown_connections 2023-06-09T12:19:16.830 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb8153880 0x7f5fb815d4c0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/1080246485 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb80c9f40 0x7f5fb815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/1080246485 >> 172.21.15.7:0/1080246485 conn(0x7f5fb80737f0 msgr2=0x7f5fb80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/1080246485 shutdown_connections 2023-06-09T12:19:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.823+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/1080246485 wait complete. 2023-06-09T12:19:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.825+0000 7f5fbd3ff700 1 Processor -- start 2023-06-09T12:19:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.825+0000 7f5fbd3ff700 1 -- start start 2023-06-09T12:19:16.833 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 0x7f5fb8153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb8153880 0x7f5fb81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8149070 con 0x7f5fb815b2d0 2023-06-09T12:19:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb81eedd0 con 0x7f5fb80c9f40 2023-06-09T12:19:16.834 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fbd3ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb8149310 con 0x7f5fb8153880 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fb77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fb77fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58246/0 (socket says 172.21.15.7:58246) 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fb77fe700 1 -- 172.21.15.7:0/127216667 learned_addr learned my addr 172.21.15.7:0/127216667 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fb67fc700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb8153880 0x7f5fb81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.826+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 0x7f5fb8153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.835 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 -- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb8153880 msgr2=0x7f5fb81571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb8153880 0x7f5fb81571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 -- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 msgr2=0x7f5fb8153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 0x7f5fb8153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 -- 172.21.15.7:0/127216667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa80094a0 con 0x7f5fb815b2d0 2023-06-09T12:19:16.836 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 0x7f5fb8153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-09T12:19:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fb77fe700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000cba0 tx=0x7f5fb000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb000aad0 con 0x7f5fb815b2d0 2023-06-09T12:19:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5fb000ac50 con 0x7f5fb815b2d0 2023-06-09T12:19:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb00070d0 con 0x7f5fb815b2d0 2023-06-09T12:19:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb8159980 con 0x7f5fb815b2d0 2023-06-09T12:19:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.827+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb81596e0 con 0x7f5fb815b2d0 2023-06-09T12:19:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.828+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb8159c60 con 0x7f5fb815b2d0 2023-06-09T12:19:16.838 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.829+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5fb0007230 con 0x7f5fb815b2d0 2023-06-09T12:19:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.830+0000 7f5f9f7fe700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.830+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 8313+0+0 (secure 0 0 0) 0x7f5fb0089760 con 0x7f5fb815b2d0 2023-06-09T12:19:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.831+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:16.839 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.831+0000 7f5fb6ffd700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 0x7f5fa406d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80090b0 tx=0x7f5fa8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:16.840 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:16.835+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f5fb0053e00 con 0x7f5fb815b2d0 2023-06-09T12:19:17.091 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.088+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5fb815a500 con 0x7f5fb815b2d0 2023-06-09T12:19:17.330 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.327+0000 7f5f9f7fe700 1 -- 172.21.15.7:0/127216667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v229) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5fb815a500 con 0x7f5fb815b2d0 2023-06-09T12:19:17.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 msgr2=0x7f5fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:17.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 0x7f5fa406d190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5fa80090b0 tx=0x7f5fa8008030 comp rx=0 tx=0).stop 2023-06-09T12:19:17.333 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 msgr2=0x7f5fb80ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000cba0 tx=0x7f5fb000ceb0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 shutdown_connections 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5fb815b2d0 0x7f5fb80ceb70 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5fb80c9f40 0x7f5fb8153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f5fa406ace0 0x7f5fa406d190 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.334 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 --2- 172.21.15.7:0/127216667 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5fb8153880 0x7f5fb81571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 >> 172.21.15.7:0/127216667 conn(0x7f5fb80737f0 msgr2=0x7f5fb814c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:17.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 shutdown_connections 2023-06-09T12:19:17.335 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.329+0000 7f5fbd3ff700 1 -- 172.21.15.7:0/127216667 wait complete. 2023-06-09T12:19:17.335 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-06-09T12:19:17.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-06-09T12:19:17.427 INFO:tasks.ceph.mds.b.smithi117.stdout:starting mds.b at 2023-06-09T12:19:17.427 INFO:tasks.ceph.mds.f.smithi117.stdout:starting mds.f at 2023-06-09T12:19:17.555 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.552+0000 7f7322bff700 1 Processor -- start 2023-06-09T12:19:17.556 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.552+0000 7f7322bff700 1 -- start start 2023-06-09T12:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f7322bff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c0c64e0 0x7f731c06b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f7322bff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7f731c06b740 0x7f731c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.557 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f7322bff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0c6220 con 0x7f731c1537a0 2023-06-09T12:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f7322bff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0c4f70 con 0x7f731c0c64e0 2023-06-09T12:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f7322bff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0c5c20 con 0x7f731c06b740 2023-06-09T12:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f731bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c0c64e0 0x7f731c06b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f731bfff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7f731c0c64e0 0x7f731c06b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58256/0 (socket says 172.21.15.7:58256) 2023-06-09T12:19:17.558 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.553+0000 7f731bfff700 1 -- 172.21.15.7:0/4012820253 learned_addr learned my addr 172.21.15.7:0/4012820253 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:17.559 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f732119c700 1 --2- 172.21.15.7:0/4012820253 >> v2:172.21.15.117:3300/0 conn(0x7f731c06b740 0x7f731c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731bfff700 1 -- 172.21.15.7:0/4012820253 >> v2:172.21.15.117:3300/0 conn(0x7f731c06b740 msgr2=0x7f731c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731bfff700 1 --2- 172.21.15.7:0/4012820253 >> v2:172.21.15.117:3300/0 conn(0x7f731c06b740 0x7f731c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731bfff700 1 -- 172.21.15.7:0/4012820253 >> v1:172.21.15.117:6789/0 conn(0x7f731c1537a0 legacy=0x7f731c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:17.560 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731bfff700 1 -- 172.21.15.7:0/4012820253 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0c4480 con 0x7f731c0c64e0 2023-06-09T12:19:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731bfff700 1 --2- 172.21.15.7:0/4012820253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c0c64e0 0x7f731c06b1b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f731c08d6c0 tx=0x7f7308009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c3d8274c3c1eb4 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731b7fe700 1 -- 172.21.15.7:0/4012820253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7308004230 con 0x7f731c0c64e0 2023-06-09T12:19:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731b7fe700 1 -- 172.21.15.7:0/4012820253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7308004390 con 0x7f731c0c64e0 2023-06-09T12:19:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.554+0000 7f731b7fe700 1 -- 172.21.15.7:0/4012820253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73080036a0 con 0x7f731c0c64e0 2023-06-09T12:19:17.561 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 -- 172.21.15.7:0/4012820253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c0c64e0 msgr2=0x7f731c06b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 --2- 172.21.15.7:0/4012820253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c0c64e0 0x7f731c06b1b0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f731c08d6c0 tx=0x7f7308009bd0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 -- 172.21.15.7:0/4012820253 shutdown_connections 2023-06-09T12:19:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 --2- 172.21.15.7:0/4012820253 >> v2:172.21.15.117:3300/0 conn(0x7f731c06b740 0x7f731c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 --2- 172.21.15.7:0/4012820253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c0c64e0 0x7f731c06b1b0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.562 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 -- 172.21.15.7:0/4012820253 >> 172.21.15.7:0/4012820253 conn(0x7f731c072040 msgr2=0x7f731c072440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 -- 172.21.15.7:0/4012820253 shutdown_connections 2023-06-09T12:19:17.563 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.555+0000 7f7322bff700 1 -- 172.21.15.7:0/4012820253 wait complete. 2023-06-09T12:19:17.564 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.557+0000 7f7322bff700 1 Processor -- start 2023-06-09T12:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.558+0000 7f7322bff700 1 -- start start 2023-06-09T12:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f731c06b740 0x7f731c14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f731c0c64e0 0x7f731c1eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.565 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c1f2fb0 con 0x7f731c1537a0 2023-06-09T12:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c1f2b20 con 0x7f731c0c64e0 2023-06-09T12:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f7322bff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c1493f0 con 0x7f731c06b740 2023-06-09T12:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f732119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f732119c700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58264/0 (socket says 172.21.15.7:58264) 2023-06-09T12:19:17.566 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f731bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f731c0c64e0 0x7f731c1eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f732119c700 1 -- 172.21.15.7:0/239564439 learned_addr learned my addr 172.21.15.7:0/239564439 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.559+0000 7f732099b700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f731c06b740 0x7f731c14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 -- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f731c06b740 msgr2=0x7f731c14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f731c06b740 0x7f731c14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.567 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 -- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f731c0c64e0 msgr2=0x7f731c1eeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f731c0c64e0 0x7f731c1eeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 -- 172.21.15.7:0/239564439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308009550 con 0x7f731c1537a0 2023-06-09T12:19:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f732119c700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7314007e40 tx=0x7f731400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400ed30 con 0x7f731c1537a0 2023-06-09T12:19:17.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7314014420 con 0x7f731c1537a0 2023-06-09T12:19:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c0c46d0 con 0x7f731c1537a0 2023-06-09T12:19:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140147f0 con 0x7f731c1537a0 2023-06-09T12:19:17.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.560+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c15ae20 con 0x7f731c1537a0 2023-06-09T12:19:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.561+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c0c49b0 con 0x7f731c1537a0 2023-06-09T12:19:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.562+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7314014950 con 0x7f731c1537a0 2023-06-09T12:19:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.564+0000 7f73197fa700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 0x7f730006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:17.570 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.564+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 8485+0+0 (secure 0 0 0) 0x7f731408a480 con 0x7f731c1537a0 2023-06-09T12:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.565+0000 7f732099b700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 0x7f730006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:17.571 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.565+0000 7f732099b700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 0x7f730006d1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7310009960 tx=0x7f73100090e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:17.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.569+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7f7314054b90 con 0x7f731c1537a0 2023-06-09T12:19:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:17.819+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f731c1f2c90 con 0x7f731c1537a0 2023-06-09T12:19:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.337+0000 7f73197fa700 1 -- 172.21.15.7:0/239564439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v230) v1 ==== 104+0+0 (secure 0 0 0) 0x7f73140581b0 con 0x7f731c1537a0 2023-06-09T12:19:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 msgr2=0x7f730006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 0x7f730006d1c0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7310009960 tx=0x7f73100090e0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 msgr2=0x7f731c1ef020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f7314007e40 tx=0x7f731400e3c0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 shutdown_connections 2023-06-09T12:19:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f731c1537a0 0x7f731c1ef020 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7f730006ad10 0x7f730006d1c0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f731c06b740 0x7f731c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 --2- 172.21.15.7:0/239564439 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f731c0c64e0 0x7f731c1eeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.339+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 >> 172.21.15.7:0/239564439 conn(0x7f731c072040 msgr2=0x7f731c15b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.340+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 shutdown_connections 2023-06-09T12:19:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.340+0000 7f7322bff700 1 -- 172.21.15.7:0/239564439 wait complete. 2023-06-09T12:19:18.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-06-09T12:19:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.564+0000 7fd5393ff700 1 Processor -- start 2023-06-09T12:19:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.564+0000 7fd5393ff700 1 -- start start 2023-06-09T12:19:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.565+0000 7fd5393ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.565+0000 7fd5393ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 0x7fd5340c1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.565+0000 7fd5393ff700 1 -- --> v1:172.21.15.117:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340b1120 con 0x7fd5341508b0 2023-06-09T12:19:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.565+0000 7fd5393ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53406bd40 con 0x7fd5340c1660 2023-06-09T12:19:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.565+0000 7fd5393ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd534070650 con 0x7fd5341504e0 2023-06-09T12:19:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.7:45758/0 (socket says 172.21.15.7:45758) 2023-06-09T12:19:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 -- 172.21.15.7:0/1445968148 learned_addr learned my addr 172.21.15.7:0/1445968148 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd5337fe700 1 --2- 172.21.15.7:0/1445968148 >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 0x7fd5340c1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.575 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 -- 172.21.15.7:0/1445968148 >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 msgr2=0x7fd5340c1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 --2- 172.21.15.7:0/1445968148 >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 0x7fd5340c1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 -- 172.21.15.7:0/1445968148 >> v1:172.21.15.117:6789/0 conn(0x7fd5341508b0 legacy=0x7fd53406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd532ffd700 1 -- 172.21.15.7:0/1445968148 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340b0b60 con 0x7fd5341504e0 2023-06-09T12:19:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.566+0000 7fd5337fe700 1 --2- 172.21.15.7:0/1445968148 >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 0x7fd5340c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-09T12:19:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd532ffd700 1 --2- 172.21.15.7:0/1445968148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd52400bb90 tx=0x7fd52400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bcc6a2b0384e5b server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd531ffb700 1 -- 172.21.15.7:0/1445968148 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400dd30 con 0x7fd5341504e0 2023-06-09T12:19:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd531ffb700 1 -- 172.21.15.7:0/1445968148 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd52400de90 con 0x7fd5341504e0 2023-06-09T12:19:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd531ffb700 1 -- 172.21.15.7:0/1445968148 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240106b0 con 0x7fd5341504e0 2023-06-09T12:19:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 -- 172.21.15.7:0/1445968148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 msgr2=0x7fd5340c10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1445968148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd52400bb90 tx=0x7fd52400bea0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 -- 172.21.15.7:0/1445968148 shutdown_connections 2023-06-09T12:19:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1445968148 >> v2:172.21.15.117:3300/0 conn(0x7fd5340c1660 0x7fd5340c1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1445968148 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd5340c10c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.567+0000 7fd5393ff700 1 -- 172.21.15.7:0/1445968148 >> 172.21.15.7:0/1445968148 conn(0x7fd5340cbd60 msgr2=0x7fd5340c7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.568+0000 7fd5393ff700 1 -- 172.21.15.7:0/1445968148 shutdown_connections 2023-06-09T12:19:18.580 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.568+0000 7fd5393ff700 1 -- 172.21.15.7:0/1445968148 wait complete. 2023-06-09T12:19:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.569+0000 7fd5393ff700 1 Processor -- start 2023-06-09T12:19:18.582 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.569+0000 7fd5393ff700 1 -- start start 2023-06-09T12:19:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5340c1660 0x7fd534158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd53414f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53414ed60 con 0x7fd5340c8ef0 2023-06-09T12:19:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53414f360 con 0x7fd5340c1660 2023-06-09T12:19:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.570+0000 7fd5393ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd534158140 con 0x7fd5341504e0 2023-06-09T12:19:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd532ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5340c1660 0x7fd534158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58266/0 (socket says 172.21.15.7:58266) 2023-06-09T12:19:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 -- 172.21.15.7:0/1139081823 learned_addr learned my addr 172.21.15.7:0/1139081823 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5327fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd53414f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 -- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 msgr2=0x7fd53414f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd53414f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 -- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5340c1660 msgr2=0x7fd534158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5340c1660 0x7fd534158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 -- 172.21.15.7:0/1139081823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c009a70 con 0x7fd5340c8ef0 2023-06-09T12:19:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.571+0000 7fd5337fe700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00efc0 tx=0x7fd52c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.572+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00ca30 con 0x7fd5340c8ef0 2023-06-09T12:19:18.586 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.572+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd52c00f040 con 0x7fd5340c8ef0 2023-06-09T12:19:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.572+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52c00cdd0 con 0x7fd5340c8ef0 2023-06-09T12:19:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.572+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52400b7f0 con 0x7fd5340c8ef0 2023-06-09T12:19:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.572+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53406b680 con 0x7fd5340c8ef0 2023-06-09T12:19:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.573+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5340b0e40 con 0x7fd5340c8ef0 2023-06-09T12:19:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.574+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd52c014480 con 0x7fd5340c8ef0 2023-06-09T12:19:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.575+0000 7fd51b7fe700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.575+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 8485+0+0 (secure 0 0 0) 0x7fd52c08a640 con 0x7fd5340c8ef0 2023-06-09T12:19:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.575+0000 7fd532ffd700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.576+0000 7fd532ffd700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd52400bee0 tx=0x7fd52401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:18.589 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.579+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd52c054ba0 con 0x7fd5340c8ef0 2023-06-09T12:19:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:18.786+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd53406fbd0 con 0x7fd5340c8ef0 2023-06-09T12:19:19.350 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.346+0000 7fd51b7fe700 1 -- 172.21.15.7:0/1139081823 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd52c0180d0 con 0x7fd5340c8ef0 2023-06-09T12:19:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.350+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 msgr2=0x7fd52006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.350+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 0x7fd52006d190 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fd52400bee0 tx=0x7fd52401c040 comp rx=0 tx=0).stop 2023-06-09T12:19:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.350+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 msgr2=0x7fd5341468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.350+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fd52c00efc0 tx=0x7fd52c00c3a0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 shutdown_connections 2023-06-09T12:19:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5340c8ef0 0x7fd5341468c0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd5340c1660 0x7fd534158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd52006ace0 0x7fd52006d190 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 --2- 172.21.15.7:0/1139081823 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd5341504e0 0x7fd53414f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.352+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 >> 172.21.15.7:0/1139081823 conn(0x7fd5340cbd60 msgr2=0x7fd534147720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:19.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.353+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 shutdown_connections 2023-06-09T12:19:19.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.353+0000 7fd5393ff700 1 -- 172.21.15.7:0/1139081823 wait complete. 2023-06-09T12:19:19.359 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2023-06-09T12:19:19.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-06-09T12:19:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.606+0000 7fd47a7ff700 1 Processor -- start 2023-06-09T12:19:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.607+0000 7fd47a7ff700 1 -- start start 2023-06-09T12:19:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd47a7ff700 1 --2- >> v2:172.21.15.117:3300/0 conn(0x7fd4740cf850 0x7fd474159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd47a7ff700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4740cfc20 0x7fd474156740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd47a7ff700 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47415a4e0 con 0x7fd4740c8e50 2023-06-09T12:19:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd47a7ff700 1 -- --> v2:172.21.15.7:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4740b2360 con 0x7fd4740cfc20 2023-06-09T12:19:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd47a7ff700 1 -- --> v2:172.21.15.117:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4740b2c00 con 0x7fd4740cf850 2023-06-09T12:19:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd478d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4740cfc20 0x7fd474156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd4737fe700 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd4740c8e50 0x7fd4740b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:43446/0 (socket says 172.21.15.7:43446) 2023-06-09T12:19:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd4737fe700 1 -- 172.21.15.7:0/95697304 learned_addr learned my addr 172.21.15.7:0/95697304 (peer_addr_for_me v1:172.21.15.7:0/0) 2023-06-09T12:19:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd478d9c700 1 --2- >> v2:172.21.15.7:3300/0 conn(0x7fd4740cfc20 0x7fd474156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58282/0 (socket says 172.21.15.7:58282) 2023-06-09T12:19:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.608+0000 7fd473fff700 1 --2- 172.21.15.7:0/95697304 >> v2:172.21.15.117:3300/0 conn(0x7fd4740cf850 0x7fd474159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd478d9c700 1 -- 172.21.15.7:0/95697304 >> v2:172.21.15.117:3300/0 conn(0x7fd4740cf850 msgr2=0x7fd474159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-09T12:19:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd478d9c700 1 --2- 172.21.15.7:0/95697304 >> v2:172.21.15.117:3300/0 conn(0x7fd4740cf850 0x7fd474159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd478d9c700 1 -- 172.21.15.7:0/95697304 >> v1:172.21.15.7:6789/0 conn(0x7fd4740c8e50 legacy=0x7fd4740b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd478d9c700 1 -- 172.21.15.7:0/95697304 --> v2:172.21.15.7:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47406b4b0 con 0x7fd4740cfc20 2023-06-09T12:19:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd472ffd700 1 -- 172.21.15.7:0/95697304 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151563446 0 0) 0x7fd468001110 con 0x7fd4740c8e50 2023-06-09T12:19:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd478d9c700 1 --2- 172.21.15.7:0/95697304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740cfc20 0x7fd474156740 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd4740d0240 tx=0x7fd46c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8001efc9fe6e401 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd472ffd700 1 -- 172.21.15.7:0/95697304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00e070 con 0x7fd4740cfc20 2023-06-09T12:19:19.616 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd472ffd700 1 -- 172.21.15.7:0/95697304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd46c00bb20 con 0x7fd4740cfc20 2023-06-09T12:19:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.609+0000 7fd472ffd700 1 -- 172.21.15.7:0/95697304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c012430 con 0x7fd4740cfc20 2023-06-09T12:19:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 -- 172.21.15.7:0/95697304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740cfc20 msgr2=0x7fd474156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/95697304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740cfc20 0x7fd474156740 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd4740d0240 tx=0x7fd46c00b330 comp rx=0 tx=0).stop 2023-06-09T12:19:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 -- 172.21.15.7:0/95697304 shutdown_connections 2023-06-09T12:19:19.617 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/95697304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740cfc20 0x7fd474156740 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/95697304 >> v2:172.21.15.117:3300/0 conn(0x7fd4740cf850 0x7fd474159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 -- 172.21.15.7:0/95697304 >> 172.21.15.7:0/95697304 conn(0x7fd474072060 msgr2=0x7fd474072430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 -- 172.21.15.7:0/95697304 shutdown_connections 2023-06-09T12:19:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.610+0000 7fd47a7ff700 1 -- 172.21.15.7:0/95697304 wait complete. 2023-06-09T12:19:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.612+0000 7fd47a7ff700 1 Processor -- start 2023-06-09T12:19:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.612+0000 7fd47a7ff700 1 -- start start 2023-06-09T12:19:19.620 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 0x7fd4740c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4740cfc20 0x7fd4740c3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741f2990 con 0x7fd4740c8e50 2023-06-09T12:19:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741f1d10 con 0x7fd4740cf850 2023-06-09T12:19:19.621 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd47a7ff700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4741f1ff0 con 0x7fd4740cfc20 2023-06-09T12:19:19.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:19.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58286/0 (socket says 172.21.15.7:58286) 2023-06-09T12:19:19.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 -- 172.21.15.7:0/178819047 learned_addr learned my addr 172.21.15.7:0/178819047 (peer_addr_for_me v2:172.21.15.7:0/0) 2023-06-09T12:19:19.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 -- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4740cfc20 msgr2=0x7fd4740c3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.622 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4740cfc20 0x7fd4740c3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 -- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 msgr2=0x7fd4740c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:19.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd4737fe700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 0x7fd4740c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:19.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 0x7fd4740c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:19.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd473fff700 1 -- 172.21.15.7:0/178819047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c009a70 con 0x7fd4740c8e50 2023-06-09T12:19:19.623 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.613+0000 7fd4737fe700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 0x7fd4740c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-09T12:19:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd473fff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9b0 tx=0x7fd46400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd464009710 con 0x7fd4740c8e50 2023-06-09T12:19:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd46400c850 con 0x7fd4740c8e50 2023-06-09T12:19:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46400cbe0 con 0x7fd4740c8e50 2023-06-09T12:19:19.624 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4740b4b70 con 0x7fd4740c8e50 2023-06-09T12:19:19.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.614+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4740b5150 con 0x7fd4740c8e50 2023-06-09T12:19:19.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.615+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4740b53f0 con 0x7fd4740c8e50 2023-06-09T12:19:19.625 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.616+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd46400cd40 con 0x7fd4740c8e50 2023-06-09T12:19:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.617+0000 7fd470ff9700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 0x7fd46006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-09T12:19:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.617+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 8047+0+0 (secure 0 0 0) 0x7fd46408a1f0 con 0x7fd4740c8e50 2023-06-09T12:19:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.617+0000 7fd4737fe700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 0x7fd46006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-09T12:19:19.626 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.618+0000 7fd4737fe700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 0x7fd46006d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd4741ec250 tx=0x7fd468009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-09T12:19:19.627 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.621+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176824 (secure 0 0 0) 0x7fd4640584d0 con 0x7fd4740c8e50 2023-06-09T12:19:19.837 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:19.833+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4741f2ad0 con 0x7fd4740c8e50 2023-06-09T12:19:20.352 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.346+0000 7fd470ff9700 1 -- 172.21.15.7:0/178819047 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd464058060 con 0x7fd4740c8e50 2023-06-09T12:19:20.355 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.351+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 msgr2=0x7fd46006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.351+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 0x7fd46006d190 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fd4741ec250 tx=0x7fd468009040 comp rx=0 tx=0).stop 2023-06-09T12:19:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.351+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 msgr2=0x7fd474070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-09T12:19:20.356 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.351+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd46400e9b0 tx=0x7fd46400ed70 comp rx=0 tx=0).stop 2023-06-09T12:19:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 shutdown_connections 2023-06-09T12:19:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:6832/2153300544,v1:172.21.15.7:6833/2153300544] conn(0x7fd46006ace0 0x7fd46006d190 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:20.357 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd4740cfc20 0x7fd4740c3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd4740c8e50 0x7fd474070d60 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 --2- 172.21.15.7:0/178819047 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4740cf850 0x7fd4740c2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-09T12:19:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 >> 172.21.15.7:0/178819047 conn(0x7fd474072060 msgr2=0x7fd47406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-09T12:19:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.352+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 shutdown_connections 2023-06-09T12:19:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2023-06-09T12:19:20.353+0000 7fd47a7ff700 1 -- 172.21.15.7:0/178819047 wait complete. 2023-06-09T12:19:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2023-06-09T12:19:20.384 INFO:teuthology.misc:Shutting down mds daemons... 2023-06-09T12:19:20.385 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-06-09T12:19:20.385 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.399 INFO:tasks.ceph.mds.a:Stopped 2023-06-09T12:19:20.400 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-06-09T12:19:20.400 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.464 INFO:tasks.ceph.mds.c:Stopped 2023-06-09T12:19:20.464 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2023-06-09T12:19:20.465 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.518 INFO:tasks.ceph.mds.e:Stopped 2023-06-09T12:19:20.519 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-06-09T12:19:20.519 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.532 INFO:tasks.ceph.mds.b:Stopped 2023-06-09T12:19:20.532 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-06-09T12:19:20.533 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.590 INFO:tasks.ceph.mds.d:Stopped 2023-06-09T12:19:20.590 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2023-06-09T12:19:20.591 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.643 INFO:tasks.ceph.mds.f:Stopped 2023-06-09T12:19:20.643 INFO:teuthology.misc:Shutting down osd daemons... 2023-06-09T12:19:20.644 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-06-09T12:19:20.644 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.731 INFO:tasks.ceph.osd.0:Stopped 2023-06-09T12:19:20.732 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-06-09T12:19:20.732 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.816 INFO:tasks.ceph.osd.1:Stopped 2023-06-09T12:19:20.817 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-06-09T12:19:20.817 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f is failed for ~0s 2023-06-09T12:19:20.881 INFO:tasks.ceph.osd.2:Stopped 2023-06-09T12:19:20.882 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-06-09T12:19:20.882 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:20.959 INFO:tasks.ceph.osd.3:Stopped 2023-06-09T12:19:20.960 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-06-09T12:19:20.960 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.049 INFO:tasks.ceph.osd.4:Stopped 2023-06-09T12:19:21.050 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-06-09T12:19:21.050 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.131 INFO:tasks.ceph.osd.5:Stopped 2023-06-09T12:19:21.132 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-06-09T12:19:21.132 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.203 INFO:tasks.ceph.osd.6:Stopped 2023-06-09T12:19:21.204 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-06-09T12:19:21.204 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.273 INFO:tasks.ceph.osd.7:Stopped 2023-06-09T12:19:21.273 INFO:teuthology.misc:Shutting down mgr daemons... 2023-06-09T12:19:21.274 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-06-09T12:19:21.274 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.340 INFO:tasks.ceph.mgr.y:Stopped 2023-06-09T12:19:21.340 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-06-09T12:19:21.340 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.399 INFO:tasks.ceph.mgr.x:Stopped 2023-06-09T12:19:21.400 INFO:teuthology.misc:Shutting down mon daemons... 2023-06-09T12:19:21.400 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-06-09T12:19:21.400 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.427 INFO:tasks.ceph.mon.a:Stopped 2023-06-09T12:19:21.428 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-06-09T12:19:21.428 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.447 INFO:tasks.ceph.mon.b:Stopped 2023-06-09T12:19:21.448 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-06-09T12:19:21.448 INFO:teuthology.orchestra.run:waiting for 300 2023-06-09T12:19:21.508 INFO:tasks.ceph.mon.c:Stopped 2023-06-09T12:19:21.508 INFO:tasks.ceph:Checking cluster log for badness... 2023-06-09T12:19:21.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v 'Replacing daemon mds' | head -n 1 2023-06-09T12:19:21.602 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T12:19:21.603 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-06-09T12:19:22.334 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T12:19:22.335 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-06-09T12:19:22.566 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T12:19:22.566 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-06-09T12:19:22.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T12:19:22.808 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-06-09T12:19:23.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T12:19:23.047 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-06-09T12:19:23.835 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T12:19:23.836 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-06-09T12:19:24.068 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T12:19:24.069 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-06-09T12:19:24.313 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T12:19:24.314 DEBUG:teuthology.orchestra.run.smithi117:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-06-09T12:19:24.556 INFO:tasks.ceph:Archiving mon data... 2023-06-09T12:19:24.557 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/data/mon.a.tgz 2023-06-09T12:19:24.558 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2023-06-09T12:19:24.605 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.tqWUmiQPmb 2023-06-09T12:19:24.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tqWUmiQPmb 2023-06-09T12:19:24.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.tqWUmiQPmb 2023-06-09T12:19:24.881 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.tqWUmiQPmb is 368KB 2023-06-09T12:19:24.973 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.tqWUmiQPmb 2023-06-09T12:19:25.014 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/data/mon.b.tgz 2023-06-09T12:19:25.015 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2023-06-09T12:19:25.059 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.33FpOJtbWj 2023-06-09T12:19:25.059 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.33FpOJtbWj 2023-06-09T12:19:25.245 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.33FpOJtbWj 2023-06-09T12:19:25.356 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.33FpOJtbWj is 375KB 2023-06-09T12:19:25.450 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.33FpOJtbWj 2023-06-09T12:19:25.493 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/data/mon.c.tgz 2023-06-09T12:19:25.493 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2023-06-09T12:19:25.575 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.kCfoF2ascQ 2023-06-09T12:19:25.575 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kCfoF2ascQ 2023-06-09T12:19:25.764 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.kCfoF2ascQ 2023-06-09T12:19:25.845 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.f has been restored 2023-06-09T12:19:25.880 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.kCfoF2ascQ is 375KB 2023-06-09T12:19:25.971 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.kCfoF2ascQ 2023-06-09T12:19:26.013 INFO:tasks.ceph:Cleaning ceph cluster... 2023-06-09T12:19:26.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-09T12:19:26.018 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-06-09T12:19:26.119 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-06-09T12:19:26.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-09T12:19:26.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-06-09T12:19:26.201 INFO:teuthology.orchestra.run.smithi117.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-09T12:19:26.208 INFO:teuthology.orchestra.run.smithi007.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-06-09T12:19:26.224 INFO:tasks.ceph:Archiving crash dumps... 2023-06-09T12:19:26.225 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi007/crash 2023-06-09T12:19:26.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-09T12:19:26.330 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi117/crash 2023-06-09T12:19:26.332 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-06-09T12:19:26.397 INFO:tasks.ceph:Shutting down logrotate 2023-06-09T12:19:26.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:26.402 DEBUG:teuthology.orchestra.run.smithi117:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:26.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:26.571 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-06-09T12:19:26.637 INFO:tasks.ceph:Compressing logs... 2023-06-09T12:19:26.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-09T12:19:26.642 DEBUG:teuthology.orchestra.run.smithi117:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-09T12:22:29.516 INFO:tasks.ceph:Archiving logs... 2023-06-09T12:22:29.517 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi007/log 2023-06-09T12:22:29.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-09T12:22:44.987 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi117/log 2023-06-09T12:22:44.988 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-09T12:23:01.047 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-09T12:23:01.057 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-09T12:23:01.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-09T12:23:01.063 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-09T12:23:01.200 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-09T12:23:01.210 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-09T12:23:01.211 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T12:23:01.215 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-09T12:23:01.253 INFO:teuthology.orchestra.run.smithi007.stderr:bash: ntpq: command not found 2023-06-09T12:23:01.255 INFO:teuthology.orchestra.run.smithi117.stderr:bash: ntpq: command not found 2023-06-09T12:23:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T12:23:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2023-06-09T12:23:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 113 -461us[ -479us] +/- 93ms 2023-06-09T12:23:01.261 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv02.front.sepia.ceph.com 3 6 377 52 -2232us[-2249us] +/- 52ms 2023-06-09T12:23:01.262 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +3405us[+3405us] +/- 71ms 2023-06-09T12:23:01.262 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T12:23:01.263 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-09T12:23:01.263 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2023-06-09T12:23:01.263 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 166 -472us[ -484us] +/- 92ms 2023-06-09T12:23:01.263 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv02.front.sepia.ceph.com 3 7 377 110 -2200us[-2214us] +/- 51ms 2023-06-09T12:23:01.264 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +3425us[+3425us] +/- 71ms 2023-06-09T12:23:01.264 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-06-09T12:23:01.265 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-09T12:23:01.275 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-09T12:23:01.275 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-09T12:23:01.287 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-09T12:23:01.495 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-09T12:23:01.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T12:23:01.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:23:01.736 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-09T12:23:01.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:23:01.810 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-09T12:23:01.819 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-09T12:23:01.831 INFO:teuthology.task.internal:Duration was 5289.519304 seconds 2023-06-09T12:23:01.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-09T12:23:01.844 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-09T12:23:01.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-09T12:23:01.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-09T12:23:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T12:23:01.964 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-09T12:23:02.286 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-09T12:23:02.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2023-06-09T12:23:02.287 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-09T12:23:02.346 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2023-06-09T12:23:02.346 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-09T12:23:02.403 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-09T12:23:02.404 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-09T12:23:02.408 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-09T12:23:02.495 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-09T12:23:02.495 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-09T12:23:02.500 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-09T12:23:02.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-09T12:23:02.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-09T12:23:02.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-09T12:23:02.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-09T12:23:02.788 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-09T12:23:02.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-09T12:23:02.804 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-09T12:23:02.866 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2023-06-09T12:23:02.870 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2023-06-09T12:23:02.916 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-09T12:23:02.956 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi007.front.sepia.ceph.com, flagging run as failed 2023-06-09T12:23:02.957 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-09T12:23:02.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:23:02.997 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-09T12:23:03.007 INFO:teuthology.task.internal:Transferring archived files... 2023-06-09T12:23:03.008 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi007 2023-06-09T12:23:03.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-09T12:24:19.444 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-06-09T12:24:19.640 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849/remote/smithi007/coredump/1686311844.137538.core.gz: gzip compressed data, was "1686311844.137538.core", last modified: Fri Jun 9 12:23:03 2023, max compression 2023-06-09T12:24:19.641 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-06-09T12:24:25.210 INFO:teuthology.task.internal: core looks like: /tmp/tmpg41j3sdt: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-fuse', platform: 'x86_64' 2023-06-09T12:24:25.333 INFO:teuthology.task.internal: dump_program: ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu 2023-06-09T12:24:25.334 DEBUG:teuthology.orchestra.run.smithi007:> which 'ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu' 2023-06-09T12:24:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:/usr/bin/which: invalid option -- 'f' 2023-06-09T12:24:25.382 INFO:teuthology.orchestra.run.smithi007.stderr:/usr/bin/which: unrecognized option '--admin-socket' 2023-06-09T12:24:25.383 INFO:teuthology.orchestra.run.smithi007.stderr:/usr/bin/which: no $cluster-$name.$pid.asok in (/var/run/ceph) 2023-06-09T12:24:25.383 INFO:teuthology.orchestra.run.smithi007.stderr:/usr/bin/which: no ubuntu in (/home) 2023-06-09T12:24:25.383 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-06-09T12:24:25.384 INFO:teuthology.orchestra.run.smithi007.stdout:/usr/bin/ceph-fuse 2023-06-09T12:24:25.385 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi007 with status 2: "which 'ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu'" 2023-06-09T12:24:25.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-09T12:24:25.408 INFO:teuthology.task.internal:Not uploading archives. 2023-06-09T12:24:25.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-09T12:24:25.421 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-09T12:24:25.421 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-09T12:24:25.446 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-09T12:24:25.482 INFO:teuthology.orchestra.run.smithi007.stdout: 524293 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest 2023-06-09T12:24:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: 524294 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive 2023-06-09T12:24:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: 667173 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive/audit 2023-06-09T12:24:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: 667747 312 -rw------- 1 ubuntu ubuntu 317760 Jun 9 12:23 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-06-09T12:24:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: 524295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 9 11:57 /home/ubuntu/cephtest/archive/coredump 2023-06-09T12:24:25.483 INFO:teuthology.orchestra.run.smithi007.stdout: 524479 756236 -rw------- 1 root root 1086242816 Jun 9 11:57 /home/ubuntu/cephtest/archive/coredump/1686311844.137538.core 2023-06-09T12:24:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: 524296 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog 2023-06-09T12:24:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: 524447 4 -rw-rw-rw- 1 ubuntu ubuntu 2882 Jun 9 12:19 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-06-09T12:24:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: 524301 148 -rw-rw-rw- 1 ubuntu ubuntu 150630 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-06-09T12:24:25.484 INFO:teuthology.orchestra.run.smithi007.stdout: 524302 128 -rw-rw-r-- 1 ubuntu ubuntu 129762 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-09T12:24:25.485 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-06-09T12:24:25.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-09T12:24:25.486 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi007 with status 2: "which 'ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu'" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_2ef0dcdee569e9851c9d2ffb6acfd4b80d031e61/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi007 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-06-09T12:24:25.495 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-09T12:24:25.507 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-09T12:24:25.524 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi007 with status 2: "which 'ceph-fuse -f --admin-socket /var/run/ceph/$cluster-$name.$pid.asok /home/ubuntu'" 2023-06-09T12:24:25.525 INFO:teuthology.nuke:Checking targets against current locks 2023-06-09T12:24:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: 524301 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest 2023-06-09T12:24:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: 524302 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive 2023-06-09T12:24:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: 666989 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive/audit 2023-06-09T12:24:25.532 INFO:teuthology.orchestra.run.smithi117.stdout: 666991 248 -rw------- 1 ubuntu ubuntu 252102 Jun 9 12:23 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-06-09T12:24:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: 524304 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog 2023-06-09T12:24:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: 524447 4 -rw-rw-rw- 1 ubuntu ubuntu 2536 Jun 9 12:19 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-06-09T12:24:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: 524305 112 -rw-rw-rw- 1 ubuntu ubuntu 113530 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-06-09T12:24:25.533 INFO:teuthology.orchestra.run.smithi117.stdout: 524306 108 -rw-rw-r-- 1 ubuntu ubuntu 109717 Jun 9 12:23 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-09T12:24:25.534 INFO:teuthology.orchestra.run.smithi117.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-06-09T12:24:25.560 DEBUG:teuthology.nuke:shortname: smithi007 2023-06-09T12:24:25.560 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-09T12:24:25.584 DEBUG:teuthology.nuke:shortname: smithi117 2023-06-09T12:24:25.584 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-09T12:24:25.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 10:19:34.133256', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-09T12:24:25.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-09_10:19:22-fs-wip-rishabh-2023Jun9-1308-testing-default-smithi/7299849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-09 10:19:34.134781', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-09T12:24:25.640 INFO:teuthology.orchestra.console:Power off smithi007 2023-06-09T12:24:25.641 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-09T12:24:25.660 INFO:teuthology.orchestra.console:Power off smithi117 2023-06-09T12:24:25.661 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-09T12:24:25.876 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:25.887 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:29.901 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:29.910 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:34.017 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi007.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:34.027 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-09T12:24:34.230 INFO:teuthology.orchestra.console:Power off for smithi007 completed 2023-06-09T12:24:34.237 INFO:teuthology.orchestra.console:Power off for smithi117 completed 2023-06-09T12:24:34.344 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse msgr-failures/osd-mds-delay objectstore-ec/bluestore-comp overrides/{frag ignorelist_health ignorelist_wrongly_marked_down prefetch_dirfrags/yes prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/3 tasks/{1-thrash/mds 2-workunit/fs/snaps}} duration: 5289.519304275513 failure_reason: 'Command failed (workunit test fs/snaps/snaptest-git-ceph.sh) on smithi007 with status 128: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=034108821cde95ca1c56d458dfccc7ef9e0d964d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/snaps/snaptest-git-ceph.sh''' flavor: default mon.a-kernel-sha1: 10b2f20a4f46c803bb730efc73498f97d45082e7 mon.b-kernel-sha1: 10b2f20a4f46c803bb730efc73498f97d45082e7 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4a265fdf331448508ad0c0be8859880f status: fail success: false 2023-06-09T12:24:34.344 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-09T12:24:34.432 INFO:teuthology.run:FAIL